COVID-19 update: PythonAnywhere is now all-remote

Scary times. We hope everyone reading this is well and keeping safe!

We thought it would be a good idea to tell you how we're managing the current crisis at PythonAnywhere. We switched over to remote working last Thursday, 12 March; there are obviously private and public health reasons why that was a good idea, but there's a reason specific to us, which we thought would be worth sharing.

Most of the team here are lucky enough to be in low risk categories, but we pair-program -- that is, we have two people working together at the same computer, all day and every day. Each day we rotate the pairs around, so that the same two people are never working together on two consecutive days. This makes sure that we spread knowledge around the team.

Unfortunately, that means that if someone caught COVID-19, we'd share that pretty quickly around the team too. And that would be bad, not just for us, but for the people who use our service. We can keep the servers running, tech support operating, and so on in the short term with just a small subset of the team, but that would be no good if all of us were unwell simultaneously, as even a mild infection could make it impossible for someone to help with tech support or system maintenance.

Right now, we're all working from home, sharing tech support, and pair-programming using shared consoles and voice chat as a way to keep development going. Tech support is going fine -- we're used to doing that from home, as that's what we do at weekends. Development is a little slower right now; over time we will probably refine the setup we're using, and maybe we'll even come up with some nice ideas for new features for PythonAnywhere to make it easier for other people to do the same kind of thing :-)

Fingers crossed, and good luck to everyone. These are challenging times -- stay safe, and, as we say when we're updating our service, we'll see you on the other side.


How to use shared in-browser consoles to cooperate while working remotely.

One of the challenges of remote work is when you need to work together on one thing.

Our in-browser consoles are one of the core features of our service. Almost since the beginning, PythonAnywhere has been able to share consoles -- you entered the name of another user or an email address, and they got an email telling them how to log in and view your Python (or Bash, or IPython) console. If you use an email, the person you invite doesn't have to be PythonAnywhere registered user.

When you open a console, there is a button on the top right that says "Share with others"...

...which should popup a dialog box when you click on it.

I may invite one or many persons. Some of them can have full access, some of them read-only.

In the shared console, you share not only what you see, but also a cursor -- so people that it's shared with can also type into it. That can be prevented if you share your console in "read-only" mode. You may later revoke the sharing using the same button and dialog box.


System updates on 3 and 5 March

On 3 March we upgraded our EU-based system at eu.pythonanywhere.com to the latest version of our code, and this morning (5 March) we upgraded our US-based system at www.pythonanywhere.com to the same version.

Everything went very smoothly, and all systems are working well. There were a bunch of infrastructure-related changes in this update:

  • We've made some improvements to the beta of our new virtualisation system, which is currently in public beta. More about that next week, we hope!
  • We've updated almost all of our machines to the most recent AWS Intel server types; the remainder will be upgraded over the coming two weeks. CPU geeks will be glad to hear that we're going to start experimenting with AMD. We might also consider ARM for our own internal systems, though right now it feels like sticking with x86-64 for the servers where our users run their code is the best option (let us know if in the comments if you disagree!)
  • A certain amount of code that was (somewhat embarrassingly) still Python 2 was upgraded to Python 3. blush

As usual, there were also a number of minor tweaks and minor bugfixes.

Onwards and upwards!


The PythonAnywhere newsletter, January 2020

So, we have managed to break another record for our longest period ever between two monthly newsletters. It has been sixteen busy months between September 2018 and now, so we have made 2019 an official Year Without a Newsletter.

Happy New Year, and a warm welcome to the January 2020 PythonAnywhere newsletter. Hooray! Here is what has happened since our last one.

Python 3.8 now available

We recently added support for Python 3.8. If you signed up after 4 December 2019, you'll have it available on your account -- you can use it just like any other Python version.

If you signed up before then, it's a little more complicated, but we can update your account to provide it -- more information here.

Always-on tasks

Always-on tasks are a feature we rolled out for paid accounts in our October 2018 update. Essentially, you specify a program and we keep it running for you all the time. If it exits for any reason, we'll automatically restart it -- even in extreme circumstances, for instance if the server that it's running on has a hardware failure, it will fail over to another working machine quickly.

Let's Encrypt certificates with automatic renewal

You can now get a free, automated HTTPS certificate for your custom domain using Let's Encrypt. Previously, there was all sorts of tedious manual mucking around with dehydrated to get that free cert. And you won't need to remember to renew the certificate anymore either! (or as was the case for some of our users, setting up a scheduled task to auto-renew your certificate!) Now this will all happen behind the scenes automatically.

Deployment of eu.pythonanywhere.com and migration option

Back in February 2019, we announced eu.pythonanywhere.com. It's a completely separate version of our site, with all of the computers and storage hosted in Frankfurt, rather than in the US for www.pythonanywhere.com.

In November 2019 we made a migration system available to our users. It allows us to move accounts from the US system to the EU one with minimal downtime. If you have an account on www.pythonanywhere.com and would like it to be moved to eu.pythonanywhere.com, just let us know via email (support@pythonanywhere.com).

Tutorials

We published some tutorials and HOW-TOs:

PythonAnywhere metrics

As in the last newsletter we wanted to share some of the metrics:

  • Web requests: we're processing on average about 375 hits/second through our systems (across all websites) with spikes at busy times of up to 450/second.
  • That's across about 49,000 websites. Of course the number of hits sites get is spread over a long tail distribution -- some of those sites are ones that people set up as part of tutorials, so they only get hits from their owners, while on the other hand the busiest websites might be processing 40 hits/second at their peak times
  • There are over 9,000 scheduled and always-on tasks.
  • Our live system currently comprises 69 separate machines on Amazon AWS in the US cluster and 17 in the EU one.

New modules

Although you can install Python packages on PythonAnywhere yourself, we like to make sure that we have plenty of batteries included.

Everything got updated for the new system image that provides access to Python 3.8, so if you're using that image, you should have the most recent (or at least a very recent) version of everything :-)

New whitelisted sites

Paying PythonAnywhere customers get unrestricted Internet access, but if you're a free PythonAnywhere user, you may have hit problems when writing code that tries to access sites elsewhere on the Internet. We have to restrict you to sites on a whitelist to stop hackers from creating dummy accounts to hide their identities when breaking into other people's websites.

But we really do encourage you to suggest new sites that should be on the whitelist. Our rule is, if it's got an official public API, which means that the site's owners are encouraging automated access to their server, then we'll whitelist it. Just drop us a line with a link to the API docs.

We keep adding new sites to the list every day.

And now for something completely different

You might not have noticed, but in August 2019 a Florida man has hand-captured a Burmese python measuring 17 feet, 9 inches.


Python 3.8 now available!

If you signed up since 26 November, you'll have Python 3.8 available on your account -- you can use it just like any other Python version.

If you signed up before then, it's a little more complicated, because adding Python 3.8 to your account requires changing your system image. Each account has an associated system image, which determines which Python versions, Python packages, operating system packages, and so on are available. The new image is called "fishnchips" (after the previous system images, "classic", "dangermouse" and "earlgrey").

What this means is that if we change your system image, the pre-installed Python packages will all get upgraded, which means that any code you have that depends on them might stop working if it's not compatible with the new versions.

Additionally, if you're using virtualenvs, because this update upgrades the point releases of the older Python versions (for example, 3.7.0 gets upgraded to 3.7.5), the update may make your envs stop working -- if so, you'll need to rebuild them.

So, long story short -- we can switch your account over to the new system image, but you may need to rebuild your virtualenvs afterwards if you're using them -- and you may need to update your code to handle newer pre-installed Python packages if you're not using virtualenvs.

There are more details about exactly which package versions are included in which system image on the batteries included page. And if you'd like to switch your account over to fishnchips, just drop us a line using the "Send feedback" button. (If you've read all of the above, and understand that you may have to make code/virtualenv changes, mention that you have in the feedback message as otherwise we'll respond by basically repeating all of the stuff we just said, and asking "are you sure?")


System update on 21 November 2019

This morning's system update went smoothly; some websites did take a bit longer than we expected to start up afterwards, but all is well now.

There are two big new features that we have introduced which are going through some final post-deploy tests before they go live -- a new system image (called fishnchips) to support Python 3.8 and to add on a number of extra OS packages that people have been asking us for, and an update to our virtualization system that will fix a number of problems with specific programs. We'll be posting more about those when they're ready to go live. [Update: we've posted about the fishnchips system image here.]

But there were a number of other things we added:

  • We've added the ability to restart an always-on task without stopping it, waiting for it to stop, and then starting it again -- there's a new "restart" button on the far right of the tasks table.
  • You can now temporarily disable a website without deleting it -- the button is near the bottom of the "Web" page.
  • And, of course, the normal set of minor tweaks, bugfixes, and the like.

Happy coding!


EU migrations are now live!

In brief: if you have an account on www.pythonanywhere.com you can have it migrated to eu.pythonanywhere.com -- just let us know via email to support@pythonanywhere.com.

If you'd like to know more about what that means, read on...

Back in February, we announced eu.pythonanywhere.com. It's a completely separate version of our site, with all of the computers and storage hosted in Frankfurt, rather than in the US like our other site at www.pythonanywhere.com.

Although our US-based systems are fully GDPR-compliant, thanks to the EU-US Privacy Shield Framework, we appreciate that some people are keen on keeping all of their data inside the EU so as to be sure that they comply with all regulations.

If you're based in the EU, there's almost no downside to using the new system. You will have the comfort of knowing your data is in the EU, and it's closer to you in network terms (in our tests, the network latency is about 8ms from Amsterdam, versus 90ms to our US servers). For paid accounts, billing is in euros so you don't need to worry about foreign exchange fees on your card payments. If we need to perform system maintenance on the servers, we'll do it late at night European time, rather than during the early morning timeslot that we use for the US-based system. The only reason you might want to stick with the US system is that it's a little cheaper; our underlying hosting costs are a bit higher for the EU service, which is reflected in the prices -- a Hacker account that costs US$5/month on the US servers is €5/month on our new system (plus VAT if applicable).

The new system has a good number of users now and is functioning well. We've also been beta-testing a migration system that allows us to move accounts from the US system to the EU one with minimal downtime, and that is working well too :-)

So it's time for us to take that migration system out of beta -- if you have an account on www.pythonanywhere.com and would like it to be moved to eu.pythonanywhere.com, just let us know via email to support@pythonanywhere.com.

Let us know if you have any questions!


Our new CPU API

We received many requests from PythonAnywhere users to make it possible to programmatically monitor usage of CPU credit, so we decided to add a new endpoint to our experimental API.

The first step when using the API is to get an API token -- this is what you use to authenticate yourself with our servers when using it. To do that, log in to PythonAnywhere, and go to the "Account" page using the link at the top right. Click on the "API token" tab; if you don't already have a token, it will look like this:

Click the "Create a new API token" button to get your token, and you'll see this:

That string of letters and numbers (d870f0cac74964b27db563aeda9e418565a0d60d in the screenshot) is an API token, and anyone who has it can access your PythonAnywhere account and do stuff -- so keep it secret. If someone does somehow get hold of it, you can revoke it on this page by clicking the red button -- that stops it from working in the future, and creates a new one for you to use.

Now you can use CPU API to track your CPU usage.

For example you could use it at the beginning and the end of your script to learn how many CPU seconds were consumed (assuming nothing else is running).

from math import factorial
from time import sleep
from urllib.parse import urljoin

import requests

api_token = "YOUR TOKEN HERE"
username = "YOUR USERNAME HERE"
pythonanywhere_host = "www.pythonanywhere.com"

api_base = "https://{pythonanywhere_host}/api/v0/user/{username}/".format(
    pythonanywhere_host=pythonanywhere_host,
    username=username,
)

resp = requests.get(
    urljoin(api_base, "cpu/"),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

initial_usage_seconds = resp.json()["daily_cpu_total_usage_seconds"]

# we burn some cpu seconds

[factorial(x) for x in range(2000)]

# cpu usage is updated every 60 seconds, so we need to wait to be sure that usage is available for us to read.

sleep(70)

resp = requests.get(
    urljoin(api_base, "cpu/"),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

final_usage_seconds = resp.json()["daily_cpu_total_usage_seconds"]

seconds_used = final_usage_seconds - initial_usage_seconds

print("Task cost {} CPU seconds to run".format(seconds_used))

...replacing "YOUR TOKEN HERE" and "YOUR USERNAME HERE" with the appropriate stuff.
If you're on our EU-based system, you should also replace www.pythonanywhere.com with eu.pythonanywhere.com.

Let us know if you have any comments or questions -- otherwise, happy coding!


Using our file API

Our API supports lots of common PythonAnywhere operations, like creating and managing consoles, scheduled and always-on tasks, and websites. We recently added support for reading/writing files; this blog post gives a brief overview of how you can use it to do that.

The first step when using the API is to get an API token -- this is what you use to authenticate yourself with our servers when using it. To do that, log in to PythonAnywhere, and go to the "Account" page using the link at the top right. Click on the "API token" tab; if you don't already have a token, it will look like this:

Click the "Create a new API token" button to get your token, and you'll see this:

That string of letters and numbers (d870f0cac74964b27db563aeda9e418565a0d60d in the screenshot) is an API token, and anyone who has it can access your PythonAnywhere account and do stuff -- so keep it secret. If someone does somehow get hold of it, you can revoke it on this page by clicking the red button -- that stops it from working in the future, and creates a new one for you to use.

Now let's use that token to access some files.

Preparing for using the API

On a machine with Python and requests installed, start a Python interpreter and run this:

from pprint import pprint
import requests
from urllib.parse import urljoin

api_token = "YOUR TOKEN HERE"
username = "YOUR USERNAME HERE"
pythonanywhere_host = "www.pythonanywhere.com"

api_base = "https://{pythonanywhere_host}/api/v0/user/{username}/".format(
    pythonanywhere_host=pythonanywhere_host,
    username=username,
)

...replacing "YOUR TOKEN HERE" and "YOUR USERNAME HERE" with the appropriate stuff. If you're on our EU-based system, you should also replace www.pythonanywhere.com with eu.pythonanywhere.com.

If you're using Python 2.7, the line

from urllib.parse import urljoin

should be

from urlparse import urljoin

Listing files

Now let's make a request to see what files you have:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

You should get a 200 status code:

>>> print(resp.status_code)
200

...and the JSON data in the response should be a reasonably-understandable description of the files and directories in your home directory -- note the type field that tells you what is what.

>>> pprint(resp.json())
{u'.bashrc': {u'type': u'file',
              u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.bashrc'},
 u'.gitconfig': {u'type': u'file',
                 u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.gitconfig'},
 u'.local': {u'type': u'directory',
             u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.local'},
 u'.profile': {u'type': u'file',
               u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.profile'},
 u'.pythonstartup.py': {u'type': u'file',
                        u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.pythonstartup.py'},
 u'.vimrc': {u'type': u'file',
             u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.vimrc'},
 u'README.txt': {u'type': u'file',
                 u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/README.txt'}}

So that shows how to get a directory listing.

Downloading a file

Let's try downloading a file:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/README.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

Again, the status code should be 200:

>>> print(resp.status_code)
200

...and the content of the response will be the contents of the file:

>>> print(resp.content)
# vim: set ft=rst:

See https://help.pythonanywhere.com/ (or click the "Help" link at the top
right) for help on how to use PythonAnywhere, including tips on copying and
pasting from consoles, and writing your own web applications.

Uploading a file

Let's upload a file. We'll put the text hello, world into a file called foo.txt in your home directory:

resp = requests.post(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    files={"content": "hello world"},
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

This time, the status code should be 201 (which means "created"):

>>> print(resp.status_code)
201

Now you can go to the "Files" page on PythonAnywhere, and you'll see the file; click on the filename to open an editor and see its contents:

The API will also show the new contents if you use it to get the file:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

You'll get a 200 status code:

>>> print(resp.status_code)
200

And the expected content:

>>> print(resp.content)
hello world

Updating an existing file

You can update your file with new contents:

resp = requests.post(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    files={"content": "some new contents"},
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

This time, the status code should be 200 (which means "successfully updated" in this context):

>>> print(resp.status_code)
200

...and if you refresh your editor window, you'll see the updated contents.

Deleting a file

Finally we can delete the file:

resp = requests.delete(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

And we get a "204" status code, meaning "successfully deleted":

>>> print(resp.status_code)
204

Refresh the editor page again, and you'll see a 404 not found error:

And if we try to download it using the API:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

We get the same status:

>>> print(resp.status_code)
404

Conclusion

That's been a quick high-level overview of our new file API. Let us know what you think! And if you have any questions, just leave a comment below.


System update on 26 June

Our system update on 26 June went pretty smoothly :-) There were a number of useful changes:

  • Our API now supports uploading, downloading and listing files in your private file storage. There is another blog post about this.
  • We now have an official system in place to migrate your MySQL data between database servers, which means that if you're on an older version, we can move you over to 5.7. Let us know if you're interested!
  • Our system for migrating user accounts from our US-based system at www.pythonanywhere.com to our EU-based one at eu.pythonanywhere.com is almost finished -- just a few final bugs to work out. We'll post about this again when it's ready to go live. Update: it's now live.

We also pushed a number of bugfixes:

  • A number of issues which meant that always-on tasks could lose access to the network have been fixed.
  • Previously, if you tried to change the command for an always-on task that was disabled, you'd get an error -- this has been addressed.
  • Some HTTP libraries (which seem particularly common on IoT devices) put the port number in the host header on requests (eg. they send www.somedomain.com:80 rather than www.somedomain.com). This is perfectly valid HTTP, albeit uncommon, but our system failed to parse them correctly and would respond with a 404 error. We've fixed that.

There were also a whole bunch of minor UI tweaks and the like.

Right now we're working on making sure that our billing system supports the Strong Customer Authentication (SCA) regulations that will come into force for all payments from European credit/debit cards this September; hopefully we can make this as seamless as possible for you.

Happy coding!


Page 1 of 18.

Older posts »

PythonAnywhere is a Python development and hosting environment that displays in your web browser and runs on our servers. They're already set up with everything you need. It's easy to use, fast, and powerful. There's even a useful free plan.

You can sign up here.