Python 3.8 now available!

If you signed up since 26 November, you'll have Python 3.8 available on your account -- you can use it just like any other Python version.

If you signed up before then, it's a little more complicated, because adding Python 3.8 to your account requires changing your system image. Each account has an associated system image, which determines which Python versions, Python packages, operating system packages, and so on are available. The new image is called "fishnchips" (after the previous system images, "classic", "dangermouse" and "earlgrey").

What this means is that if we change your system image, the pre-installed Python packages will all get upgraded, which means that any code you have that depends on them might stop working if it's not compatible with the new versions.

Additionally, if you're using virtualenvs, because this update upgrades the point releases of the older Python versions (for example, 3.7.0 gets upgraded to 3.7.5), the update may make your envs stop working -- if so, you'll need to rebuild them.

So, long story short -- we can switch your account over to the new system image, but you may need to rebuild your virtualenvs afterwards if you're using them -- and you may need to update your code to handle newer pre-installed Python packages if you're not using virtualenvs.

There are more details about exactly which package versions are included in which system image on the batteries included page. And if you'd like to switch your account over to fishnchips, just drop us a line using the "Send feedback" button. (If you've read all of the above, and understand that you may have to make code/virtualenv changes, mention that you have in the feedback message as otherwise we'll respond by basically repeating all of the stuff we just said, and asking "are you sure?")


System update on 21 November 2019

This morning's system update went smoothly; some websites did take a bit longer than we expected to start up afterwards, but all is well now.

There are two big new features that we have introduced which are going through some final post-deploy tests before they go live -- a new system image (called fishnchips) to support Python 3.8 and to add on a number of extra OS packages that people have been asking us for, and an update to our virtualization system that will fix a number of problems with specific programs. We'll be posting more about those when they're ready to go live. [Update: we've posted about the fishnchips system image here.]

But there were a number of other things we added:

  • We've added the ability to restart an always-on task without stopping it, waiting for it to stop, and then starting it again -- there's a new "restart" button on the far right of the tasks table.
  • You can now temporarily disable a website without deleting it -- the button is near the bottom of the "Web" page.
  • And, of course, the normal set of minor tweaks, bugfixes, and the like.

Happy coding!


EU migrations are now live!

In brief: if you have an account on www.pythonanywhere.com you can have it migrated to eu.pythonanywhere.com -- just let us know via email to support@pythonanywhere.com.

If you'd like to know more about what that means, read on...

Back in February, we announced eu.pythonanywhere.com. It's a completely separate version of our site, with all of the computers and storage hosted in Frankfurt, rather than in the US like our other site at www.pythonanywhere.com.

Although our US-based systems are fully GDPR-compliant, thanks to the EU-US Privacy Shield Framework, we appreciate that some people are keen on keeping all of their data inside the EU so as to be sure that they comply with all regulations.

If you're based in the EU, there's almost no downside to using the new system. You will have the comfort of knowing your data is in the EU, and it's closer to you in network terms (in our tests, the network latency is about 8ms from Amsterdam, versus 90ms to our US servers). For paid accounts, billing is in euros so you don't need to worry about foreign exchange fees on your card payments. If we need to perform system maintenance on the servers, we'll do it late at night European time, rather than during the early morning timeslot that we use for the US-based system. The only reason you might want to stick with the US system is that it's a little cheaper; our underlying hosting costs are a bit higher for the EU service, which is reflected in the prices -- a Hacker account that costs US$5/month on the US servers is €5/month on our new system (plus VAT if applicable).

The new system has a good number of users now and is functioning well. We've also been beta-testing a migration system that allows us to move accounts from the US system to the EU one with minimal downtime, and that is working well too :-)

So it's time for us to take that migration system out of beta -- if you have an account on www.pythonanywhere.com and would like it to be moved to eu.pythonanywhere.com, just let us know via email to support@pythonanywhere.com.

Let us know if you have any questions!


Our new CPU API

We received many requests from PythonAnywhere users to make it possible to programmatically monitor usage of CPU credit, so we decided to add a new endpoint to our experimental API.

The first step when using the API is to get an API token -- this is what you use to authenticate yourself with our servers when using it. To do that, log in to PythonAnywhere, and go to the "Account" page using the link at the top right. Click on the "API token" tab; if you don't already have a token, it will look like this:

Click the "Create a new API token" button to get your token, and you'll see this:

That string of letters and numbers (d870f0cac74964b27db563aeda9e418565a0d60d in the screenshot) is an API token, and anyone who has it can access your PythonAnywhere account and do stuff -- so keep it secret. If someone does somehow get hold of it, you can revoke it on this page by clicking the red button -- that stops it from working in the future, and creates a new one for you to use.

Now you can use CPU API to track your CPU usage.

For example you could use it at the beginning and the end of your script to learn how many CPU seconds were consumed (assuming nothing else is running).

from math import factorial
from time import sleep
from urllib.parse import urljoin

import requests

api_token = "YOUR TOKEN HERE"
username = "YOUR USERNAME HERE"
pythonanywhere_host = "www.pythonanywhere.com"

api_base = "https://{pythonanywhere_host}/api/v0/user/{username}/".format(
    pythonanywhere_host=pythonanywhere_host,
    username=username,
)

resp = requests.get(
    urljoin(api_base, "cpu/"),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

initial_usage_seconds = resp.json()["daily_cpu_total_usage_seconds"]

# we burn some cpu seconds

[factorial(x) for x in range(2000)]

# cpu usage is updated every 60 seconds, so we need to wait to be sure that usage is available for us to read.

sleep(70)

resp = requests.get(
    urljoin(api_base, "cpu/"),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

final_usage_seconds = resp.json()["daily_cpu_total_usage_seconds"]

seconds_used = final_usage_seconds - initial_usage_seconds

print("Task cost {} CPU seconds to run".format(seconds_used))

...replacing "YOUR TOKEN HERE" and "YOUR USERNAME HERE" with the appropriate stuff.
If you're on our EU-based system, you should also replace www.pythonanywhere.com with eu.pythonanywhere.com.

Let us know if you have any comments or questions -- otherwise, happy coding!


Using our file API

Our API supports lots of common PythonAnywhere operations, like creating and managing consoles, scheduled and always-on tasks, and websites. We recently added support for reading/writing files; this blog post gives a brief overview of how you can use it to do that.

The first step when using the API is to get an API token -- this is what you use to authenticate yourself with our servers when using it. To do that, log in to PythonAnywhere, and go to the "Account" page using the link at the top right. Click on the "API token" tab; if you don't already have a token, it will look like this:

Click the "Create a new API token" button to get your token, and you'll see this:

That string of letters and numbers (d870f0cac74964b27db563aeda9e418565a0d60d in the screenshot) is an API token, and anyone who has it can access your PythonAnywhere account and do stuff -- so keep it secret. If someone does somehow get hold of it, you can revoke it on this page by clicking the red button -- that stops it from working in the future, and creates a new one for you to use.

Now let's use that token to access some files.

Preparing for using the API

On a machine with Python and requests installed, start a Python interpreter and run this:

from pprint import pprint
import requests
from urllib.parse import urljoin

api_token = "YOUR TOKEN HERE"
username = "YOUR USERNAME HERE"
pythonanywhere_host = "www.pythonanywhere.com"

api_base = "https://{pythonanywhere_host}/api/v0/user/{username}/".format(
    pythonanywhere_host=pythonanywhere_host,
    username=username,
)

...replacing "YOUR TOKEN HERE" and "YOUR USERNAME HERE" with the appropriate stuff. If you're on our EU-based system, you should also replace www.pythonanywhere.com with eu.pythonanywhere.com.

If you're using Python 2.7, the line

from urllib.parse import urljoin

should be

from urlparse import urljoin

Listing files

Now let's make a request to see what files you have:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

You should get a 200 status code:

>>> print(resp.status_code)
200

...and the JSON data in the response should be a reasonably-understandable description of the files and directories in your home directory -- note the type field that tells you what is what.

>>> pprint(resp.json())
{u'.bashrc': {u'type': u'file',
              u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.bashrc'},
 u'.gitconfig': {u'type': u'file',
                 u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.gitconfig'},
 u'.local': {u'type': u'directory',
             u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.local'},
 u'.profile': {u'type': u'file',
               u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.profile'},
 u'.pythonstartup.py': {u'type': u'file',
                        u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.pythonstartup.py'},
 u'.vimrc': {u'type': u'file',
             u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/.vimrc'},
 u'README.txt': {u'type': u'file',
                 u'url': u'https://www.pythonanywhere.com/api/v0/user/yourusername/files/path/home/yourusername/README.txt'}}

So that shows how to get a directory listing.

Downloading a file

Let's try downloading a file:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/README.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

Again, the status code should be 200:

>>> print(resp.status_code)
200

...and the content of the response will be the contents of the file:

>>> print(resp.content)
# vim: set ft=rst:

See https://help.pythonanywhere.com/ (or click the "Help" link at the top
right) for help on how to use PythonAnywhere, including tips on copying and
pasting from consoles, and writing your own web applications.

Uploading a file

Let's upload a file. We'll put the text hello, world into a file called foo.txt in your home directory:

resp = requests.post(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    files={"content": "hello world"},
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

This time, the status code should be 201 (which means "created"):

>>> print(resp.status_code)
201

Now you can go to the "Files" page on PythonAnywhere, and you'll see the file; click on the filename to open an editor and see its contents:

The API will also show the new contents if you use it to get the file:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

You'll get a 200 status code:

>>> print(resp.status_code)
200

And the expected content:

>>> print(resp.content)
hello world

Updating an existing file

You can update your file with new contents:

resp = requests.post(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    files={"content": "some new contents"},
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

This time, the status code should be 200 (which means "successfully updated" in this context):

>>> print(resp.status_code)
200

...and if you refresh your editor window, you'll see the updated contents.

Deleting a file

Finally we can delete the file:

resp = requests.delete(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

And we get a "204" status code, meaning "successfully deleted":

>>> print(resp.status_code)
204

Refresh the editor page again, and you'll see a 404 not found error:

And if we try to download it using the API:

resp = requests.get(
    urljoin(api_base, "files/path/home/{username}/foo.txt".format(username=username)),
    headers={"Authorization": "Token {api_token}".format(api_token=api_token)}
)

We get the same status:

>>> print(resp.status_code)
404

Conclusion

That's been a quick high-level overview of our new file API. Let us know what you think! And if you have any questions, just leave a comment below.


System update on 26 June

Our system update on 26 June went pretty smoothly :-) There were a number of useful changes:

  • Our API now supports uploading, downloading and listing files in your private file storage. There is another blog post about this.
  • We now have an official system in place to migrate your MySQL data between database servers, which means that if you're on an older version, we can move you over to 5.7. Let us know if you're interested!
  • Our system for migrating user accounts from our US-based system at www.pythonanywhere.com to our EU-based one at eu.pythonanywhere.com is almost finished -- just a few final bugs to work out. We'll post about this again when it's ready to go live. Update: it's now live.

We also pushed a number of bugfixes:

  • A number of issues which meant that always-on tasks could lose access to the network have been fixed.
  • Previously, if you tried to change the command for an always-on task that was disabled, you'd get an error -- this has been addressed.
  • Some HTTP libraries (which seem particularly common on IoT devices) put the port number in the host header on requests (eg. they send www.somedomain.com:80 rather than www.somedomain.com). This is perfectly valid HTTP, albeit uncommon, but our system failed to parse them correctly and would respond with a 404 error. We've fixed that.

There were also a whole bunch of minor UI tweaks and the like.

Right now we're working on making sure that our billing system supports the Strong Customer Authentication (SCA) regulations that will come into force for all payments from European credit/debit cards this September; hopefully we can make this as seamless as possible for you.

Happy coding!


Using MongoDB on PythonAnywhere with MongoDB Atlas

This requires a paid PythonAnywhere account

Lots of people want to use MongoDB with PythonAnywhere; we don't have support for it built in to the system, but it's actually pretty easy to use with a database provided by MongoDB Atlas -- and as Atlas is a cloud service provided by Mongo's creators, it's probably a good option anyway :-)

If you're experienced with MongoDB and Atlas, then our help page has all of the details you need for connecting to them from our systems.

But if you'd just like to dip your toe in the water and find out what all of this MongoDB stuff is about, this blog post explains step-by-step how to get started so that you can try it out.

Prerequisites

The first important thing to mention is that you'll need a paid PythonAnywhere account to access Atlas. Free accounts can only access the external Internet using HTTP or HTTPS, and unfortunately MongoDB uses it's own protocol which is quite different to those.

Apart from that, in order to follow along you'll need at least a basic understanding of PythonAnywhere, writing website code and of databases in general -- if you've been through our Beginners' Flask and MySQL tutorial you should be absolutely fine.

Signing up to MongoDB atlas

Unsurprisingly, the first step in using Atlas is to sign up for an account (if you don't already have one). Go to their site and click the "Try Free" button at the top right. That will take you to a page with the signup form; fill in the appropriate details, and sign up.

Atlas' user interface may change a little after we publish this post (their sign-up welcome page has already changed between out first draft yesterday and the publication today!), so we won't give super-detailed screenshots showing what to click, but as of this writing, they present you with a "Welcome" window that has some "help getting started" buttons. You can choose those if you prefer, but we'll assume that you've clicked the "I don't need help getting started" button. That should land you on a page that looks like this, which is where you create a new MongoDB cluster.

Creating your first cluster and adding a user

A MongoDB cluster is essentially the same as a database server for something more traditional like MySQL or PostgreSQL. It's called a cluster because it can potentially spread over multiple computers, so it can in theory scale up much more easily than an SQL database.

To create the cluster:

  • Choose the "AWS" cloud provider, then select the region that is closest to where your PythonAnywhere account is hosted:
    • If your account is on our global site at www.pythonanywhere.com, choose the us-east-1 region.
    • If your account is on our EU site at eu.pythonanywhere.com, choose the eu-central-1 region.
  • Click the "Create cluster" button at the bottom of the page.

This will take you to a page describing your cluster. Initially it will have text saying something like "Your cluster is being created" -- wait until that has disappeared, and you'll have a page that will look something like this:

Now we need to add a user account that we'll use to connect to the cluster from Python:

  • Click the "Database Access" link in the "Security" section of the menu on the left-hand side, which will take you to a page where you can administer users.
  • Click the "Add new user" button on the top right of the pane that appears.
  • Enter a username and a password for the new user (make sure you keep a note of these somewhere)
  • The "User Privileges" should be "Read and write to any database"
  • The "Save as a temporary user" checkbox should not be checked.

Click the button to create the user, and you'll come back to the user admin page, with your user listed in the list.

Setting up the whitelist

Access to your MongoDB cluster is limited to computers on a whitelist; this provides an extra level of security beyond the username/password combination we just specified.

Just to get started, we'll create a whitelist that comprises every IP address on the Internet -- that is, we won't have any restrictions at all. While this is not ideal in the long run, it makes taking the first steps much easier. You can tighten things up later -- more information about that at the end of this post.

Here's how to configure the whitelist to allow global access:

  • Click on the "Network Access" link in the "Security" section of the menu on the left-hand side, which will take you to the page where you manage stuff like whitelists.
  • Click the "Add IP Address" button near the top right of the page.
  • On the window that pops up, click the "Allow access from anywhere" button. This will put "0.0.0.0/0" in the "Whitelist entry" field -- this is the CIDR notation for "all addresses on the Internet".
  • Put something like "Everywhere" in the "Comment" field just as a reminder as to what the whitelist entry means, and leave the "Save as temporary whitelist" checkbox unchecked.
  • Click the "Confirm" button.

Getting the connection string

Now we have a MongoDB cluster running, and it's time to connect to it from PythonAnywhere. The first step is to get the connection string:

  • Click on the "Clusters" link in the "Atlas" section of the menu on the left-hand side, which will take you back to the page you got when you first created the cluster:

  • Click the "Connect" button just underneath the name of the cluster (probably "Cluster0").
  • You'll get a window with different connection options; click the "Connect your application" option.
  • In "Choose your driver version", select a "Driver" of "Python" and a version of "3.6 or later".
  • Once you've done that, in the "Connection String Only" section, you'll see something like mongodb+srv://admin:<password>@cluster0-zakwe.mongodb.net/test?retryWrites=true&w=majority

  • Copy that string (there's a button to do that for you) and paste it somewhere safe for later use. You'll note that it has <password> in it; you should replace that (including the angle brackets) with the actual password that you configured for your user earlier on.
  • Now you can close the popup with the button in the bottom right.

Connecting to the cluster from a PythonAnywhere console

Next, go to PythonAnywhere, and log in if necessary. The first thing we need to do here is make sure that we have the correct packages installed to connect to MongoDB -- we'll be using Python 3.7 in this example, so we need to use the pip3.7 command. Start a Bash console, and run:

pip3.7 install --user --upgrade pymongo dnspython

Once that's completed, let's connect to the cluster from a command line. Run ipython3.7 in your console, and when you have a prompt, import pymongo and connect to the cluster:

import pymongo
client = pymongo.MongoClient("<the atlas connection string>")

...replacing <the atlas connection string> with the actual connection string we got from the Atlas site earlier, with <password> replaced with the password you used when setting up the user.

That's created a connection object, but hasn't actually connected to the cluster yet -- pymongo only does that on an as-needed basis.

A good way to connect and at least make sure that what we've done so far has worked is to ask the cluster for a list of the databases that it currently has:

client.list_database_names()

If all is well, you'll get a result like this:

['admin', 'local']

...just a couple of default databases created by MongoDB itself for its own internal use.

Now let's add some data. MongoDB is much more free-form than a relational database like MySQL or PostgreSQL. There's no such thing as a table, or a row -- instead, a database is just a bunch of things called "collections", each of which is comprised of a set of "documents" -- and the documents are just objects, linking keys to values.

That's all a bit abstract; I find a useful way to imagine it is that a MongoDB database is like a directory on a disk; it contains a number of subdirectories (collections), and each of those contains a number of files (each one being a document). The files just store JSON data -- basically, Python dictionary objects.

Alternatively, you can see it by comparison with an SQL database:

  • A MongoDB database is the same kind of thing as an SQL database
  • A MongoDB collection is a bit like a table -- it's meant to hold a set of similar kinds of things -- but it's not so restrictive, and defines no columns.
  • A MongoDB document is kind of like a row in such a table, but it's not constrained to any specific set of columns -- each document could in theory be very different to all of the others. It's just best practice for all of the documents in a collection to be broadly similar.

For this tutorial, we're going to create one super-simple database, called "blog". Unsurprisingly for a blog, it will contain a number of "post" documents, each of which will contain a title, body, a slug (for the per-article URL) and potentially more information.

The first question is, how do we create a database? Neatly, we don't need to explicitly do anything -- if we refer to a database that doesn't exist, MongoDB will create it for us -- and likewise, if we refer to a collection inside the database that doesn't exist, it will create that for us too. So in order to add the first row to the "posts" collection in the "blog" database we just do this:

db = client.blog
db.posts.insert_one({"title": "My first post", "body": "This is the body of my first blog post", "slug": "first-post"})

No need for CREATE DATABASE or CREATE TABLE statements -- it's all implicit! IPython will print out the string representation of the MongoDB result object that was returned by the insert_one method; something like this:

<pymongo.results.InsertOneResult at 0x7f9ae871ea88>

Let's add a few more posts:

db.posts.insert_one({"title": "Another post", "body": "Let's try another post", "slug": "another-post", "extra-data": "something"})
db.posts.insert_one({"title": "Blog Post III", "body": "The blog post is back in another summer blockbuster", "slug": "yet-another-post", "author": "John Smith"})

Now we can inspect the posts that we have:

for post in client.blog.posts.find():
    print(post)

You'll get something like this:

{'_id': ObjectId('5d0395dcbf76b2ab4ed67948'), 'title': 'My first post', 'body': 'This is the body of my first blog post', 'slug': 'first-post'}
{'_id': ObjectId('5d039611bf76b2ab4ed67949'), 'title': 'Another post', 'body': "Let's try another post", 'slug': 'another-post', 'extra-data': 'something'}
{'_id': ObjectId('5d039619bf76b2ab4ed6794a'), 'title': 'Blog Post III', 'body': 'The blog post is back in another summer blockbuster', 'slug': 'yet-another-post', 'author': 'John Smith'}

The find function is a bit like a SELECT statement in SQL; with no parameters, it's like a SELECT with no WHERE clause, and it just returns a cursor that allows us to iterate over every document in the collection. Let's try it with a more restrictive query, and just print out one of the defined values in the object we inserted:

for post in client.blog.posts.find({"title": {"$eq": "Another post"}}):
    print(post["body"])

You'll get something like this:

Let's try another post

MongoDB's query language is very rich -- you can see a list of the query operators here. We won't go into any more detail here -- there are many excellent MongoDB tutorials on the Internet, so if you google for "mongodb python tutorial" you're bound to find something useful!

Now we've connected to a MongoDB database, and created some data, so let's do something with that.

Connecting to the cluster from a Flask website

The next step is to connect to our cluster from a website's code. We'll use Flask for this -- because it's database-agnostic, it's a better fit for MongoDB than Django, which is quite tied to the SQL model of representing data.

We're also going to use a Flask extension called Flask-PyMongo to make our connections -- the raw PyMongo package has a few problems with the way PythonAnywhere runs websites, and while there are ways around those (see the help page), the Flask extension handles everything smoothly for us. So, in your Bash console, exit IPython and run

pip3.7 install --user Flask-PyMongo

Once that's done, let's create a website: head over to the "Web" page inside PythonAnywhere, and create yourself a Python 3.7 Flask app.

When it's been created, edit the flask_app.py file that was auto-generated for you, and replace the contents with this:

from flask import Flask, render_template
from flask_pymongo import PyMongo

app = Flask(__name__)
app.config["MONGO_URI"] = "<the atlas connection string>"

mongo = PyMongo(app)

@app.route('/')
def index():
    return render_template("blog.html", posts=mongo.db.posts.find())

@app.route('/<post_slug>')
def item(post_slug):
    return render_template("blog.html", posts=mongo.db.posts.find({"slug": post_slug}))

...replacing <the atlas connection string> with the connection string as before, with one change -- the original connection string will have /test in it, like this:

mongodb+srv://admin:iW8qWskQGJcEpZdu9ZUt@cluster0-zakwe.mongodb.net/test?retryWrites=true&w=majority

That /test means "connect to the database called test on the cluster". We've put our data into a database called blog, so just replace the test with blog, so that it looks like this:

mongodb+srv://admin:iW8qWskQGJcEpZdu9ZUt@cluster0-zakwe.mongodb.net/blog?retryWrites=true&w=majority

All of the rest of the code in that file should be pretty obvious if you're familiar with Flask -- the MongoDB-specific stuff is very similar to the code we ran in a console earlier, and also to the way we would connect to MySQL via SQLAlchemy. The only really new thing is the abbreviated syntax for searching for an exact match:

mongo.db.posts.find({"slug": post_slug})

...is just a shorter way of saying this:

mongo.db.posts.find({"slug": {"$eq": post_slug}})

To go with this Flask app, we need a template file called blog.html in a new templates subdirectory of the directory containing flask_app.py -- here's something basic that will work:

<html>
    <head>
        <meta charset="utf-8">
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" integrity="sha512-dTfge/zgoMYpP7QbHy4gWMEGsbsdZeCXz7irItjcC3sPUFtf0kuFbDz/ixG7ArTxmDjLXDmezHubeNikyKGVyQ==" crossorigin="anonymous">
        <title>My blog</title>
    </head>

    <body>
        <div class="container">
            <div class="row">
                <h1><a href="/">My Blog</a></h1>
            </div>

            {% for post in posts %}
                <div class="row">
                    <h2><a href="/{{ post.slug }}">{{ post.title }}</a></h2>

                    <p>
                    {{ post.body }}
                    </p>
                </div>
            {% endfor %}

        </div>

    </body>
</html>

Note that there's one simple difference to the way we reference the post document to how we'd do it in Python code -- in Python we have to say (for example) post["title"], while dictionary lookups in a Flask template require us to use post.title.

Once you've created that file, reload the website using the button on the "Web" page, and you should see a website with your blog posts on it:

Let's add a new post: keep the tab showing your website open, but in another tab go to your Bash console, start ipython3.7 again, connect to the database, and add a new post:

import pymongo
client = pymongo.MongoClient("<the atlas connection string>")
db = client.blog
db.posts.insert_one({"title": "Blog Post Goes Forth", "body": "...but I thought we were coding Python?", "slug": "bad-blackadder-nerd-joke"})

Head back to the tab showing the site, and hit the browser's refresh button -- your new post will appear!

All done!

So now we have a working super-simple blog running on PythonAnywhere, backed by an Atlas MongoDB cluster.

The one remaining issue is that the whitelist we specified is a little broad. If someone gets hold of your MongoDB username and password, they can access the database. It's possible to set things up so that you have an initially-empty whitelist, and then every time you run your code, it automatically whitelists the IP address it's running on using the Atlas API -- that's a slightly more advanced topic, though, so if you want to learn about that, head over to our MongoDB help page.

We hope this post has been useful -- if you have any questions or comments, please leave them below. Also, if there are other things you'd like to connect to from PythonAnywhere that you think could benefit from having a blog post explaining how to do it, please do let us know!


System update this morning

This morning's system update went smoothly :-)

It was primarily a maintenance update, bringing our US-based system up to the same version our EU-based system. There were a number of minor bugfixes, along with a bunch of improvements to our system administration tools, which won't be visible to you, but do mean that we'll be able to spend less time on admin stuff -- which gives us more time to work on adding cool new features!


Announcing eu.pythonanywhere.com

We're proud to announce today that we now have an EU-hosted PythonAnywhere system :-) You can access it at eu.pythonanywhere.com. It's completely separated from our normal system, but has all of the same features -- plus billing in euros.

Why we built it

Since the GDPR came into effect last year, a lot of people have asked us about whether our servers are compliant. The answer is a resounding "yes!" -- historically all of our servers have been in the US, but they're in a datacenter run by Amazon Web Services, which is covered by the EU-U.S. Privacy Shield Framework -- an agreement that allows certain US-based hosting solutions to be treated as "as good as" EU-based systems for the purposes of the GDPR.

But we appreciate that some people are still concerned. Agreements like Privacy Shield can potentially come to an end, and while things are looking good right now, there were some issues last year.

So now, if you want to be sure that your own account's data, and any data you're using PythonAnywhere to manage, is stored in the EU, we have a solution. The servers for eu.pythonanywhere.com are based in AWS's eu-central-1 datacenter in Frankfurt, and there's no connection between the two systems apart from a common list of usernames (which is also in Frankfurt).

Why you might want to use it

If you're based in the EU, there's almost no downside to using the new system. You will have the comfort of knowing your data is in the EU, it's closer to you in network terms (in our tests, the network latency is about 8ms from Amsterdam, versus 90ms to our US servers), and for paid accounts, billing is in euros so you don't need to worry about foreign exchange fees on your card payments. If we need to perform system maintenance on the servers, we'll do it late at night European time, rather than during the early morning timeslot that we use for the existing system. The only reason you might want to stick with the US system is that it's a little cheaper; our underlying hosting costs are a bit higher for the EU service, which is reflected in the prices -- a Hacker account that costs $5/month on the US servers is €5/month on our new system (plus VAT if applicable).

If you're not in the EU, then we'd recommend you stick with www.pythonanywhere.com unless you're handling a lot of data relating to EU citizens and are concerned about the GDPR. But that said, you're entirely welcome to use the new system if you prefer :-)

How to get started

You can sign up on the EU servers right now. If you already have an account on our existing system, you'll have to use a new username; we're blocking duplicate usernames at the moment because soon we'll be offering the ability to migrate existing accounts over from the US system to the EU. If you're interested in having an existing account moved over, email us at eusupport@pythonanywhere.com and let us know your account username. Update: we now have a system in place to migrate accounts, just drop us a line!

Any questions?

If you have any questions about the new EU systems, please do let us know -- you can post a comment below, or send us an email.


How DNS works: a beginner's guide

The blog post below has now been "promoted" to being an official help page -- you can read the most current version here. The version below is the original published version and may be out of date.

Original post

We sometimes get emails from people who are trying to point their custom domain at PythonAnywhere so that they can host their website, but are struggling to set up their DNS settings. Normally DNS setup is pretty simple, but sometimes people can get bogged down due to confusing interfaces on their registrar's site, or complexities in the terminology people use.

The parts of DNS that you need to know about in order to host a website are actually not all that complicated, but some domain registrars have complicated, hard-to-understand interfaces. Either they assume that you understand all of the technical details about how the whole thing works -- which makes it hard for first-timers -- or they try to put a simple user-friendly interface on top of it, but simplify it so much that it's actually harder to use because they're hiding important stuff from you.

Given that basic DNS stuff really isn't all that hard, we felt that it would be a good idea to post an explanation, going from the basics up to some slightly deeper stuff. This post is written so that if you only want the basics, you can just read the first part, while if you want a deeper understanding -- either out of interest, or because your domain registrar has got such a low-level interface that you need to -- then you can keep reading.

It's worth noting that for most people, you don't need to know any of this stuff to set up a website on PythonAnywhere, even with a custom domain; it's meant more as an explanation so that people who do run into problems with their registrar have the background knowledge they need to solve the problem -- or, indeed, to explain to the registrar's tech support team what the problem is. And, of course, it's a bit of light reading for people who are just interested in this stuff :-)

The basics: domain names and IP addresses

When a browser wants to connect to www.yourdomain.com, it needs to know which computer on the Internet is hosting that site. The string www.yourdomain.com is a hostname (technically it's a "fully qualified domain name"), but at the underlying network layer, all computers are identified by IP addresses, which are numerical. So somehow the browser needs to find out which numerical IP address it should use when it wants to talk to www.yourdomain.com.

It does that by asking a DNS server -- in full, a Domain Name System server. Normally this will be a server that's provided by your ISP or your local network administator; in general, when your computer joins a network, the details of the DNS server are part of the information it gets from the router. So to convert a hostname to an IP address, the browser makes a system call to the operating system saying "please get me the IP address for www.yourdomain.com", the operating system sends a message to the DNS server with the same request, and the DNS server responds with the IP address: something like "23.145.21.243" (in IP version 4, the version that's currently most widely used, IP addresses are normally written as four numbers between 0 and 255, separated by dots -- I'll use similar addresses as examples later on). Now the browser can connect to the server at that IP address, send it an HTTP message saying "please send me the contents of the front page for www.yourdomain.com", and the server will respond with the appropriate stuff.

What all this means is that when you want your custom domain to be hosted on PythonAnywhere, you're essentially setting things up so that all of the DNS servers across the Internet know what IP address to provide when someone wants to access your site. (This, by the way, is why we say that the changes you make "may take some time to propagate across the Internet" -- it's not just one central database that needs to be updated; instead, different ISPs will pick up the change at different times. A little more about that later.)

DNS records

The explanation above is, of course, a bit simplified. The DNS database isn't just a mapping from hostnames like www.yourdomain.com to IP addresses. Instead, for each "domain name", it can keep a bunch of different types of records. I put "domain name" in quotes back there because the technical meaning of the words here is slightly different to what people normally use it to mean. When we normally use the phrase "domain name", we mean something like "yourdomain.com", "google.com", or "bbc.co.uk". That is, we mean the portion of a network address that identifies a particular organisation. We'd expect (for example) Google to own the domain name google.com, to have a website at www.google.com, and for people who work there to have email addresses like alice.jones@google.com.

But in the technical parlance of DNS, a "domain name" is something a little more subtle. The Wikipedia article is (of course) a good explanation, but a summary is:

  • There is a "root" domain called .
  • There is a subdomain of the root domain called com.
  • If you buy yourdomain.com then you own yourdomain.com., which is is a subdomain of com.
  • If you create a website at www.yourdomain.com then technically you've created a subdomain of yourdomain.com. with the name www.yourdomain.com..

(You'll notice that all of the domain names above end with the . to represent the root domain. Technically domain names always should, but no-one ever bothers, so I won't use the extra . in the rest of this blog post.)

Basically, any level of the hierarchy is a domain name. A number of different kinds of "records" can be associated with a domain name. The domain yourdomain.com might have various records, and so might www.yourdomain.com. Any domain can have a number of different records, including multiple records of the same type.

Some examples of record types are:

  • A ("Address") records, which are how you specify an IPv4 address like 23.145.21.243. These tell DNS that if someone wants to treat the domain name as the name of a computer, then this is the IP address they should use.
  • MX ("Mail eXchange") records, which specify what email server should be used for a domain. If someone wants to send email to bob@yourdomain.com, the email systems involved will use the MX record for yourdomain.com to work out which computer to deliver it to.
  • TXT ("TeXT") records, which are pretty much free-form text and are used for various purposes including verifying senders of email messages.

There are a bunch of others -- NS, SOA, and so on. But apart from A records (which you can see are obviously important in the example above where we got the IP address for a hostname), there's one other interesting kind of record: CNAME.

CNAMEs

When a browser (via its operating system) asks its DNS server for the IP address corresponding to a hostname, the server can respond in a number of different ways:

  • If it doesn't know what the IP address is, it will just return a response saying so, so that the browser can display an appropriate error page. (For example Chrome will say something like "www.nonexistent.com’s server IP address could not be found.")
  • If it has an A record for the hostname, then it will just return the record and the browser will use the IP address contained in it.
  • If it doesn't have an A record, but does have a CNAME (abbreviated from "canonical name") record then it will return that. The CNAME record doesn't contain an IP address; instead, it contains another hostname. The browser can then send the DNS server a second message, saying "OK, then -- so what's the IP address of that hostname" -- and hopefully the response this time will have an IP address. (Of course, it could just respond with another CNAME record, which would require a third lookup, and so on.)

CNAME records, on the face of it, look like a somewhat roundabout and inefficient way to do things. Instead of making one request to get from a hostname to an IP address, a browser will need to make two or more. So why is it that when you create a website with a custom domain on PythonAnywhere, we ask you to set up a CNAME record to point www.yourdomain.com to a hostname like webapp-123456.pythonanywhere.com, rather than just providing you with an IP address so that you can create an A record?

The answer is that CNAMEs are actually really useful -- and also not all that inefficient in practice.

The usefulness first: let's imagine you had a website hosted at www.yourdomain.com, and you set up an A record with your registrar to point it at an IP address that belonged to PythonAnywhere. Your site would be up and running, and everything would work fine so long as that IP address was always the right one for your site. But if it changed, you'd need to log in to your registrar again and update it.

But IP addresses sometimes have to change. Sometimes they get blocked in certain countries. Sometimes a specific IP address might be subjected to a denial-of-service attack, and be unusable. Or sometimes we at PythonAnywhere might want to move your site from one IP address to another simply to balance out load across our cluster of servers.

If you're using a CNAME record, then IP address changes like that are something you don't need to worry about. We can update the A records for our own hostnames, like webapp-123456.pythonanywhere.com, because we control the DNS settings for pythonanywhere.com and all of its subdomains. Because your website is pointed at us using a CNAME, browsers that want to connect to your site will start using the new IP address without you needing to do anything. They'll ask for the IP address of www.yourdomain.com, they'll get a CNAME response saying that it's the same as the one for webapp-123456.pythonanywhere.com, so they'll look up the IP address for that and will get the correct new address from the A record that comes back.

But we can't update DNS records for your domain -- only you can do that -- so if you use a A record, when the IP address changes you'll have to update it yourself every time. If you happened to be away, or if we don't have up-to-date contact details for you, it might be some time before you knew about the problem and were able to fix it.

So what about the inefficiency? We always make sure that the webapp-XXXXXX.pythonanywhere.com addresses point to an A record rather than another CNAME, but that's still, in theory, two lookups to go from www.yourdomain.com to an IP address -- one to get the CNAME, and one to get the IP address from the hostname stored there. According to Cloudflare, the average ISP has a 70 millisecond round-trip time for queries, so that might mean that your site would load up 70 milliseconds slower if it needs to do two queries rather than one. Not a huge amount of time, but given that research shows that people are less engaged with slower websites, every millisecond counts.

The answer here is twofold -- firstly, your computer will generally cache the results of DNS lookups for some time. So this extra time will only impact some hits to your page. Secondly, many DNS servers are pretty smart -- if someone asks for a hostname, and the result is going to be a CNAME record, they know that the next request is likely to be for the CNAME's value -- so they'll attach the results for that to their response as well. So, for example, a browser might say "what's the address for www.yourdomain.com?" and the DNS server would reply "www.yourdomain.com has a CNAME pointing to webapp-123456.pythonanywhere.com. Oh, and by the way, webapp-123456.pythonanywhere.com has an A record pointing to the IP address 1.2.3.4". The browser can then just use the IP address directly; there's only one lookup, but the CNAME is fully resolved.

So using CNAME records to point your website at PythonAnywhere means less maintenance for you, and in general no real performance overhead.

There is one case where CNAMEs can be problematic, though:

Naked domains

One problem with CNAMEs is that in general, you can't use them for "naked" domains. A naked domain is the domain name that you buy from your registrar, something like yourdomain.com -- without anything like www. in front of it. For relatively arcane technical reasons a naked domain can't use a CNAME, only subdomains like www.yourdomain.com -- and so if you want yourdomain.com without the www. to point to PythonAnywhere, you have to use an A record.

That's why we suggest that if you want your site to be accessible without the www., you should host it at www.yourdomain.com, and then use a redirection service so that people who visit the address without the www. will be transparently redirected to the address with it. Most registrars have this kind of thing built in. You tell them to do the redirection, and they'll set up an A record pointing to one of their servers, and that server will do the redirect when it receives a request -- and because, unlike us, they can update A records on your behalf, they can fix things if it needs to change. If your registrar doesn't support it, there are third-party services you can point to with an A record you set up yourself.

See our help pages for details of how to set up a redirect -- we have links to the appropriate documentation for a bunch of popular domain reistrars, and also some links to third-party services if you need to use them.

How to use all of that information

Hopefully by now it should be clear what you need to do when pointing a custom domain to PythonAnywhere; you log into your domain registrar's website, and find the place where you set up your DNS configuration. Once you're there, you set up a CNAME record to point www.yourdomain.com to the webapp-XXXXXX.pythonanywhere.com value shown when you look at your website's configuration on the "Web" page inside PythonAnywhere. If there are any other A or CNAME records for www.yourdomain.com then you should delete them (because having two records for the same hostname will potentially confuse the DNS). But don't delete any other records -- there may be things like MX records, which you'll remember are used for email, or "NS" or "SOA" records, which are low-level DNS stuff that you shouldn't touch without knowing pretty clearly what you're doing.

Different registrars have different interfaces, but with most of them make it's reasonably easy to find the bit you need once you know what you're looking for. We also have some links to the appropriate documentation on the sites of popular registrars on our help pages.

One thing that does sometimes confuse people is that many registrars only require you to type in the bit that goes before your domain name when specifying a record -- that is, for www.yourdomain.com you would set up a CNAME with a "name" of www and a "value" of webapp-XXXXX.pythonanywhere.com. If you created one with a "name" of www.yourdomain.com, then the CNAME would actually point the hostname www.yourdomain.com.yourdomain.com at PythonAnywhere, which would be unhelpful :-)

Sometimes people are told by their registrar that instead of setting up a CNAME, they need to provide a name server (or even two name servers). Oddly, this sometimes happens even when the registrar in question really does support CNAMEs. Without wanting to call any registrar out in particular, if you are told that, but your registrar appears on the list on our help pages then the customer service person you're talking to is a little confused, and you should check out the documentation that we link to.

But some registrars really do not allow you to set up CNAMEs; they require you to specify name servers. This can happen, for example, with some country-level domains. Explaining what that means is what we'll move on to next.

Domain registrars versus DNS and name servers

Providing the ability to register domains, and providing the DNS stuff for those domains are, technically speaking, two different services. Given that in general it's pretty pointless to own a domain without being able to associate it with an IP address so that you can host a website there, most companies that do domain registration provide both registration and DNS services as a bundle, so you don't need to know about the separation -- but it is there, and if you find yourself having to use a registration-only service, you need to know a bit more about how things work.

The first thing to explain is what a name server is. Remember that earlier we said that when a browser wants to connect to www.yourdomain.com, it asks its local DNS server -- the one provided by the ISP -- what the IP address is. Obviously, the DNS server needs to find the answer somehow. It might have cached the DNS records for the hostname you're looking for due to some previous request, but if this is the first time it's heard about this hostname, it will have to pass the query on to a different server. This is the "authoritative nameserver" for the domain; different domains will have different nameservers set up for them. In a normal setup, when you buy yourdomain.com, there will have to be one or more computers somewhere on the Internet who, when asked for the details of www.yourdomain.com, can give the official answer -- a CNAME, an A record, or whatever. So if the DNS server belonging to your ISP doesn't know the IP address for www.yourdomain.com, it needs to ask the nameserver that is authoritative for yourdomain.com for the answer.

You might wonder how your ISP's DNS server knows how to find out what those nameservers are -- how do they know which nameserver is authoritative for yourdomain.com? The answer is that they ask the nameservers that are authoritative for "one level up" -- that is, if they want to know which nameservers are authoritative for yourdomain.com, they'll ask the nameservers that are responsible for .com to tell them. Likewise, if asked "which nameservers are authoritative for yourdomain.co.uk?", they would ask the nameservers that are authoritative for .co.uk, which might first require them to ask who is authoritative for .uk in order to make that query. (You might in turn wonder how the whole thing gets bootstrapped -- how the DNS server works out the nameservers that are authoritative for these top-level domains like .com and .uk. The answer is that there are some special servers that handle that -- that is, servers that are right at the top of the tree, and are authoritative for the . "root" domain we mentioned a while back.)

So -- when you register a domain name with a normal "bundled" registrar like GoDaddy, they will do two things:

  • Register you as the owner of the domain with the organisation who is responsible for keeping track of such things (which will be a different one for each top-level domain -- Verisign for .com, Nominet UK for .uk, and so on)
  • Set up one or more of their own nameservers so that they can give authoritative answers to questions about your domain.
  • Tell the authoritative nameservers one level up from your domain (that is, the .com ones for yourdomain.com) that their -- that is, the registrar's -- nameservers are authoritative for your domain.

Once that's done, you're all set -- you can use their web interface to update your DNS settings, and when you do that they'll pass the changes on to the appropriate nameservers.

However, if you are using a registrar which doesn't do DNS for you, they won't do the second two steps. They'll just register the domain. They will, however, have a way to tell the one-level-up nameservers which nameservers are authoritative for your domain -- but you'll need to provide them with the names of some servers in order for them to do that.

Setting up your own nameserver is really tricky, but luckily you don't need to do that. There are free services out there that can handle all of it for you, so you just need to sign up with one of them. They'll provide you with the addresses of some nameservers that you can then pass on to your registrar, and a web-based interface so that you can set up all of your DNS records like the CNAME and so on. One that consistently get good reports from our users is the FreeDNS service from Namecheap.

Conclusion

Hopefully that was all reasonably clear. You now know the basics of how DNS works, and how it interacts with domain registration. If you've got any questions, please leave them in the comments below -- and additionally, if you think there are other things that you'd like us to add to this post (or a future one), please do let us know!


Page 1 of 18.

Older posts »

PythonAnywhere is a Python development and hosting environment that displays in your web browser and runs on our servers. They're already set up with everything you need. It's easy to use, fast, and powerful. There's even a useful free plan.

You can sign up here.