The PythonAnywhere Newsletter, December 2016

Welcome to our Christmas newsletter! Featuring a selection of the Internet's very worst Christmas-themed animated Gifs!

snow with christmas tree in foreground

Here's what we've been up to:

Is it called "Reindeer Shaving" at this time of year?

dancing reindeer. sigh.

One change you might have noticed is that the Consoles page now has little Xs for killing consoles. The original idea was just to change them from being links that cause a page refresh to being ajax calls, which would let you kill multiple consoles at the same time. Somehow though, that small user interface tweak turned into the whole office deciding to treat that How it feels to learn JavaScript in 2016 comedy blog post as if it were an instruction manual, and we have now spent several days knee deep in React, ES6, promises, webpack, npm, Enzyme, fetch, promises, promises, and many, many more. Still, by the end of it, it all worked, and we have to conclude that ES6 is much nicer to work with than horrible old javascript.

Deep Tarpit

santa throwing a sack around

Santa needs some names for his "naughty or nice" list, and we've got some ideas for him!

Our tarpit has always been a soft limit, so going over your CPU allowance isn't the end of the world, it just slows your processes down, so that people who are within quota get first dibs on the precious cpu-cycles. Some people are taking the mick though, and we've now built some tools to detect these "deep tarpit" users, and kill their processes. So, more CPU-time for the good users, and naughty users now get the lowest priority of all, "kill".

The inside scoop from the forums

animated xmas tree

New modules

Santa on a reindeer

Although you can install Python packages on PythonAnywhere yourself, we like to make sure that we have plenty of batteries included.

  • We have been switching people over to the new dangermouse image! Send us an email if you want to switch.

  • You may have noticed a InsecurePlatformWarning when using pip > 9.0. This is a bug/regression in the pip code for python versions 2.7.9 or lower. We are keeping track of the issue and here are some ways to work around it for now.

New whitelisted sites

santa spinning on a rugby ball or something??

Paying PythonAnywhere customers get unrestricted Internet access, but if you're a free PythonAnywhere user, you may have hit problems when writing code that tries to access sites elsewhere on the Internet. We have to restrict you to sites on a whitelist to stop hackers from creating dummy accounts to hide their identities when breaking into other people's websites.

But we encourage you to suggest new sites that should be on the whitelist! Our rule is, if it's got an official public API (which means that the site's owners are encouraging automated access to their server) then we'll whitelist it. Just drop us a line with a link to the API docs.

Here are some sites we've added since our last newsletter:

.okta.com
.unwiredlabs.com
app.datadoghq.com
api.discogs.com
api.test.netbanx.com
api.test.paysafe.com
api.tinychat.com
apis.vworld.kr
cps.combain.com
epayment.bbs.no
epayment-test.bbs.no
repo.continuum.io
services.arcgisonline.com
steamcommunity.com
timesofindia.indiatimes.com
vizier.u-strasbg.fr
www.datapowerlearning.com
www.nbrb.by
www.rediff.com

What's coming up next year?

Over Christmas we're working on a few small tweaks and improvements, and for early next year, we're looking to start work on a feature that many of you have been requesting for a long time. But shush! No spoilers ;)

Thanks for reading our newsletter! Tune in the same time next month (ish) for more news from PythonAnywhere.


System update this morning -- why it took so long

This morning we deployed a new version of PythonAnywhere -- we've blogged about the new stuff that you can see in it, and this post is a run-down on why it took longer than we were expecting.

Our normal system updates tend to take between 15 and 20 minutes. Today's was meant to take about 40 minutes -- more about why later -- but it wound up taking over an hour and forty minutes. That definitely warrants an explanation.

Situation normal

It's worth starting by explaining how a normal system update works. There are three main classes of servers that make up a PythonAnywhere cluster:

  • File storage servers -- two types of server, one of which stores the files in your private file space, and one that stores and processes system logs (like the ones you can access from "Web" tab).
  • Database servers -- for MySQL and Postgres.
  • "Ephemeral" servers -- basically, everything else. Loadbalancers, web servers, console server, servers for IPython/Jupyter notebooks, a "housekeeping" server that does admin stuff like sending out emails, and so on.

In a normal system update, we replace just the ephemeral servers. The file storage and database servers are pretty simple -- they run the minimum code required to do what they do, and rarely need updating.

Our process for that is:

  • The day before, start up a complete new cluster of ephemeral servers, hooked up to the existing file and database storage servers, but without anything being routed to it.
  • When we get in early for the deploy, we start up a server that just responds to all requests with a "system down for maintenance" response, then switch all of our incoming traffic over to it.
  • Then we stop the old ephemeral servers, and run any database migrations we need to apply to the Django code that runs the main PythonAnywhere website.
  • We set things up so that when we access PythonAnywhere from our own office network, we bypass the "system maintenance" server and see the new cluster, and then we sanity-check to make sure everything's OK.
  • If it is, then we reroute all traffic to the new cluster, and we're done.

All of that tends to take between 15 and 20 minutes.

Sometimes, we need to update the file storage servers. That's a slightly more complicated arrangement.

  • When we start up the new cluster the day before, we create completely new storage servers as well as ephemeral servers. But we don't hook the new storage servers up to the storage itself. Storage is contained on Amazon EC2 Elastic Block Storage (EBS), which can be moved from machine to machine but can only be attached to one machine at any given time -- so we leave it on the old storage servers initially.
  • When we do the deploy, after we've stopped the old ephemeral servers, we need to:
    • disconnect the old EBS devices from the old storage servers
    • connect them to the new storage servers
    • start up the various system services that turn those block devices into a usable filesystem
    • run some system checks to make sure everything's OK and the new ephemeral servers can see the storage on the new storage servers.
  • Once all that is done, then we check out the new system end-to-end, and, if all is well, go live.

Because all of these steps are scripted, the extra work doesn't normally take all that long; maybe half an hour in total. Worst case, 40 minutes.

Today's system update was even larger than that, however.

Virtual Private Clouds

We've recently started hitting some limits to what we can do on AWS. When we started using the service, they had just one kind of server virtualization -- for those who are familiar with this stuff, they only supported paravirtualization (PV). But now they support a new kind, Hardware Virtual Machines (HVM). This new kind has a large number of benefits in terms of the kinds of servers supported and their performance characteristics in an environment like PythonAnywhere.

The problem is, in order to use HVM, we had to move PythonAnywhere out of what AWS call "EC2-classic" and into what they call a Virtual Private Cloud. (Use of a VPC is not normally technically required for HVM, but there was a nice little cascade of dependencies that meant that in our case, it was.)

We'd been testing PythonAnywhere-in-a-VPC in our development and integration environments for some time, and by this system update we felt we were as ready as we ever would be to go ahead with it.

We decided we'd move all of the servers into the VPC in one go. There's a thing called "ClassicLink" which could have allowed us to leave some servers in EC2-classic and move others to the VPC, but using it would have been complex, easy to get wrong, and would only have stored up trouble for the future. So we decided we'd do one of our updates where we'd replace the file storage server as well as the ephemeral ones. The night before, we'd start a full set of new servers inside a VPC, and then when we switched over to them, we'd have moved into it.

But there was a wrinkle -- as well as moving the file storage servers, we'd need to move the database servers. We (rightly) didn't think moving Postgres would be a huge deal, because we manage our own Postgres infrastructure. But moving the MySQL databases would require us to convert them over to the VPC using Amazon's interface.

We timed this with a number of test database servers, and found it took between five and ten minutes. We checked with Amazon to make sure that this was a typical time, and they confirmed.

So -- half an hour for a full cluster replacement, plus ten minutes at the worst case for the MySQL move, forty minutes in total.

What could possibly go wrong?

What went wrong

There were a few delays in today's update, but nothing too major. The code to migrate the EBS storage for the storage servers wasn't quite right, due to a change we'd made but hadn't tested enough, but we have detailed checklists for doing that process (after all, keeping people's data safe is the top priority in anything like this) so we were able to work around them. Everything was looking OK, and we were ready to go live after about 50 minutes -- a bit of a delay, but not too bad.

It was when we run the script to make the new cluster live that we hit the real problem.

Earlier on I glossed over how we route traffic away from the old cluster and over to the site-down server, and then back again when we go live with the new one. Now's the time for an explanation. AWS has a great feature called an "Elastic IP address", or EIP. An EIP is an IP address that's associated with an AWS account, which can be associated with any server that you're running. Our load-balancer endpoints all use EIPs. So to switch into maintenance mode, we simply change all of our EIPs so that instead of being associated with the old cluster, they're associated with the site-down system. To switch back, we move them from the site-down system to the new cluster.

So we ran the script to switch from the site-down system to the new cluster, and got a swathe of errors. For each EIP it said

You must specify an allocation id when mapping an address to a VPC instance

A bit of frantic googling, and we discovered something: EIPs are either associated with the EC2-classic system or with the VPC system. You can't use an EC2-classic EIP in a VPC, or vice versa.

There is a way to convert an EC2-classic EIP to a VPC one, however, so we started looking in to that. Weirdly, the boto API that we normally use to script our interactions with AWS doesn't support that particular API call. And there's no way to do it in the AWS web interface. However, we found that the AWS command-line tool (which we've never used before) does have a way to do it. So, a quick pip install aws-cli, then we started IPython and ran a script to convert all of our EIPs to VPC:

In [4]: for name, eip in eips.items():
   ...:     print name
   ...:     !aws ec2 move-address-to-vpc --public-ip $eip
   ...:     
web-9
{
    "Status": "MoveInProgress"
}
web-3
{
    "Status": "MoveInProgress"
}
consoles-4.pythonanywhere.com
{
    "Status": "MoveInProgress"
}
web-10
{
    "Status": "MoveInProgress"
}
consoles-6.pythonanywhere.com
{
    "Status": "MoveInProgress"
}
web-5

An error occurred (AddressLimitExceeded) when calling the MoveAddressToVpc operation: The maximum limit on Elastic IP addresses has been reached.

Uh-oh. We have a couple of dozen EIPs associated with our account -- we had a vague recollection of having had to increase the limit in the past. But it looked like (and we confirmed) that for some reason there's a completely separate limit of EIPs for VPCs -- the one we had previously increased only applied to EC2-classic.

AWS tech support to the rescue

The advantage of spending thousands of dollars a month on AWS, and paying extra for their premium support package, is that when stuff like this goes wrong, there's always someone you can reach out to. We logged a support case with "production systems down" priority, and were on a chat with a kindly support team member called George within five minutes. He was able to confirm that our VPC EIP limit was just five, and bumped it up enough to cover the EIPs we were moving across.

Unfortunately limit increases like that take some time to propagate across the AWS system, so while that was happening, we took a look at the details of the error that we'd got originally -- again, it was

You must specify an allocation id when mapping an address to a VPC instance

The boto API call that we were using took two parameters -- the IP address we wanted to move, and the instance ID of the server we wanted to move it to. Looking at the EIPs that we had successfully moved into the VPC world, they had a new number associated with them -- an "Allocation ID". And it appeared that boto now required this ID as well as the EIP's actual IP address when it was asked to associate an EIP.

So we reworked the code so that it could do that, and waited for the limit increase to come through.

Finally, it did, so we reran our little IPython script. All of the EIPs moved across. A bit of further scripting and we had allocation IDs ready for all of the EIPs, and could re-run the script to switch everything over and make the new cluster live. And everything worked.

Phew

Lessons learned

It's hard to draw much in the way of lessons from this. One obvious problem is that we didn't know that EIPs have to be moved into the VPC environment, and we didn't know that because our testing environment doesn't use EIPs. That's clearly something we need to fix. But it's unlikely that we would have spotted the fact that there was a different limit of inside-VPC EIPs associated with our account -- and because we had to move the EIPs over while the system was down, we wouldn't have discovered that until it was too late; this morning's update would have taken an hour and a half rather than an hour and forty minutes, which isn't a huge improvement.

I suppose the most important lesson is that large system updates sometimes go wrong -- so we should schedule them to happen later. The quietest time of day across our systems as a whole is from about 5am to 8am UTC.
Things ramp up pretty quickly after that, as people come in to work across western Europe. For a normal update, taking half an hour, 7am is a perfectly OK time to do it. But for future big changes, we'll start off at 5 or 6.


New release, ft. 2016-style Javascript and the Deep Tarpit.

The main driver for our release this morning was a move, behind the scenes, to put our servers into a "VPC", and despite the fact that it'll have no visible impact, it was a significant change to the infrastructure, and not without its challenges, as you'll hear in more detail from Giles later :)

One Yak, Fully Shaved.

new consoles list ui screenshot

One change you might notice is that the Consoles page has changed, and includes some little red Xs for killing consoles. The original idea was just to change them from being links that cause a page refresh to being ajax calls, which would let you kill multiple consoles at the same time. Somehow though, that small user interface tweak turned into the whole office deciding to treat that How it feels to learn JavaScript in 2016 comedy blog post as if it were an instruction manual, and we have now spent several days knee deep in React, ES6, promises, webpack, npm, Enzyme, fetch, promises, promises, and many, many more. Still, by the end of it, it all worked, and we have to conclude that ES6 is much nicer to work with than horrible old javascript.

The Deep Tarpit

The tarpit is one of the key ways we balance the resource needs of our various users. What happens when you exceed your CPU quota is that your processes still run, but they get a lower priority compared to people who are still within the amount they paid for. That's been working fairly well, but as with all things, we notice there's a power law at work, and there are a small number of users who regularly go massively over their tarpit limit. We've added some code that will automatically kill processes of these kinds of users, and send them a friendly notification email. Bad programmer! No biscuit.

We've also added some code to limit the amount of output in consoles, so that kids (and adults) whose first Python program is while True: print("farts") will have less of an impact on the system. Although plenty of farts will still be printed, fear ye not.

The upshot of all that should be that console performance will hopefully be a little more consistent from now on.

Better support for non-English keyboards

We do our best to avoid the classic Anglocentric, parochial laziness of imagining that the world ends with ASCII, but it takes work! For a while we've known that users on certain operating systems with certain keyboard types & layouts would have difficulties entering certain text into our consoles. So we've rolled out the ability to switch from hterm to xterm.js for our client-side terminal emulation.

If you'd like to try it out, give us a shout and we can switch it on for you. NB - keyboard shortcuts for copy + paste will be different, it'll be Ctrl+Ins / Shift+Ins instead of Ctrl-C & Ctrl-V.

General bugfixes and security fixes

And the usual retinue of bug fixes. Some of which were (minor) security fixes, incidentally, as reported by some enthusiastic security researchers. Find out more about our bug bounty if that describes you!


The PythonAnywhere newsletter, November 2016: Two-factor auth and a new system image

We try to get a newsletter out every month, but sometimes we just get too distracted working on our latest and greatest features to manage it. It wasn't that we were all out in Norway doing an opera, honest :-)

Here's what we were up to:

Two Factor Auth

Something you know, something you own, something borrowed, something blue...

We were very pleased to roll out two-factor authentication, meaning that you can now add a second step to your account login if you want extra security. We support the Google Authenticator token generator. More details on your accounts tab.

The inside scoop

  • A couple of people were being caught by an error in FileZilla SFTP, which happens if anything in your .bashrc echoes anything to stdout -- a particularly sneaky bug to track down. (although the most common problem with SSH is still the case-sensitive nature of usernames...)

  • Ping! Our own Harry gives some tips on disabling console chimes

  • Bossman Giles gives a quick rundown of how to do blue green deployment on PythonAnywhere

  • willpaycoin was worried about the Dirty Cow (geddit? a copy-on-write vulnerability. harhar). But he needn't have, our ever-vigilant cow security brigade were on it.

New batteries included? A whole new image more like!

Although you can install Python packages on PythonAnywhere yourself, we like to make sure that our preinstalled batteries included are nice and up-to-date. A few weeks ago we released a whole new system image which we're calling "dangermouse", which is the default for new users. If you are still on the "classic" image (see? it's alphabetical!) and want to switch, drop us an email and we'll upgrade you.

New whitelisted sites

Paying PythonAnywhere customers get unrestricted Internet access, but if you're a free PythonAnywhere user, you may have hit problems when writing code that tries to access sites elsewhere on the Internet. We have to restrict you to sites on a whitelist to stop hackers from creating dummy accounts to hide their identities when breaking into other people's websites.

But we really do encourage you to suggest new sites that should be on the whitelist. Our rule is, if it's got an official public API, which means that the site's owners are encouraging automated access to their server, then we'll whitelist it. Just drop us a line with a link to the API docs.

Here are some sites we've added since our last newsletter:

  • api.mailgun.net
  • auth0.com
  • botframework.com, pandorabots.com
  • api.easypost.com
  • api.hipchat.com
  • api.skype.net/com
  • api.wikimapia.org
  • backend.deviantart.com
  • login.microsoftonline.com
  • strawpoll.me
  • www.hipchat.com
  • xboxapi.com

So if you've ever dreamed of building a weather-forecasting chatbot that posts deviantart images on skype directly from your xbox, now's the time!

A few minor things

Behind the scenes we made some fairly hefty infrastructure upgrades to the way our fileservers and web servers balance load, but that shouldn't be visible, except in increased reliability perhaps. There were a couple of minor security patches, and we got print preview working on Ipython Notebooks, which I'm sure everyone was just dying to see.

That's about it! Thanks for reading, and tune in at the same time next month (ish) for more exciting news from your favourite Python PaaS.


Nitrous.io shuts down

We were sad to hear about the sudden departure of our frenemies* at Nitrous.

Whenever something like this happens, there's a conversation about the reliability of SaaS services in the face of VC pressure -- do you really want to trust your business to a company that has VCs breathing down their neck setting deadlines for exits, and ready to pull the plug if things take a dip?

So we want to reassure you that we haven't taken on any big VC money. And although we do occasionally wonder, from our cramped little London office above a Burrito shop**, what the lives of developers in startups with 7-figure investments and swanky San Francisco offices with their table-tennis tables and elaborate trousers, we mostly don't resent it much. Considering we're now cashflow-positive and our investor (Hi Robert!) has just guaranteed the sole cash injection we need to achieve actual-for-reals-profitability, and that that investment is two orders of magnitude less than what Nitrous took on, well, let's say we hope we're a reasonably low-risk bet as far as our customers are concerned, and we plan to be around for a while.

Anyways, so much to say that, if you were using Nitrous to code Python, and you still fancy having a place in THE CLOUD to store your stuff, head on over to our site, and we'll be happy to see you.

So, in honour of our defunct friends at Nitrous, why not enjoy this naughties dark DnB classic by the unmistakeable Bad Company. Rinseout!

* yes, "frenemies", its a thing. if you're cringing, rest assured that, being British, I cringed the first time I heard this word too. So. American. But like all things Silicon Valley, there's a nugget of truth in it, and more than that. competitors can be friends (someone should remind politicians of this), and particularly when you're out creating new markets, a competitor doing a good job can genuinely increase the size of your own customer pool. even when they don't die!.

** actually it's now a Vietnamese. Nice Banh-mi, delicious roast belly pork, could do with being a little more generous on the meat portions though


Today's upgrade: improving websites, better security

This morning's system update went smoothly, and we've made a couple of great changes :-)

Improved website routing

This one should be pretty much transparent to you, but we've revamped the way we route requests for the websites that we host; this should speed things up for some people.

Noisy neighbours always cause problems, in the real world and on the Internet. When someone writes a website that hogs system resources on PythonAnywhere, sometimes it can impact other people who happen to be on the same server. Naturally, we monitor the system, and when we find a particularly badly-behaved website we notify its owner by email and ask them to fix it -- or in extreme cases, if it's causing serious problems, we shut it down. But that's far from ideal.

Today's update makes that all a lot better. We've given ourselves, the system administrators, fine-grained control over where websites run. So now, if we see a website that's causing slowdowns for other users, as well as notifying the owner so that they can fix it, we can move it right away onto a server where it won't impact other people. We're calling it "putting them in the sin bin"...

Security is important

...as people have reminded us frequently in suspiciously-similar Tweets. And they're right! So we've implemented two-factor authentication, using Google Authenticator (or any other TOTP app). It's currently going through a short internal-only testing process (in other words, we've switched it on for our own accounts to see if it breaks anything) and if all is well, we'll provide it as an option for everyone next week.

On the subject of security, we've also fixed a couple of bugs: Nikhil Mittal reported a CSRF issue on PythonAnywhere that would have allowed an attacker who knew both your username and the internal database ID of one of your scheduled tasks to delete that task, if they tricked you into visiting a web page that they controlled while you were logged in to PythonAnywhere. It wouldn't have given the attacker access to any of your data, but it could have been really irritating, and we're glad it was reported so that we could fix it. Bug: fixed. Bug bounty: paid. Nikhil also reported some issues around our email confirmation system, which we've also fixed.

...and the rest

As always, we've put in a number of user interface tweaks, including fixing the print preview on IPython notebooks.

That's it!

Thanks for reading, and for using PythonAnywhere :-)


The PythonAnywhere newsletter, September 2016

We try to get a newsletter out every month, but in August we were all too busy working on our latest and greatest features to manage it. It wasn't that we were all out sunning ourselves, honest :-)

Here's what we were up to:

A new system image

This is actually quite a big deal, even if it might seem a bit dull at first glance. Accounts that have been created since 8 September have more recent versions of all of our batteries included. For example, while older accounts have Django 1.3.7 installed for Python 2.7, newer accounts get Django 1.10. The old image is called "classic" because we couldn't think of a better name, and the new one is called "dangermouse" because we remember the children's television of the UK in the 1980s much too well.

You can see how the two images differ by trying different filters on the batteries included page (which is now a shiny, JavaScript-driven, filterable list rather than a simple static page). If you have an older account and want to switch to the new system image, just drop us a line at support@pythonanywhere.com. Be warned: if you have old code that depends on the old Python packages, then it might stop working. So check your code first.

So, why did this take so long? And why do we think it's a big deal? Well, one of the development practices we follow here is what we call "Embarrassment-Oriented Project Management". In a nutshell, this says "if you're trying to work out what you need to work on next, ask yourself what makes you blush when you have to explain it to someone"... One of the things that was a tad embarassing was that we only supported Django 1.3.7 for Python 2.7. Sure, we supported later Django versions for later Python versions, but still.

But we had a problem. Until now, everyone on our site had to use the same system image, with the same versions of everything. We could add new things -- which is why, when Python 3.5 was installed, it had recent versions of all the system packages. But we couldn't upgrade anything for older Python versions, because it would have meant breaking any code that people had running that depended on the old system models. If you had a Django 1.3.7 website, and we upgraded to 1.10, then your site would suddenly stop working, and you'd have to change the code to make it work again. That would have been rather rude of us, so we didn't do it.

So what we needed to do was develop a way for different people to have different system images. For a while, we thought that Docker was the solution -- but unfortunately (for reasons related to scalability in our specific environment) it turned out not to be. So we've extended our virtualization system to support multiple system images.

More images will come in the future, once we've thought of an appropriately-irreverent name starting with "E" for the next one.

Right, enough of that! What else?

Let's Encrypt

Let's Encrypt is a project supported by the Linux Foundation to help secure the web by providing free HTTPS certificates for any domain you own. Its certificates are just as good as ones that you pay for in almost every way -- the only downside is that you have to renew them once every few months rather than once a year. (That's also a security feature, of course, because if someone steals your private key then they can only impersonate you for a few months rather than a year. But it is a little inconvenient.)

On PythonAnywhere, you already get free HTTPS for your websites that are subdomains of pythonanywhere.com -- you don't need to do anything -- but if you have a custom domain, you need to give us a certificate to install. We've been honing our process to make this as easy as possible, and we're now pretty confident in them. So now there's no excuse to not secure your website -- just follow the instructions here.

The inside scoop from our forums

New modules

Although you can install Python packages on PythonAnywhere yourself, we like to make sure that we have plenty of batteries included.

Because of the new system image, we've not actually added any new modules this time around -- but we've updated everything we could to the latest version for the new "dangermouse" image. You can compare and contrast the different version in each image over on the new, shiny, interactive batteries included page.

New whitelisted sites

Paying PythonAnywhere customers get unrestricted Internet access, but if you're a free PythonAnywhere user, you may have hit problems when writing code that tries to access sites elsewhere on the Internet. We have to restrict you to sites on a whitelist to stop hackers from creating dummy accounts to hide their identities when breaking into other people's websites.

But we really do encourage you to suggest new sites that should be on the whitelist. Our rule is, if it's got an official public API, which means that the site's owners are encouraging automated access to their server, then we'll whitelist it. Just drop us a line with a link to the API docs.

Here are some sites we've added since our last newsletter:

  • graphenedb.com -- Neo4j-as-a-service
  • rememberthemilk.com -- a friendly to-do list
  • wunderlist.com -- and another to-do list!
  • aavso.org -- The American Association of Variable Star Observers.
  • api.algorithmia.com -- a marketplace for algorithms.
  • api.bandsintown.com -- who's playing near you?
  • api.desktoppr.co -- desktop-wallpapers-as-a-service :-S
  • api.donorschoose.org -- support the poorest schools in the United States
  • api.msg91.com -- an Indian SMS provider.
  • api.sparkpost.com -- an email provider
  • api.thetvdb.com -- an open database for television fans
  • api.virwox.com -- an exchange site for virtual world currencies
  • api.wunderground.com -- real-time weather information
  • avwx.rest -- the aviation weather REST API.
  • datos.madrid.es -- open data from the city of Madrid, Spain.
  • dev.tescolabs.com -- the developer portal for a UK supermarket (!)
  • food2fork.com -- a recipe site.
  • myanimelist.net -- pretty much what you'd expect from the name...
  • open.live.bbc.co.uk -- open data from the BBC
  • open.mapquestapi.com -- open data from MapQuest.
  • opendata.bcn.cat -- open data from the city of Barcelona, Spain
  • opengraph.io -- get Open Graph data about websites using an API -- even for sites that don't explicitly support it.
  • openlibrary.org -- free classic eBooks
  • rss.imdb.com -- RSS feeds from the Internet Movie Database
  • togows.org -- a collection of bioinformatics web services.
  • www.warlight.net -- an online game similar to Risk.
  • www.zillow.com -- US real-estate listings.

And that's it

Thanks for reading our newsletter! Tune in the same time next month (ish) for more news from PythonAnywhere.


Latest deploy: Some nice new features and a surprise

Rename web apps

Yes, we know it's been a long time coming, but now you can rename your web apps (and, as a result change the domain they're served from) right on the web app setup page. Look for the little edit pencil icon next to your web app address.

Students can share with teacher

We've made it easier for students to share their consoles with their teacher.

List invoices on accounts page

For those of you that may be wondering how much of your hard-earned money you've spent on PythonAnywhere, we've added a list of all of your invoices to the Account page.

PDF export for Jupyter notebooks works

A helpful user pointed out that "Download as PDF" wasn't working in Jupyter notebooks on PythonAnywhere. So we fixed it.

"bash console here" on editor page

If you're ever editing a file and want to open a Bash console in the same directory as the file, now you can.

General security, usability and stability fixes

As usual. This is usually where we put all the fixes for bugs that are too embarrassing to list.

Something great that we're not telling you anything about

until we've tested it ourselves.


Back to school tips for teachers, from PythonAnywhere

Dear teachers,

If there's one thing we know, it's that teachers (and students) love, it's being reminded that the holidays is that the holidays are coming to an end. Hooray!

Here's a few ideas and pointers for some of the things that we hope will make your life, as a teacher, easier.

1. PythonAnywhere means all your students have exactly the same setup

If you're already a PythonAnywhere user, you'll already know this. But, any teacher that's struggled with getting several different students, with different hardware and operating systems, all set up and working for a basic programming class, know how much time is easily lost.

With PythonAnywhere, everyone has the same kind of terminal, the same operating system, the same versions of Python, and the same standard installation. And you can jump on and view exactly what your students are doing.


2. PythonAnywhere makes it easy to go beyond the basics, and use external libraries

We have hundreds of packages preinstalled, and it's easy to install new ones. Here's a few examples of packages, and the associated topic which you could build a lesson around:

  • Scraping with requests and beautiful soup
  • Accessing the google maps api with google-api-python-client
  • Number crunching with numpy/pandas/scipy/scikit-learn
  • Use sqlalchemy to access our provided mysql/postgres databases
  • Best practices such as version control and isolated environments with git/mercurial/virtualenv
  • Python standard library modules such as sqlite3, unittest, re, pickle, pdb

Here's our full list of preinstalled "batteries included"

And here are our instructions for installing new modules

3. PythonAnywhere makes it easy to distribute and collect assignments from your students.

Because all your students' home directories are available to you in a Bash console, it's easy to copy files from your accounts to theirs, and back again. Check out the worked example on our help page.

Once you've got the basic idea, your imagination is the only constraint on what you could achieve by building little automated scripts to help your classes. Off the top of our non-teacher heads, how about:

  • Checking last updated times, and counting line numbers, in assignments files, to see who's been working and who hasn't

  • Writing a scheduled task to copy assignments across to your own folder at a pre-specified time, thus automating the "deadline" for the assignment

  • Automatically running student scripts and emailing yourself the results, possibly even grading assignments automatically.

If any of that piques your curiosity but you're not quite sure how to go about it, don't hesitate to get in touch, and we'll happily point you in the right direction.

4. PythonAnywhere makes it really easy for your students to build a real, publicly available website

You can use python web frameworks such as django, flask, bottle or web2py while we take care of the infrastructure behind it. No need to worry about server configuration, load balancing, or keeping the operating system patched - we take care of all that for you.

5. We recently introduced Jupyter Notebook support (aka IPython notebook).

These are interactive "notebooks" that let you run bits of code step by step and get instant feedback. It allows you to include explanatory text and data visualizations directly in the notebook and is used by university researchers worldwide.

Many teachers have told us they love the interactive nature of notebooks, with the easy ability to "go back", change things, and see the effect filter down -- something that really helps students with exploratory coding.

Unfortunately notebooks are currently a premium feature, but (read on...)

6. We can create a custom "teacher" plan for you, if you need some paid features for your students.

Currently, free accounts have certain restrictions:

  • Internet access is restricted to a whitelist of sites (https://www.pythonanywhere.com/whitelist/)

  • You cannot use SSH, and your disk & CPU quota is limited

  • You cannot use Jupyter Notebooks

  • You cannot host applications on custom domains (only username.pythonanywhere.com)

However, we have started offering special pricing for teachers, whereby we can lift one or many of the above restrictions, for $1/student/month. Get in touch with us and we'll help you out.

7. You can create your own step-by-step tutorial, integrated into the PythonAnywhere UI.

Ever used one of our little green "tutorials", the step-by-step instructions that sit in green boxes at the top of a pythonanywhere screen?

They're actually open-source, and you can write your own, for your students to use, and we'll integrate it into PythonAnywhere. Get in touch for more information.

That's our top seven, but there's many more little things that (we hope) will make life for teachers easier:

  • You can work on the go from your tablet/ipad
  • We can do bulk creation of accounts for all the students in your class in one go.
  • We can also set up special "exam" accounts, for you to use in test situations, or controlled assessments.

We're always keen to go the extra mile to make sure teachers and students are getting the best out of PythonAnywhere, so don't hesitate to email us, and let us know what you're doing, and how we can help.


Latest deploy: new stylings, editor fixes, and our API beta

Morning all! A lovely day for leave-ing an old server image behind and welcoming in a new, independent, codebase. #brupgrade #brelease #breployment.

Style tweaks, improvements to responsiveness

On a bit of a whim we decided to upgrade to bootstrap 3, so you'll notice slightly different stylings. Flat buttons! Oh-so-3-years-ago. But also, there are some improvements to the way the site displays on mobile and smaller displays, which is nice.

We also upgraded to the latest version of the ace editor, which should bring a few little improvements too, like better vim keybindings, and better support for ipads (you can now scroll, yay!)

API beta

It's not ready for prime-time yet, but we've started work on a PythonAnywhere API. It may end up not being something we publish for general use, and just something for us to use behind the scenes, but if you're keen to take a look, get in touch, and we'll switch it on for you. Currently the API allows you to do stuff to your web apps, namely:

  • create new web app
  • reload web app
  • update webapp settings: virtualenv, static files.

You should bear in mind that anything you build using that api will probably break when we next do a release, we're making no guarantees about backward-compatibility, or that the api will even work as it is. So really it's just for playing around or for the curious for now. Still, email us if you're interested, we'd love to hear from you.

Other changes

  • There's now a UI for updating the working directory and source files location for your web apps.
  • The editor now gives you a useful warning when you try and save if you're over your quota (and it no longer deletes your whole file, which some consider a bonus)
  • We've made some changes to try and make it easier for us to deal with forum spam. ugh.
  • and a few assorted minor bugfixes.

Your comments and suggestions are always welcome. Enjoy!


Page 2 of 15.

« More recent postsOlder posts »

PythonAnywhere is a Python development and hosting environment that displays in your web browser and runs on our servers. They're already set up with everything you need. It's easy to use, fast, and powerful. There's even a useful free plan.

You can sign up here.