Improving PythonAnywhere's File Storage System

UPDATE 2024-11-05

As of today, we have migrated all of our US storage systems over to newer infrastructure. We’ll post again with more details about this migration once everything has had a week or so to bed in, but since we did the equivalent migration on our EU systems a few months back, we have had no issues at all there. So (touch wood) we’re feeling quietly confident :-)

Original post

PythonAnywhere has been around for over 10 years, and as our platform continues to grow with tens of thousands of users, we’re committed to keeping it in top shape. Part of this involves upgrading some of the older parts of our infrastructure, with a special focus on our file storage servers – some of the oldest systems we have.

Read more…

Serving UTF-8 static files? Headers to the rescue (an epic tutorial)!

Imagine there’s a PythonAnywhere user, homer8bc, with poetic inclinations. He wants to serve his newest poem (he believes it’s quite epic) as a static text page. He’s old school — he doesn’t believe in HTML, and as for CSS? Forget it! His friend, S. Yodos, lives in Cyme, while homer8bc resides on Ios island, so in-person communication is difficult…

Read more…

Issues after system maintenance on 2024-09-05

tl;dr

On Thursday 5 September 2024 we performed some system maintenance. It appeared to have gone well, and was completed at the scheduled time (06:20 UTC), but unfortunately there were unexpected knock-on effects that caused issues later on in the day, and further problems on Saturday 7 September. This post gives the details of why we needed to perform the maintenance, what happened, and what we will do to prevent a recurrence.

Read more…

Belated announcement of latest updates

Here is a slightly delayed (and short) run-down of the new stuff that we deployed recently.

The main change for this update is that we have updated the underlying OS running PythonAnywhere to Ubuntu 22.04. This is an LTS release so it will be supported for some time to come. This will not affect user environments, but it is setting us up for a new user environment that should be coming soon.

We have also:

  • Started the process of updating our file servers to be more robust
  • Improved our alerting so that we are alerted to many new forms of failure on PythonAnywhere
  • Made some improvements to the ASGI beta systems and their documentation
  • Fixed a number of security issues
  • Fixed various bugs

Postal code validation for card payments

tl;dr

We recently started validating that the postal codes used for paid PythonAnywhere accounts match the ones that people’s banks have on file for the card used. This has led to some confusion, in particular because banks handle postal code validation in a complicated way – charges that fail because of this kind of error can show up in your bank app as a payment that then disappears later, or even as a charge followed by a refund. This blog post is to summarise why that is, so hopefully it will make things a bit less confusing!

The long version…

Card fraud is, sadly, a fact of life on the Internet. If you have a website that accepts payments, eventually someone will try to use a stolen card on it. If your site is online for some time, hackers might even start using you to test lists of stolen cards – that is, they don’t want to use your product in particular, they’re just trying each of the cards to find the ones that are valid, so that they can use them elsewhere.

We recently saw an uptick in the number of these “card probers” (as we call them internally) on PythonAnywhere. We have processes in place to identify them, so that we can refund all payments they get through, and report them as fraudulent to Stripe – our card processor – so that the cards in question are harder for them to use on other sites. But this takes time – time which we would much rather spend on building new features for PythonAnywhere.

Looking into the recent charges, we discovered that many of them were using the wrong postal code when testing the cards. The probers had the numbers, the expiry dates, the CVVs, but not the billing addresses. So we re-introduced something that had been disabled on our Stripe account for some time: postal code validation for payments. You may be wondering why it wasn’t enabled already, or why it might even be something that anyone would disable; this blog post is an introduction to why postal codes and card payments can be more complicated than you might think.

Read more…

New help page: Playwright

We’ve had an increasing number of people asking us how to use Playwright on PythonAnywhere. Playwright is a browser automation framework that was developed by Microsoft; like the more-established Selenium it’s really useful for testing and web-scraping, and it’s getting a reputation for being a robust and fast library for that kind of work.

Getting it set up to run on PythonAnywhere is pretty easy, but you need to do the installation slightly differently to the way it’s documented on Playwright’s own site; user hcaptcha on our forums worked out what the trick was to making it work, so now we have a new help page documenting how to do it.

CPU resetting issues report: 3 - 5 May 2024

tl;dr

We have a number of background processes that execute periodically on our systems; one of these is the one that resets the amount of CPU used, so that you get a fresh allowance every day. Early in the morning of 2024-05-03, on our US-hosted system, that service failed silently.

Unfortunately, we only realized it was not working on the morning of 2024-05-04. Putting a fix in place required another day.

At the same time, our load balancing system was experiencing a DDoS attack by malicious bots, which led to an overall decline of performance.

For some of our users, who noticed the CPU issue, these two separate events correlated, leading to confusion.

These issues appeared only on our US-based system – users on our EU system were not affected.

Read more…

Five steps to create your own PythonAnywhere AI guru, on PythonAnywhere

AI is the coolest thing in tech right now, but getting an AI-powered website up and running can seem pretty daunting. Luckily, there are a bunch of useful tools to make it easier.

A while back we started seeing if we could use large language models to provide a helpful assistant for PythonAnywhere; we found that the capabilities (and perhaps more importantly, our own AI skills) aren’t quite there yet, but it was a lot of fun, and it felt like it would be a good basis for a new tutorial :-)

So, would you like to create your own personal PythonAnywhere guru – albeit one that occasionally gets things wrong or makes things up? This tutorial shows you how to set up a website that uses OpenAI’s libraries, but can answer questions about our site with more in-depth knowledge than ChatGPT has on its own. We’ll also touch on how to run much simpler AI models on PythonAnywhere itself, without needing to use external APIs. It’s meant as a jumping-off point – you’ll build something and understand it well enough that you can start customizing it to do something you want to do.

Read more…

Outage report 9 October 2023

tl;dr

On 2023-10-09 we had an unplanned outage. While we were preparing our systems for a scheduled system update the following morning, we faced some issues. These in themselves would not have caused problems, but responding to them resulted in the accidental termination of the old, running cluster’s machines at 15:22 UTC. To avoid additional downtime, we decided to do the planned update to recover the service. It took longer than expected, but we were able to get all hosted websites up and running by 17:30 UTC; unfortunately always-on tasks took longer and were only fully working by 21:42 UTC.

Read more…

We're hiring!

Now that we’re part of Anaconda, we’re growing the team so that we can do more, faster :-)

Right now we’re looking for a senior engineer with lots of experience in backend stuff, but an interest in working across the full stack from obscure kernel wrangling, custom Linux container-based virtualization, Django and Flask on the mid-tier, up to TypeScript and React on the front end. There’s even a (tiny) bit of Lua thrown in there.

We’re an Extreme Programming team so you’ll be pairing with other team members from day one. All work is remote (bar occasional team meetups), and we can currently hire people based in the UK or in Germany.

There’s more detailed information about the role on the official Anaconda jobs board, and you can also apply there – or you can also drop us a line at jobs@pythonanywhere.com.