Improving PythonAnywhere's File Storage System


UPDATE 2024-11-05

As of today, we have migrated all of our US storage systems over to newer infrastructure. We’ll post again with more details about this migration once everything has had a week or so to bed in, but since we did the equivalent migration on our EU systems a few months back, we have had no issues at all there. So (touch wood) we’re feeling quietly confident :-)

Original post

PythonAnywhere has been around for over 10 years, and as our platform continues to grow with tens of thousands of users, we’re committed to keeping it in top shape. Part of this involves upgrading some of the older parts of our infrastructure, with a special focus on our file storage servers – some of the oldest systems we have.

What You Might Have Noticed

If your files are stored on one of the older, misbehaving machines, you may have experienced issues like:

  • Your files not being visible
  • Your web app being down
  • In some cases, the entire PythonAnywhere system might feel slower or even become temporarily inaccessible

We understand how frustrating these symptoms can be, but rest assured we’re on top of it.

How We’re Handling It

Over the past year, we noticed that the m3-series AWS instances we use for file storage have become less reliable. To improve things, we’ve been migrating to newer m5-series instances, which offer significant boosts in reliability and performance.

We’ve implemented a solid monitoring system to keep a close eye on our infrastructure, and our developers are on call 24/7 to respond quickly to any problems. If a server starts misbehaving, we have a tried-and-tested procedure to fix it, which usually takes about 10 minutes to restore normal operations.

What’s Next

We’ve already completed the file system upgrade on our EU-based systems and saw great results. Now, we’re preparing for a major migration on our largest, US-based system (www.pythonanywhere.com), which serves the majority of our users.

Because this process involves syncing and backing up data for tens of thousands of users, it takes time. While we can’t provide an exact timeline for when the migration will be done, we’ll be sure to announce it once it’s ready.

Thanks for your understanding and patience as we work hard to keep PythonAnywhere running smoothly and reliably for everyone!

comments powered by Disqus