Filesystem performance problems

A number of people (including ourselves while we’re dogfooding) are seeing very slow disk access on PythonAnywhere at the moment.

The problem is twofold:

  • A smaller cause: we moved to Ubuntu as our underlying base operating system recently, and this appears to be more disk-intensive than our older Debian-based system.
  • The big problem: our system currently uses NFS to share each user’s files between their web applications, consoles, and scheduled tasks. This gives a bottleneck which hasn’t been a huge problem in the past, but is becoming quite serious as more people join PythonAnywhere.

The OS upgrade problem may be fixable in the short term – we’re working to identify what’s causing the greater disk access – but the bigger problem will need a bigger fix. We’re benchmarking a number of alternative filesystems (the current leader seems to be GlusterFS) to see which is most likely to handle scaling better, and will make the move as soon as we can.

Please bear with us!

comments powered by Disqus