I’ve always thought that our browser based consoles are one of the coolest things about PythonAnywhere. But everyone always wants them to be faster. And since we dogfood with them every single day so do we. One of the key bits of our stack, until quite recently, was Socket.IO. It’s a library that abstracts away the different type of socket transport protocols that various browsers support. It helps us support a broader range of browsers without worrying about quirks.
It is a great piece of software and we’re very grateful to the authors. However we have recently replaced it with a competing library called SockJS.
Doing so has increased performance, reduced key press latency, and improved reliability. We did a little bit of benchmarking to demonstrate the difference we’ve noticed by implementing this new library.
We ran two benchmarking tests. The first one was spitting out a large amount of text from the bash console and seeing how quickly the console could deal with it
On our standard machines /etc/bash_completion is 1641 lines long, 56Kb of text.
We ran cat /etc/bash_completion and timed how long it took for the output to
finish. We did this five times for each library and averaged the results.
And the results are…
Socket.IO took 16.2 seconds vs SockJS which took 14.5 seconds to cat the file
So not bad, definitely a consistent improvement. Remember we ran these tests five times each and and the variation was only a tenth of a second for each run.
The second benchmark was the time that we took to type 80 characters of text into a terminal. This is a test of the time that it takes for a character to round trip from the VT100 emulator in the browser, to the bash process on the server and then back to the browser to actually be displayed in the user’s browser. It’s a pretty good test of the real world latency that our users see.
Here Socket.IO took 8.75 seconds and SockJS 6.04 seconds to roundtrip 80 characters.
We used Selenium/Webdriver to run these tests in a reproducible and repeatable fashion. Shaving a couple of seconds off here is MUCH more noticable in real usage. Our consoles feel much snappier and just that bit closer to the experience you would get ssh’ing into a server across the Atlantic.
However this 2.71 second gain per 80 characters is actually a much bigger gain in our test suite. Our old test code used to take 20 seconds to send those same characters. We had to go deliberately slowly in order to avoid issues with characters missing or arriving out of order. Now we just blat them at our console as fast as possible and we are still seeing a reduction in in the number of random failures.
So to summarise. SockJS has demonstrated real world performance gains for our application. Our usage is pretty intensive and any latency really hurts our users so this is a great thing. The modesty of these timing gains hides the much larger gains in reliability and a speed boost for our functional tests. If you do anything with Websockets and performance is important then I would recommend investigating this library. If you want to see it in action visit www.pythonanywhere.com and play with one of our consoles.