Google's Python engineers have launched a new project called Unladen Swallow, which aims to bring a major performance boost to the Python programming language by making runtime speed five times faster. The project is being implemented as a branch of the conventional CPython runtime and will be fully source-compatible with regular Python applications and native extensions. This will make it possible to eventually merge the improvements into Python trunk.
The goal of the Unladen Swallow project is to use LLVM, the Low Level Virtual Machine compiler infrastructure, to build a just-in-time (JIT) compilation engine that can replace Python's own specialized virtual machine. This approach offers a number of significant advantages. As the developers describe in the project plan, the project will make it possible to transition Python to a register-based virtual machine and will pave the way for future optimizations.
Adopting LLVM could also potentially open the door for more seamlessly integrating other languages with Python code, because the underlying LLVM intermediate representation is largely language-neutral.
The Unladen Swallow project also encompasses several other initiatives to improve other characteristics of the runtime and take advantage of modern hardware. Parallelization and higher utilization of multicore processors is one area where the developers are placing strong focus. They are considering the possibility of implementing a concurrent garbage collector, for example. They are very serious about fixing multithreading in Python and hope to slay the Global Interpreter Lock, a mechanism that provides thread safety but imposes limitations on the kind of scalability improvements that applications can get from concurrency.
Although the project is at a very early stage of development, an experimental prototype is already available. The source code can be downloaded from the project's version control system at the Google Code site. The current implementation is said to already provide a performance increase in some specific testing scenarios, but it's still only a flesh wound compared to the improvements that are coming.
For their second milestone, which is scheduled for the second quarter of 2009, they hope to replace the Python virtual machine with a functionally-equivalent LLVM-based implementation.
The Unladen Swallow developers hope that the project will eventually make it possible to use Python instead of C for a wider range of performance-sensitive tasks. They are building it on top of Python 2.6 (2.x is not dead yet) because it will allow many existing projects to adopt it and will also provide a clean glide path to Python 3.
Python is gaining some traction in enterprise environments and it's used extensively by Google within the company's own infrastructure and its hosted App Engine system. Google's investment in improving Python performance will make the language even more compelling for rapid web application development.