Compiling Python Code with PyPy


I have been watching recently some interesting videos recorded from the last few editions of the PYCON US conference. In particular my passion for (programming) languages  attracted me to those from Alex Gaynor ( a VERY young and talented programmer / presenter) and other members of the PyPy project.  

Now what I had previously understood about PyPy was that it is basically a fast interpreter and JIT compiler for Python code, written in Python. This was intriguing in itself, the idea of using the advanced features of Python to write an entire interpreter/compiler with it  is quite brilliant, but Alex literally opened my eyes with this presentation (video) where in the short span of 45 minutes he develops  a  (stripped down) Javascript ( could be any generic algol-like language) virtual machine.

In  order, Alex develops a lexer, a parser (based on a simple bnf grammar description provided in a txt file), an AST traversal byte code generator (compiler), an interpreter (VM) and he runs it too.

The cleanness of the code and simplicity with which Alex proceeds slide after slide is remarkable. In fact I have no difficulty to compare his work with the most inspiring chapters from Nicklaus Wirth book “Algorithms + Data Structures = Programs”, where he develops a simplified compiler in PASCAL (link). Sign of the times, Alex work is available in video and his code is on Bitbucket rather than on paper.

Most importantly perhaps, Alex brought home the point that Python and the PyPy toolchain (all the libraries developed by the team to create the new Python compiler) are not limited to Python, but can be used very effectively to create fast virtual machines (and JIT compilers) for other languages too. The team has already developed  Ruby(!), Schema, Javascript(full), Prolog, SmallTalk, Io and even attempted a “Whole System”  virtual machine (aka an emulator) for the GameBoy gaming platform.

So the PyPy tools could help us develop (rapidly) a Python compiler that generates (JIT) code for any processor ( x86 was the main target but other architectures are in development) including possibly a MIPS or perhaps even a more humble PIC core!?

I am (day) dreaming about a world of efficient embedded applications written with the clarity and expressive power of the highest level programming languages… (brought to you by PyPy).

This entry was posted in Python. Bookmark the permalink.