github.com/google/grumpy@v0.0.0-20171122020858-3ec87959189c/README.md (about) 1 # Grumpy: Go running Python 2 3 [](https://travis-ci.org/google/grumpy) 4 [](https://gitter.im/grumpy-devel/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 5 6 ## Overview 7 8 Grumpy is a Python to Go source code transcompiler and runtime that is intended 9 to be a near drop-in replacement for CPython 2.7. The key difference is that it 10 compiles Python source code to Go source code which is then compiled to native 11 code, rather than to bytecode. This means that Grumpy has no VM. The compiled Go 12 source code is a series of calls to the Grumpy runtime, a Go library serving a 13 similar purpose to the Python C API (although the API is incompatible with 14 CPython's). 15 16 ## Limitations 17 18 ### Things that will probably never be supported by Grumpy 19 20 1. `exec`, `eval` and `compile`: These dynamic features of CPython are not 21 supported by Grumpy because Grumpy modules consist of statically-compiled Go 22 code. Supporting dynamic execution would require bundling Grumpy programs 23 with the compilation toolchain, which would be unwieldy and impractically 24 slow. 25 26 2. C extension modules: Grumpy has a different API and object layout than 27 CPython and so supporting C extensions would be difficult. In principle it's 28 possible to support them via an API bridge layer like the one that 29 [JyNI](http://jyni.org) provides for Jython, but it would be hard to maintain and 30 would add significant overhead when calling into and out of extension 31 modules. 32 33 ### Things that Grumpy will support but doesn't yet 34 35 There are three basic categories of incomplete functionality: 36 37 1. [Language features](https://github.com/google/grumpy/wiki/Missing-features#language-features): 38 Most language features are implemented with the notable exception of 39 [old-style classes](http://stackoverflow.com/questions/54867/what-is-the-difference-between-old-style-and-new-style-classes-in-python). 40 There are also a handful of operators that aren't yet supported. 41 42 2. [Builtin functions and types](https://github.com/google/grumpy/wiki/Missing-features#builtins): 43 There are a number of missing functions and types in `__builtins__` that have 44 not yet been implemented. There are also a lot of methods on builtin types 45 that are missing. 46 47 3. [Standard library](https://github.com/google/grumpy/wiki/Missing-features#standard-libraries): 48 The Python standard library is very large and much of it is pure Python, so 49 as the language features and builtins get filled out, many modules will 50 just work. But there are also a number of libraries in CPython that are C 51 extension modules which will need to be rewritten. 52 53 4. C locale support: Go doesn't support locales in the same way that C does. As such, 54 some functionality that is locale-dependent may not currently work the same as in 55 CPython. 56 57 ## Running Grumpy Programs 58 59 ### Method 1: make run: 60 61 The simplest way to execute a Grumpy program is to use `make run`, which wraps a 62 shell script called grumprun that takes Python code on stdin and builds and runs 63 the code under Grumpy. All of the commands below are assumed to be run from the 64 root directory of the Grumpy source code distribution: 65 66 ``` 67 echo "print 'hello, world'" | make run 68 ``` 69 70 ### Method 2: grumpc and grumprun: 71 72 For more complicated programs, you'll want to compile your Python source code to 73 Go using grumpc (the Grumpy compiler) and then build the Go code using `go 74 build`. Since Grumpy programs are statically linked, all the modules in a 75 program must be findable by the Grumpy toolchain on the GOPATH. Grumpy looks for 76 Go packages corresponding to Python modules in the \_\_python\_\_ subdirectory 77 of the GOPATH. By convention, this subdirectory is also used for staging Python 78 source code, making it similar to the PYTHONPATH. 79 80 The first step is to set up the shell so that the Grumpy toolchain and libraries 81 can be found. From the root directory of the Grumpy source distribution run: 82 83 ``` 84 make 85 export PATH=$PWD/build/bin:$PATH 86 export GOPATH=$PWD/build 87 export PYTHONPATH=$PWD/build/lib/python2.7/site-packages 88 ``` 89 90 You will know things are working if you see the expected output from this 91 command: 92 93 ``` 94 echo 'import sys; print sys.version' | grumprun 95 ``` 96 97 Next, we will write our simple Python module into the \_\_python\_\_ directory: 98 99 ``` 100 echo 'def hello(): print "hello, world"' > $GOPATH/src/__python__/hello.py 101 ``` 102 103 To build a Go package from our Python script, run the following: 104 105 ``` 106 mkdir -p $GOPATH/src/__python__/hello 107 grumpc -modname=hello $GOPATH/src/__python__/hello.py > \ 108 $GOPATH/src/__python__/hello/module.go 109 ``` 110 111 You should now be able to build a Go program that imports the package 112 "\_\_python\_\_/hello". We can also import this module into Python programs 113 that are built using grumprun: 114 115 ``` 116 echo 'from hello import hello; hello()' | grumprun 117 ``` 118 119 grumprun is doing a few things under the hood here: 120 121 1. Compiles the given Python code to a dummy Go package, the same way we 122 produced \_\_python\_\_/hello/module.go above 123 2. Produces a main Go package that imports the Go package from step 1. and 124 executes it as our \_\_main\_\_ Python package 125 3. Executes `go run` on the main package generated in step 2. 126 127 ## Developing Grumpy 128 129 There are three main components and depending on what kind of feature you're 130 writing, you may need to change one or more of these. 131 132 ### grumpc 133 134 Grumpy converts Python programs into Go programs and `grumpc` is the tool 135 responsible for parsing Python code and generating Go code from it. `grumpc` is 136 written in Python and uses the [`pythonparser`](https://github.com/m-labs/pythonparser) 137 module to accomplish parsing. 138 139 The grumpc script itself lives at `tools/grumpc`. It is supported by a number of 140 Python modules in the `compiler` subdir. 141 142 ### Grumpy Runtime 143 144 The Go code generated by `grumpc` performs operations on data structures that 145 represent Python objects in running Grumpy programs. These data structures and 146 operations are defined in the `grumpy` Go library (source is in the runtime 147 subdir of the source distribution). This runtime is analogous to the Python C 148 API and many of the structures and operations defined by `grumpy` have 149 counterparts in CPython. 150 151 ### Grumpy Standard Library 152 153 Much of the Python standard library is written in Python and thus "just works" 154 in Grumpy. These parts of the standard library are copied from CPython 2.7 155 (possibly with light modifications). For licensing reasons, these files are kept 156 in the `third_party` subdir. 157 158 The parts of the standard library that cannot be written in pure Python, e.g. 159 file and directory operations, are kept in the `lib` subdir. In CPython these 160 kinds of modules are written as C extensions. In Grumpy they are written in 161 Python but they use native Go extensions to access facilities not otherwise 162 available in Python. 163 164 ### Source Code Overview 165 166 - `compiler`: Python package implementating Python -> Go transcompilation logic. 167 - `lib`: Grumpy-specific Python standard library implementation. 168 - `runtime`: Go source code for the Grumpy runtime library. 169 - `third_party/ouroboros`: Pure Python standard libraries copied from the 170 [Ouroboros project](https://github.com/pybee/ouroboros). 171 - `third_party/pypy`: Pure Python standard libraries copied from PyPy. 172 - `third_party/stdlib`: Pure Python standard libraries copied from CPython. 173 - `tools`: Transcompilation and utility binaries. 174 175 ## Contact 176 177 Questions? Comments? Drop us a line at [grumpy-users@googlegroups.com](https://groups.google.com/forum/#!forum/grumpy-users) 178 or join our [Gitter channel](https://gitter.im/grumpy-devel/Lobby)