github.com/grumpyhome/grumpy@v0.3.1-0.20201208125205-7b775405bdf1/README.md (about) 1 # Grumpy: Go running Python 2 3 [](https://travis-ci.org/grumpyhome/grumpy) 4 [](https://gitter.im/grumpy-devel/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 5 [](https://opensource.org/licenses/Apache-2.0) 6 7 ## Overview 8 9 Grumpy is a Python to Go source code transcompiler and runtime that is intended 10 to be a near drop-in replacement for CPython 2.7. Grumpy has no VM, because it 11 compiles Python source code to Go source code which is then compiled to native 12 code (rather than to bytecode). The compiled Go source code uses Go library 13 called Grumpy runtime, which replaces Python C API for system calls (although 14 the API is incompatible with CPython's). 15 16 ## Limitations 17 18 ### Things that will probably never be supported by Grumpy 19 20 1. `exec`, `eval` and `compile`: These dynamic features of CPython are not 21 supported by Grumpy because Grumpy modules consist of statically-compiled Go 22 code. Supporting dynamic execution would require bundling Grumpy programs 23 with the compilation toolchain, which would be unwieldy and impractically 24 slow. 25 26 2. C extension modules: Grumpy has a different API and object layout than 27 CPython and so supporting C extensions would be difficult. In principle it's 28 possible to support them via an API bridge layer like the one that 29 [JyNI](http://jyni.org) provides for Jython, but it would be hard to maintain and 30 would add significant overhead when calling into and out of extension 31 modules. 32 33 ### Things that Grumpy will support but doesn't yet 34 35 There are three basic categories of incomplete functionality: 36 37 1. [Language features](https://github.com/google/grumpy/wiki/Missing-features#language-features): 38 Most language features are implemented with the notable exception of 39 [old-style classes](http://stackoverflow.com/questions/54867/what-is-the-difference-between-old-style-and-new-style-classes-in-python). 40 There are also a handful of operators that aren't yet supported. 41 42 2. [Builtin functions and types](https://github.com/google/grumpy/wiki/Missing-features#builtins): 43 There are a number of missing functions and types in `__builtins__` that have 44 not yet been implemented. There are also a lot of methods on builtin types 45 that are missing. 46 47 3. [Standard library](https://github.com/google/grumpy/wiki/Missing-features#standard-libraries): 48 The Python standard library is very large and much of it is pure Python, so 49 as the language features and builtins get filled out, many modules will 50 just work. But there are also a number of libraries in CPython that are C 51 extension modules which will need to be rewritten. 52 53 4. C locale support: Go doesn't support locales in the same way that C does. As such, 54 some functionality that is locale-dependent may not currently work the same as in 55 CPython. 56 57 ## Running Grumpy 58 59 ### Pre-requisites 60 Python 2.7 61 pip 62 Go 1.10+ 63 64 ##### Mac Python 2.7 65 OSX users can go to https://www.python.org/downloads/release/python-2718/ 66 67 Once you've succesfull installed Python2.7 please ensure that pip is also 68 installed on your system by running the command "python -m pip --version". 69 If pip is not installed please follow the instructions found here 70 https://pip.pypa.io/en/stable/installing/# 71 72 Once pip has succesfully been installed please run the following command 73 74 python -m pip install --upgrade pip setuptools 75 76 ##### Linux Python 2.7 77 For Debian based distos you can install Python 2 with. 78 79 sudo apt install python2 80 81 Next ensure that pip is installed by running "python -m pip --version" 82 or, "python2 -m pip version". If it returns pip [version number] then 83 pip is already installed. If it doesn't run the command "sudo apt install python-pip". 84 Note: Again, replace apt with the appropriate package tool. 85 86 ##### Go Compiler 87 88 Go can be downloaded here: https://golang.org/dl/ 89 90 Please see the official go installation documentation to install Go: https://golang.org/doc/install 91 92 If you wish to build Go from source see the documentation here: https://golang.org/doc/install/source 93 94 95 ## Installing Grumpy 96 97 The commands ahead assumes that you have Golang installed and a recent 98 version of Python 2 and `pip`. 99 100 ### Method 0: binary package 101 102 For convenience, a Python package is provided from the PyPI. During install, 103 many Grumpy will be compiled and stored inside your Python installation. 104 105 You need Golang preinstalled anyway for the installation to be successful. 106 107 ```sh 108 pip2 install -U grumpy-runtime -I --no-cache 109 (wait about 5 minutes) 110 echo "print 'hello, world'" | grumpy run 111 ``` 112 113 ### Method 1: make run: 114 115 The simplest way to execute a Grumpy program is to use `make run`, which wraps a 116 shell script called grumprun that takes Python code on stdin and builds and runs 117 the code under Grumpy: 118 119 ```sh 120 cd grumpy-tools-src 121 python2 setup.py develop 122 cd ../grumpy-runtime-src 123 echo "print 'hello, world'" | make run 124 ``` 125 126 ### Method 2: grumpc and grumprun: 127 128 For more complicated programs, you'll want to compile your Python source code to 129 Go using grumpc (the Grumpy compiler) and then build the Go code using `go 130 build`. Since Grumpy programs are statically linked, all the modules in a 131 program must be findable by the Grumpy toolchain on the GOPATH. Grumpy looks for 132 Go packages corresponding to Python modules in the \_\_python\_\_ subdirectory 133 of the GOPATH. By convention, this subdirectory is also used for staging Python 134 source code, making it similar to the PYTHONPATH. 135 136 The first step is to set up the shell so that the Grumpy toolchain and libraries 137 can be found. From the root directory of the Grumpy source distribution run: 138 139 ```sh 140 cd grumpy-tools-src 141 python2 setup.py develop 142 cd ../grumpy-runtime-src 143 make 144 export PATH=$PWD/build/bin:$PATH 145 export GOPATH=$PWD/build 146 export PYTHONPATH=$PWD/build/lib/python2.7/site-packages 147 ``` 148 149 You will know things are working if you see the expected output from this 150 command: 151 152 ```sh 153 cd grumpy-runtime-src 154 echo 'import sys; print sys.version' | grumprun 155 ``` 156 157 Next, we will write our simple Python module into the \_\_python\_\_ directory: 158 159 ```sh 160 cd grumpy-runtime-src 161 echo 'def hello(): print "hello, world"' > $GOPATH/src/__python__/hello.py 162 ``` 163 164 To build a Go package from our Python script, run the following: 165 166 ```sh 167 cd grumpy-runtime-src 168 mkdir -p $GOPATH/src/__python__/hello 169 grumpc -modname=hello $GOPATH/src/__python__/hello.py > \ 170 $GOPATH/src/__python__/hello/module.go 171 ``` 172 173 You should now be able to build a Go program that imports the package 174 "\_\_python\_\_/hello". We can also import this module into Python programs 175 that are built using grumprun: 176 177 ```sh 178 cd grumpy-runtime-src 179 echo 'from hello import hello; hello()' | grumprun 180 ``` 181 182 grumprun is doing a few things under the hood here: 183 184 1. Compiles the given Python code to a dummy Go package, the same way we 185 produced \_\_python\_\_/hello/module.go above 186 2. Produces a main Go package that imports the Go package from step 1. and 187 executes it as our \_\_main\_\_ Python package 188 3. Executes `go run` on the main package generated in step 2. 189 190 ## Developing Grumpy 191 192 There are three main components and depending on what kind of feature you're 193 writing, you may need to change one or more of these. 194 195 ### Grumpy Tools 196 197 Grumpy converts Python programs into Go programs and 198 `grumpy transpile` is the CLI tool responsible for parsing Python code 199 and generating Go code from it. `grumpy transpile` is written in Python 200 and uses the [`pythonparser`](https://github.com/m-labs/pythonparser) 201 module to accomplish parsing. 202 203 The CLI main entrypoint lives at `grumpy-tools-src/grumpy_tools/cli.py`. 204 It is supported by a number of Python modules in the 205 `grumpy-tools-src/grumpy_tools/compiler` subdir. 206 207 ### Grumpy Runtime 208 209 The Go code generated by `grumpy transpile` performs operations 210 on data structures that represent Python objects in running Grumpy programs. 211 These data structures and operations are defined in the `grumpy` Go library 212 (source is in the `grumpy-runtime-src/runtime` subdir of the source 213 distribution). This runtime is analogous to the Python C API and many of the 214 structures and operations defined by `grumpy` have counterparts in CPython. 215 216 ### Grumpy Standard Library 217 218 Much of the Python standard library is written in Python and thus "just works" 219 in Grumpy. These parts of the standard library are copied from CPython 2.7 220 (possibly with light modifications). For licensing reasons, these files are kept 221 in the `grumpy-runtime-src/third_party` subdir. 222 223 The parts of the standard library that cannot be written in pure Python, e.g. 224 file and directory operations, are kept in the `grumpy-runtime-src/lib` subdir. 225 In CPython these kinds of modules are written as C extensions. In Grumpy they 226 are written in Python but they use native Go extensions to access facilities not 227 otherwise available in Python. 228 229 ### Source Code Overview 230 231 - `grumpy-tools-src/grumpy_tools/compiler`: Python package implementating Python -> Go transcompilation logic. 232 - `grumpy-runtime-src/lib`: Grumpy-specific Python standard library implementation. 233 - `grumpy-runtime-src/runtime`: Go source code for the Grumpy runtime library. 234 - `grumpy-runtime-src/third_party/ouroboros`: Pure Python standard libraries copied from the 235 [Ouroboros project](https://github.com/pybee/ouroboros). 236 - `grumpy-runtime-src/third_party/pypy`: Pure Python standard libraries copied from PyPy. 237 - `grumpy-runtime-src/third_party/stdlib`: Pure Python standard libraries copied from CPython. 238 - `grumpy-tools-src/grumpy_tools/`: Transcompilation and utility CLI. 239 240 ## Contact 241 242 Questions? Comments? Drop us a line at [grumpy-users@googlegroups.com](https://groups.google.com/forum/#!forum/grumpy-users) 243 or join our [Gitter channel](https://gitter.im/grumpy-devel/Lobby)