gorgonia.org/gorgonia@v0.9.17/README.md (about)

     1  ![Logo](https://raw.githubusercontent.com/gorgonia/gorgonia/master/media/Logo_horizontal_small.png)
     2  
     3  [![GoDoc](https://godoc.org/gorgonia.org/gorgonia?status.svg)](https://godoc.org/gorgonia.org/gorgonia) [![GitHub version](https://badge.fury.io/gh/gorgonia%2Fgorgonia.svg)](https://badge.fury.io/gh/gorgonia%2Fgorgonia) 
     4  ![Build and Tests](https://github.com/gorgonia/gorgonia/workflows/Build%20and%20Tests%20on%20Linux/amd64/badge.svg)
     5  [![codecov](https://codecov.io/gh/gorgonia/gorgonia/branch/master/graph/badge.svg)](https://codecov.io/gh/gorgonia/gorgonia)
     6  [![Go Report Card](https://goreportcard.com/badge/gorgonia.org/gorgonia)](https://goreportcard.com/report/gorgonia.org/gorgonia) [![unstable](http://badges.github.io/stability-badges/dist/unstable.svg)](http://github.com/badges/stability-badges)
     7  
     8  #
     9  
    10  Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like [Theano](http://deeplearning.net/software/theano/) or [TensorFlow](https://www.tensorflow.org/), it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow.
    11  
    12  Gorgonia:
    13  
    14  * Can perform automatic differentiation
    15  * Can perform symbolic differentiation
    16  * Can perform gradient descent optimizations
    17  * Can perform numerical stabilization
    18  * Provides a number of convenience functions to help create neural networks
    19  * Is fairly quick (comparable to Theano and Tensorflow's speed)
    20  * Supports CUDA/GPGPU computation (OpenCL not yet supported, send a pull request)
    21  * Will support distributed computing
    22  
    23  # Goals #
    24  
    25  The primary goal for Gorgonia is to be a *highly performant* machine learning/graph computation-based library that can scale across multiple machines. It should bring the appeal of Go (simple compilation and deployment process) to the ML world. It's a long way from there currently, however, the baby steps are already there.
    26  
    27  The secondary goal for Gorgonia is to provide a platform for exploration for non-standard deep-learning and neural network related things. This includes things like neo-hebbian learning, corner-cutting algorithms, evolutionary algorithms and the like.
    28  
    29  # Why Use Gorgonia? #
    30  
    31  The main reason to use Gorgonia is developer comfort. If you're using a Go stack extensively, now you have access to the ability to create production-ready machine learning systems in an environment that you are already familiar and comfortable with.
    32  
    33  ML/AI at large is usually split into two stages: the experimental stage where one builds various models, test and retest; and the deployed state where a model after being tested and played with, is deployed. This necessitate different roles like data scientist and data engineer.
    34  
    35  Typically the two phases have different tools: Python ([PyTorch](http://pytorch.org/), etc) is commonly used for the experimental stage, and then the model is rewritten in some more performant language like C++ (using [dlib](http://dlib.net/ml.html), [mlpack](http://mlpack.org) etc). Of course, nowadays the gap is closing and people frequently share the tools between them. Tensorflow is one such tool that bridges the gap.
    36  
    37  Gorgonia aims to do the same, but for the Go environment. Gorgonia is currently fairly performant - its speeds are comparable to PyTorch's and Tensorflow's  CPU implementations. GPU implementations are a bit finnicky to compare due to the heavy cgo tax, but rest assured that this is an area of active improvement.
    38  
    39  # Getting started
    40  
    41  ## Installation #
    42  
    43  The package is go-gettable: `go get -u gorgonia.org/gorgonia`.
    44  
    45  Gorgonia is compatible with go modules.
    46  
    47  ## Documentation
    48  
    49  Up-to-date documentation, references and tutorials are present on the official Gorgonia website at [https://gorgonia.org](https://gorgonia.org).
    50  
    51  ## Keeping Updated
    52  
    53  Gorgonia's project has a [Slack channel on gopherslack](https://gophers.slack.com/messages/gorgonia/), as well as a [Twitter account](https://twitter.com/gorgoniaML). Official updates and announcements will be posted to those two sites.
    54  
    55  ## Usage
    56  
    57  Gorgonia works by creating a computation graph, and then executing it. Think of it as a programming language, but is limited to mathematical functions, and has no branching capability (no if/then or loops). In fact this is the dominant paradigm that the user should be used to thinking about. The computation graph is an [AST](http://en.wikipedia.org/wiki/Abstract_syntax_tree).
    58  
    59  Microsoft's [CNTK](https://github.com/Microsoft/CNTK), with its BrainScript, is perhaps the best at exemplifying the idea that building of a computation graph and running of the computation graphs are different things, and that the user should be in different modes of thoughts when going about them.
    60  
    61  Whilst Gorgonia's implementation doesn't enforce the separation of thought as far as CNTK's BrainScript does, the syntax does help a little bit.
    62  
    63  Here's an example - say you want to define a math expression `z = x + y`. Here's how you'd do it:
    64  
    65  [embedmd]:# (example_basic_test.go)
    66  ```go
    67  package gorgonia_test
    68  
    69  import (
    70  	"fmt"
    71  	"log"
    72  
    73  	. "gorgonia.org/gorgonia"
    74  )
    75  
    76  // Basic example of representing mathematical equations as graphs.
    77  //
    78  // In this example, we want to represent the following equation
    79  //		z = x + y
    80  func Example_basic() {
    81  	g := NewGraph()
    82  
    83  	var x, y, z *Node
    84  	var err error
    85  
    86  	// define the expression
    87  	x = NewScalar(g, Float64, WithName("x"))
    88  	y = NewScalar(g, Float64, WithName("y"))
    89  	if z, err = Add(x, y); err != nil {
    90  		log.Fatal(err)
    91  	}
    92  
    93  	// create a VM to run the program on
    94  	machine := NewTapeMachine(g)
    95  	defer machine.Close()
    96  
    97  	// set initial values then run
    98  	Let(x, 2.0)
    99  	Let(y, 2.5)
   100  	if err = machine.RunAll(); err != nil {
   101  		log.Fatal(err)
   102  	}
   103  
   104  	fmt.Printf("%v", z.Value())
   105  	// Output: 4.5
   106  }
   107  ```
   108  
   109  You might note that it's a little more verbose than other packages of similar nature. For example, instead of compiling to a callable function, Gorgonia specifically compiles into a `*program` which requires a `*TapeMachine` to run. It also requires manual a `Let(...)` call.
   110  
   111  The author would like to contend that this is a Good Thing - to shift one's thinking to a machine-based thinking. It helps a lot in figuring out where things might go wrong.
   112  
   113  Additionally, there are no support for branching - that is to say there are no conditionals (if/else) or loops. The aim is not to build a Turing-complete computer.
   114  
   115  ---
   116  More examples are present in the `example` subfolder of the project, and step-by-step tutorials are present on the [main website](https://gorgonia.org/tutorials/)
   117  
   118  ## Using CUDA ##
   119  
   120  Gorgonia comes with CUDA support out of the box.
   121  Please see the reference documentation about how cuda works on [the Gorgonia.org](https://gorgonia.org/reference/cuda/) website, or jump to the [tutorial](https://gorgonia.org/tutorials/mnist-cuda/).
   122  
   123  # About Gorgonia's development process
   124  
   125  ## Versioning ##
   126  
   127  We use [semver 2.0.0](http://semver.org/) for our versioning. Before 1.0, Gorgonia's APIs are expected to change quite a bit. API is defined by the exported functions, variables and methods. For the developers' sanity, there are minor differences to semver that we will apply prior to version 1.0. They are enumerated below:
   128  
   129  * The MINOR number will be incremented every time there is a deleterious break in API. This means any deletion, or any change in function signature or interface methods will lead to a change in MINOR number.
   130  * Additive changes will NOT change the MINOR version number prior to version 1.0. This means that if new functionality were added that does not break the way you use Gorgonia, there will not be an increment in the MINOR version. There will be an increment in the PATCH version.
   131  
   132  ### API Stability #
   133  Gorgonia's API is as of right now, not considered stable. It will be stable from version 1.0 forwards.
   134  
   135  
   136  ## Go Version Support ##
   137  
   138  Gorgonia supports 2 versions below the Master branch of Go. This means Gorgonia will support the current released version of Go, and up to 4 previous versions - providing something doesn't break. Where possible a shim will be provided (for things like new `sort` APIs or `math/bits` which came out in Go 1.9).
   139  
   140  The current version of Go is 1.13.1. The earliest version Gorgonia supports is Go 1.11.x but Gonum supports only 1.12+. Therefore, the minimum Go version to run the master branch is Go > 1.12.
   141  
   142  ## Hardware and OS supported ##
   143  
   144  Gorgonia runs on :
   145  - linux/AMD64
   146  - linux/ARM7
   147  - linux/ARM64
   148  - win32/AMD64
   149  - darwin/AMD64
   150  - freeBSD/AMD64
   151  
   152  If you have tested gorgonia on other platform, please update this list.
   153  
   154  ## Hardware acceleration
   155  
   156  Gorgonia use some pure assembler instructions to accelerate somes mathematical operations. Unfortunately, only amd64 is supported.
   157  
   158  
   159  # Contributing #
   160  
   161  Obviously since you are most probably reading this on Github, Github will form the major part of the workflow for contributing to this package.
   162  
   163  See also: [CONTRIBUTING.md](CONTRIBUTING.md)
   164  
   165  
   166  ## Contributors and Significant Contributors ##
   167  All contributions are welcome. However, there is a new class of contributor, called Significant Contributors.
   168  
   169  A Significant Contributor is one who has shown *deep understanding* of how the library works and/or its environs.  Here are examples of what constitutes a Significant Contribution:
   170  
   171  * Wrote significant amounts of documentation pertaining to **why**/the mechanics of particular functions/methods and how the different parts affect one another
   172  * Wrote code, and tests around the more intricately connected parts of Gorgonia
   173  * Wrote code and tests, and have at least 5 pull requests accepted
   174  * Provided expert analysis on parts of the package (for example, you may be a floating point operations expert who optimized one function)
   175  * Answered at least 10 support questions.
   176  
   177  Significant Contributors list will be updated once a month (if anyone even uses Gorgonia that is).
   178  
   179  # How To Get Support #
   180  The best way of support right now is to open a [ticket on Github](https://github.com/gorgonia/gorgonia/issues/new).
   181  
   182  # Frequently Asked Questions #
   183  
   184  ### Why are there seemingly random `runtime.GC()` calls in the tests? ###
   185  
   186  The answer to this is simple - the design of the package uses CUDA in a particular way: specifically, a CUDA device and context is tied to a `VM`, instead of at the package level. This means for every `VM` created, a different CUDA context is created per device per `VM`. This way all the operations will play nicely with other applications that may be using CUDA (this needs to be stress-tested, however).
   187  
   188  The CUDA contexts are only destroyed when the `VM` gets garbage collected (with the help of a finalizer function). In the tests, about 100 `VM`s get created, and garbage collection for the most part can be considered random. This leads to cases where the GPU runs out of memory as there are too many contexts being used.
   189  
   190  Therefore at the end of any tests that may use GPU, a `runtime.GC()` call is made to force garbage collection, freeing GPU memories.
   191  
   192  In production, one is unlikely to start that many `VM`s, therefore it's not really a problem. If there is, open a ticket on Github, and we'll look into adding a `Finish()` method for the `VM`s.
   193  
   194  
   195  # Licence #
   196  
   197  Gorgonia is licenced under a variant of Apache 2.0. It's for all intents and purposes the same as the Apache 2.0 Licence, with the exception of not being able to commercially profit directly from the package unless you're a Significant Contributor (for example, providing commercial support for the package). It's perfectly fine to profit directly from a derivative of Gorgonia (for example, if you use Gorgonia as a library in your product)
   198  
   199  Everyone is still allowed to use Gorgonia for commercial purposes (example: using it in a software for your business).
   200  
   201  ## Dependencies ##
   202  
   203  There are very few dependencies that Gorgonia uses - and they're all pretty stable, so as of now there isn't a need for vendoring tools. These are the list of external packages that Gorgonia calls, ranked in order of reliance that this package has (subpackages are omitted):
   204  
   205  |Package|Used For|Vitality|Notes|Licence|
   206  |-------|--------|--------|-----|-------|
   207  |[gonum/graph](https://github.com/gonum/gonum/tree/master/graph)| Sorting `*ExprGraph`| Vital. Removal means Gorgonia will not work | Development of Gorgonia is committed to keeping up with the most updated version|[gonum license](https://github.com/gonum/license) (MIT/BSD-like)|
   208  |[gonum/blas](https://github.com/gonum/gonum/tree/master/blas)|Tensor subpackage linear algebra operations|Vital. Removal means Gorgonial will not work|Development of Gorgonia is committed to keeping up with the most updated version|[gonum license](https://github.com/gonum/license) (MIT/BSD-like)|
   209  |[cu](https://gorgonia.org/cu)| CUDA drivers | Needed for CUDA operations | Same maintainer as Gorgonia | MIT/BSD-like|
   210  |[math32](https://github.com/chewxy/math32)|`float32` operations|Can be replaced by `float32(math.XXX(float64(x)))`|Same maintainer as Gorgonia, same API as the built in `math` package|MIT/BSD-like|
   211  |[hm](https://github.com/chewxy/hm)|Type system for Gorgonia|Gorgonia's graphs are pretty tightly coupled with the type system | Same maintainer as Gorgonia | MIT/BSD-like|
   212  |[vecf64](https://gorgonia.org/vecf64)| optimized `[]float64` operations | Can be generated in the `tensor/genlib` package. However, plenty of optimizations have been made/will be made | Same maintainer as Gorgonia | MIT/BSD-like|
   213  |[vecf32](https://gorgonia.org/vecf32)| optimized `[]float32` operations | Can be generated in the `tensor/genlib` package. However, plenty of optimizations have been made/will be made | Same maintainer as Gorgonia | MIT/BSD-like|
   214  |[set](https://github.com/xtgo/set)|Various set operations|Can be easily replaced|Stable API for the past 1 year|[set licence](https://github.com/xtgo/set/blob/master/LICENSE) (MIT/BSD-like)|
   215  |[gographviz](https://github.com/awalterschulze/gographviz)|Used for printing graphs|Graph printing is only vital to debugging. Gorgonia can survive without, but with a major (but arguably nonvital) feature loss|Last update 12th April 2017|[gographviz licence](https://github.com/awalterschulze/gographviz/blob/master/LICENSE) (Apache 2.0)|
   216  |[rng](https://github.com/leesper/go_rng)|Used to implement helper functions to generate initial weights|Can be replaced fairly easily. Gorgonia can do without the convenience functions too||[rng licence](https://github.com/leesper/go_rng/blob/master/LICENSE) (Apache 2.0)|
   217  |[errors](https://github.com/pkg/errors)|Error wrapping|Gorgonia won't die without it. In fact Gorgonia has also used [goerrors/errors](https://github.com/go-errors/errors) in the past.|Stable API for the past 6 months|[errors licence](https://github.com/pkg/errors/blob/master/LICENSE) (MIT/BSD-like)|
   218  |[gonum/mat](http://github.com/gonum/gonum)|Compatibility between `Tensor` and Gonum's Matrix|Development of Gorgonia is committed to keeping up with the most updated version||[gonum license](https://github.com/gonum/license) (MIT/BSD-like)|
   219  |[testify/assert](https://github.com/stretchr/testify)|Testing|Can do without but will be a massive pain in the ass to test||[testify licence](https://github.com/stretchr/testify/blob/master/LICENSE) (MIT/BSD-like)|
   220  
   221  
   222  ## Various Other Copyright Notices ##
   223  
   224  These are the packages and libraries which inspired and were adapted from in the process of writing Gorgonia (the Go packages that were used were already declared above):
   225  
   226  | Source | How it's Used | Licence |
   227  |------|---|-------|
   228  | Numpy  | Inspired large portions. Directly adapted algorithms for a few methods (explicitly labelled in the docs) | MIT/BSD-like. [Numpy Licence](https://github.com/numpy/numpy/blob/master/LICENSE.txt) |
   229  | Theano | Inspired large portions. (Unsure: number of directly adapted algorithms) | MIT/BSD-like [Theano's licence](http://deeplearning.net/software/theano/LICENSE.html) |
   230  | Caffe | `im2col` and `col2im` directly taken from Caffe. Convolution algorithms inspired by the original Caffee methods | [Caffe Licence](https://github.com/BVLC/caffe/blob/master/LICENSE)