github.com/grailbio/bigslice@v0.0.0-20230519005545-30c4c12152ad/README.md (about)

     1  # Bigslice
     2  
     3  Bigslice is a serverless cluster data processing system for [Go](https://golang.org).
     4  Bigslice exposes composable API
     5  that lets the user express
     6  data processing tasks in terms of a series of
     7  data transformations that invoke user code.
     8  The Bigslice runtime then
     9  transparently parallelizes and distributes the work,
    10  using the [Bigmachine](https://github.com/grailbio/bigmachine)
    11  library to create an ad hoc cluster on a cloud provider.
    12  
    13  - website: [bigslice.io](https://bigslice.io/)
    14  - API documentation: [godoc.org/github.com/grailbio/bigslice](https://godoc.org/github.com/grailbio/bigslice)
    15  - issue tracker: [github.com/grailbio/bigslice/issues](https://github.com/grailbio/bigslice/issues)
    16  - [![CI](https://github.com/grailbio/bigslice/workflows/CI/badge.svg)](https://github.com/grailbio/bigslice/actions?query=workflow%3ACI) [![Full Test](https://github.com/grailbio/bigslice/workflows/Full%20Test/badge.svg)](https://github.com/grailbio/bigslice/actions?query=workflow%3A%22Full+Test%22)
    17  
    18  # Developing Bigslice
    19  
    20  Bigslice uses Go modules to capture its dependencies;
    21  no tooling other than the base Go install is required.
    22  ```
    23  $ git clone https://github.com/grailbio/bigslice
    24  $ cd bigslice
    25  $ GO111MODULE=on go test
    26  ```
    27  
    28  If tests fail with `socket: too many open files` errors, try increasing the maximum number of open files.
    29  ```
    30  $ ulimit -n 2000
    31  ```