github.com/apache/arrow/go/v10@v10.0.1/README.md (about)

     1  <!---
     2    Licensed to the Apache Software Foundation (ASF) under one
     3    or more contributor license agreements.  See the NOTICE file
     4    distributed with this work for additional information
     5    regarding copyright ownership.  The ASF licenses this file
     6    to you under the Apache License, Version 2.0 (the
     7    "License"); you may not use this file except in compliance
     8    with the License.  You may obtain a copy of the License at
     9  
    10      http://www.apache.org/licenses/LICENSE-2.0
    11  
    12    Unless required by applicable law or agreed to in writing,
    13    software distributed under the License is distributed on an
    14    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    15    KIND, either express or implied.  See the License for the
    16    specific language governing permissions and limitations
    17    under the License.
    18  -->
    19  
    20  Apache Arrow for Go
    21  ===================
    22  
    23  [![GoDoc](https://godoc.org/github.com/apache/arrow/go/arrow?status.svg)](https://godoc.org/github.com/apache/arrow/go/arrow)
    24  
    25  [Apache Arrow][arrow] is a cross-language development platform for in-memory
    26  data. It specifies a standardized language-independent columnar memory format
    27  for flat and hierarchical data, organized for efficient analytic operations on
    28  modern hardware. It also provides computational libraries and zero-copy
    29  streaming messaging and inter-process communication.
    30  
    31  
    32  Reference Counting
    33  ------------------
    34  
    35  The library makes use of reference counting so that it can track when memory
    36  buffers are no longer used. This allows Arrow to update resource accounting,
    37  pool memory such and track overall memory usage as objects are created and
    38  released. Types expose two methods to deal with this pattern. The `Retain`
    39  method will increase the reference count by 1 and `Release` method will reduce
    40  the count by 1. Once the reference count of an object is zero, any associated
    41  object will be freed. `Retain` and `Release` are safe to call from multiple
    42  goroutines.
    43  
    44  ### When to call `Retain` / `Release`?
    45  
    46  * If you are passed an object and wish to take ownership of it, you must call
    47    `Retain`. You must later pair this with a call to `Release` when you no
    48    longer need the object.  "Taking ownership" typically means you wish to
    49    access the object outside the scope of the current function call.
    50  
    51  * You own any object you create via functions whose name begins with `New` or
    52    `Copy` or when receiving an object over a channel. Therefore you must call
    53    `Release` once you no longer need the object.
    54  
    55  * If you send an object over a channel, you must call `Retain` before sending
    56    it as the receiver is assumed to own the object and will later call `Release`
    57    when it no longer needs the object.
    58  
    59  Performance
    60  -----------
    61  
    62  The arrow package makes extensive use of [c2goasm][] to leverage LLVM's
    63  advanced optimizer and generate PLAN9 assembly functions from C/C++ code. The
    64  arrow package can be compiled without these optimizations using the `noasm`
    65  build tag. Alternatively, by configuring an environment variable, it is
    66  possible to dynamically configure which architecture optimizations are used at
    67  runtime.  See the `cpu` package [README](arrow/internal/cpu/README.md) for a
    68  description of this environment variable.
    69  
    70  ### Example Usage
    71  
    72  The following benchmarks demonstrate summing an array of 8192 values using
    73  various optimizations.
    74  
    75  Disable no architecture optimizations (thus using AVX2):
    76  
    77  ```sh
    78  $ INTEL_DISABLE_EXT=NONE go test -bench=8192 -run=. ./math
    79  goos: darwin
    80  goarch: amd64
    81  pkg: github.com/apache/arrow/go/arrow/math
    82  BenchmarkFloat64Funcs_Sum_8192-8   	 2000000	       687 ns/op	95375.41 MB/s
    83  BenchmarkInt64Funcs_Sum_8192-8     	 2000000	       719 ns/op	91061.06 MB/s
    84  BenchmarkUint64Funcs_Sum_8192-8    	 2000000	       691 ns/op	94797.29 MB/s
    85  PASS
    86  ok  	github.com/apache/arrow/go/arrow/math	6.444s
    87  ```
    88  
    89  **NOTE:** `NONE` is simply ignored, thus enabling optimizations for AVX2 and SSE4
    90  
    91  ----
    92  
    93  Disable AVX2 architecture optimizations:
    94  
    95  ```sh
    96  $ INTEL_DISABLE_EXT=AVX2 go test -bench=8192 -run=. ./math
    97  goos: darwin
    98  goarch: amd64
    99  pkg: github.com/apache/arrow/go/arrow/math
   100  BenchmarkFloat64Funcs_Sum_8192-8   	 1000000	      1912 ns/op	34263.63 MB/s
   101  BenchmarkInt64Funcs_Sum_8192-8     	 1000000	      1392 ns/op	47065.57 MB/s
   102  BenchmarkUint64Funcs_Sum_8192-8    	 1000000	      1405 ns/op	46636.41 MB/s
   103  PASS
   104  ok  	github.com/apache/arrow/go/arrow/math	4.786s
   105  ```
   106  
   107  ----
   108  
   109  Disable ALL architecture optimizations, thus using pure Go implementation:
   110  
   111  ```sh
   112  $ INTEL_DISABLE_EXT=ALL go test -bench=8192 -run=. ./math
   113  goos: darwin
   114  goarch: amd64
   115  pkg: github.com/apache/arrow/go/arrow/math
   116  BenchmarkFloat64Funcs_Sum_8192-8   	  200000	     10285 ns/op	6371.41 MB/s
   117  BenchmarkInt64Funcs_Sum_8192-8     	  500000	      3892 ns/op	16837.37 MB/s
   118  BenchmarkUint64Funcs_Sum_8192-8    	  500000	      3929 ns/op	16680.00 MB/s
   119  PASS
   120  ok  	github.com/apache/arrow/go/arrow/math	6.179s
   121  ```
   122  
   123  [arrow]:    https://arrow.apache.org
   124  [c2goasm]:  https://github.com/minio/c2goasm