github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/memsys/README.md (about)

     1  ## Memory Manager, Slab Allocator (MMSA)
     2  
     3  MMSA is, simultaneously, a) Slab and SGL allocator, and b) memory manager
     4  that is responsible to optimize memory usage between different (more vs less) utilized
     5  Slabs.
     6  
     7  Multiple MMSA instances may coexist in the system, each having its own
     8  constraints and managing its own Slabs and SGLs.
     9  
    10  MMSA includes a "house-keeping" part to monitor system resources,
    11  adjust Slab sizes based on their respective usages, and incrementally
    12  deallocate idle Slabs. To that end, MMSA utilizes `housekeep` (project and runner).
    13  
    14  ## Construction
    15  
    16  A typical initialization sequence includes steps, e.g.:
    17  
    18  1. Construct:
    19  
    20      ```go
    21      mm := &memsys.MMSA{Name: ..., TimeIval: ..., MinPctFree: ..., Name: ...}
    22      ```
    23  
    24      **Note** that with the only exception of `Name` all the rest member variables (above) have their system defaults and can be omitted.
    25  
    26  2. Initialize:
    27  
    28      ```go
    29      err := mm.Init(false /* don't panic on error */)
    30      if err != nil {
    31          ...
    32      }
    33      ```
    34  
    35  The example above shows initialization that ignores errors - in particular, insifficient minimum required memory (see the previous section).
    36  
    37  Alternatively, MMSA can be initialized *not* to panic on errors:
    38  
    39  ```go
    40   mm.Init(true /* panic on error */)
    41  ```
    42  
    43  In addition, there are several environment variables that can be used
    44  (to circumvent the need to change the code, for instance):
    45  
    46  ```
    47  AIS_MINMEM_FREE
    48  AIS_MINMEM_PCT_TOTAL
    49  AIS_MINMEM_PCT_FREE
    50  ```
    51  
    52  ## Minimum Available Memory
    53  
    54  MMSA will try to make sure that there's a certain specified amount of memory that remains available at all times.
    55  Following are the rules to set this minimum:
    56  
    57  1. environment `AIS_MINMEM_FREE` takes precedence over everything else listed below;
    58  2. if `AIS_MINMEM_FREE` is not defined, variables `AIS_MINMEM_PCT_TOTAL` and/or
    59   `AIS_MINMEM_PCT_FREE` define percentages to compute the minimum based on the total
    60    or the currently available memory, respectively;
    61  3. with no environment, the minimum is computed based on the following MMSA member variables:
    62      * `MinFree`     - memory that must be available at all times
    63      * `MinPctTotal` - same, via percentage of total
    64      * `MinPctFree`  - ditto, as % of free at init time
    65      * Example:
    66          ```go
    67          memsys.MMSA{MinPctTotal: 4, MinFree: cmn.GiB * 2}
    68          ```
    69  4. finally, if none of the above is specified, the constant `minMemFree` in the source
    70  
    71  ## Termination
    72  
    73  If the memory manager is no longer needed, terminating the MMSA instance is recommended.
    74  This will free up all the slabs allocated to the memory manager instance.
    75  Halt a running or initialized MMSA instance is done by:
    76  
    77  ```go
    78  mm.Terminate()
    79  ```
    80  
    81  ## Operation
    82  
    83  Once constructed and initialized, memory-manager-and-slab-allocator (MMSA) can be exercised via its public API that includes `GetSlab`, on the one hand and `Alloc`/`AllocSize` on the other.
    84  
    85  Notice the difference:
    86  * `GetSlab(fixed-bufsize)` returns Slab that contains presizely fixed-bufsize sized reusable buffers
    87  * `Alloc()` and `AllocSize()` return both a Slab and an already allocated buffer from this Slab.
    88  
    89  Note as well that `Alloc()` uses default buffer size for a given MMSA, while `AllocSize()` accepts the specified size (as the name implies).
    90  
    91  Once selected, each Slab can be used via its own public API that
    92  includes `Alloc` and `Free` methods. In addition, each allocated SGL internally
    93  utilizes one of the existing enumerated slabs to "grow" (that is, allocate more
    94  buffers from the slab) on demand. For details, look for "grow" in the iosgl.go.
    95  
    96  When running, the memory manager periodically evaluates
    97  the remaining free memory resource and adjusts its slabs accordingly.
    98  The entire logic is consolidated in one `work()` method that can, for instance,
    99  "cleanup" (see `cleanup()`) an existing "idle" slab,
   100  or forcefully "reduce" (see `reduce()`) one if and when the amount of free
   101  memory falls below watermark.
   102  
   103  ## Testing
   104  
   105  * **Run all tests in debug mode**:
   106  
   107  ```console
   108  $ go test -v -logtostderr=true -duration 2m -tags=debug
   109  ```
   110  
   111  * **Run one of the named tests for 100 seconds**:
   112  
   113  ```console
   114  $ go test -v -run=Test_Sleep -duration=100s
   115  ```
   116  
   117  * **Same as above with debug and deadbeef (build tags) enabled**:
   118  
   119  ```console
   120  $ go test -v -tags=debug,deadbeef -run=Test_Sleep -duration=100s
   121  ```
   122  
   123  * **Run each test for 10 minutes with the permission to use up to 90% of total RAM**
   124  
   125  ```console
   126  $ AIS_MINMEM_PCT_TOTAL=10 go test -v -run=No -duration 10m -timeout=1h -tags=debug
   127  ```
   128  
   129  ## Global Memory Manager
   130  
   131  In the interest of reusing a single memory manager instance across multiple packages outside the ais core package, the memsys package declares a `gMem2` variable that can be accessed through the matching exported Getter.
   132  The notable runtime parameters that are used for the global memory manager are MinFreePct and TimeIval which are set to 50% and 2 minutes, respectively.
   133  Note that more specialized use cases which warrant custom memory managers with finely tuned parameters are free to create their own separate `MMSA` instances.
   134  
   135  Usage:
   136  
   137  To access the global memory manager, a single call to `memsys.Init()` is all that is required. Separate `Init()` nor `Run()` calls should not be made on the returned MMSA instance.