agones.dev/agones@v1.54.0/build/build-image/cache/README.md (about)

     1  # Origin
     2  
     3  This `README.md` was originally based on [cloud-builders-community](https://github.com/GoogleCloudPlatform/cloud-builders-community), which contains source code for community-contributed Docker images. You can use these images as build steps for [Google Cloud Build](https://cloud.google.com/build/docs). 
     4  
     5  The key change made upon importing the [cache builder's](https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/cache) code into Agones was the update of image tags to use Google Artifact Registry instead of Google Container Registry.
     6  
     7  # Cache builders
     8  
     9  This includes a pair of builders, `save_cache` and `restore_cache`, that work together to cache files between builds to a GCS bucket (or local file).
    10  
    11  ## Using the `save_cache` builder
    12  
    13  All options that require a value use the form `--option=value` or `-o=value` so that they look nice in Yaml files.
    14  
    15  | Option           | Description                                                 |
    16  | ---------------- | ----------------------------------------------------------- |
    17  | -b, --bucket     | The cloud storage bucket to upload the cache to. [optional] |
    18  | -o, --out        | The output directory to write the cache to. [optional]      |
    19  | -k, --key        | The cache key used for this cache file. [optional]          |
    20  | -p, --path       | The files to store in the cache. Can be repeated.           |
    21  | -t, --threshold  | The parallel composite upload threshold [default: 50M]      |
    22  | -n, --no-clobber | Skips the save if the cache file already exists in GCS.     |
    23  
    24  One of `--bucket` or `--out` parameters are required.  If `--bucket` then the cache file will be uploaded to the provided GCS bucket path.  If `--out` then the cache file will be stored in the directory specified on disk.
    25  
    26  The key provided by `--key` is used to identify the cache file. Any other cache files for the same key will be overwritten by this one.
    27  
    28  The `--path` parameters can be repeated for as many folders as you'd like to cache.  When restored, they will retain folder structure on disk.
    29  
    30  The `--no-clobber` flag is used to skip creating and uploading the cache to GCS if the cache file already exists. This will shorten the time for builds when a cache was restored and is not changed by your build process. For example, this flag can be used if you are caching your dependencies and all of your dependencies are pinned to a specific version. This flag is valid only when `--bucket` is used.
    31  
    32  ## Using the `restore_cache` builder
    33  
    34  All options use the form `--option=value` or `-o=value` so that they look nice in Yaml files.
    35  
    36  | Option                 | Description                                                                            |
    37  | ---------------------- | -------------------------------------------------------------------------------------- |
    38  | -b,  --bucket          | The cloud storage bucket to download the cache from. [optional]                        |
    39  | -s,  --src             | The local directory in which the cache is stored. [optional]                           |
    40  | -k,  --key             | The cache key used for this cache file. [optional]                                     |
    41  | -kf, --key_fallback    | The cache key fallback pattern to be used if exact cache key is not found. [optional]  |
    42  
    43  One of `--bucket` or `--src` parameters are required.  If `--bucket` then the cache file will be downloaded from the provided GCS bucket path.  If `--src` then the cache file will be read from the directory specified on disk.
    44  
    45  The key provided by `--key` is used to identify the cache file.
    46  
    47  The fallback key pattern provide by `--key_fallback`, will be used to fetch the most recent cache file matching that pattern in case there is a cache miss from the specified `--key`.
    48  
    49  ### `checksum` Helper
    50  
    51  As apps develop, cache needs change. For instance when dependencies are removed from a project, or versions are updated, there is no need to cache the older versions of dependencies. Therefore it's recommended that you update the cache key when these changes occur.
    52  
    53  This builder includes a `checksum` helper script, which you can use to create a simple checksum of files in your project to use as a cache key.
    54  
    55  To use it in the `--key` arguemnt, simply surround the command with `$()`:
    56  
    57  ```bash
    58  --key=build-cache-$(checksum build.gradle)-$(checksum dependencies.gradle)
    59  ```
    60  
    61  To ensure you aren't paying for storage of obsolete cache files you can add an Object Lifecycle Rule to the cache bucket to delete object older than 30 days.
    62  
    63  ## Examples
    64  
    65  The following examples demonstrate build requests that use this builder.
    66  
    67  ### Saving a cache with checksum to GCS bucket
    68  
    69  This `cloudbuild.yaml` saves the files and folders in the `path` arguments to a cache file in the GCS bucket `gs://$CACHE_BUCKET/`.  In this example the key will be updated, resulting in a new cache, every time the `cloudbuild.yaml` build file is changed.
    70  
    71  ```yaml
    72  - name: 'us-docker.pkg.dev/$PROJECT_ID/ci/save_cache'
    73    args:
    74    - '--bucket=gs://$CACHE_BUCKET/'
    75    - '--key=resources-$( checksum cloudbuild.yaml )'
    76    - '--path=.cache/folder1'
    77    - '--path=.cache/folder2/subfolder3'
    78  ```
    79  
    80  If your build process only changes the cache contents whenever `cloudbuild.yaml` changes, then you can skip saving the cache again if it already exists in GCS:
    81  ```yaml
    82  - name: 'us-docker.pkg.dev/$PROJECT_ID/ci/save_cache'
    83    args:
    84    - '--bucket=gs://$CACHE_BUCKET/'
    85    - '--key=resources-$( checksum cloudbuild.yaml )'
    86    - '--path=.cache/folder1'
    87    - '--path=.cache/folder2/subfolder3'
    88    - '--no-clobber'
    89  ```
    90  
    91  ### Saving a cache with checksum to a local file
    92  
    93  This `cloudbuild.yaml` saves the files and folders in the `path` arguments to a cache file in the directory passed to the `out` parameter.  In this example the key will be updated, resulting in a new cache, every time the `cloudbuild.yaml` build file is changed.
    94  
    95  ```yaml
    96  - name: 'us-docker.pkg.dev/$PROJECT_ID/ci/save_cache'
    97    args:
    98    - '--out=/cache/'
    99    - '--key=resources-$( checksum cloudbuild.yaml )'
   100    - '--path=.cache/folder1'
   101    - '--path=.cache/folder2/subfolder3'
   102    volumes:
   103    - name: 'cache'
   104      path: '/cache'
   105  ```
   106  
   107  ### Restore a cache from a GCS bucket
   108  
   109  This `cloudbuild.yaml` restores the files from the compressed cache file identified by `key` on the cache bucket provided, if it exists.
   110  
   111  ```yaml
   112  - name: 'us-docker.pkg.dev/$PROJECT_ID/ci/restore_cache'
   113    args:
   114    - '--bucket=gs://$CACHE_BUCKET/'
   115    - '--key=resources-$( checksum cloudbuild.yaml )'
   116  ```
   117  
   118  ### Restore a cache from a local file
   119  
   120  This `cloudbuild.yaml` restores the files from the compressed cache file identified by `key` on the local filesystem, if it exists.
   121  
   122  ```yaml
   123  - name: 'us-docker.pkg.dev/$PROJECT_ID/ci/restore_cache'
   124    args:
   125    - '--src=/cache/'
   126    - '--key=resources-$( checksum cloudbuild.yaml )'
   127    volumes:
   128    - name: 'cache'
   129      path: '/cache'
   130  ```
   131  
   132  ### Restore a cache with a fallback key
   133  
   134  ```yaml
   135  - name: us-docker.pkg.dev/$PROJECT_ID/ci/restore_cache
   136    id: restore_cache
   137    args: [
   138      '--bucket=gs://${_CACHE_BUCKET}',
   139      '--key=gradle-$( checksum checksum.txt )',
   140      '--key_fallback=gradle-',
   141    ]
   142  ```