github.com/openshift/installer@v1.4.17/docs/design/assetgeneration.md (about)

     1  # Asset Generation
     2  
     3  The installer internally uses a directed acyclic graph to represent all of the assets it creates as well as their dependencies. This process looks very similar to many build systems (e.g. Bazel, Make).
     4  
     5  ## Overview
     6  
     7  The installer generates assets based on the [dependency graph](#dependency-graph). Each asset separately defines how it can be generated as well as its dependencies. Targets represent a set of assets that should be generated and written to disk for the user's consumption. When a user invokes the installer for a particular target, each of the assets in the set is generated as well as any dependencies. This eventually results in the user being prompted for any missing information (e.g. administrator password, target platform).
     8  
     9  The installer is also able to read assets from disk if they have been provided by the user. In the event that an asset exists on disk, the install won't generate the asset, but will instead consume the asset from disk (removing the file). This allows the installer to be run multiple times, using the assets generated by the previous invocation. It also allows a user to make modifications to the generated assets before continuing to the next target.
    10  
    11  Each asset is individually responsible for declaring its dependencies. Each asset is also responsible resolving conflicts when combining its input from disk and its state from a previous run. The installer ensures all the dependencies for an asset is generated and provides the asset with latest state to generate its own output.
    12  
    13  ## Asset
    14  
    15  An asset is the generic representation of work-item for installer that needs to be generated. Each asset defines all the other assets that are required for it to generate itself as dependencies.
    16  
    17  The asset would usually follow these steps to generate its output:
    18  
    19  1. Fetch its parent assets.
    20  
    21  2. Generate the assets either by:
    22      *  Using the parent assets
    23      *  Loading from on-disk assets
    24      *  Loading from state file
    25  
    26  3. If any of the parent assets are **dirty** (currently we think all on-disk assets are **dirty**), then use the parent assets to generate and return **dirty**.
    27  
    28  4. If none of the parent assets are **dirty**, but the asset itself is on disk, then use the on-disk asset and return **dirty**.
    29  
    30  5. If none of the parent assets or this asset is **dirty**, but the asset is found in the state file, then use the asset from state file and return **NOT dirty**.
    31  
    32  6. If none of the parent assets are **dirty**, this asset is not **dirty**, and this asset is not found in the state file, then generate the asset using its parent assets and return **NOT dirty**.
    33  
    34  An example of the Asset:
    35  
    36  ```go
    37  type Asset interface {
    38      Dependencies() []Assets
    39      Generate(Parents) error
    40      Name() string
    41  }
    42  ```
    43  
    44  ## Writable Asset
    45  
    46  A writable asset is an asset that generates files to write to disk. These files could be for the user to consume as output from installer targets, such as install-config.yaml from the InstallConfig asset. Or these files could be used internally by the installer, such as the cert/key files generated by TLS assets.
    47  A writable asset can also be loaded from disk to construct.
    48  
    49  ```go
    50  type WritableAsset interface{
    51      Asset
    52      Files() []File
    53      Load(FileFetcher) (found bool, err error)
    54  }
    55  
    56  type File struct {
    57      Filename string
    58      Data []byte
    59  }
    60  
    61  // FileFetcher is passed to every Loadable asset when implementing
    62  // the Load() function. The FileFetcher enables the Loadable asset
    63  // to read specific file(s) from disk.
    64  type FileFetcher interface {
    65      // FetchByName returns the file with the given name.
    66      FetchByName(string) (*File, error)
    67      // FetchByPattern returns the files whose name match the given glob.
    68      FetchByPattern(*regexp.Regexp) ([]*File, error)
    69  }
    70  ```
    71  After being loaded and consumed by a children asset, the existing on-disk asset will be purged.
    72  E.g.
    73  
    74  ```shell
    75  $ openshift-install create install-config
    76  # Generate install-config.yaml
    77  
    78  $ openshift-install create manifests
    79  # Generate manifests/ and openshift/ dir, also remove install-config.yaml
    80  ```
    81  
    82  ## Target generation
    83  
    84  The installer uses depth-first traversal on the dependency graph, starting at the target nodes, generating all the dependencies of the asset before generating the asset itself. After all the target assets have been generated, the installer outputs the contents of the components of the targets to disk.
    85  
    86  ### Dirty detection
    87  
    88  An asset generation reports **DIRTY** when it detects that the components have been modified from previous run. For now the asset is considered dirty when it's on-disk.
    89  
    90  ### Example
    91  
    92  ```dot
    93  digraph G {
    94      size ="4,4";
    95      A1;
    96      A2;
    97      A3;
    98      A4;
    99      A5;
   100      A6;
   101      A5 -> {A3, A4};
   102      A3 -> {A1, A2};
   103      A6 -> {A1, A2};
   104  }
   105  ```
   106  
   107  When generating targets **A5 and A6**
   108  
   109  ```
   110  load state;
   111  
   112  A5: (A3, A4)
   113      A3: (A1, A2)
   114          A1:
   115              A1.generate(state)
   116              update state
   117          A2:
   118              A2.generate(state)
   119              update state
   120          set dirty if one of A1/A2 is dirty
   121          A3.generate(state): pass dirty if set
   122          update state
   123      A4:
   124          A4.generate(state)
   125          update state
   126      set dirty if one of A3/A4 is dirty
   127      A5.generate(state): pass dirty if set
   128      update state
   129  A6: (A1, A2)
   130      A1:
   131          reuse
   132      A2:
   133          reuse
   134      set dirty if one of A1/A2 is dirty
   135      A6.generate(state): pass dirty if set
   136      update state
   137  Flush A5 and A6 to disk
   138  ```
   139  
   140  ## Dependency graph
   141  
   142  The following graph shows the relationship between the various assets that the installer generates:
   143  
   144  ![Image depicting the resource dependency graph](resource_dep.svg)
   145  
   146  This graph is generated from the using the following command:
   147  
   148  ```sh
   149  bin/openshift-install graph | dot -Tsvg >docs/design/resource_dep.svg
   150  ```