github.com/muhammadn/cortex@v1.9.1-0.20220510110439-46bb7000d03d/docs/proposals/generalize-modules.md (about)

     1  ---
     2  title: "Generalize Modules Service to make it extensible"
     3  linkTitle: "Generalize Modules Service to make it extensible"
     4  weight: 1
     5  slug: generalize-modules
     6  ---
     7  
     8  - Author: @annanay25
     9  - Reviewers: @jtlisi, @pstibrany, @cyriltovena, @pracucci
    10  - Date: April 2020
    11  - Status: Accepted
    12  
    13  ## Overview
    14  
    15  Cortex uses modules to start and operate services with dependencies. Inter-service dependencies are specified in a map and passed to a module manager which ensures that they are initialised in the right order of dependencies. While this works really well, the implementation is tied in specifically to the Cortex struct and is not flexible for use with other projects like Loki, which also require similar forms of dependency management.
    16  
    17  We would like to extend modules in cortex to a generic dependency management framework, that can be used by any project with no ties to cortex.
    18  
    19  ## Specific goals
    20  
    21  - Framework should allow for reusing cortex modules and allow us to:
    22    - Add new modules
    23    - Overwrite the implementation of a current module
    24    - Manage dependencies
    25  - Framework should allow for building an application from scratch using the `modules` package, with no dependencies on Cortex. For ex: Remove code from Loki that was copied from `pkg/cortex/cortex.go`.
    26  
    27  
    28  ## Proposed Design
    29  
    30  ### Modules package
    31  
    32  To make the modules package extensible, we need to abstract away any Cortex specific details from the module manager. The proposed design is to:
    33  
    34  - Make a new component `Manager`, which is envisioned to be a central manager for all modules of the application. It stores modules & dependencies, and will be housed under a new package `pkg/util/modules`. `Manager` has the following methods for interaction:
    35  ```
    36     func (m *Manager) RegisterModule(name string, initFn func() (Service, error))
    37     func (m *Manager) AddDependency(name string, dependsOn... string) error
    38     func (m *Manager) InitModuleServices(target string) (map[string]services.Service, error)
    39  ```
    40  
    41  - Modules can be created by the application and registered with `modules.Manager` using `RegisterModule`. The parameters are:
    42    - `name`: Name of the module
    43    - `initFn`: A function that will be used to start the module. If it returns nil, and other modules depend on it, `InitModuleServices` will return an error.
    44  
    45  - Dependencies between modules can be added using `AddDependency`. The parameters to the function are:
    46    - `name`: Name of the module
    47    - `dependsOn`: A variadic list of modules that the module depends on.
    48  
    49    These need to be added before the call to `InitModuleServices`.
    50  
    51  - The application can be initialized by running `initFn`'s of all the modules in the right order of dependencies by invoking `InitModuleServices` with the target module name.
    52  
    53  
    54  ### Changes to `pkg/cortex`:
    55  
    56  - `WrappedService` present in the current `module` design will be deprecated. All `initFn`'s will be wrapped into `WrappedService` by default.
    57  
    58  - While the process of loading modules into `modules.Manager` should be remain as part of the `Cortex.New()` function, `InitModuleServices` should be part of `Cortex.Run()` and to enable this, `modules.Manager` would be made a member of the `Cortex` struct.
    59  
    60  
    61  ## Usage
    62  
    63  Following these changes, the Modules package will be a generic dependency management framework that can be used by any project.
    64  
    65  #### To use the modules framework:
    66  
    67  - Import the `pkg/util/modules` package, and initialize a new instance of the `Manager` using `modules.NewManager()`
    68  - Create components in the system that implement the services interface (present in `pkg/util/services`).
    69  - Register each of these components as a module using `Manager.RegisterModule()` by passing name of the module and `initFn` for the module.
    70  - To add dependencies between modules, use `Manager.AddDependency()`
    71  - Once all modules are added into `modules.Manager`, initialize the application by calling `Manager.InitModuleServices()` which initializes modules in the right order of dependencies.
    72  
    73  
    74  ## Future work
    75  
    76  - Extend the module manager to allow specifying multiple targets as opposed to a single target name supported currently.
    77  - Factor out `Run()` method to make it independent of Cortex. This will help reduce replicated code in the Loki project as well as help manage `modules.Manager` outside of the Cortex struct.