github.com/tickoalcantara12/micro/v3@v3.0.0-20221007104245-9d75b9bcbab9/docs/architecture/README.md (about)

     1  ---
     2  title: Architecture
     3  keywords: micro
     4  tags: [micro]
     5  sidebar: home_sidebar
     6  permalink: /architecture
     7  summary: Architecture - Design, goals and tradeoffs
     8  ---
     9  
    10  ## Architecture
    11  {: .no_toc }
    12  
    13  This document covers the architecture of micro, design decisions and tradeoffs made.
    14  
    15  ## Contents
    16  
    17  * TOC
    18  {:toc}
    19  
    20  ## Overview
    21  
    22  Micro is a platform for cloud native development. It takes the concept of distributed systems and codifies it as a software 
    23  design pattern using microservices and related primitives. The overall goal of Micro is to abstract away cloud infrastructure 
    24  and to define a set of building blocks which can be used to write cloud services aka microservices, APIs or distributed systems.
    25  
    26  ## Rationale
    27  
    28  Micro was born out of a pain with existing development models in the cloud. The majority of time has been spent coming to grips 
    29  with complex cloud native infrastructure as opposed to software design. Cloud is in its infancy, predominantly focused on 
    30  computing and infrastructure services. Development has largely lagged behind and there has been no vertically integrated 
    31  solution to make sense of it all.
    32  
    33  PaaS in prior generations like Heroku provided a full solution for web application development using Rails as the developer angle. 
    34  As they and others iterated though, multi-language became an attractive solution and one that really muted its opinionated approach 
    35  and diluted the overall focus of developer productivity.
    36  
    37  Our focus is developer productivity and we see that as one achieved through constraints. Every dominant platform constrained itself 
    38  to a specific model and then software exploded to leverage it. We think Cloud is much like Mobile in the sense that it requires 
    39  an iPhone or Android like approach to managing its complexity. Cloud needs a development model, it needs an operating system and 
    40  we think Micro can provide that through a completely vertically integrated solution.
    41  
    42  Micro is Android for Cloud. PaaS 3.0. Or better known as a cloud operating system. As a framework Rails went far but it was incomplete 
    43  without the likes of Heroku to host it. Spring fell short as well and was acquired by VMWare to try make sense of this platform model. 
    44  We think Micro needs to take inspiration from Android, to encapsulate all the concerns of cloud, abstract away the hardware and 
    45  define a software lifecycle, development and consumption model for it.
    46  
    47  ## Design
    48  
    49  Micro is designed as 3 core components:
    50  
    51  - Server - A single server which acts as the runtime for a cloud platform
    52  - Clients - Entrypoints via command line, api gateway and gRPC proxy/sdks
    53  - Framework - A Go framework specifically designed for writing microservices
    54  
    55  Micro services are defined as:
    56  
    57  - Domain Driven - APIs are written in protobuf format and act as the API contract for the service
    58  - Engineered by Design - Written using the micro service framework so that they employ a standard model
    59  - Reusable building blocks - Each service acts as a building block for the next. In this model, everything is a service
    60  
    61  For in-depth material see the [Reference](/reference). This doc will otherwise cover things at a high level.
    62  
    63  **Server Architecture**
    64  
    65  Below is the architecture of the Micro Server which is composed of a number of independent services. Micro itself 
    66  is accessed through a HTTP or gRPC API. Micro is runtime agnostic and provides plugin for running on many 
    67  different architectures whether its local, kubernetes or beyond.
    68  
    69  <img src="../images/micro-3.0.png">
    70  
    71  **Service Architecture**
    72  
    73  A Micro service makes use of a Go Service Framework which provides Go interfaces for accessing the services provided 
    74  by the Micro Server. It also initialises your server to run on the underlying runtime and exposes your services 
    75  via a gRPC server. Micro makes it so all you have to think about is the business logic. 
    76  
    77  <img src="../images/micro-service-3.0.png">
    78  
    79  ## Server
    80  
    81  The server acts as an abstraction for the underlying infrastructure. It provides distributed systems primitives as building 
    82  blocks for writing microservices. We'll outline those below. The rationale behind defining a single server is based on 
    83  the understanding that all systems at scale inevitably need a platform with common building blocks. In the beginning this 
    84  supports monolithic apps for building CRUD and over time evolves to firstly separate the frontend from backend and then 
    85  starts to provide scalable systems for persistence, events, config, auth, etc.
    86  
    87  The server encapsulates all these concerns while embracing the distributed systems model and running each core concern 
    88  as a separate service and process. Its the unix philosophy done well for software composition. As a whole though 
    89  micro is a single monolithic codebase after many separate libraries were consolidated and inevitably go-micro got 
    90  folded into micro itself.
    91  
    92  ## Clients
    93  
    94  Clients are effectively entrypoints or forms of access for the Micro server. The server runs both an API gateway and gRPC proxy. 
    95  The API is deemed as a public facing http api gateway that converts http/json with path based routing to rpc requests. The 
    96  world still built around http and cloud services emerging with that pattern, it feels like this is the right approach.
    97  
    98  The gRPC proxy is a forward looking entrypoint, where we see it as a way of extending the service-to-service communication 
    99  to the developers laptop or other environments. At the moment it acts as a proxy for command line to remote environments.
   100  
   101  The command line interface itself is deemed as a client and the defacto way to interact with Micro. We do not offer a web UI. 
   102  The assumption is web is a context switch and we'd prefer developers to stay in the terminal. The CLI is extensible in the 
   103  sense that every service you run becomes a subcommand. By doing so we take the API path based model and service decomposition 
   104  to the CLI as well.
   105  
   106  The final piece that is a continuous work in progress is gRPC generated clients which can be found in micro/client/sdk. 
   107  Eventually they will be published to various package managers but for now are all routed in one directory. These are built from 
   108  the protos defined for each core service within Micro, enabling multi-language access. We do not assume services to be built 
   109  multi-language but consumption of Micro and services may extend outward.
   110  
   111  ## Framework
   112  
   113  The Go service framework is a core piece which comes from the original go-micro framework started in 2015. This framework offered 
   114  core distributed systems primitives as Go interfaces and made them pluggable. With its complexity and overlap with Micro we 
   115  decided the best thing was to merge the two and create a Service Framework within Micro to define the defacto standard for building 
   116  Micro Services in Go. The framework provides pluggable abstractions with pre-initialised defaults. The developer 
   117  will import and use any of the packages within the framework without any initialisation, they in turn speak to the micro server 
   118  or basically the core services via gRPC. 
   119  
   120  For the developer, this is their main point of interaction when writing code. We employ a build, run, manage philosophy where 
   121  build actually starts with putting something in the hands of the developer. Import the framework, start writing code, 
   122  import various packages as needed when you have to get config, check auth, store key-value data. Then run your service using 
   123  Micro itself. The server abstracts away the infra, the service is built to run on Micro and everything else is taken care of.
   124  
   125  ## Services
   126  
   127  The services provided by the server are below. These are identified as the core concerns for a platform and distributed systems 
   128  development. In time this may evolve to include synchronization (locking & leadership election), sagas pattern, etc but for 
   129  now we want to provide just the core primitives.
   130  
   131  - **API** - HTTP Gateway which dynamically maps http/json requests to RPC using path based resolution
   132  - **Auth** - Authentication and authorization out of the box using jwt tokens and rule based access control.
   133  - **Broker** - Ephemeral pubsub messaging for asynchronous communication and distributing notifications
   134  - **Config** - Dynamic configuration and secrets management for service level config without the need to restart
   135  - **Events** - Event streaming with ordered messaging, replay from offsets and persistent storage
   136  - **Network** - Inter-service networking, isolation and routing plane for all internal request traffic
   137  - **Proxy** - gRPC identity aware proxy used for remote access and any external grpc request traffic
   138  - **Runtime** - Service lifecyle and process management with support for source to running auto build
   139  - **Registry** - Centralised service discovery and API endpoint explorer with feature rich metadata
   140  - **Store** - Key-Value storage with TTL expiry and persistent crud to keep microservices stateless
   141  
   142  The assumptions made are by building as independent services we can scope the concerns and domain boundaries appropriately but 
   143  cohesively build a whole system that acts as a platform or a cloud operating system. Much like Android for Mobile we think 
   144  Micro could become a definitive server for the Cloud.
   145  
   146  ## Environments
   147  
   148  Environments are a concept introduced in v3 which allow you to switch between entirely different deployments of Micro. 
   149  The term 'environment' basically implies what we already understand as separate development or runtime 
   150  environments whether it be local, staging, production or something by another name. You can think of it as a deployment of 
   151  Micro targeted by the micro proxy which runs within every server.
   152  
   153  Environments as a built in concept allow us to create a workflow that let's us seamlessly switch between them using a single tool. 
   154  In kubernetes this is known as contexts, in Micro we understand these as entirely separate running environments for our software. 
   155  To us it just makes sense to build it in. An environment might be a deployment in AWS on managed k8s or multiple separate regional 
   156  deployments identified be [name]-[region] or similar.
   157  
   158  Environments are homed by the micro proxy for remote access. By default locally this runs on :8081 and you simply do `micro env set local` 
   159  to have all commands run against it. The CLI proxies all commands to the micro proxy and in turn that send the request to the relevant 
   160  services hosted by the Micro server.
   161  
   162  Micro bakes in 3 environments by default; local, dev (free cloud hosted), platform (paid cloud hosted). The goal is to move 
   163  beyond open source into something that seamless integrates the cloud for all developers everywhere.
   164  
   165  ## Namespaces
   166  
   167  Namespaces in Micro are a form of resource isolation and multi-tenancy. Most people know namespacing from linux lxc or kubernetes. 
   168  This overlaps but also evolves from how micro previously used it. Micro used to use logical namespacing through names prefixed 
   169  with a namespace. This allowed multiple tenants to exist in a single environment and using explicit naming to identify them. 
   170  
   171  As we're starting to see Micro as more of a platform and cloud operating system there was a need to codify this concept across 
   172  all resources. Namespacing is incorporated across every service so that your services are isolated in the registry and network, 
   173  so that your data is isolated in their specific databases and tables and so authentication and accounts are tied to individual 
   174  namespaces.
   175  
   176  Namespaces encapsulate accounts, resources and networking routing. Namespaces beyond this are also useful for feature and branch 
   177  development. It enables subsets of services to be deployed into an isolated namespace for testing. This enables a single 
   178  environment for development to be used in a real world context and for production to enable feature flagging where needed.
   179  
   180  ## Glossary
   181  
   182  - Server - the micro server which encapsulates infrastructure and distributed systems concerns
   183  - Clients - entrypoints for accessing the server and services run by Micro
   184  - Framework - the Go service framework used to write Micro services
   185  - Environment - an instance of the micro service running locally or in a remote environment
   186  - Namespaces - isolated grouping of resources and multi-tenancy as a concept across Micro