github.com/treeverse/lakefs@v1.24.1-0.20240520134607-95648127bfb0/docs/understand/enterprise/fluffy-configuration.md (about)

     1  ---
     2  title: Fluffy Server Configuration
     3  description: Configuration reference for Fluffy Server
     4  parent: lakeFS Enterprise
     5  grand_parent: Understanding lakeFS
     6  redirect_from:
     7  - /understand/fluffy-configuration.html
     8  ---
     9  
    10  # Fluffy Server Configuration
    11  
    12  {% include toc.html %}
    13  
    14  Configuring Fluffy using a YAML configuration file and/or environment variables.
    15  The configuration file's location can be set with the '--config' flag. If not specified, the first file found in the following order will be used:
    16  1. ./config.yaml
    17  1. `$HOME`/fluffy/config.yaml
    18  1. /etc/fluffy/config.yaml
    19  1. `$HOME`/.fluffy.yaml
    20  
    21  Configuration items can be controlled by environment variables, see [below](#using-environment-variables).
    22  
    23  
    24  ## Reference
    25  
    26  This reference uses `.` to denote the nesting of values.
    27  
    28  * `logging.format` `(one of ["json", "text"] : "text")` - Format to output log message in
    29  * `logging.level` `(one of ["TRACE", "DEBUG", "INFO", "WARN", "ERROR", "NONE"] : "INFO")` - Logging level to output
    30  * `logging.audit_log_level` `(one of ["TRACE", "DEBUG", "INFO", "WARN", "ERROR", "NONE"] : "DEBUG")` - Audit logs level to output.
    31  
    32    **Note:** In case you configure this field to be lower than the main logger level, you won't be able to get the audit logs
    33    {: .note }
    34  * `logging.output` `(string : "-")` - A path or paths to write logs to. A `-` means the standard output, `=` means the standard error.
    35  * `logging.file_max_size_mb` `(int : 100)` - Output file maximum size in megabytes.
    36  * `logging.files_keep` `(int : 0)` - Number of log files to keep, default is all.
    37  * `logging.trace_request_headers` `(bool : false)` - If set to `true` and logging level is set to `TRACE`, logs request headers.
    38  * `listen_address` `(string : "0.0.0.0:8000")` - A `<host>:<port>` structured string representing the address to listen on
    39  * `database` - Configuration section for the Fluffy key-value store database. The database must be shared between lakeFS & Fluffy
    40    + `database.type` `(string ["postgres"|"dynamodb"|"local"] : )` - Fluffy database type
    41    + `database.postgres` - Configuration section when using `database.type="postgres"`
    42      + `database.postgres.connection_string` `(string : "postgres://localhost:5432/postgres?sslmode=disable")` - PostgreSQL connection string to use
    43      + `database.postgres.max_open_connections` `(int : 25)` - Maximum number of open connections to the database
    44      + `database.postgres.max_idle_connections` `(int : 25)` - Maximum number of connections in the idle connection pool
    45      + `database.postgres.connection_max_lifetime` `(duration : 5m)` - Sets the maximum amount of time a connection may be reused `(valid units: ns|us|ms|s|m|h)`
    46    + `database.dynamodb` - Configuration section when using `database.type="dynamodb"`
    47      + `database.dynamodb.table_name` `(string : "kvstore")` - Table used to store the data
    48      + `database.dynamodb.scan_limit` `(int : 1025)` - Maximal number of items per page during scan operation
    49  
    50        **Note:** Refer to the following [AWS documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Limit) for further information
    51        {: .note }
    52      + `database.dynamodb.endpoint` `(string : )` - Endpoint URL for database instance
    53      + `database.dynamodb.aws_region` `(string : )` - AWS Region of database instance
    54      + `database.dynamodb.aws_profile` `(string : )` - AWS named profile to use
    55      + `database.dynamodb.aws_access_key_id` `(string : )` - AWS access key ID
    56      + `database.dynamodb.aws_secret_access_key` `(string : )` - AWS secret access key
    57      + **Note:** `endpoint` `aws_region` `aws_access_key_id` `aws_secret_access_key` are not required and used mainly for experimental purposes when working with DynamoDB with different AWS credentials.
    58        {: .note }
    59      + `database.dynamodb.health_check_interval` `(duration : 0s)` - Interval to run health check for the DynamoDB instance (won't run if equal to 0).
    60    + `database.local` - Configuration section when using `database.type="local"`
    61      + `database.local.path` `(string : "~/fluffy/metadata")` - Local path on the filesystem to store embedded KV metadata
    62      + `database.local.sync_writes` `(bool: true)` - Ensure each write is written to the disk. Disable to increase performance
    63      + `database.local.prefetch_size` `(int: 256)` - How many items to prefetch when iterating over embedded KV records
    64      + `database.local.enable_logging` `(bool: false)` - Enable trace logging for local driver
    65  * `auth` - Configuration section for the Fluffy authentication services, like SAML or OIDC.
    66    + `auth.encrypt.secret_key` `(string : required)` - Same value given to lakeFS. A random (cryptographically safe) generated string that is used for encryption and HMAC signing
    67    + `auth.logout_redirect_url` `(string : "/auth/login")` - The address to redirect to after a successful logout, e.g. login.
    68    + `auth.post_login_redirect_url` `(string : '')` - Required when SAML is enabled. The address to redirect after a successful login. For most common configurations, setting to `/` will redirect to lakeFS homepage.
    69    + `auth.serve_listen_address` `(string : '')` - If set, an endpoint serving RBAC requests binds to this address.
    70    + `auth.serve_disable_authentication` `(bool : false)` - Unsafe. Disables authentication to the RBAC server.
    71    + `auth.ldap`
    72      + `auth.ldap.server_endpoint` `(string : required)` - The LDAP server address, e.g. 'ldaps://ldap.company.com:636'
    73      + `auth.ldap.bind_dn` `(string : required)` - The bind string, e.g. 'uid=<bind-user-name>,ou=Users,o=<org-id>,dc=<company>,dc=com'
    74      + `auth.ldap.bind_password` `(string : required)` - The password for the user to bind.
    75      + `auth.ldap.username_attribute` `(string : required)` - The user name attribute, e.g. 'uid'
    76      + `auth.ldap.user_base_dn` `(string : required)` - The search request base dn, e.g. 'ou=Users,o=<org-id>,dc=<company>,dc=com'
    77      + `auth.ldap.user_filter` `(string : required)` - The search request user filter, e.g. '(objectClass=inetOrgPerson)'
    78      + `auth.ldap.connection_timeout_seconds` `(int : required)` - The timeout for a single connection
    79      + `auth.ldap.request_timeout_seconds` `(int : required)` - The timeout for a single request
    80    + `auth.saml` Configuration section for SAML
    81      + `auth.saml.enabled` `(bool : false)` - Enables SAML Authentication.
    82      + `auth.saml.sp_root_url` `(string : '')` - The base lakeFS-URL, e.g. 'https://<lakefs-url>'
    83      + `auth.saml.sp_x509_key_path` `(string : '')` - The path to the private key, e.g '/etc/saml_certs/rsa_saml_private.cert'
    84      + `auth.saml.sp_x509_cert_path` `(string : '')` - The path to the public key, '/etc/saml_certs/rsa_saml_public.pem'
    85      + `auth.saml.sp_sign_request` `(bool : 'false')` SPSignRequest some IdP require the SLO request to be signed
    86      + `auth.saml.sp_signature_method` `(string : '')` SPSignatureMethod optional valid signature values depending on the IdP configuration, e.g. 'http://www.w3.org/2001/04/xmldsig-more#rsa-sha256'
    87      + `auth.saml.idp_metadata_url` `(string : '')` - The URL for the metadata server, e.g. 'https://<adfs-auth.company.com>/federationmetadata/2007-06/federationmetadata.xml'
    88      + `auth.saml.idp_skip_verify_tls_cert` `(bool : false)` - Insecure skip verification of the IdP TLS certificate, like when signed by a private CA
    89      + `auth.saml.idp_authn_name_id_format` `(string : 'urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified')` - The format used in the NameIDPolicy for authentication requests
    90      + `auth.saml.idp_request_timeout` `(duration : '10s')` The timeout for remote authentication requests.
    91      + `auth.saml.external_user_id_claim_name` `(string : '')` - The claim name to use as the user identifier with an IdP mostly for logout
    92    + `auth.oidc` Configuration section for OIDC
    93      + `auth.oidc.enabled` `(bool : false)` - Enables OIDC Authentication.
    94      + `auth.oidc.url` `(string : '')` - The OIDC provider url, e.g. 'https://oidc-provider-url.com/'
    95      + `auth.oidc.client_id` `(string : '')` - The application's ID.
    96      + `auth.oidc.client_secret` `(string : '')` - The application's secret.
    97      + `auth.oidc.callback_base_url` `(string : '')` - A default callback address of the Fluffy server.
    98      + `auth.oidc.callback_base_urls` `(string[] : '[]')`
    99    + **Note:** You may configure a list of URLs that the OIDC provider may redirect to. This allows lakeFS to be accessed from multiple hostnames while retaining federated auth capabilities.
   100      If the provider redirects to a URL not in this list, the login will fail. This property and callback_base_url are mutually exclusive.
   101      {: .note }
   102      + `auth.oidc.authorize_endpoint_query_parameters` `(bool : map[string]string)` - key/value parameters that are passed to a provider's authorization endpoint.
   103      + `auth.oidc.logout_endpoint_query_parameters` `(string[] : '[]')` - The query parameters that will be used to redirect the user to the OIDC provider after logout, e.g. '[returnTo, https://<lakefs.ingress.domain>/oidc/login]'
   104      + `auth.oidc.logout_client_id_query_parameter` `(string : '')` - The claim name that represents the client identifier in the OIDC provider
   105      + `auth.oidc.additional_scope_claims` `(string[] : '[]')` - Specifies optional requested permissions, other than `openid` and `profile` that are being used.
   106    + `auth.cache` Configuration section for RBAC service cache
   107      + `auth.cache.enabled` `(bool : true)` - Enables RBAC service cache
   108      + `auth.cache.size` `(int : 1024)` - Number of users, policies and credentials to cache.
   109      + `auth.cache.ttl` `(duration : 20s)` - Cache items time to live expiry.
   110      + `auth.cache.jitter` `(duration : 3s)` - Cache items time to live jitter.
   111    + `auth.external` - Configuration section for the external authentication methods
   112      + `auth.external.aws_auth` - Configuration section for authenticating to lakeFS using AWS presign get-caller-identity request: [External Principals AWS Auth]({% link reference/security/external-principals-aws.md %})
   113        + `auth.external.aws_auth.enabled` `(bool : false)` - If true, external principals API will be enabled, e.g auth service and login api's.
   114        + `auth.external.aws_auth.get_caller_identity_max_age` `(duration : 15m)` - The maximum age in seconds for the GetCallerIdentity request to be valid, the max is 15 minutes enforced by AWS, smaller TTL can be set.
   115        + `auth.authentication_api.external_principals_enabled` `(bool : false)` - If true, external principals API will be enabled, e.g auth service and login api's.
   116        + `auth.external.aws_auth.valid_sts_hosts` `([]string)` - The default are all the valid AWS STS hosts (`sts.amazonaws.com`, `sts.us-east-2.amazonaws.com` etc).
   117        + `auth.external.aws_auth.required_headers` `(map[string]string : )` - Headers that must be present by the client when doing login request (e.g `X-LakeFS-Server-ID: <lakefs.ingress.domain>`).
   118        + `auth.external.aws_auth.optional_headers` `(map[string]string : )` - Optional headers that can be present by the client when doing login request.
   119        + `auth.external.aws_auth.http_client.timeout` `(duration : 10s)` - The timeout for the HTTP client used to communicate with AWS STS.
   120        + `auth.external.aws_auth.http_client.skip_verify` `(bool : false)` - Skip SSL verification with AWS STS.
   121        {: .ref-list }
   122  
   123  ## Using Environment Variables
   124  
   125  All the configuration variables can be set or overridden using environment variables.
   126  To set an environment variable, prepend `FLUFFY_` to its name, convert it to upper case, and replace `.` with `_`:
   127  
   128  For example, `logging.format` becomes `FLUFFY_LOGGING_FORMAT`, `auth.saml.enabled` becomes `FLUFFY_AUTH_SAML_ENABLED`, etc.