github.com/treeverse/lakefs@v1.24.1-0.20240520134607-95648127bfb0/docs/quickstart/learning-more-lakefs.md (about)

     1  ---
     2  title: Learn more about lakeFS
     3  description: Learn more about lakeFS here with links to resources including quickstart, samples, installation guides, and more. 
     4  parent: ⭐ Quickstart
     5  nav_order: 99
     6  previous: ["Using Actions and Hooks in lakeFS", "./actions-and-hooks.html"]
     7  ---
     8  
     9  # Learn more about lakeFS
    10  
    11  The [lakeFS quickstart]({% link quickstart/index.md %}) is just the beginning of your lakeFS journey 🛣️
    12  
    13  Here are some more resources to help you find out more about lakeFS. 
    14  
    15  ## Connecting lakeFS to your own object storage
    16  
    17  Enjoyed the quickstart and want to try out lakeFS against your own data? Here's how to run lakeFS locally as a Docker container locally connecting to an object store. 
    18  
    19  <div class="tabs">
    20    <ul>
    21      <li><a href="#on-aws-s3">AWS S3</a></li>
    22      <li><a href="#on-azure-blob">Azure Blob Storage</a></li>
    23      <li><a href="#on-google-gcs">Google Cloud Storage</a></li>
    24      <li><a href="#on-minio">MinIO</a></li>
    25    </ul> 
    26    <div markdown="1" id="on-aws-s3">
    27  
    28  Note: Make sure the Quickstart Docker Compose from the previous steps isn't also running as you'll get a port conflict.
    29  {: .note }
    30  
    31     ```bash
    32  docker run --pull always -p 8000:8000 \
    33     -e LAKEFS_BLOCKSTORE_TYPE='s3' \
    34     -e AWS_ACCESS_KEY_ID='YourAccessKeyValue' \
    35     -e AWS_SECRET_ACCESS_KEY='YourSecretKeyValue' \
    36     treeverse/lakefs run --local-settings
    37     ```
    38  
    39    </div>
    40    <div markdown="1" id="on-azure-blob">
    41  
    42  Note: Make sure the Quickstart Docker Compose from the previous steps isn't also running as you'll get a port conflict.
    43  {: .note }
    44  
    45     ```bash
    46  docker run --pull always -p 8000:8000 \
    47     -e LAKEFS_BLOCKSTORE_TYPE='azure' \
    48     -e LAKEFS_BLOCKSTORE_AZURE_STORAGE_ACCOUNT='YourAzureStorageAccountName' \
    49     -e LAKEFS_BLOCKSTORE_AZURE_STORAGE_ACCESS_KEY='YourAzureStorageAccessKey' \
    50     treeverse/lakefs run --local-settings
    51     ```
    52  
    53    </div>
    54    <div markdown="1" id="on-google-gcs">
    55  
    56  Note: Make sure the Quickstart Docker Compose from the previous steps isn't also running as you'll get a port conflict.
    57  {: .note }
    58  
    59     ```bash
    60  docker run --pull always -p 8000:8000 \
    61     -e LAKEFS_BLOCKSTORE_TYPE='gs' \
    62     -e LAKEFS_BLOCKSTORE_GS_CREDENTIALS_JSON='YourGoogleServiceAccountKeyJSON' \
    63     treeverse/lakefs run --local-settings
    64     ```
    65  where you will replace ```YourGoogleServiceAccountKeyJSON``` with JSON string that contains your Google service account key.
    66  
    67  If you want to use the JSON file that contains your Google service account key instead of JSON string (as in the previous command) then go to the directory where JSON file is stored and run the command with local parameters:
    68  
    69     ```bash
    70  docker run --pull always -p 8000:8000 \
    71     -v $PWD:/myfiles \
    72     -e LAKEFS_BLOCKSTORE_TYPE='gs' \
    73     -e LAKEFS_BLOCKSTORE_GS_CREDENTIALS_FILE='/myfiles/YourGoogleServiceAccountKey.json' \
    74     treeverse/lakefs run --local-settings
    75     ```
    76  This command will mount your present working directory (PWD) within the container and will read the JSON file from your PWD.
    77  
    78    </div>
    79    <div markdown="1" id="on-minio">
    80  
    81  To use lakeFS with MinIO (or other S3-compatible object storage), use the following example:
    82  
    83  Note: Make sure the Quickstart Docker Compose from the previous steps isn't also running as you'll get a port conflict.
    84  {: .note }
    85  
    86     ```bash
    87  docker run --pull always -p 8000:8000 \
    88     -e LAKEFS_BLOCKSTORE_TYPE='s3' \
    89     -e LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE='true' \
    90     -e LAKEFS_BLOCKSTORE_S3_ENDPOINT='http://<minio_endpoint>' \
    91     -e LAKEFS_BLOCKSTORE_S3_DISCOVER_BUCKET_REGION='false' \
    92     -e LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID='<minio_access_key>' \
    93     -e LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY='<minio_secret_key>' \
    94     treeverse/lakefs run --local-settings
    95     ```
    96  
    97    </div>
    98  </div>
    99  
   100  ## Deploying lakeFS
   101  
   102  Ready to do this thing for real? The deployment guides show you how to deploy lakeFS [locally]({% link howto/deploy/onprem.md %}) (including on [Kubernetes][onprem-k8s]) or on [AWS]({% link howto/deploy/aws.md %}), [Azure]({% link howto/deploy/azure.md %}), or [GCP]({% link howto/deploy/gcp.md %}). 
   103  
   104  Alternatively you might want to have a look at [lakeFS Cloud](https://lakefs.cloud/) which provides a fully-managed, SOC-2 compliant, lakeFS service. 
   105  
   106  ## lakeFS Samples
   107  
   108  The [lakeFS Samples](https://github.com/treeverse/lakeFS-samples) GitHub repository includes some excellent examples including: 
   109  
   110  * How to implement multi-table transaction on multiple Delta Tables
   111  * Notebooks to show integration of lakeFS with Spark, Python, Delta Lake, Airflow and Hooks.
   112  * Examples of using lakeFS webhooks to run automated data quality checks on different branches.
   113  * Using lakeFS branching features to create dev/test data environments for ETL testing and experimentation.
   114  * Reproducing ML experiments with certainty using lakeFS tags.
   115  
   116  ## lakeFS Community
   117  
   118  The lakeFS community is important to us. Our **guiding principles** are: 
   119  
   120  * Fully open, in code and conversation
   121  * We learn and grow together
   122  * Compassion and respect in every interaction
   123  
   124  We'd love for you to join [our **Slack group**](https://lakefs.io/slack) and come and introduce yourself on `#announcements-and-more`. Or just lurk and soak up the vibes 😎
   125  
   126  If you're interested in getting involved in the development of lakeFS, head over our [the **GitHub repo**](https://github.com/treeverse/lakeFS) to look at the code and peruse the issues. The comprehensive [contributing]({% link project/contributing.md %}) document should have you covered on next steps but if you've any questions the `#dev` channel on [Slack](https://lakefs.io/slack) will be delighted to help. 
   127  
   128  We love speaking at meetups and chatting to community members at them - you can find a list of these [here](https://lakefs.io/community/). 
   129  
   130  Finally, make sure to drop by to say hi on [Twitter](https://twitter.com/lakeFS), [Mastodon](https://data-folks.masto.host/@lakeFS), and [LinkedIn](https://www.linkedin.com/company/treeverse/) 👋🏻
   131  
   132  ## lakeFS Concepts and Internals
   133  
   134  We describe lakeFS as "_Git for data_" but what does that actually mean? Have a look at the [concepts]({% link understand/model.md %}) and [architecture]({% link understand/architecture.md %}) guides, as well as the explanation of [how merges are handled]({% link understand/how/merge.md %}). To go deeper you might be interested in [the internals of versioning]({% link understand/how/versioning-internals.md %}) and our [internal database structure]({% link understand/how/kv.md %}).
   135  
   136  
   137  [onprem-k8s]:  {% link howto/deploy/onprem.md %}#k8s