github.com/atlassian/git-lob@v0.0.0-20150806085256-2386a5ed291a/README.md (about)

     1  # git-lob #
     2  ## IMPORTANT ##
     3  **TL;DR: use [Git LFS](https://git-lfs.github.com) instead of this project.**
     4  
     5  **git-lob** was an experiment into Git large file handling by [Atlassian]
     6  (https://www.atlassian.com) in late 2014 / early 2015. Unbeknownst to us, GitHub
     7  had already started working on this problem too and announced [Git LFS]
     8  (https://git-lfs.github.com) in May 2015. 
     9  
    10  Atlassian are keenly aware that it's in the community's interest not to fragment
    11  this space any further, and GitHub's thinking turned out to be very similar to
    12  our own, even picking the same language, [Go](http://golang.org). Therefore
    13  we've shelved this project and are now contributing to 
    14  [Git LFS](https://git-lfs.github.com) instead.
    15  
    16  We're releasing the source to git-lob anyway, even though we have no 
    17  intention of developing it further. We'd always intended to make it open source, 
    18  and others might find this code useful. We'll probably be porting some of the features
    19  here to Git LFS ourselves but other people are welcome to pick some things up too.
    20  Here are some of the features that could be ported:
    21  
    22  1. SSH native API (we're porting this right now)
    23  2. Support for large file storage on S3 and file stores (no server required - probably won't be ported)
    24  3. Chunked uploads/downloads for easier resuming
    25  4. Pushing / fetching binaries separately from git commits
    26  5. Pruning old binaries
    27  6. Include / exclude paths (only download binaries in areas you work in)
    28  7. Binary deltas using VCDIFF to reduce upload/download time
    29  8. Shared local binary stores
    30  
    31  You're free to experiment with this project of course but it is 
    32  **not** supported by Atlassian, nor will it have any further updates.
    33  
    34  # Original documentation #
    35  ## About ##
    36  git-lob is a git extension for efficiently handling binary files in [Git](http://git-scm.com). Instead of storing the binary file content inside the git repo, it is hashed & externalised, with the git commit only referring to the hash. The binary store is synchronised separately from the commits. 
    37  
    38  ## Why is it useful? ##
    39  1. Adding large binaries to git slows it down & makes cloning slow
    40  2. Most people don't need or want to have every version ever of large binary files on their disk when they only use the more recent ones
    41  3. We need efficient ways to synchronise large binaries via deltas
    42  
    43  git-lob keeps your Git repository smaller and faster, while providing rich functionality for managing binary files linked to the source code. 
    44  
    45  The target audience is a project team on Windows, Mac or Linux who generally collaborate through one 'master' upstream repo (and maybe a small number of forks) and want to use the speed & power of Git for all their code & smaller resources while having lots of large binary files which are potentially inter-dependent with the history of that code.
    46  
    47  ## Why did you write it, rather than use X, and why Go? ##
    48  [Discussed in more detail...](doc/why.md)
    49  
    50  ## Prerequisites ##
    51  
    52  You must have git 1.8+ installed and on your path. 
    53  
    54  ## Installation ##
    55  ### Install from source ###
    56  
    57  If you want to build from source, make sure you have a Go environment already set up, with $GOPATH/bin already on your $PATH (on Windows, %GOPATH%\bin on your %PATH%). Then run the following in a console:
    58  ```bash
    59  > go get github.com/atlassian/git-lob
    60  > go install github.com/atlassian/git-lob
    61  ```
    62  
    63  Now edit your main .gitconfig file in your user directory and add a new filter definition. 
    64  
    65  On Mac/Linux:
    66  ```ini
    67  [filter "lob"]
    68    clean = "$GOPATH/bin/git-lob filter-clean %f"
    69    smudge = "$GOPATH/bin/git-lob filter-smudge %f"
    70    required = true
    71  ```
    72  
    73  On Windows:
    74  ```ini
    75  [filter "lob"]
    76    clean = "%GOPATH%/bin/git-lob.exe filter-clean %f"
    77    smudge = "%GOPATH%/bin/git-lob.exe filter-smudge %f"
    78    required = true
    79  ```
    80  
    81  You can expand $GOTPATH/%GOPATH% inline if you need to support usage where GOPATH is not defined. Again on Windows, always use forward slashes, for example c:/path/to/git-lob.exe
    82  
    83  ### Install From binary distribution ###
    84  If you downloaded a precompiled version for your platform, just extract git-lob[.exe] to a location of your choice.
    85  
    86  Now edit your main .gitconfig file in your user directory and add a new filter definition as shown in the 'Install from source' section but set the path to git-lob[.exe] to be wherever you extracted it
    87  
    88  ## Repository Configuration ##
    89  To start putting binary files into git-lob you need to create or modify a .gitattributes file in the root of your repository:
    90  ```ini
    91  *.png filter=lob -crlf
    92  *.jpg filter=lob -crlf
    93  *.zip filter=lob -crlf
    94  *.tiff filter=lob -crlf
    95  *.tga filter=lob -crlf
    96  *.dds filter=lob -crlf
    97  *.bmp filter=lob -crlf
    98  *.mov filter=lob -crlf
    99  ```
   100  Include a line for all file types you want to be handled by git-lob. After saving this file, every time you 'git add' on a matching file, its content will be excluded from Git and put in the separate binary store, referenced by SHA in the commit.
   101  
   102  ## Configuring remote storage ##
   103  
   104  Binaries in git-lob are not stored in the regular git repo, but a corresponding
   105  binary store must always exist to provide the actual binary content. A remote
   106  in git usually only gives you the real git repo, so git-lob needs to expand
   107  the configuration parameters to git remotes to specify the location of the 
   108  corresponding remote binary store. 
   109  
   110  The parameters depend on the type of binary storage ('provider') being used; see `git-lob providers` for a list of available providers and `git-lob provider <provider>` for specific details of one provider.
   111  
   112  As an example, let's take the 'filesystem' provider, which simply uses the OS's
   113  file system as a remote transport (obviously very simplistic):
   114  
   115  ```ini
   116  [remote "origin"]
   117      # these 2 lines are standard git
   118      url = ssh://git@bitbucket.org/something/somthing.git
   119      fetch = +refs/heads/*:refs/remotes/origin/*
   120      # these next 2 lines are required to configure the remote binary store
   121      git-lob-provider = filesystem
   122      git-lob-path = /Volumes/shared/something/something/binary/store
   123  ```
   124  Other providers may require other parameters. It's important to note that you
   125  can share a binary store among multiple remote repos if you wish, much like
   126  the local git-lob.sharedstore option, since binaries are stored by SHA. 
   127  Identical file content in multiple repos can be stored only once this way.
   128  Of course, access control may be an issue to consider here though.
   129  
   130  ## Other options ##
   131  git-lob supports a number of command-line parameters, and configuration parameters in your .gitconfig (user or repository level). Please call 'git lob help' for general help and a list of main commands, and 'git lob help topics' to list other topics.
   132  
   133  # More Information #
   134  ## Host setup ##
   135  
   136  * [How to configure Amazon S3 for team use](doc/s3_setup.md)
   137  * [The Smart Sync Protocol](doc/smart_protocol.md)
   138  
   139  ## Information for git-lob developers ##
   140  
   141  * [Initial developer setup](doc/devsetup.md)
   142  * [Coding conventions](doc/conventions.md)
   143  * [Third-party dependencies](doc/dependencies.md)
   144  * [A note about dates in fetch & prune](doc/fetch_prune_dates.md)