github.com/atlassian/git-lob@v0.0.0-20150806085256-2386a5ed291a/README.md (about) 1 # git-lob # 2 ## IMPORTANT ## 3 **TL;DR: use [Git LFS](https://git-lfs.github.com) instead of this project.** 4 5 **git-lob** was an experiment into Git large file handling by [Atlassian] 6 (https://www.atlassian.com) in late 2014 / early 2015. Unbeknownst to us, GitHub 7 had already started working on this problem too and announced [Git LFS] 8 (https://git-lfs.github.com) in May 2015. 9 10 Atlassian are keenly aware that it's in the community's interest not to fragment 11 this space any further, and GitHub's thinking turned out to be very similar to 12 our own, even picking the same language, [Go](http://golang.org). Therefore 13 we've shelved this project and are now contributing to 14 [Git LFS](https://git-lfs.github.com) instead. 15 16 We're releasing the source to git-lob anyway, even though we have no 17 intention of developing it further. We'd always intended to make it open source, 18 and others might find this code useful. We'll probably be porting some of the features 19 here to Git LFS ourselves but other people are welcome to pick some things up too. 20 Here are some of the features that could be ported: 21 22 1. SSH native API (we're porting this right now) 23 2. Support for large file storage on S3 and file stores (no server required - probably won't be ported) 24 3. Chunked uploads/downloads for easier resuming 25 4. Pushing / fetching binaries separately from git commits 26 5. Pruning old binaries 27 6. Include / exclude paths (only download binaries in areas you work in) 28 7. Binary deltas using VCDIFF to reduce upload/download time 29 8. Shared local binary stores 30 31 You're free to experiment with this project of course but it is 32 **not** supported by Atlassian, nor will it have any further updates. 33 34 # Original documentation # 35 ## About ## 36 git-lob is a git extension for efficiently handling binary files in [Git](http://git-scm.com). Instead of storing the binary file content inside the git repo, it is hashed & externalised, with the git commit only referring to the hash. The binary store is synchronised separately from the commits. 37 38 ## Why is it useful? ## 39 1. Adding large binaries to git slows it down & makes cloning slow 40 2. Most people don't need or want to have every version ever of large binary files on their disk when they only use the more recent ones 41 3. We need efficient ways to synchronise large binaries via deltas 42 43 git-lob keeps your Git repository smaller and faster, while providing rich functionality for managing binary files linked to the source code. 44 45 The target audience is a project team on Windows, Mac or Linux who generally collaborate through one 'master' upstream repo (and maybe a small number of forks) and want to use the speed & power of Git for all their code & smaller resources while having lots of large binary files which are potentially inter-dependent with the history of that code. 46 47 ## Why did you write it, rather than use X, and why Go? ## 48 [Discussed in more detail...](doc/why.md) 49 50 ## Prerequisites ## 51 52 You must have git 1.8+ installed and on your path. 53 54 ## Installation ## 55 ### Install from source ### 56 57 If you want to build from source, make sure you have a Go environment already set up, with $GOPATH/bin already on your $PATH (on Windows, %GOPATH%\bin on your %PATH%). Then run the following in a console: 58 ```bash 59 > go get github.com/atlassian/git-lob 60 > go install github.com/atlassian/git-lob 61 ``` 62 63 Now edit your main .gitconfig file in your user directory and add a new filter definition. 64 65 On Mac/Linux: 66 ```ini 67 [filter "lob"] 68 clean = "$GOPATH/bin/git-lob filter-clean %f" 69 smudge = "$GOPATH/bin/git-lob filter-smudge %f" 70 required = true 71 ``` 72 73 On Windows: 74 ```ini 75 [filter "lob"] 76 clean = "%GOPATH%/bin/git-lob.exe filter-clean %f" 77 smudge = "%GOPATH%/bin/git-lob.exe filter-smudge %f" 78 required = true 79 ``` 80 81 You can expand $GOTPATH/%GOPATH% inline if you need to support usage where GOPATH is not defined. Again on Windows, always use forward slashes, for example c:/path/to/git-lob.exe 82 83 ### Install From binary distribution ### 84 If you downloaded a precompiled version for your platform, just extract git-lob[.exe] to a location of your choice. 85 86 Now edit your main .gitconfig file in your user directory and add a new filter definition as shown in the 'Install from source' section but set the path to git-lob[.exe] to be wherever you extracted it 87 88 ## Repository Configuration ## 89 To start putting binary files into git-lob you need to create or modify a .gitattributes file in the root of your repository: 90 ```ini 91 *.png filter=lob -crlf 92 *.jpg filter=lob -crlf 93 *.zip filter=lob -crlf 94 *.tiff filter=lob -crlf 95 *.tga filter=lob -crlf 96 *.dds filter=lob -crlf 97 *.bmp filter=lob -crlf 98 *.mov filter=lob -crlf 99 ``` 100 Include a line for all file types you want to be handled by git-lob. After saving this file, every time you 'git add' on a matching file, its content will be excluded from Git and put in the separate binary store, referenced by SHA in the commit. 101 102 ## Configuring remote storage ## 103 104 Binaries in git-lob are not stored in the regular git repo, but a corresponding 105 binary store must always exist to provide the actual binary content. A remote 106 in git usually only gives you the real git repo, so git-lob needs to expand 107 the configuration parameters to git remotes to specify the location of the 108 corresponding remote binary store. 109 110 The parameters depend on the type of binary storage ('provider') being used; see `git-lob providers` for a list of available providers and `git-lob provider <provider>` for specific details of one provider. 111 112 As an example, let's take the 'filesystem' provider, which simply uses the OS's 113 file system as a remote transport (obviously very simplistic): 114 115 ```ini 116 [remote "origin"] 117 # these 2 lines are standard git 118 url = ssh://git@bitbucket.org/something/somthing.git 119 fetch = +refs/heads/*:refs/remotes/origin/* 120 # these next 2 lines are required to configure the remote binary store 121 git-lob-provider = filesystem 122 git-lob-path = /Volumes/shared/something/something/binary/store 123 ``` 124 Other providers may require other parameters. It's important to note that you 125 can share a binary store among multiple remote repos if you wish, much like 126 the local git-lob.sharedstore option, since binaries are stored by SHA. 127 Identical file content in multiple repos can be stored only once this way. 128 Of course, access control may be an issue to consider here though. 129 130 ## Other options ## 131 git-lob supports a number of command-line parameters, and configuration parameters in your .gitconfig (user or repository level). Please call 'git lob help' for general help and a list of main commands, and 'git lob help topics' to list other topics. 132 133 # More Information # 134 ## Host setup ## 135 136 * [How to configure Amazon S3 for team use](doc/s3_setup.md) 137 * [The Smart Sync Protocol](doc/smart_protocol.md) 138 139 ## Information for git-lob developers ## 140 141 * [Initial developer setup](doc/devsetup.md) 142 * [Coding conventions](doc/conventions.md) 143 * [Third-party dependencies](doc/dependencies.md) 144 * [A note about dates in fetch & prune](doc/fetch_prune_dates.md)