github.com/vanadium-archive/go.jiri@v0.0.0-20160715023856-abfb8b131290/README.md (about) 1 # Jiri 2 3 *"Jiri integrates repositories intelligently"* 4 5 Jiri is a tool for multi-repo development. 6 It supports: 7 * syncing multiple local repos with upstream, 8 * capturing the current state of all local repos in a "snapshot", 9 * restoring local project state from a snapshot, and 10 * facilitating sending change lists to [Gerrit][gerrit]. 11 12 Jiri has an extensible plugin model, making it easy to create new sub-commands. 13 14 Jiri is open-source. See the contributor guidelines [here][contributing]. 15 16 ## Jiri Basics 17 Jiri organizes a set of repositories on your local filesystem according to a 18 [manifest][manifests]. These repositories are referred to as "projects", and 19 are all contained within a single directory called the "jiri root" which is 20 assumed to be set in the `JIRI_ROOT` environment variable. 21 22 The manifest file specifies the relative location of each project within the 23 jiri root, and also includes other metadata about the project such as its 24 remote url, the remote branch it should track, and more. 25 26 The `jiri update` command syncs the master branch of all local projects to the 27 revision and remote branch specified in the manifest for each project. Jiri 28 will create the project locally if it does not exist, and if run with the `-gc` 29 flag, jiri will "garbage collect" any projects that are not listed in the 30 manifest by deleting them locally. 31 32 The `.jiri_manifest` file in the jiri root describes which project jiri should 33 sync. Typically the `.jiri_manifest` file will import other manifests, but it 34 can also contain a list of projects. 35 36 For example, here is a simple `.jiri_manifest` with just two projects, "foo" 37 and "bar", which are hosted on github and bitbucket respectively. 38 ``` 39 <?xml version="1.0" encoding="UTF-8"?> 40 <manifest> 41 <projects> 42 <project name="foo-project" 43 remote="https://github.com/my-org/foo" 44 path="foo"/> 45 <project name="bar" 46 remote="https://bitbucket.com/other-org/bar" 47 path="bar"/> 48 </projects> 49 </manifest> 50 ``` 51 When you run `jiri update` for the first time, the "foo" and "bar" repos will 52 be cloned into `$JIRI_ROOT/foo` and `$JIRI_ROOT/bar` respectively. Running 53 `jiri update` again will sync the master branch of these repos with the remote 54 master branch. 55 56 Note that the project paths do not need to be immediate children of the jiri 57 root. We could have decided to set the `path` attribute for the "bar" project 58 to "third_party/bar", or even nest "bar" inside the "foo" project by setting 59 the `path` to "foo/bar" (assuming no files in the foo repo conflict with bar). 60 61 Because manifest files also need to be kept in sync between various team 62 members, it often makes sense to keep your team's manifests in a version 63 controlled repository. 64 65 Jiri makes it easy to "import" a remote manifest from your local 66 `.jiri_manifest` file with the `jiri import` command. For example, running the 67 following command will create a `.jiri_manifest` file (or append to an existing 68 one) with an `import` tag that imports the minimal manifest from the 69 https://github.com/vanadium/manifest repo. 70 71 ``` 72 jiri import -name="manifest" minimal https://github.com/vanadium/manifest 73 ``` 74 75 The next time you run `jiri update`, jiri will sync all projects listed in the 76 Vanadium minimal manifest. 77 78 ## Quickstart 79 80 This section explains how to get started with jiri. 81 82 First we "bootstrap" jiri so that it can sync and build itself. 83 84 Then we create and import a new manifest, which specifies how jiri should 85 manage your projects. 86 87 ### Bootstrapping 88 89 You can get jiri up-and-running in no time with the help of the [bootstrap 90 script][bootstrap_jiri]. 91 92 First, pick a jiri root directory. All projects will be synced to 93 subdirectories of the root. 94 95 ``` 96 export MY_ROOT=$HOME/myroot 97 ``` 98 99 Execute the `jiri_bootstrap` script, which will fetch and build the jiri tool, 100 and initialize the root directory. 101 102 ``` 103 curl -s 104 https://raw.githubusercontent.com/vanadium/go.jiri/master/scripts/bootstrap_jiri | bash -s "$MY_ROOT" 105 ``` 106 107 The `jiri` command line tool will be installed in 108 `$MY_ROOT/.jiri_root/scripts/jiri`, so add that to your `PATH`. 109 110 ``` 111 export PATH="$MY_ROOT"/.jiri_root/scripts:$PATH 112 ``` 113 114 Next, use the `jiri import` command to import the "minimal" manifest from the 115 vanadium manifest repo. This manifest includes only the projects needed to 116 build the jiri tool itself. 117 118 You can see the minimal manifest [here][minimal manifest]. For more 119 information on manifests, read the [manifest docs][manifests]. 120 121 ``` 122 cd "$MY_ROOT" 123 jiri import -name=manifest minimal https://vanadium.googlesource.com/manifest 124 ``` 125 126 You should now have a file in the root directory called `.jiri_manifest`, which 127 will contain a single import. 128 129 Finally, run `jiri update`, which will sync all local projects to the revisions 130 listed in the manifest (which in this case will be `HEAD`). 131 132 133 ``` 134 jiri update 135 ``` 136 137 You should now see the jiri project and dependencies in 138 `$MY_ROOT/release/go/src/v.io`, and the vanadium manifest repo in 139 `$MY_ROOT/manifest`. 140 141 Running `jiri update` again will sync the local repos to the remotes, and 142 rebuild the jiri tool. 143 144 ### Managing your projects with jiri 145 146 Now that jiri is able to sync and build itself, we must tell it how to manage 147 your projects. 148 149 In order for jiri to manage a set of projects, those projects must be listed in 150 a [manifest][manifests], and that manifest must be hosted in a git repo. 151 152 If you already have a manifest hosted in a git repo, you can import that 153 manifest the same way we imported the "minimal" manifest. 154 155 For example, if your manifest is called "my_manifest" and is in a repo hosted 156 at "https://github.com/my_org/manifests", then you can import that manifest 157 as follows. 158 159 ``` 160 jiri import my_manifest https://github.com/my_org/manifests 161 ``` 162 163 The rest of this section walks through how to create a manifest from scratch, 164 host it from a local git repo, and get jiri to manage it. 165 166 Suppose that the project you want jiri to manage is the "Hello-World" repo 167 located at https://github.com/Test-Octowin/Hello-World. 168 169 First we'll create a new git repo to host the manifest we'll be writing. 170 171 ``` 172 mkdir -p /tmp/my_manifest_repo 173 cd /tmp/my_manifest_repo 174 git init 175 ``` 176 177 Next we'll create a manifest and commit it to the manifest repo. 178 179 The manifest file will include the Hello-World repo as well as the manifest 180 repo itself. 181 182 ``` 183 cat <<EOF > my_manifest 184 <?xml version="1.0" encoding="UTF-8"?> 185 <manifest> 186 <projects> 187 <project name="Hello-World" 188 remote="https://github.com/Test-Octowin/Hello-World" 189 path="helloworld"/> 190 <project name="manifest" 191 remote="/tmp/my_manifest_repo" 192 path="manifest"/> 193 </projects> 194 </manifest> 195 EOF 196 197 git add my_manifest 198 git commit -m "Add my_manifest." 199 ``` 200 201 This manifest contains a single project with the name "Hello-World" and the 202 remote of the repo. The `path` attribute tells jiri to sync this repo inside 203 the `helloworld` directory. 204 205 Normally we would want to push this repo to some remote to make it accessible 206 to other users who want to sync the same projects. For now, however, we'll 207 just refer to the repo by its path in the local filesystem. 208 209 Now we just need to import that new manifest and `jiri update`. Since we don't 210 want the new manifest repo to conflict with the minimal manifest repo, we must 211 pass the `-path` flag to the import statement. 212 213 ``` 214 cd "$MY_ROOT" 215 jiri import -path="my_manifest_repo" my_manifest /tmp/my_manifest_repo 216 jiri update 217 ``` 218 219 You should now see the Hello-World repo in `$MY_ROOT/helloworld`, and your 220 manifest repo in `$MY_ROOT/my_manifest_repo`. 221 222 ## Command-line help 223 224 The `jiri help` command will print help documentation about the `jiri` tool and 225 its subcommands. 226 227 For general documentation, including a list of subcommands, run `jiri help`. 228 To find documentation about a specific topic or subcommand, run `jiri help 229 <command>`. 230 231 You can read all the command-line documentation in a single page here: 232 http://godoc.org/v.io/jiri. 233 234 ## Filesystem 235 236 <!-- TODO(nlacasse): Figure out a way to keep the canonical documentation in 237 one place but mirror it to the README and cmdline docs. --> 238 239 See the jiri [filesystem godocs](https://godoc.org/v.io/jiri/cmd/jiri#hdr-Jiri_filesystem___Description_of_jiri_file_system_layout). 240 241 ## Manifests<a name="manifests"></a> 242 243 <!-- TODO(nlacasse): Figure out a way to keep the canonical documentation in 244 one place but mirror it to the README and cmdline docs. --> 245 246 See the jiri [manifest godocs](https://godoc.org/v.io/jiri/cmd/jiri#hdr-Jiri_manifest___Description_of_manifest_files). 247 248 ## Snapshots 249 250 TODO(nlacasse): Write me. 251 252 ## Profiles 253 254 TODO(nlacasse): Write me. 255 256 ## Gerrit CL workflow 257 258 [Gerrit][gerrit] is a collaborative code-review tool used by many open source 259 projects. 260 261 One of the peculiarities of Gerrit is that it expects a changelist to be 262 represented by a single commit. This constrains the way developers may use git 263 to work on their changes. In particular, they must use the --amend flag with 264 all but the first git commit operation and they need to use git rebase to sync 265 their pending code change with the remote master. See Android's [repo command 266 reference][android repo] or Go's [contributing instructions][go contrib] for 267 examples of how intricate the workflow for resolving conflicts between the 268 pending code change and the remote master is. 269 270 The `jiri cl` command enables interaction with Gerrit without having to use 271 such a complex and error-prone workflow. With `jiri cl`, users commit as often 272 as they want on feature branches, and `jiri cl` handles the hard work of 273 squashing all commits into a single commit and sending to Gerrit. 274 275 The rest of this section describes common development operations using `jiri 276 cl`. The term "CL" (short for "ChangeList") refers to a set of code changes 277 uploaded for review. 278 279 ### Using feature branches 280 281 The "master" branch of each local repository is reserved for tracking its 282 remote counterpart. All development should take place on a non-master 283 "feature" branch. Once the code is reviewed and approved, it is merged into 284 the remote master via the Gerrit code review system. The change can then be 285 merged into the local master branch with `jiri update`. 286 287 <!-- TODO(nlacasse): dje is changing this behavior. The plan is that "master" 288 will be the default reserved branch for each repo, but that can be overridden 289 with the `localbranch` attribute in the manifest. Update this section once 290 this change lands. --> 291 292 ### Creating a new CL 293 294 1. Sync the master branch with the remote. 295 ``` 296 jiri update 297 ``` 298 2. Create a new feature branch for the CL. 299 ``` 300 jiri cl new <branch-name> 301 ``` 302 3. Make modifications to the project source code. 303 4. Stage any changed files for commit. 304 ``` 305 git add <file1> <file2> ... <fileN> 306 ``` 307 5. Commit the changes. 308 ``` 309 git commit 310 ``` 311 6. Repeat steps 3-5 as necessary. 312 313 ### Syncing a CL with the remote 314 315 1. Sync the master branch with the remote. 316 ``` 317 jiri update 318 ``` 319 2. Switch to the feature branch that corresponds to the CL under development. 320 ``` 321 git checkout <branch-name> 322 ``` 323 3. Sync the feature branch with the master branch. 324 ``` 325 jiri cl sync 326 ``` 327 4. If there are no conflicts between the master and the feature branch, the CL 328 has been successfully synced with the remote. 329 5. If there are conflicts: 330 1. Manually [resolve the conflicts][github resolve conflict]. 331 2. Stage any changed files for a commit. 332 ``` 333 git add <file1> <file2> ... <fileN> 334 ``` 335 3. Commit the changes. 336 ``` 337 git commit 338 ``` 339 340 ### Requesting a code review 341 342 1. Switch to the feature branch that corresponds to the CL under development. 343 ``` 344 git checkout <branch-name> 345 ``` 346 2. Upload the CL to Gerrit. 347 ``` 348 jiri cl mail 349 ``` 350 351 If the CL upload is successful, this will print the URL of the CL hosted on 352 Gerrit. You can add reviewers and comments through the [Gerrit web UI][gerrit 353 web ui] at that URL. 354 355 Note that there are many useful flags for `jiri cl`. You can learn about them 356 by running `jiri cl --help`. 357 358 ### Reviewing a CL 359 360 1. Follow the link received in the code review email request. 361 2. Use the [Gerrit web UI][gerrit web UI] to comment on the CL and click the 362 "Reply" button to submit comments, selecting the appropriate code-review 363 score. 364 365 ### Addressing review comments 366 367 368 1. Switch to the feature branch that corresponds to the CL under development. 369 ``` 370 git checkout <branch-name> 371 ``` 372 2. Modify and commit the code as described above. 373 3. Reply to each Gerrit comment and click the "Reply" button to send them. 374 4. Send the updated CL to Gerrit. 375 ``` 376 jiri cl mail 377 ``` 378 379 ### Submitting a CL 380 1. Note that if the CL conflicts with any changes that have been submitted 381 since the last update of the CL, these conflicts need to be resolved before 382 the CL can be submitted. To do so, follow the steps in the "Syncing a CL 383 with the remote" section above and then upload the updated CL to Gerrit. 384 ``` 385 jiri cl mail 386 ``` 387 2. Once a CL meets the conditions for being submitted, it can be merged into 388 the remote master branch by clicking the "Submit" button on the Gerrit web 389 UI. 390 3. Delete the local feature branch after the CL has been submitted to Gerrit. 391 1. Sync the master branch to the laster version of the remote. 392 ``` 393 jiri update 394 ``` 395 2. Safely delete the feature branch that corresponds to the CL. 396 ``` 397 jiri cl cleanup <branch-name> 398 ``` 399 400 Note that deleting the feature branch with `git branch -d <branch-name>` won't 401 work in general because the git history on the local feature branch differs 402 from the history on the remote master. The local feature branch might have 403 many small commits, while the remote will have the same changes squashed into a 404 single commit. This difference in the history will prevent git from letting 405 you do `git branch -d <branch-name>`. You *can* use `git branch -D 406 <branch-name>`, but that can potentially cause you to lose work if the branch 407 has not been merged into master yet. For this reason, we recommend using `jiri 408 cl cleanup` to delete the feature branch safely. 409 410 ### Dependent CLs 411 If you have changes A and B, and B depends on A, you can still submit distinct 412 CLs for A and B that can be reviewed and submitted independently (although A 413 must be submitted before B). 414 415 First, create your feature branch for A, make your change, and upload the CL 416 for review according to the instructions above. 417 418 Then, while still on the feature branch for A, create your feature branch for B. 419 ``` 420 jiri cl new feature-B 421 ``` 422 Then make your change and upload the CL for review according to the 423 instructions above. 424 425 You can respond to review comments by submitting new patch sets as normal. 426 427 After the CL for A has been submitted, make sure to clean up A's feature branch 428 and upload a new patch set for feature B. 429 ``` 430 jiri update # fetch update that includes feature A 431 git checkout feature-B 432 jiri cl cleanup feature-A 433 git merge master # merge feature A into feature B branch 434 jiri cl mail # send new patch set for feature B 435 ``` 436 The CL for feature B can now be submitted. 437 438 This process can be extended for more than 2 CLs. You must keep two things in mind: 439 * always create the dependent feature branch with `jiri cl new` from the parent feature branch, and 440 * after a parent feature has been submitted, cleanup the parent feature branch with `jiri cl cleanup`, and merge master into all dependent CLs and upload new patch sets. 441 442 ## FAQ 443 444 ### Why the name "jiri"? 445 [Jiří][jiri-wiki] is a very popular boys name in the Czech Republic. 446 447 ### How do you pronounce "jiri"? 448 We pronounce "jiri" like "yiree". 449 450 The actual Czech name [Jiří][jiri-wiki] is pronounced something like "yirzhee". 451 452 ### Why can't I commit to my master branch? 453 454 Jiri keeps the master branch of each project in the state described in the 455 manifest. Any changes that are made to the master branch would be lost during 456 the next `jiri update`. 457 458 <!-- TODO(nlacasse): Answer these. 459 ### Why not repo/gclient/etc? 460 ### How can I test changes to a manifest without pushing it upstream? 461 --> 462 463 [android repo]: https://source.android.com/source/using-repo.html "Repo command reference" 464 [bootstrap_jiri]: scripts/bootstrap_jiri "bootstrap_jiri" 465 [contributing]: CONTRIBUTING.md "contributing" 466 [gerrit]: https://code.google.com/p/gerrit/ "Gerrit code review" 467 [gerrit web ui]: https://gerrit-review.googlesource.com/Documentation/user-review-ui.html "Gerrit review UI" 468 [github resolve conflict]: https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line/ "Resolving a merge conflict" 469 [go contrib]: https://golang.org/doc/contribute.html#Code_review "Go Contribution Guidelines - Code Review" 470 [jiri-wiki]: https://en.wikipedia.org/wiki/Ji%C5%99%C3%AD "Jiří" 471 [manifests]: #manifests "manifests" 472 [minimal manifest]: https://vanadium.googlesource.com/manifest/+/refs/heads/master/minimal "minimal manifest"