github.com/ubuntu-core/snappy@v0.0.0-20210827154228-9e584df982bb/HACKING.md (about) 1 # Hacking on snapd 2 3 Hacking on snapd is fun and straightforward. The code is extensively 4 unit tested and we use the [spread](https://github.com/snapcore/spread) 5 integration test framework for the integration/system level tests. 6 7 ## Development 8 9 ### Supported Go versions 10 11 From snapd 2.52, snapd supports Go 1.13 and onwards. From snapd 2.38 12 to 2.52, snapd requires Go 1.9+. Versions before 2.38 support Go 1.6+. 13 14 ### Setting up a GOPATH 15 16 When working with the source of Go programs, you should define a path within 17 your home directory (or other workspace) which will be your `GOPATH`. `GOPATH` 18 is similar to Java's `CLASSPATH` or Python's `~/.local`. `GOPATH` is documented 19 [online](http://golang.org/pkg/go/build/) and inside the go tool itself 20 21 go help gopath 22 23 Various conventions exist for naming the location of your `GOPATH`, but it 24 should exist, and be writable by you. For example 25 26 export GOPATH=${HOME}/work 27 mkdir $GOPATH 28 29 will define and create `$HOME/work` as your local `GOPATH`. The `go` tool 30 itself will create three subdirectories inside your `GOPATH` when required; 31 `src`, `pkg` and `bin`, which hold the source of Go programs, compiled packages 32 and compiled binaries, respectively. 33 34 Setting `GOPATH` correctly is critical when developing Go programs. Set and 35 export it as part of your login script. 36 37 Add `$GOPATH/bin` to your `PATH`, so you can run the go programs you install: 38 39 PATH="$PATH:$GOPATH/bin" 40 41 (note `$GOPATH` can actually point to multiple locations, like `$PATH`, so if 42 your `$GOPATH` is more complex than a single entry you'll need to adjust the 43 above). 44 45 Note that if you are using go 1.16 or newer you need to disable the 46 go modules feature. Use: 47 48 export GO111MODULE=off 49 50 for this. 51 52 53 ### Getting the snapd sources 54 55 The easiest way to get the source for `snapd` is to use the `go get` command. 56 57 go get -d -v github.com/snapcore/snapd/... 58 59 This command will checkout the source of `snapd` and inspect it for any unmet 60 Go package dependencies, downloading those as well. `go get` will also build 61 and install `snapd` and its dependencies. To also build and install `snapd` 62 itself into `$GOPATH/bin`, omit the `-d` flag. More details on the `go get` 63 flags are available using 64 65 go help get 66 67 At this point you will have the git local repository of the `snapd` source at 68 `$GOPATH/src/github.com/snapcore/snapd`. The source for any 69 dependent packages will also be available inside `$GOPATH`. 70 71 ### Dependencies handling 72 73 Go dependencies are handled via `govendor`. Get it via: 74 75 go get -u github.com/kardianos/govendor 76 77 After a fresh checkout, move to the snapd source directory: 78 79 cd $GOPATH/src/github.com/snapcore/snapd 80 81 And then, run: 82 83 govendor sync 84 85 You can use the script `get-deps.sh` to run the two previous steps. 86 87 If a dependency need updating 88 89 govendor fetch github.com/path/of/dependency 90 91 Other dependencies are handled via distribution packages and you should ensure 92 that dependencies for your distribution are installed. For example, on Ubuntu, 93 run: 94 95 sudo apt-get build-dep ./ 96 97 ### Building 98 99 To build, once the sources are available and `GOPATH` is set, you can just run 100 101 go build -o /tmp/snap github.com/snapcore/snapd/cmd/snap 102 103 to get the `snap` binary in /tmp (or without -o to get it in the current 104 working directory). Alternatively: 105 106 go install github.com/snapcore/snapd/cmd/snap/... 107 108 to have it available in `$GOPATH/bin` 109 110 Similarly, to build the `snapd` REST API daemon, you can run 111 112 go build -o /tmp/snapd github.com/snapcore/snapd/cmd/snapd 113 114 ### Contributing 115 116 Contributions are always welcome! Please make sure that you sign the 117 Canonical contributor license agreement at 118 http://www.ubuntu.com/legal/contributors 119 120 Snapd can be found on GitHub, so in order to fork the source and contribute, 121 go to https://github.com/snapcore/snapd. Check out [GitHub's help 122 pages](https://help.github.com/) to find out how to set up your local branch, 123 commit changes and create pull requests. 124 125 We value good tests, so when you fix a bug or add a new feature we highly 126 encourage you to create a test in `$source_test.go`. See also the section 127 about Testing. 128 129 ### Testing 130 131 To run the various tests that we have to ensure a high quality source just run: 132 133 ./run-checks 134 135 This will check if the source format is consistent, that it builds, all tests 136 work as expected and that "go vet" has nothing to complain. 137 138 The source format follows the `gofmt -s` formating. Please run this on your 139 source files if `run-checks` complains about the format. 140 141 You can run an individual test for a sub-package by changing into that 142 directory and: 143 144 go test -check.f $testname 145 146 If a test hangs, you can enable verbose mode: 147 148 go test -v -check.vv 149 150 (or -check.v for less verbose output). 151 152 Note, the yamlordereddictloader python package is needed to carry out the tests format check. 153 154 There is more to read about the testing framework on the [website](https://labix.org/gocheck) 155 156 ### Running spread tests 157 158 To run the spread tests locally via QEMU, you need the latest version of 159 [spread](https://github.com/snapcore/spread). You can get spread, QEMU, and the 160 build tools to build QEMU images with: 161 162 $ sudo apt update && sudo apt install -y qemu-kvm autopkgtest 163 $ curl https://storage.googleapis.com/snapd-spread-tests/spread/spread-amd64.tar.gz | tar -xz -C $GOPATH/bin 164 165 #### Building spread VM images 166 167 To run the spread tests via QEMU you need to create VM images in the 168 `~/.spread/qemu` directory: 169 170 $ mkdir -p ~/.spread/qemu 171 $ cd ~/.spread/qemu 172 173 Assuming you are building on Ubuntu 18.04 LTS (Bionic Beaver) (or a later 174 development release like Ubuntu 19.04 (Disco Dingo)), run the following to 175 build a 64-bit Ubuntu 16.04 LTS (Xenial Xerus) VM to run the spread tests on: 176 177 $ autopkgtest-buildvm-ubuntu-cloud -r xenial 178 $ mv autopkgtest-xenial-amd64.img ubuntu-16.04-64.img 179 180 To build an Ubuntu 14.04 (Trusty Tahr) based VM, use: 181 182 $ autopkgtest-buildvm-ubuntu-cloud -r trusty --post-command='sudo apt-get install -y --install-recommends linux-generic-lts-xenial && update-grub' 183 $ mv autopkgtest-trusty-amd64.img ubuntu-14.04-64.img 184 185 This is because we need at least 4.4+ kernel for snapd to run on Ubuntu 14.04 186 LTS, which is available through the `linux-generic-lts-xenial` package. 187 188 If you are running Ubuntu 16.04 LTS, use 189 `adt-buildvm-ubuntu-cloud` instead of `autopkgtest-buildvm-ubuntu-cloud` (the 190 latter replaced the former in 18.04): 191 192 $ adt-buildvm-ubuntu-cloud -r xenial 193 $ mv adt-xenial-amd64-cloud.img ubuntu-16.04-64.img 194 195 #### Downloading spread VM images 196 197 Alternatively, instead of building the QEMU images manually, you can download 198 pre-built and somewhat maintained images from 199 [spread.zygoon.pl](spread.zygoon.pl). The images will need to be extracted 200 with `gunzip` and placed into `~/.spread/qemu` as above. 201 202 #### Running spread with QEMU 203 204 Finally, you can run the spread tests for Ubuntu 16.04 LTS 64-bit with: 205 206 $ spread -v qemu:ubuntu-16.04-64 207 208 To run for a different system, replace `ubuntu-16.04-64` with a different system 209 name. 210 211 For quick reuse you can use: 212 213 $ spread -reuse qemu:ubuntu-16.04-64 214 215 It will print how to reuse the systems. Make sure to use 216 `export REUSE_PROJECT=1` in your environment too. 217 218 #### Running UC20 spread with QEMU 219 220 Ubuntu Core 20 on amd64 has a requirement to use UEFI, so there are a few 221 additional steps needed to run spread with the ubuntu-core-20-64 systems locally 222 using QEMU. For one, upstream spread currently does not support specifying what 223 kind of BIOS to use with the VM, so you have to build spread from this PR: 224 https://github.com/snapcore/spread/pull/95, and then use the environment 225 variable `SPREAD_QEMU_BIOS` to specify an UEFI BIOS to use with the VM, for 226 example the one from the OVMF package. To get OVMF on Ubuntu, you can just 227 install the `ovmf` package via `apt`. After installing OVMF, you can then run 228 spread like so: 229 230 $ SPREAD_QEMU_BIOS=/usr/share/OVMF/OVMF_CODE.fd spread -v qemu:ubuntu-core-20-64 231 232 This will enable testing UC20 with the spread, albeit without secure boot 233 support. None of the native UC20 tests currently require secure boot however, 234 all tests around secure boot are nested, see the section below about running the 235 nested tests. 236 237 Also, due to the in-flux state of spread support for booting UEFI VM's like 238 this, you can test ubuntu-core-20-64 only by themselves and not with any other 239 system concurrently since the environment variable is global for all systems in 240 the spread run. This will be fixed in a future release of spread. 241 242 ### Testing snapd 243 244 To test the `snapd` REST API daemon on a snappy system you need to 245 transfer it to the snappy system and then run: 246 247 sudo systemctl stop snapd.service snapd.socket 248 sudo SNAPD_DEBUG=1 SNAPD_DEBUG_HTTP=3 ./snapd 249 250 To debug interaction with the snap store, you can set `SNAP_DEBUG_HTTP`. 251 It is a bitfield: dump requests: 1, dump responses: 2, dump bodies: 4. 252 253 (make hack: In case you get some security profiles errors when trying to install or refresh a snap, 254 maybe you need to replace system installed snap-seccomp with the one aligned to the snapd that 255 you are testing. To do this, simply backup /usr/lib/snapd/snap-seccomp and overwrite it with 256 the testing one. Don't forget to rollback to the original when finish testing) 257 258 ### Running nested tests 259 260 Nested tests are used to validate features which cannot be tested on regular tests. 261 262 The nested test suites work different from the other test suites in snapd. In this case each test runs in a new image 263 which is created following the rules defined for the test. 264 265 The nested tests are executed using spread tool. See the following examples using the qemu and google backends. 266 267 . `qemu: spread qemu-nested:ubuntu-20.04-64:tests/nested/core20/tpm` 268 . `google: spread google-nested:ubuntu-20.04-64:tests/nested/core20/tpm` 269 270 The nested system in all the cases is selected based on the host system. The folloing lines show the relation between host and nested systemd (same applies for classic nested tests): 271 272 . ubuntu-16.04-64 => ubuntu-core-16-64 273 . ubuntu-18.04-64 => ubuntu-core-18-64 274 . ubuntu-20.04-64 => ubuntu-core-20-64 275 276 The tools used for creating and hosting the nested vms are: 277 278 . ubuntu-image snap is used to building the images 279 . QEMU is used for the virtualization (with kvm acceleration) 280 281 Nested test suite is composed by the following 4 suites: 282 283 classic: the nested suite contains an image of a classic system downloaded from cloud-images.ubuntu.com 284 core: it tests a core nested system and the images are generated by using ubuntu-image snap 285 core20: this is similar to core suite but tests on it are focused on UC20 286 manual: tests on this suite create a non generic image with spedific conditions 287 288 The nested suites use some environment variables to configure the suite and the tests inside it, the most important ones are the described bellow: 289 290 NESTED_WORK_DIR: It is path to the directory where all the nested assets and images are stored 291 NESTED_TYPE: Use core for ubuntu core nested systems or classic instead. 292 NESTED_CORE_CHANNEL: The images are created using ubuntu-image snap, use it to define the default branch 293 NESTED_CORE_REFRESH_CHANNEL: The images can be refreshed to a specific channel, use it to specify the channel 294 NESTED_USE_CLOUD_INIT: Use cloud init to make initial system configuration instead of user assertion 295 NESTED_ENABLE_KVM: Enable kvm in the qemu command line 296 NESTED_ENABLE_TPM: re boot in the nested vm in case it is supported (just supported on UC20) 297 NESTED_ENABLE_SECURE_BOOT: Enable secure boot in the nested vm in case it is supported (just supported on UC20) 298 NESTED_BUILD_SNAPD_FROM_CURRENT: Build and use either core or snapd snapd from current branch 299 NESTED_CUSTOM_IMAGE_URL: Download and use an custom image from this url 300 301 302 # Quick intro to hacking on snap-confine 303 304 Hey, welcome to the nice, low-level world of snap-confine 305 306 307 ## Building the code locally 308 309 To get started from a pristine tree you want to do this: 310 311 ``` 312 ./mkversion.sh 313 cd cmd/ 314 autoreconf -i -f 315 ./configure --prefix=/usr --libexecdir=/usr/lib/snapd --enable-nvidia-multiarch --with-host-arch-triplet="$(dpkg-architecture -qDEB_HOST_MULTIARCH)" 316 ``` 317 318 This will drop makefiles and let you build stuff. You may find the `make hack` 319 target, available in `cmd/snap-confine` handy, it installs the locally built 320 version on your system and reloads the apparmor profile. 321 322 Note, the above configure options assume you are on Ubuntu and are generally 323 necessary to run/test graphical applications with your local version of 324 snap-confine. The `--with-host-arch-triplet` option sets your specific 325 architecture and `--enable-nvidia-multiarch` allows the host's graphics drivers 326 and libraries to be shared with snaps. If you are on a distro other than 327 Ubuntu, try `--enable-nvidia-biarch` (though you'll likely need to add further 328 system-specific options too). 329 330 ## Submitting patches 331 332 Please run `(cd cmd; make fmt)` before sending your patches for the "C" part of 333 the source code.