github.com/fawick/restic@v0.1.1-0.20171126184616-c02923fbfc79/CHANGELOG.md (about) 1 This file describes changes relevant to all users that are made in each 2 released version of restic from the perspective of the user. 3 4 Important Changes in 0.8.0 5 ========================== 6 7 * A vulnerability was found in the restic restorer, which allowed attackers in 8 special circumstances to restore files to a location outside of the target 9 directory. Due to the circumstances we estimate this to be a low-risk 10 vulnerability, but urge all users to upgrade to the latest version of restic. 11 12 Exploiting the vulnerability requires a Linux/Unix system which saves 13 backups via restic and a Windows systems which restores files from the repo. 14 In addition, the attackers need to be able to create create files with 15 arbitrary names which are then saved to the restic repo. For example, by 16 creating a file named "..\test.txt" (which is a perfectly legal filename on 17 Linux) and restoring a snapshot containing this file on Windows, it would be 18 written to the parent of the target directory. 19 20 We'd like to thank Tyler Spivey for reporting this responsibly! 21 22 https://github.com/restic/restic/pull/1445 23 24 * The s3 backend used the subdir `restic` within a bucket if no explicit path 25 after the bucket name was specified. Since this version, restic does not use 26 this default path any more. If you created a repo on s3 in a bucket without 27 specifying a path within the bucket, you need to add `/restic` at the end of 28 the repository specification to access your repo: `s3:s3.amazonaws.com/bucket/restic` 29 https://github.com/restic/restic/issues/1292 30 https://github.com/restic/restic/pull/1437 31 32 * We've added a local cache for metadata so that restic doesn't need to load 33 all metadata (snapshots, indexes, ...) from the repo each time it starts. By 34 default the cache is active, but there's a new global option `--no-cache` 35 that can be used to disable the cache. By deafult, the cache a standard 36 cache folder for the OS, which can be overridden with `--cache-dir`. The 37 cache will automatically populate, indexes and snapshots are saved as they 38 are loaded. 39 https://github.com/restic/restic/pull/1040 40 https://github.com/restic/restic/issues/29 41 https://github.com/restic/restic/issues/738 42 https://github.com/restic/restic/issues/282 43 https://github.com/restic/restic/pull/1287 44 45 * A related change was to by default create pack files in the repo that 46 contain either data or metadata, not both mixed together. This allows easy 47 caching of only the metadata files. The next run of `restic prune` will 48 untangle mixed files automatically. 49 https://github.com/restic/restic/pull/1265 50 51 * The Google Cloud Storage backend no longer requires the service account to 52 have the `storage.buckets.get` permission ("Storage Admin" role) in `restic 53 init` if the bucket already exists. 54 https://github.com/restic/restic/pull/1281 55 56 * Added support for rate limiting through `--limit-upload` and 57 `--limit-download` flags. 58 https://github.com/restic/restic/issues/1216 59 https://github.com/restic/restic/pull/1336 60 https://github.com/restic/restic/pull/1358 61 62 * Failed backend requests are now automatically retried. 63 https://github.com/restic/restic/pull/1353 64 65 * We've added the `dump` command which prints a file from a snapshot to 66 stdout. This can e.g. be used to restore files read with `backup --stdin`. 67 https://github.com/restic/restic/issues/510 68 https://github.com/restic/restic/pull/1346 69 70 Small changes 71 ------------- 72 73 * The directory structure in the fuse mount now exposes a symlink `latest` 74 which points to the latest snapshot in that particular directory. 75 https://github.com/restic/restic/pull/1249 76 77 * The option `--compact` was added to the `forget` command to provide the same 78 compact view as the `snapshots` command. 79 https://github.com/restic/restic/pull/1269 80 81 * We've re-enabled a workaround for `minio-go` (the library we're using to 82 access s3 backends), this reduces memory usage. 83 https://github.com/restic/restic/issues/1256 84 https://github.com/restic/restic/pull/1267 85 86 * The sftp backend now prompts for the password if a password is necessary for 87 login. 88 https://github.com/restic/restic/issues/448 89 https://github.com/restic/restic/pull/1270 90 91 * The `generate` command has been added, which replaces the now removed 92 commands `manpage` and `autocomplete`. This release of restic contains the 93 most recent manpages in `doc/man` and the auto-completion files for bash and 94 zsh in `doc/bash-completion.sh` and `doc/zsh-completion.zsh` 95 https://github.com/restic/restic/issues/1274 96 https://github.com/restic/restic/pull/1282 97 98 * A bug was discovered in the library we're using to access Backblaze, it now 99 reuses already established TCP connections which should be a lot faster and 100 not cause network failures any more. 101 https://github.com/restic/restic/issues/1291 102 https://github.com/restic/restic/pull/1301 103 104 * Another bug in the `forget` command caused `prune` not to be run when 105 `--prune` was specified without a policy, e.g. when only snapshot IDs that 106 should be forgotten are listed manually. This is corrected now. 107 https://github.com/restic/restic/pull/1317 108 109 * The `check` command now explicetly prints `No errors were found` when no 110 errors could be found. 111 https://github.com/restic/restic/pull/1319 112 https://github.com/restic/restic/issues/1303 113 114 * The fuse mount now has an `ids` subdirectory which contains the snapshots 115 below their (short) IDs. 116 https://github.com/restic/restic/issues/1102 117 https://github.com/restic/restic/pull/1299 118 https://github.com/restic/restic/pull/1320 119 120 * The `backup` command was improved, it now caches the result of excludes for 121 a directory. 122 https://github.com/restic/restic/issues/1271 123 https://github.com/restic/restic/pull/1326 124 125 * We've added the `--cacert` option which can be used to pass one (or more) CA 126 certificates to restic. These are used in addition to the system CA 127 certificates to verify HTTPS certificates (e.g. for the REST backend). 128 https://github.com/restic/restic/issues/1114 129 https://github.com/restic/restic/pull/1276 130 131 * When the list of files/dirs to be saved is read from a file with 132 `--files-from`, comment lines (starting with `#`) are now ignored. 133 https://github.com/restic/restic/issues/1367 134 https://github.com/restic/restic/pull/1368 135 136 Important Changes in 0.7.3 137 ========================== 138 139 * For large backups stored in Google Cloud Storage, the `prune` command fails 140 because listing only returns the first 1000 files. This has been corrected, 141 no data is lost in the process. In addition, a plausibility check was added 142 to `prune`. 143 https://github.com/restic/restic/issues/1246 144 https://github.com/restic/restic/pull/1247 145 146 147 Important Changes in 0.7.2 148 ========================== 149 150 * We've added an official docker image and a Dockerfile to build this image in 151 `docker/`. 152 https://github.com/restic/restic/pull/1061 153 154 * The git repository layout was changed to resemble the layout typically used 155 in Go projects, we're not using `gb` for building restic any more and 156 vendoring the dependencies is now taken care of by `dep`. 157 https://github.com/restic/restic/pull/1126 158 159 * We now support saving backups on Google Cloud Storage. 160 https://github.com/restic/restic/pull/1134 161 https://github.com/restic/restic/pull/1052 162 https://github.com/restic/restic/issues/211 163 164 * We've added support for Microsoft Azure Blob Storage as a restic backend. 165 https://github.com/restic/restic/pull/1149 166 https://github.com/restic/restic/pull/1059 167 https://github.com/restic/restic/issues/609 168 169 * In the course of supporting Microsoft Azure Blobe Storage Go 1.8 is now a 170 requirement to build restic. 171 172 * The `restore` command has been improved: When dirs are excluded (or not 173 included) in a restore, they are not loaded from the repo any more. 174 https://github.com/restic/restic/pull/1044 175 176 * Name collisions are now resolved by appending a counter. 177 https://github.com/restic/restic/issues/1179 178 https://github.com/restic/restic/pull/1209 179 180 181 Small changes 182 ------------- 183 184 * The `key` command now prompts for a password even if the original password 185 to access a repo has been specified via the `RESTIC_PASSWORD` environment 186 variable or a password file. 187 https://github.com/restic/restic/issues/1132 188 https://github.com/restic/restic/pull/1133 189 190 * Properly report errors when reading files with exclude patterns. 191 https://github.com/restic/restic/pull/1144 192 193 * We now automatically generate man pages for all restic commands, see the 194 subdir `doc/man`. 195 https://github.com/restic/restic/issues/697 196 https://github.com/restic/restic/pull/1147 197 198 * The `key remove` command was corrected and now works as documented. 199 https://github.com/restic/restic/pull/1164 200 201 * When a restic command other than `init` is used with a local repository and 202 the repository directory does not exist, restic creates the directory 203 structure. That's an error, only the `init` command should create the dir. 204 https://github.com/restic/restic/issues/1167 205 https://github.com/restic/restic/pull/1182 206 207 * Restic now prints stats on all BSD systems (not only on darwin) when SIGINFO 208 is received (usually when ctrl+t is pressed). 209 https://github.com/restic/restic/pull/1203 210 https://github.com/restic/restic/pull/1082#issuecomment-326279920 211 212 * Since a few releases restic had the ability to write profiling files for 213 memory and CPU usage when `debug` is enabled. It was discovered that when 214 restic is interrupted (ctrl+c is pressed), the proper shutdown hook is not 215 run. This is now corrected. 216 https://github.com/restic/restic/pull/1191 217 218 * A new option `--exclude-caches` was added that allows excluding cache 219 directories (that are tagged as such). This is a special case of a more 220 generic option `--exclude-if-present` which excludes a directory if a file 221 with a specific name (and contents) is present. 222 https://github.com/restic/restic/issues/317 223 https://github.com/restic/restic/pull/1170 224 https://github.com/restic/restic/pull/1224 225 226 * The `forget` command now has an option `--group-by` that allows flexible 227 grouping policies. 228 https://github.com/restic/restic/pull/1196 229 230 * The date and time restic records for a new backup can now be specified 231 externally by passing `--time` to the `backup` command. 232 https://github.com/restic/restic/pull/1205 233 234 * The option `--compact` was added to the `snapshots` command to get a better 235 overview of the snapshots in a repo. It limits each snapshot to a single 236 line. 237 https://github.com/restic/restic/issues/1218 238 https://github.com/restic/restic/pull/1223 239 240 241 Important Changes in 0.7.1 242 ========================== 243 244 * The `migrate` command for chaning the `s3legacy` layout to the `default` 245 layout for s3 backends has been improved: It can now be restarted with 246 `restic migrate --force s3_layout` and automatically retries operations on 247 error. 248 https://github.com/restic/restic/issues/1073 249 https://github.com/restic/restic/pull/1075 250 251 Small changes 252 ------------- 253 254 * The local and sftp backends now create the subdirs below `data/` on 255 open/init. This way, restic makes sure that they always exist. This is 256 connected to an issue for the sftp server: 257 https://github.com/restic/rest-server/pull/11#issuecomment-309879710 258 https://github.com/restic/restic/issues/1055 259 https://github.com/restic/restic/pull/1077 260 https://github.com/restic/restic/pull/1105 261 262 * When no S3 credentials are specified in the environment variables, restic 263 now tries to load credentials from an IAM instance profile when the s3 264 backend is used. 265 https://github.com/restic/restic/issues/1067 266 https://github.com/restic/restic/pull/1086 267 268 * On Darwin and FreeBSD, restic now prints stats when SIGINFO is received 269 (usually when ctrl+t is pressed). 270 https://github.com/restic/restic/pull/1082 271 272 * The dependencies have been updated. 273 https://github.com/restic/restic/pull/1108 274 https://github.com/restic/restic/pull/1124 275 276 * A bug was found (and corrected) in the index rebuilding after prune, which 277 led to indexes which include blobs that were not present in the repo any 278 more. There were already checks in place which detected this situation and 279 aborted with an error message. A new run of either `prune` or 280 `rebuild-index` corrected the index files. This is now fixed and a test has 281 been added to detect this. 282 https://github.com/restic/restic/pull/1115 283 284 * Errors for chmod() on Unix for filesystems which do not support it (e.g. smb 285 mounted via gvfs) are now ignored. 286 https://github.com/restic/restic/pull/1080 287 https://github.com/restic/restic/pull/1112 288 289 * The semantic for the `--tags` option to `forget` and `snapshots` was 290 clarified: 291 https://github.com/restic/restic/issues/1081 292 https://github.com/restic/restic/pull/1090 293 294 Important Changes in 0.7.0 295 ========================== 296 297 * New "swift" backend: A new backend for the OpenStack Swift cloud storage 298 protocol has been added, https://wiki.openstack.org/wiki/Swift 299 https://github.com/restic/restic/pull/975 300 https://github.com/restic/restic/pull/648 301 302 * New "b2" backend: A new backend for Backblaze B2 cloud storage 303 service has been added, https://www.backblaze.com 304 https://github.com/restic/restic/issues/512 305 https://github.com/restic/restic/pull/978 306 307 * Improved performance for the `find` command: Restic recognizes paths it has 308 already checked for the files in question, so the number of backend requests 309 is reduced a lot. 310 https://github.com/restic/restic/issues/989 311 https://github.com/restic/restic/pull/993 312 313 * Improved performance for the fuse mount: Listing directories which contain 314 large files now is significantly faster. 315 https://github.com/restic/restic/pull/998 316 317 * The default layout for the s3 backend is now `default` (instead of 318 `s3legacy`). Also, there's a new `migrate` command to convert an existing 319 repo, it can be run like this: `restic migrate s3_layout` 320 https://github.com/restic/restic/issues/965 321 https://github.com/restic/restic/pull/1004 322 323 * The fuse mount now has two more directories: `tags` contains a subdir for 324 each tag, which in turn contains only the snapshots that have this tag. The 325 subdir `hosts` contains a subdir for each host that has a snapshot, and the 326 subdir contains the snapshots for that host. 327 https://github.com/restic/restic/issues/636 328 https://github.com/restic/restic/pull/1050 329 330 Small changes 331 ------------- 332 333 * For the s3 backend we're back to using the high-level API the s3 client 334 library for uploading data, a few users reported dropped connections (which 335 the library will automatically retry now). 336 https://github.com/restic/restic/issues/1013 337 https://github.com/restic/restic/issues/1023 338 https://github.com/restic/restic/pull/1025 339 340 * The `prune` command has been improved and will now remove invalid pack 341 files, for example files that have not been uploaded completely because a 342 backup was interrupted. 343 https://github.com/restic/restic/issues/1029 344 https://github.com/restic/restic/pull/1036 345 346 * restic now tries to detect when an invalid/unknown backend is used and 347 returns an error message. 348 https://github.com/restic/restic/issues/1021 349 https://github.com/restic/restic/pull/1070 350 351 Important Changes in 0.6.1 352 ========================== 353 354 This is mostly a bugfix release and only contains small changes: 355 356 * We've fixed a bug where `rebuild-index` would corrupt the index when used 357 with the s3 backend together with the `default` layout. This is not the 358 default setting. 359 360 * Backends based on HTTP now allow several idle connections in parallel. This 361 is especially important for the REST backend, which (when used with a local 362 server) may create a lot connections and exhaust available ports quickly. 363 https://github.com/restic/restic/issues/985 364 https://github.com/restic/restic/pull/986 365 366 * Regular status report: We've removed the status report that was printed 367 every 10 seconds when restic is run non-interactively. You can still force 368 reporting the current status by sending a `USR1` signal to the process. 369 https://github.com/restic/restic/pull/974 370 371 * The `build.go` now strips the temporary directory used for compilation from 372 the binary. This is the first step in enabling reproducible builds. 373 https://github.com/restic/restic/pull/981 374 375 Important Changes in 0.6.0 376 ========================== 377 378 Consistent forget policy 379 ------------------------ 380 381 The `forget` command was corrected to be more consistent in which snapshots are 382 to be forgotten. It is possible that the new code removes more snapshots than 383 before, so please review what would be deleted by using the `--dry-run` option. 384 385 https://github.com/restic/restic/pull/957 386 https://github.com/restic/restic/issues/953 387 388 Unified repository layout 389 ------------------------- 390 391 Up to now the s3 backend used a special repository layout. We've decided to 392 unify the repository layout and implemented the default layout also for the s3 393 backend. For creating a new repository on s3 with the default layout, use 394 `restic -o s3.layout=default init`. For further commands the option is not 395 necessary any more, restic will automatically detect the correct layout to use. 396 A future version will switch to the default layout for new repositories. 397 398 https://github.com/restic/restic/pull/966 399 https://github.com/restic/restic/issues/965 400 401 Memory and time improvements for the s3 backend 402 ----------------------------------------------- 403 404 We've updated the library used for accessing s3, switched to using a lower 405 level API and added caching for some requests. This lead to a decrease in 406 memory usage and a great speedup. In addition, we added benchmark functions for 407 all backends, so we can track improvements over time. The Continuous 408 Integration test service we're using (Travis) now runs the s3 backend tests not 409 only against a Minio server, but also against the Amazon s3 live service, so we 410 should be notified of any regressions much sooner. 411 412 https://github.com/restic/restic/pull/962 413 https://github.com/restic/restic/pull/960 414 https://github.com/restic/restic/pull/946 415 https://github.com/restic/restic/pull/938 416 https://github.com/restic/restic/pull/883