github.com/advanderveer/restic@v0.8.1-0.20171209104529-42a8c19aaea6/CHANGELOG.md (about) 1 This file describes changes relevant to all users that are made in each 2 released version of restic from the perspective of the user. 3 4 Important Changes in 0.X.Y 5 ========================== 6 7 * We've disabled handling SIGPIPE again. Turns out, writing to broken TCP 8 connections also raised SIGPIPE, so restic exits on the first write to a 9 broken connection. Instead, restic should retry the request. 10 https://github.com/restic/restic/pull/1459 11 https://github.com/restic/restic/issues/1457 12 https://github.com/restic/restic/issues/1466 13 14 * The command `diff` was added, it allows comparing two snapshots and listing 15 all differences. 16 https://github.com/restic/restic/issues/11 17 https://github.com/restic/restic/issues/1460 18 https://github.com/restic/restic/pull/1462 19 20 21 Small changes 22 ------------- 23 24 * We've added code to detect old cache directories of repositories that 25 haven't been used in a long time, restic now prints a note when it detects 26 that such dirs exist. Also, the option `--cleanup-cache` was added to 27 automatically remove such directories. That's not a problem because the 28 cache will be rebuild once a repo is accessed again. 29 https://github.com/restic/restic/pull/1436 30 31 * The cache directory on Windows and Darwin was not correct, instead the 32 directory `.cache` was used. 33 https://github.com/restic/restic/pull/1454 34 35 * By default, the access time for files and dirs is not saved any more. It is 36 not possible to reliably disable updating the access time during a backup, 37 so for the next backup the access time is different again. This means a lot 38 of metadata is saved. If you want to save the access time anyway, pass 39 `--with-atime` to the `backup` command. 40 https://github.com/restic/restic/pull/1452 41 42 * We've improved the s3 backend to work with DigitalOcean Spaces. 43 https://github.com/restic/restic/pull/1459 44 https://github.com/restic/restic/issues/1457 45 46 * The cancellation logic was improved, restic can now shut down cleanly when 47 requested to do so (e.g. via ctrl+c). 48 https://github.com/restic/restic/pull/1439 49 50 Important Changes in 0.8.0 51 ========================== 52 53 * A vulnerability was found in the restic restorer, which allowed attackers in 54 special circumstances to restore files to a location outside of the target 55 directory. Due to the circumstances we estimate this to be a low-risk 56 vulnerability, but urge all users to upgrade to the latest version of restic. 57 58 Exploiting the vulnerability requires a Linux/Unix system which saves 59 backups via restic and a Windows systems which restores files from the repo. 60 In addition, the attackers need to be able to create create files with 61 arbitrary names which are then saved to the restic repo. For example, by 62 creating a file named "..\test.txt" (which is a perfectly legal filename on 63 Linux) and restoring a snapshot containing this file on Windows, it would be 64 written to the parent of the target directory. 65 66 We'd like to thank Tyler Spivey for reporting this responsibly! 67 68 https://github.com/restic/restic/pull/1445 69 70 * The s3 backend used the subdir `restic` within a bucket if no explicit path 71 after the bucket name was specified. Since this version, restic does not use 72 this default path any more. If you created a repo on s3 in a bucket without 73 specifying a path within the bucket, you need to add `/restic` at the end of 74 the repository specification to access your repo: `s3:s3.amazonaws.com/bucket/restic` 75 https://github.com/restic/restic/issues/1292 76 https://github.com/restic/restic/pull/1437 77 78 * We've added a local cache for metadata so that restic doesn't need to load 79 all metadata (snapshots, indexes, ...) from the repo each time it starts. By 80 default the cache is active, but there's a new global option `--no-cache` 81 that can be used to disable the cache. By deafult, the cache a standard 82 cache folder for the OS, which can be overridden with `--cache-dir`. The 83 cache will automatically populate, indexes and snapshots are saved as they 84 are loaded. Cache directories for repos that haven't been used recently can 85 automatically be removed by restic with the `--cleanup-cache` option. 86 https://github.com/restic/restic/pull/1040 87 https://github.com/restic/restic/issues/29 88 https://github.com/restic/restic/issues/738 89 https://github.com/restic/restic/issues/282 90 https://github.com/restic/restic/pull/1287 91 https://github.com/restic/restic/pull/1436 92 93 * A related change was to by default create pack files in the repo that 94 contain either data or metadata, not both mixed together. This allows easy 95 caching of only the metadata files. The next run of `restic prune` will 96 untangle mixed files automatically. 97 https://github.com/restic/restic/pull/1265 98 99 * The Google Cloud Storage backend no longer requires the service account to 100 have the `storage.buckets.get` permission ("Storage Admin" role) in `restic 101 init` if the bucket already exists. 102 https://github.com/restic/restic/pull/1281 103 104 * Added support for rate limiting through `--limit-upload` and 105 `--limit-download` flags. 106 https://github.com/restic/restic/issues/1216 107 https://github.com/restic/restic/pull/1336 108 https://github.com/restic/restic/pull/1358 109 110 * Failed backend requests are now automatically retried. 111 https://github.com/restic/restic/pull/1353 112 113 * We've added the `dump` command which prints a file from a snapshot to 114 stdout. This can e.g. be used to restore files read with `backup --stdin`. 115 https://github.com/restic/restic/issues/510 116 https://github.com/restic/restic/pull/1346 117 118 Small changes 119 ------------- 120 121 * The directory structure in the fuse mount now exposes a symlink `latest` 122 which points to the latest snapshot in that particular directory. 123 https://github.com/restic/restic/pull/1249 124 125 * The option `--compact` was added to the `forget` command to provide the same 126 compact view as the `snapshots` command. 127 https://github.com/restic/restic/pull/1269 128 129 * We've re-enabled a workaround for `minio-go` (the library we're using to 130 access s3 backends), this reduces memory usage. 131 https://github.com/restic/restic/issues/1256 132 https://github.com/restic/restic/pull/1267 133 134 * The sftp backend now prompts for the password if a password is necessary for 135 login. 136 https://github.com/restic/restic/issues/448 137 https://github.com/restic/restic/pull/1270 138 139 * The `generate` command has been added, which replaces the now removed 140 commands `manpage` and `autocomplete`. This release of restic contains the 141 most recent manpages in `doc/man` and the auto-completion files for bash and 142 zsh in `doc/bash-completion.sh` and `doc/zsh-completion.zsh` 143 https://github.com/restic/restic/issues/1274 144 https://github.com/restic/restic/pull/1282 145 146 * A bug was discovered in the library we're using to access Backblaze, it now 147 reuses already established TCP connections which should be a lot faster and 148 not cause network failures any more. 149 https://github.com/restic/restic/issues/1291 150 https://github.com/restic/restic/pull/1301 151 152 * Another bug in the `forget` command caused `prune` not to be run when 153 `--prune` was specified without a policy, e.g. when only snapshot IDs that 154 should be forgotten are listed manually. This is corrected now. 155 https://github.com/restic/restic/pull/1317 156 157 * The `check` command now explicetly prints `No errors were found` when no 158 errors could be found. 159 https://github.com/restic/restic/pull/1319 160 https://github.com/restic/restic/issues/1303 161 162 * The fuse mount now has an `ids` subdirectory which contains the snapshots 163 below their (short) IDs. 164 https://github.com/restic/restic/issues/1102 165 https://github.com/restic/restic/pull/1299 166 https://github.com/restic/restic/pull/1320 167 168 * The `backup` command was improved, it now caches the result of excludes for 169 a directory. 170 https://github.com/restic/restic/issues/1271 171 https://github.com/restic/restic/pull/1326 172 173 * We've added the `--cacert` option which can be used to pass one (or more) CA 174 certificates to restic. These are used in addition to the system CA 175 certificates to verify HTTPS certificates (e.g. for the REST backend). 176 https://github.com/restic/restic/issues/1114 177 https://github.com/restic/restic/pull/1276 178 179 * When the list of files/dirs to be saved is read from a file with 180 `--files-from`, comment lines (starting with `#`) are now ignored. 181 https://github.com/restic/restic/issues/1367 182 https://github.com/restic/restic/pull/1368 183 184 Important Changes in 0.7.3 185 ========================== 186 187 * For large backups stored in Google Cloud Storage, the `prune` command fails 188 because listing only returns the first 1000 files. This has been corrected, 189 no data is lost in the process. In addition, a plausibility check was added 190 to `prune`. 191 https://github.com/restic/restic/issues/1246 192 https://github.com/restic/restic/pull/1247 193 194 195 Important Changes in 0.7.2 196 ========================== 197 198 * We've added an official docker image and a Dockerfile to build this image in 199 `docker/`. 200 https://github.com/restic/restic/pull/1061 201 202 * The git repository layout was changed to resemble the layout typically used 203 in Go projects, we're not using `gb` for building restic any more and 204 vendoring the dependencies is now taken care of by `dep`. 205 https://github.com/restic/restic/pull/1126 206 207 * We now support saving backups on Google Cloud Storage. 208 https://github.com/restic/restic/pull/1134 209 https://github.com/restic/restic/pull/1052 210 https://github.com/restic/restic/issues/211 211 212 * We've added support for Microsoft Azure Blob Storage as a restic backend. 213 https://github.com/restic/restic/pull/1149 214 https://github.com/restic/restic/pull/1059 215 https://github.com/restic/restic/issues/609 216 217 * In the course of supporting Microsoft Azure Blobe Storage Go 1.8 is now a 218 requirement to build restic. 219 220 * The `restore` command has been improved: When dirs are excluded (or not 221 included) in a restore, they are not loaded from the repo any more. 222 https://github.com/restic/restic/pull/1044 223 224 * Name collisions are now resolved by appending a counter. 225 https://github.com/restic/restic/issues/1179 226 https://github.com/restic/restic/pull/1209 227 228 229 Small changes 230 ------------- 231 232 * The `key` command now prompts for a password even if the original password 233 to access a repo has been specified via the `RESTIC_PASSWORD` environment 234 variable or a password file. 235 https://github.com/restic/restic/issues/1132 236 https://github.com/restic/restic/pull/1133 237 238 * Properly report errors when reading files with exclude patterns. 239 https://github.com/restic/restic/pull/1144 240 241 * We now automatically generate man pages for all restic commands, see the 242 subdir `doc/man`. 243 https://github.com/restic/restic/issues/697 244 https://github.com/restic/restic/pull/1147 245 246 * The `key remove` command was corrected and now works as documented. 247 https://github.com/restic/restic/pull/1164 248 249 * When a restic command other than `init` is used with a local repository and 250 the repository directory does not exist, restic creates the directory 251 structure. That's an error, only the `init` command should create the dir. 252 https://github.com/restic/restic/issues/1167 253 https://github.com/restic/restic/pull/1182 254 255 * Restic now prints stats on all BSD systems (not only on darwin) when SIGINFO 256 is received (usually when ctrl+t is pressed). 257 https://github.com/restic/restic/pull/1203 258 https://github.com/restic/restic/pull/1082#issuecomment-326279920 259 260 * Since a few releases restic had the ability to write profiling files for 261 memory and CPU usage when `debug` is enabled. It was discovered that when 262 restic is interrupted (ctrl+c is pressed), the proper shutdown hook is not 263 run. This is now corrected. 264 https://github.com/restic/restic/pull/1191 265 266 * A new option `--exclude-caches` was added that allows excluding cache 267 directories (that are tagged as such). This is a special case of a more 268 generic option `--exclude-if-present` which excludes a directory if a file 269 with a specific name (and contents) is present. 270 https://github.com/restic/restic/issues/317 271 https://github.com/restic/restic/pull/1170 272 https://github.com/restic/restic/pull/1224 273 274 * The `forget` command now has an option `--group-by` that allows flexible 275 grouping policies. 276 https://github.com/restic/restic/pull/1196 277 278 * The date and time restic records for a new backup can now be specified 279 externally by passing `--time` to the `backup` command. 280 https://github.com/restic/restic/pull/1205 281 282 * The option `--compact` was added to the `snapshots` command to get a better 283 overview of the snapshots in a repo. It limits each snapshot to a single 284 line. 285 https://github.com/restic/restic/issues/1218 286 https://github.com/restic/restic/pull/1223 287 288 289 Important Changes in 0.7.1 290 ========================== 291 292 * The `migrate` command for chaning the `s3legacy` layout to the `default` 293 layout for s3 backends has been improved: It can now be restarted with 294 `restic migrate --force s3_layout` and automatically retries operations on 295 error. 296 https://github.com/restic/restic/issues/1073 297 https://github.com/restic/restic/pull/1075 298 299 Small changes 300 ------------- 301 302 * The local and sftp backends now create the subdirs below `data/` on 303 open/init. This way, restic makes sure that they always exist. This is 304 connected to an issue for the sftp server: 305 https://github.com/restic/rest-server/pull/11#issuecomment-309879710 306 https://github.com/restic/restic/issues/1055 307 https://github.com/restic/restic/pull/1077 308 https://github.com/restic/restic/pull/1105 309 310 * When no S3 credentials are specified in the environment variables, restic 311 now tries to load credentials from an IAM instance profile when the s3 312 backend is used. 313 https://github.com/restic/restic/issues/1067 314 https://github.com/restic/restic/pull/1086 315 316 * On Darwin and FreeBSD, restic now prints stats when SIGINFO is received 317 (usually when ctrl+t is pressed). 318 https://github.com/restic/restic/pull/1082 319 320 * The dependencies have been updated. 321 https://github.com/restic/restic/pull/1108 322 https://github.com/restic/restic/pull/1124 323 324 * A bug was found (and corrected) in the index rebuilding after prune, which 325 led to indexes which include blobs that were not present in the repo any 326 more. There were already checks in place which detected this situation and 327 aborted with an error message. A new run of either `prune` or 328 `rebuild-index` corrected the index files. This is now fixed and a test has 329 been added to detect this. 330 https://github.com/restic/restic/pull/1115 331 332 * Errors for chmod() on Unix for filesystems which do not support it (e.g. smb 333 mounted via gvfs) are now ignored. 334 https://github.com/restic/restic/pull/1080 335 https://github.com/restic/restic/pull/1112 336 337 * The semantic for the `--tags` option to `forget` and `snapshots` was 338 clarified: 339 https://github.com/restic/restic/issues/1081 340 https://github.com/restic/restic/pull/1090 341 342 Important Changes in 0.7.0 343 ========================== 344 345 * New "swift" backend: A new backend for the OpenStack Swift cloud storage 346 protocol has been added, https://wiki.openstack.org/wiki/Swift 347 https://github.com/restic/restic/pull/975 348 https://github.com/restic/restic/pull/648 349 350 * New "b2" backend: A new backend for Backblaze B2 cloud storage 351 service has been added, https://www.backblaze.com 352 https://github.com/restic/restic/issues/512 353 https://github.com/restic/restic/pull/978 354 355 * Improved performance for the `find` command: Restic recognizes paths it has 356 already checked for the files in question, so the number of backend requests 357 is reduced a lot. 358 https://github.com/restic/restic/issues/989 359 https://github.com/restic/restic/pull/993 360 361 * Improved performance for the fuse mount: Listing directories which contain 362 large files now is significantly faster. 363 https://github.com/restic/restic/pull/998 364 365 * The default layout for the s3 backend is now `default` (instead of 366 `s3legacy`). Also, there's a new `migrate` command to convert an existing 367 repo, it can be run like this: `restic migrate s3_layout` 368 https://github.com/restic/restic/issues/965 369 https://github.com/restic/restic/pull/1004 370 371 * The fuse mount now has two more directories: `tags` contains a subdir for 372 each tag, which in turn contains only the snapshots that have this tag. The 373 subdir `hosts` contains a subdir for each host that has a snapshot, and the 374 subdir contains the snapshots for that host. 375 https://github.com/restic/restic/issues/636 376 https://github.com/restic/restic/pull/1050 377 378 Small changes 379 ------------- 380 381 * For the s3 backend we're back to using the high-level API the s3 client 382 library for uploading data, a few users reported dropped connections (which 383 the library will automatically retry now). 384 https://github.com/restic/restic/issues/1013 385 https://github.com/restic/restic/issues/1023 386 https://github.com/restic/restic/pull/1025 387 388 * The `prune` command has been improved and will now remove invalid pack 389 files, for example files that have not been uploaded completely because a 390 backup was interrupted. 391 https://github.com/restic/restic/issues/1029 392 https://github.com/restic/restic/pull/1036 393 394 * restic now tries to detect when an invalid/unknown backend is used and 395 returns an error message. 396 https://github.com/restic/restic/issues/1021 397 https://github.com/restic/restic/pull/1070 398 399 Important Changes in 0.6.1 400 ========================== 401 402 This is mostly a bugfix release and only contains small changes: 403 404 * We've fixed a bug where `rebuild-index` would corrupt the index when used 405 with the s3 backend together with the `default` layout. This is not the 406 default setting. 407 408 * Backends based on HTTP now allow several idle connections in parallel. This 409 is especially important for the REST backend, which (when used with a local 410 server) may create a lot connections and exhaust available ports quickly. 411 https://github.com/restic/restic/issues/985 412 https://github.com/restic/restic/pull/986 413 414 * Regular status report: We've removed the status report that was printed 415 every 10 seconds when restic is run non-interactively. You can still force 416 reporting the current status by sending a `USR1` signal to the process. 417 https://github.com/restic/restic/pull/974 418 419 * The `build.go` now strips the temporary directory used for compilation from 420 the binary. This is the first step in enabling reproducible builds. 421 https://github.com/restic/restic/pull/981 422 423 Important Changes in 0.6.0 424 ========================== 425 426 Consistent forget policy 427 ------------------------ 428 429 The `forget` command was corrected to be more consistent in which snapshots are 430 to be forgotten. It is possible that the new code removes more snapshots than 431 before, so please review what would be deleted by using the `--dry-run` option. 432 433 https://github.com/restic/restic/pull/957 434 https://github.com/restic/restic/issues/953 435 436 Unified repository layout 437 ------------------------- 438 439 Up to now the s3 backend used a special repository layout. We've decided to 440 unify the repository layout and implemented the default layout also for the s3 441 backend. For creating a new repository on s3 with the default layout, use 442 `restic -o s3.layout=default init`. For further commands the option is not 443 necessary any more, restic will automatically detect the correct layout to use. 444 A future version will switch to the default layout for new repositories. 445 446 https://github.com/restic/restic/pull/966 447 https://github.com/restic/restic/issues/965 448 449 Memory and time improvements for the s3 backend 450 ----------------------------------------------- 451 452 We've updated the library used for accessing s3, switched to using a lower 453 level API and added caching for some requests. This lead to a decrease in 454 memory usage and a great speedup. In addition, we added benchmark functions for 455 all backends, so we can track improvements over time. The Continuous 456 Integration test service we're using (Travis) now runs the s3 backend tests not 457 only against a Minio server, but also against the Amazon s3 live service, so we 458 should be notified of any regressions much sooner. 459 460 https://github.com/restic/restic/pull/962 461 https://github.com/restic/restic/pull/960 462 https://github.com/restic/restic/pull/946 463 https://github.com/restic/restic/pull/938 464 https://github.com/restic/restic/pull/883