github.com/mckael/restic@v0.8.3/doc/080_examples.rst (about) 1 .. 2 Normally, there are no heading levels assigned to certain characters as the structure is 3 determined from the succession of headings. However, this convention is used in Python’s 4 Style Guide for documenting which you may follow: 5 6 # with overline, for parts 7 * for chapters 8 = for sections 9 - for subsections 10 ^ for subsubsections 11 " for paragraphs 12 13 ######## 14 Examples 15 ######## 16 17 ******************************** 18 Setting up restic with Amazon S3 19 ******************************** 20 21 Preface 22 ======= 23 24 This tutorial will show you how to use restic with AWS S3. It will show you how 25 to navigate the AWS web interface, create an S3 bucket, create a user with 26 access to only this bucket, and finally how to connect restic to this bucket. 27 28 Prerequisites 29 ============= 30 31 You should already have a ``restic`` binary available on your system that you can 32 run. Furthermore, you should also have an account with 33 `AWS <https://aws.amazon.com/>`__. You will likely need to provide credit card 34 details for billing purposes, even if you use their 35 `free-tier <https://aws.amazon.com/free/>`__. 36 37 38 Logging into AWS 39 ================ 40 41 Point your browser to 42 https://console.aws.amazon.com 43 and log in using your AWS account. You will be presented with the AWS homepage: 44 45 .. image:: images/aws_s3/01_aws_start.png 46 :alt: AWS Homepage 47 48 By using the "Services" button in the upper left corder, a menu of all services 49 provided by AWS can be opened: 50 51 .. image:: images/aws_s3/02_aws_menu.png 52 :alt: AWS Services Menu 53 54 For this tutorial, the Simple Storage Service (S3), as well as Identity and 55 Access Management (IAM) are relevant. 56 57 58 Creating the bucket 59 =================== 60 61 First, a bucket to store your backups in must be created. Using the "Services" 62 menu, navigate to S3. In case you already have some S3 buckets, you will see a 63 list of them here: 64 65 .. image:: images/aws_s3/03_buckets_list_before.png 66 :alt: List of S3 Buckets 67 68 Click the "Create bucket" button and choose a name and region for your new 69 bucket. For the purpose of this tutorial, the bucket will be named 70 ``restic-demo`` and reside in Frankfurt. Because the bucket name space is 71 shared among all AWS users, the name ``restic-demo`` may not be available to 72 you. Be creative and choose a unique bucket name. 73 74 .. image:: images/aws_s3/04_bucket_create_start.png 75 :alt: Create a Bucket 76 77 It is not necessary to configure any special properties or permissions of the 78 bucket just yet. Therefore, just finish the wizard without making any further 79 changes: 80 81 .. image:: images/aws_s3/05_bucket_create_review.png 82 :alt: Review Bucket Creation 83 84 The newly created ``restic-demo`` bucket will now appear on the list of S3 85 buckets: 86 87 .. image:: images/aws_s3/06_buckets_list_after.png 88 :alt: List With New Bucket 89 90 Creating a user 91 =============== 92 93 Use the "Services" menu of the AWS web interface to navigate to IAM. This will 94 bring you to the IAM homepage. To create a new user, click on the "Users" menu 95 entry on the left: 96 97 .. image:: images/aws_s3/07_iam_start.png 98 :alt: IAM Home Page 99 100 In case you already have set-up users with IAM before, you will see a list of 101 them here. Use the "Add user" button at the top to create a new user: 102 103 .. image:: images/aws_s3/08_user_list.png 104 :alt: IAM User List 105 106 For this tutorial, the new user will be named ``restic-demo-user``. Feel free to 107 choose your own name that best fits your needs. This user will only ever access 108 AWS through the ``restic`` program and not through the web interface. Therefore, 109 "Programmatic access" is selected for "Access type": 110 111 .. image:: images/aws_s3/09_user_name.png 112 :alt: Choose User Name and Access Type 113 114 During the next step, permissions can be assigned to the new user. To use this 115 user with restic, it only needs access to the ``restic-demo`` bucket. Select 116 "Attach existing policies directly", which will bring up a list of pre-defined 117 policies below. Afterwards, click the "Create policy" button to create a custom 118 policy: 119 120 .. image:: images/aws_s3/10_user_pre_policy.png 121 :alt: Assign a Policy 122 123 A new browser window or tab will open with the policy wizard. In Amazon IAM, 124 policies are defined as JSON documents. For this tutorial, the "Visual editor" 125 will be used to generate a policy: 126 127 .. image:: images/aws_s3/11_policy_start.png 128 :alt: Create a New Policy 129 130 For restic to work, two permission statements must be created using the visual 131 policy editor. The first statement is set up as follows: 132 133 .. code:: 134 135 Service: S3 136 Allow Actions: DeleteObject, GetObject, PutObject 137 Resources: arn:aws:s3:::restic-demo/* 138 139 This statement allows restic to create, read and delete objects inside the S3 140 bucket named ``restic-demo``. Adjust the bucket's name to the name of the 141 bucket you created earlier. Next, add a second statement using the "Add 142 additional permissions" button: 143 144 .. code:: 145 146 Service: S3 147 Allow Actions: ListBucket, GetBucketLocation 148 Resource: arn:aws:s3:::restic-demo 149 150 Again, substitute ``restic-demo`` with the actual name of your bucket. Note 151 that, unlike before, there is no ``/*`` after the bucket name. This statement 152 allows restic to list the objects stored in the ``restic-demo`` bucket and to 153 query the bucket's region. 154 155 Continue to the next step by clicking the "Review policy" button and enter a 156 name and description for this policy. For this tutorial, the policy will be 157 named ``restic-demo-policy``. Click "Create policy" to finish the process: 158 159 .. image:: images/aws_s3/13_policy_review.png 160 :alt: Policy Review 161 162 Go back to the browser window or tab where you were previously creating the new 163 user. Click the button labeled "Refresh" above the list of policies to make 164 sure the newly created policy is available to you. Afterwards, use the search 165 function to search for the ``restic-demo-policy``. Select this policy using the 166 checkbox on the left. Then, continue to the next step. 167 168 .. image:: images/aws_s3/14_user_attach_policy.png 169 :alt: Attach Policy to User 170 171 The next page will present an overview of the user account that is about to be 172 created. If everything looks good, click "Create user" to complete the process: 173 174 .. image:: images/aws_s3/15_user_review.png 175 :alt: User Creation Review 176 177 After the user has been created, its access credentials will be displayed. They 178 consist of the "Access key ID" (think user name), and the "Secret access key" 179 (think password). Copy these down to a safe place. 180 181 .. image:: images/aws_s3/16_user_created.png 182 :alt: User Credentials 183 184 You have now completed the configuration in AWS. Feel free to close your web 185 browser now. 186 187 188 Initializing the restic repository 189 ================================== 190 191 Open a terminal and make sure you have the ``restic`` binary ready. First, choose 192 a password to encrypt your backups with. In this tutorial, ``apg`` is used for 193 this purpose: 194 195 .. code-block:: console 196 197 $ apg -a 1 -m 32 -n 1 -M NCL 198 I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5 199 200 Note this password somewhere safe along with your AWS credentials. Next, the 201 configuration of restic will be placed into environment variables. This will 202 include sensitive information, such as your AWS secret and repository password. 203 Therefore, make sure the next commands **do not** end up in your shell's 204 history file. Adjust the contents of the environment variables to fit your 205 bucket's name and your user's API credentials. 206 207 .. code-block:: console 208 209 $ unset HISTFILE 210 $ export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/restic-demo" 211 $ export AWS_ACCESS_KEY_ID="AKIAJAJSLTZCAZ4SRI5Q" 212 $ export AWS_SECRET_ACCESS_KEY="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" 213 $ export RESTIC_PASSWORD="I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5" 214 215 216 After the environment is set up, restic may be called to initialize the 217 repository: 218 219 220 .. code-block:: console 221 222 $ ./restic init 223 created restic backend b5c661a86a at s3:https://s3.amazonaws.com/restic-demo 224 225 Please note that knowledge of your password is required to access 226 the repository. Losing your password means that your data is 227 irrecoverably lost. 228 229 restic is now ready to be used with AWS S3. Try to create a backup: 230 231 .. code-block:: console 232 233 $ dd if=/dev/urandom bs=1M count=10 of=test.bin 234 10+0 records in 235 10+0 records out 236 10485760 bytes (10 MB, 10 MiB) copied, 0,0891322 s, 118 MB/s 237 238 $ ./restic backup test.bin 239 scan [/home/philip/restic-demo/test.bin] 240 scanned 0 directories, 1 files in 0:00 241 [0:04] 100.00% 2.500 MiB/s 10.000 MiB / 10.000 MiB 1 / 1 items ... ETA 0:00 242 duration: 0:04, 2.47MiB/s 243 snapshot 10fdbace saved 244 245 $ ./restic snapshots 246 ID Date Host Tags Directory 247 ---------------------------------------------------------------------- 248 10fdbace 2017-03-26 16:41:50 blackbox /home/philip/restic-demo/test.bin 249 250 A snapshot was created and stored in the S3 bucket. This snapshot may now be 251 restored: 252 253 .. code-block:: console 254 255 $ mkdir restore 256 257 $ ./restic restore 10fdbace --target restore 258 restoring <Snapshot 10fdbace of [/home/philip/restic-demo/test.bin] at 2017-03-26 16:41:50.201418102 +0200 CEST by philip@blackbox> to restore 259 260 $ ls restore/ 261 test.bin 262 263 The snapshot was successfully restored. This concludes the tutorial. 264 265 266 ***************************************************** 267 Backing up your system without running restic as root 268 ***************************************************** 269 270 Motivation 271 ========== 272 273 Creating a complete backup of a machine requires a privileged process 274 that is able to read all files. On UNIX-like systems this is 275 traditionally the ``root`` user. Processes running as root have 276 superpower. They cannot only read all files but do also have the power 277 to modify the system in any possible way. 278 279 With great power comes great responsibility. If a process running as 280 root malfunctions, is exploited, or simply configured in a wrong way it 281 can cause any possible damage to the system. This means you only want 282 to run programs as root that you trust completely. And even if you 283 trust a program, it is good and common practice to run it with the 284 least possible privileges. 285 286 Capabilities on Linux 287 ===================== 288 289 Fortunately, Linux has functionality to divide root's power into 290 single separate *capabilities*. You can remove these from a process 291 running as root to restrict it. And you can add capabilities to a 292 process running as a normal user, which is what we are going to do. 293 294 Full backup without root 295 ======================== 296 297 To be able to completely backup a system, restic has to read all the 298 files. Luckily Linux knows a capability that allows precisely this. We 299 can assign this single capability to restic and then run it as an 300 unprivileged user. 301 302 First we create a new user called ``restic`` that is going to create 303 the backups: 304 305 .. code-block:: console 306 307 root@a3e580b6369d:/# useradd -m restic 308 309 Then we download and install the restic binary into the user's home 310 directory. 311 312 .. code-block:: console 313 314 root@a3e580b6369d:/# mkdir ~restic/bin 315 root@a3e580b6369d:/# curl -L https://github.com/restic/restic/releases/download/v0.8.0/restic_0.8.0_linux_amd64.bz2 | bunzip2 > ~restic/bin/restic 316 317 Before we assign any special capability to the restic binary we 318 restrict its permissions so that only root and the newly created 319 restic user can execute it. Otherwise another - possibly untrusted - 320 user could misuse the privileged restic binary to circumvent file 321 access controls. 322 323 .. code-block:: console 324 325 root@a3e580b6369d:/# chown root:restic ~restic/bin/restic 326 root@a3e580b6369d:/# chmod 750 ~restic/bin/restic 327 328 Finally we can use ``setcap`` to add an extended attribute to the 329 restic binary. On every execution the system will read the extended 330 attribute, interpret it and assign capabilities accordingly. 331 332 .. code-block:: console 333 334 root@a3e580b6369d:/# setcap cap_dac_read_search=+ep ~restic/bin/restic 335 336 From now on the user ``restic`` can run restic to backup the whole 337 system. 338 339 .. code-block:: console 340 341 root@a3e580b6369d:/# sudo -u restic /opt/restic/bin/restic --exclude={/dev,/media,/mnt,/proc,/run,/sys,/tmp,/var/tmp} -r /tmp backup / 342