github.com/unigraph-dev/dgraph@v1.1.1-0.20200923154953-8b52b426f765/wiki/content/enterprise-features/index.md (about)

     1  +++
     2  date = "2017-03-20T19:35:35+11:00"
     3  title = "Enterprise Features"
     4  +++
     5  
     6  Dgraph enterprise features are proprietary licensed under the [Dgraph Community
     7  License][dcl]. All Dgraph releases contain proprietary code for enterprise features.
     8  Enabling these features requires an enterprise contract from
     9  [contact@dgraph.io](mailto:contact@dgraph.io) or the [discuss
    10  community](https://discuss.dgraph.io).
    11  
    12  **Dgraph enterprise features are enabled by default for 30 days in a new cluster**.
    13  After the trial period of thirty (30) days, the cluster must obtain a license from Dgraph to
    14  continue enjoying the enterprise features released in the proprietary code. The license can
    15  be applied to the cluster by including it as the body of a POST request and calling
    16  `/enterpriseLicense` HTTP endpoint on any Zero server.
    17  
    18  
    19  {{% notice "note" %}}
    20  At the conclusion of your 30-day trial period if a license has not been applied to the cluster,
    21  access to the enterprise features will be suspended. The cluster will continue to operate without
    22  enterprise features.
    23  {{% /notice %}}
    24  
    25  
    26  [dcl]: https://github.com/dgraph-io/dgraph/blob/master/licenses/DCL.txt
    27  
    28  ## Binary Backups
    29  
    30  Binary backups are full backups of Dgraph that are backed up directly to cloud
    31  storage such as Amazon S3 or any Minio storage backend. Backups can also be
    32  saved to an on-premise network file system shared by all alpha instances. These
    33  backups can be used to restore a new Dgraph cluster to the previous state from
    34  the backup. Unlike [exports]({{< relref "deploy/index.md#export-database" >}}),
    35  binary backups are Dgraph-specific and can be used to restore a cluster quickly.
    36  
    37  ### Configure Backup
    38  
    39  Backup is only enabled when a valid license file is supplied to a Zero server OR within the thirty
    40  (30) day trial period, no exceptions.
    41  
    42  #### Configure Amazon S3 Credentials
    43  
    44  To backup to Amazon S3, the Alpha must have the following AWS credentials set
    45  via environment variables:
    46  
    47   Environment Variable                        | Description
    48   --------------------                        | -----------
    49   `AWS_ACCESS_KEY_ID` or `AWS_ACCESS_KEY`     | AWS access key with permissions to write to the destination bucket.
    50   `AWS_SECRET_ACCESS_KEY` or `AWS_SECRET_KEY` | AWS access key with permissions to write to the destination bucket.
    51   `AWS_SESSION_TOKEN`                         | AWS session token (if required).
    52  
    53  #### Configure Minio Credentials
    54  
    55  To backup to Minio, the Alpha must have the following Minio credentials set via
    56  environment variables:
    57  
    58   Environment Variable                        | Description
    59   --------------------                        | -----------
    60   `MINIO_ACCESS_KEY`                          | Minio access key with permissions to write to the destination bucket.
    61   `MINIO_SECRET_KEY`                          | Minio secret key with permissions to write to the destination bucket.
    62  
    63  ### Create a Backup
    64  
    65  To create a backup, make an HTTP POST request to `/admin/backup` to a Dgraph
    66  Alpha HTTP address and port (default, "localhost:8080"). Like with all `/admin`
    67  endpoints, this is only accessible on the same machine as the Alpha unless
    68  [whitelisted for admin operations]({{< relref "deploy/index.md#whitelist-admin-operations" >}}).
    69  
    70  #### Backup to Amazon S3
    71  
    72  ```sh
    73  $ curl -XPOST localhost:8080/admin/backup -d "destination=s3://s3.us-west-2.amazonaws.com/<bucketname>"
    74  ```
    75  
    76  #### Backup to Minio
    77  
    78  ```sh
    79  $ curl -XPOST localhost:8080/admin/backup -d "destination=minio://127.0.0.1:9000/<bucketname>"
    80  ```
    81  
    82  #### Disabling HTTPS for S3 and Minio backups
    83  
    84  By default, Dgraph assumes the destination bucket is using HTTPS. If that is not
    85  the case, the backup will fail. To send a backup to a bucket using HTTP
    86  (insecure), set the query parameter `secure=false` with the destination
    87  endpoint:
    88  
    89  ```sh
    90  $ curl -XPOST localhost:8080/admin/backup -d "destination=minio://127.0.0.1:9000/<bucketname>?secure=false"
    91  ```
    92  
    93  #### Overriding Credentials
    94  
    95  The `access_key`, `secret_key`, and `session_token` parameters can be used to
    96  override the default credentials. Please note that unless HTTPS is used, the
    97  credentials will be transmitted in plain text so use these parameters with
    98  discretion. The environment variables should be used by default but these
    99  options are there to allow for greater flexibility.
   100  
   101  The `anonymous` parameter can be set to "true" to a allow backing up to S3 or
   102  Minio bucket that requires no credentials (i.e a public bucket).
   103  
   104  #### Backup to NFS
   105  
   106  ```
   107  # localhost:8080 is the default Alpha HTTP port
   108  $ curl -XPOST localhost:8080/admin/backup -d "destination=/path/to/local/directory"
   109  ```
   110  
   111  A local filesystem will work only if all the Alphas have access to it (e.g all
   112  the Alphas are running on the same filesystems as a normal process, not a Docker
   113  container). However, a NFS is recommended so that backups work seamlessly across
   114  multiple machines and/or containers.
   115  
   116  #### Forcing a Full Backup
   117  
   118  By default, an incremental backup will be created if there's another full backup
   119  in the specified location. To create a full backup, set the `force_full` parameter
   120  to `true`. Each series of backups can be
   121  identified by a unique ID and each backup in the series is assigned a
   122  monotonically increasing number. The following section contains more details on
   123  how to restore a backup series.
   124  
   125  ### Restore from Backup
   126  
   127  The `dgraph restore` command restores the postings directory from a previously
   128  created backup. Restore is intended to restore a backup to a new Dgraph cluster.
   129  During a restore, a new Dgraph Zero may be running to fully restore the backup
   130  state.
   131  
   132  The `--location` (`-l`) flag specifies a source URI with Dgraph backup objects.
   133  This URI supports all the schemes used for backup.
   134  
   135  The `--posting` (`-p`) flag sets the posting list parent directory to store the
   136  loaded backup files.
   137  
   138  The `--zero` (`-z`) optional flag specifies a Dgraph Zero address to update the
   139  start timestamp using the restored version. Otherwise, the timestamp must be
   140  manually updated through Zero's HTTP 'assign' endpoint.
   141  
   142  The `--backup_id` optional flag specifies the ID of the backup series to
   143  restore. A backup series consists of a full backup and all the incremental
   144  backups built on top of it. Each time a new full backup is created, a new backup
   145  series with a different ID is started. The backup series ID is stored in each
   146  `manifest.json` file stored in every backup folder.
   147  
   148  The restore feature will create a cluster with as many groups as the original
   149  cluster had at the time of the last backup. Restoring create a posting directory
   150  `p<N>` corresponding to the backup group ID. For example, a backup for Alpha
   151  group 2 would have the name ".../r32-g**2**.backup" and would be loaded to
   152  posting directory "p**2**".
   153  
   154  #### Restore from Amazon S3
   155  ```sh
   156  $ dgraph restore -p /var/db/dgraph -l s3://s3.us-west-2.amazonaws.com/<bucketname>
   157  ```
   158  
   159  #### Restore from Minio
   160  ```sh
   161  $ dgraph restore -p /var/db/dgraph -l minio://127.0.0.1:9000/<bucketname>
   162  ```
   163  
   164  #### Restore from Local Directory or NFS
   165  ```sh
   166  $ dgraph restore -p /var/db/dgraph -l /var/backups/dgraph
   167  ```
   168  
   169  #### Restore and Update Timestamp
   170  
   171  Specify the Zero address and port for the new cluster with `--zero`/`-z` to update the timestamp.
   172  ```sh
   173  $ dgraph restore -p /var/db/dgraph -l /var/backups/dgraph -z localhost:5080
   174  ```
   175  
   176  ## Access Control Lists
   177  
   178  Access Control List (ACL) provides access protection to your data stored in
   179  Dgraph. When the ACL feature is turned on, a client, e.g. dgo or dgraph4j, must
   180  authenticate with a username and password before executing any transactions, and
   181  is only allowed to access the data permitted by the ACL rules.
   182  
   183  This document has two parts: first we will talk about the admin operations
   184  needed for setting up ACL; then we will explain how to use a client to access
   185  the data protected by ACL rules.
   186  
   187  ### Turn on ACLs
   188  
   189  The ACL Feature can be turned on by following these steps
   190  
   191  1. Since ACL is an enterprise feature, make sure your use case is covered under
   192  a contract with Dgraph Labs Inc. You can contact us by sending an email to
   193  [contact@dgraph.io](mailto:contact@dgraph.io) or post your request at [our discuss
   194  community](https://discuss.dgraph.io) to get an enterprise license.
   195  
   196  2. Create a plain text file, and store a randomly generated secret key in it. The secret
   197  key is used by Alpha servers to sign JSON Web Tokens (JWT). As you’ve probably guessed,
   198  it’s critical to keep the secret key as a secret. Another requirement for the secret key
   199  is that it must have at least 256-bits, i.e. 32 ASCII characters, as we are using
   200  HMAC-SHA256 as the signing algorithm.
   201  
   202  3. Start all the alpha servers in your cluster with the option `--acl_secret_file`, and
   203  make sure they are all using the same secret key file created in Step 2.
   204  
   205  Here is an example that starts one zero server and one alpha server with the ACL feature turned on:
   206  
   207  ```bash
   208  dgraph zero --my=localhost:5080 --replicas 1 --idx 1 --bindall --expose_trace --profile_mode block --block_rate 10 --logtostderr -v=2
   209  dgraph alpha --my=localhost:7080 --lru_mb=1024 --zero=localhost:5080 --logtostderr -v=3 --acl_secret_file ./hmac-secret
   210  ```
   211  
   212  If you are using docker-compose, a sample cluster can be set up by:
   213  
   214  1. `cd $GOPATH/src/github.com/dgraph-io/dgraph/compose/`
   215  2. `make`
   216  3. `./compose -e --acl_secret <path to your hmac secret file>`, after which a `docker-compose.yml` file will be generated.
   217  4. `docker-compose up` to start the cluster using the `docker-compose.yml` generated above.
   218  
   219  ### Set up ACL Rules
   220  
   221  Now that your cluster is running with the ACL feature turned on, let's set up the ACL
   222  rules. A typical workflow is the following:
   223  
   224  1. Reset the root password. The example below uses the dgraph endpoint `localhost:9080`
   225  as a demo, make sure to choose the correct IP and port for your environment:
   226  ```bash
   227  dgraph acl -a localhost:9080 mod -u groot --new_password
   228  ```
   229  Now type in the password for the groot account, which is the superuser that has access to everything. The default password is `password`.
   230  2. Create a regular user
   231  ```bash
   232  dgraph acl -a localhost:9080 add -u alice
   233  ```
   234  Now you should see the following output
   235  ```console
   236  Current password for groot:
   237  Running transaction with dgraph endpoint: localhost:9080
   238  Login successful.
   239  New password for alice:
   240  Retype new password for alice:
   241  Created new user with id alice
   242  ```
   243  3. Create a group
   244  ```bash
   245  dgraph acl -a localhost:9080 add -g dev
   246  ```
   247  Again type in the groot password, and you should see the following output
   248  ```console
   249  Current password for groot:
   250  Running transaction with dgraph endpoint: localhost:9080
   251  Login successful.
   252  Created new group with id dev
   253  ```
   254  4. Assign the user to the group
   255  ```bash
   256  dgraph acl -a localhost:9080 mod -u alice -l dev
   257  ```
   258  The command above will add `alice` to the `dev` group. A user can be assigned to multiple groups.
   259  The multiple groups should be formated as a single string separated by `,`.
   260  For example, to assign the user `alice` to both the group `dev` and the group `sre`, the command should be
   261  ```bash
   262  dgraph acl -a localhost:9080 mod -u alice -l dev,sre
   263  ```
   264  5. Assign predicate permissions to the group
   265  ```bash
   266  dgraph acl mod -a localhost:9080 -g dev -p friend -m 7
   267  ```
   268  The command above grants the `dev` group the `READ`+`WRITE`+`MODIFY` permission on the
   269  `friend` predicate. Permissions are represented by a number following the UNIX file
   270  permission convention. That is, 4 (binary 100) represents `READ`, 2 (binary 010)
   271  represents `WRITE`, and 1 (binary 001) represents `MODIFY` (the permission to change a
   272  predicate's schema). Similarly, permisson numbers can be bitwise OR-ed to represent
   273  multiple permissions. For example, 7 (binary 111) represents all of `READ`, `WRITE` and
   274  `MODIFY`. In order for the example in the next section to work, we also need to grant
   275  full permissions on another predicate `name` to the group `dev`
   276  ```bash
   277  dgraph acl mod -a localhost:9080 -g dev -p name -m 7
   278  ```
   279  6. Check information about a user
   280  ```bash
   281  dgraph acl info -a localhost:9080 -u alice
   282  ```
   283  and the output should show the groups that the user has been added to, e.g.
   284  ```bash
   285  Running transaction with dgraph endpoint: localhost:9080
   286  Login successful.
   287  User  : alice
   288  UID   : 0x3
   289  Group : dev
   290  Group : sre
   291  ```
   292  7. Check information about a group
   293  ```bash
   294  dgraph acl info -a localhost:9080 -g dev
   295  ```
   296  and the output should include the users in the group, as well as the permissions the
   297  group's ACL rules, e.g.
   298  ```bash
   299  Current password for groot:
   300  Running transaction with dgraph endpoint: localhost:9080
   301  Login successful.
   302  Group: dev
   303  UID  : 0x4
   304  ID   : dev
   305  Users: alice
   306  ACL  : {friend  7}
   307  ACL  : {name  7}
   308  ```
   309  
   310  ### Access Data Using a Client
   311  
   312  Now that the ACL data are set, to access the data protected by ACL rules, we need to
   313  first log in through a user. A sample code using the dgo client can be found
   314  [here](https://github.com/dgraph-io/dgraph/blob/master/tlstest/acl/acl_over_tls_test.go).
   315  
   316  ### Access Data Using Curl
   317  
   318  Dgraph's HTTP API also supports authenticated operations to access ACL-protected
   319  data.
   320  
   321  To login, send a POST request to `/login` with the userid and password. For example, to log in as the root user groot:
   322  
   323  ```sh
   324  $ curl -X POST localhost:8080/login -d '{
   325    "userid": "groot",
   326    "password": "password"
   327  }'
   328  ```
   329  
   330  Response:
   331  ```
   332  {
   333    "data": {
   334      "accessJWT": "<accessJWT>",
   335      "refreshJWT": "<refreshJWT>"
   336    }
   337  }
   338  ```
   339  The response includes the access and refresh JWTs which are used for the authentication itself and refreshing the authentication token, respectively. Save the JWTs from the response for later HTTP requests.
   340  
   341  You can run authenticated requests by passing the accessJWT to a request via the `X-Dgraph-AccessToken` header. For example:
   342  
   343  ```sh
   344  $ curl -X POST -H 'Content-Type: application/graphql+-' -H 'X-Dgraph-AccessToken: <accessJWT>' localhost:8080/query -d '...'
   345  $ curl -X POST -H 'Content-Type: application/json' -H 'X-Dgraph-AccessToken: <accessJWT>' localhost:8080/mutate -d '...'
   346  $ curl -X POST -H 'X-Dgraph-AccessToken: <accessJWT>' localhost:8080/alter -d '...'
   347  ```
   348  
   349  The refresh token can be used in the `/login` POST body to receive new access and refresh JWTs, which is useful to renew the authenticated session once the ACL access TTL expires (controlled by Dgraph Alpha's flag `--acl_access_ttl` which is set to 6h0m0s by default).
   350  
   351  ```sh
   352  $ curl -X POST localhost:8080/login -d '{
   353    "refresh_token": "<refreshJWT>"
   354  }'
   355  ```