github.com/cozy/cozy-stack@v0.0.0-20240327093429-939e4a21320e/docs/data-system.md (about) 1 [Table of contents](README.md#table-of-contents) 2 3 # Data System 4 5 ## Typing 6 7 The notion of document type does not exist in Couchdb. 8 9 Cozy-stack introduce this notion through a special `_type` field. 10 11 This type name cannot contain `/`, and it should be unique among all developers, 12 it is recommended to use the Java naming convention with a domain you own. 13 14 All CozyCloud types will be prefixed by io.cozy and be pluralized. Example: 15 `/data/io.cozy.events/6494e0ac-dfcb-11e5-88c1-472e84a9cbee` Where, `io.cozy.` is 16 the developer specific prefix, `events` the actual type, and 17 `6494e0ac-dfcb-11e5-88c1-472e84a9cbee` the document's unique id . 18 19 ## Access a document 20 21 ### Request 22 23 ```http 24 GET /data/:type/:id HTTP/1.1 25 ``` 26 27 ```http 28 GET /data/io.cozy.events/6494e0ac-dfcb-11e5-88c1-472e84a9cbee HTTP/1.1 29 ``` 30 31 ### Response OK 32 33 ```http 34 HTTP/1.1 200 OK 35 Date: Mon, 27 Sept 2016 12:28:53 GMT 36 Content-Length: ... 37 Content-Type: application/json 38 Etag: "3-6494e0ac6494e0ac" 39 ``` 40 41 ```json 42 { 43 "_id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 44 "_type": "io.cozy.events", 45 "_rev": "3-6494e0ac6494e0ac", 46 "startdate": "20160823T150000Z", 47 "enddate": "20160923T160000Z", 48 "summary": "A long month", 49 "description": "I could go on and on and on ...." 50 } 51 ``` 52 53 ### Response Error 54 55 ```http 56 HTTP/1.1 404 Not Found 57 Content-Length: ... 58 Content-Type: application/json 59 ``` 60 61 ```json 62 { 63 "status": 404, 64 "error": "not_found", 65 "reason": "deleted", 66 "title": "Event deleted", 67 "details": "Event 6494e0ac-dfcb-11e5-88c1-472e84a9cbee was deleted", 68 "links": { "about": "https://cozy.github.io/cozy-stack/errors.md#deleted" } 69 } 70 ``` 71 72 ### Possible errors 73 74 - 401 unauthorized (no authentication has been provided) 75 - 403 forbidden (the authentication does not provide permissions for this 76 action) 77 - 404 not_found 78 - reason: missing 79 - reason: deleted 80 - 500 internal server error 81 82 ## Access multiple documents at once 83 84 ### Request 85 86 ```http 87 POST /data/:type/_all_docs HTTP/1.1 88 ``` 89 90 ```http 91 POST /data/io.cozy.files/_all_docs?include_docs=true HTTP/1.1 92 Content-Length: ... 93 Content-Type: application/json 94 Accept: application/json 95 ``` 96 97 ```json 98 { 99 "keys": [ 100 "7f46ed4ed2a775494da3b0b44e00314f", 101 "7f46ed4ed2a775494da3b0b44e003b18" 102 ] 103 } 104 ``` 105 106 ### Response OK 107 108 ```http 109 HTTP/1.1 200 OK 110 Date: Mon, 27 Sept 2016 12:28:53 GMT 111 Content-Length: ... 112 Content-Type: application/json 113 Etag: "3-6494e0ac6494e0ac" 114 ``` 115 116 ```json 117 { 118 "total_rows": 11, 119 "rows": [ 120 { 121 "id": "7f46ed4ed2a775494da3b0b44e00314f", 122 "key": "7f46ed4ed2a775494da3b0b44e00314f", 123 "value": { 124 "rev": "1-870e58f8a1b2130c3a41e767f9c7d93a" 125 }, 126 "doc": { 127 "_id": "7f46ed4ed2a775494da3b0b44e00314f", 128 "_rev": "1-870e58f8a1b2130c3a41e767f9c7d93a", 129 "type": "directory", 130 "name": "Uploaded from Cozy Photos", 131 "dir_id": "7f46ed4ed2a775494da3b0b44e0027df", 132 "created_at": "2017-07-04T06:49:12.844631837Z", 133 "updated_at": "2017-07-04T06:49:12.844631837Z", 134 "tags": [], 135 "path": "/Photos/Uploaded from Cozy Photos" 136 } 137 }, 138 { 139 "key": "7f46ed4ed2a775494da3b0b44e003b18", 140 "error": "not_found" 141 } 142 ] 143 } 144 ``` 145 146 ### possible errors 147 148 - 401 unauthorized (no authentication has been provided) 149 - 403 forbidden (the authentication does not provide permissions for this 150 action) 151 - 500 internal server error 152 153 ### Details 154 155 When some keys don't match an existing document, the response still has a status 156 200 and the errors are included in the `rows` field (see above, same behavior as 157 [CouchDB](http://docs.couchdb.org/en/stable/api/database/bulk-api.html#post--db-_all_docs)). 158 159 ## Create a document 160 161 ### Request 162 163 ```http 164 POST /data/:type/ HTTP/1.1 165 ``` 166 167 ```http 168 POST /data/io.cozy.events/ HTTP/1.1 169 Content-Length: ... 170 Content-Type: application/json 171 Accept: application/json 172 ``` 173 174 ```json 175 { 176 "startdate": "20160712T150000", 177 "enddate": "20160712T150000" 178 } 179 ``` 180 181 ### Response OK 182 183 ```http 184 HTTP/1.1 201 Created 185 Content-Length: ... 186 Content-Type: application/json 187 ``` 188 189 ```json 190 { 191 "id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 192 "type": "io.cozy.events", 193 "ok": true, 194 "rev": "1-6494e0ac6494e0ac", 195 "data": { 196 "_id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 197 "_type": "io.cozy.events", 198 "_rev": "1-6494e0ac6494e0ac", 199 "startdate": "20160712T150000", 200 "enddate": "20160712T150000" 201 } 202 } 203 ``` 204 205 ### possible errors 206 207 - 400 bad request 208 - 401 unauthorized (no authentication has been provided) 209 - 403 forbidden (the authentication does not provide permissions for this 210 action) 211 - 500 internal server error 212 213 ### Details 214 215 - A doc cannot contain an `_id` field, if so an error 400 is returned 216 - A doc cannot contain any field starting with `_`, those are reserved for 217 future cozy & couchdb api evolution 218 219 ## Update an existing document 220 221 ### Request 222 223 ```http 224 PUT /data/:type/:id HTTP/1.1 225 ``` 226 227 ```http 228 PUT /data/io.cozy.events/6494e0ac-dfcb-11e5-88c1-472e84a9cbee HTTP/1.1 229 Content-Length: ... 230 Content-Type: application/json 231 Accept: application/json 232 ``` 233 234 ```json 235 { 236 "_id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 237 "_type": "io.cozy.events", 238 "_rev": "1-6494e0ac6494e0ac", 239 "startdate": "20160712T150000", 240 "enddate": "20160712T200000" 241 } 242 ``` 243 244 ### Response OK 245 246 ```http 247 HTTP/1.1 200 OK 248 Content-Length: ... 249 Content-Type: application/json 250 ``` 251 252 ```json 253 { 254 "id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 255 "type": "io.cozy.events", 256 "ok": true, 257 "rev": "2-056f5f44046ecafc08a2bc2b9c229e20", 258 "data": { 259 "_id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 260 "_type": "io.cozy.events", 261 "_rev": "2-056f5f44046ecafc08a2bc2b9c229e20", 262 "startdate": "20160712T150000", 263 "enddate": "20160712T200000" 264 } 265 } 266 ``` 267 268 ### Possible errors 269 270 - 400 bad request 271 - 401 unauthorized (no authentication has been provided) 272 - 403 forbidden (the authentication does not provide permissions for this 273 action) 274 - 404 not_found 275 - reason: missing 276 - reason: deleted 277 - 409 Conflict (see Conflict prevention section below) 278 - 500 internal server error 279 280 ### Conflict prevention 281 282 The client MUST give a `_rev` field in the document. If this field is different 283 from the one in the current version of the document, an error 409 Conflict will 284 be returned. 285 286 ### Details 287 288 - If no id is provided in URL, an error 400 is returned 289 - If the id provided in URL is not the same than the one in document, an error 290 400 is returned. 291 292 ## Create a document with a fixed id 293 294 ### Request 295 296 ```http 297 PUT /data/:type/:id HTTP/1.1 298 ``` 299 300 ```http 301 PUT /data/io.cozy.events/6494e0ac-dfcb-11e5-88c1-472e84a9cbee HTTP/1.1 302 Content-Length: ... 303 Content-Type: application/json 304 Accept: application/json 305 ``` 306 307 ```json 308 { 309 "startdate": "20160712T150000", 310 "enddate": "20160712T200000" 311 } 312 ``` 313 314 ### Response OK 315 316 ```http 317 HTTP/1.1 200 OK 318 Content-Length: ... 319 Content-Type: application/json 320 ``` 321 322 ```json 323 { 324 "id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 325 "type": "io.cozy.events", 326 "ok": true, 327 "rev": "1-056f5f44046ecafc08a2bc2b9c229e20", 328 "data": { 329 "_id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 330 "_type": "io.cozy.events", 331 "_rev": "1-056f5f44046ecafc08a2bc2b9c229e20", 332 "startdate": "20160712T150000", 333 "enddate": "20160712T200000" 334 } 335 } 336 ``` 337 338 ### Possible errors 339 340 - 400 bad request 341 - 401 unauthorized (no authentication has been provided) 342 - 403 forbidden (the authentication does not provide permissions for this 343 action) 344 - 404 not_found 345 - reason: missing 346 - reason: deleted 347 - 409 Conflict (see Conflict prevention section below) 348 - 500 internal server error 349 350 ### Details 351 352 - No id should be provide in the document itself 353 354 ## Delete a document 355 356 ### Request 357 358 ```http 359 DELETE /data/:type/:id?rev=:rev HTTP/1.1 360 ``` 361 362 ```http 363 DELETE /data/io.cozy.events/6494e0ac-dfcb-11e5-88c1-472e84a9cbee?rev=1-82a7144c9ec228c9a851b8a1c1aa225b HTTP/1.1 364 Accept: application/json 365 ``` 366 367 ### Response OK 368 369 ```http 370 HTTP/1.1 200 OK 371 Content-Length: ... 372 Content-Type: application/json 373 ``` 374 375 ```json 376 { 377 "id": "6494e0ac-dfcb-11e5-88c1-472e84a9cbee", 378 "type": "io.cozy.events", 379 "ok": true, 380 "rev": "2-056f5f44046ecafc08a2bc2b9c229e20", 381 "_deleted": true 382 } 383 ``` 384 385 ### Possible errors 386 387 - 400 bad request 388 - 401 unauthorized (no authentication has been provided) 389 - 403 forbidden (the authentication does not provide permissions for this 390 action) 391 - 404 not_found 392 - reason: missing 393 - reason: deleted 394 - 409 Conflict (see Conflict prevention section below) 395 - 500 internal server error 396 397 ### Conflict prevention 398 399 It is possible to use either a `rev` query string parameter or a HTTP `If-Match` 400 header to prevent conflict on deletion: 401 402 - If none is passed or they are different, an error 400 is returned 403 - If only one is passed or they are equals, the document will only be deleted 404 if its `_rev` match the passed one. Otherwise, an error 409 is returned. 405 406 ### Details 407 408 - If no id is provided in URL, an error 400 is returned 409 410 ## List all the documents (recommended & paginated way) 411 412 We have added a non-standard `_normal_docs` endpoint since 413 `_all_docs` endpoint sends the design docs in the response 414 and that it makes it hard to use pagination on it. 415 416 This `_normal_docs` endpoint skip the design docs (and does 417 not count them in the`total_rows`). It accepts three parameters 418 in the query string: `limit` (default: 100), `skip` (default: 0), 419 and `bookmark` (default: ''). 420 421 The response format looks like a `_find` response with mango. 422 And, like for `_find`, the limit cannot be more than 1000. 423 The [pagination](https://github.com/cozy/cozy-stack/blob/master/docs/mango.md#pagination-cookbook) 424 is also the same: both `bookmark` and `skip` can be used, but 425 `bookmark` is recommended for performances. 426 427 ### Request 428 429 ```http 430 GET /data/io.cozy.events/_normal_docs?limit=100&bookmark=g1AAAAB2eJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYorGKQYpVqaJRoZm1paWFiapFkamhknGpilJiampZkYJRmC9HHA9OUAdTASpS0rCwAlah76 HTTP/1.1 431 Accept: application/json 432 ``` 433 434 ### Response 435 436 ```http 437 HTTP/1.1 200 OK 438 Content-Type: application/json 439 ``` 440 441 ```json 442 { 443 "rows": [ 444 { 445 "_id": "16e458537602f5ef2a710089dffd9453", 446 "_rev": "1-967a00dff5e02add41819138abb3284d", 447 "field": "value" 448 }, 449 { 450 "_id": "f4ca7773ddea715afebc4b4b15d4f0b3", 451 "_rev": "2-7051cbe5c8faecd085a3fa619e6e6337", 452 "field": "other-value" 453 }, 454 ... 455 ], 456 "total_rows": 202, 457 "bookmark": "g1AAAAB2eJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYorGKQYpVqaJRoZm1paWFiapFkamhknGpilJiampZkYJRmC9HHA9OUAdTASpS0rCwAlah76" 458 } 459 ``` 460 461 ## List all the documents (alternative & not pagginated way) 462 463 You can use `_all_docs` endpoint to get the list of all the documents. 464 In some cases the use of this route is legitimate, but we recommend to 465 avoid it as much as possible. 466 467 The stack also supports a `Fields` query parameter to only include those fields 468 in the response (with an uppercase `F` to avoid collusion with a possible 469 future `fields` by CouchDB), and a `DesignDocs` query parameter to skip design 470 docs. Example: `?Fields=name,metadata.title,tags&DesignDocs=false`. 471 472 ### Request 473 474 ```http 475 GET /data/io.cozy.events/_all_docs?include_docs=true HTTP/1.1 476 Accept: application/json 477 ``` 478 479 ### Response 480 481 ```http 482 HTTP/1.1 200 OK 483 Content-Type: application/json 484 ``` 485 486 ```json 487 { 488 "offset": 0, 489 "rows": [ 490 { 491 "id": "16e458537602f5ef2a710089dffd9453", 492 "key": "16e458537602f5ef2a710089dffd9453", 493 "value": { 494 "rev": "1-967a00dff5e02add41819138abb3284d" 495 }, 496 "doc": { 497 "field": "value" 498 } 499 }, 500 { 501 "id": "f4ca7773ddea715afebc4b4b15d4f0b3", 502 "key": "f4ca7773ddea715afebc4b4b15d4f0b3", 503 "value": { 504 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337" 505 }, 506 "doc": { 507 "field": "other-value" 508 } 509 } 510 ], 511 "total_rows": 2 512 } 513 ``` 514 515 ### Details 516 517 See 518 [`_all_docs` in couchdb docs](http://docs.couchdb.org/en/stable/api/database/bulk-api.html#db-all-docs) 519 520 ## List the known doctypes 521 522 A permission on `io.cozy.doctypes` for `GET` is needed to query this endoint. 523 524 ### Request 525 526 ```http 527 GET /data/_all_doctypes HTTP/1.1 528 Accept: application/json 529 ``` 530 531 ### Response 532 533 ```http 534 HTTP/1.1 200 OK 535 Content-Type: application/json 536 ``` 537 538 ```json 539 ["io.cozy.files", "io.cozy.jobs", "io.cozy.triggers", "io.cozy.settings"] 540 ``` 541 542 ## Others 543 544 - The creation and usage of [Mango indexes](mango.md) is possible. 545 - CouchDB behaviors are not always straight forward: see 546 [some quirks](couchdb-quirks.md) for more details. 547 548 549 ## Access a design document 550 551 A design document is a special CouchDB document that represents a view or Mango index definition. 552 553 ### Request 554 555 ```http 556 GET /data/:type/_design/:ddoc HTTP/1.1 557 ``` 558 559 ```http 560 GET /data/io.cozy.events/_design/c4a8fa4a4660b8eed43137881500265c HTTP/1.1 561 ``` 562 563 ### Response OK 564 565 ```http 566 HTTP/1.1 200 OK 567 Content-Type: application/json 568 ``` 569 570 ```json 571 { 572 "_id": "_design/c4a8fa4a4660b8eed43137881500265c", 573 "_rev": "1-56cf55098fc69450f84a22a632ffafb9", 574 "language": "query", 575 "views": { 576 "by-startdate-and-enddate": { 577 "map": { 578 "fields": { 579 "startdate": "asc", 580 "enddate": "asc", 581 }, 582 "partial_filter_selector": {} 583 }, 584 "reduce": "_count", 585 "options": { 586 "def": { 587 "fields": [ 588 "startdate", 589 "enddate", 590 "metadata.datetime" 591 ] 592 } 593 } 594 } 595 } 596 } 597 ``` 598 599 ## Access all design documents for a doctype 600 601 ### Request 602 603 ```http 604 GET /data/:type/_design_docs HTTP/1.1 605 ``` 606 607 ```http 608 GET /data/io.cozy.events/_design_docs HTTP/1.1 609 ``` 610 611 ### Response OK 612 613 ```http 614 HTTP/1.1 200 OK 615 Content-Type: application/json 616 ``` 617 618 ```json 619 { 620 "total_rows": 123, 621 "offset": 57, 622 "rows": [ 623 ... 624 ] 625 } 626 ``` 627 628 ## Delete a design document 629 630 ### Request 631 632 ```http 633 DELETE /data/:type/_design/:ddoc HTTP/1.1 634 ``` 635 636 ```http 637 DELETE /data/io.cozy.events/_design/c4a8fa4a4660b8eed43137881500265c?rev=1-1aa097a2eef904db9b1842342e6c6f50 HTTP/1.1 638 ``` 639 640 ### Response OK 641 642 ```http 643 HTTP/1.1 200 OK 644 Content-Type: application/json 645 ``` 646 647 ```json 648 { 649 "ok": true, 650 "id": "_design/c4a8fa4a4660b8eed43137881500265c", 651 "rev": "2-2ad11c9dc32df12a4b24f994807408ba" 652 } 653 ``` 654 655 ## Copy a design document 656 657 This is useful to duplicate a view or Mango index definition, without having to recompute the whole B-Tree. The original and the copy will both use the same index on disk. 658 659 660 ### Request 661 662 ```http 663 POST /data/:type/_design/:ddoc/copy HTTP/1.1 664 665 ``` 666 667 ```http 668 POST /data/io.cozy.events/_design/c4a8fa4a4660b8eed43137881500265c/copy?rev=1-1aa097a2eef904db9b1842342e6c6f50 HTTP/1.1 669 Destination: _design/d7349ab71c0859c408a725ebbfd453b18aa94ec7 670 ``` 671 672 ### Response OK 673 674 ```http 675 HTTP/1.1 201 Created 676 Content-Type: application/json 677 ``` 678 679 ```json 680 { 681 "ok": true, 682 "id": "_design/d7349ab71c0859c408a725ebbfd453b18aa94ec7", 683 "rev": "1-1aa097a2eef904db9b1842342e6c6f50" 684 } 685 ``` 686 687 ## Delete a database 688 689 ### Request 690 691 ```http 692 DELETE /data/:type/ HTTP/1.1 693 ``` 694 695 ```http 696 DELETE /data/io.cozy.events/ HTTP/1.1 697 ``` 698 699 ### Response OK 700 701 ```http 702 HTTP/1.1 200 OK 703 Content-Type: application/json 704 ``` 705 706 ```json 707 { 708 "ok": true, 709 "deleted": true 710 } 711 ```