github.com/cozy/cozy-stack@v0.0.0-20240327093429-939e4a21320e/docs/jobs.md (about) 1 [Table of contents](README.md#table-of-contents) 2 3 # Jobs 4 5 Jobs are designed to represent asynchronous tasks that your cozy can execute. 6 These tasks can be scheduled in advance, recurring or sudden and provide various 7 services. 8 9 At the time, we do not provide "generic" and programmable tasks. This list of 10 available workers is part of the defined API. 11 12 The job queue can be either local to the cozy-stack when used as a monolithic 13 instance (self-hosted for instance) or distributed via a redis server for 14 distributed infrastructures. 15 16 This doc introduces two cozy types: 17 18 - `io.cozy.jobs` for jobs 19 - `io.cozy.triggers` for triggers 20 21 ## Triggers 22 23 Jobs can be queued up via the `/jobs/queue/:worker-type` API, for a direct 24 execution. But it can also be convenient to schedule jobs on some conditions, 25 and the triggers are the way to do that. 26 27 Jobs can be launched by different types of triggers: 28 29 - `@at` to schedule a one-time job executed after at a specific time in the 30 future 31 - `@in` to schedule a one-time job executed after a specific amount of time 32 - `@hourly` to schedule jobs that run once an hour 33 - `@daily` to schedule jobs that run once a day 34 - `@weekly` to schedule jobs that run once a week 35 - `@monthly` to schedule jobs that run once a month 36 - `@every` to schedule periodic jobs executed at a given fix interval 37 - `@cron` to schedule recurring jobs scheduled at specific times 38 - `@event` to launch a job after a change on documents in the cozy 39 - `@webhook` to launch a job when an HTTP request hit a specific URL 40 - `@client` when the client controls when the job are launched. 41 42 These triggers have specific syntaxes to describe when jobs should be 43 scheduled. See below for more informations. 44 45 ### `@at` syntax 46 47 The `@at` trigger takes a ISO-8601 formatted string indicating a UTC time in the 48 future. The date is of this form: `YYYY-MM-DDTHH:mm:ss.sssZ` 49 50 :warning: Be aware that the `@at` trigger is removed from the doctype after it has created the associated job. 51 52 Examples 53 54 ``` 55 @at 2018-12-12T15:36:25.507Z 56 ``` 57 58 ### `@in` syntax 59 60 The `@in` trigger takes the same duration syntax as `@every` 61 62 :warning: Be aware that the `@in` trigger is removed from the doctype after it has created the associated job. 63 64 Examples 65 66 ``` 67 @in 10m 68 @in 1h30m 69 ``` 70 71 ### `@daily` syntax 72 73 The `@daily` trigger will create a job once a day. By default, the stack is 74 free to choose the day and hour when to do that, but you can add restrictions: 75 76 ``` 77 @daily # Once a day, any hour 78 @daily before 5am # Once a day, between midnight and 5am (UTC) 79 @daily after 10pm # Once a day, between 10pm and midnight (UTC) 80 @daily between 8am and 6pm # Once a day, between 8am and 6pm (UTC) 81 ``` 82 83 ### `@weekly` syntax 84 85 The `@weekly` trigger will create a job once a week. By default, the stack is 86 free to choose the day and hour when to do that, but you can add restrictions: 87 88 ``` 89 @weekly # Once a week, any day, any hour 90 @weekly on monday # Every monday 91 @weekly on mon,wed,fri # Once a week, on a monday, wednesday, or friday 92 @weekly on wed-fri # Once a week, on a wednesday, thursday, or friday 93 @weekly on weekday # Once a week, but not the week-end 94 @weekly on weekend # Every week-end (saturday or sunday) 95 @weekly before 5am # Once a week, any day, but between midnight and 5am (UTC) 96 @weekly after 10pm # Once a week, any day, but between 10pm and midnight (UTC) 97 @weekly between 8am and 6pm # Once a week, any day, between 8am and 6pm (UTC) 98 @weekly on monday before 9am # Every monday, between midnight and 9am (UTC) 99 ``` 100 101 ### `@monthly` syntax 102 103 The `@monthly` trigger will create a job once a month. By default, the stack is 104 free to choose the day and hour when to do that, but you can add restrictions: 105 106 ``` 107 @monthly # Once a month, any day, any hour 108 @monthly on the 1 # Every first day of the month 109 @monthly on the 1-5 # Once a month, on the first five days of the month 110 @monthly before 5am # Once a month, any day, but between midnight and 5am (UTC) 111 @monthly after 10pm # Once a month, any day, but between 10pm and midnight (UTC) 112 @monthly between 8am and 6pm # Once a month, any day, between 8am and 6pm (UTC) 113 @monthly on the 1 before 9am # Every first day of the month, between midnight and 9am (UTC) 114 ``` 115 116 **Note:** the current implementation is to take a random day/hour and run the 117 job each month at this day/hour. So, you should avoid 29-31 if you really want 118 the job to run each month. 119 120 ### `@every` syntax 121 122 The `@every` trigger uses the same syntax as golang's `time.ParseDuration` (but 123 only support time units above seconds): 124 125 A duration string is a possibly signed sequence of decimal numbers, each with 126 optional fraction and a unit suffix, such as "300ms", "1.5h" or "2h45m". Valid 127 time units are "s", "m", "h". 128 129 Examples 130 131 ``` 132 @every 1.5h # schedules every 1 and a half hours 133 @every 30m10s # schedules every 30 minutes and 10 seconds 134 ``` 135 136 ### `@cron` syntax 137 138 In order to schedule recurring jobs, the `@cron` trigger has the syntax using 139 six fields: 140 141 | Field name | Mandatory? | Allowed values | Allowed special characters | 142 | ------------ | ---------- | --------------- | -------------------------- | 143 | Seconds | Yes | 0-59 | \* / , - | 144 | Minutes | Yes | 0-59 | \* / , - | 145 | Hours | Yes | 0-23 | \* / , - | 146 | Day of month | Yes | 1-31 | \* / , - ? | 147 | Month | Yes | 1-12 or JAN-DEC | \* / , - | 148 | Day of week | Yes | 0-6 or SUN-SAT | \* / , - ? | 149 150 Asterisk ( `*` ) 151 152 The asterisk indicates that the cron expression will match for all values of the 153 field; e.g., using an asterisk in the 5th field (month) would indicate every 154 month. 155 156 Slash ( `/` ) 157 158 Slashes are used to describe increments of ranges. For example 3-59/15 in the 159 1st field (minutes) would indicate the 3rd minute of the hour and every 15 160 minutes thereafter. The form `"*\/..."` is equivalent to the form 161 `"first-last/..."`, that is, an increment over the largest possible range of the 162 field. The form `"N-/..."` is accepted as meaning `"N-MAX/..."`, that is, 163 starting at N, use the increment until the end of that specific range. It does 164 not wrap around. 165 166 Comma ( `,` ) 167 168 Commas are used to separate items of a list. For example, using `"MON,WED,FRI"` 169 in the 5th field (day of week) would mean Mondays, Wednesdays and Fridays. 170 171 Hyphen ( `-` ) 172 173 Hyphens are used to define ranges. For example, 9-17 would indicate every hour 174 between 9am and 5pm inclusive. 175 176 Question mark ( `?` ) 177 178 Question mark may be used instead of `*` for leaving either day-of-month or 179 day-of-week blank. 180 181 To schedule jobs given an interval: 182 183 Examples: 184 185 ``` 186 @cron 0 0 0 1 1 * # Run once a year, midnight, Jan. 1st 187 @cron 0 0 0 1 1 * # Run once a year, midnight, Jan. 1st 188 @cron 0 0 0 1 * * # Run once a month, midnight, first of month 189 @cron 0 0 0 * * 0 # Run once a week, midnight on Sunday 190 @cron 0 0 0 * * * # Run once a day, midnight 191 @cron 0 0 * * * * # Run once an hour, beginning of hour 192 ``` 193 194 ### `@event` syntax 195 196 The `@event` syntax allows to trigger a job when something occurs in the stack. 197 It follows the same syntax than [permissions 198 scope](https://docs.cozy.io/en/cozy-stack/permissions/#what-is-a-permission) 199 string: 200 201 `type[:verb][:values][:selector]` 202 203 Unlike for permissions string, the verb should be one of `CREATED`, `DELETED`, 204 `UPDATED`. It is possible to put several verbs, separated by a comma. 205 206 There is also a special value `!=`. It means that a job will be trigger only if 207 the value for the given selector has changed (ie the value before the update and 208 the value after that are different). 209 210 The job worker will receive a compound message including original trigger_infos 211 messages and the event which has triggered it. 212 213 Examples: 214 215 ``` 216 @event io.cozy.files // anything happens on files 217 @event io.cozy.files:CREATED // a file was created 218 @event io.cozy.files:DELETED:image/jpg:mime // an image was deleted 219 @event io.cozy.bank.operations:CREATED io.cozy.bank.bills:CREATED // a bank operation or a bill 220 @event io.cozy.bank.operations:CREATED,UPDATED // a bank operation created or updated 221 @event io.cozy.bank.operations:UPDATED:!=:category // a change of category for a bank operation 222 ``` 223 224 ### `@webhook` syntax 225 226 It takes no parameter. The URL to hit is not controlled by the request, but is 227 chosen by the server (and is returned as `webhook` in the `links` JSON-API 228 response). 229 230 Example: 231 232 ``` 233 @webhook 234 ``` 235 236 ### `@client` syntax 237 238 It takes no parameter and can only by used for the `client` worker. The stack 239 won't create a job unless a client calls the launch endpoint for this trigger. 240 The main goal of this trigger is keep a state, as the aggregation of job 241 results. 242 243 ## Error Handling 244 245 Jobs can fail to execute their task. We have two ways to parameterize such 246 cases. 247 248 ### Retry 249 250 A retry count can be optionally specified to ask the worker to re-execute the 251 task if it has failed. 252 253 Each retry is executed after a configurable delay. The try count is part of the 254 attributes of the job. Also, each occurring error is kept in the `errors` field 255 containing all the errors that may have happened. 256 257 ### Timeout 258 259 A worker may never end. To prevent this, a configurable timeout value is 260 specified with the job. 261 262 If a job does not end after the specified amount of time, it will be aborted. A 263 timeout is just like another error from the worker and can provoke a retry if 264 specified. 265 266 ### Defaults 267 268 By default, jobs are parameterized with a maximum of 3 tries with 1 minute 269 timeout. 270 271 These defaults may vary given the workload of the workers. 272 273 ## Jobs API 274 275 Example and description of the attributes of a `io.cozy.jobs`: 276 277 ```js 278 { 279 "domain": "me.cozy.localhost", 280 "worker": "sendmail", // worker type name 281 "options": { 282 "timeout": 60, // timeout value in seconds 283 "max_exec_count": 3, // maximum number of time the job should be executed (including retries) 284 }, 285 "arguments": { // the arguments will be given to the worker (if you look in CouchDB, it is called message there) 286 "mode": "noreply", 287 "template_name": "new_registration", 288 "template_values": { 289 "DevicesLink": "http://me.cozy.localhost/#/connectedDevices", 290 } 291 }, 292 "state": "running", // queued, running, done, errored 293 "queued_at": "2016-09-19T12:35:08Z", // time of the queuing 294 "started_at": "2016-09-19T12:35:08Z", // time of first execution 295 "error": "" // error message if any 296 } 297 ``` 298 299 Example and description of a job creation options — as you can see, the options 300 are replicated in the `io.cozy.jobs` attributes: 301 302 ```js 303 { 304 "timeout": 60, // timeout value in seconds 305 "max_exec_count": 3, // maximum number of retry 306 } 307 ``` 308 309 ### GET /jobs/:job-id 310 311 Get a job informations given its ID. 312 313 #### Request 314 315 ```http 316 GET /jobs/123123 HTTP/1.1 317 Accept: application/vnd.api+json 318 ``` 319 320 #### Response 321 322 ```json 323 { 324 "data": { 325 "type": "io.cozy.jobs", 326 "id": "123123", 327 "attributes": { 328 "domain": "me.cozy.localhost", 329 "worker": "sendmail", 330 "options": { 331 "timeout": 60, 332 "max_exec_count": 3 333 }, 334 "state": "running", 335 "queued_at": "2016-09-19T12:35:08Z", 336 "started_at": "2016-09-19T12:35:08Z", 337 "error": "" 338 }, 339 "links": { 340 "self": "/jobs/123123" 341 } 342 } 343 } 344 ``` 345 346 ### POST /jobs/queue/:worker-type 347 348 Enqueue programmatically a new job. 349 350 This route requires a specific permission on the worker-type. A global 351 permission on the global `io.cozy.jobs` doctype is not allowed. 352 353 Each [worker](./workers.md) accepts different arguments. For konnectors, the 354 arguments will be given in the `process.env['COZY_FIELDS']` variable. 355 356 #### Request 357 358 ```http 359 POST /jobs/queue/sendmail HTTP/1.1 360 Content-Type: application/vnd.api+json 361 Accept: application/vnd.api+json 362 ``` 363 364 ```json 365 { 366 "data": { 367 "attributes": { 368 "manual": false, 369 "options": { 370 "timeout": 60, 371 "max_exec_count": 3 372 }, 373 "arguments": {} // any json value used as arguments for the job 374 } 375 } 376 } 377 ``` 378 379 #### Response 380 381 ```json 382 { 383 "data": { 384 "type": "io.cozy.jobs", 385 "id": "123123", 386 "attributes": { 387 "domain": "me.cozy.localhost", 388 "worker": "sendmail", 389 "options": { 390 "timeout": 60, 391 "max_exec_count": 3 392 }, 393 "state": "running", 394 "queued_at": "2016-09-19T12:35:08Z", 395 "started_at": "2016-09-19T12:35:08Z", 396 "error": "" 397 }, 398 "links": { 399 "self": "/jobs/123123" 400 } 401 } 402 } 403 ``` 404 405 #### Permissions 406 407 To use this endpoint, an application needs a permission on the type 408 `io.cozy.jobs` for the verb `POST`. The is required to restrict its permission 409 to specific worker(s), like this (a global permission on the doctype is not 410 allowed): 411 412 ```json 413 { 414 "permissions": { 415 "mail-from-the-user": { 416 "description": "Required to send mails from the user to his/her friends", 417 "type": "io.cozy.jobs", 418 "verbs": ["POST"], 419 "selector": "worker", 420 "values": ["sendmail"] 421 } 422 } 423 } 424 ``` 425 426 ### POST /jobs/support 427 428 Send a mail to the support (email address defined by `mail.reply_to` in the 429 config file, or overwritten by context with `contexts.<name>.reply_to`). 430 431 It requires a permission on `io.cozy.support` (a permission on 432 `io.cozy.jobs:POST:sendmail:worker`, or larger, is also accepted to ease the 433 transition from sending manually a mail to the support via the sendmail queue). 434 435 #### Request 436 437 ```http 438 POST /jobs/support HTTP/1.1 439 Content-Type: application/vnd.api+json 440 ``` 441 442 ```json 443 { 444 "data": { 445 "attributes": { 446 "arguments": { 447 "subject": "Cozy is so cool!", 448 "body": "I really love Cozy. Thank you so much!" 449 } 450 } 451 } 452 } 453 ``` 454 455 #### Response 456 457 ```http 458 HTTP/1.1 204 No Content 459 ``` 460 461 ### POST /jobs/campaign-emails 462 463 Send a non transactional (or campaign) email to the user via the dedicated 464 campaign mail server (configured via `campaign_mail` attributes in the config 465 file, or overwritten by context with `campaign_mail.contexts.<name>` 466 attributes). 467 468 Both the subject and at least one part are required. 469 470 #### Request 471 472 ```http 473 POST /jobs/campaign-emails HTTP/1.1 474 Content-Type: application/vnd.api+json 475 ``` 476 477 ```json 478 { 479 "data": { 480 "attributes": { 481 "arguments": { 482 "subject": "Checkout the new cool stuff!", 483 "parts": [ 484 { 485 "body": "So many new features to check out!", 486 "type": "text/plain" 487 } 488 ] 489 } 490 } 491 } 492 } 493 ``` 494 495 #### Response 496 497 ```http 498 HTTP/1.1 204 No Content 499 ``` 500 501 #### Permissions 502 503 To use this endpoint, an application needs a permission on the type 504 `io.cozy.jobs` for the verb `POST` and the `sendmail` worker. 505 This can be defined like so: 506 507 ```json 508 { 509 "permissions": { 510 "campaign-emails": { 511 "description": "Required to send campaign emails to the user", 512 "type": "io.cozy.jobs", 513 "verbs": ["POST"], 514 "selector": "worker", 515 "values": ["sendmail"] 516 } 517 } 518 } 519 ``` 520 521 ### GET /jobs/queue/:worker-type 522 523 List the jobs in the queue. 524 525 #### Request 526 527 ```http 528 GET /jobs/queue/sendmail HTTP/1.1 529 Accept: application/vnd.api+json 530 ``` 531 532 #### Response 533 534 ```json 535 { 536 "data": [ 537 { 538 "attributes": { 539 "domain": "cozy.localhost:8080", 540 "options": null, 541 "queued_at": "2017-09-29T15:32:31.953878568+02:00", 542 "started_at": "0001-01-01T00:00:00Z", 543 "state": "queued", 544 "worker": "log" 545 }, 546 "id": "77689bca9634b4fb08d6ca3d1643de5f", 547 "links": { 548 "self": "/jobs/log/77689bca9634b4fb08d6ca3d1643de5f" 549 }, 550 "meta": { 551 "rev": "1-f823bcd2759103a5ad1a98f4bf083b36" 552 }, 553 "type": "io.cozy.jobs" 554 } 555 ], 556 "meta": { 557 "count": 0 558 } 559 } 560 ``` 561 562 #### Permissions 563 564 To use this endpoint, an application needs a permission on the type 565 `io.cozy.jobs` for the verb `GET`. In most cases, the application will restrict 566 its permission to only one worker, like this: 567 568 ```json 569 { 570 "permissions": { 571 "mail-from-the-user": { 572 "description": "Required to know the number of jobs in the sendmail queues", 573 "type": "io.cozy.jobs", 574 "verbs": ["GET"], 575 "selector": "worker", 576 "values": ["sendmail"] 577 } 578 } 579 } 580 ``` 581 582 ### PATCH /jobs/:job-id 583 584 This endpoint can be used for a job of the `client` worker (executed by a 585 client, not on the server) to update the status. 586 587 #### Request 588 589 ```http 590 PATCH /jobs/022368c07dc701396403543d7eb8149c HTTP/1.1 591 Content-Type: application/vnd.api+json 592 ``` 593 594 ```json 595 { 596 "data": { 597 "type": "io.cozy.jobs", 598 "id": "022368c07dc701396403543d7eb8149c", 599 "attributes": { 600 "state": "errored", 601 "error": "LOGIN_FAILED" 602 } 603 } 604 } 605 ``` 606 607 ### Response 608 609 ```http 610 HTTP/1.1 200 OK 611 Content-Type: application/vnd.api+json 612 ``` 613 614 ```json 615 { 616 "data": { 617 "type": "io.cozy.jobs", 618 "id": "022368c07dc701396403543d7eb8149c", 619 "attributes": { 620 "domain": "me.cozy.localhost", 621 "worker": "sendmail", 622 "options": {}, 623 "state": "errored", 624 "error": "LOGIN_FAILED", 625 "queued_at": "2021-04-12T12:34:56Z", 626 "started_at": "2021-04-12T12:34:56Z", 627 "finished_at": "2021-04-12T12:38:59Z" 628 }, 629 "links": { 630 "self": "/jobs/022368c07dc701396403543d7eb8149c" 631 } 632 } 633 } 634 ``` 635 636 ### POST /jobs/triggers 637 638 Add a trigger of the worker. See [triggers' descriptions](#triggers) to see the 639 types of trigger and their arguments syntax. 640 641 This route requires a specific permission on the worker type. A global 642 permission on the global `io.cozy.triggers` doctype is not allowed. 643 644 The `debounce` parameter can be used to limit the number of jobs created in a 645 burst. It delays the creation of the job on the first input by the given time 646 argument, and if the trigger has its condition matched again during this period, 647 it won't create another job. It can be useful to combine it with the changes 648 feed of couchdb with a last sequence number persisted by the worker, as it 649 allows to have a nice diff between two executions of the worker. Its syntax is the 650 one understood by go's [time.ParseDuration](https://golang.org/pkg/time/#ParseDuration). 651 652 #### Request 653 654 ```http 655 POST /jobs/triggers HTTP/1.1 656 Accept: application/vnd.api+json 657 ``` 658 659 ```json 660 { 661 "data": { 662 "attributes": { 663 "type": "@event", 664 "arguments": "io.cozy.invitations", 665 "debounce": "10m", 666 "worker": "sendmail", 667 "message": {}, 668 "options": { 669 "timeout": 60, 670 "max_exec_count": 3 671 } 672 } 673 } 674 } 675 ``` 676 677 **Note**: the `message` field was previously called `worker_arguments`. The 678 latter version still works but is deprecated, you should use `message` instead. 679 680 #### Response 681 682 ```json 683 { 684 "data": { 685 "type": "io.cozy.triggers", 686 "id": "123123", 687 "attributes": { 688 "type": "@every", 689 "arguments": "30m10s", 690 "debounce": "10m", 691 "worker": "sendmail", 692 "options": { 693 "timeout": 60, 694 "max_exec_count": 3 695 } 696 }, 697 "links": { 698 "self": "/jobs/triggers/123123" 699 } 700 } 701 } 702 ``` 703 704 #### Permissions 705 706 To use this endpoint, an application needs a permission on the type 707 `io.cozy.triggers` for the verb `POST`. In most cases, the application will 708 restrict its permission to only one worker, like this: 709 710 ```json 711 { 712 "permissions": { 713 "mail-from-the-user": { 714 "description": "Required to send regularly mails from the user to his/her friends", 715 "type": "io.cozy.triggers", 716 "verbs": ["POST"], 717 "selector": "worker", 718 "values": ["sendmail"] 719 } 720 } 721 } 722 ``` 723 724 ### GET /jobs/triggers/:trigger-id 725 726 Get a trigger informations given its ID. 727 728 #### Request 729 730 ```http 731 GET /jobs/triggers/123123 HTTP/1.1 732 Accept: application/vnd.api+json 733 ``` 734 735 #### Response 736 737 ```json 738 { 739 "data": { 740 "type": "io.cozy.triggers", 741 "id": "123123", 742 "attributes": { 743 "type": "@every", 744 "arguments": "30m10s", 745 "worker": "sendmail", 746 "options": { 747 "timeout": 60, 748 "max_exec_count": 3 749 }, 750 "current_state": { 751 "status": "done", 752 "last_success": "2017-11-20T13:31:09.01641731", 753 "last_successful_job_id": "abcde", 754 "last_execution": "2017-11-20T13:31:09.01641731", 755 "last_executed_job_id": "abcde", 756 "last_failure": "2017-11-20T13:31:09.01641731", 757 "last_failed_job_id": "abcde", 758 "last_error": "error value", 759 "last_manual_execution": "2017-11-20T13:31:09.01641731", 760 "last_manual_job_id": "abcde" 761 } 762 }, 763 "links": { 764 "self": "/jobs/triggers/123123" 765 } 766 } 767 } 768 ``` 769 770 #### Permissions 771 772 To use this endpoint, an application needs a permission on the type 773 `io.cozy.triggers` for the verb `GET`. A konnector can also call this endpoint 774 for one of its triggers (no permission required). 775 776 ### GET /jobs/triggers/:trigger-id/state 777 778 Get the trigger current state, to give a big picture of the health of the 779 trigger. 780 781 - last executed job status (`done`, `errored`, `queued` or `running`) 782 - last executed job that resulted in a successful executoin 783 - last executed job that resulted in an error 784 - last executed job from a manual execution (not executed by the trigger 785 directly) 786 787 #### Request 788 789 ``` 790 GET /jobs/triggers/123123/state HTTP/1.1 791 Accept: application/vnd.api+json 792 ``` 793 794 #### Response 795 796 ```json 797 { 798 "data": { 799 "type": "io.cozy.triggers.state", 800 "id": "123123", 801 "attributes": { 802 "status": "done", 803 "last_success": "2017-11-20T13:31:09.01641731", 804 "last_successful_job_id": "abcde", 805 "last_execution": "2017-11-20T13:31:09.01641731", 806 "last_executed_job_id": "abcde", 807 "last_failure": "2017-11-20T13:31:09.01641731", 808 "last_failed_job_id": "abcde", 809 "last_error": "error value", 810 "last_manual_execution": "2017-11-20T13:31:09.01641731", 811 "last_manual_job_id": "abcde" 812 } 813 } 814 } 815 ``` 816 817 #### Permissions 818 819 To use this endpoint, an application needs a permission on the type 820 `io.cozy.triggers` for the verb `GET`. A konnector can also call this endpoint 821 for one of its triggers (no permission required). 822 823 ### GET /jobs/triggers/:trigger-id/jobs 824 825 Get the jobs launched by the trigger with the specified ID. 826 827 Query parameters: 828 829 - `Limit`: to specify the number of jobs to get out 830 831 #### Request 832 833 ```http 834 GET /jobs/triggers/123123/jobs?Limit=1 HTTP/1.1 835 Accept: application/vnd.api+json 836 ``` 837 838 #### Response 839 840 ```json 841 { 842 "data": [ 843 { 844 "type": "io.cozy.jobs", 845 "id": "123123", 846 "attributes": {}, 847 "links": { 848 "self": "/jobs/123123" 849 } 850 } 851 ] 852 } 853 ``` 854 855 ### PATCH /jobs/triggers/:trigger-id 856 857 This route can be used to change the frequency of execution of a `@cron` 858 trigger, or the message of any trigger. 859 860 #### Request 861 862 ```http 863 PATCH /jobs/triggers/123123 HTTP/1.1 864 Content-Type: application/vnd.api+json 865 Accept: application/vnd.api+json 866 ``` 867 868 ```json 869 { 870 "data": { 871 "attributes": { 872 "type": "@cron", 873 "arguments": "0 0 0 * * 0" 874 } 875 } 876 } 877 ``` 878 879 #### Response 880 881 ```json 882 { 883 "data": { 884 "type": "io.cozy.triggers", 885 "id": "123123", 886 "attributes": { 887 "type": "@cron", 888 "arguments": "0 0 0 * * 0", 889 "worker": "sendmail", 890 "options": { 891 "timeout": 60, 892 "max_exec_count": 3 893 } 894 }, 895 "links": { 896 "self": "/jobs/triggers/123123" 897 } 898 } 899 } 900 ``` 901 902 #### Permissions 903 904 To use this endpoint, an application needs a permission on the type 905 `io.cozy.triggers` for the verb `PATCH`. A konnector can also call this 906 endpoint for one of its triggers (no permission required). 907 908 ### POST /jobs/triggers/:trigger-id/launch 909 910 Launch a trigger manually given its ID and return the created job. 911 912 **Note:** this endpoint can be used to create a job for a `@client` trigger. In 913 that case, the job won't be executed on the server but by the client. And the client 914 must call `PATCH /jobs/:job-id` when the job is completed (success or error). 915 916 #### Request 917 918 ```http 919 POST /jobs/triggers/123123/launch HTTP/1.1 920 Accept: application/vnd.api+json 921 ``` 922 923 #### Response 924 925 ```json 926 { 927 "data": { 928 "type": "io.cozy.jobs", 929 "id": "123123", 930 "attributes": { 931 "domain": "me.cozy.localhost", 932 "worker": "sendmail", 933 "options": {}, 934 "state": "running", 935 "queued_at": "2016-09-19T12:35:08Z", 936 "started_at": "2016-09-19T12:35:08Z", 937 "error": "" 938 }, 939 "links": { 940 "self": "/jobs/123123" 941 } 942 } 943 } 944 ``` 945 946 #### Permissions 947 948 To use this endpoint, an application needs a permission on the type 949 `io.cozy.triggers` for the verb `POST`. 950 951 ### DELETE /jobs/triggers/:trigger-id 952 953 Delete a trigger given its ID. 954 955 #### Request 956 957 ```http 958 DELETE /jobs/triggers/123123 HTTP/1.1 959 Accept: application/vnd.api+json 960 ``` 961 962 #### Permissions 963 964 To use this endpoint, an application needs a permission on the type 965 `io.cozy.triggers` for the verb `DELETE`. A konnector can also call this 966 endpoint for one of its triggers (no permission required). 967 968 ### GET /jobs/triggers 969 970 Get the list of triggers. This route only accept `Worker` and `Type` query 971 parameters and returns the trigger but also in `attributes` its `current_state` 972 (the same `current_state` returned by [GET 973 /jobs/triggers/:trigger-id](jobs/#get-jobstriggerstrigger-id)). Be warned that 974 `/data/io.cozy.triggers/_find` does not return this `current_state` attribute 975 and you'll need to query `/jobs/triggers/:trigger-id` to have it. 976 977 Query parameters (with comma-separated values): 978 979 - `Type`: to filter on the trigger type (`@cron`, `@in`, etc.) 980 - `Worker`: to filter only triggers associated with a specific worker. 981 982 #### Request 983 984 ```http 985 GET /jobs/triggers?Worker=konnector&Type=@cron,@in,@at HTTP/1.1 986 Accept: application/vnd.api+json 987 ``` 988 989 #### Response 990 991 ```json 992 { 993 "data": [ 994 { 995 "type": "io.cozy.triggers", 996 "id": "123123", 997 "arguments": "0 40 0 * * *", 998 "current_state": { 999 "last_error": "LOGIN_FAILED", 1000 "last_executed_job_id": "abc", 1001 "last_execution": "2019-01-07T08:23:22.069902888Z", 1002 "last_failed_job_id": "abcd", 1003 "last_failure": "2019-01-07T08:23:22.069902888Z", 1004 "last_manual_execution": "2019-01-07T08:23:22.069902888Z", 1005 "last_manual_job_id": "azer", 1006 "status": "errored", 1007 "trigger_id": "123123" 1008 }, 1009 "debounce": "", 1010 "domain": "xxx.mycozy.cloud", 1011 "message": { 1012 "konnector": "slug", 1013 "account": "XXX" 1014 }, 1015 "options": null, 1016 "type": "@cron", 1017 "worker": "konnector", 1018 "id": "123123", 1019 "links": { 1020 "self": "/jobs/triggers/123123" 1021 } 1022 } 1023 ] 1024 } 1025 ``` 1026 1027 #### Permissions 1028 1029 To use this endpoint, an application needs a permission on the type 1030 `io.cozy.triggers` for the verb `GET`. When used on a specific worker, the 1031 permission can be specified on the `worker` field. 1032 1033 ### POST /jobs/webhooks/:trigger-id 1034 1035 This endpoint is used for creating a job (for example executing a konnector 1036 or a service). 1037 It requires no permission, but a trigger of type `@webhook` must have been 1038 created before using this endpoint. Its body must be a JSON that will be 1039 available to the konnector or to the service through the 1040 `process.env['COZY_PAYLOAD']` variable. 1041 1042 It is possible to pass a `Manual=true` parameter in the query-string if the job 1043 is interactive. It will give it an higher priority in the queues. 1044 1045 #### Request 1046 1047 ```http 1048 POST /jobs/webhooks/f34c74d0-0c91-0139-5af5-543d7eb8149c HTTP/1.1 1049 Content-Type: application/json 1050 ``` 1051 1052 ```json 1053 { 1054 "account_id": "0672e560" 1055 } 1056 ``` 1057 1058 #### Response 1059 1060 ``` 1061 HTTP/1.1 204 No Content 1062 ``` 1063 1064 ### POST /jobs/webhooks/bi 1065 1066 This endpoint is used to create jobs for banking konnectors. It requires a 1067 payload with the format defined by [Budget 1068 Insight](https://docs.budget-insight.com/guides/webhooks) and an 1069 `Authorization` header with a Bearer token, where a trigger and an account can 1070 be found on this instance matching their data. The event type and the URL of 1071 the BI API (on the good environment) are also sent in the query string. 1072 1073 #### Request 1074 1075 ```http 1076 POST /jobs/webhooks/bi?event=CONNECTION_SYNCED&bi_url=https://.../ HTTP/1.1 1077 Content-Type: application/json 1078 Authorization: Bearer token-from-bi 1079 Host: cozy.example.com 1080 ``` 1081 1082 #### Response 1083 1084 ```http 1085 HTTP/1.1 204 No Content 1086 ``` 1087 1088 ### DELETE /jobs/purge 1089 1090 This endpoint allows to purge old jobs of an instance. 1091 Some parameters can be given to this route: 1092 1093 * `duration` is the duration of jobs to keep. This is a human-readable string 1094 (integer+suffix). 1095 For example: 1096 - "3W" will keep jobs up to 3 weeks 1097 - "2M" will keep jobs up to 2 months 1098 * `workers` is a comma-separated list of workers to apply the purge job. 1099 1100 #### Request 1101 1102 ```http 1103 DELETE /jobs/purge HTTP/1.1 1104 Accept: application/json 1105 ``` 1106 1107 #### Response 1108 1109 ```json 1110 { 1111 "deleted": 42 1112 } 1113 ``` 1114 #### Permissions 1115 1116 To use this endpoint, an application needs a permission on the type 1117 `io.cozy.jobs` for the verb `DELETE`. 1118 1119 ## Worker pool 1120 1121 The consuming side of the job queue is handled by a worker pool. 1122 1123 On a monolithic cozy-stack, the worker pool has a configurable fixed size of 1124 workers. The default value is not yet determined. Each time a worker has 1125 finished a job, it check the queue and based on the priority and the queued date 1126 of the job, picks a new job to execute. 1127 1128 ## Permissions 1129 1130 In order to prevent jobs from leaking informations between applications, we may 1131 need to add filtering per applications: for instance one queue per applications. 1132 1133 We should also have an explicit check on the permissions of the applications 1134 before launching a job scheduled by an application. For more information, refer 1135 to our [permission document](../permissions). 1136 1137 ## Multi-stack 1138 1139 When some instances are served by several stacks, the scheduling and running of 1140 jobs can be distributed on the stacks. The synchronization is done via redis. 1141 1142 For scheduling, there is one important key in redis: `triggers`. It's a sorted 1143 set. The members are the identifiants of the triggers (with the domain name), 1144 and the score are the timestamp of the next time the trigger should occur. 1145 During the short period of time where a trigger is processed, its key is moved 1146 to `scheduling` (another sorted set). So, even if a stack crash during 1147 processing a trigger, this trigger won't be lost. 1148 1149 For `@event` triggers, we don't use the same mechanism. Each stack has all the 1150 triggers in memory and is responsible to trigger them for the events generated 1151 by the HTTP requests of their API. They also publish them on redis: this pub/sub 1152 is used for the realtime API.