github.com/webonyx/up@v0.7.4-0.20180808230834-91b94e551323/docs/07-guides.md (about) 1 --- 2 title: Guides 3 --- 4 5 ## Subscribing to Up Pro 6 7 Up Pro provides additional features which are not available in the open-source version, such as encrypted environment variables, alerting support and more. 8 9 First sign into the platform with the following command – you'll receive an email for confirmation. 10 11 ``` 12 $ up team login 13 14 email: tj@apex.sh 15 ⠋ verify: Check your email for a confirmation link 16 17 ``` 18 19 Click the link in your email and you're signed in! If you're using Up Pro with one or more organizations, you should create a team to manage team members and subscriptions independently. If you plan on using Up Pro for personal use you may skip this step. 20 21 ``` 22 $ up team add "My Company" 23 ``` 24 25 Next you'll need to subscribe! You'll be asked for an optional coupon, credit card information – which never touches our servers, only Stripe via HTTPS – and finally a subscription confirmation. 26 27 ``` 28 $ up team subscribe 29 ``` 30 31 Now you and your team members may upgrade to the latest version of Up Pro, instead of the open-source distribution: 32 33 ``` 34 $ up upgrade 35 ``` 36 37 To view the status of your account at any time run the following: 38 39 ``` 40 $ up team 41 42 team: apex 43 subscription: Up Pro 44 amount: $10.00/mo USD 45 created: December 22, 2017 46 ``` 47 48 To switch to another team run the following and select the active team. 49 50 ``` 51 $ up team switch 52 53 54 ❯ apex 55 tj@apex.sh 56 ``` 57 58 At any time you can ensure you're on Up Pro `up version`: 59 60 ``` 61 $ up version 62 0.1.1-pro 63 ``` 64 65 Note that AWS charges for your resource usage, and is not associated with Up Pro's subscription. Most small to medium applications will fit within AWS' free tier, however, you should consult AWS' pricing documentation for details. The [Serverless Calc](http://serverlesscalc.com/) is a useful tool for estimating the API Gateway and Lambda charges. 66 67 ## Inviting Team Members 68 69 To invite members use the following command: 70 71 ``` 72 $ up team members add tobi@apex.sh 73 $ up team members add loki@apex.sh 74 $ up team members add jane@apex.sh 75 ``` 76 77 At any time you can view invites and members: 78 79 ``` 80 $ up team members 81 82 team: apex 83 84 Members 85 86 • tj@apex.sh 87 • tobi@apex.sh 88 • loki@apex.sh 89 90 Invites 91 92 • jane@apex.sh 93 ``` 94 95 Your team members will receive an email with installation instructions, where they run the following to sign in – with your team id of course. 96 97 ``` 98 $ up team login --email tobi@apex.sh --team apex 99 ``` 100 101 ## Development to Production Workflow 102 103 This section guides you through taking a small application from development, to production, complete with purchasing and mapping a custom domain. 104 105 ### Deploying 106 107 First, create `app.js` in an empty directory with the following Node.js app. Note that it must listen on __PORT__ which is passed by Up. 108 109 ```js 110 const http = require('http') 111 const { PORT = 3000 } = process.env 112 113 http.createServer((req, res) => { 114 res.end('Hello World\n') 115 }).listen(PORT) 116 ``` 117 118 Next, you should give your application a name and start configuring. The `profile` name should correspond to the name in `~/.aws/credentials` so that Up knows which AWS account to deploy to, and which credentials to use. 119 120 ```json 121 { 122 "name": "up-example", 123 "profile": "up-tobi" 124 } 125 ``` 126 127 Run `up` to deploy the application. 128 129 ``` 130 $ up 131 132 build: 5 files, 3.9 MB (358ms) 133 deploy: complete (14.376s) 134 stack: complete (1m12.086s) 135 ``` 136 137 Test with `curl` to ensure everything is working: 138 139 ``` 140 $ curl `up url` 141 Hello World 142 ``` 143 144 ### Purchasing a Domain 145 146 Domains can be mapped from existing services, or purchased directly from AWS via Route53. First check if the domain you'd like is available: 147 148 ``` 149 $ up domains check up.com 150 151 Domain up.com is unavailable 152 153 Suggestions: 154 155 theupwards.com $12.00 USD 156 upwardonline.com $12.00 USD 157 myupwards.com $12.00 USD 158 theastir.com $12.00 USD 159 astironline.com $12.00 USD 160 myastir.com $12.00 USD 161 myupward.net $11.00 USD 162 cleanup.tv $32.00 USD 163 myup.tv $32.00 USD 164 itup.tv $32.00 USD 165 newup.tv $32.00 USD 166 thedown.net $11.00 USD 167 theupward.net $11.00 USD 168 upwardsonline.net $11.00 USD 169 ``` 170 171 Oh no up.com is taken! Try another: 172 173 ``` 174 $ up domains check up-example.com 175 176 Domain up-example.com is available for $12.00 USD 177 ``` 178 179 Purchase it with the following command and fill out the details required by the registrar: 180 181 ``` 182 $ up domains buy 183 184 Domain: up-example.com 185 First name: TJ 186 Last name: Holowaychuk 187 Email: tj@apex.sh 188 Phone: +1.2501007000 189 Country code: CA 190 City: Victoria 191 State or province: BC 192 Zip code: X9X 9X9 193 Address: Some address here 194 ``` 195 196 It can take a few minutes for AWS to finalize the purchase after which you should receive an email. Then you'll see it in the `up domains` output along with the automatic renewal time. 197 198 ``` 199 $ up domains 200 201 gh-polls.com renews Aug 28 17:17:58 202 up-example.com renews Sep 19 19:40:50 203 ``` 204 205 By default domains purchased with Up have privacy protection enabled, hiding your contact information from [WHOIS](https://en.wikipedia.org/wiki/WHOIS). 206 207 ### Deploying to Stages 208 209 Before deploying to the staging and production stages, first tweak the application a little to include the `UP_STAGE` environment variable: 210 211 ```js 212 const http = require('http') 213 const { PORT = 3000, UP_STAGE } = process.env 214 215 http.createServer((req, res) => { 216 res.end('Hello World from ' + UP_STAGE) 217 }).listen(PORT) 218 ``` 219 220 Now deploy to staging and production. Note that `up` is an alias of `up deploy staging`. 221 222 ``` 223 $ up 224 $ up deploy production 225 ``` 226 227 Open both in the browser: 228 229 ``` 230 $ up url -o 231 $ up url -s production -o 232 ``` 233 234 You should see "Hello World from production" and "Hello World from staging". 235 236 ### Mapping Custom Domains to Stages 237 238 Now that you have an application deployed, you probably want a fancy custom domain for it right? You can map these using the `stages` and `domain` properties. 239 240 Here we let Up know that we want `up-example.com` for production and `dev.up-example` for staging. 241 242 ```json 243 { 244 "name": "up-example", 245 "profile": "up-tobi", 246 "stages": { 247 "staging": { 248 "domain": "dev.up-example.com" 249 }, 250 "production": { 251 "domain": "up-example.com" 252 } 253 } 254 } 255 ``` 256 257 Note that you could map staging to a domain like `staging-myapp.com`, it does not have to be a sub-domain of your production domain. 258 259 Now when you run `up stack plan` to preview changes to your resources, it will prompt you to verify the Let's Encrypt certificate emails that AWS sends. 260 261 ``` 262 $ up stack plan 263 264 domains: Check your email for certificate approval 265 ⠧ confirm: up-example.com 266 ``` 267 268 AWS requires email verification to prove you own the domain. AWS sends an email to the 3 contact addresses listed in WHOIS when you registered the domain, and to the following 5 common system addresses for your domain: 269 270 - administrator@your_domain_name 271 - hostmaster@your_domain_name 272 - postmaster@your_domain_name 273 - webmaster@your_domain_name 274 - admin@your_domain_name 275 276 See [Validate Domain Ownership](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-validate-email.html) for more information. 277 278 After clicking "I Approve" in one of the emails, the output will resume and you'll see some new resources Up will be creating. 279 280 ``` 281 Add AWS::ApiGateway::DomainName 282 id: ApiDomainDevelopment 283 284 Add AWS::ApiGateway::BasePathMapping 285 id: ApiDomainDevelopmentPathMapping 286 287 Add AWS::ApiGateway::DomainName 288 id: ApiDomainProduction 289 290 Add AWS::ApiGateway::BasePathMapping 291 id: ApiDomainProductionPathMapping 292 293 Add AWS::Route53::RecordSet 294 id: DnsZoneDevUpExampleComRecordDevUpExampleCom 295 296 Add AWS::Route53::RecordSet 297 id: DnsZoneUpExampleComRecordUpExampleCom 298 ``` 299 300 If you're curious, now that Up knows you want to map the domain(s), it will create: 301 302 - Registers ACM free SSL certificate(s) for your domain(s) 303 - CloudFront distribution for the API Gateway 304 - API Gateway stage mapping 305 - Route53 DNS zone and record(s) mapping to the CloudFront distribution 306 307 Now apply these changes: 308 309 ``` 310 $ up stack apply 311 ``` 312 313 After the changes have been applied, it can take roughly 10-40 minutes for CloudFront to distribute the configuration and SSL certificate globally, so until then our up-example.com domain won't work. 314 315 Once available https://up-example.com will always point to production via `up deploy production`, and https://dev.up-example.com/ will point to the latest deployment via `up`. 316 317 ### Mapping Domains from External Registrars 318 319 If you purchased a domain via `up domains buy` you can skip this step, however if you used an external registrar such as Godaddy you will need to delegate to AWS for DNS management. 320 321 To do this you'll need to sign in to your registrar's site, and configure the nameservers. To figure out what values to use for the nameservers, run `up stack`, which outputs the NS records for the apex (top-level) domains of your application. 322 323 ``` 324 $ up stack 325 326 Staging 327 328 domain: stage.up-example.com 329 endpoint: d2od0udp1p8bru.cloudfront.net 330 331 Production 332 333 domain: up-example.com 334 endpoint: d72wsqljqg5cy.cloudfront.net 335 nameservers: 336 • ns-1495.awsdns-58.org 337 • ns-103.awsdns-12.com 338 • ns-1670.awsdns-16.co.uk 339 • ns-659.awsdns-18.net 340 ``` 341 342 Save those four values in your registrar's interface, and you should be good to go! Note that altering DNS records can take some time to propagate. 343 344 ### Mapping with Third-party DNS 345 346 If you manage DNS with a third-party such as Cloudflare, and wish to use Up only for deployment you will need to manually edit or add DNS records. 347 348 For example if your top-level domain `sloths.com` is managed by Cloudflare and you'd like point `api.sloths.com` to your app, you should first add it to your `up.json`: 349 350 ```json 351 { 352 "name": "sloths" 353 "stages": { 354 "production": { 355 "domain": "api.sloths.com" 356 } 357 } 358 } 359 ``` 360 361 Next you will need to `up stack plan` and `up stack apply`, this will set up a CloudFront end-point for the application. To view the endpoint information, run `up stack`: 362 363 ``` 364 $ up stack 365 366 Production 367 368 domain: api.sloths.com 369 endpoint: d72wsqljqg5cy.cloudfront.net 370 ``` 371 372 In your DNS provider – Cloudflare in this example – you should create a `CNAME` record pointing to the production `endpoint`. Make sure that the `domain` you use matches the domain in Cloudflare. 373 374 ### Stack Changes 375 376 The "stack" is all of the resources associated with your app. You plan changes via `up stack plan` and perform them with `up stack apply`. 377 378 Suppose you wanted to map the "staging" stage, you would first add it to `up.json`: 379 380 ```json 381 { 382 "name": "up-example", 383 "profile": "up-tobi", 384 "stages": { 385 "staging": { 386 "domain": "stage.up-example.com" 387 }, 388 "production": { 389 "domain": "up-example.com" 390 } 391 } 392 } 393 ``` 394 395 Then run: 396 397 ``` 398 $ up stack plan 399 ``` 400 401 Review the output, it should be all "Add"s in this case, then apply: 402 403 ``` 404 $ up stack apply 405 ``` 406 407 ### Deleting the App 408 409 After you're done messing around, you may want to remove all the resources and the app itself. To do so simply run: 410 411 ``` 412 $ up stack delete 413 ``` 414 415 ## Deploying Applications from Continuous Integration 416 417 Up makes it easy to deploy your applications from CI, thanks to its Go binaries you can install Up in seconds in any CI provider such as Travis, Circle, Semaphore among others. 418 419 ### Environment Variables 420 421 The first step is to set up environment variables so that you have access to your AWS account. You can get these values from `cat ~/.aws/credentials`: 422 423 - `AWS_ACCESS_KEY_ID` – AWS access key 424 - `AWS_SECRET_ACCESS_KEY` – AWS secret key 425 426 If using running Up Pro you'll need your Up credentials in order to access Up Pro via the `up upgrade` command. To obtain this run `up team ci` or `up team ci --copy` to copy it directly to your clipboard, then paste this as the env var's value. 427 428 - `UP_CONFIG` – Up configuration as base64-encoded JSON 429 430 If you run into "403 Forbidden" errors this is due to GitHub's low rate limit for unauthenticated users, consider creating a [Personal Access Token](https://github.com/settings/tokens) and adding the following variable to your CI: 431 432 - `GITHUB_TOKEN` — Github personal access token 433 434 ### Commands 435 436 You may install Up in the current working directory, and deploy to production with the following commands, omitting the `up upgrade` if you are not an Up Pro subscriber. 437 438 ``` 439 $ curl -sf https://up.apex.sh/install | BINDIR=. sh 440 $ ./up upgrade 441 $ ./up production 442 ``` 443 444 Or if you prefer installing globally within `PATH`: 445 446 ``` 447 $ sudo chown -R $(whoami) /usr/local/bin 448 $ curl -sf https://up.apex.sh/install | sh 449 $ up upgrade 450 $ up production 451 ``` 452 453 ## Mastering Logging 454 455 This section describes how you can log from your application in a way that Up will recognize. In the future Up will support forwarding your logs to services such as Loggly, Papertrail or ELK. 456 457 ### Plain Text 458 459 The first option is plain-text logs to stdout or stderr. Currently writes to stderr are considered ERROR-level logs, and stdout becomes INFO. 460 461 Writing plain-text logs is simple, for example with Node.js: 462 463 ```js 464 console.log('User signed in') 465 console.error('Failed to sign in: %s', err) 466 ``` 467 468 Would be collected as: 469 470 ``` 471 INFO: User signed in 472 ERROR: Failed to sign in: something broke 473 ``` 474 475 ### JSON 476 477 The second option is structured logging with JSON events, which is preferred as it allows you to query against specific fields and treat logs like events. 478 479 JSON logs require a `level` and `message` field: 480 481 ```js 482 console.log(`{ "level": "info", "message": "User login" }`) 483 ``` 484 485 Would be collected as: 486 487 ``` 488 INFO: User login 489 ``` 490 491 The `message` field should typically contain no dynamic content, such as user names or emails, these can be provided as fields: 492 493 ```js 494 console.log(`{ "level": "info", "message": "User login", "fields": { "name": "Tobi", "email": "tobi@apex.sh" } }`) 495 ``` 496 497 Would be collected as: 498 499 ``` 500 INFO: User login name=Tobi email=tobi@apex.sh 501 ``` 502 503 Allowing you to perform queries such as: 504 505 ``` 506 $ up logs 'message = "User login" name = "Tobi"' 507 ``` 508 509 Or: 510 511 ``` 512 $ up logs 'name = "Tobi" or email = "tobi@*"' 513 ``` 514 515 Here's a simple JavaScript logger for reference. All you need to do is output some JSON to stdout and Up will handle the rest! 516 517 ```js 518 function log(level, message, fields = {}) { 519 const entry = { level, message, fields } 520 console.log(JSON.stringify(entry)) 521 } 522 ``` 523 524 For example, with the Go [apex/log](https://github.com/apex/log) package you'd use the `json` handler, which outputs this format. 525 526 ## Log Query Language 527 528 Up supports a comprehensive query language, allowing you to perform complex filters against structured data, supporting operators, equality, substring tests and so on. This section details the options available when querying. 529 530 ### AND Operator 531 532 The `and` operator is implied, and entirely optional to specify, since this is the common case. 533 534 Suppose you have the following example query to show only production errors from a the specified IP address. 535 536 ``` 537 production error ip = "207.194.32.30" 538 ``` 539 540 The parser will inject `and`, effectively compiling to: 541 542 ``` 543 production and error and ip = "207.194.38.50" 544 ``` 545 546 ### Or Operator 547 548 There is of course also an `or` operator, for example showing warnings or errors. 549 550 ``` 551 production (warn or error) 552 ``` 553 554 These may of course be nested as you require: 555 556 ``` 557 (production or staging) (warn or error) method = "GET" 558 ``` 559 560 ### Equality Operators 561 562 The `=` and `!=` equality operators allow you to filter on the contents of a field. 563 564 Here `=` is used to show only GET requests: 565 566 ``` 567 method = "GET" 568 ``` 569 570 Or for example `!=` may be used to show anything except GET: 571 572 ``` 573 method != "GET" 574 ``` 575 576 ### Relational Operators 577 578 The `>`, `>=`, `<`, and `<=` relational operators are useful for comparing numeric values, for example response status codes: 579 580 ``` 581 status >= 200 status < 300 582 ``` 583 584 ### Stages 585 586 Currently all development, staging, and production logs are all stored in the same location, however, you may filter to find exactly what you need. 587 588 The keywords `production`, `staging`, and `development` expand to: 589 590 ``` 591 stage = "production" 592 ``` 593 594 For example, filtering on slow production responses: 595 596 ``` 597 production duration >= 1s 598 ``` 599 600 Is the same as: 601 602 ``` 603 stage = "production" duration >= 1s 604 ``` 605 606 ### Severity Levels 607 608 Up provides request level logging with severity levels applied automatically. For example, a 5xx response is an ERROR level, while 4xx is a WARN, and 3xx or 2xx are the INFO level. 609 610 This means that instead of using the following for showing production errors: 611 612 ``` 613 production status >= 500 614 ``` 615 616 You may use: 617 618 ``` 619 production error 620 ``` 621 622 ### In Operator 623 624 The `in` operator checks for the presence of a field within the set provided. For example, showing only POST, PUT and PATCH requests: 625 626 ``` 627 method in ("POST", "PUT", "PATCH") 628 ``` 629 630 ### Units 631 632 The log grammar supports units for bytes and durations. For example, showing responses larger than 56kb: 633 634 ``` 635 size > 56kb 636 ``` 637 638 Or showing responses longer than 1500ms: 639 640 ``` 641 duration > 1.5s 642 ``` 643 644 Byte units are: 645 646 - `b` bytes (`123b` or `123` are equivalent) 647 - `kb` bytes (`5kb`, `128kb`) 648 - `mb` bytes (`5mb`, `15.5mb`) 649 650 Duration units are: 651 652 - `ms` milliseconds (`100ms` or `100` are equivalent) 653 - `s` seconds (`1.5s`, `5s`) 654 655 ### Substring Matches 656 657 When filtering on strings, such as the log message, you may use the `*` character for substring matches. 658 659 For example if you want to show logs with a remote ip prefix of `207.`: 660 661 ``` 662 ip = "207.*" 663 ``` 664 665 Or a message containing the word "login": 666 667 ``` 668 message = "*login*" 669 ``` 670 671 There is also a special keyword for this case: 672 673 ``` 674 message contains "login" 675 ``` 676 677 ## Hot Reloading in Development 678 679 The `up start` command uses your `proxy.command` by default, which may be inferred based on your application type, such as `node app.js` for Node.js or `./server` for Golang. 680 681 You may alter this command for `up start` with the development environment. For example with Golang you may want `go run main.go`, or hot reloading with [gin](https://github.com/codegangsta/gin) as shown here: 682 683 ```json 684 { 685 "name": "app", 686 "stages": { 687 "development": { 688 "proxy": { 689 "command": "gin --port $PORT" 690 } 691 } 692 } 693 } 694 ``` 695 696 Note that the server must always listen on `PORT` which is provided by `up start`. 697 698 ## Accessing Lambda Context 699 700 Traditional AWS Lambda functions a provided a context object which contains runtime information such as API Gateway user identity. This information is exposed as JSON in the `X-Context` header field in Up as shown here: 701 702 ```js 703 const http = require('http') 704 const { PORT } = process.env 705 706 const app = http.createServer((req, res) => { 707 const ctx = JSON.parse(req.headers['x-context']) 708 res.end(JSON.stringify(ctx, null, 2)) 709 }) 710 711 console.log('Starting app on %s', PORT) 712 app.listen(PORT) 713 ``` 714 715 Output will be similar to the following. Visit the [AWS Documentation](https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html) for details. 716 717 ```json 718 { 719 "apiId": "g4yn392afg", 720 "resourceId": "ez0z8areob", 721 "requestId": "d8314ef1-5543-11e8-a925-21fa0dd01c37", 722 "accountId": "337344593553", 723 "stage": "staging", 724 "identity": { 725 "apiKey": "", 726 "accountId": "", 727 "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36", 728 "sourceIp": "64.110.31.100", 729 "accessKey": "", 730 "caller": "", 731 "user": "", 732 "userARN": "", 733 "cognitoIdentityId": "", 734 "cognitoIdentityPoolId": "", 735 "cognitoAuthenticationType": "", 736 "cognitoAuthenticationProvider": "" 737 }, 738 "authorizer": null 739 } 740 ``` 741 742 ## Git Integration 743 744 Up automatically detects and utilizes Git commit information when you deploy, such as recording the Git commit SHA or tag, and author. This information can be used for: 745 746 - Viewing and filtering logs by version 747 - Viewing the Git SHA or tag in the deployment history 748 - Rolling back to a previous version via Git SHA or tag 749 750 ### Logging 751 752 When deploying from a Git repo the commit is also available in the logs for display and filtering, for example the tag `v1.8.2` in the following log output: 753 754 ``` 755 Mar 15th 11:32:54am INFO production v1.8.2 request: id=467991e8-287f-11e8-bd4e-0ba9f514a0f0 ip=207.194.38.50 method=GET path=/favicon.ico 756 Mar 15th 11:32:54am INFO production v1.8.2 response: duration=0s id=467991e8-287f-11e8-bd4e-0ba9f514a0f0 ip=207.194.38.50 method=GET path=/favicon.ico size=66 B status=200 757 ``` 758 759 Filtering can be performed with the `commit` field: 760 761 ``` 762 $ up logs 'commit = "v1.8.2"' 763 ``` 764 765 ### Deployment History 766 767 Up Pro allows you to view deployment history via `up deploys` which will display the Git SHA, tag, or Lambda version if you are not using Git for the project. 768 769 ``` 770 $ up deploys 771 772 Stage Version Author Date 773 774 production v1.8.0 TJ Holowaychuk 22 minutes ago 775 production v1.7.3 TJ Holowaychuk 23 minutes ago 776 staging f4f3143 TJ Holowaychuk 24 minutes ago 777 staging ca781f7 TJ Holowaychuk 24 minutes ago 778 production v1.7.2 TJ Holowaychuk 56 minutes ago 779 staging v1.7.2 TJ Holowaychuk 2 hours ago 780 production 7e62daf TJ Holowaychuk 2 hours ago 781 production v1.7.1 TJ Holowaychuk 2 hours ago 782 staging v1.7.1 TJ Holowaychuk 2 hours ago 783 staging 15c46ba TJ Holowaychuk 2 hours ago 784 staging 15c46ba TJ Holowaychuk 2 hours ago 785 staging 74f9a92 TJ Holowaychuk 2 hours ago 786 ``` 787 788 ### Rollbacks 789 790 As mentioned in the [rollback](#commands.rollback) command section, with Up Pro you may use the Git tag or SHA for reverting to an older release: 791 792 ``` 793 $ up rollback -s production v1.8.0 794 ```