github.com/in4it/ecs-deploy@v0.0.42-0.20240508120354-ed77ff16df25/README.md (about) 1 # ECS deploy 2 ECS Deploy is a REST API server written in Go that can be used to deploy services on ECS from anywhere. It typically is executed as part of your deployment pipeline. Continuous Integration software (like Jenkins, CircleCI, Bitbucket or others) often don't have proper integration with ECS. This API server can be deployed on ECS and will be used to provide continuous deployment on ECS. 3 4 * Registers services in DynamoDB 5 * Creates ECR repository 6 * Creates necessary IAM roles 7 * Creates ALB target and listener rules 8 * Creates and updates ECS Services based on json/yaml input 9 * SAML supported Web UI to redeploy/rollback versions, add/update/delete parameters, examine event/container logs, scale, and run manual tasks 10 * Support to scale out and scale in ECS Container Instances 11 12 ## The UI 13 14 <p align="center"> 15 <a href="https://d3jb1lt6v0nddd.cloudfront.net/ecs-deploy/ecs-deploy-ui.gif"> 16 <img src="https://d3jb1lt6v0nddd.cloudfront.net/ecs-deploy/ecs-deploy-ui.gif" /> 17 </a> 18 </p> 19 20 ## Usage 21 22 ### Download 23 24 You can download ecs-deploy and ecs-client from the [releases page](https://github.com/in4it/ecs-deploy/releases) or you can use the [image from dockerhub](https://hub.docker.com/r/in4it/ecs-deploy/). 25 26 ### Bootstrap ECS cluster 27 28 You can bootstrap a new ECS cluster using ecs-deploy. It'll setup a autoscaling group, ALB, IAM roles, and the ECS cluster. 29 30 ``` 31 ./ecs-deploy --bootstrap \ 32 --alb-security-groups sg-123456 \ 33 --cloudwatch-logs-enabled \ 34 --cloudwatch-logs-prefix mycompany \ 35 --cluster-name mycluster \ 36 --ecs-desired-size 1 \ 37 --ecs-max-size 1 \ 38 --ecs-min-size 1 \ 39 --ecs-security-groups sg-123456 \ 40 --ecs-subnets subnet-123456 \ 41 --environment staging \ 42 --instance-type t2.micro \ 43 --key-name mykey \ 44 --loadbalancer-domain cluster.in4it.io \ 45 --paramstore-enabled \ 46 --paramstore-kms-arn aws:arn:kms:region:accountid:key/1234 \ 47 --paramstore-prefix mycompany \ 48 --profile your-aws-profile \ 49 --region your-aws-region 50 ``` 51 52 You'll need to setup the security groups and VPC/subnets first. The ALB security group should allow port 80 and 443 incoming, the ECS security group should allow 32768:61000 from the ALB. 53 54 If you no longer need the cluster, you can remove it by specifying --delete-cluster instead of --bootstrap 55 56 ### Bootstrap with terraform 57 Alternatively you can use terraform to deploy the ecs cluster. See [terraform/README.md](https://github.com/in4it/ecs-deploy/blob/master/terraform/README.md) for a terraform module that spins up an ecs cluster. 58 59 ### Deploy to ECS Cluster 60 61 To deploy the examples (an nginx server and a echoserver), use ecs-client: 62 63 Login interactively: 64 ``` 65 ./ecs-client login --url http://yourdomain/ecs-cluster 66 ``` 67 68 Login with environment variables: 69 ``` 70 ECS_DEPLOY_LOGIN=deploy ECS_DEPLOY_PASSWORD=password ./ecs-client login --url http://yourdomain/ecs-cluster 71 ``` 72 73 Deploy: 74 ``` 75 ./ecs-client deploy -f examples/services/multiple-services/multiple-services.yaml 76 ``` 77 78 79 ## Configuration (Environment variables) 80 81 ### AWS Specific variables: 82 83 * AWS\_REGION=region # mandatory 84 85 ### Authentication variables; 86 * JWT\_SECRET=secret # mandatory 87 * DEPLOY\_PASSWORD=deploy # mandatory 88 * DEVELOPER\_PASSWORD=developer # mandatory 89 90 ### Service specific variables 91 These will be used when deploying services 92 93 * AWS\_ACCOUNT\_ENV=dev|staging|testing|qa|prod 94 * PARAMSTORE\_ENABLED=yes 95 * PARAMSTORE\_PREFIX=mycompany 96 * PARAMSTORE\_KMS\_ARN= 97 * CLOUDWATCH\_LOGS\_ENABLED=yes 98 * CLOUDWATCH\_LOGS\_PREFIX=mycompany 99 * LOADBALANCER\_DOMAIN=mycompany.com 100 101 ### DynamoDB specific variables 102 * DYNAMODB\_TABLE=Services 103 104 ### ECR 105 106 * ECR\_SCAN\_ON\_PUSH=true 107 108 ### SAML 109 110 SAML can be enabled using the following environment variables 111 * SAML\_ENABLED=yes 112 * SAML\_ACS\_URL=https://mycompany.com/url-prefix 113 * SAML\_CERTIFICATE=contents of your certificate 114 * SAML\_PRIVATE\_KEY=contents of your private key 115 * SAML\_METADATA\_URL=https://identity-provider/metadata.xml 116 117 To create a new key and certificate, the following openssl command can be used: 118 ``` 119 openssl req -x509 -newkey rsa:2048 -keyout myservice.key -out myservice.cert -days 3650 -nodes -subj "/CN=myservice.mycompany.com" 120 ``` 121 122 # Web UI 123 124 * PARAMSTORE\_ASSUME\_ROLE=arn # arn to assume when querying the parameter store 125 126 # Autoscaling (down and up) 127 128 ## Setup 129 130 * Create an SNS topic, add https subscriber with URL https://your-domain.com/ecs-deploy/webhook 131 * Create a [CloudWatch Event for ECS tasks/services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch_event_stream.html) 132 * Create an [EC2 Auto Scaling Lifecycle hook](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html), and a CloudWatch event to capture the Lifecycle hook 133 * Let the SNS topic be the trigger for the CloudWatch events 134 135 ## Usage 136 137 * Autoscaling (up) will be triggered when the largest container (in respect to mem/cpu) cannot be scheduled on the cluster 138 * Autoscaling (down) will be triggered when there is enough capacity available on the cluster to remove an instance (instance size + largest container + buffer) 139 140 ## Configuration 141 142 The defaults are set for the most common use cases, but can be changed by setting environment variables: 143 144 | Environment variable | Default value | Description | 145 | --------------------- | ------------- | ----------- | 146 | PARAMSTORE\_ENABLED | no | Use "yes" to enable the parameter store. | 147 | PARAMSTORE\_PREFIX | "" | Prefix to use for the parameter store. mycompany will result in /mycompany/servicename/variable | 148 | PARAMSTORE\_KMS\_ARN | "" | Specify a KMS ARN to encrypt/decrypt variables | 149 | PARAMSTORE\_INJECT | no | Use "Yes" to enable injection of secrets into the task definition | 150 | AUTOSCALING\_STRATEGIES | LargestContainerUp,LargestContainerDown | List of autoscaling strategies to apply. See below for different types | 151 | AUTOSCALING\_DOWN\_STRATEGY | gracefully | Only gracefully supported now (uses interval and period before executing the scaling down operation) | 152 | AUTOSCALING\_UP\_STRATEGY | immediately | Scale up strategy (immediatey, gracefully) | 153 | AUTOSCALING\_DOWN\_COOLDOWN | 5 | Cooldown period after scaling down | 154 | AUTOSCALING\_DOWN\_INTERVAL | 60 | Seconds between intervals to check resource usage before scaling, after a scaling down operation is detected | 155 | AUTOSCALING\_DOWN\_PERIOD | 5 | Periods to check before scaling | 156 | AUTOSCALING\_UP\_COOLDOWN | 5 | Cooldown period after scaling up | 157 | AUTOSCALING\_UP\_INTERVAL | 60 | Seconds between intervals to check resource usage before scaling, after a scaling up operation is detected | 158 | AUTOSCALING\_UP\_PERIOD | 5 | Periods to check before scaling | 159 | SERVICE\_DISCOVERY\_TTL | 60 | TTL for service discovery records | 160 | SERVICE_DISCOVERY_FAILURETHRESHOLD | 3 | Failure threshold for service discovery records | 161 | AWS\_RESOURCE\_CREATION\_ENABLED | yes | Let ecs-deploy create AWS IAM resources for you | 162 | SLACK\_WEBHOOKS | "" | Comma seperated Slack webhooks, optionally with a channel (format: url1:#channel,url2:#channel) | 163 | SLACK\_USERNAME | ecs-deploy | Slack username | 164 | ECS\_TASK\_ROLE\_PERMISSION\_BOUNDARY\_ARN | "" | permission boundary for ecs task roles | 165 | ECR\_SCAN\_ON\_PUSH | false | Enable ECR image scanning | 166 | DEPLOY_MAX_WAIT_SECONDS | 900 | wait 15 minutes for a deployment to complete | 167 168 ### Autoscaling Strategies 169 170 | Strategy | Description | 171 | ---------------| ----------- | 172 | LargestContainerUp | Scale when the largest container (+buffer) in the cluster cannot be scheduled anymore on a node | 173 | LargestContainerDown | Scale down when there is enough capacity to schedule the largest container (buffer) after a node is removed | 174 | Polling | Poll all services every minute to check if a task can't be scheduled due to resource constraints (10 services per api call, only 1 call per second) |