Hello, I am Lee Ji-young, working as a Solutions Architect at Amazon. In this session, we will look into Amazon ECS Auto Scaling. The agenda for this session is as follows: we will examine methods for optimizing costs in Amazon ECS, how to use ECS service auto scaling, how to manage ECS capacity providers and cluster auto scaling, and finally, we will discuss considerations for adopting spot instances before concluding.

Amazon ECS Auto Scaling is one of the ways to effectively manage costs when operating an ECS cluster. So first, we will explore how to utilize other methods for cost optimization in the Amazon ECS environment. The first aspect to consider when optimizing computing resources in the ECS environment is to visualize costs, allowing us to identify and utilize only the necessary computing resources.

Using this, next, we can manage computing resources according to changes in workload requests through auto-scaling. We can also utilize purchasing options that allow for more cost-effective use of CPU and memory, as well as various hardware options. By applying different methods, we can optimize the costs of computing resources used in the ECS environment. The usage of ECS can be visualized through various means, such as the cost explorer provided by AWS or cost dashboards offered by partners, allowing us to identify ECS costs. Additionally, based on CloudWatch container metrics information, we can determine the appropriate capacity of services required for ECS workloads. The next aspect to consider is how to use the required capacity in a cost-effective manner. In other words, costs can be reduced depending on how we choose purchasing options and hardware. First, regarding hardware, AWS has designed its own AR-based processor.

We can consider introducing the Graviton processor. The processor is optimized for the cloud, designed to reduce computing costs. In the case of Graviton 2, it can achieve cost savings of up to 40% compared to x86, and Graviton 3 offers a performance improvement of 255% over Graviton 2. Because of its cost-effectiveness, it is advisable to check whether the applications running on your workloads are compatible with Graviton before considering its adoption. 

Secondly, we can look into purchasing options. The on-demand pricing for AWS resources is based on the resources used without any conditions. Compared to AWS on-demand, there is a savings plan available for one or three-year commitments, which allows for cost reductions. The savings plan purchase option requires that a certain amount is continuously used throughout the commitment period. This means that whether you use the resources or not, costs will still incur based on the amount of resources you have committed to.

It is advisable to purchase and apply a savings plan only for the necessary resources. Lastly, let's discuss the spot options. AWS offers discounted pricing for resources that are not being used, such as EC2 instances. However, if those resources are needed by AWS, they can be reclaimed from the user, and in the event of an interruption warning, the instance will be terminated within about two minutes. This means that even if there is a sudden interruption, there will be no issues when restarting, making it suitable for workloads that do not save their state temporarily. You can consider applying this option after confirming how your workload operates. By utilizing this savings plan and spot instances, you can optimize costs in relation to performance. Finally, let's focus on auto-scaling, which allows you to manage computing resources in the ECS environment to use only what you need. By configuring ECS auto-scaling in conjunction with purchase options, you can also optimize costs, so let's take a closer look at this aspect as well.

I will look into ECS auto-scaling, so let's briefly examine the configuration of Amazon ECS. When you use Amazon ECS to serve your workloads, you can logically separate your applications by creating an Amazon ECS cluster. By using task definitions, you can define tasks that can be executed on instances or gateways within the ECS cluster. When you create an ECS service to manage tasks, it can maintain the required number of tasks. Even if tasks fail or need to be restarted due to deployments, the service can maintain the specified number of tasks. Additionally, when tasks are added or terminated, the ELB configuration is automatically updated. If the user requests for the application being served remain consistent, you can operate sufficiently with the basic features provided by ECS. However, the number of user requests for the application may vary over time and depending on the situation.

Therefore, the capacity of the CPU memory may vary depending on how the application operates. In this case, Amazon ECS provides features that can automatically manage and optimize the computing capacity needed for applications. Amazon ECS offers a service called Auto Scaling. As seen in the ECS configuration, the ECS service can maintain the desired number of tasks. However, the desired number of tasks can change according to the user's request volume, and the ECS service can manage this using service Auto Scaling. ECS service Auto Scaling can increase or decrease the number of tasks managed by the service as needed. The required number of tasks can be set based on metrics from the service. According to the defined metric settings, the policies for Auto Scaling are reflected. Based on the Auto Scaling policies, the desired...

When the number of tasks changes, the ECS service can increase or decrease the number of tasks accordingly. To calculate the required number of tasks, we can look at the types of metrics that can be utilized by the service. The most fundamental metrics include the overall CPU and memory usage of the ECS service. Additionally, we can use the average response time of the ALB associated with the ECS service or the SQS queue load. Lastly, we can also utilize custom metrics such as the SQS backlog tasks. Understanding which metric information can most significantly impact the ECS service when processing workloads is crucial. Once we have identified the baseline metrics according to the characteristics of the workload, the next step is to determine how to increase the desired coefficient for the tasks. There are two main methods for increasing the number of tasks: target tracking and step scaling. First, let’s discuss the target tracking option.

I will examine it. For target tracking, you need to select which metric to use and specify the target value. It is relatively easy to configure and maintain. Next is step scaling settings. In the case of step scaling, you can specify a metric and a threshold, and set how it will operate. Depending on the operation, you can also specify the amount of resource change. This means that for an ECS service, you can customize and set how the resources should change based on the situation. Therefore, compared to target tracking, step scaling is more difficult to set up, and after applying it, continuous monitoring is necessary to ensure it operates as desired. Let's look at an example of step scaling settings. If the average exceeds three occurrences in one minute, the scaling action will be triggered. The first setting adds two tasks when the CPU exceeds 10%. The second setting occurs when the CPU exceeds 20%.

I have configured to add three tasks. Looking at how it operates, the average CPU usage is 40%, recorded three times every minute, which triggered an alarm. According to the previously set scaling policy, when the usage is between 10% and 20%, two tasks are added, and when it exceeds 20%, three tasks are increased. The alarm that was triggered indicates that the usage has exceeded 40% three times, so when it first reaches between 10% and 20%, no action is taken. Only when it increases to 20% does the task increase event occur. If events consistently occur above 20%, tasks can increase by three at a time. However, it will not exceed the pre-set maximum task value. In this way, step scaling allows for the configuration of metrics evaluation periods, scaling actions, etc. While it is more complex compared to target scaling, it has the advantage of specifying the number of tasks needed to operate ECS services stably and quickly, depending on the workload's behavior. Lastly, there is scheduled scaling.

ECS 서비스는 날짜와 시간을 기준으로 태스크의 실행 수를 조정할 수 있습니다. 특정 시간에 사용량 증가를 예측할 수 있다면 이 스케줄 스케일링 옵션을 이용해보는 것도 좋습니다. 특정 시간은 쿨론 설정과 동일하며, UTC 시간으로 설정하시면 됩니다. 다음은 스케줄 스케일링 AWS CLI 설정 예제입니다. 오전 8시에 최소 3개, 최대 8개로 확장을 구성하였고, 오후 6시에 최소 및 최대를 1개로 설정한 예입니다. 이후 스케줄 스케일링이 필요 없는 경우, Delete Schedule Action으로 삭제하시면 됩니다. 지금까지는 ECS 서비스 관점에서 태스크를 어떻게 관리할지에 대해 이야기 나눴습니다. ECS는 컴퓨팅 용량으로 EC2 인스턴스나 게이트를 선택하여 실행할 수 있습니다. EC2의 경우 사용자가 직접 서버를 설정하고 관리합니다. 다양한 인스턴스 유형, 특히 GPU 등을 선택할 수 있으며, 인프라를 직접 관리하는 만큼 비용이 저렴할 수 있습니다. 반면에 게이트는 서버를 관리할 필요가 없습니다.

It is a serverless computing engine that allows you to run containers without the need for provisioning or management. It automatically provisions and manages the CPU and memory required to execute tasks, thereby reducing server operational overhead. ECS provides the capability to automatically manage and allocate this computing capacity. 

First, let's look at the Amazon ECS Fargate capacity provider. The Fargate capacity provider operates with two options: Fargate and Fargate Spot. As discussed in the purchasing options earlier, Fargate offers on-demand options, savings plans, and discounted prices through the Spot option. If you wish to use Fargate Spot in your ECS cluster, you can configure the Fargate Spot capacity provider.

클러스터에 추가해야 합니다. 캐파시티 프로바이더를 생성한 후에 CS 태스크를 실행합니다. 캐파시티 프로바이더 설정에 따라서 스크는 각각의 캐파시티 프로바이더에 대치됩니다. 이렇게 두 개의 게이트 캐파시티 프로바이더가 등록된 경우, 스크는 어떤 캐파시티 프로바이더에 실행할지에 대해서 어떻게 구성할 수 있을까요? 패시티 프로바이더의 베이스와 웨이트 속성을 가지고 설정할 수 있습니다. 베이스의 설정은 다른 캐파시티 프로바이더의 태스크를 배정하기 전에 먼저 배정할 태스크의 개수입니다. 여러 캐파시티 프로바이더가 있더라도 베이스를 0 이상 설정할 수 있는 캐파시티 프로바이더는 한 개밖에 없습니다. 웨이트의 경우 여러 캐파시티 프로바이더가 있는 경우에 해당 캐파시티 프로바이더 간의 태스크를 배정할 비율을 지정할 수 있습니다. 예를 들어, 일곱 개의 태스크를...

Let's assume we are executing this. Looking at the capacity provider settings, the Fargate capacity provider has a base of 1, so we first allocate the task limits. Since the weights for the gate and Fargate spot capacity providers are set to 1 and 2, respectively, the remaining six tasks will be distributed as two tasks to the gate and four tasks to the spot capacity provider. Although we can use computing resources at a discounted price, if we want to apply spots that can be removed at any time while ensuring that the workload provides stable service, we need to estimate the minimum number of tasks required for the workload to operate and set that number as the base for the gate capacity provider. In this case, it is set to 2. The rest will be allocated as Fargate spots, resulting in a weight ratio of gate to Fargate spot of 0 to 1. The capacity provider is set to 0.

Because it is set up this way, tasks are not configured, and the Fargate spot is set to 1, allowing the remaining tasks to be allocated. Therefore, except for the two tasks allocated as base settings, all others will be allocated as Fargate spots. As a result, even if the tasks in the Fargate spot end due to an inter-warning event, at least two minimal tasks can operate at the gate, ensuring that there are no issues with the service running. Additionally, there is an advantage of being able to secure more rapidly processed resources using tasks operating in the Fargate spot during normal times. As you can see, when operating with the gate as a computing resource in the ECS environment, a sudden increase in the number of tasks can be initiated relatively quickly. However, in actual operational environments, the container image size may cause variations in the time required depending on the operational situation when using Fargate as a computing resource in the ECS environment.

In cases where the operation of the cluster is abruptly terminated, resources can be returned relatively quickly, preventing unnecessary costs. By using ECS Gate, when expanding or reducing computing capacity, administrators do not need to manage the infrastructure themselves. They can secure the necessary computing resources in a timely manner while reducing costs through Gate and Fargate Spot Capacity Providers. This allows administrators to manage resources flexibly, reducing the complexity of infrastructure management and enabling cost-effective operation of container-based applications. 

Next, we have the Amazon ECS EC2 Capacity Provider. Using computing resources with EC2 means that administrators are directly responsible for setting up and managing the servers. They can choose from various instance types, especially GPU options, and since they manage the infrastructure themselves, the costs may vary depending on how they operate.

비용이 더 발생할 수도 있습니다. 지금부터는 ECS EC2를 효과적으로 운영할 수 있도록 E2 용량 제공자에 제공해 주는 기능을 살펴보도록 하겠습니다. ECS EC2 용량 제공자의 동작 방식에 대해서 살펴보겠습니다. ECS 클러스터를 생성합니다. ECS 클러스터가 생성되고 E2 용량 관리를 위한 E2 오토스케일링 그룹이 생성됩니다. 용량 제공자를 기반으로 EC2를 관리하기 위해서 용량 제공자를 생성합니다. ECS 태스크를 실행시킵니다. ECS 용량 제공자는 몇 개의 EC2 인스턴스가 필요한지에 대해서 대기 중인 태스크의 개수를 기반으로 필요한 용량을 계산합니다. 이에 따라서 E2 오토스케일링의 원하는 인스턴스 수가 변경되고 필요한 수의 EC2 인스턴스가 동시에 시작됩니다. ECS 용량 제공자의 필요한 컴퓨팅 용량이 확보되면 대기 중인 태스크가 새로 시작된 EC2 인스턴스에 배치됩니다.

The task begins. The number of instances required for the E2 Auto Scaling group is determined by the capacity provider algorithm, taking various situations into account. However, let's assume that we will decide the capacity based solely on CPU and memory. One task is set to use 256 CPU and 512 memory. The capacity provider operates with the C5 large instance limits, and one task is currently running. In this case, if we increase the desired task count to 16, how many E2 instances need to be added? The C5 large has 2 vCPUs and 3 GiB of memory. When converted for use in an ECS environment, it can be seen to provide 2048 CPU and 960 memory. Since there is already one task running and we want to start 15 additional tasks, at least two E2 instances will be required. If we are only operating with the E2 Auto Scaling group, one E2 instance will start first, and then tasks will be placed.

Pending means that issues are initiated in sequence, which can delay the start of all tasks. On the other hand, when managing capacity through the EC2 capacity provider, the provider estimates the required capacity before starting the EC2 instances and sets the Auto Scaling desired capacity to 2, allowing for the simultaneous increase of the necessary two instances. In the next log, you can see that two instances are indeed increased simultaneously. The required number of instances for the EC2 capacity provider is monitored through the CloudWatch metrics, specifically the ECS managed scaling provider ratio. This ratio is calculated by dividing the number of instances currently in operation by the number of instances required by the Auto Scaling Group (ASG), and then multiplying by 100. If we denote the number of required ASG instances as 'm', this 'm' value is not determined by a single factor alone.

The configuration of the ASG, whether using instance types or mixed instance types, will depend on factors such as the resource requirements of pending tasks in the provisioning state and the batch strategy for the tasks. The next important value regarding scaling policy settings is the target capacity. For example, I set the target capacity value to 100. This means that I intend to use 100% of the ECS resources without any valid resources. The value of the capacity reservation can determine whether to scale out or scale in the instances based on the target capacity value set. If the capacity reservation value is greater than the target capacity, it means that the required number of instances is smaller than the number currently running, so we increase the instances. However, if the target capacity is larger, the required number of instances exceeds the current number, so we remove instances. One thing to be cautious about is when the target capacity is set below 100%.

The job batching strategy has the advantage of being easier to manage when scaling down or emptying factories; however, it also has the disadvantage that if a failure occurs in an instance where tasks are concentrated, it can significantly impact the entire business operation. Next, let's look at the results of a scale-out test for cluster auto-scaling to aid in understanding. I would like to first mention that the test results I will share may vary depending on the operational circumstances in the actual operating environment. 

Looking at the details, one task is configured with a CPU of 256 and memory of 512. The capacity provider is currently running one C5 large East instance and one task. We will examine how the value of the capacity provider varies when the desired task count is increased to 16. When increasing the desired task count to 16, approximately two...

The number of instances needs to be increased. Currently, there is 1 instance in operation, and since we need to add two more, the total number of instances required for the Auto Scaling Group (ASG) becomes 3. When we calculate this, the capacity provider ratio is 300%. After adding the two instances, the current operational instances will be three, and the required number of instances will also be three, maintaining the capacity provider ratio at 100. As the capacity provider ratio increases, we can see that the number of service instances increases while the capacity reservation number decreases. With the additional server computing resources, the pending task count decreases, and we can observe that the actual desired task count of 16 is running. Based on the test results, we will continue to monitor this. Generally, when using EC2 computing resources, if additional tasks can be placed within EC2, the execution of new tasks is relatively...

It can be fast, but if there are too many idle resources, unnecessary costs may arise for the E2 valid resources. If instances are needed, additional instances will be added at the point when it is determined that instances are necessary by the city provider's regulation. Although it may take time for all 15 tasks to be executed, it is advantageous in terms of costs since you do not need to maintain valid resources on a regular basis. Considering this trade-off, it is essential to set the target capacity value. Next, let's examine how the number of required instances decreases when the desired task count is suddenly reduced. If the desired task count is reduced to 1, the required computing resources for 22 will only use the 256 CPU memory needed for one task, resulting in a reduction of the required instances from three to one.

When multiplying by 100, the target capacity provider reservation will be set to 1 for the desired capacity of E2 auto-scaling, which is 33%. On the other hand, when scaling in, to prevent unnecessary computing resources from being added or removed repeatedly, the system will monitor for a certain period to confirm whether the instance is truly unnecessary before removing it. When a running instance is removed, the capacity provider will increase again to maintain the same level as the target capacity. In actual tests, observing the changes in the number of service instances compared to the capacity provider reservation shows a reduction from three to two, and then to one. At this point, it can be confirmed that the provider reservation is reset back to 100. The status information of the respective instances will change to a draining state. When an instance changes to draining, the instance...

The script that was running in the background is moved to another instance, and the running task count changes to zero. After that, this instance automatically shuts down. Stopping 15 tasks varies depending on how quickly the task's application terminates, but in the tested environment, it included a very simple call processing logic, so it took about 2 minutes and 24 seconds for the 15 tasks to stop.

Now, let's look at the considerations for scaling in cluster auto-scaling. Even in the case of 22 instances that are not running any tasks, if we look at the scaling-in alarm settings, it is configured to exclude the actual instance from the E2 auto-scaling group when 15 alarms are triggered. Next, during the scaling process of the E2 instance, when it is terminated, there may still be tasks running that need to continue being serviced on that instance. Before the E2 instance shuts down, the tasks that are currently being serviced should be moved to another instance.

After moving to the instance first, you can terminate 22. This feature is part of the managed instance training function. By using this option, you can move tasks to the instance before termination, allowing for more stable operation of ECS services compared to the existing method of adding tasks based on the desired count after the E2 instance has been terminated due to scaling. 

Another option to consider is scale-in protection. This option prevents the termination of an important ECS service task when the instance is scaled in unexpectedly. If a task in the service with this option activated is running on instance 2, that instance will not be terminated. While this provides the advantage of delivering a stable service by ensuring that important tasks do not get terminated, it may lead to inefficient management of computing resources, as it would prevent the termination of instance 22, which is operating with available resources. Consequently,

These options have trade-offs, so it is recommended that managers understand and configure the workload. So far, I have explained the operational principles of cluster autoscaling within a single capacity provider. Next, let's look at the case of using multiple E2 capacity providers together. Assume we are using both the capacity provider for Fight and the E2 Spot capacity provider. To reliably service the workload, we can see that the capacity provider operating on the E2 On-Demand option is set to a base of 3, with the remaining tasks distributed in a 2:1 ratio between East and East Spot. To handle the necessary user requests, the tasks that are essential are operated on-demand, while utilizing spot resources to secure idle resources, allowing for cost-efficient workload management.

E2 Capacity Provider 사용 시 고려 사항을 살펴보겠습니다. 첫 번째로, 하나 이상의 작업을 시작할 때 용량 사용 가능 여부에 따라 작업은 실행되거나 대기 상태가 될 수 있습니다. 이런 경우 클러스터 오토 스케일링을 먼저 상태 확인해 보시는 게 좋습니다. 충분한 용량이 없는 경우 프로비저닝 상태로 전환되어 있는 것을 볼 수 있습니다. 두 번째는, 스케일 인으로 E2 인스턴스의 여유 자원을 줄이고 싶은 경우 인스턴스의 스케일 인 보호 옵션이 비활성화되어 있는지 확인이 필요합니다. 세 번째는, Capacity Provider의 오토 스케일링이 연결되어 있을 때, 이 오토 스케일링은 Capacity Provider가 관리하고 있기 때문에 관리자가 직접 수정하지 않는 것이 좋습니다. 네 번째는 CPU, 메모리 이외에도 ENI, GPU 등 다른 제약 사항이 없는지 확인해야 합니다. CPU, 메모리 이외에 필요한 자원이 부족한 경우에도 스케일 인이 이루어질 수 없습니다.

There are cases where tasks remain in a pending state. For example, even if there is a lot of CPU memory available, if the task is running in AWS VPC mode, it may remain pending due to the limit on the number of ENIs (Elastic Network Interfaces) for the instance. If multiple tasks want to be served simultaneously, you may need to enable AWS VPC trunking mode to eliminate the quota limit on ENI resources. Lastly, to minimize instance preparation time due to the capacity provider, you can consider adding the warm pool feature to the E2 Auto Scaling group. For reference, looking at how the warm pool feature operates, when ASG instances are needed, it does not prepare instances on demand but rather prepares E2 instances in advance and keeps them in a hyper-nated stop state. When E2 instances are needed, these warm instances can be launched immediately, thus reducing the preparation time for instances. Using spot capacity providers in ECS Auto Scaling.

We talked about the considerations when using Spot Capacity Providers. Spot allows you to use unused resources from AWS EC2 or Fargate at a discounted price. If you need those resources from AWS EC2, they can be reclaimed, and the instance will terminate within about two minutes. Generally, it is recommended to use Spot for stateless tasks, as they can be unexpectedly terminated without impacting the running tasks. In the event of a reclaim event, it is necessary to quickly clean up the running applications and develop them in a way that ensures the overall workload is not affected, even if the service is terminated. If you are introducing Spot for the first time, it might be helpful to consider a scenario where we typically use about 75% of CPU while operating servers. The remaining 25% of unused resources can incur costs, but they can be utilized even when the number of user requests increases.

To operate normally, we have secured resources. In the case of these available resources, their absence does not impact the overall workload. Therefore, to prepare for unexpected increases in user requests, we can strategize by utilizing spot resources at discounted prices. One of the advantages of Amazon ECS is that it automatically handles spot interruptions. It adjusts task termination and automatically removes connections to the load balancer. When unexpected interruptions occur in the workload due to spot instances, it is recommended to design the system to allow for graceful termination. By applying the use of spot instances alongside ECS auto-scaling and on-demand resources to the savings plan purchase option, we can further reduce costs. In summary, we can effectively use auto-scaling to utilize only the necessary resources while minimizing the overhead of instance management.

I have introduced the resources, which can establish scaling strategies according to workload characteristics. It would be good to utilize initial target tracking first. Additionally, you can use multiple capacity providers together to schedule tasks. By appropriately placing on-demand spots, you can manage not only stability but also cost optimization. If you plan to use the workload for a long time, you might consider entering into a contract to receive discounts on the on-demand capacity through a savings plan purchase option. Lastly, when using spots, it is important to design for graceful termination in response to interruptions at the application level. In this session, I discussed various features provided by ECS and how to consider cost-saving measures while maintaining workload performance. Thank you.

Amazon ECS 알아보기 - AutoScaling

Amazon Web Services(AWS)는 웹 서비스의 형태로 기업에 IT인프라를 제공하는 클라우드 컴퓨팅 서비스 전문 기업입니다. 엔터프라이즈 애플리케이션, 빅 데이터 프로젝트, 소셜 게임 및 모바일 앱 시장에 이르기까지 다양한 분야의 기업에 안정적이고 저렴한 확장형 인프라 플랫폼을 제공하고 있으며, 현재 전 세계 190여 개국에 있는 수십만 개의 비즈니스를 지원 중입니다.

고지 사항(Disclaimer)
본 채널의 모든 동영상 컨텐츠는 AWS 서비스 교육 및 설명을 위해 온라인 세미나용으로 별도로 제작, 제공된 것입니다. 만약 AWS 사이트와 컨텐츠상에서 차이나 불일치가 있을 경우, AWS 사이트(aws.amazon.com)가 우선합니다. 또한 AWS 사이트 상에서 한글 번역문과영어 원문에 차이나 불일치가 있을 경우(번역의 지체로 인한 경우 등 포함), 영어 원문이 우선합니다. AWS는 본 컨텐츠에 포함되거나 컨텐츠를 통하여 사용자에게 제공된 일체의 정보, 콘텐츠, 자료, 제품(소프트웨어 포함) 또는 서비스를 이용함으로 인하여 발생하는 여하한 종류의 손해에 대하여 어떠한 책임도 지지 아니하며, 이는 직접 손해, 간접 손해, 부수적 손해, 징벌적 손해및 결과적 손해를 포함하되 이에 한정되지 아니합니다.

Amazon ECS 알아보기 - AutoScaling

Scripts