Skip to content

Conversation

patrick-ogrady
Copy link
Contributor

@patrick-ogrady patrick-ogrady commented May 27, 2025

During the latest alto deploy, I ran into a number of flaky failures. After making this change, it was able to complete successfully:

2025-05-27T03:43:28.635747Z ERROR deployer: failed to create EC2 deployment error=AwsEc2(Unhandled(Unhandled { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: nodename nor servname provided, or not known" })), connection: Unknown } }), meta: ErrorMetadata { code: None, message: None, extras: None } }))

@patrick-ogrady patrick-ogrady added this to the v0.0.54 milestone May 27, 2025
@patrick-ogrady patrick-ogrady changed the title [deployer] Perform Deletion Concurrently [deployer] Perform Deletion Concurrently + Automate Retry May 27, 2025
@patrick-ogrady patrick-ogrady requested a review from Copilot May 27, 2025 05:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves the EC2 resource deletion process by executing deletions concurrently and enhances the AWS client configuration with an adaptive retry mechanism.

  • Introduces concurrent deletion jobs for EC2 resources using try_join_all
  • Configures an adaptive retry strategy with defined max attempts and backoff parameters in the AWS client
  • Minor update to inline comments for punctuation consistency

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
deployer/src/ec2/destroy.rs Adds asynchronous job creation to perform concurrent resource deletion
deployer/src/ec2/aws.rs Updates the AWS client configuration to use an adaptive retry strategy with backoff limits
Comments suppressed due to low confidence (1)

deployer/src/ec2/destroy.rs:48

  • [nitpick] Consider renaming the variable 'job' to something more descriptive like 'teardown_task' to clarify its purpose in scheduling EC2 resource deletion.
let job = async move {

@patrick-ogrady patrick-ogrady changed the title [deployer] Perform Deletion Concurrently + Automate Retry [deployer] Perform Deletion Concurrently + Activate Automatic Retry May 27, 2025
@patrick-ogrady patrick-ogrady merged commit 2efe85f into main May 27, 2025
13 checks passed
@patrick-ogrady patrick-ogrady deleted the reuse-ami branch May 27, 2025 05:21
Copy link

codecov bot commented May 27, 2025

Codecov Report

Attention: Patch coverage is 0% with 124 lines in your changes missing coverage. Please review.

Project coverage is 90.79%. Comparing base (300af4d) to head (5c9e0b2).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
deployer/src/ec2/destroy.rs 0.00% 119 Missing ⚠️
deployer/src/ec2/aws.rs 0.00% 5 Missing ⚠️
@@            Coverage Diff             @@
##             main     #999      +/-   ##
==========================================
- Coverage   90.82%   90.79%   -0.03%     
==========================================
  Files         190      190              
  Lines       52617    52630      +13     
==========================================
  Hits        47787    47787              
- Misses       4830     4843      +13     
Files with missing lines Coverage Δ
deployer/src/ec2/aws.rs 0.00% <0.00%> (ø)
deployer/src/ec2/destroy.rs 0.00% <0.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 300af4d...5c9e0b2. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant