Skip to content

http: add support for HTTP 429 rate limit retries #2008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vaidas-shopify wants to merge 2 commits into
base: master
Choose a base branch
from
+750 −57
Open

http: add support for HTTP 429 rate limit retries #2008

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d80ce07
http: add support for HTTP 429 rate limit retries
vaidas-shopify Nov 26, 2025
ad4495f
http: add trace2 logging for retry operations
vaidas-shopify Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
http: add trace2 logging for retry operations 
Add trace2 instrumentation to HTTP 429 retry operations to enable
monitoring and debugging of rate limit scenarios in production
environments.

The trace2 logging captures:

  * Retry attempt numbers (http/429-retry-attempt) to track retry
    progression and identify how many attempts were needed

  * Retry-After header values (http/429-retry-after) from server
    responses to understand server-requested delays

  * Actual sleep durations (http/retry-sleep-seconds) within trace2
    regions (http/retry-sleep) to measure time spent waiting

  * Error conditions (http/429-error) such as "retries-exhausted",
    "exceeds-max-retry-time", "no-retry-after-config", and
    "config-exceeds-max-retry-time" for diagnosing failures

  * Retry source (http/429-retry-source) indicating whether delay
    came from server header or config default

This instrumentation provides complete visibility into retry behavior,
enabling operators to monitor rate limiting patterns, diagnose retry
failures, and optimize retry configuration based on real-world data.

Signed-off-by: Vaidas Pilkauskas <[email protected]>
  • Loading branch information
@vaidas-shopify
vaidas-shopify committed Dec 18, 2025
commit ad4495fc94a4bcdcf7f299ffd8514afce88f2d6c
23 changes: 22 additions & 1 deletion http.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
#include "odb.h"
#include "tempfile.h"
#include "date.h"
#include "trace2.h"

static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
static int trace_curl_data = 1;
Expand Down Expand Up @@ -1738,6 +1739,8 @@ void run_active_slot(struct active_request_slot *slot)

if (waiting_for_delay) {
warning(_("rate limited, waiting %ld seconds before retry"), slot->retry_delay_seconds);
trace2_data_intmax("http", the_repository, "http/retry-sleep-seconds",
slot->retry_delay_seconds);
start_time = slot->retry_delay_start;
}

Expand All @@ -1753,6 +1756,7 @@ void run_active_slot(struct active_request_slot *slot)
}

if (elapsed_time.tv_sec >= slot->retry_delay_seconds) {
trace2_region_leave("http", "retry-sleep", the_repository);
slot->retry_delay_seconds = -1;
waiting_for_delay = 0;

Expand Down Expand Up @@ -1995,6 +1999,8 @@ static int handle_curl_result(struct slot_results *results)
return HTTP_REAUTH;
}
} else if (results->http_code == 429) {
trace2_data_intmax("http", the_repository, "http/429-retry-after",
results->retry_after);
return HTTP_RATE_LIMITED;
} else {
if (results->http_connectcode == 407)
Expand Down Expand Up @@ -2421,10 +2427,16 @@ static void sleep_for_retry(struct active_request_slot *slot, long retry_after)
static long handle_rate_limit_retry(int *rate_limit_retries, long slot_retry_after)
{
int retry_attempt = http_max_retries - *rate_limit_retries + 1;

trace2_data_intmax("http", the_repository, "http/429-retry-attempt",
retry_attempt);

if (*rate_limit_retries <= 0) {
/* Retries are disabled or exhausted */
if (http_max_retries > 0) {
error(_("too many rate limit retries, giving up"));
trace2_data_string("http", the_repository,
"http/429-error", "retries-exhausted");
}
return -1;
}
Expand All @@ -2439,6 +2451,10 @@ static long handle_rate_limit_retry(int *rate_limit_retries, long slot_retry_aft
error(_("rate limited (HTTP 429) requested %ld second delay, "
"exceeds http.maxRetryTime of %ld seconds"),
slot_retry_after, http_max_retry_time);
trace2_data_string("http", the_repository,
"http/429-error", "exceeds-max-retry-time");
trace2_data_intmax("http", the_repository,
"http/429-requested-delay", slot_retry_after);
return -1;
}
return slot_retry_after;
Expand All @@ -2448,16 +2464,21 @@ static long handle_rate_limit_retry(int *rate_limit_retries, long slot_retry_aft
/* Not configured - exit with error */
error(_("rate limited (HTTP 429) and no Retry-After header provided. "
"Configure http.retryAfter or set GIT_HTTP_RETRY_AFTER."));
trace2_data_string("http", the_repository,
"http/429-error", "no-retry-after-config");
return -1;
}
/* Check if configured default exceeds maximum allowed */
if (http_retry_after > http_max_retry_time) {
error(_("configured http.retryAfter (%ld seconds) exceeds "
"http.maxRetryTime (%ld seconds)"),
http_retry_after, http_max_retry_time);
trace2_data_string("http", the_repository,
"http/429-error", "config-exceeds-max-retry-time");
return -1;
}

trace2_data_string("http", the_repository,
"http/429-retry-source", "config-default");
return http_retry_after;
}
}
Expand Down
Loading