Revert "Parallelize submodule checkout (#41335)" #41425
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 82e6d0e. Signed-off-by: Maciej Grela <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
This PR reverts a previous commit that parallelized submodule checkout, addressing a critical issue where git's parallel submodule operations fail ungracefully during DNS failures, potentially leaving some submodules with empty working directories even after subsequent successful runs.
- Removes the
--jobs 4parameter from all submodule checkout commands across the codebase - Ensures reliable submodule initialization by reverting to sequential processing
- Prevents silent failures that could compromise build integrity
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/darwin/Framework/chip_xcode_build_connector.sh | Removed --jobs 4 from darwin platform submodule checkout |
| scripts/build/gn_gen_cirque.sh | Removed --jobs 4 from linux platform submodule checkout |
| integrations/docker/images/chip-cert-bins/Dockerfile | Removed --jobs 4 from docker build submodule checkout |
| integrations/cloudbuild/smoke-test.yaml | Removed --jobs 4 from cloud build smoke test |
| integrations/cloudbuild/chef.yaml | Removed --jobs 4 from cloud build chef configuration |
| .github/workflows/examples-telink.yaml | Removed --jobs 4 from telink workflow submodule checkouts |
| .github/actions/checkout-submodules/action.yaml | Removed --jobs 4 from reusable checkout action |
There was a problem hiding this comment.
Code Review
This pull request reverts the parallel submodule checkout feature. The description provides a clear and detailed explanation of the issue with git clone --jobs failing ungracefully during DNS failures, leading to inconsistent submodule states. The changes correctly remove the --jobs argument from checkout_submodules.py calls across various build and CI scripts. This is a necessary change to ensure the stability and reliability of the build process. The changes are correct and well-justified.
|
PR #41425: Size comparison from 3108862 to f7a7679 Full report (37 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, nxp, psoc6, qpg, realtek, stm32, telink)
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #41425 +/- ##
==========================================
- Coverage 50.94% 50.94% -0.01%
==========================================
Files 1378 1378
Lines 100698 100698
Branches 13058 13058
==========================================
- Hits 51303 51302 -1
- Misses 49395 49396 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
bzbarsky-apple
approved these changes
Oct 13, 2025
andy31415
approved these changes
Oct 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts commit 82e6d0e.
Summary
When running git clone with --jobs github does not fail gracefully when DNS fails. If after that the checkout is started again it seemingly finishes correctly. This is not true however as some submodules end up with empty workdirs. This has caused at least one job to fail: https://github.com/project-chip/connectedhomeip/actions/runs/18407027799/job/52449279674
Testing
Steps to reproduce the problem:
Start devcontainer:
Start parallel checkout:
After around 10 seconds cut off DNS (run all
iptablescommands outside of the docker container):Wait for the checkout to fail as expected:
Enable DNS again and start a new checkout:
This checkout succeeds like expected:
But some submodules have empty workdirs:
third_party/nlassert/repo,third_party/nlio/repo,third_party/jsoncpp/repoandthird_party/editline/repohave empty workdirs.If you repeat the steps above without
--jobsthe problem does not occur.This looks to be a bug in git.
Readability checklist
The checklist below will help the reviewer finish PR review in time and keep the
code readable:
descriptive
“When in Rome…”
rule (coding style)
See: Pull Request Guidelines