Skip to content

Conversation

@mohsenmottaghi
Copy link
Contributor

@mohsenmottaghi mohsenmottaghi commented Dec 5, 2025

What this PR does / why we need it:
This change explicitly sets RESTScopeStrategy to RESTScopeNamespace in both wait() and waitForDelete() functions. This ensures:

  • Namespace-scoped resources are watched per-namespace, avoiding cluster-wide LIST operations
  • Cluster-scoped resources continue to work correctly because Kubernetes client-go properly handles empty namespace for cluster-scoped resources
  • Service accounts with only namespace-scoped permissions can successfully use Helm wait and rollback-on-failure functionality

Fixes permission errors like:

Error: UPGRADE FAILED: an error occurred while rolling back the release. original upgrade error: failed to list /v1, Kind=Service: services is forbidden: User "system:serviceaccount:default:cicd-robot" cannot list resource "services" in API group "" at the cluster scope: release core-service failed: failed to list networking.k8s.io/v1, Kind=Ingress: ingresses.networking.k8s.io is forbidden: User "system:serviceaccount:default:cicd-robot" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope

I think that maybe some issues, like ref #31526, are related to this problem.

If applicable:

  • this PR contains user facing changes (the docs needed label should be applied if so)
  • this PR contains unit tests
  • this PR has been tested for backwards compatibility

@pull-request-size pull-request-size bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Dec 5, 2025
@mattfarina
Copy link
Collaborator

@mohsenmottaghi would you be able to write tests for this?

@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Dec 8, 2025
@mohsenmottaghi
Copy link
Contributor Author

@mohsenmottaghi would you be able to write tests for this?

Sure @mattfarina , I've put together a few tests for different scenarios, like deploying across multiple namespaces, working with limited namespace access, and combining namespace-scoped and cluster-scoped resources.

@mohsenmottaghi mohsenmottaghi force-pushed the fix-helm-4-watch-strategy branch from 464d558 to 0fb7cd7 Compare December 8, 2025 19:55
@mohsenmottaghi
Copy link
Contributor Author

@mattfarina, Are these tests sufficient and appropriate, or should I consider changing them?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes permission issues in Helm's wait and rollback functionality by explicitly setting RESTScopeStrategy to RESTScopeNamespace in the wait() and waitForDelete() functions. This ensures namespace-scoped resources are watched per-namespace instead of cluster-wide, allowing service accounts with only namespace-scoped permissions to successfully use Helm wait operations.

Key Changes:

  • Modified wait() and waitForDelete() to use namespace-scoped watching strategy
  • Added comprehensive test coverage with a mock restricted RBAC client
  • Tests verify proper behavior across multiple namespaces and with restricted permissions

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
pkg/kube/statuswait.go Updated wait() and waitForDelete() to explicitly set RESTScopeStrategy: watcher.RESTScopeNamespace in watcher options
pkg/kube/statuswait_test.go Added extensive test coverage including: multi-namespace tests, restricted RBAC client mock, tests for Wait/WaitForDelete/WatchUntilReady with namespace-scoped and cluster-scoped resources

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 600 to 602
go func() {
time.Sleep(timeUntilDelete)
err := fakeClient.Tracker().Delete(gvr, u.GetNamespace(), u.GetName())
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goroutine captures the loop variables u and gvr which may cause a race condition or incorrect behavior. The variables should be passed as parameters to the goroutine or copied to local variables before the goroutine starts to ensure each goroutine operates on the correct values.

Suggested change
go func() {
time.Sleep(timeUntilDelete)
err := fakeClient.Tracker().Delete(gvr, u.GetNamespace(), u.GetName())
uCopy := u
gvrCopy := gvr
go func() {
time.Sleep(timeUntilDelete)
err := fakeClient.Tracker().Delete(gvrCopy, uCopy.GetNamespace(), uCopy.GetName())

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed all goroutines to pass loop variables as parameters to the anonymous function, ensuring each goroutine gets its own copy

@mohsenmottaghi mohsenmottaghi force-pushed the fix-helm-4-watch-strategy branch from 601a6d6 to 0fb7cd7 Compare December 16, 2025 10:24
Adding some tests for multi namespace deployment, simulate restrcited rbac access and mixed namespace scope and cluster scope resources

Signed-off-by: Mohsen Mottaghi <[email protected]>
TerryHowe
TerryHowe previously approved these changes Dec 16, 2025
Copy link
Contributor

@TerryHowe TerryHowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

add missing coverage for the positive case where cluster-scoped resources (like ClusterRole or Namespace) should work correctly

Signed-off-by: Mohsen Mottaghi <[email protected]>
@mohsenmottaghi mohsenmottaghi force-pushed the fix-helm-4-watch-strategy branch from 0fb7cd7 to d6b35ce Compare December 16, 2025 14:14
@pull-request-size pull-request-size bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 16, 2025
@mohsenmottaghi
Copy link
Contributor Author

/lgtm

Thanks @TerryHowe , I updated the branch with the latest version from the main branch and also tried to resolve Copilot's code review suggestions.

Copy link
Contributor

@TerryHowe TerryHowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Tests pass for me.

@gjenkins8 gjenkins8 added the Has One Approval This PR has one approval. It still needs a second approval to be merged. label Dec 20, 2025
Copy link
Member

@gjenkins8 gjenkins8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A chart can create objects in any namespace (and can even create namespaces themselves).

Watching a specific namespace will miss objects created in other namespaces?

@gjenkins8
Copy link
Member

A chart can create objects in any namespace (and can even create namespaces themselves).

Watching a specific namespace will miss objects created in other namespaces?

Ah, the change here is telling the watch library to only watch namespaces associated with the objects (if I have understood correctly)

@gjenkins8 gjenkins8 merged commit 24e8297 into helm:main Dec 22, 2025
5 checks passed
@gjenkins8 gjenkins8 removed the Has One Approval This PR has one approval. It still needs a second approval to be merged. label Dec 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants