Skip to content

Conversation

@999eagle
Copy link
Contributor

@999eagle 999eagle commented Feb 7, 2024

Closes: #26851

This PR changes the admin/realms/{realm}/clients/{client-uuid}/authz/resource-server/policy/ endpoints by splitting the .../policy/{type}/ endpoint (which could return two different sub services depending on whether a policy type or policy uuid was given as {type}) into two separate endpoints .../policy/by-type/{type}/ and .../policy/by-id/{policy-id}/. This allows us

  1. to correctly document those endpoints in the OpenAPI spec as the generator previously couldn't descend into the sub services as the method just returned Object instead of concrete services and
  2. to prevent the infinite recursion for the PolicyTypeService which inherited the old endpoint from the PolicyService thus allowing API calls like .../policy/{type}/{type}/{type}/{type}/... to actually work.

This is a backend-only change and keeps the current dual-use endpoint working to prevent introducing breaking API changes (except for disallowing the infinite recursion which hopefully no one is actually using). This endpoint is still undocumented in the OpenAPI spec though as different sub services and thus different endpoints can be returned depending on the {type}-argument.

For this change, I've extracted the common code of PolicyService and PolicyTypeService into a new abstract base class BasePolicyService and changed PolicyTypeService to extend that instead of the PolicyService to make sure the dual-use endpoint is only available for the PolicyService and not for the PolicyTypeService. An easier change would've been to override the method in PolicyTypeService to throw an exception or something but this sadly is impossible to communicate to the OpenAPI spec generator thus documenting the endpoint twice.
The new .../policy/by-type/{type}/ and .../policy/by-id/{policy-id}/ endpoints are added to the PolicyService only and also would've been documented for the PolicyTypeService if it still extended that instead of the common base class.

@ahus1
Copy link
Contributor

ahus1 commented Feb 13, 2024

Thank you for this contribution. This moves around quite some bits of the code. I also read your comment in the parent issue:

this might be a breaking change in the API and require changes in the admin UI to separate those two endpoints.

Could you please

  • summarize how the API behavior changed due to this PR, and
  • share the overall idea of the refactoring you did?

This would help me a lot when reviewing this. Thanks!

@ahus1 ahus1 self-assigned this Feb 13, 2024
@999eagle
Copy link
Contributor Author

Sorry, I've updated the PR description now to give more details about this.

Copy link
Contributor

@pedroigor pedroigor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@999eagle The refactoring makes sense but I'm not 100% sure about the new endpoints you are introducing.

If you can elaborate a bit more about why they are here, I appreciate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this (and the other endpoint) endpoint?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a look at this PR as well, and I am (and OpenAPI) is struggling with the Object return type. By splitting the policy-by-type and policy-by-ID to separate endpoints, it would allow a more specific return type than Object. Happy to hear more direction from you about this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's exactly why I've introduced these two new endpoints.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern is that we now have those two endpoints plus the current one returning an object and that does not work well with Open API. Shall we at least deprecate the current endpoint in favor of these two and remove it in 1 or 2 releases from now?

I was wondering if we could somehow handle this using a org.eclipse.microprofile.openapi.OASFilter so that we could add both variants (type/id) to the resulting model tree. But I don't see how we can do that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open API doesn't really document the current one anyway as it's returning Object and thus the generator cannot actually document all the endpoints in those services.
If you're okay with that deprecating would be a good idea but that would necessarily involve changes to the admin UI as well, so I didn't do it right away in this PR.

@ahus1
Copy link
Contributor

ahus1 commented Mar 4, 2024

I had a look at this PR and I still need to understand more of the AUTHZ stuff and how the URLs and UI work together. I agree that the current API has its flaws and it should be changed. Still I'm not sure how, although I know we should deprecate some of the usages.

I found that the Java Admin CLI offers a little bit different APIs to this, as the types are named there, and they return different representations: https://github.com/keycloak/keycloak/blob/3f6925143aa8a3e61c30782029ca5ea781d60ce0/integration/admin-client-jee/src/main/java/org/keycloak/admin/client/resource/PoliciesResource.java

Again, I need more time to look at this.

@999eagle 999eagle force-pushed the fix/openapi-authz-policies branch from 76a4f00 to 08339da Compare March 27, 2024 09:15
@999eagle
Copy link
Contributor Author

Rebased onto current main so I can use this patch in my build.

@999eagle 999eagle requested review from ahus1 and pedroigor April 8, 2024 11:31
@pedroigor
Copy link
Contributor

I found that the Java Admin CLI offers a little bit different APIs to this, as the types are named there, and they return different representations: https://github.com/keycloak/keycloak/blob/3f6925143aa8a3e61c30782029ca5ea781d60ce0/integration/admin-client-jee/src/main/java/org/keycloak/admin/client/resource/PoliciesResource.java

@ahus1 @999eagle FTR, originally we did not have specific endpoints for each permission or policy type but only a generic API to handle all of them using a generic representation. At that time we also did not have OpenAPI support and we did not change the Authorization API to fit into it.

Having a proper OpenAPI is even more tricky if you consider that each policy type has its own representation to make it easier to manage those different policies. Are we changing this here?

@999eagle
Copy link
Contributor Author

@pedroigor I have not changed how policies themselves are represented and how the representations of different types of policies are documented in the OpenAPI spec as that would be (as you already said yourself) quite tricky. Policies are still documented to be returned as the generic representation.

This is basically the smallest possible API change to be able to at least document all the policy-related API endpoints (not including the specific bodies returned of course). The only thing this should change is to separate the existing API endpoint /admin/realms/{realm}/clients/{client-uuid}/authz/resource-server/policy/{type} (where {type} can actually be either a type or an id which already returns different sub-services depending on which one it is) into two separate endpoints /admin/realms/{realm}/clients/{client-uuid}/authz/resource-server/policy/by-type/{policy-type} and /admin/realms/{realm}/clients/{client-uuid}/authz/resource-server/policy/by-id/{policy-id} that always return the same type of sub-service (PolicyTypeService and PolicyResourceService respectively).

@999eagle
Copy link
Contributor Author

@pedroigor @ahus1 ping
did you have any time to review this yet?

@999eagle 999eagle force-pushed the fix/openapi-authz-policies branch from 08339da to 4540cec Compare June 5, 2024 09:03
@999eagle
Copy link
Contributor Author

999eagle commented Jun 5, 2024

rebased onto current main again.

@999eagle 999eagle force-pushed the fix/openapi-authz-policies branch from 4540cec to 7d6dd41 Compare July 3, 2024 07:06
@999eagle
Copy link
Contributor Author

999eagle commented Jul 3, 2024

rebased again to be able to use this. also pinging @ahus1 @pedroigor again, did you have time to review this?

@ahus1
Copy link
Contributor

ahus1 commented Jul 5, 2024

@999eagle - sorry for the long time I kept you waiting on this PR. I appreciate all the enhancements you provided in the previous PRs by adding the missing annotations to Keycloak's code.

This one is IMHO different as it adds new API methods that allow for documentation, while it keeps the old API that is then still undocumented. In addition to that, I struggle with the policy endpoints as I'm not familiar with this area of the code.

The newly added API is then still incomplete in at least two ways:

  • The responses are not documented in the API description (you mention that above, but I didn't check the results).
  • The Java admin client's API description still uses the old API. While the backend is dynamic, the client is explicitly typed - see
    @Path("role")
    RolePoliciesResource role();
    @Path("user")
    UserPoliciesResource user();
    @Path("js")
    JSPoliciesResource js();
    @Path("time")
    TimePoliciesResource time();
    @Path("aggregate")
    AggregatePoliciesResource aggregate();
    @Path("client")
    ClientPoliciesResource client();
    @Path("group")
    GroupPoliciesResource group();
    @Path("client-scope")
    ClientScopePoliciesResource clientScope();
    @Path("regex")
    RegexPoliciesResource regex();
    }

This week I stumbled across a similar problem that we don't document some of the Realm Admin endpoints that are dynamically added - see

@Path("{extension}")
public Object extension(@PathParam("extension") String extension) {
AdminRealmResourceProvider provider = session.getProvider(AdminRealmResourceProvider.class, extension);
if (provider != null) {
Object resource = provider.getResource(session, realm, auth, adminEvent);
if (resource != null) {
return resource;
}
}
throw new NotFoundException();
}

This indicates to me that we should look for a different approach to solve this. Even if we redesign this API, the other Realm Admin endpoint will still give us a headache.

We could reach out to the https://github.com/smallrye/smallrye-open-api project to ask if there is an annotation we could add to indicate different possible subresoures or return types. IMHO in the case of the Realm Admin Subresources it won't help us much as those classes live in a different Maven module and are not known there at compile time, although it would help us here.

I searched the OpenAPI ecosystem, and found several tools which claim to merge OpenAPI specs. See https://github.com/robertmassaioli/openapi-merge?tab=readme-ov-file for one example. There also seems to be a Java Maven plugin, although it might be less popular: https://github.com/kpramesh2212/openapi-merger-plugin/tree/main/openapi-merger-maven-plugin

So I'd suggest a new Proof-of-concept: As the OpenAPI annotations in the Java Code don't seem to be a good fit here, create a manual OpenAPI spec for those parts and merge it with the automatically created spec. I would assume you could do that even locally without changing the Keycloak code, at least in the proof-of-concept mode.

If that proves to work, the next challenge would be to automate that in the Keycloak build: The Node plugin might not be a good fit in the Java build, and the Java Maven plugin doesn't seem to be popular and I didn't find something the main OpenAPI/Swagger project for that (might need to look again). So we would rather look into build our own Maven plugin which is then maintained in the Keycloak repository.

A possible future enhancement would then to migrate the manually created OpenAPI spec to something that is at least half-automated, or we can stick with merging an automatically generated and a second manually maintained file.

I know this is very different from what you originally suggested. I'd be happy to hear your thoughts on this about the possible advantages and disadvantages you see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAPI: API endpoints for client/{id}/authz/resource-server/policy/ are undocumented

3 participants