I think it makes sense, and it would simplify the whole thing. Not sure if we should have that scoped in this PR, or open one just to handle the authorization events and do the necessary cleanup.

Someone suggested to remove of all the InvalidationEvent implementations and using this event type everywhere. I would make this more future-proof to avoid future issues. If you don't agree, the PR looks good to me!

public class CacheKeyInvalidatedEvent extends InvalidationEvent implements RealmCacheInvalidationEvent, UserCacheInvalidationEvent, AuthorizationCacheInvalidationEvent {

    @ProtoFactory
    public CacheKeyInvalidatedEvent(String id) {
        super(id);
    }

    @Override
    public void addInvalidations(RealmCacheManager realmCache, Set<String> invalidations) {
        addCacheKeyInvalidations(realmCache, invalidations);
    }

    @Override
    public void addInvalidations(StoreFactoryCacheManager storeFactoryCache, Set<String> invalidations) {
        addCacheKeyInvalidations(storeFactoryCache, invalidations);
    }

    @Override
    public void addInvalidations(UserCacheManager userCache, Set<String> invalidations) {
        addCacheKeyInvalidations(userCache, invalidations);
    }

    private void addCacheKeyInvalidations(CacheManager cacheManager, Set<String> invalidations) {
        cacheManager.invalidateCacheKey(getId(), invalidations);
    }
}

Sounds like a refactoring and I don't see any harm doing it here. But if you are not pushing for it I can send a separate PR.