Also I've noticed that due to timing we still can end up requesting semantic tokens from both sessions if LSP-typescript starts first so it makes sense that in the scope popup we still don't limit anything to single session.

The code block at https://github.com/sublimelsp/LSP/pull/2715/changes#diff-982c110867d9809c978a5597e4da66ce7538b327675a66b8f27214413b7c0ec1R267-R270 that you have re-added here should make that semantic highlighting from a previous session is cleared whenever another session, which is considered as the new "best" session, initializes. So there should only ever be semantic tokens be rendered from a single session, even if they were also requested previously from another session. The reason why you still see them in the scope name popup is because internally the token data itself is still stored in a variable per SessionBuffer when you query that in

That data might even become outdated after some more buffer changes, when the new requests only go to one of the sessions, so I think this needs to be fixed.

If we revert that to use the best_session, it should show only the token from the correct session that is currently rendered in the view. Then I'd slightly prefer the formatting with two separate lines for token type and token modifiers from before, however that's not too important to me and I also like that the name of the session for semantic highlighting is displayed in the updated popup.

Switched to best_session.

Also changed popup output so that semantic tokens section is only shown when there are tokens. Otherwise it's not clear where to put session name

I've just realized something. best_session actually returns different session depending on where the cursor is in the file. So I'm still seeing tokens from LSP-typescript when inside "bindings" while LSP-vue when outside. So based on what you said, I think this does not reflect reality. In both cases it should indicate that the token is from LSP-vue...

Perhaps it would be best to iterate over all sessions but ensure that tokens are cleared properly when session is no longer considered "best".

I've just realized something. best_session actually returns different session depending on where the cursor is in the file. So I'm still seeing tokens from LSP-typescript when inside "bindings" while LSP-vue when outside. So based on what you said, I think this does not reflect reality. In both cases it should indicate that the token is from LSP-vue...

Yes, I've realized the same previously in my PR and set the position offset to 0 in 8e034b0 to ensure a stable session. Here you found another place that I have missed in that PR. So you can just pass 0 to the best_session method here as well to fix it.

I've just realized something. best_session actually returns different session depending on where the cursor is in the file. So I'm still seeing tokens from LSP-typescript when inside "bindings" while LSP-vue when outside. So based on what you said, I think this does not reflect reality. In both cases it should indicate that the token is from LSP-vue...

Yes, I've realized the same previously in my PR and set the position offset to 0 in 8e034b0 to ensure a stable session. Here you found another place that I have missed in that PR. So you can just pass 0 to the best_session method here as well to fix it.

Before I saw your comment I've come up with a solution where I clear pending requests on canceling semantic tokens (there was a timing issue where SessionBuffer.clear_semantic_tokens_async() was called while there was a pending request so tokens were not actually cleared) and clear the whole semantic tokens state in SessionBuffer. Not sure if there can be bad implications of that but it feels more robust than setting offset to 0.

Also as far as formatting in popup feel free to provide better suggestion or even just push changes but since technically it's a list of tokens again, it's harder to say where the session name should be.

Setting the offset to 0 is the right thing to do IMO because the semantic tokens are requested for the entire file anyway. Using the cursor position makes sense for things like "Goto definition" where different parts of the file might be handled by different sessions.

I would revert the last commit and keep clearing the rendering separate from the data handling. The previous semantic tokens data needs to be used for the delta requests, and I think we can still make use of that even after a request got cancelled (at least in theory - I haven't checked if that case could happen in practice).

Without something like

(which happens to be the case for me with LSP-vue & LSP-typescript combo).

the regions still won't be cleared correctly (or rather will be re-added soon after) in case when there is active request

Okay, good observation. Then I would add it under the sb.clear_semantic_tokens_async() line (instead of inside that function itself):

if request_id := sb.semantic_tokens.pending_response:
    sb.session.cancel_request_async(request_id)