When using Voiceover + Safari, expected behaviour would be that I can open the select and use up and down arrow keys to navigate. The screen reader focus is the same as the visual/programmatic focus.
The focus is on the listbox. User presses arrow key. Component updates aria-activedescendant. Screen reader sees change in accessibility API. Screen reader reads new active option.

What actually happens is that screen reader focus might stay the same for 1 or 2 key presses, but then derails, even going outside of the select.
What seems to happen is more something like. User presses arrow key. Screen reader tries to navigate. Screen reader starts to read something new. Component updates aria-activedescendant. Screen reader sees change in accessibility API. Screen reader may or may not read new active option.

It is kind of hard to describe as it's feels slightly random.

I've investigated if making changes to the html/aria structure changes things, but I was unable to affect the outcome. It seems to me as this might be some sort of timing issue but I am unable to point to a root cause right now.