The reason for this is because of the Resolution in the coarse matching. This means that a coarse match is done at every 8 pixels. Another reason is due to the input into the Vision transformer as the token size is reduced. This is why it is recomended to use a Image size that is divisible by 8.