Heyas! Just saw your hackaday, this is a super cool project.

I once built an arduino VT220 emulator, I wonder if (some of) that could be leveraged somehow?