@TristonianJones Yeah, the lexis says:

WHITESPACE ::= [\t\n\f\r ]+

I actually ran into a scenario in the browser where U+00A0 (non-breaking space) was produced and wouldn't parse in cel-es (which follows the grammar pretty precisely).

However, doing a little more research, it seems like the logical definition of whitespace in the grammar is the purpose-made Pattern_White_Space property in Unicode: https://www.unicode.org/reports/tr31/

This is what Rust defers to: https://doc.rust-lang.org/reference/whitespace.html

Unfortunately, this wouldn't have resolved what I encountered because non-breaking spaces are not in Pattern_White_Space. And although UTR31 does note:

Similarly, each programming language can define its own whitespace characters or syntax characters relative to the Unicode Pattern_White_Space or Pattern_Syntax characters, with some specified set of additions or subtractions.

...I am a little wary of Chesterton's fence.

I'm happy to make a PR to the grammar if we agree on the right whitespace character set. I mainly raised this because it is a frustrating sharp edge when a program appears perfectly valid but isn't. (And I anticipate the use of CEL by relatively unsophisticated users whose workflows might introduce non-ASCII whitespace but will be poorly-equipped to debug the result of resulting errors.)

Also, I really created this issue as a record of a potential gap — this is NOT urgent.

Allow arbitrary Unicode whitespace #484

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions