Skip to content

Conversation

vviers
Copy link
Collaborator

@vviers vviers commented Jul 21, 2025

Context

This PR tries to solve the issue described in #1702 through the introduction of 2 env variables :
GRIST_TRUTHY_VALUES and GRIST_FALSY_VALUES

Proposed solution

I introduce 2 env variables that are passed to the python sandbox. I know sandbox security is quite crucial, and so I'm looking for feedback as to whether that is safe. it is :)

Related issues

Closes #1702

Has this been tested?

  • 👍 yes, I added tests to the test suite
  • 💭 no, because this PR is a draft and still needs work
  • 🙅 no, because this is not relevant here
  • 🙋 no, because I need help

@vviers vviers requested a review from fflorent July 21, 2025 15:33
Copy link
Contributor

github-actions bot commented Jul 21, 2025

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@vviers
Copy link
Collaborator Author

vviers commented Jul 21, 2025

I have read the CLA Document and I hereby sign the CLA

github-actions bot added a commit that referenced this pull request Jul 21, 2025
@vviers
Copy link
Collaborator Author

vviers commented Jul 21, 2025

I'm on the fence between introducing a new env variable vs. trying to use document locale (but that would mean storing those lists directly in the Grist code for all possible language and is probably more work to set up and maintain i18n). Any opinion on this ?

Copy link
Collaborator

@fflorent fflorent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

(I hesitated to ask a test to check that the intersection between the truthy and the falsy values is empty. But it sounds pretty unlikely that this case occur often and we can't be responsible of covering every admins mistakes.)

@fflorent
Copy link
Collaborator

Another question: Is it intended not to document it in the README? (both answers are OK to me, feels like an edge case)

I'm on the fence between introducing a new env variable vs. trying to use document locale (but that would mean storing those lists directly in the Grist code for all possible language and is probably more work to set up and maintain i18n). Any opinion on this ?

Agree, also it could make the behavior unclear if I understand the alternative correctly.
I prefer much more using an env variable like you did.

@vviers vviers changed the title Intruduce extra truthy and falsy values at instance level Introduce extra truthy and falsy values at instance level Jul 21, 2025
@vviers vviers force-pushed the issue_1702 branch 2 times, most recently from fa010b6 to 6749af1 Compare July 21, 2025 16:36
@vviers vviers marked this pull request as ready for review July 21, 2025 16:38
@vviers
Copy link
Collaborator Author

vviers commented Jul 21, 2025

@fflorent and I couldn't find where the usertypes' do_convert tests are (if there are any). We looked at the ValueGuesser tests but realized that they were more related to when a user starts typing values in a new column or imports a file into Grist.

Conversion from text column is a good first step imo, we can look at using the env variables in ValueGuesser too but that's more work and should be a separate PR.

@vviers
Copy link
Collaborator Author

vviers commented Jul 22, 2025

The CI failure seems unrelated to my work

@jordigh
Copy link
Contributor

jordigh commented Jul 23, 2025

I'm on the fence between introducing a new env variable vs. trying to use document locale (but that would mean storing those lists directly in the Grist code for all possible language and is probably more work to set up and maintain i18n). Any opinion on this ?

It wasn't a lot of work to setup, if you trust the LLMs. It's probably correct here:

localized_bools = {
    'ar': {'true': ['نعم', 'صحيح', '1'], 'false': ['لا', 'خطأ', '0']},
    'bg': {'true': ['да', 'вярно', '1'], 'false': ['не', 'невярно', '0']},
    'ca': {'true': ['sí', 'cert', '1'], 'false': ['no', 'fals', '0']},
    'cs': {'true': ['ano', 'pravda', '1'], 'false': ['ne', 'nepravda', '0']},
    'de': {'true': ['ja', 'wahr', '1'], 'false': ['nein', 'falsch', '0']},
    'el': {'true': ['ναι', 'σωστό', '1'], 'false': ['όχι', 'λάθος', '0']},
    'en': {'true': ['yes', 'true', '1'], 'false': ['no', 'false', '0']},
    'en_GB': {'true': ['yes', 'true', '1'], 'false': ['no', 'false', '0']},
    'es': {'true': ['sí', 'verdadero', '1'], 'false': ['no', 'falso', '0']},
    'eu': {'true': ['bai', 'egia', '1'], 'false': ['ez', 'gezurra', '0']},
    'fa': {'true': ['بله', 'درست', '1'], 'false': ['نه', 'نادرست', '0']},
    'fi': {'true': ['kyllä', 'tosi', '1'], 'false': ['ei', 'epätosi', '0']},
    'fr': {'true': ['oui', 'vrai', '1'], 'false': ['non', 'faux', '0']},
    'hu': {'true': ['igen', 'igaz', '1'], 'false': ['nem', 'hamis', '0']},
    'ig': {'true': ['ee', 'ezigbo', '1'], 'false': ['mba', 'ụgha', '0']},
    'it': {'true': ['sì', 'vero', '1'], 'false': ['no', 'falso', '0']},
    'ja': {'true': ['はい', '真', '1'], 'false': ['いいえ', '偽', '0']},
    'ko': {'true': ['예', '참', '1'], 'false': ['아니요', '거짓', '0']},
    'nb_NO': {'true': ['ja', 'sant', '1'], 'false': ['nei', 'usant', '0']},
    'nl': {'true': ['ja', 'waar', '1'], 'false': ['nee', 'onwaar', '0']},
    'pl': {'true': ['tak', 'prawda', '1'], 'false': ['nie', 'fałsz', '0']},
    'pt': {'true': ['sim', 'verdadeiro', '1'], 'false': ['não', 'falso', '0']},
    'pt_BR': {'true': ['sim', 'verdadeiro', '1'], 'false': ['não', 'falso', '0']},
    'ro': {'true': ['da', 'adevărat', '1'], 'false': ['nu', 'fals', '0']},
    'ru': {'true': ['да', 'истина', '1'], 'false': ['нет', 'ложь', '0']},
    'sk': {'true': ['áno', 'pravda', '1'], 'false': ['nie', 'nepravda', '0']},
    'sl': {'true': ['da', 'resnica', '1'], 'false': ['ne', 'napačno', '0']},
    'ta': {'true': ['ஆம்', 'உண்மை', '1'], 'false': ['இல்லை', 'தவறு', '0']},
    'th': {'true': ['ใช่', 'จริง', '1'], 'false': ['ไม่', 'เท็จ', '0']},
    'tr': {'true': ['evet', 'doğru', '1'], 'false': ['hayır', 'yanlış', '0']},
    'uk': {'true': ['так', 'істина', '1'], 'false': ['ні', 'неправда', '0']},
    'vi': {'true': ['đúng', 'có', '1'], 'false': ['sai', 'không', '0']},
    'zh_Hans': {'true': ['是', '真', '1'], 'false': ['否', '假', '0']},
    'zh_Hant': {'true': ['是', '真', '1'], 'false': ['否', '假', '0']},
}

@vviers
Copy link
Collaborator Author

vviers commented Jul 23, 2025

Thanks @jordigh !

What do you think of merging this PR as-is and starting a more comprehensive work that's based on document locale in a separate PR ? Room for improvement also includes the ValueGuesser.

Paging @paulfitz since we discussed this on Slack as well.

@vviers vviers moved this to Needs feedback in French administration Board Jul 23, 2025
@vviers vviers added enhancement New feature or request gouv.fr labels Jul 23, 2025
Copy link
Member

@paulfitz paulfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a pragmatic solution to an awkward problem. One comment (from Dmitry actually)

@paulfitz
Copy link
Member

@vviers it looks good, would be nice to have a test. Looked through tests and closest is in test/server/lib/ActiveDoc.ts, the sandbox passes in docUrl test. Here's a quick stub you could clean up to test your change:

  it('sandbox passes in truthy/falsy settings', async function() {
    const env = new EnvironmentSnapshot();
    try {
      process.env.GRIST_TRUTHY_VALUES = 'meep';
      const activeDoc = new ActiveDoc(docTools.getDocManager(), 'truthyTest');
      await activeDoc.createEmptyDoc(fakeSession);
      await activeDoc.applyUserActions(fakeSession, [
        ["AddTable", "Info", [{id: 'Flag', type: 'Bool'}]],
        ["AddRecord", "Info", null, {Flag: 'meep'}],
        ["AddRecord", "Info", null, {Flag: 'moop'}],
      ]);
      const data = (await activeDoc.fetchTable(fakeSession, 'Info', true)).tableData[3];
      console.log(data); // shows { manualSort: [ 1, 2 ], Flag: [ true, 'moop' ] }
      await activeDoc.shutdown();
    } finally {
      env.restore();
    }
  });

@vviers
Copy link
Collaborator Author

vviers commented Jul 25, 2025

@paulfitz done ✅ thank you !

Copy link
Member

@paulfitz paulfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vviers !

@paulfitz paulfitz merged commit 173d48b into gristlabs:main Jul 25, 2025
14 checks passed
@github-project-automation github-project-automation bot moved this from Needs feedback to Done in French administration Board Jul 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request gouv.fr

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Boolean detection not recognized in French

5 participants