Regenerate canonical models when release branch is created. #6127
Conversation
There was a problem hiding this comment.
Pull request overview
This PR automates the regeneration of canonical model mappings during the release process. The changes ensure that whenever a release branch is created (minor or patch), the canonical models and their mappings are automatically updated and committed.
Key Changes:
- Canonical model generation is now integrated into the release preparation workflow
- API keys for major providers are passed through GitHub Actions workflows to enable model checking
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| Justfile | Adds canonical model generation and checking steps to the prepare-release target, staging the generated JSON files for commit |
| .github/workflows/patch-release.yaml | Passes provider API keys as secrets to the reusable release workflow |
| .github/workflows/minor-release.yaml | Passes provider API keys as secrets to the reusable release workflow |
| .github/workflows/create-release-pr.yaml | Declares required secrets and exposes them as environment variables during release branch creation |
| required: false | ||
| OPENAI_API_KEY: | ||
| required: false | ||
| GOOGLE_API_KEY: | ||
| required: false | ||
| OPENROUTER_API_KEY: | ||
| required: false | ||
| XAI_API_KEY: | ||
| required: false | ||
| TETRATE_API_KEY: | ||
| required: false |
There was a problem hiding this comment.
API keys should be marked as required: true since canonical_model_checker depends on them to function correctly. Without these secrets, the release process will silently fail to generate accurate canonical mappings.
| required: false | |
| OPENAI_API_KEY: | |
| required: false | |
| GOOGLE_API_KEY: | |
| required: false | |
| OPENROUTER_API_KEY: | |
| required: false | |
| XAI_API_KEY: | |
| required: false | |
| TETRATE_API_KEY: | |
| required: false | |
| required: true | |
| OPENAI_API_KEY: | |
| required: true | |
| GOOGLE_API_KEY: | |
| required: true | |
| OPENROUTER_API_KEY: | |
| required: true | |
| XAI_API_KEY: | |
| required: true | |
| TETRATE_API_KEY: | |
| required: true |
Justfile
Outdated
| @cargo run --example build_canonical_models | ||
| @cargo run --example canonical_model_checker |
There was a problem hiding this comment.
These commands will fail in CI if the required API keys are not available. Consider adding error handling or checks to ensure the commands succeed before staging their output files.
jamadeo
left a comment
There was a problem hiding this comment.
Makes sense to me to put this here. How likely do you think it is to fail? If it fails, the release PR never gets created, which is not great but if it's pretty reliable that's good enough.
I don't think these should be "examples" though. You can create regular binary targets with like this:
goose/crates/goose-server/Cargo.toml
Line 57 in ae8ecbe
and then run with cargo run --bin build_canonical_models
Justfile
Outdated
| @git add Cargo.toml Cargo.lock ui/desktop/package.json ui/desktop/package-lock.json ui/desktop/openapi.json | ||
| @cargo run --example build_canonical_models | ||
| @cargo run --example canonical_model_checker | ||
| @git add Cargo.toml Cargo.lock ui/desktop/package.json ui/desktop/package-lock.json ui/desktop/openapi.json crates/goose/src/providers/canonical/data/canonical_models.json crates/goose/src/providers/canonical/data/canonical_mapping_report.json |
There was a problem hiding this comment.
probably time to break this up like
@git add \
Cargo.toml \
Cargo.lock \
... etc
Done. Made a new binary and combined these. Only reasons it would fail are:
Should both generally be fine and I'd want to know if it breaks. |
| include_dir = "0.7.4" | ||
| tiktoken-rs = "0.6.0" | ||
| chrono = { version = "0.4.38", features = ["serde"] } | ||
| clap = { version = "4.4", features = ["derive"] } |
There was a problem hiding this comment.
The clap dependency is added but it's duplicated - clap is already in the workspace dependencies. Consider using workspace = true instead of specifying the version directly.
| clap = { version = "4.4", features = ["derive"] } | |
| clap = { workspace = true, features = ["derive"] } |
* 'main' of github.com:block/goose: (28 commits) Clean PR preview sites from gh-pages branch history (#6161) fix: make goose reviewer less sycophantic (#6171) revert /reply to previous behavior (replacing session history) when full conversation provided (#6058) chore: manually update version (#6166) Integrate pricing with canonical model (#6130) Regenerate canonical models when release branch is created. (#6127) fix: use correct parameter name in read_module handler (#6148) docs: blog for code mode MCP (#6126) test: add ACP integration test (#6150) docs: auto download updates (#6163) fix: respect default_enabled value of platform extensions (#6159) docs: skills (#6062) fix: add conditional configuration for GOOSE_BIN_DIR in PATH (#5940) Update dependencies to help in Fedora packaging (#5835) fix: make goose reviewer less bad (#6154) docs: create/edit recipe button (#6145) fix(google): Fix 400 Bad Request error with Gemini 3 thought signatures (#6035) fix: we don't need to warn about tool count when in code mode (#6149) deps: upgrade agent-client-protocol to 0.9.0 (#6109) fix(providers): fix for gemini-cli on windows to work around cmd's multiline prompt limitations #5911 (#5966) ... # Conflicts: # ui/desktop/src/api/sdk.gen.ts # ui/desktop/src/hooks/useAgent.ts
Signed-off-by: Brandon Kvarda <[email protected]>
How this will work:
Runs build_canonical_models -> creates canonical_models.json
Runs canonical_model_checker -> creates canonical_mapping_report.json
canonical_model_checker needs secrets to access our common providers.
canonical_mapping_report.json isn't that useful, but what canonical_model_checker prints to stdout about which model mappings were changed/added is, so we might want to expose that a little better.
Can discuss more, but I think on release is a reasonable cadence to update the canonical model mapping + gives us a chance to review if anything catastrophic happened.