Skip to content

abcxyz/github-token-minter

 
 

GitHub Token Minter

GitHub Token Minter (Minty) is a service that acts as a GitHub App that can generate tokens with elevated privileges to allow a GitHub workflow to act upon other private repositories in an organization or on organization level resources such GitHub teams.

This documents the current state of github-token-minter as of v2.x. For details on legacy deployments check the document at docs/guides/LEGACY.md.

If you are looking to migrate to v2.x from a v0.x release please see the migration guide at docs/migration/README.md.

Architecture

Architecture

Setup

Create a GitHub App and Install it

Follow the steps to create a new GitHub App. You'll want to capture the application id and private key that are generated during the creation process.

Once created it needs to be installed in your organziation. When installing you will grant the app the level of access you deem that it needs.

Deploy the service

You can use the provided Terraform module to setup the basic infrastructure needed for this service. You can refer to the provided module to see how to build your own Terraform from scratch.

module "github_token_minter" {
  source = "git::https://github.com/abcxyz/github-token-minter.git//terraform?ref=main" # Should pin this to a SHA

  project_id = "YOUR PROJECT ID"

  domain     = "YOUR DOMAIN NAME"
  dataset_id = "NAME OF YOUR AUDIT DATASET"
  service_iam = {
    admins = []
    developers = [
      "serviceAccount:<YOUR CI SERVICE ACCOUNT>@<YOUR PROJECT ID>.iam.gserviceaccount.com",
    ]
    invokers = [
      "serviceAccount:<YOUR WIF SERVICE ACCOUNT>@<YOUR PROJECT ID>.iam.gserviceaccount.com",
    ]
  }
  dataset_iam = {
    owners = ["group:<some group>@<your org>.com"]
    editors = [
        "serviceAccount:<YOUR WIF SERVICE ACCOUNT>@<YOUR PROJECT ID>.iam.gserviceaccount.com",
    ]
    viewers = ["group:<some other group>@<your org>.com"]
  }
}

By default, the Terraform module deploys the service with Workload Identity Federation (WIF) enabled, requiring authenticated requests. You can disable this behavior by setting the enable_wif variable to false. This will make the Cloud Run service publicly accessible and remove the need for the WIF-related resources.

module "github_token_minter" {
  source = "git::https://github.com/abcxyz/github-token-minter.git//terraform?ref=main" # Should pin this to a SHA

  enable_wif = false

  # ... other variables
}

Service Configuration

The service relies on a number of environment variables. Most of them have defaults that map to the public GitHub environment and are exposed only to allow customers who use private GitHub Enterprise environments to configure them.

The two that are not optional are the GITHUB_APP_ID and the GITHUB_PRIVATE_KEY.

Both of those values come from creating and installing your GitHub app within your organization. These values are sensitive and must be treated as secrets. The default Terraform module stores them in Secret Manager and they are mapped into the Cloud Run service from there.

ENV VAR name Description
GITHUB_APP_ID GitHub App Id created following the app creation instructions at https://docs.github.com/en/developers/apps/building-github-apps/creating-a-github-app
GITHUB_PRIVATE_KEY Private key generated as part of the GitHub App creation
GITHUB_API_BASE_URL GitHubAPIBaseURL is the base URL for the GitHub installation. It should include the protocol (https://) and no trailing slashes.
PORT Port to run the service on. Defaults to 8080
CONFIGS_DIR Location of local configuration files on the filesystem. Defaults to configs
ISSUER_ALLOWLIST The list of OIDC token issuers that GitHub Token Minter will accept. Defaults to accepting GitHub and Google tokens.
JWKS_CACHE_DURATION The duration for which to cache the JWKS for an OIDC token issuer.
REPO_CONFIG_PATH The location within a repository to look for configuration files. Defaults to .github/minty.yaml
ORG_CONFIG_REPO The respository that contains the configuration file for an organization. Defaults to .minty
ORG_CONFIG_PATH The location within an organization to look for configuration files. Defaults to minty.yaml
REF The ref (sha, branch, etc.) to look for configuration files at. Defaults to main

CLI Usage

The minty CLI provides several commands to manage the server and perform related tasks.

minty server run

This command starts the GitHub Token Minter server.

Flag Environment Variable Description
--source-system-auth SOURCE_SYSTEM_AUTH The URI for authenticating with a source system. This matches a custom URI like gha://<app_id>?private_key=<private_key> or gha://<app_id>?kms_id=<kms_id> and supports comma separation for configuring multiple source systems.
--port PORT The port that this server runs as.
--github-app-id GITHUB_APP_ID DEPRECATED: Please use SOURCE_SYSTEM_AUTH instead. The ID of the GitHub App that this server runs as.
--github-private-key GITHUB_PRIVATE_KEY DEPRECATED: Please use SOURCE_SYSTEM_AUTH instead. The private key of the GitHub App that this server runs as.
--source-system-api-base-url SOURCE_SYSTEM_API_BASE_URL The base URL for the Git[Hub
--config-dir CONFIGS_DIR The directory containing local configuration files.
--repo-config-path REPO_CONFIG_PATH The path to the minty configuration file in a repository.
--org-config-path ORG_CONFIG_PATH The path to the minty configuration file for an organization.
--org-config-repo ORG_CONFIG_REPO The repository that contains the configuration file for an organization.
--ref REF The ref (sha, branch, etc.) to look for configuration files at.
--config-cache-minutes CONFIG_CACHE_MINUTES The number of minutes to cache configuration files before retrieving fresh ones. Defaults to 15 minutes.
--jwks-cache-duration JWKS_CACHE_DURATION The duration for which to cache the JWKS for an OIDC token issuer.
--issuer-allowlist ISSUER_ALLOWLIST The list of OIDC token issuers that GitHub Token Minter will accept. Format is a comma-separated list of URLs or the flag can be specified multiple times.

minty tools validate-cfg

This command validates a minty configuration file.

Flag Environment Variable Description
--minty-file MINTY_FILE The minty config file to inspect.
--scope SCOPE The scope to test.
--token TOKEN The token to test with.

minty tools mint

This command mints a token.

Flag Environment Variable Description
--request REQUEST The token request to mint a token for.
--token TOKEN The OIDC token to exchange. This could be a GCP service account or GitHub token.
--mintyURL MINTY_URL The URL of the minty server.

Usage:

minty tools mint \
  --token=$(gcloud auth print-identity-token --impersonate-service-account=my_service_account@iam.gserviceaccount.com --audiences='https://<token minter cloud run service url>' --include-email) \
  --mintyURL=https://minty.url
  --request='{"scope": "read-issues", "org_name": "some-org", "repositories": ["some-repo"], "permissions": {"issues": "read"}}'

minty tools import-pk

This command imports a private key to Google Cloud KMS.

Flag Environment Variable Description
--project-id PROJECT_ID The GCP project ID.
--location LOCATION The Cloud KMS location of the key ring.
--key-ring KEY_RING The name of the key ring that contains the key.
--key KEY The name of the key.
--import-job-prefix IMPORT_JOB_PREFIX The prefix of the import job name.
--private-key The private key to import. By default accept a filepath, and if input is exactly "-", read the value from stdin instead.

Configuring Repository Access

NOTE: We recommend that configuration for single repository access is stored within that repository and configuration that spans multiple repositories is stored in a protected organization repository, typically named .google-github.

Each repository that needs to mint a token must be configured with an allowed set of permissions. Any request from a repository that is not configured will default to denying all access.

Each configuration file can have 1 or more rules that it can match against and it will evaluate them in order, top down, until it finds a matching condition.

The repository configuration file looks like below and is made up of two primary sections:

# github-token-minter/.github/minty.yaml

version: 'minty.abcxyz.dev/v2'

rule:
  if: |-
    (
      assertion.iss == issuers.github &&
      assertion.repository_owner_id == '93787867' &&
      assertion.repository_id == '576289489'
    ) || (
      assertion.iss == issuers.google &&
      assertion.email == "github-automation-bot@gha-ghtm-ci-i-647d53.iam.gserviceaccount.com" // integration CI service account
    )

scope:
  create-tag:
    rule:
      if: |-
        assertion.job_workflow_ref == "abcxyz/pkg/.github/workflows/create-tag.yml@refs/heads/main" &&
        assertion.workflow_ref.startsWith("abcxyz/github-token-minter/.github/workflows/create-tag.yml") &&
        assertion.ref == 'refs/heads/main' &&
        assertion.event_name == 'workflow_dispatch'
    repositories:
      - 'github-token-minter'
    permissions:
      contents: 'write'

  integ:
    rule:
      if: |-
        assertion.workflow_ref.startsWith("abcxyz/github-token-minter/.github/workflows/ci.yml") || (
          assertion.iss == issuers.google &&
          assertion.email == "github-automation-bot@gha-ghtm-ci-i-647d53.iam.gserviceaccount.com" // integration CI service account
        )
    repositories:
      - 'github-token-minter'
    permissions:
      issues: 'read'

Rule

The rule statement is a top level filter which is applied first to ensure that requests meet a minimun standard required by this repository. This typically denotes which issuers are permitted, what organization is allowed access and which repositories.

As you can see in the minty.yaml file in this repository, this rule only allows requests from GitHub if they come from the abcxyz organization (referenced by id to prevent name squatting attacks), and that the request to come from the repository id for github-token-minter, there is an additionl OR statement that allows access from a particular GCP service account (more details on that below).

The if clause uses the Google Common Expression Language to match an inbound OIDC token against a series of rules. Any attributes from the assertion object are available to match against and you can make this expression as simple or complicated as required.

The object mirrors what is available in an OIDC token for a GitHub Workflow, below is an example token.

{
  "jti": "example-id",
  "sub": "repo:octo-org/octo-repo:environment:prod",
  "environment": "prod",
  "aud": "https://github.com/octo-org",
  "ref": "refs/heads/main",
  "sha": "example-sha",
  "repository": "octo-org/octo-repo",
  "repository_owner": "octo-org",
  "actor_id": "12",
  "repository_visibility": "private",
  "repository_id": "74",
  "repository_owner_id": "65",
  "run_id": "example-run-id",
  "run_number": "10",
  "run_attempt": "2",
  "actor": "octocat",
  "workflow": "example-workflow",
  "head_ref": "",
  "base_ref": "",
  "event_name": "workflow_dispatch",
  "ref_type": "branch",
  "job_workflow_ref": "octo-org/octo-automation/.github/workflows/oidc.yml@refs/heads/main",
  "iss": "https://token.actions.githubusercontent.com",
  "nbf": 1632492967,
  "exp": 1632493867,
  "iat": 1632493567
}

OIDC tokens for Google Cloud Service Accounts are also accepted. These tokens include fewer claims and can be distinguished by the issuer field iss, example below. Note that GitHub tokens include a repository claim, but Google tokens specify the repository using the audience claim aud.

{
  "aud": "testorg/testrepo",
  "azp": "123456789012345678901",
  "email": "example-service-account@example-project-id.iam.gserviceaccount.com",
  "email_verified": "true",
  "exp": "1729210450",
  "iat": "1729206850",
  "iss": "https://accounts.google.com",
  "sub": "123456789012345678901",
  "alg": "RS256",
  "kid": "ffffffffffffffffffffffffffffffffffffffff",
  "typ": "JWT"
}

Scopes

The scopes section contains named entry points into the configuration file. Each request contains a scope parameter which directs the service to look at the particular rules defined by that scope.

As you can see from the example, the create-tag scope defines another if statement which is used to verify that the caller is permitted to request this scope.

The repositories and permissions attributes mirror the schema defined for requesting a GitHub app installation access token.

  • Repositories is an array of strings representing the name of the repository. The repository attribute supports a prefix wildcard match so you can select multiple repositories at once. A single * would capture all repositories for the organization but something like github-* would only capture repositories that started with github-. For single level repository configuration this field can be omitted and it will default to the current repository.

  • Permissions is a map of permission name to access level. For example, to generate a token that can read issues you would specify "issues": "read".

Using the service

The service provides a GitHub action that can be used to access the service from a GitHub workflow. It is located in the .github/actions/minty directory of this repository.

The action can be used in two ways depending on how your github-token-minter service is deployed.

Method 1: Authenticating with Google Cloud (WIF)

If your service was deployed with Workload Identity Federation enabled (enable_wif = true), the caller must first authenticate with Google Cloud using the google-github-actions/auth action.

      - id: 'minty-auth'
        uses: 'google-github-actions/auth@6fc4af4b145ae7821d527454aa9bd537d1f2dc5f' # ratchet:google-github-actions/auth@v2
        with:
          create_credentials_file: false
          export_environment_variables: false
          workload_identity_provider: '${{ vars.TOKEN_MINTER_WIF_PROVIDER }}'
          service_account: '${{ vars.TOKEN_MINTER_WIF_SERVICE_ACCOUNT }}'
          token_format: 'id_token'
          id_token_audience: '${{ vars.TOKEN_MINTER_SERVICE_AUDIENCE }}'
          id_token_include_email: true

The main parameters are the workload_identity_provider and service_account you want to authenticate with and the service_audience that the github-token-minter server expects for your environment.

Once you have authenticated you then call the minty action like this:

      - id: 'mint-token'
        uses: 'abcxyz/github-token-minter/.github/actions/minty@main' # ratchet:exclude
        with:
          id_token: '${{ steps.minty-auth.outputs.id_token }}'
          service_url: '${{ vars.TOKEN_MINTER_SERVICE_URL }}'
          requested_permissions: |-
            {
              "scope": "draft-release",
              "repositories": ["${{ github.event.repository.name }}"],
              "permissions": {
                "pull_requests": "write",
                "contents": "write"
              }
            }

The id_token from the auth action is passed to the minty action.

Method 2: Authenticating with the GitHub Workflow Token

If your service was deployed without Workload Identity Federation (enable_wif = false), you can call the minty action directly without the google-github-actions/auth step.

      - id: 'mint-token'
        uses: 'abcxyz/github-token-minter/.github/actions/minty@main' # ratchet:exclude
        with:
          service_url: '${{ vars.TOKEN_MINTER_SERVICE_URL }}'
          requested_permissions: |-
            {
              "scope": "read-issues",
              "repositories": ["some-repo-in-other-org"],
              "permissions": {
                "issues": "read"
              }
            }

In both methods, the final parameter is the request object itself. It contains the scope that is being targeted, an optional org_name to specify a different GitHub organization, an optional set of repositories that the workflow is requesting access to (this defaults to the current repo), and the set of permissions that the workflow requires. The permissions map can be a subset of the permissions defined in the config file or can be omitted to request whatever permissions are defined for the scope.

Example Workflow

 use-a-token-example:
    runs-on: 'ubuntu-latest'
    needs:
      - 'deployment'
    permissions:
      contents: 'write'
      packages: 'write'
      id-token: 'write'
    steps:
      - name: 'Checkout'
        uses: 'actions/checkout@ac593985615ec2ede58e132d2e21d2b1cbd6127c' # ratchet:actions/checkout@v3
      - id: 'minty-auth'
        uses: 'google-github-actions/auth@6fc4af4b145ae7821d527454aa9bd537d1f2dc5f' # ratchet:google-github-actions/auth@v2
        with:
          create_credentials_file: false
          export_environment_variables: false
          workload_identity_provider: '${{ vars.TOKEN_MINTER_WIF_PROVIDER }}'
          service_account: '${{ vars.TOKEN_MINTER_WIF_SERVICE_ACCOUNT }}'
          token_format: 'id_token'
          id_token_audience: '${{ vars.TOKEN_MINTER_SERVICE_AUDIENCE }}'
          id_token_include_email: true
      - id: 'mint-token'
        uses: 'abcxyz/github-token-minter/.github/actions/minty@main' # ratchet:exclude
        with:
          id_token: '${{ steps.minty-auth.outputs.id_token }}'
          service_url: '${{ vars.TOKEN_MINTER_SERVICE_URL }}'
          requested_permissions: |-
            {
              "scope": "read-issues",
              "repositories": ["${{ github.event.repository.name }}"],
              "permissions": {
                "issues": "read"
              }
            }

      - name: 'list-issues'
        run: |
          curl --fail \
            -H "Accept: application/vnd.github+json" \
            -H "Authorization: Bearer ${{ steps.mint-token.outputs.token }}"\
            -H "X-GitHub-Api-Version: 2022-11-28" \
            https://api.github.com/repos/abcxyz/github-token-minter/issues/events

Known Limitations

Note that minting multiple tokens in the same workflow job will cause the post-job cleanup step to fail.

Example:

Post job cleanup.
Error: HttpError: Bad credentials - https://docs.github.com/rest

About

No description, website, or topics provided.

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Packages

No packages published

Contributors 13

Languages