11 KiB
Add a GitHub organization or repository to ARC
Harmony deploys GitHub Actions Runner Controller (ARC) as one controller per
Kubernetes cluster and one runner scale set per GitHub destination. A
destination is a repository, organization, or enterprise selected by
githubConfigUrl.
A scale set cannot serve a second destination by adding another URL. Keep an existing scale set and add another one when onboarding a repository or organization. The scale sets share the cluster controller.
This guide covers repository and organization destinations authenticated by a GitHub App. Enterprise destinations require a personal access token (classic), not a GitHub App. See GitHub's ARC authentication documentation.
ARC runners execute workflow code. Harmony's Docker-in-Docker configuration gives that code control of a privileged container and must be treated as a node compromise boundary. Admit only trusted repositories and workflows. Keep runner nodes isolated from production workloads and restrict their network access.
Choose the scope
Use the smallest scope that matches the current consumers:
| Scope | githubConfigUrl |
Use when |
|---|---|---|
| Repository | https://github.com/<org>/<repo> |
One repository needs the runners |
| Organization | https://github.com/<org> |
Several authorized repositories in one organization need the runners |
Workflows select the scale set by name:
jobs:
build:
runs-on: <runner-scale-set-name>
Runner scale-set names must be unique within a GitHub runner group. The same
name can be used by separate organizations. Put those scale sets in separate
Kubernetes namespaces so their AutoscalingRunnerSet, ServiceAccount, and
Secret names do not collide.
For organization scope, configure the runner group's repository access in GitHub. A scale set's organization URL does not override that policy. If only one repository needs the runner, repository scope avoids this extra access boundary.
Collect the inputs
Set these values before provisioning:
export GITHUB_CONFIG_URL="https://github.com/example-org/example-repo"
export GITHUB_APP_ID="123456"
export GITHUB_APP_INSTALLATION_ID="654321"
export GITHUB_APP_PRIVATE_KEY_FILE="/secure/path/github-app.pem"
export RUNNERS_NAMESPACE="arc-runners-example-org"
export GITHUB_CONFIG_SECRET="github-arc-app"
export RUNNER_SCALE_SET_NAME="okd-cluster-1"
export CONTROLLER_NAMESPACE="arc-systems"
export ARC_CHART_VERSION="$(kubectl -n "$CONTROLLER_NAMESPACE" get deployment \
-l app.kubernetes.io/name=gha-rs-controller \
-o jsonpath='{.items[0].metadata.labels.app\.kubernetes\.io/version}')"
| Variable | Source |
|---|---|
GITHUB_CONFIG_URL |
Repository or organization URL in GitHub |
GITHUB_APP_ID |
GitHub App settings, under App ID |
GITHUB_APP_INSTALLATION_ID |
The final number in https://github.com/organizations/<org>/settings/installations/<id> after installing the App |
GITHUB_APP_PRIVATE_KEY_FILE |
PEM file generated under the GitHub App's Private keys settings |
RUNNERS_NAMESPACE |
A new namespace chosen for this destination |
GITHUB_CONFIG_SECRET |
A chosen Kubernetes Secret name |
RUNNER_SCALE_SET_NAME |
The workflow runs-on label; at most 39 characters with Harmony's github-runner- Helm prefix, and unique within its runner group |
CONTROLLER_NAMESPACE |
Namespace of the shared ARC controller; Harmony defaults to arc-systems |
ARC_CHART_VERSION |
app.kubernetes.io/version on the existing controller Deployment |
The App must be installed in the destination organization. Each organization installation has its own installation ID even when it uses the same App ID and private key. If the installation is limited to selected repositories, add the target repository to the installation.
For organization-scoped runners, grant the App organization permission Self-hosted runners: Read and write. For repository-scoped runners, also grant repository permission Administration: Read and write. GitHub requires Metadata: Read-only for both scopes.
Create the namespace and credential
Apply a Namespace resource from the chosen variable:
kubectl create namespace "$RUNNERS_NAMESPACE" \
--dry-run=client -o yaml | kubectl apply -f -
Create the App credential directly from its sources. The Secret must be in the same namespace as the runner scale set.
kubectl create secret generic "$GITHUB_CONFIG_SECRET" \
--namespace "$RUNNERS_NAMESPACE" \
--from-literal=github_app_id="$GITHUB_APP_ID" \
--from-literal=github_app_installation_id="$GITHUB_APP_INSTALLATION_ID" \
--from-file=github_app_private_key="$GITHUB_APP_PRIVATE_KEY_FILE" \
--dry-run=client -o yaml | kubectl apply -f -
Do not put the private key, a rendered Secret, or Helm values containing the
private key in version control. GithubRunnerScore references the Secret by
name and does not read its value.
Add the Harmony Score
Add a GithubRunnerScore to the cluster's private deployment crate:
use harmony::modules::github_runner::GithubRunnerScore;
let arc_chart_version = "<app.kubernetes.io/version from the controller>";
let runner = GithubRunnerScore {
github_config_url: "https://github.com/example-org/example-repo".into(),
github_config_secret: "github-arc-app".into(),
runner_scale_set_name: "okd-cluster-1".into(),
runners_namespace: "arc-runners-example-org".into(),
controller_namespace: "arc-systems".into(),
min_runners: 0,
max_runners: 2,
cpu_limit: "8".into(),
memory_limit: "16Gi".into(),
chart_version: arc_chart_version.into(),
..GithubRunnerScore::defaults()
};
Onboarding converges only runner.runner_set(). The cluster platform owns the
shared runner.controller() release and its upgrade procedure. Do not install,
upgrade, or downgrade the controller as part of destination onboarding.
The runner-set chart version must match the cluster-owned controller version.
GithubRunnerScore::defaults() currently pins 0.9.3; override
chart_version with the collected ARC_CHART_VERSION if they differ. ARC
upgrades are coordinated platform operations, not ordinary Helm upgrades.
The runner-set release is github-runner-<runner_scale_set_name> in
runners_namespace. Use a Kubernetes-compatible scale-set name because it is
also used in generated resource names. Keep min_runners less than or equal to
max_runners.
min_runners: 0 permits scale-to-zero. Set a positive minimum when an idle
runner must remain available. max_runners caps concurrent runner pods. The
Score applies the CPU and memory limits to the runner. ARC chart 0.9.3
generates the Docker-in-Docker container and does not preserve its requested
resource limits.
The scale-set Helm chart owns the AutoscalingRunnerSet, listener,
ServiceAccount, Role, and RoleBinding resources. Do not maintain copies of
those generated resources by hand.
Account for Docker-in-Docker on OKD
The current Score does not produce a complete working OKD runner pod. Before
onboarding an OKD destination, the platform deployment must supply the existing
OKD overlay or GithubRunnerScore must be extended to own it. The required pod
settings are:
- runner process as UID 1001 and GID 123;
dindas privileged root;dockerd --storage-driver=vfs, because nested overlay storage fails with CRI-O;- the
privilegedSCC granted only to the generated runner ServiceAccount.
After the scale-set release creates its ServiceAccount, identify it before granting the SCC:
kubectl -n "$RUNNERS_NAMESPACE" get serviceaccount
oc adm policy add-scc-to-user privileged \
-z "${RUNNER_SCALE_SET_NAME}-gha-rs-no-permission" \
-n "$RUNNERS_NAMESPACE"
The generated ServiceAccount name can change with ARC chart naming. Use the
name reported by kubectl if it differs from the example. Do not treat the SCC
command alone as a complete OKD installation.
The ARC runner ServiceAccount needs no deployment RBAC. This does not reduce the node-level risk of privileged Docker-in-Docker. Provide deployment jobs a separately brokered, least-privilege kubeconfig.
Verify the scale set
Check that the destination, listener, and runner pods agree:
kubectl -n "$RUNNERS_NAMESPACE" get autoscalingrunnersets.actions.github.com \
-o custom-columns='NAME:.metadata.name,URL:.spec.githubConfigUrl,MIN:.spec.minRunners,MAX:.spec.maxRunners'
kubectl -n "$CONTROLLER_NAMESPACE" get pods
kubectl -n "$RUNNERS_NAMESPACE" get pods
The URL column must show the intended repository or organization. A listener
pod for the new scale set should run in the controller namespace. With
min_runners: 0, no runner pod is expected until a job is queued.
Add or run a workflow using the configured label:
jobs:
arc-check:
runs-on: okd-cluster-1
steps:
- run: uname -a
- run: docker info
If the job remains queued, check these boundaries in order:
- The workflow's
runs-onvalue matchesrunnerScaleSetNameexactly. spec.githubConfigUrlnames the repository or organization that owns the workflow.- The App installation ID belongs to that organization.
- The App has the required repository and organization permissions.
- A selected-repository App installation includes the target repository.
- The GitHub runner group permits the target repository.
- The listener logs show a session for the new scale set.
- Runner pod events show no SCC, scheduling, or image-pull failure.
export LISTENER_POD="$(kubectl -n "$CONTROLLER_NAMESPACE" get pods \
-l "actions.github.com/scale-set-name=$RUNNER_SCALE_SET_NAME,actions.github.com/scale-set-namespace=$RUNNERS_NAMESPACE" \
-o jsonpath='{.items[0].metadata.name}')"
kubectl -n "$CONTROLLER_NAMESPACE" logs "$LISTENER_POD"
kubectl -n "$RUNNERS_NAMESPACE" get events --sort-by=.lastTimestamp
Remove a destination
Stop workflows from targeting the scale set and wait for active jobs to finish. Then uninstall its Helm release and wait for ARC finalizers to remove the scale set:
helm uninstall "github-runner-${RUNNER_SCALE_SET_NAME}" \
--namespace "$RUNNERS_NAMESPACE"
kubectl -n "$RUNNERS_NAMESPACE" wait \
--for=delete "autoscalingrunnerset/${RUNNER_SCALE_SET_NAME}" \
--timeout=5m
Remove the runner-set Score from the private deployment crate so a later
convergence does not reinstall it. Delete RUNNERS_NAMESPACE only if it was
created for this scale set and contains no unrelated resources. Revoke the
GitHub App installation only after ARC cleanup if the organization no longer
uses the App. Do not remove the shared controller while other scale sets use
it.
GitHub's supported deployment model and Helm options are documented in Deploying runner scale sets with ARC.