Skip to content

Add health and readiness probes to controller manager #4459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Okabe-Junya wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add health and readiness probes to controller manager #4459

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions charts/actions-runner-controller/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,18 @@ spec:
name: metrics-port
protocol: TCP
{{- end }}
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
Comment thread
Okabe-Junya marked this conversation as resolved.
resources:
{{- toYaml .Values.resources | nindent 12 }}
securityContext:
Expand Down
12 changes: 12 additions & 0 deletions charts/gha-runner-scale-set-controller-experimental/templates/_controller_template.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,18 @@ args:
ports:
{{- toYaml $ports | nindent 2 }}
{{- end }}
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
Comment on lines +95 to +104
Copy link

Copilot AI Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These probes hard-code port 8081, but the manager now supports --health-probe-bind-address and this chart allows users to append arbitrary extraArgs. If a user sets a different bind address/port via extraArgs, the probes will start failing and repeatedly restart the pod. Suggest adding a chart value for the health probe bind address/port, wiring it into both the container args and the probe port field (or at least deriving the probe port from the configured bind address).

Suggested change
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
{{- $healthProbeBindAddress := ":8081" -}}
{{- with .Values.controller.manager.extraArgs }}
{{- if kindIs "slice" . }}
{{- range . }}
{{- if and (kindIs "string" .) (hasPrefix "--health-probe-bind-address=" .) }}
{{- $healthProbeBindAddress = trimPrefix "--health-probe-bind-address=" . -}}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- $healthProbePort := 8081 -}}
{{- $healthProbeBindParts := splitList ":" $healthProbeBindAddress -}}
{{- if gt (len $healthProbeBindParts) 1 }}
{{- $healthProbePort = (last $healthProbeBindParts | int) -}}
{{- end }}
livenessProbe:
httpGet:
path: /healthz
port: {{ $healthProbePort }}
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: {{ $healthProbePort }}

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Author

@Okabe-Junya Okabe-Junya Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since (technically) we can use extraArgs in experimental chart, this feeedback mighy be reasonable, but I think it would introduce unnecessary complexity and a lack of consistency with other charts.

But, if this change is necessary, I will create a fixup

initialDelaySeconds: 5
periodSeconds: 10
env:
- name: CONTROLLER_MANAGER_CONTAINER_IMAGE
value: "{{ .Values.controller.manager.container.image }}"
Expand Down
12 changes: 12 additions & 0 deletions charts/gha-runner-scale-set-controller/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,18 @@ spec:
protocol: TCP
name: metrics
{{- end }}
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
Comment thread
Okabe-Junya marked this conversation as resolved.
env:
- name: CONTROLLER_MANAGER_CONTAINER_IMAGE
value: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
Expand Down
12 changes: 12 additions & 0 deletions config/manager/manager.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,18 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8081
initialDelaySeconds: 5
periodSeconds: 10
Comment thread
Okabe-Junya marked this conversation as resolved.
volumeMounts:
- name: controller-manager
mountPath: "/etc/actions-runner-controller"
Expand Down
19 changes: 16 additions & 3 deletions main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ import (
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/healthz"
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
"sigs.k8s.io/controller-runtime/pkg/webhook"
// +kubebuilder:scaffold:imports
Expand Down Expand Up @@ -83,6 +84,7 @@ func main() {
listenerMetricsEndpoint string

metricsAddr string
probeAddr string
autoScalingRunnerSetOnly bool
enableLeaderElection bool
disableAdmissionWebhook bool
Expand Down Expand Up @@ -121,6 +123,7 @@ func main() {
flag.StringVar(&listenerMetricsAddr, "listener-metrics-addr", ":8080", "The address applied to AutoscalingListener metrics server")
flag.StringVar(&listenerMetricsEndpoint, "listener-metrics-endpoint", "/metrics", "The AutoscalingListener metrics server endpoint from which the metrics are collected")
flag.StringVar(&metricsAddr, "metrics-addr", ":8080", "The address the metric endpoint binds to.")
flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the health probe endpoint binds to.")
flag.BoolVar(&enableLeaderElection, "enable-leader-election", false,
"Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.")
flag.StringVar(&leaderElectionID, "leader-election-id", "actions-runner-controller", "Controller id for leader election.")
Expand Down Expand Up @@ -239,9 +242,10 @@ func main() {
SyncPeriod: &syncPeriod,
DefaultNamespaces: defaultNamespaces,
},
WebhookServer: webhookServer,
LeaderElection: enableLeaderElection,
LeaderElectionID: leaderElectionID,
WebhookServer: webhookServer,
HealthProbeBindAddress: probeAddr,
LeaderElection: enableLeaderElection,
LeaderElectionID: leaderElectionID,
Client: client.Options{
Cache: &client.CacheOptions{
DisableFor: []client.Object{
Expand All @@ -256,6 +260,15 @@ func main() {
os.Exit(1)
}

if err := mgr.AddHealthzCheck("healthz", healthz.Ping); err != nil {
log.Error(err, "unable to set up health check")
os.Exit(1)
}
if err := mgr.AddReadyzCheck("readyz", healthz.Ping); err != nil {
log.Error(err, "unable to set up ready check")
os.Exit(1)
}
Comment thread
Okabe-Junya marked this conversation as resolved.

if autoScalingRunnerSetOnly {
if err := actionsgithubcom.SetupIndexers(mgr); err != nil {
log.Error(err, "unable to setup indexers")
Expand Down
Loading