sdlc.dev

Kubernetes path · State, security and extension · Topic 13 of 13

Operators and custom resources

Not started

Everything built into Kubernetes follows one pattern: an API type that stores desired state, and a controller that reconciles reality to match it. Custom resources let you add your own types to the API, and an operator is a controller for such a type that encodes how to run a particular piece of software: creating a database cluster, failing over, taking backups, rotating certificates, upgrading safely. It is the same control loop as a Deployment, applied to your domain.

Why it matters: Databases, certificates, message queues and GitOps tools all arrive in clusters as operators. Knowing they are just CRDs plus a control loop tells you how to use, debug and judge them.

  1. 1A CRD adds a new kind

    A CustomResourceDefinition teaches the API server a new type, with a schema; kubectl, RBAC and watches then work on it like any built-in object.

  2. 2A controller gives it meaning

    Without a controller a custom resource is only stored data. The controller watches it and reconciles the world to match its spec.

  3. 3An operator encodes operations

    An operator is a controller with domain knowledge: provisioning, failover, backups, upgrades for one kind of software.

  4. 4It runs with power

    Operators often hold broad RBAC rights across the cluster; choose maintained ones and review what they can do.

1

How it works

The explanation, with examples you can run.

Step 1: a new type

apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:  name: backups.example.comspec:  group: example.com  scope: Namespaced  names:    kind: Backup    plural: backups    singular: backup  versions:    - name: v1      served: true      storage: true      subresources:        status: {}      schema:        openAPIV3Schema:          type: object          properties:            spec:              type: object              required: [database, schedule]              properties:                database: { type: string }                schedule: { type: string }                keep: { type: integer, minimum: 1 }            status:              type: object              properties:                lastSuccess: { type: string, format: date-time }

Once the CRD is applied, the API server serves /apis/example.com/v1/namespaces/*/backups. Objects are validated against the schema, stored in etcd, can be listed and watched, and are covered by RBAC like everything else:

apiVersion: example.com/v1kind: Backupmetadata:  name: shop-nightlyspec:  database: db  schedule: "0 2 * * *"  keep: 7
kubectl get backups

Step 2: a controller

So far nothing happens: a custom resource is just data. The behaviour lives in a controller, usually a Deployment in the cluster built with a framework such as Kubebuilder or controller-runtime (Go), the Operator SDK, or kopf (Python). Its core is a reconcile function, called whenever a watched object changes and periodically:

reconcile(backup):  desired = backup.spec                         # what the user asked for  actual  = look at the CronJob, the bucket, recent runs  if the CronJob is missing or differs: create or update it  delete backups beyond spec.keep  write backup.status.lastSuccess               # report what was observed

Good reconcilers are idempotent (running twice changes nothing more), level-triggered (they act on the current state, not on a sequence of events), and set owner references on what they create, so deleting the Backup garbage-collects its CronJob. Finalizers let a controller clean up external resources, such as a cloud bucket, before the object disappears.

Operators you will meet

  • cert-manager: Certificate objects become TLS secrets that renew themselves.
  • Database operators such as CloudNativePG or the operators for MySQL, Kafka (Strimzi) and Redis: a Cluster object becomes StatefulSets, Services, replication, failover and backups.
  • Argo CD and Flux: Application or Kustomization objects keep the cluster in sync with git.
  • Prometheus Operator: ServiceMonitor objects become scrape configuration.
  • Gateway API itself ships as CRDs with controllers from each implementation.

Judging an operator

An operator is privileged automation running inside your cluster, often with cluster-wide rights. Before adopting one, check that it is maintained and widely used, what RBAC it requests, how it upgrades its own CRDs, and what happens to your data if you uninstall it. When writing one, keep the scope small: if a Helm chart or a CronJob does the job, you may not need a controller at all.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.You install only the CRD for backups.example.com, not its controller, and create a Backup object. What happens?
  2. 2.What does a controller do on each reconcile?
  3. 3.Why prefer an operator such as CloudNativePG over a bare StatefulSet for Postgres?

0 of 3 right so far. Get all three to finish this topic.

Go deeper