Kubernetes

This guide walks you through installing the Defakto Agent on a Kubernetes cluster using the official Helm chart. The Defakto Agent runs as a DaemonSet and provides SPIFFE identity services to workloads running in your cluster.

Prerequisites

Before installing the Defakto Agent, ensure you have:

A running Kubernetes cluster
kubectl configured to access your cluster
Helm 3.x installed
A deployed Trust Domain Server
Cluster registration details from your Trust Domain Server

Step 1 — Choose an agent attestation method

The recommended attestation method for Kubernetes clusters is the Kubernetes Service Account Token. No secret material is required; the agent presents its pod's service account token to authenticate.

See Agent Attestation Methods for the full list of available methods and server-side configuration.

Step 2 — Create the Helm values

Create a values.yaml file:

agent:
  endpoint:
    endpoint: "td-server.spirl.example.com"
  auth:
    clusterId: c-xxxxxx
    attestors:
      - type: k8s_token

Step 3 — Download and install the Helm chart

helm pull oci://ghcr.io/spirl/charts/spirl-system -d .

helm upgrade --install --namespace spirl-system \
  --create-namespace \
  --values ./values.yaml \
  spirl-system ./spirl-system-${versions.spirl_system_version}.tgz

You should see:

Release "spirl-system" does not exist. Installing it now.
NAME: spirl-system
LAST DEPLOYED: Tue Dec 19 14:02:09 2023
NAMESPACE: spirl-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

Step 4 — Verify

Check your target namespace for the Defakto Agent pods and verify that they are running.

When the agent is up and running, you'll see a health check confirmation log:

{
  "logger": "agent.readyChecker",
  "msg": "Health check passed - all systems operational"
}

Check the logs of the Defakto Agent pods to verify that they are connected to the Trust Domain Server:

kubectl -n spirl-system logs $(kubectl -n spirl-system get pods -o name | grep "spirl-agent" | head -n 1)

When a deployment is receiving and processing bundle updates successfully, you should see the following log message:

{
  "logger": "agent.bundleRefresher",
  "msg": "Bundle added or updated",
  "trustDomain": "spirl.example.com"
}

The trustDomain key will contain the name of your trust domain.

Disable TLS (optional)

By default, the agent and Trust Domain Server communicate over TLS. To disable TLS (for example, in lab environments), set the SPIRL_ENDPOINT_ENABLE_TLS environment variable in your values file:

agent:
  env:
    - name: SPIRL_ENDPOINT_ENABLE_TLS
      value: "false"

Working with Node Taints and Tolerations

If your Kubernetes cluster has tainted nodes that require tolerations for pod scheduling, you can configure tolerations for the Defakto Agent components. This is particularly useful when deploying on:

Nodes with dedicated workloads
Master/control plane nodes
Nodes with special hardware requirements
Nodes with specific scheduling constraints

Component Scheduling Requirements

Different Defakto components have different scheduling requirements:

Agent & CSI Driver: Must run on every node that hosts workloads requiring SPIFFE identities. These components need the same tolerations to ensure they can be scheduled on all workload nodes.
Controller: Only needs to run somewhere in the cluster and can be scheduled independently. You may want different (or no) tolerations for the controller, allowing it to run on dedicated management nodes.

Example: Agent and CSI Driver Tolerations

For workloads running on tainted nodes, configure the agent and CSI driver with matching tolerations:

agent:
  endpoint:
      endpoint: "td-server.spirl.example.com"
  auth:
      clusterId: c-xxxxxx
      attestors:
        - type: k8s_token
  tolerations:
      - key: "node.kubernetes.io/unschedulable"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "dedicated"
        operator: "Equal"
        value: "workload"
        effect: "NoSchedule"

csiDriver:
  tolerations:
      - key: "node.kubernetes.io/unschedulable"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "dedicated"
        operator: "Equal"
        value: "workload"
        effect: "NoSchedule"

Example: Controller with Different Tolerations

The controller can be configured with different tolerations, or run on untainted nodes:

# Option 1: Controller on management nodes
controller:
  tolerations:
      - key: "node-role.kubernetes.io/master"
      operator: "Exists"
      effect: "NoSchedule"

# Option 2: Controller with no specific tolerations (runs on any untainted node)
# controller: {}

# Option 3: Controller tolerating all taints
controller:
  tolerations:
      - operator: "Exists"

Common Toleration Patterns

Tolerate all taints:

agent:
  tolerations:
    - operator: "Exists"
csiDriver:
  tolerations:
    - operator: "Exists"

Tolerate specific key with any value:

agent:
  tolerations:
    - key: "dedicated"
      operator: "Exists"
      effect: "NoSchedule"
csiDriver:
  tolerations:
    - key: "dedicated"
      operator: "Exists"
      effect: "NoSchedule"

Tolerate specific key-value pair:

agent:
  tolerations:
    - key: "node-type"
      operator: "Equal"
      value: "compute"
      effect: "NoSchedule"
csiDriver:
  tolerations:
    - key: "node-type"
      operator: "Equal"
      value: "compute"
      effect: "NoSchedule"

A note for users of AWS EKS with Fargate

If you use AWS EKS with some workloads scheduled on Fargate, you may see some agent or CSI driver pods stuck in Pending if you use a broad toleration that matches the taint used by Fargate nodes. You can use node affinity in addition to your toleration to ensure that Fargate nodes are excluded. For example:

agent:
  tolerations:
    - operator: "Exists"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: eks.amazonaws.com/compute-type
            operator: NotIn
            values:
            - fargate
csiDriver:
  tolerations:
    - operator: "Exists"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: eks.amazonaws.com/compute-type
            operator: NotIn
            values:
            - fargate

Working with Custom Docker Registries

Many enterprise environments require pulling images from private registries or through registry proxies. The Defakto Agent Helm chart supports custom registries and image pull secrets for secure image access.

Registry Configuration

Defakto Agent uses multiple container images from different sources:

Defakto Images: Agent, Controller, and Reflector (default: ghcr.io/spirl)
SPIFFE CSI Driver: From the SPIFFE project (default: ghcr.io/spiffe)
Kubernetes CSI Registrar: From Kubernetes SIG Storage (default: registry.k8s.io/sig-storage)

You can configure each component to use your custom registry independently.

Creating Image Pull Secrets

Before deploying, create the required image pull secrets:

kubectl create secret docker-registry my-registry-secret \
  --namespace spirl-system \
  --docker-server=my-registry.company.com \
  --docker-username=myuser \
  --docker-password=mypassword

Configuration Examples

Private Registry with Authentication:

images:
  repository: "my-registry.company.com/spirl"
  pullSecrets:
    - name: "my-registry-secret"
  csiDriver:
    repository: "my-registry.company.com/spiffe"
  csiNodeDriverRegistrar:
    repository: "my-registry.company.com/sig-storage"

Registry Proxy/Mirror:

images:
  repository: "proxy.company.com/ghcr.io/spirl"
  csiDriver:
    repository: "proxy.company.com/ghcr.io/spiffe"
  csiNodeDriverRegistrar:
    repository: "proxy.company.com/registry.k8s.io/sig-storage"

Air-Gapped Environment (Pin Versions):

images:
  repository: "internal-registry.local/spirl"
  pullSecrets:
    - name: "internal-registry-secret"
  agent:
    tag: "v1.2.3"
  controller:
    tag: "v1.2.3"
  csiDriver:
    tag: "0.2.7"
    repository: "internal-registry.local/spiffe"
  csiNodeDriverRegistrar:
    tag: "v2.14.0"
    repository: "internal-registry.local/sig-storage"

Helm Values Reference

This section provides a comprehensive reference for all configuration options available in the Defakto Agent Helm chart. These values can be used to customize your Defakto Agent deployment to meet specific requirements.

Platform Configuration

platform: Platform type (default: "K8S"). Supported values:
- k8s - Standard Kubernetes
- istio - Kubernetes with Istio service mesh
- eks - Amazon EKS
- eks-istio - Amazon EKS with Istio
- linux - Linux environments

ArgoCD Configuration

argocd.enabled: Enable ArgoCD annotations for Kubernetes resources (default: false). When enabled, all Kubernetes resources will be annotated with ArgoCD-specific annotations that govern resource lifecycle management.

Images Configuration

images.repository: Base repository for all images (default: "ghcr.io/spirl")
images.pullPolicy: Image pull policy (default: "IfNotPresent")
images.pullSecrets: Array of image pull secrets (default: [])
images.pullCredentials: Not recommended, use images.pullSecrets instead. Array of registries and corresponding secrets in the form base64("username:password"). The array will be used to generate a dockerconfigjson secret that will be used as an imagePullSecret (default: [])
images.controller.name: Controller image name (default: "spirl-controller")
images.controller.tag: Controller image tag (default: Chart appVersion)
images.controller.repository: Controller-specific repository (optional)
images.agent.name: Agent image name (default: "spirl-agent")
images.agent.tag: Agent image tag (default: Chart appVersion)
images.agent.repository: Agent-specific repository (optional)
images.csiDriver.name: CSI driver image name (default: "spiffe-csi-driver")
images.csiDriver.tag: CSI driver image tag (default: "0.2.7")
images.csiDriver.repository: CSI driver repository (default: "ghcr.io/spiffe")
images.csiNodeDriverRegistrar.name: CSI node driver registrar image name (default: "csi-node-driver-registrar")
images.csiNodeDriverRegistrar.tag: CSI node driver registrar image tag (default: "v2.14.0")
images.csiNodeDriverRegistrar.repository: CSI node driver registrar repository (default: "registry.k8s.io/sig-storage")
images.reflector.name: Reflector image name (default: "reflector")
images.reflector.tag: Reflector image tag (default: Chart appVersion)
images.reflector.repository: Reflector-specific repository (optional)

Agent Configuration

Endpoint Configuration

agent.endpoint.endpoint: Primary trust domain server endpoint (default: "")
agent.endpoint.endpoints: Array of additional fallback endpoints (default: [])
agent.endpoint.supplementalRootsPEM: Additional root CAs for endpoint trust (PEM format)
agent.endpoint.connectionMaxAge: Maximum age for endpoint connections (default: "30m")
agent.endpoint.useGRPCFastRedial: Enable fast redialing for gRPC connections (default: true)

Authentication Configuration

Kubernetes Service Account Token

agent.auth.attestors: List of attestation methods. Add an entry with type: k8s_token to enable Kubernetes Service Account Token authentication.

Cluster Version Key (CVK)

agent.auth.key.id: Cluster version ID
agent.auth.key.pem: Private key for cluster authentication (PEM format)
agent.auth.suppliedSecretName: Name of pre-existing secret for authentication

Security and Resources

agent.runWithoutPrivileged: Run the agent without privileged: true
agent.supplementalRootsPEM: Additional trust anchors for trust bundle
agent.podSecurityContext: Pod security context configuration
agent.containerSecurityContext: Container security context configuration
agent.resources: Resource requests and limits
agent.priorityClassName: Priority class name for pod scheduling

Scheduling and Placement

agent.affinity: Node affinity rules
agent.tolerations: Array of node tolerations for tainted nodes (default: [])
agent.annotations: Map of pod annotations (default: {})
agent.additionalLabels: Map of additional pod labels (default: {})

Advanced Configuration

agent.env: Array of environment variables (default: [])
agent.additionalVolumes: Array of additional volumes (default: [])
agent.additionalVolumeMounts: Array of additional volume mounts (default: [])
agent.allowedAttributes: Array of allowed workload attributes (default: [])

Controller Configuration

controller.env: Array of environment variables (default: [])
controller.podSecurityContext: Pod security context configuration
controller.containerSecurityContext: Container security context configuration
controller.resources: Resource requests and limits
controller.affinity: Node affinity rules
controller.tolerations: Array of node tolerations (default: [])
controller.annotations: Map of pod annotations (default: {})
controller.priorityClassName: Priority class name

CSI Driver Configuration

csiDriver.env: Array of environment variables (default: [])
csiDriver.podSecurityContext: Pod security context configuration
csiDriver.containerSecurityContext: Container security context configuration
csiDriver.resources: Resource requests and limits
csiDriver.affinity: Node affinity rules
csiDriver.tolerations: Array of node tolerations (default: [])
csiDriver.annotations: Map of pod annotations (default: {})
csiDriver.priorityClassName: Priority class name

CSI Node Driver Registrar Configuration

csiNodeDriverRegistrar.env: Array of environment variables (default: [])
csiNodeDriverRegistrar.containerSecurityContext: Container security context configuration
csiNodeDriverRegistrar.resources: Resource requests and limits

Node Configuration

node.kubeletRootDirectory: Kubelet root directory path (default: /var/lib/kubelet)
node.spirlAgentSocketDirectory: Defakto agent socket directory path

Platform-Specific Configuration

Kubernetes (`k8s`)

k8s.spiffeSocketPath: SPIFFE socket path (default: "/run/spirl/sockets/agent.sock")
k8s.skipKubeletVerification: Skip kubelet verification (default: true)
k8s.kubeletHost: Kubelet host override
k8s.includeAnnotations: Array of pod annotation prefixes to include in attestation (default: [])
k8s.updateStrategy: DaemonSet update strategy configuration (default: RollingUpdate with maxUnavailable: 0, maxSurge: 10%)

Istio (`istio`)

istio.spiffeSocketPath: SPIFFE socket path (default: "/run/spirl/sockets/socket")
istio.skipKubeletVerification: Skip kubelet verification (default: true)
istio.socketVolumeName: Socket volume name (default: "workload-socket")
istio.socketVolumeMountPath: Socket volume mount path (default: "/run/secrets/workload-spiffe-uds")
istio.socketFileName: Socket file name (default: "socket")
istio.includeAnnotations: Array of pod annotation prefixes to include in attestation (default: [])

Amazon EKS (`eks`)

eks.spiffeSocketPath: SPIFFE socket path (default: "/run/spirl/sockets/agent.sock")
eks.includeAnnotations: Array of pod annotation prefixes to include in attestation (default: [])

Amazon EKS with Istio (`eksIstio`)

eksIstio.spiffeSocketPath: SPIFFE socket path (default: "/run/spirl/sockets/socket")
eksIstio.socketVolumeName: Socket volume name (default: "workload-socket")
eksIstio.socketVolumeMountPath: Socket volume mount path (default: "/run/secrets/workload-spiffe-uds")
eksIstio.socketFileName: Socket file name (default: "socket")
eksIstio.includeAnnotations: Array of pod annotation prefixes to include in attestation (default: [])

Telemetry Configuration

telemetry.enabled: Enable telemetry collection (default: false)
telemetry.metricsAPI.port: Port for metrics API exposure (default: 9090)
telemetry.collectors.grpc.emitLatencyMetrics: Enable gRPC latency metrics (default: false)
telemetry.collectors.workload.emitSVIDExpiry: Enable workload SVID expiry timestamp metrics (default: false)

Webhook Configuration

webhook.namespaceSelector: Namespace selector configuration for admission webhook (default: {})
webhook.objectSelector: Object selector configuration for admission webhook (default: matches labels with k8s.spirl.com/spiffe-csi: enabled)

Reflector Configuration

Basic Configuration

reflector.enabled: Enable reflector component (default: false)
reflector.additionalLabels: Map of additional labels for reflector pods (default: {})

Endpoint Configuration

reflector.endpoint.endpoint: Reflector endpoint URL
reflector.endpoint.endpoints: Array of fallback endpoints
reflector.endpoint.supplementalRootsPEM: Additional root CAs (PEM format)
reflector.endpoint.connectionMaxAge: Maximum connection age
reflector.endpoint.useGRPCFastRedial: Enable fast redialing

Service Configuration

reflector.service.type: Kubernetes service type (default: "ClusterIP")
reflector.service.port: Service port number (default: 9190)
reflector.service.annotations: Map of service annotations (default: {})

Deployment Configuration

reflector.deployment.replicaCount: Number of replicas (default: 2)
reflector.deployment.env: Array of environment variables (default: [])
reflector.deployment.resources: Resource requests and limits
reflector.deployment.affinity: Node affinity rules (default: {})

Pod Disruption Budget

reflector.deployment.podDisruptionBudget.enabled: Enable PDB (default: false)
reflector.deployment.podDisruptionBudget.minAvailable: Minimum available pods
reflector.deployment.podDisruptionBudget.maxUnavailable: Maximum unavailable pods

Horizontal Pod Autoscaler

reflector.deployment.hpa.enabled: Enable HPA (default: false)
reflector.deployment.hpa.minReplicas: Minimum number of replicas (default: 2)
reflector.deployment.hpa.maxReplicas: Maximum number of replicas (default: 7)
reflector.deployment.hpa.targetCPUUtilizationPercentage: Target CPU utilization (default: 50)
reflector.deployment.hpa.behavior: HPA behavior configuration

Authentication

reflector.auth.key.id: Cluster version ID for reflector
reflector.auth.key.pem: Private key for reflector authentication
reflector.auth.suppliedSecretName: Pre-existing secret name

Cloud Provider Configuration

AWS Configuration

reflector.aws.kmsKeyARN: AWS KMS key ARN for encryption
reflector.aws.endpoints.kms: Custom KMS endpoint URL

Azure Configuration

reflector.azure.keyVault.url: Azure Key Vault URL
reflector.azure.keyVault.keyName: Key name in Key Vault
reflector.azure.keyVault.keyVersion: Key version (optional)

GCP Configuration

reflector.gcp.projectID: GCP project ID
reflector.gcp.location: GCP location/region
reflector.gcp.kms.keyRing: KMS key ring name
reflector.gcp.kms.keyName: KMS key name

Experimental Features

experimental.supplementalAttestation.jwt.issuer: JWT attestation issuer
experimental.supplementalAttestation.jwt.jwksUrl: JWKS URL for JWT validation
experimental.supplementalAttestation.jwt.supplementalRootsFile: Supplemental roots file path
experimental.useLegacyEnvoyValidation: Use legacy Envoy validation (default: false)

Prerequisites​

Step 1 — Choose an agent attestation method​

Step 2 — Create the Helm values​

Step 3 — Download and install the Helm chart​

Step 4 — Verify​

Disable TLS (optional)​

Working with Node Taints and Tolerations​

Example: Agent and CSI Driver Tolerations​

Example: Controller with Different Tolerations​

Common Toleration Patterns​

Working with Custom Docker Registries​

Creating Image Pull Secrets​

Configuration Examples​

Helm Values Reference​

Platform Configuration​

ArgoCD Configuration​

Images Configuration​

Agent Configuration​

Endpoint Configuration​

Authentication Configuration​

Kubernetes Service Account Token​

Cluster Version Key (CVK)​

Security and Resources​

Scheduling and Placement​

Advanced Configuration​

Controller Configuration​

CSI Driver Configuration​

CSI Node Driver Registrar Configuration​

Node Configuration​

Platform-Specific Configuration​

Kubernetes (k8s)​

Istio (istio)​

Amazon EKS (eks)​

Amazon EKS with Istio (eksIstio)​

Telemetry Configuration​

Webhook Configuration​

Reflector Configuration​

Basic Configuration​

Endpoint Configuration​

Service Configuration​

Deployment Configuration​

Pod Disruption Budget​

Horizontal Pod Autoscaler​

Authentication​

Cloud Provider Configuration​

AWS Configuration​

Azure Configuration​

GCP Configuration​

Experimental Features​

Prerequisites

Step 1 — Choose an agent attestation method

Step 2 — Create the Helm values

Step 3 — Download and install the Helm chart

Step 4 — Verify

Disable TLS (optional)

Working with Node Taints and Tolerations

Example: Agent and CSI Driver Tolerations

Example: Controller with Different Tolerations

Common Toleration Patterns

Working with Custom Docker Registries

Creating Image Pull Secrets

Configuration Examples

Helm Values Reference

Platform Configuration

ArgoCD Configuration

Images Configuration

Agent Configuration

Endpoint Configuration

Authentication Configuration

Kubernetes Service Account Token

Cluster Version Key (CVK)

Security and Resources

Scheduling and Placement

Advanced Configuration

Controller Configuration

CSI Driver Configuration

CSI Node Driver Registrar Configuration

Node Configuration

Platform-Specific Configuration

Kubernetes (`k8s`)

Istio (`istio`)

Amazon EKS (`eks`)

Amazon EKS with Istio (`eksIstio`)

Telemetry Configuration

Webhook Configuration

Reflector Configuration

Basic Configuration

Endpoint Configuration

Service Configuration

Deployment Configuration

Pod Disruption Budget

Horizontal Pod Autoscaler

Authentication

Cloud Provider Configuration

AWS Configuration

Azure Configuration

GCP Configuration

Experimental Features