Kubernetes cluster management

This guide covers the complete lifecycle of managing Kubernetes (K8s) clusters with spirlctl, including adding clusters to trust domains, configuring them, monitoring their status, and removing them.

Cluster Lifecycle Overview

A typical workflow includes:

1. Registering - Register a cluster to a trust domain
2. Configure - Customize cluster settings (e.g. path templates)
3. Monitor - View cluster status, workloads, and configurations
4. Disable - Disable cluster configurations before removal
5. Delete - Remove the cluster from the trust domain

Registering clusters to a trust domain

"register" vs. "add"

There are two ways to add a cluster to a trust domain:

• spirlctl cluster register - Creates the cluster metadata in the trust domain and returns credentials, but does not deploy anything to the cluster. It is the method used in the Defakto Terraform provider.
• spirlctl cluster add - In addition to the steps in spirlctl cluster register, this command also deploys Defakto components to the cluster. It is the method used in the Quick Start example.

In nearly all cases, you will want to use spirlctl cluster register to register a cluster to the trust domain, so this documentation is focused on that method.

For options on installing the SPIRL Server into a K8s cluster, see the Kubernetes instructions.

Using cluster register

To register a cluster with a trust domain and deploy it later (e.g. via Helm chart):

spirlctl cluster register workloads-cluster \
  --trust-domain example.com \
  --platform k8s

The output includes critical information to save. Here is an example:

Cluster registered successfully!

Cluster Version ID: cv-abc123def456
Private Key:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC...
-----END PRIVATE KEY-----

warning

Save these credentials securely! You'll need them to configure your self-hosted agent. The private key cannot be retrieved later.

Output Formats

For programmatic processing, use JSON output:

spirlctl cluster register workloads-cluster \
  --trust-domain example.com \
  --platform k8s \
  --output json

The output will include the following fields:

{
  "cluster_name": "workloads-cluster",
  "cluster_version_id": "cv-9xa1bk0e66",
  "cluster_private_key": "-----BEGIN PRIVATE KEY-----\nMC4CAQAwBQYDK2VwBCIEIDO0RfKhdKYP01u9Tc7LKxyjm4Jfk7vh1XLhKVcpaF0s\n-----END PRIVATE KEY-----\n",
  "trust_domain": "example.com",
}

As before, the cluster private key is only output once, and is required later.

Listing Clusters

List All Clusters

To list clusters across all trust domains, use the following command:

spirlctl cluster list

Here is example output:

Name                     ID            Configs  Trust Domain             Created
p2i-application          c-8mdxko5rzb  1        hardened.example.com     2025-10-21T21:22:26Z
development              c-ch2er74c0d  1        example.com              2025-08-26T22:43:55Z
staging                  c-qrq9akemxy  1        example.com              2025-10-21T17:24:53Z
data-analysis            c-xmhcdqme4k  1        example.com              2025-08-27T19:44:22Z
production-us-west-2     c-xvb70zwob2  1        example.com              2025-10-21T21:11:10Z
production-eu-central-1  c-xx6owoyoom  1        example.com              2025-10-21T21:10:46Z
integration-testing      c-yhgvdkk31x  4        example.com              2025-08-29T19:19:23Z

7 clusters found.

Filter clusters by Trust Domain

Clusters can also be listed for a single trust domain:

spirlctl cluster list --trust-domain example.com

Here is sample output:

Name             ID            Configs  Created
p2i-application  c-8mdxko5rzb  1        2025-10-21T21:22:26Z

Viewing Cluster Details

Get Cluster Information

Use cluster info to view detailed information about a specific cluster:

spirlctl cluster info p2i-application --trust-domain hardened.example.com

This displays:

• Cluster ID
• Creation timestamp
• Active configuration versions
• Trust domain association
• Count of active agents
• Count of active workload IDs
• Cluster path template

Example output:

ID:                      c-8mdxko5rzb
Cluster:                 p2i-application
Trust Domain:            hardened.example.com (td-888butgzdc)
Active agents:           0
Active workload IDs:     0
Last Used:               1970-01-01 00:00:00 +0000 UTC

Configs (1)
ID             Created                            Active agents  Active workload IDs  Last Used                      Platform    Path Template
cv-l0qy4x6luq  2025-10-21 21:22:26.791 +0000 UTC  0              0                    1970-01-01 00:00:00 +0000 UTC  Kubernetes  /{{cluster.name}}/ns/{{kubernetes.pod.namespace}}/sa/{{kubernetes.pod.service_account}}

Cluster Configuration

View Cluster Configurations

List all configuration versions for a cluster:

spirlctl cluster config list integration-testing --trust-domain example.com

Example output:

ID             Enabled  Created                            Active agents  Active workload IDs  Last Used                      Platform    Path Template
cv-hi7azkawm2  true     2025-08-29 23:45:01.332 +0000 UTC  1              0                    2025-10-21 23:40:20 +0000 UTC  Kubernetes  /{{kubernetes.pod.namespace}}/{{kubernetes.pod.label.app}}
cv-srokiwo9cm  true     2025-08-29 23:39:55.708 +0000 UTC  0              0                    2025-08-29 23:43:22 +0000 UTC  Kubernetes  /kubernetes.pod.namespace/{{kubernetes.pod.label.app}}
cv-t8qssrmu64  true     2025-08-29 22:58:55.46 +0000 UTC   0              0                    1970-01-01 00:00:00 +0000 UTC  Kubernetes  /{{kubernetes.pod.namespace}}/{{kubernetes.pod.container.name}}
cv-b2zoaqo0m0  true     2025-08-29 19:30:03.112 +0000 UTC  0              0                    2025-08-29 22:56:46 +0000 UTC  Kubernetes  /kubernetes.pod.namespace/{{kubernetes.pod.service_account}}/{{kubernetes.pod.uid}}/{{kubernetes.pod.name}}/{{kubernetes.pod.node.name}}

4 configs found.

Customization Options

When adding or registering a cluster, you can customize various settings:

Path Templates

Control how SPIFFE IDs are structured for workloads:

spirlctl cluster register production \
  --trust-domain example.com \
  --platform k8s \
  --path-template "/{{cluster.name}}/ns/{{kubernetes.pod.namespace}}/sa/{{kubernetes.pod.service_account}}"

See SPIFFE ID Templates for more details.

X.509 SVID Customization

Customize the Subject and SANs for X.509 certificates:

spirlctl cluster register production \
  --trust-domain example.com \
  --platform k8s \
  --x509-customization-template "CN=CustomCN"

See X.509 SVID Customization for more details and the template format.

JWT SVID Customization

Add custom claims to JWT tokens:

spirlctl cluster register production \
  --trust-domain example.com \
  --platform k8s \
  --jwt-customization-template "claim_name={{kubernetes.pod.uid}},another_claim={{kubernetes.pod.namespace}}"

See JWT SVID Customization for more details and the template format.

Provider Attestation

See Provider Attestation for available options.

Updating a cluster configuration

Cluster configuration changes are immutable, so when any value changes the Defakto control plane creates a new configuration version. This new version must be applied to the agent before it takes effect. Here is an example of updating the path template:

$ spirlctl cluster config change-path-template production \
   "/ns/{{kubernetes.pod.namespace}}/uid/{{kubernetes.pod.uid}}" \
   --trust-domain example.com

Success! To prevent your agents using the old version of the path template, they must be updated to use the following private key.
If you do not update your agents, they will still be able to use the old identify format. If something goes wrong, you can switch back to using an old key.
To view keys available to revert back to use 'spirlctl cluster key list CLUSTER_NAME'
Any agents on older config versions will still be able to authenticate until these config versions are blocked.
To block a config version use 'spirlctl cluster config disable CLUSTER_NAME --config-version KEY_VERSION_ID'

Cluster: production
Cluster Version ID: cv-4vknt6lpp6
Private Key:
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEII5XgpmpzYc9+dkfgOFwZl2jlyEtMkUlSx6cRTwXkUmS
-----END PRIVATE KEY-----

Now there are two configurations:

$ spirlctl cluster config list workloads-example --trust-domain example.com
ID             Enabled  Created                            Active agents  Active workload IDs  Last Used                      Platform    Path Template                                                                            Provider Attestation
cv-4vknt6lpp6  true     2025-10-22 19:11:49.205 +0000 UTC  0              0                    1970-01-01 00:00:00 +0000 UTC  Kubernetes  /ns/{{kubernetes.pod.namespace}}/uid/{{kubernetes.pod.uid}}                              n/a
cv-x29tt0nyji  true     2025-10-21 21:10:46.835 +0000 UTC  0              0                    1970-01-01 00:00:00 +0000 UTC  Kubernetes  /{{cluster.name}}/ns/{{kubernetes.pod.namespace}}/sa/{{kubernetes.pod.service_account}}  n/a

2 configs found.

All agents must be updated to use the new key which will apply the new configuration. After the agent restarts, the change takes effect.

Change Path Template

See SPIFFE ID Templates for more details:

spirlctl cluster config change-path-template production \
   "/ns/{{kubernetes.pod.namespace}}/uid/{{kubernetes.pod.uid}}" \
   --trust-domain example.com

Change X.509 Template

See X.509 SVID Customization for more details and the template format.

spirlctl cluster config change-x509-template production \
  "CN=CustomCN" --trust-domain example.com

Change JWT Template

See JWT SVID Customization for more details and the template format.

spirlctl cluster config change-jwt-template production \
  "claim_name={{kubernetes.pod.uid}},another_claim={{kubernetes.pod.namespace}}"
  --trust-domain example.com

Configuration Version Management

Configurations are enabled by default. Before they can be deleted, they must be disabled. This step allows you to verify that all the agents have picked up the new configuration. A disabled configuration can be re-enabled.

Disabling a Configuration

Note that the --trust-domain flag is not required for either of these commands.

spirlctl cluster config disable production \
  --config-id cv-abc123

To re-enable a disabled configuration:

spirlctl cluster config enable production \
  --config-id cv-abc123

Delete a Configuration

danger!

It is not possible to undo configuration deletion.

Once a configuration has been disabled, it can be deleted:

spirlctl cluster config delete production \
  --config-id cv-abc123

Key Rotation

Rotate the cluster's private key to generate a new key pair:

spirlctl cluster config rotate-key production \
  --trust-domain example.com

This creates a new cluster configuration version with a new key pair. The output includes the new private key, which must be saved securely.

tip

Key rotation is a critical security practice. Rotate keys regularly and after any suspected compromise.

Viewing Cluster Workloads

List all workloads running in a cluster:

spirlctl cluster workloads list production --trust-domain example.com

This shows known workload identities issued for a cluster, grouped by active, expiring, and expired status. The following attributes are included:

• Type (JWT or X.509)
• SPIFFE ID
• Number issued for each ID
• Date last issued

An expiring workload is one which will expire in the next 6 hours.

Removing a cluster

Disable a Cluster

warning

If a cluster is disabled in error, each configuration must be individually re-enabled.

Before deleting a cluster, disable all its configurations:

spirlctl cluster disable production --trust-domain example.com

This disables all active cluster configs, preventing new SVID issuance. Existing SVIDs will continue to work until they expire.

Delete a Cluster

danger!

It is not possible to undo Defakto cluster deletion. Workloads running on the Kubernetes cluster can continue to use previously-issued SVIDs until expiration, but will not receive new ones. Ensure workloads are migrated or stopped before deleting.

Remove the cluster from the trust domain:

spirlctl cluster delete production --trust-domain example.com

If the cluster has active configurations, you'll receive an error. Use the --force flag to override:

spirlctl cluster delete production \
  --trust-domain example.com \
  --force

danger!

Force deletion bypasses all safety mechanisms and deletes clusters immediately.

Common Workflows

Registering an EKS cluster with custom settings

spirlctl cluster register eks-prod \
  --trust-domain prod.example.com \
  --platform eks \
  --description "Production EKS cluster in us-east-1" \
  --path-template "/ns/{{kubernetes.pod.namespace}}/uid/{{kubernetes.pod.uid}}" \

Registering an Istio cluster

spirlctl cluster register istio-mesh \
  --trust-domain example.com \
  --platform istio \
  --description "Production Istio service mesh"

Generate deployment manifests

Running spirlctl cluster add with the --no-deploy flag is useful for generating Kubernetes manifests:

# Generate manifests without deploying
spirlctl cluster add production \
  --trust-domain example.com \
  --platform k8s \
  --no-deploy

# Review generated files
ls -l spirl-*.yaml

Troubleshooting

Cluster Shows Inactive Status

Check cluster configuration:

spirlctl cluster info production --trust-domain example.com
spirlctl cluster config list production --trust-domain example.com

Ensure at least one config is enabled:

spirlctl cluster config enable production \
  --trust-domain example.com \
  --config-id cv-abc123

Workloads Not Receiving SVIDs

1. Check cluster is active: spirlctl cluster info production --trust-domain example.com
2. Verify cluster has active configurations
3. Verify that the Agent is running
4. Review agent logs for errors

Cannot Delete Cluster

danger!

It is not possible to undo cluster deletion. Ensure workloads are migrated or stopped before deleting.

If you get an error about active versions:

# Disable all configs first
spirlctl cluster disable production --trust-domain example.com

# Then delete
spirlctl cluster delete production --trust-domain example.com

Or use force deletion (with caution):

spirlctl cluster delete production \
  --trust-domain example.com \
  --force

Best Practices

1. Use Descriptive Names: Name clusters clearly (e.g., prod-us-east-1, staging-eu-west-1)
2. Add Descriptions: Include --description for easier management
3. Secure Private Keys: Export and securely store cluster private keys.
4. Regular Key Rotation: Rotate cluster keys periodically for security
5. Test Path Templates: Verify path templates match your workload patterns before deploying
6. Monitor Cluster Status: Check cluster health and usage with cluster info and cluster list
7. Disable Before Delete: Always disable clusters and configurations before deletion to verify expected operation, and gracefully handle SVID expiration.

Command Reference

Cluster Commands

Command	Description
cluster add	Add cluster and deploy SPIRL components
cluster register	Register cluster without deploying
cluster list	List all clusters
cluster info	Get detailed cluster information
cluster disable	Disable all cluster configurations
cluster delete	Delete cluster from trust domain
cluster config	Manage cluster configurations
cluster workloads	View cluster workloads

Cluster Config Commands

Command	Description
config list	List cluster configurations
config enable	Enable a configuration
config disable	Disable a configuration
config delete	Delete a configuration
config rotate-key	Rotate cluster key pair
config change-path-template	Update path template
config change-x509-template	Update X.509 customization
config change-jwt-template	Update JWT customization

Next Steps

• Configure AWS Federation
• Set up CI/CD Integration
• Configure Istio Integration
• Customize Path Templates
• Customize X.509 SVIDs
• Customize JWT SVIDs