Provider Attestation

When the spirl-agents authenticate to the spirl-servers, besides checking they own the cluster-version keys, there’s an optional attestation process that verifies the agents are running on authorized hardware - specifically, the cloud provider provisioned VMs.

This extra step also makes the provider attributes available to the SPIFFE ID template, to incorporate cloud-provider-specific attributes into the SPIFFE IDs.

This doc covers how to set up this attestation for the SPIRL clusters.

AWS IMDSv2 attestation

AWS attestation is done by the Instance Metadata Service Version 2 (IMDSv2), where the spirl-servers challenges the agents to provide the instance identity document, and validates it.

Permission setup

The spirl-servers need permission to query the EC2 instance states in the AWS account where the spirl-agents are running. Since the spirl-servers and spirl-agents typically run in separate AWS accounts, this requires setting up cross-account IAM role assumption.

Note: In this doc, we will use the following placeholders

• <spirl-server-aws-account-id>: the spirl-server AWS account
• <spirl-server-role>: the name of the role assigned to the spirl-servers
• <spirl-agent-aws-account-id>: the spirl-agent AWS account (this is also where the workloads are)
• <spirl-attestor-role>: the name of the role with the attestor permissions
• <trust-domain-id>: the trust-domain ID

The spirl-agent AWS account

This is the same AWS account where the workloads are running.

A role <spirl-attestor-role> needs to be created in this account with the following:

• Permissions to query EC2 instances

{
    "Statement": [
        {
            "Action": "ec2:DescribeInstances",
            "Effect": "Allow",
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}

• A trust policy that allows assumption by the spirl-server's IAM role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<spirl-server-aws-account-id>:root"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "<trust-domain-id>"
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<spirl-server-aws-account-id>:root"
            },
            "Action": "sts:TagSession"
        }
    ]
}

Note: the trust policy above delegates management of what principals can assume the role to the spirl-server aws account. For example, if there's an IAM user in the spirl-server aws account, it can also assume the role if the spirl-server aws account grants it that permission. This covers the cases like in eks, the from-role is dynamic, e.g. "arn:aws:sts::<account-id>:assumed-role/<role-name>/<dynamic-session-name>". Principals like "AWS": "arn:aws:iam::<spirl-server-aws-account-id>:role/<spirl-server-role>" won't work for those cases.

The spirl-server AWS account

This is assuming the spirl-servers already run with existing IAM roles, e.g. configured by EKS Pod Identity, IAM roles for service accounts, or AWS access keys.

Add the following to the role for explicit permission to assume the <spirl-attestor-role> in the spirl-agent account. Note that both the account-ID and the role-name must match what's created in the spirl-agent account.

{
    "Statement": [
        {
            "Action": [
                "sts:TagSession",
                "sts:AssumeRole"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:iam::<spirl-agent-aws-account-id>:role/<spirl-attestor-role>"
        }
    ],
    "Version": "2012-10-17"
}

Enable AWS provider attestation in SPIRL

Create a provider attestation config

This requires the ARN of the <spirl-attestor-role> role in the spirl-agent account.

spirlctl attestation provider create <attestor-name> --type aws --attestor-role-arn arn:aws:iam::<spirl-agent-aws-account-id>:role/<spirl-attestor-role>

Create a cluster/node-group with the provider attestor

spirlctl node-group add <ng-name> --trust-domain <trust-domain-name> --provider-attestation <attestor-name>

spirlctl cluster register <c-name> --trust-domain <trust-domain-name> --platform k8s --provider-attestation <attestor-name>

Use the AWS specific attributes in the SPIFFE ID template

With AWS attestation turned on, the SPIFFE ID template can use the following AWS specific attributes:

• provider.aws.region
• provider.aws.account_id
• provider.aws.instance_id
• provider.aws.tags.<TAG_NAME>

For example

spirlctl node-group add <ng-name> --trust-domain <trust-domain-name> --provider-attestation <attestor-name> --path-template "/prefix/{{provider.aws.account_id}}/{{provider.aws.region}}/{{provider.aws.instance_id}}"

Or to create a new config to update the template

spirlctl node-group config change-path-template <ng-name> "/prefix/{{provider.aws.account_id}}/{{provider.aws.region}}/{{provider.aws.instance_id}}" --trust-domain <trust-domain-name>

Troubleshooting

Agent failing with code = Internal desc = failed to get a platform verifier

This means the role-assumption is misconfigured. Check the spirl-server logs for more detailed errors.

One commonly missed config is "sts:ExternalId": "<trust-domain-id>". Make sure it matches the trust-domain servers that the agents are talking to.