Netflix Bridges AWS IAM and Internal Identity with Workload Attestation

Netflix built a workload attestation mechanism that lets Apache Spark jobs on Amazon EMR exchange their AWS IAM execution role for a short-lived internal identity in its Metatron PKI. The Spark driver attests once and distributes credentials to executors over encrypted Spark RPC, while Data Projects map to IAM roles spread across a pool of dedicated AWS accounts.

• Data Projects map to dedicated IAM roles distributed across a pool of AWS accounts
• Identity service matches AWS STS GetCallerIdentity response against signed control-plane metadata
• Driver attests once and distributes credentials to executors via Spark RPC
• Driver re-attests periodically before its short-lived certificate expires