Enforce artifact compliance with HCP Terraform
Using Terraform data sources in your configuration to retrieve artifacts lets your configuration dynamically use more up-to-date artifacts as you create them and stops you from using revoked artifacts. The resource artifact validation HCP Terraform run task feature helps you implement this best practice by scanning your Terraform configuration to prevent usage of revoked artifacts, even when they are referenced by hard-coded image IDs.
Note
HCP Terraform Free Edition includes one run task integration that you can apply to up to ten workspaces. Refer to HCP Terraform pricing for details.
The resource artifact validation run task feature parses your Terraform resources for machine image IDs and checks if artifacts are tracked by HCP Packer. If the artifact is associated with an artifact version, the run task will fail if it is a revoked version. Additionally, it will warn you if the artifact is scheduled for revocation. The run task also detects any hard-coded image IDs, and prompts users to use the HCP Packer data sources to better track and manage artifacts. This encourages your organization to track artifact lifecycles in HCP Packer and dynamically query artifact IDs from artifact channels with Terraform data sources.
Note
The resource artifact validation currently supports this list of resources.
In this tutorial, you will use the HCP Terraform run task for HCP Packer to enforce Terraform configuration compliance. You will do this by associating the run task with an HCP Terraform workspace. Then, you will iteratively update the workspace's configuration until the resource artifact validation reports that the configuration uses compliant artifacts.
Prerequisites
This tutorial assumes that you are familiar with:
- The standard Packer and HCP Packer workflows. If you are new to Packer, complete the Get Started tutorials first. If you are new to HCP Packer, complete the Get Started HCP Packer tutorials first.
- The Terraform and HCP Terraform plan/apply workflows. If you are new to Terraform itself, refer first to the Getting Started tutorials. If you are new to HCP Terraform, refer to the Get Started - HCP Terraform tutorials.
To follow along with this tutorial, you will need:
- Packer 1.10.1+ installed locally
- Terraform 1.1.7 or later installed locally
- An AWS account with credentials set as local environment variables
- A HCP account with the HCP Packer Plus tier
- An HCP Terraform account.
- HCP Terraform workspace admin permissions to associate run tasks to a workspace.
This tutorial relies on an HCP Terraform run task integrated with HCP Packer. If you have not yet created one, follow the Set Up HCP Terraform Run Task for HCP Packer tutorial to do so.
Create HCP service principal and set as environment variables
In HCP Packer, go to Access control (IAM) in the left navigation menu, then select the Service principals tab.
Create a service principal named packer
with the Contributor role.
Once you create the service principal, HCP shows you a detailed overview page. Click Keys in the left navigation bar, then click Generate key to create a client ID and secret.
Copy and save the client ID and secret; you will not be able to retrieve the secret later. You will use these credentials in the next step.
Once you generate the keys for the service principal, set the client ID and secret as environment variables so that Packer can authenticate with HCP.
In your terminal, set an environment variable for the client ID.
$ export HCP_CLIENT_ID=
Then, set an environment variable for the client secret.
$ export HCP_CLIENT_SECRET=
Next, navigate to your HCP project settings page to get your project's ID.
Use this value to set an environment variable for your project's ID.
$ export HCP_PROJECT_ID=
Login to HCP Terraform
In this tutorial, you will use the Terraform CLI to create an HCP Terraform workspace and trigger remote plan and apply runs.
Log into your HCP Terraform account in your terminal.
$ terraform login
Terraform will request an API token for app.terraform.io using your browser.
If login is successful, Terraform will store the token in plain text in
the following file for use by subsequent commands:
/Users/<USER>/.terraform.d/credentials.tfrc.json
Do you want to proceed?
Only 'yes' will be accepted to confirm.
Enter a value:
Confirm with a yes
and follow the workflow in the browser window that automatically opens. Paste the generated API key into your Terminal when prompted. For more detailed instructions on logging in, review the Authenticate the CLI with HCP Terraform tutorial.
Clone repository
In your terminal, clone the example repository. This repository contains a Packer template that defines an Ubuntu AMI and two directories with Terraform configuration that you will use to test the run task.
$ git clone https://github.com/hashicorp-education/learn-hcp-packer-run-tasks
Navigate to the cloned repository.
$ cd learn-hcp-packer-run-tasks
Create artifact version in HCP Packer
Open ubuntu-focal.pkr.hcl
to review the template. This template will build an Ubuntu 20.04 AMI in the us-east-2
region and push the metadata to the learn-packer-run-tasks
in HCP Packer.
ubuntu-focal.pkr.hcl
build {
hcp_packer_registry {
bucket_name = "learn-packer-run-tasks"
## ...
}
sources = [
"source.amazon-ebs.basic-example-east"
]
}
Initialize your Packer template.
$ packer init .
Now, build your artifact.
$ packer build ubuntu-focal.pkr.hcl
Tracking build on HCP Packer with fingerprint "01HMH2E8ENJR3NB6PBJBT2F69B"
amazon-ebs.basic-example-east: output will be in this color.
==> amazon-ebs.basic-example-east: Prevalidating any provided VPC information
==> amazon-ebs.basic-example-east: Prevalidating AMI Name: packer_AWS_1705675008_v1.0.0
amazon-ebs.basic-example-east: Found Image ID: ami-03...
# ...
==> Wait completed after 4 minutes 34 seconds
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs.basic-example-east: AMIs were created:
us-east-2: ami-0c16bec5441261947
--> amazon-ebs.basic-example-east: Published metadata to HCP Packer registry packer/learn-packer-run-tasks/versions/01HMH2E9VE1RQ20E2CTV9SX4ZT
In your HCP dashboard, go to the learn-packer-run-tasks
bucket to confirm Packer pushed the build metadata to HCP Packer.
Create channel and schedule revocation
On the Channels page, create a channel named production
and set it to the first version.
Next, go to the Versions page. Schedule a revocation date for the first version by clicking on ..., then Revoke version.
Select Revoke at a future date and enter the time for 1 minute from your current time. The time is in UTC (current time in UTC). For example, if it is currently 10:00
, enter 10:01
. Then, enter Assign artifact channel to revoked version
for the revocation reason, then click Revoke Version to revoke the version.
You are setting a short revocation window so that your artifact channel uses a revoked artifact to test validation workflows. This is for the educational purposes of the tutorial.
Set up HCP Terraform workspace
Go to the tf-resource-validation
directory. This directory contains Terraform configuration that you will use to create your HCP Terraform workspace to test the HCP Packer resource artifact validation run task.
$ cd tf-resource-validation
Open main.tf
. This configuration defines an EC2 instance that references the community Ubuntu 20.04 AMI ID for us-east-2
. This AMI ID is not tracked in your HCP Packer registry.
tf-resource-validation/main.tf
provider "aws" {
region = var.region
}
resource "aws_instance" "app_server" {
ami = "ami-039af..."
instance_type = "t2.micro"
tags = {
Name = "Learn-HCP-Packer"
}
}
Update configuration
Open terraform.tf
. In the cloud
block, update the organization
to point to your HCP Terraform organization.
tf-resource-validation/terraform.tf
terraform {
## ...
cloud {
organization = "hashicorp-training"
hostname = "app.terraform.io"
workspaces {
name = "learn-hcp-packer-run-tasks-resource-validation"
}
}
}
Create HCP Terraform workspace
Initialize your Terraform configuration. This will create an HCP Terraform workspace named learn-hcp-packer-run-tasks-resource-validation
in your HCP Terraform organization.
$ terraform init
Initializing HCP Terraform...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v4.2.0...
- Installed hashicorp/aws v4.2.0 (signed by HashiCorp)
HCP Terraform has been successfully initialized!
You may now begin working with HCP Terraform. Try running "terraform plan" to
see any changes that are required for your infrastructure.
If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.
In HCP Terraform, open the learn-hcp-packer-run-tasks-resource-validation
workspace.
Add AWS and HCP credentials to workspace variables
Go to the Variables page.
Under Workspace variables, add your AWS_ACCESS_KEY_ID
, AWS_SECRET_ACCESS_KEY
, HCP_CLIENT_ID
, HCP_CLIENT_SECRET
, and HCP_PROJECT_ID
as environment variables. You generated the HCP client ID and secret in the prerequisites. Alternatively, you can create variable sets with these environment variables and reuse them across multiple workspaces.
Note
Be sure to mark the AWS_SECRET_ACCESS_KEY
and HCP_CLIENT_SECRET
as sensitive
.
Note
The AWS_SESSION_TOKEN
is optional unless your organization requires it.
Enable run tasks in workspace
Click on Settings then Run Tasks.
Under Available Run Tasks, click on HCP-Packer.
HCP Terraform run tasks have two enforcement levels:
- Advisory: If this run task fails, the run will proceed with a warning in the UI.
- Mandatory: If this run task fails, the run will return an error and stop.
Select the Mandatory enforcement level, then click Create.
The Run Task page will now display the run task for HCP Packer. This run task will parse resources for hard-coded machine image IDs and check if they are tracked and unrevoked in HCP Packer. If the run task detects an machine image ID that is associated with a revoked version, both the run task and the HCP Terraform run will fail.
Trigger HCP Terraform run
In your terminal, apply your configuration. When prompted to confirm the apply, press Enter
to discard the run.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-hcp-packer-run-tasks-resource-validation/runs/run-REDACTED
Waiting for the plan to start...
Terraform v1.1.6
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
## ...
Plan: 1 to add, 0 to change, 0 to destroy.
## ...
Do you want to perform these actions in workspace "learn-hcp-packer-run-tasks-resource-validation"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
╷
│ Error: Apply discarded.
│
│
Verify untracked artifact validation
In HCP Terraform, open the latest run and expand the Tasks passed box.
The run task passed with the following message:
HCP Packer does not currently track the image ami-039af3bfc52681cd5. Tracking images reduces security risk by preventing deployments of decommissioned images.
Since you have the HCP Packer Plus tier, the run task will perform both data source and resource artifact validation. This will check and validate whether the HCP Packer data sources and hard-coded image IDs reference revoked artifact versions in HCP Packer.
The run task parsed the aws_instance
resource but did not find the AMI it uses in your HCP Packer registry. Since the task cannot verify the compliance of untracked artifacts, it passes. The run task prompts you to use HCP Packer to track and manage your artifacts for more accurate validation.
Hard-code AMI to test validation
In HCP Packer, go to the learn-packer-run-tasks
bucket's revoked version. Under Builds, click on us-east-2 to view more information about the artifact.
Copy the External Identifier, you will update your Terraform configuration to use this AMI ID.
In tf-resource-validation/main.tf
, update the aws_instance
's ami
attribute to the revoked version's image ID.
tf-resource-validation/main.tf
resource "aws_instance" "app_server" {
ami = "ami-0e87b..."
instance_type = "t2.micro"
tags = {
Name = "Learn-HCP-Packer"
}
}
Apply your configuration. After Terraform creates the plan, it will return an error because the run task failed.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-hcp-packer-run-tasks-resource-validation/runs/run-REDACTED
## ...
Post-plan Tasks:
All tasks completed! 0 passed, 1 failed (4s elapsed)
│ HCP-Packer ⸺ Failed (Mandatory)
│ 1 image scanned. 1 failure.
│
Error: the run failed because the run task, HCP-Packer, is required to succeed
│
│ Overall Result: Failed
------------------------------------------------------------------------
Verify hard-coded AMI artifact validation
In HCP Terraform, open the latest run and expand the Tasks failed box.
The run task failed with the following message:
Your configuration hardcodes image ami-0e87bb27f39d20b5c from the revoked version v1 in the bucket learn-packer-run-tasks. Update your configuration to use the HCP Terraform provider to dynamically source up-to-date images.
The run task parsed the aws_instance
resource and found the machine image ID in a revoked version. This configuration uses a revoked (compromised or outdated) artifact. As a result, because the resource was being created, the run task failed and blocked the deployment of revoked artifacts.
Note
The run task will only fail if the configuration uses a revoked image for creating new resources. If an existing resource uses a revoked artifact, the run task will succeed but still report that the resource is not compliant.
If the run task identifies a newer artifact version, it will suggest that you use it. If you are the artifact maintainer, you can then assign the channel to the newer version.
The run task also detected a hard-coded image ID in your configuration. The error message recommends updating the configuration to use HCP Packer data sources, so you do not have to manually update hard-coded image IDs.
Restore artifact version
In the HCP Packer dashboard, go to the learn-packer-run-tasks
bucket and select the revoked version. Click Manage, then Restore version to restore the revoked version.
Confirm the action by clicking on Restore version.
Use HCP Packer data source
In tf-resource-validation/main.tf
, add the following data sources to the top of the file so Terraform can dynamically query the artifact ID from HCP Packer.
tf-resource-validation/main.tf
provider "hcp" {}
data "hcp_packer_version" "ubuntu" {
bucket_name = "learn-packer-run-tasks"
channel_name = "production"
}
data "hcp_packer_artifact" "ubuntu_us_east_2" {
bucket_name = "learn-packer-run-tasks"
platform = "aws"
version_fingerprint = data.hcp_packer_version.ubuntu.fingerprint
region = "us-east-2"
}
Then, update the aws_instance
's ami
attribute to reference the hcp_packer_artifact
data source.
tf-resource-validation/main.tf
resource "aws_instance" "app_server" {
ami = data.hcp_packer_artifact.ubuntu_us_east_2.external_identifier
instance_type = "t2.micro"
tags = {
Name = "Learn-HCP-Packer"
}
}
In your terminal, apply your configuration. When prompted to confirm the apply, press Enter
to discard the run.
$ terraform apply
Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.
Preparing the remote apply...
To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp-training/learn-hcp-packer-run-tasks-resource-validation/runs/run-REDACTED
Waiting for the plan to start...
Terraform v1.1.6
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
## ...
Plan: 1 to add, 0 to change, 0 to destroy.
## ...
Do you want to perform these actions in workspace "learn-hcp-packer-run-tasks-resource-validation"?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
╷
│ Error: Apply discarded.
│
│
Verify artifact validation
In HCP Terraform, open the latest run and expand the Tasks passed box.
The run task passed with the following message:
Your configuration uses current and approved images.
The run task parsed both the data sources and the aws_instance
resource. Since the data sources and the resource reference a d.
Next steps
In this tutorial, you associated the HCP Terraform run task for HCP Packer with an HCP Terraform workspace, then used the run task to ensure your Terraform configuration uses compliant artifacts and follows HCP Packer best practices.
For more information on topics covered in this tutorial, check out the following resources:
- Read more about the HCP Terraform run task integration in the HCP Packer documentation.
- Complete the data source artifact validation run task tutorial to learn how to identify compromised and outdated artifacts referenced by the HCP Packer data sources (
hcp_packer_version
andhcp_packer_artifact
).