Building AMIs using Jenkins

This article explains a Jenkins pipeline that automates the creation of Amazon Machine Images (AMIs) using HashiCorp Packer. It is designed for building FME Flow custom AMIs from a source AMI, based on the installer path and other inputs. These AMIs can then be used for Provisioning High Availability Flow infrastucture on AWS using Terraform. 

Prerequisites

The following prerequisites must be in place to successfully execute this pipeline. Please be advised that this is a basic pipeline script that requires review for any security concerns. Any customization in the setup may require editing the Jenkins pipeline script. A PDF document of the full pipeline script is attached to this article. 

Jenkins Setup

• A Jenkins server with a build agent labelled “build-server-1” was set up. In our test, we are running a distributed Jenkins installation [master and worker setup] on Linux-based AWS EC2 instances. We also installed Nginx reverse proxy to access Jenkins on port 80 [Jenkins runs on port 8080 by default].
• Required plugins were installed on Jenkins:
  • Git plugin
  • Pipeline plugin
  • Terraform Plugin
  • Github Integration Plugin
  • Credentials Binding plugin
  • Pipeline Utility Steps Plugin
• The job type “pipeline” was configured. 

Packer 

• Packer was installed on the Jenkins build agent
• The Packer template fme_flow_aws.pkr.hcl is located under AWS/packer/ in the GitHub repo- GitHub - safesoftware/fme-server-iac-templates. 

AWS Access and Credentials

• AWS CLI was installed on the Jenkins build server.
• AWS Access Key [aws-access-key] and Secret Key [aws-secret-key] for an AWS user account were created and stored as Jenkins credentials.
• Ensure the AWS user has appropriate permissions to create AMIs and launch EC2 instances via Packer.
• The source AMI needed to create the custom AMI must be valid and located in the region.

GitHub Repository

• A GitHub repository was created by forking Safe Software’s official IaC repository GitHub - safesoftware/fme-server-iac-templates.
• A Jenkins credential entry was created for access to the repo [using webhook and personal access token] to ensure Jenkins can access Git.
• Git was installed on the Build server

Network and Permissions

• The Jenkins server must have internet access to download the FME Flow installer.

Pipeline Structure

Agent Declaration

agent { label 'build-server-1' }

This ensures the job runs on the designated Jenkins agent (build-server-1) that has all the required tools: Git, Packer, AWS CLI, etc.

Parameters Block

parameters {
    string(name: 'INSTALLER_URL', ...)
    string(name: 'OWNER', ...)
    string(name: 'FME_BUILD', ...)
    string(name: 'SOURCE_AMI', ...)}

This accepts user-defined inputs needed for the AMI build process:

• INSTALLER_URL: Location of the FME installer from FME Downloads
• OWNER: Metadata tag for the owner of the to-be-created custom AMI
• FME_BUILD: Tag to identify the build version of FME
• SOURCE_AMI: Base AMI ID that Packer will use from the AWS AMI Catalog

Environment Variables

environment {
  AWS_REGION = 'ca-central-1'}

This sets the region where the AMI will be built. This is passed into Packer via the -var option.

Validate User Inputs

stage('Validate User Inputs') {
  script {
    if (!params.INSTALLER_URL?.trim()) ...}}

This prevents the pipeline from proceeding if any required parameters are missing or incorrectly formatted. It is important for SOURCE_AMI validation to avoid launching a misconfigured Packer build.

Checkout Code

stage('Checkout Code') {
  git branch: 'main', url: 'https://github.com/...'}

This pulls the latest packer template and associated files needed to build the AMI from the GitHub repository.

Initialize Packer

stage('Initialize Packer') {
  dir('AWS/packer') {
    sh 'packer init fme_flow_aws.pkr.hcl'}}

It sets up required Packer plugins and validates that the environment is ready for the build.

Validate Packer Template

packer validate -var "..." fme_flow_aws.pkr.hcl

This step ensures the template is correct before attempting a build. Catching syntax or variable issues early avoids wasting build time.

Build AMI with Packer

packer build -var "..." fme_flow_aws.pkr.hcl | tee output.log

This triggers the actual AMI creation process. Packer uses the parameters and builds an EC2 instance, installs FME using the installer_url and creates a new AMI from the customized instance. This tee output.log ensures logs are saved for AMI ID extraction [This is an optional step].

Extract AMI ID [Optional]

grep -o 'ami-[a-zA-Z0-9]*' output.log | tail -n 2

This pulls the last created AMI IDs from the Packer output log. This AMI ID can be passed to downstream jobs (like Terraform provisioning) or saved for audit.

Additional configuration

By default, the first login on the FME Flow Web UI as admin prompts a password change. If you want to access FME Flow via a pipeline using Flow CLI immediately after deployment, you need to set the silent install parameter FIRSTLOGINCHANGEPASSWORD=false when installing FME core. 

On line 7 of the PowerShell script here [config/powershell/install-flow-core.ps1], add FIRSTLOGINCHANGEPASSWORD=false as shown below.

After this change, the user login can persist as admin/admin after install. FME Flow does not enforce an immediate password change, allowing you to use the Flow CLI within your pipeline right after deployment without any additional login configuration.

Conclusion

This pipeline should provide you with two AMIs -one for Flow Core and one for Flow Engine.