Automating FME Flow Post Installation Tasks using Jenkins

This article walks through a Jenkins pipeline that automates the process of:

1. Logging in to an FME Flow instance via the Flow CLI and administering a health check
2. Requesting and applying the Flow license
3. Scaling engine hosts (standard and dynamic)
4. Running a sample workspace

This setup supports dynamic infrastructure, typically after deploying FME Flow via Terraform or another IaC method.

Prerequisites

To successfully execute this pipeline, the following prerequisites must be in place. Please be advised that this is a basic pipeline script that requires review for any security concerns. Any customization in the setup may require editing the Jenkins pipeline script.  A PDF document of the full pipeline script is attached to this article.

Jenkins Setup

• A Jenkins server with a build agent labelled “build-server-1” was set up. In our test, we are running a distributed Jenkins installation [master and worker setup] on Linux-based AWS EC2 instances. We also installed Nginx reverse proxy to access Jenkins on port 80 [Jenkins runs on port 8080 by default].
• Required plugins were installed on Jenkins:
  • Git plugin
  • Pipeline plugin
  • Terraform Plugin
  • Github Integration Plugin
  • Credentials Binding plugin
  • Pipeline Utility Steps Plugin
• The job type “pipeline” was configured.
• FME Flow CLI was installed at /usr/local/bin/fmeflow on the build server. Follow GitHub - safesoftware/fmeflow-cli: A Command Line Interface for interacting with FME Flow for more information on installing the CLI.
• Network access to the FME Flow instance must be provided.

FME Flow Setup

• REST API version 4 was used for the API calls.
• The license request service was accessible from the Flow Core VMs.
• A Jenkins credential entry was created for the Flow admin user fmeflow-login and it must contain a valid FME Flow username/password.

Pipeline Structure

Agent Declaration

agent { label 'build-server-1' }

This runs the job on a specific agent where CLI tools and network access are preconfigured.

Parameters Block

parameters {
  string(name: 'FMEFLOW_URL', defaultValue: ...)
  ...}

These parameters make the pipeline flexible and allow runtime customization:

• FMEFLOW_URL: Base URL of the deployed FME Flow instance
• STANDARD_ENGINE_COUNT: Number of standard engines to scale to
• DYNAMIC_ENGINE_COUNT: Number of dynamic engines to scale to
• FIRST_NAME, LAST_NAME, EMAIL, COMPANY, SERIAL_NUMBER: Required to programmatically request a license

Environment Block

environment {
    CLI_BIN = '/usr/local/bin/fmeflow'
    CLI_CONFIG_FILE = "${WORKSPACE}/.fmeflow-cli.yaml"
    PASSWORD_FILE = "${WORKSPACE}/fmeflow_pass.txt"}

These define commonly reused paths:

• CLI_BIN: Where the CLI binary is located
• CLI_CONFIG_FILE: Where to save the YAML file that stores the API token generated after login.
• PASSWORD_FILE: Where to temporarily store the admin login password (this is deleted later for security).

Log in to FME Flow

fmeflow login <url> --user <user> --password-file ... --config ...

This step logs in using the CLI and creates a YAML file with a new API token for subsequent commands. This avoids using plaintext passwords in every command and stores a session token for reuse.

Health Check

fmeflow healthcheck --config ...

This step confirms that the target FME Flow is up and API endpoints are functional. It is useful for early error detection before making changes.

Extract API Token

awk '/^token:/{print \$2}' .fmeflow-cli.yaml

This grabs the generated API token from the YAML file and saves it to an environment variable for raw curl commands used later. CLI handles most operations, but some might require API calls instead.

Request License

fmeflow license request --first-name ... --serial-number ...

This triggers a license request with the provided user and organization information.

Check License Status

It waits 30 seconds and checks the status of the license request. The FME Flow backend needs time to complete licensing. This polling pattern helps confirm whether the license was successfully activated.

Scale & Summarize Engine Hosts

This step queries available engine hosts, scales each one to the desired count for both standard and dynamic engine types and summarizes scaling in the console log.

Key API Endpoints Used:

• GET /fmeapiv4/enginehosts:Retrieves all hosts configured to run FME Engines.
• GET /fmeapiv4/enginehosts/${name}/engines : Retrieves all FME Engines configured to run on the specified FME Engine host.
• POST /fmeapiv4/enginehosts/{name}/engines/scale : Adjusts the number of FME Engines to the number specified in the request. 

Run FME Workspace

fmeflow run --repository Samples --workspace austinApartments.fmw --wait ...

This step executes a known sample FME workspace as a test to validate that the system is functioning properly post-license and post-engine-scaling.

Use Token (Optional)

This is an optional step that demonstrates that the API token is available for any secure operations downstream. It’s not printed for security, but can be used if you need it in other stages.

Post Block: Cleanup

post {
    always {
        sh "rm -f ${PASSWORD_FILE}"}}

This ensures that the password file is deleted even if the job fails.

Conclusion 

You should have a healthy Flow installation with a valid license and the intended number of standard/ CPU engines running.