FME Flow and Docker - Managing Engines and Queues

Introduction

When deploying FME Flow (formerly FME Server) using Docker Compose, there is one FME engine service by default. This is beneficial as it allows for easy engine scaling. To scale engines using Docker Compose, use the command below, changing the number of engines to however many you’d like - license permitting.

docker-compose up --scale fmeflowengine=4

The downside of this: in order for Docker Compose to scale engines, the engine container hostnames are set to the container ID.

This makes it very difficult to create and assign FME Flow engines to queues as every time the container restarts or engines are scaled the container IDs will be different. As engines are assigned to queues by engine name, this can lead to an FME Flow engine ‘graveyard’, where the list of engines will continuously grow as FME Flow does not know to remove engines that have been shut down (and will never come back).

In order to be able to use FME Flow Job Queues with a Docker deployment, we recommend that you follow one of the following approaches - depending on how you use your FME Flow.

FME Flow 2021+:

FME Server on Docker: Utilizing Engine Assignment and Job Routing