Introduction
This tutorial series provides instructions for setting up Queue Control on FME Flow. In general, a job queue contains a list of jobs waiting to be processed by a subsystem. Jobs are held here until a number of conditions are met and the job is selected for processing.
The job queueing mechanism provides greater control and flexibility in how FME Flow processes jobs and uses the available engines. You can create engine assignment rules, associated active periods, and job routing rules to dynamically assign engines to queues and send jobs to specific queues based on criteria you set.
In FME Flow 2025.2, the Engine Management sidebar in the Web Interface has been redesigned to provide a simplified UX and improved control (and visibility) of the configurations made.
- The “Overview” page provides a summary of queues, engines, hosts, and streams. They are displayed in different panels with different sorting options. A font color scheme has been incorporated to differentiate between standard and CPU-usage engines. We also have colored icons representing the running/waiting/idle status of queues and engines.
- The “Queues” page has search, sorting, and filtering capabilities [by Job status and Engine type]. Queue Priority 1 is the lowest priority, and 10 is the highest. There is a batch edit feature for queue priorities. The side panel [accessed by clicking the individual queue] displays queue statistics such as the numbers of successful, failed, and canceled jobs, and the average and total wait and run times. There is also an option to reset these statistics.
When creating a queue, you can preview the queue priority of the already configured queues.
- The “Engines” page shows Standard and CPU-usage engines with sorting and filtering options. The side panel [accessed by clicking the individual engine] displays information like the most recent jobs, jobs currently running, and engine properties.
-
The “Host” page can be accessed from the “Hosts” panel on the “Overview” page by clicking “View All”.
Terminology
In FME Flow, Queue Control is made up of the following components that work together to ensure jobs are routed to the correct engines for processing:
-
Job Routing Rules: These rules determine which queue any submitted job should be sent to. There are two types of Job Routing Rules that can be configured -
- Properties: rule(s) based on one of these properties or a combination of these job-specific properties: processing time [CPU and Elapsed], system CPU usage [in %], peak memory usage, job repository, related workspace, source type, source name, user name, or user role.
- Repository: rule(s) based on the repository that a workspace is stored in.
-
Engine Assignment Rules: These rules control which engine(s) are assigned to each queue. There are two types of Engine Assignment Rules that can be configured-
- Property: Properties fall under three categories: Custom, Engine, or Queue. Custom properties are user-defined values assigned to engines; they are commonly used in Docker/K8S deployments. Engine properties are pre-populated based on the attributes of the engine host, including physical memory and processor count. Queue properties hold information about the jobs already waiting in that queue, for example, the number of queued jobs and the time to finish for all queued jobs.
- Name: rule(s) based on the name of the engine.
In FME Flow, to assist in balancing engine load, you can choose to set a limit on the number of queues to which engines can be assigned. For more information, please follow the linked documentation.
- Queues: Queues hold jobs sent to them by job routing rules and direct them to engines based on engine assignment rules. Every queue is assigned a priority from 1 to 10, with 10 being the highest. During installation, a “Default” queue is auto-created with a priority of 5(Medium). As a middle value, it allows creating queues with higher or lower priority than the default.
- Active periods: Optionally, if you would like to specify periods of time during which engine assignment rules are active, Active periods allow you to do so. You can specify a start date/time and an end date/time, and set recurrence options. If an engine assignment rule is assigned to any active periods, the rule applies to the associated engines only during those periods. Any job submitted to a queue that does not have an available engine to run it because the active period has not yet started remains in the queue until the active period begins. For more information on how to create an active period, please follow our documentation.
FME Flow Job Processing without Queue Control
By default, no engine assignment rules or job routing rules are created/setup by the system. All jobs to be processed will be routed to the Default queue [it is automatically created during installation] with a priority of 6 (Medium). The jobs are considered equal and will be run by any available engine, in the order they were submitted.
Articles
Please be advised that these instructions are based on FME Flow 2026.2. If you are working with an older version of FME, refer to the version-specific documentation to learn which queue functionality was supported.
Please follow the links below to set up queue control on your FME Flow Web UI -
Job Routing Based on Job Priority
Set up time-sensitive jobs to be processed with higher priority than jobs with less urgency.
Job Routing Based on Job Metrics
Set up jobs with high CPU/Memory utilization to be routed to engines with sufficient hardware/ system resources.
Job Routing Based on Engine Properties
Set up jobs to be routed to specific engines based on the type of engines [standard or CPU-Usage] or engine host’s name or resources.
Job Routing Based on Queue Metrics
Set up jobs to be routed to specific engines based on the number of jobs waiting in the queue.
Additional Resources
Queues and Data Processing Jobs: FME Server and the Art of Queuing
Guide to Manage Esri Jobs using Queue Control
Defining FME Engines, Queue Control, and Streams to a Kubernetes Deployment
Did the Queue Priority Order Change in FME Flow