As of FME Flow 2025.1, the V3 REST API has been deprecated. The current API will remain accessible, but no new features or general bug fixes will be implemented. Starting with FME Flow 2026.1, the V3 API will be removed. We encourage all users to prepare their systems for this change.
The V4 REST API offers enhanced security and usability, including new and reorganized endpoints, as well as the removal of certain endpoints. For information on migrating from V3 to V4, please see the FME Flow REST API V4 documentation and click on the heading Migrating from REST API V4.
Introduction
Monitoring the activity of FME Flow (formerly FME Server) enables you to make more informed decisions about how you would like to respond to increased demand.
In FME Flow 2021.0+, additional job statistics have been exposed to show job performance. These metrics are available to view through the Web UI on the Jobs Completed, Workspace, Automations, and Schedule pages as well as accessible via the REST API.
This tutorial will demonstrate how to use the REST API to retrieve these statistics and build a visual report that can be displayed in FME Flow Dashboards to help monitor FME Flow activity, performance, or usage over time. First, let’s take a look at what these new statistics are!
The REST API V4 documentation can be found here.
Job Statistics Overview
Statistics Available Per Job
- Elapsed Time: Total processing time for an FME Workspace (End Time - Start Time).
- CPU Time: Total CPU time for an FME Workspace (User time + System time).
CPU Time is time spent doing work on the CPU (as opposed to waiting for
input, disk, network, etc.).
CPU Time can be further subdivided between User (FME doing work) and System
(the
OS doing work on FME's behalf).
You may observe discrepancies between the CPU time recorded in an FME Form/Flow
job log and the time presented through Job Statistics (on the Jobs Completed
page or via the REST API). This is expected because the time reported by
Job
Statistics includes the additional start and end scripting that cannot be
captured
by FME Form/Flow logging processes.
- CPU Percent: The percentage of total processing time that is recorded as CPU time (CPU Time / Elapsed Time).
- Peak Memory Usage: The net memory usage is calculated as the peak memory used by an FME Engine process at the end of translation, subtracting the memory in use by the FME Engine before the translation begins.
Unlike in FME Form, the FME Flow Engine processes do not shut down after every job, and this can lead to unintentional memory leaks between translations. To account for this accumulation and return a more accurate value for peak memory consumption, the above calculation is performed. This can result in different values presented through Job Statistics (on the Jobs Completed page or via the REST API) and in the FME Form/Flow log files. The latter shows only the peak memory used by the FME Engine process at any time during the translation, without taking into consideration the memory already in use at the start.
To fetch this information through the REST API, use:
-
All Jobs:
[GET] /fmeapiv4/jobs -
Per Job
[GET] /fmeapiv4/jobs/<id>
More information about each API call: https://docs.safe.com/fme/html/fmeapiv4/docs/index.html
Statistics Available Per Workspace, Schedule, or Automation
- Total Runs: The number of times that the workspace has been run.
- Average Elapsed Time: The average elapsed time across all runs.
- Average CPU Time: The average CPU time across all invocations.
- Average CPU Percent: The average CPU percent across all invocations.
- Average Peak Memory Usage: The average amount of memory an FME Workspace required while it was processing. This is the average of the Peak Memory Usage above.
To fetch this information through the REST API, use:
-
All Workspaces by Repository
[GET] /fmeapiv4/repositories/<repo>/items -
Per Workspace
[GET] /fmeapiv4/repositories/<repo>/items/<item/workspace> -
All Schedules
[GET] fmeapiv4/schedules -
Per Schedule
[GET] fmeapiv4/schedules/<id> - Per Automation It is not currently possible to retrieve this information via the REST API.
More information about each API call: https://docs.safe.com/fme/html/fmeapiv4/docs/index.html
If you would like to reset the statistics, this is an option per workspace or schedule through the Web UI. For Automations, statistics are reset automatically whenever you stop and restart the Automation.
Use Cases
CPU-Usage Engines (formerly Dynamic Engines)
As opposed to paying for a fixed license per engine, CPU-Usage Engines run on credits, giving you the option to purchase additional FME Flow processing power in a whole new way. One credit equals one hour of CPU time on the engine. To understand when it might be appropriate to route jobs to CPU-Usage Engines, administrators need to have a good understanding of the average CPU time their workspaces are using. Check out the Getting Started with CPU-Usage (Dynamic) Engines article to learn more about this feature.
Queue Control
FME Flow Queues were rewritten in 2021.0+ to provide users with the ability to configure advanced job routing rules, enabling workspaces to be directed to engines based on their previous performance, as indicated by average job statistics. To make informed decisions about configuring job routing rules, a clear understanding of how these jobs are performing is necessary.
Schedules
Understanding how long a workspace configured on a schedule typically takes to run can help FME Flow users set schedules to minimize long queue wait times.
Managing Hardware
Your FME Flow architecture may include engines distributed across hosts with different hardware specifications. By understanding the peak memory usage of different workspaces, you can route jobs accordingly to improve performance and even consider autoscaling new compute resources.
Performance Tuning
By understanding when your engines are most or least busy and how much time is spent performing certain tasks ,this information may help to create and modify workflows to utilize engines more efficiently.
Auto-Scaling Engines
This lays the groundwork for autoscaling, enabling organizations to optimize how engines are deployed, so watch this space!
Step-by-step Instructions
The following example uses a pre-made workspace that will be published to FME Flow to generate a dashboard. The dashboard will use a third-party tool Plotly.js to generate a Box Plot showing the peak memory usage per job, and information that has been retrieved from the jobs/completed endpoint. If you want to build your own workspace from scratch, check out the Tips for Working with the FME REST API in FME Workbench article for some things to look at whilst getting started.
1. Open Starting Workspace
In FME Workbench, open PeakMemUsage_BoxPlot.fmw starting workspace, which is available for download from the Files section of this article.
2. Explore the Workspace
Explore the workspace and make sure you understand what each step is doing. Set the AUTH_NAMED_CONNECTION published parameter value to your FME Flow Web Connection.
3. Publish to FME Flow
Publish the workspace to the Dashboards Repository (You may need to ask your Admin to update your user to have write permissions to this repository)
If not already on FME Flow, upload the FME Flow Web Connection.
Register the workspace with the Job Submitter and Data Streaming Service. This way, the workspace can optionally be run on the fly and stream the dashboard directly back to the user who requested the information.
4. Run Workspace
To confirm everything is working correctly, run the workspace in FME Flow
5. View Dashboard
Go to Jobs > Dashboards and from the Name select PeakMemoryUsage. This is the name of the HTML file that this workspace creates and can be modified in the Writer dataset in FME Workbench.
The third-party tool, Plotly, used to generate this workspace has an interactive interface. Explore your result by highlighting a section to zoom in, or hover over a box/point to get additional details about the job statistics.
6. Configure Schedule
For the dashboard to remain up to date and show the latest statistics, the workspace must be run at a repeat interval. Select Schedules > Build Schedule
Specify the interval at which you want the workspace to run (i.e., how often the dashboard should be updated)
Now the dashboard is all set up and will keep up-to-date!
Dashboard Examples
Attached to this article are workspaces for a series of simpler job statistic reports that utilize the HTMLReportGenerator. To set these up on your FME Flow, follow the instructions above for each workspace. All the workspaces include minimal filtering options, which can be used if a Workspace App is built to return the report on demand. Please add a comment below if you have any other dashboards you’d like to see.
Average CPU Time/CPU % and Peak Memory Usage Per Workspace
A basic bar chart to cover each new statistic using the /jobs endpoint. Specify the number of workspaces to return and a date range (e.g., up to 7 days ago). Additionally, filter results for a single repository.
Job Statistics Per Repository Overview
Dashboard to display all the new job statistics using the /repositories/<repo>/items endpoint. A report has been created for each repository, divided by a tab. Each report contains a bar graph displaying the average CPU usage versus elapsed time per workspace, and underneath the table, it lists the CPU percentage and peak memory usage, as well as the number of job runs this information is based on. Optionally filter by Repository name. Results cover the period since the statistics were last reset.
Workspace Status per User
As a user, you may want to perform a health check of all the jobs you own, or, as an administrator, you might like to see how active each of your FME Flow users is. This workspace utilizes the /jobs endpoint to generate a report of all workspaces belonging to a user, along with the number of times they’ve been run, sorted by job status.
Troubleshooting
If you are experiencing issues please consult our FME Flow Dashboards troubleshooting guide, if this does not resolve your problem then contact Safe Software Support.
Additional Resources
Documentation: FME Flow Dashboards
Tips for Working With the FME Flow REST API in FME Workbench
Data Attribution
Plotly.js is free and open-source software, licensed under the MIT license.