Search

Using the FME Flow REST API V4 to Create Job History Reports

Sienna Emery
Sienna Emery
  • Updated

Files

As of FME Flow 2025.1, the V3 REST API has been deprecated. The current API will remain accessible, but no new features or general bug fixes will be implemented. Starting with FME Flow 2026.1, the V3 API will be removed. We encourage all users to prepare their systems for this change. 

The V4 REST API offers enhanced security and usability, including new and reorganized endpoints, as well as the removal of certain endpoints. For information on migrating from V3 to V4, please see the FME Flow REST API V4 documentation and click on the heading Migrating from REST API V4.
 

Introduction

The FME Flow REST API can be used for many things, from managing FME Flow Engines to submitting FME Flow jobs programmatically. However, one of the most useful tasks can be generating job history reports. These reports can give you valuable information- for example, which workspace is using the most memory or CPU. This can be useful for making crucial scheduling decisions on a busy FME Flow. In this article, we will discuss the ins and outs of building a workspace to access the FME Flow REST API and create reports. This article, Monitoring FME Flow Job Activity using the REST API, shows how to use this workspace to build a dashboard in FME Flow.

Please note this tutorial uses the FME Flow REST API V4. The steps are similar for V3, but the endpoints may be different. 

Requirements

  • Access to FME Flow 2025.0+
  • An FME Flow User account with permissions to view the job history

Step-by-Step Instructions 

To begin this tutorial, download the workspace (AverageStatistic_Starting.fmw). This workspace is mainly complete, but we will be walking through how to create some essential parts of it.

Part 1: Fetch Job Statistics with Pagination

Part 1 illustrates how to fetch job history from FME Flow using the FME Flow REST API. This exercise uses HTTPCallers to access the FME Flow REST API. For more information on how to use the HTTPCaller transformer, see HTTP Requests with the HTTPCaller.

First, we need to use the FME Flow REST API to determine how many jobs were run on the FME Flow. 
This is because we use the REST API to retrieve job history from FME Flow, and the response can only contain a maximum of 1000 jobs. If the number of jobs exceeds this limit, we need to make multiple calls with the HTTPCaller. This is called pagination and can be found in most APIs. 

The FME Flow REST API uses two parameters to handle pagination. These parameters are called limit and offset. Let's say you have a busy server with 10,000 jobs, and you want to retrieve the full job history. For the first call, you would want to set the limit to 1000 and the offset to 0. This will retrieve the newest 1000 jobs. The next call would have to set the limit to 1000 and the offset to 1000. This would retrieve the next 1000 jobs after the first 1000.

1. Review the FME Flow API Documentation

To start, review the REST API documentation by going to http://<yourServerHost>/fmeapiv4/docs/index.html where <yourServerHost> is your server hostname. If you access the REST API documentation from your FME Flow through the Help Resources icon, you can even test REST API calls out through the web interface.

Alternatively, you can access the documentation here.
HelpfulResources.png

Once on the REST API documentation, select the API tab. Then, scroll and select jobs. This section shows all REST API calls related to job routing and history.

Find GET /jobs. Once there, review the documentation. Notice the response is listed and it shows where to find the total count.


2. Get the Total Record Count of all Jobs Run on FME Flow

Open up the workspace AverageStatistic_Starting.fmw found in the files section. 
Find bookmark 1: Fetch Job Statistics with pagination. Inside it is another bookmark, 1a. Get Total Record Count. 

This is where we will add an HTTPCaller and a JSONExtractor to get the record count.

If you are using FME 2025.2+, the HTTPCaller no longer requires a creator transformer to run. You can simply add the HTTPCaller to the canvas and configure it. 

As of FME 2025.2, many transformers have been updated to not require input from the creator transformer. For a list of all the transformers with this improvement, please see Transformers with an Optional Input Port


AddHTTPCaller.png
Add an HTTPCaller to the workspace and attach it to Creator_2. Open up the HTTPCaller and set up the following parameters:

  • Request URL: $(FME_SERVER_WEB_URL)/fmeapiv4/jobs?status=success
  • HTTP Method: GET
  • Use Authentication: Enabled
    • Authentication Method: Web Connection
    • Web Connection: $(AUTH_NAMED_CONNECTION)
      • Click the drop-down arrow, hover over User Parameters, and select $(AUTH_NAMED_CONNECTION)

Then click OK. 

3. Set the FME Flow User Parameter

To run this workspace on FME Form, you need to first set the FME Flow User Parameter. The parameter we set in the HTTPCaller allows this workspace to run on any FME Flow, as the Flow URL will be set automatically in FME Flow.

$(FME_SERVER_WEB_URL)

However, this means the workspace won’t run on FME Form without setting up the parameter. In the Navigator panel, expand the FME Flow Parameters, then find the [FME_SERVER_WEB_URL] FME Flow Web URL parameter. Double-clicking on the parameter will open a text box to add your FME Flow URL. 

Make sure not to include a final slash at the end of the URL. 

Click OK to exit the dialog.
FMEFLowParameters.png


4. Run the FME Workspace

Run the Workspace with Data Caching Enabled (formerly Feature Caching). Review the attribute from the output called _response_body.

RunTheCaller.png

To get a better view of the JSON, click the ABCXYZ button, then select JSON.OpenJSONFormat.png

Scroll down to the bottom of the page to view the totalCount from your Flow.
NiceJSON.png
5. Get the totalCount from the JSON response

Add a JSONExtractor to the canvas. Connect the JSONExtractor to the output port of the HTTPCaller. Then, connect the Evaluated port from the JSONExtractor to the ExpressionEvaluater_3.
Open the JSONExtractor to modify the parameters:

  • Input Source: JSON Document
  • JSON Document: _response_body
  • Extract Queries:
    • Target Attribute: totalCount
    • JSON Query: json["totalCount"] 

JSONExtractor.png

Run the workspace again and note that a new attribute called totalCount is created.

6. Run the 1b. Create Offset Bookmark

Click the bookmark and select Run to Contained. This bookmark will create the requests needed to fetch all of the job records. If the total count exceeds 1000 records, then we need to submit multiple calls to get the results.

RunCreateOffsetBookmark.png

The ExpressionEvaluator is used to determine the offset. The total count is divided by 1000 to determine the number of pages we’d need to fetch. This number is then rounded to ensure there are no decimal places.

Next, a Tester is used. If the first call retrieves fewer than 1000 results, we do not need to make additional calls, so we bypass the Cloner. The Cloner is used to create multiple features based on the offset number. For instance, if the offset is set to 3, we will create three copies of a feature with a copy number (a count of the copies) we call offset.

Next, the AttributeManager is used to increase the offset by 1000, since we want to grab the next 1000 records instead of the next record.

7. Run the 1c. Fetch Results Bookmark

RunFetchResultBookmark.png
Here, the HTTPCaller is used to fetch the remaining results, if there are any. The offset is used to tell the API where to start fetching the next 1000 records.



Next, the JSONFragmenter is used to parse the results. Take a look at the original _response_body from the HTTPCaller's output. Notice that the first item is “items,” and all the information about the job is held within this item.

PaginationItems.png

Open up the JSONFragmenter_2; the JSON query is set to json["items"][*]

This action allows the user to expose all attributes inside the item's tag.

Next, the attributes to expose section will now expose the attributes listed. These attributes can be found in the response body.

ExposeCPUTime.png

These attributes are manually entered in the “Attribute to Expose” parameter, or they can be imported if you have a file saved with the JSON using the Import… button.

JSONFragmenter2.png

Part 2: Prepare Data and Create Reports

Currently, the data is retrieved by the REST API, but it is not in a format conducive to creating informative reports. The stars of this next section are the StatisticCalculator and the HTMLReportGenerator. The StatisticsCalculator calculates statistics based on a designated attribute or set of attributes. The HTMLReportGenerator takes this data and turns it into an HTML report.

1. Configure the StatisticsCalculator 

Open the StatisticsCalculator and set it up to calculate the following attributes:

  • Group Processing: Enabled
    • Group By: Workspace Name, Repository
  • Statistics to Calculate
    • Attribute 1:
      • Attribute: cpuTime
      • Mean: Enabled
    • Attribute 2:
      • Attribute: cpuPct
      • Mean: Enabled
    • Attribute 3:
      • Attribute: peakMemUsage
      • Mean: Enabled

By using the Group by option, the statistics are calculated for the group. In our case, statistics will be calculated per workspace. Instead of the statistics being calculated for all the workspaces together. 

StatsCalcParams.png
Please note that you can calculate multiple statistics for an attribute. For example, on cpuTime, you could calculate the Mean and the Total Count.

2. Sort and Calculate the CPU Time

Next, review bookmark 3b. Sample by the number of workspaces to show. We will only review one of the bookmarks here because the rest are similar.
ReturnTopCPUTime.png

Take a look at the Return Top CPU Time bookmark. Here, the Sorter sorts the longest-running workspaces in descending order.

Then, the Sampler fetches the number of workspaces specified by the user parameter SamplingRate. This enables the user to select the number of workspaces they’d like to see in the report.

Next, the ExpressionEvaluator converts the CPU Time, which is currently in milliseconds, to seconds. This result is then rounded. 

After this, the data is ready to go into the HTMLReportGenerator.

3. Configure the HTMLReportGenerator 
Take a look at the bookmark 4a. Create Average CPU Time HTML Report. Open up the HTMLReportGenerator and configure the following parameters:

  • Page Title: Average CPU Time Per Workspace
  • Page Contents: Header
    • Click the plus sign and select Header, then click on the right side to access the settings
    • Content Settings:
      • Text: Average CPU Time Per Workspace
      • Header Level: H1
      • Text Alignment: Center
      • Color: 0,0,0

HTMLReportGenerator1.png

Next, under Page Contents, add another Header. In the Content Settings, set the following:

  • Page Contents: Header
  • Content Settings:
    • Text: Top $(SamplingRate) Largest CPU Workspaces (@DateTimeFormat(@DateTimeAdd(@DateTimeNow(),-P7D),"%b %d, %Y") - @DateTimeFormat(@DateTimeNow(),"%b %d, %Y"))
    • Header Level: H3
    • Text Alignment: Center
    • Color: 102,102,102
Top $(SamplingRate) Largest CPU Workspaces (@DateTimeFormat(@DateTimeAdd(@DateTimeNow(),-P7D),"%b %d, %Y") - @DateTimeFormat(@DateTimeNow(),"%b %d, %Y"))

Notice how the DateTime functions are used within the text editor. These functions get a timestamp and format it so that it is easier to read. This can also be handled in the DateTime transformers.

HTMLReportGenerator2.png

  • Page Contents: Chart (Bar)
  • Content Settings:
    • X Axis Label: <leave blank>
    • X Tick Label Attribute: userName
    • Y Axis Label: Average CPU Time in Seconds
    • Data Series:
      • Data Attribute: cpuTime.mean
      • Color: 0,0, 255


HTMLReportGenerator3.png

Note, StringReplacers are used after the HTMLReportGenerator to change the styling; however, this isn’t necessary. It’s just a personal adjustment. 
 
4. Run the Workspace and Review the Results

Finally, click the drop-down on the Run Button and select Rerun Entire Workspace.

RerunEntireWorkspace.png

This will prompt you to set the parameter values. Make sure to set your FME Flow Web Connection and select locations for the HTML Reports. Then, select Run.

TranslationParameterValues.png

Now, review the HTML files created! Below is an example of a report created through this workspace.

image21.png

Congratulations! You have completed this exercise. To continue with this series, see Monitoring FME Flow Job Activity using the REST API to learn how to take this workspace and create a dashboard from it.

Was this article helpful?

We're sorry to hear that.

Please tell us why.

As of January 14th, 2026, comments on knowledge base articles have been closed. To make sure questions don’t get missed and to enable more community support, we’ve moved discussions to the FME Community. If you have a question or a comment about this article, please create a new post or create a support ticket.