Submitting a Job through the FME Flow REST API

Introduction

The FME Flow REST API allows third-party applications to run jobs on FME Flow. By connecting your systems and users over the web, data and information can be exchanged in real time between otherwise unrelated systems.

All workspaces submitted by REST API use the FME Flow REST Service. The REST Service allows you to run jobs on FME Flow, much like using the job submitter service. To use the transformation services, like data streaming or data download, use a webhook URL instead. 

Jobs may be submitted using two different REST API requests: Transact and Submit. These two endpoints represent synchronous and asynchronous requests, respectively. On a practical level, these requests differ in how and when job results are returned. Which you choose will be determined by your specific workflow needs. 

The Transact request is synchronous. A synchronous task must be completed before the next task begins. After submitting a synchronous job, the API request is not finished until the job is complete. The translation results are returned in the API response. The Transact request is a good choice for short workflows or when results are required as soon as possible. 

The Submit request is asynchronous. An asynchronous task does not need to be completed before the next task begins. After submitting an asynchronous job, the API request is finished, regardless of the job status. A job ID is returned in the API response. To find out the translation's status or retrieve any results, the job ID can be used in subsequent requests. The Submit request is a good choice for long workflows or when immediate results are not required.

In this tutorial, we'll run a job on FME Flow by both Submit and Transact requests. The requests will be built and made from the API development platform Postman. However, the same requests can be made from any HTTP client, like the HTTPCaller in FME Workbench. Please note: for applications outside of your network to submit requests by REST API, please ensure that your FME Flow instance is publicly available. 

Please note this tutorial was written in an earlier version of FME. The steps are still accurate, but some of the screenshots might look different. 

Requirements

• FME Flow
  
  • An FME Flow API Token with necessary permissions (see Authorizing Requests)
• Postman

Step-by-Step Instructions

In this step-by-step tutorial, we will demonstrate submitting both synchronous and asynchronous jobs through the FME Flow REST API. We’ll use the "austinDownloads.fmw" workspace for both requests. This workspace is included inside the “Samples” repository of every FME Flow installation. 

Preparation is key to building successful workflows. There are some steps we can take before creating a job request that can help us be successful on the first try.
 

Review the Workflow Options

Whenever possible, review the workflow or data before building an API request. From the FME Flow Run Workspace page, select the “Samples” Repository and “austinDownloads.fmw” as the Workspace. 


Published Parameters can be set inside the request body of any REST API job submission. For any workspace, there are a couple of ways to find which parameters are available to use. 

From the Run Workspace page, scroll to find and expand the Advanced section. At the bottom of this section, you will find a table named Published Parameters. The Parameter column contains each available parameter. The Options column lists its accepted inputs. These values can be used with our API job requests.


Workspace published parameters may also be fetched by API request. Read more about how to retrieve and submit different types of published parameters in the Published Parameter Data Model tab of the FME Flow REST API Documentation.

If the parameter is of the geometry type, the JSON has to be escaped: 

"publishedParameters": [ 
   { 
     "name": "PARAMETER", 
     "value": "{\"type\":\"Polygon\",\"coordinates\":[[[-97.75332212445375,30.524413269916312],[-97.76362180707092,30.485367404225624],[-97.7629351615631,30.485367404225624],[-97.8570055961333,30.44571390561845],[-97.85219907757862,30.522638797976853],[-97.75332212445375,30.524413269916312]]]}" 
   },

Documentation

The best source of guidance for all your web workflows is usually the API documentation. Visit the FME Flow REST API Documentation to get started, follow along with these requests, and explore the many endpoints available to use with FME Flow. 

Alternatively, you can access REST API documentation personalized to your own FME Flow through the Help menu. This page allows you to test the listed API requests directly inside the browser.
 

Authorizing Requests

Just like a user needs to log into FME Flow before running a workspace, an application needs to provide identification before running a job. The FME Flow REST API uses tokens to authenticate requests. For any workflow, multiple tokens can be created with permissions to only what they need to complete their request. 
This tutorial assumes you already have a token. If you do not already have a token, create one by following our FME Flow REST API Authorization tutorial or from the FME Flow Token Management page.  

Part 1: Running a Synchronous Job With Standard Parameters

This section demonstrates the Transact request that runs a job synchronously on FME Flow. A Transact request instructs FME Flow to submit the job and only return a response upon the job’s completion. This means that you will wait until the translation is complete and the response message contains the job results.
The FME Flow REST API documentation contains more information about the Transact request in the Transformations section. 



1. Create a New Request in Postman
In Postman, a new request can be started by selecting New > HTTP Request or the plus sign ("+") button above the workspace window. 



A URL is the calling card of any REST API Request. We can use a URL to target the specific FME Flow domain, endpoint, and workspace. Copy this request URL to use with our request:
 

Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Flow’s domain name. 

Oftentimes, workspaces require parameter values to run the translation. The POST method allows us to submit these parameters and other data with each request. Using the dropdown menu, change the method from "GET" to "POST".

 

2. Create a Request Body
The body of every POST request contains data to send to FME Flow. The transact request body will contain the parameters and values of our workspace published in JSON. 

Select and copy the following JSON body to use with our request:

{
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ]
}

In Postman, select the Body tab below the POST request. Select the raw option, change the type to JSON by clicking on the drop-down arrow, then paste the JSON in the space below. 


 

3. Create Headers
Headers allow us to tell FME Flow more information about our API request. 

We can start by telling FME Flow that our Body content is in JSON format with a “Content-Type” header.

Next, we can ask FME Flow to return its response in JSON format with the “Accept” header.

Lastly, with an “Authorization” header, we can supply our unique token value. Replace <yourTOKEN> with your token value. If you do not already have a token, create one by following our FME Flow REST API Authorization tutorial or from the FME Flow Token Management page.

Select the Headers tab in Postman to add these headers and their values to this job request:

The Headers configuration should resemble the screenshot below. Note that Postman may include other default headers that can be ignored or hidden.



4. Run the Job by Transact REST API Request
In Postman, click Send. Wait 5-10 seconds while FME Flow receives the request and submits the job. Upon completion of the translation, Postman will display a response message from FME Flow.

The response from a Transact request contains information about the completed job: the job ID, status, how many features were handled, the time and duration of the request, and any messages. A synchronous request waits until the translation finishes before a response is returned so FME Flow can immediately share the job summary. 

5. View the Job in FME Flow
The FME Flow Web UI allows you to view all job requests, including those sent from the REST API. 

Log into FME Flow. From the left sidebar, navigate to the Jobs > Completed page. Find the recently run austinApartments.fmw job by matching the “id” number returned in the API response message.

6. Review the Job Log 
Click on the job to find more details.

An FME Flow job log contains a detailed description of the job translation. Inside, you can review information such as workspace parameters, the number of features read and written, which engine was used, error or warning messages, etc.

At the top of the page, the COMPLETED section contains high-level summary information. Expand Request Data to see the PUBLISHED PARAMETERS sent from our API request. These parameters should align with the JSON values supplied inside our request body: 



Now, let's modify our Transact request to run another job with different parameters.

7. Send a Transact Request with Different Parameters
Back in Postman, select the Body tab. This time, we’re going to submit different values to the “THEMES” parameter. Update the value list to contain “railroad” and “streetcl” instead of “airports” and “cenart”. 

Click Send to submit the updated request. Wait 5-10 seconds to receive a successful response. Note the job’s “id” in the response message.

8. Find the Job in FME Flow
In FME Flow, navigate back to the Jobs > Completed page. Find the last submitted job by ID and select it to review the log.

Under REQUEST DATA, find the updated parameters. 



Scroll down the log to find the Features Written Summary line to review the final output data, as determined by the "THEMES" parameter.



Congratulations! You’ve now sent a synchronous job request via the FME Flow Transact endpoint. Use this request to run jobs and receive the results in the same request.
 

Part 2: Running an Asynchronous Job With Standard Parameters

This section demonstrates the Submit request that runs a job asynchronously on FME Flow. A Submit request instructs FME Flow to submit the job and return a response immediately without waiting for the translation to be completed. The response message does not contain the job results. This means you will have to make subsequent requests to FME Flow to find out the job results.

The FME Flow REST API documentation contains more information about the Submit request in the Transformations section. 



Part 2 assumes that you have already completed Part 1: Running a Synchronous Job with Standard Parameters. For more detail and context for each step, review Part 1. 

1. Create a New Request in Postman
In Postman, a new request can be started by selecting New > HTTP Request or the plus sign ("+") beside our last request. 

Copy this URL to use with our request:

Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Flow’s domain name. Using the dropdown menu, change the method from "GET" to "POST".

2. Create a Request Body
Select and copy the following JSON body to use with our request:

{
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ]
}

In Postman, select the Body tab. Select the raw option, change the type to JSON by clicking on the drop-down arrow, then paste the JSON in the space below. 

3. Create Headers
Select the Headers tab in Postman to add these headers and their values to this job request:
 

The Headers configuration should resemble the screenshot below. Remember to change <yourTOKEN> to your actual token. Note that Postman may include other default headers that can be ignored or hidden. 

4. Run the Job by Submit REST API Request
In Postman, click Send. Wait ~5 seconds while FME Flow receives the request and submits the job. Immediately after the job is received, Postman will display a response message from FME Flow. 

The response from a Submit request only contains the job’s “ID”. Because an asynchronous request doesn’t wait until the translation finishes, there is no other information to report. However, the job ID can be used in subsequent API requests to retrieve the job results and status. 

5. View the Job in FME Flow
Log into FME Flow. From the left sidebar, navigate to the Jobs > Completed page. 

First, look for the recently run austinApartments.fmw job by matching the “id” returned in the API response message. However, since we’ve submitted an asynchronous request, the translation may still be in progress. If the job is not found on the Completed page, look for it on the Running or Queued pages.



Select the job to view the translation log. Expanding REQUEST DATA displays the parameters used in the Submit API request.



Scrolling further down the log, we can also view the total number of Features Written and other summary statistics.

 
Congratulations! You’ve now sent an asynchronous job request via the FME Flow Submit endpoint. Use this request to run jobs without waiting for the results.

Continue to Step 6 to learn how to retrieve job results with the REST API.

6.  Create a new Request in Postman
Now that we have a Job ID, we can use a Job ID REST API request to retrieve our job’s status and results.
 

Back in Postman, open a new tab. Copy this URL to use with our request: 

Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Flow’s domain name. Replace <JobId> with the “id” you received from the response in the previous call. 

This time, leave the method as "GET". A GET method requests information from FME Flow.

7. Create Headers
Select the Headers tab in Postman. When retrieving job information, we don't need to send any data with GET requests, eliminating the need for the Content-type Header.
 

The Headers configuration should resemble the screenshot below. Remember to replace <yourTOKEN> with your actual token. Note that Postman may include other default headers that can be ignored or hidden.

8. Retrieve Job Information by REST API Request
In Postman, click Send. Wait ~5 seconds while FME Flow receives the request and performs the query. When the request is finished, a response message returns information such as the translation summary, workspace details, user, and resources. Find the job id near the bottom of the response.



Congratulations! You’ve now sent a request via the FME Flow Job ID endpoint. Use this request to fetch translation statuses and results after a Submit request.
 

Troubleshooting 

Failed Requests and Error Codes 

The FME Flow REST API returns an error when it rejects a request. The API response contains the error’s code and message. 

For an overview of general error codes and their corresponding messages, review the HTTP Response Codes and Errors section of the FME Flow REST API documentation.



Error messages may also be specific to the request type. For example, a job request might be rejected when a published parameter is invalid. Select a request in the documentation to view its specific Response Status Codes:

Authentication Errors

When a token is rejected, you’ll receive a “401 Unauthorized” or “403 Forbidden” error code. 



Tokens are likely to be rejected when they expire or if permission is not given to access the workflow dependencies. For authentication options and troubleshooting, please review the Authorization section of the FME Flow REST API documentation or the Authorization section of the FME Flow REST API documentation.