Submitting a Job through the FME Server REST API

Sanae Mendoza
Sanae Mendoza
  • Updated

FME Version

  • FME 2022.0

Introduction

The FME Server REST API allows third-party applications to run jobs on FME Server. By connecting your systems and users over the web, data and information can be exchanged in real-time between otherwise unrelated systems.

All workspaces submitted by REST API use the FME Server REST Service. The REST Service allows you to run jobs on FME Server, much like using the job submitter service. To use the Transformation services, like data streaming or data download, use a Webhook URL instead. 

Jobs may be submitted by two different REST API requests: Transact and Submit. These two endpoints represent synchronous and asynchronous requests, respectively. On a practical level, these requests differ in how and when job results are returned. Which you choose will be determined by your specific workflow needs. 

The Transact request is synchronous. A synchronous task must be completed before the next task begins. After submitting a synchronous job, the API request is not finished until the job is complete. The translation results are returned in the API response. The Transact request is a good choice for short workflows or when results are required as soon as possible. 

The Submit request is asynchronous. An asynchronous task does not need to be completed before the next task begins. After submitting an asynchronous job, the API request is finished, regardless of the job status. A job ID is returned in the API response. To find out the translation's status or retrieve any results, the job ID can be used in subsequent requests. The Submit request is a good choice for long workflows or when immediate results are not required.

In this tutorial, we'll run a job on FME Server by both Submit and Transact requests. The requests will be built and made from the API development platform, Postman. However, the same requests can be made from any HTTP client, like the HTTPCaller in FME Workbench. Please note: for applications outside of your network to submit requests by REST API, please ensure that your FME Server instance is publicly available. 

Requirements

 

Step-by-Step Instructions

In this step-by-step tutorial, we will demonstrate submitting both synchronous and asynchronous jobs through the FME Server REST API. We’ll use the "austinDownloads.fmw" workspace for both requests. This workspace is included inside the “Samples” repository of every FME Server installation. 

Preparation is key to building successful workflows. There are some steps we can take before creating a job request that can help us be successful on the first try.
 

Review the Workflow Options

Whenever possible, review the workflow or data before building an API request. From the FME Server Run Workspace page, select the “Samples” Repository and “austinDownloads.fmw” as the Workspace. 

WebRunJob.png
 
Published Parameters can be set inside the request body of any REST API job submission. For any workspace, there are a couple of ways to find which parameters are available to use. 

From the Run Workspace page, scroll to find and expand the Advanced section. At the bottom of this section, you will find a table named Published Parameters. The Parameter column contains each available parameter. The Option column lists its accepted inputs. These values can be used with our API job requests.

WebParameterOptions.png
 
Workspace published parameters may also be fetched by API request. Read more about how to retrieve and submit different types of published parameters in the Published Parameter Data Model tab of the FME Server REST API Documentation.

 

DocPP.png

Documentation

For all your web workflows, the best source of guidance will usually be the API documentation. Visit the FME Server REST API Documentation to get started, follow along with these requests, and explore the many endpoints available to use with FME Server. 

Alternatively, you can access REST API documentation personalized to your own FME Server through the Help menu. This page allows you to test the listed API requests directly inside the browser.
 

Authorizing Requests

Just like a user needs to log into FME Server before running a workspace, an application needs to provide identification before running a job. The FME Server REST API uses tokens to authenticate requests. For any workflow, multiple tokens can be created with permissions to only what they need to complete their request. 
 
This tutorial assumes you already have a token. If you do not already have a token, create one by following our FME Server REST API Authorization tutorial or from the FME Server Token Management page.  
 

Part 1: Running a Synchronous Job With Standard Parameters

This section demonstrates the Transact request that runs a job synchronously on FME Server. A Transact request instructs FME Server to submit the job and only return a response upon the job’s completion. This means that you will wait until the translation is complete and the response message contains the job results.

More information about the Transact request can be found under the Transformations section of the FME Server REST API documentation. 

DocTransact.png

1. Create a New Request in Postman
In Postman , a new request can be started by selecting New > HTTP Request or the plus sign ("+") button above the workspace window. 

PostmanRequest.png

A URL is the calling card of any REST API Request. We can use a URL to target the specific FME Server domain, endpoint, and workspace. Copy this request URL to use with our request:
 

POST HTTP://<yourServerHost>/fmerest/v3/transformations/transact/Samples/austinDownload.fmw


Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Server’s domain name. 

Oftentimes, workspaces require parameter values to run the translation. The POST method allows us to submit these parameters and other data with each request. Using the dropdown menu, change the method from "GET" to "POST".

PostmanTransact.png 

TIP: To learn more about the building blocks of an API Request and how to send requests from FME Workbench, review HTTP Requests with the HTTPCaller.

 
2. Create a Request Body
For every POST request, the Body contains data to send to FME Server. The transact request body will contain our workspace published parameters and values in JSON. 

Select and copy the following JSON body to use with our request:

{
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ]
}


In Postman, select the Body tab below the POST request. Select the raw option, change the type to JSON by clicking on the drop-down arrow, then paste the JSON in the space below. 

PostmanBody.png
 

Notes on the request JSON: You may notice that our JSON does not specify all the parameters available for austinApartments.fmw. Whenever parameter values are not specified like this, FME Server uses default values. Additionally, the name value refers to the workspace User Parameter name, not the label (as you might see from the Run Workspace page). 

 
3. Create Headers
Headers
allow us to tell FME Server more information about our API request. 

We can start by telling FME Server that our Body content is in JSON format with a “Content-Type” header.

Next, we can ask FME Server to return its response in JSON format with the “Accept” header.

Lastly, with an “Authorization” header, we can supply our unique token value. Replace <yourTOKEN> with your token value. If you do not already have a token, create one by following our FME Server REST API Authorization tutorial or from the FME Server Token Management page.  


Select the Headers tab in Postman to add these headers and their values to this job request:

Key Value
Content-Type application/json
Accept application/json
Authorization fmetoken token= <yourTOKEN>

 
The Headers configuration should resemble the screenshot below. Note that Postman may include other default headers that can be ignored or hidden.

Headers.png

4. Run the Job by Transact REST API Request
In Postman, click Send. Wait 5-10 seconds while FME Server receives the request and submits the job. Upon the translation completion, Postman will display a response message from FME Server.

The response from a Transact request contains information about the completed job: the job id, status, how many features were handled, the time and duration of the request, and any messages. A synchronous request waits until the translation finishes before returning a response, so FME Server can immediately share the job summary. 

PostmanTransactResponse.png

 

 
5. View the Job in FME Server
Any job requests, including those sent from the REST API, can be viewed from the FME Server Web UI. 

Log into FME Server. From the left sidebar, navigate to the Jobs > Completed page. Find the recently run austinApartments.fmw job by matching the “id” number returned in the API response message.

JobHistory.png


Note: a job submitted by REST API will be listed as run by the User account that created the authentication token. 

 
6. Review the Job Log 
Click on the job to find more details.

An FME Server job log contains a detailed description of the job translation. Inside, you can review information such as workspace parameters, the number of features read and written, which engine was used, error or warning messages, etc.

At the top of the page, the COMPLETED section contains high-level summary information. Expand Request Data to see the PUBLISHED PARAMETERS sent from our API request. These parameters should align with the JSON values supplied inside our request body: 

WebParameters.png

Now, let's modify our Transact request to run another job with different parameters.
 
7. Send a Transact Request with Different Parameters
Back in Postman, select the Body tab. This time, we’re going to submit different values to the “THEMES” parameter. Update the value list to contain “railroad” and “streetcl” instead of “airports” and “cenart”. 

PostmanNewParameters.png


TIP: Refer to the Introduction section, Review Workflow Options, to locate all the parameter values available to use with this workspace. 


Click Send to submit the updated request. Wait 5-10 seconds to receive a successful response. Note the job’s “id” in the response message.
 
8. Find the Job in FME Server
In FME Server, navigate back to the Jobs > Completed page. Find the last submitted job by the id. Select the job to review the log.

Under REQUEST DATA, find the updated parameters. 

WebRequestParameters2.png

Scroll down the log to find the Features Written Summary line to review the final output data, as determined by the "THEMES" parameter.

JobLogFeaturesWritten.png

Congratulations! You’ve now sent a synchronous job request via the FME Server Transact endpoint. Use this request to run jobs and receive the results in the same request.
 

Part 2: Running an Asynchronous Job With Standard Parameters

This section demonstrates the Submit request that runs a job asynchronously on FME Server. A Submit request instructs FME Server to submit the job and return a response immediately, without waiting for the translation to complete. The response message does not contain the job results. This means you will have to make subsequent requests to FME Server to find out the job results.

More information about the Submit request can be found under the Transformations section of the FME Server REST API documentation. 

DocSubmit.png

Part 2 assumes that you have already completed Part 1: Running a Synchronous Job with Standard Parameters. For more detail and context for each step, review Part 1. 
 
1. Create a New Request in Postman
In Postman, a new request can be started by selecting New > HTTP Request or the plus sign ("+") beside our last request. 

Copy this URL to use with our request:

POST HTTP://<yourServerHost>/fmerest/v3/transformations/submit/Samples/austinDownload.fmw


Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Server’s domain name. Using the dropdown menu, change the method from "GET" to "POST".

PostmanSubmit.png
 
2. Create a Request Body
Select and copy the following JSON body to use with our request:

{
  "publishedParameters": [
    {
      "name": "MAXY",
      "value": "42"
    },
    {
      "name": "THEMES",
      "value": [
        "airports",
        "cenart"
      ]
    }
  ]
}

 
In Postman, select the Body tab. Select the raw option, change the type to JSON by clicking on the drop-down arrow, then paste the JSON in the space below. 

PostmanBody.png
 
3. Create Headers
Select the Headers tab in Postman to add these headers and their values to this job request:
 

Key Value
Content-Type application/json
Accept application/json
Authorization fmetoken token= <yourTOKEN>

 
The Headers configuration should resemble the screenshot below. Remember to change <yourTOKEN> to your actual token. Note that Postman may include other default headers that can be ignored or hidden. 

Headers.png
 
4. Run the Job by Submit REST API Request
In Postman, click Send. Wait ~5 seconds while FME Server receives the request and submits the job. Immediately after the job is received, Postman will display a response message from FME Server. 

The response from a Submit request only contains the job’s “id”. Because an asynchronous request doesn’t wait until the translation finishes, there is no other information to report. However, the job id can be used in subsequent API requests to retrieve the job results and status. 

PostmanSubmitJobResponse.png

TIP: Take a moment to compare this response to the Transact (synchronous) response (Part 1, Step 4). How does it differ?

 
5. View the Job in FME Server
Log into FME Server. From the left sidebar, navigate to the Jobs > Completed page. 

First, look for the recently run austinApartments.fmw job by matching the “id” returned in the API response message. However, since we’ve submitted an asynchronous request, the translation may still be in progress. If the job is not found on the Completed page, look for it on the Running or Queued pages.

SubmitJobCompleted.png

Select the job to view the translation log. The parameters used in the Submit API request can be viewed by expanding REQUEST DATA.

SubmitJobStatus.png

Scrolling further down the log, we can also view the total number of Features Written and other summary statistics.

SubmitJobLogFeaturesWritten.png 
Congratulations! You’ve now sent an asynchronous job request via the FME Server Submit endpoint. Use this request to run jobs without waiting for the results.

Continue to Step 6 to learn how to retrieve job results with the REST API.
 
6.  Create a new Request in Postman
Now that we have a Job ID, we can use a Job Id REST API request to retrieve our job’s status and results.
 SubmitJobIDRequest.png

Back in Postman, open a new tab. Copy this URL to use with our request: 

GET http://<yourServerHost>/fmerest/v3/transformations/jobs/id/<JobID>


Paste the URL into the space provided in Postman. Update <yourServerHost> to your FME Server’s domain name. Replace <JobId> with the “id” you received from the response in the previous call. 

This time, leave the method as "GET". A GET method requests information from FME Server.

SubmitJobIDRequestURL.png
 
7. Create Headers
Select the Headers tab in Postman. We don't need to send any data with GET requests, like when retrieving job information. This eliminates the need for the Content-type Header.
 

Key Value
Accept application/json
Authorization fmetoken token= <yourTOKEN>

 
The Headers configuration should resemble the screenshot below. Remember to replace <yourTOKEN> with your actual token. Note that Postman may include other default headers that can be ignored or hidden.

SubmitJobIDRequestHeaders.png
 
8. Retrieve Job Information by REST API Request
In Postman, click Send. Wait ~5 seconds while FME Server receives the request and performs the query. When the request is finished, a response message returns information such as the translation summary, workspace details, user, and resources. Find the job id near the bottom of the response.

SubmitJobIDRequestResponse.png

Congratulations! You’ve now sent a request via the FME Server Job Id endpoint. Use this request to fetch a translation status and results after a Submit request.
 

Troubleshooting 

Failed Requests and Error Codes 

The FME Server REST API returns an error when it rejects a request. The API response contains the error’s code and message. 

Review the HTTP Response Codes and Errors section of the FME Server REST API documentation for an overview of general error codes and their corresponding messages.

DocErrorCodes.png

Error messages may also be specific to the request type. For example, a job request might be rejected when a published parameter is invalid. Select a request in the documentation to view its specific Response Status Codes:

ErrorsSubmitCodes.png

Authentication Errors

When a token is rejected, you’ll receive a “401 Unauthorized” or “403 Forbidden” error code. 

TS_401.png

Tokens are likely to be rejected when they are expired or do not have permissions to the workflow dependencies. Please review Authorization in the FME Server REST API or the Authorization section of the FME Server REST API documentation for authentication options and troubleshooting.
 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.