Send Source Data to FME Flow with HTTP POST

Files

Sample.zip
- 10 KB
- Download

As of FME Flow 2025.1, the V3 REST API has been deprecated. The current API will remain accessible, but no new features or general bug fixes will be implemented. Starting with FME Flow 2026.1, the V3 API will be removed. We encourage all users to prepare their systems for this change.

The V4 REST API offers enhanced security and usability, including new and reorganized endpoints, as well as the removal of certain endpoints. For information on migrating from V3 to V4, please see the FME Flow REST API V4 documentation and click on the heading Migrating from REST API V4.

Introduction

To support real-time data applications, you may want to POST data to FME Flow and trigger a workspace that reads this data as a source dataset. For example, you can create triggers for sensors that can send sensor data to an FME Workspace for processing, or you could create your own application that submits form data to FME Flow.

Please Note:

Using the method described in this article, FME Flow accepts raw data in the POST body only but does not accept form data. Form data can be used to submit workspace parameters simply by sending parameter names and values as form element names and values.
Currently, this method is best for simple text data such as XML or JSON and does not support multipart file upload. If you want to upload file data to be used as the source dataset for an FME Workspace, please see the article Data Upload with FME Flow.

The following Web Services can accept raw data sent to FME Flow using HTTP POST:

FME Data Download
FME Data Streaming
FME Job Submitter

You will use the FME Flow REST API and create your own application to solve data challenges and ultimately leverage the power of FME. A REST API enables users to interact with the server by creating requests and receiving responses using HTTP Protocols. Requests are made through URLs, and responses are formatted in JSON. HTTP Methods are essential to every call to the REST API. In other words, HTTP Methods represent an action to FME Flow that you would like to complete. The HTTP Method includes GET, POST, PUT, or DELETE. We will be focusing on the HTTP POST request, which is used to create a publication, project, or notification.

Visit the FME Flow REST API page to learn more about the different actions and requests that are available. You can also access this page via your FME Flow web interface under the Help icon by opening the REST API tab.

Postman is a REST Client Tool that makes calls to FME Flow. If you are familiar with another REST client tool, you may use it as an alternative. The instructions in this article will refer to the Postman interface.

Webhooks are URLs that enable external software to interact with workspaces. To learn more about webhooks and FME Flow, read the Webhooks and FME Flow article.

Requirements

Download Postman
Sample.zip attached to this article

Step-by-step Instructions

Prepare the Data and Create a Workspace

You may use the sample workspace, ReceivePost.fmw, in the “Sample” folder instead of creating your own workspace.

1. Create a simple text file that is structured to represent a sample of data that you will be sending (e.g., an XML or JSON snippet). In the “Samples” folder attached to this tutorial, a very simple data file called Sample.txt may be used. This text file will be called later on when posting to FME Flow with the Data Streaming service.

2. Build an FME workspace that will read the data you POST to it. In most cases, using the Text File Reader is best. Even if the data is XML or JSON, you can still use the Text File reader. Make sure to set the reader parameter "Read Whole File at Once" to Yes. You can then parse the data further along in the workspace using the appropriate XML or JSON transformers.

3. Add a writer to the workspace for output data - which will be sent to the Data Download or Data Streaming services. In the sample workspace attached (ReceivePost.fmw), we simply use a Text File writer to return the same data.

4. From the Navigator Panel, right-click on User Parameters and select "Manage Parameters".
Ensure that the Required checkbox is disabled for the Source Text File(s). In order for the call to work, the source file parameter needs to be optional, so ensure that the Required box is not checked. The Destination Text File can remain enabled for the Required checkbox.

Create Folders in FME Flow & Link to FME Workbench

5. Log in to your FME Flow web interface using your admin account. Your admin account should be enabled with sufficient permissions for the following steps. The default permissions for the fmeauthor role will work too, but for best practice, use your admin account here. On the Resources page, select the Data folder and click New. Name the new folder "SendSourceData" and click Create. This will be the folder that the workspace will be writing out to.

6. Select the newly created folder "SendSourceData", click "Actions" and go to "Properties".

A new window will appear, giving the folder's file properties. Copy the SystemPath:

 "$(FME_SHAREDRESOURCE_DATA)/SendSourceData"

7. Return to FME Workbench. In the Navigator panel, expand User Parameters, right-click the [DestDataset_TEXTLINE] Destination Text File parameter, and select Edit Value. Paste the copied System Path into the Destination Text File value:

 $(FME_SHAREDRESOURCE_DATA)/SendSourceData

Finally, specify the output file name and format by appending "/out.txt" to the end of the System Path:

$(FME_SHAREDRESOURCE_DATA)/SendSourceData/out.txt

Save the workspace.

Publish the Workspace to FME Flow

8. In FME Workbench, click the "Publish to FME Flow" button or select File > Publish to FME Flow from the menu bar. Connect to your FME Flow and publish the workspace into a new repository named “Training”.

Select a valid FME Flow connection, then click "New..." to create a new repository for the workspace. Name this new repository "Training" and click OK.

For complete instructions on how to publish to FME Flow, please see the information below:

To publish a workspace from FME Workbench to FME Flow, follow the generic instructions below using the specific details listed in the article. If the article does not have a specific repository name listed, we recommend creating a Training repository. If a specific service is not listed in the article, please use the default Job Submitter.

1. Click Publish

From the File menu, click Publish to FME Flow, or from the toolbar ribbon, click Publish.

Toolbar Ribbon:

File Menu:

Deploy Menu (2025.1+):

2. Add a Web Connection

In the Publish to FME Flow wizard, a FME Flow connection needs to be selected or created. To create an FME Flow connection, click the drop-down under FME Flow Connection and select Connect to FME Flow.

The following instructions are for a basic connection, but if your FME Flow is set up for Microsoft Azure, SAML, or Windows Credentials, please see the Using Web Connections to Connect to FME Flow documentation.

An FME Flow Connection dialog will appear; here, we will set up the connection. The connection information and credentials can be obtained from your FME Flow Administrator. If you have not yet logged into FME Flow after installing, please do so prior to continuing, as you will be prompted to change the password from the default:
Username: admin
Password: admin
- • Web Service: FME Flow
  
  • Server URL: http://localhost
  
  • Connection Name: Training FME Flow
  
  • Authentication: Basic
  
  • Username: your username
  
  • Password: your password
  
  Click OK. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
- • Web Service: FME Server
  
  • Connection Name: Training FME Server
  
  • Server URL: http://localhost
  
  • Username: your username
  
  • Password: your password
  
  Click Authenticate. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
  
  Click Next to move to the Create Repository page of the dialog
3. Create Repository

Repositories are used to store groups of related items that have been published to FME Flow (similar to directories). Items such as workspaces, source data, custom formats, custom transformers, and templates can be published to a repository.

If you need to create a new repository, click on the New button next to Repository Name. Enter the repository name and an optional description. Click OK to close the Create New Repository dialog.

To select an existing repository, click the drop-down next to Repository Name and select the desired repository.

4. Upload Files

Some workspaces require additional files to run correctly on FME Flow, so these need to be uploaded along with the workspace. To upload files, enable Upload data files at the bottom of the dialog.

When files are uploaded through the publishing menu, they are stored within the repository. If you want to upload folder files (such as geodatabases) or to specify where the files stored, the Select Files button allows you to do that. For detailed instructions on how to use the Select Files button, see the documentation.

5. Register Services

The final step before publishing a workspace is to register services. Services control how the data will be output after the workspace is run.

Services return results in different forms:

• Data Download Service returns results as a downloadable zip file

• Data Streaming Service returns results as a data stream

• Job Submitter Service accepts and runs workspace job requests

• KML Network Link returns a KML Network Link that can be used in Google Earth

• Notification Service allows for event-driven messaging

Typically, the Job Submitter is always enabled, but multiple services can be enabled. Once services have been enabled, click Publish.

6. Review Translation Log

To confirm whether or not the workspace was published successfully, check the Translation Log. The Translation log will show which repository you published to, which files were included, and a quick link to run the workspace.

9. Ensure "Upload data files" is enabled. Click "Select Files..." to ensure the path to your Sample.txt file is correct. Select OK and click Next.

10. In the last dialog box of the Publish to FME Flow wizard, click on the Edit button beside the Data Download service.

First, ensure that the "Send HTTP Message Body to Reader" parameter is set to your reader. Below this is another parameter called "Writers in Download". Ensure this is set to your Text File Writer.

11. Repeat step 10 for the Data Streaming and the Job Submitter services. Ensure Data Download, Data Streaming, and Job Submitter services are selected and click Publish.

Create a Webhook

12. On the FME Flow web interface, go to Run Workspace. Select the Repository "Training" and the Workspace "ReceivePost.fmw" that was just published. Select the Job Submitter service and ensure that the Destination Text File ends with ".../out.txt".

Click Run to ensure the Workspace runs successfully.

13. Once the ReceivePost.fmw is complete and the translation is successful, go to Resources > Data > SendSourceData > out.txt and delete this file. To delete the file, select it, go to Actions, and click Remove. Next, we will run the workspace via REST API.

16.5 - remove txt.png

14. Return to the Run Workspace page. Open the Advanced panel and select "Create a Webhook". The webhook enables third-party software to run this workspace programmatically.

The API token associated with this webhook allows access to run any other workspaces in the same repository that are registered to the same service.

15. On the Create Webhook page, a Webhook URL for that repository will be automatically filled with the Token Name, Description, and Expiration date. You may choose to change the expiration date here. Enable "User can upload", and select Create. We will manage the Token's permissions after the webhook is created.

16. Download the webhook to save the information. This is the only opportunity you will have to access the token and instructions for using this webhook, so please don't skip this step.

17. Now that the webhook is created, go to the FME Flow User Settings (top right corner) and select Manage Tokens.

This will bring you to the API Tokens page. To update the permissions, click on the "Webhook-ReceivePost from training" token that was created.

18. In the token's Permissions panel, enable Access to Repositories, and expand with the down arrow to reveal additional options. Enable Download, Read, and Run for the "training" repository.

Next, enable Create for Resources, and expand with the down arrow to enable Access, List, Write, and Upload for the Data Folder.

Select Advanced for Run Workspace.
22.2 run workspace.png

Finally, click the down arrow for Services to select the type of services you need with this webhook token. Enable Full Access to Data Download, Data Streaming, Data Upload, and Job Submitter if you want to only use one webhook for these services. At the bottom of the page, click Save to apply the new permissions to your webhook token.

Using Postman to Make an HTTP POST Request

19. Download Postman using this link https://www.postman.com/downloads/. Postman is a REST client tool to make calls to FME Flow.

20. Open Postman and select "Continue without an account".

From the main Postman workbench view, click the "+" icon to create a new HTTP request.

21. Before posting to FME Flow, let's start with a GET request to do a health check from FME Flow using the REST API.

Use the GET method and the following URL to send the request to your FME Flow. Please note, you will need to replace <yourServerHost> with your FME Flow hostname.

http://<yourServerHost>/fmerest/v3/healthcheck?textResponse=false

Click Send. The Body should return a JSON response of "status" : "ok" (200), which is a standard response for successful HTTP requests.

Note: This will be the only call that does not require a token.

22. Let's proceed with the HTTP POST request. In Postman, create a new request by clicking the + on the tab. Change the HTTP request from GET to POST.

From the webhook file that you downloaded in Step 16 (fmewebhook-Webhook-ReceivePost from training -<yourToken>.txt), copy the Authorization with Header URL and paste into the "Enter request URL" parameter in Postman.

http://<yourServerHost>/fmejobsubmitter/training/ReceivePost.fmw?SourceDataset_TEXTLINE=%24(FME_MF_DIR)Sample.txt&DestDataset_TEXTLINE=%24(FME_SHAREDRESOURCE_DATA)%2FSendSourceData%2Fout.txt&opt_showresult=false&opt_servicemode=sync

Go to the Headers tab and add a new header. For the Header Key, enter "Authorization", and for the Header Value, enter "fmetoken token=<yourToken>".

23. Click Send in Postman, and a Status 200 OK should return, signifying a successful HTTP request.

24. Since our token also has permissions enabled for the Data Download and Data Streaming services, we can access those services by simply changing a string in the request URL from "fmejobsubmitter" to "fmedatadownload".

In Postman, create a new request by clicking the + on the tab. Ensure the HTTP request is set to POST. Copy the Authorization with Header URL from your downloaded webhook file into the field. This time, we will edit the URL by changing the "fmejobsubmitter" string to "fmedatadownload".

http://<yourServerHost>/fmedatadownload/training/ReceivePost.fmw?SourceDataset_TEXTLINE=%24(FME_MF_DIR)Sample.txt&DestDataset_TEXTLINE=%24(FME_SHAREDRESOURCE_DATA)%2FSendSourceData%2Fout.txt&opt_showresult=false&opt_servicemode=sync

Navigate to the Headers tab and enter the same values that were used previously: for the Header Key, enter "Authorization", and for the Header Value, enter "fmetoken token=<yourToken>". Click Send.

A Status 200 OK should return. Under Body, select "Preview" to format the HTML. You will find the link to download the zip file. Paste the zip file link to your web browser and the data will be downloaded onto your machine.

25. Repeat step 25 for the Data Streaming service, except this time, replace the "fmedatadownload" string in the URL with "fmedatastreaming."

http://<yourServerHost>/fmedatastreaming/training/ReceivePost.fmw?SourceDataset_TEXTLINE=%24(FME_MF_DIR)Sample.txt&DestDataset_TEXTLINE=%24(FME_SHAREDRESOURCE_DATA)%2FSendSourceData%2Fout.txt&opt_showresult=false&opt_servicemode=sync

Once you click Send, a Status Code 200 OK should be returned. In the return Body, the Data Source will be visible in the text format that was uploaded from the Sample.txt file.

To upload a text file as your source data, return back to the Headers section. Under the Authorization Key, we will be adding two more Keys-Value pairs:

Key: "Content-Type", Value: "text/plain"
Key: "Accept Key", Value: "application/json"

In the Body tab, select binary and upload the test.txt file from the Sample folder. You may choose to create your own text file or edit the test.txt file. Click "Send" and the Status should return 200 OK with the return Body "Hello this is sending source to FME Flow with HTTP POST".

Congratulations! You’ve successfully posted data to FME Flow and triggered a workspace that reads this data as a source dataset using several different FME services.