Automating Workflow Submission from a Third Party Application To FME Flow

Introduction

FME Flow web integrations allow you to connect workflows across multiple applications, users, databases, and systems, making information available wherever and whenever it’s needed. In addition to being a powerful integration platform, FME offers the tools to validate, enrich, and transform data so it always reaches its destination regardless of format, schema, or quality. 

Third-party applications can run jobs and Automations in FME Flow using either the REST API or a webhook. Both options follow HTTP standards and can be used in combination.
 

What is the Difference Between REST API and Webhook?

A REST API defines standards that allow applications to communicate over the web. Many applications provide an API for other applications to request and exchange information, events, and data.

Third-party applications can use the FME Flow REST API to make a variety of requests, including submitting a job or running an automation. FME Flow can also make requests to third-party applications by their REST APIs; see Tutorial: Getting Started with APIs.

Using APIs, an application must proactively request event information from another application.



Webhooks use the same communication as a REST API but are lightweight and specific to event-driven requests. Where an application API is often comprised of many kinds of requests, a webhook is typically just a URL that receives data from another application.

When an event occurs in a third-party application, it can send that event information to FME Flow via a webhook. FME Flow does not need to request that information; it can just wait until it arrives. FME Flow can also use another application's webhook to send event information in real time.

With a webhook, an application can receive event information from another application in real time.




Think of a webhook as a courier who rings your doorbell to deliver a package, as opposed to a package left at the post office. The difference is whether you receive the package as it arrives or go pick it up yourself at the post office.

The courier (webhooks) represents a single, convenient service: it delivers packages right to your door. The post office (REST API) represents more services and functionality: you go there to pick up a package, buy stamps, send mail, etc. However, you have to go there to check for packages yourself, and the package may or may not have arrived yet.

Webhooks can be favorable if you prefer to get real-time updates, and they can be less taxing on the client and server because they don’t have to repeatedly make the same request. However, REST APIs tend to offer more comprehensive interfaces.
 

Requirements

In order for a third-party application to reach your FME Flow from outside of your network, FME Flow must be publicly accessible. Your IT team will be responsible for choosing the best configuration for your organization, but please see Enabling FME Flow for Public Access for our recommendations and requirements. FME Flow Hosted, our cloud-hosted and managed FME Flow deployment, is publicly accessible out of the box and requires no additional configuration. 

Running workflows with the FME Flow REST API

Third-party applications can initiate workflows in FME Flow using the FME Flow REST API. For an overview of the FME Flow REST API and step-by-step instructions for how to use it, see Getting Started with the FME Flow REST API.

Running Jobs by API Request

The FME Flow REST API offers the most comprehensive solution for submitting jobs over the web. Parameters, authentication methods, and datasets are sent through POST requests. Jobs submitted via the API are limited to the FME Flow REST Service (i.e., it does not have access to Transformation Services).

To learn how to use the REST API to submit a job to FME Flow with custom parameters, see Submitting a Job through the FME Flow REST API.
 

Running Automations by API Request

In FME Flow Automations, a Manual Trigger can be used to run an automated workflow upon receiving a request from an application. Data can be received in JSON, XML, or text. Data is sent through POST requests, and this method requires token or user-based authentication.

To learn how to create an FME Flow Automation with the Manual Trigger that can be initiated by the REST API Trigger request, see Running an Automation using the Manual Trigger.

Running Workflows with FME Flow Webhooks

Third-party applications can run jobs and Automations in FME Flow using a webhook URL. Any FME workspace can be initiated from a URL with parameters and authentication by GET or POST request. To learn how webhooks work in FME and for a list of step-by-step tutorials, see Working with FME and Webhooks.

Running Jobs by Webhook URL

Any workspace uploaded to FME Flow can be initiated from a URL. A webhook URL allows jobs to be run, with parameters and authentication, by GET or POST request. This makes it a versatile integration solution for custom web applications.

Webhook URLs are the only option for using the FME Flow Transformation Services, allowing the results to be delivered back in different ways. 

Running Automations by Webhook Trigger

A Webhook Trigger allows applications to send real-time messages and data to an FME Flow Automation. Applications can send messages, data, or notifications to initiate an automated workflow. Webhooks are increasingly supported by many applications and databases. Webhook data can be received in JSON and a variety of other formats.


 

Web Services

Web Services extend your FME environment to be fully customizable, programmable, and capable of delivering results in dynamic ways. Web Services can be divided into Transformation, Notification, and Utility services, depending on their function. Collectively, they represent FME's service-oriented architecture (SOA).

FME’s Web Services can be used for individual tasks or in combination.

Transformation Services

Deciding which services are needed for your workflow is important when choosing whether to run jobs through the REST API or Webhook. The most important distinction is that the REST API submits jobs through the REST Service, whereas Webhook URLs submit jobs through the Transformation Services. Automations do not use transformation services.

For a comparison of three transformation services (Job Submitter Service, Data Download Service, Data Streaming Service) and how to use them, see Which FME Flow Service Should I Use?

Utility Services 

The Data Upload service is a Utility Service that allows an application or user to upload source data or other file-based resources by API. This advanced tutorial demonstrates uploading datasets through the API. See the documentation and Data Upload Tutorial: REST API for instructions.
 

Authenticating Third-Party Requests

Tokens allow third-party applications and users to authenticate with FME Flow, even without an account. Access to FME Flow can be limited so applications have access only to a specified workflow. REST and Webhook job submissions can pass tokens in the headers or query string parameters.
 
See Token Management in FME Flow for an overview and instructions for creating and configuring tokens that may be used for job requests.
 

Directing Submitted Jobs with Job Directives

Third-party requests can include Job Directives that determine how FME Flow runs the job. They allow an application to set a priority, queue, maximum processing time, or expiration for the job submission. Both REST API and Webhooks have standardized ways of including job directives.
 

Resources

Documentation: FME Flow Rest API V3 
Tutorial: Getting Started with the FME Flow REST API
Blog: Connect to APIs and Webhooks in No Time 
Landing Page: Getting Started with FME and the Web
Landing Page: Working with Web Services and FME