Tutorial: Custom Data Delivery using FME Flow

Introduction

There are several ways to customize data delivery with FME Flow. You may make use of the following resources when creating a solution:

• FME Flow Services
• FME Flow Automations
• FME Flow Apps
• Webhooks
• API Requests

FME Flow Services

FME Flow comes with five different services. Each one delivers data in a different way. You can select the one(s) that best meets your needs when publishing your workspace to FME Flow. This tutorial will cover data delivery customization options using the Data Download and Data Streaming services.

Data Download Service

The data download service provides users with a file download link. Any written files from the workspace will be included in the resulting .zip. The most thorough explanation of how to build a good workspace for data download is found in the Getting Started with FME Flow Tutorial. 

This tutorial covers three common techniques for customizing data delivery with the data download service: using generic writers, dynamic workspaces, and changing the zip file contents.

Generic Writers

Generic Writers allow workspace outputs' file formats to be specified by the user when running the workspace. To make use of them, simply route the workspace output to a Generic Writer with the Writer’s “Output Format” parameter linked to a published parameter. 

Dynamic Workspaces

Dynamic workspaces let you apply the same transformation to files of the same format but with different schemas. This is a great option for applying a common transformation to files with different attributes.

Changing the Zip File Contents

Create a more complex download zip file by writing additional files and directories to the output directory.

• Include additional files
• Include the translation log in the results zip file

Data Streaming Service

The Data Streaming service accepts and carries out transformation requests as specified by a workspace, returning the results as a data stream. This allows data to be sent to a client application or viewed by users directly in the browser. The data streaming service is great for PDF, KML, HTML, GeoRSS and more.

Providing Data to Client Applications

Workspaces that write XML, GML, or GeoJSON can utilize the data streaming service and Webhooks in FME Flow to provide data directly to client applications and web services. 

• Stream an XML or GML response within a web service
• Stream GeoJSON and use it in a client application
• Creating a WFS Service using FME Flow

Streaming Data In Browser

The FME Flow Data Streaming Service can also be used to stream data such as an HTML page, PDF, or image directly in a user's browser so they do not have to download anything on their machine to view workspace outputs. For an example, review the Stream a PDF tutorial. Most users will be able to view the PDF output directly in their browser. If the data streaming service cannot stream the output file in the user's browser, it will provide a data download link instead.

FME Flow Automations

FME Flow Automations allow you to create workflows for a wide range of purposes from running workspaces in sequence to scheduling jobs that run at a specific time each day to sending emails upon job completion to using HTTP requests. Automations are comprised of a trigger, internal actions, and external actions. You can use data from the trigger or upstream actions to build complex workflows and outputs.

FME Flow Apps

FME Flow Apps are customizable and intuitive web applications built to run either workspaces or automations. Permission to use FME Flow Apps can be granted to guests, users, and roles.

Workspace Apps

FME Flow Workspace Apps allow users to configure and run workspaces using a registered service. Users do not need FME Flow accounts to use Workspace Apps unless you require them to log in, which makes them a great way to share your workspaces with a wide group of users. Review the Getting Started with FME Flow Workspace Apps tutorial to create your first Workspace App.

Automation Apps

FME Flow Automation Apps act as a manual trigger for FME Flow Automations. Users can fill out manual parameters in a customizable web page, then submit the fields to the Automation. Unlike Workspace Apps, which can only run a single workspace at a time, Automation Apps allow users to run a series of workspaces at once or benefit from other downstream internal and external actions. For example, you may choose to send them an email confirmation after a chain of workspaces finishes running.

Gallery Apps

Gallery Apps can be used to provide users with a collection of related Workspace Apps and Automation Apps on a single, customizable web page. Users must have either guest access or access through their FME Flow account to both the Gallery App and the linked Workspace Apps and Automation Apps. Review the Getting Started with FME Flow Gallery Apps tutorial to create your first Workspace App.

Webhooks 

FME Flow Webhooks and can be used to create a static URL for streaming workspace results to other applications. Workspace webhook URLs can also be bookmarked for easy access to a custom version of a dataset, emailed to a client or colleague, or placed on a static web page for self-serve access to data downloads.

REST API Requests

The FME Flow REST API allows for in-depth programmatic interaction with FME Flow. You can run a workspace, retrieve a workspace's published parameters, monitor job activity, check engine health, and more with the REST API. Responses are available in JSON or XML - set the Accept header, or use the parameter "accept", which can be set to "json" or "xml". For more information, see the API documentation at http://<host>/fmerest/apidoc/v3/

Next Steps

Send a Customized Email

Send a custom email with an attachment or a link to the data. Read the overview of the Email action in FME Flow Automations, then work through the Send Emails with Attachments From FME Flow Automations tutorial.

Upload to FTP

FME Flow includes an FTP Subscriber. This means you can automatically upload the results of a transformation to an FTP site. See the FTP subscriber documentation for more details.

Integrate a Web Map

Allow users to interactively select an area of interest, and send them only the relevant data. See the interactive Data Distribution Demo for an example. Scroll to the bottom of the page to access a tutorial on how to recreate the workspace.

Tweak FME Flow Settings & Configuration

In some cases, you may wish to change some settings or configure components differently for better data distribution:

• Increase the amount of time results are kept before being cleaned up
• Use a reverse proxy