Tutorial: Custom Data Delivery using FME Server

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2021.1

Components

Data delivery may make use of the following components:

  • Data Download service
  • Data Streaming service
  • Workspace to extract data
  • Notification topic with subscriber (email, FTP etc.)
  • Custom web form to request data


Exploring the FME Server Data Download Service

As a learning exercise, here are two different ways to access the Data Download service. Try them out to get a feel for working with it directly and through the REST API.

 

The Data Download URL format

  1. Open your FME Server web interface
  2. Go to Workspaces > Samples > austinDownload.fmw > Service: Data Download > Run
  3. Select the Data Download URL which will download as a zip file. Unzip on your machine. 

 

REST API

To retrieve a workspace’s published parameters using REST API, you can visit
http://<host>/fmerest/v3/repositories/Samples/items/austinDownload.fmw/parameters
Responses are also available in JSON or XML - set the Accept header, or use the parameter "accept", which can be set to "json" or "xml". For more information, see the API documentation at https://<host>/fmerest/apidoc/v3/

 

Building the workspace

The most thorough explanation of how to build a good workspace for data download is found in the FME Server Tutorial

Common techniques for building a Data Download workspace include:

  • Routing output to a Generic Writer, with the Writer’s “Output Format” parameter linked to a published parameter, type “Choice with Alias”
  • Reading from a format with layers, such as a MapInfo TAB file, Esri File Geodatabase or Oracle Spatial Database. This allows selection of datasets, through the Reader > Parameters > Advanced > Feature Types to Read parameter in the Navigator Pane.
  • Dynamic schema, since the datasets being written do not necessarily have the same attribute fields
  • Grouping layers using display names

 

Building a static data download page

Once you have built a workspace, you will need to build a web form to collect user input and submit the request to run the workspace.

Using the instructions for the REST interface above, retrieve the parameters for the workspace. You will need the “name” of each parameter, and its “value” options.

See the resources at W3Schools for more information about HTML forms.

For a working example of a simple form you can modify and interact with live, see the Run Job Synchronously demo on the FME Server Developer Playground.


Option A: Synchronous - Parsing the response

If you just want to link to the data after submitting the form, you can request an HTML response from FME Server. This will return a web page with a link to the data. The styling of the response page can be customized.

See “Next Steps” below if you want to do something more complex.

If you use the Data Streaming service, you can create a workspace that outputs XML or JSON, and parse it using JavaScript, or simply display the data immediately. For more information about parsing XML and JSON, see these resources on XML and JSON.


Option B: Asynchronous - Sending an email

You can also run a workspace through the data download service in “async” mode to have it send an email upon completion. You will need to configure an email subscriber to the DATADOWNLOAD_ASYNC_JOB_SUCCESS topic. See Sending Email Part 1 for more details.


Configuring Security

To restrict the use of the web form to authenticated users, you can also request a username and password, and generate a token. The token must be appended as a parameter. A token can be generated on the fly by asking for a username and password. See Request a Token on the FME Server Developer Playground.

Alternatively, for public access, you could assign the fmeguest role permissions to run the workspace through the REST interface. The Guest user is disabled by default, so you may need to enable it first through the User settings. See the FME Server Documentation on Permissions.

Or, if you already have Active Directory setup in your organization, leverage it to define security roles in FME Server using Active Directory groups.

 

Next steps

Do more with your links

So far, we’ve hinted at what you can do with a workspace URL request link once you’ve generated it, but here are some specific suggestions:

  • Bookmark the link for easy access to a custom version of a dataset
  • Email it to a client or colleague. If they can access the FME Server, they can get the most up-to-date version of the data with a single click.
  • Put it on a static web page. No need to run a scheduled process to update infrequently used data downloads - let users generate it on-demand.


Use Data Streaming

Stream the data instead of downloading. The Data Streaming service accepts and carries out transformation requests as specified by a workspace, returning the results as a data stream, instead of providing a link to it. To use streaming, simply publish to the Data Streaming service instead of Data Download. Great for PDF, KML, HTML, GeoRSS and more.


Change zip file contents

Create a more complex download zip file by writing additional files and directories to the output directory.


Send customized email

Send a custom email with an attachment or a link to the data. Read the introductory material at Email in FME Server and then work through the Sending email tutorial.


Upload to FTP

Since FME Server 2013, an FTP Subscriber is included. This means you can automatically upload the results of a transformation to an FTP site. See the FTP subscriber documentation for more details.


Integrate a web map

Allow users to interactively select an area of interest, and send them only the relevant data. Demos are available using:


Tweak FME Server Settings & Configuration

In some cases, you may wish to change some settings or configure components differently for better data distribution:

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.