Create Self-Serve Access to Data with FME Flow

Introduction

In the first tutorial in this series, you learned how to publish a workspace to FME Flow (formerly FME Server) and run it from the Web User Interface. Now, you will modify that workspace to allow for “self-serve” data access. This will be accomplished by creating published parameters in FME Workbench, publishing the workspace to FME Flow, and seeing how to interact with the published parameters.

Step-by-Step Instructions

1. Open FME Workbench

Open the template attached to this article (CommunityMapping.fmwt) in FME Workbench.

The Generic writer's default writer format is Esri Shapefile. However, the goal is for the user to select the format of their choice from a small list at run time, which is accomplished with published parameters.

FME parameters control how FME operates. They exist in many places, such as readers, writers, and transformers.

2. Open the Generic Writer Published Parameters

In the Navigator window, expand the Output [GENERIC] writer by clicking the arrow to the left, then expand Parameters.

Double-click the Output Format parameter to open the parameters. The user can currently select from any format in the formats gallery.

But that’s not what we want. The goal is to present the user with a list of four formats. So, we will create a new user parameter and define it accordingly.

3. Create a New User Parameter

In the Navigator window, right-click on User Parameters and select Manage User Parameters.

Click on the green plus sign, then select Choice. 


On the right-hand side, fill in the following parameter properties:

• Parameter Identifier: Output_Format
• Prompt: Select an output format
• Published: Enabled
• Required: Disabled
• Disable Attribute Assignment: Enabled
• Choice Configuration: Drop-down
  • Then for Choices, enter the following:

Then select Esri Shapefile as the Default Value and click OK. 

4. Link Published Parameter to Output Format Parameter

Our new Output_Format parameter appears under User Parameters > Published Parameters, but we still have to link it to the Generic Writer’s Output Format parameter. Locate the Output [GENERIC] Writer, and expand it with the arrow to view the Parameters, the first one being Output Format. Right-click on Output Format, select Link to a Parameter.

Select the new Output_Format parameter we just created.

5. Run the Workspace

If you run the workspace immediately, it will use the default value and output Esri Shapefile. We need to be prompted to change the parameters. Click the drop-down arrow next to Run and enable Prompt for User Parameters, now click Run. You will be prompted to choose the output format. Notice that only those four formats are presented now in the drop-down list. Pick GML and click Run. 

6. Publish Features Types to Read Parameter

Now, let’s set up the workspace so that the user can choose which feature types to request.

In the Navigator, expand the CommunityMap [FILEGDB] File Geodatabase Reader > Parameters > and double click Features to Read. This parameter lets the user decide which feature types to process when the workspace runs. Close the dialogue.

Right-click on Feature Types to Read parameter and select Create User Parameter. Use the defaults for the definition, and click OK. Confirm the new parameter appears under Published Parameters.

7. Run Workspace Again

Ensure that Prompt for User Parameters is still enabled and rerun the workspace using Run > Rerun Entire Workspace.

Notice that this time, there are two parameters to set. Set it to whatever parameters you wish, and then click Run. Confirm that the workspace runs correctly.

8. Publish to FME Flow

Now that we have the parameters set up and confirmed that the workspace runs correctly, we can publish it to FME Flow.

Before publishing, save the workspace. Once saved, go to File > Publish to FME Flow on the top menu bar.

Connect to the same connection as the previous tutorial. Select the Flow Tutorial repository that we also created in the previous tutorial. We will keep the workspace name CommunityMapping.fmw and overwrite the previously uploaded workspace.

If you did not complete the previous tutorial, see the instructions below to learn how to create an FME Flow connection and upload a workspace:

• To publish a workspace from FME Workbench to FME Flow, follow the generic instructions below using the specific details listed in the article. If the article does not have a specific repository name listed, we recommend creating a Training repository. If a specific service is not listed in the article, please use the default Job Submitter.

  1. Click Publish

  From the File menu, click Publish to FME Flow, or from the toolbar ribbon, click Publish.

  Toolbar Ribbon:
  
  
  File Menu:
  

  2. Add a Web Connection

  In the Publish to FME Flow wizard, a FME Flow connection needs to be selected or created. To create an FME Flow connection, click the drop-down under FME Flow Connection and select Connect to FME Flow. For more information on creating or managing web connections, see the documentation.

  
   

  The following instructions are for a basic connection, but if your FME Flow is set up for Microsoft Azure, SAML, or Windows Credentials, please see the Using Web Connections to Connect to FME Flow documentation.

  An FME Flow Connection dialog will appear; here, we will set up the connection. The connection information and credentials can be obtained from your FME Flow Administrator. If you have not yet logged into FME Flow after installing, please do so prior to continuing, as you will be prompted to change the password from the default:
  Username: admin
  Password: admin

  • • Web Service: FME Flow

    • Server URL: http://localhost

    • Connection Name: Training FME Flow

    • Authentication: Basic

    • Username: your username

    • Password: your password

    Click OK. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.

  • • Web Service: FME Server

    • Connection Name: Training FME Server

    • Server URL: http://localhost

    • Username: your username

    • Password: your password

    Click Authenticate. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.

    Click Next to move to the Create Repository page of the dialog

  3. Create Repository

  Repositories are used to store groups of related items that have been published to FME Flow (similar to directories). Items such as workspaces, source data, custom formats, custom transformers, and templates can be published to a repository.

  If you need to create a new repository, click on the New button next to Repository Name. Enter the repository name and an optional description. Click OK to close the Create New Repository dialog.
  

  To select an existing repository, click the drop-down next to Repository Name and select the desired repository.
  

  4. Upload Files

  Some workspaces require additional files to run correctly on FME Flow, so these need to be uploaded along with the workspace. To upload files, enable Upload data files at the bottom of the dialog.

  When files are uploaded through the publishing menu, they are stored within the repository. If you want to upload folder files (such as geodatabases) or to specify where the files stored, the Select Files button allows you to do that. For detailed instructions on how to use the Select Files button, see the documentation.

  5. Register Services

  The final step before publishing a workspace is to register services. Services control how the data will be output after the workspace is run.

  

  Services return results in different forms:

  • Data Download Service returns results as a downloadable zip file

  • Data Streaming Service returns results as a data stream

  • Job Submitter Service accepts and runs workspace job requests

  • KML Network Link returns a KML Network Link that can be used in Google Earth

  • Notification Service allows for event-driven messaging

  Typically, the Job Submitter is always enabled, but multiple services can be enabled. Once services have been enabled, click Publish.

  6. Review Translation Log

  To confirm whether or not the workspace was published successfully, check the Translation Log. The Translation log will show which repository you published to, which files were included, and a quick link to run the workspace.

9. Open the FME Flow Web User Interface

Open and log into the FME Flow Web User Interface. If this is your first time logging into FME Flow, please see the instructions in Tutorial: Getting Started with FME Flow to learn how to do this.

10. Run Workspace

On the side menu bar in FME Flow, click on Run Workspace. On the Run Workspace page, select Flow Tutorial for the Repository and the CommunityMapping workspace. Select Data Download for the Service.

Review the Published Parameters. The two parameters that were set up and just tested in FME Workbench are shown. Just as before, select an output format, and select one or more feature types.

Once you have selected the parameters, click Run.

10. View the Results

The Data Download service presents a URL link to the translation results in a zip file. A quick inspection of the contents of the zip file confirms the results are what we expected.

Additional Resources

Working with User Parameters

Publishing to FME Flow

Running and Configuring a Workspace

FME Flow Troubleshooting Guide

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.