Creating Parameters and Uploading Data for Self-Serve Access

Introduction

Welcome to the second tutorial of the Getting Started with FME Flow (formerly FME Server) series. In this tutorial, we will modify the workspace from the previous tutorial to add published parameters, which when published to FME Flow, allows for self-serve data access. 

Step-by-Step Instructions

Part 1 of this tutorial assumes you have FME Form (formerly FME Desktop) installed. If you do not have access to FME Form, please download the AddressesToNotify-SelfServe.fmwt from the Files section on this article, then follow the instructions in the Uploading to FME Flow section of the documentation. After the workspace has been uploaded to FME Form, then proceed to Part 2 of this tutorial.  
 

Part 1: Create User Parameters in FME Workbench 

1. Open FME Workbench
Download the AddressesToNotify.fmwt workspace that is available in the Files section of this article. Open FME Workbench (FME Form) and open the downloaded workspace.


This workspace is compiling a list of customers to notify that were affected by a water pump failure. The workspace is reading a CSV containing information about local addresses in the city of Surrey, BC, as well as a shapefile containing a polygon area of interest where water service maintenance will occur. 


If you have completed Publish a Workspace to FME Flow and Run It, this is the same workspace. 
 
2. Open the Parameter Manager
 In the Navigator window, right-click on User Parameters and select Manage User Parameters. 


The Parameter Manager contains all of the user parameters in a workspace. This workspace only has the default parameters from when a reader and writer are added. Since we want to provide the user an option to upload a new water_service_area file, we should clean these parameters up.  For more information on the Parameter Manager, see the Using the Parameter Manager article. 


3. Remove the Parameter
When a workspace is run on FME Flow, any of the published parameters can be edited by the user. We want the CSV file of addresses to stay the same, so let’s remove that parameter. Click on the Source CSV parameter on the left-hand side (1) to select it and then click on the trash icon at the top to remove the parameter (2).​​​​​​​

 
4. Modify the Source Parameter
Now we can modify the prompt of the Source Esri ShapefileSelect so the user understands what they should upload. Click on the Source Esri Shapefile parameter to select it, then on the right-hand side, edit the Prompt to be:

Service Areas Input Shapefile Dataset

5. Create New Parameter
While we’re in the Parameter Manager, let’s create a parameter for the service date. The user will set this when the workspace is run and will be attached to the AddressesToNotify.xlsx spreadsheet. 
Click on the green plus sign (+) in the top left and select Datetime.


Then on the right-hand side, enter the following parameter properties:

• Parameter Identifier: SERVICE_DATE
• Prompt: Date of Water Service
• Published: Enabled
• Required: Enabled
• Disable Attribute Assignment: Enabled
• Time Format: Date Only
• Conditional Visibility (new for FME 2023.0): Disabled
• Default Value: <blank>

Click OK to close with the Parameter Manager. 
 
6. Modify the Shapefile Reader
Since we won’t know if the filename of the updated file will be different, let’s change our shapefile reader to accept all filenames with the same schema. 

Double-click on the water_service_area reader feature type to open the parameters. 


In the parameters, enable Merge Feature Type and leave the other parameters set at the default, then click OK. 


The reader feature type will now display <All> which means that any shapefile can be accepted into the reader, as long as the schema is the same.


 7. Link Published Parameter to Service Date Attribute
Now that we have a SERVICE_DATE parameter, we need to create an attribute to utilize it in the workspace. Double-click on the AttributeManager transformer to open the parameters. In the parameters, click in the <Add new attribute> box and then type in SERVICE_DATE as a new attribute. Next, click in the box for Value next to SERVICE_DATE to show the drop-down button. Click on the drop-down button and expand User Parameter, then select SERVICE_DATE.


For FME 2023.0 and newer, click in the box for Type and select date as the data type. Click OK to confirm the AttributeManager parameters. 


8. Run the Workspace
If you run the workspace immediately, it will use the original input shapefile. However, we want to test with the updated shapefile. Click on the drop-down arrow next to the Run button and ensure that Prompt for Parameters is enabled.  


Now, click the Run button to run the workspace. A User Parameter Value dialog will appear. Click the ellipsis (or file icon) next to the Service Areas Input Shapefile Dataset and select the water_service_area_updated.zip, which is available to download from the Files section of this article. 

This zip file contains all of the sidecar shapefile files and has a new service area that customers will have to be notified about. In the File Browser, change the Shapefiles (*.shp) drop-down to All File(s) to see the zip file.  
 
Next, set the Date of Water Service to a date within the next week by clicking on the drop-down arrow to display the calendar. For the Destination Microsoft Excel File parameter, confirm the output location. If you want a different file destination, click the ellipsis [...] button (or file icon) and select a new folder. 

Finally, disable the Save As User Parameter Default Values before clicking Run. We don’t want these parameters to be saved as the default because this is just a test run to ensure the workspace is working correctly. 


After running the workspace, review the Translation Log for the Translation was SUCCESSFUL message. If you did not receive this message, review the translation log for any red error messages and resolve them. 
 

Part 2: Publish to FME Flow and Run Workspace ​​​​​​​

If you do not have a connection to FME Flow created, please complete Part 1 of Publish a Workspace to FME Flow and Run It, prior to continuing. 

1. Publish to FME Flow
Now that we have the parameters set up and confirmed that the workspace runs correctly, we can publish it to FME Flow.

Before publishing, save the workspace. Once saved, go to File > Publish to FME Flow on the top menu bar.

Select an FME Flow connection, then select the Flow Training repository created in the previous tutorial or create a new repository. To differentiate between the workspace uploaded from the previous tutorial, add -SelfServe to the end of the workspace file name:

AddressesToNotify-SelfServe.fmw 

Enable Upload Files, then click Next, and then confirm that the Data Download service is selected, then click Publish. 

 
2. Upload the Updated Dataset
Open and log into the FME Flow Web Interface.  When we run the workspace in FME Flow, we have the option to upload the updated shapefile directly through the published parameter. The only issue with this method is that it gets uploaded to the Temporary files folder, which gets purged every month (by default). We want to reference this in an automation we will set up in the next tutorial, so let’s upload it to the permanent Data folder. 
On the side menu bar, click on Resources. (FME 2022.2 or older: expand Resources Files & Connections, then click on Resources). 


Resources are where files are stored, such as log files, temp files, connections, and backup files. The Data folder is where users can store data outside of repositories for use across FME Flow. Open the Data folder (1), then click New in the top corner (2). Create a new folder called Flow Training, then click OK.


Open the new Flow Training folder, and click on Upload > File(s). Then select water_service_area_updated.zip, and click OK. 


Now, this file can be accessed anywhere on FME Flow. After uploading data to FME Flow, the Resources Uploaded notification can be seen in the top-right corner, which will indicate how many resources were uploaded. Clicking on it, you can see all of the resources uploaded during the session. 



3. Run Workspace
On the side menu bar, click on Run Workspace. On the Run Workspace page, select Flow Tutorial for the Repository and the AddressesToNotify-SelfServe.fmw workspace. Select Data Download for the Service.


In the Published Parameters section, there are the two parameters that were set up in FME Workbench. For the Service Areas Input Shapefile Dataset parameter, click on Browse Resources.


In the Resource browser, click Data, then Flow Training. Next to water_service_area_update.zip, click on the plus sign (+) to select the dataset (1). It will now appear on the side under Selected Items. Click the x next to $(FME_MR_DIR)water_service_area.shp to remove it (2), as we are only interested in our updated dataset. Click OK to close the Resource browser. 



Note: If you are working with shapefiles not contained in a zip file or any other file types that have sidecar files, be sure to select ALL of the files when adding them to a parameter or workspace. If you are missing a file, the workspace won’t run correctly. 

Next, click the calendar icon next to the Date of Water Service parameter, and select a date within the next week. 

Once all of the parameters have been set, click Run at the bottom of the page to run the workspace. 


4. View the Results
Click on the URL to download the zip file. Open up the zip file to confirm there is a Microsoft Excel file. 


Optional: Open the file in Microsoft Excel or in FME Data Inspector to review the output. 



 

Continue Your FME Journey  

• Next Article: Schedule a Workspace to Run with FME Flow Automations

Interested in exploring another practical example similar to this tutorial?

• Using the Generic Format for Self-Serve Data Access
• Getting Started with FME Flow Apps
• Using the Geometry Parameter in FME Flow

Want to continue learning FME Flow basics?

• The FME Academy is a guided, on-demand way to learn the entire FME Platform, and it's FREE! 

Interested in data integration and workspace authoring?

• Tutorial: Getting Started with FME Form (formerly FME Desktop)

Have a specific question?

• Reach out to the FME Community. 

​​​​​​​

Data Attribution 

The data used here originates from data made available by the City of Surrey , British Columbia. It contains information licensed under the Open Government License - Surrey.