Files
Introduction
Welcome to the second tutorial in the Getting Started with FME Flow series. In this tutorial, we will modify the workspace from the previous tutorial to add published parameters that, when published to FME Flow, enable self-serve data access.
Step-by-Step Instructions
Part 1 of this tutorial assumes you have FME Form installed. If you do not have access to FME Form, please download the AddressesToNotify-SelfServe.fmwt from the Files section of this article, then follow the instructions in the Uploading to FME Flow section of the documentation. After the workspace has been uploaded to FME Form, proceed to Part 2 of this tutorial.
Part 1: Create User Parameters in FME Workbench
1. Open FME Workbench
Download the AddressesToNotify.fmwt workspace that is available in the Files section of this article. Open FME Workbench (FME Form) and open the downloaded workspace.
This workspace is compiling a list of customers to notify who were affected by a water pump failure. The workspace is reading a CSV containing information about local addresses in the city of Surrey, BC, and a shapefile containing a polygon area of interest for water service maintenance.
If you have completed Publish a Workspace to FME Flow and Run It, this is the same workspace.
2. Open the Parameter Manager
In the Navigator window, right-click on User Parameters and select Manage User Parameters.
The Parameter Manager contains all of the user parameters in a workspace. This workspace only has the default parameters from when a reader and writer are added. Since we want to give the user the option to upload a new water_service_area file, we should clean up these parameters. For more information on the Parameter Manager, see the Using the Parameter Manager article.
3. Remove the Parameter
When a workspace is run on FME Flow, the user can edit any published parameters. We want the CSV file of addresses to stay the same, so let’s remove that parameter. Click the Source CSV parameter on the left-hand side (1) to select it, then click the trash icon at the top to remove it (2).
4. Modify the Source Parameter
Now, we can modify the prompt for the Source Esri ShapefileSelect so the user understands what to upload. Click on the Source Esri Shapefile parameter to select it, then on the right-hand side, edit the Prompt to be:
Service Areas Input Shapefile Dataset5. Create New Parameter
While we’re in the Parameter Manager, let’s create a parameter for the service date. The user will set this when the workspace runs, and it will be attached to the AddressesToNotify.xlsx spreadsheet.
Click the green plus sign (+) in the top-left corner, then select Datetime.
Then, on the right-hand side, enter the following parameter properties:
- Parameter Identifier: SERVICE_DATE
- Prompt: Date of Water Service
- Published: Enabled
- Required: Enabled
- Disable Attribute Assignment: Enabled
- Time Format: Date Only
- Conditional Visibility: Disabled
- Default Value: <blank>
Click OK to close the Parameter Manager.
6. Modify the Shapefile Reader
Since we won’t know whether the updated file's filename will change, let’s change our shapefile reader to accept all filenames with the same schema.
Double-click on the water_service_area reader feature type to open the parameters.
In the parameters, enable Merge Feature Type and leave the other parameters set to their defaults, then click OK.
The reader feature type will now display <All>, meaning that any shapefile can be accepted by the reader as long as the schema matches.
7. Link Published Parameter to Service Date Attribute
Now that we have a SERVICE_DATE parameter, we need to create an attribute to utilize it in the workspace.
Double-click on the AttributeManager transformer to open the parameters. In the parameters, click in the <Add new attribute> box and then type in SERVICE_DATE as a new attribute.
Next, click the Value box next to SERVICE_DATE to display the drop-down. Click the drop-down button, expand User Parameter, then select SERVICE_DATE.
Click the box for Type, then select Date as the data type. Click OK to confirm the AttributeManager parameters.
8. Run the Workspace
If you run the workspace immediately, it will use the original input shapefile. However, we want to test with the updated shapefile. Click on the drop-down arrow next to the Run button and ensure that Prompt for Parameters is enabled.
Now, click the Run button to run the workspace. A User Parameter Value dialog will appear. Click the ellipsis (or file icon) next to the Service Areas Input Shapefile Dataset and select the water_service_area_updated.zip, which can be downloaded from the Files section of this article.
This zip file contains all the sidecar shapefiles and includes a new service area that customers need to be notified about. In the File Browser, change the Shapefiles (*.shp) drop-down to All File(s) to see the zip file.
Next, set the Date of Water Service to a date within the next week by clicking on the drop-down arrow to display the calendar. For the Destination Microsoft Excel File parameter, confirm the output location. If you want a different file destination, click the ellipsis [...] button (or file icon) and select a new folder.
Finally, disable the Save As User Parameter Default Values before clicking Run. We don’t want these parameters saved as the default, since this is just a test run to ensure the workspace is working correctly.
After running the workspace, review the Translation Log for the Translation was SUCCESSFUL message. If you did not receive this message, review the translation log for any red error messages and resolve them.
Part 2: Publish to FME Flow and Run Workspace
If you do not have a connection to FME Flow created, please complete Part 1 of Publish a Workspace to FME Flow and Run It prior to continuing.
1. Publish to FME Flow
Now that we have the parameters set up and confirmed that the workspace runs correctly, we can publish it to FME Flow.
Before publishing, save the workspace. Once saved, go to File > Publish to FME Flow on the top menu bar.
Select an FME Flow connection, then select the Flow Training repository created in the previous tutorial or create a new repository. To differentiate between the workspace uploaded from the previous tutorial, add -SelfServe to the end of the workspace file name:
AddressesToNotify-SelfServe.fmw Enable Upload data files.
Click Next.
Confirm that the Data Download service is selected, then click Publish.
2. Upload the Updated Dataset
Open and log in to the FME Flow Web Interface. When we run the workspace in FME Flow, we can upload the updated shapefile directly via the published parameter. The only issue with this method is that it gets uploaded to the Temporary files folder, which gets purged every month (by default). We want to reference this in an automation we will set up in the next tutorial, so let’s upload it to the permanent Data folder.
On the side menu bar, click on Resources.
Resources are where files are stored, such as log files, temp files, connections, and backup files. The Data folder is where users can store data outside of repositories for use across FME Flow. Open the Data folder (1), then click New in the top corner (2). Create a new folder called Flow Training, then click OK.
Open the new Flow Training folder, and click on Upload > File(s). Then select water_service_area_updated.zip, and click OK.
Now, this file can be accessed anywhere on FME Flow. After uploading data to FME Flow, the Resources Uploaded notification appears in the top-right corner, indicating how many resources were uploaded. Clicking it shows all the resources uploaded during the session.
3. Run Workspace
On the side menu bar, click on Run Workspace. On the Run Workspace page, select Flow Training for the Repository and the AddressesToNotify-SelfServe.fmw workspace. Select Data Download for the Service.
In the Published Parameters section, there are two parameters that were set up in FME Workbench. For the Service Areas Input Shapefile Dataset parameter, click on Browse Resources.
In the Resource browser, click Data, then Flow Training. Remove the existing dataset by clicking the X next to $(FME_MF_DIR)water_service_area.shp on the right (1). Then, change the file ending filter to All Files so you can select the .ZIP file (2). Then, next to water_service_area_update.zip, click on the plus sign (+) to select the dataset (3). Click OK to close the Resource browser (4).
Note: If you are working with shapefiles that are not contained in a zip file, or with any other file types that have sidecar files, be sure to select ALL of the files when adding them to a parameter or workspace. If you are missing a file, the workspace won’t run correctly.
Next, click the calendar icon next to the Date of Water Service parameter and select a date within the next week.
Once all of the parameters have been set, click Run at the bottom of the page to run the workspace.
4. View the Results
Click on the URL to download the zip file. Open up the zip file to confirm there is a Microsoft Excel file.
Optional: Open the file in Microsoft Excel or in FME Data Inspector to review the output.
Continue Your FME Journey
- Next Article: Schedule a Workspace to Run with FME Flow Automations
Interested in exploring another practical example similar to this tutorial?
- Using the Generic Format for Self-Serve Data Access
- Getting Started with FME Flow Apps
- Using the Geometry Parameter in FME Flow
Want to continue learning FME Flow basics?
- The FME Academy is a guided, on-demand way to learn the entire FME Platform, and it's FREE!
Interested in data integration and workspace authoring?
Have a specific question?
- Reach out to the FME Community.
Data Attribution
The data used here originates from data made available by the City of Surrey, British Columbia. It contains information licensed under the Open Government License - Surrey.