Files
As of FME 2024.0, support for Esri ArcGIS Online (AGOL), ArcGIS Enterprise Portal, and ArcGIS Server Feature Service formats has transitioned to a unified approach using the new Esri ArcGIS Feature Service (Format). This format replaces the legacy reader/writer formats for each of the three services.
Starting in FME 2026.1, the legacy formats will be hidden in the Quick Add menu. Instead, equivalent functionality is now provided by the downloadable Esri ArcGIS Connector package, available on FME Hub. The package also includes new web services and key transformers such as the ArcGISOnlineConnector, ArcGISAttachmentConnector, and ArcGISBranchVersionManager. The package can be installed directly through the Quick Add menu in FME Workbench. Once installed, the new Esri Feature Service format will appear in the Gallery as a unified method for interacting with all three types of Esri ArcGIS Feature Services.
For details on this transition, including guidance on updating existing workspaces, please refer to the article: Working with Esri ArcGIS Feature Services in FME.
Introduction
As demonstrated in the article Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks, Automations can process survey form data in near real-time. This tutorial will show how an Automation can also email a link to a formatted Survey123 report for a single response. This approach can be helpful when external parties, who may not have access to ArcGIS Online or Survey123, need to be notified of a survey submission and provided with the details.
Before You Begin
- Complete the Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks tutorial, which demonstrates creating an Automation with a webhook trigger
- ArcGIS Online account registered as, minimum, a Creator user type to have access to Survey123
- Available credits on your ArcGIS Online account, as Survey123 reports cost credits
- Microsoft Word docx single-record report template uploaded to Survey123. Please see Esri’s documentation for more information.
- Publicly accessible FME Flow in order to receive a webhook payload from Survey123
Step-By-Step Instructions
This tutorial uses an existing survey and Automation created in the Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks article. Please ensure you have completed that first.
Part 1: Modify the Survey123 Webhook
We need the Survey123 webhook payload to include the survey information necessary to run the report.
1. Open the Invasive Species Reporting Survey
Open ArcGIS Online and navigate to Survey123. Open the Invasive Species Reporting survey. If you do not see the Invasive Species Reporting survey, ensure you have completed Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks.
2. Edit Webhook
In the survey, click on Settings, then click Webhooks. Next, click the pencil to edit the Invasive Species webhook.
3. Add Survey Info
Under Event data, enable Survey info and click Save.
4. Log the New Webhook Payload
With the Survey123 webhook modified, let’s take a look at the new payload in FME Flow by logging the entire webhook message content.
In a new browser tab, open FME Flow and log in. Open the Invasive Species Automation that was completed as part of the article Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks. If the Automation is running, stop it.
Connect a log action to the Webhook trigger success port. For the Formatted Message, from the drop-down menu, choose Webhook > Message Content. Click Apply.
Start the Automation.
5. Submit a Test Invasive Species Survey in Survey123
Back in Survey123, submit a test Invasive Species survey. The test survey will trigger the webhook. View the Automation Log File in FME Flow.
In the action_fmelogaction.log, the webhook message now contains a surveyInfo key with the information we need to create a report from the response data.
Part 2: Configure the Workspace
1. Open the InvasiveSpeciesReport.fmw Workspace
Download the InvasiveSpeciesReport.fmw workspace from the Files section of this article and open it in FME Workbench. The workspace has one parameter: webhook_message. It takes the webhook_message JSON, extracts the attributes needed to create the report from the JSON, and uses a custom transformer, the Survey123ReportGetter, to run and get a link to the report. The link is passed back to the FME Flow Automation using an FME Flow Automations Writer at the end.
2. Configure the Survey123ReportGetter Transformer
Open the Survey123ReportGetter parameters and set them as follows:
- Select an ArcGIS Online Web Connection: You should have already created a connection to complete the article Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks. If not, please follow How to Create an ArcGIS Online Web Connection (OAuth 2.0) to create a connection.
- Service URL: Previously Exposed JSON attributes
- Survey123 Object ID: Previously Exposed JSON attributes
- Survey123 Form Item ID: Previously Exposed JSON attributes
- Survey123 Template ID: Copy the ID from the template item’s URL in ArcGIS Online
- Maximum Credits to Spend: single-record reports use 0.5 credits, although this is subject to change by ESRI at any time. Leave the value as 0.5
- Report Filename: enter a filename for the report. You can use the text editor to access FME Functions. In this example, we’ll append the current datetime to the end of the report filename using the @DateTimeNow() function
- Output format: pdf
- Persist Report File: for this example, we'll leave it unchecked. This will produce a temporary link valid only for 5 minutes. When the report is persisted in ArcGIS Online, the user will need to login to access the file. Another option is to modify this workflow to save the resulting report file to your own public file hosting service and email that URL.
- ArcGIS Online Output Folder ID: this can be left blank since the report file is not being persisted in ArcGIS Online.
Click OK.
3. Copy/Paste the Webhook Message into FME Workbench
Copy the entire webhook message JSON from the FME Flow Automation log file in Part 1, Step 4, from the opening curly brace to the closing curly brace, and paste it into the webhook_message value. We will use this to test the workspace.
4. Run the workspace
The workspace may take a minute to run, as Survey123 takes time to create the report file. Confirm the workspace finished successfully with no errors, and then in Visual Preview, inspect the Survey123ReportGetter output feature. You should see the report filename, filesize and url.
If you receive a 400 Unknown Error from the Survey123ReportGetter, try reauthenticating the web connection. Go to FME Options, Web Connections, right-click the connection, and choose reauthenticate. Make sure to authenticate with an account that has access to the survey and report template in Survey123.
5. Publish to FME Flow
Before publishing, let’s embed the Survey123ReportGetter, since it is a custom transformer. To learn more about working with custom transformers in FME Flow, please read Custom Transformers and FME Flow.
Right-click on the Survey123ReportGetter and choose Embed.
The transformer will turn green, indicating that it has been embedded.
In FME Workbench, click Publish to connect to your FME Flow. In the publish wizard, choose a repository to upload the workspace to (or create a new repository) and click Next.
You should have already published an ArcGIS Online Web Connection in the article Push Data from ArcGIS Survey123 to an Application in Real-time with Webhooks. If not, click the checkbox beside the connection to publish it to FME Flow. Click Next.
Finally, register the workspace with the Job Submitter service and click Publish.
Check the Translation Log to make sure everything was published to FME Flow successfully.
Part 3: Complete the Automation
With the workspace published to FME Flow, we can now modify the Automation to use it.
1. Add the InvasiveSpeciesReport.fmw Workspace to the Automation
Open the Invasive Species Automation. Delete the Log action created in Part 1 that is connected to the Webhook trigger success port. Add a new action to the canvas and connect it to the InvasiveSpecies workspace action.
Configure the new action to run a workspace. Choose the InvasiveSpeciesReport.fmw that was published in Part 2. For the webhook_message parameter, click the drop-drop and choose Webhook > Message Content. Click Apply.
2. Add an Email Action to the Automation
Since the InvasiveSpeciesReport.fmw workspace contains an FME Flow Automation Writer, there is a “Report” output port available below the action. The port outputs a feature with the report URL that we can pass to an external email action.
Add an external action to the canvas and connect it to the Report port. Choose Email (send) as the Action and configure the SMTP settings for your outgoing email. In this example, we’ve loaded a Gmail template and entered the SMTP Account and Password.
Enter your email address in Email To and the SMTP Account Email in Email From. For the Email Subject, enter “Invasive Species Reported”.
For the Email Body, open the Text Editor.
In the Text Editor, enter some body text for the receiver and then add the report_url. The report_url can be accessed by expanding Workspace > Report. Click OK.
Click Validate, confirm the email action configuration is valid, and then click Apply to save the changes.
3. Start and Test the Automation
Save the Automation and click Start.
With the Automation running, submit another test Invasive Species survey in Survey123. Wait a couple of minutes, and then check your email. You should see an email containing a link to a report.
Click the link in the email to view the report. Below is a sample.
If you do not receive an email, check the Automation Log File for any errors.