Creating Looping workflows within FME Flow Automations

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2023.0

Introduction

Looping in FME Flow (formerly FME Server) Automations enhances the functionality of automations by allowing FME Flow users to re-run actions that have failed (due to external factors) and create advanced ‘Process Manager’ integrations. 
Warning: Enabling loops in automations is an advanced feature. Care must be taken to avoid the creation of an infinite loop that could overload your server's engines. Creating a loop upstream of a merge node is also not recommended and may lead to unexpected behavior.


​​​​​​Examples of Looping

Creating a loop in an automation workflow will be useful when users want to re-try actions if they fail. Two examples of looping scenarios are:

 

Resubmitting Failed Jobs

Workspaces may fail due to external data sources being unable to cope with concurrent connections or experiencing network interruptions. If the workspace fails, it can be connected back to the input port to try again. (Note: if the failure is due to another reason, an endless loop may occur). Logic must be built into the workspace to handle retrying failures or use the automations Writer to control when a workspace might loop.

image.png

 

Re-trying HTTP Actions

The HTTP Request action gives the following output attributes (formerly output keys): Error Type, Error Message, Status Code, Response Headers, and Response Body. You could use the Filter action after a failed HTTP Request to determine whether to add in a loop. For example, if the returned Status Code was a 503 Service Unavailable (or any other applicable status code) loop it back, with the intention that when the service becomes available the HTTP Request will succeed. You may want to consider making the HTTP Request within a workspace where logic can be added to avoid looping for an excessively long time.

image.png

 

Step-by-step Instructions

 Part 1: Create a Simple Looping Automation

Scenario
Using the FAA Airport Status Web Service API, send an email containing delay information on a schedule. Create an automation loop to re-try the API call if it fails to retrieve any delay information for the selected airport. More information on the API call can be found in the FAA Airport Status Service SwaggerHub documentation

 

1. Enable Automations Looping

Import the Simple Looping.fsproject (built with FME Flow 2023.0) to FME Flow, then open the Airport Delay Tracker automation. 
To use Looping, it must be enabled from the Automations Menu dropdown. After selecting ‘Enable Looping’, read the warning message and select ‘Enable Loops’. Looping will allow downstream output ports to be connected to upstream input ports. 
image.png

 

2. Add Workspace to the Automation
Connect the airportstatus.fmw workspace to the success port of the FME Flow Schedule. Choose the desired airport to monitor in the workspace parameters, such as San Francisco International Airport (SFO). Connect the Delay output port to the Email action and the NoDelays port to the Log action. Configure the Email action
image.png

 

3. Add Loop
The airportstatus.fmw workspace is configured to terminate and fail if no delay information is pulled from the API call. 
image.png
To run the workspace and make the API call again, click and drag the airportstatus.fmw failure port to its input port. 
image.png
The dashed gray line is used to show a looping scenario, which occurs on a workspace failure in this simple example. 

4. Start Automation
Save the automation and then select Start Automation.

 

 Part 2: Created an Advanced Looping Scenario

Adding looping capability to FME Flow Automations allows users to create more complex looping workflows that resemble a ‘Process Manager’ Enterprise Integration Pattern. A Process Manager is an Integration Pattern (for more information on Integration Patterns see this article) that relies on a central processing unit to dynamically route tasks or messages to different ‘processing units’ (workspaces), without having a sequence or processing order predefined. 

A Process Manager is an Integration Pattern (for more information on Integration Patterns see this article) which relies on a central processing unit to dynamically route tasks or messages to different ‘processing units’ (workspaces), without having a sequence or processing order predefined. This central processing unit needs to have some memory mechanism that stores information for which step in the sequence was executed last.

Scenario
Create a looping automation that mimics spending a day at FME Amusements going on rides, playing games, and eating snacks until the money runs out. Uploading the FME Amusements Tracker.xlsx file into a watched directory triggers the automation which flows through the Ticket Booth, Amusement Activities, and Check Wallet workspaces. The Ticket Booth workspace purchases tickets, 10 tickets maximum at a time, for a cost of $0.50 per ticket. The Amusement Activities deducts 3 tickets and allows the participant to go on an amusement ride, play a carnival game, or get some food, chosen at random, and records the activity on the spreadsheet. The Check Wallet workspace tests whether the participant has enough tickets to do more activities, needs to go to the ticket booth to get more tickets, or doesn’t have enough money to continue and their day is over at FME Amusements.

1. Prepare Automation for Looping
Import the Advanced Looping.fsproject (built with FME Flow 2023.0), then open the FME Amusements automation. Enable Looping for the automation (see Part 1, Step 1). 

2. Add Loops
Currently, the automation is set up so that a trip to FME Amusements consists of visiting the ticket booth, doing one activity that costs 3 tickets, and checking for money and tickets before ending. Adding loops to the automation will allow more tickets to be purchased and more activities to be completed. 
The Check Wallet workspace is set to direct the automation flow based on how much money and tickets remain for the participant. The Continue Activities port is used if there are enough tickets to complete another activity and the Need More Tickets port is used if there isn’t enough tickets to do an activity but there’s enough money to buy tickets to do another activity. The Days Over port is used when there’s not enough money or tickets to do any more activities. 
Connect the Continue Activities output port to the input port of the Amusement Activities workspace action to create a loop.
image.png

Next, connect the Need More Tickets output port to the Ticket Booth input port to create the next loop and complete the process. 
image.png
Save and start the automation.

3. Run the Automation
Create or modify the FME Amusements Tracker.xlsx file in the directory being watched by the trigger to ensure the FME Amusements participant has some money to spend at FME Amusements. The default directory being watched is $(FME_SHAREDRESOURCE_DATA)/FME Amusements Data.
image.png
This will trigger the automation to run, repeating through the workspaces until there is not enough tickets or money. The automation log file will record each workspace running which will occur multiple times. 
After completion, the original file will be overwritten to show the current money and ticket amounts and the random activities completed at FME Amusements. 
image.png

If looping functionality is no longer required for your automation, select Disable Looping from the automation Menu to prevent any unwanted continuous loops from accidentally being created. 

 

Data Attribution

The data used in this article comes from the Federal Aviation Administration’s Airport Status Web Service (ASWS) API, which is licensed Creative Commons 0 (CC0).

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.