FME Flow: Automations Log Changes

Introduction

FME Flow (previously FME Server) Automations logs have been updated in FME 2023.2 Instead of having one log for the entire automation, each component has a separate log file; each trigger, action, and external action now has a dedicated log file. This was done so each component could be analyzed individually if any errors occurred.

What’s New

There are now two modes in which an automation can be run: Production and Debug mode.

• Production mode will create separate log files for each trigger and action but contains information from multiple automations
• Debug mode will produce a single log file, with all the triggers and actions logged in order of when they happened. This is the same log file from versions 2023.1 and earlier.

We do not recommend running automations continuously in Debug mode as these logs will be lost once the automation has been stopped. Production mode should be used when automations are running after they have been developed and tested.

Production Mode

When running an automation in Production mode, a green notification bar will appear at the top of the canvas.

When you restart an automation in Production mode after running in Debug, you’ll be notified that the Debug log file will be removed once the automation starts.

To find the logs for Production mode, you can access them through the drop-down Menu in the Automations canvas or by navigating to Resources > Logs > automations > current. From there, you can find each .log file for all the actions and triggers

Debug Mode

To turn on Debug mode, click the Debug button to the right of the Start button at the top of the canvas.

When you start Debug mode, a warning message will appear with the following information on it:

• We do not recommend this mode for production environments
• Logs will not be saved in the Resources, but previous production logs are accessible
• Where to find the debug log
• The debug log will be cleared if the automation is restarted

Once the automation is running in Debug mode, you will see an orange notification at the top of the Automations canvas letting you know you are in Debug mode and providing a link to the log file. This notification will remain the entire time Debug mode is on.

In Debug mode, the log files can be accessed through the Menu drop-down on the Automations Canvas.

How to Combine Logs for One Automation

Reading multiple files to determine where an error occurred can be tricky and time-consuming. Here is a workbench configured to read each log file and write it into one file for a single automation. For FME Flow versions 2023.2 to 2024.1. Please follow the instructions below. For FME Flow 2024.2, the instructions are in the next section

FME Flow 2023.2 to 2024.1

1. Download the automation log files
To download all the automation log files in FME Flow, navigate to the automations log folder under Resources > Logs > automations > current. Click the check box on the top left to select all the automations, then click “Actions” on the top right and select “Download”. This will create a .zip folder containing all the automations logs.

2. Edit the workspace for the specific automation and time frame
Open CombineAutomationLogs_2024.fmw in FME Workbench. First, we need to edit the reader named “current” and the writer called “output”

a. In the Navigator window, click on the arrow drop-down CSV2 reader named “current”. Right-click on the Source CSV and select “Edit Parameter Value"

b. Navigate to the location of the .zip folder containing all the automation log files. Click ok.

c. Click on the drop-down arrow for the CSV2 writer named “output”. Right-click on Destination CSV and select “Edit parameter value”

d. select an output location for the combined log file
e. Save the workbench

3. Adjust user parameters
Three user parameters can be set at run time or through the Navigator window. These are StartDateTime, EndDateTime, and AutomationID. If you want to set them now, right-click on the parameter and select Edit Value

• The most important User Parameter to set is the AutomationID. This is an alphanumeric string found in the automation URL on FME Flow. This parameter will need to be changed with each run if you want to combine the logs of other Automations.
  
• StartDateTime and EndDateTime can be customized if you want to see a specific timeframe; otherwise, you can put the current DateTime in for EndDateTime

4. Run the workspace and create the combined log file
Select Run > Rerun Entire Workspace from the menu bar, which opens the Translation Parameter Values dialog box.

a. To select the start or end dates, you can click on the down arrow in the box to bring up a calendar

b. Click Run, and a log file will be created in the location you specified for the output parameter in step 2C

FME Flow 2024.2

FME Flow 2024.2 now allows users to download the combined automation log. To access this, open any automation running in production mode and select “Download log” from the menu drop-down.

A pop-up window will appear, and you must select a start date before the download button is clickable. Click download to receive the combined log file.