Logging with Python scripts

Introduction

Writing messages to the FME log file in a Python script can be tricky. For example, if you use a simple print() function to add a log message, the message will be printed in the Translation Log window in FME Workbench. However, the message may not be saved to the workspace log file depending on where the Python script is run and the FME product.

The procedure for writing to the log file differs based on where you are running the script.

Methods of Logging

fmeobjects.FMELogFile()

Log messages can be added to the log using fmeobjects.FMELogFile() function. You will need to create an FME log file object before sending messages to the log file object.

import fmeobjects
logger = fmeobjects.FMELogFile() 
logger.logMessageString("Hello, I am logging now")

Log messages are written as an INFORM level message by default. It is possible to set the severity of log message using message severity constants.

import fmeobjects
logger = fmeobjects.FMELogFile() 
logger.logMessageString("This is a warning", fmeobjects.FME_WARN)
logger.logMessageString("This is an error", fmeobjects.FME_ERROR)
logger.logMessageString("This is a statistic", fmeobjects.FME_STATISTIC)

Messages logged using this method will appear in both the Translation Log window and in the log file.

Python's print() function

Log messages can be added to the log file using Python's built-in print() function. All log messages added using print() will be an INFORM level message. 

print("Hello, I am logging now")

Log messages added to the log file using print() appears in the translation log window but may not appear in the log file depending on where the Python script is run.

Python's open() function

Log messages can be added to the log file by opening the log file and writing or appending to the log using Python's built-in open() function.

import fme, os
fmeLogFile = f"{os.path.join(fme.macroValues['FME_MF_DIR'],fme.macroValues['WORKSPACE_NAME'])}.log"
with open(fmeLogFile,"a") as f:
  f.write("Hello, I am logging now")

The log file path can be customized in each workspace using the Log File parameter found in Workspace Parameters Log and Troubleshoot in the Navigator window. If a custom log file name or path has been set, modify the value of the fmeLogFile variable in the Python snippet to your custom log file.

Messages logged using the open() function appears in the log file but does not appear in the translation log window.

Where

Python Scripted Parameter

The evaluation of a Python scripted parameter occurs so early in the workspace translation that a log may not exist yet.

Using the print() function method to add log messages will cause the log message to appear in the translation log window but not the log file itself.

Startup Python Script

PythonCaller Transformer

In a PythonCaller (or PythonCreator) transformer, messages are added to the log file with the FMELogFile object, which is part of the fmeobjects. You need to create a logger object:

logger = fmeobjects.FMELogFile() 

and then you will need to send messages to the logger:

logger.logMessageString("Hello I am Logging Now")

Shutdown Python Script

Logging in a shutdown script is a bit trickier. At this point in the lifecycle of the workspace, the workspace has been disconnected from the fme process, so the FMELogfile object no longer works. Instead, you will need to get the location of the log file on disk with FME_LogFileName parameter and use Python's built-in file functionality to add lines to the log file, for example:

logger = open(FME_LogFileName,'a')
logger.write("wow - this message will actually be written to log file")
logger.close()

Note: Lines logged in this way will NOT appear in the log window in FME Workbench, but will be appended to the end of the log file on disk. If you want something to appear in both places, you will need to write it to the log file, as well like this:

print("wow - this message will only go to log window")

If you are running Python scripts on FME Flow, any error messages produced by the interpreter will be written out to the Process Monitor Engine log file rather than the Workspace log. This file can be found in

<fmeserversystemshare>\Logs\Engine\Current\.</fmeserversystemshare> 

Examples of logging in a PythonCaller and in a shutdown script can be found in the attached workspace.



For a complete overview of using Python with FME, please see the article Tutorial: Python and FME Basics