Logging in Python scripts

Introduction

Writing messages to the FME log file in a Python script can be tricky. For example, if you use a simple print() function to add a log message, the message will be printed in the Translation Log window in FME Workbench. However, the message may not be saved to the workspace log file depending on where the Python script is run and the FME product.

The procedure for writing to the workspace log file differs based on where you are running the script.

Methods of Logging

fmeobjects.FMELogFile()

Log messages can be added to the log using fmeobjects.FMELogFile() function. You will need to create an FME log file object before sending messages to the log file object.

import fmeobjects
logger = fmeobjects.FMELogFile() 
logger.logMessageString("Hello, I am logging now")

Log messages are written as an INFORM level message by default. It is possible to set the severity of log message using message severity constants.

import fmeobjects
logger = fmeobjects.FMELogFile() 
logger.logMessageString("This is a warning", fmeobjects.FME_WARN)
logger.logMessageString("This is an error", fmeobjects.FME_ERROR)

Messages logged using this method will appear in both the Translation Log window and in the workspace log file.

Python's print() function

Log messages can be added to the log file using Python's built-in print() function. All log messages added using the print() function will be an INFORM level message. 

print("Hello, I am logging now")

Log messages added to the log file using print() appear in the Translation Log window but may not appear in the workspace log file depending on where the Python script is run.

Python's open() function

Log messages can be added to the log file by opening the workspace log file and writing or appending to the log file using Python's built-in open() function.

import fme, os
fmeLogFile = f"{os.path.join(fme.macroValues['FME_MF_DIR'],fme.macroValues['WORKSPACE_NAME'])}.log"
with open(fmeLogFile,"a") as f:
  f.write("Hello, I am logging now")

The workspace log file path can be customized in each workspace using the Log File parameter found in Workspace Parameters Log and Troubleshoot in the Navigator window. If a custom log file name or path has been set, modify the value of the fmeLogFile variable in the Python snippet to your custom log file location.

Messages logged using the open() function appear in the workspace log but do not appear in the Translation Log window.

fmetools.logfile.get_configured_logger()

The fmetools.logfile Python module is part of the fmetools library maintained by Safe Software. In select contexts, it is recommended to use this module to write log messages instead of fmeobjects.FMELogFile().

Base classes in fmetools.plugins such as FMEEnhancedTransformer already include preconfigured loggers. To use the preconfigured loggers, call the get_configured_logger() method of the fmetools.logfile module. 

The fmetools.logfile module bridges an fmeobject.FMELogFile() object with the standard Python logging library. Methods for logging with the fmetools.logfile module are based on those provided by the standard Python logging library. 

The following is a table mapping the get_configured_logger methods to the corresponding FME log message severity level and an example code snippet demonstrating their use.

# From transformer.py

from fmeobjects import FMEFeature
from ._vendor.fmetools.plugins import FMEEnhancedTransformer
from ._vendor.fmetools.paramparsing import TransformerParameterParser
from ._vendor.fmetools.logfile import get_configured_logger

class TransformerImpl(FMEEnhancedTransformer):
    params: TransformerParameterParser
    version: int

    def setup(self, first_feature: FMEFeature):
        super().setup(first_feature)
        # Get transformer version from internal attribute on first feature,
        # and load its parameter definitions.
        # Note: TransformerParameterParser requires >=b24145 when running on FME Flow.
        self.version = int(first_feature.getAttribute("___XF_VERSION"))
        self.params = TransformerParameterParser(
            "example.my-package.DemoGreeter",
            version=self.version,
        )
        """ Get preconfigured logger """
        self.sdkLogger = get_configured_logger()

    def receive_feature(self, feature: FMEFeature):
        # Pass internal attributes on feature into parameter parser.
        # Then get the parsed value of the First Name parameter.
        # By default, these methods assume a prefix of '___XF_'.
        self.params.set_all(feature)
        first_name = self.params.get("FIRST_NAME")
        
        """ Log using preconfigured logger """
        self.sdkLogger.debug("This is debug")
        self.sdkLogger.info("This is informational")
        self.sdkLogger.warning("This is a warning")
        self.sdkLogger.error("This is an error")
        
        # Set the output attribute, and output the feature.
        feature.setAttribute("_greeting", "Greetings, {}!".format(first_name))
        self.pyoutput(feature, output_tag="Output")

fmetools is not installed with FME installations, so this method of logging will only work in limited contexts. FME 2023.0 and newer, along with Python 3.8 and newer, is required.

Messages logged using this module appear in the Translation Log window and workspace log.

Location of Python Logging

Scripted Value Parameter

Using the print() function method to add log messages will cause the log message to appear in the Translation Log window but not the workspace log itself in a scripted value parameter. 

It is not possible to add a message to the workspace log within a scripted value parameter. This is because the evaluation of a scripted value parameter occurs so early in the workspace translation that a workspace log may not exist yet.
It is recommended that logging in scripted value parameters be limited to debugging purposes when authoring workspaces.

Startup Python Script

Logging in a startup Python script can be performed with both the print() function and with the fmeobjects.FMELogFile() method.

Both methods will add log messages to both the Translation Log window and the workspace log.

PythonCaller/PythonCreator Transformer

In a PythonCaller or PythonCreator transformer, both the print() function and fmeobjects.FMELogFile() method will work. Both methods will add log messages to both the Translation Log window and the workspace log file.

Shutdown Python Script

FME 2025.0 and newer

Recent changes to FME have made logging in Python shutdown scripts easier. As of FME 2025.0 and newer, it is possible to use the fmeobject.FMELogFile() and print() methods to log messages in a shutdown Python script. 

Log messages will appear in both the Translation Log pane and the workspace log file.

The open() function for logging messages recommended for FME 2024.2 and older will not work in FME 2025.0 and newer.

FME 2024.2 and older

Shutdown Python scripts cannot log messages using the fmeobjects.FMELogFile() method in FME 2024.2 and older. At this point in the lifecycle of the workspace, the workspace has been disconnected from the FME process, so calling fmeobjects.FMELogfile() does not work, and the workspace will error. 

Instead, use Python's built-in open() functionality to manually write lines to the end of the log file on disk. Messages logged using this method does not appear in the Translation Log window in FME Workbench. 

If you want log messages to also appear in the Translation Log window, use the print() function in addition to the open() function.

FME Packages SDK custom transformer

Logging in a custom transformer created using the FME Packages SDK should be performed using the fmetools.logfile.get_configured_logger() method. 

Every FME Package includes its own copy of the fmetools library through vendorization, so users don't need to install it separately.

Messages will appear in both the workspace log and Translation Log window when using the preconfigured logger. Log messages will be prepended with the text "fmelog: ", as fmelog is the default preconfigured logger name. Debug-level messages are also additionally prepended with "DEBUG: ".

Logging on FME Flow

The process for logging with Python scripts is similar on FME Flow as FME Form, with a few exceptions. 

In locations where the print() method only outputs to the translation log window and not the translation log, log messages will be written to the process monitor engine log file rather than the workspace log. Example locations include scripted value parameters in all FME versions and shutdown Python scripts for FME 2024.2 and older.

The process monitor engine log file can be found at:

<fmeServerSystemShare>\Logs\Engine\Current\<fmeFlowHost>_fmeProcessMonitorEngine.log

When manually appending log messages to the job file using the open() method in FME Flow 2024.2 and older, use the following Python script to write to the default location for a job log:

import fme,os

logFileBase = os.path.join(fme.macroValues['FME_SHAREDRESOURCE_LOG'],"engine/current/jobs/")
jobLogDir = str((int(fme.macroValues['FME_JOB_ID'])//1000)*1000)
jobLogName = f"job_{fme.macroValues['FME_JOB_ID']}.log"
fmeLogPath = os.path.join(logFileBase,jobLogDir,jobLogName)

with open(fmeLogPath,"a") as logger:
    logger.write("Hello, I am logging now")

Summary

A workspace with examples of logging in a scripted value parameter, startup Python script, PythonCaller, and shutdown Python script for FME 2025.0 and newer can be found in loggingWithPythonScripts_20250.fmw, from the Files section.

If you are working with FME 2024.2 and older, please see loggingWithPythonScripts_20242.fmw instead, also available from the Files section.

The following is a table that summarizes the recommended methods of logging based on where a Python script is located within FME and the output location on FME Form and FME Flow.

For a complete overview of using Python with FME, please see the article Tutorial: Python and FME Basics

See the FME Packages SDK Guide for tutorials on working with the FME Packages SDK.