Search

FME Flow Troubleshooting: Python

Sanae Mendoza
Sanae Mendoza
  • Updated

FME Version

Are you encountering issues while using Python with FME Flow? 

There are several places where Python can be used in FME: This article discusses common troubleshooting tips, questions and resources for troubleshooting Python in FME Flow. See the complete list of topics in the FME Flow Troubleshooting Guide. 
 

Content Overview

Initial Troubleshooting
Common Issues


Initial Troubleshooting

Even when you have an error message in hand, it is often useful to review the recommended initial troubleshooting steps. These questions can help identify the origin and context for the issue. 
 
Are there errors in the job log or system logs? 
Review both the FME Flow job log and fmeprocessmonitorengine.log (in the Engine folder) for Python-related errors. 
 
Where and when does the Python workflow fail?
If the workflow fails in all FME applications, begin troubleshooting in FME Workbench. This scenario often indicates an issue at a workspace or environment level. 

If the workspace succeeds in FME Workbench but fails in FME Flow, resume troubleshooting in FME Flow. This scenario often indicates an issue at an FME Flow configuration or environment level.  

In either scenario, testing with an FME Workbench installed on the same machine as FME Flow is often useful. Make sure to use the same build and version as your FME Flow installation. This eliminates any variability between the application environments. 

Only when a workflow can run successfully in FME Workbench should you advance to troubleshooting in FME Flow. 
 
Does the Python script execute successfully outside of the FME application?
If using a standalone script, consider executing it outside of FME (e.g. command line, PowerShell, or an IDE). If it still fails to execute, the problem is likely not with FME but with the script itself. 
 
Are you using the correct Python version for your workflow and environment? 
Python is set at the workspace level by the Python Compatibility setting.

The FME Engine Python interpreter is reported in the log file. Search the logs (both Workbench and Flow) for “Python”.

Searching a Log file for Python Errors
Using Search to find Python errors.

If your log file reports Esri Python, see our FME and Esri Troubleshooting Guide’s section on ArcPy for common problems and solutions.
 
Do you have environment variables set? 
On the machine where FME Flow is installed, review the system environment variables for any Python-related variables (e.g. “PYTHONHOME”) that may force a particular interpreter to be used. 
 
Is the FME Python script using the correct paths?
Whenever a Python script imports a package, it finds it in a list of known directory paths. The path information may be default values, environment variables, or the directory from which it’s working from. Consider adding “print” statements to your Python script to confirm that the correct paths are being used. 

For example, to list the directories/paths that are searched for modules, add the following 'print' statement to the script:  
import sys
print(sys.path)
Review “How Can I Modularize My Python Code” for strategies on specifying module locations in FME. 
 

Common Issues 

“My Python script fails on FME Flow, but error messages are missing or insufficient” 
In FME Form (Workbench), Python errors are written to the translation log. However, in FME Flow, Python errors are not written in the job log. Instead, review the fmeprocessmonitorengine.log (in the Engine folder) for a more detailed error message. 

See Logging with Python scripts for how to add additional logging tools to your workflow. 
 
‘The log file for Job ID <n> does not exist’ 
On the FME Flow job log page, expand the RESULT DATA section (located at the top of the page, before the translation log begins).

Result Data section
The 'RESULT DATA' section expanded on an FME Flow job log page

The Status Message may contain an error such as: 
INCLUDE -- failed to evaluate Python script
If you see this error, the Python script is likely the cause of job failure. However, specific Python errors are not written in the job log in FME Flow. Instead, review the fmeprocessmonitorengine.log (in the Engine folder) for a more detailed error message. 

If the RESULT DATA contains an error unrelated to Python, please see ‘The log file for Job ID <n> does not exist’ for more possible causes.
 
Python Exception <ModuleNotFoundError>: No module named 'x'
If the message is triggered by a reader, writer, or transformer and references an FME-related module, a package may be missing on FME Flow. See ‘FME Flow and Packages: Python Exception <ModuleNotFoundError>’.

If the message is triggered by a Python script and references a third-party module, an additional module may be missing on FME Flow. See Using Additional Python Modules in the FME Flow documentation for importing dependencies. Ensure the module is compatible with your FME Flow operating system.
 
Python Exception <ModuleNotFoundError>: No module named 'arcpy'
If your log file reports Esri Python, see our FME and Esri Troubleshooting Guide’s section on ArcPy for common problems and solutions.
 
Workspace requiring Python x.x is requested to run on an FME Engine configured to use a Python x.x+ interpreter. FME cannot continue given the Python version mismatch"’
When an FME Engine does not have the correct Python interpreter loaded for the translation: the job initiation fails, the Engine restarts, and FME reattempts the translation. 

The preferred Python Interpreter for the translation can be configured at the workspace level with the Python Compatibility parameter. Ensure that FME Flow has access to the chosen interpreter. 

Some known issues can prevent FME Flow from automatically resolving this issue:   
My Python scripts fail on FME Flow Hosted
Python scripts may fail once published to FME Flow Hosted due to differences in operating system. FME Flow Hosted runs on Linux, therefore it may use a different Python interpreter than your local computer. 

See Using Python with FME Flow Hosted for more information. 
 
A Python interpreter could not be loaded.
Ensure that FME Flow has access to the correct Python interpreter. 

As of FME 2022.0, Python 2.7 has been deprecated and is no longer available within FME. Please see the Python 2.7 Deprecation article. 
 
There is a known Windows bug that may cause instability when using Python in the current locale. Please update to Windows 10 version 2004 and greater, or to Windows Server 2022
See Known Issue: Instability Running Python in FME 2022.0 on Windows 10 and Windows Server 2019 Version 1909 or Older. 
 
Python scripts fail on my FME Flow machine in a different locale 
In 2022+, FME will default to UTF-8, which can cause problems while running scripts that force a different locale. See Known Issue: Executing Python scripts that change locale will result in unexpected errors in FME 2022.0 for more information. 
 

Are you still experiencing issues? 

Please consider posting to the FME Community Q&A if you are still experiencing issues that are not addressed in this article. There are also different support channels available.

When creating an FME Flow Support case for Python-related issues, please include the following: 
  • Relevant FME logs (see Debugging Toolbox for finding logs on FME Flow):
    • A copy of the failed job log (and successful, if it exists)
    • The fmeprocessmonitorengine.log (engine folder)
  • The workspace (*.fmw file)


Have ideas on how to improve this? 

You can add ideas or product suggestions to our Ideas Exchange.
 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.