How to Control FME Server Engine Memory Usage

FME Server Engines are periodically restarted by the FME Process Monitor to control any growing memory concerns. There are two parameters that control the frequency of FME Server Engine restarts, and there is a third parameter for controlling how aggressive FME Server Engines will consume memory resources.

Controlling the Automatic Restart of FME Engines

Why? Adjusting the settings noted below will control how often the FME Server Engines are restarted – thus clearing the memory they may build up.

# MAX_TRANSACTION_RESULT_SUCCESSES - maximum number of successful result
# transactions to accept from the service before shutting down or restarting the service.
#
# MAX_TRANSACTION_RESULT_FAILURES - maximum number of failed result transactions
# to accept from the service before shutting down or restarting the service.

MAX_TRANSACTION_RESULT_SUCCESSES=100
MAX_TRANSACTION_RESULT_FAILURES=10

These settings are found in /Server/ fmeServerConfig.txt – note that FME Server must be restarted for any changes to take effect.

Controlling How FME Server Engines Consume Memory (2017 and Older)

NOTE: The value of this parameter should NOT be adjusted unless you have exceptional circumstances or have been directed by Safe Software. Changing this value can cause unexpected behaviour and reduce performance.

FME_ENGINE_MEMORY_REDLINE <factor>

The Resource Manager automatically determines the optimal total memory the FME Engine process should use. It also dynamically allocates this total memory optimally to the algorithms within FME requesting it.

The FME_ENGINE_MEMORY_REDLINE directive is a hint to the FME Engine on how aggressive it should be in consuming memory. It takes a value between 0 and 1 (0.5 is the default value). For more aggressive memory usage, a value above 0.5 should be used. For reduced memory usage, a value below 0.5 should be used. The risk in being too aggressive is the process running out of memory or the machine thrashing. The risk in being too conservative is that the process may take longer to complete.

WARNING: The effectiveness of the value of FME_ENGINE_MEMORY_REDLINE will vary between releases. Additionally, adjusting this value for FME 2018 or newer will have less influence on resources compared to older releases – the memory management improvements to FME Engine have removed the need to tweak this parameter value.

:: Value = 0.5 (Default)

The Resource Manager will stash "at a reasonable point", with the goal of running as fast as possible without risking stability.

:: Value = 0.0

"Optimizing Memory Usage" should occur immediately. This practice will incur longer processing time as it is more costly to write data to disk than using memory resources.

:: Value = 1.0

No memory limits on individual processes. Stashing will only occur if the entire system is dangerously low on memory.

Please note that when lowering the parameter value below 0.5 it is important that there exists a sufficient amount of temporary (physical) space in your /tmp or FME_TEMP directories.

Note: In FME 2017.1.0 and earlier, Linux was biased towards never stashing due to performance issues. Linux behaviour was harmonized with Windows in FME 2017.1.1

Where to find and set FME_ENGINE_MEMORY_REDLINE?

NOTE: FME Server must be restarted for any changes to take effect.

FME Server 2014 SP2 and Newer:
FME_ENGINE_MEMORY_REDLINE parameter is referenced in the /Server/ fmeEngineConfig.txt file.

FME Server 2014 SP1 and Older:
FME_ENGINE_MEMORY_REDLINE parameter is referenced in the <FMEServer>/Server/fmeEngineConfig_Engine#.txt files.
You will need to change this parameter for each fmeEngineConfig_Engine#.txt file.

FME Desktop:
To set the FME_ENGINE_MEMORY_REDLINE in FME Desktop see the article Performance Tuning FME .