A Guide to FME Flow on Linux

Introduction

The speed and price point of Linux installations make them an enticing option for an FME Flow Installation. This article will discuss considerations when planning an installation on a Linux operating system. While a good understanding of Linux is helpful, a less experienced Linux user can easily have FME Flow up and running. This is a hands-on guide designed to help you plan and install FME Flow so it can work well in your organization.

Format Integrations

Linux FME has a few unsupported formats and even fewer transformers, but the vast majority of both are supported on Linux. In most cases, if there is no third-party dependency, the format or transformer will be available on both Windows and Linux installations. The most common of these unsupported formats are ArcGIS formats, which include SDE Geodatabases and access to Arcpy Python libraries. Customers can handle ArcGIS integration on Linux Installations in several ways.

Distribute Engine Installation

FME Flow allows you to have a complete FME Flow installation on a Linux machine and then add another Windows machine to that installation for a distributed engine installation that can process the Geodatabase SDE and other ArcGIS-dependent formats. This way, you can have the Linux side handle the other workflows that require more memory and CPU and then route only the ArcGIS jobs to the distributed Engine.

Use ArcGIS Online / Portal

You can still have the automated capabilities of FME when you host your data on ArcGIS online. Beyond ArcGIS Online and ArcGIS Portal Readers, we have several custom transformers designed for ArcGIS Online Integration that allow you to get more out of your data. 

Use an ArcGIS format supported on Linux (File Geodatabase, Shapefile) 

Some formats are still supported on Linux installations the Shapefile reader has recently undergone a speed overhaul, however, the API File Geodatabase reader lacks some of the advanced features such as relationship table integration. 
 

Pre Installation Considerations

File-Based Data Storage

Before you install FME Flow in a Linux environment, it's important to decide where you are going to put your data. If you are storing your data in a file share, there are a couple of options.

• Use cloud storage like an S3 bucket. This will allow you to set the same location for your data on  Windows desktops and Linux. 
• Mount a file share to the Linux Machine and set it up as an FME FlowNetwork Resource. Please see our Linux Mounting article for examples on mounting and adding network resources, 

Python

Custom Python interpreters are not supported on a Linux Server. However, adding custom modules into FME’s Python interpreter is simple and flexible.
 

Desktop Compatibility

Workspaces can be authored on a Windows, Mac, or Linux machine and run the same on Linux. There are very few formats and transformers that are not supported on Linux. The normal considerations do apply, though, when publishing, and it's important to make sure the paths to data are accessible from the FME Flow machine. You can, of course, upload data at run time, but uploading data can be time-consuming and lead to a congested server, so it's not recommended for every process.

Installation and Upgrade

FME Flow Linux installations have a few more considerations than typical Windows installations, however, installation can be simple and fast on default environments.

2022 Changes

Starting in 2022, FME has included an NGINX reverse proxy in front of the FME Flow Web Application server. This allows users to select privileged ports under 1024 without having to worry about giving the web application root permissions. Leading to consistent permissions for logs and files across the installation. Secondly, it allows for easier SSL configuration. Lastly, NGINX can provide more security.  This component is not mandatory and if you want to disable or uninstall Nginx you can. See “Disabling FME Flow's NGINX reverse proxy in favor of your own ” for a method for removing Nginx and reconfiguring Tomcat. 
 

Linux FME Flow Checklists

This article goes through several checklists to help with the Online, Offline and distributed Installations.

Uninstalling

Backup Configurations:

Before uninstalling, back up your existing FME Flow configuration, especially if you’ve made custom modifications (e.g., changes to Tomcat or other components). Refer to the upgrade documentation if you're upgrading.

Custom Tomcat Modifications:

Uninstalling FME Flow removes the contents of your Tomcat directory if you installed the default Web Application Server. This folder is located in the FME Flow installation directory under Utilities\tomcat. If you made custom modifications to this Tomcat, these modifications should be backed up before uninstalling.

Components Not Automatically Removed:

The following components are reconfigured but not removed during uninstallation. You will need to manually uninstall them if you wish to completely remove them:

• PostgreSQL FME Flow Database:
  • If present, the existing database will be reused during reinstallation as long as it was installed from system packages.
• NGINX Reverse Proxy:
  • This will also be reconfigured during a reinstall but not uninstalled.

Note: If you wish to use a newer version of these components, remove them manually before reinstalling FME Flow. If NGINX remains installed, it may prompt to use port 81 instead of 80 on reinstall, but you can still select port 80 or another port.

Uninstallation Procedure:

1. Ensure that FME Flow, FME Flow Application Server, or any other web application servers are completely shut down.
2. Run the uninstall.sh script located in the FME Flow installation directory. Execute this script as root to ensure everything gets uninstalled properly.
3. After uninstallation, you may proceed with reinstalling or upgrading FME Flow, where the PostgreSQL and NGINX configurations will be automatically reused unless manually removed.

Customization 

Changing the FME Flow user account running the Server services

There are instances, especially in Distributed installations, where data is hosted in multiple machines, and you might have a specific account created to access data across a network. 

You can also manage this user account with your enterprise IAM and would like it to be a domain account instead of a local account. We can achieve this by changing the account that runs the FME Flow services. The default service account on Linux is the “fme flow” account. 

Steps:

1. Add or identify a new user account- You can create a new user and assign it to a group with the command  useradd -m -g <groupname> <username>
2. Confirm that the account has read/write rights to the FME Flow System Share and the FME Flow install directory 
3. Change the owner of all files and folders located under Logs to allow the new account to write to the FME Flow log directories.
4. Edit the startServer.sh and stopServer.sh script to replace all the super user “su” references from default “fmeflow” to the said account
5. Optionally, change file ownership and super user references from default “fmeflow” to the said account if Systemd scripts or SysV scripts are configured to run at system start-up

Disabling FME Flow's NGINX Reverse Proxy in Favor of your Own

If you have a native reverse proxy already set up and would like to use it instead of the Nginx reverse proxy shipped with FME Flow, we can disable the Nginx proxy. 

Installing proxies on all the FME Flow Web Applications in a fault-tolerant installation may be redundant and might increase complexity and points of failure. You can follow these steps to remove NGINX after installation.

Steps

1.  Remove the below line [loopback address] from the /opt/fmeflow/Utilities/tomcat/conf/server.xml file to allow traffic from outside of the machine. 
   address="127.0.0.1" 

2. Optionally, replace Connector port="8080" to 80/any port number if required.

3. Run the following commands to stop, uninstall NGINX, and remove the configuration files from the system

   1. sudo systemctl stop nginx

   2. sudo apt-get remove nginx nginx-common

   3. sudo apt-get purge nginx nginx-common

   4. sudo apt-get autoremove

   5. Reboot the host.

Moving the Engines to a New Directory

FME Flow engines can’t be installed in a different directory. However, it is possible to move the engines to a new directory after installation. The steps are outlined below.

Steps

1. Install FME Flow with Engine
2. Stop FME Flow
3. Move Engine directory  
   1. cp -R  /opt/fme-engine-2022 /EngineDirectory
4. Remove old Engine
   1. rm -rf /opt/fme-engine-2022
5. Edit the <FMEFlowDir>\Server\processMonitorConfigEngines.txt file:
   1. Under the Admin section at the bottom of the file, add the following line: NODE_OVERWRITE=true
   2. Change TEMPLATE_START_ENGINE to use the new directory. 
      1. TEMPLATE_START_ENGINE="/EngineDirectory/fme-engine-2022/fme
6. Start FME Flow

Starting and Stopping FME Flow

FME Flow provides a template you can use for setting up the FME Flow processes as services. These not only provide a method for starting the services on boot but also allow you to restart an FME Flow service in the event of a crash. This is also a location where you can add custom variables such as FME_TEMP and change the owner starting the services.

 Common Questions

What mix and match( Windows / Linux) options are available?
You should be able to install any operating system combination as Server core/Engines in a Distributed or a Fault-tolerant environment. A common setup is a Linux-distributed engine with a Windows core server distributed installation. 

Does Windows IWA / SAML, and Azure AD work on FME Flow?
Yes, all of the Authentication Services IWA, SAML, and Azure AD work on Linux installations.

Do readers or workspace connections with IWA work?
No, unfortunately, there is no way to allow the engines to work on behalf of a Windows user or service account.

I have Postgres already installed on my machine, will an express version work?
Yes, FME Flow should be able to pick up the existing Postgres installation and use it, but only if it was installed from the system package and was not manually installed from a direct Postgres download or another source.

What versions of Linux do you support?

We support different versions of Linux, please see FME Platform Technical Specifications for an up-to-date list. 

Documentation

System Requirements
Provide a Remote File System for the FME Flow System Share
Silent Installation
Uninstalling