FME Flow Administration: Planning and Performing an FME Flow Installation

Liz Sanderson
Liz Sanderson
  • Updated

Introduction

There are a number of factors that need to be considered when you are planning an FME Flow (formerly FME Server) installation. This guide is intended to provide all the resources you need to help you make decisions about how to implement your FME Flow. 

As you walk through this document, we recommend you keep in mind the following questions. How you answer these will help determine what FME Flow deployment is right for you. 

  • Why are you using FME Flow - what are your jobs going to be doing? 
  • Where are your data sources located? 
  • Have you considered a CICD environment cycle? 
  • What third-party technology do you need to connect with? 
  • Do you need to use a specific Database Provider?
  • Will you need to scale FME Flow? 
  • Do you need a public-facing FME Flow?

 

Planning your FME Flow Deployment 

Deployment Options/Installation Types

The FME Flow Installer offers a variety of deployment options. There is no right or wrong option, and the model you choose to deploy will vary depending on your organization’s requirements and planned usage. You should base your decision around what your jobs will be doing, security, and redundancy. There are three different deployment models to be aware of: 

  • Express
  • Distributed
  • Fault-Tolerant

For an in-depth guide of what these options are and how they behave, please refer to A Guide to Choosing your FME Flow Deployment Architecture.


Disaster Recovery 

Disaster recovery is designing with the intention of being able to regain FME Flow operations and data in the event of a major failure of a data center. For more information, see: Planning for Disaster Recovery.  


Operating System (OS) Considerations

FME Flow can be installed on Windows or Linux OS. However, not all formats are supported on Linux platforms: FME Applications and Format Support on Linux.

If you opt to install FME Flow on a Linux OS, we recommend that your FME Flow Administrator has Linux experience. The Technical Support Team has put together some resources that should be reviewed before proceeding with this setup: 

If you are mixing operating systems, for example, FME Form (formerly FME Desktop) is Windows, but FME Flow will be Linux, you may encounter connection issues accessing file paths. Workspaces will need to be set up to use network shares and may need to be modified before they run successfully on FME Flow.  

Note: Other installation options that are not covered by this document include Containerization (Docker & Kubernetes), Cloud Marketplace (Azure & GCP), and FME Cloud. For more information, please see the following: 


Hardware & Resource Requirements

A common question is, what resources should I have available on my server? There is not one right answer as the resource requirements are dependent on how many engines you have running and what your workspaces have been built to do. To aid these decisions, if these workspaces are already built and running in FME Form, view the peak memory and CPU usage information at the bottom of each translation log file.

The following resources have been put together to try and help make those decisions: 


FME writes many temporary files to the system temp directory by default. If there are space limitations, you may wish to set the FME_TEMP environment variable to override this file path following these instructions: Setting a temporary file location for FME to use via the FME_TEMP environment variable.

The job activity on FME Flow should be continually monitored and your FME Flow should be scaled accordingly to meet any influx in usage. FME Flow job activity can be monitored using the REST API. This article has some suggestions for setting up Dashboards: Monitoring FME Flow Job Activity using the REST API.


Engine Type

Engines are the component of FME Flow that processes your workspaces. FME Flow supports two different engine types. These two engine types are technically identical. However, their pricing model differs. See FME Flow Pricing

Hence, this is a consideration for when you are planning your FME Flow deployment. The jobs that you are planning to run on FME Flow will likely determine which engine type is most appropriate for your installation. In particular, think about your job duration, frequency, and how the jobs will be triggered.  
 

Engine Type Standard CPU-Usage (Dynamic)
Pricing Purchase a perpetual fixed engine. Purchase credits and charged by the CPU time the engine spends processing a job. That is, no cost for idle engines.
Scalability Launch the number of Engines assigned to your FME Flow license. If you require additional capacity, you’ll need to contact your Account Manager.  Launch as many FME Engines as you want when you need them. The only restrictions are the ability of your hardware to support this processing power. 
Best for processing jobs that are... Predictable and consistent workloads. Includes but is not limited to: 
  • Scheduled jobs
  • Automations
Unpredictable workloads. Includes but is not limited to: 
  • Infrequent big data loads
  • FME Workspace Apps
  • Streams
  • Automations



Standard and CPU-Usage engines can be used together to provide ultimate flexibility. To learn more, see Getting Started with CPU-Usage (Dynamic) Engines

For better performance, your FME Flow Engines should be as close to your data as possible. This reduces the effect of network latency and wait times. Keep this in mind when choosing whether FME Flow should be installed on-premise or in a virtual.
Note: Since these engines are technically identical, there are no changes to the install instructions to deploy one or the other, rather, you’ll need to make sure this is accounted for when you purchase FME Flow. 


CI/CD Lifecycle

Your organization may require you to support Continuous Integration/Continuous Delivery (CI/CD) by providing multiple FME Flow environments through the lifecycle, for example, from DEV > UAT > STAGE > PROD. When deploying multiple environments Safe recommends setting up identical architecture to reduce the risk of any errors migrating between each environment. This includes: 

Workspaces should be developed and tested on the same build of FME Form and the same operating system, and then Projects can be used to migrate contents between environments. 

For more information, please see the following: 


Installing FME Flow

Now that you’ve reviewed this guide, you should be ready to proceed with your FME Server installation following the appropriate documentation: 

When installing FME Flow, there are some important considerations to be aware of: 


Licensing

The license file is stored in <FMEServerSystemShare>\licenses.

For help requesting a license, review the instructions in the Documentation.

There are two methods available to license FME Flow through the Web UI; Automatic and Manual. 


Automatic Mode

This method of licensing is the recommended route; you must have your serial number to hand and input this information into a form in the FME Flow UI. To license automatically requires access to the internet. If your environment is locked down, follow the FME Flow Automatic Licensing URLs article for guidance on what outbound connections to allow. 

To use Dynamic Engines, you must be able to license automatically. 


Manual Mode

If you are not able to use an internet connection using the manual method, you can download a JSON file and email this to codes@safe.com, where a license file will be generated using the information provided. The response is not immediate, and you should prepare to wait up to half an hour to receive the license file that can then be uploaded through the Web UI. 

Dynamic Engine licensing is not supported by manual offline requests. 


Silent Install Licensing

Alternatively, if you would like to license FME Flow as part of a silent install script, you can make a call to the FME Flow REST API.

Automatic POST /licensing/request

Manual POST /licensing/requestfile

Note: If you are using FME 2020.0 or newer, you must set the installer property FIRSTLOGINCHANGEPASSWORD=false. This will disable the requirement to update the password on the first login, meaning you can use the admin\admin credentials to authorize the request. As part of your script, after the license has been successfully requested, you should update your Administrator password for better security. 


Processes, Ports, and Firewall

FME Flow components and processes require communication over a variety of ports. If you have a firewall in place, you’ll need to create inbound/outbound rules for FME Flow to function as expected. For more information, see FME Flow Ports


Account Permissions

On a distributed/fault-tolerant install, or if you are accessing data stored outside FME Flow System Share, you’ll be required to run FME Flow Services under a service account. For information on the required permissions, see Directory and Account Permissions.


Third-party Licensing and Workspace Dependencies

To make use of Esri-related formats (ArcSDE Geodb, File Geodb, Personal Geodb), Esri software must be installed and licensed on the same machine as your FME Flow Engine Service. For more information, please see: Using FME Flow with Esri ArcGIS Software

Once FME Flow is installed, depending on the components in use within your workspaces, other provisions may be required on FME Flow. Some common examples include: 

  • Python Modules
  • Custom Coordinate Systems
  • Packages
  • Web /Database Connections
  • Format Dependencies e.g. JDBC Drivers 
    Note: This information is available in the ‘Quick Facts’ on the format help page, and if a dependency is missing, there should be a clear error in the translation log when it is run on FME Flow.


Optimization

Optimizing FME Flow is an iterative process and requires continuously analyzing your existing setup. Some of the deciding factors for system improvement are:

  • Job processing time
  • System resource usage
  • Number of engines available
  • Business requirements 

Check out this webinar where we dive into Best Practices for Optimizing Jobs on FME Flow.


What's Next? 

After FME Flow has been installed, you may wish to perform some additional configurations; these are for enhanced security on FME Flow. For further guidance, see FME Flow Administration: Post-Installation Configuration.

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.