FME Server Deployment Life Cycle for the Enterprise

Introduction

For the purpose of this article, the deployment life cycle refers to the framework that describes the planning and maintenance involved in implementing FME Server software within your organization. This article introduces the deployment life cycle of an enterprise FME Server, from Development to Production, and any system between.
 

Life Cycle Stages

Deploying FME Server in the enterprise may involve several systems before the application goes live in a production environment. Typically, FME Server begins in a Development (DEV) environment, migrates to a Testing (TEST) environment, and then proceeds to a live, or Production (PROD) environment for everyday use in the enterprise. Other environments such as STAGE and PREPROD are also common.

The chosen FME Server architecture model and configuration should be replicated across each environment in the life cycle and the essential contents migrated correctly to ensure consistent operation of the FME workflows. 
 

Life Cycle Deployment & Migration

When migrating an FME Server through the enterprise life cycle, the efforts undertaken in the DEV environment must carry through to the next system. These efforts may include new workspaces or updates to existing ones, and changes to user management, security, automations/notifications, schedules, resources, connections, projects, streams, server apps, engine & queues and other third-party integrations with FME Server. These changes may have been made via the FME Server Web User Interface or might involve one of FME Server’s configuration or properties files. To account for these changes, each scenario requires somewhat different considerations, or workflows, for implementing a migration.

As well, replicating FME Server from one environment to the next may involve a regularly scheduled software upgrade process or fewer software upgrades that only occur when new features and enhancements are added to FME Server Application. As an enterprise, you must determine the breadth of the migration that suits your business requirements.
 

Migration Considerations

• Do you have the required software installers?
  • FME Desktop, FME Server and any other required 3rd party software? Have you recorded or acquired the FME Server Licensing information (serial number) so that the new software can be licensed after a software upgrade?  Does the system where FME Server is being installed have access to the internet (Automatic Licensing URLs) to permit automatic licensing or will manual licensing be required? [Licensing Options]  
• Have you read the latest online documentation for software installer options?
  • [Installation Options] 
• Are you using an external web application server as the FME Server Web Application?
  • This does not apply to an Express install, and is optional for a distributed/fault-tolerant install
  • [Provide a Web Application Server]
• Are you using an external database - PostgreSQL, SQL Server, or Oracle? 
  • This does not apply to an Express install and is optional for a distributed install
  • [FME Server Database]
• If providing your own database, have optional Database Client Software and FME Server database scripts been identified?  
  • Have the database login credentials been obtained?
    • The required database scripts to create the FME Server Database are accessible after installing FME Server.  
    • [Provide a Database Server]
• Will you be using domain service accounts for FME Server services?
  • Distributed environments often require the use of domain service accounts.  
  • If upgrading an existing FME Server environment, make a note of the services 'log on as' settings.
  • [Directory and Account Permissions]
• Have any changes been made to the configuration files of FME Server or to the properties file of the Web Application Server?
  • E.g. Has FME Server been configured for HTTPS?
  • [Configuring for HTTPS]
• Are you required to maintain the Job logs for historical records?
• For existing systems
  • Backup job history (if necessary).
  • Backup job log files (if necessary).
  • Record 3rd party software installed that is used by FME Server.
  • Record OS system environment variables; for example, FME_TEMP
  • Record OS system PATH entries, if applicable; for example, Oracle Instant Client.
• For any outstanding questions or clarifications, contact your local reseller or Safe Software Support.

Migration Limitations

The following are key configuration considerations when migrating FME Server, and the limitations for each.

Configurations Stored in the FME Server Database

There is no way to automatically determine the difference between any two FME Server Configurations at this time. Typically there should be no changes made to the configuration files but there are instances when manual changes are made, HTTPS configuration, for example.  In this case, it is necessary to identify the differences and reinforces why it is recommended to track changes to the environments.
Some configuration is stored with the FME Server Database, including processMonitorConfigEngines.txt and processMonitorConfigCore.txt. These configurations are migrated with a default backup and restore operation.  If however, the NODE_OVERWRITE directive is set to true in the configuration file(s), it means the config file is read during each startup of FME Server and not stored in the database.  
Ensure you are aware of the changes made to the config file and apply those changes to the new environment, but only when necessary.  As new versions of the software may not require the change.  If in doubt confirm with Safe Software Support or your FME Reseller.

FME Server Configuration Files ​​​​​​​
Much information about an FME Server installation remains in configuration files, which are not transferred during a migration. Changes to these files must be tracked and made to other systems in your life cycle. Consider backing up copies and using versioning software to track changes. DO NOT replace config files from older versions of FME Server, as newer config files may have additional settings that do not exist in the older versions.  We continue to improve the product and move the configuration to the FME Server Database.  It is important to ensure the configuration changes in the files are applicable to the new software.

Web Applet Properties Files
These files do not migrate automatically, and any changes must be recorded. Consider backing up copies and using versioning software to track changes. DO NOT replace the properties files from older versions of FME Server. Often these files are not modified and can be ignored.  

HTTPS/SSL and Single Sign-On
If you have made SSL configuration changes, then ensure you have the necessary information to apply the settings to the new software.  We do not support copying any files to the new software or new system.  We recommend repeating the Configuring for HTTPS in the new software for each system. Likewise, SSO configuration will need to be performed again if restoring to a new system. If FME Server is being installed to the same system, you’ll need to reapply changes to the Tomcat Properties File for SSO.

Security
User Configuration, Users, Roles, Tokens, Azure AD, SAML and Active Directory settings are backed up.  The new ‘admin’ superuser will not be affected in the new system, for example, the password will not get reset to the old system’s ‘admin’ password.  After migration, confirm the registered apps or IDP are using the correct hostnames (applies to Azure AD & SAML).  

Database and Web Connections for Workspaces
Database Connections for workspaces are migratable; however, be aware of any database aliases used in the connection syntax - for example, Oracle’s tnsnames entries or database instance names that may not exist in the next environment and may present problems. Using EZConnect syntax for Oracle databases avoids the use of tnsnames and may be the preferred approach.
Other Database types (SQL Server, PostgreSQL) may also be affected between environments.  It is important to understand how the Engine Service “Log On As” user was set and ensure the new system is configured in the same way, especially if your connections are using Windows Authentication.  Web Connections will be migrated as well. There are instances where you’ll need to revalidate the Web Connections after migration.

External Resources
Often there are supporting data sources or other template files used in FME workspaces. If these resources are not published to FME Server, they are not migrated. If these files are not in a network location, for example, accessed via a UNC path, you must manually copy or provide other automated provisions to move these files. (An external FME workspace can use FileCopy to move files if network connectivity exists between environments).

FME Server Resources
Depending on how users have published data to the FME Server, there can be very large files/folders within the FME Server Data Resource Folder or Workspace Repositories.  This can impact the file size of the backup file, and in some cases cause the backup or the restore to fail.  If the backup results in a very large .fsconfig file (> 4GB), you may want to investigate if there are files within the Data Resource folder or in the Repository Folder that can be temporarily removed before doing the backup and restore process.  Additionally, if Resources consist of AWS S3 Buckets or UNC Paths, only the configuration is backed up and not the files as these are outside the storage of FME Server. Please contact Safe Software Support if you are encountering problems with either the backup or restore processes and you have a very large .fsconfig file.

FME 32-bit and 64-bit Engines
NOTE: If your legacy environment is using 32-bit engines, that is no longer supported in FME Server 2022+. You will need to migrate any 32-bit dependant workspaces to 64-bit supported formats.
 

Pre-requisites: Admin Guide reading:

• Reviewed Before you Begin section.
• Reviewed Express Install for Windows or Linux section.
• Reviewed Upgrading FME Server section.
• Reviewed Workspace Life Cycle for the Enterprise.