FME Flow Workspace life cycle for the Enterprise

Introduction

Developing and deploying a new FME Workspace in the Enterprise may involve several systems before the workspace goes live in a production environment. The workspace may begin in a Development (DEV) environment and then be migrated to a Testing (TEST and/or QA) environment for checks or quality assurance. After passing all the tests and checks, it can be promoted to the Enterprise Production (PROD) environment.

There are four methods available for publishing and migrating workspaces through the DEV/TEST/PROD environments, each with its pros and cons:

• FME Form (formerly FME Desktop) Publish Wizard
• FME Flow (formerly FME Server) Backup and Restore
• FME Flow Projects
• FME Flow REST API

Recommendations and Best Practices

FME Form Publish Wizard

The recommended workflow is to use FME Form to publish a workspace to FME Flow. FME Form takes care of all the settings and registrations for the different FME Flow web services (e.g., Data Streaming, Data Download, Job Submitter, and others).

Pros

• Easily register workspaces to any of the available web services.
• Configure workspaces to interact with the Notification Service.
• No custom or third-party applications are required.

Cons

• Automation is not possible. Requires manual migration to move from DEV to TEST to PROD.
• Inefficient when dealing with large numbers of workspaces and different systems.
• When publishing a new workspace, a review of service registrations, security policies, and resources may be required.
• For more complicated workspace workflows that involve multiple workspaces (parent-child), notifications, and file resources, the admin must follow documented, repeatable steps that produce a consistent result.

Publishing using FME Flow's Backup and Restore

FME Flow provides backup and restore tools that allow you to migrate your existing FME Flow configuration (workspaces, service registrations, resources, etc.) from one environment to the next in the workspace life cycle, for example, from a DEV to a QA environment. These tools are available on the FME Server Web User Interface and do not require the FME Form.

Pros

• Easy migration of all workspaces in one step.
• Ensures all workspaces are migrated - no missing pieces. Maintains configurations in the previous environment, including service registrations, security policies, and resources.
• Can update existing workspaces to newer versions, if desired.
• Can insert new workspaces only without overwriting existing workspaces if desired.

Cons

• Cannot track details about changes (updates vs inserts) - no visibility of what was replaced or inserted.
• Cannot select a subset of workspaces to restore.
• The option to overwrite existing items is all-or-nothing. Cannot pick and choose.
• When using Active Directory security, considerations for different FME Flow security modes must be considered. For example, you may develop a workspace in DEV where Active Directory is not used. After restoring to the PROD environment, security must be reviewed in the FME Flow Web User Interface to ensure the appropriate users and roles have access as designed.

Migration using FME Flow Projects (similar to Backup and Restore)

FME Flow allows you to create a Project and assign different kinds of objects. Object types include Repositories, Users, Automations, Server Apps, API Tokens, Resources, Resource Connections, Database and web connections, Cleanup Tasks, Topics, subscriptions, publications, Schedules, and Other projects. You can then export and import the project into another FME Flow instance.

Pros

• Easy migration of workspaces and related objects
• Chunk workflows into smaller, more manageable parts to migrate
• Wrap Projects up into a master project
• Can update existing objects to newer versions
• Pause the Notifications System during import
• Can review the history of Project Import and Export Activity

Cons

• May need to deal with multiple projects, and it could be challenging to manage

Publishing using the REST API

FME Flow's REST API allows you to interact with FME Flow programmatically. In addition to publishing, the REST API provides methods to control a range of FME Flow functionality, including: 

• Sending, receiving, and modifying notifications
• Running jobs and viewing job history
• Uploading and managing file resources

Pros

• Efficiency - Can create an automated, repeatable process for workspace migration.
• Desirable results for workspaces that use the FME Flow Job Submitter service.
• Integration with external source control applications via webhooks.

Cons

• Cannot configure how workspace Readers and Writers interact with the Notification with different Services.
• It requires customizing or developing an application to interface with the FME Flow REST API and the source location of workspaces. A developer resource is likely required.
• All configurations in the previous environment are not automatically maintained. When publishing a new workspace, this requires reviewing service registrations, security policies, and resources.