Getting Started With CI/CD (DevOps)

Sanae Mendoza
Sanae Mendoza
  • Updated

FME Version

Introduction

FME offers tools for CI/CD (DevOps) that improve communication, speed up development and enhance data quality. FME supports infrastructure as code for consistent deployments, making data workflows more efficient and reliable. 
 

Why Use FME for DevOps?

Use all the FME DevOps tools together for a fully integrated enterprise development cycle, or pick and choose tools that complement existing workflows. 

You might use FME for DevOps because you want to:

  • Seamlessly move workflows between environments, e.g.  development through production
  • Standardize configurations and settings between users and environments
  • Enhance collaborative workflow development
  • Deploy testing and monitoring for better data quality and delivery
  • Automate workflow deployment

 

Environmental Considerations

When deploying multiple environments, Safe recommends setting up identical architecture to reduce the risk of any errors migrating between each environment. This includes: 

Workspaces should be developed and tested on the same build of FME Form and the same operating system. 

When importing a project into the target FME installation or performing a backup and restore after an upgrade, if the source FME Flow has “Standard” encryption, you will need to set the target FME Flow's encryption as Standard and provide the target FME with the unique encryption key from the source FME.

 

Terminology

DevOps: A set of practices that combine software development (Dev) and IT operations (Ops) to automate and streamline the entire development lifecycle, fostering collaboration, efficiency, and faster delivery of high-quality software.

CI/CD: “Continuous Integration/Continuous Deployment” is a software development approach that involves automatically and frequently integrating new code changes into a shared repository, followed by automated testing and deployment of those changes to production environments. 
 

FME Tools for DevOps

Workspace Compare

FME Form and FME Flow simplify the continuous integration process by supporting source control integration with the Compare Workspace tool, enabling efficient management of workspace versions, tracking changes, and resolving differences using a graphical interface. 

Workspace Compare Git Integration

The Compare Workspace tool is compatible with git tools, such as Git on the command line, TortoiseGit, and Sourcetree, and can be configured to interface directly with them for viewing differences (git diff) and resolving conflicts following a merge attempt (git merge).

Deployment Parameter Store

The Deployment Parameter Store provides a centralized location for storing configurations such as database connection details, web credentials, URLs, domains, and usernames. Deployment Parameters can be referenced throughout FME Form and FME Flow, facilitating seamless migrations and standardization for all users.

FME Flow CLI

The FME Flow Command Line Interface (CLI) allows you to perform actions in FME Flow without accessing the Web UI. Using CLI commands for FME Flow commands, automate tasks in FME Flow like project management, configurations, backup and restore, and migrations.


FME Flow Projects 

Use Projects to save FME Flow items as a group. Projects can be used to migrate contained workflows between environments and through a development cycle.

 

FME Flow Version Control

Version Control in FME Flow allows access to previously saved workspace files. When a workspace is updated, previous versions are saved to a Git repository. Note that Version Control is separate functionality from Workspace Compare Git Integration.  
 

Monitoring and Logging

FME provides monitoring and logging capabilities to track installation, job status, system processes, and performance metrics, helping users maintain reliable and efficient workflows. 

Infrastructure as Code (IaC)

Templates to create reproducible FME Flow deployments for the cloud. For advanced users comfortable with IaC workflows. 

Additional Resources

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.