Getting Started With CI/CD (DevOps)

Sanae Mendoza
Sanae Mendoza
  • Updated

FME Version

  • FME 2023.1

Introduction

FME offers tools for CI/CD (DevOps) that improve communication, speed up development and enhance data quality. FME supports infrastructure as code for consistent deployments, making data workflows more efficient and reliable. 
 

Why Use FME for DevOps?

Use all the FME DevOps tools together for a fully integrated enterprise development cycle, or pick and choose tools that complement existing workflows. 

You might use FME for DevOps because you want to:

  • Seamlessly move workflows between environments, e.g.  development through production
  • Standardize configurations and settings between users and environments
  • Enhance collaborative workflow development
  • Deploy testing and monitoring for better data quality and delivery
  • Automate workflow deployment

 

Terminology

DevOps: A set of practices that combine software development (Dev) and IT operations (Ops) to automate and streamline the entire development lifecycle, fostering collaboration, efficiency, and faster delivery of high-quality software.

CI/CD: “Continuous Integration/Continuous Deployment” is a software development approach that involves automatically and frequently integrating new code changes into a shared repository, followed by automated testing and deployment of those changes to production environments. 
 

FME Tools for DevOps

Workspace Compare

FME Form and FME Flow simplify the continuous integration process by supporting source control integration with the Compare Workspace tool, enabling efficient management of workspace versions, tracking changes, and resolving differences using a graphical interface. 

​​Tutorial: Using FME's Comparison Tool
 

Workspace Compare Git Integration

The Compare Workspace tool is compatible with git tools, such as Git on the command line, TortoiseGit, and Sourcetree, and can be configured to interface directly with them for viewing differences (git diff) and resolving conflicts following a merge attempt (git merge).

Git Documentation: https://github.com/safesoftware/workspace-comparison-configuration
 

Deployment Parameter Store

The Deployment Parameter Store provides a centralized location for storing configurations such as database connection details, web credentials, URLs, domains, and usernames. Deployment Parameters can be referenced throughout FME Form and FME Flow, facilitating seamless migrations and standardization for all users.

Tutorial: Getting Started with Deployment Parameters 
 

FME Flow CLI

The FME Flow Command Line Interface (CLI) allows you to perform actions in FME Flow without accessing the Web UI. Using CLI commands for FME Flow commands, automate tasks in FME Flow like project management, configurations, backup and restore, and migrations.

Git Documentation: https://github.com/safesoftware/fmeflow-cli 
Tutorial: Getting Started with the FME Flow Command Line Interface (CLI)


FME Flow Projects 

Use Projects to save FME Flow items as a group. Projects can be used to migrate contained workflows between environments and through a development cycle.

Tutorial: Getting Started With FME Flow Projects
 

FME Flow Version Control

Version Control in FME Flow allows access to previously saved workspace files. When a workspace is updated, previous versions are saved to a Git repository. Note that Version Control is separate functionality from Workspace Compare Git Integration.  
 

Monitoring and Logging

FME provides monitoring and logging capabilities to track installation, job status, system processes, and performance metrics, helping users maintain reliable and efficient workflows. 

Tutorial: FME Flow Enterprise Log Integration 
Tutorial: Monitoring FME Server Job Activity using the REST API
Guide: FME Flow Debugging Toolbox 
 

Infrastructure as Code (IaC)

Templates to create reproducible FME Flow deployments for the cloud. For advanced users comfortable with IaC workflows. 

Guide: Infrastructure as Code (IaC) for Distributed FME Server Deployments in the Cloud
 

Additional Resources

Tutorial: Getting Started with FME Flow Administration
Tutorial: Getting Started with Enterprise Integration Patterns

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.