Workbench Git Integration from the Command Line

FME also supports Github integration through the FME Flow, which allows you to commit to Github when publishing to FME Flow, as well as view versions, push, and commit using the FME Flow user interface. Please see Publish a Workspace, Transformer, or Custom Format. Please note that this article focuses on using Git commands with FME Workbench and that these steps will not enable version control functionality in the FME Flow web user interface or when publishing new versions from FME Workbench to FME Flow.

Introduction

FME Form's Git integration enables seamless support for CI/CD pipelines. This integration helps to streamline version control, and collaboration, leading to more efficient development processes and reliable deployments. You have the option to configure a git tool of your choice such as TortoiseGit , Sourcetree , and on the command-line to interface directly with the Compare Workspaces tool. You are able to view differences (git diff) and resolve conflicts (git merge). This article will provide step-by-step instructions to create, version, compare, and resolve conflicts in your workspace on the command-line

You can find a repository containing sample shell scripts, workspaces, and sample data to set up and test your git environment on Safe’s public Github space. 

Requirements

• FME Form 2023.0+
• Download attached Data containing a starter workspace
• Install Git on your local machine

Step-by-Step Instructions

Part 1: Set Up your Git Environment

1. Create Custom diff/merge Shell Scripts

You can find a copy of the following custom diff/merge shell scripts on Safe’s public github space. Create a copy of these on your local machine. We recommend placing them in a separate directory together on your PATH variable. 
Workspace_diff.sh

#!/bin/bash
# Arguments are: path old-file old-hex old-mode new-file new-hex new-mode

echo "Comparing changes on '" $1 "' using FME Workspace Compare"

fmeworkbench -TITLE-OVERRIDE "[$1] Compare Changes" -COMPARE-TITLE1 "Left" -COMPARE-TITLE2 "Right" -COMPARE $5 $2
exit 0

Workspace_merge.sh

#!/bin/bash
# Arguments are: ancestor-file current-file other-file path
echo "Launching FMEWorkbench to merge " $4 

fmeworkbench -TITLE-OVERRIDE "[$4] Merge" -COMPARE-BASE $1 -COMPARE-BASE-TITLE "Base" -COMPARE-TITLE1 "Ours" -COMPARE-TITLE2 "Theirs" -COMPARE $2 $3

2. Edit PATH Variables

Edit your PATH environment variable and add: 

• the directory holding the shell scripts (/Users/admin/Documents/mergefiles)
• the FME Installation Directory (/Library/FME/2023.0)

Your PATH variable should look something like this. Notice that the FME installation directory and the directory holding the shell scripts, /mergefiles, are present. 

echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/FME/2023.0:/Users/admin/Documents/mergefiles:/Applications/Contents/Public:/Library/Apple/usr/bin

3. Edit gitconfig

Register the custom diff/merge shell scripts as the custom drivers called “fmeworkbench” by adding the following to your gitconfig file. 
 

[diff "fmeworkbench"]
	command = workspace_diff.sh
	binary = true

[merge "fmeworkbench"]
	name = FME Workbench Merge Driver
	driver = workspace_merge.sh %O %A %B %P

4. Create a Repository to Store Workspaces

Create a local repository where you will keep track of versioned workspaces. This example uses a repository called “vesrionedfmw”. 

git init

Git init allows git to track changes  made to this repository.

5. Edit gitattributes

Since workspaces will eventually be compared through the FME Compare Workspaces tool, we don’t want Git to automatically try to merge workspaces as text files as things can get mixed up in a way such that FME Workbench will no longer be able to parse them.

To ensure this, we will configure our repository to use drivers to launch FME Workbench.  
Add the following line to your gitattributes file which can reside in any directory in the repository that contains your workspaces. The root directory of the repository is the simplest place to put it.
 

*.fmw diff=fmeworkbench merge=fmeworkbench 

Part 2: Compare Workspaces

1. Add a Workspace

Navigate to your repository on the command line. 

cd /users/fmeuser/versionedfmw 

Place the provided starter workspace (Version1.fmw) into your designated git repository.

2. Checkout a New Branch 

We will be adding an Inspector transformer to the starter workspace. Follow these steps to checkout a new branch where you will add the Inspector

git checkout -b addInspector

3. Make Changes to the Workspace

Using the following command, launch FME Workbench, passing a single file to open Version1.fmw

fmeworkbench Version1.fmw

This will open up Version1.fmw in FME Workbench. Add an Inspector transformer to the canvas, and connect it to the PostalAddress reader feature type. 


Save the workspace and exit out of FME Workbench. Stage and commit these changes

git add Version1.fmw
git commit -main

4. Compare Changes (git diff) 

Follow these steps in your command-line terminal to compare your changes

1) Passing “git status” will show that Version1.fmw has been modified. 
2) Now to compare these changes pass the command “git diff”. This will launch the FME Workbench in diff mode. 



3) You can now visually confirm that changes have been made to the workspace. 
4) We are happy with our changes so we will exit out of FME Workbench. 
5) Stage and commit these changes to git. 

git add Version1.fmw
git commit -main

6) return to your main branch

git checkout main

5. Checkout another branch

Next, we will make changes to the SpatialFilter. Checkout another branch and call it editSpatialFilter.

git checkout -b editSpatialFilter

6. Make Changes to the Spatial Filter

Launch Workbench to open the Workspace. You will notice your Inspector is not present in the workspace

fmeworkbench Version1.fmw

Edit the Spatial Filter to match the following parameters. 



Save your changes and exit out of Workbench. Stage and commit these changes

git add Version1.fmw
git commit -main

7. Merge Changes (git merge) 

We want to merge changes we’ve made to the Spatial Filter with the Inspector we added earlier. Pass the following command to open the Compare Tool in Merge mode. 

git merge addInspector

8. View Changes

You now see a three way merge of your changes. We want to merge all our changes onto “Our” side. Select the Inspector on “Their” side and click the Copy to Left button. Save and close Workbench.




The Workspace on the editSpatialFilter branch should have an Inspector added to it now. Repeat this process for workspaces you would like to track using git. Happy FMEing!