Change Detection

Introduction

Change detection is used to identify differences between a set of input features. It can also identify features that have not undergone changes or are unchanged. Change detection is useful when comparing datasets or databases to see what changes have been implemented in a revised edition.
FME provides users with different transformers that can be used to detect changes depending on the data and what changes need to be detected. Such transformers include: ChangeDetector, UpdateDetector, Matcher, and CRCCalculator.

Change Detection Transformers

ChangeDetector 

The ChangeDetector gives you the option of detecting features that have been:

• Updated
• Inserted
• Upserted
• Deleted
• Unchanged

If you have a persistent key (detection key attribute) on both the Original & Revised datasets then the ChangeDetector will add the fme_db_operation attribute to the features. See the article Updating Databases: Using Change Detection to Set fme_db_operation . For line matching, ChangeDetector also includes matching using the Frechet distance.

In FME 2022.1, we have introduced the ability to perform UPSERT operations. Find a step-by-step guide in using UPSERT operations in these tutorials. 

• Updating a PostgreSQL Database Using UPSERT
• Updating ArcGIS Online Feature Services using UPSERT

Webinar series on Managing Changing Data with FME Part 1 and Part 2 gives a good background on change detection with FME.

Matcher

The Matcher is a more general approach to change detection and can be more convenient to use in some workflows. The Matcher is able to produce Unchanged (SingleMatched) and Changed (NotMatched) outputs.

CRCCalculator

Using the CRCCalculator with the Matcher can lessen the cost of the Matcher reading in geometry intense datasets on its own. By pairing the two transformers, the CRCCalculator can create unique values based on coordinates and attributes which then the Matcher can use to compare.

The CRCCalculator can also be used with the ChangeDetector and UpdateDetector to test against the unique values. Additionally, the CRCCalculator can be used with the FeatureMerger and Tester to produce Unchanged, Updated, Deleted, and Inserted outputs as shown in Creating a Unique Identifier.

Change Detection within Databases

Change detection within ArcSDE Geodatabases is possible by comparing the different versions of tables to see changes. This is further detailed in Extracting Differences from an ArcSDE Geodatabase.

Updating Databases shows an example of using the fme_db_operation to perform updates on a PostGIS database. Updates and inserts with the fme_db_operation can also be used within a Smallworld Database as in Database Operations in Smallworld.

See also Tutorial: Updating Databases for a more thorough description of database updates