Data QA: Identifying Self-Intersections with FME

Introduction

Self-intersections (also known as "loops", "bowties" or "fishtails") are features whose boundary twists around such that it intersects itself, causing a loop:

The left example shows where the endpoint of a polygon did not meet the start point, causing the polygon to close itself with a loop.

The right-hand feature is an example of points that may be out of sequence, or have mistakes. It comes very close to being a line that directly reverses back on itself, and very close to being a spike, but the layout actually causes a self-intersecting loop to form.

In most cases, self-intersections are not as obvious as these examples. A mismatched start and end point can be so close together that the self-intersection is nearly invisible. Disordered or stray points often have angles that are so acute they look like a single line.

FME incorporates the ability to locate and fix self-intersections using the GeometryValidator transformer.

The following workflow identifies and fixes self-intersecting polygons in a building outline dataset. The dataset contains over 50,000 buildings. To keep this example basic, three buildings are used that have been reported as possibly self-intersecting (ID numbers 102521333, 309854487, and 301873712). The following steps investigate whether these polygons self-intersect.

The workspace template and source data can be downloaded from the Files section.

Step-by-Step Instructions

The following steps walk through locating, counting, and fixing self-intersecting features using the GeometryValidator transformer.

Part 1: Locating Self-Intersections

Follow these steps to locate self-intersecting features with a GeometryValidator transformer.

1. Add Source Data
Open FME Workbench and begin with an empty canvas. On the Toolbar, click Add Reader. In the Add Reader dialog, set the following:

• Format: OpenStreetMap (OSM) XML
• Dataset: building.zip
• Workflow Options:
  • Single Merged Feature Type: Enabled

2. Add a Tester Transformer
On the Toolbar, click Add Transformer. In the Add Transformer dialog, search for Tester and click OK to add it to the canvas. Connect it to the building OpenStreetMap (OSM) XML reader output. Double-click the Tester transformer to open its parameters, and set the following:

• Test Clauses:
  • Clause 1:
    • Left Value: id
    • Operator: =
    • Right Value: 102521333
  • Clause 2:
    • Logic: OR
    • Left Value: id
    • Operator: =
    • Right Value: 301873712
  • Clause 3:
    • Logic: OR
    • Left Value: id
    • Operator: =
    • Right Value: 309854487

All buildings could be passed into the GeometryValidator transformer and only failed features recorded, but this example uses a subset of suspected features to improve speed and clarity.

3. Inspect the Data
Click the Tester transformer on the canvas to open the Mini Toolbar, then click Run To This. Once the run is complete, click View Source Data in the Mini Toolbar to view the data in Visual Preview. At a glance, the features may not be obviously self-intersecting, and it is difficult to identify where the self-intersection sites may be.

4. Add a GeometryValidator Transformer
On the Toolbar, click Add Transformer. In the Add Transformer dialog, search for GeometryValidator and click OK to add it to the canvas. Connect it to the Tester Passed port. Double-click the GeometryValidator transformer to open its parameters, and set the following:

• Validation:
  • Issues To Detect: Self-Intersections in 2D
• Output:
  • Attempt Repair: No

Re-run the workspace. Once the workspace finishes running, click the green inspect button next to each output port to inspect the output in Visual Preview. To see the Failed and IssueLocations outputs simultaneously, hold CTRL while clicking on their green inspect buttons. It looks like this:

The building on the left, if it is a single polygon, has obvious problems, with the outer perimeter crossing at several locations. Query the feature to confirm that it is a single polygon with a single perimeter.

The building on the right has less obvious problems. Zoom in closely to the top-right (north-east) corner of the building to see the problem:

This small offset is 0.7mm (about 0.03 in), demonstrating how a tiny mistake can cause a geometry problem. The output now contains all of the features with a self-intersection, along with a point feature marking where each self-intersection occurs.

Part 2: Counting Self-Intersections

In Part 2, we will count the number of self-intersections and failed features using the filtered features from the GeometryValidator. When the workspace is run with caching enabled, the outport ports show feature counts and give us insight into the number of issues:

The StatisticsCalculator transformer can be used to generate an attribute containing data QA insights, such as the number of problem features or self-intersections.

1. Add a StatisticsCalculator Transformer
On the Toolbar, click Add Transformer. In the Add Transformer dialog, search for StatisticsCalculator and click OK to add it to the canvas. Connect it to the GeometryValidator IssueLocations output port. Double-click the StatisticsCalculator transformer to open its parameters, and set the following:

• Statistics to Calculate:
  • Attribute: Any Attribute (e.g. area)
  • Total Count: Enabled

2. Re-run the Workspace

Run the workspace again. This time, the output includes an attribute indicating how many Issue Locations there are. 

To change the name of the output attribute from StatisticsCalculator, use the AttributeRenamer transformer. This can be repeated for multiple data QA issues and used to generate a report.

Part 3: Fixing Self-Intersections

There are a number of ways to fix a self-intersection. For example, a gap could be opened at the point of intersection so that the two pieces of line work do not cross:

Another solution is to divide the feature into two or more polygons, splitting them at the intersection point. This is the technique that FME uses with the GeometryValidator transformer.

The following steps use GeometryValidator to automatically fix self-intersecting features. If automatic cleaning is not desirable, the IssueLocations output port of the GeometryValidator transformer can be used to identify places to check where problems might be manually resolved.

1. Repair the Self-Intersecting Features
Double-click the GeometryValidator transformer to open its parameters, and set the following:

• Output:
  • Attempt Repair: Yes

2. Re-run the Workspace

Re-run the workspace and inspect the Repaired output. FME splits the original single-polygon features into multi-part features, each consisting of several polygons or outlines:

The data has now been cleaned of self-intersections.

Additional Resources

Data QA: Identifying Duplicate Features [Article]

Data Attribution

The data used here originates from open data made available by OpenStreetMap contributors. It contains information licensed under the Open Database License (ODbL) v1.0.