Data QA Identifying Sliver Overlaps and Gaps in Polygon Coverage

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2021.1

Introduction

Overlaps and gaps are problems that occur with invalid polygon coverages. A polygon coverage should be a single, continuous surface. However, this is not always the case.

An overlap occurs where two or more polygons intersect each other, creating an area covered by multiple polygons. For example, here the red feature extends over the orange feature:

overlapsandgaps1.png

 

A gap occurs where two or more polygons fail to meet at the boundary, creating a hole in the coverage. For example, here the red and green features fail to meet, creating a white gap in the coverage:

 

overlapsandgaps2.png

In general, these overlaps and gaps are very small and narrow - else they would be easily detected without the need for special data validation techniques - and this common shape and size is why they are sometimes called a sliver.

Overlaps can be located using the Small Polygon method, but that won't identify gaps at the same time, and neither can be fixed using the same methods. So, to handle both slivers and gaps FME has a special transformer called the AreaGapAndOverlapCleaner.

 

Source Data

The dataset for this example is a set of polygon features (in a MapInfo TAB dataset) representing planning zones in the city of Vancouver.

The dataset looks like this in the FME Data Inspector:

overlapsandgaps3.png

 

Logically a piece of land cannot fall into two zones (i.e. the zones should not overlap) but equally a piece of land cannot be unzoned (i.e. there should not be any gaps in the zoning coverage).

The scenario here is to simply locate, count, and fix any overlaps or gaps in our coverage.
 

Step-by-Step Instructions

Part 1: Locating Overlaps and Gaps

The AreaGapAndOverlapCleaner transformer, although designed for handling gaps and overlaps, will only fix them. It will not highlight or locate them to be fixed elsewhere. To simply locate such features we'll use the AreaOnAreaOverlayer transformer, in combination with the Dissolver and DonutHoleExtractor.

Follow these steps to learn how to identify overlaps and gaps.
 

1. Start FME Workbench and create a new Workspace

Select Readers > Add Reader from the menu bar.

Set the data format to MapInfo TAB (MITAB). Select the attached tab file as the source dataset and click OK to add the reader.
 

2. Creating a Single Surface and comparing it with an AreaOnAreaOverlayer

To locate gaps and overlaps the idea is to create a single surface that covers the area of interest, and then compare the data with an AreaOnAreaOverlayer. Add a Dissolver transformer connected to the Zones data. This will create a single surface feature. However, it can be created as a donut feature, which is not what we need. Therefore, add a DonutHoleExtractor transformer and connect the Dissolver Area Output Port to it.

overlapsandgaps4.png

The default parameters are correct for this current exercise.
 

3. Now place an AreaOnAreaOverlayer transformer. Connect both the DonutHoleExtractor:Outershell port, and the Zones data to its input port:

overlapsandgaps5.png

Again, any parameters can be left in their default state for this exercise.

Note that in FME2018 or newer, the AreaOnAreaOverlayer transformer has an extra Remnants output port.
 

4. Add an Inspector transformer.
Connect the Inspector transformer to the AreaOnAreaOverlayer:Area output port. Check the Inspector's parameters and set the Group By setting by selecting the _overlaps attribute.

Now run the workspace. The Display Control Window in the output looks like this:

overlapsandgaps6.png

Where the number of overlaps is equal to 1, we know that these are gaps (because the surface created by the Dissolver is not overlapped by Zones data).

Where the number of overlaps is greater than 2, we know that these are overlaps (because the surface created by the Dissolver is overlapping at least two other polygons in the Zones data).

Only where the number of overlaps is equal to 2 is the data correct.

To confirm the issues, add the original source Zones data as a backdrop and zoom into one of the problem features:

overlapsandgaps7.png

The grey line here is a gap between two polygons. It's only 0.0022 metres (2mm or 1/12th of an inch) so you will have to zoom in very close to be able to see it.

So now we have isolated very small and very narrow polygons that represent gaps and overlaps in the polygon coverage.
 

Part 2: Counting Overlaps and Gaps

Counting the number of bad features is quite easy because we have already filtered them out. We just need to create a count of them using the StatisticsCalculator transformer.

Follow these steps to learn how to count overlaps and gaps features.
 

5. Add a StatisticsCalculator
Add a StatisticsCalculator between the AreaOnAreaOverlayer transformer and the Inspector. Check the StatisticsCalculator transformer's parameters.

Select the _overlaps attribute for both the Group By parameter and the Attributes to Analyze parameter.

Set the Total Count Attribute parameter to NumFeatures and accept the changes.
image2.png
 

6. Turn off the Group By in the Inspector transformer and re-run the translation.
The output will now have a count of features for each overlap number. This will tell us how many features there are in each category:

overlapsandgaps8.png

So in this example, we have 6 gaps, 6 overlaps, and 430 features that are correct.

NB: If you connected the StatisticsCalculator:Summary output port to the Inspector, there will only be a single output feature per group. To get all output features use the Complete port instead.
 

Part 3: Fixing Overlaps and Gaps

Fixing overlaps and gaps is a simple task using the AreaGapAndOverlapCleaner transformer, and to confirm that it has worked we can simply repeat the location process above.

So, follow these steps to learn how to fix overlaps and gaps using the AreaGapAndOverlapCleaner transformer.
 

7. The help information for the AreaGapAndOverlapCleaner transformer states that it requires clean, valid data and that self-intersecting or degenerate polygons will not be repaired.

So, to ensure we clean the data of these issues first, place a GeometryValidator transformer. Connect it to a new output from the Zones feature type:

overlapsandgaps9.png

Check the GeometryValidator parameters and add the tests for "Degenerate or Corrupt Geometries" and "Self-Intersection in 2D". Ensure that the setting Attempt Repair is set to Yes. Leave the rest as default. 
image1.png
 

8. Add an AreaGapAndOverlapCleaner transformer.
Connect it to the Passed and Repaired ports on the GeometryValidator transformer.

overlapsandgaps10.png

Check the AreaGapAndOverlapCleaner parameters. The only one of importance is the Repair Method. There are various methods, which the transformer documentation explains in more detail. For now, leave the Repair Method as "Longest Boundary".
 

9. Connect an Inspector transformer to the AreaGapAndOverlapCleaner:Repaired port and run the translation.

The output will be the polygons with all gaps and overlaps repaired to create a single, continuous surface. However, it will be difficult to prove this without very close inspection of the data. So, let's recreate the prior check of data to ensure there are no remaining problems.

To do this, select the existing Dissolver, DonutHoleExtractor, and AreaOnAreaOverlayer transformers. Press Ctrl+D to create duplicate copies of these.

Now connect these to the AreaGapAndOverlapCleaner in the same configuration as above (i.e. there is one output from the AreaGapAndOverlapCleaner to the Dissolver and another to the AreaOnAreaOverlayer):

overlapsandgaps11.png


10. Add an Inspector transformer
Connect it to the new AreaOnAreaOverlayer and run the translation. This time you'll be able to see that there are no features that overlap, and no gaps.

You can also see that there is the same number of output features as before. The overlaps and gaps have been distributed to other features depending on the method chosen in the AreaGapAndOverlapCleaner transformer.

Also, do notice that the AreaOnAreaOverlayer causes attributes to be invalid. All output takes the attributes from one feature. For this reason, only take the AreaGapAndOverlapCleaner repaired output for further use; never use the AreaOnAreaOverlayer output except for verification of the polygon coverage.
 

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.

 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.