Files
-
- 2 MB
- Download
Introduction
A linear network should consist of lines that meet at a point, without crossing. However, this is not always the case. There are various problems that can occur.
A misaligned point occurs where two lines are meant to meet at a known point, but one line does not properly connect:
An overshoot occurs where two lines are meant to meet at a known point, but one line extends beyond that intersection:
An undershoot occurs where two lines are meant to meet at a known point, but one line fails to reach that intersection:
A missing node occurs where two lines are meant to meet at a known point, but only one of those lines contains the intersection node:
In this case, the red (horizontal) line has an end node. The green (vertical) line has no node or vertex at the same location.
These issues are usually very small and invisible to the eye. Otherwise, they would be easy to detect without special data validation techniques. There is no single transformer for these scenarios, but a combination of general geometry-handling transformers can do the job.
The term "network" is used here, but these issues also apply to any linear features that are meant to form a closed structure. The parcel boundary lines used in this example are one such case.
The dataset for this example is a set of line features, in a MicroStation DGN dataset, representing property parcel boundaries in the city of Vancouver.
The dataset looks like this in the Data Preview:
The scenario here is to clean up the line features, ensuring all property parcels close correctly. This is proven by turning the lines into polygon features with an AreaBuilder transformer.
The template workspace and source data can be downloaded from the files section.
Step-by-Step Instructions
Follow these steps to identify and fix overshoots, undershoots, misaligned points, and missing nodes using the DGN dataset of Vancouver parcel boundaries.
Part 1: Locating Bad Linear Geometry
To assess the state of linear geometry, the simplest method is to use an AreaBuilder transformer. If the geometry can be turned into polygon features, then it all connects correctly.
A more complicated method uses a TopologyBuilder transformer. This turns the network into a series of nodes and edges. If nodes exist that are only used by a single edge, this indicates an unconnected line. If the line itself is very short, or the gap between a node and a neighboring line is short, this indicates an overshoot, undershoot, or misaligned point.
This example uses the simpler AreaBuilder method.
1. Open FME Workbench and Add Source Data
Open FME Workbench and start a new workspace. On the top menu, click Reader. In the Add Reader dialog:
- Format: Bentley MicroStation Design (V8)
-
Dataset: /2dparcels.dgn
- Browse to downloaded dataset
Click OK to add the reader
.
2. Add an AreaBuilder Transformer
Add an AreaBuilder transformer and connect its Input port to the ParcelLines reader feature type.
3. Add a FeatureColorSetter Transformer
For clarity, add a FeatureColorSetter transformer and connect its Input port to the AreaBuilder Area output port:
Open the FeatureColorSetter parameters and set the following:
- Color Scheme: Fixed
-
Pen Color: 0,249,0
- Click the ellipsis to select desired color
- Fill Color: 0,249,0
This sets the area color of the polygon. Run the workspace with caching enabled and inspect the output features. Incomplete features, meaning those that are invalid in some way, will appear in a separate layer and color in the AreaBuilder Incomplete output port:
Inspect the features by zooming in close to their endpoints. This shows whether the features are overshoots, undershoots, misaligned points, or missing nodes.
In a road network, some features may be highlighted but not actually incorrect - for example, a cul-de-sac or dead-end street. Manual assessment may be necessary. Alternatively, a LineOnAreaOverlayer or SpatialFilter transformer can show whether the lines overlap a polygon. If they do, they are more likely to be incorrect.
Part 2: Counting Bad Linear Geometry
Assuming the isolated features are actually incorrect, counting them can quickly be done using a StatisticsCalculator transformer.
4. Add a StatisticsCalculator Transformer
Add a StatisticsCalculator transformer and connect it to the AreaBuilder Incomplete output port. If no attributes are available, expose one on the source feature type (in the Format Attributes tab of the Feature Type parameters dialog), or use an AttributeManager to create one — the values are not important.
Open the StatisticsCalculator parameters and set the following:
- Attribute to Analyze: <any available attribute>
- Total Count: Enabled
5. Re-run the Workspace
The output now has a count of incorrect features. This is a total count, not a count per type of error.
Part 3: Fixing Bad Linear Geometry
Fixing bad linear geometry can be a case of trial and error. It’s good to have a specific tolerance value in mind, but also to use the transformers in the order given below. A different order can produce different results, and may introduce small pieces of unwanted linework.
6. Add a Snapper Transformer Between the ParcelLines and AreaBuilder
Add a Snapper transformer between the ParcelLines reader feature type and the AreaBuilder transformer. Connect both the Snapped and Untouched Snapper output ports to the AreaBuilder:
Examine the Snapper parameters. The default values will be fine for most parameters, but in this case you will have to set the following:
- Snapping Distance: 0.2
Re-run the workspace and inspect the output of the Snapper Snapped port by clicking the green inspect button. The result is that there are two fewer invalid features. Both of these were misaligned points that are now snapped into position.
Undershoots and overshoots might also be fixed by snapping, but none appeared within the specified tolerance in this dataset. The Snapper can handle those scenarios, but its primary use is fixing misaligned points.
For one feature, the misaligned feature had other features snapped to it, rather than being snapped to something else. That is why three features were snapped when only two needed fixing. This is not ideal, but the tolerance parameter keeps such snapping from being too extreme.
7. Add a LineExtender Transformer Between the Snapper and AreaBuilder
Add a LineExtender transformer between the Snapper and AreaBuilder transformers. Ensure the LineExtender Stretched output port is the one connected to the AreaBuilder Input port:
Open the parameters and set the following:
- Extension Length: 0.5
This extends all lines by 0.5m, which fixes undershoots. It also turns other features into overshoots, which the next step resolves.
8. Add an Intersector Transformer Between the LineExtender and AreaBuilder
Add an Intersector transformer between the LineExtender and AreaBuilder transformers. The Intersector Intersected output port is the data we want to keep:
This transformer cuts off overshoots at their intersection points and creates missing nodes.
Now re-run the workspace. A large number of features will appear as incomplete, but these are the overshoots that were cut off. All other features now form correct polygons, with no overshoots, undershoots, misaligned points, or missing nodes.
9. Recreate Linear Features with an Intersector Transformer
If the data must still be used as a network of lines, recreate the linear features with an Intersector transformer to convert the data back to properly noded lines.
For lines that are meant to remain incomplete, such as cul-de-sacs, use a LengthCalculator and Tester combination to filter out lines that are shorter than what is normally expected, since these are either an overshoot or an added extension, not a real feature:
Additional Resources
Data QA: Identifying Short Line Features [Article]
AreaBuilder Transformer [Documentation]
Data Attribution
The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.