Introduction
While moving, integrating, and transforming data, it's also important to ensure that the data quality is of a suitable standard.
FME offers a range of tools and transformers for data validation. Some, such as the GeometryValidator and AttributeValidator transformers, are specifically designed for data validation. Others, as this tutorial will show, are not specific to data validation but can be used for that purpose.
When assessing data quality ,there are three techniques that can be applied:
- Identifying Problems: Identifying features that do not conform to the required standard
- Counting Problems: Assessing overall data quality by quantifying the number of substandard features
- Fixing Problems: Improving data quality by fixing the issues found in substandard features
Each example in this tutorial will include information on how to identify, quantify, and address the problem being discussed.
Tutorials
Geometric Inconsistencies
These are geometries that are inherently flawed, rather than being part of a substandard network or coverage. This includes:
Duplicates
These are features that have some form of duplication. This includes:
- Data QA: Identifying Duplicate Features with FME
- Data QA: Identifying Consecutive Duplicate Vertices with FME and Data QA: Identifying Non-Consecutive Duplicate Vertices with FME
- Data QA: Identifying Duplicate Attribute Values
Small Features
Often, very small features are indicative of poor geometry and/or topology. This includes:
- Data QA: Identifying Short Line Features
- Data QA: Identifying Small Polygon Features
- Data QA: Identifying Features Closer than a Minimum Distance (features too close to one another)
Area Topology
These are issues related to a continuous polygon coverage. This includes:
Linear Topology
These are issues relating to a linear network. This includes:
Spatial Concepts
These are geographic features containing some form of logical issue, for example, a road represented by a polygon or a bicycle path that runs through a lake. This includes: