SchemaMapper: Generating a Lookup Table from Source Data

Introduction

When starting a schema mapping project it may be helpful to load all the feature type data into lookup tables for use with the SchemaMapper transformer. This creates a great starting point and eliminates the need to manually copy data by hand. Using this technique should dramatically speed up the schema mapping setup process, especially when there are multiple input files being mapped.

This article displays two separate methods to build a lookup table. Each lookup table is separated by the type of schema mapping being performed: Attribute Mapping, Feature Type Mapping, and Filtering.

Examples

Attribute and Feature Type Lookup Tables

In this example, a series of shapefiles are going to have their attributes mapped using the SchemaMapper. They are read into the workspace using the merge filter.

The Schema (Any Format) reader pulls out all the attribute names from the files and then writes them to the lookup table. Since the reader pulls the attributes into a list, a ListExploder transformer is used to grab each attribute name.

At the same time, a separate table is generated listing each input Feature Type’s name using the incoming fme_feature_type name attribute.

Example feature type table:

Example attribte table:

The resulting lookup tables both contain a blank column enabling a user to add their own schema mapping rules. Once completed, the lookup table can be processed by SchemaMapper transformer.

Filter Lookup Table

This type of lookup table is used with the SchemaMapper’s Filter Action. In order to set up a filter, the SchemaMapper must know attribute names and their unique values for the Feature Type. This workspaces produces a lookup table that contains unique values for each attribute contained by the input feature type.

In this example, a shapefile of water main data is going to be used to generate the filter lookup table.

First, the dataset is read into the workspace using a Generic reader. An AttributeRemover is used to strip out any unnecessary data (for example, the OBJECTID) and then each attribute is exploded and duplicates are removed. Finally, the data is sorted and written to the lookup table.

Example filter lookup table:

In the lookup table output, extra blank columns (Source, Destination) are added, enabling a user to add their own schema mapping rules. Once completed, the lookup table can be processed by SchemaMapper transformer