SchemaMapper: Generating a Lookup Table from Source Data

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2017.x

Introduction

When starting a schema mapping project it may be helpful to load all the feature type data into lookup tables for use with the SchemaMapper transformer. This creates a great starting point and eliminates the need to manually copy data by hand. Using this technique should dramatically speed up the schema mapping setup process, especially when there are multiple input files being mapped.

This article displays two separate methods to build a lookup table. Each lookup table is separated by the type of schema mapping being performed: Attribute Mapping, Feature Type Mapping, and Filtering.

 

Examples

Attribute and Feature Type Lookup Tables

In this example, a series of shapefiles are going to have their attributes mapped using the SchemaMapper. They are read into the workspace using the merge filter.

-yGh1Xt4AO_sm_Bfieudel7UvDZhIwMrAHZti-XwVbcaD_A2gAxkhEpqBTq0iWKyiSwqvoc58E8BtuQa_a9oDUqsCmpJXnfaNJeZOSZj_WVxh_ZR3IMEGtWPfjC3AWusiIY8bs4

The Schema (Any Format) reader pulls out all the attribute names from the files and then writes them to the lookup table. Since the reader pulls the attributes into a list, a ListExploder transformer is used to grab each attribute name.

U1P8GkeDaHOGj9QsV1VX1QviHTCDeOPfpa0nl9j9_0pv6SV67B1gyO3N_LHA3HS0jIlt1F5ZywVN13mZ7yK49AdREo81zd3WY08dwJERFCY9qsOHVW6Va8G1xQQudc8MXaNVnNY

At the same time, a separate table is generated listing each input Feature Type’s name using the incoming fme_feature_type name attribute.

Example feature type table:


Example attribte table:

GIknTyWdA-ehpEqowUVJJ_zYz8pv6O9UqfyDGmYDIlmwn2H9U03sd5U2XNxbpxuoTFS_B5-58eGLVTdeKmhLSb0trbH9LXpL4PbDT9KySbRaoS1KN0MnzBUsA1Awy8hRVUQqrHE

The resulting lookup tables both contain a blank column enabling a user to add their own schema mapping rules. Once completed, the lookup table can be processed by SchemaMapper transformer.

 

Filter Lookup Table

This type of lookup table is used with the SchemaMapper’s Filter Action. In order to set up a filter, the SchemaMapper must know attribute names and their unique values for the Feature Type. This workspaces produces a lookup table that contains unique values for each attribute contained by the input feature type.

7lT3XkqUg6fEUYFtURNt2dQtZu-vO_G7cwMg4eOPl1wVAnojCV0cPssgpDpN8H8s_XZ8fwnnQpn-YQlU4OnpdJGEwxDIK85lz5026VKA-goHdb_aP8E8PPPQDi5l6_LWQxtqj0c

In this example, a shapefile of water main data is going to be used to generate the filter lookup table.

First, the dataset is read into the workspace using a Generic reader. An AttributeRemover is used to strip out any unnecessary data (for example, the OBJECTID) and then each attribute is exploded and duplicates are removed. Finally, the data is sorted and written to the lookup table.

Example filter lookup table:

7OxLbm1xdkDx5SLQV1gJUIn4Oj5O3Wbhsyxwrm3tAn0Ghkc1wvT1DLFxqZHIZz_ozrGIpidXCuhpxQUFe1OJoGRtRIcRNPuVDFvmi9P77AVq5HvpQZy1ZOa5ySeo1Z7BtHPdWD4

In the lookup table output, extra blank columns (Source, Destination) are added, enabling a user to add their own schema mapping rules. Once completed, the lookup table can be processed by SchemaMapper transformer

 

 

 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.