SchemaMapper: Feature Type Mapping

Introduction

Feature type mapping maps one or more reader feature types to one or more writer feature types. It’s similar to attribute mapping, except it deals with FME Feature Types - layers or tables with their schema, rather than attributes within a schema.

Feature types generally represent layers, files, feature classes, or tables, depending on the source data and FME reader parameters. In the example below, two distinct sewer feature types (defined as ‘feature classes’ within ESRI’s Geodatabase structure) are merged into one outgoing generic sewer feature type simply by connecting the two reader feature types to one writer feature type. The same is done for the two water system feature types.

This direct reader-to-writer schema mapping is possible within FME, but may not be suitable for complex feature type mapping or if attribute mapping needs to be performed at the same time as feature type mapping. In the following demo, we’ll introduce a simple example of Feature Type mapping using the SchemaMapper.

Step-by-step Instructions

In the image above, multiple Sewer and Water data sets are combined into two tables in a single dataset. To map these features with a SchemaMapper, use a simple lookup table to define the inbound and outbound relationships. This type of lookup table can be created in an Excel spreadsheet or a text editor of your choice.

1) Create an Attribute Map

In the text editor of your choice, enter the following information separated by commas. (Feel free to skip this step and open the completed workspace to see how the final product works!)

1 SourceFeatureType,DestinationFeatureType
2 Sewers_Sanitary,Sewer
3 Sewers_Storm,Sewer
4 water_distribution_mains,Water
5 water_transmission_mains,Water

The first line contains the field names for our lookup table. The following lines contain pairs of feature type names, with the reader feature type on the left and the writer feature type to map it to on the right.

Spaces before or after commas in your lookup table can confuse FME during Feature Type mapping. The best practice is to omit all white space unless it exists in the attribute values.

2) Configure the SchemaMapper Transformer

Back in the workspace, open the SchemaMapper parameters dialog and set up the transformer to read in the CSV lookup table:

At this point, you may look at the lookup table reader parameters. Here you will find basic and advanced parameters that dictate how FME reads your lookup table, and how FME will use the lookup table to map attributes.

The default parameter values should be adequate for our example, and you should see an accurate tabular representation of your lookup table in the Preview pane.

3) Create Feature Type Map Parameters

Continue to the following SchemaMapper parameters, which define the “Actions” the transformer will take on your data.

Since we’re mapping feature types, click the Add (+) button and select Feature Type Map from the drop-down list. You’ll immediately be prompted for another set of parameters. Select your source and destination fields from the dropdown menus at the top. These fields are named according to the first row in the CSV file.

Similar to the Preview pane you saw earlier in the transformer reader Parameters, FME should display your lookup table, with the two columns in bold once they are successfully mapped:

Click OK to exit this dialog. The SchemaMapper transformer is now correctly set up.

4) Expose Attributes

The features have been mapped to their correct feature types so that the FME attribute fme_feature_type has been correctly set. This attribute can now connect the features to the correct feature type in the workspace.

To do so, place an AttributeExposer transformer (which will expose the fme_feature_type attribute) and an AttributeFilter transformer (which will divide the data based on its fme_feature_type value:

This image displays the result of each step described in the previous paragraph. We now have a filter that will check the fme_feature_type for the Sewer and Water values:

In the AttributeExposer parameters, pick fme_feature_type from the available list of attributes and click OK. Next, open the AttributeFilter parameters. The attribute we just exposed is the one we want to use as a filter, so select it from the dropdown list. Since we just mapped the fme_feature_type in the SchemaMapper from whatever our input was to either Sewer or Water, we need to add these two values to our list of “Possible Attribute Values”.

After you add the values, remember to connect each one to its appropriate writer feature type!

Conclusion

We are now about to run our Feature Type mapping workspace! Let’s go over what we’ve set up:

1. We have four incoming feature types.
2. We created a lookup table defining how we want our four feature types mapped and read it into the SchemaMapper transformer.
3. We exposed the fme_feature_type attribute, which the SchemaMapper redefined, so it could be used as a filter inside the workspace.
4. We filtered our features by an fme_feature_type of Sewer or Water.
5. We write the two new consolidated feature types.

This may seem like a lot of work and transformers compared to simply connecting multiple reader feature types to a single writer. However, if you imagine a larger production workspace with many dynamic inputs from various source datasets, combined with the need for attribute mapping before writing, this kind of workflow offers the most flexibility, organization, and control.

Splitting the Feature Type mapping process into multiple steps will also allow a greater degree of quality control and error reporting, as it is possible to see the routing of features and to track any features which are not successfully mapped, filtered, or written.

Tutorial Series

Configuring the SchemaMapper Transformer

The SchemaMapper requires an external lookup table to be configured. This article describes how to create the lookup table and connect it to the SchemaMapper.

SchemaMapper: Attribute Mapping

Attribute mapping is when one or more attributes on a feature are mapped by renaming their existing attribute names to a new one. This article demonstrates the design of a simple lookup table and how it can be used to map attributes.

SchemaMapper: Conditional Attribute Mapping

Conditional mapping uses the SchemaMapper’s conditional filtering capabilities. It is a way to define a simple clause to perform attribute mappings based on specific conditions. This article describes how to map a schema based on the value of attributes within the feature type.

SchemaMapper: Conditional Feature Type Mapping

This article describes how to use the SchemaMapper’s conditional filtering capabilities to define a clause to map feature types based on the value of attributes within the feature type.

SchemaMapper: Advanced Schema Mapping

This article demonstrates some of the advanced uses of the SchemaMapper, including using multiple clauses to filter features, adding new attributes, and modifying existing attribute values. The article also discusses the order of entries in the lookup table, which is useful for debugging.

SchemaMapper: Generating a Lookup Table from Source Data

The method of generating lookup tables using data sourced from input feature types is discussed in this article.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.