Files
Introduction
The workflow in the previous example reads any data and dynamically copies the schema from the source to the destination, generating an output that mirrors the input.
It is also possible to map the source data to a new schema and generate it dynamically. This is useful for enforcing a specific output or when the desired schema already exists. Any reader dataset, in any format, can serve as the template for the destination schema by adding it as a resource reader.
Workspace and Data
A finished version of the workspace created in this example can be found in the zip file attached to this tutorial, titled "2-ExternalDataset".
Step-by-Step Instructions
In this example, a project provides updated city data as a standardized set of shapefiles. However, updates are received in the OGC Geopackage format. To handle this scenario, a dynamic workspace is created using the standardized shapefiles as schema resources.
1. Add the Source Dataset
Open a new FME Workspace. From the Readers menu, select "Add Reader" and configure it as follows:
- Format: OGC Geopackage
- Source: <Path>\resources\CityDatabase.gpkg
-
Parameters:
- Tables: BusStops, CityParks, Roads
-
Workflow Options:
- Individual Feature Types: enabled
This will read the three selected feature types in the dataset.
2. Add the Destination Dataset
From the Writers menu, select "Add Writer" and configure it as follows:
- Format: Esri Shapefile
- Dataset: <Path>\output
- Shapefile Definition: Dynamic (Advanced)
3. Use Workspace Resources to set the Schema
In the Readers menu, select "Add Reader as Resource."
Configure it as follows:
- Format: Esri Shapefile
-
Dataset: <Tutorial Download>\resources\schema_templates\BusStops.shp,<Tutorial Download>\resources\schema_templates\CityParks.shp,<Tutorial Download>\resources\schema_templates\Roads.shp
- Hold shift to select all three at the same time
Once the resources are added, an entry is placed in the Navigator window:
4. Set the Dynamic Writer to use the New Schema
To map the source data into a new dynamic schema, open the Writer Feature Type parameters and use the “Schema Sources” setting. By default, the schema from the original source dataset, in this case 'CityDatabase', is entered into the dialog. Change the setting to point to the new schema:
Click OK and close the dialog.
5. Map the Schema
Finally, configure the workspace to map the source to the new schema. Add an AttributeManager to the canvas, connect it to all three reader feature types, and connect the output port to the writer. In the parameters, change the following:
Attributes to Rename:
-
Input Attribute: PRIMARYINDEX
- Output Attribute: PID
-
Input Attribute: class
- Output Attribute: ROAD_CLASS
-
Input Attribute: featureID
- Output Attribute: FEATUREID
-
Input Attribute: numberOfLanes
- Output Attribute: ROAD_LANES
-
Input Attribute: routeNumber
- Output Attribute: ROUTE_NUM
-
Input Attribute: routeType
- Output Attribute: ROUTE_TYPE
-
Input Attribute: type
- Output Attribute: ROAD_TYPE
-
Input Attribute: name
- Output Attribute: PARK_NAME
-
Input Attribute: name_alt
- Output Attribute: NAME_ALT
-
Input Attribute: STOPABBR
- Output Attribute: STOP_ABBR
-
Input Attribute: STOPNAME
- Output Attribute: STOP_NAME
Attributes to Remove:
- id
- ATSTREET
- BAY
- CORNER
- FACILITY_T
- GIS_ID
- ONSTREET
- PLACEMENT
6. Run the Workspace
Below is an image of the final workspace. Run the workspace and check that the output files contain the correct attribute schema.
Debugging Tip
The feature types defined by each selected schema source will be logged. For example:
Reader 'SHAPEFILE_1' of type 'SHAPEFILE' provided schema for types: BusStops,CityParks,Roads
If features are destined for a Feature Type and that Feature Type is not defined in the Schema Sources, the following type of error message will appear near the bottom of the log file:
Additional Resources
Data Attribution
The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.