Files
Introduction
A dynamic workflow that mirrors the source and destination schema is simple to set up and offers maximum flexibility. The user can input any dataset in the chosen source format, and the workflow will create an output that mirrors the input dataset’s schema. For example, a workflow can be created so that any table from a geodatabase can be read and converted to a CityGML file while maintaining its schema.
The important part is that a user can change the source dataset, and the translation will still work. The user will not need to make changes to the workspace. This approach is perfect for when a dataset has many feature types with different schemas, or when new feature types will be added to the workflow in the future.
In the following example, an OGC Geopackage dataset (CityDatabase.gpkg) contains multiple feature types, each with a different schema. Each feature type will be reprojected and written to a shapefile dynamically using its own schema.
Workspace and Data
A finished version of the workspace created in this example can be found in the zip file attached to this tutorial, titled "1-MirrorImage".
CityDatabase.gpkg contains several tables (feature types), which can be seen in the Select Tables dialog:
Step-by-Step Instructions
1. Open FME and Select the Generate Workspace Tool
Open FME Workbench and select the Generate Workspace tool from the start page. We will configure this tool over the next three steps.
Configure the reader as follows:
- Format: OGC Geopackage
- Dataset: <Path>\resources\CityDatabase.gpkg
-
Parameters
- Tables: BusStops, CityParks, Roads
Configure the writer as follows:
- Format: Esri Shapefile
- Dataset: <Path>\output
2. Set the Workspace to be Dynamic
In the dialog, there are two Workflow Options: Static Schema and Dynamic Schema. If the Static Schema option is selected, each feature type would appear separately on the Workbench canvas, like this:
However, if the Dynamic Schema option is chosen, a schema-less workspace with both a dynamic reader and writer is created.
Select 'Dynamic Schema'.
3. Click OK to Generate the Workspace
Click OK to generate the workspace. Both the dynamic reader and writer will have a single feature type displayed on the canvas, regardless of the number of reader feature types selected.
The reader feature type is named ‘<All>’ while the writer feature type is named ‘<Dynamic>’. As well, the dynamic feature type does not contain any attributes.
Behind the scenes, the reader feature type has been configured to read all features regardless of the feature type name by setting the Merge Filter to an asterisk (*). This can be seen by opening the reader feature type parameters.
The writer feature type has been configured to write all features with the schema defined by the source dataset.
Note: Individual readers and writers can be made dynamic without using the Generate Workspace dialog. This is done by setting the options above in each feature type’s parameters dialog.
4. Add a Reprojector Transformer
Click anywhere on the canvas, then type 'Reprojector'. Place this transformer between the reader and writer feature types.
Double-click the Reprojector to open the parameters, and set the Destination Coordinate System to LL84.
5. Run the Workspace
When the workspace is run, all CityDatabase source data is read through a single feature type.
On the writer side, the incoming “fme_feature_type” attribute contains the feature type name and is used to determine which schema to use. For example, a feature with “fme_feature_type” set to “Roads” will search the CityDatabase for the “Roads” dataset and, when found, use that schema. The data will then be dynamically divided back into its component layers, using the appropriate schema to organize the original attributes and geometry type.
Advanced Techniques
Using the Feature Types to Read Parameter
When a dynamic workflow is created, an additional User Parameter called “Feature Types to Read” is automatically published in the workspace.
This parameter creates a quick way to change the number of feature types to be read. When the workspace is run using “Prompt for User Parameters”, the user can select which feature types to read in the dialog that appears.
This is especially useful if the workspace references dozens of source feature types (for example, a database) and not all of them are required during a dynamic translation run.
Adding Additional Attributes to a Dynamic Writer
Since the destination feature type is a mirror of the source dataset’s schema, attributes created within the workspace are not automatically added to the output. Any new attributes must be added in the User Attributes tab of the writer feature type parameters. For example, the "_timestamp" attribute created by the DateTimeStamper transformer has been manually added to the writer feature type here:
And it appears on the writer feature type:
SHAPEFILE Writer requires attribute names to start with a letter, which includes both Latin and non-Latin alphabets (such as Japanese, Cyrillic, or Arabic scripts). If an attribute name begins with a non-letter character (such as numbers or special characters), the Writer will automatically add a 'Q_' prefix to ensure compatibility.
Generic Workspace
Adding a Generic Reader and/or a Generic Writer to this workflow could produce a single workspace capable of reading and writing any data format, regardless of its schema.
Additional Resources
Data Attribution
The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.