Dynamic Workflows: Definitions

Introduction

Let's look at the difference between a static and a dynamic workflow. This is a static workflow:

A "traditional" static workspace like this is bound to a static schema.

When you press Run:

• The reader schema is what the workspace accepts as input.
• The writer schema is what the workspace creates as output.

By default, the two schemas are the same, but they don't have to be. The writer schema can be manually edited to create different outputs.

Definition: Schema is another term for "Data Model". It's the data structure. In a static workflow, the reader schema defines the data you have, and the writer schema defines the data you want to get.

This is a dynamic workflow:

A dynamic workflow like this is NOT bound to a specific schema.

At the moment that you run the workspace:

• The workspace can accept any input, regardless of schema.
• The workspace accepts a new output schema from one of many different sources.
• The writer schema source may be:
  • Any incoming dataset
  • Any external dataset
  • A lookup table
  • Defined within the workspace

In other words, the schemas do not need to be defined beforehand. They are fetched dynamically at run time.

Step-by-step Instructions

The aim of this exercise is to set up a simple dynamic translation and demonstrate how it is dynamic.

Please download the ZIP file from the article's Files section to follow along with the exercises.

1. Generate Workspace

Start FME Workbench and choose File > Generate Workspace.

In the dialog that opens, set the following:

• Reader:
  • Format: MapInfo TAB (MITAB)
  • Dataset: <path>\Parks.tab
    • Click on the ellipsis to navigate to the location of the file on your computer
• Writer:
  • Format: OGC GML (Geography Markup Language)
  • Dataset: <path>\output.gml
    • Click on the ellipsis to navigate to the location of the file on your computer
• Workflow Options:
  • Dynamic Schema: enabled

Click OK to generate the workspace.

2. Run the Workspace

Enable Data Caching by clicking the drop-down arrow next to the Run button. Once feature caching is enabled, click the Run button. 

Inspect the output by clicking on the green magnifying glass on the writer feature type. This will display the output in the Visual Preview window. Notice that it contains the parks data and is identical in structure to the original Parks dataset; it's just now in GML format.

Question) The content earlier in this article said that a writer schema may be defined by either:

• Any incoming dataset
• Any external dataset
• A lookup table
• Defined within the workspace

Where do you think the schema is coming from in this workspace?

3. Re-run the Workspace

With Prompt for User Parameters enabled, rerun the workspace. If you have Feature Caching enabled, you'll need to choose Re-Run, not Run. This time, you are prompted to pick the source dataset. Pick the Zoning dataset. You can find this at C:\FMEData2022\Data\Zoning\Zones.tab

Click Run. Again, the workspace runs, and the output is created. This time, the output is the zoning data, identical in structure to the original Zoning dataset.

Answer) This shows how the writer schema is defined:

• Any incoming dataset
• Any external dataset
• A lookup table
• Defined within the workspace

This is a dynamic workspace that fetches the output schema from the incoming dataset. Every time we change the source dataset (which still has to be in MapInfo format), the writer copies its schema to the output.

4. Reproject Data (Optional)

If you have time, add a Reprojector transformer into the workspace and connect it between the reader and writer feature types. Set it up to reproject the data from UTM83-10 to LL84; leave the Source Coordinate System set to <Read from Feature>.

This is an example of why you might want to use a dynamic workspace. Now we have a workspace that converts any MapInfo TAB dataset to GML, including a coordinate reprojection.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.