Dynamic Workflows: Definitions

Introduction

Let's first look at the difference between a static and a dynamic workflow. This is a static workflow:



A "traditional" static workspace like this is bound to a static schema.

When you press Run:

• The reader schema is what the workspace accepts as input.
• The writer schema is what the workspace creates as output.

By default, the two schemas are the same, but they don't have to be. The writer schema can be manually edited to create different output.

Definition: Schema is another term for "Data Model". It's the structure of the data. In a static workflow, the reader schema defines the data you have, and the writer schema the data you want to get.

Dynamic Workflow Definitions

This is a dynamic workflow:

A dynamic workflow like this is NOT bound to a specific schema.

At the moment that you run the workspace:

• The workspace can accept any input, regardless of schema.
• The workspace accepts a new output schema from one of many different sources.
• The writer schema source may be:
  • Any incoming dataset
  • Any external dataset
  • A lookup table
  • Defined within the workspace

In other words, the schemas do not need to be defined beforehand. They are fetched dynamically at run time.

Step-by-step Instructions

The aim of this exercise is to set up a simple dynamic translation and demonstrate in what way it is dynamic.

1. Generate Workspace
Start FME Workbench and choose File > Generate Workspace from the menubar. An alternative is Ctrl+G.

In the dialog that opens, set the Format to MapInfo TAB (MITAB) then choose the Parks dataset for the Reader Dataset. You can find it at C:\FMEData2022\Data\Parks\Parks.tab.

Set the Writer Format to OGC GML (Geography Markup Language) then browse to a folder to save the Dataset. Name the dataset Output.gml.  



Most importantly, be sure to set the "Workflow Option" to Dynamic Schema. Click OK and the workspace is created.

2. Run the Workspace
Enable Feature Caching by clicking the drop-down arrow next to the Run button. Once feature caching is enabled, click the Run button. 



Inspect the output by clicking on the green magnifying glass on the writer feature type. This will display the output in the Visual Preview window. Notice that it contains the parks data and is identical in structure to the original Parks dataset; it's just now in GML format.

Question) The content earlier in this article said that a writer schema may be defined by either:

• Any incoming dataset
• Any external dataset
• A lookup table
• Defined within the workspace

Where do you think the schema is coming from in this workspace?

3. Re-Run the Workspace
With Prompt for User Parameters enabled, run the workspace again. If you have Feature Caching enabled, you'll need to choose Re-Run, not Run. This time you are prompted to pick the source dataset. Pick the Zoning dataset. You can find this at C:\FMEData2022\Data\Zoning\Zones.tab



Click Run. Again the workspace runs and the output is created. This time the output is the zoning data, identical in structure to the original Zoning dataset.

Answer) This shows how the writer schema is defined:

• Any incoming dataset
• Any external dataset
• A lookup table
• Defined within the workspace

This is a dynamic workspace that fetches the output schema from the incoming dataset. Every time we change the source dataset (it still has to be in the MapInfo format) then at run time the writer copies its schema for the output.

4. Reproject Data (Optional)
If you have time, add a Reprojector transformer into the workspace and connect it between the reader and writer feature types. Set it up to reproject the data from UTM83-10 to LL84, you can leave the Source Coordinate System to <Read from Feature>.



This is an example of why you might want to use a dynamic workspace. Now we have a workspace that converts any MapInfo TAB dataset to GML, including a coordinate reprojection.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.