Extracting Data within a Clipping Polygon using FeatureReader

Introduction

With FME Flow and FME Form, we can use an area of interest polygon (e.g., a city boundary) to clip a set of user-specified layers that fall within it. This is different from the more traditional approach of reading the entire set of layers and then clipping the features.

To perform this task, the FeatureReader transformer will be used. This transformer enables us to use any format and allows spatial filters to be applied to the user-specified layer.

As well, the FeatureReader allows you to write data dynamically into destination feature types for any table that it reads. This occurs via a <Schema> port, which provides the writer with the schema definition to use. This method is incredibly flexible, as different data layers can be read into the workflow and then written out.

The source data and complete workspace can be downloaded from the Files section.

Step-by-Step Instructions

1. Read the Boundary Dataset

The zoning data will be used to spatially filter the desired data. All features within this boundary will be incorporated into the workflow.

Add a new MapInfo Tab reader to the canvas:

• Format: MapInfo TAB (MITAB)
• Dataset: /SourceData/Zoning/Zones.tab
  • Browse to downloaded source data

2. Add a User Parameter for Zone Type

This option provides flexibility for the end user, allowing them to select different boundaries based on the type of zone they choose (e.g., industrial vs. residential). 

In the Navigator, right-click on the User Parameters and select ‘Manage User Parameters’. 

In the User Parameter dialog, click the green plus (+) icon and select ‘Choice’ to add a choice user parameter. For the new parameter, enter the following:

• Parameter Identifier: Type
• Label: Type

We will import the zone types from the MapInfo file so the user can easily select a type when running the workspace. Under 'Choice Configuration', click 'Import > From Dataset...'

In the Import Wizard dialog, set the format and dataset:

• Format: MapInfo TAB (MITAB)
• Dataset: /SourceData/Zoning/Zones.tab

Click Continue. Specify the import mode and ensure Zones is selected as the feature type:

• Import from: Attribute Values

Click Continue. Now, we will choose the column to import data from.

• Value: ZoneCategory
• Display: ZoneCategory

Click Import. The categories will populate the table. The User Parameter dialog should look like this:

Optional:

Using a Web Mapping Tool: If your users are drawing the area of interest in some kind of web mapping tool, you could send the vertices of this polygon to a new user parameter, then create a polygon from these vertices using a combination of transformers.

Using a Where Clause: If your users select an area of interest by adding a reader, you can publish a Where Clause for that reader (if applicable for the format) to provide users with flexibility. In our example, MapInfo does not have this functionality, so we must create a parameter.

Both methods can be combined in a single workspace, allowing users to choose either one.

3. Dissolve Zones

Each zone category in the zones dataset contains multiple polygons. We want to use a single zone as a boundary, so we need a way to aggregate these features into an single feature. To do this, we will use the Dissolver, Aggregator, and Tester transformers.

First, add a Dissolver transformer to the canvas and connect it to the Zones reader feature type. Double-click the transformer to open the parameters dialog and set the following:

• Group Processing: Enabled
  • Group By: ZoneCategory
    • Click the ellipsis to select the attribute

4. Aggregate Zones

Add an Aggregator transformer and connect the Dissolver Area output port to the Aggregator Input port. Set the Aggregator parameters:

• Group Processing: Enabled
  • Group By: ZoneCategory

5. Test Zones with User Parameter

We will now use a Tester transformer to find the zones with the category selected in the user parameter.

Connect the Aggregator Aggregated output port to a Tester Input port and set the following parameters:

• Left Value: ZoneCategory
  • Click the drop-down arrow to find the list of Attribute Values
• Operator: =
• Right Value: $(Type)
  • Click the drop-down arrow to find the list of User Parameters

The workspace should now look like this:

6. Read Data to be Filtered

At this stage, we will read the data layers that we are interested in through PostGIS. This is the data that we want to be filtered by the boundary polygon. 

Add a FeatureReader to the Canvas and connect its Initiator input port to the Tester Passed output port. Then set the following parameters:

• Format: PostGIS
• Connection: Embed Connection Parameters
• Parameters:
  • Host: postgis.train.safe.com
  • Port: 5432
  • Database: fmedata
  • Username: fmedata
  • Password: fmedata
• Feature Types to Read: public.Parks
  • Click the ellipsis to browse feature types
• Spatial Filter: Initiator OGC-Contains Results
• Output Ports: Single Output Port - <Generic>
• Attribute and Geometry Handling:
  • <Generic> Port:
    • Attributes to Expose: fme_feature_type

Run the workspace and inspect the output. You should notice that only the park data that exists inside your selected boundary polygon (zone type) remains. At this stage, only 'public.Parks' would be read into the workspace, so we can improve the flexibility by giving the users a choice of layers.

7. Create a New User Parameter for Layer Choice

We will create a second user parameter to allow users to select the specific layers (i.e., feature types) they want to use. 

In the FeatureReader parameters, select the dropdown arrow next to ‘Feature Types to Read’ and choose ‘User Parameter > Create User Parameter…’ 

In the Add/Edit User Parameter dialog, set the parameter identifier and label:

• Parameter Identifier: FEATURETYPE
• Label: Choose a feature type

Under 'Choice Configuration', choose ‘List’ and enter the following Values:

These are layers that exist in the database and will be used if selected by the user. Click OK to add the user parameter.

Optional:

You can also create the parameter as a text field. A text field allows options to be entered that didn't exist when the workspace was authored; they need to be manually input or passed in via another program, such as a web application that uses FME Flow. You will send the user's choices to this parameter as a space-delimited list. For example: “AddressPoints Parcels Neighborhoods”.

8. Write the Data in Dynamic Mode

Since the workspace is flexible in what it can read, the writer must be set up in dynamic mode to ensure that the data can be written out. The Generic format allows users to choose from various formats.

Type ‘Generic writer’ on the canvas and press 'Enter' on the keyboard to add a Generic writer. Enter the following:

• Format: Generic (Any Format)
• Dataset: /Output
  • Browse to desired output location
• Parameters:
  • Output Format: Autodesk Autocad DWG/DXF
    • When the workspace is run, you can change this to any format you’d like when prompted

The workspace will work as is for file and folder-based formats. For other formats, such as cloud-based and databases, it is necessary to set additional writer parameters to specify and connect to the desired destination.

Click OK. In the Feature Type dialog, set:

• Dynamic Schema Definition: Enabled
  • Schema Sources: “Schema From Schema Feature”
  • Schema Definition Name: fme_feature_type

To write data in FME, the writer feature type must know the schema it will use. In this case, we will use the data from the FeatureReader as our schema. 

Connect the FeatureReader <Schema> and <Generic> output ports to the <Dynamic> Generic writer:

To use fixed schema mode (not shown in this example), you must add a writer and destination feature types for any table that the user may read. Use the FeatureTypeFilter transformer to route features to their corresponding destination feature type.

9. Prompt and Run to Test with Different Choices of Tables.

Before running the workspace, ensure ‘Prompt for Parameters’ is enabled in the Run menu:

Click the green Run button to run the workspace. Test different zone types (‘Type’ parameter) and tables (‘Choose a feature type’ parameter) to generate different sets of outputs. 

The FeatureReader transformer is triggered by the Initiator feature. In our case, this refers to the set of bounding polygons associated with the Zones data. If we have 10 bounding polygons, then the FeatureReader is executed 10 times. Performance is best if there are fewer Initiator features.

When the FeatureReader is initiated, it connects to the output format and reads the feature types specified in the FeatureType parameter. The transformer allows for spatial filtering, so using the "Contains" clause means that the features returned are contained within the Initiator polygons. Then, the writer dynamically takes all the features and writes them to their respective layers based on the incoming layer name and schema. Any features that fell outside of our bounding box are not included.

This is the output with the zone type parameter set to One Family Dwelling and the feature type parameter set to public.AddressPoints:

Additional Resources

Dissolver Transformer Documentation

Aggregator Transformer Documentation

Tester Transformer Documentation

FeatureReader Transformer Documentation

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.