Fanout | How to Separate Features Based on an Attribute

Introduction

A fanout is a way within FME to split output data based upon the value of an attribute. The data is divided at the time of writing, rather than within the workspace itself (so like an on-the-fly AttributeFilter). There are two different kinds of fanout: feature type fanout and dataset fanout. This article will demonstrate both kinds of fanout using a CSV file containing information about community trees.

Step-by-step Instructions

Feature Type Fanout

A feature type fanout writes a single dataset but divides the data up into a number of layers/themes/feature types/objects/classes in that dataset.

A feature type fanout. Data is divided up into a number of layers on the basis of an elevation attribute.

1. Open the Starting Workspace

Open the FanoutTrees-Start.fmwt template file. This template reads in the CommunityTrees.csv file and then extracts the Year from the DATE_PLANTED attribute using the SubstringExtractor transformer. It then sorts the data in Numeric Descending order using the Sorter transformer. The Sorter is used so that when viewing the data in Microsoft Excel, the most recent year will be the first sheet and the oldest year will be the last sheet.

2. Write Out to Microsoft Excel

Add a Microsoft Excel writer: 

• Format: Microsoft Excel
• Dataset: TreesByYear.xlsx
• Sheet Definition: Automation

Click OK to close the Feature Type dialog. We will return to it in a minute. Once the writer has been added, connect it to the Sorter.

3. Create Feature Type Fanout

Once the writer feature type is connected to the Sorter, open up the parameters:

• Sheet Name: YEAR_PLANTED attribute
  • Click the drop-down arrow next to Sheet Name, expand Attributes, then select YEAR_PLANTED

Creating the fanout this way will create a different sheet for each of the years contained within a single Microsoft Excel file.

4. Run the Workspace and View the Results

Save and run the workspace. You can view the results in either the FME Data Inspector or Microsoft Excel. The output will be a single Microsoft Excel file with a sheet for each year.

Dataset Fanout

A dataset fanout divides the data and writes a different dataset for each division.

A dataset fanout. Data is divided up into a number of datasets on the basis of an elevation attribute.

1. Open starting workspace (or continue working in the same workspace)

The starting workspace template for Dataset Fanout is the same as the one for Feature Type fanout, so you can open a fresh workspace or continue in the same one.

2. Write Out to Microsoft Excel

Add a (another) Microsoft Excel writer:

• Format: Microsoft Excel
• Dataset: Output.xlsx
• Sheet Definition: Automation

In the Feature Type dialog: 

• Sheet: Trees. 

Once the writer has been added, connect it to the Sorter.

3. Set Dataset Fanout in the Navigator Pane

In the Navigator Pane, expand the Output [XLXSW] writer, then double-click on Fanout Dataset to open the parameters.

• Fanout Dataset: Enabled
• Fanout Expression:

TreesByNeighbourhood\@Value(NEIGHBOURHOOD_NAME)_Trees.xlsx

This expression will create a folder named TreesByNeighbourhood and a Microsoft Excel file for each neighborhood using NEIGHBOURHOOD_NAME. To ensure we know what is in this file, _Trees is added at the end. It is important to include the file extension when creating a dataset fanout.

4. Run the Workspace and View the Results

Save and run the workspace. You can view the results in either the FME Data Inspector or a file manager. The output will be a different Microsoft Excel file for each neighbourhood, each with a single sheet called Trees.

Combination Fanouts

Setting both a feature type fanout AND a dataset fanout in the same workspace is no problem. The dataset fanout will produce several output datasets, each with a number of layers created by the feature type fanout.

To create a Combination Fanout, follow the steps in both Feature Type Fanout to add the Microsoft Writer and Dataset Fanout for the datasets in Navigator. 

Raster Data and Fanouts

Each new Raster format writer generally includes a Feature Type Fanout automatically.

This is so that each outgoing feature (file) is written with the same filename as was read on input; the fanout attribute being fme_basename

One drawback is that the FME Data Inspector’s open dataset dialog becomes confused when opening the FME Data Inspector from the writer. You will have to manually select the raster files when trying to inspect or by using the asterisk (*) wildcard, ie. *.jpg if all the files are contained in the same folder.

Multi-Attribute Fanouts

Each fanout setting only allows a user to select a single attribute to fan out by. However, multi-attribute fanouts can be created by merging attributes together using the StringConcatenator transformer.

File vs Folder Datasets

The behavior of fanouts changes based on whether you are working with a file-based format or a directory-based format. For instance, an AutoCAD DWG writer (file-based) will create new layers with a feature type fanout and new drawings with a dataset fanout. An Esri Shapefile writer (directory-based) will create new shapefiles on a feature-type fanout and new directories on a dataset fanout.

This table explains the different combinations you can get:

Batch Processing

When combined with input aggregation features you can often get away with doing all of your translations in a single workspace, in a sort of pseudo-batch process.

"Batch" processing with fanouts. For each dataset that enters the workspace, a similarly named dataset is output.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.