Fanout | How to Separate Features Based on an Attribute

Introduction

A fanout is a way within FME to split output data based upon the value of an attribute. The data is divided at the time of writing, rather than within the workspace itself (so like an on-the-fly AttributeFilter). There are two different kinds of fanout: feature type fanout and dataset fanout. This article will demonstrate both kinds of fanout using a CSV file containing information about community trees.

Feature Type Fanout

A feature type fanout writes a single dataset but divides the data up into a number of layers/themes/feature types/objects/classes in that dataset.

A feature type fanout. Data is divided up into a number of layers on the basis of an elevation attribute.

Example:

1) Open starting workspace

Open the FanoutTrees-Start.fmwt template file. This template reads in the CommunityTrees.csv file and then extracts the Year from the DATE_PLANTED attribute using the SubstringExtractor transformer. It then sorts the data in Numeric Descending order using the Sorter transformer. The Sorter is used so that when viewing the data in Microsoft Excel, the most recent year will be the first sheet and the oldest year will be the last sheet.

Starting workspace, this workspace will be used in all three examples.

2) Write out to Microsoft Excel

Add a Microsoft Excel writer and name it TreesByYear.xlsx, and then set the Sheet Definition to Automatic. Just close the Feature Type dialog, we will come back to that in a minute. Once the writer has been added, connect it to the Sorter.

3) Create feature type fanout

Once the writer feature type is connected to the Sorter, open up the parameters. For Sheet Name, select the YEAR_PLANTED attribute. Creating the fanout this way will create a different sheet for each of the years contained within a single Microsoft Excel file.

Feature type fanout set in the writer feature type parameters

4) Run the workspace and view the results

Save and run the workspace. You can view the results in either the FME Data Inspector or Microsoft Excel. The output will be a single Microsoft Excel file with a sheet for each year.

A single Microsoft Excel file is created with a different sheet per year

Dataset Fanout

A dataset fanout divides the data up and writes a different dataset for each division.

A dataset fanout. Data is divided up into a number of datasets on the basis of an elevation attribute.

Example:

1) Open starting workspace (or continue working in the same workspace)

The starting workspace template for Dataset Fanout is the same as the one for Feature Type fanout, so you can open a fresh workspace or continue in the same one.

2) Write out to Microsoft Excel

Add a (another) Microsoft Excel writer and name it Output.xlsx, and then set the Sheet Definition to Automatic. In the Feature Type dialog, name the Sheet to Trees. Once the writer has been added, connect it to the Sorter.

3) Set dataset fanout in the Navigator Pane

In the Navigator Pane, expand the Output [XLXSW] writer then click on Fanout Expression to open the parameters.

Set dataset fanout in the Navigator Pane

Click the checkbox next to Fanout Expression to enable dataset fanout, then for the Fanout Expression paste in the expression below:

TreesByNeighbourhood\@Value(NEIGHBOURHOOD_NAME)_Trees.xlsx

This expression will make a folder named TreesByNeighbourhood and then a Microsoft Excel file will be created for each of the neighborhoods using NEIGHBOURHOOD_NAME. Then so we know what is in this file, _Trees is added at the end. It is important to include the file extension when creating a dataset fanout.

Fanout expression for dataset fanout

4) Run the workspace and view the results

Save and run the workspace. You can view the results in either the FME Data Inspector or a file manager. The output will be a different Microsoft Excel file for each neighbourhood each with a single sheet called Trees.

Each neighbourhood is a single file with a single sheet

Combination Fanouts

There's no problem with setting both a feature type fanout AND a dataset fanout in the same workspace. The dataset fanout will produce a number of output datasets, each of which has a number of layers created by the feature type fanout.

Example:

1) Open starting workspace (or continue working in the same workspace)

The starting workspace template for Dataset Fanout is the same as the one for Feature Type fanout, so you can open a fresh workspace or continue in the same one.

2) Write out to Microsoft Excel

Add a (another) Microsoft Excel writer and name it Combination.xlsx, and then set the Sheet Definition to Automatic. Just close the Feature Type dialog, we will come back to that in a minute. Once the writer has been added, connect it to the Sorter.

3) Create feature type fanout

Once the writer feature type is connected to the Sorter, open up the parameters. For Sheet Name, select the YEAR_PLANTED attribute. Creating the fanout this way will create a different sheet for each of the years contained within a single Microsoft Excel file.

4) Set dataset fanout in the Navigator Pane

In the Navigator Pane, expand the Combination [XLXSW] writer then click on Fanout Expression to open the parameters. Click the checkbox next to Fanout Expression to enable dataset fanout, then for the Fanout Expression paste in the expression below:

TreesByNeighbourhoodByYear\@Value(NEIGHBOURHOOD_NAME)_TreesByYear.xlsx

5) Run the workspace and view the results

Save and run the workspace. You can view the results in either the FME Data Inspector or a file manager. There will be a separate file for each of the neighbourhoods and then within each of the files, there will be separate sheets for each year.

Combination dataset and feature type fanouts.

Raster Data and Fanouts

Each new Raster format writer generally includes a Feature Type Fanout automatically.

This is so that each outgoing feature (file) is written with the same filename as was read on input; the fanout attribute being fme_basename

One drawback is that the FME Data Inspector’s open dataset dialog becomes confused when opening the FME Data Inspector from the writer. You will have to manually select the raster files when trying to inspect or by using the asterisk (*) wildcard, ie. *.jpg if all the files are contained in the same folder.

Multi-Attribute Fanouts

Each fanout setting only allows a user to select a single attribute to fan out by. However, multi-attribute fanouts can be created by merging attributes together using the StringConcatenator transformer.

File vs Folder Datasets

The behavior of fanouts changes based on whether you are working with a file-based format or a directory-based format. For instance, an AutoCAD DWG writer (file-based) will create new layers with a feature type fanout, and new drawings with a dataset fanout. An Esri Shapefile writer (directory-based) will create new shapefiles on a feature-type fanout, and new directories on a dataset fanout.

This table explains the different combinations you can get:

Batch Processing

When combined with input aggregation features you can often get away with doing all of your translations in a single workspace, in a sort of pseudo-batch process.

"Batch" processing with fanouts. For each dataset that enters the workspace, a similarly named dataset is output.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.