Control Your Data's Attributes with the AttributeManager Transformer

Introduction

The AttributeManager is the powerhouse transformer for attribute manipulation. Almost all of the attribute manipulation can be done with other transformers, but for a cleaner workspace as well the ability to see all of the attribute modifications in a single window. 

Some of the tasks handled by the AttributeManager are:

• Create attributes
• Rename attributes
• Copy attributes
• Remove attributes
• Set attribute values
• Re-order attributes
• Concatenate attributes
• Arithmetic expressions
• Set conditional values
• Change data types (new for 2023.0) 

In this tutorial, we will read a CSV dataset containing emissions data from New York City. Using the AttributeManager, we will remove and change the order of attributes, create a new attribute that utilizes a datetime function, round the values of an attribute, and update the attribute data types.  
 

Step-by-step Instructions 

1. Open a Blank Workspace
In FME Workbench, open a blank workspace. Add a CSV (Comma Separated Value) reader to the canvas and browse to the NYC_Municipal_Building_Energy_Benchmarking_Results.csv dataset that is available in the Files section of this article.
The CSV and a completed workspace are contained within the <FMEYear>ControlingAttributesWithTheAttributeManager.zip folder. If you are using FME 2022 or older, please download the 2022 version of the folder, as the AttributeManager was updated for 2023. 


2. Run Workspace with Feature Caching 
Before we begin, we should run the workspace with Feature Caching enabled and inspect the data in Visual Preview to see what attributes to modify. 

Ensure Feature Caching is enabled by clicking the drop-down arrow next to the run button. After confirming Feature Caching is enabled, click Run. 

View the data in Visual Preview. There are a few attribute modifications that we will be making, as well as some others that we might discover while working through the updates. 

• Move BIN and Building to the beginning
• Remove Campus
• Fix the number of decimal places for 2012 GHG Emissions Intensity (kgCO2e/ft2)
• Add a DateMigrated attribute
• Review the data types and update as needed (FME 2023.0 and newer)

3. Add an AttributeManager - Remove Attribute
Add an AttributeManager transformer to the canvas and connect it to the CSV reader feature type. In the parameters, the first change we will make is to remove the Campus attribute, as it is not needed. Scroll down to the bottom of the AttributeManager, click on Campus to select it, then click on the negative (-) button at the bottom to remove the attribute. 


After removing an attribute, the Action column will update from Do Nothing to Remove. 


To bring back an attribute, click on Remove and select Do Nothing. 


The order in which attributes appear in the AttributeManager also affects how they are utilized within the AttributeManager. If an attribute is removed at the top of the AttributeManager but used later in a calculation, it will not work, as the attribute has already been removed. FME acts on the attributes from top to bottom. So if an attribute needs to be used but also removed, move the attribute to be removed to the bottom of the AttributeManager, so the last action FME does is to remove it. 

4. Change Attribute Order
The next step is to move two of the attributes to the top of the list. On your keyboard, press CTRL or CMD then click on both BIN and Building to select them. Next, click on the up arrow with the line to move to them to the very top of the list. 


5. Import Data Types 
Data Types within the AttributeManager were introduced in FME 2023.0. The following four steps can only be accomplished in FME 2023.0 or newer. For older versions of FME, please skip to step 10.

The Type field in the AttributeManager is auto-populated with the data types from the original dataset, which can be seen in the reader. If the data type is unassigned from the original data, FME will assume a character string with a maximum length of 200. 

FME can review the data in the Feature Cache and select a data type that best fits the data. To do this, click the Import button and select Types From Feature Cache. 


In the Import Types from Feature Cache dialog, click Select All, then click Import. There are additional Attribute Type Handling parameters, please see the documentation for more information. 


The (data) Types that were updated are now shown in black. This can also be looked at as a form of data validation in that we can see that the Source EUI Types are still set to varchar, which indicates that there is something in the data that isn’t numeric. Let’s look at the data in Visual Preview to see if we can determine why. Click OK to save our changes so far and close the AttributeManager. 


6. Inspect Source EUI 
Inspect the CSV reader feature type cache again to view the data in Visual Preview. Reviewing the data, a comma separates the thousands' place in the data, causing it to be classified as varchar. We need to remove the comma as we want the data to be real32. 


7. Remove Comma 
Although this step could be completed in the AttributeManager using the Text Editor and @ReplaceString() function, sometimes it is easier to use another transformer, especially when multiple attributes need to be modified the same way. 
 

StringReplacer Method:

Add a StringReplacer transformer between the CSV reader feature type and the AttributeManager. 


In the StringReplacer parameters, click on the ellipsis (...) next to Attributes and select:

• 2010 Source EUI (kBtu/sqft)
• 2012 Source EUI (kBtu/sqft)

Next, set the Text to Replace to , then click OK. We don’t want to replace it with anything, so the Replacement Text is left blank. 

 

AttributeManager Method:

If you want to complete this step in the AttributeManager, the @ReplaceString() function can be used:

@ReplaceString(<string>,<before>,<after>,caseSensitive=TRUE)

Where <string> is the attribute you are updating, <before> is the string you want to replace, and <after> is what it is being replaced with. The comma needs to be escaped by enclosing it in double quotes. Since we are replacing it with nothing, double quotes are needed as well. 

For the 2010 Source EUI (kBtu/sqft) attribute, in the Value box enter:

@ReplaceString(@Value(2010 Source EUI (kBtu/sqft)),",","",caseSensitive=TRUE)

Then for  2012 Source EUI (kBtu/sqft), enter:

@ReplaceString(@Value(2012 Source EUI (kBtu/sqft)),",","",caseSensitive=TRUE) 

8. Re-Import Types
Run the workspace to the StringReplacer to update the Feature Caches, then open the AttributeManager. With the commas removed from the two attributes, we can re-import the (data) Types to see if the data is now real32. 

Click on the Import button and select Types from Feature Caches. The 2010 Source EUI and 2012 Source EUI should now be real32. 


Note: If you replaced the strings in the AttributeManager instead of the StringReplacer, the Type field for 2010 Source EUI (kBtu/sqft) and 2012 Source EUI (kBtu/sqft) will need to be manually updated to real32. This is because the AttributeManager cannot read its own feature caches, it can only read the previous feature cache. 

9. Manually Update Data Types
Currently, the BIN (data) Type is set to uint32. We need to update this to varchar, as it is an ID for each building. Setting it to a string data type prevents other applications from incorrectly modifying it by adding decimals or treating it as a date. We will also change 2013 Rating to varchar as N/A is often used as a value for this field. To manually update the Type, click on the box for Type to open the drop-down menu, then select the desired data type. For BIN, select varchar(width). The Type will change to varchar(200), we don’t need 200 characters for BIN. Manually update varchar(200) to varchar(9). Next, for 2013 Rating, change it to varchar(9) as well. 


10. Round Attribute Values
When we first inspected the data, we noted that the 2011 GHG Emissions wasn’t rounded to 1 decimal point, which can be fixed in the AttributeManager. Click in the Value box next to 2011 GHG Emissions to show the drop-down arrow. Click on the drop-down arrow, then select Arithmetic Editor. 


In the Arithmetic Editor, on the side, expand Math Functions, then double-click on round to add it to the editor.


The @round() function requires a <number> which in this case will be the value of 2012 GHG Emissions, and <precision> which is the number of decimal places, which we will set to 1. 

To add the 2012 GHG Emissions attribute value, on the side, expand FME Feature Attributes and double-click on 2012 GHG Emissions. As long as <number> was highlighted, the @Value(2012 GHG Emissions…) attribute value will be added to the correct spot. Now highlight all of <precision> and type in 1. Click OK to close the Arithmetic Editor. 

@round(@Value(2012 GHG Emissions Intensity (kgCO2e/ft2)),1)

This step could have been completed with an ExpressionEvaluator transformer. 

11. Create a New Attribute
The final step is to create a new attribute to contain the date that this information was added to the output dataset (database, excel spreadsheet, etc.). At the bottom of the AttributeManager, click in the <Add new attribute> box and type in DateMigrated. 


We will use the date that the workspace is run as the value. In the Value box for the newly created DateMigrated attribute, either click on the ellipsis or click on the drop-down and select Text Editor to open the Text Editor dialog.


On the side, expand Date/Time Functions and double-click on DateTimeFormat to add it. If we only did DateTimeNow, the formatting would be in Local Unzoned and would not be very readable. Add DateTimeNow to the <datetime> parameter of the DateTimeFormat function. Next, for <format> enter in the following:

%Y%m%d%H%M

The output date will be 202306081604 (Year, Month, Day, Hour, Minute). If you would like seconds, add %S at the end. Click OK to close the Text Editor. 

@DateTimeFormat(@DateTimeNow(),%Y%m%d%H%M)

Set the DateMigrated Type to datetime, and then click OK to close the AttributeManager. 

This step could have been completed using the DateTimeStamper and the DateTimeConverter transformers. 

12. Run Workspace
Run the workspace and view the AttributeManager feature cache in Visual Preview. 

• BIN and Building are now the first two attributes
• If you hover over the attribute heading, the data types have been updated
• 2012 GHG Emissions now only has 1 decimal place
• A new attribute called DateMigrated as a value for today's date
• Campus has been removed. 

This data can now be written out to a database or Excel spreadsheet if desired. See Tutorial: Updating Databases with FME or Tutorial: Getting Started with Excel for more information. 

Additional Resources

[Video] FME 2023: What's Your Type? The AttributeManager Transformer Has the Answer!
[Article] Tutorial: Working with Date and Time Attributes
[FME Academy] Create and Modify Attributes Module
 

Data Attribution 

The data used here originates from open data made available by New York City.