Attribute Processing Example Workspace (CSV to MapInfo)

Introduction

This tutorial demonstrates one approach to manipulating and managing attributes in FME Form. Starting from a CSV of cell-signal records, you will cleanse and classify attributes, create geometry from latitude/longitude, style features by quality, and write the results to an OGC GeoPackage.

Step-by-step Instructions

1. Read in the DataPoints CSV file

As of FME 2025.2, the Coordinate System parameter is now configured within the Parameters dialog of each reader/writer format. For more information, including details about the change and affected transformers, please see Coordinate System Parameter Location Change.

• In a new workspace, click the Add Reader icon in the Toolbar and add a reader with the following parameters:

    • Format: CSV (Comma Separated Value)

    • Dataset: /CellSignal-Small.csv

  
  

  Click the Parameters button to set the coordinate system:

    • Attribute Definition: Manual

  
  

    • Name: longitude

    • Type: x_coordinate

  
  

    • Name: latitude

    • Type: y_coordinate

  
  

    • Create Point Geometry from Attributes: Enabled

    • Coordinate System: LL84

  

  
  

  Click OK to add the reader to the workspace.

• In a new workspace, click the Add Reader icon in the Toolbar and add a reader with the following parameters:

    • Format: CSV (Comma Separated Value)

    • Dataset: /CellSignal-Small.csv

    • Coordinate System: LL84

  
  

  Click the Parameters button and confirm that the longitude and latitude attributes are set to x_coordinate and y_coordinate, respectively. If they are not, change the Attribute Definition to Manual, then manually set.

  

  
  

  Click OK to add the reader to the workspace.

2. Map Null/Missing/Empty Values

Add a NullAttributeMapper.

• Map: All Attributes

• If Attribute Value Is: Empty, Missing (Selected Attributes Only), Null

  • Click the ellipsis and enable Select All 

• Map To: Missing

This normalizes blank-like values so they’re easy to filter later.

3. Remove Records with Missing Key Fields

Add a Tester and connect it after the NullAttributeMapper. Create tests for the following attributes: Latitude, Longitude, Power, Quality, and num_measures.

• For each attribute: 

  • Left Value: <attribute name>

  • Operator: Attribute Is Missing

  • Logic: OR

4. Remove Duplicate Coordinates

We have a large number of records, and sometimes duplicates occur; fortunately, we can utilize the DuplicateFilter to easily remove duplicate latitude/longitude pairs. Add a DuplicateFilter to the Failed port of the Tester.

• Key Attributes: Latitude, Longitude

5. Reformat Date/Time

Next, we need to reformat the date field record_tstamp because it contains hours and minutes and we are only interested in the date. Add a DateTimeConverter and connect it to the Unique output port on the DuplicateFilter. Assume input like 4/12/2011 1:04.

• Datetime Attributes: record_tstamp

• Input Format: %e/%N/%Y %k:%M

• Output Format: %m/%d/%Y

This keeps only the date (e.g., 04/12/2011).

6. Extract Code Prefix (Letters) with RegEx

The ‘Code’ attribute coming from the CSV file has a specific pattern. The first three characters are letters and the remaining characters are numbers (ABC3098330997). A simple regular expression can be used to extract them and create our codeTag and codeValue attributes, respectively.

Add a StringSearcher to isolate the first three letters from Code.

• Search In: Code

• Regular Expression: [A-Z][A-Z][A-Z]

• Matched Result: codeTag

7. Extract Code Remainder (Numbers)

Now that we have created codeTag using the StringSearching and Regular Expression, we can use the same Regular Expression to extract the remainder of Code. Using the StringReplacer transformer, we will replace the alphabetic characters with nothing to isolate a codeValue attribute. 

Add a StringReplacer to remove the leading letters from Code.

• Attributes: Code

• Mode: Replace Regular Expression

• Text to Replace: [A-Z][A-Z][A-Z]

• Replacement Text: (leave blank)

This leaves only the numeric portion in Code.

8. Rename Code to CodeValue

We changed the values of Code using the StringSearcher, but the StringSearcher doesn’t allow for attribute renaming, so we will need to use the AttributeRenamer transformer. 

Add an AttributeRenamer.

• Input Attribute: Code

• Output Attribute: codeValue

9. Keep Only Specific Station Codes

We are only interested in the Stations with a CodeTag of ABC or ABD, so we will need to filter them out. To do this, we can use an AttributeFilter transformer, which routes values to specific output ports. This process can also be achieved with other transformers like the Tester. Add an AttributeFilter to route by station code.

• Attribute to Filter By: codeTag

• Possible Attribute Values: 

  • ABC

  • ABD

10. Categorize by Power Ranges

Now we would like to organize and filter our data by Power level at each station, to do this we will use an AttributeRangeFilter transformer. With the AttributeRangeFilter, we can create groups based on ranges using the Power attribute and then process the groups individually or as a group. 

Add an AttributeRangeFilter and connect both AttributeFilter ABC and ABD output ports to it.

In the parameters, set:

• Source Attribute: Power

Then click the Generate button. In the Generate Range Values dialog, enter:

• Minimum: -100

• Maximum: -70

• Number of Ranges: 3

Then click OK. 

Finally, rename the Output Ports to Very High, High, and Medium, respectively. 

11. Categorize by Quality

Next, we need to test for the quality of the signal. Add a Tester and connect it to the Very High output port on the AttributeRangeFilter. In the parameters, create the following tests:

• Test Clause: Quality > -20 AND Quality <= 0 

Then click OK. 

Next, add another Tester to the canvas and connect it to the High output port. In the parameters, create the following tests:

• Test Clause: Quality > -40 AND Quality <= -20

Finally, add one more Tester to the canvas and connect it to the Medium output port. In the parameters, create the following tests:

• Test Clause: Quality > -60 AND Quality <= -40 

11. Create a QualityOfSignal Attribute

Now that we have filtered out which Power levels are required and then tested for the quality of the signal, we need to identify these in an attribute. Add an AttributeCreator after each Tester Passed output port. 

AttributeCreator (Connected to the Tester_2)

• Output Attribute: QualityOfSignal
• Value: Best 

AttributeCreator_2 (Connected to the Tester_3)

• Output Attribute: QualityOfSignal
• Value: Resonable 

AttributeCreator_3 (Connected to the Tester_4)

• Output Attribute: QualityOfSignal
• Value: Worst 

12. Create Geometry from Attributes

Now that the data is cleaned up, we can visualize each cell signal on a map using the Latitude and Longitude attributes. Add a VertexCreator and connect it to all three AttributeCreators.

• Mode: Add Point

• X Value: longitude

• Y Value: latitude

This generates a point feature for each record.

13. Color Points by Quality

Now that the points are created, we want to visualize the QualityOfSignal attribute by using different colors. Add an AttributeValueMapper transformer.

• Input Attribute: QualityOfSignal

• Output Attribute: fme_color

• Value Map:

This assigns a color value to each point based on signal quality.

14. Write to OGC GeoPackage

Add an OGC GeoPackage writer to export the results.

• Format: OGC GeoPackage

• Attribute Definition: Automatic

Connect it to the Output port of the AttributeValueMapper, then run the workspace.
Inspect the output to confirm that points are color-coded by QualityOfSignal.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.