Using Arcpy for FME Feature Processing

Introduction

Although FME offers close to 500 different transformers, in some cases a user may wish to apply a specific ArcGIS geoprocessing tool to their data. Using ArcPy within FME's PythonCaller transformer a workspace author may incorporate such a tool directly into their FME workflow, effectively extending FME's geoprocessing capabilities.

There are two types of ArcGIS geoprocessing tools we can use:

• Tools that manipulate individual features
• Tools that manipulate entire feature classes

We must use different approaches with these types. 

In our first example, we will measure the geodesic area of individual features, using the Arcpy geometry.getArea() method. In our second example, we will dissolve the polygons in a feature class, using the Arcpy Dissolve tool.

Note: Functionality mentioned in this article requires a licensed version of ArcGIS be installed on the same computer as FME.

Individual FeatureProcessing

In this example we are going to measure the geodesic area of individual features with the Arcpy geometry.getArea() method, using the following process:

• Extract the feature geometry as OGC Well Known Binary using a GeometryExtractor transformer
• Extract the feature coordinate system and convert to Esri WKT, using a CoordinateSystemExtractor and CoordinateSystemDescriptionConverter
• In a PythonCaller:
  • create a new Arcpy geometry from the WKB
  • set its coordinate system to the Esri WKT
  • measure the geodesic area in hectares
  • create new attribute to store the geodesic area

Process Transformers:


PythonCaller code:

import fme
import fmeobjects
import arcpy

def calcGeodesicArea(feature):
    
    # get geometry and coordinate system from feature attributes
    esriWKB = feature.getAttribute('_geometry')
    esriWKT = feature.getAttribute('_esri_wkt')
    
    # create arcpy geometry from WKB annd set coordinate system to WKT
    geom = arcpy.FromWKB(esriWKB).projectAs(esriWKT)
    
    # calculate geodesic area
    area = geom.getArea("GEODESIC","HECTARES")
    
    # output the geodesic area to feature attribute
    feature.setAttribute('_geodesic_area', area)

PythonCaller Parameters:

The new '_geodesic_area' attribute created by the Python code needs to be exposed in the PythonCaller's Attributes to Expose section.
 

Feature Class Processing

In this example, we are going to use the ArcGIS Dissolve tool on a set of input polygons, using the following process:

• Use the FeatureWriter transformer to write data to a feature class in a temporary Geodatabase
• Run the ArcPy operation on that feature class using the PythonCaller transformer
• Read the results back into the FME workspace with the FeatureReader transformer

The source dataset is a set of zoning polygons that we want to dissolve on the basis of a common Zoning Category.

Input

Output

To do this, we will use the Arcpy Dissolve tool (documentation).
 

Workspace

The completed workspace wraps the entire process into an FME custom transformer:

This creates a generic solution where, for example, it is easier for the user to choose attributes to group features to be dissolved.

Inside the custom transformer, a number of steps must be carried out to prepare the data for geoprocessing.

The first step is to change the feature type (layer/table name) to something consistent. In this case we use an AttributeCreator to tag each feature with a feature type value called ‘pre-dissolve’:

This will be the name of the table created in the temporary geodatabase.

The next step is to create a temporary file location for this data. A special transformer called the TempPathnameCreator is used to create the temporary location. However, because we only wish to generate a single location, we first separate a single feature from the rest with a Sampler and send that one feature to the TempPathnameCreator.

A VariableSetter and VariableRetriever are then used to copy the temporary location to all the other features.

NB: The temporary folder created by the TempPathnameCreator will be automatically cleaned up when the workspace finishes, removing all the files you have written there. Since we are going to put an entire Geodatabase folder in there, this is very useful.

The final step required to create the temporary Geodatabase is to create a schema. To do this, the sampled feature is passed through an FME Hub transformer called the SchemaSetter.

After creating the schema, all features are sent to a FeatureWriter, to write them to the temporary Geodatabase:

Notice that the temporary pathname is used to define the location to write to, and fme_feature_type is used to define the table name. Also notice that the Dynamic Schema Definition setting is applied, with the source schema (‘Schema from Schema Feature’) coming from the schema created by the SchemaSetter transformer.

NB: Using a dynamic schema is another way in which we produce a generic solution that can be used on any source data.

Before starting the ArcGIS geoprocessing, two more transformers are required. After creating the Geodatabase, the FeatureWriter outputs a summary feature containing the output dataset location as an attribute. Because (on a Windows computer) this has a combination of forward and backward slashes, a StringReplacer transformer is used to clean up the path.

Finally, a ParameterFetcher transformer is used to retrieve the list of attributes to dissolve by, extracting it from the User Parameter 'Group By'.

Python Code

A PythonCaller transformer carries out the Arcpy processing. The code is as follows:

import fme
import fmeobjects
import arcpy
 
def processFeature(feature):
    # Get dissolve FC and settings from feature attributes
    dataset = feature.getAttribute('_dataset')
    dissolveFields = feature.getAttribute('_Group_Attrs').split(",")
    arcpy.env.workspace = dataset
 
    # Set local variables
    inFeatures = "pre_dissolve"
    outFeatureClass = dataset + "/dissolved"
 
    # Execute Dissolve using group by attributes as Dissolve Fields
    arcpy.Dissolve_management(inFeatures, outFeatureClass, dissolveFields, "", "MULTI_PART", "DISSOLVE_LINES")

This code fetches information about the dataset from FME attributes (the feature.getAttribute function) and then executes the Arcpy dissolve tool using that information.

Once the Python script's work is complete, and the dissolve done, to get the features back into the FME workspace requires a FeatureReader transformer. The summary feature that triggered the Python script is used as input, providing the source Geodatabase location through an attribute.

We specify the dissolved feature class in the Feature Types to Read so we don’t also read the original feature class as well. After cleaning up the temporary attributes, the dissolved features are output from the transformer.

This same process can be used for any Arcpy feature processing that does an entire feature class at a time.
 

Conclusion

If ArcGIS is available on the same computer as FME, its geoprocessing tools can be used to extend FME's functionality through the use of the PythonCaller, FeatureWriter and FeatureReader.