Adding Fields to an Existing Feature Class with FME and ArcPy

As of FME 2022.0, Python 2.7 has been deprecated and is no longer available within FME. Please see the Python 2.7 Deprecation article. This article has not yet been updated to use Python 3+, to continue with this article, please use FME 2020 or older.

Introduction

A scenario that one may encounter when working with file geodatabase is that one wishes to write data to an existing feature class while altering its schema without having to drop the existing feature class table. This would allow a user to maintain the field names of their existing feature class while adding new fields either manually or from that of a source dataset. While we do recommend that any schema alterations to existing feature classes be handled by your ArcGIS client, the following example will show how to use both FME and ArcGIS Python modules in FME to add fields to an existing geodatabase feature class.

Requirements

The Python interpreter MUST be set to the interpreter installed by ArcGIS. See: No Module Named ArcPy: Importing Esri's ArcPy for Use with FME

ArcGIS installed on the same machine as FME

Source

Garbage Schedule (Source - Esri shapefile)

In the above image, we see a visualization of the source Esri shapefile containing the new field "Truck" of which we want to add to an existing feature class.

Garbage Schedule (Destination - Esri geodatabase feature class)

In the above image, we see a visualization of the destination feature class. As one can see, the data is essentially identical except for that the shapefile has an additional attribute field.

Step-by-step Instructions

1. Read Source Data and Expose Attributes

Garbage Schedule geometry is read from the shapefile source file with the Esri shapefile reader. This shapefile contains an attribute field that does not appear on the corresponding feature class to which we will write the data. Once the shapefile reader has been added to the workspace, open the reader parameters dialogue and click on the format parameters tab. Expose the following attributes (i.e. check the box):

fme_feature_type

fme_geometry

SHAPE_GEOMETRY

fme_type

* Note - The attributes required to be exposed will vary from dataset to dataset, format to format.

2. Rename/Remove Attributes

The AttributeManager will be used used to rename and remove various attributes from the source shapefile dataset. Connect the shapefile reader to the AttributeManager and in the AttributeManager properties, change the Output Attribute "NumAddress" to "NumAddresses". In addition, change the Action for fme_geometry, fme_type, fme_feature_type and SHAPE_GEOMETRY (i.e. the attributes exposed in the previous step) to "Remove".

* Note - The exposed format attributes are being removed so that they do not get appended to our destination feature class via our upcoming Python code.

3. Add Field to Destination Feature Class

In order for the new field contained on the source shapefile to get included onto our existing destination feature class, it must be added programmatically. This will allow schema changes to take place without having to drop the existing feature class table. The PythonCaller is used to make these alterations by calling functionality of both FME (fmeobjects) and ArcGIS (arcpy).

First, python modules for both FME and ArcGIS are imported wherein the destination geodatabase path is then read from the corresponding FME Macro. We then hard code the name of our destination feature class to the geodatabase path. The arcpy function "List Fields" is then used to get individual strings of the field names currently contained in the destination feature class. An empty list variable is then created and the strings from our List Fields call are appended to the empty list.

Secondly, the fmeobjects feature.getAllAttributeNames() function is used to get a list of the fields on the source data set. We then compare the destination attribute list to the values contained in the source attributes, removing instances from the source attribute list where the source attribute already exist on the destination dataset.

Finally, we loop through the remaining values in the source attribute list and add fields to the destination feature class based on the data type of the attribute in question.

The following contains the entire code definition that will appear in the PythonCaller:

import fme
import fmeobjects
import arcpy


# Template Function interface:
# When using this function, make sure its name is set as the value of
# the 'Class or Function to Process Features' transformer parameter

def processFeature(feature):
    # Get the location of the Destination GDB from the FME Macro value.
    Dest_GDB = fme.macroValues['DestDataset_GEODATABASE_FILE']
    
        
    # Call in the Destination feature class from the Destination Geodatabase and     # get a list of field names from the destination feature class.
    
    dest_featureclass = Dest_GDB + '\\GarbageSchedule'
      
    dest_attr = arcpy.ListFields(dest_featureclass)
    
    # Create empty list and append field names from destination feature class.
    dest_list = []
    for a in dest_attr:
        dest_list.append(a.name)
    
    # Get attribute names from the source file.
    source_attr = feature.getAllAttributeNames()
    
    
    
    # Compare the values in dest_list to the values in the source_attr list and
    # remove duplicates
    for val in dest_list:
        if val in source_attr:
            source_attr.remove(val)
    
    
    # For the remaining attributes (i.e. source does not exist in destination), 
    # add the corresponding
    # field with the proper data type. The user can add more if statements for
    # data types as their data requires.
    for value in source_attr:
        if type(value) == str:
            arcpy.AddField_management(dest_featureclass, value, "TEXT")
        if type(value) == int:
            arcpy.AddField_management(dest_featureclass, value, "LONG")

Be sure to change the Class or Function to Process Features to "processFeature" so that the proper function will execute.

4. Write Data

Add the Esri Geodatabase (File Geodb) writer to the workspace. For the writer dataset, navigate to the geodatabase provided (please see add_fields.zip in the download section). For the Feature Class or Table Definition, select "Import from Dataset". The Import from Dataset option is selected because the feature class already exists in the geodatabase and we will write data and alter its schema.

When prompted, set the format and select the file geodatabase as noted in the paragraph above (it may – should – already be set by FME). FME will now scan the geodatabase to confirm what tables exist. When prompted with a list of classes, select GarbageSchedule.

Once the writer has been added to the canvas, double click on the writer feature type to enter the Feature Type Properties dialog. Click the Format Parameters tab and set Truncate table to "Yes".

Save and Run the workspace and inspect the data from the shapefile, but also now has the "Truck" field added while maintaining the original attribute field names from the feature class.

Completed Workspace

Results in ArcMap

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.