Using a Python Startup/Shutdown Script or PythonCaller to Perform Geoprocessing with Arcpy

As of FME 2022.0, Python 2.7 has been deprecated and is no longer available within FME. Please see the Python 2.7 Deprecation article. This article has not yet been updated to use Python 3+, to continue with this article, please use FME 2020 or older.

Introduction

Adding a python startup script to your workspace is a great way to extend the functionality of workbench. Although we encourage you to make use of FME’s transformers wherever possible, sometimes it is necessary to take advantage of the greater control that a custom script provides. If you don’t have a lot of prior programming experience python is a great language to start with as its simple, clean syntax makes it easy to use and understand. Python has also been chosen as the default scripting language for ArcGIS; If you have a valid ArcGIS license on your machine, you can make use of the many geoprocessing functions in the arcpy module in your python scripts, you can also use ModelBuilder to create python scripts from your models. For more information on using Python with FME, please see Tutorial: Python and FME Basics.

Requirements

FME Desktop

FME Background Geoprocessing (if using 64-bit FME)

Licensed Esri ArcGIS

Demos

Demo 1: Using a Python Startup Script

Background

In this example, we will be creating a python startup script to do a little bit of geodatabase cleanup before our workspace runs. The dataset we are using is a file geodatabase that contains an existing polygon feature class linked via a relationship class to an annotation feature class. The issue arises when we try to overwrite the polygon class; the truncate table setting on the feature type parameters will only truncate the table of polygon features, the corresponding annotation features will not be deleted as desired. In order for our workspace to run correctly, we must ensure that both tables are properly emptied before our workspace executes so that new features can be written and properly linked. To do this, we will call the Delete Rows method in the arcpy python module from a python startup script to drop all features from both tables before the workspace is run.

Creating a python script

First, we need to create a python script. If you are unfamiliar with writing python scripts using arcpy, an easy way to get started is to use ArcGIS ModelBuilder to export a model to a python script. In this example, we’ll create a simple model that runs the Delete Rows tool from the Data Management Toolbox on two tables in our geodatabase. We’ll add this tool to a blank model and set its single parameter to the name of the table we wish to clear. We do this for both our polygon feature class and our annotation feature class. We can then export our model to a python script by choosing Export>To Python Script from the Model menu and specifying a file location as shown below:

We can then examine the contents of our script in a text editor such as notepad. The contents should look something like this:

# Import arcpy module
import arcpy
# Local variables:
Parcels = "C:\\Users\\lsanderson\\Documents\\Safe Projects\\Esri\\ArcPy\\Data\\CityData.gdb\\PropertyData\\Parcels"
Parcels__2_ = Parcels
Parcel_Ids = "C:\\Users\\lsanderson\\Documents\\Safe Projects\\Esri\\ArcPy\\Data\\CityData.gdb\\PropertyData\\Parcel_Ids"
Parcel_Ids__2_ = Parcel_Ids

# Process: Delete Rows
arcpy.DeleteRows_management(Parcels)

# Process: Delete Rows (2)
arcpy.DeleteRows_management(Parcel_Ids)

This script imports the arcpy module, assigns the feature class locations we specified in ModelBuilder to variables, and calls the Delete Rows geoprocessing tool.

Adding a startup script in FME Workbench

Workbench allows us to add a python script to be run at startup before the main workspace executes, or at shutdown, after your workspace has finished running (successfully or not), by adding a script as a workspace parameter. To set this parameter, we locate the workspace parameters entry in the Navigator window and then expand the advanced parameters as shown below:

Double-click on 'Startup Python Script'. This opens up a simple text editor, where we can paste in the contents of our python script. Select and copy the contents of the exported python script from notepad or the text editor of your choosing and paste into the Startup Python Script editor. Then click OK. Now your python script should be all set to run before your workspace.

Demo 2: Using a Python Shutdown Script

In the attached Python_Shutdown_Script.fmw workspace, a Python shutdown script has been set up. This script creates spatial indexes on all Shapefiles written in the workspace. This is an older script to demonstrate this topic. This functionality can not be accomplished by using various transformers.

PythonCaller

You can also use the PythonCaller transformer to call your python and arcpy scripts. In the examples described here, you could use the PythonCaller in conjunction with the FeatureReader instead of the Startup Python Script. Similarly, a FeatureWriter + PythonCaller could replace the Shutdown script. For an example of this, please see the article Using Arcpy for FME Feature Processing.

Additional Resources

Tutorial: Python and FME Basics

Using Arcpy for FME Feature Processing

No Module Named ArcPy: Importing Esri's ArcPy for Use with FME