Improving Performance when working with Esri Geodatabases

Question

How can I improve my performance when working with Esri Geodatabases?

Answer

Choosing a Format

Esri Geodatabase (File Geodatabase Open API) Reader/Writer
Try using the Esri Geodatabase (File Geodatabase Open API) reader/writer if you're only reading or writing simple features to a File Geodatabase. It avoids the overhead of the ArcGIS Geodb API.
  

Improving Writer Performance

Increase Transaction Interval

You can speed up translations involving all writers by increasing the interval between committing transactions. Committing transactions is an expensive operation, and therefore it is recommended that you make the transaction interval as big as possible. In speed tests performed at Safe Software, changing the transaction interval from 500 to 1000 resulted in a specific translation being 2.5% faster. Changing the transaction interval to 5000 resulted in the same translation running 5.5% faster. Turning transactions OFF resulted in an improvement of either 12% or 19%. The performance advantages of changing the transaction interval or turning transactions off will differ between various datasets.

Set Transaction Type

For Esri Geodatabase, if possible, use the Transaction Type TRANSACTIONS or NONE instead of EDIT_SESSION (Personal, File) or VERSIONING (ArcSDE). EDIT_SESSION or VERSIONING has a significant impact on performance. Your Geodatabase only requires this mode if it is versioned or if you have certain custom behaviors or triggers (i.e., loading network topology, having relationship tables, etc.).

Plan Ahead

You can also increase the speed of writing features by creating all the feature datasets, feature classes, and tables ahead of time so that the Geodatabase Writer only needs to open them, rather than create them. Try using an Esri XML Workspace Document as the Geodatabase template to create the entire Geodatabase before FME starts to write. More information on using a Geodatabase Template or Esri XML Workspace Document can be found here.
 
Make sure the update key field has an attribute index when you are updating.
 

Set Simplify Geometry and Validate Features

Setting Simplify Geometry to NO will speed up geometry writing. Setting Validate Features to Write to NO will speed up writing features whose attributes are set to domain or subtype values. The values can be validated after the load using the Esri tools.

Avoid Annotations

Avoid or minimize the number of annotation features, because writing to these is slow. It has been improved when writing annotations with no user-defined fields.

Correct Precision and Resolution

Only use as much precision and spatial resolution as you need. If all you need is 32-bit precision, set this on the writer feature type parameters instead of 64-bit. If all you need is decimeter-level accuracy, don't use such a large scale factor that you are giving yourself 0.00001 mm accuracy. In both cases, you will be carrying around and processing more data than you need to. Reducing your effective precision and resolution should speed things up, especially for large datasets. Be forewarned that reducing your precision/resolution can cause problems with out-of-bounds errors or polygons collapsing into points if you don't give your data sufficient precision.

Exploit FME Caching

Make sure your database writer is the first writer in the FME Workspace navigation tree (right-click on the writer and use Move Up). FME caches the data for the second and subsequent writers, so having the database writer first ensures features are passed straight through to the writer.

Avoid Updating Spatial Indices

For the Geodatabase writers, set the grid 1 size to 0. This means that a spatial index is not built during the load but will automatically be built after the load is completed.

Incremental Updates

If you are updating the database using Feature Operation: fme_db_operation or Feature Operation: Update | Delete, you need to set the Match Columns parameter to create a key for the updates and deletes. Make sure that the Match Column attribute is indexed in your Geodatabase. This can improve update performance 4 to 5 times.

Improving Reader Performance

Check your constraints

In the reader parameters, if you don't require domain or subtype information then turn off the option for this information

• Uncheck Resolve Domains
• Uncheck Resolve Subtypes

Search Envelope

Set a search envelope to reduce the number of features read by the reader and passed into the workspace. The search envelope can also be used to clip incoming features to further reduce the volume of data. This action is also performed at the database level, so it is further optimized for performance.

Ignore Unneeded Complexity

Check that the settings on the Geodatabase reader, Ignore Network Info and Ignore Relationship Info, are set to YES if you don't require Network or Relationship information to be read from the Geodatabase. When this setting is set to YES, reading feature classes with no network information has even been found to improve performance.