Search

How to Create and Manage Esri Geodatabase (ArcSDE) Connections in FME

Matt Meeboer
Matt Meeboer
  • Updated

FME Version

Introduction

Esri Geodatabase (ArcSDE) database connections are unique in FME in that all connection information is stored in a separate file, the SDE (.sde extension) file. While most database connections in FME have connection parameters, such as the server host and port, an Esri Geodatabase connection includes the path to the dependent SDE file with this information. The SDE file is created using Esri software, like ArcGIS Pro. Please see Esri’s documentation for creating SDE files. 

This article will discuss considerations for managing the underlying SDE file so that FME can access it. It will also demonstrate how to create an Esri Geodatabase connection in FME. Windows will be used as the operating system, as it is required when working with the Esri Geodatabase format in FME. 

 

Requirements

 

SDE File Storage Considerations

Before creating an Esri Geodatabase connection in FME, consideration should be given to where you want to store the SDE file. There are two primary options.

 

Option 1: Store Copies of the SDE File for FME Form and FME Flow

Store a local copy of the SDE file on your computer to use with FME Form. Upload a copy of the SDE file to an FME Flow Resource folder. This option has a higher overhead for management, but is more reliable.

 

Pros

  • FME Form doesn’t need to transverse a network to use the SDE file, which improves connection reliability.
  • For an FME Flow Express Installation, the FME Flow system share is hosted on the same machine as the engines and core, so FME Flow also doesn’t need to transverse a network to use the SDE file.
  • When uploading the SDE file to FME Flow, you can browse to and choose the Resource location using the UI, which lets you confirm the path is valid.

Cons 

  • All FME Form users must have a copy of the SDE file.
  • If you are using the Esri Geodatabase connection in FME Flow, you must remember to upload the SDE file either during or after publishing from FME Form. Note that FME Flow Resource files are not accessible from within FME Form Geodatabase Connections. 
  • If the SDE file changes often, you may need to frequently update your local copy and the copy used by FME Flow.
  • Potential for different results between Form and Flow if the SDE files are not well managed and kept in sync or if a different SDE is inadvertently chosen on FME Flow.

 

Option 2: Network Share (UNC Path)

Store the SDE file in a shared network location that is accessible to all FME Form users as well as FME Flow. For example, //<myfileserver>/myshare/sdeconnectionfiles.

 

Pros

  • Only one copy of the SDE file to manage.
  • The SDE file can be updated on the network share without needing to redistribute it to all users and reupload it to FME Flow. This is especially helpful if the file changes often.
  • When publishing to FME Flow, you do not need to remember to upload the SDE file and modify the connection on FME Flow. 
  • Reduced risk of different results between FME Form and FME Flow, since both use the same SDE file.

Cons 

  • FME Form and FME Flow both need access to the shared network location.
  • If the SDE file becomes invalid, or is moved or renamed, it breaks all Esri Geodatabase connections using the file for all FME Form and FME Flow users.
  • If read permissions are inadvertently removed from Active Directory users or groups that need access to the SDE file, those users’ Esri Geodatabase connections will stop working. If permissions are removed from the FME Flow service account, then this can cause a production outage.
  • Need to consider that other applications, like ArcGIS Pro, may lock the SDE file in the network location and prevent FME from accessing it.
  • Need to consider who has write, modify, and full control permissions to the network locations. This should ideally be a small list of users and groups to avoid issues.
  • The enterprise ecosystem may interfere with SDE file access. Examples include security software, complex networking rules, proxies, and simultaneous user limits on network shares. These can all lead to timeouts or total loss of access and be difficult to troubleshoot.
  • Often requires IT department assistance to set up and troubleshoot.

 

The best option will depend upon your organization and environment. You may also choose to combine the options above. For example, keeping SDE files on a network share for FME Form users, but uploading a copy to a Resource folder on FME Flow to limit chances of production issues.

 

Step-by-Step Instructions

Before completing these steps, consider where you want to store your SDE file. See SDE File Storage Considerations. Make sure that a valid SDE file has been placed in an accessible location and that you have met the licensing requirements. See Requirements.

These steps will use Option 1 from the SDE File Storage Considerations as an example. This involves storing the SDE file in an FME Flow resource folder. 

 

Part 1: FME Form

Create and test the Esri Geodatabase Connection in FME Form.

1. Add the Database Connection

In FME Form, go to Tools > FME Options and click Database Connections. Click + to add a new connection.

 

2. Enter the Connection Parameters

On ‘Add Database Connection’, choose ‘Esri ArcSDE Geodatabase’ and enter:

  • Name: a unique connection name
  • Connection File: click the ellipsis (...) to browse to and select the SDE file.

For Override Credentials and Version options, please refer to the Database Connection documentation.

 

3. Test the Connection

Click the ‘Test…’ button to confirm the connection works. 

 

If you receive an error, see the Troubleshooting section.

Click Save to save the connection.

You can now add an Esri Geodatase reader or writer to a new workspace in FME Workbench and use your connection. 

 

Part 2: FME Flow

Publish the connection to FME Flow, upload the SDE file, and modify the connection.

1. Publish to FME Flow

In FME Workbench, connect to the FME Flow instance that you want to publish to. Click the Publish button.

 

Follow the steps to publish to FME Flow. On the Upload Connections step, make sure that the Enterprise Geodatabase connection is selected.

 

2. Upload the SDE File

The SDE file does not get uploaded to FME Flow with the Esri Geodatabase connection. The file can be uploaded before, during, or after the publish process. In this example, the assumption is made that the SDE file does not yet exist on FME Flow. 

Log into the FME Flow web user interface and go to Resources. Click on the Engine folder to open it. SDE files can be stored in any Resource folder, but since the FME Flow engines need to access them, the Engine folder is a logical place. If you do not have access, the Data folder can be used. It’s recommended to discuss the resource folder location with your FME Flow Administrator.

 

Click New to create a subfolder to store your SDE Files. In this example, ‘SDEFiles’.

 

Open the new folder and click Upload > Files to select and upload the SDE file to FME Flow.

 

3. Update the Esri Geodatabase Connection

Go to Connections & Parameters > Database Connections. Click on your Esri Geodatabase connection to open it.

 

Notice that the Connection File is still referencing the old location of the SDE file that was set in FME Form when the connection was created. Since this is a location on the C:\ drive, running a workspace with this connection would likely fail. Click the X to clear the Connection File.

Click Browse Resources to browse to and select the SDE file that was uploaded to FME Flow. 

The Esri Geodatabase connection is now referencing the SDE file on FME Flow. Click OK

 

Run a test workspace on FME Flow that uses the Esri Geodatabase connection to confirm it is working as expected.

 

Troubleshooting

Below is a general checklist for troubleshooting. For more detailed troubleshooting, including specific errors, please see the detailed articles found under Additional Resources.

  • Is the database server hosting the Esri Enterprise Geodatabase running? 
  • Can the SDE file be used outside of FME Flow to connect to the Geodatabase? For example, in ArcGIS Pro?
  • Is the path to the SDE file in your Esri Geodatabase connection correct in FME Form and FME Flow? 
  • Can the path to the SDE file be accessed in Windows Explorer from the machine where FME Form or FME Flow is installed? 
  • Is the SDE file still valid? Check with the creator and ask for a new SDE file to test.
  • FME Flow: if a different user is trying to run your workspace, have you shared the Database Connection with them? Do they have access to the Resource folder where the SDE file is located?
  • FME Flow: if the SDE file is hosted on a network share, is FME Flow running under a Windows service account and does it have access to the network location? See Running the FME Flow System Services Under Different Accounts (Windows)
  • Are you receiving Esri licensing errors? See Using FME Flow with Esri ArcGIS Software

 

Additional Resources

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.