Reading and Writing SharePoint Libraries with FME

As of FME 2024.0, support for Esri ArcGIS Online (AGOL), ArcGIS Enterprise Portal, and ArcGIS Server Feature Service formats has transitioned to a unified approach using the new Esri ArcGIS Feature Service (Format). This format replaces the legacy reader/writer formats for each of the three services.

Starting in FME 2026.1, the legacy formats will be hidden in the Quick Add menu. Instead, equivalent functionality is now provided by the downloadable Esri ArcGIS Connector package, available on FME Hub. The package also includes new web services and key transformers such as the ArcGISOnlineConnector, ArcGISAttachmentConnector, and ArcGISBranchVersionManager. The package can be installed directly through the Quick Add menu in FME Workbench. Once installed, the new Esri Feature Service format will appear in the Gallery as a unified method for interacting with all three types of Esri ArcGIS Feature Services.

For details on this transition, including guidance on updating existing workspaces, please refer to the article: Working with Esri ArcGIS Feature Services in FME.

Introduction

The SharePointOnlineConnector enables users to seamlessly access their SharePoint Libraries within FME. This empowers users to efficiently perform actions like listing, downloading, uploading, and deleting contents stored in their SharePoint Site. In this article, we will explore three scenarios where the SharePointOnlineConnector can be used to optimize your workflow.

If you are unsure whether you should be using the SharePoint Reader/Writer or the SharePoint Online Connector, please refer to Getting Started with Microsoft SharePoint.

If you are having trouble configuring the safe.microsoft-sharepoint.Microsoft SharePoint Online Web Service please refer to one of these articles depending on your App Registration settings. 

• How to Create a SharePoint Web Connection for FME Form [Single Tenant]
• How to Create a SharePoint Web Connection for FME Form [Multitenant]

Step-by-step Instructions

Part 1: Download and Migrate Data from SharePoint

By setting the Action parameter to Download in the SharePointOnlineConnector, you can download contents from your SharePoint Library as an Attribute, File, or Folder. This simplifies data migration, supports various data types, and directs data to a chosen source.

In this scenario, we will combine the SharePointOnlineConnector and the FeatureReader to migrate .shp files stored in SharePoint Libraries to ArcGIS Online.

1. Add the SharePointOnlineConnector

Add the SharePointOnlineConnector and attach a Creator to initiate the workflow. From your configured Web Service, choose the Web Connection you've set up, granting access to your SharePoint Site.

Set the Action to "Download". Navigate to your intended Site, Document Library, and file path by clicking the "..." next to Parameters.

Configure the destination for the download as "File" and designate your preferred location.

Execute the SharePointOnlineConnector. You should observe that the file has been successfully downloaded to your designated location.

2. Add the FeatureReader

Connect the Output from the SharePointOnlineConnector to the FeatureReader.

Choose the Format as Esri Shapefile. Set the Dataset parameter to "_download_path"; this will utilize the download destination specified in your SharePointOnlineConnector.

 

3. Write to ArcGIS Online

Add the ArcGIS Online Feature Writer. Connect the Output port from the FeatureReader to the Input port of the Writer. Configure the Writer to write or create your desired Feature Service. Learn more about using the ArcGIS Online Feature Writer here.

If you are using 2025.1+, please download the Esri ArcGIS Connector from FME Hub.

4. Run your Workspace

Run your workspace. Your SharePoint files have now been successfully migrated to ArcGIS Online. The same principle applies to migrations to other destinations, such as Snowflake, ArcGIS Portal, and various databases supported by FME.

Part 2: Clean up your SharePoint Documents

By setting the Action parameter to Delete in the SharePointOnlineConnector you have the option to clean up documents stored in your SharePoint Site.

1. Add SharePointOnlineConnector (List Action)

Add the SharePointOnlineConnector and attach a Creator to initiate the workflow. From your configured Web Service, choose the Web Connection you've set up, granting access to your SharePoint Site.

Set the Action to "List". Navigate to your intended Site and Document Library.

Run the SharePointOnlineConnector. You should observe all files in the selected Document Library have been Listed.

 

2. Filter the File Path Using the Tester

Add a Tester Transformer. Connect the output of the SharePointOnlineConnector to the Tester. In this example, we will delete all files with the ".csv" extension. Run the Tester; you should notice that all files with the ".csv" extension are produced as output.

• Left Value: _name
• Operator: Ends With
• Right Value: .csv

3. Add another SharePointOnlineConnector (Delete Action)

Add another SharePointOnlineConnector. Choose the same operational Web Connection. Select the action "Delete." Navigate to the identical Site and Document Library.

For the Path parameter, select the attribute "_full_path" from the dropdown arrow on the right. This will utilize the complete path of files with the ".csv" extension.
 

4. Run your Workspace

Run your workspace and inspect the contents of your designated SharePoint Document library. All files ending with the ".csv" extension will have been removed.

Part 3: Use your SharePoint Web Connection in a Reader

An alternative to using your SharePoint Web Connection in the SharePointOnlineConnector is directly in a Reader.

Add a Reader of your choice. Click the dropdown arrow to the very right of the Dataset parameter, Select File from Web, and Browse Microsoft SharePoint.

This will prompt you to choose a functional Web Connection, then proceed to your desired Site and Document Library where the file is stored. Click "OK." You will notice that the Dataset parameter now directly reflects your SharePoint Site.

However, this approach has a limitation: it only allows reading the dataset. If your intention involves migration, deletion, or direct modifications on your SharePoint Site, it's advisable to use the SharePointOnlineConnector.