Reading and Writing SharePoint Libraries with FME

Introduction

The SharePointOnlineConnector enables users to seamlessly access their SharePoint Libraries within FME. This empowers users to efficiently perform actions like listing, downloading, uploading, and deleting contents stored in their SharePoint Site. In this article, we will explore three scenarios where the SharePointOnlineConnector can be used to optimize your workflow.

If you are unsure whether you should be using the SharePoint Reader/Writer or the SharePoint Online Connector, please refer to Getting Started with Microsoft SharePoint.

If you are having trouble configuring the safe.microsoft-sharepoint.Microsoft SharePoint Online Web Service please refer to one of these articles depending on your App Registration settings. 

• How to Create a SharePoint Web Connection for FME Form [Single Tenant]
• How to Create a SharePoint Web Connection for FME Form [Multitenant]

Step-by-step Instructions

Part 1: Download and Migrate Data from SharePoint

By setting the Action parameter to Download in the SharePointOnlineConnector you are able to download contents from your SharePoint Library as an Attribute, File, or Folder. This simplifies data migration, supporting various data types, and directing it to a chosen source.

In this scenario we will use a combination of the SharePointOnlineConnector and the FeatureReader to migrate .shp files stored in SharePoint Libraries to ArcGIS Online.
 

1. Add the SharePointOnlineConnector

Add the SharePointOnlineConnector and attach a Creator to initiate the workflow. Choose the Web Connection you've set up from your configured Web Service, granting access to your SharePoint Site.

Set the Action to "Download". Navigate to your intended Site, Document Library, and file path by clicking the "..." next to Parameters.

Configure the destination for the download as "File" and designate your preferred location.

Execute the SharePointOnlineConnector. You should observe the file has been successfully downloaded to your designated location.


 

2. Add the FeatureReader

Connect the Output from the SharePointOnlineConnector to the FeatureReader.

Choose the Format as Esri Shapefile. Set the Dataset parameter to "_download_path"; this will utilize the download destination specified in your SharePointOnlineConnector.


 

3. Write to ArcGIS Online

Add the ArcGIS Online Feature Writer. Connect the Output port from the FeatureReader to the Input port of the Writer. Configure the Writer to write or create your desired Feature Service. Learn more about using the ArcGIS Online Feature Writer here.


 

4. Run your Workspace

Run your workspace. Your SharePoint files have now been successfully migrated to ArcGIS Online. The same principle applies to migrations to other destinations like Snowflake, ArcGIS Portal, and various databases supported by FME.

Part 2: Clean up your SharePoint Documents

By setting the Action parameter to Delete in the SharePointOnlineConnector you have the option to clean up documents stored in your SharePoint Site.
 

1. Add SharePointOnlineConnector (List Action)

Add the SharePointOnlineConnector and attach a Creator to initiate the workflow. Choose the Web Connection you've set up from your configured Web Service, granting access to your SharePoint Site.

Set the Action to "List". Navigate to your intended Site and Document Library. 

Run the SharePointOnlineConnector. You should observe all files in the selected Document Library have been Listed.


 

2. Filter file path using the Tester

Add a Tester Transformer. Connect the output of the SharePointOnlineConnector to the Tester. In this example, we will delete all files with the ".csv" extension. Run the Tester; you should notice that all files with the ".csv" extension are produced as output.


 

3. Add another SharePointOnlineConnector (Delete Action)

Add another SharePointOnlineConnector. Choose the same operational Web Connection. Select the action "Delete." Navigate to the identical Site and Document Library.

For the Path parameter, select the attribute "_full_path" from the dropdown arrow on the right. This will utilize the complete path of files with the ".csv" extension.
 

4. Run your Workspace

Run your workspace and inspect the contents of your designated SharePoint Document library. All files ending with the ".csv" extension will have been removed.

Part 3: Use your SharePoint Web Connection in a Reader

An alternative to using your SharePoint Web Connection in the SharePointOnlineConnector is directly in a Reader. 

Add a Reader of your choice. Navigate to the dropdown arrow on the very right of the Dataset parameter, Select File from Web, Browse Microsoft SharePoint.


This will prompt you to choose a functional Web Connection, then proceed to your desired Site and Document Library where the file is stored. Click "OK." You will notice that the Dataset parameter now directly reflects your SharePoint Site.





However, this approach has a limitation: it only allows reading the dataset. If your intention involves migration, deletion, or direct modifications on your SharePoint Site, it's advisable to use the SharePointOnlineConnector.