Document Library Migration & Metadata Updates with Microsoft SharePoint

Evie Lapalme
Evie Lapalme
  • Updated

FME Version

  • FME 2022.0


With the SharePoint Online Connector from the FME Hub, you can automate updates, migrate your documents, and automate administrative tasks on your Microsoft SharePoint instance. 

What is Microsoft SharePoint?

SharePoint is used as a secure place to store, organize, share, and access information across an organization in a web-based service. A key feature of SharePoint is lists that can be linked to other documents and edited simultaneously across the organization. There are many uses for SharePoint, but it is commonly used as an intranet site for documents shared internally for an organization. 




Step-By-Step Instructions

Example 1: Uploading to SharePoint and Email Notifications

In this hypothetical situation, we have some metadata (XML) for an organization’s electronic document management system on a local or network drive.  This organization wants to migrate their document management system to SharePoint using FME.  Using a metadata file will allow us to mirror the current directory structure in the Document Library in SharePoint.

1. Open FME Workbench
Open FME Workbench and click New to create a new workspace.

2. Read in Metadata
This is an example of how you might upload multiple files at once from your directory using a metadata file with path and file information. For this exercise, we have not provided the actual documents that are being migrated. Instead, we will illustrate the steps required to build the required workflow for you to have a reference for your own scenario, environment, and SharePoint Instance.

In this example we’ve initiated the workspace with a reader to read in metadata containing information about the document management system:


The path portion of the metadata is the most important, make sure to have it point to the correct files in your directory.


3. Add the SharePointOnlineConnector
Add a SharePointOnlineConnector to the canvas and connect it to the feature type used to read in your metadata file.
Open the parameters of the SharePointOnlineConnector by clicking on the cog wheel. Fill out all of the fields that appear in red with your own SharePoint Online Web Connection and file system. Set the Request Action to “Upload” and the File to Upload as the “path” attribute you created in your metadata file. Set your upload options by selecting a site, document library, and path from your SharePoint instance. Here, we have chosen to set the “If File Exists” option to Overwrite when importing new files. 


Set the Attributes to Add to _full_path, _id, _name, and _url. 


This is what the workspace would look like so far:


4. Setting up Email Notifications
Add two StatisticsCalculators to the canvas. Connect one StatisticsCalculators to the output port and the other to the Rejected port of the SharePointOnlineConnector. We will calculate the total count of each of these.


Add an Aggregator to the canvas to merge the two StatisticsCalculators and connect the output port to a new AttributeCreator. The AttributeCreator will have two values: success and failure. Have these values equal and fme_rejection_code.total_count respectively. 


Add an Emailer to the canvas and connect it to the output port of the AttributeCreator. Fill out your own credentials where the inputs are red. In the subject name it: “Document Migration Summary”.


In the body of the email paste this snippet:

<h2>eDoc to SharePoint Migration Summary</h2> 
<p>@DateTimeFormat(@DateTimeNow(),%B %e), @DateTimeFormat(@DateTimeNow(),%Y) at @DateTimeFormat(@DateTimeNow(), %l:%M%p)</p>

 <p><strong>Uploaded Documents: </strong>@Value(success)</p>
 <p><strong>Rejected Documents: </strong>@Value(failure)</p>

5. Run your workspace
You should both have migrated your files that mirrors your current directory structure, and have gotten an email that summarizes the number of documents that were successfully uploaded to SharePoint and how many failed to upload similar to this:



Example 2: Updating SharePoint List Records

Suppose we have created a custom attribute or column in our SharePoint Document Library list called “Custodian”  that contains the name of the owner of each document.  Let’s see how we can use FME to populate this value.

1. Open FME Workbench
Open FME Workbench and click New to create a new workspace.
Add a Creator to the canvas.

2. Add a SharePointOnlineConnector
The SharePointOnlineConnector will be used to read in the Document Library list, so that we can obtain and use the List Item ID downstream in our workflow. 
Add a SharePointOnlineConnector to the canvas and connect it to the Creator.
Open the parameters by clicking on the cog wheel, and fill out all of the fields that appear in red with your own SharePoint Online Web Connection and file system. Select “List” from the Request Action. 


Make sure to add the attributes: _full_path, _id, _list_id, _list_item_id, _name, _url


3. Attribute Creation
Add the AttributeCreator to the canvas and create a new attribute called “Custodian” with the value of your choosing. The value that we set is “Ernie”. 

4. Writing the output
Add a Microsoft SharePoint List writer to the canvas and configure it with your SharePoint connection and the file system parameters. Make sure to set the List Name to the _list_id attribute and the Item ID Attribute to the  _list_item_id attribute. 


Set the User Attributes to Manual and configure the attribute Custodian as a choice.


The workspace should look something like this:


Final output in SharePoint: 



Additional Resources 


Data Attribution

Data used in this tutorial originates from data created by Safe Software

Was this article helpful?



Please sign in to leave a comment.