Automate ArcGIS Enterprise Portal Administration with Organization Webhooks and FME Flow

As of FME 2024.0, support for Esri ArcGIS Online (AGOL), ArcGIS Enterprise Portal, and ArcGIS Server Feature Service formats has transitioned to a unified approach using the new Esri ArcGIS Feature Service (Format). This format replaces the legacy reader/writer formats for each of the three services.

Starting in FME 2026.1, the legacy formats will be hidden in the Quick Add menu. Instead, equivalent functionality is now provided by the downloadable Esri ArcGIS Connector package, available on FME Hub. The package also includes new web services and key transformers such as the ArcGISOnlineConnector, ArcGISAttachmentConnector, and ArcGISBranchVersionManager. The package can be installed directly through the Quick Add menu in FME Workbench. Once installed, the new Esri Feature Service format will appear in the Gallery as a unified method for interacting with all three types of Esri ArcGIS Feature Services.

For details on this transition, including guidance on updating existing workspaces, please refer to the article: Working with Esri ArcGIS Feature Services in FME.

Introduction

Esri’s ArcGIS Portal is a component of ArcGIS Enterprise that allows you to host your organization’s maps and data on the web. Anyone in the organization with the right permissions and a web browser can view and manage spatial data, which makes sharing spatial data easy.

ArcGIS Portal is often just one of many applications in an organization’s ecosystem. An administrator often has the responsibility of managing all of their datasets, users, groups, and items. FME offers a solution for automating this process.

FME Flow Automations run automated workflows in response to various events, including incoming messages. Organization Webhooks can be used to monitor item, group, and user activity in the ArcGIS Portal and send messages about that activity. Connecting organization webhooks to automation allows us to build workflows that respond to activity and events occurring in the ArcGIS Portal in near real-time.

For example, when a Feature Layer is uploaded to Portal, we may want to initiate the following workflows:

Send out a notification to stakeholders
Synchronize with another database
Validate and clean up datasets

In the following tutorial, we will set up an FME Flow Automation to receive a message every time an item in ArcGIS Portal is shared externally. However, organization webhooks supports dozens of other different trigger events. If you’re interested in using FME to monitor other ArcGIS Enterprise portal items, groups, and users, you can implement those using a similar approach to this example.

Requirements

An ArcGIS Enterprise Portal Web Connection
An ArcGIS Portal account with the Organization webhooks privilege or assigned the default administrator role .
FME Flow and ArcGIS Enterprise Portal must be able to communicate within the same network OR be configured for public access. For more information, refer to your organization’s IT and the Enabling FME Flow for Public Access article.
FME Flow must have HTTPS enabled (the Portal webhook payload is delivered via HTTPS)
ArcGIS Enterprise Portal must trust the certificate used by your FME Flow

Step-by-Step Instructions

In this scenario, an ArcGIS admin works for an organization that provides datasets to the general public. As part of their responsibilities, the admin must ensure all employees are publishing datasets that adhere to governance and accessibility guidelines. However, as the organization grows, manually reviewing each shared dataset has become too time-consuming.

After learning that FME can be used to receive real-time messages about ArcGIS Portal item activity, the admin is interested in building an automated solution with FME. This workflow will send out notifications the moment each item is shared, so that it can be reviewed and validated.

This tutorial will provide instructions for creating the organization webhook in ArcGIS Portal, building an FME Flow Automation to receive webhook messages, and creating a workspace that processes the webhook messages.

Part 1: Create the ArcGIS Enterprise Portal Organization Webhook

In Part 1, we build an automation in FME Flow that receives webhook messages from ArcGIS Portal. In ArcGIS Portal, we create the organization webhook that will send a message every time a feature layer is shared externally.

1. Configure the Automation in FME Flow

Start by building a new Automation in FME Flow that will receive messages from the ArcGIS Enterprise Portal.

Set the Trigger to Webhook (triggered) and click Apply. Attach a Log a Message Action to the Success port (the checkmark) of the Trigger.

Click the drop-down menu next to the Formatted Message parameter. From the Webhook dropdown, select Message Content. This will let us view the message received from the Portal webhook in our Automation Log.

2. Get the Webhook URL

Save the Automation and give it a name (E.g., “Portal External Sharing”). Once saved, double-click the Webhook trigger to reopen its Parameters. Copy the new Webhook URL to your clipboard. Later in the tutorial, this URL will connect our Automation to the ArcGIS Enterprise Portal webhook.

Start the Automation. If you try to create the webhook without first starting the Automation, you will receive an error in the ArcGIS Enterprise Portal! Leave FME Flow open, as we will come back to it shortly.

3. Create the Webhook

In a new window or tab in your browser, log in to your ArcGIS Enterprise Portal’s REST API Directory as an administrator. The URL will be in the format:

https://webadaptorhost.domain.com/webadaptorname/sharing/rest

For example, if your ArcGIS Portal web adaptor machine name is gis, domain is safeville.gov, and web adaptor name is portal, then the URL would be:

https://gis.safeville.gov/portal/sharing/rest

After logging in, you will see the admin home page. If you do not, click Admin in the top right.

Scroll down on the Organization ID page. From the list of Child Resources, click Webhooks.

Scroll down on the Organization ID page. From the list of Child Resources, click Webhooks.

Click Create Webhook.

All ArcGIS Enterprise Portal webhooks for the organization are created on this page.

Paste the Webhook URL from the clipboard (copied in step 2) into the Payload URL.
Enter a unique name for the webhook.
Under Which events would you like to trigger this webhook?, select “Let me define manually”.
Enter the ArcGIS Portal Trigger Event for which you want to send a message to FME Flow. In this example, we’ll enter ”/items/share” in order to receive a message whenever any item in our organization is shared.

Click Create Webhook. The webhook is now active.

Notes:

If you get the error “Unable to create webhook for the account due to 'null’” when creating the webhook, your login token likely expired. Simply click Login in the top right corner and re-login as an administrator.
If you get an error related to SSL configuration, e.g., ‘Exception: PKIX path building failed: host unable to find valid certification path to requested target’, please ensure that your FME Flow SSL certificate has been imported into your ArcGIS Portal following the instructions from Esri .

In FME Flow, view the Automation Log File from within the automation we just created.

There is a message showing that the ArcGIS Enterprise Portal Webhook has been registered.

4. Test the webhook

Share an item in your ArcGIS Enterprise Portal

Note: To share an editable feature layer with Everyone (public), you must first enable Public Data Collection in the feature layer’s settings

Check the Automation Log File again in FME Flow (you may need to refresh the page).
FME Flow has received a webhook message (in JSON) letting us know which item has been shared and who shared it. The message only contains the item ID, though; in the next part, we will create a workspace that gets the item name and uses it in a notification.

Part 2: Create a Workspace to Process the Webhook Payload

In Part 2, we’ll create a workspace in FME Workbench that can be run by the FME Flow Automation every time the webhook trigger receives a message from ArcGIS Portal. The workspace will get the item details via an ArcGIS REST API call and write them to an FME Flow Automation writer so that they can be included in an email.

1. Create a New Workspace and User Parameters

Open a blank workspace in FME Workbench. Save the workspace as portalwebhook.fmw.

In the Navigator, right-click User Parameters and select Manage User Parameters. Click the Insert button and choose Text.

On the right side of the Parameter Manager, enter the following parameter properties:

Parameter Identifier: webhook_message
Prompt: Webhook Message
Published: Enabled
Required: Disabled
Disable Attribute Assignment: Disabled
Default Value: Paste the webhook JSON payload, from the opening curly brace to the closing one, from the Automation’s log file (see Part 1, Step 4). This will be used to test our workspace.

Click OK.

2. Get the Webhook Response Message

Search for the Creator transformer and add it to the canvas. This will start the workspace in the Automation when the webhook is triggered.

Using the ParameterFetcher transformer, we can create a feature from the webhook message. Attach a ParameterFetcher transformer to the Creator.

Open the ParameterFetcher parameters. For the Parameter Name, open the drop-down menu and select the “webhook_message” user parameter that was created in the prior step.
Name the Target Attribute “_webhook_message”.

Click OK.

Run the workspace with Feature Caching Enabled and inspect the ParameterFetcher output in the Visual Preview window. A new webhook_message attribute has been created with the response text we pasted into the webhook_message’s default value. Now, the FME Flow Automation Webhook Trigger can pass the ArcGIS Portal message into our workspace for processing.

3. Parse the JSON

The webbook’s JSON keys and values need to be parsed into attributes. To view the webhook message as JSON, double-click the _webhook_message value in Visual Preview, click on the Syntax Highlighting button in the bottom left corner, and select JSON.

For this example, the only JSON keys we need are in the “events” array: “id”, “operation”, and “sharedToGroups” (under “properties”).

Add a JSONFragmenter and connect it to the ParameterFetcher.

Configure the JSONFragmenter:

JSON Attribute: webhook_message
JSON Query: json[“events”][*]
Flatten Query Result into Attributes: Yes
Recursively Flatten Objects/Arrays: Yes

In the “Attributes to Expose” parameter, enter the following keys on new lines:

id
operation
properties.sharedToGroups{}
source
username

Select OK to close the transformer.

Run the workspace again to build the feature cache.

Inspect the JSONFragmenter output in Visual Preview. Select the feature and enable the Feature Information window to see the attributes created from the JSON.

4. Check if the Item Has Been Shared with Everyone

In this example, we only want to send a notification if an item has been shared with Everyone. Before letting the workflow continue, let’s check that condition. Connect a ListSearcher to the JSONFragmenter. Search the list attribute properties.sharedToGroup for the string ‘Everyone’.

Run the workspace. A feature is output from the Found port. If a feature is not found, we won’t continue the transformation.

Next, flatten the sharedToGroups list into a single attribute, to make it easier to work with and display to the user.

Connect a ListConcatenator to the ListSearcher Found port. For the List Attribute, choose the properties.sharedToGroups{} list and for the Destination attributes, enter sharedToGroups.

The new attribute can be seen in the Visual Preview when the workspace is run.

5. Create the ArcGIS Enterprise Portal Web Connections

The ArcGIS REST API URL that we will call to get more information about the shared item (provided in the webhook message) requires the item id.

An ArcGIS Portal Web Connection is required to connect to ArcGIS Portal. If you do not already have an ArcGIS Portal Web Connection, please read How to Create an FME Web Connection for ESRI ArcGIS Portal Feature Service for instructions.

Add an HTTPCaller to the canvas and connect it to the ListConcantenator.

Instead of hardcoding the value, the request URL will reference the item id attribute value since it will be different each time. For the Request URL, give it the value of https://yourportaldomain.com/portal/sharing/rest/content/items/@Value(id)

where http://yourportaldomain.com/"yourportaldomain.com”is the domain for your Portal.

Configure the rest of the HTTPCaller as follows:

HTTP Method: GET
Query String Parameters:
- Name: f
- Value: json (this will return the data in JSON format)
Response Body Attribute: _response_body
Use Authentication: Yes
- Authentication Method: Web Connection
- Web Connection: the ArcGIS Enterprise Portal Web Connection you setup for FME Workbench

Run the workspace again and inspect the HTTPCaller output in Visual Preview. Preview the _response_body attribute; it should contain details about the item that was shared, such as the name and description.

6. Parse JSON and Cleanup Attributes

Connect a JSONFlattener to the HTTPCaller output. Set the JSON Document to the _response_body attribute returned from the HTTPCaller and set Recursively Flatten Objects/Arrays to Yes. Expose the following attributes from the JSON: title, description and type.

To keep the output clean, connect an AttributeKeeper to the JSONFlattener and only keep the following attributes:

description
id
operation
sharedToGroups
title
type
username

7. Write to FME Flow Automation and Upload to FME Flow

Add an FME Flow Automation writer to the canvas. The Feature Type Definition can be set to Automatic… .

In the Feature Type dialog, name the Feature Type SharedItem. Then click OK.

Connect the AttributeKeeper to the SharedItem Automations writer.

Run the workspace again and make sure you receive no errors.

Bookmarks can be used to clean up the workspace. Add a bookmark around all the transformers from the ParameterFetcher to the ListConcaternator and name it “Process Webhook”. Add another bookmark around the transformers from the HTTPCaller to the AttributeKeeper and name it “Get Item Information”.

The workspace should now look something like this:

8. Publish to FME Flow

Save the workspace and click Publish to open the Publishing Wizard. Then, connect to and publish the workspace to FME Flow.

Select your FME Flow connection and choose a repository to upload the workspace to (or create a new repository). Click Next.

In the next window, select your ArcGIS Enterprise Portal Web Connection if it has not already been published to FME Flow. Click Next.

In Register Services, select Job Submitter and click Publish.

Check the Translation Log to make sure everything was published to FME Flow successfully.

Please see the drop-down below for detailed instructions on how to create an ME Flow connection and publish workspaces.

To publish a workspace from FME Workbench to FME Flow, follow the generic instructions below using the specific details listed in the article. If the article does not have a specific repository name listed, we recommend creating a Training repository. If a specific service is not listed in the article, please use the default Job Submitter.

1. Click Publish

From the File menu, click Publish to FME Flow, or from the toolbar ribbon, click Publish.

Toolbar Ribbon:

File Menu:

Deploy Menu (2025.1+):

2. Add a Web Connection

In the Publish to FME Flow wizard, a FME Flow connection needs to be selected or created. To create an FME Flow connection, click the drop-down under FME Flow Connection and select Connect to FME Flow.

The following instructions are for a basic connection, but if your FME Flow is set up for Microsoft Azure, SAML, or Windows Credentials, please see the Using Web Connections to Connect to FME Flow documentation.

An FME Flow Connection dialog will appear; here, we will set up the connection. The connection information and credentials can be obtained from your FME Flow Administrator. If you have not yet logged into FME Flow after installing, please do so prior to continuing, as you will be prompted to change the password from the default:
Username: admin
Password: admin
- • Web Service: FME Flow
  
  • Server URL: http://localhost
  
  • Connection Name: Training FME Flow
  
  • Authentication: Basic
  
  • Username: your username
  
  • Password: your password
  
  Click OK. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
- • Web Service: FME Server
  
  • Connection Name: Training FME Server
  
  • Server URL: http://localhost
  
  • Username: your username
  
  • Password: your password
  
  Click Authenticate. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
  
  Click Next to move to the Create Repository page of the dialog
3. Create Repository

Repositories are used to store groups of related items that have been published to FME Flow (similar to directories). Items such as workspaces, source data, custom formats, custom transformers, and templates can be published to a repository.

If you need to create a new repository, click on the New button next to Repository Name. Enter the repository name and an optional description. Click OK to close the Create New Repository dialog.

To select an existing repository, click the drop-down next to Repository Name and select the desired repository.

4. Upload Files

Some workspaces require additional files to run correctly on FME Flow, so these need to be uploaded along with the workspace. To upload files, enable Upload data files at the bottom of the dialog.

When files are uploaded through the publishing menu, they are stored within the repository. If you want to upload folder files (such as geodatabases) or to specify where the files stored, the Select Files button allows you to do that. For detailed instructions on how to use the Select Files button, see the documentation.

5. Register Services

The final step before publishing a workspace is to register services. Services control how the data will be output after the workspace is run.

Services return results in different forms:

• Data Download Service returns results as a downloadable zip file

• Data Streaming Service returns results as a data stream

• Job Submitter Service accepts and runs workspace job requests

• KML Network Link returns a KML Network Link that can be used in Google Earth

• Notification Service allows for event-driven messaging

Typically, the Job Submitter is always enabled, but multiple services can be enabled. Once services have been enabled, click Publish.

6. Review Translation Log

To confirm whether or not the workspace was published successfully, check the Translation Log. The Translation log will show which repository you published to, which files were included, and a quick link to run the workspace.

Part 3: Complete the FME Flow Automation

In Part 3, we finish building the FME Flow Automation so that it runs our workspace and sends an email notification each time it receives a webhook message from ArcGIS Portal.

Firstly, an ArcGIS Enterprise Portal web connection needs to be authorized and working on FME Flow. If the ArcGIS Portal web connection was uploaded for the first time with the workspace, finish the process for authorization on FME Flow.

1. Create a Workspace Action

Open the Automation we created in Part 1. Stop the Automation if it is still running.

Add a new Action to the canvas and connect it to the Webhook Trigger’s Action Success port. Keep the log action attached to the success port so we can see the webhook response in case the workspace fails.

Open the Action and choose Run a Workspace. Choose the repository where your workspace was published and select the workspace.

Once the workspace is selected, you will notice that the Webhook Message parameter is already filled in with the test payload used as our workspace’s user parameter default value. Instead, we want to run the automated workspace with real-time message content. Open the drop-down menu for Webhook Message, expand Webhook Keys, and select “Message Content”. Click Apply.

After adding the workspace, you will see a port named SharedItem on the Run Workspace Action. This port is available because the workspace we created writes to an FME Flow Automation Writer, which creates a port from the Feature Type name. For more information about the FME Flow Automation Writer, see the Routing Data Between Workspaces in Automations article.

2. Create an External Action to Send the Email Notification

Connect an external action to the SharedItem port of the FME Workspace action. For the Action, choose Email (send).

The example below uses Gmail to send the email. Click Load Template and choose Gmail. Enter the SMTP Account and Password. In the Email To field, enter the email(s) to notify when a Portal item is shared with everyone (use your own email for testing).

Create an Email Subject using the Text Editor to access the attributes from the workspace.

In the Text Editor, click an attribute name to add it to the subject.

You can take the same approach to create a formatted Email Body.
The following item from the ArcGIS Enterprise Portal has been shared with {route.SharedItem.sharedToGroups}

Title: {route.SharedItem.title}
Description: {route.SharedItem.description}
Type: {route.SharedItem.type}
ID: {route.SharedItem.id}

Shared By: {route.SharedItem.username}

Click Apply to save the External Action parameters. Save and Start the Automation.

3. Test the Portal External Sharing Automation

Share any item from your ArcGIS Enterprise portal. Give a few seconds and check your email.

If you don’t receive an email, check the Automation Log for errors.

Your ArcGIS Portal organization webhook Automation is complete!

In this workflow, we built an FME Flow Automation to send an email whenever an item in your Portal is shared with everyone. The ArcGIS admin is able to remain vigilant about what gets shared with the public, while saving valuable time out of their day.

Using ArcGIS Enterprise Portal webhooks with the ArcGIS Portal Reader/Writer, ArcGISOnlineConnector, and the HTTPCaller (for ArcGIS REST API requests) you can automate many other administrative tasks.