Search

How to Write Updates to an Existing ArcGIS Online Feature Service

Liz Sanderson
Liz Sanderson
  • Updated

Files

As of FME 2024.0, support for Esri ArcGIS Online (AGOL), ArcGIS Enterprise Portal, and ArcGIS Server Feature Service formats has transitioned to a unified approach using the new Esri ArcGIS Feature Service (Format). This format replaces the legacy reader/writer formats for each of the three services.

Starting in FME 2026.1, the legacy formats will be hidden in the Quick Add menu. Instead, equivalent functionality is now provided by the downloadable Esri ArcGIS Connector package, available on FME Hub. The package also includes new web services and key transformers such as the ArcGISOnlineConnector, ArcGISAttachmentConnector, and ArcGISBranchVersionManager. The package can be installed directly through the Quick Add menu in FME Workbench. Once installed, the new Esri Feature Service format will appear in the Gallery as a unified method for interacting with all three types of Esri ArcGIS Feature Services.

For details on this transition, including guidance on updating existing workspaces, please refer to the article: Working with Esri ArcGIS Feature Services in FME.

Introduction

The ArcGIS Online Feature Service writer supports four modes: insert, update, delete, and upsert. These modes can be set either at the feature type level (to apply to every feature written to the corresponding feature service) or at the feature level using the fme_db_operation attribute. For more information on using the UPSERT mode, refer to the article "Updating ArcGIS Online Feature Services using UPSERT."

This writer mode setting works in much the same way on this writer as it does for any of the database writers in FME, as documented here: Database Writer Mode. One important difference with the ArcGIS Online Feature Service writer, however, is that it does not allow for manually selecting the attribute to be used as a unique identifier for making updates and deletes. Updates and deletes are done using an OBJECTID field that is automatically added when a new Feature Service is created. Any features being sent to the writer to be updated or deleted, therefore, must have this field set on them. Alternatively, if the Feature Service has edit tracking enabled, GlobalID must be present as the unique identifier for any edits involving update or delete.

One way to attach the OBJECTID or GlobalID field to your features is by adding an ArcGIS Online Feature Service reader or FeatureReader transformer into your workflow. In this example, following the ChangeDetector, a FeatureJoiner is used to merge the unique identifiers to the updated features before writing to the Feature Service.  

The attached workspace shows an example of how to write updates to an ArcGIS Online Feature Service. This same method is also used for inserts and deletes. In the following sections, we'll use the WHERE clause to query edits and demonstrate incremental edit tracking with the REST API.

Step-by-step Instructions

Part 1: Writing Data to ArcGIS Online

1. Open FME Workbench

Open FME Workbench and click New to create a new workspace.

2. Add a Shapefile Reader to the Canvas

Drag and drop the PlayingFieldStatus.zip to the canvas. This will add a reader to your workspace. Since the file is zipped, FME will not be able to determine which format to select. In the Add Reader dialog, set the following parameters:

  • Format: Esri Shapefile
  • Dataset: PlayingFieldStatus.zip

Click OK to add the reader.

2. Shapereader.png

3. Inspect the Data

Click on the reader on the canvas. Then, click on the Run button above it with data caching enabled. This will read the data. Click on the green data cache icon and inspect the data within the Data Preview tab. Note that our shapefile data consists of point geometries.

3. Shapefile inspect.png

4. Add an Esri ArcGIS Feature Service Writer

Select the canvas again and start typing ArcGIS, then scroll down and select the ArcGIS Feature Service writer. This writer is included in the downloadable Esri ArcGIS Connector package, available on FME Hub. 

Within the parameters window, select an ArcGIS Online web connection. If you need to create a new ArcGIS Online connection, you can create a new connection by selecting Add Web Connection. This gives you the option to authenticate either through OAuth 2.0 or by generating a token. A folder was created specifically for this tutorial within ArcGIS Online. If applicable, select the destination folder within your ArcGIS Online account. Since we are creating a new feature service, proceed with the following parameters:

  • Format: Esri ArcGIS Feature Service
  • Parameters:
    • Source Type: ArcGIS Online
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
    • Folder: Open the ellipses (...) and select the destination folder (if applicable)
    • Feature Service Handling: Create If Needed
    • Feature Service Name: PlayingField_Status
    • If Feature Service Exists: Overwrite Existing

Click OK.

In the Feature Type window, specify a Layer Name that will be displayed in your canvas. This name will also be applied to the point feature layer within the ArcGIS Online Feature Service. For consistency, use the same name as your feature service.

  • Layer Name: PlayingField_Status
  • Geometry: arcgisfeatureservice_point

Connect the reader and writer together.



6. Write the Data to ArcGIS Online

Select Run Workspace to write the data to ArcGIS Online. Log in to ArcGIS Online to confirm that the data has been written out.

Before proceeding to the next section, we will enable edit tracking on the Feature Service. On the Feature Layer's item page, click on the Settings tab and scroll to the Editing Options section. Toggle on the following options:

  • Enable editing: Enabled (default)

  • Keep track of changes to the data (add, update, delete features): Enabled

    • This option enables the GlobalID field and Change Tracking capability, along with system-maintained change logs. 

  • Keep track of who edited the data (editor name, date and time): Enabled

    • This option enables Editor Tracking, creating the CreationDate, EditDate, Creator, and Editor fields.

As per Esri's documentation, hosted feature Services cannot have both the Sync and ChangeTracking capabilities enabled at the same time. Please ensure that the Sync option is disabled before proceeding with this exercise.

Navigate to the Data tab and reinspect the data. GlobalID and OBJECTID fields are hidden by default, but you can toggle them on with the Column Visibility menu located at the top right bar.

Part 2: Detecting Changes and Uploading to ArcGIS Online

1. Add readers to the workspace

Select the canvas and start typing ArcGIS, then scroll down and select the ArcGIS Feature Service reader. Apply the following parameters. 

  • Format: Esri ArcGIS Feature Service

  • Parameters:
    • Source Type: ArcGIS Online
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
    • Folder: Select a source folder (if applicable)
    • Feature Service: Select PlayingField_Status
    • Layers: Select PlayingField_Status

Next, add a JSON Reader to the workspace.

  • Format: JSON (JavaScript Object Notation)

  • Dataset: playingfieldsupdate.JSON

Attach the JSON reader to an AttributeManger to remove attributes that do not match the schema of the existing Feature Service, ensuring only necessary fields are present for comparison.  In the AttributeManager parameters, set the following:

  • Remove Attribute:
    • Input attribute: json_ogc_wkt_crs
      • Action: Remove
    • Input attribute: json_geometry.type
      • Action: Remove
    • Input attribute: json_geometry.coordinates{}
      • Action: Remove

2. Add a ChangeDetector to detect changes.

Add a ChangeDetector to the workspace and attach the AttributeManger to the Revised port on the ChangeDetector. Next, attach the ArcGIS Feature Service Reader to the Original port of the ChangeDetector. This allows the transformer to compare selected attributes between the revised and original features to identify changes.

Within the ChangeDetector, set the following parameters:

  • Update Detection Key Attributes: RecordID
  • Selected Attributes: Open ellipses (...) to select attributes to match or exclude
    • Closure_No
    • Park_Name
    • RecordID
    • Site_Area
    • Weekend_St
  • Check Geometry: Disabled
  • Detailed Change List Name: _changes

If you run the current workspace and examine the data caches of features output from the Updated port. Note the fme_db_operation attribute and the _changes list attribute that was created for each feature. 

3. Add FeatureJoiner to prepare data for Feature Service update

Next, add a FeatureJoiner transformer to add the required unique identifiers (OBJECTID and/or GlobalID) to the features to be updated. Connect the ChangeDetector's Updated output to the Right input port and the initial ArcGIS Feature Service Reader to the Left input port. 

Set the following parameters on the FeatureJoiner:

  • Join Mode: Inner
  • Attribute Conflict Resolution: Prefer Right
  • Geometry Handling: Prefer Left
  • Join On:
    • Left: RecordID
    • Right: RecordID
    • Comparison Mode: Automatic

Using a ListExploder, you can expand the changes list attribute derived from ChangeDetector and generate a consolidated summary of edits, which can be written to a separate log for tracking purposes. 

Connect the features from the FeatureJoiner's Joined output port into the ListExploder and add a writer of your choice.

  • List Attribute: _changes{}

4. Add a writer to update the Feature Service.

Lastly, we will add the ArcGIS Feature Service writer to the workspace to write updates back to the same Feature Service from which we read. Connect the writer to the FeatureJoiner's Joined output.

Use the following parameters:

  • Format: Esri ArcGIS Feature Service

  • Parameters:
    • Source Type: ArcGIS Online
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
    • Folder: Select destination folder (if applicable)
    • Feature Service Handling: Use Existing
    • Feature Service Name: Select PlayingField_Status

Click OK. 

In the Feature Type window, set the following: 

  • Feature Type Parameters:
    • Layer Name: PlayingField_Status
    • Geometry: arcgisfeatureservice_point
    • Feature Operation: fme_db_operation

Click OK twice.

 

Since Enterprise/Portal's Data Store uses a PostgreSQL database, attribute names cannot contain uppercase letters. Failure to use lowercase names may result in blank values. Therefore, prior to writing to an Enterprise or Portal Feature Service, ensure all attribute names are lowercase, typically by using a BulkAttributeNamer.

5. Write to ArcGIS Online and Check Output

Click the Run Workspace button with feature caching enabled to inspect all the features in the workspace. 


Next, check the output in ArcGIS Online to ensure the appropriate records were updated.


 

Part 3: Tracking Changes in ArcGIS Online

While transformers like ChangeDetector can generate detailed change logs when updates are performed within your FME workflow, tracking changes becomes more complex when those edits are made externally. This includes changes made directly by other users or applications within the ArcGIS Online/Portal environment.

Fortunately, enabling edit tracking on the Feature Service allows us to audit and track these external updates. We will explore two distinct approaches to fetching recently modified features. The first, simpler method involves utilizing a WHERE clause to query only for data modified (inserts and updates) after a certain time. The second, more advanced method, utilizes the HTTPCaller transformer to access the REST API's extractChanges operation, which retrieves more detailed changes. These methods offer a way to capture incremental updates, serving as a backup alternative to event-driven webhooks.

Method A: WHERE Clause

This method will leverage the WHERE clause within an ArcGIS Feature Reader. Instead of reading and comparing your entire dataset, you can query exactly which features have been modified since a specified date and time. As discussed in earlier steps, when you toggle on the Feature Service tracking option: "Keep track of who edited the data (editor name, date and time)", this will generate CreationDate, EditDate, Creator, and Editor fields.

1. Create a DateTime User Parameter 

In a new workspace, before defining the query filter, add a user parameter first. This allows for flexibility in storing a specific timestamp to be used in the WHERE clause for retrieving updated features. In the Navigator pane, right-click on User Parameters, select Manage User Parameters

Click the green plus icon to create a user parameter. Set the following in the Parameters:

  • User Parameter type: Datetime
  • Parameter Identifier: LastEdit_datetime
  • Prompt: Filter features modified after
  • Published: Enabled
  • Required: Enabled
  • Enabled Attribute Assignment: Enabled
  • Time Format: Datetime
  • Time Zone for Saved Value: Local

2. Read the Feature Service

While the standalone Feature Service Reader works well for static queries, we recommend using a FeatureReader for this workflow. This ensures that dynamic parameters are fully resolved at runtime before the query is submitted.

Start by adding a Creator and a FeatureReader to the workspace. This acts as the initiator feature to activate the FeatureReader. Connect the Creator to the Initiator port of the FeatureReader. 

If you are using FME 2025.2 and newer, you can start your workflow with just the FeatureReader.

As of FME 2025.2, many transformers have been updated to not require input from the creator transformer. For a list of all the transformers with this improvement, please see Transformers with an Optional Input Port

Set the following parameters for the FeatureReader:

  • Format: Esri ArcGIS Feature Service
  • Feature Service Parameters:
    • Source Type: ArcGIS Online
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
    • Folder: Select source folder (if applicable)
    • Feature Service: Select PlayingField_Status
  • FeatureReader Parameters:
    • Feature Types to Read: Select PlayingField_Status
    • WHERE Clause: EditDate >= timestamp '@DateTimeFormat(@TimeZoneSet(@TimeZoneSet($(LastEdit_datetime), local), utc),%Y-%m-%d %H:%M:%S)'
      • Returns only features that were edited after the user-specified datetime
      • Converts user-specified local datetime into UTC format required for ArcGIS REST API date-time queries

Date-time queries for standard esriFieldTypeDate fields must use TIMESTAMP functions in the WHERE clause.

<DateField> = timestamp 'yyyy-mm-dd HH:mm:ss'

In our expression, if user enters 2025-11-24 17:00:00' into the user parameter:

  • @TimeZoneSet(@TimeZoneSet($(LastEdit_datetime), local), utc) produces a zoned datetime with local offset (20251124170000-08:00), then converts that value to UTC (20251125010000+00:00)
  • @DateTimeFormat(<datetime>,"%Y-%m-%d %H:%M:%S") formats the value into 2025-11-25 01:00:00 

For more information on date/time functions available in the Text Editor, refer to our Date/Time Functions documentation. For more advanced queries, such as adding AND or BETWEEN operators, refer to Esri's Query documentation for guidance.

Next, run the workspace and use a timestamp from before the updates made in Part 2. Inspect the data cache on the PlayingField_Status port.

You will notice that the Feature Service Reader currently returns date fields from Feature Services in a 13-digit UNIX Epoch Milliseconds (UTC) format. To verify our query is accurate, we will convert these fields into a readable local time format in the next step.

3. Clean and Verify Results From the Query

Add an AttributeManager to the workspace and connect it to the FeatureReader's PlayingField_Status port. 

To work with FME’s date/time functions, we must first convert the value by dividing it by 1000. This can be done with the @Evaluate() function in the Text Editor, or alternatively with the Arithmetic Editor in a separate AttributeManager if preferred. 

In the AttributeManager, set the following:

  • Attribute: EditDate
  • Value:@DateTimeFormat(@TimeZoneSet(@DateTimeParse(@Evaluate(@Value(EditDate)/1000),%s),local),"%Y-%m-%d %H:%M")) 

Here is what each piece of the value query is calculating:

  • @Evaluate(@Value(EditDate)/1000) 

    • Convert Epoch Milliseconds into Epoch Seconds

  • @DateTimeParse(…, %s) 

    • Parse into FME datetime with UTC offset

  • @TimeZoneSet(…, local) 

    • Converts to equivalent local datetime

  • @DateTimeFormat(…, "%Y-%m-%d %H:%M") 

    • Formats into readable datetime

Run the workspace and inspect the output. EditDate now reflects the correct local time of updates made in Part 2, restoring the same features we had updated, confirming that the query is functioning correctly.

4. Next Steps

Now that the workspace has been configured to fetch only incremental updates, you can automate this process on a schedule in FME Flow and connect your AttributeManager to a writer. The returned modified features can then be synced downstream to a backup database, following a similar workflow to Part 2, or used for incremental QA checks.  

However, deletions are not detected using this method. In the next section, we'll explore a more advanced option for capturing slightly more comprehensive change tables, including deletions, using the extractChanges REST API endpoint.

Method B: Rest API - extractChanges

Method B takes a more advanced approach by directly calling the Feature Service's extractChanges REST API endpoint. This retrieves a snapshot of the changes table, providing a distinct breakdown of features that were added, updated, or deleted between two time checkpoints.

This operation is only available when the Feature Service has Change Tracking capability enabled. Before making any edits, ensure that the Feature Service Layer setting "Keep track of changes to the data (add, update, delete features)" is enabled and the "Enable sync" option is disabled, as outlined in Part 1.  

In this section, we will use HTTPCaller to communicate with the REST API, focusing solely on retrieving the OBJECTID of the changed features for simplicity. To reduce the complexity of handling JSON responses and attribute cleanup, a custom transformer has been created to streamline the extraction of the changes table. 

Requesting additional details like attribute values, geometry updates, or full information of deleted features, often triggers an asynchronous response which may require polling and looping to resolve which we will not cover in this article. For more advanced configurations, refer to Esri’s extractChanges documentation for guidance.

1. Retrieve Feature Service's Metadata

Before calling the extractChanges endpoint, we must first retrieve the Feature Service's metadata using an HTTPCaller. This metadata includes the Feature Service's server generation number, serverGen, and a version number corresponding to the dataset’s edit generation. Each time an edit is performed, the server's generation number increments. The extractChanges operation requires this value to determine what has changed since that generation, allowing the server to return only incremental updates occurring after that specific generation. 

The serverGen retrieved from the metadata response of the first HTTPCaller is static and serves only as an initial baseline. To retrieve the latest serverGen for future runs, we must rely on the returnUpdates parameter within the extractChanges operation in the next step. Because the Feature Service does not maintain an accessible history log of server generation numbers, our workflow needs to cache the latest serverGen after each run if we wish to maintain the incremental tracking chain.

Log into ArcGIS Online, browse to your Feature Layer item Overview page, and copy the Feature Service's URL.

Download the workspace template and run the FeatureServiceChangeExtractor custom transformer. When prompted for user parameter values, paste your Feature Service URL. Since the server generation numbers are currently unknown, leave those fields blank. When these range parameters are omitted, the REST API will default to their respective limits. Since both limits are blank, this defaults the query range to the earliest serverGen and the latest serverGen available.

Once translation completes, right-click the custom transformer on the canvas, then select 'Edit Embedded Transformer' to inspect and preview the internal workspace. 

The following are the parameters for the initial HTTPCaller:

  • Request URL: Feature Service URL
    • In the custom transformer, we created a user parameter for the Feature Service URL for greater flexibility
  • HTTP Method: GET
  • Use Authentication: Enabled
    • Authentication Method: Web Connection
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
  • Query String Parameters: 
    • Name: f 
      • Value: json
  • Response Body Attribute: _meta_json

The HTTPCaller returns a JSON response, which we need to parse to extract the relevant generation values. We will use minServerGen, which represents the minimum generation number available for the Feature Service. A JSONExtractor is used to pull both the LayerName and the corresponding minServerGen from the response’s changeTrackingInfo section.

Next, a ParameterFetcher retrieves the optional user parameters that define the minimum and maximum server generations for our main query. When no values are supplied, an AttributeManger handles the conditional logic and defaults to the minServerGen value retrieved from the metadata.

2. Retrieve the Changes Table

With our serverGen values established, we can now issue the main request to the extractChanges REST API endpoint with a second HTTPCaller. This call retrieves the changes table, a structured list of features that were added, updated, or deleted within the specified server generation range. To simplify the workflow, the request is configured with returnIdsOnly=true, returning only the OBJECTIDs of changed features. This keeps the response small and avoids the asynchronous handling required for returning full attribute or geometry details. A full list of other extractChangesCapabilities can be found in the metadata response from the first HTTPCaller: 

The second HTTPCaller is configured as follows:

  • Request URL: $(Request_Feature Service URL)/extractChanges
  • HTTP Method: POST
  • Use Authentication: Enabled
    • Authentication Method: Web Connection
    • Web Connection: Esri ArcGIS Online OAuth (safe.esri-agol)
  • Body parameters:
    • Upload Body: f=json&layers=0&serverGens=[@Value(user_serverGen_min),@Value(user_serverGen_max)]&returnIdsOnly=true&returnInserts=true&returnUpdates=true&returnDeletes=true
      • Return a JSON list of OBJECTIDs for all added, updated, and deleted features occurring within the specified server generation range
    • Content Type: URL Encoded (application/x-www-form-urlencoded)

The returned JSON response provides two key pieces of data: a list of OBJECTIDs of edits (additions, deletions, updates) and a serverGen value representing the upper boundary of the retrieved changes.

{
	"edits": [
		{
			"id": 0,
			"objectIds": {
				"adds": [
				],
				"deletes": [
				],
				"updates": [
					4,
					7,
					149,
					151
				]
			}
		}
	],
	"layerServerGens": [
		{
			"id": 0,
			"serverGen": 3297085
		}
	],
	"responseType": "esriReplicaResponseTypeEdits",
	"transportType": "esriTransportTypeUrl"
}

In our example, because we did not define specific minimum or maximum serverGen values, the returned response automatically defaulted to the full available range, fetching all changes from the earliest to the latest server generation. In other words, the serverGen returned (3297085) is the latest server generation available and becomes the new baseline minimum server generation for future extractChanges requests to continue incremental tracking.

3. Parse Response and Cache the New Baseline serverGen

A JSONExtractor parses the serverGen value from the response and routes it through a TestFilter to check whether it is new. New values are forwarded to both caching and downstream labelling the retrieved changes. Existing values bypass caching and proceed directly downstream. This new baseline serverGen exits the serverGen_new output port, which then can be stored, either in a file or database, and supplied back into the user_serverGen_min attribute for retrieving the next set of incremental changes.

Features continue downstream into the changes table section, where a series of JSONFragmenters explode the JSON arrays and extract the edits. The resulting Changes Table is shown below, providing a snapshot of all edits from the query range, matching the edits we performed earlier in the exercise. 

The ChangesTable output can be written to a file or database for backup and incremental edit tracking purposes. While this example focuses on returning only OBJECTIDs, you can retrieve additional details by following a similar workflow outlined in earlier steps, passing features into a FeatureReader, using a WHERE clause such as OID = OBJECTID to join geometry, attributes, and EditDate information back onto the changes table. To enrich deleted features, since they no longer exist in the Feature Service, their attributes must be retrieved through a modified API call or by comparing against a previous dataset using the Part 2 ChangeDetector workflow.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.

Was this article helpful?

We're sorry to hear that.

Please tell us why.

As of January 14th, 2026, comments on knowledge base articles have been closed. To make sure questions don’t get missed and to enable more community support, we’ve moved discussions to the FME Community. If you have a question or a comment about this article, please create a new post or create a support ticket.