Streaming XML with FME Flow

Files

StreamingXML.zip
- 3 MB
- Download
StreamingXML_25-2.zip
- 3 MB
- Download

Introduction

XML is a markup language that enables users to exchange data across web services. In FME Flow, a workspace can be published using the Data Streaming Service to easily create a web application to stream spatial data stored as XML.

The example below demonstrates how to create a workspace that processes spatial data and writes it out to XML, which can then be published to an FME Flow to generate a direct URL that can be used to stream the data within a web service.

The second section of this tutorial provides an overview of the data streaming service with GML format, an OGC specification for representing geographic data in XML.

Step-by-step Instructions

Example: Streaming XML

Follow the instructions below to create a workspace that writes data in XML, which can then be published to an FME Flow using the Data Streaming Service.
An XML document usually comprises at least two components:

Root: This is the first element to appear in any markup language and defines what kind of document the file will be; it also encapsulates all other elements that will appear in the file.
Sub: This is any additional element that needs to fall within the opening and closing root tag.

1. Create a CityName Attribute

Open a blank workspace in FME Workbench.

Add an AttributeCreator transformer to the canvas to start the workflow.
Add a Creator transformer to the canvas and then connect an AttributeCreator to it.

As of FME 2025.2, many transformers have been updated to not require input from the creator transformer. For a list of all the transformers with this improvement, please see Transformers with an Optional Input Port

In the AttributeCreator parameters, set the following:

Output Attribute: CityName
- Value: Austin Bus Routes
- Type: char(200)

This will be the Root element we will refer to later in the XML template.

2. Add an OGC GeoPackage Reader

In this example, the Sub Template will list the geometry of the bus routes in Austin. First, we will need to read in the Transit.gpkg dataset, which is available in the files section on the right.

Add a reader to the canvas and set the following parameters:

Format: OGC GeoPackage
Dataset: Transit.gpkg
- Click on the ellipses to navigate to the location of the file on your computer

Click OK to add the reader to the canvas.

3. Extract Feature Geometry

Add a GeometryExtractor transformer and connect it to the busroutes reader feature type. In the parameters, change the Geometry Encoding to FME XML. This transformer will convert the geometry associated with each bus route into XML structure.

If you were to run the workspace with feature caching enabled, each feature would have an additional geometry attribute that would look like this:

Notice the <multicurve name> element, which indicates that the feature is composed of multiple lines.

4. Set XML Template

Now that we have created both requirements for the final XML file, we can combine them using the XMLTemplater transformer. When this transformer is first added to the canvas, it only has a Root input, so connect the AttributeCreator to the Root input port. In the XMLTemplater parameters, copy the following expression and paste it into the Template section of the ROOT element.

<?xml version="1.0" encoding="UTF-8"?>
<CityName>{fme:get-attribute("CityName")}
<BusRoutes>{fme:process-features("BUSROUTES")}
</BusRoutes>
</CityName>

This template states that the document will be in XML format and will also display the value of the CityName attribute created in Step 1.

You can edit this template to include additional attribute data as long as the Root port is receiving that data

Note: Line {fme:process-features("BUSROUTES")} is a placeholder that corresponds to the Sub-Template we are about to create. Every bus route will be inserted as a child of the <BusRoutes> node.

Next, enable Sub Template and add a new sub template by clicking the plus ( + ) sign, then copy the following expression and paste it into the Template section for the SUB element.

<BusRoute>
<LINE_ID>{fme:get-attribute("LINE_ID")}</LINE_ID>
<LINE_NAME>{fme:get-attribute("LINE_NAME")}</LINE_NAME>
{fme:get-xml-attribute("_geometry")}
</BusRoute>

This sub-template instructs the document to add the Bus route ID and Name before detailing the route's geometry.

The line {fme:get-xml-attribute("_geometry")} pulls in the XML geometry feature created in the GeometryExtractor. Note that this is a get-xml-attribute function rather than the usual get-attribute; this is to inform the template that these attributes are already written in XML and to discard everything except the geometry itself.

If you have an XML Schema (XSD) file, you can use the XMLSampleGenerator transformer to create an XML Template that can be used as a base in the XMLTemplater.

Before closing the parameter editor, change the SUB port name to BUSROUTES. Renaming this port is important, as it is referenced in the Root template. Finally, change the XML Result attribute name to text_line_data.

After closing the XMLTemplater parameters, connect the GeometryExtractor to the BUSROUTES input port. \

5. Format the XML File

Add an XML Formatter to the canvas and connect it to the XML Templater. In the parameters, change the Attribute With XML Text to text_line_data and do the same to the Attribute to contain XML output. We can use this transformer to clean up the lines and appearances before writing the data out.

6. Add a Text File Writer

The XML Document is now ready to be exported to a text file. Add a writer to the workspace and set the following:

Format: Text File
Dataset: Busroutes.xml
- Click on the ellipses to navigate to a location on your computer

Next, click the Parameters button to continue configuring the writer.

In the Parameters, set the following:

FME Flow Parameters:
- MIME Type: text/xml

Here we are using the Text File writer instead of the XML writer because we have already generated the entire XML document in a single attribute, so all that is needed is to direct the writer to this text_line_data attribute. It is important to set the correct MIME Type when using the data streaming service.

7. Run the Workspace

Before publishing the workspace to FME Flow run the workspace and inspect the output in a text editor to see what you have created.

8. Publish the Workspace to FME Flow

Publish the workspace to FME Flow. Create a new repository called XML and ensure you check the box Upload data files.

For more in-depth instructions on how to create a web connection and upload to FME Flow, please see the information below:

To publish a workspace from FME Workbench to FME Flow, follow the generic instructions below using the specific details listed in the article. If the article does not have a specific repository name listed, we recommend creating a Training repository. If a specific service is not listed in the article, please use the default Job Submitter.

1. Click Publish

From the File menu, click Publish to FME Flow, or from the toolbar ribbon, click Publish.

Toolbar Ribbon:

File Menu:

Deploy Menu (2025.1+):

2. Add a Web Connection

In the Publish to FME Flow wizard, a FME Flow connection needs to be selected or created. To create an FME Flow connection, click the drop-down under FME Flow Connection and select Connect to FME Flow.

The following instructions are for a basic connection, but if your FME Flow is set up for Microsoft Azure, SAML, or Windows Credentials, please see the Using Web Connections to Connect to FME Flow documentation.

An FME Flow Connection dialog will appear; here, we will set up the connection. The connection information and credentials can be obtained from your FME Flow Administrator. If you have not yet logged into FME Flow after installing, please do so prior to continuing, as you will be prompted to change the password from the default:
Username: admin
Password: admin
- • Web Service: FME Flow
  
  • Server URL: http://localhost
  
  • Connection Name: Training FME Flow
  
  • Authentication: Basic
  
  • Username: your username
  
  • Password: your password
  
  Click OK. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
- • Web Service: FME Server
  
  • Connection Name: Training FME Server
  
  • Server URL: http://localhost
  
  • Username: your username
  
  • Password: your password
  
  Click Authenticate. FME will try to authenticate the connection. Once authenticated, this connection will be available whenever you publish a workspace. The connection will only need to be created once.
  
  Click Next to move to the Create Repository page of the dialog
3. Create Repository

Repositories are used to store groups of related items that have been published to FME Flow (similar to directories). Items such as workspaces, source data, custom formats, custom transformers, and templates can be published to a repository.

If you need to create a new repository, click on the New button next to Repository Name. Enter the repository name and an optional description. Click OK to close the Create New Repository dialog.

To select an existing repository, click the drop-down next to Repository Name and select the desired repository.

4. Upload Files

Some workspaces require additional files to run correctly on FME Flow, so these need to be uploaded along with the workspace. To upload files, enable Upload data files at the bottom of the dialog.

When files are uploaded through the publishing menu, they are stored within the repository. If you want to upload folder files (such as geodatabases) or to specify where the files stored, the Select Files button allows you to do that. For detailed instructions on how to use the Select Files button, see the documentation.

5. Register Services

The final step before publishing a workspace is to register services. Services control how the data will be output after the workspace is run.

Services return results in different forms:

• Data Download Service returns results as a downloadable zip file

• Data Streaming Service returns results as a data stream

• Job Submitter Service accepts and runs workspace job requests

• KML Network Link returns a KML Network Link that can be used in Google Earth

• Notification Service allows for event-driven messaging

Typically, the Job Submitter is always enabled, but multiple services can be enabled. Once services have been enabled, click Publish.

6. Review Translation Log

To confirm whether or not the workspace was published successfully, check the Translation Log. The Translation log will show which repository you published to, which files were included, and a quick link to run the workspace.

9. Run the Workspace Using the Data Streaming Service

Open the FME Flow Web Interface and navigate to the Run Workspace page on the side menu. Select the XML repository and then click Run.

After a few moments, the XML will be returned in your web browser.

To create a direct URL for other users to access, please see the Webhook instructions in How to Create a WFS Service Using FME Flow or create an FME Flow App.

Example: Streaming GML

This example demonstrates how to write a spatial dataset to GML format, generating your own XML schema that can be used to stream data in a web service. Writing to GML is much simpler than writing to XML because it doesn’t require any formatting.

1. Create a New Workspace

Start a new workspace in FME Workbench. If you are continuing from the previous example, please save and close that workspace.

2. Add an OGC GeoPackage Reader

Add an OGC GeoPackage reader to the canvas and browse to the Transit.gpkg dataset, which can be downloaded from the Files section of this article. Click OK to add the reader to the canvas. This GeoPackage only has the Busroutes table.

3. Add an OGC GML Writer

As of FME 2025.2, the Coordinate System parameter is now configured within the Parameters dialog of each reader/writer format. For more information, including details about the change and affected transformers, please see Coordinate System Parameter Location Change.

Add a writer to the canvas and set the following:

- Format: OGC GML (Geography Markup Language)
- Dataset: Busroutes.gml
  - Click on the ellipses to navigate to a location on your computer
Click on the Parameters button to set the Coordinate System

In the Parameters window, set the following:
- Spatial:
  - Coordinate System: LL84
Click OK twice to add the writer to the canvas.
- Format: OGC GML (Geography Markup Language)
- Dataset: Busroutes.gml
  - Click on the ellipses to navigate to a location on your computer
- Coord. System: LL84
Click OK to add the writer to the canvas.

After adding the writer to the canvas, connect it to the reader feature type.

4. Run the Workspace

Run the workspace and click on the writer feature type to open the popup menu. On the popup menu click on the Open Containing Folder button.

In the folder, note that by running this workspace, two files were created: the GML file containing the data and an XML schema file. The other Busroutes.xml dataset shown in the screenshot is from the previous exercise.

5. Update Writer to Use Schema

Return to the workspace, and in the Navigator panel, right-click on the BusRoutes writer and select Update Writer.

Open the Parameters and change the GML Version to GML Application Schema. Next, click on the ellipsis for Application Schema and select the Busroutes.xsd file that was just created.

6. Publish to FME Flow

Publish the workspace to FME Flow. Create a new repository called XML and ensure you check the box Upload data files.

7. Run the Workspace Using the Data Streaming Service

Open the FME Flow Web Interface and navigate to the Run Workspace page on the side menu. Select the XML repository and then click Run. After a few moments, the GML will be streamed to your web browser.

To create a direct URL for other users to access, please see the Webhook instructions in How to Create a WFS Service Using FME Flow or create an FME Flow App.

The OGC Web Feature Service (WFS) is supported by FME Flow and has the ability to return features with spatial geometry like the examples covered in this article. For detailed instructions, see How to Create a WFS Service Using FME Flow.

Alternatively, you may be interested in streaming other data formats; in that case, this article may be of interest: Streaming GeoJSON with FME Flow.

Data Attribution

The data used here originates from open data made available by the City of Austin, Texas. It contains information licensed under the Public Domain Dedication License - City of Austin.

Search