Streaming XML with FME Flow

Introduction

XML is a markup language that enables users to exchange data across web services. In FME Flow, a workspace can be published using the Data Streaming Service to easily create a web application to stream spatial data stored as XML.
The example below shows how to create a workspace that takes spatial data and writes it out to XML, which can then be published to FME Flow to generate a direct URL that can be used to stream the data within a web service. 
The second section of this tutorial will provide an overview of the data streaming service with GML format - an OGC specification for representing geographic data in XML.

Step-by-step Instructions

Example: Streaming XML

Follow the instructions below to create a workspace that will write out data in XML that can then be published to FME Flow using the Data Streaming Service. 
An XML document usually comprises of at least two components:

• Root: is the first element to appear in any markup language and defines what kind of document the file will be; it also encapsulates all other elements that will appear in the file.
• Sub: this is any additional element that needs to fall within the opening and closing root tag.

1. Create a CityName Attribute
Open a blank workspace in FME Workbench. Add a Creator transformer to the canvas and then connect an AttributeCreator to it. 

In the AttributeCreator parameters, create a New Attribute called CityName and then set the Attribute Value to Austin Bus Routes. For FME 2023+, set the Type to Char(200). This will be the Root element we will refer to later in the XML template. 


2. Add an OGC GeoPackage Reader
In this example, the Sub Template will list the geometry of the bus routes in Austin. First, we will need to read in the Transit.gpkg dataset. Add an OGC GeoPackage reader to the canvas and browse to the Transit.gpkg dataset, which can be downloaded from the Files section of this article. Click OK to add the reader to the canvas. This GeoPackage only has the Busroutes table. 


3.  Extract Feature Geometry
Add a GeometryExtractor transformer and connect it to the busroutes reader feature type. In the parameters,  change the Geometry Encoding to FME XML. This transformer will convert the geometry associated with each bus route into XML structure.


If you were to run the workspace with feature caching enabled, each feature would have an additional geometry attribute that will look like this:


Notice the <multicurve name> element that indicates the feature is made up of multiple lines.

4. Set XML Template
Now we have created both of the requirements for the final XML file we can put them together using the XMLTemplater transformer. When this transformer is first added to the canvas it only has a Root input, so connect the AttributeCreator to the Root input port. In the XMLTemplater parameters, copy the following expression and paste it into the Template section of the ROOT element. 

<?xml version="1.0" encoding="UTF-8"?>
<CityName>{fme:get-attribute("CityName")}
<BusRoutes>{fme:process-features("BUSROUTES")}
</BusRoutes>
</CityName>

This template states that the document will be XML and will also write out the value of the CityName attribute created in Step 1.
You can edit this template to include additional attribute data as long as the Root port is receiving that data.
Note: Line {fme:process-features("BUSROUTES")} is a placeholder that correlates to the Sub Template we are about to create. Every bus route will be inserted as a child of the <BusRoutes> node.

Next, enable Sub Template and add a new sub template by clicking the plus ( + ) sign, then copy the following expression and paste it into the Template section for the SUB element. 

<BusRoute>
<LINE_ID>{fme:get-attribute("LINE_ID")}</LINE_ID>
<LINE_NAME>{fme:get-attribute("LINE_NAME")}</LINE_NAME>
{fme:get-xml-attribute("_geometry")}
</BusRoute>

This sub template is telling the document to add the Bus route ID and Name before detailing the geometry of the route.
Note: Line {fme:get-xml-attribute("_geometry")} pulls in the XML geometry feature created in the GeometryExtractor. Notice here that this is a get-xml-attribute function rather than the usual get-attribute, this is to tell the template that these attributes are already written in XML and to drop everything but the geometry itself.
Note: If you have an XML Schema (XSD) file you can use the XMLSampleGenerator transformer to create an XML Template that can be used as a base in the XMLTemplater.
Before we close the parameter editor, change the SUB port name to BUSROUTES; renaming this port is important as it is referenced in the Root template. Finally, change the XML Result attribute name to text_line_data.


After closing the XMLTemplater parameters, connect the GeometryExtractor to the BUSROUTES input port. 



5. Format the XML File
Add an XMLFormatter to the canvas and connect it to the XMLTemplater. In the parameters,  change the Attribute With XML Text to text_line_data and do the same to the Attribute to contain XML output. We can use this transformer to clean up the lines and appearances before writing the data out.


6. Add a Text File Writer
The XML Document is now ready to be exported to a text file. Add a Text File writer to the workspace and set the dataset to Busroutes.xml. In the Parameters, change the MIME Type to text/xml from the drop-down list.


Note: Here we are using the Text File writer instead of the XML writer because we have already generated the entire XML document in a single attribute so all that is needed is to direct the writer to this text_line_data attribute. It is important to set the correct MIME Type when using the data streaming service.

7. Run the Workspace
Before publishing the workspace to FME Flow run the workspace and inspect the output in a text editor to see what you have created.


8. Publish the Workspace to FME Flow
Publish the workspace to FME Flow. Create a new repository called XML and ensure you check the box Upload data files. 


Register the workspace to the Data Streaming Service only.


9. Run the Workspace Using the Data Streaming Service
Open the FME Flow Web Interface and navigate to the Run Workspace page on the side menu. Select the XML repository and then click Run.


After a few moments, the XML will be returned in your web browser. 



To create a direct URL for other users to access, please see the Webhook instructions in How to Create a WFS Service Using FME Flow or create an FME Flow App. 
 

Example: Streaming GML

This example demonstrates how you can write a spatial dataset out to GML format to generate your own XML schema, which can be used to stream data in a web service. Writing to GML is much simpler than writing to XML because it doesn’t require any formatting. 

1. Create a New Workspace
Start a new workspace in FME Workbench. If you are continuing from the previous example, please save and close that workspace. 

2. Add an OGC GeoPackage Reader
Add an OGC GeoPackage reader to the canvas and browse to the Transit.gpkg dataset, which can be downloaded from the Files section of this article. Click OK to add the reader to the canvas. This GeoPackage only has the Busroutes table. 


3. Add an OGC GML Writer
Add an OGC GML (Geography Markup Language) writer to the canvas and browse to a location to save the GML file. Name the file Busroutes.gml and set the Coord. System to LL84. Click OK to finish adding the writer to the canvas. 


After adding the writer to the canvas, connect it to the reader feature type. 



4. Run the Workspace
Run the workspace and click on the writer feature type to open the popup menu. On the popup menu click on the Open Containing Folder button. 


In the folder, note that by running this workspace, two files were created: the GML file containing the data and an XML schema file. The other Busroutes.xml dataset shown in the screenshot is from the previous exercise. 


5. Update Writer to Use Schema
Return to the workspace, and in the Navigator panel, right-click on the BusRoutes writer and select Update Writer. 


Open the Parameters and change the GML Version to GML Application Schema. Next, click on the ellipsis for Application Schema and select the Busroutes.xsd file that was just created. 


6. Publish to FME Flow
Publish the workspace to FME Flow. Create a new repository called XML and ensure you check the box Upload data files. 


Register the workspace to the Data Streaming Service only.


7. Run the Workspace Using the Data Streaming Service
Open the FME Flow Web Interface and navigate to the Run Workspace page on the side menu. Select the XML repository and then click Run. After a few moments, the GML will be streamed to your web browser.


To create a direct URL for other users to access, please see the Webhook instructions in How to Create a WFS Service Using FME Flow or create an FME Flow App. 

The OGC Web Feature Service (WFS) is supported by FME Flow and has the ability to return features with spatial geometry like the examples covered in this article. For detailed instructions, see How to Create a WFS Service Using FME Flow.
Alternatively, you may be interested in streaming other data formats, in which case this article may be of interest: Streaming GeoJSON with FME Flow.
 

Data Attribution

Data used in this article originates from open data made available by the City of Austin, Texas. It contains data licensed under the Public Domain Dedication License - City of Austin.