WebSockets and FME Server

As of FME Server 2019+, sending messages to/reading from FME Server's WebSocket Server can be completed using FME Server Automations. For more information, please see Getting Started with Automations. 

The WebsocketSender and WebsocketReciever transformers will be deprecated in FME 2024.0. Please update all workflows to use FME Flow Automation Websockets or Python. 

Introduction

WebSockets are a relatively new technology, implemented on top of HTTP, that allows for bidirectional browser-based client-server communication without the overhead of polling (AJAX requests with HTTP overhead), nor the inconvenience and unreliability of client plugins. Modern browsers include built-in implementations of a JavaScript API for connecting to a WebSockets server.

WebSocket modes in FME Server

There are two main patterns in which WebSockets can be used with FME Server, and which one you choose will depend upon your application. It is also possible to blend these patterns for different parts of the communication flow.

In all modes, a named stream is shared between clients. This means that FME Server can easily broadcast the same message to multiple clients, and receive communication from them on a single stream.

Notification Mode

Use this when your communication is low volume, or if you need the flexibility of the Notification Service.

In this case, communication between the engine (i.e. a running workspace) and the WebSocket server is routed through an FME Server topic by means of a WebSocket publisher and subscriber. Jobs are typically launched in response to a message on a topic, and complete when they have finished processing it.

Message Streaming Mode

Use this when you have a high volume of messages, and are dealing with browser-server communication only.

In this case, the engine (i.e. running workspace) connects directly to the WebSocket server, and keeps running until you cancel the job. This does tie up one engine license, but removes the overhead of starting up an engine and making connections.

Hybrid Mode

If different parts of your communication flow fit different descriptions, you may wish to blend these modes in a combined mode.

In this case, the engine will either respond to notification events and send directly on WebSocket, or vice-versa. This is useful when there aren’t many requests, but you still want the server to respond without the overhead of polling from the client, or you don’t want to dedicate a continuously running FME Engine to the job.

Trying out Notification Mode

In this mode, FME Server integrates WebSocket-based communication into its existing notification framework.

Configuring FME Server

In order to send and receive messages over WebSockets, we need to configure a Topic, a Publication and a Subscription, using the FME Server web interface. First, create a new topic called "topic_ws_sample". Then create a new publication called "pub_ws_sample" with a stream ID "sample_stream_in". See below for an example:

Note that the target URL should have the publicly accessible host name of the FME Server machine (the URL is configured to use localhost by default), and the default port and path should be kept as-is. The WebSocket server will communicate on port 7078.

Building the Workspace

FME Server has generic WebSocket transformers which must be configured to communicate using FME Server’s protocol. To send messages to FME Server, use WebSocketSender. To receive messages, use WebSocketReceiver.

First, we will simply send a message to FME Server via WebSockets, and view it in the Monitoring tab. Set up your workspace as follows, or use send_sample.fmw:

Ensure that you are using the correct hostname for the server - the WebSocketSender is set up to use localhost by default.

The connection preamble is

 {    ws_op : "open",    ws_stream_id : "sample_stream_in" }

This opens the communication with FME Server on the stream id specified when creating the publication.

The data to send is

 {     ws_op: "send",     ws_msg: "FME Lizard reporting for duty." }

In the FME Server Web Interface, set up Monitoring to view incoming messages. Keeping the browser window open, run the workspace in FME Desktop, and you should see the following:

Trying out Message Streaming Mode

In this mode, the engine (i.e. running workspace) connects directly to the WebSocket server, and keeps running until you cancel the job.

Building the workspace

The workspace “displacer.fmw” from the included files is configured to run in Message Streaming mode.

The key components in the workspace are the WebSocketReceiver and the WebSocketSender. The Receiver reads incoming messages, which are then extracted to points. The points are displaced, and written out by the Sender.

In this example, the outgoing messages are formatted as follows:

 { "ws_topic":"disp_pnts" "ws_msg":"{\"lat\": \"49.1468009923073\", \"lon\": \"-122.672091674805\" }" }

The ws_topic and ws_msg elements are required. ws_msg can contain any text you wish to send.

Building the Front-End

In the www folder of the tutorial files, “index.html” has all the JavaScript code needed to interface with FME Server in the web browser via web sockets.

The front-end web page includes a Google Maps interface and logs for incoming and outgoing messages. It makes two WebSocket connections - one for sending the point the user selects, and the other for receiving the displaced point. Access Live Demo

Installing the application

 FMEServer.init({             server : "http://<host>:<port>",             token : ""

• Configure your server in the JavaScript section of the index.html file. If you have not enabled SSL on your server, use "http://" instead of "https://" (Note: you can leave the token blank):
• Copy all the files from the www directory to <FMEServer>\Utilities\tomcat\webapps\ROOT\websockets-tutorial
• Change the “server_url” published parameter in “displacer.fmw” to point to your own server.
• Publish it to your FME Server, on the Job Submitter service.
• Start it using the Job Submitter service through FME Server’s web interface. The job will run continuously, so you can navigate away from the page. There will not be a success message.

Putting it together: Modify the setup to use hybrid mode

Finally, we will change the above workspace to use hybrid mode.

For a completed version, take a look at “displacer_modified.fmw” in the included files for this tutorial. As might be surmised, it is a modified version of displacer.fmw.

Starting with “displacer.fmw”, try changing the workspace, FME Server setup, and JavaScript code to use the following workflow. You’ll need to remove the WebSocketReceiver, and subscribe the workspace to a topic. The topic will need to have a WebSocket publication.

Building the workspace

The modified workspace should have a Text File Reader. You can specify “sample-json.json” as a data file, but it will be replaced by the incoming notification.

• Depending on your configuration, you may need to configure cross-origin resource sharing (CORS).
• Visit http://:/websockets-tutorial>:<port>/websockets-tutorial in a browser and click anywhere on the map.
• When you are done, you can just cancel the job from the Running tab of the Jobs page on the web interface.
• Remove the Creator and the WebSocketReceiver.
• Replace these with text file reader, and configure it to read the entire file at once.
• Add a JSONExtractor - it should extract the JSON attribute ws_publisher_content.
• Change the SubstringExtractor’s configuration to use the extracted content as its text attribute.
• Publish it to your FME Server, on the Job Submitter and Notification service. Edit the Notification service configuration to subscribe to a new topic called "points" and configure the rest as below:

Configuring FME Server

Create the WebSocket publication.

• Create a new publication called "points" which is published to the "points" topic.
• Configure the WebSocket protocol using your own server Target URL and a Stream ID called "points" See below:

Running the Application

Visit http://<host>:<port>/websockets-tutorial in a browser and click anywhere on the map.

Conclusion

Congrats! You now know how to use HTML5 WebSockets with FME Server - minimal overhead for communication between the browser and the server, and no server-side code required. Use WebSockets directly in message streaming mode when you are dealing with a large volume of messages via WebSocket only. Couple WebSockets with FME Server’s Notification Service in notification mode when you want to send and receive using other protocols in addition to WebSockets. Or, combine the techniques in hybrid mode when you don’t want to occupy an engine full-time, but still need high-throughput messaging in one direction.

If you’re not sure which mode to use in your web application, or you have any other questions about WebSockets and FME Server, feel free to contact us.