Webhooks and FME Server with Slack Integrations

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2016.x

Introduction

There is a huge benefit to using Webhooks as means of instant notification - if your client needs updates for specific events, creating a Webhook for those events will enable them to receive and disseminate information about that event into their automated workflows. A Webhook allows one system to receive data from another system in real-time. Instead of polling a source system for new data, events in the source system automatically send data to the target system.

Webhooks are HTTP (POST, GET) callbacks that are sent to user-defined URLs in response to triggers or events. The use of Webhooks allow for third party data integration with existing services, such as FME Server!

Using Webhooks is nearly as simple as entering a URL into the address bar of any web browser. The URLs are almost always contained on one line, and there is no requirement to write code!

As an example, imagine that you have a webpage where users can post data. You could setup a Webhook to track new postings. By inserting a URL of your choosing into this Webhook, and when new data is posted, it will use the HTTP POST method to send the data to your URL, typically as a JSON payload. This information can be digested to trigger workflows or propagated to other systems using subsequent Webhooks.

 

FME Server Providing a Webhook

Every topic created in FME Server automatically generates a direct URL that can be used with a Webhook. When an event configured to the direct URL triggers, it will send information to that topic in FME Server. Subscriptions can be configured to monitor a topic and then FME Server can perform additional events, such as delivering data via email or updating a database, based on the protocols you have set up.

(Navigate to this documentation for information on the supported subscriber protocols in FME Server and how to properly configure each.)

Obtaining the direct URLs to trigger topics from FME Server is really easy:

  • Navigate to Topics in the FME Server Web UI (under Manage > Notifications)
  • Choose any existing topic
  • Select Developer Information to reveal the direct URL

W0VcpWDFmETCwqWsLXePSZWSRHxGVP342uDgJ5CN4OuCYxNhEiAsjUMFc55BSyVA0M05yO6K3WYBeGtE0p1J7xxdHPvXBCsjuFRaNLZ5Z9EdyKoHqB-IZzdpIeOsmUOEl_-guPeX

 

FME Server Sending Notifications to a Webhook

The power of FME Server doesn’t stop with enabling third-party apps to send messages to FME Server via Webhooks. In fact, FME Server has the ability to submit data to URLs, which includes Webhooks – this means that FME Server can be used with any third-party services that utilize Webhooks!

In this example, we will be using a rich messaging tool. Through some simple configurations, and without writing any code, FME Server can be configured to send information about successful (or failed) jobs to Slack. This provides an easy and effective way to be informed about and monitor scheduling, automation, or general activity with FME Server.

A workspace can be authored to create and format data to send off to a Webhook. This example workspace pictured below is driven by the HTTPCaller – which is the transformer used to actually submit data to the Webhook.

LhVw3XECQGVjN41m-LhXPHz6Xz-xKRx6ZSxJBxSTGTZyL_FDFYDhH8TNw-eZL3aewBLmpUyvRW5E66tthZgh-IHjxJY5n_-Sprx1Q_aMBMSsdguwdkux941UqtcfaYYfPdgGOi2q


 

Below is an example of FME Server sending to a Webhook. In this example, you will:

  • Configure a Slack Webhook URL as the endpoint for delivering a message
  • Publish an example workspace to FME Server designed for automating Slack messages
  • Create and manage notifications in FME Server for the example workspace
  • Publish and run a workspace to trigger the automated messages

 

Step-by-step Instructions

1. Configure the Third-Party Service | Slack

Slack keeps you organized with purposeful channels. This example requires you to setup a channel to use for the purpose of receiving messages sent by a workspace run by FME Server. After creating a new channel, or selecting an existing one, add an “Incoming WebHooks Integration”. The channel fme_notifications is used in this example.

  • Navigate to Slack Integrations and locate the Incoming WebHooks integration.

Z-x-Fm7x9PKDuy1KvRqqOduQ4LGWyva6t5-v5wRAHFwAEcvKEtAEBvRJqMtSOqCAjuh_H6eVHMo2vAewNmk3gHy-RhIq0WVT_XZh47gxeNu8GB_Kr6vbSycAbUxQeVfkN1gKYAkK

  • Slack will then take you to a configuration page where you can select an existing Channel, or create a new one. Click on Add Incoming WebHooks Integration to retrieve the Webhook URL.

HwxFU1c0XWnSCUie0JTiLxIJSr3lMJLZrHnRfYkwutvVp9Nl9yPE8pta5IfUzoNlHrGguVtpykrD366WSi3iBBdmj6Y0Fn7qVMUmi1q9typdADBknMdlbwn3jZrCHTaWxEEowr7c

The URL will look similar to https://hooks.slack.com/services/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 and will be used as the Slack Webhook URL parameter in the example FME Workspace.

SmqKllS2_axfrrzoRYRmpFr91cc4He-XjadcBVrTfo14sPnrpDLq375-DuTj5ohWIyZL8ZxzURSwRQsrQSsOmJCvjeCxyBDYl7nrdN5tr2mYddm7m6Td0JcgowJ2C6WhM8qmEIVt

2. Configure the Workspace | FME Desktop

This example workspace has been configured to provide a link to the log file within the Slack message for any failed job. For these links to work, the Token Authentication parameter must be valid (see below for an example).

OhLgfc46K6tBO-hf-jU5JWvpcRfjIQYm7AYeK63NLJBMkYyp8k9ZVpXB8K-laLziRl0mWwMu_lOZIRv7S3ojthetcl4T19bOxxzTdwG2VPWHHXyHKXwIFyt6zuXRSadzrgHxN4AM

Slack Webhook URL: <this is obtained from Step 1>

FME Server Hostname: The hostname of your FME Server.

Token Authentication: A valid token can be obtained by selecting Manage Token in the FME Server Web UI (see image below) or by accessing the REST API of your FME Server via http://<host>:<port>/fmerest/ and selecting Get Token in the upper-right corner.

For more information about the Token Service, please see this documentation.

EM118T81GeHUaPEHNRDFmPsFivor1GrMcHYACBpz_1dph8kC5esFgHM0f-CxdfhAgLM9cvpktgGquHQCeYaoT1N0G2MCHy6-d3oA-Ds-j1-tlbvbFP-Wch-_0POdUpjOWqYgBQJJ

 

3. Publish the Workspace | FME Desktop

Publish the example workspace fmeserver_slack_webhook_2016.fmw to your FME Server. This workspace contains all the logic required for Slack to post a message to a channel. Since this workspace will be configured with the Workspace Subscriber, it does not need to be registered with any FME Server Services.

  • Select an existing repository or create a new one. Do not select “Upload data files”.NNxDa3iqzra3Sye9ogkLiaWtxwNYYCB7H8N0AENzTloXCwqlKZQIdfUDrBK0kSdE5DrtNkClgPy7Z7JYJPu9gmAPr6Ahe-bwLCXP7BpT3GzPVDYPdopIciQb8fxMTgtIJEAJ3mw4
  • Unselect all Services. When you click Publish to complete this process, the warning message “No services are currently selected.” can be safely ignored.MUV-6Q5IxlLHSJH7RD-8Om4hrU0vAfupHvmXsWkUi8bs5GXtDlPdArdcXmJx1eJ1AsSWq5izUWqrdejYLP_dMAfTL5LCso415HUHYoFVGp1lqoIJ9VCtM0-0IYpD19hSVnpH-bBv

 

4. Subscribe Workspace to Topic | FME Server Web UI

In order for the example workspace to send messages to Slack, it needs to subscribe to a topic that will be triggered by other workspaces run in FME Server. The topic will notify the Workspace Subscription configured in this step and pass data to the JSON Reader of the workspace.

  • Create the topic named SLACK_WEBHOOK using the FME Server Web UI (Manage > Notifications > Topics).yU9CN75f09GdHzFCZQaG3vGvKbomg23db4OGlV7GUB_y8Yl1I_TwlBEIbbytzTjtxQPp1uAfGr-XBVIK8hr1j8OzzASwzR6rfbTTzqlbu1VmE6aPhcStLTD1pfV5FAYTL9bwdYut
  • Create a new subscription named SLACK_TRIGGER using the FME Server Web UI (Manage > Notifications > Subscriptions).
  • Subscribe to the SLACK_WEBHOOK topic and set the protocol to Workspace.ueJFm4stm1Ge5HR7lfD4887NLc62duoKcfEOWTWamCu-8HW7EnxvewFp7mEox00kA1UfQPnwzIxkThNP229waaux9hMUYcdkcQMjio3OvWuamguFuzJk7qpQWuR076v-xCpcbnLU
  • Select the fmeserver_slack_webhook_2016.fmw workspace.PE32tLAiU9ncFS9xF8iEZcL5xlV9d3hAuPuyOxQWlOBnQjUHQnDmbTMWl34Tyq6yk3uCvnPxNaV_Vpqyu0NefmoPnny4G4r-4e4tgSE3hx9E94zTN5uo5FBB0cbZbhSeVRj3DFaM
  • Set the Notification Message Mapping parameter to point at the JSON Reader.85SQN7tAubSdamsZzrHda5yNOhGElUCcy4TT6H6O5QwO1GWT92oMPMx7kzahKxnL2B78MmbVAgzOTAU_wskl6gnDfS8XvVUg-V7gGvgYAYiZdRQ6-7wol0HP9JjftoQCgL54maNv
  • After making this change, the JSON Reader parameter is disabled and the following message is displayed. This ensures the JSON passed by the SLACK_WEBHOOK topic will be directed to the JSON Reader in the example workspace.wved96idkc4--5SUAegxb8IjVRUgcxOVG9kJO-yc09rsB_BLxK5PA7LJId2Ba69nP7kHGKHXk2U-BoQIFHqknyLyCOZEbg1RhFFBPRDyl0Xp2NGc-ExvCiMKWorHMVF_vdD8SHIZ

 

5. Configure Workspaces to Post to Topic | FME Desktop

The last step is configuring workspaces to post job status notifications to the topic, and thus messages to your Slack channel. This can be achieved in FME Desktop when publishing another workspace to FME Server.

Two workspaces have been included in this example package to test the Slack messages: simple_success.fmw and simple_fail.fmw. Select the Job Submitter Service, edit the properties and set the Notify on Job Completion to the SLACK_WEBHOOK topic.

HYTS4oY5V_J7dWMet5Fs0bVX1c2o1MDfroG5P44JX3Y6tvKzbVvIjmoO3DNOM4jtJVkQ_nm9y8L_XQ-X2Ma-VBoesbWZK9ehLn8eYu0vrbUt9TYteKOXLVGy8U6PExOG4sU9UyOZ

 

6. Verify the Slack Webhook Integration | FME Server Web UI

Run either workspace (simple_success.fmw or simple_fail.fmw) that was just published and registered with the Job Submitter Service. Observe the results in Slack after the job completes.

HyyRR_KV3Hx_ziS-5E6y4oHFHGGzXNvQ3vhHrvc8pIjJlC0vYWKZdRPRSSSnZX6mAXsXrr6JfH9zr4OUea-5hFdqNzyQL6kyM-ETR41kZQyHSokUEze40iHJ1ljOFRMBzyqYDeII

 

Testing and Debugging Webhooks

Verifying the Webhook Output

RequestBin provides a URL that can used in place of the Slack Webhook URL. This service can be used to inspect exactly what FME Server is sending.

 

Providing Sample Webhook Input

Hurl generates HTTP requests that can be directed towards Webhook URLs. Instead of using FME to create and send the JSON message, this tool can be used to send your own message to any third-party service. This is particularly useful for testing scenarios where FME Server is providing the Webhook.

 

Verifying the Webhook Input

RequestBincan also be used to see what other third-party services are providing. FME Server topic monitoring can show the message, but not all of the HTTP details; RequestBin can be used to further debug each request.

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.