How to Make API Calls with the FeatureReader, JSONExtractor, or HTTPCaller

Introduction

This article will explore making API calls with FME. For this example, we will use a hardcoded request URL to retrieve air quality and pollution data from AirVisual’s REST API. Since readers do not have input ports, we could not make a dynamic API call using attribute values from other datasets. We will explore several transformers to create API workflows that offer greater flexibility and functionality.

The following tutorial will provide instructions on accessing AirVisual’s API using three different transformers, all producing a KML file. This tutorial compares some transformers that allow users to access APIs and shows you how to set them up.

We will make API calls through the FeatureReader, the JSONExtractor, and the HTTPCaller. We will go through each approach from the simplest to the most complex, describing the differences between each transformer. The usability of each method will depend on the response format and the API requirements. AirVisual’s API can be accessed from each of these transformers because:

1. The response format is JSON
2. The only request method AirVisual offers is GET
3. There are no header or body parameters to fill out
4. Authentication is done through a token attached to the request URL

A quick comparison of the three methods is listed in the table below and further explained in each of the parts. 

To access the AirVisual API, you’ll need a free account and an API key. If you already have an account and API key from AirVisual, please continue to Part 1 below. 

Please create a free personal account on AirVisual. Once you have created an account, go to your dashboard, select the Air Quality API link, and click the “Create new API key” button. Please note the exact steps stated here may not be accurately reflected on AirVisual’s website if they make changes. 

If you would like to follow along using the coordinates of Safe Software’s head office, please use the following request URL:

http://api.airvisual.com/v2/nearest_city?lat=49.178&lon=-122.842&key=<YOUR API KEY>

Step-by-step Instructions

Part 1: Make API calls using a FeatureReader

The FeatureReader is a transformer that acts similarly to a reader but is incorporated into the workflow mid-translation. This means that there is an input port that requires initiation for the feature to be read. The FeatureReader can read JSON and output values into an attribute with a specified coordinate system. The main difference between a FeatureReader and a reader is the presence of the input port, which allows for reading data mid-translation. 

The best use cases for this API approach would include situations where:

• The API request is GET
• The header and body parameters do not need to be modified
• Every value in the response will be used
• Dynamic request URLs are used
• API authentication is required

1. Start FME Workbench 

Select Blank Workspace from the start page to open a blank canvas.

2. Add a Creator Transformer

To add a transformer to your canvas, directly type the transformer's name or select the transformer button in the menu bar. Add a Creator transformer to the workspace. 

The parameters can be left as default as this transformer is only used to initiate the workspace without having to input data. To open the parameter dialog, double-click on the transformer or select the cog in the mini menu.

This tutorial will use a Creator to ensure that each method is comparable and consistent. If you wanted to create a dynamic request URL, you could replace the Creator with an existing workflow containing valid GPS coordinates and an API key, as these are the only three elements of the request URL needed.

3. Add a FeatureReader and Configure the API Call to AirVisual

Connect your Creator to the Initiator port of a FeatureReader.

Double-click the FeatureReader transformer to open the parameters. In the parameters of the FeatureReader transformer, set the Format to JSON (JavaScript Object Notation) and the Coordinate System to LL84. This will ensure that the latitude and longitude values are properly interpreted when we create a point feature.

The Dataset will be the request URL with your API key substituted where <YOUR API KEY> appears.

http://api.airvisual.com/v2/nearest_city?lat=49.178&lon=-122.842&key=<YOUR API KEY>

4. Run Workspace to Test the API 

Run the workspace with Feature Caching enabled and view your output in Visual Preview.

Select the JSONFeature output port on the FeatureReader and open the Feature Information window in Visual Preview (Ctrl + Alt + F) to view a list of the attributes. Like a JSON reader, every value from the response has been flattened into an attribute.

5. Add a VertexCreator to Create a Point Feature From the Coordinate Array

Connect a VertexCreator to the JSONFeature output port on the FeatureReader.

AirVisual returns the coordinates of the nearest monitoring station in a JSON array. Since these values are read into FME as list attributes, we need to specify which value to use by its index, which is indicated between the curly brackets { }. Note that JSON indexes begin at 0 and not 1.

To select the value, click the drop-down arrow to the right of the X Value field, expand Attribute Value, then select dat.location.coordinates{}, which will open another dialog box where you can select which array element you want. For this tutorial, X is 0, and Y is 1. 

After selecting the coordinate values for X and Y, the VertexCreator Parameters should look like the following image. Click OK.

6. Refine the Attributes

Add an AttributeManager to the canvas after the VertexCreator. Connect the AttributeManager to the VertexCreator, and open the parameters. 

In the Output Attribute column, rename the following attributes:

• data.city -> City
• data.state -> Province
• data.country -> Country
• data.current.weather.tp -> Temperature (C)
• data.current.weather.hu -> Humidity (%)
• data.current.weather.ws -> Wind Speed (m/s)
• data.current.pollution.aqius -> AQI (US)
• data.current.pollution.ts -> Date/Time

Use the black arrows below the table to arrange the order of your output attributes. To remove extra attributes, select their rows using Ctrl or Shift-click, then click the minus button below the table. Remove the remaining attributes and click OK.

7. Add a Google KML Writer 

Click the Add Writer button on the menu bar and select the OGC/Google KML for the Format. Select the destination where you want the kml file saved. For this example, the dataset has been named  FeatureReaderOutput.kml. 

Click OK, and the Feature Type dialog will open. Rename the Feature Type Name to MonitoringStation_FeatureReader and click OK.

Connect the AttributeManager Output port to the MonitoringStation_FeatureReader KML writer feature type. Your canvas will look like the one below.

8. Run the Workspace and Open the .kml File in Google Earth

Run the workspace and check the Translation Log for errors. Most warnings can be ignored. After the workspace has been successfully run, click on the MonitoringStation_FeatureReader writer to open the mini menu and select the folder icon. This will open a file browser where the .kml file has been saved. 

Double-click on the file in your browser to open Google Earth. If you do not have Google Earth, you can view the data in Visual Preview. 

The view should automatically zoom to the location of the nearest monitoring station. Click on the pin to view the attribute data.

Part 2: Make API Calls Using a JSONExtractor

FME Form has a variety of JSON-specific transformers. Most of them allow the JSON input source to be a URL, meaning these tools are perfect for reading, parsing, and validating JSON data obtained from the web. The JSONExtractor provides the best parsing capability, allowing users to easily create feature attributes from the nested values buried in JSON format. 

Unlike the JSON Reader or the FeatureReader, the JSONExtractor will not automatically create attributes out of every value in the JSON text. Users must specify specific output attributes by selecting them from a JSON query. The benefit of using this transformer is that you will only get what you need from the API response. This feature limits the amount of attribute handling later in the workspace and is especially useful when an API response returns many attribute values. 

The best use cases for this API approach would include situations where:

• Only select attributes are needed from the API response
• The API request method is GET
• The header and body parameters do not need to be modified
• API Authentication is a token in the URL
• The API response is JSON

1. Start a Blank Workspace

You can start a blank workspace or continue in the same workspace from Part 1. If you start a new workspace, add a Creator to the canvas.

2. Add a JSONExtractor and Configure the Parameters

Add a JSONExtractor to the canvas and connect it to the Creator. Double-click on the JSONExtractor transformer to open the parameters. Change the Input Source to File/URL and copy the request URL into the File/URL parameter, substituting your AirVisual key where <YOUR API KEY> appears.

http://api.airvisual.com/v2/nearest_city?lat=49.178&lon=-122.842&key=<YOUR API KEY>

3. Create Feature Attributes by Extracting JSON Queries

JSON queries (JavaScript property accessorsin this case) describe the path to your attribute by fragmenting your data into its nested levels. If the value you need is deeply nested, your JSON query will be longer. Rather than manually typing in your query, we will select each one from the JSON tree.

The AirVisual API has a rate limit of 5 calls/minute, and setting up the query using this method uses a call. Try setting a few attributes this way, but to finish this step, all the queries will be posted below this step so you can copy and paste them.

First, fill out the Target Attribute column in the JSONExtractor parameters. This includes the eight attribute names we’ve previously used in Part 1, plus Latitude and Longitude.

• City
• Province
• Country
• Temperature (C)
• Humidity (%)
• Wind Speed (m/s)
• AQI (US)
• Date/Time
• Longitude
• Latitude

Next, click the drop-down arrow beside the blank JSON Query box and select Select JSON Query to open the JSON tree. Use the arrows to expand the JSON until you find the right attributes. 

For example, to find “city,” we only need to expand “data”. When you click OK, the blank space will automatically populate with your JSON query.

Some attributes, such as city, state, and country, are easily identified because they are well-labeled and not nested deeply. Other values are abbreviated, and you can reference the AirVisual API documentation for the abbreviated values to decode your response.

Other attributes, such as the coordinate values, will be returned in an array. First, select <array values> as your JSON query for latitude and longitude to specify which array value you need. Once populated under the JSON Query column, manually change the asterisk to 0 for longitude and 1 for latitude.

4. Add a VertexCreator to Create a Point Feature from the Coordinate Values

Unlike the JSON reader and the FeatureReader, the Latitude and Longitude attributes can be directly inputted into the VertexCreator. We do not need to worry about arrays because we separated the coordinates in the previous step. Set the X Value as the Longitude attribute and the Y Value as the Latitude attribute.

5. Add a CoordinateSystemSetter to Set the Coordinates

The JSONExtractor does not offer an option to set a coordinate system. We must use a CoordinateSystemSetter to specify how the X and Y values will be read. Set the coordinate system to LL84.

6. Refine the Attributes

Add an AttributeManager and open the parameters. Notice there are only the ten original attributes you extracted, plus the _creation_instance, which is a product of the Creator. Remove the _creation_instance, Latitude, and Longitude attributes using the drop-down menu in the cells under the Action column.

7. Add a Google KML Writer 

Click the Add Writer button in the menu bar and select the OGC/Google KML for the Format. Select the destination where you want the kml file saved. For this example, the dataset is named JSONExtractorOutput.kml.

Click OK, and the Feature Type dialog will open. Rename the Feature Type Name to MonitoringStation_JSONExtractor and click OK.

Connect the AttributeManager Output port to the KML writer feature type. Your canvas will look like the one below.

8. Run the Workspace and Open the .kml File in Google Earth

Run the workspace and check the Translation Log for errors. Most warnings can be ignored. After the workspace has been successfully run, click on the MonitoringStation_JSONExtractor writer to open the mini menu and select the folder icon. This will open a file browser where you choose to save the .kml file. 

Double-click on the file in your browser to open Google Earth. If you do not have Google Earth, you can view the data in Visual Preview. 

The view should automatically zoom to the location of the nearest monitoring station. Click on the pin to view the attribute data.

Part 3: Make API Calls Using an HTMLCaller

The HTTPCaller is the most robust transformer used to call APIs. It is designed to handle thousands of APIs without coding. It accesses a request URL using a specified HTTP/HTTPS method, with the option to authenticate by username/password, named connection, headers, or uploaded bodies. 

In the HTTPCaller transformer, users can modify the query string parameters and headers and use additional methods such as PUT, POST, PATCH, or DELETE. Additionally, the HTTPCaller allows users to upload a body where the previous examples did not have this parameter. Responses can be saved to an attribute or file while providing error and status codes in the response header. 

The use cases for this API approach would include situations where:

• The API requires authentication methods such as Basic, Digest, or OAuth2
• HTTP client options need to be modified, such as timeouts or saving cookies
• The request URL is long or unreadable
• HTTP methods other than GET are needed
• The response is not JSON/XML
• There are required headers

1. Start a Blank Workspace

Start a blank workspace, or continue in the same workspace from Part 2. If starting a new workspace, add a Creator to the canvas.

2. Add an HTTPCaller and Set up the Call to AirVisual

Add an HTTPCaller to the canvas and connect it to the Creator. Open the HTTPCaller parameters.

To improve readability and ease of troubleshooting, components of the URL can be broken down using query string parameters

Enter the following URL as the Request URL with the HTTP Method set to GET.

http://api.airvisual.com/v2/nearest_city?

The HTTPCaller lets you add URL query strings as transformer parameters. For example, we can add the latitude and longitude coordinates and the API key as query string parameters. 

These parameters could be set as User Parameters to make this workspace run for multiple API keys or to select the coordinates at run time. For more information on User Parameters, see our tutorial on Published Parameters in FME Form. In this example, we will just type in the values for the parameters:

Run the workspace with Feature Caching enabled and view the response body in Visual Preview. Double-click in the _response_body column to see the whole response body.  If you used the provided coordinates, the _response_body attribute should return something similar to this:

{"status":"success","data":{"city":"New Westminster","state":"British Columbia","country":"Canada","location":{"type":"Point","coordinates":[-122.91091999999999,49.20678]},"current":{"pollution":{"ts":"2025-01-13T19:00:00.000Z","aqius":45,"mainus":"p2","aqicn":12,"maincn":"p2"},"weather":{"ts":"2025-01-13T19:00:00.000Z","tp":3,"pr":1035,"hu":87,"ws":1.54,"wd":110,"ic":"04d"}}}} 

3. Format the Response to Improve Readability 

We have to manually create queries because the JSONExtractor does not display the JSON tree when reading a JSON document from an attribute. In the example in Part 2, the JSON response was coming from a URL and was already formatted. We must format our JSON to read its structure when building our queries.

We can format our JSON response with a JSONFormatter. Connect the HTTPCaller output to a JSONFormatter transformer input. Open the properties of the JSONFormatter and set the JSON Document as your _response_body attribute. Leave the remaining parameters as default and click OK. 

Run the workspace with Feature Caching enabled and view the reformatted JSON in Visual Preview. A new column called _formatted should be created, and this is where the output will go. You should see something similar to this:

{ 
   "status" : "success", 
   "data" : { 
      "city" : "New Westminster", 
      "state" : "British Columbia", 
      "country" : "Canada", 
      "location" : { 
         "type" : "Point", 
         "coordinates" : [ -122.91091999999999, 49.20678 ] 
      }, 
      "current" : { 
         "pollution" : { 
            "ts" : "2025-01-13T19:00:00.000Z", 
            "aqius" : 45, 
            "mainus" : "p2", 
            "aqicn" : 12, 
            "maincn" : "p2" 
         }, 
         "weather" : { 
            "ts" : "2025-01-13T19:00:00.000Z", 
            "tp" : 3, 
            "pr" : 1035, 
            "hu" : 87, 
            "ws" : 1.54, 
            "wd" : 110, 
            "ic" : "04d" 
         } 
      } 
   } 
}

The JSON text is formatted with indentation and spaces to improve readability. The nested structure is clearly laid out, making it easier to construct JSON queries. For example, if we wanted to query the main pollutant according to the Air Quality Index of the United States, we could see that “mainus” is nested under “pollution” which is nested under “current” which is nested under “data”. Using this logic, we can create a JSON query:

json["data"]["current"]["pollution"]["mainus"]

To query values in an array, such as the coordinates of the monitoring station, you would use the same logic, but to specify an object in the array, a place number has to be indicated. For example, the JSON query for longitude is:

json["data"]["location"]["coordinates"][0]

We use zero to indicate longitude because it is the first value in the array. If we wanted to specify latitude, we would instead use 1 as the place number between the square brackets. This array resembles how we extracted the latitude and longitude information from the JSON response in Part 2.

4. Add a JSONExtractor Transformer and Extract the Attributes from the JSON Response Body

Connect the JSONExtractor input to the output of the JSONFormatter transformer. Open the JSONExtractor properties and set the JSON Document as _response_body or _formatted. The formatting of the JSON body does not impact how the JSON queries are formed or executed. We formatted the response so we could read it easily and determine how to extract the data we wanted. 

Fill in the Target Attribute column using the name of each output attribute as follows:

• City
• Province
• Country
• Temperature (C)
• Humidity (%)
• Wind Speed (m/s)
• AQI (US)
• Date/Time
• Longitude
• Latitude

Under the JSON Query column, populate each cell with its corresponding query using the logic described in the previous step. Opening the reformatted response body in another window, such as Notepad, may be helpful. Refer to the API documentation to decode the abbreviated values in the response body.

Once the attributes and queries are filled in, click OK.

5. Add a VertexCreator and Create a Point Feature

The Latitude and Longitude attributes can be directly inputted into the VertexCreator because we have already separated the coordinates in the previous step with the JSON query. Set the X Value to the Longitude attribute and the Y Value to the Latitude attribute.

6. Add a CoordinateSystemSetter 

The HTTPCaller does not have the option to set a coordinate system, so we have to use a CoordinateSystemSetter to specify how the X and Y values will be read. Set the coordinate system to LL84.

7. Add an AttributeManager and Remove Unnecessary Attributes

Add an AttributeManager and open its parameters. Use the drop-down menu under the Action column to select Remove to remove _creation_instance, _response_body, _http_status_code, Longitude, Latitude, and _formatted.

8. Add a Google KML Writer

Click the Add Writer button in the menu bar and select the OGC/Google KML for the format. Select the destination where you want the kml file saved. For this example, the dataset has been named HTTPCallerOutput.kml. 

Click OK, and a Feature Type dialog will open. Rename the Feature Type Name to MonitoringStation_HTTPCaller and click OK. 

Connect the AttributeManager Output port to the KML writer feature type. Your canvas will look like the one below.

9. Run the workspace and Open the .kml File in Google Earth

Run the workspace and check the Translation Log for errors. Most warnings can be ignored. After the workspace has been successfully run, click on the MonitoringStation_HTTPCaller writer to open the mini menu and select the folder icon. This will open a file browser where you choose to save the .kml file. 

Double-click on the file in your browser to open Google Earth. If you do not have Google Earth, you can view the data in Visual Preview. 

The view should automatically zoom to the location of the nearest monitoring station. Click on the pin to view the attribute data.

Conclusion

The output .kml file contains eight attributes parsed, renamed, and reordered using different transformers in FME. When considering the final product, the most efficient method was the JSONExtractor because it allowed us to filter out select values, limiting the amount of attribute handling later in the workflow. Furthermore, the JSON tree allowed us to easily create queries without previous knowledge of the JSON format. If we wanted every response value to be an output attribute, then the FeatureReader would be the best choice. 

These two transformers are limited in the types of API calls they can make. Authentication can only be done by an API key in the URL, and only GET methods can be used. The HTTPCaller transformer is more robust and has options for more parameters and advanced authentication types. 

Although the HTTPCaller method contained a JSONExtractor, we had to manually enter queries because the JSON tree is not an available feature when the input data source is dynamic. With this method, it is necessary to have a working knowledge of both APIs and JSON because there are more manual inputs.

The HTTPCaller offers the most user control in accessing an API, but for the purposes of this article, these features were not highlighted. For an example of leveraging the features of the HTTPCaller,  see Download Files from a REST API Using FME. You will learn how to access the Movebank REST API, which returns a .csv file containing live animal movement data.

Additional Resources

Webinar: How to Connect to Any REST API with No-Code
Article: Getting Started with the Google Calendar API
Article: Getting Started with JSON