Making Weather Maps Using the GeoNames Weather Web Service

Introduction

The GeoNames database is a free, crowdsourced geospatial resource of over 12 million unique features located across the entire globe, sorted into nine feature classes. These features are readily accessible, via several web service endpoints, to those with a (free) valid user account. This article provides a demo of the Weather web service offered by GeoNames, and shows a use case for the JSONExploder and JSONExtractor transformers. 

Requirements

As implied above, you must sign up for a free GeoNames user account before using these web services. Once you submit your request for a new user account, you must confirm your user account using the link sent to the email address you used at account registration. Next, login to GeoNames using your newly verified account credentials, click on your user name at top right, and enable the free web services at the bottom of the user profile page. Once enabled, the web service APIs become available for your use.  

Step-by-Step Instructions

The GeoNames web service of interest for this demo is the Weather Service. We will tailor the example shown under the Weather Stations with most recent Weather Observation section of the Weather service documentation to retrieve the most recent observations from weather stations located within the continental United States. The retrieved weather data will be used to build a series of contours, based on temperature values rather than elevation, to visualize the most current temperature readings from across the continental United States. 

Please download the attached template, WeatherMap_2022_1.fmwt, open it in FME Workbench, and follow the article. The workspace contains numbered bookmarks matching each step described below.
 

1. Read In the Bounding Box Data
 The first step of this process is to supply a bounding box area feature, which will restrict the results of the weather service API call to a focal area. Essentially, this bounding box dataset serves the same purpose as a spatial filter. In this demo, the bounding box is an area feature representing the continental United States. This feature can be replaced with any 2D bounding box feature as required, including a polygon derived from a Creator transformer.

2.  Create a Linear Feature from the Bounding Box Polygon
The bounding box is used in two steps of the demo workspace. In this step, the bounding box geometry is converted from a polygon to a linear feature using the GeometryCoercer transformer. The end result of this operation is a linear outline of the original bounding box, which will be used in a later step to extract contours.  

3.  Build the URL to Pass to the GeoNames Weather Service
In this step, the irregular shape of the bounding box feature is turned into a square feature by the BoundingBoxReplacer transformer. This new bounding box square is passed through the Buffer transformer, which applies a buffer all around the input to ensure the entire area of interest is captured. Next, the BoundsExtractor transformer extracts the minimum and maximum x-, y-, and z- coordinate values, and assigns these as new attributes of the buffered bounding box. 

These new attributes are passed into the Concatenator transformer as north, east, south, and west parameters used in building the URL for the HTTPCaller transformer. The full URL can be viewed by selecting the gear icon of the Concatenator, and viewing the Concatenated Result in the parameters dialog that opens.



A few key things to note about this URL:

• The GeoNames Weather API endpoint is http://api.geonames.org/weatherJSON;
• The extracted x- and y- min / max values are included in the URL as north, south, east, and west parameters;
• A limit on the number of records returned has been set at 500. This limit can be adjusted as needed;
• The required username parameter is derived from a published parameter prompted at runtime. Make sure your workspace is set to Prompt for User Parameters (under the Run tab), and fill in the GeoNames username that you set for your GeoNames user account.

The concatenated URL is stored as the new _url attribute of the buffered bounding box feature.

4.  Send the URL as an HTTP GET Request to the GeoNames Weather Service
Use the HTTPCaller transformer to send the GET request using the _url attribute created in the previous step.


 
As you may have noticed, the API endpoint for this web service finishes with the text JSON. This means that the results of an API call to this web service are sent back in JSON format.  This demo saves this JSON response as an attribute called  _url_contents.

5.  Extract Attributes from the JSON Response Body
The attribute containing the JSON response (_url_contents) is, by default, a list attribute since FME handles JSON-formatted attributes as lists. You can see the contents of the JSON response by partially running the workspace to the HTTPCaller, viewing the cached results in Visual Preview, and then selecting the ellipses next to the _url_contents value.



Currently, the _url_contents attribute value is challenging to interpret. We will use the JSONFragmenter transformer to extract information from this JSON list attribute, making this information more accessible for further processing. This transformer will split the attribute into separate features based on the value of the JSON Query parameter. 

Click on the gear icon of this transformer to see its parameters. In particular, note the value of the JSON Query parameter: 

json["weatherObservations"][*]

This query specifies that all elements (denoted by the * wildcard) one level down from the “weatherObservations” element should be split out as separate features. In the example JSON text below, individual elements within the “weatherObservations” element are denoted by { }. The first element that is to become a separate FME feature is shown in bold; the next feature is underlined.  
      

"weatherObservations":[{"clouds":"n/a","weatherCondition":"n/a","observation":"K1P1 190540Z AUTO 00000KT 10SM CLR 01/00 A3004 RMKAO2","windDirection":0,"ICAO":"K1P1","lng":-71.75,"temperature":"1","dewPoint":"0","windSpeed":"00","humidity":93,"stationName":"Plymouth,Plymouth Municipal Airport","datetime":"2008-04-19 07:40:00","lat":43.7833333333333},{"clouds":"n/a","weatherCondition":"n/a","observation":"KAFP 190540Z AUTO 21004KT 10SM CLR 16/07 A3002 RMKAO2","windDirection":210,"ICAO":"KAFP","lng":-80.0677777777778,"temperature":"16","dewPoint":"7","windSpeed":"04","humidity":55,"stationName":"Wadesboro, Anson County Airport","datetime":"2008-04-19  07:40:00","lat":35.0169444444444}...

In the demo workspace, running a partial translation to the JSONFragmenter will produce 500 new features, corresponding to 500 individual weather observations. You may recall that the Concatenator transformer’s maxRows parameter was set to 500, meaning we would only expect to see 500 individual weather observations returned from the HTTPCaller. So our results are as expected here.  

The consecutive JSONExtractor transformers create new feature attributes from each feature’s corresponding _url_contents list attribute. For example, the JSON Query parameter of the JSONExtractor_3 transformer is set to json["temperature"], and the Target Attribute is set to temperature. This setup will create an attribute named temperature for each weather observation feature, and populate its value with the temperature recorded at the associated weather station. Click the gear icon for each JSONExtractor to examine each JSON Query. Of particular note are the extracted latitude and longitude values for each weather observation. These coordinate values, corresponding to the location of a weather station, will be used to generate contours in a later step. 

6.  Format the Date and Filter for Weather Observations No Older Than One Day
In this demo, we are only interested in visualizing the most recent temperature observations from weather stations across the continental United States. Thus, we should filter out any weather observation records that are more than one day old. First, the DateFormatter and TimeStamper transformers format the extracted datetime attribute so that other transformers and FME functions can read it.. The AttributeCreator and DateFormatter_2 transformers create and populate a _destDate attribute that holds yesterday’s date (relative to the system date). Finally, the Tester transformer filters out any observations that are more than one day old by comparing the original observation date, datetime, to _destdate.  

7.  Create Points for Filtered Weather Observations
The VertexCreator transformer receives the filtered weather observations and generates point data from each observation’s latitude and longitude attributes. Since each observation derives from a weather station, the resulting output displays point locations of weather stations. The points generated are 3D points, with the temperature attribute set as the z-value. 



These points are used in two separate streams of processing: one stream to generate temperature-based labels for weather stations (Step 8), and the other stream to generate contours (Step 9).

8.  Generate Labels for Weather Stations Based on Temperature
Temperature values for each station are rounded to the nearest degree Celsius by the AttributeRounder transformer. Rounded values are then split by the Tester transformer into two streams: one stream for features with temperature values at or above 0, and a stream for those with temperature values below 0. Positive temperatures are prefixed with a + and then saved to a new _label attribute by the Concatenator transformer, while a new _label attribute is created by the AttributeCopier and applied to the negative temperature stream. Finally, a LabelPointReplacer transformer is used to create label points, using the _label attribute of each stream of input features as the label value.

9.  Create DEM Points From Temperature Z-Values, then Generate and Smooth Contours
The points derived from the VertexCreator transformer are passed to a SurfaceModeller transformer to generate a series of DEM points based on temperature values rather than elevation values. These DEM points are passed through a CoordinateFetcher transformer to retrieve their z-values, and then through a Tester transformer to filter out any undefined z-values (a.k.a. qNaN values). 

These filtered DEM points then enter the ContourGenerator transformer. This transformer builds lines of equal temperature, called isotherms, from the input DEM points. The output contour lines are, however, jagged and require smoothing; this is accomplished by the Generalizer transformer. This transformer provides many smoothing algorithm options. For this translation, we will use the McMaster(Smooth) algorithm.    



10. Extract and Process the Generalized Contours
The bounding box feature is passed to the Intersector transformer, along with the generalized contours, to extract only those contours which fall within its bounds (for this demo, within the continental United States). Next, the AreaBuilder transformer uses the extracted contours to build isotherm area features, or areas of equal temperature. This transformer also creates a list attribute, called weatherObs, containing all of the attributes of the input contour lines, including the temperature z-values. 

The ListHistogrammer transformer uses the weatherObs attribute to calculate how often each temperature value occurs within the contour area dataset and stores it as a list attribute called _histogram. The ListExploder transformer generates the new attributes count and value for each area feature, by exploding the _histogram list attribute. These two new attributes are used to create a color ramp for visualizing the temperature variations across the continental United States.


11.  Build and Apply a Color Ramp for the Isotherm Area Features
A color ramp is helpful for visualizing the gradual changes in a continuous phenomenon across an area of interest. A suitable color ramp will use colors that viewers intuitively associate with the phenomenon of interest. In this step, we will build a red-blue color ramp to show the gradual shifts in temperature across the continental United States. As red is a color often associated with higher temperatures, and blue is often associated with lower temperatures, this color ramp will progress from red to blue as temperature values shift from higher to lower. 

First, the StatisticsCalculator transformer computes the minimum, maximum, and range values for the input isotherm areas based on the value attribute created by the ListExploder in Step 10. The two ExpressionEvaluator transformers calculate values for _red and _blue attributes, to be used to set the RGB fill value for each area. The Concatenator transformer creates a new attribute, fme_color, which contains the _red and _blue values of each isotherm area separated by a constant value (for green). 



Lastly, an AttributeCopier transformer copies these RGB-formatted values for each area to the fme_fill_color format attribute. This attribute, when properly formatted as <redvalue>,<greenvalue>,<bluevalue>, controls the fill color that FME applies to area features. In this demo, the RGB values passed to the fme_fill_color attribute all specify colors within the red-blue color range. The result is a series of isotherm areas that gradually change from red, through violet, and to blue as temperatures change from higher values to lower values.


12. Clean Up the Temporary Attributes and Write Out Isotherm Areas
This last step uses the AttributeRemover transformer to remove all the extraneous or unnecessary attributes from the area isotherm features except for fme_color, fme_fill_color, and value. Next, the AttributeRenamer transformer changes the name of the value attribute to temperature, better indicating what phenomenon this attribute represents. Finally, the isotherm area features are written in a format of your choosing. The demo shows a Logger transformer connected to the AttributeRenamer. Replace this Logger with a writer of your choosing.

The resulting isotherm areas should look similar to the following, keeping in mind that the pattern you see in your results will likely differ based on temporal differences in temperature readings obtained from the HTTPCaller. Notice how there are still areas included in the isotherms which fall outside the bounds of the bounding box. These areas can be removed with a Clipper transformer. 


 

Additional Considerations

Using this template workspace, you can easily replace the bounding box data with an input polygon of your area of interest. Another change to consider is the inclusion of a method for handling temperature outliers. For example, it would be highly unusual to receive an August temperature reading of 30° Celsius from a weather station in the Arctic region of Canada. If this temperature reading is included in DEM point generation, the resulting contours may contain an increased degree of error. To reduce this error, a Tester transformer could be used to exclude identified outliers before DEM points are generated. Finally, suppose a visualization of temperature shifts over a period of time is required; an ImageRasterizer transformer could be used to generate a series of rasters depicting temperature change over time. 

Consider this demo workspace as a starting point for exploring and utilizing the variety of web services available through GeoNames. Although this demo is specific to the GeoNames Weather web service, you can use the same general process to form an HTTP GET request for the variety of other web services that GeoNames provides. A look through the GeoNames web service documentation will provide details of the different service endpoints and expected request parameters.