How to Create a Heatmap Using the RCaller

Introduction

Although heatmaps are commonly referred to as hotspot maps, they are different. A heatmap identifies clusters of data without taking any other attributes into consideration and is not statistically significant. They are good for quick visualizations of data clusters. This tutorial will show you how to create a heatmap using R, the ggmap package, and FME. To get started with FME and R please read through the Tutorial: Getting Started with the R Caller as well as the RCaller Transformer documentation.

Requirements

• R Installed
• R Studio or any R console

About ggmap

ggmap is an R package that creates spatial visualizations on top of static maps. If you are already familiar with the ggplot2 package the functions are the same.

Update: Stamen Maps shutdown

The example below originally used Stamen Maps for the base map, since ggmap is presently limited to Stamen Maps or Google Maps. Stamen Maps is being shut down on October 31, 2023, with its maps now being hosted on Stadia. The example has been updated to use Stadia instead of Stamen Maps. 

At the time of this update, ggmap had not been officially updated to connect to Stadia, but Stadia is supplying a fork with this capability. We expect this fork to be merged into the official ggmap in the future.

If you find that the RCaller fails using the instructions below, please follow the instructions below to update your ggmap installation to the Stadia fork:
https://docs.stadiamaps.com/guides/migrating-from-stamen-map-tiles/#ggmap

Step-by-step Instructions

Please ensure that you have R installed on your computer before beginning. We will be using R Studio to install the ggmap package but you can use any R console.

1. Open R Studio (or Another R Console)

Open R Studio and create a new R Script. We will need to install the ggmap package before we can continue in FME.

In the new R script type and run:

install.packages("ggmap")

It is also a good idea to test your script in R console first before using the RCaller in FME. The console gives better debugging messages and allows you to visually see what is happening with your data in real time. For this tutorial, we will jump right into FME Workbench since the script has already been debugged. If you are using your own data, use an R console first.

2. Get Stadia API Key
Stadia requires an API key to use their base maps in ggmap. The API key can be obtained by signing up for a free account at stadiamaps.com.

After creating a Stadia account, log in and create a Property on the welcome page. In the Authentication Configuration for the new Property, choose +Add API Key. The new key will be displayed and can be copied for use in the RCaller

3. Read in Crimes CSV Data

In FME Workbench, open a blank workspace and add a CSV reader. Browse to the crimes2012-17.csv dataset. Set the Coord. System to LL84. In the Parameters, set the Attribute Definition to Manual then change the Type for X and Y to x_coordinate and y_coordinate, respectively and click OK.

This dataset has already been cleaned up, if you are using your own data, ensure that the data is in LL84 and that there are columns for Latitude and Longitude or X and Y.

Input CSV Crime data

4. Create User Parameters

We will need two user parameters to help select what type of crime we want to analyze as well as where to save our final image file.

In the Navigation window, right-click on User Parameter and select Manage User Parameters. Then click the + button to add the following parameter:

Expand the Choice Configuration section, then click Import at the bottom of the Edit Choices List. This will open up the Import Wizard. Select CSV as the Format and the crimes2012-17.csv file as the Dataset, click Next. Then select Attribute Values for Import From and ensure CSV is selected, then click Continue. Select TYPE for the Choice List then click Import. A list of crime types will populate the Edit Choices List. Click Ok to save the parameter.

TypeOfCrime user parameter creation

Create a new text parameter and set the following:

ComUserName user parameter creation

Create a new Password parameter and set the following:

ApiKey user parameter creation

5. Add an RCaller

Now that we’ve created our two published parameters, we can add our R script. Add an RCaller to the canvas and connect it to the CSV reader.

In the parameters, change the Input Table from CSV to crime; you might have to reconnect the RCaller to the Reader.

Change the table in the RCaller from CSV to crime, then reconnect the Reader.

Now paste in the following script into the R Script area:

library(ggmap)

crime <- subset(crime, TYPE == "$(TypeOfCrime)")

bounding_box <- c(bottom = 49.2,
                  top    = 49.4,
                  left   = -123.4,
                  right  = -123.0) 

register_stadiamaps("$(ApiKey)")
HeatMap <- get_stadiamap(bounding_box, maptype="stamen_toner", zoom=10)

ggmap(HeatMap) + geom_point(aes(x=X, y=Y), data=crime, alpha=.5, color="red", size=3, shape=".") +
  stat_density2d(data=crime,
                 aes(x = X, y = Y, fill= ..level.., alpha= 0.5), size = 0.01,
                 bins = 30, geom = "polygon") + scale_fill_gradient(low = "green", high = "red") +
  facet_wrap(~YEAR) +
  ggtitle("$(TypeOfCrime)")

  ggsave("C:/Users/$(ComUserName)/Documents/$(TypeOfCrime)From2012-2017.png")

fmeOutput<-data.frame(1)

If you don't want to save to the documents folder, you can change the path. Just be aware the direction of the slashes.

6. Run the Workspace

Now run the workspace with prompt enabled to enter your parameter values. Then open up the folder where you saved the png.

Final output, heatmap for Theft of Bicycle

R Script Breakdown

The following is a breakdown of the FME Specific for the Heatmap R Script. For a more detailed look at how to use ggmap, see the ggmap package documentation.

This subsets the crime data table by type of crime.

crime <- subset(crime, TYPE == "$(TypeOfCrime)")

This creates the bounding box of the area of interest in latitude/longitude where top and bottom are latitude and right and left are longitude. Can easily be turned into a Published Parameter, see the completed workspace.

bounding_box <- c(bottom  = 49.2,
                 top     = 49.4,
                 left    = -123.2,
                 right   = -123.0)

Sets the base map to Stadia maps, can change the maptype to stamen_toner, stamen_terrain or stamen_watercolor.

register_stadiamaps("$(ApiKey)")
HeatMap <- get_stadiamap(bounding_box, maptype="stamen_toner", zoom=$(Zoom))

Anywhere you see x= and y= these values reflect the attribute names for your Latitude and Longitude columns. The data= will reflect what you named the table in the RCaller. Since we already subsetted the data, we can just call crime.

geom_point(aes(x=X, y=Y), data=crime, alpha=.5, color="red", size=3, shape=".")

Delete this line if you don’t want to facet. Alternatively, you can add an additional attribute to facet by two attributes: just change the facet from wrap to grid, i.e. facet_grid(NEIGHBOURHOOD~YEAR):

facet_wrap(~YEAR)

ggtitle is the title at the top of your image. You can use the user parameters or constant values to change this:

ggtitle("$(TypeOfCrime)")

This line saves the map as an image. You can use a direct file path or the published parameter. The file can also be saved as a PDF.

ggsave("C:/Users/$(ComUserName)/Documents/$(TypeOfCrime) From 2012-2017.png")

fmeOutput is a mandatory line in the RCaller. You can change the data.frame() value to extract data. For more information on fmeOutput see the RCaller documentation.

fmeOutput<-data.frame(1)

Troubleshooting

If you get the following error message, but you installed ggmap through an R console, ensure that the ggmap package is installed in the same location that FME is looking for the R Interpreter Executable. This can be found by going to Tools > FME Options > Translation then under R Interpreter:

If you get a Google Maps 403 error, it just means that the base map service is currently busy, just wait a minute and try re-running the workspace.

If you get any other error messages, check and ensure your code is correct. Double check that your code works in an R Console by replacing the published parameters with actual values.

Additional Resources

[Webinar] Ins and Outs of Using R with FME

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.