How To Optimize Your Raster Data Using the RasterInterpretationCoercer

Introduction

You have a raster that looks absolutely black in the FME Data Inspector:



Closer inspection shows that the pixels have a good variety of values, so why are the contents not visible?

The information window under "Raster Point Info" shows an image consisting of three 16 bit bands, which defines an RGB48 interpretation. This means that the range of available values for each band goes from 0 to 65535. However, the actual values on the sample image do not exceed a few hundred.



In other words, this raster contains 8-bit values stored in 16-bit bands.

The completely black display is partly due to how the FME Data Inspector interprets the data; 0 is black and 65535 is white, so even a value of 256 is not going to register very highly. But, really, this is a problem with the data. The correct interpretation needs to be set on the existing values or the values need adjusting to match the current interpretation.

Solution 1: RasterExpressionEvaluator

One possible solution is to use the RasterExpressionEvaluator. The following band list and expression could be applied:
RED16;GREEN16;BLUE16 A[0]*200;A[1]*200;A[2]*200
However, this requires some knowledge of the transformer and a few experiments to find an optimal multiplier.

Solution 2: RasterInterpretationCoercer

Another solution is to use the RasterInterpretationCoercer, which may be a better choice for this situation. It contains four conversion options, giving users much better control over the whole process.

Conversion options

The conversion options include:

• Cast
• Bounded Cast
• Scale by data values
• Scale by data type

If the interpretation of the raster is changed from RGB48 to RGB24 using the 'Scale by data value' option, the following image is produced:



An explanation of how these options work, can be found in the transformer's documentation, but provided below are some tangible results for those who prefer a good visual explanation.

For this purpose, a simple contour file was prepared and a profile, in brown, was created along the red line. The profile was made with the ProfileBuilder custom transformer found in the RasterInterpretationCoercer.fmwt workspace:


Based on contour data, a one band DEM raster and two tiles were created. The DEMGenerator makes Real64 rasters, however, here it's in UINT8 - rasters can be really big and this will save on disk space.

There are four parameters for the conversion mode. Workbench has no way of knowing ahead of time what type of rasters will be passed through this transformer - whether it will be a color image (e.g. RGB24, RGBA64, Green16, etc.) or a numeric raster (Real64, Int32, etc.), and our destination can be either color or numeric. All possible conversion combinations are listed as separate parameters. Based on the choice in "Destination Interpretation Type" only two of four parameters can be enabled.



Below is one of the tiles sent through the RasterInterpretionCoercer transformer using the "Cast" method:

Before

After


Inspection of different areas shows that the image (now UINT8) matches the source DEM (the only difference is that the values were rounded), so it looks like "Cast" is a pretty good option.

Next is the second tile sent through the RasterInterpretationCoercer using "Cast" parameter:

Before

After


That's definitely wrong. For better understanding of what is going on see the profile of the new DEM (yellow) and compare it with the original profile (brown):
The elevation gradually goes up until it reaches 255 meters, which is the maximum value possible for for the type UINT8, and then abruptly it drops down to 0, after that, this pattern repeats again. The new elevation can be expressed with the following formula:
elevation % range_for_the_bit_depth = cast_elevation
or, take the elevation of the red dot on the UINT8 image and use the following expression:
575 % 256 = 63
This proves that the "Cast" option does not allow a good conversion for the entire dataset.

"Bounded Cast" works nicely for the left tile. For the right tile it makes something different, but it is hardly any better - the top of the hill is simply chopped off (that is, everything above 255 meters) and everything below 0 is set 0:


This illustrates the limitation of "Cast" and "Bounded Cast" parameters which work well as long as the values of the raster do not exceed the chosen destination interpretation type. If all of the data being transformed is below the maximum value for the interpretation, these options are a good choice, otherwise, both pixel values and visual representation may be compromised.
To avoid this, consider another interpretation type with higher bit depth or another interpretation option.

The option "Scale by Data Value" is visually close to the originals.

Before

After


However, the elevations on both tiles are scaled - expanded on one tile and shrunk on another:


The graphical representation of this conversion type could look as follows:



The range between minimum and maximum values found in the source is scaled to the full range of available values of the destination interpretation type. The tiles in the example have ranges 25 to 200 (left tile) and -45 to 575. Both resulting raster's have values scaled to the range between 0 and 255.

This option won't keep the true representation of the source raster, however, it can be useful for making color images:


The image above was made by converting Real64 to Red8, and then two extra bands were added - Green8 set to 100, and Blue8 set to 50. Vector contours were then overlayed on the image.

The last option, "Scale by Data Type" results in a simple gray raster with all the pixels having the value of 128.



The full range of the source type is scaled to the full range of the destination type. Because the type of Real64 has a range between ~-1.78e+308 and ~1.78e+308, but the actual values vary from -45 to 575, all our destination values are squeezed into a tiny range somewhere very close to 128.

If the values had been more or less evenly spread through the entire range, this method can help preserve visual representation:

Before

After

To summarize:
The first two options, "Cast" and "Bounded Cast" are better for keeping the original values (or rounded if we go from Real to Integer) under the condition that the destination range is big enough to fit all the values. These options are useful for Numeric to Numeric conversions.

The last two options, "Scale by Data Value" and "Scale by Data Type" help preserve the visual representation and are useful for Numeric-to-Color or Color-to-Color conversions.

As for the test tiles used, picking another interpretation would be helpful in order to preserve the correct elevation. INT16 seems to be a reasonable alternative because even if the project covers the entire world, it still won't exceed the maximum and the minimum (in meters) found on our planet.

The attached template, RasterInterpretationCoercer_2015.fmwt, shows all the options explained above.

RGBA to RGB Option

The "RGBA to RGB" option tells the transformer what to do when converting an RGBA image (an image with three color bands and an alpha band responsible for the transparency) into a simple RGB (color) image, that is, if the alpha band is removed.

In the RGBA32 image, the portion outside the area of interest is fading gradually into full invisibility. The RGB values reflect actual pixel values while the alpha pixels reflect the degree of transparency value. The value of the alpha pixel under the red pin, is equal to 69 and the area underneath is semitransparent.:




Selecting the "Apply the Alpha Band" option, still produces the gradual disappearance of the image contents, however it now has the interpretation RGB24 and the remaining band values (red8, green8 and blue8 (RGB)) have changed to reflect the application of the alpha band:




"Drop Alpha Band" removes the alpha band and reveals what was hidden by the transparency of the alpha pixels. Note that the RGB band values have changed again and reflect original and not 'alpha' influenced values: