Image Compression in FME

Introduction

Compression is a process used on image or raster data to help manage file size. By compressing images, redundant data can be removed or suppressed to help address memory bloating, storage capacity limitations, and performance. This article will walk through what compression is, the types of compression available, as well as how it can be applied in FME Form.

What is Compression? 

Image data is composed of photo elements called pixels. Compression is the method in which pixels are stored, removed, or reconstructed on output to reduce the size of the image on disk. Note that applying compression does not guarantee a smaller file size will be achieved.

According to USDA, image compression can be useful for multiple different reasons, such as:

• Sharing files through online platforms, such as email, websites, or social media
• Storing images on physical devices or in the cloud
• Reducing storage space to save on costs
• Archival purposes
• Reducing the resources and time required to open the image

Types of Compression

There are two types of compression:

• Lossy
• Lossless

Lossy compression will permanently remove redundant data from a file to reduce file size. Here, image quality degrades as the less important information is filtered and discarded from the image. Once data is removed from an image using lossy compression, it cannot be recovered.

Examples of lossy compression:

• LERC (lossy is a compression level is < 0)
• JPEG

Lossless compression works differently. Lossless compression does not discard any data, even if pixels appear redundant. Instead, to maintain data quality, bit-depth is used to store multiple bytes of data on a single pixel. Because of this, images usually need to be uncompressed when imported or read.

Examples of lossless compression:

• LERC DEFLATE
• DEFLATE
• ZSTD

Compression and Format Support

The types of compression available depend on specific format support. For example, JPEG (Joint Photographic Experts Group) compression is a lossy compression method primarily used for photographic images. Further, JPEG compression would not be used on an ECW dataset, as ECW uses its own proprietary compression method, Wavelet

We often see lossy compression used for things like web images or digital photography, where users do not necessarily need all the information represented. Conversely, datasets used for analytical purposes benefit from lossless compression, which allows high detail to be maintained.

It is encouraged to consult the documentation or do research before applying compression in FME Form. For example, some compression methods are best suited for particular datasets, such as CCITT FAX, a lossless method used for bi-level (2 color / black and white) data, which are normally scanned or faxed documents.

How to Apply Compression in FME Form?

In this tutorial, you will learn how to convert and compress an orthophoto. The data and workspace can be found in the Files section of this article. In this example, a GeoTIFF file will be read in and then written out to a GeoTIFF with compression applied.

1. Observe the Original File Size of the Orthophoto
Before opening FME, review the original size of the image. You can do this by right-clicking on the downloaded image and opening the properties menu in File Explorer. The file is 4.60MB.

2. Add a GeoTIFF Reader
Open FME Workbench and start a new workspace. Add a GeoTIFF reader to the canvas. For the the source Dataset, select 02-03-JK.tif. Click OK to finish adding the reader.

3. Add a GeoTIFF Writer
Next, add a GeoTIFF writer to the canvas. Browse to a location to save the Dataset, and add _compressed to the original file name. Ensure that the Raster File Definition is set to Automatic before clicking OK.

After the writer has been added to the canvas, connect the reader and writer feature types.

4. Define Compression
Double-click on the writer feature type to open the parameters. Expand the Compression section and select ZSTD as the Compression Method. ZSTD or Zstandard compression is a lossless compression method. Leave the default ZSTD Compression Level as 9. A ratio of 9 will adequately compress data, while not impacting performance. As compression levels increase, there is more potential for FME to reduce the file size, but performance will decrease. Higher compression levels will produce greater reductions, whereas lower compression levels can be used for faster throughput.

5. Run the Workspace
On the top menu bar, click the Run button to execute the workspace and write the compressed GeoTIFF to disk.

6. Observe Output File Size
Single-click on the writer feature type to open the mini menu. Click on the Folder icon to open the output data location. In the File Explorer, right-click on the output image and open the properties menu. The file size has been reduced from 4.60MB to 3.79MB.

Additional Resources

• USDA Farm Service Agency: Image compression and formats: Why it matters. U.S. Department of Agriculture. 
• Adobe: What is lossless compression? 
• TechTarget: Image compression.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.