Data Distribution for Raster Data

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

Safe Software has announced plans to deprecate the FME Server JavaScript API. Please review the JavaScript API Deprecation notice for updates.

The FME Server Playground is also scheduled for permanent removal. Content may be removed from the website without notice. Moving forward, interactive FME demos will be accessible from www.engage.safe.com/demos.


Live Demo | Fork on GitHub

Introduction

This tutorial will go through the steps of creating an FME Server Data Download service to distribute raster data. The application will allow users to do the following:
 

  • Download specific tiles of Vancouver aerial imagery using the Geometry Picker
  • Choose a  raster format output
  • Choose between a greyscale or color image
  • Build a raster pyramid with the selected number of levels
  • Create a thumbnail with a specified size

You will create this application using FME Workbench and FME Server Workspace Apps.

 

Step-by-Step Instructions

Part 1: Load the Data

1. Start FME Workbench and Save the workspace

When FME Workbench starts up, select New under Create a Workspace. Click Save and choose where you want to save your workspace in. Call the workspace RasterDistribution.fmw. Download the RasterData folder from the Files section on this article and move it to the same folder location as your workspace.

2. Insert the Creator transformer

We will use the Creator transformer to create a dummy feature since we don’t have a Reader. If we did not have this, the workspace would not run. 

Add a Creator to the canvas and open the parameters. Set the coordinate system to EPSG:4326. Leave the rest of the settings as default.

CreatorParams.png

3. Add a published parameter for the Geometry Picker

In our FME Server app, we want to let the user specify the area on the map for which they want raster data. 

Under Navigator, right-click User Parameters. Select Manage User Parameters and a new window will pop up. Click on the green plus icon, and in the drop-down options select Geometry. Fill in the parameter with the following:
 

Parameter Identifier: GEOM
Prompt: Create Clipping Area
Published: Enabled
Required: Enabled

Geometry Encoding: GeoJSON

Geometry Types: Polygon
Top: 49.2548
Left: -123.244
Bottom: 49.3034
Right: -123.071

Default Value:     
{"type":"Polygon","coordinates":[[[-123.109703,49.280212],[-123.089447,49.27954],[-123.09494,49.268788],[-123.110046,49.273492],[-123.109703,49.280212]]]}

GeomParams2.png

Click OK.

4. Add GeometryReplacer

The GeometryReplacer will change the raster image to the user selected area using GeoJSON coordinates.

Add the GeometryReplacer and connect the Creator to it. Open settings and set the Geometry Encoding to GeoJSON. The Geometry Source can be set to our new GEOM parameter by selecting the drop-down arrow, and User Parameter. Click OK.

GeomReplacerParams.png

5. Match the coordinate system of the user selected area to the raster data

Insert the Reprojector transformer and connect the GeometryReplacer Output to it. 

In the parameters, enter the following:
 

Source Coordinate System: LL84
Destination Coordinate System: UTM83-10


Click OK.

6. Find the data

Add a FeatureReader to the canvas and then connect the Reprojector Reprojected port to the Initiator input port. In the parameters, set the format as TIFF (Tagged Image File Format). 

We will temporarily set the Dataset to look for the data locally to make sure everything is running smoothly. Click on the ellipsis and locate the file which contains all of the data. Select one of the images and click Open. Once this is added to your Dataset parameter, change the image file extension to \*.tif. This will allow the transformer to select any of the images in the folder. 

Under Feature Types to Read, click the ellipsis and check the TIFF option. Click OK.

For the Spatial Filter, set it as Bounding Boxes OGC-Intersect.

In the Output, select Specified and after clicking on the ellipsis, select the TIFF option again. Click OK to close the Select Specified dialog. Finally, click on Parameters then expand Schema Attributes. Click on the ellipsis next to Attributes to Expose and select fme_basename, click OK twice and OK again to close the transformer dialog. 

7. Add AttributeExposer

We need to expose the fme_basename attribute using the AttributeExposer. This will give us some metadata information on the rasters. Connect the TIFF port of the FeatureReader to the AttributeExposer. In the parameter dialog, type fme_basename as the Attributes to Expose.

8. Run the workspace

Before we continue, we must make sure that the workspace is working properly. Run the workspace with Feature Caching Enabled. On the FeatureReader, click on the TIFF green magnifying glass to view the data in Visual Preview. The result should look like this:

GraphicsView.PNG

9. Upload the data to FME Server

Log into your FME Server account and under Files & Connections, select Resources.

Step8.PNG

Click on the Data folder and create a new folder called RasterData. Open up this folder and select Upload, Files to upload all of the image files. You can use Ctrl-A to select all of the images at once. 

After the upload is complete, at the top select Data to go back to the Data folder. Check the box beside the RasterData folder and under actions select Properties. Copy the System Path. 

SystemPath.png

10. Change the local path to the System Path

Back in FME Workbench, open the FeatureReader Parameters and paste the System Path in the Dataset. Append to the path /* so it looks like the following:
 

Dataset: $(FME_SHAREDRESOURCE_DATA)/RasterData/*    

Step9.PNG

Note: Your RasterData folder must be in the same folder as your workspace in order for this to work.

11. Run the workspace

Let’s make sure that the workspace still works after this change. Rerun the workspace and you should see the same results.
 

Part 2: Create Mosaic

12. Mosaic the Rasters

Add the RasterMosaicker and connect the FeatureReader TIFF output port to it. This will merge the raster images together. Leave the default settings as they are.
 

Part 3: Create Conditions

13. Create Greyscale Published Parameter

Right-click User Parameters in the Navigator and select Manage User Parameters. Click on the green plus icon, and select Choice. Set the parameters as follows:
 

Parameter Identifier: Greyscale
Prompt: Output as Greyscale
Published: Enabled
Required: Enabled
    
Drop Down: Selected
 

Value Display
Yes Yes
No No


Allow Choice Edit: No
Default Value: Yes

Step12.PNG

14. Create Thumbnail Published Parameter

Create another Choice parameter using the following parameters:
 

Parameter Identifier: Thumbnail
Prompt: Create Thumbnail
Published: Enabled
Required: Enabled

Drop Down: Selected
 

Value Display
Yes Yes
No No
 

Allow Choice Edit: No
Default Value: Yes

Step13.PNG

Click OK to close the Parameter Manager. 

15. Filter the user thumbnail choice

The user has the choice between creating a thumbnail or not. We need to create a filter for that to make the workspace do what we want. 

Add a TestFilter transformer and connect the RasterMosaicker Output port to it. Open the settings. Fill out the Test Condition by double-clicking on the Test Condition If box and also fill in the Output Port Else box. The Test Condition window should look like this:

 
  Test Condition Output Port
If $(Thumbnail) = Yes Thumbnail
Else If    
Else   No Thumbnail

ThumbTestFilter.PNG

ThumbTestFilter2.png

Click OK to close the TestFilter Parameters.

16. Filter the image choice

Add a second TestFilter to the canvas and connect the Thumbnail and No Thumbnail port from the first TestFilter. Fill out the first Test Condition by double-clicking on the box under it. The Test Condition window should look like this:

 
Logic Left Value Operator Right Value
  $(Greyscale) = Yes

GryscaleTestFilter.png

Fill out the rest of the TestFilter as follows:

 
  Test Condition Output Port
If $(Greyscale) = Yes Greyscale
Else If $(Greyscale) = No Color


GryscaleTestFilter2.png
 

Part 4: Create Thumbnail Image

Create a thumbnail size published parameter

In the User Parameters, right click and select Manage User Parameters. Click on the green plus icon and select Number. Fill in the parameters as follows and make sure Required is unchecked:
 

Parameter Identifier: PERCENTAGE
Prompt: Thumbnail Size (as percentage)
Published: Enabled
Required: Disabled
Default Value: 1
 

PercentParam.PNG

Click OK.

18. Add a RasterResampler

Add a RasterResampler and connect the TestFilter Thumbnail port to it. This transformer will create a thumbnail image of the raster with the size specified by the user.

Change Resolution Specification to Percentage from the drop-down. Under Percentage, select the drop-down arrow >User Parameters > PERCENTAGE, which is the parameter we just created. Click OK.

RstrResampler.png

19. Add an AttributeCreator

We will need to create a new attribute value to add to the metadata for any thumbnail images. Add an AttributeCreator and connect the  RasterResampler Resampled port to it. 

Open the parameters, double-click on the box under New Attribute and select fme_basename. Under the Attribute Value, select the drop-down arrow and click on Open Text Editor. Paste the following inside:
 

@Value(fme_basename)_thumb


The Text Editor should look like this:

TextEditor.png

Click OK and OK again.
 

Part 5: Create Greyscale Image

20. Add a RasterExpressionEvaluator

Add a RasterExpressionEvaluator and connect the  TestFilter Greyscale port to the A port.

To create a greyscale image from the color image, we need to multiply the red, green and blue bands by an expression. Copy the following expression for each the Red8, Green8 and Blue8 Interpretations:
 

0.2989*A[0] + 0.5870*A[1] + 0.1140*A[2]

RasterExpression.png

Click OK.

21. Run the workspace

Click on the RasterExpressionEvaluator and select Run to This from the popup menu. Click Run again in the Translation Parameter Values window, be sure that the Greyscale parameter is set to Yes.

Check the result by clicking on RasterExpressionEvaluator and selecting the green magnifying glass. Your result should look like this:

GraphicsViewGrey.png

Part 6: Convert raster from three bands to one

This part of the workflow will only run if the user chooses a GIF format because it does not support 3 bands. 

22. Create output format published parameter

Right-click User Parameters in the Navigator and select Manage User Parameters. After clicking the green plus icon, select Choice. This parameter will allow the user to choose a format for their output raster. Notice that the values for each format are the short name of the Writer type. If you want to add a format but are not sure of the short name, you can search for the Writer and the short name will be listed. 

WriterShortname.png

Fill in the parameters as follows:

Parameter Identifier: OutputFormat
Prompt: Output Format
Published: Enabled
Required: Enabled

Drop Down: Selected
 

Value Display
GEOTIFF GeoTIFF (Geo-referenced Tagged Image File Format)
GIFRASTER GIF (Graphics Interchange Format)
JPEG JPEG (Joint Photographic Experts Group)
PDF2D Adobe Geospatial PDF
PNGRASTER PNG (Portable Network Graphics)

 

Allow Choice Edit: No
Default Value: GeoTIFF (Geo-referenced Tagged Image File Format)

OutputFormat.PNG

23. Add a third TestFilter

Add a third TestFilter and connect the  RasterExpressionEvaluator Result port and the  TestFilter Color port to it. 

Configure the settings like in the image below:
 

  Test Condition Output Port
If $(OutputFormat) = GIFRASTER GIFRASTER
Else   Other Formats


GifrasterTestFilter.png

24. Add a RasterPaletteGenerator

The RasterPaletteGenerator transformer will merge three of the bands into one. Add this transformer to the canvas and connect the TestFilter_3 GIFRASTER port to it. 

View the parameters for the RasterPaletteGenerator. We will leave them as default since our bands are 8-bit as well.
 

Part 7: Create Raster Pyramid

We will need to create a raster pyramid if the user chooses to have that.

25. Create a published parameter for the pyramid choice

In the Navigator, right-click on User Parameter and select Manage User Parameters. Click on the green plus icon and select Choice. Fill in the parameters as follows:
 

Parameter Identifier: Pyramid
Prompt: Create Raster Pyramid

Published: Enabled
Required: Enabled

Drop Down: Selected
 

Value Display
Yes Yes
No No

PyramidChoice.PNG


 26. Add a fourth TestFilter 

Add a fourth TestFilter and connect the RasterPaletteGenerator Output port and the  TestFilter_3 Other Formats port to it.
 
We will want to test for the condition where the user wants a pyramid and they have not selected the PDF output format. In the settings, double-click the box under Test Condition. Fill out the condition as follows:
 

Logic Left Value Operator Right Value
  $(Pyramid) = Yes
AND $(OutputFormat) != PDF2D

 

Output Port: Pyramid
 

IfPyramid.PNG

Click OK. Make sure to set the Output Port for Else as No Pyramid. Click OK.

27. Create a published parameter for pyramid levels

Right-click on User Parameters and open the Manage User Parameters. Click on the green plus icon and choose Number. Fill in the parameters as follows (make sure required is unchecked):

Parameter Identifier: NUMBER_OF_LEVELS
Prompt: Number of Levels in Pyramid
Published: Enabled
Required: Disabled

Lower Limit: None
Upper Limit: None
Numeric Precision: Integer

Default Value: 3
 

PyramidLvls.PNG

28. Add a RasterPyramider

Add a RasterPyramider and connect the TestFilter_4 Pyramid port to it. This transformer will create the number of levels that the user has specified.

Open the parameters and change the Smallest Level Size to Number of Levels. For Number of Levels, in the drop-down menu select User Parameters >NUMBER_OF_LEVELS.

RasterPyramider.png

Click OK.

29. Add a second AttributeCreator 

Here we will create a new attribute value for the metadata of any images that have a pyramid. Add a second AttributeCreator and connect the RasterPyramider Pyramids port to it. 

Open the parameters for the transformer. Under New Attribute select fme_basename. Open the Text Editor for the Attribute Value and paste this into it:

@Value(fme_basename)_pyraLvl_@Value(_pyramid_level)


Click OK and OK.
 

Part 8: Add Writer

In our workspace we will want to have a writer that will write the same format as the user specified. 

30. Add a Generic (Any Format) Writer

Add a Generic (Any Format) writer to the canvas. For the Dataset, select the ellipsis and choose a folder location for your data to download. 

Click on the Parameter button and select Generic (Any Format) for the Output Format. Click OK and OK.

GenericWriterParams.png

The Feature Type dialog will pop up. For Geometry, select fme_any. Click OK.

GenericWriterGeometry.png

Connect the AttributeCreator Output port,  TestFilter_4 No Pyramid port, and  AttributeCreator_2 Output port to the Writer.

In the Navigator, right-click Downloads [GENERIC] and select Edit ‘Downloads [GENERIC]’ Parameters. Select the down arrow, Writer Parameters >OutputFormat. This will set the output format to the same as the user specified. Click OK.

WriterOutputFormat.png

We want to make sure the writer does not appear as a published parameter on the application. Right-click [FORMAT_GENERIC] in User Parameters, and select Manage User Parameters. Unselect the Published and Required checkboxes. Click OK.

In the writer feature type parameters, change the Feature Type Name by selecting the drop-down arrow >Select Attribute Value > fme_basename. Click OK.

GenericWriterFtType.png

 31. Run workspace

Before you run the workspace, make sure the order of your User Parameters in the Navigator are as follows:

UserParams.PNG

Click the drop-down arrow beside Run and select Rerun Entire Workspace. The Translation Parameter Values should look like this:

RunParams.PNG

Click Run.

To check your result, click on the writer feature type to open the popup menu and select Open Containing Folder. This should take you to the folder that contains the final raster image. 

 

Part 9: Building Workspace App

32. Publish the workspace

In FME Workbench, at the top click on File >Publish to FME Server. Make sure you have set up your connection before clicking Next. Click New to create a new repository and call it RasterDistribution. Click Next.

For Service, choose Data Download, Data Streaming, and Job Submitter. We are using the Data Download because the user will be able to download the raster data. Since our data is on FME Server, we want to use Data Streaming. In this case, we are also using Job Submitter to run the workspace we published. 

Click Publish.

33. Create the app

Go back to FME Server, expand Server Apps on the side menu and click on Build Workspace App. Fill out the parameters as follows:
 

Name: RasterDistribution
Title: Data Distribution for Raster Data
Description:  Allows users to select an area and download raster data according to their preferences. The user can specify the output format, thumbnail choice, color choice, pyramid choice, and number of pyramid levels.
Repository: RasterDistribution
Workspace: RasterDistribution.fmw
Service: Data Download
 

ServerApp.png

Click Parameters to open the drop-down menu. Look at the parameters and make sure it has all of the published parameters we specified. 

Click Customize to open the drop-down menu. Here you have the option to customize the application interface.

Click OK. Click on the link that FME Server creates to open the application. Test it out.

ServerAppFinal.png

Congratulations, you have successfully completed the tutorial!
 

Additional Resources

Getting Started with FME Server Workspace Apps
Geometry Picker
 

Data Attribution

The data used here originates from data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.

 

 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.