Data Distribution for Point Cloud Data

Introduction

With FME Server Workspace Apps, it’s easy to create a front-facing interface for users to download point cloud data using the FME Server Data Download Service.

This article emphasizes the capabilities of the GeometryReplacer transformer, allowing the user to download a selected area. The user parameter manager is also highlighted as a key tool in creating a flexible workflow. The parameters let the user choose the output point cloud format, output coordinate system, an area of interest to which the results will be clipped, whether or not to split the output, and whether or not to reduce the number of points in the output by using a thinning algorithm.

Step-By-Step Instructions

Part 1: Creating the FME Workbench Workspace

The first part of this article explains how to make the FME Desktop workspace that will be used to create the FME Server Workspace App. It will carry all the duties of transforming the data as well as being the main platform where we can set up user parameters.

1. Add Creator Transformer
Start FME Workbench and open a new blank workspace. Begin typing on the blank workspace and enter “Creator”. Select the Creator transformer to add it to the workspace. We can use the default parameters. 
 
2. Add GeometryReplacer
Add a GeometryReplacer transformer to the canvas. Connect the Creator to the GeometryReplacer. The GeometryReplacer will be used later in the FME Server Workspace App to allow the user to clip the Point Cloud data to their desired area.

3. Setup GeometryReplacer Parameters
Next, we need to create a new Geometry parameter that can be used as the Geometry Source in the GeometryReplacer. To do this, we need to right-click on User Parameters in the Navigator window and click Manage User Parameters. 

In the User Parameters window, click on the green “plus” symbol in the top left and select Geometry to create a new Geometry user parameter. Use the following parameters:

• Identifier: GEOM_COORDS
• Prompt: Select Area
• Disable Attribute Assignment: Yes

Click the ellipsis to the right of Geometry Types and un-check Point.

Check the box next to “Specify initial bounds for map display” and set the values as such:

• ​Top: 49.2856
• Left: -123.146
• Bottom: 49.2625
• Right: -123.107

Click OK to save the parameters. Next, open up the GeometryReplacer and set the Geometry Encoding parameter to GeoJSON and change Remove Attribute to No. Click on the drop-down arrow next to Geometry Source then select the GEOM_COORDS user parameter we just created.

4. Add CoordinateSystemSetter 
Add a CoordinateSystemSetter transformer and connect the GeometryReplacer to it. Open up the transformer and set the Coordinate System parameter to LL84. 

5. Add Reprojector
Add a Reprojector transformer and connect the CoordinateSystemSetter to it. Open up the Reprojector and set the Destination Coordinate System to UTM83-10.

Your workspace should look like this now:

6. Add FeatureReader

Add a FeatureReader transformer to the canvas and connect the Reprojector to it. The FeatureReader will be used to read in the Point Cloud file.

Open the FeatureReader and set the parameters as follows:

• Format: ASPRS Lidar Data Exchange Format (LAS)
• Dataset: $(FME_SHAREDRESOURCE_DATA)/PointCloud/*.zip
• Coord. System: UTM83-10
• Feature Types to Read: LAS
• Spatial Filter: Initiator is OGC-Intersects Result 

Click the Parameters box and click on Schema Attributes to reveal Additional Attributes to Expose. Assign this the value of “fme_basename,char(50)”.

 
7. Add Clipper
Add a Clipper transformer to the canvas. The Clipper transformer will be used to clip the Point Cloud dataset to the area of interest selected by the user. 

Connect the LAS output port from the FeatureReader to the Clippee port of the Clipper transformer. Connect the Reprojector to the Clipper port. 

 

Open the Clipper transformer. Under Vector, change Tolerance to None.


8. Add AttributeExposer
We want to add the AttributeExposer to expose fme_basename, which will later be used for the file naming convention. Connect the Inside port to the AttributeExposer. 

 

Open the Attribute Exposer and add “fme_basename” to Attributes To Expose.

 

9. Add a Tester
The next step is to create a workflow that allows the user to select whether or not they want to merge the data into one point cloud. This requires the use of a Tester to check whether the user says Yes or No.

Add a Tester and connect the AttributeExposer to it. 

10. Setup Tester Parameters
Because we want the user to be able to select Yes or No for this Tester, we need to create a new User Parameter. 

Open up the User Parameter Manager and add a new Choice parameter. Set the name to TILED and change the Prompt to “Return Tiled Output”. Uncheck the Required box.

Under the Choices area on the right of the User Parameter Manager, add two new Values. These will be YES and NO. Set the Default Value to No. 

Go back and open the Tester transformer. Now we will create the Test Clause that will decide whether or not the dataset will be merged into one point cloud or not. Set the test to $(TILED) = NO.

 11. Add PointCloudCombiner
Add a PointCloudCombiner transformer and connect the Tester Passed port to it. We can use the default parameters.
 
12. Add PointCloudVerticalFilter 
Next, we need to add a PointCloudVerticalFilter which will allow us to keep the point cloud files as separate tiles. The PointCloudVerticalFilter is an FME Hub transformer that will automatically be downloaded via FME Workbench. For more information on this transformer, see the FME Hub.

Connect the Tester Failed port to the PointCloudVerticalFilter.

Open up the PointCloudVerticalFilter and set the Adjustment Parameter (# of Vertical Tiles) to 10. 
 
13. Add Tester
Next, we want to create a workflow that will test if the user wants to split their point cloud. To do this we first want to add another Tester transformer. Then connect the PointCloudCombiner and the PointCloudVerticalFilter Output ports to it. 

 

14. Setup Tester Parameter
Similar to Step 9, we want to set up a new user parameter that will be used in the Tester_2. This will test whether the user wants to split the point cloud or not. 

Go to the User Parameter Manager and add a new Choice parameter. Name it SPLIT_PC and change the Prompt to “Split Point Cloud”. Next, un-check the Required box. Under the Choices section, add Values YES and NO. Use the Display values it provides. Set the Default Value to No. 

Go back and open the Tester_2 and set the test to $(SPLIT_PC) = YES.

15. Add PointCloudSplitter 
Add a PointCloudSplitter transformer and connect the Tester_2 Passed port to it. 

Open the PointCloudSplitter and create a new User Parameter for the Split By parameter by clicking on the drop-down arrow, hovering over User Parameter, and clicking Create user Parameter. Call it SPLIT_BY_DISPLAY, un-check the Required box, and press OK. 

Go to the User Parameter Manager and click on the SPLIT_BY_DISPLAY parameter just created. Click the Choice Configuration to expand the Choices window and delete all apart from First Return Only and Last Return Only. 

The Parameters should be as follows:

• Identifier: SPLIT_BY_DISPLAY
• Prompt: Split By
• Published: Yes
• Required: No

Set the Default value to First Return Only. We have to set this because the workspace won’t run in the Workspace App we build later if this value is left empty. 

Go back to the PointCloudSplitter transformer and set Split By to the parameter just created and change Output Attribute to "_point_source_id".

16. Add Tester
Next, we need to use another Tester transformer to check whether or not the user wants to thin the output point cloud. This is useful for reducing the number of points in the point cloud and producing a smaller file. Add another Tester transformer and connect the Tester_2 Failed port and the PointCloudSplitter Split port to it.

17. Setup Tester Parameter
Again we need to set up a user parameter in the Manage User Parameter window. Create a new Choice parameter and use the following values:

• Identifier: THIN_PC
• Prompt: Thin Point Cloud
• Required: No

Press OK to close.

Open up Tester_3 and set the test to $(THIN_PC) = YES.

18. Add PointCloudThinner
Add a PointCloudThinner transformer and connect the Tester_3 Passed port to it. 

There are three user parameters we need to establish to be used in the PointCloudThinner transformer. Open the PointCloudThinner transformer. To create a new user parameter for each parameter, remember that we can click the drop-down arrow, hover over the User Parameter, and click Create User Parameter. 

Create a User Parameter for Points to Keep and call it POINTS_TO_KEEP. Un-check the Required box. 

Create a User Parameter for Interval and call it THIN_INTERVAL. Un-check the Required box and clear the Default Value.
 

​​​​​​​Create a User Parameter for Maximum Number of Points and call it MAX_NUM_POINTS. Un-check the Required box and clear the Default Value.

​​​​​​​Your PointCloudThinner Parameters should now look like this:

​​​​​​​19. Add AttributeCreator
Add an AttributeCreator transformer. Connect the Tester_3 Failed port and the PointCloudThinner Thinned port to it. 

Open up the AttributeCreator and set the parameters as follows:

• New Attribute: _point_source_id
• Attribute Value: @Value(_point_source_id)

​​​​​​​20. Add Tester
The fourth and final Tester transformer is used to check whether or not the point cloud was split into multiple point clouds. If it was split using the PointCloudSplitter, then an ID (_point_source_id) was created from the AttributeCreator used in the last step. This ID will be added to the output file name.

Add a new Tester transformer and connect the AttributeCreator to it. Open the Tester and set the following values:

• Left Value: _point_source_id
• Operator: =
• Right Value: _

​​​​​​​Expand Advanced and set Preserve Feature Order to Across Output Ports.

 
21. Add AttributeCreator
Add an AttributeCreator transformer. Connect the Tester_4 Failed port to it. 

Open up AttributeCreator_2 and set the parameters as follows:

• New Attribute: fme_basename
• Attribute Value: @Value(fme_basename)@Value(_point_source_id)

​​​​​​​22. Add Writer
Finally, we want to add a Generic Writer to write our Point Cloud file out to. Add a new Writer to the Workspace and set the parameters as follows:

• Format: Generic (Any Format)
• Dataset: <any directory of your choice>

Click on Parameters and set the Output Format to ASPRS Lidar Data Exchange Format (LAS). Press OK to close the Parameters window and press OK again to Add Writer. 

In the Feature Type dialog that pops up, set Geometry to fme_any and press OK. Connect the Tester_4 Passed port and the AttributeCreator_2 Output port to the Writer. 

​​​​​​​Open back up the Writer, click the drop-down arrow next to Feature Type Name, hover over Attribute Value and select fme_basename

​​​​​​​We don’t need the User Attributes or Format Attributes. Click on the User Attributes tab, set the Attribute Definition to Manual and remove all the values.

Click on Format Attributes and un-check fme_basename so it is no longer exposed.
 
23. Add Point Cloud XYZ Support
Because we want to let the user have the ability to export the output Point Cloud to either LAS or Point Cloud XYZ format, we have to create another writer that will be hidden in the background that will handle the Point Cloud XYZ parameters. This is because the writer needs to know which components are to be written out and the order of those fields in the output file (specified in the Component Mapping parameter for the writer). The Generic writer will use the parameter values set in the point cloud XYZ writer in the workspace when writing out to that format.
 
Add a Writer to the workspace. Set the parameters as follows:

• Format: Point Cloud XYZ
• Dataset: Any directory of your choice

Click on Parameters and add the following values to the Point Cloud Components to Output section:

​​​​​​​Press OK to close the window and then press OK again to add the writer. Accept the default name. Since we don’t need the Writer on the workspace we can right-click it and press Delete. When the window comes up asking you if you want to remove these readers and writers click No. This won’t delete the Writer we just created, it will just hide it in the background. 

24. Add User Parameter for Output Format
We need to create a parameter that will allow the user to choose whether they want to output a LAS file or a Point Cloud XYZ file.

Open up the User Parameter Manager window (again by right-clicking on User Parameters and selecting Manage User Parameters). Add a new Choice User Parameter. Set the following parameters:

• Identifier: OUTPUT_FORMAT
• Prompt: Output_Format
   

Expand Choice Configuration and add two new Values as follows:

​​​​​​​Set the Default Value to ASPRS Lidar Data Exchange Format (LAS).

​​​​​​​Don’t close the User Parameter Manager as we will be adding a parameter in the next step.

25. Add User Parameter for Output Coordinate System
We also want the user to be able to choose what coordinate system they output their data to. Create a new Choice User Parameter. Use the following parameters:

• Identifier: COORD_SYS
• Prompt: Output Coordinate System

Expand Choice Configuration and use the following values:

​​​​​​​Set the Default Value to NAD83 datum, UTM Zone 10.

​​​​​​​26. Update Output Writer Parameters
To make sure that the option to choose which format to output to appears in the Workspace App we will create, we need to go to the Navigator window and expand Output [GENERIC]. Expand Parameters and right-click Output Format and select Edit Parameter Value.

For Output Format click the drop-down arrow on the right, hover over User Parameter and select OUTPUT_FORMAT. 

In this same window expand Coordinate System and change Coordinate System to the COORD_SYS User Parameter. Press OK to close the window.​​​​​​​


 

​​​​​​​27. Organize User Parameters
The order of User Parameters in the User Parameters Manager window determines the order that they appear in the FME Server Workspace App we create. We want parameters used for the same tool to be grouped together so we need to organize them and move things around. You can press the green plus sign to create a new group. Other User Parameters can be dragged and dropped into these groups, and the order of these groups can be changed. The User Parameters should be grouped and ordered as follows:

• Output Format:
• Output Coordinate System:
• Return Tiled Output:
• Split Point Cloud [Group]
  • Split Point Cloud:
  • Split By:
• Thin Point Cloud [Group]
  • Use Point Cloud Thinning:
  • Points to Keep
  • Interval:
  • Maximum Number of Points:
• Select Area:

28. Add Bookmarks (Optional)
Bookmarks are a good way to organize your workspace and make it easier for yourself and other users to understand what is going on in the workflow. 

To add a bookmark, select an area of the workspace (for example, a group of transformers) that you want the bookmark to encompass. Click the bookmark tool or right-click and select Create Bookmark. A bookmark appears around the selected area. For more information on bookmarks, see the documentation.
 

Part 2: Creating an FME Server Workspace App

The second part of this article explains how you can make an FME Server Workspace App using the workspace created in Part 1 of this article. This section of the article is relatively quick but rewarding as you create a user-friendly interface for point cloud data distribution. 

1. Publish your Workspace to FME Server
Save your workspace, then click Publish in the toolbar at the top of FME Workbench. Establish a new connection or use an existing connection. Choose a Repository of your choice and set the Workspace Name to PointCloudDataDownload.fmw. 

Press Next, click the box to check off Data Download and then click Publish.

2. Log in to FME Server
Go to your web browser of choice and then log in to FME Server. If you are running on a local machine, use http://localhost as the URL.

3. Upload the Point Cloud Data
Before the Workspace App can run the workspace, it needs the point cloud data we are using for this tutorial. On the left side of the FME Server home page, click on Files & Connections to expand the menu and click Resources. Create a new folder called Data. Click the newly created folder to go inside and create another folder, this time called PointCloud. Click the PointCloud folder we just created. Click the Upload button in the top right and select Files to upload the CoV_PointCloud.zip file which is available for download from the Files section on this article.

4. Build Workspace App
On the side menu, click on Server Apps to expand the menu and click Build Workspace App.

​​​​​​​In the next page that loads, set the parameters as follows:

• Name: PointCloudDataDownload
• Title: Point Cloud Data Download Service
• Repository: <Repository where you published your PointCloudDataDownload.fmw>
• Workspace: PointCloudDataDownload.fmw
• Service: Data Download

​​​​​​​Once everything looks right, press OK to create your Workspace App. You should be redirected to a new page that includes the link to your newly created Workspace App.

4. The finished Workspace App
You’ve now created a workspace app using published user parameters, that can transfer point cloud data smoothly over the Workspace App interface. Below you can see what your workspace app should look like. 

Data Attribution

The data used here originates from data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.