How to Map Point Data without Latitude and Longitude: Leveraging Linear Referencing Principles

Introduction

In the previous tutorial, we introduced the AdobeGeospatialPDFReader and demonstrated how it can be configured to read and extract information from a PDF. By the end of the tutorial, you were able to generate an output feature containing all the relevant data being extracted. Now, in this follow-up tutorial, we will delve deeper into the GIS side of this project and explore methods for mapping all the Average Daily Traffic (ADT) collection points, even in the absence of geographical coordinates. To accomplish this, we will leverage the fundamentals of Linear Referencing System, which involves measuring positions and distances along linear features. So, without further ado, let's dive in!

Scenario

You have completed the initial phase of an FME Workspace, which involves reading a Traffic Speed Report in PDF format, extracting the desired data, and subsequently joining it to the nearest intersection point. However, it's essential to understand that the actual collection point may not be directly located at the intersection itself. Instead, it often happens at an adjacent mid-block location, depending on the direction of travel being recorded. For instance, when gathering vehicle counts for the Northbound direction, the collection point will be positioned north of the intersection. Conversely, for the Southbound direction, the collection point will be situated south of the intersection. While the distance between the intersection point and the collection point can vary, your GIS team has made a decision to maintain consistency by utilizing an arbitrary distance of 50 feet. As a result, you have chosen to leverage your expertise in the Linear Referencing System to explore an alternative mapping method and further enhance the previous section of this workspace.

Step-by-step Instructions  

Review Linear Referencing Fundamentals 

Let's start by reviewing the fundamentals of Linear Referencing:
Linear referencing is a technique employed in geographic information systems (GIS) and transportation engineering to locate and analyze features along a linear element, such as a road, pipeline, or river. It entails measuring positions and distances along the linear feature, typically using a linear reference system (LRS). In linear referencing, distances are measured from a known starting point, often referred to as a reference point or zero milepost. These distances are typically expressed in linear units, such as meters or miles. By associating attributes or events with specific positions or distances along the linear feature, linear referencing enables the analysis and management of information in relation to the linear feature.

Linear referencing finds applications in various domains, including mapping, route analysis, asset management, and transportation planning. For instance, it can help determine the location of road signs or utility poles along a road, measure the distance between two points on a pipeline, or analyze the distribution of accidents along a highway segment.
For more information on Linear Referencing, please refer to the blog article Linear Referencing 101: Managing Infrastructure Data

Now, let's explore how this applies to our ADT mapping project. We already have the reference point, which is our intersection point. The distance along the linear feature will be a fixed distance of 50 feet. However, in terms of the linear feature, we still don't know the specific street segment on which our collection point is located. Hence, our initial task is to select all the street segments that intersect the intersection point before identifying the segment of interest.

1. Add an Esri Shapefile Reader to Read in the Streets Shapefile
To start, let’s open up the workspace you were working on from the previous tutorial . Alternatively, you can also open the starting workspace (part2_startingworkspace) provided under Workspace Downloads 
Add an Esri Shapefile reader to the canvas and put it into the same bookmark of the Street_Intersections shapefile.



For Dataset, browse to the demo folder and select the file named Streets.shp. Make sure that Workflow Options is kept as default (Individual Feature Types) then click OK and Run. Confirm that 35,699 street segments were read in. 



2. Add a PointOnLineOverlayer for Spatial Join
Next, we will use the PointOnLineOverlayer to perform a spatial join between the intersections with the street segments shapefile. Double-click on the canvas and add a PointOnlineOverlayer transformer. Connect the FeatureJoiner Joined output port to the Point input port, and the Streets reader feature type to the Line input port. 



Open the parameters and fill in as follows: 

• Point Tolerance: 10
• Expand the Attribute Accumulation dropdown and check the Merge Attributes box
  • Accumulation Mode: Merge Incoming
  • Conflict Resolution: Use Original

Click OK then Run To This. Double check that the Line output port has 35,699 features and the Point output port has only 1 feature. 



Browse to the Table under Visual Preview, drag the bottom scroll bar to the right end and check if a new attribute named _overlaps is created. Double-click on the attribute name to sort values in descending order. You can see that this is a 4-way intersection, with four street segments intersecting at the intersection point. 



3. Add a Tester to Filter Out Irrelevant Street Segments 
Now, we will have to filter out all other street segments that do not overlap our intersection point. This can be done easily with the Tester transformer. Add a Tester to the canvas and connect the Line port from the PointOnLineOverlayer to it. Fill in the following for the parameters:

• Left Value: _overlaps
• Operator: >
• Right Value: 0 

Click OK then click Run To This and confirm that only 4 features passed through the Tester’s output port. Before determining the main linear feature along which to move the intersection point, it is crucial to ensure that the base vertex (the first or starting vertex) of all street segments overlaps with our intersection point. In other words, all street segments should originate from the same intersection point. To verify this, we will extract the coordinates of the intersection itself, as well as the coordinates of all the vertices that make up each street segment. Subsequently, we will utilize a Tester in our FME Workspace to filter out segments that do not have their base vertex overlapping with the intersection.

4. Add Two CoordinateExtractors to Extract Intersection Point and Street Segments Coordinates
First, add a CoordinateExtractor between the FeatureJoiner and the PointOnLineOverlayer. You can do this by clicking on the connection line between these two transformers, when the connection line is highlighted in blue, type in Coordinate Extractor and press Enter. After that, add another CoordinateExtractor and connect it with the Tester’s Passed port. 


For the first CoordinateExtractor, fill in the following parameters: 

• Mode: Specify Coordinate
• Output Attribute Names:
  • X: _x
  • Y: _y

This will output only one pair of coordinates since our input is only one intersection point. You can double-check that by selecting the feature in the preview table and looking at the Feature Information window, under Attributes. Please note down the value coordinates’ (_x and _y) values so we can use them for the next step. 



For the CoordinateExtractor_2, fill in the following Parameters: 

• Mode: All Coordinates
• Coordinate List Name: _segmentvertex

This will output several pairs of list attributes nested in every street segment feature. These attributes indicate the coordinates of each vertex forming the segment. Now if you pay attention to the last two list attributes from this screenshot (_segmentvertex{4}.x and _segmentvertex{4}.y ), do they look similar to the _x and _y list attributes extracted from the intersection point above? Well, that means this segment has its last vertex overlapping with the intersection point. In other words, the segment does not originate from the same intersection. However, we want the base vertex (_segmentvertex{0}) of every segment to have the same coordinates as our intersection point. Let’s see how many of these segments are having the same issue and find a way to fix that.

5. Add a Tester to Filter Out Segments that don't Share the Same Base Vertex with the Intersection Point
Add another Tester to the canvas and connect it to the Output port from the CoordinateExtractor_2. Set the parameters as follows: 

To explain, we are testing if the base vertex (_segmentvertex{0}) of the street segments have the same coordinates as the intersection point. Click OK then Run To This. You can see that there are two Passed and two Failed features, meaning that two out of four street segments do not have their base vertices overlapping with the intersection point. 



6. Add an Orientor to Flip the Vertex Order 
Now, let’s use a transformer named Orientor to flip the vertex order of the two segments at the Failed port above, making their last vertices become the base vertices. Connect the Failed port from the Tester_2 to the newly added Orientor. Fill in the following parameters:

• Orientor Type: Reverse
• Flip Appearance: Yes

Click Ok then Run To This. At this stage, you may not observe any immediate changes in terms of attributes or geometry. To verify if the vertex order has been flipped, we would need to extract the coordinates again. Alternatively, you can also select one of the segments from the preview table, then take a look at the Geometry dropdown at the bottom of the Feature Information windows to quickly check the first vertex coordinates. Are the _segmentvertex(4) coordinates the same as coordinates 0? That means we successfully flipped the vertex order of that street segment.
 


If you find the usage of the Orientor tool a bit perplexing, you can refer to the following illustration for a clearer understanding.


7. Add Another CoordinateExtractor to Extract the Flipped Vertex Coordinates
Add another CoordinateExtractor to the canvas and connect it with the Orientor. 
When prompted, use the following for its parameters: 

• Mode: All Coordinates
• Default Z Value: (leave blank)
• Coordinates List Name: _flippedvertex.

It is a good time to save the workspace again so click Save then click Run. Now, take a look at the Feature Information windows and check if the first pair of list attributes (_flippedvertex{0}) of either output feature is as follows:

Make sure that these numbers are the same as the intersection point coordinates you noted down from above, before moving on to the next step. If not, double-check the connection between the Tester_2 and Orientor, and confirm that the Failed port of Tester_2 is connected to the Orientor, not the Passed port. 

Save the workspace again and document every step with annotations and bookmarks. Your workspace should be similar to the following screenshot. 



8. Add a HorizontalAngleCalculator to Calculate the Offset Angles 
Up to this point, we have successfully obtained all the street segments where the base vertex is located at the intersection point. This ensures the consistency of our angle calculations and aids in determining the correct offset angle. Now, let's proceed to calculate the offset angle to determine the direction in which the intersection point will be moved towards. To do this, add a HorizontalAngleCalculator to the canvas. Connect the Tester_2 Passed port and the CoordinateExtractor_3 Output port to the HorizontalAngleCalculator. The HorizontalAngleCalculator transformer is distinguished by its green color, indicating that it is a custom transformer. In essence, a custom transformer is a combination of standard transformers condensed into a single transformer, which can be customized in a separate tab on the canvas.

More information on Custom Transformers can be found in this documentation About Custom Transformers.

When prompted, fill in the parameters as follows: 

• Calculation Angle from: Feature
• Point 1 X-Coord: _segmentvertex{0}.x
• Point 1 Y-Coord: _segmentvertex{0}.y
• Point 2 X-Coord: _flippedvertex{0}.x
• Point 2 Y-Coord: _flippedvertex{0}.y
• Angle Unit Measurement: Degree

Click OK then Run To This and confirm that the output has a total of 4 features. Next, we would have to remove the irrelevant attributes using another AttributeKeeper. Add an AttributeKeeper to the canvas and connect it with the HorizontalAngleCalculator. For Attributes to Keep parameter, click the ellipsis button and select the following: _angle, _azimuth, ASTREETDIR, BSTREETDIR, BSTREETNAME, CollectionPoint, FULLNAME, and TravelDirection. The List to keep parameters can be left blank.

Click OK, then Run To This and double-check that the output only has the attributes listed above. 



9. Add a TestFilter to Filter Out Segments per Cardinal Direction
Now that we have calculated the angles for all the segments, it's time to define the range of angles for each cardinal direction: North, South, East, and West so we can tell which segment is going towards which direction. To be specific, segment angle that falls within 45 degrees to 135 degrees will be assigned as North, 135 to 225 degrees will be West, 225 to 315 degrees will be South, and the rest is East. Note that the angle direction is counterclockwise starting from X axis to match the Polar Angle output from the HorizontalAngleCalculator. 



We will also consider the CollectionPoint attribute in this process. To accomplish this, add another TestFilter transformer to the canvas and configure its parameters using the instructions provided below. 
First, double-click on the first row of the If clause, under the Test column to open up the Test Conditions dialog box, and fill in the following: 

• Left Value: _angle
• Operator: In Range
• Right Value: Click the ellipsis button and specify the range as follows:
  • Lower Limit: Greater than value
  • Value: 135
  • Upper Limit: Less than value
  • Value: 225 
• Mode: Numeric
• Logic (second row): AND
• Left Value (second row): CollectionPoint
• Operator (second row): Begins With
• Right Value (second row): W
• Mode (second row): Case Insensitive
• Output port: West

Click OK, then double-click on the second row to configure the next Else If clause for the West direction: 

• _angle In Range [225,315] Numeric. (Note that the square brackets indicate an inclusion of the limit values. Greater than or equal to, and less than or equal to) 
• AND CollectionPoint Begins with S Case Insensitive. 
• Output Port: South

Next, configure the second Else If clause (third row) for the East direction: 

• _angle In Range [45,135] Numeric. (Note that the square brackets indicate an inclusion of the limit values. Greater than or equal to, and less than or equal to) 
• AND CollectionPoint Begins with N Case Insensitive. 
• Output Port: North

Similarly, the last Else If clause should be configured for the North direction:

• _angle In Range [315,360] Numeric. (Note that the square brackets indicate an inclusion of the limit values. Greater than or equal to, and less than or equal to) 
• AND CollectionPoint Begins with E Case Insensitive. 
• Output Port: East

Click OK and use the following screenshot to double-check all the parameters for TestFilter_2:



Save the Workspace then Click Run. Confirm that the North port has one output feature and the Graphics window is showing only the North segment of N 1st street. Does this segment correspond to the CollectionPoint and TravelDirection we extracted from the PDF? (N of intersection, Northbound). 



10. Add an Extra a TestFilter to Differentiate Segments with Non-cardinal Direction
The TestFilter we previously set up is quite effective in filtering most of the street segments with cardinal directions. However, there may be cases where it fails to accurately identify segments with non-cardinal directions. For instance, if we examine a different location from another PDF file, the previously configured TestFilter may become slightly confused and produce two segments through the South port. This happens when both azimuths meet the test condition, which can lead to an incorrect offset angle calculation. To address this issue, we will create an additional TestFilter to capture such instances and ensure accurate filtering.

First, we need to add a customer transformer named FeatureCounter to count the number of segments that passed through the TestFilter_2. Connect all four ports (North, South, East, West) of the Testfilter_2 to the new FeatureCounter. Set the Attribute Name to something more intuitive (_segmentcount)
Once done, click OK to confirm the configuration.



Next, let's add another Tester transformer to filter out all the non-cardinal segments. Our objective is to verify if only one segment passed through the TestFilter_2. To achieve this, we will condition the Count Attribute created from the FeatureCounter above (_segmentcount). If the count is 1, indicating that our TestFilter_2 succeeded, the segments will be directed to the Passed port. On the other hand, if the count is greater than 1, indicating that multiple segments have passed, they will all be output to the Failed port and subsequently be directed to an additional TestFilter thereafter. Now, let's configure the parameters for Tester_3 as follows:

• Left Value: _segmentcount
• Operator: =
• Right Value: 1

Lastly, add another TestFilter and connect it with the Failed Port of the Tester_3. Open up its parameters window to start configuring the Test Conditions. 



Click on the first If Statement to open the Test Conditions dialog box, the first Test Clause will be configured as follows: 

• Comparison Mode: Case Insensitive
• Output Port: StreetA_NS

The purpose of this Test Clause is to compare the street direction attributes (ASTREETDIR and BSTREETDIR) from the Intersection shapefile with the CollectionPoint attribute extracted from the PDF. Specifically, we are looking for segments whose direction aligns with the collection point. For instance, in the given location, Street A has an East-West Direction (ASTREETDIR), but the CollectionPoint is located south of the intersection. As a result, Street A will be eliminated due to a direction mismatch. The additional clause (FULL NAME Contains ASTREETNAM) ensures that the filtered segment has a matching name with the one recorded in the intersection shapefile. Consequently, no feature will be written to the first output port in this case.




Click on the second row of the Port Definitions to configure the Else If statement, the second Test Clause will be slightly different from the first one, which will test the direction and name of Street B instead of Street A:

• Comparison Mode: Case Insensitive
• Output Port: StreetB_NS

The result from this test clause is indeed a match and one feature will be written out to this port. Because the CollectionPoint is South of Intersection and Street B Direction is North-South, street B name from the Intersection feature and the segment name from Street Segments feature is also a match. 



For the opposite direction, East-West, continue to populate the next two Else If statements as follows:
Second Else If statement:

• Comparison Mode: Case Insensitive
• Output Port: StreetA_EW

Last Else If statement:

• Comparison Mode: Case Insensitive
• Output Port: StreetB_EW

Use the following screenshot to double-check your parameters: 



In case you want to check if our extra filters are working correctly, use the SENTER RD S OF CAPITOL EXPY SN.PDF as input for the AdobeGeospatialPDF Reader then Run the entire workspace. There should be only one feature being output through the TestFilter_3. Now we are ready for the next step, merging the selected segment with the intersection point. 

11. Add a FeatureMerger to Merge the Filtered Segment with the Intersection Point
Before offsetting the intersection point, we need to merge it with the specific segment output from the previous step. Let's add another FeatureMerger transformer to the canvas. Connect its Requestor port to the FeatureJoiner inside bookmark 6 (Summarize ADT and Join attributes to Intersection point), which can be found by scrolling to the left of the canvas. For the Supplier port, connect all four cardinal direction ports from the TestFilter_3, as well as the Passed port from Tester_3. There's no need to worry about connecting multiple ports to the Supplier, as there will only be one output feature that successfully passes through all the filters we've set up previously.



Open up the parameters and fill in the following: 

• Supplier First: No
• Join On
  • Requestor: 1
  • Supplier: 1
  • Comparison Mode: Automatic

This configuration is similar to what we did in the previous demo, performing a table join even when there is no common key attribute. Because we want to attach all the attributes from the street segment to our intersection, especially the _azimuth attribute calculated from above. 
Confirm that the intersection point is output at the Merged port. We are now ready to offset the intersection point.

12. Add an Offsetter to offset the intersection point
Connect the FeatureMerger_5’s Merged Port to the newly added Offsetter, then click on the cogwheel icon to configure its parameters as follows: 

• Mode: Polar Coordinate
• Polar Coordinate:
  • Distance: 50 (Since we are using the NAD83 California zone III projection, FME will automatically recognize feet as the distance unit. You can confirm the Coordinate System under the Geometry dropdown in the Feature Information window.)

• Polar Angle: _angle. (This _angle attribute is created by the HorizontalAngleCalculator, along with the _azimuth attribute. Note that its' direction is counterclockwise, starting from X axis, which aligns with the Polar Angle used for the Offsetter.)

Below is an illustration of the entire offsetting procedure. 

Click OK, then click Run Entire Workspace and confirm that the Offseter output is 50 feet away from the intersection point, towards the North direction. Next, let’s have a final attribute cleanup before outputting the result to Esri’s File Geodatabase.  



13. Add an AttributeManager to Clean up Attributes
After adding another AttributeManager to the canvas, it to the Offseter and configure the parameters as follows: 

• FACILITYID: Remove
• FULLNAME: Remove
• ASTREETDIR: Remove
• ASTREETNAM: Rename to StreetOne
• BSTREETDIR: Remove
• BSTREETNAM: Rename to StreetTwo
• INTNUM: Rename to NearestIntersection
• LONGITUDE, LATITUDE, _azimuth: Remove. 
• Make sure to use the up and down arrows at the bottom to rearrange the order of the attribute like the screenshot below. 

Click OK, then save the workspace and click Run Entire Workspace. Ensure that the output feature has all the relevant attributes including Date, IntersectionID, TravelDirection, ADTOne, ADTTwo, NearestIntersection, StreetOne, and StreetTwo. Our last step is to write out the result ADT point to a File Geodatabase using the Esri Geodatabase (File Geodb Open API).

14. Add an Esri Geodatabase (File Geodb Open API) Writer
Click Add Writer on the toolbar and set the format as Esri Geodatabase (File Geodb Open API). For Dataset, browse to the demo OutputGDB folder and key in AverageDailyTraffic to create a new file geodatabase. 
C:\SJDOT_ADT_Mapping\OutputGDB\AverageDailyTraffic.gdb
Leave all other parameters as default then Click OK. When the Select Feature Type dialog box appears, select Street_Intersections then click OK. After the Writer is added, click on the cogwheel icon to open the Feature Type parameters window, fill in the following information:

• Feature Class or Table Name: ADT_points_mapped
• Geometry: geodb_point
• Feature Operation: Insert
• Table Handling: Create If Needed
• Click on the User Attributes tab and make sure Attribute Definition is set to Automatic.

Click OK, make sure the workspace is saved, then click Run Entire Workspace. Select the writer on the canvas then click the View Written Data button and double-check that our ADT point was written out to a new feature class named ADT_points_mapped inside the AverageDailyTraffic file geodatabase. 


Don’t forget to document the entire workspace with annotations and bookmarks. You can use the screenshot below as a reference. 



Congratulations on completing the second part of the PDF Reader and ADT mapping workspace! You have successfully mapped point data without having longitude and latitude coordinates. By utilizing the principles of Linear Referencing, such as a reference point (intersection), a linear feature (street segment), and a specified distance along the linear feature (50 feet), you were able to overcome complex GIS challenges using FME Form. You should now feel more confident in working with list attributes, using Custom Transformers, and configuring various Testers and TestFilters to suit your project requirements. Well done!

Data Attribution

The data used is made available by the City of San Jose’s Department of Transportation.