ArcGIS Utility Network Data Generating HTML Report

Introduction 

Generating reports from utility network data in ArcGIS is an essential task for utilities and organizations managing complex network assets. These reports provide valuable insights into the state of the network, asset inventory, and other critical information. With FME, users can create comprehensive reports in various output formats, such as PDF, Microsoft Word, HTML, and other formats supported by FME.

This tutorial will provide a brief demonstration of FME's report generation capabilities by guiding users through the process of creating a simple HTML report that includes charts, tables, and a web map.

While the scope of the report can encompass any desired attributes and information, this tutorial will specifically focus on generating a report that highlights attributes with dirty areas within the utility network data.

For a deeper understanding of generating reports or customizing HTML reports, refer to the hyperlinked documentation.

Brief Description of this Tutorial 

This tutorial is designed to demonstrate how to create simple reports of Utility Network Data using the HTMLReportGenerator transformer in FME. The tutorial will utilize a pre-prepared workspace that retrieves updated, inserted, and deleted features from a data source and rectifies the data for human readability.

However, you can also follow along with this tutorial using your own configured workspace. Your workspace should be set up to retrieve updated, inserted, and deleted features from a File Geodatabase, ArcSDE Geodatabase, and/or Esri ArcGIS Portal Feature Service. Additionally, ensure that your workspace resolves domain values and rectifies the data for human readability before generating the report.

By following this tutorial, you will learn how to leverage the HTMLReportGenerator transformer to create informative and visually appealing reports from your Utility Network Data, with a specific focus on using Esri ArcGIS Portal Feature Service for change detection and identifying features within dirty areas. This knowledge can be applied regardless of whether you use the provided workspace or your own configured workspace.

Requirements

• FME Form Workbench installed and licensed
• Esri ArcGIS Pro or ArcGIS Enterprise installed and licensed
• Basic understanding of Utility Network data structure and dataset, FME transformers, and FME and geodatabase
• Utility Network-configured ArcGIS Portal Feature Service item
• Complete the following FME & Esri ArcGIS Utility Network Tutorials:
  
  1. Reading Utility Network Data from ArcGIS Portal Feature Service and Writing to ArcGIS Portal Feature Service
  2. Reading Utility Network Data from ArcGIS Portal Feature Service and Writing to File Geodatabase
  3. Retrieving Updated/ Inserted/ Deleted Features in ArcGIS Utility Network Data
  4. Updating/ Inserting/ Deleting Features in ArcGIS Utility Network Data

Sources

• Report_Part1.fmw contains the completed workspace for this part of tutorial. Portal URL and Authentication for Original and Revised Version of Feature Classes will need to be re-configured to make the workspace usable.
• Report_Completed.fmw contains the completed workspace for this tutorial. Portal URL and Authentication for Original and Revised Version of Feature Classes and Output location of HTML will need to be re-configured to make the workspace usable.

Step-by-Step Instructions

Part 1: Loading Workspace 

1. Load the Workspace that is Configured to Retrieve Updated, Inserted, and Deleted Features from ArcGIS Utility Network Data and Rectify Resolved Domains
Open FME Workbench and configure the workspace for retrieving updated, inserted, and deleted features from a File Geodatabase, ArcSDE Geodatabase, and/or Esri ArcGIS Portal Feature Service, resolving for domains and rectified for human readability.

Alternatively, you can open FME Workbench and load the workspace from the source that has been prepared for this tutorial. This workspace should already have the updated, inserted, and deleted features retrieved using the ChangeDetector transformer and resolved for domain values. 

However, you cannot run this workspace due to the portal feature service configuration for this workspace, unless you specify your own data sources. This workspace will be used for this tutorial.

2. Review the Workspace 
If you are using the tutorial configured workspace, to rectify the data sources and make the workspace runnable, you need to configure the reader parameters to match your specific data sources:

• For the ArcGIS Portal Feature Service reader, update the "Dataset" parameter to point to your organization's ArcGIS Portal URL and specify the appropriate feature service.

Part 2: Preparing Attributes for Human Readability and Elements for HTML

1. Add an AttributeManager to the workspace
This step prepares for the popup information for the web map in the HTML. Continuing in the same workspace as Part 1, add an AttributeManager to the workspace canvas and connect it to AttributeManager_2, which is connected to the PythonCaller.

In the AttributeManager_3 Parameters dialog: 
A. Create a New Output Attribute and specify name as: Information 
B. For the value, open text editor and specify: Asset ID:@Value(assetid), Global ID:@Value(GLOBALID)
C. Click on “OK”. 

2. Add a DateTimeConverter to the workspace
Add a DateTimeConverter to the workspace canvas and connect it to AttributeManager_3. 

In the DataTimeConverter Parameters dialog: 
A. Select desired Date Attributes for DateTime Attributes

• For this tutorial, all the Date Attributes will be selected, these include:
  • creationdate
  • inservicedate
  • installdate
  • lastupdate
  • retireddate  

B. Set the Output Format to: %Y-%m-%d 
C. Enable Passthrough Nulls, Empties, or Missing 
D. Click on “OK”. 

3. Add a Clipper to the Workspace

This step retrieves modified features within Dirty Areas. Add a Clipper to the workspace canvas and 
A. Connect the Clipper Input port to the AttributeManager (connected to Dirty Areas reader feature type).
B. Connect the Candidate Input port to the DateTimeConverter.

4. Add a Sorter to the Workspace

This step prepares for the table in the HTML. Add a Sorter to the workspace canvas and
A. Connect the Inside output port from Clipper to the input of Sorter. 

5. Add StatisticsCalculator to the Workspace

Add a StatisticsCalcuator to the workspace and  
A. Connect output of the Sorter to one of the StatisticsCalculator   

Repeat Step 5 two more times. 

6. Edit StatisticsCalculator Parameter 

In the StatisticsCalculator parameter dialog for StatisticsCalculator:
A. Enable Group Processing and set ASSETGROUP for Group By 
B. Set Calculation Method to String
C. Specify ASSETGROUP for Attribute and enable Total Count
D. ASSETTYPE for Attribute and enable Total Count, Numeric Count, and Value Count
E. Click on “OK”.

In the StatisticsCalculator parameter dialog for StatisticsCalculator_2:
A. Enable Group Processing and set SystemSubnetworkName for Group By 
B. Set Calculation Method to String
C. Specify SystemSubnetworkName for Attribute and enable Total Count, Numeric Count, and Value Count
D. Click on “OK”.

In the StatisticsCalculator parameter dialog for StatisticsCalculator_3:
A. Enable Group Processing and set lifecyclestatus for Group By 
B. Set Calculation Method to String
C. Specify lifecyclestatus for Attribute and enable Total Count, Numeric Count, and Value Count
D. Click on “OK”.

7. Add HTMLReportGenerator to the Workspace

Add four HTMLReportGenerator to the workspace and  
A. Connect output of the StatisticsCalculator to HTMLReportGenerator 
B. Connect output of the StatisticsCalculator_2 to HTMLReportGenerator_2 
C. Connect output of the StatisticsCalculator_3 to HTMLReportGenerator_3
D. Connect output of the Sorter to HTMLReportGenerator_4

Your workspace should look similar to this:

8. Edit HTMLReportGenerator Parameter 
For this tutorial, two pie charts, one bar chart, table, and map will be created

In the HTMLReportGenerator Parameters dialog for HTMLReportGenerator:
A. Create Two Header and one Chart (Pie) under Page Contents
B. For the first Header, specify the Text as ‘Report of Utility Network Dirty Area Features’, set the Header Level as H1, and set the Text Alignment as Center
C. For the second Header, specify the Text as ‘Breakdown of Asset Types’, set the Header Level as H2, and set the Text Alignment as Center 
D. For the Chart (Pie), set the Label Attribute as ASSETGROUP and set the Data Attribute as ASSETTYPE.total_count
E. Click on “OK”.

In the HTMLReportGenerator Parameters dialog for HTMLReportGenerator_2:
A. Create one Header and one Chart (Pie) under Page Contents
B. For the header, specify the Text as ‘Breakdown of System Subnetwork’, set the Header Level as H2, and set the Text Alignment as Center 
C. For the Chart (Pie), set the Label Attribute as SystemSubnetworkname and set the Data Attribute as SystemSubnetworkname.total_count
D. Click on “OK”.

In the HTMLReportGenerator Parameters dialog for HTMLReportGenerator_3:
A. Create one Header and one Chart (Bar) under Page Contents
B. For the Header, specify the Text as ‘Breakdown of Life Cycle Status’, set the Header Level as H2, and set the Text Alignment as Center 
C. For the Chart (Bar), set the X Tick Label Attribute as lifecyclestatus and set the Data Attribute as lifecyclestatus.total_count
D. Click on “OK”.

In the HTMLReportGenerator Parameters dialog for HTMLReportGenerator_4:
A. Following the order, create Header, Table, Header, and Map (Esri Leaflet) under Page Contents.
B. For the first Header, specify the Text as ‘Table of Modified Features in Dirty Area’, set the Header Level as H2, and set the Text Alignment as Center 
C. Select the following Column Contents and specify the Column Name. 

D. For the second Header, specify the Text as ‘Locations of Modified Features within Dirty Areas’, set the Header Level as H2, and set the Text Alignment as Center 
E. For the Map (Esri Leaflet), set the Label Attribute as Information  
F. Click on “OK”.

9. Add AttributeCreator to the Workspace 

Add four AttributeCreator to the workspace to the workspace and 
A. Connect output of the HTMLReportGenerator to AttributeCreator 
B. Connect output of the HTMLReportGenerator_2 to AttributeCreator_2
C. Connect output of the HTMLReportGenerator_3 to AttributeCreator_3
D. Connect output of the HTMLReportGenerator_4 to AttributeCreator_4

10. Edit AttributeCreator Parameters

In the AttributeCreator Parameters dialog:
A. Specify ‘_order’ for Output Attribute
B. Set the Value to 1

Repeat Step 10 three more times, but set the value to

• 2 for AttributeCreator_2
• 3 for AttributeCreator_3
• 4 for AttributeCreator_4

11. Add a Sorter to the workspace

This step prepares for the order of elements for HTML. Add a Sorter to the workspace canvas and
A. Connect all the AttributeCreators to Sorter_2

Your workspace should look similar to this:

12. Edit Sorter Parameters

In the Sorter Parameter dialog: 
A. Specify ‘_order’ for Attribute
B. Set Numeric for Alpha/Num
C. Set Ascending for Order
D. Click on “OK”. 

Part 3: Consolidating HTML Elements and Writing HTML File

1. Add an HTMLLayouter to the Workspace

Add an HTMLLayouter to the workspace canvas and
A. Connect the output of Sorter_2 to the HTMLLayouter

2. Edit HTMLLayouter Parameters

In the HTMLLayouter Parameters dialog: 
A. Specify the Page Title as ‘Utility Network Dirty Area Dashboard’

3. Add a Writer to the Workspace
Add an HTML writer to the canvas. 

In the parameters, set the following: 
A. Set the Format as: HTML
B. Specify the file path and name you want to save to by clicking on the ellipsis (next to Dataset).
C. Click on “OK”. 

4. Connect the HTMLLayouter to the Writer 

Your final workspace should look similar to this:

5. Save and Run the Workspace
Now, save and run the workspace. View the created HTML file. 

Your HTML should look similar to this:

Data Attribution

Water Distribution Utility Network Foundation data is copyright 2024 Esri. Licensed under Apache-2.0. Source: ArcGIS Solutions.