Search

FME & CIM XML for Electric Utilities

Liz Sanderson
Liz Sanderson
  • Updated

Files

Introduction

CIM (Common Information Model) is an open standard developed by the electric power transmission and distribution industry. One of the CIM’s key objectives is to provide a way for application software, such as GIS, ADMS, and SCADA, to exchange information about the configuration and status of electric utility networks.

The CIM defines a common set of electric data objects for both transmission and distribution. There are several IEC standards related to CIM, listed here. The CIM data model is extensive and complex, and covers all aspects of electric transmission and distribution. Typically, the CIM XML data model is narrowed down through a ‘profile’ that limits it to a specific data exchange. For example, you might have a profile for transmission or distribution, or even a profile that limits the exchange of data to the facilities managed in a GIS. Ideally, an organization would adopt a single CIM profile that the majority of its applications can work with.  CIM is usually serialized as RDF XML. 

FME has always been able to read and write CIM XML data exchange documents. However, the introduction of the XSD-Driven XML reader & writer format to FME greatly simplifies working with CIM XML (and XML in general). In earlier versions of FME, users had to use the XMLTemplater to create CIM Classes and populate the appropriate attributes. This approach is summarized in earlier presentations on CIM XML. The disadvantage of this approach meant that a change in the CIM ‘profile’ would require detailed changes to the XML Templates. With the advent of the XSD-Driven XML format, all you need is an XSD that represents the XML data model you need to create. The CIM Classes are then represented in FME as FME Feature Types, just like the majority of other FME formats.

CIM XML ETL

As with all ETL (Extract, Transform & Load) workflows, there are three main steps. In the context of creating CIM XML, these are:

  • Extract: reading the source data - usually the electric utility facilities data that is housed in the utility GIS. Examples are Esri Utility Network, Esri Geometric Network, GE Smallworld, and Intergraph G/Tech. 
  • Transform: map the source data model to the target (CIM) data model. The CIM target data model is usually constrained by a specific  ‘profile’. The profile might be specific for an application (i.e. OMS system) or domain (transmission vs. distribution). The source and target data models are often widely divergent, so the mapping of feature classes, attribute names, values, domains, and classes might be detailed and complex. Some objects might not exist in the source data, so these have to be created during the ETL workflow, for example, CIM Terminals and ConnectivityNodes
  • Load: serialize the CIM XML using the FME XSD-Driven XML writer

Depending on the source data, additional steps might be added:

  • Source data cleanup - i.e., removing unused attribution
  • Building topology - source data might not include all the network topology elements required, such as connectivity nodes. Complex edges might need to be split. 
  • Key Mapping: converting derived UUIDs into UUIDs that conform to the CIM profile in use to create persistent IDs.

Writing CIM XML Examples

Safe has created CIM XML writing examples for both the Esri Geometric Network and the Esri Utility Network data models. These can easily be restructured for other GIS data sources, such as Hexagon G/Technology or GE Smallworld.

Esri Utility Network data model to CIM XML example

The example workflow (attached FME 2022.2) is based on EPRI IOP (2022) sample data but defined in an Esri Utility Network database. To access Esri Utility Network datasets (e.g., a feeder), Esri provides an export tool that generates a JSON document representing the Utility Network data for the region of interest. The JSON exports can also be automated either using FME or the Esri Data Interoperability Extension. This JSON document of the UN data is the starting point for the FME conversion to CIM XML. For Esri Geometric Data Models, FME can read directly from the Esri Geodatabase. For this example, the output CIM XML is defined by a CIM profile developed during the EPRI IOP event. 

The workspace for Utility Network source data is divided into several sections that are indicated by bookmarks and annotations in the workspace:

  • JSON Interpretation (bookmark “JSON processing”)
    • JSON is read as a text file, and attribution is created. Attributes are supplied in two forms - fieldValues and networkAttributeValues. 
    • Bookmarks - JSON Cleanup. GlobalID cleanup - also annotated Schema Mapping: 
      • New JSON attributes must be exposed in this bookmark
      • Clean-up source data: remove unused attribution and set some fundamental attributes, i.e. GlobalID
  • Create UN Feature Classes (bookmark “Geometry builders + UN Feature Classes”)
    • Esri UN feature classes are created for features, connectivity, and associations. 
    • Also creates geometry that is not needed for CIM but helps with troubleshooting.
  • UN Connectivity & Associations: (bookmark “UN Connectivity”)
    • Joins Edges & connectivity objects
    • Joins UN assemblies and connectivity objects. 
    • Joins assemblies to devices
    • There may be duplicate connectivity records in the UN JSON, so they are resolved by aggregating them.
  • Lookup Tables: (bookmark “Lookup tables”)
    • Domain Mapping from Excel spreadsheet lookup tables: Phases, Voltage, etc.
  • Final Schema Mapping (bookmark “CIM XML & Schema Mapping”)
    • This bookmark normalizes all Esri UN features into CIM XML classes and performs schema mapping.  ‘Intelligent’ CIM mRIDs are created - see note below. 
    • Coordinate System, Locations, Position Points are all in one bookmark: “Location”
    • Terminals and ConnectivityNodes ("Terminals & ConnectivityNodes"" bookmark) are largely derived from the UN connectivity. There is pretty well a one-to-one equivalence between  Esri UN junctions and CIM connectivity nodes. So, anywhere there is an existing Esri junction, those are mapped directly to the equivalent CIM Connectivity Node. Esri UN Junctions are usually used at junctions of conductors. In some cases, junctions are not required in Esri UN, for example, where a conductor connects to a device (i.e. switch). 

In these cases, FME's UN to CIM XML migration workspace will insert a new connectivity node at the conductor line end and then add the logic to connect the line end to the connectivity node to the device, i.e.: 

  • Conductor > C.N. > Device

Terminals in Esri UN also have a roughly one-to-one correspondence with the CIM XML terminals. Esri UN lines and devices all have terminals. Where FME has to insert a connectivity node (as above), it will also create the corresponding C.N. terminals to maintain the connectivity, i.e.:

  • Conductor-T > T-CN-T > T-Device 

The exception for terminals in Esri UN is for Transformers: In CIM XML, the transformer terminals are on the TransformerTankEnd - so the Esri Transformer terminals have to be "shifted" to the new TransformerTankEnd objects (which also don't exist in the Esri UN)

  • Special Cases & Supplementary Data: i.e., asset management is added for Transformer Info classes from a secondary data source (simulated in this workspace as an Excel spreadsheet). Usually, this would be read from an asset management database -  "Asset Database" bookmark
  • Topology: topology is based on the UN connectivity classes, which are used to create CIM Terminals and ConnectivityNodes using the from/to connectivity - bookmark “Terminal & ConnectivityNode”.
  • Boundary Classes: CIM XML supports boundary classes where different datasets can be joined into a model. In this example, the Feeder Breaker is in a separate CIM XML file (transmission) so a boundary node must be included to link to the Breaker ("Boundary Classes" bookmark).
  • Writer Feature Types: This is where the XML is serialized. These need to be kept up-to-date with the CIM profile. Use Update Feature Types to keep attribution up-to-date. Instructions for adding a new XSD-Driven XML Writer are in the workspace.

Note: Derived IDs

FME derives UUIDs from the original Esri UN GlobalId. For example, ACLineSegment:

  • Original GlobalId: {2451835f-7ab0-49e0-9399-917b96c39e51}
  • Derived segment : GlobalId = urn:uuid:2451835f-7ab0-49e0-9399-917b96c39e51_434_1000
  • Corresponding ACLineSegmentPhase: GlobalId = urn:uuid:2451835f-7ab0-49e0-9399-917b96c39e51_434_1000_C etc.
  • Corresponding Location: GlobalId = urn:uuid:a582abf6-862f-46bb-85a2-297e7cf221bd_0_1000_L0

The original AC Line Global ID is: {2451835f-7ab0-49e0-9399-917b96c39e51}

  • Suffix _434_1000 is added to lines from measure attributes since Esri has segmented the AC Lines.
  • Suffix _C is added to the ACLineSegementPhase
  • Suffix _L0 added for the first location, and so on. 

For EPRI, only 'pure' UUIDs are valid IDs, so the FME “derived” UUIDs are mapped to confirming UUIDs in the Key Mapping workspace. Subsequent runs of the key-mapping workspace will preserve the conforming UUIDs so target systems can get persistent IDs.

Generating CIM XML XSD

The CIM XML workspace needs an XSD file to define the CIM RDF schema (equivalent to the RDF Schema, but NOT the same as the Enterprise Architect XML XSD). FME can generate an appropriate XSD file from a sample dataset using the CIM XML to XSD Creator workspace discussed below.

Esri UN to CIM XML workflows

The attached zip file (in the Files section of this article) contains several FME workspaces (FME 2022.2) and sample data. These are summarized below.

Workspaces (in order they are typically executed):

Esri UN JSON to CIM XML

EsriUN_to_CIMXML.fmw.  Workspace for reading Esri UN JSON and creating a CIMXML file against a CIM XML profile
Inputs: 

  • CIM XML Instance Set Dataset Name: <any text describing this data  extract> 
  • Source Esri Utility Network JSON File: EsriUN_ExampleData.json. UN JSON export from Esri UN 
  • Source Asset Info Excel File:

Asset Management data: GIS does not contain all the transformer info, so the example workspace simulates an asset management system in this spreadsheet:  AssetManager_TankInfo.xlsx

  • Source TD Boundary Model XML File:

TD Boundary Model Basic Golden InstanceSet (XML)  - this gives the boundary node that will connect to the substation Breaker (which is NOT modeled in the Distribution profile): TD Basic Golden InstanceSet.xml. This is the Transmission to Distribution Boundary (note: Transmission substation contains the Switch Breaker). 

  • Domain Lookup Tables (Schema Mapping): 

SchemaMapping spreadsheet - phase & base voltage lookup. Used in Reader and DatabaseJoiners. Also has vectorGroup lookup for secondary XFR phases & number of  Tanksends (EsriUN_to_CIMXML_Schema_Mapping.xlsx).

  • CIM XML Application Schema (XSD): XSD for the current CIM profile: cim.xsd

Outputs:

  • Destination CIM XML File:

Other CIM XML Workspace Parameters: 

  • CIM XML Resource XML Name Space:  the CIM namespace that matches the profile being used. 
  • XSD XML Document:  Adds the CIM XML namespace headers as well as the CIM custom : 

<?iec61970-552 version="3.0" method="DataSet"?>

  • RDF ID Prefix: urn:uuid: - prefix added to all UUIDs - currently under review by the CIM standards since it creates an invalid XML ID.
  • Output CIM XML Coordinate System: NAD83 / Illinois East (ftUS) [EPSG:3435]
  • Include Point of Common Coupling: (these are the CommonCoupling objects sometimes used to model meter connections) Usually No.

UUID Key Mapping

CIMXML_KeyMapping.fmw: FME create 'derived' rdfIDs as described above. Some CIM XML applications will only accept a true UUID. The key mapping workspace creates a key mapping table in a SQLite database to create UniqueIDs for each object, as well as preserving the original FME-generated IDs. This means that subsequent exports of the same data (i.e., the same feeder) will result in persistent UUIDs for the target application. Make a copy of the "KeyMapping - Template.sqlite" database file and then load the copy with new key maps.

CIM XML Validation

CIMXMLFile_Validator.fmw - CIM XML Validation workspace. The validation workspace complements validation by other applications such as CIMSpy. Validation results output by this workspace: as an Excel spreadsheet, contains several  tabs:

  • MissingReference: shows if any reference errors are present (generally tied to if the Feeder is not present in the XML file). 
  • Reference Detail:  - shows which objects are affected by the above Missing Reference. 
  • UniqueObjectClassStats:  Similar to the CIMSPY Object Summary, feature counts of each object. These should match up against the CIM SPY Object Summary. 
  • Schema Validation tabs

CIM XSD Creator

cim_sample_to_xsds_with rdf_about.fmw: This workspace will generate the CIM.xsd from a sample CIM XML dataset.

Changing the CIM XML profile

This example UN JSON to CIM XML workflow uses a specific CIM XML profile (data model or schema) developed during the 2022 EPRI IOP project for electric distribution. Ideally, within an organization, a single CIM XML profile can be shared across all target applications. But, different organizations or target applications may have different CIM XML data requirements and hence different CIM XML profiles. If the CIM XML profile changes, then the UN to CIM XML workflow needs to change. 

  • If the data model of the source Esri UN changes, then additional attribution and feature classes may be represented in the Esri UN JSON export. Check that any new, required attributes are exposed in the "JSON processing" bookmark
  • Update FME cim.xsd 
    • Use a sample of CIM XML data for the required profile to generate a new CIM XML cim.xsd.
    • Make sure you copy the new XSDs to the CIM XML workspace XSD folder
  • Update the FME XSD-Driven XML writer feature types in CIM XML workspace
    • Point FME at the revised cim.xsd file in the XSD-Driven XMLwriter
    • Check all attribute values mapped correctly in the appropriate AttributeManagers for each feature class. 
  • Re-run workspace
    • Make sure you update all the parameters - particularly the new cim.xsd location
  • FME Validation: run the FME CIM XML Validation workspace. This checks for:
    • Summary class count statistics (should match what otters import and the Jun validator)
    • Duplicate primary keys (mRID / rdf:about)
    • Missing references (all rd:resource’s should link to an rdf:about). Exception is the AdditionalEquipmentContainer, which is an external reference to the Feeder Class in the TD)
  • Key Mapping
    • Run the key mapping workspace
    • Either start from scratch (Truncate option) or update the key mapping if new data is added
  • Validate using GMDMValidator available from EPRI or use CIMSpy

Reading CIM XML

CIM XML is usually serialized as RDF XML, which is a very flat, highly normalized XML data structure - not too dissimilar to a normalized relational database. This makes it very straightforward to read CIM XML. In FME, you can use the XML reader to read back the CIM XML and use an  InLineQuerier transformer to join together the necessary CIM XML classes that can then be mapped to the target output schema.

Was this article helpful?

We're sorry to hear that.

Please tell us why.

As of January 14th, 2026, comments on knowledge base articles have been closed. To make sure questions don’t get missed and to enable more community support, we’ve moved discussions to the FME Community. If you have a question or a comment about this article, please create a new post or create a support ticket.