Getting Started with BC MoF ESF

Introduction

BC Ministry of Forests (MoF) Electronic Submission Framework (ESF) is a set of XML/GML formats that allow clients to submit data electronically to BC Ministry of Forests and Range and Ministry of Agriculture and Lands. This article supplies some resources for getting started with this format.

ESF Training Resources

In addition to the ESF templates available on the website, the ESF training also includes a range of ESF examples of FTA and Results submissions, complete with source data, workspaces and output datasets. This way you can see what FME starts with and what it needs to generate. You can download this from: MoF_ESF_Training.zip

There are now ESF specific readers available such as FTA, ABR and Results.

FME Basics

The first question is, have you used FME much before? If not, I would highly recommend starting off by first downloading the tutorial from: http://www.safe.com/support/onlinelearning/overview.php

This will help you get a feel for how FME Viewer and Workbench work. In particular, it will help you get your feet wet with some basic examples.

Remember that FME is a data flow translation tool and its important to understand how features flow from the source datasets and their feature types to your destination feature types. For example, in FME, spliting data flow is the same as copying and combining is the same as appending. If you want records to actually merge attributes or features together, then you need to use combining transformers such as Aggregators, FeatureMergers and Joiners, etc.

ESF Tutorial Examples

Once you have reviewed the FME tutorial, it would probably be a good idea to see if you can get one of the ESF training examples to work. I would suggest starting with the FTA_HA example. Make a backup copy of the ESF example output and make sure you can generate similar results.

Note that the key to ESF submissions is building the right id keys so that there are logical relational links between the various esf_ids (esf_legal_description_id, esf_harvest_application_id etc in the case of FTA). Usually if a submission fails its because one of these ids is missing or doesnt match up with the associated table record or geometry feature its supposed to be associated with. You need to populate the various esf_ application and block ids so that the nested object structure and relationships are defined correctly.

Harvest Application Example

For example, in the FTA_HA example from the ESF tutorial (HarvestApplication_wCP_submission.xml):

 The harvest application has :
 esf_harvest_application_id hawcp-1
 esf_legal_description_id legal-desc-1

 and cutblock A has:
 esf_harvest_application_id hawcp-1
 esf_legal_description_id legal-desc-3

 and cutblock B has:
 esf_harvest_application_id hawcp-1
 esf_legal_description_id legal-desc-2

This means that cutblock A and B both belong to harvest application hawcp-1 and hence define the relationship between the cutblock objects and the harvest application object. Also, all the legaldescription ids need to be defined as individual records in the legaldescription feature table.

Viewing Output

When you do generate a result, try opening it with the FME Viewer using the appropriate ESF reader format. If the results look good, you can always go to the settings subform and check the validate box. This applies the same schema check that the MOF ESF submission website subjects your online submission to, and will fail unless everything is ok with your xml.

Once you understand the esf tutorial, you can take another shot at the ESF submission. As the movie explains, the template is not a solution but rather a starting point that just shows the destination datasets that need to be populated (some are optional). You may choose instead to start with one of the FTA or Results examples (whichever you need) from the ESF training, delete the source node feature types (data set layers or feature classes), and import feature type definitions from your source shape and excel files.

Reading Nonspatial Tabular Data

Normally, FME can read csv or excel files that are simple flat tables with one header row at the top that contain the field names. You may need to copy and paste some of your tables from your multi tab excel table into a seperate excel or csv file and remember to go into the reader settings to choose: read fieldnames from first row (or skip first row). See FME tutorial chapter 3 for an example of this.

Problems

If you run into trouple, try going through BC MoF ESF format Problem Solving.

Summary

So to sum up, you need to read your submission metadata from a table such as a csv or dbf file and write that to the ESF metadata feature classes. Then you need to read your cut block and harvest application polygons and then build a the necessary nonspatial and spatial feature classes and use attribute renamers to map your input fields to the correct output fields in the destination template. Also, you need to populate the various esf_ application and block ids so that the nested object structure and relationships are defined correctly. The idea is not to create your own gml, which wouldnt be compliant with the ESF schema, but rather to write both attribute records and geometry to the appropriate feature types according to the ESF schema so that all the information is contained in one xml document file.