Importing into Smallworld

Introduction

Welcome to the FME for Smallworld 5 tutorial. This article covers importing data into your Smallworld 5 database, including how to import a simple geometry object.
 

Considerations for Import

FME can’t create new objects in the Smallworld database. You can only load into existing objects. This means that on import, the FME Smallworld writer data schema must accurately match the existing Smallworld object schema. This includes making sure you know which fields and geometries are mandatory, and also ensure that you load valid enumerated list (domain) values. It’s useful to obtain a descriptive CASE report of the Smallworld objects that will receive the imported data.

Before starting any data import, it's a good idea to inspect both the source dataset in the FME Data Inspector and the Smallworld objects to obtain a full understanding of the data schemas
 

Steps for Successful Import

1. Identify the correct Smallworld alternative or version to import into. FME will make the alternative writable. But FME cannot create a new alternative.
2. Use FME Data Inspector with the FMENOFACTORY service to inspect existing Smallworld records in the target object. Take note of the data schema that will need to be created in the FME workspace.
3. Create the FME Workspace in FME Workbench to translate the source dataset schema into the destination schema on the Smallworld objects.
4. Test the workspace using the ‘Redirect to Inspection Application’ menu option, to ensure that the data is restructured as planned.
5. Import the data into a test alternative and check the results.
6. Import the data into the production alternative or “top” alternative – consider setting a checkpoint before you run FME so you can rollback if needed.
7. Inspect the updated Smallworld objects to verify that the imported data is loaded correctly.

Import Simple Geometries

The Electricity Cable objects in the Smallworld database have a single Centreline geometry. You’re going to import some Proposed cables for a new sub-division into the Cambridge database from an AutoCAD 3D Map dataset. First, inspect the destination Smallworld objects and the source AutoCAD dataset.
 

Step-by-step Instructions

1. Inspect Source Dataset

Download the electric-networkproposed.dwg dataset.

Open the FME Data Inspector. Click on File > Open Dataset:

Using FME Data Inspector, open the source electric_networkProposed.dwg using the AutoCAD RealDWG reader. On the Parameters, set AttributeSchema.

Note: FME 2017 32bit no longer supports AutoCAD 3D Object. AutoCAD 3d Object is only available on 64bit FME in FME 2017.

The AutoCAD file electricity_cable layer has a status attribute “In service” and “Proposed”. You’ll only import the "Proposed" cables as the “In service” cables already exist in the Cambridge database.

In this case, because there is a .prj file in the same directory as the .dwg file, the coordinate system for the AutoCAD drawing can be determined – OSGB-GPS-2015 (meters)

2. Compare Electricity Cable in Smallworld and FME Data Inspector

Using the Smallworld Application Manager, open the Professional - Smallworld Core application and inspect the Cambridge sample database.

• Use the bookmark: Electric Network to zoom in on the Cambridge electric network data
• Choose the Select Mode tool and click on the “Union Lane LV” cable object
• Check the object Editor for the cable attributes and geometry

In the FME Data Inspector. Click on File > Open Dataset:

• Zoom in to the same area as above
• Choose the Select Features tool and click on the “Union Lane LV” cable feature

Compare the Smallworld and FME representations of the electricity.cable.

In FME the cable geometries are represented by a single geometry (centerline) - in the case of “Union Lane LV” a multi-segment path geometry.

3. Inspect Destination Schema

In the Smallworld GIS, use the Editor to inspect the electricity.cable objects schema. Notice that the sys_id field is named ‘id’ is the key field. If you look at a CASE report of the electricity.cable it is a mandatory field.

The Status attribute is an enumerated list with possible values:

• In service
• Abandoned
• Accepted
• Out of service
• Proposed
• Removed

The voltage attribute is an enumerated list with possible values:

• 11kV
• 132kV
• LV

When loading the Status or Voltages attribute, you need to ensure that one of these mandatory enumerated values is set.

Take note how the source schema relates to the destination schema:

In this example there is not much schema mapping is involved. In most cases you’d need to add some FME schema mapping transformers such as AttributeManager, AttributeCreator, AttributeRenamer, AttributeValueMapper, or SchemaMapper.

Note that the Smallworld geometry has a name: centreline

4. Select the Smallworld Destination Alternative

In the Professional - Smallworld Core application, click on File > Versions to open the Version Management application.

In the Version Management application, add a new ‘alternative’, if it doesn’t already exist:

Double click on the alternative to make it the current alternative.

5. Generate Workspace

Open FME Workbench and on the Start tab, choose the Generate Workspace option.

(Be sure to use the | symbol as the first character for Alternative)

When you uncheck the Expand Blocks into Entities, FME will only read the insertion point for that block, and will not read the block definition. This is usually what you want when loading CAD data into a GIS.

In the Generate Workspace dialog, place a check mark on the “Import feature type definitions” checkbox. Click OK.

Click OK

6. Import Writer Feature Types from Smallworld

The “Import Writer Feature Types” dialog will appear. Enter the usual Smallworld connection parameters.

Click on the Parameters button, and select:

• Table List: electricity.cable

Click OK to close the import feature types dialog.

7. Select AutoCAD Layers

The Select Feature Types dialog will open. This is the AutoCAD layer selection. Select electricity_cable and OK, to generate the workspace.

It is advisable to use the Import Writer Feature Types option whenever a Smallworld writer is added. The Smallworld schema will generally be different to the reader schema, and the correct schema can only be obtained by explicitly importing the Smallworld feature type schema.

8. Save Workspace

Save the workspace locally at

<My Documents>\My FME Workspaces\Smallworld4.fmw

9. Connect Reader and Writer Feature Types

Connect the electricity_cable reader feature type to the new electricity.cable writer feature type.

Expand the attribute lists for the reader and the writer.

Note: The FME Smallworld 5 writer handles Smallworld geometries in a completely different way to Smallworld 4. The Smallworld 4 versions of the workspaces are all attached at the bottom of this article for reference. Geometry types are no longer exposed as attributes and no longer use the sworld_geometry{} list attributes. Old workspaces are still compatible in Smallworld 5.1.

10. Filter the Proposed cables

You only need to insert the proposed cables. Add a Tester transformer to keep the “Proposed” cables.

11. Set Geometry Name

In the Data Inspector, we noticed that the cable object has a geometry name 'centreline'. You need to set the name of the FME geometry to match the name of the Smallworld geometry column.

Add a GeometryPropertySetter after your Tester and set:

Click OK on the GeometryPropertySetter parameter dialog.

Your workspace should look something like this:

12. Save and Run

Save the workspace. Run the translation.

Inspect the updated data in the Professional - Smallworld Core application. The proposed cables have been added along Oak Tree Avenue. You might have to select View – Refresh.

In the Smallworld GIS, open the Smallworld Explorer. Open the folders Electricity, Electricity Network and select Cable.

Problems?

The most common data load error is not loading the Smallworld geometry correctly.

In Smallworld Explorer, if the geometry column is not flagged or the Length is 0.0, then you have probably not set the geometry name correctly in the GeometryPropertySetter. Under the Writers menu, set the “Redirect to Inspection Application” and re-run the workspace. Inspect the results in Data Inspector to ensure the geometry name is configured correctly.
 

13. Clean-up

In the next section you’re going to reload the data. In Smallworld Explorer version manager, select the 'fme_training' alternative, rollback to the 'begin' checkpoint by double clicking on the check point and selecting Rollback to Checkpoint. Or in Smallworld explorer select all the Oak Tree Avenue LV records you just inserted and delete them and commit the deletes. This will give you a fresh start.

Database Operations

At this point you let FME default to a database insert operation. If we wanted to update the existing cable features we’d need to use an update operation. The fme_db_operation format attribute is used to do updates. FME uses the ‘fme_db_operation’ attribute on the Smallworld writer features to specify the database operation for each feature. The fme_db_operation attribute can be set to:

• INSERT
• UPDATE
• DELETE

We’re going to discuss the fme_db_operation attribute in more detail in a later article in this tutorial series.

In the next steps, you’re going to set the value of the 'known_as' attribute to “connector”, if it has a null value in the AutoCAD dataset.

1. Edit Workspace

Continue editing the workspace you’ve been working with, or open the workspace smallworld4a-complete.fmwt.

The completed workspace for this section is: smallworld4b-complete.fmwt.

2. Modify Tester

You do not need to update the named cables such as “Union Lane LV” or “Lower Milton Rd LV” so you can filter them out. Open the Tester properties and add the test:

known_as Attribute Is Empty String

3. Add Conditional Tests to AttributeCreator

Insert an AttributeCreator transformer between the Tester and the GeometryPropertySetter. Now add a couple of conditional expressions to the AttributeCreator. To create a conditional expression in AttributeCreator, use the menu on the Attribute Value column and select Conditional Value…

The first conditional expression will set the known_as attribute:

if known_as  is “attribute is empty sting” then Output Value to “connector”

and then to set the fme_db_operation attribute:

if status = “In service” then Output Value is UPDATE

else if status = ”Proposed” then Output Value is INSERT

(Remember that FME is case sensitive for both attribute names and attribute values)

The workspace will now look something like:

4. Test the workflow

When making more complicated workflows, it’s a very good idea to try and test them before attempting to load the data into the database. In workbench, on the Writers menu, set Redirect to Inspection Application.

Then Run the workspace and check the feature counts on the workspace connectors and also check the results in FME Data Inspector. You should see all the cables with the exception of those that had a value for “known_as”. All the “known_as” values should be either “connector” or “Oak Tree Avenue LV”. “In service” cables should have an fme_db_operation = UPDATE. The “Proposed” cables should have an fme_db_operation = INSERT.

5. Save and Run

In workbench, on the Writers menu, unset Redirect to Inspection Application. Save the workspace. Run the translation.

Inspect the updated data in the Professional - Smallworld Core GIS application.

You have loaded some cables into the fme_training alternative and updated some of the existing cables.

Older Versions of FME and Smallworld

The FME Smallworld 5 writer handles Smallworld geometries in a completely different way to Smallword 4. Geometry types are no longer exposed as attributes and no longer use the sworld_geometry{} list attributes. The good news: your old workspaces are still compatible in FME 2017 / Smallworld 5.