Using the GE Energy Smallworld 4.0 reader / writer with FME

Description

Versions: This article applies to Smallworld 4.0 and earlier. If you are using a more recent version of Smallworld refer to the article Working with GE Smallworld Databases. 
The GE Energy Smallworld Reader/Writer is for use with Smallworld v4 databases.

Getting Started

The general process for using FME and Smallworld 4 is:

• Open the Smallworld FME Interface and select the Objects/Feature Types you wish to work with by copying them into the Export to FME node in Smallworld Explorer
• In Workbench create a new workspace to go to or from Smallworld.
• Choose the option for 'Import Feature Type Definitions' and the objects defined in the Smallworld Interface are made available within the workspace.
• Complete the workspace as usual. You may then run it as you would a regular FME workspace.

Upgrading from v3.x

There are several differences between FME's support for Smallworld v4.x and v3.x:

• The reader/writer for Smallworld v4.x uses different technology (SWAF) which permits FME to access the data directly.
• Translations can now be run from within Workbench.
• Smallworld data can be viewed in the FME Viewer.
• For you mapping file types, the writer type name has changed from sworld to sworld400
• The Smallworld FME Interface is a different installation.
• It isn't necessary to set sworld_type within Workbench for v4 - presumably because you define the geometry name as an attribute and assign a type there.
• Datastore is now added to the feature type name; eg instead of rail_line you now have gis.rail_line
• The Smallworld SWAF 4 FME Translator shows significant performance improvements over the Smallworld 3 translator.

v4.2 Support

FME supports v4.2, and in fact from FME2010 onwards this is the version that FME will expect by default. TSB 3 for Translators supports Smallworld Core 4.2.
A new reader is available for Smallworld 4.2 users. Contact your GE Smallworld distributor for the CD.

v4.1.2 Support

FME supports v4.1.2 and earlier, and in fact from FME2010 onwards this is the version that FME will expect by default. TSB 2 for Smallworld Translators (from Smallworld) contains the most recent version of the dll and magik changes for Smallworld 4.1.2 and earlier.

v4.0 Support

To revert back to Smallworld 4.0 look in your FME installation folder and you will see we ship three DLLs: swordswaf.dll, sworldswaf40.dll and sworldswaf41.dll.
swordswaf.dll is the one FME uses. By default it is a copy of sworldswaf41.dll. So, if you wish to get support for 4.0 you need to overwrite swordswaf.dll with sworldswaf40.dll.

Creating Attributes and Attribute Joins

The SpatialBiz FME plugin supports user defined fields as the target for Smallworld translations through the fme_pseudo_field_defs shared constant. This is used internally when creating relationships (joins) in data where it is loaded from source systems which model the relationships differently.

User defined fields can be used in other ways through hook points in the translation process. This functionality is being implemented for SpatialBiz 3.3, scheduled for release with FME 2009.

Also, with the standard Smallworld reader/writer it isn't possible to create a join between geometry and attributes when writing to Smallworld. Because FME doesn't have access to the sys_id there isn't any way to join information from an external datasource to existing Smallworld data.

In other words you can load a parent table, and you can load a child table but you can't make a join because Smallworld assigns the value for sys_id and not FME.

Apparently it is possible to to access sys_id from a Magik application. Therefore you could write a join key to a spare attribute and then, using a Magik application, copy this value into the sys_id to create the join. I'm told that a good Magik programmer would find this a fairly straightforward task.

The SpatialBiz FME plugin supports the creation of joins when importing data to Smallworld through a number of approaches:

1: The user can export then import the key, foreign key and intermediate tables directly. During import, SpatialBiz tracks sys_id values and can build the relationships between records based on the source key values. i.e. the source key values only have to be unique and SpatialBiz will then be able to find and replace the foreign key values with the actual generated key values.

2: User defined fields can be specified in the translation which store the value and means to find related records. The @recordFinder() macro can be directed to look for related records and build the relationships during import.

The SpatialBiz FME plugin will return the sys_id field values generated during a translation run (either the Int32 or Int64 forms) for use in FMEObjects applications. In addition the generated sys_id values are retained and automatically used when looking up foreign key values to create Smallworld joins between related records.

SmallworldGeometryFactory

When FME reads data from Smallworld it runs it through the SmallworldGeometryFactory to turn the raw Smallworld content into more easily handled features. If the user wishes to view the data in its raw form, then the way to do this is by turning off the SmallworldGeometryFactory. This can be done by changing the 'Service' parameter in the workspace dialog from 'FME' to 'FMENOFACTORY'.

Localization

Local character sets (Thai, Cyrillic etc) are not handled by the GE Reader/Writer. You would need to have their Magik code amended for your particular purpose.

You could contact your local Smallworld supplier who might be able to help, or have a Smallworld consultant edit the Magik code for you.

Safe Software is not responsible for supporting the Smallworld-FME Interface Magik code.

However, the SpatialBiz FME Plugin for Smallworld supports FME Text encoding for import / export of Smallworld data. Data represented in local character sets can be translated as with other FME supported formats.

Coordinate Scaling

With Smallworld 4, the data is imported or exported in the application coordinate system units. For the Cambridge database, this is in Britsh National Grid metres, so no coordinate system scaling should be required. In the Cambridge database, you can change the applciation coordainte system units in the register.magik file, i.e.:

  ....\Smallworld4\fme400\modules\swaf_fme_application\source\register.magik file

ArcGIS and Smallworld

If you are using Smallworld 4 then you should be able to view data in ArcCatalog by entending ArcGIS with FME.

Smallworld and FME Objects

If you are using Smallworld 4, then the Smallworld SWAF reader/writer for FME will let you read Smallworld data with FME Objects.

Alternatively if you already have the SpatialBiz plugin then this continues to support Smallworld versions 4 and above.

Smallworld FME SOM

SOM is short for Spatial Objects Manager. A SOM provides the ability to 'plug-in' data readers for a variety of external data formats - The Smallworld FME SOM permits the reading of a number of data formats via FME technology.

Updating v3.x Mapping Files

There are two main issues to be resolved with using a v3 mapping file with a v4 installation.

Feature Types

Between v3 and v4, Feature Type names will have changed.

In the SWORLD v3.x reader/writer datastore was not used, for example the Feature Type name would be...

rail_line

In the SWORLDSWAF reader/writer, the datastore is now added to the feature type name, for example...

gis.rail_line

So all existing mapping files will have to be changed to include the datastore name. This will occur in at least two places for each feature type, the DEF line & the correlation lines:

SWORLD_DEF gis.rail_line \
annotation sworld_text \
centre_line sworld_chain \
name char(30) \
type enum(rail_type)

and

DWG CAMBRIDGE_RAILWAY \
name %name \
type %type \
LabelRotation %LabelRotation
# autocad_entity autocad_line \

# Change the sworld_name from route to "centre_line"
SWORLD gis.rail_line \
name %name \
type %type \
@SupplyAttributes(sworld_geometry{0}.sworld_type,sworld_text) \
@SupplyAttributes(sworld_geometry{0}.sworld_name,annotation) \
... etc

One consideration is that we will add a keyword to define the datastore, for example...

SWORLDSWAF_DATASTORE_NAME gis

...so the SWORLDSWAF writer would take "rail_line" and make it "gis.rail_line". This hasn't been implemented at this time.

Reader/Writer Keyword

Between v3 and v4, reader or writer keywords will have changed. i.e. you will need to change:

WRITER_TYPE SWORLD

...to...

WRITER_TYPE SWORLD400
WRITER_KEYWORD SWORLD