Search

Startup and Shutdown TCL Script Examples

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

Tcl will no longer be available in FME 2024.0; this includes the TclCaller, startup and shutdown scripts, Tcl user parameters, and Tcl in the Transformer Designer. Please use FME 2023.2 or older to continue using Tcl in FME. If possible, switch to using Python. 
This article will no longer be maintained.

Simple Log Window Messages

When placed into a workspace header or mapping file, these fragments use the translation begin/end hooks to cause the translation start and end times to be written to standard output (the FME log window).

FME_BEGIN_TCL set gTranslationStartTime [clock format [clock seconds]]
FME_END_TCL puts \"Translation started time: $gTranslationStartTime\nTranslation end time: [clock format [clock seconds]]\"

 

Testing for Translation Status

When placed into a workspace header or mapping file, this post-processing fragment uses the translation end hook and the global FME_Status variable to output a translation success or failure message.

Notice that it first defines a procedure which will do this testing and output, and then it actually calls the procedure to do the work.

  FME_END_TCL proc finally {} {                      \
		 global FME_Status;                          \
if {$FME_Status == "1"} {  \
puts \"Translation was successful\";  \
} else {  \
puts \"Translation was NOT successful\"; \
};  \
};


This same example could also be written without using a procedure:

FME_END_TCL if {$FME_Status == "1"} {  \
puts \"Translation was successful\";  \
} else {  \
puts \"Translation was NOT successful\"; \
}

 

Use of Mapping File Macros

This example shows how macros used by the mapping file can be passed into a beginning or ending script as an argument to a procedure. This could be used to move, copy, email, backup or otherwise post/pre-process the destination datasets. In this example, the destination dataset is backed up prior to the start of translation to save any existing output file that would normally be overwritten.

When using MACRO or Published Parameters it usually better to use the global variable $FME_MacroValues() i.e. file delete $FME_MacroValues(DestDataset)

The mapping file fragment is:

  # The macro being used in this mapping file to set the destination
  # dataset directory is DestDataset. 
 
DWG_DATASET "$(DestDataset)"

# Source in the Tcl script to run at the conclusion of this translation.
# The script is stored in the same directory as this mapping file.

FME_BEGIN_TCL source $(FME_MF_DIR_UNIX)/backup.tcl; backup {$(DestDataset)}


And the contents of the "backup.tcl" file are:

proc backup {filename} {
  if {[file exists $filename]} {
	  file copy -force $filename $filename.bak
  }
}

 

Chaining Translations

This example shows how the FME_END_TCL could be used to chain together translations, based on the results of a particular translation. In this example, if the original translation fails, then another fme session will be initiated.

Note that in the Tcl file, the 2> NUL: routes any log and error messages to the null device, which effectively causes them to be ignored. The mapping file fragment is:

# Source in the Tcl script to run at the conclusion of this translation. 
# The script is stored in the same directory as this mapping file.

FME_END_TCL source $(FME_MF_DIR_UNIX)/tryAnother.tcl


And the contents of the "tryAnother.tcl" file are:

  set outputFile [open c:/temp/status.txt w+]
 
if { $FME_Status == "1" } {
puts $outputFile "Translation was successful"
} else {
puts $outputFile "Translation failed -- running alternate translation"
exec fme.exe alternateTranslation.fme 2> NUL:
}

close $outputFile

 

Catching a Script Failure

If a startup script fails with an error message, FME will report the error and halt execution of the workspace. However, you can stop FME from halting the workspace execution by catching the error as it occurs.
For example, if you were trying to write to a custom log file, but wanted to make sure that an error finding this file did not stop the overall process you might use:

if [catch {set outputFile [open c:/temp/logfile.log {RDWR CREAT EXCL}]}] {
    puts stderr "ERROR - cannot open the log file."
} else {
    set gTranslationStartTime [clock format [clock seconds]]
    puts $outputFile "Translation started: $gTranslationStartTime
    close $outputFile
}

 

Raise Error to Stop Workspace

To purposely stop the translation, even when the script does not cause an error , then you can use the TCL 'error' command to force FME to halt processing, for example:

if {[file exists $outputFilename]} {
    error "The output file already exists - choose a different destination filename."
}

 

Startup Script

Customizing the FME Log File

Writing custom (user-defined) messages to the log file varies depending on whether it is during the startup or shutdown phase.

In a startup script - because FME already has the log file open, and it is dangerous to open it more than once - you should simply use the function FME_LogMessage, for example:

FME_LogMessage fme_inform {hi dale inline }

 

Shutdown Scripts

Customizing the FME Log File

Writing custom (user-defined) messages to the log file varies depending on whether it is during the startup or shutdown phase.

In a shutdown script - because FME has already closed the log file - the function FME_LogMessage will not work. You should therefore use FME_LogFileName to extract the name of the log file, open it, and write to it, for example:

FME_END_TCL set outputFile [open $FME_LogFileName a] ;   \
       puts $outputFile {};                                    \
       puts $outputFile {Writing some stuff at the end}; \
       puts $outputFile {};                                    \
       close $outputFile;

 

Creating a Custom Log File

This example uses an external script to output a complete set of translation statistics to a summary file at the end of translation.

The mapping file fragment is:

  # Set a mapping file id so that the Tcl finalization procedure
# "knows" which mapping file was being run

MAPPING_FILE_ID Shape to AutoCAD

# Log all the message numbers so that we can later pull out only those
# we are interested in.

LOG_MESSAGE_NUMBERS yes

# Source in the Tcl script to run at the conclusion of this translation.
# The script is stored in the same directory as this mapping file.

FME_END_TCL source $(FME_MF_DIR_UNIX)/tclFinalization.tcl


And the contents of the "tclFinalization.tcl" file are:

  # Open a file for writing out the translation stats
 
  set outputFile [open c:/temp/stats_out.txt w+]
 
  # Check for the translation status and output the desired message
 
if { $FME_Status == "1" } {
puts $outputFile "Translation was successful"
} else {
puts $outputFile "Translation failed"
puts $outputFile "Error message was: $FME_FailureMessage"
};

puts $outputFile ""

# Output the unique Mapping file identifier set in the mapping file using MAPPING_FILE_ID

puts $outputFile "Translation summary for $FME_MappingFileId"

# Output the number of features read\written per feature type.

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile "  Features read summary"
puts $outputFile "-------------------------------------------------------------------------------"

# Loop through a sorted listed of the feature types that were read, and output the
# count for each one

set formatSpec "%-65s: %12s"

foreach featType [lsort [array names FME_FeaturesRead]] {
puts $outputFile [format $formatSpec $featType
$FME_FeaturesRead($featType)]
}

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile [format $formatSpec "Total Features Read" $FME_TotalFeaturesRead];
puts $outputFile "-------------------------------------------------------------------------------" 

puts $outputFile ""

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile "  Features Written summary"
puts $outputFile "-------------------------------------------------------------------------------"

# Loop through a sorted listed of the feature types that were written, and output the
# count for each one

foreach featType [lsort [array names FME_FeaturesWritten]] {
puts $outputFile [format $formatSpec $featType $FME_FeaturesWritten($featType)]
}

puts $outputFile ""

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile [format $formatSpec "Total Features Written" $FME_TotalFeaturesWritten]
puts $outputFile [format $formatSpec "Total Coordinates Written" $FME_TotalCoordinates]
puts $outputFile "-------------------------------------------------------------------------------"

puts $outputFile ""

# Look for any lines in the logfile that were warnings, and output a count of them to
# the summary file.  Also, check if there was any "unexpected input remover"
# statistics line and report a non-zero count if there was (this may happen
# when workbench generated mapping files are run against input datasets
# with different feature types than those that were expected)
# And also fish out the system status log message (which is #246014) and copy
# it into the output file

set logFileExists [file exists $FME_LogFileName]
set warnings 0

if {$logFileExists} {
set logFile [open $FME_LogFileName r]
while {[gets $logFile line] >= 0} {
if {[regexp {WARN} $line]} {

incr warnings

} elseif {[regexp {#246014} $line]} {

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile $line
puts $outputFile "-------------------------------------------------------------------------------"

puts $outputFile ""

} elseif {[regexp {Unexpected Input Remover} $line]} {

set totalFeatures 0
set acceptedFeatures 0
set rejectedFeatures 0

set line [regsub {^.*Unexpected} $line {Unexpected}]

catch {scan $line "Unexpected Input Remover(TestFactory): Tested %d input features -- %d features passed, %d features failed." totalFeatures acceptedFeatures rejectedFeatures}

if {$rejectedFeatures > 0} {

puts $outputFile "----------------------------------------------------------------------------  ---"
puts $outputFile [format $formatSpec "Features with Unexpected Feature Types" $rejectedFeatures]
puts $outputFile "----------------------------------------------------------------------------  ---" 
puts $outputFile ""
}
}
}
close $logFile
}

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile [format $formatSpec "Logfile $FME_LogFileName Warnings" $warnings]
puts $outputFile "-------------------------------------------------------------------------------"

puts $outputFile ""

puts $outputFile "-------------------------------------------------------------------------------"
puts $outputFile [format $formatSpec "Total Elapsed Time (seconds)" $FME_ElapsedTime]
puts $outputFile [format $formatSpec "Total CPU Time (seconds)" $FME_CPUTime]
puts $outputFile "-------------------------------------------------------------------------------"

puts $outputFile ""

# And close the file
close $outputFile

 

Database Log Files with TclODBC

This example uses an external script to insert a record of the translation's activity into a database using the TclODBC package. In this example, an Access database is created (if necessary) and populated with a row for each translation that completes.


TclODBC

To use TclODBC within the FME environment, follow these steps:

  1. Read about the TclODBC package at http://wiki.tcl.tk/2151
  2. Download the TclODBC package from http://sourceforge.net/projects/tclodbc
  3. Unzip the result
  4. Edit the supplied setup.tcl to change: if [string match *lib $i] { to if [string match *lib* $i] {
  5. In a command prompt in the directory holding setup.tcl, run "fme setup.tcl". This will install the TclODBC package in the FME's tcl environment.
  6. Ensure any scripts which require the TclODBC package start with: package require tclodbc

The mapping file fragment is:

FME_END_TCL source $(FME_MF_DIR_UNIX)/recordTranslationODBC.tcl


And the contents of the "recordTranslationODBC.tcl" file are:

  # =========================================================================
  #
  # recordTranslationODBC.tcl
  #
  # This script records the execution of a translation using
  # the TclODBC package.  This example includes the creation of the
  # Access database and table for storing the translation results.
  # One row for each translation run is inserted into the table.
  # Note that for an acutal production system the "puts" statements
  # would be removed.
 
  package require tclodbc
 
  # =========================================================================
  #
  # Set up some variables that are used within to create and connect
  # to the database via ODBC
 
  set driver "Microsoft Access Driver (*.mdb)"
  set dbfile c:/translations.mdb
  set dsn XLATION
 
  # =========================================================================
  # Create the database if it isn't already there. We wouldn't do this if the
  # database was a server-based one, but for this example we're just using Access.
 
  if {![file exists $dbfile]} {
 
	  puts "Creating database $dbfile"
 
	  database configure config_dsn $driver [list "CREATE_DB=\"$dbfile\"General"]
 
  } else {
 
	  puts "Using existing database $dbfile"
  }
 
  # =========================================================================
  # Make an ODBC datasource for this database, and connect to it
  # First always remove the DSN if it was there already.  If we were
  # using ODBC to connect to a "real" database, then we'd just assume
  # the DSN is already valid
 
  catch {database configure remove_dsn $driver "DSN=$dsn"}
  database configure add_dsn $driver [list "DSN=$dsn" "DBQ=$dbfile"]
  database db $dsn
 
  # =========================================================================
# Create the table we want to insert into if it wasn't there already

if {[llength [db tables XLATION_RESULTS]] == 0} {

puts "Creating XLATION_RESULTS table in database $dbfile"

db "CREATE TABLE XLATION_RESULTS (
MappingFileID VARCHAR(50),
StartTime TIMESTAMP,
EndTime TIMESTAMP,
CpuTime DOUBLE,
Successful CHAR(3),
NumFeatures INTEGER)"
} else {

puts "XLATION_RESULTS table present in database $dbfile"

}

# =========================================================================
# All of that was just setup, now we actually can insert our row,
# commit it, and disconnect from the datasource

set Success yes
if {$FME_Status == 0} {set Success no}

db "INSERT INTO XLATION_RESULTS
(MappingFileID, StartTime, EndTime, CpuTime, Successful, NumFeatures)
VALUES ('$FME_MappingFileId', \{ts '$FME_StartingTimeStamp'\},
\{ts '$FME_EndingTimeStamp'\},
$FME_CPUTime, '$Success', $FME_TotalFeaturesWritten)"

db commit
db disconnect

# =========================================================================
# If you were connecting to a server-based database, you probably
# would NOT remove the ODBC DSN at the end. But since we were file based
# we will remove it

database configure remove_dsn $driver "DSN=$dsn"

 

Database Log Files with OraTcl

This example uses an external script to insert a record of the translation's activity into an Oracle database using the Oratcl package.

Oratcl

To use Oratcl within the FME environment, follow these steps:

  1. Install and configure Oracle client software on your computer. Verify you can access your database using sqlPlus.
  2. Read about and download the Oratcl package at http://oratcl.sourceforge.net
  3. Unzip the result into $FME_HOME/tcl_library/oratcl4.4 (you will have to create this directory, and the 4.4 may need to change to match version of Oratcl you've downloaded).
  4. Ensure any scripts which require the Oratcl package start with: package require Oratcl

Note -- it is important that the O be uppercase in Oratcl


The mapping file fragment is:

FME_END_TCL source $(FME_MF_DIR_UNIX)/recordTranslationOracle.tcl

And the contents of the "recordTranslationOracle.tcl" file are:

  # =========================================================================
  #
  # recordTranslationOracle.tcl
  #
  # This script records the execution of a translation using
  # the Oratcl package.  This example includes the creation of the
  # XLATION_RESULTS table for storing the translation results.
  # One row for each translation run is inserted into the table.
 
  package require Oratcl
 
  # =========================================================================
  # Login to the Oracle service
 
  set username scott
  set password tiger
  set service  amidala
 
  set loginHandle [oralogon $username/$password@$service]
 
  # =========================================================================
  # Determine if the xlation_results table we wish to record results to exists
 
  set tableExists no
  set statementHandle [oraopen $loginHandle]
  orasql $statementHandle "select * from user_tables where table_name = 'XLATION_RESULTS'"
  while {[orafetch $statementHandle -datavariable row] == 0} {
	   set tableExists yes
  }      
 
  # =========================================================================
  # Create the xlation_results table if we need to
 
  if {$tableExists == "no"} {
	   orasql $statementHandle "CREATE TABLE XLATION_RESULTS (
								   MappingFileID VARCHAR(50),
								   StartTime TIMESTAMP,
								   EndTime TIMESTAMP,
								   CpuTime FLOAT,
								   XlationSuccessful CHAR(3),
								   NumFeatures INTEGER)"
  }
 
  # =========================================================================
  # Insert the translation results into the table
 
  set Success yes
  if {$FME_Status == 0} {set Success no}
 
  orasql $statementHandle "INSERT INTO XLATION_RESULTS
				(MappingFileID, StartTime, EndTime,
				 CpuTime, XlationSuccessful, NumFeatures)
				VALUES ('$FME_MappingFileId',
						timestamp '$FME_StartingTimeStamp',
						timestamp '$FME_EndingTimeStamp',
						$FME_CPUTime, '$Success',
$FME_TotalFeaturesWritten)" -commit

# =========================================================================
# Shut down the statement handle

oraclose $statementHandle

# =========================================================================
# Logout of our oracle service

oralogoff $loginHandle

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.