Known Issue: Older Workspaces Opened and Saved in FME 2021.0 or Newer May No Longer Run Due to Workspace Character Encoding Conflicts

* As of April 2023, FME Desktop is now FME Form. 

Symptom

When a workspace authored in FME Desktop pre-2021.0 is opened and saved in FME Workbench 2021.0 or newer, users may experience failed translations with the following translation log error messages:

“The dataset ‘C:\Path\To\Dataset’ could not be located. Please check that it exists.”

“<FormatName>: The encoded string is not a valid UTF-8 string: ‘\x83g\x83\x8A\x83A’”

Cause

Workspaces built in FME Workbench pre-2021.0 did not store the character encoding system of the machine used to create the workspace. Workspaces, including template workspaces, built or saved in FME Workbench 2021.0 or newer do store such an encoding indicator, as the FME_NAMES_ENCODING field. This field can be viewed in translation logs where Log Debug has been enabled from the Translation menu of FME Options. The value of FME_NAMES_ENCODING for workspaces built and / or saved in FME Workbench 2021.0 or newer is set to the current OS locale encoding system.


Log file generated from a workspace run in FME Workbench 2021.2, Log Debug enabled

When an older workspace without FME_NAMES_ENCODING has been authored on a machine with a specific OS locale, eg. Shift_JIS, and then edited and saved in FME Workbench 2021.0 or newer on a machine with a different OS locale, eg. win-1252, the newly generated value of FME_NAMES_ENCODING will conflict with the original encoding system of the workspace. This encoding conflict will, in turn, cause errors when FME 2021.0 or newer tries to interpret string values in the workspace.   

Workaround

If a workspace is known to have been built in a version of FME pre-2021.0, ensure there is a backup copy of the workspace available before accessing it in FME 2021.0 or newer. This backup copy should be kept on a machine whose character encoding (OS locale) matches the character encoding that the original workspace was built in.

If the older workspace, built in a specific OS locale, has already been opened and saved in FME Workbench 2021.0 or newer on a machine with a different OS locale, users should regenerate workspace readers, writers, and schema-manipulating transformers. After these actions, users must update any other workspace elements / parameters which make reference to or directly use the outputs of these regenerated objects.