SHAPEFILE Writer: Truncates attribute names/values to one character

Liz Sanderson
Liz Sanderson
  • Updated

FME Version

  • FME 2019.x

This article is applicable to FME 2019.1 and newer with the SHAPEFILE format introduced to replace the deprecated ESRISHAPE format.

Symptom

Using the SHAPEFILE writer the attribute names and values of my dataset have been truncated to a single character.

expectedvsactual.png

SHAPEFILE writer, expected vs actual output


If I inspect the output file in FME Data Inspector using the SHAPEFILE format it appears correct, however opening the file using the ESRISHAPE format or other applications is when the attribute data is missing.


Viewing the translation log warnings are reported similar to:

WARN  |Shapefile writer: A standard Codepage name was not found for encoding 'UTF-16LE', writing encoding name to .cpg. Other applications may not correctly identify this encoding, if problems are encountered, select a different encoding


Cause

If a user has not specified the SHAPEFILE writer’s encoding parameter, by default, it is set to use fme-source-encoding. This means the writer will attempt to detect what encoding to use based on the encodings of incoming attributes. The first one it finds will be the one it uses to write the file out.


When the source data are tagged with an encoding type such as UTF-16LE, this contains embedded nulls and whilst the old writer was fairly loose with encoding, the new writer is unable to identify a standard code page and this can result in problems with the output.


Resolution

Override the SHAPEFILE writer’s decision by explicitly setting the Writer Character Encoding to use something other than fme-source-encoding.


shapefilewritercharacterencoding.png

Add Writer Parameters: set Character Encoding


If the writer has already been added to the canvas, you can locate the Character Encoding setting under the writer parameter section in the Navigator.


shapefilenavigator.png

Navigator Writer Parameters: set Character Encoding

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.