FME_MF_DIR predefined parameter (macro)

Introduction

FME_MF_DIR is one of several predefined Macro (parameter) that refers to the folder location of an FME workspace - other parameters are discussed in the FME documentation. It is unique for each workspace (so is more a parameter than an environment variable).

FME_MF_DIR has one specific role that is important to be aware of, and can be used to organize projects to great effect.

FME_MF_DIR within Workspaces

By default, all source data is referenced from a workspace using absolute paths - for example c:\myFMEproject\source.xxx

However, whenever you save a workspace FME checks to see if any source data is contained in the same folder. If so the paths that reference them in the workspace will get changed from absolute paths to relative, making use of FME_MF_DIR. The FME developers call this 'packing'.

For example, you have source data...

 Source Data: c:\myFMEproject\source.xxx

You add that source to a new workspace then save the workspace to the same folder as the data. Internally the workspace will now refer to the source data as...

 Source Data: $(FME_MF_DIR)source.xxx

Advantages

On one hand this is a great function. It means that if you ever need to move a whole project - workspace and data - to a different disk for example, then all of the workspaces will still operate correctly without needing to be edited. This is because the source data paths are relative to the location of the workspace and not tied to a particular folder.

When people send in workspaces to the support team at Safe we can tell which ones were saved in this way because we open them up and all the source datasets are already set up correctly, for example the sources all point to r:\pserv\rt\tickets\12345\data.xxx - this is not because the user knew our file structure and predicted the ticket number in advance but because they saved workspace and data to the same folder. When this isn't the case we get paths like q:\projects\FME\roadNetwork\data.xxx which we have to edit to the correct location.

Disadvantages

On the other hand this behaviour can cause problems for users who are not expecting it. Again using the example of data and workspace both stored in c:\myFMEproject\, problems occur when the data or workspace are separated when one of them is moved to a different location.

Admittedly, even the most inexperienced user will realize that a workspace cannot be expected to read data that has been moved, but even the most experienced user might wonder what is happening when he moves the workspace and the translation fails with an error message saying the source data is missing.

The problem is, of course, that moving a workspace invalidates relative paths, so that the data is no longer at $(FME_MF_DIR)source.xxx - this is a particular problem when a user is not even aware that the paths are relative.

How FME helps avoid problems

Firstly, the above problem could only occur when the workspace has been saved in the same folder as the data. Without this FME will never use relative paths and the problem is impossible.

Secondly, although FME can't be aware of users who move workspaces using - for example - Windows Explorer, it can be aware of when a workspace is moved using the Workbench Save-As function. So, whenever a workspace is saved to a different name/location using File > Save-As, FME will change relative path names back to absolute ones (or Unpack them as the developers would say). That way the relative path problem can't occur.

Gotchas

One thing to be aware of is that this does not just refer to source or destination datasets. Any path in a workspace is fair game for replacement with FME_MF_DIR, including - for example - source files for the Joiner transformer or output folders for the AttributeFileWriter transformer. If they are the same as the workspace path they will be referenced by FME_MF_DIR with all the benefits and costs that this brings.