FME Workspace Statistics (You Can Help!)

Introduction to FME Workspace Statistic

Workspace statistics help you understand how you and others in your organization use FME. This will allow you to:

• Focus organizational training on commonly used functionality (formats or transformers).
• Follow best practices for building and structuring workspaces consistently across your organization.
• Help document your workspaces.
• Identify workspaces for possible improvement, including transformers that can be updated.
• Make a more comprehensive upgrade plan by understanding which transformers can be updated.
• Help Safe understand your FME usage so we can improve the FME products appropriately.

Attached is a workspace that will read one or more workspaces and generate an Excel spreadsheet with a variety of workspace statistics, including:

• Summary statistics for each reader/writer format and transformers (includes readers/writers in transformers)
• Summary statistics for each workspace (number of readers/writers/transformers, etc.)
• List the transformer versions and compare them to the transformer version of the current version of FME
• detailed statistics for each workspace (optional)
• connectivity between transformers with frequency counts

Output is an Excel spreadsheet with the following sheets:

• Header: General information about this analysis.
• Workspace Summary: summary statistics for each workspace, including the last save build number. Workspace names can be obfuscated.
• Reader/Writer Summaries: These provide the format counts for all readers & writers across all workspaces, including those in transformers such as FeatureReader. There is a second set of sheets with statistics by family of formats (e.g., Oracle includes Oracle spatial, Oracle non-spatial, etc.).
• Transformer Summary: Counts for each transformer (and optionally custom transformers). Includes the list of transformer versions encountered in the workspaces and the current transformer version - this can help you decide if it's worth updating some transformers. Embedded custom transformers will have their contents included in the analysis. Linked Custom Transformers will not have their contents included unless you include the *.fmx files.
• ConnectionSummary: this shows a summary of the from-to connections between transformers (or readers/writers) and their frequency.
• Readers/Writers/Transformers by Workspace (optional) - this shows the breakdown of functionality by workspace (workspace names can be obfuscated).

Share your results

Please share your results with us at fmestats@safe.com - this will help us determine which formats and transformers are more important to FME users. Please differentiate between a desktop analysis and an FME Flow repository analysis.

The statistics workspace allows you to obfuscate the workspace names (optional) if you have sensitive projects (recommended). You can also choose to ignore Custom Transformers (if these are sensitive). We will consolidate results from all users and publish them here.

How to use

Run the Statistics Workspace (WorkspaceStatistics.fmw): Point at a directory that contains your workspaces (or at a root directory with a set of directories containing workspaces). The Statistics Workspace will then retrieve statistics for all workspaces and/or custom transformers it finds.

It's more efficient to point to a single directory rather than scan all directories on your system (which can be pretty slow). To help with this, we've included a second workspace ( FileCopyWorkspace.fmw) that will create a catalog of your workspaces by copying all .fmw/.fmx files from a set of folders into a single directory. The Workspace Reader (FMW) may fail if an invalid or corrupt workspace is read, so it's better to copy all workspaces into a single folder, where you can then remove or investigate the problematic workspace.

File Copy

Run the FileCopyWorkspace.fmw in two steps.

1. Enable the PATH reader and use the File Browser to point at your 'root' directory. The File and DirectoryPath reader will scan all directories below this root folder and create a catalog of your FME workspaces. Edit the catalog to remove any duplicate workspaces and related entries.
2. Disable the PATH reader and enable the Excel reader. The workspace will read the workspace catalog and copy all the workspaces into a common folder.

Statistics Workspace

Input prompts:

• Your Name, Organization, Comments (all optional)
• Hide workspace Names: Select Yes if you would prefer workspace names to be hidden
• Is this an FME Flow Repository: Yes|No 
  • This will help us better understand FME Flow usage.
• Include Disabled objects: Yes | No
• Include workspace Detailed Statistics: Yes | No. 
  • Set to 'No' if you'd prefer not to share details of individual workspaces.
• Include Custom Transformers: Yes|No
• Source FME Workspace Directory. Use the Select Multiple Folders/Files Browser if you skipped the file copy and intend to scan multiple folders.
• Destination Microsoft Excel File. Excel report location.

Usage Notes

• Try to avoid creating statistics of versions of workspaces, i.e., workspaceV01.fmw, workspaceV02.fmw
• Please indicate if these statistics are for an FME Flow repository (otherwise, we'll assume they are desktop statistics). Generally, in the FME Flow workspaces are in the repository folder, which is the file share, not the install folder, i.e., C:\ProgramData\Safe Software\FME Server\repositories\ where C:\ProgramData\

Results

• Reader/writer counts. If multiple readers/writers access the same workspace, they are counted as a single reader or writer in the reader/writer summary stats.
• Summary of connections: connections between all reader feature types to all writer feature types are counted as a single connection - reader to writer.
• Summary connections, skip junctions, and 'tunnels'.

FAQ

What's the difference in usage stats?

When you install FME, you can opt to send us usage statistics. This gives us a picture of what formats and transformers you're adding to your workspace. But some workspaces rarely get edited, they just run - particularly those workspaces you might have on a production server. We think these are probably the most important workspaces you have, and we're not getting their statistics.

Transformer Connections

There are more transformer connections available on the Transformer Gallery.

Analysis

We have run the workspace statistics for example workspaces used in Safe webinars to illustrate the results - attached.