Q&A: Do the ‘Directory modified’ Automation triggers support a single event for multiple file datasets?

Please note the following:

1. This question refers to the Directory modified trigger but the same information applies to all the file modified triggers, i.e. S3 Bucket, Dropbox, FTP.
2. In FME Server 2022.0 and newer, the Directory modified triggers are referred to as Directory (updated).

Question

Is it possible to get only one trigger event when adding/creating multiple folders and files to a directory with the Directory modified trigger?

Answer

The Directory modified trigger does not support multiple file dataset. The file modified triggers are designed to send one message to the Automation per create/modify/delete event, this means the downstream action will run for every single file change regardless of whether they were uploaded together. There is no way to merge these but here are two potential workarounds:

Option 1. Schedule Trigger

If you would like a workspace (or other downstream action) to only run once but to process multiple file changes, one approach would be swap the trigger to use a Schedule trigger set to run at a regular interval (similar to the polling interval in the Directory watch). Then in the workspace you will need to build in some logic to fetch only the files that have changed in this interval. You can achieve this by using the Directory and File Path Names reader to get a list of all files in the Directory and set the Retriever File Properties parameter to yes. Using a Tester transformer, you can check if the path_created_date or path_modified_date attribute is within the last schedule interval - this can be calculated using date/time functions. Please see this community post for more information.

Option 2. Zip File

Request your users to compress the files into one zip file before uploading into the directory so that it triggers a single event. Then have the workspace read in the zip file to be processed in the workspace.