Known Issue: FME Flow shows a job has failed, but it is still running on a distributed engine host

Issue

FME Flow indicates a job that has failed, but there is evidence in the job log and/or downstream databases that it is still running or completed successfully. 

Cause

In an FME Flow distributed or fault-tolerant installation, the architecture is designed to continue processing jobs even if there is a disconnection from the FME Flow Core. However, the job status is no longer accurately reported on the web UI when this occurs. 

In a fault-tolerant installation:

• The FME Flow web UI will still be accessible due to the redundant core and web application.

• The Jobs page indicates that the job failed, but it may still be running. 

• For Streams jobs, the Streams page indicates that the job is queued, but it may still be running.

  • When the Core becomes available again, the Jobs page indicates that the job was cancelled, but it may still be running.

In a distributed installation:

• There is no redundant core and the web UI will be inaccessible. The error "Could not connect to FME Core" will be displayed, provided the web application is still running. 

• When the core becomes available again and the web UI is accessible, the same behavior above is observed on the Jobs and Streams pages.

If the job is still running when the engine host reconnects to the core, the engine will also be missing from the Engine Management and Streams pages until the job completes or the fmeengine.exe process is terminated manually.

Workaround

If FME Flow has reported that the job failed, but the job log doesn't show that the translation completed, check the failed job log for the ProcessID. For example: 

START - ProcessID: 6864

Check the distributed engine host for that same ProcessID (PID). For example:

If there is a process still running with the same ID, then the job is still running.

If the engine hasn't lost connection to the FME Flow system share, then job log will continue to be updated. Refresh the job log to monitor job progress and check for successful completion. Note that the details above the job log (Feature Written, Time Finished etc...) will be inaccurate as well as the status. 

Resolution

This issue is unresolved.