Known Issue: Job Submitted from Automation is Stuck Running and Cannot be Cancelled

Symptom

A running job is stuck and cannot be canceled via the FME Flow Web UI or REST API. The job remains in a running status after FME Flow is restarted. The engine remains occupied by the job.

Log files contain errors similar to the ones below:

job_##.log and fmeprocessmonitorengine.log

AUTOMATIONS writer: FME Flow error: FME Flow error: UNPROCESSABLE_REQUEST(Automations) Route "AutomationWriterFeatureType" does not exist''

fmeserver.log

Wed-28-Aug-2024 12:56:55.858 PM ERROR engine-localhost-localhost_Engine1 SQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00
Where: unnamed portal parameter $18
etc...

Cause

This is caused by the following conditions:

• The job is submitted from an Automation's Run Workspace action
• The workspace used in the action contains an FMEFlowAutomationWriter
• After the Automation has started, the workspace is republished with a new FMEFlowAutomationWriter or the existing FMEFlowAutomationWriter feature type has been renamed
• The Automation is not modified. It is triggered and the Run Workspace action runs with the updated workspace
• The Automation is unaware of the new or changed feature type for the FMEFlowAutomationWriter

This results in FME Flow becoming unstable. 

Workaround

Option 1

From the FME Flow Web UI:

• In Engine Management, bring the active engines down to zero
  • If you are not able to bring the count to zero, please try Option 2
• Restart FME Flow, which queues the job
• In Jobs > Queued, cancel the job
• Return to Engine Management and bring the engine count back up from zero

To avoid the issue, stop the Automation before publishing the workspace. After publishing the workspace, click the refresh button on the Parameters tab Workspace Action. In addition to refreshing parameters, this will also refresh the FMEFlowAutomationWriter's output ports. Then, the Automation can be restarted. 

Option 2

If the Automation has not yet been modified, this workaround below may resolve the issue:

• Stop the Automation
• Stop the FME Flow Engines service
• Download a copy of the workspace used in the Automation Run Workspace action
• In FME Workbench, rename the FMEFlowAutomationWriter feature type name to what it was when the Automation was last started
• Remove any new FMEFlowAutomationWriters that were added after the Automation was last started
• Republish the workspace with the same .fmw name to the same Repository (overwrite the current workspace on FME Flow)
• Modify the Automation and reselect the workspace in the Run Workspace action. Confirm that the output port name (FMEFlowAutomationWriter feature type) gets reverted to what was entered in FME Workbench.
• Start the FME Flow Engines service
• Start the Automation
• Trigger the Automation
• The stuck job should complete

To avoid the issue, stop the Automation before publishing the workspace. After publishing the workspace, click the refresh button on the Parameters tab Workspace Action. In addition to refreshing parameters, this will also refresh the FMEFlowAutomationWriter's output ports. Then, the Automation can be restarted. 

Option 3

If neither of the options above resolves the issue, please contact Safe Support.