Symptom
If you see the following error message, then you have exceeded the limit of 8 concurrent fme.exe processes running with a single FME Form license:
Error message was: FME license system failure: The maximum number of concurrent FME instances specified in the license has been reached (-508)The setting is system-wide so having multiple installs on the same machine will not allow for more simultaneous processes.
This limit differs from that imposed on the parallel processing of "group by" transformers. These transformers limit the number of processes based on license level. More information on this can be found in the product documentation.
Cause
1. Running more than 8 FME processes
FME Form licenses are restricted to allow a maximum of 8 fme.exe processes to run at any one time. This includes any running Workspaces and any Data Inspectors, either run through the command line or FME Workbench, including those launched from another workspace using the WorkspaceRunner transformer.
2. FME.exe not terminating
If you receive this error but are not attempting to run more than 8 concurrent workspaces, then check in your Task Manager for any idle fme.exe processes that have failed to terminate upon earlier job completion.
Resolution
If FME.exe processes are hanging
- Check the Task Manager / Activity Monitor for any idle fme.exe processes that may have failed to terminate upon job completion and manually shut these down to clear the error.
- If you are using the command line to launch a workspace, ensure that there are no enabled Inspector transformers within this workspace - these processes will still launch, but since the Data Inspector is not actually opened, they will not terminate, thus taking up one or more of your 8 processes.
- If an fme.exe process is continually hanging when a specific workspace is run, do the following:
- Report the issue along with a copy of the workspace through our website: Report a Problem, as this should not be happening.
If this is a recurring issue, a workaround is to set up a Python Shutdown Script to end all fme.exe processes after the workspace in question is done. You can find an example script here.
You will not be able to do this if there are other fme.exe processes still running as these will also be terminated, but you could set up a schedule for a script to run at times you know the there are no other running workspaces.
If you are using a WorkspaceRunner
The WorkspaceRunner transformer was developed so that running a single workspace would trigger the execution of a series of child workspaces. This works by spawning a new FME process and is useful for batch processing; however, using this transformer can generate a licensing error. You can control the spawning of new FME processes in the transformer parameters to help reduce the risk of this error occurring.
The key parameter to be aware of is ‘Wait For Job to Complete’:
- If this is set to Yes (default) then the transformer will wait for each job to complete before processing the next feature so will not spawn off additional FME processes.
- If this is set to No, then the user is essentially allowing parallel processing, and two more parameters become available:
- ‘Maximum Number of Concurrent FME Processes’ - defines the number of FME processes to be launched so that multiple workspaces can run simultaneously. The main workspace uses a single fme.exe process, leaving 7 additional processes available for the child workspace to launch. If no value is set, then the maximum of 7 concurrent processes will be used.
- ‘Number of Workspaces per FME Process’ - controls the number of workspaces to be run consecutively on a single FME process before it is shut down. This may help prevent the chance of hanging fme.exe processes.
Adjusting these parameter settings should help you to avoid further encounters with the licensing error. For example, setting the ‘Maximum Number of Concurrent FME Processes’ to 5 would leave 2 FME processes available to run other workspaces.
Why not try FME Flow?
If you are interested in running more concurrent processes, we suggest you look into FME Flow, a scalable solution that lets you scale up to your needs by adding FME Flow Engines.
What is FME Flow?
FME Flow is a networked data transformation application that uses the same underlying technology as FME Form. It can operate on a local-area network or over the Internet.
FME Flow supplies two major features that would help you overcome this licensing error:
-
Complex Scheduling - Scheduling in FME Flow allows for more complex task management than simply running workspaces from the command line. Advantages include:
- Automated job submission 24/7, reducing the need for many processes to be run in parallel.
- Attempt recovery if a job initially fails; it will be resubmitted up to three times.
- Enables job submission in response to other processes. For example, if job X runs successfully, run job Y, or watch a Directory/Database to trigger a job based on an event.
-
Queues
- In FME Flow, if a job is run and no engines are available, rather than failing, it will join a queue and run once an engine becomes available.
- You can create queues with different priority levels and assign different engines and repositories to them. Therefore, a single engine could be allocated to high-priority jobs, eliminating the need for them to join the queue behind less significant or urgent jobs.
From WorkspaceRunner to FMEFlowJobSubmitter
If you make the transition to run a workspace on FME Flow, you will need to replace any WorkspaceRunner transformers with the FMEFlowJobSubmitter.
The FMEFlowJobSubmitter has the same underlying role as the WorkspaceRunner, but provides additional parameters, such as designating a Job Queue. This transformer submits one job per feature that enters it, but key parameters to be aware of here are found under Submission Mode:
- ‘Submit Jobs’ - this transformer determines whether a request is submitted as soon as an initiating feature has arrived (In Sequence) or whether it waits until all input features have been received before sending out the requests (In Parallel).
- ‘Wait for Jobs to Complete’ -
- If this is set to Yes, then the jobs are run synchronously; that is, the transformer will wait until the job is completely processed, and the initiating feature is output via the Succeeded port with additional attributes added to it.
If this is set to No, then the jobs are run asynchronously, meaning the job ID is output via the Submitted port as soon as the server receives the request.
FMEFlowJobSubmitter parameter editor
Known Issues with FMEFlowJobSubmitter
- When a job is submitted on FME Flow via the FMEFlowJobSubmitter with 'Submit Jobs' = In Sequence and 'Wait for Jobs to Complete' = Yes, the job gets stuck in the queued state. As a workaround, ensure the Python interpreters match between parent and child workspaces. More information on this limitation is available here.
For any further troubleshooting help, please refer to the article: FME Flow Troubleshooting: FMEFlowJobSubmitter.