Introduction
When deploying FME Flow on VMs in the cloud, a common question is what VM size and type to use. This is hard to answer because we often do not have the data we need to make informed, solid decisions about VM size. Additionally, each project or scenario is unique in its requirements and needed resources, making it hard to recommend a one-size-fits-all solution.
The solution to this problem is to start with the smallest, general-purpose size that can run production workflows and collect data to improve and adjust the selection over time. This can better inform decisions for future deployments and, at the same time, continuously learn about the requirements of our individual scenarios.
Starting point
Technical specifications & general recommendations for FME Flow
The first thing to consider is the Technical Specifications of FME, and to review the existing guides regarding hardware requirements and FME Flow host system sizing.
VM size
A general guideline is to plan for 1 CPU core per FME Engine, given its single-threaded implementation and a minimum of 8 GB of memory. Additional CPUs are recommended for the web server and the FME Flow Core. For example, an express install with a VM with 2 virtual cores (vCPU) and 8GB provides 1 core for the web server & FME Flow Core and 1 core for engines. This should be the minimum used for FME Flow. This is also similar to the smallest FME Flow Hosted instance size recommended for production workflows.
In a distributed setup (FME engines deployed on machines other than the FME Flow Core), a similar size is recommended, but it is possible to experiment with 1-CPU machines for FME engines if only 1 engine per VM is desired. It is also recommended to allocate more resources for a Windows deployment, as FME Flow runs more efficiently on Linux distributions.
VM type
After sizing a VM, there is still a wide range of VM type options because cloud providers offer different types tailored to specific scenarios. The best choice for a new FME Flow is usually the “General Purpose” option. These VM types are well balanced between CPU & Memory. For more details about the specifications and traits of the general-purpose VM type, please refer to the respective documentation for Microsoft Azure, Amazon Web Services & Google Cloud Platform.
Recommended general-purpose VM types:
| Microsoft Azure | Amazon Web Services | Google Cloud Platform |
|---|---|---|
| D-series | M-series | N-series |
Newer instance generations in each series usually deliver a better value, so choosing the newest available generation is a good idea.
Important: When choosing VM types, make sure to choose an Intel/AMD architecture for FME Flow
CPU, Memory & storage optimized
VM types other than “General Purpose”, like compute-, memory-, or storage-optimized VM types, should only be used if there is clear evidence that a majority of the workloads on your FME Flow will benefit from a different VM type. For example, very memory-intensive raster/point cloud processing could benefit from memory optimization, or workflows that rely heavily on writing lots of small files to disk might benefit from switching to a storage-optimized VM type.
Monitoring resources in the cloud
Once a VM type has been chosen, the next step is to monitor the instance to ensure resources are used efficiently and, if needed, adjust the VM type based on this data. Our built-in monitoring functionality for FME Flow Hosted is a good starting point for determining which metrics to monitor and when to resize or change VM types. To get started with monitoring VMs, refer to the respective documentation for Microsoft Azure, Amazon Web Services & Google Cloud Platform.
Conclusion
When deploying FME Flow in the cloud, it is recommended to start with general-purpose VMs with 2 virtual CPUs and a minimum of 8 GB of memory. Monitor the VMs to inform decisions on adjusting the deployment's size and VM type.
Additional Resources
Please consider posting to the FME Community Q&A if you have any further questions or are experiencing performance issues that are not addressed in this article. There are also different support channels available.
Make sure to include as much information as possible:
- What type of workspaces are you running on FME Flow?
- What VM sizes and types have you tried?
- What version of FME Flow are you running?
- What do the VM performance metrics show?