How to request FME Flow container resources in Kubernetes

One of the benefits of Kubernetes is automatic bin packing, so that the Kubernetes scheduler will only assign pods to a node if that node has enough resources. 

In FME Flow (formerly FME Server), the biggest consumer of node resources will most likely be the FME Flow engine(s). If you haven’t specified any resource requests for your engines in Kubernetes, you may find that the Kubernetes scheduler places more engine pods on a node than it can handle (once they start processing jobs). This will result in engines failing to process jobs and possibly getting themselves into an unhealthy state.

By default, we request 512Mi and 200m CPU for each engine. You view what the defaults resource requests are for different FME Flow containers, and how to change them, on Github.
 

Setting FME Engine Resource Requests

To mitigate failing engines due to lack of resources, you can use the values.yaml for your deployment to request resources that you know your engines will need. 
You can see how and why you may want to set up an engine deployments yaml file in our documentation.

To request resources for your engine pods, you can do this per engine set: 

fmeflow:
  engines:
  - name: "engineset1"
    type: "STANDARD"
    engines: 1
    queues: "Default"
    resources:
      requests:
        memory: 10Gi
        cpu: 800m
  - name: "engineset2"
    type: "DYNAMIC"
    engines: 2
    queues: "Default"
    resources:
      requests:
        memory: 10Gi
        cpu: 800m

You can reference the Kubernetes documentation for more information on managing container resources. The following information is taken from that page:

Limits and requests for memory are measured in bytes. You can express memory as a plain integer or as a fixed-point number using one of these suffixes: E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki.

Limits and requests for CPU resources are measured in cpu units. One cpu, in Kubernetes, is equivalent to 1 vCPU/Core for cloud providers and 1 hyperthread on bare-metal Intel processors.
 

Tips to Determine the Correct Resource Request

Assigning the correct memory and CPU resource requests may be trial and error at first, but if you’ve previously run these jobs you may be able to use historic metrics to guide you. Luckily the nature of Kubernetes means it doesn't take long to change your values.yaml file and redeploy FME Flow.

We recommend 1 cpu per engine (you may be ok with less). FME engine is single threaded so you will not get any benefit from assigning more than 1 cpu.
For memory usage, if you have some bigger workspaces that you’ve run previously, you will find the peak process memory usage at the bottom of the log file. Analysing the biggest jobs your engines are likely to process should allow you to request adequate resources.
 

|INFORM|END - ProcessID: 30524, peak process memory usage: 4335784 kB, current process memory usage: 4174368 kB

 
In this example, the peak memory usage is 4.3gb. Therefore, it’d be a sensible choice to set a memory request of 5Gi. If you request less than 4.3Gi you may find that your engine will be unable to process that job due to lack of memory if Kubernetes has filled that node to capacity and other pods are using up the memory. If the node has extra available resources the engine will be able to consume them, even if the request was set to a lower amount.
 

Considerations for Nodes

You will need to make sure that you have enough nodes, or big enough nodes that have enough resources to run your FME Flow deployment, otherwise pods will get stuck in a pending state until Kubernetes can successfully schedule them.
 
Another benefit of using Kubernetes is the different scaling options. If engines pods are frequently pending, the Cluster Autoscaler may be useful.