Introduction
Infrastructure as Code (IaC) is the process of provisioning and managing resources, typically found in a data center, via machine-readable definitions. For cloud deployments, IaC is a critical tool for delivering reliable, reproducible solutions across multiple environments. In this article, we want to share templates for provisioning distributed deployments of FME Flow on Windows in the cloud using common IaC tools.
Example IaC templates
The provided templates can be used as boilerplate code for custom deployments, making it easier to implement a reliable, reproducible solution and allowing the integration of FME Flow deployment into existing CI/CD workflows.
These examples are not complete solutions and cannot be used for production out of the box. To create a production-ready deployment, the examples can be used as a reference, but it is mandatory to complete at least the following checklist:
- Review templates to understand the entire architecture, the services used, and the associated cost
- Review network-related templates to avoid overlapping network space and add network security groups depending on requirements
- Update the templates where necessary (e.g., existing infrastructure, custom VM images with 3rd party components)
- Implement secure secret management for passwords (e.g, Azure Key Vault, AWS Secrets Manager)
- Implement update & upgrade strategy for Windows Server VMs and databases
- Implement resource monitoring and alerts for VMs, DBs, and storage (e.,g Azure Monitor & Log Analytics, AWS CloudWatch)
- Perform security hardening and implement security policies (e.g, Azure Security Baseline)
IaC tools
Here is a list with links to documentation and code examples for a distributed FME Flow Windows deployment using different tools. This is not an exhaustive list, but it should serve as a good starting point.
None of these examples is production-ready and requires further adjustments based on individual requirements and environments. This is the end-user's responsibility to understand and modify the templates before deployment.
packer
To deploy FME Flow as infrastructure, the first step is to create reusable VM images. In our examples and for the Azure Marketplace deployment, Packer was the tool of choice.
Here are examples to create FME Flow Core & Engine images for Windows:
- Amazon Web Services: FME Flow Core & Engine AMIs with Packer
- Microsoft Azure: FME Flow Core & Engine VM images with Packer
Additional Resources
- VM sizing for FME Flow in the cloud
- Deploy a distributed FME Flow in the cloud with 3rd-party components
terraform
Terraform is one of the most popular IaC tools, allowing the creation of templates for multiple cloud providers. The examples closely follow the architecture of the Azure Marketplace deployment:
- Amazon Web Services: FME Flow (Distributed deployment, Windows) with Terraform
- Microsoft Azure: FME Flow (Distributed deployment, Windows) with Terraform
Additional Resources:
- Load Balancing Concepts for FME Flow in the Cloud
- TLS termination and end-to-end TLS for FME Flow
- Scaling Examples
CloudFormation
A very popular AWS IaC tool is CloudFormation. If there is no multi-cloud strategy and it is expected that future deployments will also be on AWS only, then CloudFormation may be used:
Amazon Web Services: FME Flow (Distributed deployment, Windows) with CloudFormation
Bicep (ARM templates)
Bicep is a domain-specific language that uses a declarative syntax to deploy Azure resources, similar to Terraform. If there is no multi-cloud strategy and it is expected that future deployments will also be on Azure, only then Bicep may be used:
Microsoft Azure: FME Flow (Distributed deployment, Windows) with Bicep
Additional Resources
Please consider posting to the FME Community Q&A if you have further questions or issues that are not addressed in this article. There are also different support channels available.
For issues or limitations specific to your cloud provider or IaC tool of choice, please refer to their documentation.
The FME Flow documentation has some guidance on setting up load balancers. You may also find it useful to browse other resources or forum questions regarding load balancers in case this addresses our concerns.