FME Version
Introduction
FME Flow’s Data Virtualization feature enables users to build fully functional REST APIs without writing any code. APIs can be created from the ground up, allowing complete control over the API’s structure, endpoints, request/response formats, and configuration settings. This capability is particularly useful for organizations that need to expose internal data in a standardized and accessible way.
Approaches to Creating a Data Virtualization API
There are two ways to create a Data Virtualization API in FME Flow: from scratch or using an OpenAPI Specification (OAS) template. The main difference lies in the level of manual configuration required.
- From Scratch: Manually define the API structure, endpoints, request/response formats, and settings using the FME Flow interface. This method provides full control over the API and is ideal for building custom APIs without pre-existing documentation.
- From a Template:Import an OpenAPI Specification (OAS) file, which automatically generates the API structure, including endpoints and methods. This approach reduces manual work but requires alignment with an existing OAS, such as when migrating APIs from another system.
Both methods require the author to configure responses and integrate FME workflows that process API requests.
Data Virtualization is currently in technical preview within the 2025.1 beta and should not be used for production. Note that documentation may change rapidly and not reflect the current build. This article was written with FME 2025.1 b25562.
Learning Objectives
After completing this lesson, you will be able to:
- Create a Data Virtualization API from Scratch in FME Flow
Step-by-Step Instructions
In this exercise, you will begin building the EnvironData API, a standardized interface for accessing real-time environmental and disaster-related data. In this scenario, the EnvironData Coordination Office requires an API that consolidates information from multiple internal sources into a single access point. By exposing this data through a Data Virtualization API, the organization can support internal systems, external agencies, and public applications, reducing manual requests and improving data accessibility and efficiency. You will configure the API from the ground up, including metadata, namespace, versioning, and access controls.
1. Create a New API
Open FME Flow and select Data Virtualization from the sidebar. Click the “Create” button to open the Create API window.
The Create API page is where the API documentation, contact information, security endpoints, caching, and asynchronous processing will be set.
The API Details section allows us to set the basic information about our API. This information will be used to structure the API and be shared with end users as a reference.
| Field | Value | Description |
| API Title | Environmental Impact and Response API | The header for the documentation page. It will label any automatically generated Data Virtualization workspaces. |
| Namespace | EnvironData |
A unique identifier for your API. It is included in the API's URL path. Since it forms part of the endpoint URLs, it's recommended that you do not change it after it's been set. For easier integration and cleaner URLs, it is best practice to choose a short, clear, and descriptive namespace. Based on this, all requests to the Data Virtualization API will take the general URL form: https://<FME Flow>/api/EnvironData/<endpoint> |
| Versioning | 1.0.0. | Allows you to track iterations of the Data Virtualization API |
| Summary | EnvironData API provides real-time access to environmental and disaster-related data for effective response and analysis. | A short description of the API that will appear in the documentation. |
| Description | The EnvironData API provides essential tools for accessing real-time environmental and disaster data. It supports integration into emergency systems, monitoring platforms, and research projects to enhance preparedness and mitigate impacts. | A longer form description of the Data Virtualization API. This is an opportunity to include any information about the usage of the API, its data sources, organizational information, etc. |
| Job Queues | (unselected) | Routes all requests to the API through a specific engine. Leaving this blank will use the Default queue. |
| Job Expiry Time | 2 minutes | Determines how long can elapse between the initial request and delivery of the response. User requests will receive an expiry error if their requests exceed 2 minutes. For an API with requests that will run processes exceeding 2 minutes, like complex transformations or large data sources, extend this time. |
Contact information appears on the API documentation page and helps end users know who to reach out to with questions, issues, or feedback. This can be the details of an organization or a specific administrator responsible for the API. While optional, providing contact information adds professionalism and supports user trust. You can enter your own details or use the (fictional) sample data provided below.
| Field | Value |
| Name | EnvironData Coordination Office (EDCO) |
| URL | https://environdatagov.example |
| support@environdatagov.org |
Endpoint Security determines the users and roles that can access all endpoints in the API, ensuring consistent security policies without requiring separate configurations. After API creation, access can be managed and updated per individual endpoint.
| Field | Value | Why |
| Access Level | Authenticated | This means all endpoints require authentication by default. |
| Allowed Roles | “fmeadmin”, “fmeauthor”, and “fmeuser” | This will allow all users in these roles (and the tokens they create) to access API endpoints by default. |
| Allowed Users | Select your own user account, for example, “dvauthor” in this example. |
By default, only selected users and the tokens they generate are granted access to specific API endpoints. Your own account is not automatically granted access. To ensure you can access all necessary endpoints, explicitly assign access to your user account. In some cases, you may choose not to grant yourself access to all endpoints. For example, if different departments manage distinct areas of the API and your role is limited to overall administration rather than direct interaction with specific data. |
Caching temporarily stores the results of API calls, reducing the number of requests to the underlying data source. By serving cached data, API caching enhances application performance, decreases latency, and reduces the load on the API server, leading to a more efficient and scalable system. Caching can be enabled at an API level or at an individual endpoint level. Leave as default, No.
Asynchronous Processing allows clients to submit requests to the Data Virtualization API without waiting for immediate completion. Instead of blocking the client until the request finishes, the API returns a request identifier that can be used to check the status or retrieve the results later. This is ideal for long-running operations, improving system responsiveness and scalability. Leave as default, No.
Asynchronous processing is disabled by default for the EnvironData API endpoints.
To finish, click Create.
2. View Documentation
Upon creation, the API Details page opens.
The API Details page for the Environmental Impact and Response API provides access to key configuration settings. From this page, you can manage general API administration options, update settings as needed, and view integration-related configurations such as CORS policies to support cross-application communication.
API Details are used to automatically generate documentation. Use the View Documentation link to automatically open the Swagger documentation in a new browser tab.
The Swagger docs will be the main reference for users. It will show available endpoints, data formats, security rules, and other API details. Right now, there are no endpoints. The next step in our Data Virtualization work is to build and publish the first one.
3. Continue to the Next Exercise
Close the Swagger documentation to return to FME Flow. Continue to the next exercise to build the first endpoint: Create a Manual Endpoint in Data Virtualization.
Comments
0 comments
Please sign in to leave a comment.