Enhancing Data Virtualization Endpoints with Parameters

Introduction

Data Virtualization supports endpoint parameters, which allow users to refine API requests and customize responses. These parameters help specify which resources to access (path parameters), how to filter or modify the results (query parameters), and include additional instructions or metadata (header parameters), such as authentication details.

Parameters are first defined when creating the endpoint metadata in FME Flow. Then, inside the Data Virtualization workspace, you use readers, writers, and transformers to apply the parameters. These components handle the filtering, transformation, and routing based on the values sent in the request.

Request Parameters 

Data Virtualization supports all major types of HTTP parameters:

• Path parameters modify the URL path to identify a specific resource (e.g., /users/{userId}).
• Query parameters are appended to the URL to filter, sort, or control response content (e.g., /search?query=keyword).
• Header parameters provide additional instructions, such as authentication tokens or metadata (e.g., Authorization: Bearer {token}).

Most parameters can be optional, giving users the flexibility to tailor API responses to their specific needs while improving performance and usability.

Data Virtualization is currently in technical preview within the 2025.1 beta and should not be used for production. Note that documentation may change rapidly and not reflect the current build. This article was written with FME 2025.1 b25562.

Learning Objectives

After completing this lesson, you’ll be able to:

• Update an existing endpoint workspace
• Utilize endpoint parameters effectively in a workflow
• Use templates to test and run parameter values
• Validate request data 

Step-by-Step Instructions

In this lesson, the EnvironData Coordination Office will update the GET /wildfires endpoint by adding query parameters that give users more control over the data they retrieve. This involves configuring the parameters in FME Flow, updating the workspace logic in FME Form, and republishing the changes.

Two query parameters will be added:

• limit: Specifies the maximum number of results returned
• status: Filters results based on the wildfire status attribute
  
  

These enhancements improve the API’s flexibility, making it easier for users to request targeted and manageable data.

Part 1: Update an Endpoint with Query Parameters

In this part, the EnvironData Coordination Office defines the limit and status query parameters in FME Flow. This step sets up how the API will accept input from users, including the parameter names, data types, and where they appear in the HTTP request. These definitions prepare the endpoint for dynamic filtering and controlled result sets in later stages of the workflow.

Updating a Data Virtualization Endpoint

Once a data virtualization workspace is published, any changes to its endpoint metadata in FME Flow will change the workspace status to “Needs Updating.” This means you must redownload the workspace into FME Workbench. The changes will appear in the reader, writer, and templates. From there, it’s up to the author to update the workflow to support the new metadata and then republish the workspace to FME Flow.

1. Add Query Parameters to an Existing Endpoint

In FME Flow, open the Data Virtualization EnvionData API and navigate to the Endpoints tab. Select the  GET /wildfires endpoint. Navigate to the Parameters tab. 

Next to Query Parameters, click Add. Add and complete two query parameters for ‘limit’ and ‘status’. Leave these parameters as not required, the end user can choose to include them in the request as needed:

2. Create a New Status Code in FME Flow

Introducing parameters to the request also introduces potential failure points. To handle cases where users provide invalid parameter values, we will implement an additional status code to ensure clear and appropriate responses.

Navigate to the Response tab. Click Add Status Code

Configure a "400 - Bad Request" status code, which will apply when a user submits an invalid value for the “status” parameter.  

Instead of using predefined schemas, responses can also be defined by creating custom properties. This approach is best suited for endpoints with simple or unique response structures that do not require a shared schema. Properties function the same way as schemas in terms of how responses are structured and returned. They will appear in your API documentation as examples of expected responses, providing clarity for end users without the need for a separate, reusable schema definition.

Click the Create button next to Properties to define a new property.

Review the completed parameters and Save. 

Updating the endpoint metadata in FME Flow automatically triggers a warning message. This simply indicates that the associated workspace must be updated to reflect the new configuration changes.

Click Save to continue. 

3. Configure an Updated Data Virtualization Workspace

After updating an endpoint metadata in FME Flow, the endpoint workspace must also be updated and republished for the changes to take effect. Until this is done, the configuration updates remain incomplete in the Web UI.

To verify, go to the Workspaces tab and locate the GET /wildfires endpoint. The Workspace Status will have changed from "Assigned" (its previous status) to "Needs Updating", indicating that the workspace requires an update before the changes are applied. Proceed to Part 2 to update the workspace. 

Part 2: Control Response Size with Limiting Parameters

In this part, the EnvironData Coordination Office configures the limit query parameter to control how many wildfire records are returned in a single API response. This parameter improves usability by preventing overly large result sets and giving users more control over the volume of data retrieved. The limit parameter is implemented in FME Form by updating the workspace logic to read the input value and apply it to the SQLite query.

1. Update an Endpoint Workspace in FME Form

Open FME Workbench. 

When an endpoint metadata is updated in FME Flow, the associated workspace must be redownloaded or manually updated to reflect those changes. In fact, you will be prevented from publishing a workspace that is out-of-sync with the FME Flow Endpoint metadata. Selecting Update Reader in the publishing wizard will sync the workspace.  

Additionally, there are two other ways to proceed, depending on your setup:

If the WildfireEndpoints.fmw workspace is not already open in FME Workbench:

Open FME Workbench, then go to the Data Virtualization menu and download the workspace. This will retrieve the updated version of the workspace based on the latest endpoint metadata in FME Flow. Once downloaded, proceed to the next step.

If the workspace is already open or you are using a locally saved copy:

If your workspace is open, right-click the Data Virtualization reader in the Navigator and select Update Reader. Once updated, you can continue authoring the workspace as needed.

Once updated, expand the GET /wildfires feature type on the Data Virtualization reader type. Notice the addition of two new attributes: query.limit and query.status. 

In the next steps, the workflow will be updated to handle these additional parameters. 

2. Update the Input Template

Select “Run Just This” on the GET /wildfires feature type. 

In the Translation Parameter Values window, update the placeholder values with test values. Replace “<integer>” with “3” and delete the “status” parameter. Click Run. 

{
   "path": "/wildfires",
   "method": "GET",
   "parameters": {
      "limit": ["3"]
   }
}

This template serves as a substitute for the request: http://<FME Flow>/api/EnvironData/wildfires?limit=3 

Templates can also be saved for later use. Select Save As Sample…

Save the template sample as get_wildfires-limit.json

This sample file will be automatically saved to a “Samples” folder in your local “EnvironData” directory. 

Select Run. 

When the translation is finished, review the Visual Preview window to find the new attributes: the query parameters and any test values from the input template.

Now that the query parameters are available in the FME workspace as attributes, we can use them to author an endpoint response. 

3. Use a Query Parameter to Limit the Number of Results

The limit query parameter lets users set the maximum number of results they want in the response.

There are different ways to filter data in FME, but the best way is to filter it before it loads into the workspace. You can do this by using built-in reader options like WHERE clauses, SELECT statements, or spatial filters. You can also use database transformers like InlineQuerier, SQLExecutor, or DatabaseJoiner. These methods are faster than using downstream tools like the Tester or Sampler, which only filter data after it's already loaded.

In this example, we will apply the filter directly on the FeatureReader.

Open the FeatureReader. In the Constraints section, click the drop-down menu next to Max Features to Read. From Attribute Values, select the query.limit attribute. 

Click OK to apply. 

To test the parameter, run the entire workspace. 

Once the translation is finished, locate the written data in the “Response” folder.

Open dv_output.json in a text editor. 

This time, only 3 of the 6 rows in the SQLite fires table have been returned in the response. This confirms the limit parameter has been successfully applied in the workspace.  

We can also confirm this directly in the workspace. Note that the FeatureReader has only passed 3 features from the fires port. 

The FeatureReader’s Max Features to Read parameter can filter by any value greater than 1, even those that are missing. Setting the value to 0 or removing the limit query parameter value returns all records by default.

4. Building in Logic for Optional Query Parameters

Now that we've set up the limit parameter, we can focus on status.

The status query parameter lets users filter the wildfire results by their status values stored in the SQlite database. We'll eventually use a WHERE clause in the FeatureReader to apply it.

But first, we need to handle two things. One, status is optional, so users might not include it. A WHERE clause needs a value to work, so we have to check if one was provided. Two, if a value is sent, it must match a valid status in the dataset.

The TestFilter transformer is one transformer that is great for this kind of conditional logic. 

Add a TestFilter to the workspace. If needed, make space on the canvas and connect it right after the GET /wildfires feature type.

Open the TestFilter to configure. 

The first condition will determine if the query.status has a valid value, or not. Recall, the valid values for status in our ‘fires’ table are: “active”, “contained”, or “resolved”. 

Double-click on the If row to open the Test Conditions dialog. Create the following tests:

The completed Test Clauses table should have a test for all the values. Update the Output Port value to “Status” and click OK. 

Back in the main TestFilter parameters, open the Else If row to configure another condition. 

This time, create a test for a request that does not have a value for the status query parameter with the “Attribute is Missing” Operator.

Set the Output Port value to “No Status” and click OK. 

To handle all other requests, label the last Output Port as Invalid. 

Click OK to save. 

The updated TestFilter now has 3 Output ports, one for each type of request. Connect the Status port to the FeatureReader.

All requests that contain a value for the status query parameter will be sent to the FeatureReader. 

5. Complete the ‘limit’ Query Parameter Workflow

If the request doesn’t include a status value, we need to use a second FeatureReader.

Both FeatureReaders will be almost the same, except for the WHERE clause. To save time, just duplicate the first one. Right-click it and choose Duplicate.

FeatureReader_2 will be added to the canvas. Connect the TestFilter No Status output port to FeatureReader_2.

Configure the FeatureReader_2 downstream connections. They will be identical to the original FeatureReader.  

• Connect the FeatureReader_2 <Initiator> output port to the Tester
• Connect the FeatureReader_2 fires output port to the JSONTemplator WILDFIRES port. 

Recall our input template (GETwildfireslimit.json) does not contain a value for the status query parameter. Therefore, we can use it to test our workspace logic for a “No Status” request. Rerun the workspace.

The workspace now successfully processes a request where the client does not submit a status query parameter.

Part 3: Customize Response Data with Filtering Parameters

In this part, the EnvironData Coordination Office authors the second query parameter, status, to filter wildfire records by their status (like active, contained, or resolved). This lets users request only the data that matters to them.

We’ll also add error handling. If a user sends an invalid status, the workspace will catch it and return a 400 Bad Request. This gives clear feedback when input doesn’t match what’s expected.

These updates make the EnvironData API more reliable, flexible, and user-friendly.

1. Update the Input Template

To author the workflow for the status query parameter, we will update the input template with a test value. 

Select “Run Just This” on the GET /wildfires feature type. 

This time, remove the “limit” query parameter and update the “status” value to “active”:

{
   "path": "/wildfires",
   "method": "GET",
   "parameters": {
      "status": ["active"]
   }
}

This template serves as a substitute for the request: http://<FME Flow>/api/EnvironData/wildfires?status=active 

To use again later, consider saving the template as a sample (e.g. get _wildfires-status.json). 

Click Run and review the Visual Preview window to confirm the attribute values have been updated. Now the query.limit value is missing and the query.status value is active. 

2. Use a Query Parameter to Filter Results by Attribute 

We’ll use the query.status attribute to filter the SQLite fires table. 

Open the original FeatureReader transformer (connected to the “Status” port). Within the Constraints group, find the WHERE Clause. Enter a statement that is compatible with SQLite:

"status" = '@Value(query.status)'

Click OK to save. 

On the FeatureReader, click Run To This. Inspect the cache on the fires port. Only 2 features are returned, each with an “active” value for its status attribute.

This confirms that our WHERE Clause has successfully been applied. 

Similarly, rerunning the entire workspace and inspecting the dv_output.json file confirms that the response body contains only the filtered results. 

3. Testing with Data Entry Errors 

It’s good practice to test with as many endpoint parameters as possible to build strong, reliable workspaces. This not only helps ensure accurate status codes, but also gives a chance to handle common input errors or data mismatches between systems. To do this, we will purposely test with an invalid value.

Rerun the workspace. Replace the status value with “ACTIVE” and set the limit value to “3”. 

{
   "path": "/wildfires",
   "method": "GET",
   "parameters": {
      "limit":["3"]
      "status": ["ACTIVE"]
   }
}

To use again later, consider saving the template as a sample (e.g. get _wildfires-status-limit.json). 

This template represents the request: http://<FME Flow>/api/EnvironData/wildfires?limit=2&status=ACTIVE 

Notice that the status value doesn't match our data source, as it uses uppercase instead of lowercase. Database queries, like a WHERE Clause, are often case sensitive. 

Run the workspace with the new values. The workspace does not complete, because the uppercase query.status value is routed to the “Invalid” port by the TestFilter.

Formatting issues can come from user mistakes or differences between systems. For example, one app might use uppercase for status values, while another uses lowercase. FME can fix this by adjusting the data before querying the SQLite database, making it easier to connect systems that don’t match exactly.

4. Build in Solutions for Expected Errors

To handle mismatched cases, add a StringCaseChanger to the canvas. Making room if needed, connect it between the GET /wildfires feature type and the TestFilter.

Open the StringCaseChanger to configure the parameters.

Use the ellipsis to open Selected Attributes. Expand ‘query’ to select ‘status’.

Click OK. Beside Case Change, select “lowercase”. 

The updated workflow can now process query parameter values sent in an incorrect case. This prevents clients from receiving inaccurate error responses due to incorrect data entry.

Rerun the entire workspace to verify. 

5. Create an Invalid Query Parameter Response

So far, the workspace handles requests with no query parameters, valid ones, and small input errors. But if a client sends an invalid value, the API should respond with a clear error.

In this final step, we’ll set up the workspace to catch invalid query values and return a 400 Bad Request. We already set up this status code in FME Flow. Sending a 400 lets the client know their input isn’t valid, which improves error handling and protects data quality.

Add another AttributeCreator to the workspace. Connect the new AttributeCreator_3 to the TestFilter Invalid output port.

Open the AttributeCreator_3 to configure the response attributes. 

Review the parameters.

Recall that for the '200' status code response, we configured the response.body.content attribute value using the JSONFormatter. 

This time, we've manually set up the response.body.content attribute directly in the AttributeCreator. This approach is best suited for shorter responses that do not require dynamic formatting, such as responses without arrays or nested objects. It offers a straightforward alternative when the response structure is simple and static. 

Click OK to close. 

Finally, connect the AttributeCreator_3 Output port to the http_response writer feature type to complete the workflow. 

To test the invalid response, update the template to use the status value “Closed” 

This template represents the request: http://<FME Flow>/api/EnvironData/wildfires?limit=3&status=CLOSED 

Run. View the dv_output.json to confirm the expected error response: 

Save the workspace. 

6. Republish to FME Flow

Publish the workspace to FME Flow to apply the endpoint workspace updates. 

Once published, open FME Flow and navigate to the Workspaces tab. Once again, the GET /wildfires endpoint Workspace Status is Assigned. 

7. Test Request in Documentation

Use the Swagger documentation or another API client to make a request to the endpoint that includes parameters. In the GET /wildfires endpoint, notice the query parameters that we created are now available. Click Try it Out to enter the query parameters value:

• limit: 2
• status:  RESOLVED

For example, test the results for the request: http://<FME Flow>/api/EnvironData/wildfires?limit=2&status=RESOLVED

Click Execute

Review the response to confirm the requested data. 

We can also test our error messages. Try entering an invalid query parameter value for status, like closed. The “400” code and error message is returned. 

Next: Creating a POST Endpoint in a Data Virtualization API