Managing Projects with the REST API

Introduction

Projects in FME Flow are convenient packages used to migrate items between instances. While usually managed in the FME Flow Web UI, REST API calls can also be used to create, edit, export, and import project files. 

Creating a New Project

Overview

1. Create a project using POST /projects.
2. Find the new project's ID using GET /projects.
3. Get IDs of items to be added to the project.
4. Add items to a project using POST /projects/{id}/items/batch/add

Example

In this example, a project will be created containing all workspaces in the Samples repository.

1. Create New Project

 Send a POST request to /projects with a body such as {"name": "example", "description": "Example to demonstrate project management with the API."}.

A 201 response should be returned:

2. Get the New Project's ID

Send a GET request to /projects and note the ID corresponding to the name just specified.

3. Get IDs of Items to Add to the Project

The API endpoints needed for this step differ depending on the item type - see the API docs for details. In this example, workspaces from the Samples repository will be used, so just one API request is required: GET /repositories/Samples/items.

In FME Flow, item IDs take different forms. Since this is repository/workspace.fmw for workspaces, names must be gathered to construct the workspaces' IDs.

Item type should also be gathered at this stage - this is required in the next step.

For this example, the ID is Samples/austinApartments.fmw.

4. Add items to Project

First, use the ID and types gathered in the last step to construct the request body. The schema is viewable in the docs, matching this format:

{
  "items": [
    {
      "type": "workspace",
      "id": "Samples/austinDownload.fmw"
    },
        {
      "type": "workspace",
      "id": "Samples/austinApartments.fmw"
    },
    ...
  ]
}

As mentioned, item IDs take different forms. Workspace IDs take the form repository/workspace.fmw which is seen here.

Send a POST request to /projects/{id}/items/batch/add with the body just constructed. {id} is the project ID found in step 2.

If successful, the response status code will be 200:

If something went wrong, the response will be 207 and the errors will be shown within the response:

5. Verify the Items were Added Successfully.

This can either be done by sending a GET request to /projects/{id}/items or within the FME Flow UI.

Exporting a Project

Exporting a project involves creating a .fsproject file from an existing project. This file can be saved locally or in Resources.

Overview

1. Get the project's ID from its name using GET /projects.
2. Export as a download using POST /projects/{id}/export/download, or export to Resources using POST /projects/{id}/export/resource.

Example

1. Get the Project's ID

See step 2 above for guidance on this.

2. Export as a Download or to Resources. 

To export as a download, use POST /projects/{id}/export/download. Calling this from within an HTTPCaller in Workbench, the response can either be saved to a file, attribute or folder:

An example request body is {"exportPackageName":"example.fsproject"}. Consult the endpoint's schema for more information on this.

To export to Resources, use POST /projects/{id}/export/resource. An example request body is {"exportPackage":"example.fsproject","resourceName":"FME_SHAREDRESOURCE_TEMP"} - consult the schema for more information.

Importing a Project

Importing a project involves adding an existing .fsproject file to a FME Flow instance. 

Overview

1. Upload the project file using POST /projects/imports/upload or POST /projects/imports/resource. Get the job ID from the response header.

2. Run the import using POST /projects/imports/{id}/run.

3. Check the import was successful using GET /projects/imports/{id}.

Example

1. Upload the .fsproject File

If uploading from outside of FME Flow, use POST /projects/imports/upload?skipPreview=true with the project file in the body:

If importing from Resources, use POST /projects/imports/resource with a body of the following format:

{
  "resourceName": {resource connection},
  "resourcePath": {path to .fsproject from within connection},
  "skipPreview": true
}

Parse the ID from the Location header, highlighted in both screenshots above.

This header does not appear in the API documentation, so it is subject to change without notice. See the Troubleshooting section for guidance on an implementation suitable for production environments.

2. Run the Import

Send a POST request to /projects/imports/{id}/run to run the import, using the schema in the API docs to construct the request body. If initiated successfully, a 202 response is returned. If a generic 500 error message is returned, see the troubleshooting section.

3. Verify the Import was Successful

After submitting the import, another API call is required to check if the project imported successfully or not. To view this status, send a GET request to /projects/imports/{id} and check the value of the "status" attribute. It will be one of the following:

• "importing" if the import is still in progress. Repeat this step in a minute or so.
• "imported" if the import was successful.
• "failed" if the import was unsuccessful. 

If the status is "generatingPreview" or "ready", the project hasn't been imported yet. Go back to step 2.

If the import was unsuccessful, view the log in the FME Flow Web UI or using GET /projects/tasks/{id}/log/parsed?error=true. 

Demo Workspace

The attached workspace api_project_management.fmw runs through the three tasks mentioned. Here's how to use it:

1. Download and Open the Workspace

Download api_project_management.fmw and open in FME Form. 

2. Enable the Relevant Creator

Choose which task to run (Creating, Exporting or Importing), and enable the relevant Creator. To enable, right-click and press Enable:

3. Set User Parameters
Press Run, and set the relevant parameters by looking at the headers. Some of them may have default values, which can be kept. Set FME Flow Web URL to the domain of your FME Flow instance (without /fmeserver).

4. Check Results

Inspect Data Caches or view results in the FME Flow UI to verify behaviour is as expected:

Simpler Alternative: Using the CLI

Depending on the use case, it might be preferable to use the FME Flow Command-Line Interface instead of the REST API. The following commands can be used to achieve some of the tasks covered above.

• To create a project, use the API. This isn't currently possible with the CLI. 
• To export a project, use fmeflow projects download --name [project name]
• To import a project, use fmeflow projects upload --file [name.fsproject] 

Here's an example of the CLI used for project management. In the screenshot, the user authenticates, imports and exports a project. The command fmeflow projects is used to list projects currently on FME Flow.

Troubleshooting

Importing without Use of the Location header

Since the Location header is not documented, it is subject to change without notice in future versions of FME Flow. To perform the first step of importing using stable methods, do the following.

1. Upload the file as before, using POST /projects/imports/upload?skipPreview=true or POST /projects/imports/resource.

2. Find the ID of the most recently uploaded project by the current user, using the request GET /projects/tasks?limit=1&type=projectImport&filterString={username}.

The query parameters in step 2 aren't strictly necessary, but they prevent the wrong ID being fetched by simultaneous tasks on FME Flow.

500 errors when running an import

The endpoint /projects/imports/{id}/run is known to return generic 500 errors for valid requests. This is a known issue FMEFLOW-27108.

For most versions of FME Flow, using the request body {"notification":{"type":"topic"}} will import the project successfully.

If this doesn't work, inspect and copy the request body sent by FME Flow in DevTools: