FME Version
Introduction
You can utilize projects in FME Flow to group objects together under a common 'project' within your installation, making it easier to reference and manage them as a single entity.
The main advantage of using projects lies in facilitating the seamless transfer of configurations between FME hosts, particularly when migrating through the various stages of the enterprise life cycle, from development to testing/staging to production in your CI/CD workflow. Instead of backing up and restoring an entire FME Flow configuration, you can simply create and export the project and then import it into another FME Flow. This ensures that the target server's configuration remains intact while also incorporating the new project and all its associated components.
For an in-depth look at our most recent updates in 2023 to FME Flow Projects and to discover how to utilize these projects for constructing CI/CD workflows, please read through this article.
In this article, we will test creating, exporting, and importing projects as the owner of the objects, an FME Superuser, and as an FME Author on the installation. We will also explore “Weak” versus
“Standard” encrypted FME Flow. The FME Object types used to test this functionality are Automation, Automation App, Cleanup Task, Connection, Gallery App, Package, Project, Repository, Resource Connection, Resource Path, Role, Schedule, Stream, Subscription, API Token, Topic, User, Workspace and Workspace App.
Please note that this article and FME objects were created and tested with FME Flow 2023.2.
Step-by-Step Instructions
Part 1: Working with “Weak” encrypted FME Flow instance
The default Encryption Mode for the System Encryption setting is “Standard,” which employs a custom encryption key specific to your FME Flow installation.
If you decide to include sensitive data in your project export, that data will always be encrypted. But the security level depends on whether you use the weak encryption key shipped with every FME Flow [Encryption Mode=Weak] or use a custom stronger encryption key specific to your install [Encryption Mode=Standard].
In the case of Standard mode, when importing the project into a new FME installation, you will need to provide the unique encryption key from the source FME Flow instance. However, in this section, we will discontinue the use of custom encryption by opting for "Encryption Mode: Weak" within the Admin > System Configuration > Security > System Encryption settings on both the source and target FME. So, we do not need to provide any key during project import.
Creating the project
As an FME Superuser
You will be able to add any test FME object types mentioned above and any object irrespective of owner.
If the object has any dependencies [indicated by the green check icon under “Dependencies”], you will be able to view it by clicking the “Actions” ellipsis on the individual object. You do not have to manually add these dependencies to this project. In previous versions, the user would have to manually add all dependencies to the project at the time of export and then manually check that everything is expected on the target FME Flow once the import is complete.
As an owner FME Author/non-owner FME Author
You are only provided with the option to include the objects that you are an owner of or the objects that have been explicitly shared with the user or the role.
You are allowed to add your own user account only.
Role and Clean up task Objects are not provided as an option to add to the project if you are not a superuser.
Tokens, Workspace App, Automation App and Gallery App can only be included if you are the owner. Users assigned as “Allowed users” in Flow Apps will still be unable to add the object to the project.
If you are an FME Author who is not the owner of the project, you need to be granted at least “can view”/ “can run” share permissions in order to add an object to the project. Similarly, Schedules, Databases and Web connections that you do not own, will require “full access” sharing [the only option available for the object].
As in the case of the superuser creating the project, all dependencies are automatically added by default.
Exporting the project
To export a project, please follow the documentation Exporting and Importing Projects.
As an FME superuser/owner FME Author/non-owner FME Author
You can opt to “Include all sensitive information and use unique system encryption key” or not include them. Although the encryption mode is selected as “Weak” in this case, the sensitive information is still included and encrypted but with a weaker encryption key, which is shipped with the FME Flow installer.
All applicable object types will be exported with their current dependencies automatically, although it is not explicitly indicated in the “Total items” field under “Project Details.” During export, all dependencies are also updated by default to ensure you have the most updated version of the objects. In previous versions, there were issues with dependencies not being updated during export, and items required for the workflow to work could be out of date or not included.
Importing the project
To import a project, please follow the documentation Exporting and Importing Projects.
In the newer versions, there is clearer visibility about what the project contains. In previous versions, when a project was imported, there wasn’t a way to see how and what was going to be impacted.
There is clearer transparency about who owns the items contained in the project in the new FME Flow post-import. In previous versions, when a Project was imported by a user, the owner of the items on the new FME Flow was unclear. So, a user might have imported a project but might not be able to access it.
Please make sure that the new FME Flow [import recipient] should be, like the origin FME Flow, unencrypted [i.e. Encryption mode= Weak]. If there is a mismatch in the encryption mode between the two FME installs, you will encounter an error during project upload.
As an FME Superuser who did not export the project and does not exist on the source FME
In this test, the superuser that exported the project does not exist on the target FME. Also, the user was not imported with the project file. A different Superuser on the target FME has imported this project. Therefore, this Superuser importing will be the new owner of the imported project. If you would prefer, you will get an option to change the owner during the import process.
You are also provided with the option to “Allow items to be overwritten if they already exist”. Web and database connections are no longer imported and overwritten by default.
The owner of each imported item will remain the same as the origin FME unless the user doesn't exist on the target FME Flow instance or the user was not included in the project. For new items being imported without an existing owner, the fmesuperuser importing will become the owner unless changed.
As an FME Author who did not export the project and does not exist on the source FME
In this test, the FME Author that exported the project does not exist on the target FME. The user was also not imported with the project. A different FME author on target FME has imported this project. Therefore, this user importing will be the new owner of the imported project. Unlike the fmesuperuser importing, you will not get an option to change the owner during the import process.
User accounts, roles and user/role tokens [for workspace app, automation app] that were automatically included in the project file as object dependencies would fail to import as this operation requires that the importer have the fmesuperuser role.
Also before importing the objects into existing Repositories and Resources in target FME, please make sure that the user importing the project has been given permission to upload.
You are also provided the option to “Allow items to be overwritten if they already exist.” Web and database connections are no longer imported and overwritten by default.
The owner of each imported item will remain the same as the origin FME unless the user doesn't exist on the target FME Flow instance or the user was not included in the project. For new items being imported without an existing owner, the fmeauthor importing will become the owner unless changed.
Part 2: Working with “Standard” encrypted FME Flow instance
In this case, the source and target FME Flow have “Standard” encryption. When importing the project into the target FME installation, you will need to provide the target FME with the unique encryption key from the source FME.
System Encryption is set up as "Encryption Mode: Standard (Recommended)" under the Admin > System Configuration > Security > System Encryption settings. We can generate a new key, upload your organization’s custom key or download the current key from this webpage. Please download the key from source FME so that you can use it during project import.
The difference between working with the “Weak” versus “Standard” encrypted project is evident during the Project import. If you try to import the project without the correct key, you will see the error -
Unable to import the migration package because it contains encrypted data that cannot be decrypted. Please apply the encryption key to FME Flow then try again.
To successfully upload the project, you will have to upload the source FME’s key to the target FME.
Part 3: Removing a Project
In the new version, more options are provided when removing a project from FME Flow:
- Only remove the project
- Remove the project and all project items from FME Flow
- Remove the project, all project items, and their dependencies from FME Flow
For the latter two options, you can review Project Items & Dependencies before deleting. Only items you have permission to delete will be deleted.
Comments
0 comments
Please sign in to leave a comment.