FME Server Resources - Uploading, Referencing and Configuration

Introduction

Resource folders are a convenient storage area for uploading and storing data, and data being stored in a resources folder can be used when publishing or running workspaces. In addition, existing directories (such as UNC shares) can be mounted and used when publishing from FME Workbench, or browsed when submitting a job through the web interface.
Please download the tutorial data before starting the examples below.
 

Examples

Example 1: Upload Data Via the FME Server Web Interface

This example will demonstrate how to upload data to a Resources folder on FME Server through the web interface, and then utilize the data when running a job from a published workspace

1. Upload data to FME Server through the web interface

• Open the FME Server Web Interface > Files & Connections > Resources
• Select the Data folder > click + New > enter your first name
• Select the new folder and click Upload > Folder
• Browse to the downloaded tutorial data and select CityData.gdb

2. Generate a simple workspace

• Open FME Workbench > File > New > Generate Workspace
• Source: Esri Geodatabase (File Geodb Open API), <TutorialDownloadPath>\data\CityData.gdb
• Open the Parameters and select only CITY_GRID from the Table List
• Destination: Esri Geodatabase (File Geodb Open API)

Generate Workspace wizard

• Run the workspace to ensure that it is working correctly
• File > Publish to FME Server > create new repository named "Resources Tutorial" > Use workspace name: example1.fmw (Alternatively you can use the Publish icon at the far end of the ribbon to get to this wizard )

Create a New Repository in the Publish to FME Server wizard

• Under the Register Services window, you can leave the Job Submitter service checked, but also select the Data Download service.

3. Browse and use data uploaded to FME Server

• Open the FME Server Web Interface. On the homepage, under Last Published Workspaces, select the workspace you just published. This will display the Run Workspace page.
• Under Published Parameters, click the Browse Resources button

• Browse to the folder with your name that you created in the Data folder. Click the + button to choose the CityData.gdb folder and then click OK.

• You will see the path to the geodatabase has changed to the Shared Resources path. Run the workspace.

 

Example 2: Upload Data Via FME Workbench

This example will demonstrate how to upload data to a Resources folder when first publishing a workspace from FME Workbench using the ‘select files’ option. This data then becomes available for use at any time by other published workspaces.

1. Generate a simple workspace

• Open Workbench > File > New > Generate workspace
• Source: MapInfo TAB (MITAB), <TutorialDownloadPath>\data\parks.tab
• Destination: MapInfo TAB (MITAB), data folder
• Once the workspace has been generated, open the writer feature type parameters and change the table name to Parks2 to prevent a naming conflict. 
• Run the workspace to test that it is working correctly

2. Publish a workspace and upload data

• File > Publish to FME Server > Repository: Resources Tutorial > Use workspace name: example2.fmw
• Click "Select Files..."
• Check the ‘Files’ checkbox to select all the files
• Click "Select Location..."
• Check "Upload to a shared resource folder"
• Click Location… > Data > first name.
• Register with the Job Submitter service and finish publishing

Select the location for files to be uploaded to when publishing a workspace

• Return to FME Server and run example2.fmw
• Under the Published Parameters notice this time the resource folder is already specified for the Source MapInfo TAB file

• Now that the data has been uploaded into the Resources folder you can reference it in any future Workspace that requires the Parks.tab dataset.

Note: the example2 workspace can be used for examples 4 and 5 below.
 

Example 3: Upload Data Via the FME Server System Share

This example demonstrates how to upload data into the Resources folder without logging on to the FME Server Web Interface by looking directly into the operating system’s file system.

1. Copy data into the Resources Folder Within the File Directory

• Open the File Directory and navigate to the data folder you downloaded at the beginning of this article. Right-click on the data folder and select Copy.
• Now return to the Local Disk drive and select View and check the box for Hidden Items

Locate Hidden Items in the file directory

• The folder ProgramData has now been revealed - this is where the FME System Share is located and is home to the Resources folders. Navigate to ProgramData > Safe Software > FME Server > resources > data.
• If you have already completed the previous examples there will already be a folder under your name located here, if not then now you can Paste the data folder that we just copied and rename it to <your name>. When you return to the Web Interface, this data is now available.

Uploading a folder to FME Server Resources through FME System Share

Example 4: Using a Default Resource Folder in a Workspace

This example demonstrates reading data from a default FME Server Shared Resource folder using a parameter, so that it doesn’t need to be selected at runtime. You can view the parameters available for default FME Server Shared Resource folders in Navigator.


1. Add the Resource Location to a Reader Using a Parameter 

• Open example2.fmw, or create a new workspace following the steps in Example 2 above, if you have not already done so
• Right-click the MITAB reader and select Edit Parameters
  • Or, expand the MITAB Reader in the Navigator panel and double-click on the Source Parameter
• For the Source MapInfo File(s), click the drop-down arrow and then User Parameter > FME Server Parameters > FME_SHAREDREDRESOURCE_DATA. This parameter corresponds to Data folder path under Resources in FME Server.  

• Add \<your first name>\Parks.tab to the end of the FME Server parameter to complete the path to the Parks.tab file uploaded in Example 2.

Use a parameter to read data saved in FME Server Resources

• You can test the reader dataset path in FME Workbench by setting a temporary value for the FME_SHAREDRESOURCE_DATA parameter. For example, if the local copy of the Parks.tab file was in C:\Data\<your name>, then set C:\Data as the parameter value and run the workspace from FME Workbench. The temporary parameter value will be overwritten once published to FME Server. 

3. Publish and run the workspace

• File > Publish to FME Server > Repository: Resources Tutorial > Use workspace name: example2.fmw > Register Services: Job Submitter > Publish
• Open FME Server and run the example2.fmw workspace. From the Last Published Workspaces, navigate to and select example2.fmw. Run the workspace.

Note: you can also browse the Resources available on FME Server by expanding the drop-down tab next to Dataset and choosing Select File From Web > Browse FME Server…. This can also be done in the Add Reader Wizard when the reader is first added to the workspace, as shown below.

Select an FME Server Resource using the Select File From Web option
 

Example 5: Using a Custom Resource Folder Within a Workspace

This example demonstrates how to use a parameter to that contains a resource folder you created within an FME Workspace. Note: if you see an error message such as "Failed to open...", you may need to change the user account that the FME Server Engines service is using. See the guide here.

5a: Reading Data

1. Get the Parameter Name

• In FME Server, Under Files & Connections, click Resources.
• Click the check box next to the resource folder. In the example below, a custom network-based resource named My Data.
• From the Actions menu, select Properties.

 

• The System Path displays the parameter name that can be used in an FME Desktop Workspace.

 2. Testing the Parameter for Reading Data

• Open example2.fmw in FME Workbench
• In the Navigator panel, right-click User Parameters, and then select Manager User Parameters….

 

• Modify the linked parameter created by FME Desktop when the MITAB reader was added. Uncheck Published to make the parameter private and change the Default Value to use the parameter name for your resource, followed by the file name.

• Now, expand the Reader. Notice the source file location has changed since the linked parameter default value was updated. Note: the parameter may appear with a red cog, as shown below - this is because FME Desktop is not aware of parameter names for new resources created in FME Server. You can disregard this - as long as the parameter name is correct, the workspace will still run successfully once published to FME Server.  
• Publish the workspace to FME Server with the Job Submitter service enabled. Since this Workspace has already been published, you may be able to republish it to save repeating the entire publishing process.
• Return to the web interface and run the workspace example2.fmw. This time, you will notice that there is no option to point FME Server to a Source file as this is no longer a parameter accessible to the User (because Published was unchecked).

5b: Writing Data

This example demonstrates writing data using a parameter to specify file location.

1. Add the Parameter to a simple workspace writer

• Open example2.fmw, if you do not already have it open
• Add a writer to the workspace and set the Format to Esri Shapefile. Temporarily set the Destination Esri Shapefile Folder to a local folder, for example, C:\Data. You will receive an error if you set the destination with the parameter during creation, since FME Desktop is not aware of it.
• Open the writer and enter the resource folder’s parameter, followed by parks, for the destination folder. This will cause the shapefile and its sidecar files to be created in a new Parks subfolder within your resource folder. 

2. Publish and run the workspace

• Publish the Workspace to FME Server, making sure that the Upload data files option is left unchecked.
• Return to the FME Server Web Interface and run the newly published workspace.
• Upon completion return to your Resources folder (for example, Resource > My DATA and you should see a new folder containing the Park shapefile and sidecar files.

Folder created once the workspace was run containing the new shapefile and sidecar files