FME Flow Troubleshooting: Proxies

Full Guide: FME Flow Troubleshooting Guide

Are you encountering issues connecting to FME Flow (formerly FME Server) through a proxy server?
 
A proxy server is a computer on your LAN that allows you to connect to external servers without compromising the security of your internal network. If you are consuming FME Flow Web Services and your organization uses a proxy server on your network, you can configure FME Flow to connect through the proxy.

HTTP Status 404 Not Found error can be an indication that you need to set the FME Flow up for your proxy server. Please read below for common troubleshooting tips, questions and resources. 

Content Overview

• Initial Troubleshooting
• Common Issues  
  • “I want the FME Flow Engines to use a proxy but not the system-wide proxy.”
  • “Instead of defining an FME-wide proxy address, I would like it to be set in HTTP Caller, so different HTTP Callers could use different proxies.”
  • “In FME Flow, I am trying to use a basic proxy and version control but I am getting errors for Proxy Authentication Required.”
  • “HTTPCaller and FMEFlowJobSubmitter not respecting proxy exceptions if Python is used in the workspace”
  • “How can I run jobs when my FME Flow Core cannot send traffic to my Load Balancer directly?”
  • "I'm seeing authorization and permission errors when I have FME Flow transformers in my workspace"
  • "I've set up FME Flow to use a reverse proxy and when I try and run jobs I receive a timeout error"
• Additional Resources
• Are you still experiencing issues?
• Have ideas on how to improve this?

Initial Troubleshooting 

• Confirm the following steps from the documentation :
  • Check that the proxy has been set up for the account you wish to use (Local, Non-Local)
  • Restart FME Flow Engines
  • Check that the engines are aware of the proxy server
• Confirm the authentication mechanism being used by the proxy setting configuration (NTML, Basic, or Digest).
• When executing the provided commands, they need to be executed as an Administrator.

Common Issues

“I want the FME Flow Engines to use a proxy but not the system-wide proxy.”

In this linked documentation, we mention how to add a custom proxy. This will override any system proxy that may be set. 
The following syntax specifies how to map access to one or more URLs through a custom proxy. Run the following command, repeating all parameters for each URL you want to map through the custom proxy, as shown:

fme.exe APPLY_SETTINGS SYSTEM "Proxy/Custom Proxy Map" "<Source_URL>,<Proxy_URL>,<Proxy_Port>,<Requires_Authentication>,<UserName>,fme_encrypt(<Password>),<Authentication_Method>[,<Source_URL>,<Proxy_URL>,<Proxy_Port>,<Requires_Authentication>,<UserName>,fme_encrypt(<Password>),<Authentication_Method>]"
Where:

• <Source_URL> is the URL you want to access. The URL must be an exact match with the URL in the request; for example, http://www.lib.uwaterloo.ca/locations/umd/digitization_project/Roads_NTDB.zip.
• <Proxy_URL> is the URL of the proxy server on your system that will handle requests to the <Source_URL>.
• <Proxy_Port> is the port number on the proxy server through which requests are handled.
• <Requires_Authentication> is true or false, depending on whether the proxy server requires authentication to access. If true, provide <UserName>, <Password>, and <Authentication_Method> (Basic, Digest, or NTLM).

You'll notice the use of SYSTEM in the syntax above. This controls where in the registry the reg key and value are stored - under HKEY_LOCAL_MACHINE [HKLM] and is associated with the FME Flow software and will only affect the FME Flow and the proxy it will use.
 

“Instead of defining an FME-wide proxy address, I would like it to be set in HTTP Caller, so different HTTP Callers could use different proxies.”

We do not have the ability to define proxy settings per workspace, per repository or per feature. An alternative could be to define an exclusion list using wildcards for any internal URLs.
 

“In FME Flow, I am trying to use a basic proxy and version control but I am getting the error below. How do I fix this?”  

Could not push to remote repository._3586
Remote ref update was rejected:
https://github.com/test.git: 407 Proxy Authentication Required 

To be able to push to an external Version Control while using the Basic Proxy, the following configurations must be made:

1. Open <FMEFlowDirectory>\Server\processMonitorConfigCore.txt
2. Look for # Start FME Flow Core
3. Add this to the START SYSTEM variable: -Djdk.http.auth.tunneling.disabledSchemes=
4. Place this string anywhere near the -DJava arguments
5. Restart Server

“HTTPCaller and FMEFlowJobSubmitter not respecting proxy exceptions if Python is used in the workspace”

If you have enabled a proxy in the UI and set up exceptions for the FME Flow, FME Flow will respect them, unless there is a Python in the workspace. 
If you open fmeFlowConfig.txt, under the heading FME SERVER SETTINGS START, add the following line: ENABLE_ENGINEPROXY=false
 

“How can I run jobs when my FME Flow Core cannot send traffic to my Load Balancer directly?”

FME Flow can be configured with a proxy server to enable the Web Application/Core/Engines to send outgoing traffic. However, the FME Services that are used to submit jobs were designed to remain internal, so they are not accounted for in this proxy configuration. In some installations, your Load Balancer may have the ability to send traffic to the core, but the core may not be permitted to send traffic back to the Load Balancer. In this case, you will be unable to submit jobs via the run workspace page. 



The file ‘hosts’ is an operating system file that maps hostnames to IP addresses. For each core machine, add a line in the host file, mapping the hostname of the Load Balancer to the current machine. The mapping consists of two values “127.0.0.1” which will reference the local machine and the name of the load balancer.  This entry will allow the calls needed to submit a service (run a job) to stay local to the machine. 

On a Windows OS, this file can be found at C:\Windows\System32\drivers\etc\hosts. 
For Linux, it is /etc/hosts.

 Example line to add to hosts file:
127.0.0.1        MyLoadBalancerName

"I'm seeing authorization and permission errors when I have FME Flow transformers in my workspace"

In certain proxy environments, FME Flow fails to run FME Flow transformers. The error message returned may look something like this:

FMEFlowNotifier: Unauthorized request by user due to lack of proper permissions for connection: <My FME Flow Connection Name>. Please see: http://fme.ly/zn8
FMEFlowLogFileRetriever: user not authorized to perform this action (HTTP 403)

To resolve this, go into FME Flow proxy settings (System Configuration > Network & Email > Proxy) and create an exception for the FME Flow URL using the fully-qualified domain name. In the "No Proxy For" parameter, include for example:

<hostname>.<domain>.*
*.<domain>

"I've set up FME Flow to use a reverse proxy and when I try and run jobs I receive a timeout error"

Additional Resources 

1. FME Flow Best Practices Guide for IT Ops and Flow Admins
2. Using a Reverse Proxy with FME Flow
3. Using IIS and ARR as a reverse proxy
4. Configure FME Flow for SSL using NGINX as a reverse proxy

Are you still experiencing issues?

Please consider posting to the FME Community Q&A if you are still experiencing issues that are not addressed in this article. There are also different support channels available.

Have ideas on how to improve this?

You can add ideas or product suggestions to our Ideas Exchange.