Hi everyone, this blog took a bit longer to get out than we’d planned but we hope it’s worth the wait.
A common question we’ve been seeing from various sources is how to configure Hardware Load Balancer health checks to work correctly with Web Application Proxy. There are some key considerations you need to be aware of when configuring your Load Balancer health checks, but if you know about these things, then it should be pretty straight-forward.
We will also look at new functionality introduced in the August 2014 Windows Server 2012 R2 Update rollup (KB.2975719) to provide HTTP Health Check endpoints for your AD FS 2012 R2 and Web Application Proxy servers.
Key Properties of a Health Check request
First, let’s look at the different properties at various protocol layers that we are interested in for a health check request and the recommended values.
Protocol Level | Property | Recommend Value |
TCP/IP | IP address | Dedicated IP of each Web Application Proxy farm member |
TCP/IP | Port | Port of service being queried e.g. 80, 443 |
| SSL | SNI Header | FQDN of service being health checked |
| HTTP | Host Header | FQDN of service being health checked (not IP address) |
| HTTP | Uri | URL path to test |
While some of these are self-explanatory let’s drill into the details of what each of these mean and the considerations to take into account. The main ones we have seen some confusion and issues around are SNI Header and Host Header.
1) TCP level – the IP address and port the health check request will be sent to
Probably an obvious one but You will want your health requests to be sent to the dedicated IP address of each of the Web Application Proxy farm members.
2) TCP level – the port the health check request will be sent to
The health check requests should be sent to the relevant port that the service/application is listening on - most likely port 443 in a Web Application Proxy / AD FS scenario.
3) SSL level – does your load balancer support Server Name Indication (SNI) and if not what are your options?
If you are making a health check to an SSL endpoint, after the TCP connection has been established, the Load Balancer will send an SSL Client Hello to initiate the establishment of the SSL tunnel.
Web Application Proxy and AD FS 2012 R2 both use Server Name Indication (SNI) http.sys certificate bindings. SNI allows multiple certificates to be used on the same IP/port pair and http.sys uses the SNI header in the SSL Client Hello to determine which certificate should be served. This relies on the client sending the Server Name extension in the SSL Client Hello.
If the SSL Client Hello does not contain the SNI header then http.sys is unable to determine which certificate to serve and will reset the connection.
You therefore need to determine if your Load Balancer supports SNI in its health check requests. If it does support SNI then ensuring that the Load Balancer sends the SNI header in the SSL Client Hello is our recommendation.
Unfortunately there seems to be limited information available around configuring SNI with the common hardware load balancer solutions.
The following F5 article has some good details though especially around configuring the F5 Health Checks to send an SNI header - https://devcentral.f5.com/articles/big-ip-and-adfs-part-5-working-with-adfs-30-and-sni
Note– while this article has some great information on SNI and Health Check requests it also recommends using SSL Bridging which can break Web Application / AD FS traffic especially if bridging is used between Web Application Proxy and the AD FS 2012 R2 servers. Please see the following blog for more details on when and why SSL termination can cause issues:-
For more details on SNI, how to detect if SNI headers are being sent and for options if your load balancer does not support SNI then please see the following blog article:-
Using the 0.0.0.0 fallback binding is something we are seeing needed in a number of deployments although our recommendation would be to use SNI where you can.
4) HTTP Host Header – this should be the FQDN of the service being tested
This might seem quite an obvious one, but we have seen a number of issues where the Hardware Load Balancer is sending the dedicated IP address of the farm member as the Host Header. This can cause health check failures due to the following:-
- A Web Application Proxy published application will register the external application FQDN with http.sys and will not have a registration for the local dedicated IP address
- AD FS Proxy on the Web Application Proxy will not handle requests where the host header is the local dedicated IP address
- When SNI is used, http.sys will validate that the host header in the request matches the SNI header used during the SSL negotiation
It is therefore very important to ensure that your Load Balancer sends the correct service FQDN as the Host Header in the health check request e.g. if you are health checking the AD FS Proxy service you would send the ADFS Service FQDN as the host header e.g.
Host: adfs.contoso.com
For a Web Application Proxy published application you would set the host header to the External FQDN of the published application.
Not specifying the Host Header or, sending the node IP address as the host header will lead to unpredictable and/or failing health check behavior.
The F5 article mentioned in the SNI section above has the correct host header behavior so if you are using F5 load balancers it’s definitely the place to start. The following Cisco forum discussion talks about setting the Host header in a Cisco health check probe:
It’s important to note that when you are health checking either a Web Application Proxy published application or the AD FS Proxy service, that content is served from the respective back-end server. You are therefore health checking not just the availability of the local service, but also the communications to, and availability of, the relevant back-end service. A failure at the back-end could therefore potentially cause your load balancer to mark a Web Application Proxy node as offline even if other applications are available depending on your HLB configuration.
UPDATE - as Jaydee points out in his comment if you are using pre-authentication for your Web Application Proxy published application(s) then you will get a 307 Redirect back when you health check your published application URL. This is expected and is Web Application Proxy redirecting to AD FS for authentication.
5) Which URL path to specify in the health check request
As mentioned above, neither Web Application Proxy, or the AD FS Proxy service serve content themselves and content will be retrieved from the back-end service.
For a Web Application Proxy published application, the path specified should be a relevant and valid path on the published application that will return a known response that the hardware load balancer can use for validation e.g. https://extapp.contoso.com/test.html
For health checks against the AD FS Proxy service (or in fact AD FS itself) the current recommendation from the AD FS Product group is to health check against the following URL:-
https://adfsservicefqdn/FederationMetadata/2007-06/FederationMetadata.xml
Does it really have to be this hard?
While we’ve covered a lot of technical detail in the above the health check issues we’ve been seeing people hit generally come down to one or two of the above areas:
- Does the Hardware Load Balancer support SNI?
If it does then make sure the HLB is correctly configured to send an SNI header. If it does not then using the 0.0.0.0 fallback binding on your AD FS / Web Application Proxy server should provide a workaround.- Ensuring that the correct Host Header is defined in the health check probe config
Host headers have been in use for many years as a way for a server to differentiate between requests for different sites. All HLB’s should be able to have a host header defined in their health check probes. If you are health checking against AD FS / AD FS Proxy then make sure you send the AD FS Service FQDN as the host header.
If you are aware of these 2 points and ensure these are taken into account when configuring your Hardware Load Balancer then getting your deployment up and running should be a lot smoother.
Disclaimer– while we have provided a couple of links out to HLB vendor sites the best people to advise you how to correctly configure your Hardware Load Balancer will be the relevant vendor. If you have any questions or doubts about how to correctly configure your Hardware Load Balancer please contact your vendor for advice.
No, I said “Does it really have to be this hard?”
Well, it is a good question even if you did have to ask it twice :-)
We’ve definitely heard the feedback that configuring Hardware Load Balancer probes for Web Application Proxy and AD FS 2012 R2 can be tricky. The issues generally boil down to a couple of key points and SNI and hardware load balancers particularly seems to fall into the realm of the mysterious and unknown.
We’re therefore pleased to be able to tell you about some new functionality that shipped in the August 2014 Windows Update rollup for Server 2012 R2 - http://support.microsoft.com/kb/2975719/en-us
*NEW* - HTTP based Health Check Probe functionality
The August 2014 Windows Update rollup (http://support.microsoft.com/kb/2975719/en-us) shipped new HTTP based health check probe functionality in ADFS 2012 R2 and the AD FS Proxy service that runs as part of the Web Application Proxy server role.
This avoids any issues with SSL certificates and SNI. It also supports the use of both FQDN’s and IP addresses in the probe helping avoid any potential host header issues. The new HTTP probe can be accessed over HTTP using the path /adfs/probe e.g.
http://<Web Application Proxy name>/adfs/probe
http://<ADFS server name>/adfs/probe
http://<Web Application Proxy IP address>/adfs/probe
http://<ADFS IP address>/adfs/probe
The response to these probe endpoints is an HTTP 200 OK and is server locally with no dependence on back-end services and allows the availability of the local AD FS and AD FS Proxy services to be checked. Health checks for Web Application Proxy published services would still work as detailed in the earlier part of this blog.
These probe endpoints are available by default but on a Web Application Proxy server you will need to create an Inbound Windows Firewall rule to allow the port 80 traffic in to the probe e.g.:
AD FS 2012 R2 already has an Inbound Port 80 rule so you do not need to create a manual Windows Firewall rule to use the HTTP probe against AD FS.
While resolving issues with health-checking over SSL due to SNI or host headers issues is relatively straight-forward, it does require that the issues are understood and an accompanying level of information about the associated Hardware Load Balancer configuration.
We hope that providing an HTTP based health check probe option gives an alternate solution that has less complexity to ease your deployments.
Summary
Hardware Load Balancer health check requests and AD FS 2012 R2 and Web Application Proxy should be fairly straight-forward but we have seen some challenges in a couple of specific areas:-
- Does the Hardware Load Balancer support SNI?
- Ensuring that the correct Host Header is defined in the health check probe config
Understanding these and taking them into account during your deployment is important to avoid health check request failures. If you are unsure about whether your load balancer supports SNI or, how to configure your load balancer to meet the above requirements, please do reach out to your HLB vendor. We’ll try to share what we can here but the best people to advise on how to configure your HLB correctly is your HLB vendor.
If you come across any good info on configuring any of the Hardware Load Balancer variants out there or links to relevant vendor configuration steps please feel free to share and comment below.
We have also added new functionality in the August 2014 Windows Update rollup for Server 2012 R2 (http://support.microsoft.com/kb/2975719/en-us) to provide HTTP based health check probes to provide another health check option, which avoids some of the complexities such as SNI and host header configuration.
We’ll look forward to any feedback, comments and questions you have and we’ll be back with more content in the near future.
Ian Parramore, Senior Escalation Engineer, Web Application Proxy support team