High Availability with HAProxy: Ensuring Fault Tolerance and Redundancy in Your Infrastructure

9 min readJul 14, 2023

Introduction

It is essential to provide high availability, scalability, and dependability of online services in the complex and demanding digital world of today. In order to accomplish these objectives, load balancing is essential. For TCP and HTTP-based applications, HAProxy is an open-source software solution that offers high-performance load balancing, proxying, and SSL/TLS termination. An overview of HAProxy’s functionality, architecture, and many use cases is provided in this article.

What is HAProxy?

The acronym HAProxy means high availability proxy. It is a load balancer and reverse proxy that distributes incoming network traffic among many backend servers or services to enhance performance and offer fault tolerance. It is free, quick, and dependable. High throughput, low latency, and effective resource usage are attributes of HAProxy.

Key Features

HAProxy offers a wide range of features that make it a popular choice for load balancing and proxying needs. Some of its key features include:

Layer 4 and Layer 7 load balancing: HAProxy can operate at both the transport layer (Layer 4) and application layer (Layer 7), allowing it to make load balancing decisions based on IP addresses and ports (TCP mode) or content and URLs (HTTP mode).
SSL/TLS termination: The CPU-intensive work of SSL/TLS processing may be offloaded from backend servers to HAProxy, which can handle SSL/TLS encryption and decryption. By terminating SSL/TLS connections at the load balancer, HAProxy offloads SSL/TLS processing from backend servers. It is capable of managing SSL/TLS certificates, including certificate rotation, renewal, and validation. This feature enhances speed and makes setting up SSL/TLS for online apps simpler.
Health checking and failover: HAProxy continuously monitors the health of backend servers by performing health checks and automatically removes or adds servers based on their availability. This ensures that only healthy servers receive traffic.
Load balancing algorithms: HAProxy supports various load balancing algorithms, such as round-robin, least connections, source IP affinity, and more. These algorithms distribute incoming requests to backend servers in a balanced and optimized manner.
Logging and monitoring: HAProxy provides comprehensive logging capabilities, allowing administrators to analyze traffic patterns, troubleshoot issues, and generate performance reports. It also integrates with monitoring tools for real-time visibility into the system’s health and performance.

Load Balancing Algorithms

Round-robin: Requests are distributed evenly across backend servers in a circular manner.
Least connections: Requests are sent to the server with the fewest active connections.
Source IP affinity: Requests from the same client IP are consistently sent to the same backend server.
URI hashing: Requests are distributed based on the URI, ensuring that requests for the same resource are consistently sent to the same backend server.

Installations

Here I have used Ubuntu 20.04.6 LTS and haproxy version 2.0.31.

sudo apt update
sudo apt install haproxy -y

Backup the original haproxy conf and overwrite the haproxy.cfg

sudo mv /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak

We will change all the configurations in this config. Edit the haproxy.cfg and override the values as given below:

sudo vi /etc/haproxy/haproxy.cfg

global
    log         /dev/log local0
    log         /dev/log local1 notice
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     10000
    user        haproxy
    group       haproxy
    daemon
    stats socket /var/lib/haproxy/stats

    tune.ssl.default-dh-param 2048
    ssl-default-bind-ciphers HIGH:!aNULL:!MD5
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
    ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
    ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets


defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option                  http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    60s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 60s
    timeout check           10s
    maxconn                 10000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 429 /etc/haproxy/errors/429.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http


frontend fe-http
    bind *:80
    mode http
    acl cloudflare_ips src 173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32
    http-request deny if !cloudflare_ips
    http-request redirect scheme https code 301 if !{ ssl_fc }

frontend fe-https
    bind *:443 ssl crt /etc/ssl/all_certs ssl-min-ver TLSv1.2 ssl-max-ver TLSv1.3 alpn h2 force-tlsv12
    mode http

    # Allow Only CloudFlare IP
    acl cloudflare_ips src 173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32
    http-request deny if !cloudflare_ips

    # Hide Headers
    http-request del-header X-Powered-By
    http-request del-header Server
    option forwardfor
    http-request set-header X-Forwarded-For %[src]

    #### Security Policy and Headers ####
    http-response set-header Content-Security-Policy upgrade-insecure-requests
    http-response set-header Permissions-Policy geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()
    http-response del-header ^Server:.*$
    http-response del-header ^X-Powered-By:.*$
    http-response del-header ^X-AspNet-Version:.*$
    http-response add-header X-Frame-Options SAMEORIGIN # Or DENY
    http-response add-header Strict-Transport-Security max-age=31536000;
    http-response add-header X-XSS-Protection 1;mode=block
    http-response add-header X-Content-Type-Options nosniff
    http-response add-header Referrer-Policy same-origin
    option forwardfor
    option http-server-close

    # Letsencrypt Path
    acl letsencrypt-acl path_beg /.well-known/acme-challenge/
    use_backend letsencrypt-backend if letsencrypt-acl


    # Haproxy for application
    acl example_api hdr(host) -i api.example.com
    use_backend be-scaling if example_api

    acl example_api_new hdr(host) -i apinew.example.com
    use_backend be-scaling if example_api_new

    default_backend be-scaling


backend letsencrypt-backend
    server letsencrypt 127.0.0.1:8888


backend be-scaling
    mode http
    balance roundrobin
    stats enable
    stats refresh 5s
    stats hide-version
    stats scope .
    stats uri     /admin?stats
    stats realm   Haproxy\ Statistics
    stats auth    admin:password
    server server1 192.168.10.100:8080
    server server2 192.168.10.101:8080

Explanations

Let’s go through the configuration options and find out each configs in details:

Global

log: Specifies the logging configuration. HAProxy logs will be sent to /dev/log with the facility local0 and to /dev/log with the facility local1 for notices.
chroot: Sets the chroot directory for the HAProxy process. It restricts the process to a specific directory for security purposes.
pidfile: Specifies the file path where the process ID (PID) of HAProxy will be stored.
maxconn: Sets the maximum number of concurrent connections allowed by HAProxy.
user and group: Sets the user and group under which HAProxy will run.
daemon: Instructs HAProxy to run in the background as a daemon.
stats socket: Defines the Unix socket path for accessing HAProxy statistics.
tune.ssl.default-dh-param: Specifies the default Diffie-Hellman parameters size for SSL/TLS.
ssl-default-bind-ciphers: Sets the allowed ciphers for SSL/TLS connections between clients and HAProxy.
ssl-default-bind-options: Configures SSL/TLS options for the bind-side (client-side) of the connection.
ssl-default-server-ciphers: Sets the allowed ciphers for SSL/TLS connections between HAProxy and backend servers.
ssl-default-server-options: Configures SSL/TLS options for the server-side of the connection.

Default

This keyword marks the beginning of the default configuration section.

mode http: Sets the operating mode of HAProxy to HTTP, indicating that it will be used for HTTP traffic.
log global: Specifies that logging should be enabled globally for all frontend and backend sections.
option httplog: Enables detailed HTTP request logging.
option dontlognull: Prevents logging of requests with no content.
option http-server-close: Closes the server-side connection after the response, allowing HAProxy to free up resources faster.
option forwardfor except 127.0.0.0/8: Adds an X-Forwarded-For header to forwarded requests, indicating the original client IP address. However, it excludes requests originating from the local loopback network.
option redispatch: Allows redistributing requests to other servers if the originally selected server becomes unavailable.
retries 3: Sets the number of connection retries to 3 before considering a server as unavailable.
timeout http-request 60s: Defines the maximum time allowed for an HTTP request to be received.
timeout queue 1m: Specifies the maximum time a request can spend in the queue waiting for a server connection.
timeout connect 10s: Defines the maximum time allowed for establishing a connection to a server.
timeout client 1m: Sets the maximum inactivity time on the client side.
timeout server 1m: Sets the maximum inactivity time on the server side.
timeout http-keep-alive 60s: Specifies the maximum time allowed for an HTTP keep-alive session.
timeout check 10s: Sets the maximum time for a health check to complete.
errorfile code file: Associates HTTP error codes with custom error pages stored at the specified file locations. For example, 400 errors will use the file at /etc/haproxy/errors/400.http.

frontend fe-http

Begins the definition of the frontend section with the name “fe-http”.

bind *:80: Binds the frontend to listen on all IP addresses (*) on port 80. This means it will accept incoming traffic on port 80 from any available network interface.
acl cloudflare_ips src IPs: Defines an access control list (ACL) named “cloudflare_ips” that allows requests originating from the specified IP ranges. These IP ranges correspond to Cloudflare’s network.
http-request deny if !cloudflare_ips: Denies (rejects) any HTTP request that does not match the “cloudflare_ips” ACL. This ensures that only requests coming from Cloudflare’s IP ranges are allowed.
http-request redirect scheme https code 301 if !{ ssl_fc }: Redirects any HTTP request to HTTPS by sending a 301 redirect response. This line checks if the request does not have the ssl_fc (frontend connection is SSL/TLS) flag set. If the flag is not set, indicating an HTTP request, the redirect is triggered.

frontend fe-https

Begins the definition of the frontend section with the name “fe-https”.

bind *:443 ssl crt /etc/ssl/all_certs: Binds the frontend to listen on all IP addresses (*) on port 443 for HTTPS traffic. The SSL/TLS certificates used for encryption are specified in the directory /etc/ssl/all_certs.
ssl-min-ver TLSv1.2 ssl-max-ver TLSv1.3: Sets the minimum and maximum supported SSL/TLS protocol versions to TLSv1.2 and TLSv1.3, respectively.
alpn h2: Enables HTTP/2 (h2) as an application layer protocol negotiation option for ALPN (Application-Layer Protocol Negotiation).
force-tlsv12: Forces the use of TLSv1.2 for SSL/TLS connections.
Https Headers
http-request del-header X-Powered-By: Removes the “X-Powered-By” header from the HTTP request.
http-request del-header Server: Removes the “Server” header from the HTTP request.
option forwardfor: Adds the client’s IP address to the “X-Forwarded-For” header in the HTTP request.
http-request set-header X-Forwarded-For %[src]: Sets the “X-Forwarded-For” header in the HTTP request to the source IP address of the client.
http-response set-header Content-Security-Policy upgrade-insecure-requests: Sets the Content-Security-Policy header to upgrade insecure (HTTP) requests to secure (HTTPS) requests.
http-response set-header Permissions-Policy: Sets the Permissions-Policy header with specific permissions for various APIs.
http-response del-header ^Server:.*$, http-response del-header ^X-Powered-By:.*$, http-response del-header ^X-AspNet-Version:.*$: Deletes the “Server,” “X-Powered-By,” and “X-AspNet-Version” headers from the HTTP response.
http-response add-header X-Frame-Options SAMEORIGIN: Adds the X-Frame-Options header to the HTTP response with the SAMEORIGIN value, allowing embedding the page in frames from the same origin.
http-response add-header Strict-Transport-Security max-age=31536000: Adds the Strict-Transport-Security header to the HTTP response, enforcing secure (HTTPS) connections for the specified duration.
http-response add-header X-XSS-Protection 1;mode=block: Adds the X-XSS-Protection header to enable XSS (Cross-Site Scripting) protection in the browser.
http-response add-header X-Content-Type-Options nosniff: Adds the X-Content-Type-Options header to prevent MIME type sniffing.
http-response add-header Referrer-Policy same-origin: Adds the Referrer-Policy header to limit the referrer information to the same origin.
option forwardfor, option http-server-close: Specifies HAProxy options related to forwarding and closing HTTP connections.
acl letsencrypt: These lines define an ACL named “letsencrypt-acl” that matches requests with a path beginning with /.well-known/acme-challenge/. If a request matches this ACL, it uses the backend named “letsencrypt-backend”.
acl example_api and example_api_new: These lines define ACLs for matching the host header of the HTTP request. If a request matches one of these ACLs, it uses the backend named “be-scaling-new”. The ACLs here are checking for specific host headers.

Backend

Backend Letsencrypt:

This section defines a backend named “letsencrypt-backend” that specifies the server letsencrypt running on 127.0.0.1:8888. This backend is used when requests match the letsencrypt-acl ACL.

Backend Scaling

Begins the definition of the backend section with the name “be-scaling”

balance roundrobin: Specifies the load-balancing algorithm as round-robin, meaning that requests will be evenly distributed among the available servers.
stats enable: Enables statistics reporting for this backend.
stats refresh 5s: Sets the refresh interval for updating the statistics to 5 seconds.
stats hide-version: Hides the HAProxy version from the statistics page.
stats scope: Specifies that the statistics should cover all servers in this backend.
stats uri /admin?stats: Defines the URI path for accessing the statistics page.
stats realm Haproxy\ Statistics: Sets the authentication realm for accessing the statistics page.
stats auth admin:password: Specifies the username and password required for authentication when accessing the statistics page. In this case, the username is “admin” and the password is “password”.
server: This line defines a server entry within the backend. It specifies that requests should be forwarded to the server located at the IP address 192.168.10.100 on port 8080 and 192.168.10.101 on port 8080 on a round robin algorithm.

Conclusion

High availability, scalability, and performance optimization are all features that HAProxy’s strong and adaptable load balancing and proxying technology offers for web applications. It is a well-liked option for controlling traffic in contemporary infrastructure installations due to its broad feature set, sturdy design, and broad community support. HAProxy provides the tools and capabilities to satisfy even the most demanding requirements, whether it is load balancing, SSL/TLS termination, or advanced traffic control.

References

HAProxy Documentation: https://www.haproxy.org/documentation/
HAProxy Configuration Manual: https://cbonte.github.io/haproxy-dconv/2.2/configuration.html
HAProxy Technologies: https://www.haproxy.com/