Setting Load Balancer HAProxy on OPNsense – GNS3 LAB59

Setting Load Balancer HAProxy pada OPNsense – GNS3 LAB59

  • OPNsense WAN: 220.10.10.10/24
  • OPNsense LAN: 192.168.1.1/24
  • OPNsense OPT1: 192.168.2.1/24
  • Virtual IPs: 220.10.10.11/32, 220.10.10.12/32
  • OPNsense Gateway: 220.10.10.1
  • MikroTik WAN interface: 220.20.20.20/24
  • MikroTik LAN interface: 192.168.88.1/24
  • MikroTik Gateway: 220.20.20.1
  • Server-A: 192.168.1.100/24 – Frontend-server-AB (220.10.10.11)
  • Server-B: 192.168.1.200/24 – Frontend-server-AB (220.10.10.11)
  • Server-C: 192.168.2.100/24 – Frontend-server-CD (220.10.10.12)
  • Server-D: 192.168.2.200/24 – Frontend-server-CD (220.10.10.12)
  • User-X: 192.168.88.20

  • Setting Virtual IPs
  • Interfaces > Virtual IPs > Settings > Add (+)
  • Mode: IP Alias > Interface: WAN > Type: Single Address > Address: 220.10.10.11/32 > ceklis: Allow service binding > Gateway: 220.10.10.1 > Save
  • Tambahkan Certificate Authority
  • System > Trust > Authorities > Add (+)
  • Description name: OPNsense-CA > Method: Create an Internal Certificate Authority
  • Key type: RSA > Key lenght: 2048 > Digest algorithm: SHA1 > Lifetime (days): 1000
  • Country code: ID (Indonesia) > State: DKI Jakarta > City: Jakarta Pusat > Organization: SYS-OPS.ID > Email: [email protected] > Common Name: opnsense-ca > Save
  • Tambahkan Certificate
  • System > Trust > Certificates > Add (+)
  • Server Certificate
  • Method: Create an Internal Certificate > Description name: HAproxy-cert
  • Certificate Authority: OPNsense-CA > Type: Server Certificate > key type: RSA > Key lenght: 2048 > Digest algorithm: SHA1 > Lifetime (days): 1000 > Private key location: Save on this Firewall
  • Common Name: haproxy> Save
  • – – –
  • Client Certificate
  • Method: Create an Internal Certificate > Description name: Client-cert
  • Certificate Authority: OPNsense-CA > Type: Client Certificate > key type: RSA > Key lenght: 2048 > Digest algorithm: SHA1 > Lifetime (days): 1000 > Private key location: Save on this Firewall
  • Common Name: client> Save

  • Install plugin HAProxy
  • System > Firmware > Plugins > os-haproxy > Install
  • Setting HAProxy
  • Services > HAProxy > Settings

  • Tambahkan Health check untuk monitoring web server
  • Tab Rules & Checks > Health Monitors > Add (+)
  • Name: Health Check > Check type: HTTP [default] > SSL preferences: Use server settings > Check interval: 3s
  • HTTP method: OPTIONS [default] > Request URI: / > HTTP version: HTTP/1.0 [default] > HTTP host: localhost > Save > Apply
  • Tambahkan user untuk akses authentikasi
  • Tab User management > Users > Add (+)
  • Masukan Name: admin dan Password: admin > Save > Apply
  • Aktifkan service HAproxy
  • Tab Settings > Service > Enable HAproxy: checklis > Apply
  • Setting Global Parameters HAProxy
  • Tab Settings > Global Parameters
  • HAProxy threads: 10 > Verify SSL server Certificate: no preference [default] > Maximum SSL DH Size: 2048 > Spread checks: 2 > Enable SSL default setting: checklis > Minimum SSL Version: TLSv1.1 > Maximum SSL Version: TLSv1.3 > Apply
  • Setting Default Parameters HAProxy
  • Tab Settings > Default Parameters
  • Maximum Connection (Public Services): 5000 > Maximum Connection (Servers): 5000 > Client Timeout: 30s > Connection Timeout: 30s > Server Timeout: 30s > Retries: 5 > Apply
  • Settings Statistics HAProxy
  • Tab Settings > Statistiscs
  • advanced mode > Enable: checklis > Local stats TCP port: 8080 > Enable remote access: checklis > Remote listen addresses: 220.10.10.10:8080 > Enable authentication: checklis > Allow users: admin > Apply
  • Setting Cache HAProxy
  • Tab Settings > Cache
  • Cache enabled: checklis > Maximum Size of Cache (MB): 256 > Maximum Object Age (sec): 60 > Maximum Object Size (byte): 1000000 > Apply

  • Tambahkan Real Servers (server web)
  • Tab Real Servers > Add (+)
  • advanced mode > Enable: checklis > Name: Server-A > Type: static > FQDN or IP: 192.168.1.100
  • Port: 80 > Mode: active [default] > Multiplexer Protocol: auto-selection (recommended) > Prefer IP Family: prefer IPv4
  • Verify SSL Certificate: checklis > SSL Verify CA: OPNsense-CA > SSL Client Certificate: Client-cert > Max Connections: 2500 > Weight: 1 > Check interval: 3 > Port to check: 80 > Save > Apply
  • Tambahkan Backend Pools
  • Tab Virtual Services > Backend Pools > Add (+)
  • advanced mode > Enable: checklis > Name: Backend-server-AB > Mode: HTTP (Layer 7) [default] > Balancing Algorithm: Round Robin > Proxy Protocol: none > Servers: Server-A , Server-B > Fast CGI Application: none > Resolver: none > Prefer IP Family: prefer IPv4
  • Enable Health Checking: checklis > Health Monitor: Health Check > Log Status changes: checklis > Check Interval: 3s > Down Interval: 6s
  • Enable HTTP/2: checklis > Advertise Protocols (ALPN): HTTP/2 , HTTP/1.1
  • Persistence type: Stick-table persistence [default] > Table type: Source-IP [default] > Stored data types: Connection count, Current connections, Connection rate, Session count, Session rate, HTTP request count, HTTP request rate, HTTP error count, HTTP error rate > Expiration time: 30m > Size: 50k > Connection rate, Session rate, HTTP request rate, HTTP error rate: 10s
  • Connection Timeout: 60s > Server Timeout: 300s > Save > Apply
  • Tambahkan Public Services (Frontend HTTP & HTTPS)
  • Tab Virtual Services > Public Services > Add (+)
  • Frontend HTTP
  • advanced mode > Enable: checklis > Name: Frontend-server-HTTP-AB > Listen Addresses: 220.10.10.11:80 > Type: HTTP / HTTPS (SSL offloading) [default] > Default Backend Pool: Backend-server-AB
  • Enable HTTP/2: checklis > Advertise Protocol (ALPN): HTTP/2, HTTP/1.1 > X-Forwarded-For header: checklis > Maximum Connections: 5000 > HTTP Keep-Alive Timeout: 300s
  • Detailed Logging & Separated Statistics: checklis > Table Type: only store IPv4 addresses [default] > Stored data types: Current connections, HTTP error count, HTTP error rate > Expiration time: 30m > Size: 50k > Enable sticky counter: checklis > Type: http-keep-alive [default]> Save > Apply
  • – – –
  • Frontend HTTPS
  • advanced mode > Enable: checklis > Name: Frontend-server-HTTP-AB > Listen Addresses: 220.10.10.11:80 > Type: HTTP / HTTPS (SSL offloading) [default] > Default Backend Pool: Backend-server-AB
  • Enable SSL offloading: checklis > Certificates: HAproxy-cert > Default certificate: HAproxy-cert > Enable Advanced settings: checklis > Minimum SSL version: TLSv1.1 > Maximum SSL version: TLSv1.3 > Enable HSTS, HSTS includeSubDomains, HSTS preload: checklis > HSTS max-age: 15768000 > Bind options: prefer-client-ciphers
  • Enable HTTP/2: checklis > HTTP/2 without TLS: checklis > Advertise Protocol (ALPN): HTTP/2, HTTP/1.1 > X-Forwarded-For header: checklis > Maximum Connections: 5000 > HTTP Keep-Alive Timeout: 300s
  • Detailed Logging & Separated Statistics: checklis > Table Type: only store IPv4 addresses [default] > Stored data types: Current connections, HTTP error count, HTTP error rate > Expiration time: 30m > Size: 50k > Enable sticky counter: checklis > Type: http-keep-alive [default]> Save > Apply

  • Tambahkan Rule Firewall pada interface WAN untuk allow port 80 dan 443 pada virtual ip yang digunakan oleh frontend HAProxy
  • Firewall > Rules > WAN > Add (+)
  • Action: Pass > Interface: WAN > Direction: In > TCP/IP Version: IPv4 > Protocol: TCP/UDP > Source: any > Destination: Single host or network: 220.10.10.11/32 > Destination port range: From: HTTP to: HTTP (untuk port 80) sedangkan From: HTTPS to: HTTPS (untuk port 443) > Save

Pengujian

  • Akses Server-A dan Server-B dengan virtual ip: 220.10.10.11 > https://220.10.10.11
  • Akses Server-C dan Server-D dengan virtual ip: 220.10.10.12 > https://220.10.10.12
  • Cek status statistic HAProxy dengan url: http://220.10.10.10:8080/haproxy?stats
  • Cek status server: Services > HAProxy > Statistics > Status
  • Cek konfigruasi HAProxy
#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    10
    hard-stop-after             60s
    no strict-limits
    tune.ssl.default-dh-param   2048
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua
    ssl-default-bind-options prefer-client-ciphers ssl-min-ver TLSv1.1 ssl-max-ver TLSv1.3
    ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
cache opnsense-haproxy-cache
    total-max-size 256
    max-age 60
    max-object-size 1000000
    process-vary off

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 5
    default-server init-addr last,libc
    default-server maxconn 5000

# autogenerated entries for ACLs

# userlists generated from groups

# autogenerated entries for config in backends/frontends

# autogenerated entries for stats
userlist stats_auth
    user admin insecure-password admin
    # NOTE: UserlistAddUsers called with empty group data


# Frontend: Frontend-server-HTTP-AB (220.10.10.11:80)
frontend Frontend-server-HTTP-AB
    bind 220.10.10.11:80 name 220.10.10.11:80 
    mode http
    option http-keep-alive
    default_backend Backend-server-AB
    option forwardfor
    # tuning options
    maxconn 5000
    timeout client 30s
    timeout http-keep-alive 300s
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cur,http_err_cnt,http_err_rate(10s) 
    tcp-request connection track-sc0 src
    # logging options
    option httplog
    option socket-stats

# Frontend: Frontend-server-HTTPS-AB (220.10.10.11:443)
frontend Frontend-server-HTTPS-AB
    http-response set-header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
    bind 220.10.10.11:443 name 220.10.10.11:443 ssl prefer-client-ciphers ssl-min-ver TLSv1.1 ssl-max-ver TLSv1.3 ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/6376ca82384113.08874068.certlist 
    mode http
    option http-keep-alive
    default_backend Backend-server-AB
    option forwardfor
    # tuning options
    maxconn 5000
    timeout client 30s
    timeout http-keep-alive 300s
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cur,http_err_cnt,http_err_rate(10s) 
    tcp-request connection track-sc0 src
    # logging options
    option httplog
    option socket-stats

# Frontend: Frontend-server-HTTP-CD (220.10.10.12:80)
frontend Frontend-server-HTTP-CD
    bind 220.10.10.12:80 name 220.10.10.12:80 
    mode http
    option http-keep-alive
    default_backend Backend-server-CD
    option forwardfor
    # tuning options
    maxconn 5000
    timeout client 30s
    timeout http-keep-alive 300s
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cur,http_err_cnt,http_err_rate(10s) 
    tcp-request connection track-sc0 src
    # logging options
    option httplog
    option socket-stats

# Frontend: Frontend-server-HTTPS-CD (220.10.10.12:443)
frontend Frontend-server-HTTPS-CD
    http-response set-header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
    bind 220.10.10.12:443 name 220.10.10.12:443 ssl prefer-client-ciphers ssl-min-ver TLSv1.1 ssl-max-ver TLSv1.3 ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/6377443b2b42b9.02655761.certlist 
    mode http
    option http-keep-alive
    default_backend Backend-server-CD
    option forwardfor
    # tuning options
    maxconn 5000
    timeout client 30s
    timeout http-keep-alive 300s
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cur,http_err_cnt,http_err_rate(10s) 
    tcp-request connection track-sc0 src
    # logging options
    option httplog
    option socket-stats

# Backend: Backend-server-AB ()
backend Backend-server-AB
    option log-health-checks
    # health check: Health Check
    option httpchk
    http-check send meth OPTIONS uri / ver HTTP/1.0
    mode http
    balance roundrobin
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cnt,conn_cur,conn_rate(10s),sess_cnt,sess_rate(10s),http_req_cnt,http_req_rate(10s),http_err_cnt,http_err_rate(10s) 
    stick on src
    # tuning options
    timeout connect 60s
    timeout server 300s
    retries 5
    http-reuse safe
    server Server-A 192.168.1.100:80 check inter 3s downinter 6s port 80  maxconn 2500 weight 1 resolve-prefer ipv4
    server Server-B 192.168.1.200:80 check inter 3s downinter 6s port 80  maxconn 2500 weight 1 resolve-prefer ipv4

# Backend: Backend-server-CD ()
backend Backend-server-CD
    option log-health-checks
    # health check: Health Check
    option httpchk
    http-check send meth OPTIONS uri / ver HTTP/1.0
    mode http
    balance roundrobin
    # stickiness
    stick-table type ip size 50k expire 30m store conn_cnt,conn_cur,conn_rate(10s),sess_cnt,sess_rate(10s),http_req_cnt,http_req_rate(10s),http_err_cnt,http_err_rate(10s) 
    stick on src
    # tuning options
    timeout connect 60s
    timeout server 300s
    retries 5
    http-reuse safe
    server Server-C 192.168.2.100:80 check inter 3s downinter 6s port 80  maxconn 2500 weight 1 resolve-prefer ipv4
    server Server-D 192.168.2.200:80 check inter 3s downinter 6s port 80  maxconn 2500 weight 1 resolve-prefer ipv4


listen local_statistics
    bind            127.0.0.1:8080
    mode            http
    stats uri       /haproxy?stats
    stats realm     HAProxy\ statistics
    stats admin     if TRUE

listen  remote_statistics
    bind            220.10.10.10:8080
    mode            http
    stats uri       /haproxy?stats
    stats hide-version
    acl auth_ok http_auth(stats_auth)
    stats http-request allow if auth_ok
    stats http-request auth realm HAProxy\ statistics

herdiana3389

A system administrator with skills in system administration, virtualization, linux, windows, networking, cloud computing, container, etc.