Elasticsearch & NGINX – Better Together | NGINX

Monitoring NGINX Plus Statistics with ELK on our blog.

The Advantages of Deploying NGINX Plus with Elasticsearch

Elasticsearch has a few features designed for scalability, but offloading the load balancing of requests to the open source NGINX product or the commercial NGINX Plus product, which has even more enterprise‑grade features, frees up resources on the Elasticsearch servers.

Putting NGINX or NGINX Plus in front of a single Elasticsearch server provides benefits, such as request logging, but the real value comes when you scale to multiple Elasticsearch servers.

In addition to logging each API request, NGINX and NGINX Plus

• Support a very large number of client connections (with or without keepalives enabled) with a much smaller of number of persistent connections (using keepalives) to the Elasticsearch servers.
• Load balance requests across the Elasticsearch servers.
• Cache data locally to reduce repeated requests to the Elasticsearch servers for the same content. NGINX Plus further provides an HTTP API for purging objects from the cache.
• (NGINX Plus only) Provides application health checks to continually check whether the Elasticsearch servers are up and functioning. Down servers are automatically removed from the load balancing rotation and returned to it when they come back up.
• (NGINX Plus only) Reports a wealth of live activity metrics to improve monitoring and management. A demonstration of the metrics is available at demo.nginx.com.
• (NGINX Plus only) Supports dynamic reconfiguration of upstream server groups via an HTTP API. You can add and remove servers from the upstream group, mark them up or down, and change their weights.

The following table compares the functionality available with Elasticsearch by itself, combined with open source NGINX, and combined with NGINX Plus:

	Elasticsearch client	Open source NGINX	NGINX Plus
Horizontal scalability	✅	✅	✅
Keepalive optimization	❌	✅	✅
Centralized HTTP access logs	❌	✅	✅
Queuing and concurrency control	❌	❌	✅
Response caching	❌	✅	✅
Failover of failed connections	✅	✅	✅
Active monitoring of Elasticsearch nodes	✅ (some clients)	❌	✅
Advanced load balancing methods	❌	✅	✅
Weighted load balancing	❌	✅	✅
Dynamic reconfiguration	❌	❌	✅
Status monitoring	❌	❌	✅
General‑purpose HTTP load balancing	❌	✅	✅

When you use NGINX Plus in a high‑availability active/passive configuration to load balance a cluster of Elasticsearch servers, another benefit is that you can configure the Elasticsearch clients to direct requests to NGINX Plus, rather than to the Elasticsearch servers. This means you can scale the Elasticsearch server cluster as needed without having to update the clients.

Deploying NGINX Plus with Elasticsearch

Deploying NGINX Plus with Elasticsearch is very straightforward. The directives in the following sample NGINX Plus configuration define settings for load balancing two Elasticsearch servers:

• upstream – Load balance requests across two Elasticsearch servers with IP addresses 192.168.187.132 and 192.168.187.133 that listen on port 9200.
• proxy_cache_valid – Cache valid responses for 10 minutes and 404 responses for 1 minute.

• health_check – Actively check the health of the servers, taking them out of the load‑balancing rotation when they are down and adding them back into the rotation when are healthy again. The health check references the match block, which not only makes sure that the server returns a status code of 200, but checks that the response is formatted in JSON and that it returns the string expected from Elasticsearch:

  "status" : 200

• Final server block – Collect statistics and listen on port 8080 for requests either for the status.html page which displays the statistics, or for status, which tells NGINX Plus to return the raw data in JSON format.

# in the 'http' context
proxy_cache_path /var/cache/nginx/cache keys_zone=elasticsearch:10m inactive=60m;

upstream elasticsearch_servers {
    zone elasticsearch_servers 64K;
    server 192.168.187.132:9200;
    server 192.168.187.133:9200;
}

match statusok {
    status 200;
    header Content-Type ~ "application/json";
    body ~ '"status" : 200';
}

server {
    listen 9200;
    status_zone elasticsearch;
    location / {
        proxy_pass http://elasticsearch_servers;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_cache elasticsearch;
        proxy_cache_valid 200 302 10m;
        proxy_cache_valid 404 1m;
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
        health_check interval=5s fails=1 passes=1 uri=/ match=statusok;
    }

    # redirect server error pages to the static page /50x.html
    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root /usr/share/nginx/html;
    }
    access_log logs/es_access.log combined;
}

server {
    listen 8080;
    root /usr/share/nginx/html;

    location / {
        index status.html;
    }

    location =/status {
        status;
    }
}

Summary

Elasticsearch, a powerful and flexible search engine, and NGINX Plus, an enterprise‑grade application delivery platform, fit naturally together as components of a modern scalable web architecture. As software products, they provide the same features and functions wherever you install them: on bare‑metal servers, in virtual machines, in the cloud, or in containers.

To try out NGINX Plus with Elasticsearch for yourself, start your free 30‑day trial of NGINX Plus today or contact us for a live demo.

Further Reading

Playing HTTP Tricks with Nginx on the Elasticsearch blog

Securing Elasticsearch with Nginx by James McFadden

Monitoring NGINX Plus Statistics with ELK on our blog