We first compared the price and performance of NGINX Plus vs. F5 BIG‑IP back in 2016. Our testing revealed that NGINX Plus on commodity hardware exceeded the performance of F5® BIG‑IP® while providing cost savings of up to 84%.
Since we published that report, F5 Networks has refreshed its BIG‑IP line of hardware load balancers, introducing the new BIG‑IP iSeries. The iSeries hardware promises better price and performance compared to previous BIG‑IP hardware models, along with additional upgrades, such as onboard field‑programmable gate arrays (FPGAs).
At the same time, the price‑performance of commodity servers has also improved: new Intel® Xeon® processors are faster than their predecessors, and prices have come down significantly as well. So, are the improvements in the new BIG‑IP iSeries enough to keep up with the combination of commodity hardware and open source‑based software?
In this blog, we’ll again compare three simple, unambiguous performance metrics:
- HTTP requests per second (RPS)
- SSL/TLS transactions per second (TPS)
- HTTP throughput reported as gigabits per second (Gbps)
The F5 metrics are from the company’s published datasheet, with pricing from CDW and Carahsoft (click to download).
The NGINX Plus metrics are from our Sizing Guide, and we based hardware pricing on the list prices of Dell PowerEdge servers with the same specs as the Intel hardware that achieved the indicated results in our tests.
The bottom line: the NGINX Plus price‑performance advantage continues, with cost savings for NGINX Plus over F5 ranging from 78% to 87%. At the same time, the flexibility advantages of software over F5 and other hardware ADCs, as described in our recent blog post on agile development, continue to grow ever more important.
Let’s review the findings in detail.
NGINX Plus vs. F5 BIG-IP i2600
The table compares F5’s entry‑level ADC, the F5 BIG‑IP i2600, with NGINX Plus running on the Dell PowerEdge R330 with an 8‑core Intel Xeon 4110 @ 2.1GHz CPU and an Intel XL710 2×40 Gbe network interface card (NIC).
| F5 BIG‑IP i2600 | NGINX Plus (Dell R330) | Cost |
|---|---|---|
| One-time hardware cost | $19,175 | $2,200 |
| Annual 8×5 support and software subscription costs | $2,300 | $2,500 |
| Total Cost (Year 1) | $21,475 | $4,700 (78% savings) |
| Total Cost (Year 3) | $26,075 | $9,700 (63% savings) |
| Total Cost (Year 5) | $30,675 | $11,700 (59% savings) |
| Performance Metrics | ||
| HTTP RPS | 350,000 | 350,000 |
| SSL/TLS TPS | 2,500 | 14,0001 (5.6x) |
| Throughput (Gbps) | 10 | 40 (4x) |
1 Using OpenSSL 1.0.2d
The F5 configuration includes a single 10GbE NIC. The NGINX Plus/Dell PowerEdge configuration is specified with 1x40GbE NIC, which may be replaced with less expensive 10GbE NICs if desired.
NGINX Plus vs. F5 BIG-IP i5600
The table compares a mid‑range BIG‑IP appliance, the F5 BIG‑IP i5600, with NGINX Plus running on a similarly sized bare‑metal server, the Dell PowerEdge R630, with dual 22‑core Intel Xeon E5‑2699 v4 @ 2.2GHz CPUs and dual Intel XL710 2×40 Gbe NICs.
| F5 BIG‑IP i5600 | NGINX Plus (Dell R630) | |
| Cost | ||
|---|---|---|
| One-time hardware cost | $53,000 | $10,000 |
| Annual 24×7 support and software subscription costs | $9,540 | $3,500 |
| Total Cost (Year 1) | $62,540 |
$13,500 (78% Savings) |
| Total Cost (Year 3) | $81,620 | $20,500 (75% savings) |
| Total Cost (Year 5) | $100,700 | $27,500 (73% savings) |
| Performance Metrics | ||
| HTTP RPS | 1.1M | 1.2M (1.1x) |
| SSL/TLS TPS | 20,000 | 61,000 (3.1x)1 |
| Throughput (Gbps) | 60 | 70 (1.2x) |
1 Using OpenSSL 1.0.2d
NGINX Plus provides more than three times the SSL/TLS TPS compared to the equivalent F5 BIG‑IP.
Scaling Horizontally for High Availability
Whether you select F5 hardware or NGINX Plus running on an x86 server, you most likely want to run a pair of devices for high availability. This doubles the cost of your deployment, making the absolute price difference even greater.
Furthermore, you need a recovery plan if a hardware appliance or server fails and you need to restore your previous level of high availability. Because the NGINX Plus software is portable, the cost of maintaining spare hardware is significantly lower than with proprietary hardware. In the unlikely event of total hardware failure, NGINX Plus can be redeployed promptly on any other available x86 server, on bare metal, in a virtual machine, or in a container instance.
On the other hand, F5 cold spares are an additional sunk cost. The alternative of relying on a warranty turnaround for hardware failure may leave you exposed for several days without the protection of high availability.
NGINX Plus, leveraging general‑purpose hardware, provides a high‑availability solution that is significantly more cost‑effective and offers much faster recovery in the event of hardware failure.
Scaling Horizontally for Additional Performance
A single NGINX Plus instance running on a high‑end x86 server can achieve 1.2M RPS, 61K SSL/TLS TPS, and 70 Gbps of throughput. If you wish to specify a cluster that can handle more than this level of traffic, you can deploy NGINX Plus in a multiply active, multiply redundant fashion. This is commonly referred to as an N+1 deployment, where there are N active instances and one redundant system.
NGINX Plus in an N+1 deployment is much more cost‑effective than running a highly available pair (1+1) of high‑end F5 devices. For example, to achieve 2M RPS, the costs compare as follows, using the Dell PowerEdge R630 with dual 22‑core Intel Xeon E5‑2699 v4 @ 2.2GHz CPUs and dual Intel XL710 2×40 Gbe NICs:
| F5 BIG‑IP i11600 1+1 appliances (1 active device) |
NGINX Plus (Dell R630) 2+1 servers (2 active devices) |
|
| Cost | ||
|---|---|---|
| One-time hardware cost | $264,000 ($132,000 each) |
$30,000 ($10,000 each) |
| Annual 24×7 support and software subscription costs | $47,520 ($23,760 each) |
$10,500 ($3,500 each) |
| Total Cost (Year 1) | $311,520 | $40,500 (87% Savings) |
| Total Cost (Year 3) | $226,492 | $61,500 (85% savings) |
| Total Cost (Year 5) | $501,600 | $82,500 (84% savings) |
| Performance Metrics | ||
| HTTP RPS | 2.5M | 2.4M |
| SSL/TLS TPS | 37,000 | 122,0001 |
| Throughput (Gbps) | 80 | 140 |
1 Using OpenSSL 1.0.2d
NGINX Plus Flexibility
There are numerous “soft” benefits to NGINX Plus as well. Here are a few of the most important ones:
- Complementary solution. It’s easy to use NGINX Plus alongside existing F5 boxes. You can mix and match as needed. Of course, with the kind of cost savings described here, most customers choose to gradually decrease their dependence on F5 and similar hardware ADCs, eventually eliminating them entirely.
- Removing a layer. When you use NGINX Plus instead of F5, you are adding more of something that has multiple uses (NGINX Plus) and removing something that’s single‑purpose (F5 and other hardware ADCs). Over time, this can greatly simplify your stack – first for a given application, and then company‑wide.
- Operational effectiveness. Our customers who have moved completely to NGINX Plus tell us that their developers and operations people would rebel immediately if told to go back to F5. The ease, simplicity, fungibility, and flexibility of NGINX Plus make their daily work that much faster, easier, and more efficient.
- Cloud‑readiness. F5 was designed for private data centers and has not effectively made the move to the cloud, meaning that an F5 shop that wants to leverage the cloud – that is, most of them – needs a completely different load balancing paradigm, and different people, depending on where data and code lives.
- Who you hire. Knowledge about NGINX Open Source and NGINX Plus is widespread throughout the application delivery world and growing, a standard part of the toolkit in most relevant roles. F5 knowledge is specialized to a smaller number of, well, specialists, and is stagnant. We’re told that it’s easier to find and (due to the frustration level involved in F5) keep people in an NGINX Plus shop.
Increasingly, F5 BIG‑IP and other hardware ADCs stick out like a sore thumb in an otherwise fungible, flexible, and fast‑moving application delivery and development environment.
Conclusion
F5 hardware has served the IT industry well for more than 20 years. But the industry has moved away from hardware appliances and towards cloud‑native software solutions, leaving F5 BIG‑IP as a legacy component in the data center. Whereas the F5 BIG‑IP was once a solution that helped reduce costs for organizations, it is now cost‑prohibitive.
Our own performance measurements and pricing analysis support this. For the simple use cases we examined, we saw cost savings in Year 1 ranging from 78% to 88% when comparing F5 BIG‑IP to NGINX Plus.
Our customers report that they see similar cost savings when switching from hardware appliances to equivalent NGINX Plus solutions. They also enjoy related advantages, such as greater flexibility and easier manageability.
The price‑performance advantages of an NGINX Plus‑based solution, while immense, are only one reason to switch from hardware to software. Manageability, flexibility, the need for cloud solutions, the availability of trained personnel, and reducing the risk of using technology that may be nearing or reaching end of life are other important reasons for moving to software.
With NGINX Plus you are free to select the most cost‑effective hardware for your needs. We don’t force you to accept hardware that doesn’t meet your company’s internal standards, nor are you obliged to overprovision hardware now in anticipation of growth in traffic or application complexity that may arise in two to three years.
And lastly, a special thank you to Intel for providing the servers we used to complete this testing.
Appendix
Testing Details
The data used to create this cost comparison was gathered from multiple sources:
- All NGINX Plus testing was done using three servers, with dual Intel Xeon CPUs (E5‑2699 v4, running @ 2.20GHz) in each. The servers were configured in a standard client → proxy → server topology.
- To get metrics for different numbers of CPU cores, we adjusted the number of CPU cores in use.
- Hardware specifications and performance metrics for BIG‑IP hardware are from the BIG‑IP datasheet provided by F5; we did not test F5 hardware ourselves.
The hardware used to benchmark NGINX Plus was loaned by Intel.
Performance Metrics
The following performance metrics are compared in this report:
- Requests/sec (RPS) – Measures the ability to process HTTP requests. In our tests for NGINX Plus, clients send requests over keepalive connections. NGINX Plus processes each request and forwards it to the web server over another keepalive connection.
- SSL/TLS transactions per second (TPS) – Measures the ability to process new SSL/TLS connections. In our tests for NGINX Plus, clients send a series of HTTPS requests, each on a new connection. NGINX Plus parses the requests and forwards them to the web server over an established keepalive connection. The web server sends back a 0‑byte response for each request.
- Throughput – Measures the throughput sustained when serving large files over HTTP.
Perfect Forward Secrecy
In accordance with current SSL/TLS best practices, we measured NGINX Plus’ SSL/TLS TPS using the ECDHE-RSA-AES256-GCM-SHA384 cipher suite, which uses Ephemeral Elliptic curve Diffie‑Hellman key exchange (ECDHE), AES, and SHA‑384. We also used an RSA 2048‑bit key for valid comparison with the performance figures on the F5 datasheets.
This cipher provides Perfect Forward Secrecy (PFS, also called simply forward secrecy), which ensures that encrypted traffic captured now can’t be decrypted at a later time, even if the private key is compromised. PFS is becoming a ‘must have’ in the current security climate. For example, starting with iOS9, Apple mandates App Transport Security (ATS), which among other things requires that “communication through higher-level APIs … be encrypted using TLS version 1.2 with forward secrecy”.
F5 does not reveal the cipher used in their datasheet performance tests, and previous F5 benchmarks have not used PFS, which adds a performance penalty.
Readers should bear in mind the challenge of comparing SSL/TLS performance when different ciphers offer a trade‑off between security and speed.