今天测试了Nginx在Linux、FreeBSD、DragonflyBSD
三个操作系统中的性能差异

0x00 TL;DR

性能从高到低依次是: FreeBSD > DragonflyBSD > Debian

三个操作系统都安装于虚拟机中，硬件配置如下：

CPU: 2 sockets, 2 cores
Memory: 1GB
Network: virtio
Disk: VirtIO SCSI

0x01 操作系统的版本信息

  1. Debian 12 bookworm x86_64 gcc
  2. FreeBSD 14-RELEASE amd64 llvm
  3. DragonflyBSD 6.4-SYNTH x86_64 gcc

操作系统都是默认参数，Nginx 1.24.0在每个系统中手动编译
编译参数如下：

--with-stream --with-http_ssl_module

0x02 NGINX配置

本次测试主要是关注Nginx在不同操作系统中的性能差异
为了排除磁盘读写的影响，关闭了日志
HTTP响应也改成简单的return
得到的结果就是Nginx自身和操作系统内核带来的性能差异

BSD系统中Nginx参数

user  www;
worker_processes  auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 400000;
pcre_jit on;

events {
    worker_connections  20000;
    multi_accept on;
    use kqueue;
}

http {
    access_log off;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    connection_pool_size 1024;
    request_pool_size 8k;
    
    server_tokens off;

    server {
        listen       80;
        server_name  localhost;

        location / {
            default_type application/json;
            return 200 "{}";
            root   html;
        }
    }
}

Debian系统中Nginx参数

user  www-data;
worker_processes  auto;
worker_cpu_affinity auto;
worker_rlimit_nofile 400000;
pcre_jit on;

events {
    worker_connections  20000;
    multi_accept on;
    use epoll;
}

http {
    access_log off;

    sendfile        on;
    tcp_nopush      on;
    tcp_nodelay     on;
    connection_pool_size 1024;
    request_pool_size 8k;

    server_tokens off;

    server {
        listen       80;
        server_name  localhost;

        location / {
            default_type application/json;
            return 200 "{}";
            root   html;
        }
    }
}

0x03 测试命令

wrk -t 4 -c 4000 --latency http://host.local/

0x04 测试结果

每个操作系统测试三次，取三次中的最高值。

  1. Debian

    Running 10s test @ http://debian.local/
  4 threads and 4000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
 Latency   106.01ms  153.25ms   1.80s    89.23%
 Req/Sec    16.16k     3.45k   23.85k    67.25%
  Latency Distribution
  50%   32.76ms
  75%  143.90ms
  90%  279.70ms
  99%  707.57ms
  644063 requests in 10.07s, 100.73MB read
  Socket errors: connect 0, read 0, write 0, timeout 19
Requests/sec:  63936.47
Transfer/sec:     10.00MB

  2. FreeBSD

    Running 10s test @ http://freebsd.local/
  4 threads and 4000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
 Latency    97.54ms  154.30ms   1.95s    89.62%
 Req/Sec    20.84k     3.60k   29.08k    78.45%
  Latency Distribution
  50%   27.11ms
  75%  126.72ms
  90%  264.34ms
  99%  717.26ms
  829037 requests in 10.10s, 117.01MB read
  Socket errors: connect 0, read 0, write 0, timeout 18
Requests/sec:  82122.38
Transfer/sec:     11.59MB

  3. DragonflyBSD

    Running 10s test @ http://dragonflybsd.local/
  4 threads and 4000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
 Latency    35.05ms   96.83ms   1.98s    94.49%
 Req/Sec    17.81k     9.37k   38.46k    60.50%
  Latency Distribution
  50%   10.58ms
  75%   20.41ms
  90%   85.93ms
  99%  357.30ms
  708720 requests in 10.05s, 105.44MB read
  Socket errors: connect 0, read 13, write 0, timeout 1415
Requests/sec:  70510.98
Transfer/sec:     10.49MB

0x05 结果分析

以Debian为性能基准数(100%)，性能对比结果如下：

FreeBSD (128%) > DragonflyBSD (110%) > Debian (100%)

在测试过程中，
Linux和DragonflyBSD的CPU占用率都会上升到95%-100%之间，
FreeBSD始终维持在60-70%之间,
在CPU占用上FreeBSD有优势。

三个系统中Nginx进程内存占用情况：

  • Linux中一个Nginx Worker平均占用20M虚拟内存
  • FreeBSD中一个Nginx Worker平均占用39MB虚拟内存
  • DragonflyBSD中一个Nginx Worker平均占用41MB虚拟内存

Linux在内存占用上拥有很大的优势，而FreeBSD和DragonflyBSD基本持平。
另外需要提一下的是，当有大量并发访问时，
Linux系统的内存占用总量会有20MB左右的上下波动。
FreeBSD和DragonflyBSD在测试中则无变化。