Skip to content

gonwan/webframework-benchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Benchmark for Web Frameworks

See Techempower. This repository contains homemade benchmarks for Http servers. Only a single text response is considered, since some servers do not have their builtin Json implementation. Basically, I am benchmarking event loops, with Http implementation.

All server applications are running in a 4C docker environment. After packaging, a server can be launched using docker compose:

$ docker compose -f docker-compose.yml up actix

wrk is used as the client. Http keep-alive is enabled in the benchmark.

# benchmark plaintext
$ wrk -c 1000 -t 10 -d 30s http://127.0.0.1:90xx/text

Dependencies

If you are using Ubuntu 24.04 or later, all packages can be installed via apt-get:

$ sudo apt-get install libssl-dev libevent-dev libevhtp-dev libjsoncpp-dev \
    libtrantor-dev libdrogon-dev libboost-dev wrk \
    golang openjdk-21-jdk maven dotnet-sdk-10.0 nodejs rust-all

Environment

  • Server: 4C docker
  • Client: 12C host machine, AMD 7840HS, LinuxMint 22.3 / Ubuntu 24.04.
  • All applications are tweaked to work with 4C.
  • All Java & .NET applications are properly warmed up before benchmark.
  • The userland proxy of docker is disabled to eliminate the networking bottleneck. In /etc/docker/daemon.json, set the value and restart docker.
{
    "userland-proxy": false
}

Benchmark Result

Server Language Version Throughput
nginx C 1.28.2 336k /s
libevhtp (libevent) C 1.2.18 / 2.1.12 254k /s
drogon C++ 1.8.7 320k /s
beast C++ 1.83 220k /s
gin Go 1.22 1.10.1 86k /s
gin (GOMAXPROCS=4) Go 1.22 1.10.1 194k /s
gofiber Go 1.22 2.52.12 98k /s
gofiber (GOMAXPROCS=4) Go 1.22 2.52.12 272k /s
hertz Go 1.22 0.10.4 138k /s
hertz (GOMAXPROCS=4) Go 1.22 0.10.4 268k /s
spring-webflux (netty) Java 21 3.5.10 / 4.1.130 170k /s
spring-webmvc (tomcat) Java 21 3.5.10 / 10.1.50 47k /s
spring-webmvc (tomcat with virtual thread) Java 21 3.5.10 / 10.1.50 68k /s
spring-webmvc-undertow Java 21 3.5.10 / 2.3.22 61k /s
aspnetcore .NET 10 10.0 125k /s
node Javascript 24.14 178k /s
bun Javascript 24.14 / 1.3.10 254k /s
actix Rust 1.75 4.12.1 334k /s
---
config:
  xyChart:
    showDataLabel: true
  themeVariables:
    xyChart:
      plotColorPalette: '#5294e2'
---
xychart-beta horizontal
title "Http Server Benchmark"
x-axis [nginx, libevhtp, drogon, beast, gin, gofiber, hertz, spring-flux, spring-mvc, spring-mvc-vt, spring-mvc-udt, aspnetcore, node, bun, actix]
y-axis "Throughput (k/s)" 0 --> 350
bar [336, 254, 320, 220, 194, 272, 268, 170, 47, 68, 61, 125, 178, 254, 334]
Loading

It is amazing that actix in rust wins the first price. It runs even faster than any C/C++ servers. It does gain a bit more QPS when enabling LTO optimization. But I choose to not have it. Since all C/C++ servers do not have it, and the optimization can introduce bugs in a production environment. nginx is here for reference.

Then, drogon is the second. It is also an application level framework. libevhtp is not actively maintained and somehow hard to use. beast from boost is just another choice in C++. In my previous benchmark, I can remember it did not scale well and got a much lower throughput. It got almost same throughput running in a 4C and 24C Linux machine. Not knowing why, maybe It was running on an outdated CPU and OS(CentOS 7).

In the Go world, all 3 frameworks give competitive throughput. Setting GOMAXPROCS makes large difference in performance. Starting 1.25, Go automatically detects and respects docker CPU limits to set GOMAXPROCS.

Java servers all have bad performance. spring-webmvc gives the worst performance by default. It does get better when switching to use undertow or the virtual thread feature introduced in Java 21, but no so much. There is almost no difference when running under Java 21 and Java 25 in this case, regarding JEP 491(Synchronize Virtual Threads without Pinning). spring-webflux gives the best performance. The drawback is that, it is hard to write and debug reactive code. The Java world seems to have chosen virtual thread, say stackful coroutine, as their future of non-blocking programming. Even Spring has deprecated their reactor-kafka component. By the way, Java detects the docker environment with limited CPU resource. It not necessary to set IO or worker count manually for these servers.

Regarding ASP .NET Core, it beats only Java servers.

Regarding Javascript, both vanilla node and bun servers run smooth. They both have higher throughput than Java and .NET servers.

The benchmark only evolves event loop with Http implementation, without any business logics. The QPS can decrease dramatically if cross-service calls or database operations are added in a real-world application. But the benchmark does expose difference among languages and their potentials.

Maybe it time to say goodbye to Java and .NET in 2026. I almost have no knowledge about rust, so I think Go frameworks can be the first choice for application-level servers, as they are easier to get started.

Releases

No releases published

Packages

 
 
 

Contributors