Quarkus vs. WildFly--Requests per Second

A MicroProfile application (https://github.com/AdamBien/quarkus_vs_war) was used to compare the requests / second behaviour of Quarkus to WildFly:

Both, WildFly and Quarkus were used with out-of-the-box configuration. The test was started two times and the second (better) result was:

WildFly


Running 10s test @ http://localhost:8080/q/resources/hello
10 threads and 100 connections
Thread Stats   Avg      Stdev     Max   +/- Stdev
Latency     5.96ms   12.27ms 184.41ms   91.79%
Req/Sec     6.49k     2.10k   23.34k    73.23%
646210 requests in 10.10s, 112.78MB read
Requests/sec:  63997.47
Transfer/sec:     11.17MB

Quarkus JVM


Running 10s test @ http://localhost:8080/resources/hello
10 threads and 100 connections
Thread Stats   Avg      Stdev     Max   +/- Stdev
Latency     6.30ms   18.71ms 265.39ms   93.18%
Req/Sec     7.93k     2.88k   27.08k    77.00%
788462 requests in 10.09s, 91.73MB read
Requests/sec:  78147.92
Transfer/sec:      9.09MB

Interestingly both runs of the native GraalVM version produced nearly identical results:

Quarkus Native


Running 10s test @ http://localhost:8080/resources/hello
10 threads and 100 connections
Thread Stats   Avg      Stdev     Max   +/- Stdev
Latency     2.85ms    3.81ms 128.14ms   90.41%
Req/Sec     4.84k   563.06     6.33k    64.90%
481609 requests in 10.01s, 56.03MB read
Requests/sec:  48105.39
Transfer/sec:      5.60MB
Native executable q-1.0-SNAPSHOT-runner size: 28MB.

WRK was used for load generation and the reports above.

Comments:

Hi Adam,

First of all, thanks for sharing this measurements.

As I see comparing the quarkus native numbers with the other tests, the average and standard deviation in latency is inferior, this shows more consistency, but due to that, fact and because of the native compilation, I was expecting that quarkus was able to achieve more request per second in native mode than in JVM mode.

Do you know if the JVM optimizations during runtime are more effective than fact of executing native program, or this is just because the graalvm requires more work to achieve greater performance?

Thank you kindly

Posted by Javier Toja on March 12, 2020 at 11:20 AM CET #

@Adam, Many thanks for sharing. Please can we format output from WRK to HTML. histograms and all same way we do on Vegeta. There is seems to be limited documentation on WRK. PLease which do you think one should go with?

Posted by Taysay on March 21, 2020 at 08:34 PM CET #

Post a Comment:
  • HTML Syntax: NOT allowed
...the last 150 posts
...the last 10 comments
License