Quarkus vs. WildFly--Requests per Second
A MicroProfile application (https://github.com/AdamBien/quarkus_vs_war) was used to compare the requests / second behaviour of Quarkus to WildFly:
Both, WildFly and Quarkus were used with out-of-the-box configuration. The test was started two times and the second (better) result was:
WildFly
Running 10s test @ http://localhost:8080/q/resources/hello
10 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 5.96ms 12.27ms 184.41ms 91.79%
Req/Sec 6.49k 2.10k 23.34k 73.23%
646210 requests in 10.10s, 112.78MB read
Requests/sec: 63997.47
Transfer/sec: 11.17MB
Quarkus JVM
Running 10s test @ http://localhost:8080/resources/hello
10 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 6.30ms 18.71ms 265.39ms 93.18%
Req/Sec 7.93k 2.88k 27.08k 77.00%
788462 requests in 10.09s, 91.73MB read
Requests/sec: 78147.92
Transfer/sec: 9.09MB
Interestingly both runs of the native GraalVM version produced nearly identical results:
Quarkus Native
Running 10s test @ http://localhost:8080/resources/hello
10 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.85ms 3.81ms 128.14ms 90.41%
Req/Sec 4.84k 563.06 6.33k 64.90%
481609 requests in 10.01s, 56.03MB read
Requests/sec: 48105.39
Transfer/sec: 5.60MB
Native executable q-1.0-SNAPSHOT-runner
size: 28MB.
WRK was used for load generation and the reports above.
Hi Adam,
First of all, thanks for sharing this measurements.
As I see comparing the quarkus native numbers with the other tests, the average and standard deviation in latency is inferior, this shows more consistency, but due to that, fact and because of the native compilation, I was expecting that quarkus was able to achieve more request per second in native mode than in JVM mode.
Do you know if the JVM optimizations during runtime are more effective than fact of executing native program, or this is just because the graalvm requires more work to achieve greater performance?
Thank you kindly
Posted by Javier Toja on March 12, 2020 at 11:20 AM CET #
@Adam, Many thanks for sharing. Please can we format output from WRK to HTML. histograms and all same way we do on Vegeta. There is seems to be limited documentation on WRK. PLease which do you think one should go with?
Posted by Taysay on March 21, 2020 at 08:34 PM CET #