Last active
June 10, 2024 17:47
-
Star
(127)
You must be signed in to star a gist -
Fork
(16)
You must be signed in to fork a gist
-
-
Save omnibs/e5e72b31e6bd25caf39a to your computer and use it in GitHub Desktop.
Revisions
-
omnibs revised this gist
Jun 24, 2015 . 1 changed file with 6 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -210,7 +210,9 @@ Bonus Track ASP.Net 4.5.1 MVC 5 on IIS 8 -- I've benchmarked ASP.Net, since I do .Net at my day to day job. Rackspace doesn't offer an OnMetal instance with windows, only virtualized instances, so I've done the benchmarks on the most powerful one I could find. On a Rackspace `I/O v1 120GB` instance is listed as: ``` CPU: 32 vCPUs @@ -233,4 +235,6 @@ Running 30s test @ http://10.223.241.211/showdown 581748 requests in 30.02s, 1.39GB read Requests/sec: 19381.05 Transfer/sec: 47.37MB ``` Results were pretty horrid, but maybe it's the virtualization overhead, or maybe Rackspace's Windows virtualization is bad, or even the 10gbit network between OnMetal instances and virtualized instances is not that good. I might do more testing this weekend to try to figure that out. -
omnibs revised this gist
Jun 24, 2015 . No changes.There are no files selected for viewing
-
omnibs revised this gist
Jun 24, 2015 . 1 changed file with 6 additions and 6 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -8,15 +8,15 @@ Results | Framework | Throughput (req/s) | Latency (ms) | Consistency (σ ms) | |-------------------|-------------------:|-------------:|-------------------:| | Plug | 198328.21 | 0.63ms | 2.22ms | | Phoenix 0.13.1 | 179685.94 | 0.61ms | 1.04ms | | Gin | 176156.41 | 0.65ms | 0.57ms | | Play | 171236.03 | 1.89ms | 14.17ms | | Phoenix 0.9.0-dev | 169030.24 | 0.59ms | 0.30ms | | Express Cluster | 92064.94 | 1.24ms | 1.07ms | | Martini | 32077.24 | 3.35ms | 2.52ms | | Sinatra | 30561.95 | 3.50ms | 2.53ms | | Rails | 11903.48 | 8.50ms | 4.07ms | Environment == -
omnibs revised this gist
Jun 24, 2015 . 1 changed file with 54 additions and 13 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,11 +1,37 @@ Comparative Benchmark Numbers @ Rackspace == I've taken the benchmarks from [Matthew Rothenberg](https://github.com/mroth)'s [phoenix-showdown](https://github.com/mroth/phoenix-showdown), updated Phoenix to 0.13.1 and ran the tests on the most powerful machines available at Rackspace. Results -- | Framework | Throughput (req/s) | Latency (ms) | Consistency (σ ms) | |-------------------|-------------------:|-------------:|-------------------:| | Phoenix 0.9.0-dev | 169030.24 | 0.59ms | 0.30ms | | Phoenix 0.13.1 | 179685.94 | 0.61ms | 1.04ms | | Plug | 198328.21 | 0.63ms | 2.22ms | | Play | 171236.03 | 1.89ms | 14.17ms | | Rails | 11903.48 | 8.50ms | 4.07ms | | Sinatra | 30561.95 | 3.50ms | 2.53ms | | Gin | 176156.41 | 0.65ms | 0.57ms | | Martini | 32077.24 | 3.35ms | 2.52ms | | Express Cluster | 92064.94 | 1.24ms | 1.07ms | Environment == | Environment | Version | |-------------|-----------| | Erlang | 17.5 | | Elixir | 1.1.0-dev | | Ruby | 2.2.0 | | Node | 0.12.4 | | Go | 1.3.3 | | Java | 8u45 | Server -- ``` Rackspace OnMetal IO @@ -18,7 +44,8 @@ Disk I/O Good Price $2.4658 / hr ``` Client -- ``` Rackspace OnMetal Compute @@ -30,11 +57,15 @@ Disk I/O Good Price $0.7534 / hr ``` Detailed Results == PHOENIX 0.9.0-dev (original) -- ``` server:~# MIX_ENV=prod elixir -pa _build/prod/consolidated -S mix phoenix.server client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" Running 30s test @ http://10.184.11.239:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -48,7 +79,8 @@ PHOENIX 0.13.1 (current) -- ``` server:~# MIX_ENV=prod elixir -pa _build/prod/consolidated -S mix phoenix.server client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.13.193:4000/showdown" Running 30s test @ http://10.184.13.193:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -63,7 +95,8 @@ PLUG -- ``` server:~# MIX_ENV=prod elixir -pa _build/prod/consolidated -S mix server client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" Running 30s test @ http://10.184.11.239:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -78,7 +111,9 @@ PLAY -- ``` server:~# ./activator clean stage server:~# ./target/universal/stage/bin/play-scala -Dhttp.port=5000 client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:5000/showdown" Running 30s test @ http://10.184.11.239:5000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -93,7 +128,8 @@ RAILS -- ``` server:~# PUMA_WORKERS=20 MIN_THREADS=1 MAX_THREADS=80 RACK_ENV=production bundle exec puma client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -108,7 +144,8 @@ SINATRA -- ``` server:~# RACK_ENV=production bundle exec puma -t 1:16 -w 20 --preload client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:9292/showdown" Running 30s test @ http://10.184.11.239:9292/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -123,7 +160,8 @@ GIN -- ``` server:~# GOMAXPROCS=20 GIN_MODE=release go run server.go client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -138,7 +176,8 @@ MARTINI -- ``` server:~# GOMAXPROCS=20 MARTINI_ENV=production go run server.go client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev @@ -153,6 +192,8 @@ EXPRESS (20-node cluster) -- ``` server:~# NODE_ENV=production node server.js -w 20 client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev -
omnibs revised this gist
Jun 20, 2015 . 1 changed file with 31 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -161,4 +161,35 @@ Running 30s test @ http://10.184.11.239:3000/showdown 2771185 requests in 30.10s, 5.65GB read Requests/sec: 92064.94 Transfer/sec: 192.37MB ``` Bonus Track == ASP.Net 4.5.1 MVC 5 on IIS 8 -- On a Rackspace `I/O v1 120GB` instance listed as: ``` CPU: 32 vCPUs RAM: 120 GB System Disk: 40 GB Data Disk: 1.2 TB (4 disks) Network: 10 Gb / s Disk I/O: Best ``` Results: ``` root@client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.223.241.211/showdown" Running 30s test @ http://10.223.241.211/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 62.98ms 104.13ms 962.77ms 81.84% Req/Sec 1.02k 435.75 2.92k 70.76% 581748 requests in 30.02s, 1.39GB read Requests/sec: 19381.05 Transfer/sec: 47.37MB ``` -
omnibs revised this gist
Jun 19, 2015 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -149,7 +149,7 @@ Requests/sec: 32077.24 Transfer/sec: 63.84MB ``` EXPRESS (20-node cluster) -- ``` -
omnibs revised this gist
Jun 19, 2015 . 1 changed file with 16 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -21,6 +21,7 @@ Price $2.4658 / hr Client: ``` Rackspace OnMetal Compute CPU 2.8 Ghz, 10 core Intel® Xeon® E5-2680 v2 RAM 32 GB System Disk 32 GB @@ -29,7 +30,7 @@ Disk I/O Good Price $0.7534 / hr ``` PHOENIX 0.9.0-dev (original) -- ``` @@ -43,6 +44,20 @@ Running 30s test @ http://10.184.11.239:4000/showdown Requests/sec: 169030.24 Transfer/sec: 354.47MB ``` PHOENIX 0.13.1 (current) -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.13.193:4000/showdown" Running 30s test @ http://10.184.13.193:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 611.61us 1.04ms 95.57ms 99.05% Req/Sec 9.04k 656.30 10.98k 71.84% 5408349 requests in 30.10s, 11.08GB read Requests/sec: 179685.94 Transfer/sec: 376.82MB ``` PLUG -- -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 4 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,7 @@ Phoenix-Showdown == Benches https://github.com/mroth/phoenix-showdown Erl 17.5 | Elixir 1.1.0-dev -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 8 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -27,6 +27,7 @@ Price $0.7534 / hr ``` PHOENIX -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" @@ -41,6 +42,7 @@ Transfer/sec: 354.47MB ``` PLUG -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" @@ -55,6 +57,7 @@ Transfer/sec: 408.34MB ``` PLAY -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:5000/showdown" @@ -69,6 +72,7 @@ Transfer/sec: 334.12MB ``` RAILS -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" @@ -83,6 +87,7 @@ Transfer/sec: 26.28MB ``` SINATRA -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:9292/showdown" @@ -97,6 +102,7 @@ Transfer/sec: 63.04MB ``` GIN -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" @@ -111,6 +117,7 @@ Transfer/sec: 350.61MB ``` MARTINI -- ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" @@ -125,6 +132,7 @@ Transfer/sec: 63.84MB ``` EXPRESS -- ``` Running 30s test @ http://10.184.11.239:3000/showdown -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 21 additions and 8 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -4,14 +4,27 @@ Erl 17.5 | Elixir 1.1.0-dev Server: ``` Rackspace OnMetal IO CPU Dual 2.8 Ghz, 10 core Intel® Xeon® E5-2680 v2 RAM 128 GB System Disk 32 GB Data Disk Dual 1.6 TB PCIe flash cards Network Redundant 10 Gb / s connections in a high availability bond Disk I/O Good Price $2.4658 / hr ``` Client: ``` CPU 2.8 Ghz, 10 core Intel® Xeon® E5-2680 v2 RAM 32 GB System Disk 32 GB Network Redundant 10 Gb / s connections in a high availability bond Disk I/O Good Price $0.7534 / hr ``` PHOENIX -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 1 addition and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,6 +5,7 @@ Erl 17.5 | Elixir 1.1.0-dev Server: |CPU|Dual 2.8 Ghz, 10 core Intel® Xeon® E5-2680 v2| |-|-| |RAM| 128 GB| |System Disk| 32 GB| |Data Disk| Dual 1.6 TB PCIe flash cards| -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 10 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -2,6 +2,16 @@ Benches from https://github.com/mroth/phoenix-showdown Erl 17.5 | Elixir 1.1.0-dev Server: |CPU|Dual 2.8 Ghz, 10 core Intel® Xeon® E5-2680 v2| |RAM| 128 GB| |System Disk| 32 GB| |Data Disk| Dual 1.6 TB PCIe flash cards| |Network| Redundant 10 Gb / s connections in a high availability bond| |Disk I/O| Good| |Price| $2.4658 / hr| PHOENIX ``` -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,3 +1,7 @@ Benches from https://github.com/mroth/phoenix-showdown Erl 17.5 | Elixir 1.1.0-dev PHOENIX ``` -
omnibs revised this gist
Jun 6, 2015 . 1 changed file with 13 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -94,4 +94,17 @@ Running 30s test @ http://10.184.11.239:3000/showdown 962648 requests in 30.01s, 1.87GB read Requests/sec: 32077.24 Transfer/sec: 63.84MB ``` EXPRESS ``` Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.24ms 1.07ms 26.92ms 83.39% Req/Sec 4.63k 2.37k 13.34k 74.60% 2771185 requests in 30.10s, 5.65GB read Requests/sec: 92064.94 Transfer/sec: 192.37MB ``` -
omnibs renamed this gist
Jun 6, 2015 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
omnibs created this gist
Jun 6, 2015 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,97 @@ PHOENIX ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" Running 30s test @ http://10.184.11.239:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 595.50us 398.08us 62.05ms 99.05% Req/Sec 8.50k 304.25 9.20k 70.34% 5087667 requests in 30.10s, 10.42GB read Requests/sec: 169030.24 Transfer/sec: 354.47MB ``` PLUG ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:4000/showdown" Running 30s test @ http://10.184.11.239:4000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 631.83us 2.22ms 217.98ms 99.05% Req/Sec 9.99k 761.02 18.66k 78.99% 5969613 requests in 30.10s, 12.00GB read Requests/sec: 198328.21 Transfer/sec: 408.34MB ``` PLAY ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:5000/showdown" Running 30s test @ http://10.184.11.239:5000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.89ms 14.17ms 349.18ms 98.57% Req/Sec 8.80k 1.58k 23.45k 86.04% 5154091 requests in 30.10s, 9.82GB read Requests/sec: 171236.03 Transfer/sec: 334.12MB ``` RAILS ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 8.50ms 4.07ms 61.71ms 70.94% Req/Sec 598.10 40.81 740.00 71.03% 357320 requests in 30.02s, 788.91MB read Requests/sec: 11903.48 Transfer/sec: 26.28MB ``` SINATRA ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:9292/showdown" Running 30s test @ http://10.184.11.239:9292/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 3.50ms 2.53ms 56.23ms 74.62% Req/Sec 1.54k 414.73 5.33k 65.96% 919897 requests in 30.10s, 1.85GB read Requests/sec: 30561.95 Transfer/sec: 63.04MB ``` GIN ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 655.59us 573.14us 10.11ms 89.21% Req/Sec 8.86k 759.03 15.02k 82.84% 5302230 requests in 30.10s, 10.31GB read Requests/sec: 176156.41 Transfer/sec: 350.61MB ``` MARTINI ``` root@onmetal-client:~# wrk -t20 -c100 -d30S --timeout 2000 "http://10.184.11.239:3000/showdown" Running 30s test @ http://10.184.11.239:3000/showdown 20 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 3.35ms 2.52ms 32.73ms 60.45% Req/Sec 1.61k 99.91 1.98k 68.77% 962648 requests in 30.01s, 1.87GB read Requests/sec: 32077.24 Transfer/sec: 63.84MB ```