git.iko.soy/distributed-process
on ff8d9c6 help sign in
distributed-process/benchmarks/results_notes.md
111 lines 4.0 kB history raw
3[2012.02.22] {File for notes on results}
4============================================================
6Here's a run of the PingTCP benchmark on a 3.1 ghz Intel Westmere running RHEL 6.2:
8 ./PingTCP.exe server 8080 &
9 sleep 1
10 server: creating TCP connection
11 server: awaiting client connection
12 ./PingTCP.exe client 0.0.0.0 8080 1000
13 server: listening for pings
14 warming up
15 estimating clock resolution...
16 mean is 1.963031 us (320001 iterations)
17 found 38180 outliers among 319999 samples (11.9%)
18 32741 (10.2%) low severe
19 5439 (1.7%) high severe
20 estimating cost of a clock call...
21 mean is 43.17696 ns (15 iterations)
22 found 2 outliers among 15 samples (13.3%)
23 1 (6.7%) high mild
24 1 (6.7%) high severe
26 benchmarking PingTCP
27 collecting 100 samples, 1 iterations each, in estimated 8.073997 s
28 mean: 77.01177 ms, lb 76.92822 ms, ub 77.14009 ms, ci 0.950
29 std dev: 526.1251 us, lb 366.2692 us, ub 803.1751 us, ci 0.950
31(That was without -O2)
33 -----
35Running a C benchmark on the same machine it takes 5.6 seconds to do
36200K ping pongs.
38 => 28 us avg latency for C
40I got 72 us avg latency using PingTCP. The named pipes backend took
4119.8s and was using 100% CPU. That's a 99 us latency.
43Pipes with unix-bytestring take 11.0 seconds, but still use 100% CPU.
45 => 55 us latency for Pipes with unix-bytestring
49[2012.02.24] {After Adam's strictness refactoring.}
50============================================================
52100K ping/pongs, 1 trial:
54 PingTCP: 7.34s >80%CPU
55 PingTCPTransport: 19s ~60%CPU
56 PingPipes: 6.36s >=100%CPU
58Timing instead using 100 trials of 1000 pingpongs via criterion gives
59results consistent with the above. Here are the mean latencies:
61 PingTCP: 72 us
62 PingTCPTransport: 191 us
63 PingPipes: 65 us
65And here is my first attempt at running the SendTransport benchmark (TCP backend):
67 ./SendTransport.exe client 0.0.0.0 8081 sourceAddr 1000
68 mean: 38.10901 ms, lb 35.92353 ms, ub 39.30006 ms, ci 0.950
69 std dev: 8.305210 ms, lb 4.935312 ms, ub 11.97144 ms, ci 0.950
70 found 5 outliers among 100 samples (5.0%)
71 5 (5.0%) low severe
72 variance introduced by outliers: 94.703%
73 variance is severely inflated by outliers
75 ./SendTransport.exe client 0.0.0.0 8081 sourceAddr 5000
76 mean: 32.87646 ms, lb 29.69947 ms, ub 35.62485 ms, ci 0.950
77 std dev: 15.21419 ms, lb 12.39949 ms, ub 17.39441 ms, ci 0.950
78 found 18 outliers among 100 samples (18.0%)
79 18 (18.0%) low severe
80 variance introduced by outliers: 98.928%
81 variance is severely inflated by outliers
83 ./SendTransport.exe client 0.0.0.0 8081 sourceAddr 10000
84 mean: 32.87737 ms, lb 29.34305 ms, ub 35.62680 ms, ci 0.950
85 std dev: 15.22060 ms, lb 12.40398 ms, ub 17.57401 ms, ci 0.950
86 found 18 outliers among 100 samples (18.0%)
87 18 (18.0%) low severe
88 variance introduced by outliers: 98.929%
89 variance is severely inflated by outliers
91 ./SendTransport.exe client 0.0.0.0 8081 sourceAddr 50000
92 mean: 1.351308 ms, lb 1.249680 ms, ub 1.548209 ms, ci 0.950
93 std dev: 696.6768 us, lb 448.3538 us, ub 1.314057 ms, ci 0.950
94 found 24 outliers among 100 samples (24.0%)
95 3 (3.0%) high mild
96 21 (21.0%) high severe
97 variance introduced by outliers: 98.942%
98 variance is severely inflated by outliers
100 ./SendTransport.exe client 0.0.0.0 8081 sourceAddr 100000
101 mean: 2.184967 ms, lb 2.030858 ms, ub 2.641543 ms, ci 0.950
102 std dev: 1.235358 ms, lb 518.3995 us, ub 2.666540 ms, ci 0.950
103 found 15 outliers among 100 samples (15.0%)
104 4 (4.0%) low severe
105 9 (9.0%) high severe
106 variance introduced by outliers: 98.952%
107 variance is severely inflated by outliers
109This is quite odd. so in addition to the high variance it would seem
110that messages of size 50K or 100K are much FASTER to send than smaller
111massages?