You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-28 - 02:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Thu Sep 28, 2023 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25821211693,18sleep00-21swapper/019:06:150
257772114103,6sleep20-21swapper/219:05:412
26015211396,12sleep10-21swapper/119:08:451
25848211393,15sleep30-21swapper/319:06:353
128822810,1sleep312888-21latency00:00:283
26140993416,17cyclictest0-21swapper/100:20:161
26141992917,10cyclictest0-21swapper/200:20:162
242482240,1sleep024251-21aten2.4_r3power00:30:220
26139992117,2cyclictest0-21swapper/020:30:190
2614199200,2cyclictest0-21swapper/222:35:222
2614199200,2cyclictest0-21swapper/221:30:272
2614299172,6cyclictest0-21swapper/320:05:003
2614199172,13cyclictest0-21swapper/219:25:162
2614299160,6cyclictest0-21swapper/319:40:593
2614299160,6cyclictest0-21swapper/319:40:583
2614099160,14cyclictest0-21swapper/100:10:211
2614299150,6cyclictest0-21swapper/300:27:593
2614299150,13cyclictest10366-21cpuspeed_turbos23:55:193
26141991512,2cyclictest0-21swapper/220:00:342
26141991512,2cyclictest0-21swapper/200:30:302
26141991510,3cyclictest4696-21df_abs21:00:202
26141991510,3cyclictest0-21swapper/221:05:172
2614199150,13cyclictest0-21swapper/220:30:302
2613999151,8cyclictest0-21swapper/020:30:000
26139991510,3cyclictest30604-21munin-node0
2614299149,3cyclictest18678-21latency_hist20:15:003
2614299142,6cyclictest0-21swapper/320:30:003
2614299141,7cyclictest0-21swapper/320:35:013
2614299140,12cyclictest0-21swapper/320:05:213
26141991410,2cyclictest0-21swapper/200:25:052
26141991410,2cyclictest0-21swapper/200:15:202
2614199140,12cyclictest21794-21cut20:20:282
2614199140,12cyclictest0-21swapper/200:05:232
2614099140,12cyclictest0-21swapper/120:59:591
2613999149,3cyclictest0-21swapper/022:35:380
26139991411,2cyclictest0-21swapper/020:15:190
2613999140,2cyclictest0-21swapper/020:55:180
26142991310,2cyclictest0-21swapper/323:10:273
2614199139,3cyclictest0-21swapper/223:45:282
2614199139,3cyclictest0-21swapper/219:35:252
2614199131,11cyclictest301ktimersoftd/223:15:312
26141991310,2cyclictest0-21swapper/223:30:212
26141991310,2cyclictest0-21swapper/223:10:252
26141991310,2cyclictest0-21swapper/222:25:172
26141991310,2cyclictest0-21swapper/221:20:252
26141991310,2cyclictest0-21swapper/200:10:242
26141991310,2cyclictest0-21swapper/200:00:292
2614199130,2cyclictest0-21swapper/223:35:192
2614199130,2cyclictest0-21swapper/200:35:272
26140991310,2cyclictest0-21swapper/122:50:181
26140991310,2cyclictest0-21swapper/121:50:371
26140991310,2cyclictest0-21swapper/119:35:341
2614099130,2cyclictest0-21swapper/123:55:261
2614099130,2cyclictest0-21swapper/121:40:161
2613999138,3cyclictest0-21swapper/023:10:230
2613999132,6cyclictest0-21swapper/023:05:010
2613999132,6cyclictest0-21swapper/022:15:000
26139991310,2cyclictest0-21swapper/023:30:230
26139991310,2cyclictest0-21swapper/022:30:300
26139991310,2cyclictest0-21swapper/022:20:240
208792130,2sleep10-21swapper/123:00:251
2614299121,3cyclictest20479-21munin-node3
2614299121,2cyclictest0-21swapper/322:05:153
2614299120,3cyclictest0-21swapper/322:40:263
2614299120,11cyclictest0-21swapper/323:05:193
2614299120,11cyclictest0-21swapper/322:25:293
2614299120,11cyclictest0-21swapper/300:35:183
2614199129,2cyclictest0-21swapper/223:00:302
2614199129,2cyclictest0-21swapper/222:24:262
2614199129,2cyclictest0-21swapper/221:15:202
2614199129,2cyclictest0-21swapper/220:40:002
2614199123,7cyclictest0-21swapper/221:40:282
2614199120,2cyclictest0-21swapper/223:50:272
2614199120,2cyclictest0-21swapper/221:45:232
2614199120,2cyclictest0-21swapper/221:35:222
2614199120,2cyclictest0-21swapper/221:25:222
2614199120,11cyclictest0-21swapper/220:25:282
2614199120,10cyclictest0-21swapper/220:10:292
2614099129,2cyclictest0-21swapper/123:50:211
2614099129,2cyclictest0-21swapper/121:25:271
2614099129,2cyclictest0-21swapper/121:10:321
2614099128,2cyclictest10517-21munin-node1
2614099128,2cyclictest0-21swapper/122:55:321
2614099127,3cyclictest0-21swapper/119:55:201
2614099121,9cyclictest0-21swapper/120:50:341
2614099121,3cyclictest26359-21munin-node1
2614099121,10cyclictest0-21swapper/120:15:231
2614099120,2cyclictest0-21swapper/123:40:351
2614099120,2cyclictest0-21swapper/121:35:201
2614099120,2cyclictest0-21swapper/120:40:221
2614099120,2cyclictest0-21swapper/119:25:141
2614099120,11cyclictest0-21swapper/121:05:231
2614099120,11cyclictest0-21swapper/120:45:251
2614099120,11cyclictest0-21swapper/120:25:221
2614099120,11cyclictest0-21swapper/119:30:331
2613999129,2cyclictest0-21swapper/022:50:020
2613999129,2cyclictest0-21swapper/021:30:400
2613999129,2cyclictest0-21swapper/020:00:330
2613999129,2cyclictest0-21swapper/000:20:350
2613999123,6cyclictest0-21swapper/022:56:160
2613999122,7cyclictest0-21swapper/021:03:590
2613999122,7cyclictest0-21swapper/020:43:010
2613999122,7cyclictest0-21swapper/020:08:580
2613999122,7cyclictest0-21swapper/019:10:020
2613999122,6cyclictest0-21swapper/019:50:290
2613999120,2cyclictest0-21swapper/023:55:180
2613999120,2cyclictest0-21swapper/023:45:340
2613999120,2cyclictest0-21swapper/023:40:260
2613999120,2cyclictest0-21swapper/022:15:250
2613999120,2cyclictest0-21swapper/022:00:240
2613999120,11cyclictest0-21swapper/023:35:340
2613999120,11cyclictest0-21swapper/021:25:230
2613999120,11cyclictest0-21swapper/020:50:260
2613999120,11cyclictest0-21swapper/019:55:240
2613999120,11cyclictest0-21swapper/019:15:300
2614299118,2cyclictest0-21swapper/322:35:393
2614299118,2cyclictest0-21swapper/322:30:203
2614299118,2cyclictest0-21swapper/321:20:193
2614299118,2cyclictest0-21swapper/321:00:203
2614299115,4cyclictest12294-21sh21:19:593
2614299112,7cyclictest0-21swapper/322:50:593
2614299112,7cyclictest0-21swapper/320:42:583
2614299112,6cyclictest5473-21runrttasks00:05:343
2614299112,6cyclictest0-21swapper/322:58:583
2614299112,6cyclictest0-21swapper/319:16:583
2614299111,7cyclictest0-21swapper/323:21:583
2614299111,7cyclictest0-21swapper/323:21:583
2614199118,2cyclictest0-21swapper/223:25:292
2614199118,2cyclictest0-21swapper/222:10:002
2614199118,2cyclictest0-21swapper/220:35:232
2614199118,2cyclictest0-21swapper/220:15:202
2614199110,10cyclictest0-21swapper/222:45:322
2614099118,2cyclictest24201-21aten2.4_r3power00:30:221
2614099118,2cyclictest0-21swapper/122:25:171
2614099118,2cyclictest0-21swapper/121:00:221
2614099113,6cyclictest26138-21cyclictest00:35:321
2614099112,7cyclictest0-21swapper/123:29:591
2614099111,8cyclictest6708-21apt-get19:45:211
2614099111,8cyclictest32407-21munin-node1
2614099111,8cyclictest0-21swapper/120:30:211
2614099111,3cyclictest0-21swapper/120:35:191
2614099111,2cyclictest0-21swapper/100:00:231
2614099110,3cyclictest22570-21munin-node1
2614099110,3cyclictest0-21swapper/123:35:341
2614099110,2cyclictest0-21swapper/122:05:191
2614099110,2cyclictest0-21swapper/120:00:031
2614099110,10cyclictest0-21swapper/122:30:211
2613999117,2cyclictest28355-21munin-run21:59:590
2613999117,2cyclictest0-21swapper/023:20:310
2613999117,2cyclictest0-21swapper/023:20:300
2613999116,3cyclictest16967-21needreboot00:10:230
2613999112,7cyclictest0-21swapper/021:07:580
2613999112,5cyclictest0-21swapper/021:15:590
2613999111,6cyclictest0-21swapper/023:16:190
2613999111,6cyclictest0-21swapper/021:40:160
2613999111,3cyclictest27980-21munin-run00:39:590
2613999110,2cyclictest0-21swapper/022:45:360
2613999110,10cyclictest0-21swapper/023:25:350
245562110,2sleep10-21swapper/123:10:231
2614299107,2cyclictest0-21swapper/321:40:183
2614299107,2cyclictest0-21swapper/320:23:283
2614299107,2cyclictest0-21swapper/319:35:203
2614299105,3cyclictest30691-21cut23:25:203
2614299105,3cyclictest22985-21grep21:45:243
2614299105,3cyclictest1909-21/usr/sbin/munin22:15:013
2614299105,3cyclictest19063-21tr20:15:213
2614299105,3cyclictest0-21swapper/323:49:553
2614299102,7cyclictest0-21swapper/321:32:013
2614299102,6cyclictest0-21swapper/319:10:333
2614299101,7cyclictest0-21swapper/322:17:583
2614299101,6cyclictest27271-21latency21:55:243
2614299101,6cyclictest0-21swapper/321:10:193
2614199107,2cyclictest0-21swapper/222:55:262
2614199107,2cyclictest0-21swapper/222:30:362
2614199107,2cyclictest0-21swapper/222:19:272
2614199107,2cyclictest0-21swapper/220:05:282
2614199107,2cyclictest0-21swapper/219:55:302
2614199104,4cyclictest4601-21rsyslogd23:44:552
2614199104,2cyclictest10657-21cat19:55:002
2614199102,6cyclictest0-21swapper/223:55:302
2614199102,6cyclictest0-21swapper/219:40:282
2614199102,6cyclictest0-21swapper/219:40:272
2614199101,7cyclictest22454-21aten2.4_r3power23:05:182
2614199101,5cyclictest0-21swapper/221:55:012
2614199101,5cyclictest0-21swapper/220:54:592
2614199100,8cyclictest0-21swapper/222:05:002
2614099107,2cyclictest0-21swapper/100:05:231
2614099105,3cyclictest0-21swapper/119:40:231
2614099105,3cyclictest0-21swapper/119:40:221
2614099102,6cyclictest0-21swapper/119:50:391
2614099101,7cyclictest0-21swapper/120:20:211
2614099101,3cyclictest0-21swapper/123:05:241
2614099101,2cyclictest21931-21apt-get00:25:061
2614099101,2cyclictest15247-21ntp_states22:45:281
2614099100,8cyclictest0-21swapper/123:20:181
2614099100,8cyclictest0-21swapper/123:20:171
2614099100,8cyclictest0-21swapper/122:00:201
2614099100,8cyclictest0-21swapper/120:05:201
2614099100,8cyclictest0-21swapper/119:15:171
2614099100,3cyclictest3540-21uptime22:15:351
2614099100,3cyclictest16769-21munin-node1
2614099100,3cyclictest0-21swapper/121:55:241
2614099100,3cyclictest0-21swapper/121:30:281
2614099100,2cyclictest12358-21munin-node1
2613999106,3cyclictest4889-21ntpd21:45:270
2613999105,3cyclictest24983-21fschecks_count21:50:230
2613999105,3cyclictest14996-21tr00:05:290
2613999105,3cyclictest12387-21cpuspeed_turbos00:00:230
2613999105,3cyclictest1053-21fschecks_time19:30:240
2613999103,6cyclictest0-21swapper/020:20:460
2613999102,6cyclictest0-21swapper/021:10:320
2613999102,6cyclictest0-21swapper/019:45:340
2613999102,6cyclictest0-21swapper/019:25:260
2613999101,6cyclictest0-21swapper/022:41:590
2613999101,6cyclictest0-21swapper/020:45:510
30126290,6sleep30-21swapper/319:20:013
261429994,3cyclictest8814-21kernelversion23:50:253
261429994,3cyclictest7630-21python21:05:303
261429994,3cyclictest5028-21head22:20:243
261429994,3cyclictest495-21cpu20:50:193
261429994,3cyclictest32175-21timerandwakeup19:25:263
261429994,3cyclictest26371-21chrt23:15:013
261429994,3cyclictest2613-21tr20:55:213
261429994,3cyclictest24599-21aten2.4-expect21:50:203
261429994,3cyclictest19149-21cut00:15:303
261429994,3cyclictest15235-21latency21:25:243
261429994,3cyclictest14373-21aten2.4_r3power22:45:193
261429994,3cyclictest10027-21unixbench_multi19:50:323
261429994,3cyclictest0-21swapper/300:15:003
261429993,4cyclictest6938-21tr19:45:213
261429993,4cyclictest5476-21chrt23:40:523
261429993,4cyclictest26812-21aten2.4-expect20:35:173
261429993,4cyclictest1113-21grep23:30:353
261429991,3cyclictest28874-21cpuspeed_turbos22:00:233
261429991,3cyclictest0-21swapper/320:45:043
261429990,8cyclictest0-21swapper/319:30:213
261429990,7cyclictest18432-21copy21:35:033
261419996,2cyclictest0-21swapper/219:21:072
261419995,2cyclictest17317-21proc_pri22:50:272
261419994,3cyclictest5473-21runrttasks23:20:232
261419994,3cyclictest5473-21runrttasks23:20:232
261419994,3cyclictest509-21munin-node2
261419994,3cyclictest27158-21http_loadtime21:55:222
261419993,4cyclictest904-21sshd22:10:272
261419993,4cyclictest5473-21runrttasks19:11:042
261419993,4cyclictest31034-21cpuspeed_turbos20:45:202
261419993,4cyclictest28604-21grep19:15:172
261419993,4cyclictest12398-21aten2.4_r3power22:40:192
261419992,5cyclictest10425-21latency_hist21:15:002
261419991,7cyclictest0-21swapper/220:55:122
261419990,2cyclictest0-21swapper/219:45:002
261409994,3cyclictest19287-21switchtime00:15:341
261409991,6cyclictest0-21swapper/122:10:271
261409990,8cyclictest0-21swapper/123:15:191
261409990,8cyclictest0-21swapper/122:20:221
261409990,7cyclictest0-21swapper/123:45:011
261409990,7cyclictest0-21swapper/122:35:191
261399994,3cyclictest9508-21sed23:50:340
261399994,3cyclictest7526-21timerandwakeup22:25:320
261399994,3cyclictest5473-21runrttasks00:16:100
261399994,3cyclictest304912sleep019:21:450
261399994,3cyclictest27101-21df_abs20:35:190
261399994,3cyclictest22502-21grep00:25:190
261399994,3cyclictest16834-21head20:10:210
261399993,4cyclictest13395-21tr21:20:240
261399991,6cyclictest18432-21copy21:35:010
261399991,3cyclictest0-21swapper/019:40:190
261399991,3cyclictest0-21swapper/019:40:190
261399990,4cyclictest0-21swapper/023:05:000
261429983,3cyclictest24169-21chrt00:30:083
261429983,3cyclictest2076-21aten2.4_r3power23:35:203
261429983,3cyclictest20257-21aten2.4_r3power00:20:183
261429980,7cyclictest0-21swapper/319:55:073
261409981,5cyclictest0-21swapper/119:20:331
261399983,3cyclictest2519-21chrt19:35:160
12520280,2sleep10-21swapper/121:20:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional