You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 12:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Tue Dec 09, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20145999624,51cyclictest0-21swapper/322:53:093
20144998929,13cyclictest30462-21grep23:33:262
2014499885,27cyclictest0-21swapper/221:24:422
2014499885,27cyclictest0-21swapper/221:24:422
227012870,3sleep2281ktimersoftd/223:18:212
20145998730,50cyclictest29241-21unixbench_multi18:58:513
2014499873,26cyclictest0-21swapper/223:50:532
2014499864,34cyclictest15179-21tail19:38:382
20144998638,9cyclictest18695-21grep23:08:282
20144998626,12cyclictest1369-21runrttasks21:53:262
20144998622,12cyclictest0-21swapper/219:28:412
20145998426,38cyclictest756-21latency_hist20:18:153
2014499834,27cyclictest0-21swapper/222:13:112
20145998221,53cyclictest9076-21fschecks_time20:33:353
20144998212,33cyclictest0-21swapper/219:23:182
2014599813,69cyclictest1369-21runrttasks19:03:403
20145998125,49cyclictest25400-21cpuspeed_turbos21:08:293
20145998113,12cyclictest0-21swapper/323:08:373
2014599804,68cyclictest0-21swapper/320:43:153
2014599804,68cyclictest0-21swapper/320:43:153
2014599803,70cyclictest31053-21needreboot21:18:393
2014599803,70cyclictest1369-21runrttasks22:20:023
2014499803,23cyclictest17648-21hwlatdetect19:43:372
20144998018,12cyclictest0-21swapper/219:13:502
20145997938,34cyclictest30642-21sed23:33:283
2014599793,23cyclictest0-21swapper/300:08:393
2014599793,23cyclictest0-21swapper/300:08:393
20145997924,48cyclictest14038-21crond21:53:113
2014599792,70cyclictest407-21sort21:23:313
2014599792,70cyclictest407-21sort21:23:313
2014599783,69cyclictest0-21swapper/323:43:323
2014599783,68cyclictest14262-21cut22:58:363
2014599783,67cyclictest8535-21sendmail_mailqu21:38:413
2014599783,67cyclictest0-21swapper/321:54:463
20145997823,48cyclictest1558-21unixbench-2d19:08:503
20145997822,49cyclictest12157-21cpuspeed_turbos00:03:223
2014599773,66cyclictest24551-21taskset21:04:383
20145997722,49cyclictest6829-21idleruntime22:43:313
20145997722,48cyclictest7773-21fschecks_time23:53:253
20145997722,48cyclictest29047-21unixbench-2d23:28:413
20145997722,48cyclictest29047-21unixbench-2d23:28:413
2014599772,68cyclictest16896-21telnet19:43:303
2014499777,31cyclictest12889-21latency_hist20:43:152
2014499777,31cyclictest12889-21latency_hist20:43:152
2014499774,27cyclictest0-21swapper/222:58:272
20144997734,12cyclictest0-21swapper/218:48:352
2014599763,22cyclictest0-21swapper/322:29:423
20145997625,44cyclictest12852-21awk19:33:393
20145997624,46cyclictest18495-21munin-node20:53:333
20145997623,46cyclictest1369-21runrttasks18:48:043
20145997623,45cyclictest1369-21runrttasks19:32:333
20145997622,48cyclictest17778-21load21:58:363
20145997622,45cyclictest15432-21munin-node23:03:313
20145997621,47cyclictest1369-21runrttasks23:52:313
20145997620,48cyclictest1369-21runrttasks18:51:063
20145997616,53cyclictest0-21swapper/321:33:133
2014599753,65cyclictest6934-21crond19:23:173
2014599753,64cyclictest0-21swapper/323:23:353
2014599753,64cyclictest0-21swapper/320:29:233
2014599753,24cyclictest0-21swapper/322:15:443
2014599753,21cyclictest0-21swapper/320:06:323
2014599753,20cyclictest23431-21gltestperf23:18:283
20145997522,47cyclictest28771-21cpu22:23:263
20145997522,46cyclictest6485-21unixbench_singl21:33:453
20145997521,48cyclictest29117-21df_abs20:08:313
20145997521,48cyclictest0-21swapper/323:38:233
20145997521,47cyclictest20619-21cpuspeed_turbos20:58:283
20145997516,9cyclictest5268-21grep20:23:483
2014599743,64cyclictest1243-21snmpd22:04:483
2014599743,22cyclictest0-21swapper/320:51:173
2014599743,21cyclictest0-21swapper/321:48:073
20145997427,41cyclictest11019-21cut22:53:243
20145997425,43cyclictest14339-21needreboot20:43:423
20145997425,43cyclictest14339-21needreboot20:43:423
20145997424,44cyclictest1369-21runrttasks23:16:243
20145997417,8cyclictest5180-21unixbench-2d22:38:443
20145997417,8cyclictest1083-21aten_r1power_vo22:33:223
2014599733,63cyclictest0-21swapper/320:01:423
2014599733,63cyclictest0-21swapper/320:01:423
20145997329,38cyclictest3260-21tail19:13:383
20145997313,8cyclictest2859-21unixbench_multi20:18:503
2014499734,26cyclictest0-21swapper/200:08:322
2014499734,26cyclictest0-21swapper/200:08:322
243799720,6rtkit-daemon2436-21rtkit-daemon18:43:412
2014599723,63cyclictest0-21swapper/318:53:543
20145997222,43cyclictest22640-21missed_timers19:53:403
2014599713,27cyclictest9444-21munin-node23:58:403
20145997124,41cyclictest1369-21runrttasks19:27:173
20145997120,44cyclictest15760-21rt-features19:38:483
2014599703,20cyclictest0-21swapper/321:13:323
2014599693,60cyclictest0-21swapper/319:48:303
2014599693,21cyclictest0-21swapper/322:09:313
251502650,2sleep1201ktimersoftd/121:08:271
2014399654,52cyclictest0-21swapper/120:13:411
62612613,49sleep00-21swapper/023:48:420
2014599613,27cyclictest20170-21cat18:43:193
270732603,12sleep00-21swapper/018:55:360
243799600,4rtkit-daemon2436-21rtkit-daemon19:06:482
2013899604,38cyclictest0-21swapper/021:14:400
286382594,48sleep00-21swapper/023:28:350
286382594,48sleep00-21swapper/023:28:350
243799580,5rtkit-daemon2436-21rtkit-daemon23:06:462
243799570,5rtkit-daemon2436-21rtkit-daemon21:33:122
20143995713,22cyclictest2380-21tail23:43:211
188402573,11sleep00-21swapper/023:08:320
92732564,43sleep09296-21iostat_ios22:48:330
267232563,7sleep026795-21diskstats22:18:270
243799560,5rtkit-daemon2436-21rtkit-daemon23:33:072
243799560,5rtkit-daemon2436-21rtkit-daemon23:33:072
243799560,5rtkit-daemon2436-21rtkit-daemon19:23:472
243799560,5rtkit-daemon2436-21rtkit-daemon18:53:482
243799550,5rtkit-daemon2436-21rtkit-daemon20:03:462
243799550,5rtkit-daemon2436-21rtkit-daemon20:03:162
243799550,5rtkit-daemon2436-21rtkit-daemon20:03:162
205032553,12sleep00-21swapper/022:03:440
2014399557,26cyclictest0-21swapper/122:03:241
20143995514,13cyclictest0-21swapper/123:33:071
20143995514,13cyclictest0-21swapper/123:33:071
2013899555,35cyclictest0-21swapper/022:43:420
29292542,44sleep20-21swapper/222:34:092
243799540,5rtkit-daemon2436-21rtkit-daemon20:43:372
243799540,5rtkit-daemon2436-21rtkit-daemon20:43:372
243799540,5rtkit-daemon2436-21rtkit-daemon19:53:312
243799540,5rtkit-daemon2436-21rtkit-daemon19:38:172
243352544,13sleep00-21swapper/022:13:270
2014399545,42cyclictest0-21swapper/120:23:141
2014399544,42cyclictest0-21swapper/119:53:331
20143995412,21cyclictest0-21swapper/120:38:271
20143995412,21cyclictest0-21swapper/120:38:271
243799530,5rtkit-daemon2436-21rtkit-daemon22:28:252
243799530,5rtkit-daemon2436-21rtkit-daemon21:13:282
2014399533,43cyclictest6940-21munin-run19:23:171
20143995322,14cyclictest0-21swapper/100:03:391
20143995322,13cyclictest0-21swapper/120:23:431
2013899534,31cyclictest0-21swapper/020:13:280
20138995337,10cyclictest13584-21unixbench_singl00:03:410
199462532,44sleep00-21swapper/020:56:580
177152532,43sleep00-21swapper/021:58:360
38372523,8sleep03850-21python19:13:490
222362523,10sleep00-21swapper/018:44:470
2014399525,29cyclictest0-21swapper/119:03:411
20143995216,27cyclictest0-21swapper/121:38:251
20143995210,26cyclictest0-21swapper/122:28:101
20138995236,10cyclictest12261-21grep21:48:270
20138995235,11cyclictest31919-21hwlatdetect22:28:330
17402523,10sleep10-21swapper/121:25:171
17402523,10sleep10-21swapper/121:25:171
115132522,43sleep00-21swapper/000:02:080
101532522,10sleep10-21swapper/122:49:431
55032512,41sleep00-21swapper/020:25:430
54302512,42sleep00-21swapper/022:40:230
43132512,41sleep00-21swapper/021:31:080
282292512,42sleep00-21swapper/020:05:360
243799510,5rtkit-daemon2436-21rtkit-daemon21:48:272
20144995123,22cyclictest9209-21idleruntime-cro23:58:062
2014399515,28cyclictest0-21swapper/123:58:401
2014399515,28cyclictest0-21swapper/123:03:281
2014399515,28cyclictest0-21swapper/121:18:481
2014399515,28cyclictest0-21swapper/118:48:551
2014399514,29cyclictest0-21swapper/122:58:381
2014399514,29cyclictest0-21swapper/120:28:531
2014399514,27cyclictest0-21swapper/119:38:171
2014399514,26cyclictest4785-21ntp_states22:38:381
20143995131,13cyclictest26313-21aten_r1power_cu20:03:271
20143995114,29cyclictest0-21swapper/122:08:241
2013899514,30cyclictest0-21swapper/019:48:290
2013899514,28cyclictest0-21swapper/020:33:140
20138995135,10cyclictest362-21grep23:38:270
20138995135,10cyclictest29877-21tail20:08:410
2013899513,42cyclictest0-21swapper/021:38:240
138352513,9sleep10-21swapper/121:50:351
47842503,40sleep00-21swapper/019:18:290
29762502,42sleep00-21swapper/022:34:310
258602502,42sleep00-21swapper/020:00:480
258602502,42sleep00-21swapper/020:00:480
243799500,5rtkit-daemon2436-21rtkit-daemon20:23:302
221602502,41sleep00-21swapper/023:15:380
20144995036,8cyclictest5257-21grep22:38:442
20144995020,14cyclictest0-21swapper/220:53:462
2014499502,43cyclictest0-21swapper/223:46:072
2014399505,28cyclictest0-21swapper/100:13:061
2014399505,28cyclictest0-21swapper/100:13:061
2014399505,23cyclictest0-21swapper/122:33:361
2014399504,29cyclictest0-21swapper/121:53:231
2014399504,28cyclictest0-21swapper/123:13:241
2014399504,28cyclictest0-21swapper/122:28:341
2014399504,28cyclictest0-21swapper/121:28:291
2014399504,28cyclictest0-21swapper/120:58:391
2014399504,28cyclictest0-21swapper/120:48:391
2014399504,28cyclictest0-21swapper/120:33:321
2014399504,28cyclictest0-21swapper/119:58:351
2014399504,28cyclictest0-21swapper/119:58:351
2014399504,28cyclictest0-21swapper/119:13:331
2014399504,28cyclictest0-21swapper/119:08:341
2014399504,28cyclictest0-21swapper/118:53:381
20143995031,11cyclictest29048-21cpuspeed_turbos20:08:301
20143995020,13cyclictest0-21swapper/119:38:301
20143995019,13cyclictest0-21swapper/123:38:081
2013899504,29cyclictest0-21swapper/020:58:400
2013899504,28cyclictest0-21swapper/022:08:420
2013899504,28cyclictest0-21swapper/020:38:430
2013899504,28cyclictest0-21swapper/020:38:430
2013899504,28cyclictest0-21swapper/018:48:430
20138995019,13cyclictest0-21swapper/020:18:440
168552502,40sleep00-21swapper/020:48:440
138292503,41sleep00-21swapper/019:34:410
319342492,41sleep00-21swapper/021:22:340
243799490,5rtkit-daemon2436-21rtkit-daemon22:05:182
229592492,40sleep00-21swapper/019:53:470
2014499494,29cyclictest0-21swapper/220:13:482
2014499494,27cyclictest0-21swapper/221:38:232
2014499494,27cyclictest0-21swapper/218:58:502
2014499493,28cyclictest0-21swapper/223:28:082
20144994921,13cyclictest0-21swapper/220:38:152
20144994919,13cyclictest0-21swapper/220:48:392
2014399498,33cyclictest201ktimersoftd/120:43:381
2014399498,33cyclictest201ktimersoftd/120:43:381
2014399495,25cyclictest28712-21munin-node21:13:391
2014399494,28cyclictest0-21swapper/123:38:411
2014399494,28cyclictest0-21swapper/119:28:421
2014399494,28cyclictest0-21swapper/118:43:441
2014399494,26cyclictest0-21swapper/123:28:091
2013899495,27cyclictest0-21swapper/022:58:300
2013899494,28cyclictest0-21swapper/023:18:190
2013899494,28cyclictest0-21swapper/021:03:350
2013899494,28cyclictest0-21swapper/020:43:420
2013899494,28cyclictest0-21swapper/020:43:420
2013899494,27cyclictest0-21swapper/023:23:330
2013899494,27cyclictest0-21swapper/021:23:270
2013899494,27cyclictest0-21swapper/021:23:270
2013899494,27cyclictest0-21swapper/019:23:340
2013899494,27cyclictest0-21swapper/019:08:470
2013899494,26cyclictest0-21swapper/000:08:260
2013899494,26cyclictest0-21swapper/000:08:260
20138994920,11cyclictest0-21swapper/021:53:280
198712494,39sleep10-21swapper/118:40:181
127432492,39sleep00-21swapper/022:57:200
101832493,39sleep00-21swapper/019:28:360
95002483,9sleep00-21swapper/021:43:170
305082482,39sleep00-21swapper/022:25:000
299022483,10sleep00-21swapper/019:03:210
295212483,10sleep10-21swapper/119:01:211
2014499485,27cyclictest0-21swapper/220:28:522
2014499485,26cyclictest0-21swapper/220:09:412
2014499484,29cyclictest0-21swapper/221:58:322
2014499484,27cyclictest0-21swapper/220:18:502
20144994831,10cyclictest576-21grep23:38:292
20144994814,25cyclictest0-21swapper/221:08:282
2014399489,22cyclictest24633-21tr22:13:301
2014399486,23cyclictest26226-21cut22:18:221
2014399485,35cyclictest12487-21taskset22:53:591
2014399485,25cyclictest0-21swapper/121:58:431
2014399484,27cyclictest0-21swapper/122:43:321
2014399484,26cyclictest0-21swapper/121:48:121
2013899484,27cyclictest0-21swapper/023:33:200
2013899484,27cyclictest0-21swapper/023:03:320
2013899484,27cyclictest0-21swapper/020:33:500
2013899484,27cyclictest0-21swapper/019:38:420
2013899483,27cyclictest0-21swapper/023:53:270
200682484,38sleep00-21swapper/018:42:500
294362473,10sleep00-21swapper/019:00:150
247692473,39sleep10-21swapper/121:07:271
2014499474,28cyclictest0-21swapper/222:43:212
2014499474,27cyclictest0-21swapper/220:58:332
2014499474,27cyclictest0-21swapper/219:48:362
20144994730,11cyclictest11868-21needreboot22:53:352
2014499472,38cyclictest0-21swapper/223:13:212
20144994717,13cyclictest0-21swapper/222:48:252
20143994718,13cyclictest0-21swapper/121:34:431
20143994717,13cyclictest0-21swapper/123:53:181
20143994710,8cyclictest114550irq/46-eth023:18:291
2013899474,26cyclictest0-21swapper/023:43:280
167312472,37sleep00-21swapper/019:43:230
209492462,10sleep10-21swapper/119:49:041
2014499465,35cyclictest0-21swapper/222:13:252
20144994626,14cyclictest0-21swapper/222:18:222
2014399464,26cyclictest0-21swapper/120:57:591
20143994628,11cyclictest16831-21aten_r1power_fr19:43:291
65002453,36sleep10-21swapper/123:48:591
2014499455,33cyclictest0-21swapper/221:03:482
2014499454,24cyclictest29999-21aten2.4-expect21:18:272
20144994532,8cyclictest10659-21needreboot23:58:332
20144994528,11cyclictest28966-21cpuspeed_turbos22:23:272
2014399455,13cyclictest0-21swapper/119:24:221
2013899454,25cyclictest0-21swapper/021:09:390
20138994519,11cyclictest0-21swapper/021:34:430
184692454,34sleep20-21swapper/218:38:342
2014499445,13cyclictest0-21swapper/219:08:382
2014499444,13cyclictest0-21swapper/200:08:062
2014499443,25cyclictest0-21swapper/221:33:392
20144994427,11cyclictest9811-21cpuspeed_turbos21:43:272
198772443,9sleep10-21swapper/123:12:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional