You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 02:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Thu Sep 10, 2026 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28206991095,12cyclictest1227-21runrttasks22:21:161
2820699994,12cyclictest26353-21ssh21:17:171
2820699974,32cyclictest0-21swapper/120:42:131
2820099955,73cyclictest0-21swapper/020:57:270
2820699944,23cyclictest0-21swapper/118:29:171
28206999414,15cyclictest0-21swapper/122:57:231
28206999311,24cyclictest25582-21ssh22:52:331
28200999322,34cyclictest23742-21ssh22:51:100
2820699924,26cyclictest14267-21users20:07:421
28200999221,50cyclictest5504-21ssh22:22:230
28200999216,9cyclictest107550irq/46-eth021:57:220
2820899914,80cyclictest4796-21cpuspeed_turbos22:22:152
2820899914,80cyclictest16776-21df21:02:222
28200999116,11cyclictest1124-21snmpd19:49:290
2820899904,50cyclictest0-21swapper/221:32:192
2820899903,80cyclictest30576-21latency_hist17:37:152
2820899903,52cyclictest0-21swapper/219:42:442
2820899903,52cyclictest0-21swapper/219:42:442
2820699905,14cyclictest1227-21runrttasks21:49:171
2820699905,14cyclictest1227-21runrttasks21:49:171
2820699904,31cyclictest0-21swapper/122:02:351
28206999022,13cyclictest0-21swapper/120:22:531
28206999022,13cyclictest0-21swapper/120:22:531
2820099903,75cyclictest0-21swapper/018:32:240
2820099903,68cyclictest0-21swapper/020:49:530
2820699895,29cyclictest0-21swapper/121:52:221
2820099895,22cyclictest0-21swapper/021:29:550
2820099894,24cyclictest0-21swapper/020:17:260
2820099893,22cyclictest0-21swapper/022:36:470
2820099893,22cyclictest0-21swapper/022:36:470
2820099893,21cyclictest0-21swapper/020:04:110
28200998917,55cyclictest0-21swapper/020:32:260
28200998913,12cyclictest0-21swapper/022:27:180
28200998913,11cyclictest0-21swapper/021:19:360
2820899885,77cyclictest0-21swapper/221:07:432
2820899885,77cyclictest0-21swapper/221:07:432
2820899884,78cyclictest0-21swapper/219:47:342
2820899884,49cyclictest12059-21cat18:02:442
2820899883,48cyclictest0-21swapper/220:31:312
2820099884,67cyclictest0-21swapper/022:19:510
2820099883,64cyclictest0-21swapper/019:53:040
2820099883,23cyclictest0-21swapper/020:52:380
28200998818,53cyclictest0-21swapper/020:42:230
28200998816,11cyclictest1124-21snmpd22:57:240
28200998814,13cyclictest0-21swapper/021:38:270
28200998811,58cyclictest0-21swapper/020:27:300
2820899874,75cyclictest14600-21gltestperf21:47:232
2820899874,75cyclictest14600-21gltestperf21:47:232
2820899873,77cyclictest0-21swapper/219:34:052
2820699875,29cyclictest0-21swapper/121:07:241
2820699875,29cyclictest0-21swapper/121:07:241
2820099873,75cyclictest0-21swapper/022:13:250
2820099873,69cyclictest0-21swapper/021:04:410
2820899864,76cyclictest0-21swapper/220:17:222
2820899864,75cyclictest0-21swapper/221:24:322
2820899864,48cyclictest0-21swapper/221:13:052
2820699865,35cyclictest0-21swapper/121:22:491
2820699865,30cyclictest0-21swapper/120:22:051
2820099864,59cyclictest0-21swapper/022:07:200
2820099864,24cyclictest0-21swapper/019:37:340
2820099864,22cyclictest0-21swapper/021:35:310
28200998620,50cyclictest0-21swapper/019:22:270
28200998610,12cyclictest0-21swapper/021:22:260
2820899855,74cyclictest0-21swapper/221:21:232
2820899854,75cyclictest0-21swapper/220:32:562
2820899854,64cyclictest4113-21ssh22:21:462
2820899853,75cyclictest6734-21cut19:57:262
2820099854,74cyclictest0-21swapper/019:42:550
2820099854,74cyclictest0-21swapper/019:42:550
2820099854,65cyclictest0-21swapper/020:16:260
2820099854,14cyclictest0-21swapper/020:22:220
2820099854,14cyclictest0-21swapper/020:22:220
2820099853,75cyclictest0-21swapper/019:57:270
2820099853,73cyclictest0-21swapper/018:57:280
2820099853,73cyclictest0-21swapper/018:57:280
28200998520,49cyclictest0-21swapper/021:16:410
28200998517,51cyclictest19869-21ssh21:56:340
28209998438,38cyclictest4008-21awk19:52:373
28209998436,40cyclictest30312-21ssh19:44:283
28209998436,40cyclictest30312-21ssh19:44:283
2820899846,61cyclictest0-21swapper/222:29:232
2820899845,73cyclictest0-21swapper/222:07:142
2820899844,75cyclictest0-21swapper/220:57:342
2820899844,73cyclictest32255-21ssh20:37:162
2820899844,73cyclictest0-21swapper/220:46:512
2820899844,73cyclictest0-21swapper/218:39:282
2820899844,73cyclictest0-21swapper/217:57:322
2820899844,49cyclictest0-21swapper/221:29:022
2820099848,58cyclictest0-21swapper/022:02:110
2820099843,8cyclictest107550irq/46-eth021:52:010
2820099843,8cyclictest107550irq/46-eth021:52:010
2820099843,22cyclictest0-21swapper/022:43:170
2820099843,20cyclictest1124-21snmpd19:34:270
28200998414,8cyclictest10851-21ssh21:42:190
28200998413,12cyclictest0-21swapper/020:37:370
28209998337,39cyclictest11005-21chrt21:42:203
2820899833,74cyclictest0-21swapper/222:56:242
2820899833,74cyclictest0-21swapper/222:45:242
2820899833,74cyclictest0-21swapper/219:52:232
2820899833,73cyclictest0-21swapper/222:38:292
2820099834,62cyclictest0-21swapper/022:55:340
2820099834,15cyclictest0-21swapper/021:07:280
2820099834,15cyclictest0-21swapper/021:07:280
28200998331,46cyclictest19493-21tune2fs19:27:270
2820099833,21cyclictest15231-1kworker/0:2H19:22:090
28200998318,10cyclictest16091-21grep22:37:330
28200998317,7cyclictest107550irq/46-eth018:09:140
28200998316,8cyclictest29701-21telnet18:42:240
2820899825,69cyclictest1227-21runrttasks18:22:232
2820899824,73cyclictest0-21swapper/222:12:232
2820899824,72cyclictest495-21crond17:42:142
2820899824,72cyclictest0-21swapper/222:32:112
2820899824,72cyclictest0-21swapper/222:32:112
2820899824,72cyclictest0-21swapper/222:02:212
2820899824,72cyclictest0-21swapper/220:47:372
2820899824,72cyclictest0-21swapper/220:15:372
2820899824,72cyclictest0-21swapper/217:47:142
2820899824,71cyclictest0-21swapper/219:37:592
2820899823,72cyclictest0-21swapper/221:57:292
2820899823,58cyclictest0-21swapper/222:52:022
2820099823,23cyclictest0-21swapper/020:07:400
28200998222,54cyclictest1227-21runrttasks19:16:500
28200998213,14cyclictest0-21swapper/018:12:350
28200998213,14cyclictest0-21swapper/018:12:350
28209998140,33cyclictest30759-21fschecks_count22:12:183
28209998136,39cyclictest3568-21awk21:32:043
2820899814,71cyclictest0-21swapper/223:02:012
2820899814,71cyclictest0-21swapper/221:52:142
2820899814,71cyclictest0-21swapper/220:22:522
2820899814,71cyclictest0-21swapper/220:22:522
2820899814,71cyclictest0-21swapper/218:57:122
2820899813,70cyclictest107301ssh20:02:402
2820899813,61cyclictest20945-21diskmemload20:53:442
28200998121,9cyclictest1124-21snmpd18:56:310
28209998034,40cyclictest0-21swapper/321:12:503
2820899804,70cyclictest0-21swapper/219:22:102
2820099805,12cyclictest28205-21basename17:32:140
2820099805,12cyclictest28205-21basename17:32:140
28200998022,51cyclictest5009-21unixbench_multi17:47:460
28200998016,8cyclictest1124-21snmpd18:24:160
2820899794,69cyclictest0-21swapper/221:43:132
28200997922,50cyclictest32255-21smartctl17:37:460
28200997918,8cyclictest1124-21snmpd18:49:200
28209997837,34cyclictest7766-21runrttasks22:27:113
28209997837,34cyclictest22010-21crond20:22:083
2820899784,68cyclictest0-21swapper/218:17:312
2820099783,21cyclictest0-21swapper/018:39:180
28200997822,50cyclictest7058-21cpuspeed19:02:220
28209997738,32cyclictest25456-21wc19:37:323
28209997730,40cyclictest20906-21grep21:57:223
28209997730,39cyclictest0-21swapper/321:03:163
28209997729,40cyclictest31791munin-node20:42:253
28209997727,44cyclictest0-21swapper/318:42:223
2820899775,66cyclictest0-21swapper/221:40:542
2820899774,67cyclictest0-21swapper/220:07:282
2820899774,67cyclictest0-21swapper/218:47:322
2820899774,67cyclictest0-21swapper/218:33:362
2820099773,21cyclictest0-21swapper/017:52:250
2820099773,21cyclictest0-21swapper/017:34:080
28200997725,46cyclictest9495-21find17:57:410
28209997640,29cyclictest3978-21runrttasks22:21:153
28209997636,33cyclictest7544-21sshd17:53:253
28209997633,13cyclictest27542-21sort22:07:203
28209997629,40cyclictest0-21swapper/322:06:233
28209997629,40cyclictest0-21swapper/320:51:193
28209997628,40cyclictest11672-21ssh22:32:223
28209997628,40cyclictest11672-21ssh22:32:223
2820899765,65cyclictest17327-21runrttasks18:16:302
2820899765,65cyclictest17327-21runrttasks18:16:302
2820899763,64cyclictest5671-21netstat18:57:362
2820899763,64cyclictest5671-21netstat18:57:362
258522760,5sleep325984-21grep22:52:393
28209997537,15cyclictest31910-21rm21:24:383
28209997534,34cyclictest31629-21cpuspeed_turbos19:47:223
28209997530,38cyclictest29047-21rm21:21:233
28209997529,39cyclictest756-21munin-node20:37:263
28209997525,43cyclictest15157-21chrt20:11:033
2820099753,63cyclictest11321-21grep18:02:320
2820099753,21cyclictest18238-21tune2fs18:17:300
28209997433,35cyclictest23659-21taskset19:35:133
28209997433,34cyclictest9457-21cpuspeed_turbos20:02:223
28209997433,34cyclictest21079-21rm22:47:143
28209997431,37cyclictest6791-21rm22:24:183
28209997431,37cyclictest1227-21runrttasks21:41:173
28209997429,38cyclictest124621ssh20:55:513
28209997416,52cyclictest0-21swapper/320:23:143
28209997416,52cyclictest0-21swapper/320:23:143
2820899745,64cyclictest0-21swapper/217:48:302
2820899744,49cyclictest13875-21uname18:07:352
28200997425,43cyclictest737-21munin-node17:42:320
28200997414,9cyclictest9225-21munin-node19:07:300
28209997330,36cyclictest6489-21ssh21:35:413
28209997326,41cyclictest0-21swapper/317:37:283
28209997320,47cyclictest0-21swapper/323:00:403
2820899733,64cyclictest0-21swapper/219:17:102
28200997312,53cyclictest23690-21ntpq18:27:390
2820899724,61cyclictest0-21swapper/218:27:402
2820899723,64cyclictest0-21swapper/219:27:392
2820899723,40cyclictest8550-21grep19:02:412
28209997134,13cyclictest12614-21latency_hist18:07:143
28209997132,33cyclictest0-21swapper/321:00:133
219499710,6rtkit-daemon17199-21grep19:22:291
28209997030,34cyclictest31232-21ssh20:34:333
28209997030,33cyclictest17620-21ssh20:12:493
2820899704,60cyclictest0-21swapper/217:52:462
2820899704,52cyclictest29921-21cpuspeed_turbos18:42:252
2820899703,58cyclictest0-21swapper/219:26:022
28209996930,33cyclictest16988-21munin-node21:52:183
28209996927,17cyclictest0-21swapper/320:01:293
2820899694,59cyclictest0-21swapper/217:32:152
2820899694,59cyclictest0-21swapper/217:32:152
28209996833,29cyclictest14639-21grep22:37:143
28209996829,32cyclictest9318-21head19:07:213
28209996829,32cyclictest14633-21ssh21:47:243
28209996829,32cyclictest14633-21ssh21:47:243
28209996828,15cyclictest0-21swapper/321:08:353
28209996828,15cyclictest0-21swapper/321:08:353
28209996826,20cyclictest0-21swapper/318:26:363
28209996738,23cyclictest28239-21kernelversion18:37:353
28209996735,26cyclictest18555-21ls18:17:333
28209996724,36cyclictest1124-21snmpd17:49:113
2820899674,53cyclictest0-21swapper/219:07:282
28209996642,17cyclictest24554-21crond18:32:113
28209996641,18cyclictest16480-21aten_r1power_fr19:22:223
28209996640,19cyclictest6320-21runrttasks18:58:333
28209996640,19cyclictest6320-21runrttasks18:58:333
28209996627,32cyclictest11708-21tail19:12:223
28206996615,33cyclictest0-21swapper/121:27:261
28209996540,17cyclictest17384-21crond18:17:123
28209996540,17cyclictest17384-21crond18:17:123
28209996536,23cyclictest20237-21munin-node19:27:393
28209996520,39cyclictest0-21swapper/318:02:133
28209996519,40cyclictest0-21swapper/322:42:053
28209996444,14cyclictest14856-21sed19:17:303
28209996441,16cyclictest26474-21sed18:32:453
28209996422,36cyclictest0-21swapper/317:42:293
28209996419,39cyclictest0-21swapper/318:07:293
2820699645,37cyclictest0-21swapper/119:40:091
219499640,8rtkit-daemon0-21swapper/020:59:331
219499640,6rtkit-daemon0-21swapper/019:17:301
28209996344,12cyclictest4107-21runrttasks18:55:523
28209996343,13cyclictest1285-21unixbench_singl18:47:453
28209996342,13cyclictest7435-21fschecks_count19:02:263
28209996342,13cyclictest26455-21gltestperf20:27:273
28209996337,20cyclictest29015-21munin-node17:32:333
2820699615,36cyclictest0-21swapper/119:52:121
2820699588,24cyclictest0-21swapper/122:32:101
2820699588,24cyclictest0-21swapper/122:32:101
2820699578,19cyclictest0-21swapper/119:52:101
2820699574,35cyclictest0-21swapper/121:57:441
2820699574,35cyclictest0-21swapper/119:42:391
2820699574,35cyclictest0-21swapper/119:42:391
28206995737,11cyclictest18150-21head21:02:411
28206995716,27cyclictest0-21swapper/121:40:061
28209995637,13cyclictest28380-21sort17:32:153
28209995637,13cyclictest28380-21sort17:32:153
2820699564,28cyclictest0-21swapper/120:12:231
2820699563,36cyclictest0-21swapper/120:41:331
2820699555,29cyclictest0-21swapper/121:36:221
2820699555,29cyclictest0-21swapper/118:52:431
2820699554,36cyclictest0-21swapper/122:08:001
2820699554,35cyclictest0-21swapper/120:28:111
2820699554,35cyclictest0-21swapper/119:58:301
2820699554,31cyclictest0-21swapper/120:07:081
2820699554,29cyclictest0-21swapper/122:51:131
2820699553,28cyclictest0-21swapper/117:42:141
28206995522,13cyclictest0-21swapper/122:22:221
2820699546,29cyclictest0-21swapper/122:12:351
2820699545,31cyclictest0-21swapper/117:53:241
2820699545,30cyclictest0-21swapper/118:57:291
2820699545,30cyclictest0-21swapper/118:57:291
2820699545,28cyclictest0-21swapper/117:59:131
2820699545,27cyclictest0-21swapper/118:07:341
2820699544,29cyclictest0-21swapper/122:27:151
28206995419,14cyclictest11002-21diskstats21:42:201
2820699535,29cyclictest0-21swapper/119:12:271
2820699535,29cyclictest0-21swapper/119:02:411
2820699535,29cyclictest0-21swapper/118:47:371
2820699535,28cyclictest0-21swapper/120:47:161
2820699535,28cyclictest0-21swapper/117:47:411
2820699534,29cyclictest0-21swapper/120:53:201
2820699534,29cyclictest0-21swapper/119:07:331
2820699534,28cyclictest17404-21sh22:42:021
2820699534,28cyclictest0-21swapper/119:29:261
28206995334,11cyclictest15542-21tr18:12:261
28206995334,11cyclictest15542-21tr18:12:261
28206995322,13cyclictest0-21swapper/118:32:301
28206995321,14cyclictest0-21swapper/118:42:391
249592533,42sleep10-21swapper/121:13:211
199862533,11sleep10-21swapper/122:44:221
288932524,8sleep128971-21hostname20:32:231
2820699525,27cyclictest0-21swapper/119:33:271
2820699525,27cyclictest0-21swapper/117:34:081
2820699524,28cyclictest0-21swapper/117:42:461
28206995220,13cyclictest0-21swapper/118:17:471
28206995220,12cyclictest0-21swapper/118:39:191
2820699515,27cyclictest0-21swapper/118:22:491
126302512,11sleep10-21swapper/118:07:141
2820699412,23cyclictest0-21swapper/117:32:151
2820699412,23cyclictest0-21swapper/117:32:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional