You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 17, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2016721190,3sleep20-21swapper/209:24:122
148782910,6sleep33066099cyclictest09:16:313
60032880,6sleep13065899cyclictest12:30:581
60032880,6sleep13065899cyclictest12:30:581
166412820,3sleep10-21swapper/109:20:001
3065799802,60cyclictest12049-21latency_hist09:55:220
200912800,2sleep10-21swapper/112:08:291
3065799771,48cyclictest20487-21grep10:45:540
3065799765,56cyclictest0-21swapper/010:45:230
80872750,3sleep10-21swapper/107:35:491
3065799759,56cyclictest0-21swapper/009:50:230
3065799759,56cyclictest0-21swapper/009:50:230
184242740,2sleep20-21swapper/208:05:392
184242740,2sleep20-21swapper/208:05:392
30657997356,6cyclictest7950irq/56-dwc_otg_07:10:360
3065799732,59cyclictest3574-21sh11:05:500
3065799732,47cyclictest32573-21sh10:20:430
3065799704,52cyclictest0-21swapper/007:55:400
3065799702,56cyclictest0-21swapper/012:25:490
3065799702,52cyclictest0-21swapper/009:00:440
3065799702,48cyclictest26718-21fschecks_time11:35:400
3065799694,51cyclictest0-21swapper/011:30:360
30657996916,39cyclictest22586-21processes08:15:490
3065799691,58cyclictest0-21swapper/008:55:390
3065799683,53cyclictest0-21swapper/007:35:220
3065799683,52cyclictest0-21swapper/010:02:550
3065799682,54cyclictest0-21swapper/009:25:380
3065799681,55cyclictest9036-21rm12:35:430
3065999678,29cyclictest0-21swapper/211:10:362
3065799671,9cyclictest730-21fschecks_time09:40:410
3065799671,9cyclictest730-21fschecks_time09:40:410
30657996716,47cyclictest114542chrt07:45:430
30657996644,5cyclictest16203-21systemd-journal10:30:220
3065799662,49cyclictest0-21swapper/012:10:490
30657996622,29cyclictest29344-21fschecks_time08:35:420
3065799652,49cyclictest0-21swapper/008:15:220
3065799652,48cyclictest0-21swapper/010:53:330
3065799651,50cyclictest23707-21aten_rbpower_po08:20:380
30657996511,39cyclictest19568-21ssh12:07:260
30660996420,19cyclictest18966-21ssh12:06:143
3065799646,42cyclictest8369-21unixbench_multi07:35:520
3065799642,50cyclictest0-21swapper/010:05:380
3065799642,48cyclictest0-21swapper/011:15:370
3065799642,48cyclictest0-21swapper/011:00:430
3065799642,46cyclictest1180-21runrttasks12:20:290
30657996416,30cyclictest19234-21sh09:22:220
3065799641,53cyclictest12984-21latency_hist09:15:230
3065799641,46cyclictest10484-21ls11:15:240
3065799634,39cyclictest0-21swapper/007:40:310
3065799632,52cyclictest0-21swapper/008:40:500
3065799632,52cyclictest0-21swapper/008:40:500
3065799632,50cyclictest0-21swapper/010:56:070
3065799632,47cyclictest5249-21ls12:30:430
3065799632,47cyclictest5249-21ls12:30:430
3065799632,47cyclictest0-21swapper/009:35:470
3065799631,49cyclictest0-21swapper/009:05:370
3065799631,18cyclictest24711-21latency10:10:440
3065799622,48cyclictest2189-21fschecks_time07:20:410
3065799621,52cyclictest0-21swapper/012:15:390
30660996122,19cyclictest26760-21ssh12:16:083
30660996118,24cyclictest12972-21ssh10:36:343
3065799612,48cyclictest0-21swapper/010:30:530
3065799611,48cyclictest12394-21wget09:55:370
3065799611,42cyclictest5983-21sh11:50:400
3065799611,42cyclictest5983-21sh11:50:400
30657996113,37cyclictest0-21swapper/010:36:220
30660996021,13cyclictest10129-21ls07:40:483
3065799602,49cyclictest0-21swapper/012:00:460
30657996017,32cyclictest4062-21if_err_wlan007:25:410
30657996017,32cyclictest2828-21/usr/sbin/munin11:45:500
127162600,2sleep30-21swapper/311:17:253
3065799595,45cyclictest0-21swapper/010:17:280
3065799592,49cyclictest0-21swapper/008:46:290
3065799591,47cyclictest15286-21ntp_kernel_pll_11:20:460
3065799591,45cyclictest0-21swapper/011:55:450
30660995824,14cyclictest4559-21ls10:25:513
30660995818,22cyclictest28623-21ssh11:37:403
30660995815,13cyclictest22847-21taskset09:26:433
3065799584,43cyclictest0-21swapper/011:45:230
3065799582,48cyclictest0-21swapper/008:54:070
3065799582,45cyclictest0-21swapper/011:30:230
3066099578,30cyclictest31100-21ssh11:00:293
30660995624,15cyclictest5879-21ssh11:09:393
30660995619,13cyclictest7030-21ssh12:33:003
30660995619,13cyclictest7030-21ssh12:33:003
30660995617,22cyclictest7750irq/56-dwc_otg10:57:153
30660995615,14cyclictest6795-21ssh09:47:593
30660995615,14cyclictest6795-21ssh09:47:593
3065799565,43cyclictest0-21swapper/008:04:380
302672563,44sleep00-21swapper/007:08:490
302672563,44sleep00-21swapper/007:08:490
3066099558,30cyclictest0-21swapper/307:35:363
30660995522,13cyclictest790-21ps07:15:483
30660995520,14cyclictest900-21/usr/sbin/munin12:25:383
30660995518,22cyclictest10880-21ssh10:35:133
30660995517,22cyclictest20139-21ssh09:24:093
30660995517,13cyclictest11563-21sed07:45:443
30660995512,27cyclictest20970-21ssh10:46:433
3065799552,42cyclictest0-21swapper/008:28:570
14512550,9sleep1221ktimersoftd/109:40:521
14512550,9sleep1221ktimersoftd/109:40:521
3066099548,30cyclictest0-21swapper/308:00:343
30660995421,21cyclictest27540-21ssh10:15:023
30660995417,14cyclictest9134-21ntp_states09:50:483
3065999547,30cyclictest0-21swapper/208:45:352
3065799548,27cyclictest0-21swapper/009:15:350
3065799542,42cyclictest0-21swapper/008:05:470
3065799542,42cyclictest0-21swapper/008:05:470
3065799542,39cyclictest0-21swapper/009:30:260
293082540,9sleep121-21rcuc/107:05:411
293082540,9sleep121-21rcuc/107:05:411
30660995321,14cyclictest1190-21ssh11:44:513
30660995318,14cyclictest631-21/usr/sbin/munin10:20:513
3065799531,37cyclictest0-21swapper/007:50:390
3066099528,28cyclictest0-21swapper/308:50:373
3066099528,27cyclictest0-21swapper/308:15:363
3066099521,34cyclictest20150-21ssh10:05:293
3065999521,46cyclictest9992-21ls09:10:452
3065799524,10cyclictest0-21swapper/008:30:350
3065799521,4cyclictest531-21latency07:15:420
30660995123,14cyclictest28301-21ssh09:34:463
30660995118,13cyclictest25911-21sh11:35:053
30660995114,25cyclictest13810-21ssh12:00:393
30660995110,12cyclictest7750irq/56-dwc_otg10:00:353
3066099508,27cyclictest0-21swapper/309:35:353
30660995019,9cyclictest30023-21ntp_states12:20:473
30660995015,21cyclictest3198-21chrt07:22:373
30660995015,15cyclictest13760-21taskset09:56:143
30660995015,13cyclictest9283-21ssh09:10:323
3065999509,5cyclictest7950irq/56-dwc_otg_12:05:452
3065999508,30cyclictest25035-21ssh09:30:352
3065999501,25cyclictest0-21swapper/210:35:392
3066099497,26cyclictest0-21swapper/311:10:293
30660994918,13cyclictest4906-21ssh11:49:263
30660994916,14cyclictest26649-21seq08:26:243
30660994913,9cyclictest4005-21munin-run09:45:233
30660994913,9cyclictest4005-21munin-run09:45:233
30660994912,11cyclictest7950irq/56-dwc_otg_07:50:343
3065999497,26cyclictest0-21swapper/208:35:372
3065899497,28cyclictest0-21swapper/111:40:351
35592480,3sleep10-21swapper/108:55:221
30660994817,15cyclictest17869-21ssh11:24:523
30660994817,13cyclictest8295-21ssh11:53:233
30660994817,13cyclictest8295-21ssh11:53:233
30660994812,24cyclictest7950irq/56-dwc_otg_12:15:103
3065999488,28cyclictest0-21swapper/208:00:342
3065999487,27cyclictest0-21swapper/208:15:362
3065899488,24cyclictest0-21swapper/109:10:331
30660994723,13cyclictest27956-21ssh10:15:243
30660994720,18cyclictest7750irq/56-dwc_otg10:41:333
30660994712,13cyclictest26338-21ssh10:54:363
30660994711,16cyclictest7850irq/56-dwc_otg_11:55:513
30660994711,13cyclictest7950irq/56-dwc_otg_09:10:223
3065999478,21cyclictest0-21swapper/210:05:372
3065999478,21cyclictest0-21swapper/208:50:372
30658994733,6cyclictest31677-21idleruntime-cro10:20:221
30658994727,12cyclictest17051-21ssh11:23:151
30658994721,5cyclictest22127-21ssh11:30:231
30658994712,4cyclictest0-21swapper/111:51:061
30658994712,4cyclictest0-21swapper/111:51:061
30660994619,16cyclictest20566-21ssh11:27:273
30660994616,13cyclictest19490-21runrttasks08:07:143
30660994616,13cyclictest19490-21runrttasks08:07:143
30660994615,13cyclictest30492-21taskset08:39:173
30660994613,13cyclictest32289-21taskset08:44:083
30660994613,13cyclictest32289-21taskset08:44:083
3065999469,28cyclictest0-21swapper/207:55:362
3065999468,27cyclictest0-21swapper/210:50:372
3065999468,26cyclictest0-21swapper/207:40:292
30659994627,5cyclictest467950wget12:30:352
30659994627,5cyclictest467950wget12:30:352
30659994620,16cyclictest7950irq/56-dwc_otg_12:15:512
30658994620,13cyclictest31965-21/usr/sbin/munin10:20:431
30658994616,13cyclictest1792-21sh11:45:331
235452461,4sleep123552-21ssh10:09:541
30660994519,13cyclictest5298-21seq08:59:443
30660994517,13cyclictest8974-21diskmemload12:36:333
30660994516,10cyclictest16203-21systemd-journal07:29:233
3065999458,22cyclictest0-21swapper/208:55:352
3065999451,34cyclictest0-21swapper/211:06:212
3066099449,15cyclictest14569-21/usr/sbin/munin07:55:393
3066099443,5cyclictest0-21swapper/309:00:333
30660994410,10cyclictest7950irq/56-dwc_otg_08:21:463
3065999442,31cyclictest0-21swapper/209:55:232
30659994414,19cyclictest301ktimersoftd/209:15:372
30658994425,11cyclictest5435-21ssh11:08:461
3065899442,31cyclictest14543-21switchtime12:00:501
30658994419,13cyclictest28954-21sh09:35:341
30658994417,13cyclictest27703-21cron10:15:221
30658994411,4cyclictest0-21swapper/109:31:441
30660994328,12cyclictest27205-21/usr/sbin/munin08:30:413
3066099431,9cyclictest7850irq/56-dwc_otg_07:30:343
30660994314,9cyclictest20457-21irqstats08:10:433
3065999431,6cyclictest60-21kswapd012:10:332
30658994319,5cyclictest21108-21ssh09:25:351
3065999427,27cyclictest0-21swapper/209:25:352
3065999426,24cyclictest736-21ssh12:25:372
3065999422,9cyclictest7850irq/56-dwc_otg_07:35:352
30659994214,14cyclictest7750irq/56-dwc_otg07:15:462
3065899421,8cyclictest23785-21cpuspeed08:20:391
30658994214,16cyclictest7739-21fschecks_time10:30:411
3065899421,31cyclictest16551-21ls08:00:351
3065999419,16cyclictest7950irq/56-dwc_otg_11:40:402
3065999411,27cyclictest16737-21ls10:00:432
3065899419,18cyclictest32334-21grep11:00:511
3065899416,7cyclictest7850irq/56-dwc_otg_11:35:361
3065899416,13cyclictest23-21ksoftirqd/109:50:331
3065899414,6cyclictest7950irq/56-dwc_otg_08:30:351
30658994116,8cyclictest221ktimersoftd/108:08:591
30658994116,8cyclictest221ktimersoftd/108:08:591
3065899411,31cyclictest17755-21ls09:20:431
30660994010,17cyclictest7950irq/56-dwc_otg_08:47:073
3065999403,20cyclictest29-21rcuc/211:27:412
30659994019,7cyclictest18837-2110-uname10:45:212
3065999401,26cyclictest2858-21ls07:20:522
3065899403,19cyclictest9260-21aten_rbpower_po07:40:341
3065899401,6cyclictest7950irq/56-dwc_otg_12:40:221
3065899401,28cyclictest5263-21ls07:30:231
3065899401,27cyclictest10317-21ssh11:15:231
30658994012,15cyclictest0-21swapper/109:55:531
3065999395,15cyclictest7950irq/56-dwc_otg_09:55:372
3065999394,15cyclictest29-21rcuc/212:20:422
30659993916,7cyclictest32541-21cat10:20:432
3065999391,29cyclictest20529-21grep10:45:542
30659993910,15cyclictest301ktimersoftd/210:25:512
3065899396,8cyclictest7850irq/56-dwc_otg_11:15:401
30658993916,15cyclictest19602-21ssh10:45:421
3065899391,25cyclictest22427-21ls12:10:511
30658993911,12cyclictest31921-21unixbench_multi08:40:531
30658993911,12cyclictest31921-21unixbench_multi08:40:531
3065999389,18cyclictest0-21swapper/210:15:362
3065999388,18cyclictest0-21swapper/211:00:292
3065999387,17cyclictest0-21swapper/211:55:342
3065999387,14cyclictest301ktimersoftd/211:25:152
3065999381,5cyclictest0-21swapper/209:40:562
3065999381,5cyclictest0-21swapper/209:40:562
30659993813,15cyclictest57042sleep211:50:342
30659993813,15cyclictest57042sleep211:50:342
3065999381,26cyclictest29522-21latency09:35:432
3065999381,26cyclictest0-21swapper/207:50:342
3065899389,13cyclictest0-21swapper/110:51:111
3065899388,15cyclictest22799-21fschecks_count11:30:401
3065899387,7cyclictest7950irq/56-dwc_otg_12:25:531
30658993817,14cyclictest1364-21date07:17:221
291492380,3sleep20-21swapper/207:05:382
291492380,3sleep20-21swapper/207:05:382
3065999379,16cyclictest0-21swapper/211:45:512
3065999378,17cyclictest0-21swapper/210:10:372
3065999377,18cyclictest0-21swapper/209:00:352
3065999375,6cyclictest31-21ksoftirqd/211:17:402
3065999374,15cyclictest7750irq/56-dwc_otg12:39:542
3065999371,8cyclictest27012-21systemd-cgroups10:55:272
3065999371,18cyclictest4878-21ssh09:45:422
3065999371,18cyclictest4878-21ssh09:45:422
3065899379,6cyclictest0-21swapper/111:59:451
3065899373,25cyclictest0-21swapper/112:22:461
30658993720,6cyclictest7250-21ssh09:48:541
30658993720,6cyclictest7250-21ssh09:48:541
30658993716,13cyclictest0-21swapper/110:38:591
30658993714,11cyclictest0-21swapper/110:29:501
30658993711,13cyclictest0-21swapper/107:15:221
30658993710,8cyclictest30735-21munin-run11:00:221
30658993710,4cyclictest0-21swapper/110:00:561
30658993710,13cyclictest9093-21latency_hist09:10:231
3066099369,15cyclictest0-21swapper/307:13:243
3065999366,19cyclictest25655-21fschecks_count08:25:402
3065999362,5cyclictest0-21swapper/211:36:472
3065999362,18cyclictest0-21swapper/210:34:062
3065999361,19cyclictest7850irq/56-dwc_otg_12:04:092
3065899366,8cyclictest7850irq/56-dwc_otg_07:30:481
3065899366,6cyclictest7850irq/56-dwc_otg_08:35:431
3065899366,10cyclictest7950irq/56-dwc_otg_07:20:441
3065899362,10cyclictest6196-21/usr/sbin/munin09:00:441
3065899361,8cyclictest25678-21uniq12:15:421
3065899361,25cyclictest0-21swapper/110:42:091
3065899360,26cyclictest10969-21/usr/sbin/munin07:45:431
224292360,6sleep23065999cyclictest11:30:302
3065999354,5cyclictest4267-21ls07:25:442
3065999352,9cyclictest0-21swapper/208:10:342
30659993513,10cyclictest301ktimersoftd/208:30:232
3065899352,12cyclictest4149-21fschecks_time08:55:401
3065899352,11cyclictest7950irq/56-dwc_otg_08:10:371
30658993519,5cyclictest1180-21runrttasks07:55:421
3065999344,11cyclictest29-21rcuc/209:10:092
3065999342,23cyclictest0-21swapper/207:12:022
30659993416,10cyclictest23718-21aten_rbpower_vo08:20:392
30659993415,9cyclictest7228-21latency_hist07:35:232
30659993413,5cyclictest12539-21latency_hist07:50:232
3065999341,15cyclictest31-21ksoftirqd/208:41:352
3065999341,15cyclictest31-21ksoftirqd/208:41:352
136332340,2sleep10-21swapper/107:50:471
3065899335,8cyclictest22865-21grep08:15:521
3065899331,8cyclictest1761-21idleruntime-cro08:50:221
30658993210,12cyclictest27076-21latency_hist08:30:221
289302304,13sleep30-21swapper/307:05:363
289302304,13sleep30-21swapper/307:05:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional