You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 09:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Jul 22, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4438991602,143cyclictest0-21swapper/023:00:390
4438991312,100cyclictest23954-21/usr/sbin/munin21:20:460
4438991307,108cyclictest0-21swapper/019:50:330
4438991233,99cyclictest0-21swapper/022:35:400
4438991162,97cyclictest20170-21ssh21:55:220
44389911057,24cyclictest0-21swapper/022:50:480
4438991065,79cyclictest0-21swapper/022:40:350
44389910616,13cyclictest7850irq/56-dwc_otg22:10:520
443899973,67cyclictest0-21swapper/021:40:460
443899955,55cyclictest0-21swapper/022:45:480
4438999128,5cyclictest32615-21fschecks_time00:10:400
4438998823,46cyclictest9-21ksoftirqd/021:30:470
444199873,11cyclictest1773-21ssh22:10:523
443899877,67cyclictest0-21swapper/022:20:350
443899877,67cyclictest0-21swapper/022:20:350
4438998727,6cyclictest7950irq/56-dwc_otg_23:20:520
252692870,5sleep225233-21ssh21:20:542
443899853,62cyclictest0-21swapper/021:55:240
443899853,62cyclictest0-21swapper/021:55:240
443899841,35cyclictest13852-21sed23:45:530
443899841,35cyclictest13852-21sed23:45:530
30642840,3sleep20-21swapper/223:32:512
443899822,37cyclictest0-21swapper/023:09:530
4440998022,36cyclictest0-21swapper/222:40:362
443899802,39cyclictest0-21swapper/023:15:320
443899801,36cyclictest0-21swapper/000:25:510
443899801,36cyclictest0-21swapper/000:25:510
443899772,31cyclictest0-21swapper/020:50:240
443899772,31cyclictest0-21swapper/020:50:240
4438997721,16cyclictest0-21swapper/023:51:390
443899771,35cyclictest0-21swapper/020:50:230
4440997614,17cyclictest9573-21aten_rbpower_en23:00:392
443899762,34cyclictest6479-21timerwakeupswit22:55:510
443899761,63cyclictest0-21swapper/023:35:400
444199759,45cyclictest7950irq/56-dwc_otg_21:40:463
443899752,36cyclictest0-21swapper/023:40:570
444199742,49cyclictest7850irq/56-dwc_otg23:00:393
443899742,36cyclictest0-21swapper/000:21:510
444099728,47cyclictest0-21swapper/220:45:352
444199709,27cyclictest26522-21runrttasks22:40:513
443899701,48cyclictest0-21swapper/023:25:430
443899701,48cyclictest0-21swapper/023:25:430
444199694,40cyclictest0-21swapper/322:50:473
443899692,35cyclictest0-21swapper/000:16:260
444199683,38cyclictest0-21swapper/321:20:473
444099675,39cyclictest7950irq/56-dwc_otg_22:35:412
4438996711,10cyclictest0-21swapper/020:15:340
444099663,33cyclictest7850irq/56-dwc_otg22:45:482
4439996612,26cyclictest8050irq/56-dwc_otg_23:00:351
444199654,40cyclictest0-21swapper/322:45:453
444199652,55cyclictest20276-21ssh21:55:233
41272643,15sleep10-21swapper/119:09:511
443999637,22cyclictest7950irq/56-dwc_otg_22:35:401
4438996220,15cyclictest0-21swapper/000:00:210
198232620,4sleep219821-21diskstats00:35:382
444099600,25cyclictest2050-21ssh22:50:472
443999603,14cyclictest0-21swapper/122:45:471
4439996012,26cyclictest0-21swapper/122:40:341
178542600,3sleep017856-21sh00:32:500
444099596,31cyclictest7950irq/56-dwc_otg_21:40:442
4439995934,17cyclictest8730-21ssh21:40:461
443899593,47cyclictest13-21rcuc/023:13:370
4438995914,28cyclictest27274-21unixbench_singl20:10:510
4440995813,39cyclictest20260-21ls21:55:222
443999587,32cyclictest8050irq/56-dwc_otg_22:50:481
443999582,30cyclictest8050irq/56-dwc_otg_21:30:441
4439995813,9cyclictest7850irq/56-dwc_otg21:20:481
4438995817,7cyclictest0-21swapper/021:25:350
4438995816,21cyclictest28853-21/usr/sbin/munin00:05:430
443899581,46cyclictest1033-21/usr/sbin/munin23:30:390
443899572,30cyclictest13005-21unixbench_singl21:00:500
444199564,37cyclictest0-21swapper/321:55:243
444199564,37cyclictest0-21swapper/321:55:243
4439995625,16cyclictest7950irq/56-dwc_otg_21:55:231
443899565,38cyclictest24750-21fschecks_time00:00:410
444099554,35cyclictest0-21swapper/222:10:522
4438995536,7cyclictest28773-21acpi22:05:350
443899552,37cyclictest31824-21fschecks_count20:25:410
443899551,40cyclictest25068-21/usr/sbin/munin22:00:410
443899551,35cyclictest20699-21zcat21:15:460
444099543,28cyclictest8050irq/56-dwc_otg_21:30:452
444199536,7cyclictest0-21swapper/323:30:323
444199536,27cyclictest37-21rcuc/322:35:393
4438995332,5cyclictest101ktimersoftd/019:20:320
175542530,2sleep20-21swapper/219:45:422
443899522,38cyclictest0-21swapper/019:48:210
443899522,35cyclictest16130-21rm21:10:420
4438995219,5cyclictest20100-21grep00:35:420
4438995214,20cyclictest29981-21fschecks_count20:20:390
443899521,38cyclictest0-21swapper/022:27:400
4441995125,6cyclictest381ktimersoftd/323:15:443
444099519,31cyclictest0-21swapper/223:15:382
444099519,28cyclictest0-21swapper/223:35:282
444099518,28cyclictest0-21swapper/200:05:362
4438995114,15cyclictest15318-21diskmemload22:15:370
444199503,26cyclictest0-21swapper/321:30:443
444099506,8cyclictest8050irq/56-dwc_otg_19:15:352
443999502,41cyclictest7950irq/56-dwc_otg_21:55:241
443999502,41cyclictest7950irq/56-dwc_otg_21:55:241
4439995016,26cyclictest7950irq/56-dwc_otg_22:10:511
4438995025,6cyclictest13997-21cut21:05:410
4438995025,3cyclictest0-21swapper/020:40:210
4438995023,4cyclictest5632-21unixbench_multi20:40:540
443899501,39cyclictest0-21swapper/020:34:540
443899501,39cyclictest0-21swapper/020:34:540
443899501,38cyclictest13604-21ssh21:45:590
443899501,36cyclictest0-21swapper/019:16:120
444199491,15cyclictest0-21swapper/320:15:343
444099499,25cyclictest0-21swapper/223:45:362
444099499,25cyclictest0-21swapper/223:45:362
4438994932,5cyclictest7850irq/56-dwc_otg21:36:460
443899491,41cyclictest0-21swapper/019:30:040
443899491,38cyclictest0-21swapper/019:55:370
443899491,27cyclictest11781-21aten_rbpower_po19:30:350
258762490,5sleep125878-21unixbench_singl22:00:511
444199487,26cyclictest0-21swapper/319:55:363
444099488,26cyclictest0-21swapper/223:20:362
443899481,33cyclictest15338-21basename19:40:350
443899481,33cyclictest15338-21basename19:40:350
101652480,9sleep2301ktimersoftd/223:41:092
4438994723,5cyclictest17255-21aten_rbpower_en22:30:370
4438994723,5cyclictest17255-21aten_rbpower_en22:30:370
443899471,33cyclictest0-21swapper/020:57:050
443899471,31cyclictest0-21swapper/020:06:270
443899471,23cyclictest516-21in:imuxsock20:05:210
444099467,24cyclictest0-21swapper/221:05:272
444099467,24cyclictest0-21swapper/219:20:342
40992463,35sleep00-21swapper/019:09:300
322302460,2sleep20-21swapper/222:09:462
4441994521,6cyclictest727-21runrttasks00:37:223
444099458,26cyclictest0-21swapper/200:10:352
444099457,23cyclictest0-21swapper/220:10:362
444099457,23cyclictest0-21swapper/220:05:352
444099452,9cyclictest0-21swapper/220:55:342
4439994518,5cyclictest15542-21fschecks_count19:40:391
4439994518,5cyclictest15542-21fschecks_count19:40:391
443999451,27cyclictest15906-21ls21:10:381
443899452,29cyclictest0-21swapper/019:35:400
4441994419,6cyclictest12878-21sshd23:45:393
4441994419,6cyclictest12878-21sshd23:45:393
444099441,35cyclictest8050irq/56-dwc_otg_19:50:452
443899442,33cyclictest0-21swapper/019:10:450
444199438,24cyclictest0-21swapper/300:25:373
444199438,24cyclictest0-21swapper/300:25:373
444199437,28cyclictest37-21rcuc/323:40:353
4440994315,16cyclictest7950irq/56-dwc_otg_00:34:272
4440994314,18cyclictest4437-21cyclictest20:40:352
444099431,28cyclictest20950-21cut19:55:402
443999431,10cyclictest16898-21fschecks_time23:50:411
38812433,31sleep20-21swapper/219:06:452
444199427,24cyclictest0-21swapper/300:00:343
4441994216,5cyclictest15835-21ssh00:30:393
444099421,29cyclictest14583-21ls23:05:502
4439994219,5cyclictest20073-21fschecks_count21:15:381
444199415,26cyclictest0-21swapper/320:55:363
444199411,28cyclictest21506-21ls23:55:503
444099419,9cyclictest31-21ksoftirqd/223:15:222
444099417,25cyclictest0-21swapper/220:20:352
444099416,15cyclictest8050irq/56-dwc_otg_21:55:242
444099416,15cyclictest8050irq/56-dwc_otg_21:55:242
4440994112,12cyclictest301ktimersoftd/200:15:542
4440994112,10cyclictest7850irq/56-dwc_otg22:15:332
443999411,27cyclictest20540-21ls00:35:491
444199407,23cyclictest0-21swapper/321:25:373
444099407,11cyclictest301ktimersoftd/221:50:212
444099406,6cyclictest8050irq/56-dwc_otg_00:05:222
444099406,4cyclictest8050irq/56-dwc_otg_22:00:332
444099406,19cyclictest7950irq/56-dwc_otg_22:25:502
4441993917,5cyclictest727-21runrttasks22:25:273
444199391,27cyclictest32536-21uniq00:10:393
444099398,15cyclictest7950irq/56-dwc_otg_23:50:532
444099397,15cyclictest7950irq/56-dwc_otg_23:25:432
444099397,15cyclictest7950irq/56-dwc_otg_23:25:432
444099395,17cyclictest8050irq/56-dwc_otg_20:55:222
444099395,17cyclictest8050irq/56-dwc_otg_20:55:222
4440993918,12cyclictest8133-21if_err_wlan000:20:412
444199382,19cyclictest0-21swapper/300:10:223
444199381,25cyclictest6444-21ls19:15:393
444199381,18cyclictest28956-21grep23:25:223
4441993811,7cyclictest8050irq/56-dwc_otg_00:20:513
444099389,21cyclictest13154-21rm00:26:192
444099389,21cyclictest13154-21rm00:26:192
444099383,23cyclictest0-21swapper/220:15:362
444099383,11cyclictest7850irq/56-dwc_otg21:00:412
4440993818,4cyclictest4102-21seq20:38:132
4440993814,10cyclictest301ktimersoftd/221:25:392
444099381,3cyclictest0-21swapper/221:15:332
4440993810,14cyclictest301ktimersoftd/222:32:382
4440993810,14cyclictest301ktimersoftd/222:32:382
443999389,16cyclictest8050irq/56-dwc_otg_23:30:381
443999387,20cyclictest27996-21modprobe00:05:221
4439993815,14cyclictest221ktimersoftd/123:38:161
4439993810,15cyclictest25875-21ssh23:20:431
444099375,20cyclictest3962-21ssh21:35:362
443999375,5cyclictest8050irq/56-dwc_otg_22:20:321
443999375,5cyclictest8050irq/56-dwc_otg_22:20:321
443999373,16cyclictest8050irq/56-dwc_otg_23:55:361
4439993716,6cyclictest13252-21sh23:45:451
4439993716,6cyclictest13252-21sh23:45:451
443999371,24cyclictest13760-21taskset23:05:371
4439993711,12cyclictest0-21swapper/100:15:511
184082370,3sleep118398-21ssh23:10:501
101052370,2sleep30-21swapper/319:25:383
4441993625,8cyclictest381ktimersoftd/323:25:463
4441993625,8cyclictest381ktimersoftd/323:25:463
4441993620,11cyclictest16658-21aten_rbpower_po23:50:373
4441993619,7cyclictest510750ssh00:15:583
444099368,18cyclictest0-21swapper/222:23:242
444099368,18cyclictest0-21swapper/222:23:242
444099367,20cyclictest0-21swapper/219:10:352
443999366,15cyclictest8050irq/56-dwc_otg_23:20:221
4439993618,6cyclictest4105-21cpu21:35:381
443999361,23cyclictest12744-21ssh00:25:521
443999361,23cyclictest12744-21ssh00:25:521
443999361,21cyclictest7850irq/56-dwc_otg19:15:421
4441993517,6cyclictest830850ssh23:40:163
4441993512,4cyclictest7022-21sh22:56:293
444199351,23cyclictest0-21swapper/319:50:333
4441993511,8cyclictest39-21ksoftirqd/323:05:373
444099357,7cyclictest31-21ksoftirqd/219:25:482
444099354,19cyclictest0-21swapper/223:55:352
444099353,13cyclictest8050irq/56-dwc_otg_20:25:412
444099351,7cyclictest0-21swapper/222:55:352
444099351,13cyclictest0-21swapper/221:11:262
443999359,14cyclictest0-21swapper/121:05:351
443999356,7cyclictest8132-21ssh00:20:411
443999356,14cyclictest8050irq/56-dwc_otg_00:30:361
443999354,7cyclictest7850irq/56-dwc_otg19:45:371
443999353,21cyclictest0-21swapper/123:25:311
443999353,21cyclictest0-21swapper/123:25:311
443999353,11cyclictest0-21swapper/123:40:321
443999351,8cyclictest28675-21ssh00:05:411
4439993511,5cyclictest0-21swapper/100:14:081
234302350,2sleep30-21swapper/320:00:483
444199346,10cyclictest8050irq/56-dwc_otg_22:08:253
444199345,20cyclictest15703-21ssh21:10:373
4441993418,8cyclictest203202sleep323:13:553
444199341,22cyclictest0-21swapper/321:45:363
444199341,19cyclictest8377-21uniq19:20:383
444099347,15cyclictest0-21swapper/220:30:352
444099347,15cyclictest0-21swapper/220:30:352
443999348,11cyclictest8050irq/56-dwc_otg_19:20:381
443999347,6cyclictest4854-21ssh22:15:391
443999347,12cyclictest8050irq/56-dwc_otg_19:35:501
443999346,15cyclictest8050irq/56-dwc_otg_20:45:211
443999343,18cyclictest24311-21/usr/sbin/munin20:05:361
208342340,5sleep1443999cyclictest22:35:101
208342340,5sleep1443999cyclictest22:35:101
444199336,15cyclictest0-21swapper/321:05:273
444099331,22cyclictest24079-21latency_hist20:05:222
443999336,14cyclictest8050irq/56-dwc_otg_19:10:361
443999333,5cyclictest7850irq/56-dwc_otg20:55:371
443999331,25cyclictest21070-21ls19:55:411
443999331,24cyclictest30812-21rm21:28:241
444199328,4cyclictest0-21swapper/322:20:513
444199328,4cyclictest0-21swapper/322:20:513
444199327,21cyclictest0-21swapper/320:20:353
444199326,11cyclictest7950irq/56-dwc_otg_22:20:223
444199323,9cyclictest8050irq/56-dwc_otg_20:30:403
444199323,9cyclictest8050irq/56-dwc_otg_20:30:403
4441993217,4cyclictest26335-21sh20:10:383
443999326,5cyclictest8050irq/56-dwc_otg_21:45:401
443999325,13cyclictest8050irq/56-dwc_otg_20:35:371
443999325,13cyclictest8050irq/56-dwc_otg_20:00:371
443999321,6cyclictest32678-21sed20:25:521
443999321,15cyclictest31173-21ssh22:07:511
444199318,5cyclictest0-21swapper/322:30:363
444199318,5cyclictest0-21swapper/322:30:363
444199314,21cyclictest8050irq/56-dwc_otg_20:40:353
444199311,18cyclictest7950irq/56-dwc_otg_20:25:493
443999316,12cyclictest8050irq/56-dwc_otg_22:25:371
443999315,13cyclictest0-21swapper/120:15:381
443999313,8cyclictest9795-21modprobe20:55:221
443999313,8cyclictest9795-21modprobe20:55:221
443999313,8cyclictest6692-21fschecks_count20:45:391
443999312,5cyclictest7850irq/56-dwc_otg22:55:401
443999312,12cyclictest7850irq/56-dwc_otg20:30:441
443999312,12cyclictest7850irq/56-dwc_otg20:30:441
443999311,23cyclictest12843-21ls21:00:491
443999311,19cyclictest0-21swapper/119:50:351
4439993111,4cyclictest9965-21aten_rbpower_po19:25:371
444199307,5cyclictest7950irq/56-dwc_otg_19:30:383
444199302,12cyclictest8050irq/56-dwc_otg_20:50:403
444199302,12cyclictest8050irq/56-dwc_otg_20:50:403
4441993012,4cyclictest0-21swapper/321:18:293
4441993011,5cyclictest13004-21awk21:00:503
4441993010,6cyclictest7950irq/56-dwc_otg_21:37:243
443999304,7cyclictest26814-21tr20:10:451
443999303,5cyclictest7850irq/56-dwc_otg20:25:221
4439993011,7cyclictest0-21swapper/119:30:381
444099292,12cyclictest8050irq/56-dwc_otg_19:35:482
444099291,7cyclictest15333-21aten_rbpower_cu19:40:352
444099291,7cyclictest15333-21aten_rbpower_cu19:40:352
248422290,4sleep324841-21df22:00:373
444199287,18cyclictest0-21swapper/319:45:363
4441992819,5cyclictest6803-21if_err_eth020:45:403
444199281,14cyclictest8050irq/56-dwc_otg_20:05:443
444099281,17cyclictest0-21swapper/219:30:372
38772283,15sleep30-21swapper/319:06:423
444199279,9cyclictest727-21runrttasks19:41:203
444199279,9cyclictest727-21runrttasks19:41:203
444199275,4cyclictest8050irq/56-dwc_otg_19:35:503
444199261,15cyclictest0-21swapper/319:10:393
444199252,7cyclictest37-21rcuc/320:35:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional