You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 14:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat Jul 04, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1004021190,2sleep00-21swapper/000:09:380
165821100,2sleep10-21swapper/100:38:061
165821100,2sleep10-21swapper/100:38:061
180192930,4sleep218013-21ssh23:00:312
180192930,4sleep218013-21ssh23:00:312
11559999032,47cyclictest9-21ksoftirqd/000:35:220
11559999032,47cyclictest9-21ksoftirqd/000:35:220
1155999895,81cyclictest11-21rcu_preempt23:05:230
1155999827,23cyclictest9-21ksoftirqd/020:10:240
11559997723,16cyclictest0-21swapper/022:00:190
11559997613,9cyclictest7950irq/56-dwc_otg_23:20:250
162962750,3sleep121-21rcuc/122:58:491
1155999744,67cyclictest0-21swapper/021:25:230
195672720,2sleep21156799cyclictest23:40:412
195672720,2sleep21156799cyclictest23:40:412
1156899723,47cyclictest7850irq/56-dwc_otg00:00:253
305502690,1sleep30-21swapper/322:36:083
1156899691,53cyclictest22282-21ssh22:26:103
1156799699,44cyclictest0-21swapper/223:31:282
1156799691,7cyclictest12117-21grep19:10:322
11559996811,30cyclictest0-21swapper/021:50:230
1156899676,46cyclictest8050irq/56-dwc_otg_21:26:203
1156899674,51cyclictest7950irq/56-dwc_otg_20:30:183
1156799675,5cyclictest9650irq/80-mmc023:20:132
1156799674,5cyclictest7950irq/56-dwc_otg_21:40:262
11567996710,40cyclictest0-21swapper/200:15:352
210852660,2sleep10-21swapper/119:35:311
300602650,2sleep10-21swapper/120:00:271
11568996513,36cyclictest11624-21aten_rbpower_cu19:10:263
1155999651,52cyclictest0-21swapper/019:50:220
1156899647,41cyclictest8050irq/56-dwc_otg_21:40:243
1156899641,48cyclictest12445-21ssh00:11:263
1156799646,46cyclictest0-21swapper/222:00:352
1156799642,45cyclictest31651-21/usr/sbin/munin20:05:322
1156799642,45cyclictest31651-21/usr/sbin/munin20:05:322
1156799641,46cyclictest8050irq/56-dwc_otg_20:50:122
1155999642,52cyclictest0-21swapper/020:55:230
1156899638,43cyclictest24222-21sed00:25:403
1155999631,45cyclictest24839-21rm22:30:230
1156899621,50cyclictest28094-21ssh22:34:123
1156799621,50cyclictest6848-21grep00:05:292
11567996210,32cyclictest17380-21/usr/sbin/munin20:55:412
1155999621,45cyclictest20334-21cron22:25:120
1156899612,49cyclictest11765-21ssh23:31:313
1156899612,46cyclictest24624-21ssh23:08:523
1156899611,49cyclictest23965-21ssh23:46:083
1156799613,4cyclictest7850irq/56-dwc_otg22:35:332
11559996136,14cyclictest0-21swapper/023:35:120
1155999611,40cyclictest0-21swapper/000:30:240
1156899603,47cyclictest0-21swapper/300:20:493
1156899603,36cyclictest7850irq/56-dwc_otg21:35:303
1156899603,36cyclictest7850irq/56-dwc_otg21:35:303
1156899601,41cyclictest0-21swapper/300:08:533
11568996010,35cyclictest17418-21basename23:00:233
11568996010,35cyclictest17418-21basename23:00:233
11567996010,35cyclictest0-21swapper/222:45:272
1156399602,15cyclictest0-21swapper/119:20:241
1155999602,47cyclictest0-21swapper/022:10:290
1155999602,44cyclictest0-21swapper/000:11:060
1156899597,40cyclictest0-21swapper/321:13:133
1156899597,39cyclictest0-21swapper/322:20:293
1156899594,36cyclictest0-21swapper/300:36:463
1156899594,36cyclictest0-21swapper/300:36:463
1156799596,36cyclictest7950irq/56-dwc_otg_20:30:392
11559995938,11cyclictest7850irq/56-dwc_otg22:50:330
11559995932,16cyclictest0-21swapper/023:25:280
1155999591,42cyclictest13349-21aten_rbpower_po22:55:240
11559995914,22cyclictest32226-21missed_timers20:05:320
11559995914,22cyclictest32226-21missed_timers20:05:320
1155999591,37cyclictest27136-21latency_hist00:30:130
1156899586,42cyclictest8050irq/56-dwc_otg_22:48:353
1156899583,40cyclictest7850irq/56-dwc_otg23:16:473
1156899581,6cyclictest28228-21/usr/sbin/munin21:55:283
1156899581,42cyclictest25770-21seq19:48:423
1156899581,40cyclictest17112-21rm00:17:233
1156799589,36cyclictest7850irq/56-dwc_otg00:25:412
1156799589,25cyclictest0-21swapper/221:12:072
1156799586,39cyclictest8050irq/56-dwc_otg_21:15:312
1156799582,43cyclictest0-21swapper/221:32:192
1156799581,9cyclictest0-21swapper/222:20:312
1156799581,3cyclictest7850irq/56-dwc_otg23:48:412
1156799581,10cyclictest7850irq/56-dwc_otg22:17:412
1156799581,10cyclictest7850irq/56-dwc_otg22:17:412
1156899574,44cyclictest7950irq/56-dwc_otg_21:46:003
1156899574,43cyclictest8050irq/56-dwc_otg_22:03:063
1156899571,5cyclictest11645-21ssh22:52:543
1156799571,8cyclictest7950irq/56-dwc_otg_23:37:372
1156799571,6cyclictest22846-21ssh21:48:492
11563995717,8cyclictest23-21ksoftirqd/100:25:241
1155999572,41cyclictest29596-21ssh23:15:130
1155999572,33cyclictest28253-21cut21:55:280
1155999571,46cyclictest0-21swapper/021:07:460
1156899565,40cyclictest7950irq/56-dwc_otg_19:32:163
1156899562,3cyclictest8050irq/56-dwc_otg_20:40:263
1156899561,6cyclictest11908-21ssh22:15:003
1156899561,4cyclictest7950irq/56-dwc_otg_21:54:353
1156899561,4cyclictest7850irq/56-dwc_otg21:21:053
1156899561,40cyclictest7950irq/56-dwc_otg_23:20:253
1156899561,40cyclictest0-21swapper/321:15:253
1156799568,44cyclictest3501-21grep20:15:332
1156799568,31cyclictest0-21swapper/223:27:052
1156799562,6cyclictest8050irq/56-dwc_otg_19:35:252
1156799562,41cyclictest9103-21ssh22:50:242
1156799561,6cyclictest8050irq/56-dwc_otg_00:00:282
11559995639,6cyclictest7950irq/56-dwc_otg_00:15:410
11559995637,5cyclictest0-21swapper/019:40:400
1155999563,44cyclictest15701-21ssh21:40:260
11559995627,17cyclictest727-21runrttasks22:15:170
11559995627,17cyclictest727-21runrttasks22:15:170
1155999561,43cyclictest0-21swapper/022:49:230
1156899556,30cyclictest0-21swapper/300:31:483
1156899555,38cyclictest8050irq/56-dwc_otg_22:40:243
1156899553,43cyclictest8050irq/56-dwc_otg_23:51:313
1156899552,41cyclictest8050irq/56-dwc_otg_23:11:273
1156899551,4cyclictest28613-21memory19:55:313
1156799557,29cyclictest7950irq/56-dwc_otg_00:40:122
1156799557,29cyclictest7950irq/56-dwc_otg_00:40:122
1156799555,40cyclictest8050irq/56-dwc_otg_21:29:312
1156799552,4cyclictest7950irq/56-dwc_otg_00:34:302
1156799551,41cyclictest28398-21ssh23:51:392
1155999551,42cyclictest0-21swapper/023:44:020
1155999551,42cyclictest0-21swapper/023:44:020
1155999551,39cyclictest0-21swapper/023:50:360
1155999551,34cyclictest0-21swapper/000:00:120
1156899541,4cyclictest13499-21sed22:15:373
1156899541,4cyclictest13499-21sed22:15:373
1156899541,40cyclictest0-21swapper/323:26:353
1156799547,3cyclictest0-21swapper/200:10:532
1156799547,37cyclictest8050irq/56-dwc_otg_22:40:232
1156799545,3cyclictest7950irq/56-dwc_otg_00:23:122
1156799545,38cyclictest8050irq/56-dwc_otg_19:51:502
1156799542,7cyclictest1691-21sh23:20:252
1156799541,6cyclictest8050irq/56-dwc_otg_23:57:372
1156799541,6cyclictest22593-21aten_rbpower_en19:40:262
1156799541,44cyclictest0-21swapper/221:21:482
11559995431,7cyclictest3122-21fschecks_time20:15:290
11559995428,16cyclictest0-21swapper/022:25:290
11559995427,13cyclictest28574-21grep21:15:400
1155999542,42cyclictest0-21swapper/021:23:440
1155999542,39cyclictest7296-21/usr/sbin/munin21:30:370
1155999542,38cyclictest20148-21tune2fs21:45:280
1155999542,33cyclictest0-21swapper/023:15:270
1155999541,41cyclictest3573-21grep00:00:390
31282530,5sleep13123-21ssh00:00:331
1156899538,40cyclictest21138-21aten_rbpower_en21:05:263
1156899531,5cyclictest18476-21grep20:55:403
1156899531,4cyclictest13602-21perl22:55:283
1156799536,41cyclictest8050irq/56-dwc_otg_22:12:002
1156799532,5cyclictest24674-21ssh23:08:572
1156799532,5cyclictest12885-21taskset21:35:492
1156799532,5cyclictest12885-21taskset21:35:492
1156799532,37cyclictest0-21swapper/222:33:412
1156799531,9cyclictest16968-21ssh23:00:032
1156799531,7cyclictest750-21snmpd19:26:482
1156799531,6cyclictest30378-21taskset21:57:242
1155999533,32cyclictest0-21swapper/019:45:250
11559995326,15cyclictest0-21swapper/019:15:400
11559995321,14cyclictest8404-21wget20:30:250
1156899522,37cyclictest0-21swapper/320:05:303
1156899522,37cyclictest0-21swapper/320:05:303
11568995217,32cyclictest14350-21ssh23:35:263
1156899521,5cyclictest11192-21sed21:35:133
1156899521,4cyclictest1029-21apt-get20:10:233
1156899521,47cyclictest3857-21ls20:15:383
1156899521,3cyclictest7950irq/56-dwc_otg_23:40:283
1156899521,3cyclictest7950irq/56-dwc_otg_23:40:283
1156899521,38cyclictest0-21swapper/319:40:263
1156799527,30cyclictest0-21swapper/223:14:132
1156799524,4cyclictest7850irq/56-dwc_otg21:50:392
1156799521,9cyclictest0-21swapper/222:28:162
1156799521,8cyclictest0-21swapper/222:08:472
11559995227,8cyclictest20566-21latency_hist19:35:130
11559995223,14cyclictest4589-21cron22:45:120
1155999521,38cyclictest0-21swapper/021:35:390
1155999521,38cyclictest0-21swapper/021:35:390
1155999521,35cyclictest12109-21tr20:40:270
1155999521,13cyclictest0-21swapper/022:37:380
11568995110,30cyclictest0-21swapper/322:07:463
11559995129,9cyclictest18672-21sh23:00:380
11559995129,9cyclictest18672-21sh23:00:380
11559995127,14cyclictest79582chrt22:10:120
1155999511,42cyclictest727-21runrttasks23:38:580
1155999511,37cyclictest0-21swapper/021:10:300
1155999511,36cyclictest22191-21cron19:40:120
110662512,44sleep00-21swapper/019:07:110
110662512,44sleep00-21swapper/019:07:110
154902500,3sleep10-21swapper/122:18:551
154902500,3sleep10-21swapper/122:18:551
1156899505,36cyclictest8050irq/56-dwc_otg_20:27:293
1156899501,5cyclictest1652-21ssh23:59:293
1156899501,3cyclictest0-21swapper/319:54:123
1156799501,8cyclictest21535-21if_wlan021:05:312
1156799501,6cyclictest30139-21cut20:00:292
11567995011,36cyclictest15287-21aten_rbpower_po19:20:262
1156399507,23cyclictest61-21kswapd021:50:231
1155999502,30cyclictest23256-21ssh23:45:370
11559995022,16cyclictest10693-21latency20:35:320
11559995022,16cyclictest10693-21latency20:35:320
11559995020,6cyclictest9650irq/80-mmc020:30:130
1156899492,43cyclictest17328-21fschecks_time19:25:283
11568994917,30cyclictest15208-21/usr/sbin/munin19:20:273
1156799495,41cyclictest7532-21grep20:25:392
1156799492,5cyclictest0-21swapper/220:55:132
1156799491,7cyclictest15007-21latency_hist19:20:122
1156799491,6cyclictest0-21swapper/219:34:592
1156799491,3cyclictest0-21swapper/220:41:472
1155999492,35cyclictest125-21systemd-journal20:00:330
1155999492,30cyclictest0-21swapper/020:50:330
1156899482,6cyclictest30054-21cpuspeed20:00:263
1156899481,5cyclictest4865-21/usr/sbin/munin20:20:273
1156899481,4cyclictest0-21swapper/319:35:193
1156799482,5cyclictest0-21swapper/220:20:372
1156799481,7cyclictest19626-21fschecks_time21:00:292
1156799481,6cyclictest1116-21aten_rbpower_po20:10:262
1156799481,3cyclictest0-21swapper/220:35:512
1156799481,3cyclictest0-21swapper/220:35:512
11567994812,33cyclictest28518-21uname19:55:312
1156399481,10cyclictest0-21swapper/119:30:231
1156899473,40cyclictest15649-21aten_rbpower_en20:50:263
1156899472,42cyclictest15092-21latency_hist19:20:123
1156899472,2cyclictest0-21swapper/320:35:243
1156899472,2cyclictest0-21swapper/320:35:243
1156899471,4cyclictest18959-21apt-get21:00:183
1156799471,5cyclictest26062-21latency_hist19:50:122
1156399476,26cyclictest1611-21ssh23:20:181
11563994713,21cyclictest23-21ksoftirqd/122:50:231
11563994710,24cyclictest0-21swapper/121:25:251
11559994732,6cyclictest7950irq/56-dwc_otg_19:55:350
11559994726,5cyclictest15740-21cat19:20:320
11559994724,14cyclictest0-21swapper/019:10:460
1155999472,35cyclictest0-21swapper/020:22:290
1155999471,25cyclictest19301-21fschecks_count00:20:280
1156399464,10cyclictest7850irq/56-dwc_otg21:15:231
1156399462,6cyclictest8050irq/56-dwc_otg_23:25:241
11563994626,14cyclictest23-21ksoftirqd/123:05:231
11559994624,15cyclictest0-21swapper/019:25:250
11559994624,12cyclictest0-21swapper/020:45:340
1155999462,29cyclictest0-21swapper/021:00:260
1156899451,4cyclictest14807-21timerwakeupswit20:45:393
11563994514,14cyclictest23-21ksoftirqd/120:10:241
1156399447,24cyclictest0-21swapper/119:50:251
11563994416,13cyclictest21-21rcuc/122:00:191
104352440,5sleep20-21swapper/219:05:352
104352440,5sleep20-21swapper/219:05:352
1156399437,24cyclictest0-21swapper/122:40:191
1156399432,4cyclictest0-21swapper/120:30:181
1156399417,24cyclictest0-21swapper/119:10:261
1156399416,5cyclictest0-21swapper/123:30:251
132282400,2sleep10-21swapper/100:12:531
1156399404,6cyclictest7950irq/56-dwc_otg_20:40:371
1156399402,4cyclictest7950irq/56-dwc_otg_22:05:281
11563994018,6cyclictest29284-21fschecks_count22:35:291
1156399396,23cyclictest0-21swapper/121:00:251
1156399391,26cyclictest29672-21ls23:15:131
1156399382,23cyclictest5176-21rm22:45:271
1156399382,15cyclictest8050irq/56-dwc_otg_22:25:341
11563993814,6cyclictest22669-21diskmemload21:12:151
1156399379,15cyclictest0-21swapper/119:25:221
1156399375,7cyclictest0-21swapper/123:00:221
1156399375,7cyclictest0-21swapper/123:00:221
11563993719,4cyclictest14203-21sh21:38:121
11563993719,4cyclictest14203-21sh21:38:121
11563993716,5cyclictest27478-21sensors23:50:371
11563993712,7cyclictest2822450wget21:55:271
1156399371,25cyclictest22673-21latency_hist21:10:131
1156399368,6cyclictest7950irq/56-dwc_otg_21:45:121
1156399365,5cyclictest7950irq/56-dwc_otg_19:55:391
1156399364,9cyclictest8050irq/56-dwc_otg_21:30:351
1156399364,10cyclictest8050irq/56-dwc_otg_20:15:341
11563993616,4cyclictest30149-21fschecks_count23:15:301
11563993612,6cyclictest14917-21aten_rbpower_en00:15:261
1156399361,15cyclictest10411-21fschecks_count20:35:281
1156399361,15cyclictest10411-21fschecks_count20:35:281
11563993611,11cyclictest0-21swapper/123:46:481
1156399357,5cyclictest7850irq/56-dwc_otg23:40:251
1156399357,5cyclictest7850irq/56-dwc_otg23:40:251
1156399357,24cyclictest0-21swapper/122:20:251
1156399356,6cyclictest7950irq/56-dwc_otg_21:20:261
11563993517,4cyclictest24330-21/usr/sbin/munin19:45:391
1156399351,22cyclictest6701-21uniq20:25:291
103822355,17sleep110434-21/usr/sbin/munin19:05:351
103822355,17sleep110434-21/usr/sbin/munin19:05:351
1156399349,14cyclictest0-21swapper/123:35:421
1156399342,20cyclictest0-21swapper/122:35:091
1156399342,16cyclictest0-21swapper/121:48:021
1156399341,22cyclictest4849-21cpuspeed20:20:261
11563993411,10cyclictest0-21swapper/122:11:151
1156399334,20cyclictest30683-21ssh23:55:231
11563993315,8cyclictest215702chrt00:22:561
1156399329,10cyclictest0-21swapper/100:34:001
1156399324,3cyclictest0-21swapper/120:50:251
1156399321,8cyclictest873-21latency_hist20:10:131
1156399321,8cyclictest873-21latency_hist20:10:131
11563993216,3cyclictest8050irq/56-dwc_otg_00:05:261
1156399316,4cyclictest7850irq/56-dwc_otg20:55:401
1156399316,14cyclictest0-21swapper/119:40:181
11563993112,4cyclictest13388-21/usr/sbin/munin19:15:301
11563992920,6cyclictest13825-21aten_rbpower_cu20:45:261
111632264,15sleep30-21swapper/319:08:283
111632264,15sleep30-21swapper/319:08:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional