You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 14:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Dec 10, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
412999933,70cyclictest0-21swapper/019:10:180
306572920,2sleep30-21swapper/300:13:113
412999836,67cyclictest0-21swapper/019:25:290
4129997411,50cyclictest20796-21fschecks_time19:55:320
412999739,50cyclictest0-21swapper/023:35:290
148822690,2sleep30-21swapper/319:40:163
148822690,2sleep30-21swapper/319:40:163
412999683,36cyclictest0-21swapper/023:55:300
413199674,18cyclictest7950irq/56-dwc_otg_19:40:162
413199674,18cyclictest7950irq/56-dwc_otg_19:40:162
4129996719,27cyclictest5494-21unixbench_singl20:40:440
412999662,54cyclictest16606-21rm23:55:160
412999652,44cyclictest0-21swapper/000:10:350
261582650,12sleep07950irq/56-dwc_otg_22:45:390
412999642,45cyclictest0-21swapper/000:15:350
412999641,53cyclictest21015-21taskset00:00:330
412999632,37cyclictest0-21swapper/022:44:330
4129996233,15cyclictest0-21swapper/021:20:160
4129996233,15cyclictest0-21swapper/021:20:160
4129996214,25cyclictest4777-21aten_rbpower_cu23:40:300
242732620,7sleep1221ktimersoftd/121:20:411
412999611,46cyclictest0-21swapper/021:42:330
412999611,46cyclictest0-21swapper/021:42:330
412999611,44cyclictest7310-21head20:45:450
412999601,46cyclictest25254-21sed20:05:450
413199596,14cyclictest0-21swapper/221:30:222
412999594,38cyclictest27546-21ssh21:25:360
4129995933,8cyclictest13448-21apt-config22:30:170
4129995929,16cyclictest19070-21awk21:15:160
412999591,42cyclictest28969-21latency_hist23:30:160
4129995911,27cyclictest0-21swapper/020:30:430
4129995838,6cyclictest7850irq/56-dwc_otg_00:05:350
4129995836,5cyclictest1134-21aten_rbpower_cu22:55:320
412999583,45cyclictest0-21swapper/019:15:290
4129995830,15cyclictest0-21swapper/000:36:400
4129995823,10cyclictest0-21swapper/022:10:270
4129995819,32cyclictest9-21ksoftirqd/023:05:170
412999581,37cyclictest19692-21sed21:55:530
4129995811,27cyclictest0-21swapper/020:20:150
412999577,8cyclictest0-21swapper/019:45:280
4129995736,10cyclictest7850irq/56-dwc_otg_23:45:340
412999571,38cyclictest23437-21aten_rbpower_en21:20:300
162762570,2sleep00-21swapper/022:33:270
413199566,5cyclictest7850irq/56-dwc_otg_20:15:322
412999562,45cyclictest0-21swapper/023:15:300
412999562,31cyclictest12545-21unixbench_singl19:30:440
412999561,44cyclictest0-21swapper/022:20:270
4129995611,5cyclictest0-21swapper/021:35:150
413299552,10cyclictest7950irq/56-dwc_otg_20:20:283
412999556,30cyclictest32452-21unixbench_multi20:25:460
412999555,27cyclictest29224-21aten_rbpower_en23:30:300
4129995525,12cyclictest11951-21grep00:30:430
412999551,44cyclictest14531-21grep21:05:440
412999551,38cyclictest0-21swapper/020:35:260
159912550,6sleep3413299cyclictest23:13:483
413099543,15cyclictest7750irq/56-dwc_otg21:40:261
413099543,15cyclictest7750irq/56-dwc_otg21:40:261
412999546,42cyclictest0-21swapper/020:00:310
412999546,42cyclictest0-21swapper/020:00:310
4129995441,5cyclictest7750irq/56-dwc_otg22:15:270
4129995431,6cyclictest13267-21ssh23:10:320
4129995428,5cyclictest7850irq/56-dwc_otg_21:00:460
4129995425,13cyclictest0-21swapper/020:50:460
412999542,37cyclictest0-21swapper/023:20:320
4129995419,17cyclictest757-21latency_hist22:55:160
412999541,39cyclictest26504-21/usr/sbin/munin22:05:320
412999541,34cyclictest0-21swapper/022:40:170
413199531,7cyclictest0-21swapper/200:15:262
412999533,31cyclictest0-21swapper/019:45:150
412999531,40cyclictest10951-21grep21:45:330
413099525,8cyclictest0-21swapper/119:10:171
4130995210,12cyclictest0-21swapper/120:20:271
412999527,28cyclictest0-21swapper/020:20:270
412999521,35cyclictest1120-21runrttasks19:50:280
175592520,2sleep10-21swapper/123:15:361
4129995123,8cyclictest7850irq/56-dwc_otg_00:20:310
4129995123,8cyclictest7850irq/56-dwc_otg_00:20:310
4129995116,20cyclictest27022-21grep20:10:470
4129995114,19cyclictest13465-21fschecks_count19:35:340
4129995114,19cyclictest13465-21fschecks_count19:35:340
412999511,29cyclictest9308-21wget23:05:300
413299503,30cyclictest0-21swapper/319:10:183
413099507,28cyclictest0-21swapper/122:35:321
4129995034,3cyclictest0-21swapper/019:20:330
4129994930,4cyclictest7750irq/56-dwc_otg21:50:380
412999491,34cyclictest0-21swapper/000:25:330
412999491,31cyclictest3112-21ssh21:35:350
37992496,35sleep00-21swapper/019:09:310
413299487,7cyclictest0-21swapper/320:00:313
413299487,7cyclictest0-21swapper/320:00:313
413299487,26cyclictest0-21swapper/322:05:303
4131994822,4cyclictest9944-21aten_rbpower_po20:55:322
413099487,27cyclictest0-21swapper/119:25:291
412999484,12cyclictest0-21swapper/020:55:300
412999481,37cyclictest0-21swapper/022:00:400
413299477,25cyclictest0-21swapper/320:55:213
413199468,23cyclictest0-21swapper/219:10:182
413199467,25cyclictest0-21swapper/220:35:282
413199467,25cyclictest0-21swapper/219:20:292
413099468,8cyclictest0-21swapper/121:45:271
413299457,25cyclictest0-21swapper/319:45:293
413299457,24cyclictest0-21swapper/323:00:213
413299455,27cyclictest7850irq/56-dwc_otg_21:34:113
4131994515,11cyclictest0-21swapper/219:50:272
413099456,9cyclictest7750irq/56-dwc_otg20:10:321
4130994519,11cyclictest7950irq/56-dwc_otg_21:55:291
413299448,25cyclictest0-21swapper/322:50:213
413299447,25cyclictest0-21swapper/322:35:323
413299447,25cyclictest0-21swapper/320:10:323
413099449,23cyclictest0-21swapper/120:25:311
413099447,24cyclictest0-21swapper/119:15:311
4132994318,5cyclictest28068-21latency_hist00:10:163
4131994319,13cyclictest0-21swapper/223:55:322
413199427,22cyclictest0-21swapper/221:35:312
413099427,23cyclictest0-21swapper/123:00:221
413099427,23cyclictest0-21swapper/122:50:211
413099427,22cyclictest0-21swapper/120:15:341
413099427,22cyclictest0-21swapper/119:30:291
4132994117,5cyclictest31943-21/usr/sbin/munin00:15:303
413199416,3cyclictest7950irq/56-dwc_otg_20:20:292
413099417,23cyclictest0-21swapper/120:55:321
413099415,7cyclictest7950irq/56-dwc_otg_22:10:291
413099411,6cyclictest565750ssh22:20:301
198922410,1sleep30-21swapper/321:56:023
125832410,2sleep20-21swapper/222:29:072
413299406,22cyclictest0-21swapper/322:10:293
413199407,25cyclictest0-21swapper/222:30:322
413199407,23cyclictest0-21swapper/221:50:292
413199407,21cyclictest0-21swapper/221:45:292
4131994029,5cyclictest301ktimersoftd/200:20:312
4131994029,5cyclictest301ktimersoftd/200:20:312
413199402,4cyclictest7750irq/56-dwc_otg21:20:272
413199401,9cyclictest22640-21sed22:40:502
413099406,21cyclictest0-21swapper/122:30:211
413099401,7cyclictest29505-21fschecks_time23:30:341
247512400,2sleep10-21swapper/122:02:571
4132993916,15cyclictest29174-21basename23:30:303
4131993915,5cyclictest4652-21sh22:19:052
413199391,23cyclictest0-21swapper/223:35:302
4130993919,5cyclictest1092-21sh22:55:301
4130993916,5cyclictest11121-21sh23:06:591
413299387,21cyclictest0-21swapper/322:40:293
413299386,14cyclictest0-21swapper/320:25:313
413299385,5cyclictest7850irq/56-dwc_otg_22:55:303
413199388,13cyclictest7850irq/56-dwc_otg_19:45:382
413199385,20cyclictest10896-21ssh00:30:292
4131993820,5cyclictest21470-21netstat00:00:382
413199381,31cyclictest2083-21ls22:55:442
4131993812,17cyclictest0-21swapper/221:42:392
4131993812,17cyclictest0-21swapper/221:42:392
4130993814,9cyclictest7850irq/56-dwc_otg_22:05:281
292602380,2sleep20-21swapper/200:10:442
413299378,16cyclictest0-21swapper/321:40:283
413299378,16cyclictest0-21swapper/321:40:283
413299377,20cyclictest0-21swapper/323:35:303
413299376,21cyclictest0-21swapper/319:15:313
413299371,5cyclictest5002-21grep23:40:343
413299371,26cyclictest6383-21uniq20:45:343
413299371,23cyclictest2059-21kworker/u8:021:10:153
413299371,16cyclictest7850irq/56-dwc_otg_19:50:313
413199371,7cyclictest21137-21grep22:40:162
413199371,7cyclictest11696-21grep21:00:312
4130993722,7cyclictest7950irq/56-dwc_otg_00:25:411
413299367,24cyclictest0-21swapper/322:30:213
413299367,16cyclictest0-21swapper/300:25:313
413199369,7cyclictest26619-21fschecks_time22:05:342
413199369,17cyclictest0-21swapper/223:25:302
413199367,6cyclictest7750irq/56-dwc_otg00:35:332
413099369,4cyclictest7950irq/56-dwc_otg_20:55:161
413099369,10cyclictest0-21swapper/122:15:261
413099365,19cyclictest8792-21ssh23:45:291
4130993629,5cyclictest23-21ksoftirqd/123:40:281
4130993612,8cyclictest16741-21munin-run23:15:151
413099360,26cyclictest0-21swapper/121:30:301
413299357,22cyclictest0-21swapper/319:30:293
413299356,14cyclictest28557-21grep21:25:483
413299355,21cyclictest7850irq/56-dwc_otg_23:50:373
413299353,6cyclictest16624-21grep21:10:453
4132993517,8cyclictest1226150ssh22:28:283
4132993516,5cyclictest17173-21aten_rbpower_po23:15:303
413299351,26cyclictest11172-21uniq00:30:333
413199359,18cyclictest0-21swapper/221:15:282
413199359,18cyclictest0-21swapper/221:15:282
413199352,4cyclictest27068-21grep20:10:472
413199352,3cyclictest0-21swapper/221:10:332
413099357,17cyclictest7850irq/56-dwc_otg_20:50:161
413099356,13cyclictest23-21ksoftirqd/122:42:401
4130993514,4cyclictest9658-21basename22:25:321
29022350,2sleep10-21swapper/120:35:331
413299347,4cyclictest0-21swapper/322:20:273
413299345,5cyclictest7850irq/56-dwc_otg_00:00:333
4132993419,8cyclictest7850irq/56-dwc_otg_20:40:163
413299341,4cyclictest16864-21ssh23:55:313
4132993412,11cyclictest0-21swapper/323:46:503
413299341,10cyclictest0-21swapper/323:25:303
413199347,25cyclictest7269-21ssh00:25:312
413199347,14cyclictest0-21swapper/223:40:302
413199346,14cyclictest7850irq/56-dwc_otg_22:50:192
413199341,8cyclictest10746-21unixbench_singl19:25:442
413199341,7cyclictest0-21swapper/220:05:162
413199341,7cyclictest0-21swapper/220:05:162
413199341,5cyclictest18434-21/usr/sbin/munin21:55:422
413199341,24cyclictest10255-21ssh23:05:442
4131993411,12cyclictest31-21ksoftirqd/223:25:162
4131993410,5cyclictest0-21swapper/222:49:272
413099349,10cyclictest0-21swapper/121:10:371
413099347,4cyclictest7850irq/56-dwc_otg_23:50:371
413099347,14cyclictest0-21swapper/121:25:321
413099346,4cyclictest7950irq/56-dwc_otg_21:15:281
413099346,4cyclictest7950irq/56-dwc_otg_21:15:281
413099345,4cyclictest15032-21diskmemload23:20:331
413099344,5cyclictest7950irq/56-dwc_otg_23:55:361
413099342,4cyclictest7950irq/56-dwc_otg_21:50:291
4130993413,10cyclictest19700-21sort19:50:431
413099341,28cyclictest16629-21ls19:45:151
4130993411,6cyclictest21103-21fschecks_time00:00:331
4130993411,5cyclictest149602chrt21:09:381
4130993411,10cyclictest0-21swapper/100:39:391
413299336,9cyclictest247522ssh22:02:573
413299336,15cyclictest0-21swapper/320:50:323
413299334,15cyclictest7750irq/56-dwc_otg22:45:343
4132993312,12cyclictest39-21ksoftirqd/321:50:153
413199335,14cyclictest7850irq/56-dwc_otg_19:15:312
413199331,9cyclictest7985-21wget20:50:322
413199331,7cyclictest963-21sort20:30:322
413199331,5cyclictest23566-21tail22:00:452
4131993312,7cyclictest301ktimersoftd/222:24:042
413199331,26cyclictest24638-21latency20:05:372
413199331,24cyclictest17703-21missed_timers23:15:382
413099338,5cyclictest23-21ksoftirqd/123:35:371
413099338,13cyclictest0-21swapper/121:39:071
413099335,6cyclictest7850irq/56-dwc_otg_00:05:371
413099333,10cyclictest7950irq/56-dwc_otg_23:25:371
413099333,10cyclictest7950irq/56-dwc_otg_00:30:331
4130993311,14cyclictest15032-21diskmemload00:21:471
4130993311,14cyclictest15032-21diskmemload00:21:471
413299327,22cyclictest0-21swapper/319:25:293
413299326,14cyclictest0-21swapper/321:00:303
413299321,20cyclictest7850irq/56-dwc_otg_23:05:293
413199327,7cyclictest7850irq/56-dwc_otg_22:15:152
413199327,4cyclictest0-21swapper/223:10:252
4131993215,5cyclictest29-21rcuc/200:05:492
413099328,4cyclictest21-21rcuc/121:05:151
413099326,10cyclictest7950irq/56-dwc_otg_20:45:151
413099323,14cyclictest7950irq/56-dwc_otg_00:10:401
413099322,7cyclictest7750irq/56-dwc_otg19:55:281
4130993215,5cyclictest1120-21runrttasks19:37:331
4130993215,5cyclictest1120-21runrttasks19:37:331
4130993210,5cyclictest21-21rcuc/120:07:121
413299319,4cyclictest0-21swapper/321:51:003
4132993116,10cyclictest0-21swapper/323:20:313
413299311,3cyclictest7750irq/56-dwc_otg19:40:263
413199318,11cyclictest0-21swapper/223:30:422
413199316,15cyclictest0-21swapper/219:40:282
413199315,6cyclictest4624-21grep20:40:332
4131993121,7cyclictest30914-21latency_hist21:30:152
4131993110,4cyclictest0-21swapper/223:01:562
413099317,14cyclictest0-21swapper/120:00:311
413099317,14cyclictest0-21swapper/120:00:311
4130993112,4cyclictest0-21swapper/100:15:371
4130993112,11cyclictest11-21rcu_preempt19:45:271
4130993110,6cyclictest7850irq/56-dwc_otg_20:35:151
221412310,1sleep30-21swapper/321:18:303
221412310,1sleep30-21swapper/321:18:303
413299309,6cyclictest3926-21awk00:20:373
413299309,6cyclictest3926-21awk00:20:373
413299307,4cyclictest0-21swapper/322:16:093
413299307,20cyclictest0-21swapper/320:40:303
413299306,9cyclictest7850irq/56-dwc_otg_00:35:463
413299305,12cyclictest7850irq/56-dwc_otg_19:20:373
413299301,20cyclictest0-21swapper/320:10:163
413199302,4cyclictest0-21swapper/223:50:302
4131993013,4cyclictest12376-21sh23:50:052
413099309,10cyclictest25669-21cut22:45:331
4130993015,4cyclictest7988-21sh19:20:311
392123011,10sleep13889-21latency_hist19:10:171
413299297,4cyclictest0-21swapper/321:37:043
413299292,18cyclictest7850irq/56-dwc_otg_20:30:323
413199296,9cyclictest7750irq/56-dwc_otg20:25:362
413199292,8cyclictest11666-21fschecks_count19:30:322
413199292,16cyclictest6200-21cut20:45:312
413199292,12cyclictest7850irq/56-dwc_otg_21:05:332
413299288,3cyclictest0-21swapper/319:55:333
413299283,4cyclictest7850irq/56-dwc_otg_20:15:423
413299281,20cyclictest0-21swapper/321:23:563
413199282,12cyclictest0-21swapper/219:58:412
35512273,14sleep30-21swapper/319:06:263
36542253,14sleep20-21swapper/219:07:472
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional