You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 02:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri Jan 09, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
42832900,2sleep00-21swapper/020:55:160
30923998413,43cyclictest9317-21grep22:35:430
30923998312,57cyclictest12681-21grep21:15:430
3092399826,64cyclictest0-21swapper/020:05:280
30923998010,46cyclictest17320-21cut23:00:280
3092399786,50cyclictest0-21swapper/023:25:170
30923997612,42cyclictest29323-21grep23:30:430
30923997513,41cyclictest5656-21latency_hist19:30:160
3092399749,51cyclictest9749-21latency_hist21:10:160
3092399746,58cyclictest0-21swapper/023:44:400
30923997314,38cyclictest21189-21latency21:40:340
3092399716,44cyclictest19378-21wget00:35:280
3092399715,57cyclictest7269-21grep00:00:370
3092399708,11cyclictest128-21systemd-journal20:40:160
3092399708,11cyclictest128-21systemd-journal20:40:160
3092399707,45cyclictest0-21swapper/020:40:400
30923997010,50cyclictest20455-21aten_rbpower_cu20:10:280
30923996915,12cyclictest14838-21sort19:55:160
3092399679,33cyclictest1440-21/usr/sbin/munin20:45:390
3092399668,47cyclictest7969-21fschecks_time19:35:320
30923996513,38cyclictest3694-21cron19:25:150
3092399646,23cyclictest0-21swapper/022:50:290
3092399646,23cyclictest0-21swapper/022:50:290
3092399645,47cyclictest0-21swapper/023:25:150
3092399641,30cyclictest0-21swapper/000:05:250
30923996410,33cyclictest128-21systemd-journal23:46:150
3092699635,4cyclictest7319-21switchtime22:30:423
3092399637,45cyclictest0-21swapper/000:25:090
30923996312,39cyclictest0-21swapper/021:35:270
3092399627,40cyclictest0-21swapper/019:30:400
30923996228,19cyclictest0-21swapper/021:55:280
30923996216,28cyclictest10642-21expr21:10:390
3092399621,26cyclictest0-21swapper/021:50:290
3092699611,3cyclictest7950irq/56-dwc_otg_22:05:333
209072610,1sleep10-21swapper/123:10:251
3092399603,40cyclictest2018-21tail22:15:440
3092399603,40cyclictest2018-21tail22:15:440
3092699592,4cyclictest7950irq/56-dwc_otg_00:30:353
30926995912,6cyclictest11907-21fschecks_time21:15:323
3092399597,38cyclictest11294-21mv22:41:100
30923995914,31cyclictest5181-21tune2fs23:55:330
30923995911,38cyclictest6689-21fschecks_count22:30:320
30923995910,31cyclictest0-21swapper/020:55:350
159592590,1sleep20-21swapper/219:55:412
3092399584,36cyclictest30710-21wc22:05:420
30923995810,39cyclictest29623-21cat22:05:160
3092699572,11cyclictest7950irq/56-dwc_otg_20:05:323
3092399575,40cyclictest0-21swapper/022:10:500
303042573,14sleep10-21swapper/119:05:511
3092399568,38cyclictest0-21swapper/019:11:380
3092399568,38cyclictest0-21swapper/019:11:380
3092399568,35cyclictest29006-21sshd20:31:080
3092399565,38cyclictest0-21swapper/000:14:070
3092399562,44cyclictest0-21swapper/019:43:360
30923995624,14cyclictest7850irq/56-dwc_otg_00:15:450
3092699554,7cyclictest7950irq/56-dwc_otg_20:05:163
3092699554,7cyclictest7950irq/56-dwc_otg_20:05:163
3092699551,9cyclictest7950irq/56-dwc_otg_19:30:283
3092699551,4cyclictest7950irq/56-dwc_otg_19:25:293
3092599553,32cyclictest7850irq/56-dwc_otg_20:55:252
3092399558,37cyclictest0-21swapper/022:24:410
3092399556,33cyclictest0-21swapper/022:55:220
3092399555,38cyclictest0-21swapper/021:24:000
3092699543,6cyclictest7950irq/56-dwc_otg_23:25:443
3092699542,12cyclictest0-21swapper/320:29:333
30926995416,35cyclictest30738-21unixbench_singl20:35:433
30926995416,35cyclictest30738-21unixbench_singl20:35:433
3092699541,5cyclictest8849-21users19:35:433
3092699541,5cyclictest7950irq/56-dwc_otg_21:35:353
3092699541,4cyclictest7950irq/56-dwc_otg_00:35:363
3092699541,12cyclictest7950irq/56-dwc_otg_20:40:293
3092399546,35cyclictest0-21swapper/023:14:180
30923995427,7cyclictest25260-21sed20:20:430
30923995411,29cyclictest128-21systemd-journal23:36:130
3092699532,4cyclictest7950irq/56-dwc_otg_23:20:163
3092699531,6cyclictest24285-21/usr/sbin/munin21:50:423
3092699531,3cyclictest0-21swapper/322:55:313
3092699531,15cyclictest0-21swapper/300:25:153
3092399537,39cyclictest0-21swapper/022:30:120
3092399537,35cyclictest17057-21taskset00:27:280
3092399536,35cyclictest0-21swapper/020:26:430
3092399535,37cyclictest7492-21taskset21:02:020
3092399535,32cyclictest4388-21taskset23:52:310
3092699528,5cyclictest2809-21fschecks_count20:50:323
3092699525,4cyclictest7950irq/56-dwc_otg_20:15:443
30926995215,34cyclictest31785-21fschecks_count22:10:323
30926995213,5cyclictest11684-21if_eth019:45:343
30926995212,4cyclictest31109-21cpu19:10:293
30926995212,4cyclictest31109-21cpu19:10:293
3092499527,10cyclictest7750irq/56-dwc_otg22:30:291
3092399525,37cyclictest0-21swapper/021:26:500
3092399525,37cyclictest0-21swapper/021:26:500
3092399524,37cyclictest0-21swapper/022:45:450
30923995222,15cyclictest7750irq/56-dwc_otg21:45:330
30923995217,20cyclictest17574-21tr00:30:290
131002520,2sleep30-21swapper/300:15:443
3092699519,4cyclictest0-21swapper/319:25:153
3092699511,7cyclictest2712-21sh23:50:153
3092699511,3cyclictest0-21swapper/323:55:383
3092699511,2cyclictest0-21swapper/323:35:313
3092699511,10cyclictest0-21swapper/320:32:363
3092399517,37cyclictest0-21swapper/023:06:040
3092399513,38cyclictest0-21swapper/019:48:550
30923995110,3cyclictest0-21swapper/019:55:160
3092699503,4cyclictest7950irq/56-dwc_otg_21:40:443
3092699501,8cyclictest7850irq/56-dwc_otg_21:02:283
3092699501,4cyclictest7950irq/56-dwc_otg_23:14:263
3092699501,3cyclictest7950irq/56-dwc_otg_22:16:523
3092699501,3cyclictest7950irq/56-dwc_otg_22:16:523
30926995012,4cyclictest742-21/usr/sbin/munin20:45:333
3092699501,12cyclictest0-21swapper/323:03:263
3092599506,27cyclictest1125-21runrttasks22:45:302
3092599501,11cyclictest0-21swapper/219:25:282
30924995010,6cyclictest7750irq/56-dwc_otg22:05:261
3092399507,39cyclictest24238-21chrt23:19:420
3092699492,3cyclictest0-21swapper/321:24:113
30926994913,4cyclictest17171-21aten_rbpower_vo21:30:293
3092399496,28cyclictest0-21swapper/021:35:040
3092399494,35cyclictest0-21swapper/019:15:380
3092399492,26cyclictest0-21swapper/020:15:360
3092699488,5cyclictest1880-21modprobe19:20:153
3092699482,3cyclictest7750irq/56-dwc_otg21:45:323
3092699482,11cyclictest7850irq/56-dwc_otg_21:57:353
3092699481,5cyclictest7950irq/56-dwc_otg_23:50:163
3092699481,4cyclictest20833-21iwlist20:10:333
30926994810,5cyclictest9240-21cut00:05:393
3092599487,25cyclictest0-21swapper/200:25:282
3092599482,22cyclictest7750irq/56-dwc_otg19:20:262
3092599482,19cyclictest7850irq/56-dwc_otg_21:55:272
3092499487,24cyclictest0-21swapper/119:15:291
3092699472,8cyclictest28029-21wget22:00:293
3092699471,6cyclictest7950irq/56-dwc_otg_21:10:293
3092699471,4cyclictest4621-21/usr/sbin/munin22:25:333
3092699471,4cyclictest11865-21/usr/sbin/munin22:45:413
3092599477,25cyclictest0-21swapper/222:20:282
3092699461,6cyclictest843-21chrt23:44:493
3092699461,4cyclictest2711-21/usr/sbin/munin22:20:333
3092699461,3cyclictest7850irq/56-dwc_otg_20:24:073
3092599469,5cyclictest29-21rcuc/200:00:282
3092599467,25cyclictest0-21swapper/222:25:202
3092599467,24cyclictest0-21swapper/220:10:282
30925994623,6cyclictest7850irq/56-dwc_otg_20:30:272
167182460,2sleep30-21swapper/300:25:423
3092699456,24cyclictest0-21swapper/322:35:283
3092599457,24cyclictest0-21swapper/220:20:212
3092599457,23cyclictest0-21swapper/221:30:292
3092599457,23cyclictest0-21swapper/221:05:292
30924994524,6cyclictest20266-21grep23:05:451
306052452,14sleep00-21swapper/019:09:390
3092699447,23cyclictest0-21swapper/322:40:283
3092699446,23cyclictest0-21swapper/320:55:283
3092599447,25cyclictest0-21swapper/222:15:292
3092599447,25cyclictest0-21swapper/222:15:292
3092599446,25cyclictest0-21swapper/220:15:292
3092599444,5cyclictest0-21swapper/221:45:282
30925994417,5cyclictest0-21swapper/220:05:292
3092499447,23cyclictest0-21swapper/121:15:281
3092499443,11cyclictest1544-21ntpd21:00:261
30923994413,20cyclictest0-21swapper/020:00:310
30923994413,20cyclictest0-21swapper/020:00:310
3092599437,23cyclictest0-21swapper/221:00:282
3092599431,4cyclictest0-21swapper/200:20:272
3092499437,22cyclictest0-21swapper/119:50:291
3092499435,23cyclictest0-21swapper/100:15:281
3092499434,7cyclictest7750irq/56-dwc_otg23:15:271
3092699421,3cyclictest7750irq/56-dwc_otg23:31:123
30925994211,11cyclictest31-21ksoftirqd/222:55:162
30925994211,11cyclictest31-21ksoftirqd/222:55:162
150022421,35sleep10-21swapper/121:25:161
3092699411,3cyclictest7850irq/56-dwc_otg_00:00:433
3092499413,4cyclictest0-21swapper/119:40:271
30924994116,6cyclictest3987-21aten_rbpower_en19:25:291
227502411,33sleep20-21swapper/223:15:312
3092599405,5cyclictest7950irq/56-dwc_otg_21:35:332
3092499401,23cyclictest0-21swapper/120:25:281
3092599394,19cyclictest0-21swapper/221:20:272
3092599392,6cyclictest0-21swapper/219:45:292
3092599391,27cyclictest1488-21ls23:45:322
30925993910,7cyclictest7922-21fschecks_count19:35:322
3092499399,5cyclictest7950irq/56-dwc_otg_23:55:281
3092599386,22cyclictest0-21swapper/222:10:282
3092599386,22cyclictest0-21swapper/220:35:282
3092599386,22cyclictest0-21swapper/220:35:282
3092599384,27cyclictest7950irq/56-dwc_otg_23:25:272
3092599382,21cyclictest0-21swapper/222:00:282
30925993811,7cyclictest0-21swapper/200:05:252
3092499387,22cyclictest0-21swapper/119:30:211
3092499386,10cyclictest0-21swapper/123:50:161
3092499382,7cyclictest7850irq/56-dwc_otg_23:20:241
3092499382,3cyclictest7850irq/56-dwc_otg_21:25:261
3092499382,3cyclictest7850irq/56-dwc_otg_21:25:261
30924993814,5cyclictest19808-21kernelversion00:35:341
3092499381,28cyclictest28638-21ls20:30:431
3092499381,22cyclictest0-21swapper/122:50:291
3092499381,22cyclictest0-21swapper/122:50:291
3092599378,22cyclictest7850irq/56-dwc_otg_21:15:432
3092599374,6cyclictest31-21ksoftirqd/220:45:392
3092599374,16cyclictest29-21rcuc/223:08:422
30925993718,12cyclictest27171-21seq20:25:492
30925993710,12cyclictest29-21rcuc/200:38:022
30924993711,5cyclictest7850irq/56-dwc_otg_20:10:421
3092699361,4cyclictest8007-21apt-get21:05:273
3092699361,4cyclictest13666-21/usr/sbin/munin22:50:293
3092699361,4cyclictest13666-21/usr/sbin/munin22:50:293
3092699361,11cyclictest0-21swapper/321:28:193
3092699361,11cyclictest0-21swapper/321:28:193
3092599369,14cyclictest7950irq/56-dwc_otg_23:35:362
3092599366,14cyclictest29-21rcuc/220:40:362
3092599363,14cyclictest7850irq/56-dwc_otg_21:40:342
3092599361,24cyclictest22506-21ls23:15:152
3092499367,14cyclictest0-21swapper/123:35:291
3092499366,24cyclictest0-21swapper/100:30:291
3092499365,14cyclictest7950irq/56-dwc_otg_21:50:341
3092499361,25cyclictest23104-21switchtime20:15:411
3092599355,5cyclictest7850irq/56-dwc_otg_20:00:272
3092599355,5cyclictest7850irq/56-dwc_otg_20:00:272
3092599353,9cyclictest14830-21latency_hist19:55:162
3092599352,21cyclictest0-21swapper/200:10:342
30925993520,5cyclictest7850irq/56-dwc_otg_22:30:412
3092599351,5cyclictest2829-21fschecks_count20:50:322
3092499354,6cyclictest10138-21fschecks_time21:10:311
3092499352,3cyclictest0-21swapper/123:50:341
3092499352,20cyclictest0-21swapper/119:35:401
3092699341,9cyclictest0-21swapper/319:51:133
3092699341,9cyclictest0-21swapper/319:44:373
3092699341,4cyclictest0-21swapper/319:56:033
3092699341,2cyclictest0-21swapper/323:23:593
3092599347,24cyclictest0-21swapper/219:10:282
3092599347,24cyclictest0-21swapper/219:10:282
3092599343,5cyclictest0-21swapper/200:30:332
3092599343,12cyclictest7950irq/56-dwc_otg_19:20:152
3092599342,7cyclictest10043-21/usr/sbin/munin22:40:372
3092599341,3cyclictest7850irq/56-dwc_otg_21:25:262
3092599341,3cyclictest7850irq/56-dwc_otg_21:25:262
3092599341,15cyclictest7850irq/56-dwc_otg_22:55:312
3092499348,13cyclictest7950irq/56-dwc_otg_23:00:151
3092499344,18cyclictest12424-21unixbench_singl19:45:431
3092499342,7cyclictest0-21swapper/120:20:271
30924993419,4cyclictest1869450wget20:05:291
30924993410,9cyclictest9550irq/80-mmc020:00:161
307022340,3sleep230701-21grep22:05:422
3092699337,5cyclictest7750irq/56-dwc_otg23:05:363
3092599338,7cyclictest31-21ksoftirqd/221:10:322
3092599337,15cyclictest29-21rcuc/219:30:402
30925993315,9cyclictest7850irq/56-dwc_otg_23:41:422
3092499337,24cyclictest0-21swapper/121:05:291
3092499337,23cyclictest0-21swapper/123:00:281
3092499337,23cyclictest0-21swapper/122:00:281
3092499337,23cyclictest0-21swapper/121:55:291
3092499336,6cyclictest7850irq/56-dwc_otg_21:30:291
3092499334,15cyclictest7950irq/56-dwc_otg_19:20:281
3092499332,5cyclictest7850irq/56-dwc_otg_22:35:281
3092499331,24cyclictest0-21swapper/120:50:281
3092599328,12cyclictest301ktimersoftd/221:54:382
3092599323,21cyclictest7850irq/56-dwc_otg_23:00:282
3092599322,12cyclictest7950irq/56-dwc_otg_23:55:302
3092499329,5cyclictest7750irq/56-dwc_otg20:40:321
3092499328,11cyclictest7850irq/56-dwc_otg_23:25:301
3092499327,6cyclictest7950irq/56-dwc_otg_00:05:431
3092499327,15cyclictest0-21swapper/123:40:291
30924993217,8cyclictest31815-21fschecks_time22:10:321
3092499321,6cyclictest0-21swapper/122:15:311
3092499321,6cyclictest0-21swapper/122:15:311
3092499321,5cyclictest7244-21ls00:00:371
3092499321,16cyclictest0-21swapper/120:55:321
3092599316,22cyclictest0-21swapper/223:20:262
3092599315,16cyclictest7950irq/56-dwc_otg_23:50:342
3092599312,9cyclictest0-21swapper/222:35:392
30925993124,5cyclictest13678-21cron00:20:152
3092599312,14cyclictest29-21rcuc/223:30:392
3092499318,4cyclictest7750irq/56-dwc_otg00:10:341
3092499316,16cyclictest0-21swapper/122:25:211
3092499312,19cyclictest0-21swapper/120:03:411
3092499312,19cyclictest0-21swapper/120:03:411
30924993112,5cyclictest21-21rcuc/119:10:371
30924993112,5cyclictest21-21rcuc/119:10:371
3092499311,21cyclictest0-21swapper/122:45:341
3092499311,17cyclictest7950irq/56-dwc_otg_21:45:161
3092699301,9cyclictest0-21swapper/300:14:593
3092599306,15cyclictest29-21rcuc/219:41:132
3092499306,14cyclictest0-21swapper/123:30:291
3092499304,13cyclictest7950irq/56-dwc_otg_20:40:001
3092499304,13cyclictest7950irq/56-dwc_otg_20:40:001
3092499302,18cyclictest0-21swapper/122:41:301
3092499302,18cyclictest0-21swapper/100:26:381
30924993010,4cyclictest0-21swapper/122:20:291
30924992922,5cyclictest1391-21ls20:45:381
3092499291,5cyclictest128-21systemd-journal21:35:481
3092499291,10cyclictest22730-21cut21:45:331
3092499284,4cyclictest7750irq/56-dwc_otg00:25:151
304052273,18sleep20-21swapper/219:07:102
303892262,15sleep30-21swapper/319:06:583
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional