You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 18:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Wed Aug 12, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1069699272258,9cyclictest7950irq/56-dwc_otg_11:25:261
10695991643,127cyclictest18004-21ssh12:20:340
10695991363,51cyclictest0-21swapper/009:35:230
10695991354,120cyclictest9201-21unixbench-2d11:30:350
10695991333,58cyclictest6254-21kworker/u8:310:35:340
10695991333,120cyclictest0-21swapper/011:25:260
10695991293,53cyclictest25989-21/usr/sbin/munin09:15:320
10695991293,53cyclictest25989-21/usr/sbin/munin09:15:320
10695991273,58cyclictest12059-21ssh10:55:270
10696991246,88cyclictest11801-21fschecks_count10:55:211
10695991232,55cyclictest0-21swapper/011:05:280
10695991144,90cyclictest13757-21grep12:15:320
10695991125,75cyclictest0-21swapper/012:05:290
10695991125,75cyclictest0-21swapper/012:05:290
10696991114,79cyclictest26808-21ntp_states09:15:301
10696991114,79cyclictest26808-21ntp_states09:15:301
10696991112,73cyclictest8256-21df_inode11:30:201
10695991104,75cyclictest0-21swapper/011:45:180
10696991076,79cyclictest18196-21grep12:20:371
10695991074,81cyclictest3716-21unixbench_multi10:05:340
106959910746,29cyclictest0-21swapper/011:20:250
10695991071,77cyclictest0-21swapper/010:15:150
10695991071,77cyclictest0-21swapper/010:15:150
10695991054,82cyclictest0-21swapper/010:00:350
10696991037,63cyclictest29394-21memory11:55:231
277432990,2sleep30-21swapper/311:53:473
1069699992,74cyclictest10426-21grep09:35:231
1069599995,70cyclictest0-21swapper/009:25:210
10695999918,59cyclictest22217-21diskstats09:10:210
1069599985,64cyclictest0-21swapper/012:25:280
1069599973,71cyclictest0-21swapper/010:40:100
10697999437,38cyclictest0-21swapper/210:35:322
1069599946,71cyclictest0-21swapper/010:25:270
1069599936,50cyclictest29382-21/usr/sbin/munin11:55:220
1069699914,57cyclictest19689-21ssh10:25:291
1069699913,79cyclictest32151-21/usr/sbin/munin11:20:271
1069699893,79cyclictest13581-21/usr/sbin/munin12:15:271
1069699892,80cyclictest30166-21chrt09:20:181
1069699876,55cyclictest2064-21df_inode09:25:201
107922840,2sleep310694-21cyclictest07:10:183
1069599844,38cyclictest0-21swapper/009:55:140
1069699833,74cyclictest21959-21sh12:25:291
1069899822,62cyclictest7850irq/56-dwc_otg_11:55:233
1069599823,54cyclictest0-21swapper/009:20:170
1069699813,69cyclictest22201-21ssh09:10:191
1069699803,70cyclictest3744-21grep10:05:351
112792790,5sleep21069799cyclictest10:55:042
1069699793,69cyclictest21026-21head11:05:341
1069699782,69cyclictest20522-21apt-get11:45:171
1069699781,40cyclictest31634-21wc08:05:321
131932760,2sleep30-21swapper/310:56:143
10698997616,30cyclictest7950irq/56-dwc_otg_11:25:263
10698997613,42cyclictest9230-21/usr/sbin/munin11:30:353
1069699763,66cyclictest32107-21sh10:00:341
1069699763,66cyclictest28552-21grep10:35:321
1069699762,68cyclictest31463-21ssh10:40:081
1069699761,55cyclictest8381-21ssh09:32:351
34062750,7sleep11069699cyclictest12:03:311
1069699755,51cyclictest7950irq/56-dwc_otg_08:00:241
1069699755,51cyclictest7950irq/56-dwc_otg_08:00:241
1069699753,64cyclictest5167-21/usr/sbin/munin12:05:281
1069699753,64cyclictest5167-21/usr/sbin/munin12:05:281
1069699731,55cyclictest14640-21apt-get08:50:171
1069899723,46cyclictest18211-21unixbench_multi12:20:373
10698997227,21cyclictest19567-21ssh10:25:263
94062710,2sleep20-21swapper/211:30:432
1069599717,10cyclictest10493-21ssh10:15:040
1069599716,5cyclictest0-21swapper/007:45:150
10697996929,10cyclictest31-21ksoftirqd/211:05:282
1069699681,40cyclictest15467-21sed09:40:311
1069599672,49cyclictest0-21swapper/007:35:310
88282660,2sleep20-21swapper/210:12:132
10698996611,27cyclictest21975-21ssh12:25:303
1069699662,40cyclictest12493-21idleruntime-cro08:45:041
1069699661,50cyclictest20226-21latency_hist11:45:031
1069899653,45cyclictest13528-21ssh12:15:263
1069799657,22cyclictest7950irq/56-dwc_otg_09:35:202
1069699652,42cyclictest0-21swapper/107:21:001
1069699652,42cyclictest0-21swapper/107:21:001
1069699651,43cyclictest27219-21df_inode07:55:181
10698996423,19cyclictest381ktimersoftd/312:05:303
10698996423,19cyclictest381ktimersoftd/312:05:303
1069799642,45cyclictest11799-21ssh10:55:222
1069699641,42cyclictest7059-21latency_hist10:50:041
10695996442,5cyclictest7950irq/56-dwc_otg_12:10:160
1069899632,6cyclictest0-21swapper/311:35:173
1069899632,6cyclictest0-21swapper/311:35:173
10697996311,24cyclictest29-21rcuc/212:25:272
1069599632,16cyclictest0-21swapper/008:25:040
10697996213,26cyclictest0-21swapper/210:05:332
1069699627,10cyclictest7750irq/56-dwc_otg11:10:161
1069699622,43cyclictest7950irq/56-dwc_otg_10:34:011
1069699621,44cyclictest28423-21runrttasks09:56:241
1069699621,43cyclictest26801-21sort07:55:041
10695996248,6cyclictest7750irq/56-dwc_otg10:50:220
1069899617,25cyclictest7750irq/56-dwc_otg11:05:343
1069899612,33cyclictest28484-21ssh10:35:313
1069699613,43cyclictest7850irq/56-dwc_otg_10:15:201
1069699613,43cyclictest7850irq/56-dwc_otg_10:15:201
1069699611,48cyclictest13758-21ssh11:35:551
1069699611,48cyclictest13758-21ssh11:35:551
1069699611,47cyclictest25599-21ssh09:53:471
1069899609,26cyclictest0-21swapper/311:45:163
1069899603,37cyclictest0-21swapper/309:25:343
1069899603,34cyclictest319-21ssh11:20:243
10698996014,40cyclictest31477-21ssh10:40:113
1069699604,36cyclictest1174-21sshd08:10:451
1069699603,35cyclictest16206-21latency11:00:261
1069699601,48cyclictest21618-21diskmemload12:32:141
10695996037,4cyclictest0-21swapper/009:00:030
1069599602,43cyclictest16851-21ssh11:00:340
1069599601,42cyclictest0-21swapper/012:31:240
1069899593,41cyclictest0-21swapper/310:00:353
1069799593,12cyclictest18019-21timerandwakeup12:20:342
1069699595,39cyclictest7750irq/56-dwc_otg11:51:501
1069699591,46cyclictest0-21swapper/108:45:261
1069699591,44cyclictest10674-21rm12:11:431
10695995919,5cyclictest19256-21ssh09:45:300
10695995919,5cyclictest19256-21ssh09:45:300
1069599591,41cyclictest0-21swapper/011:35:300
1069599591,41cyclictest0-21swapper/011:35:300
174222580,2sleep20-21swapper/208:55:312
1069899581,32cyclictest7950irq/56-dwc_otg_09:35:213
1069799583,23cyclictest0-21swapper/210:40:112
1069699581,50cyclictest23470-21grep07:45:181
1069699581,44cyclictest10039-21ssh10:14:241
1069699581,38cyclictest0-21swapper/108:29:101
1069599581,44cyclictest9108-21/usr/sbin/munin08:35:210
1069699571,6cyclictest3370-21apt-get08:20:061
1069699571,47cyclictest0-21swapper/108:55:411
1069699571,46cyclictest21618-21diskmemload10:22:411
1069699571,42cyclictest0-21swapper/107:36:331
1069699571,35cyclictest18655-21fschecks_count09:45:211
1069699571,35cyclictest18655-21fschecks_count09:45:211
1069599571,42cyclictest12278-21idleruntime-cro07:15:040
1069599571,41cyclictest0-21swapper/011:15:260
1069899563,25cyclictest30148-21apt-get09:20:173
10698995618,14cyclictest0-21swapper/309:10:193
1069799562,21cyclictest7950irq/56-dwc_otg_10:25:262
10697995615,35cyclictest26323-21cut09:15:232
10697995615,35cyclictest26323-21cut09:15:232
1069699561,38cyclictest0-21swapper/110:52:171
1069599562,41cyclictest0-21swapper/009:00:220
1069599561,41cyclictest0-21swapper/009:10:040
10697995515,19cyclictest7850irq/56-dwc_otg_11:25:252
1069699551,36cyclictest0-21swapper/112:37:461
1069599552,45cyclictest0-21swapper/008:50:180
1069899543,28cyclictest7950irq/56-dwc_otg_09:15:323
1069899543,28cyclictest7950irq/56-dwc_otg_09:15:323
1069799543,24cyclictest0-21swapper/212:05:292
1069799543,24cyclictest0-21swapper/212:05:292
1069699543,27cyclictest7750irq/56-dwc_otg07:25:211
1069699542,35cyclictest0-21swapper/107:33:571
1069699541,39cyclictest0-21swapper/111:17:571
1069599542,7cyclictest29303-21ssh12:35:140
10695995425,14cyclictest140-21systemd-journal07:25:040
10695995425,14cyclictest140-21systemd-journal07:25:040
10695995416,7cyclictest0-21swapper/012:00:160
1069599540,20cyclictest7750irq/56-dwc_otg08:45:160
1069899538,27cyclictest0-21swapper/307:45:183
10695995329,5cyclictest7850irq/56-dwc_otg_08:40:280
1069599532,41cyclictest0-21swapper/007:40:150
1069599532,27cyclictest15445-21/usr/sbin/munin10:20:260
1069799525,30cyclictest444-21ssh12:00:162
10697995214,8cyclictest7850irq/56-dwc_otg_09:25:342
1069699521,39cyclictest21501-21chrt09:08:501
1069599523,37cyclictest0-21swapper/007:26:360
10695995232,6cyclictest7850irq/56-dwc_otg_11:10:240
10695995214,24cyclictest0-21swapper/008:05:280
1069599521,28cyclictest4494-21unixbench_singl10:45:330
1069799514,24cyclictest0-21swapper/211:59:492
1069799514,16cyclictest407-21ls11:20:262
1069799513,26cyclictest13764-21ssh12:15:322
10697995112,9cyclictest7850irq/56-dwc_otg_09:20:172
1069699512,45cyclictest22267-21awk07:40:261
1069699511,34cyclictest0-21swapper/108:37:071
1069599516,31cyclictest0-21swapper/010:30:200
1069599514,33cyclictest1819-21cut08:15:180
10695995134,10cyclictest0-21swapper/009:50:160
1069599513,26cyclictest0-21swapper/008:10:240
10695995131,6cyclictest7850irq/56-dwc_otg_07:50:270
1069599512,35cyclictest0-21swapper/007:30:270
10695995110,6cyclictest13628-21unixbench_multi07:15:310
1069899505,39cyclictest0-21swapper/310:05:343
1069699501,45cyclictest8928-21latency_hist08:35:041
1069699501,40cyclictest0-21swapper/108:18:581
1069599501,39cyclictest0-21swapper/009:30:270
1069799497,25cyclictest0-21swapper/207:10:172
1069699492,43cyclictest11062-21grep07:10:211
1069699492,35cyclictest0-21swapper/107:15:531
1069699491,44cyclictest18462-21diskstats09:00:191
10695994925,4cyclictest7568-21grep08:30:200
1069599492,36cyclictest0-21swapper/011:53:000
1069599491,34cyclictest0-21swapper/009:40:210
1069899488,25cyclictest0-21swapper/308:20:193
1069899488,25cyclictest0-21swapper/307:30:173
1069799486,26cyclictest12153-21ssh11:35:092
1069799486,26cyclictest12153-21ssh11:35:092
10695994817,15cyclictest16369-21acpi11:40:150
1069899478,23cyclictest0-21swapper/311:40:173
10698994716,7cyclictest7750irq/56-dwc_otg08:45:043
1069799475,5cyclictest0-21swapper/207:45:152
1069599474,25cyclictest29561-21proc_pri08:00:300
1069599474,25cyclictest29561-21proc_pri08:00:300
1069799466,20cyclictest0-21swapper/211:45:172
1069599462,34cyclictest0-21swapper/007:55:300
100902453,35sleep00-21swapper/007:05:510
1069899447,22cyclictest0-21swapper/308:35:173
1069899445,25cyclictest0-21swapper/307:40:173
1069599441,36cyclictest0-21swapper/008:25:200
1069899438,24cyclictest0-21swapper/311:00:193
1069899437,22cyclictest0-21swapper/307:20:183
1069899437,22cyclictest0-21swapper/307:20:183
10698994319,7cyclictest7950irq/56-dwc_otg_09:30:153
10698994311,18cyclictest381ktimersoftd/308:45:193
1069799437,22cyclictest0-21swapper/208:05:182
1069799433,13cyclictest0-21swapper/209:10:192
10697994325,11cyclictest15448-21grep09:40:312
1069799424,21cyclictest0-21swapper/210:00:362
1069799421,5cyclictest0-21swapper/208:45:162
1069899418,23cyclictest0-21swapper/307:55:153
1069899417,22cyclictest0-21swapper/308:30:173
1069799417,23cyclictest0-21swapper/208:50:102
1069799414,4cyclictest0-21swapper/212:10:162
10698994013,21cyclictest381ktimersoftd/310:20:293
10698994013,15cyclictest7850irq/56-dwc_otg_08:50:213
1069799401,8cyclictest0-21swapper/210:15:172
1069799401,8cyclictest0-21swapper/210:15:172
1069799401,25cyclictest0-21swapper/208:10:172
1069899397,23cyclictest0-21swapper/308:00:183
1069899397,23cyclictest0-21swapper/308:00:183
10698993920,8cyclictest7750irq/56-dwc_otg07:50:233
10698993920,11cyclictest3209-21/usr/sbin/munin10:45:343
10698993919,10cyclictest7950irq/56-dwc_otg_10:30:193
10697993918,5cyclictest3025-21seq08:16:082
1069799391,5cyclictest22544-21ssh09:50:172
10698993818,5cyclictest24500-21sh11:10:243
10697993818,9cyclictest29653-21grep12:35:202
1069799381,22cyclictest21070-21ls09:05:302
1069799381,21cyclictest12754-21cut07:15:202
1069899375,16cyclictest7950irq/56-dwc_otg_07:35:173
1069899372,22cyclictest19988-21ssh09:46:093
1069899372,22cyclictest19988-21ssh09:46:093
10698993720,4cyclictest1180-21runrttasks08:05:233
10698993714,4cyclictest0-21swapper/309:55:303
1069799375,4cyclictest7950irq/56-dwc_otg_08:03:412
1069799375,4cyclictest7950irq/56-dwc_otg_08:03:412
10697993728,3cyclictest7750irq/56-dwc_otg11:55:042
1069799371,25cyclictest18507-21cut09:00:202
1069799371,21cyclictest15902-21latency_hist07:25:052
1069799371,21cyclictest15902-21latency_hist07:25:052
1069899369,16cyclictest381ktimersoftd/312:39:493
10698993618,15cyclictest7950irq/56-dwc_otg_12:30:253
10698993613,11cyclictest381ktimersoftd/312:00:333
10697993615,5cyclictest19740-21/usr/sbin/munin07:35:322
1069799361,15cyclictest7950irq/56-dwc_otg_07:40:322
10698993513,4cyclictest7750irq/56-dwc_otg10:15:153
10698993513,4cyclictest7750irq/56-dwc_otg10:15:153
1069899350,21cyclictest7850irq/56-dwc_otg_07:25:333
1069799351,4cyclictest0-21swapper/207:30:172
1069899342,23cyclictest381ktimersoftd/310:12:133
10697993416,8cyclictest7850irq/56-dwc_otg_09:50:042
10697993416,8cyclictest7850irq/56-dwc_otg_09:50:042
1069899336,24cyclictest0-21swapper/308:15:173
1069899334,5cyclictest0-21swapper/309:40:153
1069899332,20cyclictest0-21swapper/309:52:073
1069899332,14cyclictest29986-21rm11:16:443
10698993310,4cyclictest7750irq/56-dwc_otg12:11:373
1069799338,12cyclictest0-21swapper/209:58:082
1069799336,24cyclictest16603-21switchtime11:00:322
1069799332,16cyclictest25239-21aten_rbpower_cu07:50:172
1069799331,3cyclictest7950irq/56-dwc_otg_11:40:192
1069899326,5cyclictest7950irq/56-dwc_otg_09:00:213
1069899326,14cyclictest0-21swapper/307:15:163
1069899322,13cyclictest9624-21ssh10:52:153
1069799326,11cyclictest7950irq/56-dwc_otg_08:20:272
1069799322,12cyclictest7750irq/56-dwc_otg10:35:032
1069799321,20cyclictest10000-21ls08:35:292
10697993210,5cyclictest0-21swapper/209:30:162
1069799316,4cyclictest7850irq/56-dwc_otg_11:15:212
1069799311,20cyclictest25826-21ssh12:30:262
1069799311,14cyclictest7750irq/56-dwc_otg08:30:292
1069899306,14cyclictest0-21swapper/308:25:183
1069899301,23cyclictest20615-21missed_timers09:05:243
1069799304,9cyclictest7950irq/56-dwc_otg_07:25:182
1069799301,5cyclictest11224-21uniq08:40:222
1069899292,19cyclictest0-21swapper/308:12:153
1069899291,16cyclictest17295-21ls08:55:303
1069799296,4cyclictest7750irq/56-dwc_otg10:20:192
1069799293,10cyclictest7850irq/56-dwc_otg_10:45:172
96052283,12sleep28866-21/usr/sbin/munin07:05:282
93222283,13sleep19395-21latency07:05:231
1069799286,12cyclictest0-21swapper/211:10:172
1069799274,4cyclictest7850irq/56-dwc_otg_07:55:232
87612263,14sleep30-21swapper/307:05:053
1069799264,7cyclictest1180-21runrttasks08:26:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional