You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 22:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri May 15, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211255621530,3sleep3451irq_work/307:36:013
22952112660,6chrt0-21swapper/210:40:592
22952112660,6chrt0-21swapper/210:40:592
2081703996557,5cyclictest2153954-21cat08:20:002
2081700996353,8cyclictest2187685-21ntpq08:51:041
21049802600,2sleep0201rcuc/007:30:570
2081700995750,5cyclictest2263578-21/usr/sbin/munin10:11:011
2081698995649,5cyclictest2081562-21kworker/u8:0+flush-8:008:35:540
2081698995649,5cyclictest2081562-21kworker/u8:0+flush-8:008:35:540
2081700995547,6cyclictest2193698-21apt-get09:00:001
2081700995547,6cyclictest2167821-21ntpq08:31:021
2081698995546,6cyclictest2407399-21latency_hist12:35:000
2081698995544,9cyclictest2187640-21ntpq08:51:030
2081698995450,3cyclictest2327688-21kworker/u8:2+events_unbound11:46:070
2081698995449,3cyclictest2347520-21kworker/u8:0+events_unbound11:56:080
2081698995346,5cyclictest2081562-21kworker/u8:0+flush-8:007:20:010
2081700995243,7cyclictest2313370-21/usr/sbin/munin11:01:011
2081698995245,5cyclictest2081562-21kworker/u8:0+flush-8:007:46:060
2081698995244,6cyclictest2038822-21kworker/u8:3+flush-8:007:25:550
2081703995140,8cyclictest2087218-21ntpq07:15:032
2081698995146,4cyclictest2392247-21kworker/u8:2+events_unbound12:20:000
2081698995146,4cyclictest2392247-21kworker/u8:2+events_unbound12:19:590
2081698995143,6cyclictest2252574-21ntpq09:56:030
2081703995040,8cyclictest2238394-21apt-get09:45:002
2081698995043,5cyclictest2268540-21apt-get10:15:500
2081703994941,5cyclictest280-21systemd-journal12:31:062
2081700994943,4cyclictest2347520-21kworker/u8:0+flush-8:011:55:541
24031822480,3sleep10-21swapper/112:30:541
2081703994841,5cyclictest280-21systemd-journal11:40:352
2081703994837,8cyclictest2193698-21apt-get09:00:012
2081700994839,6cyclictest2223545-21latency_hist09:30:011
2081698994841,5cyclictest2082167-21diskstats07:10:540
2081700994741,4cyclictest280-21systemd-journal07:43:351
2081700994640,5cyclictest2183309-21kworker/u8:1+flush-8:011:36:031
2081700994640,4cyclictest2158534-21kworker/u8:2+flush-8:009:35:591
2081703994540,4cyclictest2081562-21kworker/u8:0+flush-8:007:36:062
2081700994537,6cyclictest2318375-21apt-get11:05:491
2081698994536,6cyclictest2352754-21latency_hist11:40:010
2081698994439,3cyclictest2362703-21kworker/u8:3+flush-8:012:06:050
208169899441,40cyclictest1-21systemd08:25:000
21416672430,3sleep375-21kworker/3:1+events08:05:593
2081700994334,6cyclictest2238395-21latency_hist09:45:011
2081700994333,7cyclictest2173657-21latency_hist08:40:011
2081700994333,7cyclictest2173657-21latency_hist08:40:011
2081698994337,4cyclictest2133231-21kworker/u8:3+flush-8:009:05:550
2081698994336,5cyclictest2119075-21latency_hist07:45:010
21961542420,4sleep00-21swapper/009:00:570
2081700994232,7cyclictest2099002-21latency_hist07:25:011
2081698994235,5cyclictest280-21systemd-journal09:49:580
2081698994231,8cyclictest2113191-21ntpq07:36:040
2081698994230,9cyclictest280-21systemd-journal08:00:000
2081700994134,5cyclictest2038822-21kworker/u8:3+flush-8:007:11:071
2081698994136,4cyclictest2133231-21kworker/u8:3+flush-8:010:00:530
2081700994035,4cyclictest2327688-21kworker/u8:2+events_unbound12:05:001
2081700994034,4cyclictest2099310-21apt-get07:25:501
2081698994035,4cyclictest2158534-21kworker/u8:2+flush-8:009:30:580
2081698994028,10cyclictest280-21systemd-journal08:20:000
2081700993931,6cyclictest2273496-21apt-get10:20:481
2081700993931,5cyclictest2392567-21latency_hist12:20:011
2081700993931,5cyclictest2392567-21latency_hist12:20:001
2081700993928,8cyclictest2327083-21ntpq11:11:021
2081698993935,3cyclictest2183309-21kworker/u8:1+flush-8:011:01:050
2081698993930,7cyclictest2401588-21ntpq12:26:030
2081703993832,4cyclictest2133231-21kworker/u8:3+flush-8:009:31:072
2081700993832,4cyclictest280-21systemd-journal08:06:351
2081700993829,6cyclictest2081541-21latency_hist07:10:001
2081698993832,4cyclictest2158534-21kworker/u8:2+flush-8:010:26:070
2081700993730,5cyclictest280-21systemd-journal07:45:361
2081698993733,3cyclictest2327688-21kworker/u8:2+events_unbound11:30:520
2081698993730,5cyclictest2183309-21kworker/u8:1+flush-8:011:21:100
2081698993728,7cyclictest2377485-21latency_hist12:05:010
2081700993629,5cyclictest2153633-21apt-get08:20:001
2081700993626,7cyclictest2263294-21latency_hist10:10:001
2081698993632,2cyclictest2284112-21irqrtprio10:30:560
2081698993626,7cyclictest2347520-21kworker/u8:0+flush-8:012:11:040
2081703993529,4cyclictest2133231-21kworker/u8:3+events_unbound09:00:022
2081700993529,4cyclictest2347520-21kworker/u8:0+flush-8:011:51:101
2081700993528,5cyclictest280-21systemd-journal10:05:001
2081700993527,6cyclictest2283366-21/usr/sbin/munin10:31:011
2081700993526,5cyclictest2327688-21kworker/u8:2+events_unbound11:45:531
2081698993531,3cyclictest2133231-21kworker/u8:3+events_unbound09:51:080
2081698993528,5cyclictest2347520-21kworker/u8:0+events_unbound11:51:060
2081698993528,5cyclictest2183309-21kworker/u8:1+events_unbound10:11:050
2081698993522,9cyclictest280-21systemd-journal08:35:000
2081700993429,4cyclictest2347520-21kworker/u8:0+events_unbound12:11:081
2081700993428,4cyclictest2243893-21apt-get09:50:491
2081700993428,4cyclictest2124291-21apt-get07:50:501
2081698993430,3cyclictest2183309-21kworker/u8:1+events_unbound09:20:580
2081698993429,4cyclictest2038822-21kworker/u8:3+events_unbound07:21:040
2081698993428,4cyclictest2103931-21kworker/u8:1+flush-8:008:03:080
2081698993426,6cyclictest2081562-21kworker/u8:0+flush-8:008:06:040
2081700993329,3cyclictest2327688-21kworker/u8:2+flush-8:011:45:011
2081700993328,3cyclictest2081562-21kworker/u8:0+flush-8:007:30:421
2081700993323,8cyclictest280-21systemd-journal08:01:371
2081698993327,4cyclictest2081562-21kworker/u8:0+flush-8:008:10:530
2081698993323,7cyclictest2219864-21kthreadcore09:25:590
2081700993230,1cyclictest2327688-21kworker/u8:2+flush-8:011:31:081
2081698993227,4cyclictest2283104-21kworker/u8:0+events_unbound10:45:540
2081698993226,4cyclictest2393199-21fschecks_count12:20:550
2081698993226,4cyclictest2183309-21kworker/u8:1+flush-8:010:56:060
2081698993226,4cyclictest2158534-21kworker/u8:2+flush-8:010:06:100
2081698993222,8cyclictest2332060-21ntpq11:16:020
2081703993123,6cyclictest280-21systemd-journal10:57:352
2081700993126,4cyclictest2081562-21kworker/u8:0+flush-8:008:11:061
2081700993124,5cyclictest2132998-21ntpq07:56:021
2081700993123,6cyclictest280-21systemd-journal10:36:351
2081700993123,6cyclictest280-21systemd-journal10:36:351
2081700993122,7cyclictest2283104-21kworker/u8:0+flush-8:010:51:081
2081698993125,4cyclictest2347520-21kworker/u8:0+events_unbound11:40:570
2081698993124,4cyclictest2081562-21kworker/u8:0+flush-8:008:45:590
2081698993116,12cyclictest0-21swapper/009:16:020
2081700993026,2cyclictest2183571-21systemd-run08:50:011
2081700993025,4cyclictest2133231-21kworker/u8:3+flush-8:009:32:391
2081698993023,5cyclictest2408063-21fschecks_time12:35:520
2081703992924,3cyclictest2298536-21aten2_r5power_e10:45:522
2081700992924,4cyclictest2313015-21munin-plugin-st11:00:011
2081698992925,3cyclictest2283104-21kworker/u8:0+events_unbound10:51:070
2081698992924,3cyclictest2163375-21unixbench_multi08:26:090
2081698992923,4cyclictest2342477-21unixbench_singl11:26:090
2081698992922,5cyclictest2327643-21unixbench_singl11:11:090
2081698992922,5cyclictest2273763-21fschecks_count10:20:530
2081703992824,2cyclictest2119603-21cpuspeed_turbos07:45:522
2081703992823,4cyclictest2133231-21kworker/u8:3+events_unbound09:46:042
2081703992820,6cyclictest2318642-21fschecks_count11:05:532
2081700992825,2cyclictest2253439-21munin-run10:00:001
2081700992821,6cyclictest2158534-21kworker/u8:2+flush-8:010:26:011
2081700992821,6cyclictest2088863-21kworker/u8:2+flush-8:007:16:031
2081700992821,5cyclictest2183309-21kworker/u8:1+flush-8:009:21:051
2081700992818,7cyclictest280-21systemd-journal12:37:341
2081698992822,4cyclictest2178595-21cron08:45:000
2081698992821,5cyclictest2125214-21kthreadcore07:50:580
2081698992821,4cyclictest2283104-21kworker/u8:0+flush-8:010:41:000
2081698992821,4cyclictest2283104-21kworker/u8:0+flush-8:010:40:590
2081698992820,7cyclictest2183309-21kworker/u8:1+flush-8:011:06:040
2081703992722,3cyclictest2188532-21apt-get08:55:012
2081700992723,2cyclictest0-21swapper/111:25:561
2081698992723,3cyclictest2133231-21kworker/u8:3+flush-8:009:11:070
2081698992721,4cyclictest2229163-21fschecks_time09:35:540
2081703992622,3cyclictest2378258-21cpuspeed_turbos12:05:532
2081703992622,3cyclictest2183309-21kworker/u8:1+flush-8:011:25:012
2081700992620,4cyclictest2333275-21apt-get11:20:501
2081700992620,4cyclictest2183309-21kworker/u8:1+flush-8:010:20:011
2081700992620,4cyclictest2159137-21apt-get08:25:501
2081700992617,7cyclictest2202926-21ntpq09:06:041
2081698992616,6cyclictest0-21swapper/008:55:500
2081703992518,5cyclictest2086739-21cp07:15:002
2081700992522,2cyclictest2332982-21systemd-run11:20:001
2081700992521,2cyclictest2109583-21cpuspeed07:35:541
2081700992521,2cyclictest0-21swapper/112:25:201
2081700992520,3cyclictest2208362-21unixbench_multi09:11:111
2081700992518,5cyclictest280-21systemd-journal09:46:341
2081698992520,3cyclictest2238551-21apt-key09:45:010
2081703992421,2cyclictest2214357-21fschecks_time09:20:552
2081703992420,2cyclictest2243951-21aten2_r5power_e09:50:512
2081703992419,4cyclictest20977-21cron12:15:002
2081703992419,3cyclictest2099597-21fschecks_count07:25:542
2081703992416,5cyclictest2402676-21cat12:30:022
2081700992420,3cyclictest0-21swapper/110:45:001
2081700992420,3cyclictest0-21swapper/110:45:001
2081703992319,3cyclictest2173316-21unixbench_singl08:36:102
2081703992319,3cyclictest2173316-21unixbench_singl08:36:102
2081703992317,4cyclictest2392219-21unixbench_singl12:16:102
2081703992317,4cyclictest2392219-21unixbench_singl12:16:102
2081700992319,3cyclictest2133231-21kworker/u8:3+flush-8:008:41:031
2081700992318,3cyclictest2393819-21kernelversion12:20:581
2081700992315,6cyclictest2209046-21apt-get09:15:481
2081698992317,4cyclictest413-21systemd-logind10:40:000
2081698992317,4cyclictest413-21systemd-logind10:39:590
21031162220,5sleep30-21swapper/307:26:063
2081703992219,2cyclictest2114643-21cpuspeed_turbos07:40:532
2081703992218,2cyclictest0-21swapper/212:05:002
2081703992217,3cyclictest2314319-21kthreadcore11:00:582
2081703992217,3cyclictest2283081-21apt10:30:002
2081703992217,3cyclictest2273166-21idleruntime-cro10:20:002
2081700992218,2cyclictest0-21swapper/110:46:011
20814772223,13sleep00-21swapper/007:09:270
2081703992118,2cyclictest2098994-21idleruntime-cro07:25:002
2081703992111,8cyclictest2372550-21cron12:00:002
208170399210,8cyclictest21314551kthreadcore07:55:582
2081700992117,2cyclictest0-21swapper/112:05:571
20812202214,10sleep20-21swapper/207:06:152
2081703992017,2cyclictest2148302-21unixbench_multi08:11:092
2081703992016,2cyclictest0-21swapper/211:30:002
2081703992016,2cyclictest0-21swapper/207:35:012
2081703992015,4cyclictest2327688-21kworker/u8:2+flush-8:011:49:452
2081703992014,4cyclictest2327688-21kworker/u8:2+flush-8:011:35:052
2081703992010,7cyclictest2250930-21tr09:55:582
2081700992016,2cyclictest0-21swapper/108:20:561
2081700992015,3cyclictest0-21swapper/109:00:521
20812342204,9sleep30-21swapper/307:06:163
208170399192,12cyclictest2258693-21apt-get10:05:512
2081703991916,2cyclictest2392994-21aten2_r5power_p12:20:522
2081703991915,3cyclictest0-21swapper/210:51:102
2081703991915,2cyclictest2328557-21fschecks_count11:15:532
2081703991915,2cyclictest0-21swapper/211:50:152
2081703991914,4cyclictest0-21swapper/210:36:072
2081703991914,4cyclictest0-21swapper/210:36:062
2081703991910,7cyclictest2125933-21tr07:50:582
2081703991815,2cyclictest2323646-21fschecks_count11:10:542
2081703991813,3cyclictest2213672-21sh09:20:002
208170399177,8cyclictest280-21systemd-journal10:23:352
2081703991714,2cyclictest2219292-21fschecks_time09:25:562
2081703991713,2cyclictest0-21swapper/208:06:102
2081703991710,5cyclictest2411973-21unixbench_singl12:36:072
2081703991710,5cyclictest2285760-21tr10:30:582
23480852163,8sleep30-21swapper/311:35:513
22140092162,9sleep30-21swapper/309:20:513
208170399168,6cyclictest2137247-21tr08:00:592
208170399166,7cyclictest2266500-21tr10:10:592
208170399162,13cyclictest0-21swapper/208:25:302
208170399162,10cyclictest2167933-21ntpq08:31:072
2081703991613,2cyclictest0-21swapper/209:35:152
2081703991613,2cyclictest0-21swapper/209:11:022
2081703991612,3cyclictest0-21swapper/208:40:572
2081703991612,2cyclictest0-21swapper/208:46:012
2081703991612,2cyclictest0-21swapper/208:20:532
2081703991611,3cyclictest0-21swapper/209:06:002
208170399157,6cyclictest2254600-21sed10:00:572
2081703991511,2cyclictest0-21swapper/211:30:472
23063772142,8sleep30-21swapper/310:51:013
22735002140,10sleep30-21swapper/310:20:513
24074182132,6sleep30-21swapper/312:35:023
24030392132,7sleep30-21swapper/312:30:523
23966812132,6sleep30-21swapper/312:21:053
23828732132,6sleep30-21swapper/312:10:013
23080722132,6sleep30-21swapper/310:55:023
22887242133,6sleep30-21swapper/310:35:563
22887242133,6sleep30-21swapper/310:35:563
22585022132,6sleep30-21swapper/310:05:023
22087622133,6sleep30-21swapper/309:15:023
21835662132,6sleep30-21swapper/308:50:023
21536222132,7sleep30-21swapper/308:20:003
24023852124,5sleep30-21swapper/312:28:523
23894732122,6sleep30-21swapper/312:16:003
23894732122,6sleep30-21swapper/312:16:003
23578102122,6sleep30-21swapper/311:45:023
23431552122,6sleep30-21swapper/311:30:533
23428162122,5sleep30-21swapper/311:30:013
23329842123,5sleep30-21swapper/311:20:013
23321242122,6sleep30-21swapper/311:16:053
23133002122,6sleep30-21swapper/311:00:363
23029722122,6sleep30-21swapper/310:48:363
22872392122,6sleep30-21swapper/310:31:063
22635572122,6sleep30-21swapper/310:10:353
22533012122,6sleep30-21swapper/309:58:363
22393282122,6sleep30-21swapper/309:45:563
22334942122,6sleep30-21swapper/309:40:023
22056882123,5sleep30-21swapper/309:11:013
21890762122,6sleep30-21swapper/308:55:523
21752102122,6sleep30-21swapper/308:40:593
21596632122,6sleep30-21swapper/308:25:583
21542062122,6sleep30-21swapper/308:20:533
21437782122,5sleep30-21swapper/308:10:013
21340072122,6sleep30-21swapper/308:00:023
21291812122,6sleep30-21swapper/307:55:373
21183162122,6sleep30-21swapper/307:41:083
21041772122,6sleep30-21swapper/307:30:363
20960242122,6sleep30-21swapper/307:21:013
20872192122,6sleep30-21swapper/307:15:043
20816942122,6sleep30-21swapper/307:10:023
23780392112,5sleep30-21swapper/312:05:533
23739872112,5sleep30-21swapper/312:00:593
23676242112,5sleep30-21swapper/311:55:003
23630032112,5sleep30-21swapper/311:50:523
23536052112,5sleep30-21swapper/311:40:573
23237272112,5sleep30-21swapper/311:10:553
23182942112,5sleep30-21swapper/311:05:083
22930162112,5sleep30-21swapper/310:40:013
22930162112,5sleep30-21swapper/310:40:013
22682302111,5sleep30-21swapper/310:15:013
22577532112,6sleep30-21swapper/310:01:093
22482372112,6sleep30-21swapper/309:51:543
22293072112,6sleep30-21swapper/309:35:563
22238562112,5sleep30-21swapper/309:30:513
22212352112,5sleep30-21swapper/309:26:013
22030092112,5sleep30-21swapper/309:06:073
21980972110,7sleep30-21swapper/309:01:053
21788532112,6sleep30-21swapper/308:45:193
21687372112,5sleep30-21swapper/308:35:013
21687372112,5sleep30-21swapper/308:35:003
21640392112,5sleep30-21swapper/308:30:283
21255662112,5sleep30-21swapper/307:50:593
21196272112,5sleep30-21swapper/307:45:523
22783482102,5sleep30-21swapper/310:25:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional