You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 13:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Wed May 27, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
203276821940,5sleep30-21swapper/319:50:393
20933022840,4chrt0-21swapper/120:50:491
1991053996857,7cyclictest1704890-21systemd-journal19:45:470
1991055996555,7cyclictest2082049-21latency_hist20:40:011
1991055996555,7cyclictest2082049-21latency_hist20:40:001
1991053996157,3cyclictest2076901-21kworker/u8:1+events_unbound23:55:590
1991053996154,5cyclictest2017546-21kworker/u8:0+flush-8:019:41:000
1991053995956,2cyclictest2304567-21kworker/u8:3+events_unbound00:30:050
1991055995848,8cyclictest2026602-21ntpq19:40:521
1991053995854,3cyclictest2131712-21kworker/u8:0+events_unbound23:45:560
1991053995848,8cyclictest2186565-21/usr/sbin/munin22:25:590
1991055995750,5cyclictest2017546-21kworker/u8:0+flush-8:019:45:521
1991053995652,3cyclictest1971342-21kworker/u8:2+events_unbound21:45:460
1991053995551,3cyclictest2190600-21kworker/u8:3+events_unbound22:50:430
1991053995551,3cyclictest2076901-21kworker/u8:1+events_unbound23:05:450
1991053995450,3cyclictest1971342-21kworker/u8:2+events_unbound20:30:440
1991053995450,3cyclictest1971342-21kworker/u8:2+events_unbound19:25:420
1991053995349,3cyclictest2076901-21kworker/u8:1+events_unbound23:15:470
1991053995344,7cyclictest2265288-21latency_hist23:45:020
1991055995243,6cyclictest1704890-21systemd-journal22:50:471
1991053995249,2cyclictest2076901-21kworker/u8:1+events_unbound00:25:470
1991053995148,2cyclictest1971342-21kworker/u8:2+events_unbound21:40:490
1991053995147,3cyclictest1971342-21kworker/u8:2+events_unbound20:50:490
1991053995144,5cyclictest1971342-21kworker/u8:2+flush-8:022:20:530
1991053995047,2cyclictest1938639-21kworker/u8:3+events_unbound19:11:040
1991053994945,3cyclictest2190600-21kworker/u8:3+events_unbound23:00:580
1991053994945,3cyclictest2190600-21kworker/u8:3+events_unbound22:56:030
1991053994945,3cyclictest1938639-21kworker/u8:3+events_unbound19:31:030
1991053994845,2cyclictest2076901-21kworker/u8:1+events_unbound22:19:480
1991053994744,2cyclictest2131712-21kworker/u8:0+events_unbound22:41:010
1991053994740,3cyclictest2304567-21kworker/u8:3+events_unbound00:36:480
1991053994643,2cyclictest2017546-21kworker/u8:0+events_unbound19:52:490
1991053994642,3cyclictest2279669-21kworker/u8:2+events_unbound00:09:490
1991053994642,3cyclictest2131712-21kworker/u8:0+events_unbound23:30:570
1991053994641,4cyclictest1938639-21kworker/u8:3+flush-8:021:15:540
1991053994640,2cyclictest2279669-21kworker/u8:2+events_unbound00:14:480
1991053994639,5cyclictest2017546-21kworker/u8:0+events_unbound20:05:020
1991053994542,2cyclictest2076901-21kworker/u8:1+events_unbound21:22:480
1991053994541,3cyclictest2131712-21kworker/u8:0+events_unbound22:07:290
1991053994541,3cyclictest2131712-21kworker/u8:0+events_unbound22:07:290
1991053994541,3cyclictest2131712-21kworker/u8:0+events_unbound21:52:480
1991053994541,3cyclictest1938639-21kworker/u8:3+events_unbound21:27:490
1991053994541,3cyclictest1938639-21kworker/u8:3+events_unbound20:39:490
1991053994541,3cyclictest1938639-21kworker/u8:3+events_unbound20:39:490
1991053994538,5cyclictest2006824-21ntpq19:20:540
1991053994441,2cyclictest2076901-21kworker/u8:1+events_unbound23:27:430
1991053994440,3cyclictest2131712-21kworker/u8:0+events_unbound21:34:480
1991053994440,3cyclictest2076901-21kworker/u8:1+events_unbound22:14:480
1991053994440,3cyclictest1971342-21kworker/u8:2+events_unbound21:07:490
1991053994440,3cyclictest1971342-21kworker/u8:2+events_unbound20:56:060
1991053994440,3cyclictest1938639-21kworker/u8:3+events_unbound20:12:100
1991055994333,8cyclictest2072422-21/usr/sbin/munin20:30:591
1991053994340,2cyclictest2017546-21kworker/u8:0+events_unbound20:17:100
1991053994340,2cyclictest1938639-21kworker/u8:3+events_unbound20:22:100
1991053994339,3cyclictest2131712-21kworker/u8:0+events_unbound23:21:490
1991053994339,3cyclictest2131712-21kworker/u8:0+events_unbound00:18:000
1991053994339,3cyclictest2076901-21kworker/u8:1+events_unbound00:01:010
1991053994339,3cyclictest1971342-21kworker/u8:2+events_unbound21:57:490
1991053994339,3cyclictest1938639-21kworker/u8:3+events_unbound21:03:460
1991053994339,3cyclictest1938639-21kworker/u8:3+events_unbound20:27:480
1991061994234,6cyclictest1704890-21systemd-journal21:15:472
1991055994226,9cyclictest2052688-21/usr/sbin/munin20:11:001
1991053994238,3cyclictest2190600-21kworker/u8:3+events_unbound22:46:010
1991053994238,3cyclictest2131712-21kworker/u8:0+events_unbound23:36:030
1991053994238,3cyclictest2076901-21kworker/u8:1+events_unbound22:31:490
1991053994238,3cyclictest1938639-21kworker/u8:3+events_unbound19:56:490
199076524228,9sleep10-21swapper/119:08:381
1991061994130,9cyclictest2240615-21awk23:20:002
1991061994124,14cyclictest2315011-21apt-get00:35:002
1991055994132,7cyclictest2146571-21/usr/sbin/munin21:45:581
1991053994137,3cyclictest2304567-21kworker/u8:3+events_unbound00:23:470
1991053994137,3cyclictest2190600-21kworker/u8:3+events_unbound23:53:540
1991053994137,3cyclictest2017546-21kworker/u8:0+events_unbound19:39:490
1991053994137,3cyclictest1971342-21kworker/u8:2+events_unbound21:37:220
1991053994137,3cyclictest1971342-21kworker/u8:2+events_unbound20:47:490
1991053994137,3cyclictest1971342-21kworker/u8:2+events_unbound20:41:480
1991053994137,3cyclictest1938639-21kworker/u8:3+events_unbound20:03:480
1991061994021,16cyclictest1704890-21systemd-journal21:35:462
1991055994031,7cyclictest2111988-21/usr/sbin/munin21:10:531
1991055994031,7cyclictest1991225-21/usr/sbin/munin19:10:551
1991053994037,2cyclictest2076901-21kworker/u8:1+events_unbound23:14:470
199106199392,16cyclictest2186590-21apt-get22:25:442
1991055993931,6cyclictest2226120-21apt-get23:05:401
1991053993935,3cyclictest2131712-21kworker/u8:0+events_unbound22:04:280
1991053993935,3cyclictest2131712-21kworker/u8:0+events_unbound22:04:270
1991053993834,3cyclictest2131712-21kworker/u8:0+events_unbound22:37:060
1991053993834,3cyclictest1938639-21kworker/u8:3+events_unbound21:12:390
199106199371,17cyclictest2260655-21apt-get23:40:432
1991055993731,4cyclictest2240919-21apt-get23:20:401
1991055993730,5cyclictest2047273-21latency_hist20:05:001
1991061993621,13cyclictest2122123-21cpuspeed_turbos21:20:432
1991061993620,14cyclictest2201141-21cron22:40:012
1991055993628,6cyclictest2225830-21latency_hist23:05:001
1991055993628,5cyclictest1704890-21systemd-journal21:31:471
1991053993632,3cyclictest1938639-21kworker/u8:3+events_unbound19:16:200
1991061993523,10cyclictest2018172-21fschecks_time19:35:442
1991061993521,12cyclictest2022887-21aten2_r5power_c19:40:422
1991055993529,4cyclictest1971342-21kworker/u8:2+flush-8:020:41:521
1991055993521,7cyclictest401-21dbus-daemon00:05:011
20435282340,2sleep375-21kworker/3:1+events20:00:473
199106199342,25cyclictest2275457-21apt-get23:55:432
1991055993428,4cyclictest2254599-21ntpq23:30:561
1991055993428,4cyclictest2106744-21cron21:05:001
1991055993426,6cyclictest2170969-21apt-get22:10:001
1991055993426,6cyclictest2170969-21apt-get22:10:001
1991061993319,12cyclictest2086612-21unixbench_multi20:41:012
1991055993327,4cyclictest1938639-21kworker/u8:3+flush-8:021:15:531
1991055993324,7cyclictest2274310-21ntpq23:50:531
1991055993323,8cyclictest2185218-21ntpq22:20:541
1991061993224,6cyclictest1704890-21systemd-journal19:12:472
199106199322,23cyclictest2295291-21apt-get00:15:422
199106199322,22cyclictest2240919-21apt-get23:20:402
1991061993217,13cyclictest0-21swapper/220:05:012
199105599326,10cyclictest0-21swapper/121:25:491
1991055993226,4cyclictest2007712-21sh19:25:011
199106199312,24cyclictest2185218-21ntpq22:20:542
199106199312,22cyclictest2206370-21apt-get22:45:402
199106199312,22cyclictest2131774-21apt-get21:30:422
199106199312,21cyclictest2216240-21apt-get22:55:432
1991055993126,3cyclictest2211034-21basename22:49:591
1991055993124,3cyclictest0-21swapper/122:05:001
1991055993124,3cyclictest0-21swapper/122:04:591
19862452313,9sleep20-21swapper/219:05:092
199106199302,23cyclictest2289161-21ntpq00:05:552
1991061993018,9cyclictest2300221-21munin-node00:20:412
199106199301,23cyclictest2007082-21/usr/sbin/munin19:20:592
199106199301,22cyclictest2156489-21apt-get21:55:422
199106199301,22cyclictest2047274-21apt-get20:05:022
1991061993012,15cyclictest2253126-21seq23:30:512
199106199300,7cyclictest2235495-21chrt23:12:392
199105599305,11cyclictest2195329-21ntpq22:30:561
199105599304,13cyclictest2039747-21kthreadcore19:55:511
1991055993024,5cyclictest2131712-21kworker/u8:0+flush-8:000:20:511
1991055993023,5cyclictest2161111-21cron22:00:001
1991061992924,3cyclictest2092519-21fschecks_count20:50:452
199106199291,24cyclictest2105880-21ntpq21:00:552
199106199291,22cyclictest2201457-21apt-get22:40:402
199105599298,10cyclictest0-21swapper/122:15:541
1991055992922,5cyclictest2091934-21latency_hist20:50:011
199105599291,13cyclictest2309028-21ntpq00:25:561
199105599290,16cyclictest1995913-21munin-node19:15:031
199106199287,18cyclictest2316369-21sed00:35:492
199106199282,20cyclictest2076267-21ntpq20:30:572
199106199281,23cyclictest2165219-21ntpq22:00:552
199106199281,23cyclictest2165219-21ntpq22:00:552
199106199281,22cyclictest2107976-21ps21:05:482
199106199281,21cyclictest2226120-21apt-get23:05:412
199105599284,10cyclictest30-21ksoftirqd/123:40:451
199105599284,10cyclictest2155289-21ntpq21:50:521
199105599283,11cyclictest2275431-21/usr/sbin/munin23:55:461
1991055992824,2cyclictest2071785-21grep20:26:021
1991055992823,3cyclictest2315890-21cut00:35:441
1991055992823,3cyclictest2126528-21cron21:25:001
1991055992822,4cyclictest2200767-21unixbench_singl22:36:001
1991055992822,4cyclictest2131712-21kworker/u8:0+flush-8:023:16:511
199105599282,14cyclictest2141627-21taskset21:40:151
199105599282,13cyclictest0-21swapper/120:15:551
1991055992820,6cyclictest2246145-21fschecks_time23:25:451
1991055992819,7cyclictest2101789-21cron21:00:011
199105599281,17cyclictest2066905-21taskset20:21:101
199105599281,12cyclictest2110890-21ntpq21:05:571
199105599280,18cyclictest2294373-21smartctl00:11:051
1991061992724,2cyclictest2091559-21unixbench_singl20:46:012
1991061992722,3cyclictest2270811-21fschecks_count23:50:442
1991061992712,12cyclictest0-21swapper/221:40:552
199106199271,11cyclictest2270230-21munin-run23:50:012
199105599274,11cyclictest2269369-21ntpq23:45:541
199105599273,12cyclictest2220012-21gcc22:55:551
1991055992717,2cyclictest161rcu_preempt19:30:561
199105599271,15cyclictest2259453-21ld23:35:531
199105599271,13cyclictest2140562-21collect221:35:561
199105599271,11cyclictest2201414-21taskset22:40:261
199105599270,14cyclictest2011998-21python319:26:001
199105599270,13cyclictest2036571-21ntpq19:50:541
199105399279,3cyclictest161rcu_preempt19:10:010
199106199268,15cyclictest0-21swapper/219:45:462
199106199263,18cyclictest2130663-21ntpq21:25:562
199106199261,8cyclictest2151936-21/usr/sbin/munin21:50:452
199106199261,8cyclictest2077754-21/usr/sbin/munin20:35:472
199106199261,8cyclictest2077754-21/usr/sbin/munin20:35:462
199106199261,21cyclictest2178981-21kthreadcore22:15:492
1991061992612,12cyclictest0-21swapper/219:54:592
199106199261,20cyclictest2308925-21ntpq00:25:522
199106199261,19cyclictest2171434-21apt-get22:10:032
1991061992611,13cyclictest0-21swapper/220:25:542
199106199260,9cyclictest2053164-21if_enp4s0.220:10:472
199105599263,12cyclictest2048289-21cat20:05:481
199105599263,11cyclictest2231493-21if_enp4s023:10:471
199105599263,10cyclictest2299704-21taskset00:17:171
1991055992621,3cyclictest0-21swapper/119:35:471
199105599262,10cyclictest30-21ksoftirqd/100:30:021
1991055992620,4cyclictest2285536-21aten2_r5power_v00:05:431
199105599261,14cyclictest2175315-21ntpq22:10:521
199106199257,9cyclictest20994392cut20:55:512
199106199257,16cyclictest2063535-21tr20:20:482
199106199252,19cyclictest2294190-21ntpq00:10:592
1991061992520,3cyclictest2250070-21unixbench_multi23:26:002
199106199251,8cyclictest2221593-21/usr/sbin/munin23:00:462
199106199251,8cyclictest2112011-21/usr/sbin/munin21:10:412
1991061992514,9cyclictest2171215-21gpgv22:10:012
1991061992514,9cyclictest2171215-21gpgv22:10:002
199106199251,20cyclictest2280735-21diskstats00:00:442
199106199251,20cyclictest2008264-21/usr/sbin/munin19:25:452
199106199250,9cyclictest2057997-21sort20:15:452
199106199250,10cyclictest2147882-21kthreadcore21:45:482
199105599253,9cyclictest2187038-21if_enp4s0.222:25:481
199106199246,16cyclictest2213069-21cut22:50:502
1991061992412,10cyclictest0-21swapper/219:55:592
199106199240,7cyclictest2256530-21irqrtprio23:35:462
199106199238,13cyclictest0-21swapper/219:15:402
199106199231,8cyclictest2013339-21awk19:30:452
199106199230,3cyclictest2191523-21/usr/sbin/munin22:30:402
19866882193,9sleep30-21swapper/319:05:473
22702112175,8sleep30-21swapper/323:50:013
22606532162,9sleep30-21swapper/323:40:423
21013142155,6sleep30-21swapper/320:56:063
22765722132,6sleep30-21swapper/323:55:493
22359622132,7sleep30-21swapper/323:15:423
22110532132,6sleep30-21swapper/322:50:013
21865642132,7sleep30-21swapper/322:25:423
21367082132,7sleep30-21swapper/321:35:423
21118062132,6sleep30-21swapper/321:10:023
21075442130,9sleep30-21swapper/321:05:493
20564622132,7sleep30-21swapper/320:10:563
20225852132,6sleep30-21swapper/319:40:023
23098582122,5sleep30-21swapper/300:30:013
23001312122,5sleep30-21swapper/300:20:023
22714092122,6sleep30-21swapper/323:50:493
22521212122,6sleep30-21swapper/323:30:513
22413892124,5sleep30-21swapper/323:20:463
22214532122,6sleep30-21swapper/323:00:453
22162432122,6sleep30-21swapper/322:55:443
22066152122,6sleep30-21swapper/322:45:443
21614442122,6sleep30-21swapper/322:00:423
21614442122,6sleep30-21swapper/322:00:423
21560812122,6sleep30-21swapper/321:53:483
21463142122,5sleep30-21swapper/321:45:013
21455852122,6sleep30-21swapper/321:40:593
21312892122,6sleep30-21swapper/321:27:473
21215732122,5sleep30-21swapper/321:20:013
21169532122,6sleep30-21swapper/321:15:443
21021212122,6sleep30-21swapper/321:00:433
20962292122,6sleep30-21swapper/320:50:593
20919002122,6sleep30-21swapper/320:49:483
20828392123,6sleep30-21swapper/320:40:493
20790402123,5sleep30-21swapper/320:35:513
20790402123,5sleep30-21swapper/320:35:503
20724452122,6sleep30-21swapper/320:30:423
20674472124,5sleep30-21swapper/320:25:073
20221742122,6sleep30-21swapper/319:36:003
20129262122,6sleep30-21swapper/319:30:433
19981192122,6sleep30-21swapper/319:15:423
19951442122,6sleep30-21swapper/319:11:013
23155452112,5sleep30-21swapper/300:35:433
23086492112,6sleep30-21swapper/300:25:523
22958092112,5sleep30-21swapper/300:15:493
22903312112,5sleep30-21swapper/300:10:423
22889512112,5sleep30-21swapper/300:05:543
22804602112,5sleep30-21swapper/300:00:403
22593892112,5sleep30-21swapper/323:35:523
22460812112,5sleep30-21swapper/323:25:453
22340192111,6sleep30-21swapper/323:10:523
22261522112,6sleep30-21swapper/323:05:423
22013842112,5sleep30-21swapper/322:40:033
21964852112,5sleep30-21swapper/322:35:403
21915292112,5sleep30-21swapper/322:30:443
21813642112,6sleep30-21swapper/322:20:163
21769102112,5sleep30-21swapper/322:15:473
21710312112,5sleep30-21swapper/322:10:013
21663692112,6sleep30-21swapper/322:05:443
21663692112,6sleep30-21swapper/322:05:433
21565482111,6sleep30-21swapper/321:55:443
21317482112,5sleep30-21swapper/321:30:413
20625052112,5sleep30-21swapper/320:20:023
20612112112,5sleep30-21swapper/320:15:533
20480532112,5sleep30-21swapper/320:05:463
20376082112,5sleep30-21swapper/319:55:033
20283102111,6sleep30-21swapper/319:45:483
20082562112,5sleep30-21swapper/319:25:453
20070812112,6sleep30-21swapper/319:20:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional