You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 02:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Mon Apr 27, 2026 00:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
197368899543,40cyclictest2031170-21ssh21:35:470
1973689995343,6cyclictest0-21swapper/100:05:471
1973690995243,6cyclictest2171473-21latency_hist00:30:022
1973690995141,8cyclictest2136746-21unixbench_singl23:45:552
1973688995139,9cyclictest1-21systemd00:10:010
1973690995039,8cyclictest421-21dbus-daemon00:10:012
1973688995043,4cyclictest2086489-21latency_hist22:45:020
1973688995017,17cyclictest2053835-21sh22:04:300
1973688994919,10cyclictest2007582-21ntpq21:00:420
1973688994919,10cyclictest2007582-21ntpq21:00:420
1973690994840,5cyclictest2033880-21idleruntime-cro21:40:012
1973689994837,7cyclictest548-21gdbus23:20:011
1973689994835,7cyclictest0-21swapper/123:20:571
1973688994818,8cyclictest2011080-21ntpq21:10:500
1973688994817,9cyclictest2104007-21ntpq23:05:490
1973688994817,9cyclictest2104007-21ntpq23:05:490
1973690994735,5cyclictest0-21swapper/223:30:442
1973690994640,3cyclictest0-21swapper/221:40:442
1973690994640,3cyclictest0-21swapper/221:40:442
1973690994640,3cyclictest0-21swapper/220:00:002
1973690994640,3cyclictest0-21swapper/220:00:002
1973690994637,5cyclictest2082416-21idleruntime-cro22:40:002
1973688994619,9cyclictest2125854-21ssh23:33:130
1973688994618,7cyclictest2176723-21ntpq00:35:450
1973688994618,18cyclictest0-21swapper/020:30:440
197368999453,32cyclictest2047337-21ssh21:55:471
1973688994518,9cyclictest2087675-21ntpq22:45:430
1973688994516,6cyclictest2058903-21grep22:10:340
1973688994515,10cyclictest2020590-21ssh21:22:430
1973690994427,12cyclictest0-21swapper/221:50:542
1973690994427,12cyclictest0-21swapper/221:50:542
1973688994433,8cyclictest0-21swapper/023:40:010
1973688994432,7cyclictest2079629-21ntpq22:35:440
1973688994419,16cyclictest0-21swapper/020:40:460
1973688994418,9cyclictest2003145-21ntpq20:45:450
1973688994418,8cyclictest2135113-21sh23:45:040
1973688994418,14cyclictest1996704-21cpuspeed20:25:360
1973688994418,10cyclictest2165578-21sh00:21:390
1973690994335,4cyclictest0-21swapper/222:25:492
1973690994335,4cyclictest0-21swapper/222:25:492
1973689994336,3cyclictest0-21swapper/121:50:491
1973689994336,3cyclictest0-21swapper/121:50:491
1973688994329,8cyclictest2009156-21ntpq21:05:450
1973688994324,4cyclictest2140205-21ntpq23:50:460
1973688994323,9cyclictest2157311-21ssh00:11:220
1973688994323,18cyclictest2037249-21ssh21:43:410
1973688994323,18cyclictest2037249-21ssh21:43:410
1973688994319,8cyclictest1983629-21ntpq19:40:460
1973688994318,8cyclictest2039328-21ntpq21:45:490
1973688994318,8cyclictest2039328-21ntpq21:45:490
1973688994318,8cyclictest2024633-21sh21:27:430
1973688994316,8cyclictest2091821-21ntpq22:50:470
1973690994236,3cyclictest0-21swapper/221:10:472
1973690994235,4cyclictest2163261-21munin-plugin-st00:20:012
1973690994235,4cyclictest0-21swapper/200:00:492
1973690994232,4cyclictest0-21swapper/223:15:572
1973688994230,9cyclictest2043606-21switchtime21:50:580
1973688994230,9cyclictest2043606-21switchtime21:50:580
1973688994229,8cyclictest1986545-21ntpq19:50:430
1973688994224,16cyclictest2069233-21ssh22:22:540
1973688994224,16cyclictest2069233-21ssh22:22:540
1973688994218,7cyclictest2096963-21sh22:56:550
1973688994217,7cyclictest2108063-21ntpq23:10:490
1973688994217,7cyclictest2108063-21ntpq23:10:490
1973690994134,4cyclictest0-21swapper/200:30:482
1973690994134,4cyclictest0-21swapper/200:30:482
1973690994133,5cyclictest1984199-21systemd-run19:45:012
1973688994128,8cyclictest1985025-21ntpq19:45:390
1973688994124,15cyclictest2161234-21ssh00:16:080
1973688994124,15cyclictest2028089-21ssh21:31:370
1973688994118,8cyclictest1979150-21ntpq19:25:470
1973688994116,9cyclictest2071541-21ntpq22:25:440
1973688994116,9cyclictest2071541-21ntpq22:25:440
1973688994116,8cyclictest1988839-21cat20:00:020
1973688994116,8cyclictest1988839-21cat20:00:020
1973688994116,7cyclictest2114271-21ssh23:19:130
1973690994028,3cyclictest0-21swapper/222:55:452
1973689994031,7cyclictest2160066-21/usr/sbin/munin00:15:391
1973688994030,7cyclictest2172635-21ntpq00:30:440
1973688994030,7cyclictest2172635-21ntpq00:30:440
1973688994030,7cyclictest0-21swapper/021:55:350
1973688994027,9cyclictest2055336-21ntpq22:05:430
1973688994027,9cyclictest2006124-21ntpq20:55:460
1973688994027,9cyclictest2006124-21ntpq20:55:460
1973688994027,8cyclictest1991065-21ntpq20:05:430
1973688994017,7cyclictest2101965-21ssh23:03:430
1973688994016,9cyclictest1975966-21needreboot19:15:390
1973688994016,10cyclictest2004082-21hostname20:50:340
1973688994013,9cyclictest2063437-21ntpq22:15:420
1973688994013,8cyclictest2168733-21ntpq00:25:510
1973690993931,5cyclictest2008235-21cut21:05:002
1973690993931,5cyclictest2008235-21cut21:05:002
1973689993929,6cyclictest171855-21cups-browsed23:37:591
1973688993931,3cyclictest2132171-21ntpq23:40:500
1973688993927,8cyclictest2148454-21ntpq00:00:500
1973688993927,8cyclictest2144204-21ntpq23:55:430
1973688993927,8cyclictest2144204-21ntpq23:55:430
1973688993927,8cyclictest2000191-21ntpq20:35:470
1973688993923,13cyclictest2016853-21ssh21:18:190
1973688993918,9cyclictest2120917-21sh23:26:310
1973688993916,8cyclictest1992173-21/usr/sbin/munin20:10:350
1973688993916,16cyclictest2075671-21ssh22:30:450
1973690993831,4cyclictest2179512-21munin-run00:40:002
1973690993829,4cyclictest0-21swapper/221:15:502
1973690993827,4cyclictest0-21swapper/223:10:462
1973690993827,4cyclictest0-21swapper/223:10:462
197369099382,27cyclictest2051261-21ssh22:00:422
1973690993816,5cyclictest36-21ksoftirqd/223:35:412
197369099381,31cyclictest2063558-21ssh22:15:472
1973689993830,5cyclictest2147050-21grep00:00:011
1973689993830,5cyclictest2147050-21grep00:00:011
1973689993830,5cyclictest2098761-21latency_hist23:00:021
1973688993826,8cyclictest1994112-21ntpq20:15:460
1973688993825,9cyclictest1974585-21ntpq19:10:430
1973688993825,8cyclictest1995598-21ntpq20:20:480
1973690993727,6cyclictest171855-21cups-browsed21:48:592
1973690993727,6cyclictest171855-21cups-browsed21:48:592
197369099371,27cyclictest2156344-21rm00:10:402
1973689993731,3cyclictest0-21swapper/122:55:011
1973689993729,5cyclictest1-21systemd21:30:011
1973689993729,4cyclictest0-21swapper/122:40:011
1973689993728,6cyclictest2054955-21fschecks_time22:05:371
1973689993725,9cyclictest2132453-21switchtime23:40:571
197368999372,28cyclictest2071599-21ssh22:25:451
197368999372,28cyclictest2071599-21ssh22:25:451
1973688993727,7cyclictest1980529-21ntpq19:30:430
1973688993725,8cyclictest1982117-21ntpq19:35:450
1973688993724,10cyclictest1989592-21ntpq20:00:430
1973688993724,10cyclictest1989592-21ntpq20:00:430
1973688993723,11cyclictest2117469-21ssh23:22:410
1973688993718,16cyclictest1977619-21ntpq19:20:460
1973690993629,3cyclictest0-21swapper/221:25:432
1973690993625,8cyclictest441-21gpm19:25:452
1973689993630,3cyclictest1056-21mta-sts-daemon20:45:311
1973689993629,4cyclictest0-21swapper/122:50:011
197369099356,18cyclictest2104034-21ssh23:05:502
197369099356,18cyclictest2104034-21ssh23:05:502
1973690993527,5cyclictest2070365-21systemctl22:25:012
1973690993527,5cyclictest2070365-21systemctl22:25:012
1973690993526,6cyclictest2005222-21idleruntime-cro20:55:002
1973690993524,7cyclictest1990848-21kernelversion20:05:382
197368999353,24cyclictest2051295-21ssh22:00:431
1973689993528,4cyclictest2173056-21rm00:30:551
1973689993528,4cyclictest2173056-21rm00:30:551
1973689993528,4cyclictest0-21swapper/119:40:011
1973689993525,7cyclictest2066269-21idleruntime-cro22:20:011
1973689993525,6cyclictest2019360-21switchtime21:20:551
197368999351,10cyclictest0-21swapper/121:00:441
197368999351,10cyclictest0-21swapper/121:00:441
1973690993427,3cyclictest0-21swapper/200:00:012
1973690993427,3cyclictest0-21swapper/200:00:012
1973690993426,5cyclictest2091343-21gltestperf22:50:382
1973690993414,5cyclictest36-21ksoftirqd/221:30:452
1973690993413,5cyclictest36-21ksoftirqd/223:20:382
1973689993427,4cyclictest1055-21mta-sts-daemon21:40:331
1973689993427,4cyclictest1055-21mta-sts-daemon21:40:331
1973689993426,5cyclictest421-21dbus-daemon23:29:441
1973689993425,6cyclictest1-21systemd20:35:011
1973689993425,4cyclictest0-21swapper/100:20:501
1973690993326,4cyclictest2120517-21unixbench_multi23:25:572
1973690993325,5cyclictest2086571-21latency_hist22:45:022
1973690993325,5cyclictest2002223-21cron20:45:012
1973690993325,3cyclictest0-21swapper/221:20:502
1973690993322,5cyclictest0-21swapper/200:20:422
1973690993320,3cyclictest171rcu_preempt20:14:592
1973690993311,7cyclictest36-21ksoftirqd/220:30:442
1973690993311,5cyclictest36-21ksoftirqd/220:35:462
197369099331,11cyclictest2055418-21ntpq22:05:462
1973690993310,5cyclictest36-21ksoftirqd/222:10:372
1973689993325,5cyclictest2107436-21cpuspeed23:10:341
1973689993325,5cyclictest2107436-21cpuspeed23:10:341
1973689993324,6cyclictest2103806-21needreboot23:05:411
1973689993324,6cyclictest2103806-21needreboot23:05:411
1973689993324,6cyclictest0-21swapper/121:10:531
1973689993324,4cyclictest0-21swapper/121:15:481
1973690993227,2cyclictest0-21swapper/220:20:012
1973690993212,6cyclictest36-21ksoftirqd/221:55:372
1973690993212,4cyclictest36-21ksoftirqd/223:50:492
1973690993211,5cyclictest36-21ksoftirqd/223:41:292
1973689993227,2cyclictest0-21swapper/100:05:011
1973689993226,3cyclictest0-21swapper/122:10:501
1973689993224,5cyclictest2004235-21fschecks_count20:50:371
197368999322,22cyclictest2136149-21sh23:45:451
1973689993219,3cyclictest1992628-21ntpq20:10:461
1973688993220,9cyclictest1973643-21munin-run19:10:010
1973688993220,9cyclictest1973643-21munin-run19:10:010
1973690993126,2cyclictest0-21swapper/222:45:372
1973690993125,3cyclictest0-21swapper/219:50:012
1973690993121,7cyclictest2005548-21aten_r2power_fr20:55:342
1973690993121,7cyclictest2005548-21aten_r2power_fr20:55:342
1973690993114,11cyclictest1977545-21ntpq19:20:422
1973690993111,5cyclictest36-21ksoftirqd/220:25:452
1973690993110,5cyclictest36-21ksoftirqd/223:00:112
1973690993110,5cyclictest36-21ksoftirqd/222:31:272
1973689993125,3cyclictest0-21swapper/119:20:001
1973689993125,3cyclictest0-21swapper/100:40:001
1973689993124,4cyclictest1051-21mta-sts-daemon20:06:181
1973689993123,5cyclictest1985901-21/usr/sbin/munin19:50:461
1973689993122,6cyclictest420-21avahi-daemon22:40:431
197368999312,25cyclictest2039256-21ssh21:45:461
197368999312,25cyclictest2039256-21ssh21:45:461
197368999312,22cyclictest2156412-21ssh00:10:431
1973689993113,3cyclictest171rcu_preempt23:01:591
197369099309,6cyclictest36-21ksoftirqd/219:35:342
197369099309,5cyclictest36-21ksoftirqd/220:20:492
1973690993024,3cyclictest0-21swapper/221:10:002
1973690993011,5cyclictest36-21ksoftirqd/219:30:482
1973689993023,4cyclictest2123542-21fschecks_count23:30:351
1973689993023,4cyclictest1054-21mta-sts-daemon19:45:411
1973689993023,4cyclictest1051-21mta-sts-daemon22:20:291
1973689993023,4cyclictest1051-21mta-sts-daemon22:20:291
197369099299,4cyclictest36-21ksoftirqd/219:10:442
197369099298,4cyclictest36-21ksoftirqd/220:00:422
197369099298,4cyclictest36-21ksoftirqd/220:00:422
197369099298,4cyclictest36-21ksoftirqd/219:50:392
1973689992922,4cyclictest2139900-21wc23:50:381
1973689992922,4cyclictest1054-21mta-sts-daemon22:31:411
1973689992921,5cyclictest171855-21cups-browsed21:05:081
1973689992921,5cyclictest1-21systemd21:00:011
1973689992921,5cyclictest1-21systemd21:00:011
1973689992918,9cyclictest1985692-21kworker/u8:12+events_unbound20:25:491
197369099289,4cyclictest36-21ksoftirqd/220:45:472
1973690992820,5cyclictest1055-21mta-sts-daemon19:20:012
1973689992823,2cyclictest0-21swapper/120:20:011
1973689992819,6cyclictest1988328-21switchtime19:55:531
1973689992819,6cyclictest1988328-21switchtime19:55:531
197368999281,4cyclictest171rcu_preempt20:35:471
197368999276,16cyclictest2027028-21ssh21:30:431
197368999272,20cyclictest2031062-21ssh21:35:431
1973689992719,5cyclictest171855-21cups-browsed19:31:461
1973689992715,3cyclictest2168565-21ntpq00:25:441
1973689992618,5cyclictest2001790-21sendmail_mailqu20:40:531
1973689992617,6cyclictest171855-21cups-browsed19:11:001
1973689992517,5cyclictest1983329-21kernelversion19:40:391
1973689992517,5cyclictest171855-21cups-browsed20:23:291
1973689992516,6cyclictest171855-21cups-browsed20:03:301
1973689992516,6cyclictest171855-21cups-browsed20:03:301
1973689992417,4cyclictest1978492-21aten2.4-expect19:25:331
197368999241,12cyclictest1977592-21ntpq19:20:451
197369099199,7cyclictest421-21dbus-daemon19:10:022
197369099199,7cyclictest421-21dbus-daemon19:10:022
197368999183,3cyclictest171rcu_preempt19:10:021
197368999183,3cyclictest171rcu_preempt19:10:021
197369199175,8cyclictest0-21swapper/323:30:433
197369199173,7cyclictest0-21swapper/321:15:483
197369199167,6cyclictest0-21swapper/321:50:423
197369199167,6cyclictest0-21swapper/321:50:423
197369199163,7cyclictest0-21swapper/322:20:423
197369199163,7cyclictest0-21swapper/322:20:423
197369199162,9cyclictest0-21swapper/300:30:483
197369199162,9cyclictest0-21swapper/300:30:483
197369199161,7cyclictest0-21swapper/322:40:453
197369199153,6cyclictest0-21swapper/300:00:433
197369199152,6cyclictest0-21swapper/322:55:433
197369199152,6cyclictest0-21swapper/319:50:493
197369199151,8cyclictest0-21swapper/323:45:423
197369199151,8cyclictest0-21swapper/322:15:413
197369199151,7cyclictest0-21swapper/323:55:463
197369199151,7cyclictest0-21swapper/323:55:463
197369199151,7cyclictest0-21swapper/323:00:443
197369199151,7cyclictest0-21swapper/321:25:423
197369199147,5cyclictest0-21swapper/319:15:013
197369199144,4cyclictest0-21swapper/320:22:003
197369199143,5cyclictest0-21swapper/321:00:463
197369199143,5cyclictest0-21swapper/321:00:463
197369199142,5cyclictest0-21swapper/322:05:493
197369199141,7cyclictest0-21swapper/323:40:493
197369199141,7cyclictest0-21swapper/323:35:433
197369199141,7cyclictest0-21swapper/322:35:433
197369199141,7cyclictest0-21swapper/322:00:433
197369199141,7cyclictest0-21swapper/300:20:503
197369199141,7cyclictest0-21swapper/300:10:493
197369199141,6cyclictest0-21swapper/323:05:433
197369199141,6cyclictest0-21swapper/323:05:433
197369199141,6cyclictest0-21swapper/321:40:413
197369199141,6cyclictest0-21swapper/321:40:413
197369199141,6cyclictest0-21swapper/320:40:473
197369199141,6cyclictest0-21swapper/320:30:463
197369199141,6cyclictest0-21swapper/319:55:473
197369199141,6cyclictest0-21swapper/319:55:473
197369199132,7cyclictest0-21swapper/323:25:003
197369199132,7cyclictest0-21swapper/319:35:483
197369199132,6cyclictest0-21swapper/322:45:423
197369199132,6cyclictest0-21swapper/300:35:383
197369199132,5cyclictest0-21swapper/323:50:443
197369199131,7cyclictest0-21swapper/322:50:433
197369199131,7cyclictest0-21swapper/322:10:433
197369199131,7cyclictest0-21swapper/300:05:463
197369199131,6cyclictest0-21swapper/321:55:433
197369199131,6cyclictest0-21swapper/321:45:413
197369199131,6cyclictest0-21swapper/321:45:413
197369199131,6cyclictest0-21swapper/320:45:473
197369199131,6cyclictest0-21swapper/320:00:403
197369199131,6cyclictest0-21swapper/320:00:403
197369199131,6cyclictest0-21swapper/319:15:413
197369199131,6cyclictest0-21swapper/300:25:493
197369199131,5cyclictest0-21swapper/320:55:453
197369199131,5cyclictest0-21swapper/320:55:453
197369199130,6cyclictest0-21swapper/320:05:423
197369199122,6cyclictest0-21swapper/323:10:433
197369199122,6cyclictest0-21swapper/323:10:433
197369199122,6cyclictest0-21swapper/321:10:393
197369199121,7cyclictest0-21swapper/323:25:473
197369199121,7cyclictest0-21swapper/321:30:443
197369199121,7cyclictest0-21swapper/300:15:343
197369199121,6cyclictest0-21swapper/321:35:333
197369199121,6cyclictest0-21swapper/321:25:033
197369199121,6cyclictest0-21swapper/320:15:493
197369199115,4cyclictest0-21swapper/322:25:393
197369199115,4cyclictest0-21swapper/322:25:393
197369199112,6cyclictest0-21swapper/323:15:443
197369199112,5cyclictest0-21swapper/321:10:003
197369199112,5cyclictest0-21swapper/320:35:423
197369199111,6cyclictest0-21swapper/320:50:443
197369199111,6cyclictest0-21swapper/319:30:443
197369199111,5cyclictest0-21swapper/320:10:443
197369199111,4cyclictest0-21swapper/322:30:473
197369199101,5cyclictest0-21swapper/320:25:503
197369199100,6cyclictest0-21swapper/319:25:423
197369199100,5cyclictest0-21swapper/319:20:373
19736919991,5cyclictest0-21swapper/319:45:443
19736919991,5cyclictest0-21swapper/319:40:433
19736919983,3cyclictest431ktimers/319:10:013
19736919983,3cyclictest431ktimers/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional