You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 06:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed May 13, 2026 00:43:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4116785995443,7cyclictest2254995-21cups-browsed22:20:510
4116786995245,3cyclictest0-21swapper/122:55:541
4116786995238,9cyclictest261-21systemd-journal00:38:381
4116785995037,8cyclictest151ktimers/021:55:560
4116785995035,10cyclictest0-21swapper/020:40:510
411678799485,33cyclictest113608-21ssh00:20:542
4116786994840,5cyclictest19183-21latency_hist22:25:011
411678599482,20cyclictest28647-21ssh22:35:550
4116786994637,5cyclictest43514-21sh22:55:021
4116786994635,7cyclictest96104-21cups-browsed00:01:461
4116787994534,8cyclictest124311-21sh00:35:012
4116787994534,8cyclictest124311-21sh00:35:012
4116786994535,7cyclictest4178011-21missed_timers21:40:471
4116786994534,7cyclictest96104-21cups-browsed00:05:151
4116785994537,5cyclictest4177636-21cpu21:40:410
4116785994518,3cyclictest0-21swapper/000:40:010
4116785994517,25cyclictest0-21swapper/000:16:150
4116785994427,14cyclictest1055-21mta-sts-daemon00:30:130
4116785994427,14cyclictest1055-21mta-sts-daemon00:30:130
4116785994423,18cyclictest0-21swapper/023:20:510
4116785994421,21cyclictest0-21swapper/021:12:560
4116785994421,20cyclictest51580-21cron23:05:010
4116787994335,5cyclictest4142250-21cron20:35:012
4116787994335,4cyclictest0-21swapper/223:26:082
4116787994333,5cyclictest0-21swapper/223:10:572
4116787994333,5cyclictest0-21swapper/223:10:572
4116785994335,5cyclictest1-21systemd22:20:010
4116785994329,8cyclictest12350irq/32-eth021:25:540
4116785994328,12cyclictest4181974-21latency21:45:470
4116785994320,20cyclictest0-21swapper/022:06:030
4116785994319,2cyclictest0-21swapper/000:30:020
4116785994315,3cyclictest0-21swapper/022:56:030
4116787994234,5cyclictest4131798-21systemd-run20:00:012
4116787994232,8cyclictest79935-21latency_hist23:40:022
4116787994232,6cyclictest2254990-21cupsd23:24:092
4116787994230,9cyclictest106040-21timerwakeupswit00:11:102
4116786994233,5cyclictest65653-21grep23:21:041
4116785994222,17cyclictest0-21swapper/021:20:580
4116785994218,22cyclictest0-21swapper/021:40:000
4116785994218,22cyclictest0-21swapper/021:40:000
4116785994216,2cyclictest0-21swapper/019:20:010
411678799413,30cyclictest4174368-21awk21:35:582
411678799413,30cyclictest4174368-21awk21:35:582
4116787994127,11cyclictest4182725-21timerwakeupswit21:46:032
4116787994120,14cyclictest2254995-21cups-browsed22:15:542
411678599419,6cyclictest89417-21ssh23:50:560
411678599412,9cyclictest60975-21ssh23:15:510
411678599412,16cyclictest20011-21df_inode22:25:440
411678599412,12cyclictest2254990-21cupsd23:11:410
411678599412,12cyclictest2254990-21cupsd23:11:410
4116785994119,19cyclictest25083-21unixbench_multi22:31:020
4116785994115,7cyclictest12350irq/32-eth019:30:560
4116785994112,20cyclictest36731-21ssh22:45:540
411678599411,15cyclictest4160928-21munin-run21:20:020
411678599411,15cyclictest32734-21ntpq22:40:560
4116786994031,5cyclictest0-21swapper/123:50:511
4116785994032,6cyclictest40832-21ssh22:50:560
4116785994030,6cyclictest12350irq/32-eth022:10:530
411678599402,10cyclictest105567-21ssh00:11:000
4116785994017,2cyclictest0-21swapper/022:00:470
4116787993931,5cyclictest4169742-21fschecks_time21:30:462
4116787993931,5cyclictest2254995-21cups-browsed21:40:512
4116787993924,10cyclictest4188959-21systemctl21:55:012
411678799391,32cyclictest20450-21ssh22:25:522
4116786993931,5cyclictest27868-21ssh22:35:421
4116786993931,5cyclictest112225-21sendmail00:20:011
4116786993930,4cyclictest0-21swapper/122:00:571
4116786993928,8cyclictest117600-21ntp_kernel_pll_00:25:571
4116785993931,6cyclictest4142274-21sh20:35:010
411678599391,8cyclictest100744-21aten_r2power_cu00:05:420
4116785993918,2cyclictest0-21swapper/020:00:010
4116785993914,4cyclictest4148765-21fschecks_count20:55:430
411678599391,3cyclictest0-21swapper/023:45:510
4116785993910,10cyclictest151ktimers/020:20:530
4116787993830,5cyclictest4152761-21diskmemload22:30:502
4116787993829,6cyclictest439-21systemd-logind00:00:002
4116787993829,6cyclictest439-21systemd-logind00:00:002
4116787993828,6cyclictest420-21avahi-daemon22:03:372
4116787993827,7cyclictest548-21gdbus22:45:262
4116786993832,3cyclictest0-21swapper/121:00:011
4116786993830,4cyclictest0-21swapper/100:20:541
4116786993830,3cyclictest0-21swapper/122:10:501
4116786993828,4cyclictest0-21swapper/123:05:541
4116785993831,4cyclictest1-21systemd20:30:010
4116785993829,7cyclictest81171-21ntp_kernel_err23:40:520
4116785993816,3cyclictest4116784-21cyclictest21:35:020
4116785993812,5cyclictest75863-21latency_hist23:35:020
4116785993812,5cyclictest113873-21grep00:21:020
411678599381,17cyclictest4128178-21ntpq19:45:510
411678599381,17cyclictest4128178-21ntpq19:45:510
4116787993729,4cyclictest1051-21mta-sts-daemon00:05:572
4116787993728,6cyclictest261-21systemd-journal22:59:072
411678699373,26cyclictest547-21gmain21:05:511
4116786993729,5cyclictest35564-21latency_hist22:45:021
4116786993729,4cyclictest0-21swapper/123:10:521
4116786993729,4cyclictest0-21swapper/123:10:521
4116786993728,5cyclictest4143807-21latency_hist20:40:021
4116785993730,5cyclictest55670-21systemd-run23:10:020
4116785993717,3cyclictest0-21swapper/000:02:580
4116785993715,6cyclictest4130248-21sshd19:54:060
411678599371,15cyclictest77331-21ssh23:36:030
411678599371,14cyclictest4125144-21ntpq19:35:510
4116787993626,6cyclictest98070-21unixbench_multi00:01:112
411678799362,27cyclictest4166175-21ssh21:25:542
411678699364,23cyclictest121524-21ssh00:30:521
411678699364,23cyclictest121524-21ssh00:30:521
4116786993628,4cyclictest0-21swapper/122:32:241
4116786993627,6cyclictest93914-21unixbench-2d23:56:031
4116786993627,6cyclictest93914-21unixbench-2d23:56:031
4116786993627,5cyclictest4156828-21grep21:15:001
4116786993627,5cyclictest1-21systemd21:30:011
4116786993624,9cyclictest4186175-21ntp_kernel_err21:50:511
4116786993623,10cyclictest439-21systemd-logind19:30:011
4116785993629,4cyclictest54139-21kworker/u8:8-rpciod23:25:490
411678599362,14cyclictest4186347-21ssh21:50:560
411678599361,15cyclictest93347-21ntpq23:55:510
411678599361,15cyclictest93347-21ntpq23:55:510
411678799356,18cyclictest428-21mta-sts-daemon20:20:562
4116787993528,4cyclictest112223-21cut00:20:012
4116787993527,5cyclictest4139807-21fschecks_time20:25:452
4116787993527,5cyclictest4136787-21fschecks_time20:15:442
411678699353,23cyclictest4182215-21ssh21:45:521
4116786993527,5cyclictest4151260-21cron21:05:001
4116786993525,7cyclictest2254995-21cups-browsed21:31:301
4116786993524,8cyclictest4190672-21timerandwakeup21:56:021
4116786993524,7cyclictest420-21avahi-daemon21:19:571
4116786993524,7cyclictest2254995-21cups-browsed21:20:291
4116786993515,11cyclictest4152761-21diskmemload23:25:571
4116787993425,6cyclictest420-21avahi-daemon22:11:412
4116787993425,5cyclictest2254995-21cups-browsed23:19:352
4116787993423,8cyclictest16190-21kernelversion22:20:462
4116787993423,7cyclictest547-21gmain23:43:112
411678799342,24cyclictest4162162-21ssh21:20:522
4116786993428,3cyclictest0-21swapper/121:36:021
4116786993428,3cyclictest0-21swapper/121:36:021
4116786993426,5cyclictest88102-21latency_hist23:50:021
4116786993425,6cyclictest0-21swapper/123:00:441
4116786993424,6cyclictest659-21snmpd22:45:421
4116787993326,4cyclictest83952-21kworker/u8:9-xprtiod00:35:532
4116787993326,4cyclictest0-21swapper/222:50:452
4116787993325,5cyclictest31296-21munin-run22:40:012
4116787993324,6cyclictest261-21systemd-journal00:27:092
4116787993324,5cyclictest4146754-21cron20:50:012
4116786993326,4cyclictest0-21swapper/123:40:011
4116786993325,5cyclictest60737-21kernelversion23:15:471
4116785993323,6cyclictest151ktimers/020:15:550
411678599332,13cyclictest4117002-21/usr/sbin/munin19:11:070
411678799324,20cyclictest89380-21ssh23:50:552
4116787993226,3cyclictest0-21swapper/220:40:012
4116787993225,4cyclictest1056-21mta-sts-daemon20:50:412
4116787993224,5cyclictest2254995-21cups-browsed21:57:332
4116787993223,7cyclictest4149749-21/usr/sbin/munin21:00:012
4116787993223,6cyclictest2254995-21cups-browsed23:30:242
4116787993223,5cyclictest0-21swapper/221:10:562
4116787993223,4cyclictest0-21swapper/221:10:002
4116787993222,7cyclictest1-21systemd20:15:012
411678699323,20cyclictest105543-21ssh00:10:591
4116786993226,3cyclictest0-21swapper/120:25:021
4116786993225,4cyclictest0-21swapper/123:30:081
4116786993224,5cyclictest4129695-21ntp_offset19:50:531
4116786993224,4cyclictest0-21swapper/122:08:451
4116786993223,6cyclictest4124305-21latency_hist19:35:021
4116785993225,4cyclictest4133120-21grep20:01:030
411678799314,17cyclictest85218-21ssh23:45:512
4116787993126,2cyclictest0-21swapper/219:35:012
4116787993123,6cyclictest351ktimers/219:20:502
4116787993121,8cyclictest4130007-21timerandwakeup19:51:022
4116786993125,3cyclictest0-21swapper/120:20:011
4116786993122,6cyclictest4146773-21sh20:50:011
411678699312,21cyclictest12504-21ssh22:15:541
4116785993124,4cyclictest4116884-21latency_hist19:10:020
411678799303,21cyclictest4531-21ssh22:05:572
411678799303,19cyclictest48816-21ssh23:00:522
4116787993022,3cyclictest0-21swapper/223:05:472
4116787993020,4cyclictest4119421-21switchtime19:16:052
4116787993019,3cyclictest171rcu_preempt22:40:322
4116787993018,3cyclictest171rcu_preempt19:45:562
4116787993018,3cyclictest171rcu_preempt19:45:562
4116786993024,3cyclictest0-21swapper/119:40:511
4116786993021,6cyclictest439-21systemd-logind19:25:011
4116786993021,6cyclictest2254995-21cups-browsed23:42:301
4116786993021,5cyclictest4134771-21idleruntime-cro20:10:011
4116786993019,8cyclictest4140765-21munin-plugin-st20:30:021
4116785993016,4cyclictest4127207-21unixbench_singl19:41:000
4116787992919,7cyclictest439-21systemd-logind21:20:022
4116786992923,3cyclictest0-21swapper/119:20:011
4116786992919,5cyclictest0-21swapper/120:40:561
4116786992917,6cyclictest0-21swapper/122:25:421
4116785992923,4cyclictest4136372-21latency_hist20:15:010
4116785992922,5cyclictest0-21swapper/021:10:010
4116786992822,3cyclictest0-21swapper/120:01:021
4116785992823,2cyclictest0-21swapper/019:30:010
411678799272,22cyclictest1056-21mta-sts-daemon19:35:522
4116787992719,4cyclictest4104047-21kworker/u8:6+rpciod19:30:012
4116787992617,7cyclictest4150965-21switchtime21:01:022
4116787992611,3cyclictest171rcu_preempt19:40:502
4116786992619,5cyclictest2254995-21cups-browsed19:35:421
4116786992618,5cyclictest4142142-21unixbench_multi20:31:011
4116785992620,4cyclictest4143754-21cron20:40:010
411678799258,13cyclictest421-21dbus-daemon19:10:552
4116787992517,5cyclictest4134759-21cron20:10:012
4116786992519,3cyclictest0-21swapper/119:45:441
4116786992519,3cyclictest0-21swapper/119:45:441
4116786992519,3cyclictest0-21swapper/119:11:001
411678599253,13cyclictest428-21mta-sts-daemon20:05:530
4116785992517,6cyclictest4150988-21timerandwakeup21:01:020
411678799249,2cyclictest171rcu_preempt20:00:552
411678699244,14cyclictest428-21mta-sts-daemon20:50:511
4116786992418,4cyclictest1051-21mta-sts-daemon19:58:291
4116786992418,3cyclictest0-21swapper/120:11:031
4116785992419,3cyclictest1056-21mta-sts-daemon19:23:570
4116785992319,2cyclictest0-21swapper/020:55:010
411678799223,3cyclictest171rcu_preempt20:40:512
4116785992114,4cyclictest4146117-21ntpq20:45:500
411678799203,4cyclictest4104047-21kworker/u8:6+xprtiod19:10:022
411678899183,7cyclictest0-21swapper/300:30:563
411678899183,7cyclictest0-21swapper/300:30:563
411678899182,8cyclictest0-21swapper/321:55:543
411678899182,7cyclictest0-21swapper/323:05:543
411678899181,8cyclictest0-21swapper/322:40:523
411678899172,6cyclictest0-21swapper/322:30:493
411678899171,7cyclictest0-21swapper/323:15:513
4116786991712,3cyclictest0-21swapper/119:10:021
411678899163,8cyclictest0-21swapper/300:01:023
411678899163,7cyclictest0-21swapper/300:25:593
411678899163,6cyclictest0-21swapper/320:50:533
411678899162,8cyclictest0-21swapper/323:45:403
411678899162,8cyclictest0-21swapper/300:15:533
411678899161,7cyclictest0-21swapper/323:00:563
411678899161,6cyclictest0-21swapper/319:55:563
411678899161,6cyclictest0-21swapper/319:30:523
411678899155,7cyclictest0-21swapper/322:15:543
411678899153,8cyclictest0-21swapper/321:20:533
411678899153,6cyclictest0-21swapper/323:30:523
411678899153,5cyclictest0-21swapper/319:40:513
411678899152,8cyclictest0-21swapper/323:20:503
411678899152,8cyclictest0-21swapper/322:25:533
411678899152,8cyclictest0-21swapper/321:35:403
411678899152,8cyclictest0-21swapper/321:35:403
411678899152,7cyclictest0-21swapper/321:15:533
411678899152,6cyclictest0-21swapper/320:55:503
411678899152,6cyclictest0-21swapper/320:35:513
411678899151,8cyclictest0-21swapper/323:55:493
411678899151,8cyclictest0-21swapper/323:55:493
411678899151,7cyclictest0-21swapper/321:45:543
411678899151,6cyclictest0-21swapper/322:50:513
411678899143,8cyclictest0-21swapper/323:10:523
411678899143,8cyclictest0-21swapper/323:10:523
411678899143,7cyclictest0-21swapper/322:20:413
411678899142,8cyclictest0-21swapper/321:10:563
411678899142,7cyclictest0-21swapper/322:55:423
411678899142,7cyclictest0-21swapper/322:45:513
411678899142,7cyclictest0-21swapper/322:00:533
411678899141,8cyclictest0-21swapper/323:50:583
411678899141,7cyclictest0-21swapper/322:35:513
411678899141,7cyclictest0-21swapper/321:50:503
411678899141,7cyclictest0-21swapper/320:45:493
411678899141,7cyclictest0-21swapper/320:05:573
411678899141,7cyclictest0-21swapper/300:05:433
411678899141,6cyclictest0-21swapper/320:40:513
411678899141,6cyclictest0-21swapper/320:30:513
411678899141,6cyclictest0-21swapper/319:50:533
411678899141,6cyclictest0-21swapper/319:20:523
411678899141,6cyclictest0-21swapper/319:10:533
411678899141,6cyclictest0-21swapper/300:25:033
411678899132,7cyclictest0-21swapper/323:40:573
411678899132,7cyclictest0-21swapper/323:36:023
411678899132,7cyclictest0-21swapper/300:35:533
411678899131,7cyclictest0-21swapper/323:25:493
411678899131,7cyclictest0-21swapper/322:05:493
411678899131,7cyclictest0-21swapper/321:30:413
411678899123,6cyclictest0-21swapper/322:10:513
411678899122,7cyclictest0-21swapper/300:10:553
411678899122,6cyclictest0-21swapper/321:40:513
411678899122,6cyclictest0-21swapper/320:25:523
411678899121,8cyclictest0-21swapper/321:25:483
411678899121,6cyclictest0-21swapper/321:05:483
411678899121,6cyclictest0-21swapper/321:00:513
411678899121,5cyclictest0-21swapper/320:20:503
411678899121,5cyclictest0-21swapper/319:15:553
411678899121,3cyclictest0-21swapper/320:15:563
411678899111,8cyclictest0-21swapper/320:15:023
411678899104,4cyclictest0-21swapper/319:25:523
411678899101,7cyclictest0-21swapper/319:35:443
411678899101,6cyclictest0-21swapper/320:00:523
411678899101,5cyclictest0-21swapper/319:45:493
411678899101,5cyclictest0-21swapper/319:45:493
41167889991,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional