You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 11:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 17, 2026 00:43:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4068259996552,9cyclictest12650irq/32-eth022:06:010
4068261995748,6cyclictest4081266-21sshd19:51:032
406826199564,39cyclictest4166789-21ssh22:50:582
4068259995440,11cyclictest4118142-21threads21:31:100
4068259995140,6cyclictest637-21switchtime23:36:120
4068259994943,3cyclictest1002-21mta-sts-daemon19:15:550
4068259994928,4cyclictest171rcu_preempt23:15:570
4068259994536,6cyclictest27755-21needreboot00:20:570
4068259994532,10cyclictest0-21swapper/000:40:010
4068261994434,6cyclictest4193152-21latency_hist23:35:022
4068261994432,7cyclictest0-21swapper/222:26:022
4068259994431,3cyclictest0-21swapper/023:51:030
4068259994429,9cyclictest151ktimers/021:21:020
4068259994429,9cyclictest151ktimers/021:21:020
4068261994335,5cyclictest4159507-21basename22:40:012
4068260994335,5cyclictest14415-21munin-run00:00:011
4068259994336,4cyclictest439-21systemd-logind20:00:010
4068261994234,5cyclictest4129938-21needreboot21:50:552
406825999423,30cyclictest4130059-21ssh21:50:590
4068259994229,9cyclictest12650irq/32-eth022:38:460
406825999422,32cyclictest31074-21ssh00:26:050
4068261994135,3cyclictest0-21swapper/223:00:012
4068260994133,5cyclictest4101231-21munin-plugin-st21:00:011
4068259994129,9cyclictest623-21sshd23:02:040
4068259994129,9cyclictest623-21sshd23:02:040
4068259994129,9cyclictest0-21swapper/023:20:490
4068259994115,18cyclictest12650irq/32-eth022:51:010
4068259994114,7cyclictest0-21swapper/021:41:010
4068261994032,5cyclictest4144143-21grep22:15:012
406826199402,33cyclictest4124004-21sh21:41:002
4068260994033,4cyclictest29718-21grep00:25:011
4068260994032,5cyclictest4145221-21needreboot22:15:541
4068260994032,5cyclictest1-21systemd23:50:021
4068260994031,6cyclictest265-21systemd-journal21:48:041
4068259994019,11cyclictest0-21swapper/020:40:560
406825999400,9cyclictest4175980-21ssh23:05:570
406825999400,9cyclictest4175980-21ssh23:05:570
4068261993931,4cyclictest0-21swapper/221:20:022
4068261993930,5cyclictest4141115-21ssh22:10:012
4068260993931,5cyclictest421-21dbus-daemon23:45:021
4068260993930,6cyclictest4162759-21latency_hist22:45:031
4068259993930,3cyclictest0-21swapper/023:41:070
4068259993911,20cyclictest665-21snmpd21:26:000
4068261993831,4cyclictest4122768-21kworker/u8:10-rpciod21:55:502
4068260993830,5cyclictest4166113-21aten_r2power_cu22:50:471
4068260993830,5cyclictest2341350-21cups-browsed21:22:431
4068260993830,5cyclictest2341350-21cups-browsed21:22:431
406825999382,28cyclictest4151431-21sh22:25:590
4068259993818,13cyclictest4163693-21ssh22:45:560
4068259993817,17cyclictest12650irq/32-eth023:11:010
4068259993817,13cyclictest421-21dbus-daemon19:46:020
4068259993813,17cyclictest421-21dbus-daemon00:13:460
4068259993813,17cyclictest421-21dbus-daemon00:13:460
4068261993732,2cyclictest0-21swapper/200:20:012
4068261993729,5cyclictest4088890-21switchtime20:16:072
4068260993730,4cyclictest0-21swapper/100:35:011
4068260993730,3cyclictest4166923-21kworker/u8:4-xprtiod23:55:031
4068260993729,5cyclictest4180886-21grep23:15:001
4068259993730,5cyclictest4122724-21cron21:40:000
406825999372,29cyclictest4133176-21ssh21:56:010
4068261993628,5cyclictest4147202-21grep22:20:002
4068261993628,5cyclictest4104466-21latency_hist21:10:032
4068261993627,6cyclictest485-21in:imuxsock23:55:012
406826099365,28cyclictest420-21avahi-daemon21:51:001
4068260993629,5cyclictest0-21swapper/120:45:021
4068260993628,5cyclictest4107322-21latency_hist21:15:011
4068259993628,6cyclictest17503-21cron00:05:010
4068259993623,10cyclictest14402-21cron00:00:010
4068259993610,5cyclictest0-21swapper/020:05:580
4068261993528,4cyclictest4172521-21ssh23:00:532
4068261993528,4cyclictest4172521-21ssh23:00:532
4068261993527,4cyclictest4174801-21kworker/u8:10+xprtiod00:00:022
406826199351,29cyclictest27973-21ssh00:21:062
4068260993529,3cyclictest0-21swapper/100:40:011
4068260993528,4cyclictest4187132-21latency_hist23:25:021
4068260993528,4cyclictest4083232-21sendmail20:00:011
4068260993527,5cyclictest4181833-21latency23:15:551
4068260993527,5cyclictest4135025-21wc22:00:021
4068260993524,8cyclictest4159500-21idleruntime-cro22:40:011
406825999357,18cyclictest428-21mta-sts-daemon20:45:590
4068259993525,7cyclictest4154067-21ls22:30:500
4068259993525,7cyclictest4093811-21latency_hist20:35:010
406825999352,26cyclictest4169855-21sh22:56:020
4068259993521,11cyclictest4089910-21kernelversion20:20:530
4068261993429,2cyclictest0-21swapper/223:36:082
4068261993428,3cyclictest0-21swapper/222:50:012
4068261993427,3cyclictest0-21swapper/223:11:112
4068261993426,4cyclictest0-21swapper/221:46:022
4068260993427,4cyclictest4125883-21latency_hist21:45:021
4068260993427,4cyclictest0-21swapper/121:20:011
4068260993426,5cyclictest4153352-21true22:30:011
4068260993423,9cyclictest4097561-21needreboot20:45:561
406825999343,23cyclictest24823-21ssh00:16:040
4068259993425,7cyclictest4069737-21cron19:15:010
4068259993425,5cyclictest0-21swapper/023:30:160
406825999342,17cyclictest4138053-21idleruntime-cro22:05:010
4068259993420,5cyclictest1001-21mta-sts-daemon22:44:090
4068259993416,14cyclictest2341350-21cups-browsed20:26:000
4068259993416,14cyclictest2341350-21cups-browsed20:26:000
4068261993325,5cyclictest31491-21unixbench_singl00:26:112
4068261993325,5cyclictest1-21systemd00:35:022
4068260993327,3cyclictest4084750-21cron20:05:011
4068260993325,5cyclictest4104262-21systemd-run21:10:011
4068260993324,6cyclictest4144142-21munin-run22:15:011
4068260993320,9cyclictest420-21avahi-daemon19:46:021
4068259993324,6cyclictest2341350-21cups-browsed20:56:060
4068259993317,9cyclictest12650irq/32-eth022:20:580
4068261993226,3cyclictest4081963-21kworker/u8:3-rpciod20:05:012
4068261993224,5cyclictest38897-21idleruntime-cro00:40:012
4068261993224,5cyclictest1002-21mta-sts-daemon22:40:552
4068261993222,7cyclictest4115255-21tail21:26:122
406826199322,22cyclictest4117752-21sh21:30:592
4068260993225,4cyclictest4094256-21fschecks_count20:35:511
4068260993225,4cyclictest1002-21mta-sts-daemon22:30:281
4068260993224,5cyclictest4190609-21aten_r2power_en23:30:491
4068260993224,5cyclictest4188680-21/usr/sbin/munin23:26:121
4068260993224,5cyclictest4122726-21ssh21:40:001
4068260993224,5cyclictest4086260-21systemd-run20:10:011
4068260993224,5cyclictest2188-21cron23:40:011
4068260993222,7cyclictest1002-21mta-sts-daemon20:20:471
406825999323,24cyclictest6569-21ssh23:46:020
4068259993225,4cyclictest1003-21mta-sts-daemon20:20:010
4068259993223,6cyclictest4075744-21sh19:35:010
406825999322,23cyclictest4108641-21ssh21:16:030
406825999322,20cyclictest33936-21ssh00:31:000
4068259993220,9cyclictest265-21systemd-journal20:55:010
4068259993218,10cyclictest12650irq/32-eth021:10:590
4068259993213,15cyclictest4126985-21ntp_kernel_pll_21:45:580
4068261993124,4cyclictest6273-21missed_timers23:45:522
4068261993124,4cyclictest4066926-21kworker/u8:11+xprtiod21:00:022
4068261993122,6cyclictest18711-21/usr/sbin/munin00:05:592
406826199312,19cyclictest4148528-21sshd22:21:042
4068261993120,8cyclictest435-21rsyslogd23:40:122
4068260993124,4cyclictest23200-21kworker/u8:6-xprtiod00:15:011
4068260993124,4cyclictest23200-21kworker/u8:6-xprtiod00:15:011
4068260993124,4cyclictest1000-21mta-sts-daemon23:05:441
4068260993124,4cyclictest1000-21mta-sts-daemon23:05:441
4068260993124,4cyclictest0-21swapper/122:01:011
4068259993126,2cyclictest0-21swapper/000:05:520
4068259993122,5cyclictest2341350-21cups-browsed22:10:170
406825999312,23cyclictest4145378-21ssh22:16:000
4068261993025,2cyclictest0-21swapper/223:20:012
4068261993023,4cyclictest0-21swapper/200:05:012
4068261993022,4cyclictest0-21swapper/222:01:052
4068261993021,5cyclictest4190230-21latency_hist23:30:022
4068261993020,7cyclictest4077257-21sendmail19:40:012
4068261993019,8cyclictest4175575-21gltestperf23:05:492
4068261993019,8cyclictest4175575-21gltestperf23:05:492
4068260993024,4cyclictest4068258-21cyclictest22:56:071
4068260993023,4cyclictest4077291-21latency_hist19:40:021
4068260993023,3cyclictest0-21swapper/122:50:011
4068260993021,6cyclictest4114656-21ntpq21:25:591
4068259993024,3cyclictest0-21swapper/020:40:010
4068259993023,5cyclictest4055555-21kworker/u8:1-events_unbound19:51:020
4068259993023,5cyclictest0-21swapper/019:30:010
4068259993014,12cyclictest151ktimers/020:01:000
406825999301,25cyclictest4103734-21sshd21:06:030
4068261992922,4cyclictest4095237-21awk20:40:002
4068261992922,3cyclictest0-21swapper/200:11:042
4068261992922,3cyclictest0-21swapper/200:11:042
4068261992921,5cyclictest0-21swapper/221:05:012
4068261992920,7cyclictest4104242-21diskmemload23:20:542
4068261992920,5cyclictest4075662-21unixbench_singl19:31:082
406826099293,21cyclictest24994-21sh00:16:091
4068260992921,6cyclictest4172826-21needreboot23:00:571
4068260992921,6cyclictest4172826-21needreboot23:00:571
4068260992921,5cyclictest4139793-21tail22:06:101
4068260992921,5cyclictest4086762-21fschecks_count20:10:511
4068260992920,6cyclictest439-21systemd-logind19:35:011
4068260992919,7cyclictest4102747-21sh21:05:011
4068259992922,4cyclictest4078749-21sh19:45:010
4068261992822,3cyclictest0-21swapper/221:25:002
4068261992822,3cyclictest0-21swapper/221:25:002
4068261992821,4cyclictest4071453-21latency_hist19:20:032
4068261992820,5cyclictest4093623-21unixbench_multi20:31:122
4068261992819,6cyclictest4153801-21/usr/sbin/munin22:30:492
4068261992818,7cyclictest4098050-21timerwakeupswit20:46:092
4068261992817,8cyclictest4105795-21switchtime21:11:092
406826199281,3cyclictest171rcu_preempt21:35:542
406826099283,19cyclictest428-21mta-sts-daemon00:00:571
4068260992823,3cyclictest1001-21mta-sts-daemon21:30:471
4068260992820,5cyclictest4092644-21telnet20:30:501
4068260992820,5cyclictest4078632-21tail19:41:091
4068260992818,7cyclictest1003-21mta-sts-daemon19:10:041
4068259992821,4cyclictest420-21avahi-daemon19:21:380
4068259992821,3cyclictest0-21swapper/019:10:010
4068261992719,5cyclictest4086598-21aten_r2power_po20:10:482
4068260992719,5cyclictest1003-21mta-sts-daemon00:05:571
4068259992720,3cyclictest0-21swapper/023:26:030
4068261992620,3cyclictest0-21swapper/219:46:022
4068261992619,4cyclictest0-21swapper/220:50:532
4068261992618,5cyclictest4083268-21systemd-run20:00:012
4068261992618,5cyclictest4078704-21head19:41:092
4068260992619,4cyclictest4072599-21unixbench-2d19:21:111
4068260992619,4cyclictest1003-21mta-sts-daemon20:15:451
4068260992618,5cyclictest4150273-21sh22:25:001
4068260992617,6cyclictest420-21avahi-daemon19:55:011
4068259992614,9cyclictest0-21swapper/021:01:010
406826199253,16cyclictest428-21mta-sts-daemon19:26:002
4068261992518,5cyclictest4096609-21unixbench-2d20:41:082
4068260992517,5cyclictest4091266-21grep20:25:511
4068260992517,5cyclictest4091266-21grep20:25:511
4068259992519,2cyclictest0-21swapper/019:40:010
406826199247,5cyclictest62750lldpd19:23:082
4068261992418,3cyclictest0-21swapper/219:11:002
4068260992417,5cyclictest31452-21unixbench_multi00:26:101
4068260992411,8cyclictest4070535-21/usr/sbin/munin19:15:531
406825999249,10cyclictest12650irq/32-eth020:11:030
4068261992316,4cyclictest4089792-21fschecks_time20:20:512
4068260992315,5cyclictest4099914-21latency_hist20:55:031
4068261992214,5cyclictest4092155-21tail20:26:092
4068261992214,5cyclictest4092155-21tail20:26:092
406826199212,14cyclictest4085251-21diskstats20:05:502
406826099211,15cyclictest4073684-21ntpq19:26:021
4068261991915,2cyclictest0-21swapper/219:10:012
406826299182,8cyclictest0-21swapper/321:50:593
406826299181,8cyclictest0-21swapper/322:45:563
406826299171,8cyclictest0-21swapper/300:35:593
4068260991711,3cyclictest0-21swapper/119:10:031
406826299163,8cyclictest0-21swapper/323:31:063
406826299163,7cyclictest0-21swapper/321:15:593
406826299163,7cyclictest0-21swapper/300:06:033
406826299162,5cyclictest0-21swapper/320:51:033
406826299162,5cyclictest0-21swapper/320:25:563
406826299162,5cyclictest0-21swapper/320:25:563
406826299161,9cyclictest0-21swapper/323:50:593
406826299161,9cyclictest0-21swapper/323:36:043
406826299161,8cyclictest0-21swapper/323:05:573
406826299161,8cyclictest0-21swapper/323:05:573
406826299161,8cyclictest0-21swapper/322:26:023
406826299161,8cyclictest0-21swapper/322:15:503
406826299161,7cyclictest0-21swapper/323:10:573
406826299153,7cyclictest0-21swapper/322:31:013
406826299152,8cyclictest0-21swapper/323:56:033
406826299152,7cyclictest0-21swapper/323:46:023
406826299152,7cyclictest0-21swapper/321:46:023
406826299152,7cyclictest0-21swapper/321:35:563
406826299152,6cyclictest0-21swapper/319:26:023
406826299151,8cyclictest0-21swapper/322:01:053
406826299151,8cyclictest0-21swapper/321:20:563
406826299151,8cyclictest0-21swapper/321:20:563
406826299151,7cyclictest0-21swapper/321:25:573
406826299151,7cyclictest0-21swapper/320:40:583
406826299151,7cyclictest0-21swapper/320:35:563
406826299151,6cyclictest0-21swapper/321:05:563
406826299150,7cyclictest0-21swapper/321:00:583
406826299144,6cyclictest0-21swapper/322:06:013
406826299142,8cyclictest0-21swapper/321:11:093
406826299142,7cyclictest0-21swapper/322:21:033
406826299142,7cyclictest0-21swapper/322:10:473
406826299142,6cyclictest0-21swapper/323:26:013
406826299141,8cyclictest0-21swapper/321:55:553
406826299141,7cyclictest0-21swapper/322:35:573
406826299141,7cyclictest0-21swapper/321:31:053
406826299141,7cyclictest0-21swapper/319:46:023
406826299141,7cyclictest0-21swapper/300:21:013
406826299141,6cyclictest0-21swapper/319:51:013
406826299141,3cyclictest0-21swapper/319:16:003
406826299135,5cyclictest0-21swapper/300:01:013
406826299132,6cyclictest0-21swapper/300:10:583
406826299132,6cyclictest0-21swapper/300:10:583
406826299131,8cyclictest0-21swapper/323:40:483
406826299131,8cyclictest0-21swapper/323:16:043
406826299131,7cyclictest0-21swapper/322:56:083
406826299131,7cyclictest0-21swapper/322:50:573
406826299131,7cyclictest0-21swapper/321:40:573
406826299131,7cyclictest0-21swapper/320:20:593
406826299131,7cyclictest0-21swapper/300:30:523
406826299131,6cyclictest0-21swapper/320:55:593
406826299131,6cyclictest0-21swapper/320:01:053
406826299131,6cyclictest0-21swapper/319:40:573
406826299131,6cyclictest0-21swapper/319:35:553
406826299131,5cyclictest0-21swapper/320:30:593
406826299131,5cyclictest0-21swapper/320:15:583
406826299131,5cyclictest0-21swapper/319:20:583
406826299130,7cyclictest0-21swapper/323:01:033
406826299130,7cyclictest0-21swapper/323:01:033
406826299130,7cyclictest0-21swapper/322:40:573
406826299130,5cyclictest0-21swapper/319:10:563
406826299122,5cyclictest0-21swapper/320:10:573
406826299122,4cyclictest0-21swapper/323:20:453
406826299122,4cyclictest0-21swapper/320:45:573
406826299121,7cyclictest0-21swapper/300:26:003
406826299121,6cyclictest0-21swapper/320:05:573
406826299121,5cyclictest0-21swapper/319:30:563
406826299111,6cyclictest0-21swapper/300:16:033
406826299101,5cyclictest0-21swapper/319:55:513
40682629981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional