You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 13:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Aug 25, 2026 12:42:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3493031995949,6cyclictest1000-21mta-sts-daemon09:46:080
3493033995143,4cyclictest1002-21mta-sts-daemon11:30:592
3493032995143,5cyclictest3494531-21systemd-run07:15:011
3493031994932,11cyclictest12650irq/32-eth012:26:050
3493032994841,4cyclictest1000-21mta-sts-daemon11:58:331
3493032994841,4cyclictest1000-21mta-sts-daemon11:58:331
3493032994839,3cyclictest0-21swapper/111:31:051
3493032994738,6cyclictest1001-21mta-sts-daemon09:01:051
3493033994637,3cyclictest0-21swapper/209:46:002
3493032994536,6cyclictest3599764-21aten_r2power_fr11:05:511
3493032994531,4cyclictest0-21swapper/111:11:071
3493033994435,6cyclictest421-21dbus-daemon08:30:012
3493032994436,5cyclictest3606363-21needreboot11:15:591
3493032994335,3cyclictest0-21swapper/110:46:051
3493031994330,4cyclictest3538233-21latency_hist09:25:010
3493033994235,3cyclictest0-21swapper/211:00:012
3493033994234,5cyclictest3647749-21sh12:25:012
3493033994234,5cyclictest3512516-21munin-plugin-st08:15:012
349303299424,32cyclictest3569983-21sh10:16:041
3493032994234,5cyclictest3545388-21missed_timers09:35:571
3493032994234,5cyclictest3545388-21missed_timers09:35:571
3493032994233,6cyclictest3623619-21latency_hist11:45:021
349303299422,32cyclictest3609556-21ssh11:21:041
3493031994235,4cyclictest1000-21mta-sts-daemon08:31:080
3493033994129,9cyclictest1-21systemd10:25:012
349303299415,28cyclictest1002-21mta-sts-daemon10:31:011
3493032994130,8cyclictest3647743-21cron12:25:011
3493033994031,6cyclictest1-21systemd07:55:012
3493033994031,5cyclictest0-21swapper/210:16:052
3493032994031,5cyclictest1001-21mta-sts-daemon10:24:461
3493033993930,6cyclictest3521680-21latency_hist08:45:022
3493033993930,5cyclictest0-21swapper/210:35:012
3493032993932,4cyclictest1002-21mta-sts-daemon12:26:011
3493032993931,5cyclictest3502029-21basename07:40:011
3493032993930,7cyclictest3523034-21sh08:50:011
3493032993918,12cyclictest3529020-21diskmemload10:56:011
3493031993929,6cyclictest151ktimers/007:56:020
3493031993920,12cyclictest151ktimers/008:06:010
3493033993830,5cyclictest3650790-21latency_hist12:30:022
3493033993830,5cyclictest3622312-21unixbench_singl11:41:152
3493032993832,3cyclictest0-21swapper/109:55:011
3493032993826,9cyclictest3495357-21ntp_kernel_err07:16:031
349303299382,31cyclictest3560838-21ssh10:01:041
3493031993832,3cyclictest0-21swapper/011:26:070
3493031993831,4cyclictest1001-21mta-sts-daemon09:40:470
3493031993830,5cyclictest421-21dbus-daemon07:45:010
3493031993829,7cyclictest3559584-21idleruntime-cro10:00:010
349303399371,31cyclictest3588299-21ssh10:46:062
3493032993730,4cyclictest3627409-21gltestperf11:50:561
3493032993729,5cyclictest3653786-21systemd-run12:35:021
3493032993729,5cyclictest3516952-21unixbench_singl08:26:131
3493032993729,4cyclictest3521641-21latency_hist08:45:021
349303299372,31cyclictest2702384-21cups-browsed12:11:001
349303199374,24cyclictest3582103-21ssh10:36:030
349303199374,24cyclictest3582103-21ssh10:36:030
3493031993731,4cyclictest3541877-21aten_r2power_fr09:30:520
3493031993729,5cyclictest0-21swapper/010:07:480
3493031993728,7cyclictest421-21dbus-daemon10:25:010
3493031993722,13cyclictest3566635-21latency10:10:570
349303199371,26cyclictest3651999-21ssh12:31:050
3493033993630,3cyclictest0-21swapper/210:01:142
3493033993628,5cyclictest3559562-21ssh10:00:012
3493033993627,6cyclictest2702353-21cupsd07:40:012
3493033993625,8cyclictest3638762-21munin-plugin-st12:10:022
3493032993626,8cyclictest0-21swapper/108:40:011
3493032993626,6cyclictest750-21ntpd07:21:041
3493031993629,4cyclictest3578952-21needreboot10:30:590
3493031993628,4cyclictest3547637-21latency_hist09:40:030
3493031993628,4cyclictest3547637-21latency_hist09:40:030
3493033993527,5cyclictest3596939-21fschecks_time11:00:572
3493033993527,5cyclictest3591257-21ssh10:51:072
3493033993527,5cyclictest3532100-21ssh09:15:012
3493033993527,4cyclictest0-21swapper/211:56:022
3493033993527,4cyclictest0-21swapper/211:56:022
3493033993527,4cyclictest0-21swapper/211:51:482
3493033993511,5cyclictest171rcu_preempt08:21:042
3493032993526,4cyclictest0-21swapper/110:40:561
3493031993528,4cyclictest3501044-21fschecks_time07:35:540
3493031993527,3cyclictest151ktimers/009:11:070
3493031993524,8cyclictest420-21avahi-daemon11:51:110
3493033993428,3cyclictest0-21swapper/211:20:002
3493033993426,5cyclictest3614410-21ssh11:30:012
3493033993426,5cyclictest3576196-21proc_pri10:26:122
3493033993426,5cyclictest0-21swapper/207:45:002
3493033993425,4cyclictest0-21swapper/209:41:042
3493032993423,7cyclictest3636217-21munin-node12:05:521
349303299342,25cyclictest3563843-21sh10:06:031
349303299342,24cyclictest3536348-21ssh09:21:011
349303299342,22cyclictest3642995-21ssh12:16:061
349303199345,20cyclictest12650irq/32-eth010:46:060
3493031993428,4cyclictest3621472-21latency11:40:580
3493031993427,4cyclictest3643416-21unixbench-2d12:16:140
3493031993426,6cyclictest3586991-21idleruntime-cro10:45:000
3493031993426,5cyclictest420-21avahi-daemon11:03:510
349303199342,26cyclictest3633905-21ssh12:01:060
3493033993326,4cyclictest1001-21mta-sts-daemon07:21:052
3493033993326,4cyclictest1000-21mta-sts-daemon10:39:482
3493033993326,4cyclictest1000-21mta-sts-daemon10:39:482
3493033993325,5cyclictest3527516-21cron09:05:012
3493033993324,4cyclictest0-21swapper/210:06:162
3493033993322,8cyclictest3633340-21aten_r2power_vo12:00:542
3493032993327,4cyclictest3493030-21cyclictest08:56:051
3493032993327,3cyclictest0-21swapper/111:40:011
3493032993326,4cyclictest3541284-21cron09:30:011
3493032993326,3cyclictest0-21swapper/109:20:011
3493032993325,4cyclictest1000-21mta-sts-daemon09:34:401
3493032993325,3cyclictest0-21swapper/111:26:011
3493032993325,3cyclictest0-21swapper/110:36:021
3493032993325,3cyclictest0-21swapper/110:36:021
3493032993324,6cyclictest265-21systemd-journal08:25:011
3493032993323,4cyclictest0-21swapper/109:11:001
3493031993325,4cyclictest3541311-21kworker/u8:6-xprtiod10:01:030
3493031993325,3cyclictest0-21swapper/012:36:050
3493031993325,3cyclictest0-21swapper/012:36:050
3493031993324,6cyclictest1002-21mta-sts-daemon11:47:260
349303199332,26cyclictest3615544-21sh11:30:590
3493031993321,8cyclictest12650irq/32-eth012:21:070
3493033993225,4cyclictest1001-21mta-sts-daemon07:16:162
3493033993225,3cyclictest3514051-21kworker/u8:13-xprtiod08:40:012
3493033993224,5cyclictest3656771-21munin-run12:40:012
3493033993224,5cyclictest3656771-21munin-run12:40:012
3493033993224,5cyclictest3603208-21latency11:10:572
3493033993224,5cyclictest3545825-21mailstats09:36:092
3493033993224,5cyclictest3545825-21mailstats09:36:092
349303399322,23cyclictest3618734-21ssh11:36:052
3493032993225,4cyclictest1003-21mta-sts-daemon09:46:071
3493032993225,3cyclictest0-21swapper/109:10:011
3493032993224,5cyclictest3498060-21fschecks_time07:25:541
3493032993223,5cyclictest3555513-21kworker/u8:1-events_unbound09:55:521
349303299322,23cyclictest3576014-21ssh10:26:051
3493031993226,3cyclictest0-21swapper/009:15:540
3493031993225,5cyclictest3553981-21/usr/sbin/munin09:51:130
3493031993224,6cyclictest3539934-21timerwakeupswit09:26:160
3493031993224,5cyclictest2702384-21cups-browsed09:01:100
3493033993124,4cyclictest3602205-21cron11:10:012
3493033993124,4cyclictest3523547-21fschecks_time08:50:542
3493033993123,5cyclictest3568749-21latency_hist10:15:022
3493033993123,4cyclictest1003-21mta-sts-daemon09:30:032
3493033993123,4cyclictest0-21swapper/211:21:042
349303399312,23cyclictest3640046-21ssh12:11:062
349303299314,4cyclictest3591194-21ssh10:51:031
3493032993125,3cyclictest1000-21mta-sts-daemon09:41:101
3493032993121,7cyclictest3567431-21grep10:11:131
3493031993125,4cyclictest0-21swapper/012:11:070
3493031993121,4cyclictest3593050-21cron10:55:010
3493031993118,9cyclictest151ktimers/008:11:000
3493031993118,8cyclictest3603539-21sh11:11:080
349303199311,25cyclictest3606432-21ssh11:16:010
3493033993023,4cyclictest0-21swapper/210:45:012
3493033993023,3cyclictest0-21swapper/209:16:142
3493033993022,5cyclictest3644718-21cron12:20:012
3493033993021,4cyclictest0-21swapper/209:23:542
349303299303,22cyclictest3633960-21ssh12:01:081
3493032993025,2cyclictest0-21swapper/108:20:011
3493032993023,4cyclictest3597899-21unixbench_multi11:01:161
3493032993022,5cyclictest3512505-21cron08:15:011
349303299302,20cyclictest3655052-21ssh12:36:061
349303299302,20cyclictest3655052-21ssh12:36:061
3493032993020,5cyclictest3518498-40mandb08:34:391
3493031993024,4cyclictest3600982-21unixbench_singl11:06:130
3493031993024,3cyclictest0-21swapper/010:55:570
3493031993023,4cyclictest0-21swapper/011:56:060
3493031993023,4cyclictest0-21swapper/011:56:060
3493031993022,6cyclictest3496011-21cron07:20:010
3493031993019,3cyclictest0-21swapper/010:20:010
3493031993012,14cyclictest12650irq/32-eth011:36:050
349303399294,15cyclictest428-21mta-sts-daemon07:26:072
3493033992924,2cyclictest0-21swapper/208:45:592
3493033992922,3cyclictest0-21swapper/209:30:592
3493033992921,6cyclictest0-21swapper/209:50:592
3493033992918,8cyclictest3651700-21kernelversion12:30:582
3493032992922,3cyclictest0-21swapper/111:46:051
3493032992921,5cyclictest421-21dbus-daemon07:55:011
3493032992921,4cyclictest3524391-21unixbench_multi08:51:121
3493032992920,7cyclictest3504426-21ntp_kernel_pll_07:46:031
3493032992920,7cyclictest3504426-21ntp_kernel_pll_07:46:031
3493032992912,11cyclictest265-21systemd-journal07:59:561
3493031992922,4cyclictest3611350-21cron11:25:000
3493031992922,4cyclictest3520007-21cron08:40:010
3493031992922,4cyclictest1001-21mta-sts-daemon07:31:000
3493031992919,7cyclictest151ktimers/010:26:120
3493033992820,5cyclictest3626534-21sh11:49:542
3493033992719,5cyclictest1003-21mta-sts-daemon07:30:592
3493031992720,4cyclictest3521990-21cpuspeed08:45:520
3493033992620,4cyclictest3509518-21kworker/u8:11-xprtiod08:34:372
3493033992620,3cyclictest0-21swapper/209:10:022
3493033992620,3cyclictest0-21swapper/208:00:012
3493032992620,3cyclictest1001-21mta-sts-daemon07:41:061
3493032992619,4cyclictest1001-21mta-sts-daemon08:01:531
3493032992617,5cyclictest3497517-21kworker/u8:2+rpciod07:35:011
3493031992621,3cyclictest1003-21mta-sts-daemon08:05:000
3493031992619,4cyclictest3638753-21ssh12:10:010
3493031992618,6cyclictest3521305-21timerwakeupswit08:41:110
3493031992618,6cyclictest3514008-21cron08:20:010
3493033992515,7cyclictest2702384-21cups-browsed08:56:342
3493031992520,3cyclictest151ktimers/007:51:000
3493031992518,4cyclictest3497628-21latency_hist07:25:020
3493031992517,6cyclictest3504294-21missed_timers07:45:590
3493031992517,6cyclictest3504294-21missed_timers07:45:590
3493031992515,7cyclictest0-21swapper/008:55:570
3493033992418,3cyclictest0-21swapper/208:10:022
3493033992417,4cyclictest1003-21mta-sts-daemon07:15:012
349303299242,17cyclictest566-21gmain08:06:031
349303199242,16cyclictest3523919-21ntpq08:51:020
3493031992417,4cyclictest2702384-21cups-browsed07:29:350
3493031992416,6cyclictest3494464-21unixbench_singl07:11:170
3493033992315,5cyclictest3513532-21wc08:16:102
349303199232,14cyclictest3514963-21ntpq08:21:030
3493031992318,2cyclictest0-21swapper/009:10:010
349303399222,14cyclictest3508948-21ntpq08:01:022
3493031992216,3cyclictest0-21swapper/008:26:130
3493033992115,3cyclictest0-21swapper/207:46:012
3493033992115,3cyclictest0-21swapper/207:46:012
3493032992113,5cyclictest3493043-21systemd-run07:10:011
349303499204,9cyclictest0-21swapper/312:26:063
349303499183,5cyclictest0-21swapper/307:21:053
349303499182,9cyclictest0-21swapper/309:45:563
3493033991812,3cyclictest0-21swapper/207:10:012
3493031991811,5cyclictest3489159-21kworker/u8:0-rpciod07:10:010
349303499172,9cyclictest0-21swapper/311:51:053
349303499172,9cyclictest0-21swapper/309:41:123
349303499172,9cyclictest0-21swapper/309:31:083
349303499163,6cyclictest0-21swapper/308:34:383
349303499162,8cyclictest0-21swapper/312:06:003
349303499162,7cyclictest0-21swapper/311:11:013
349303499161,8cyclictest0-21swapper/310:26:063
349303499161,8cyclictest0-21swapper/309:36:033
349303499161,8cyclictest0-21swapper/309:36:033
349303499161,7cyclictest0-21swapper/307:41:073
349303499161,7cyclictest0-21swapper/307:31:043
349303499161,6cyclictest0-21swapper/308:56:023
349303499161,6cyclictest0-21swapper/307:16:033
349303499153,7cyclictest0-21swapper/310:45:593
349303499153,6cyclictest0-21swapper/310:05:543
349303499153,6cyclictest0-21swapper/309:56:023
349303499153,5cyclictest0-21swapper/308:21:003
349303499152,7cyclictest0-21swapper/312:01:043
349303499151,9cyclictest0-21swapper/312:16:133
349303499151,9cyclictest0-21swapper/311:35:293
349303499151,8cyclictest0-21swapper/311:21:013
349303499151,8cyclictest0-21swapper/310:20:573
349303499144,5cyclictest0-21swapper/310:11:033
349303499143,7cyclictest0-21swapper/312:11:033
349303499143,6cyclictest0-21swapper/311:40:523
349303499143,6cyclictest0-21swapper/310:41:003
349303499142,8cyclictest0-21swapper/311:00:013
349303499142,8cyclictest0-21swapper/309:51:023
349303499142,7cyclictest0-21swapper/312:21:013
349303499142,7cyclictest0-21swapper/311:15:553
349303499142,7cyclictest0-21swapper/309:21:033
349303499142,7cyclictest0-21swapper/307:55:543
349303499142,6cyclictest0-21swapper/310:01:013
349303499141,7cyclictest0-21swapper/311:56:003
349303499141,7cyclictest0-21swapper/311:56:003
349303499141,7cyclictest0-21swapper/311:00:593
349303499141,7cyclictest0-21swapper/310:30:293
349303499141,7cyclictest0-21swapper/309:15:533
349303499141,6cyclictest0-21swapper/308:51:043
349303499141,6cyclictest0-21swapper/308:36:033
349303499141,6cyclictest0-21swapper/307:35:593
349303499135,5cyclictest431ktimers/307:10:013
349303499133,6cyclictest0-21swapper/312:36:023
349303499133,6cyclictest0-21swapper/312:36:023
349303499133,6cyclictest0-21swapper/310:35:543
349303499133,6cyclictest0-21swapper/310:35:543
349303499132,7cyclictest0-21swapper/311:05:553
349303499132,7cyclictest0-21swapper/309:26:023
349303499132,6cyclictest0-21swapper/312:31:013
349303499131,8cyclictest0-21swapper/309:10:093
349303499131,7cyclictest0-21swapper/311:45:593
349303499131,7cyclictest0-21swapper/311:30:543
349303499131,7cyclictest0-21swapper/310:50:543
349303499131,7cyclictest0-21swapper/309:06:073
349303499131,6cyclictest0-21swapper/308:00:583
349303499122,6cyclictest0-21swapper/309:05:013
349303499122,6cyclictest0-21swapper/308:11:033
349303499121,8cyclictest0-21swapper/311:26:043
349303499121,7cyclictest0-21swapper/310:16:013
349303499121,6cyclictest0-21swapper/308:50:013
349303499121,6cyclictest0-21swapper/307:26:033
349303499112,5cyclictest0-21swapper/307:51:043
349303499111,6cyclictest0-21swapper/308:16:043
349303499111,6cyclictest0-21swapper/308:06:043
349303499101,6cyclictest0-21swapper/307:46:083
349303499101,6cyclictest0-21swapper/307:46:083
349303499101,5cyclictest0-21swapper/308:25:573
34930349995,3cyclictest0-21swapper/307:10:523
34930349994,3cyclictest0-21swapper/308:40:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional