You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 23:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Aug 23, 2026 12:42:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3837680997869,6cyclictest3934701-21kworker/u8:11-xprtiod11:25:591
383768099524,39cyclictest3975483-21ssh11:56:011
3837680995136,9cyclictest420-21avahi-daemon11:42:541
3837680994838,6cyclictest3963904-21unixbench_singl11:36:181
383768099481,34cyclictest3929749-21sh10:41:001
3837679994636,6cyclictest3032758-21cups-browsed08:36:020
3837681994535,6cyclictest3032758-21cups-browsed09:55:502
3837680994534,7cyclictest420-21avahi-daemon10:18:241
3837680994534,6cyclictest3867669-21munin-plugin-st08:50:011
3837680994436,5cyclictest3907306-21ssh10:05:011
3837681994333,6cyclictest3895290-21latency_hist09:45:022
3837680994334,4cyclictest0-21swapper/109:16:071
383768199422,29cyclictest3032758-21cups-browsed07:56:012
3837680994234,4cyclictest0-21swapper/112:20:021
3837680994231,6cyclictest3951732-21unixbench_singl11:16:151
3837679994234,5cyclictest3032758-21cups-browsed11:16:030
3837679994229,9cyclictest0-21swapper/011:10:010
3837679994229,11cyclictest3866151-40mandb08:43:400
3837679994221,15cyclictest12650irq/32-eth010:16:060
3837681994136,2cyclictest0-21swapper/211:00:012
3837681994133,5cyclictest485-21in:imuxsock11:22:492
3837681994132,5cyclictest4001668-21cut12:40:012
3837681994132,5cyclictest3869165-21cut08:55:012
383768099404,26cyclictest3032755-21cupsd09:36:011
3837680994032,5cyclictest3960000-21kernelversion11:30:551
3837680994032,5cyclictest3960000-21kernelversion11:30:551
3837680994030,5cyclictest0-21swapper/111:46:011
3837680994030,4cyclictest0-21swapper/111:01:051
383768099402,29cyclictest3948106-21ssh11:11:021
3837679994022,13cyclictest12650irq/32-eth011:56:020
383768199397,17cyclictest1002-21mta-sts-daemon08:43:462
3837681993931,4cyclictest0-21swapper/210:01:022
3837680993927,9cyclictest3899946-21sed09:51:131
383768099391,6cyclictest171rcu_preempt07:51:021
383768099391,31cyclictest3926772-21ssh10:36:031
3837679993931,6cyclictest3904254-21sendmail10:00:010
3837681993832,3cyclictest0-21swapper/210:21:052
3837681993831,3cyclictest0-21swapper/212:05:562
3837681993830,5cyclictest3873671-21diskmemload11:30:502
3837681993830,5cyclictest3873671-21diskmemload11:30:502
383768199381,30cyclictest3966462-21ssh11:41:082
383768099389,19cyclictest428-21mta-sts-daemon08:51:031
383768099387,5cyclictest0-21swapper/107:31:001
3837680993830,5cyclictest3934706-21systemd-run10:50:011
3837680993829,5cyclictest3990543-21missed_timers12:20:561
3837680993828,7cyclictest3980524-21latency_hist12:05:021
3837680993828,5cyclictest1002-21mta-sts-daemon10:59:591
3837680993828,5cyclictest0-21swapper/109:46:041
3837680993824,9cyclictest0-21swapper/108:43:481
3837680993823,12cyclictest1001-21mta-sts-daemon07:40:591
3837680993821,13cyclictest271ktimers/111:06:031
3837679993820,12cyclictest12650irq/32-eth009:21:010
3837681993731,3cyclictest0-21swapper/210:30:582
3837681993731,3cyclictest0-21swapper/210:30:582
3837681993730,4cyclictest3857173-21systemd-run08:15:012
3837681993727,6cyclictest567-21gdbus10:16:012
3837680993729,3cyclictest0-21swapper/110:00:011
3837680993728,6cyclictest420-21avahi-daemon09:05:011
3837680993728,6cyclictest420-21avahi-daemon09:05:011
3837680993728,5cyclictest3885969-21systemd-run09:30:011
3837680993727,7cyclictest3846657-21sendmail07:40:001
3837680993723,9cyclictest3910340-21cut10:10:011
3837679993730,4cyclictest3937738-21systemd-run10:55:010
3837679993730,4cyclictest3860173-21cron08:25:010
3837679993729,5cyclictest420-21avahi-daemon09:30:530
3837681993628,5cyclictest3879805-21idleruntime-cro09:20:012
3837681993628,5cyclictest3843675-21basename07:30:002
3837681993628,5cyclictest3843675-21basename07:30:002
3837681993627,5cyclictest1-21systemd11:10:012
383768099365,16cyclictest3972493-21ssh11:51:031
3837680993631,2cyclictest0-21swapper/108:00:011
3837680993627,6cyclictest265-21systemd-journal08:10:011
3837680993627,5cyclictest3876742-21munin-plugin-st09:15:001
3837680993626,6cyclictest1003-21mta-sts-daemon08:01:011
383768099362,20cyclictest3881165-21ssh09:21:071
3837680993615,16cyclictest566-21gmain07:46:051
3837679993630,4cyclictest3943837-21latency_hist11:05:020
383768199353,28cyclictest3963395-21ssh11:36:092
3837681993530,2cyclictest0-21swapper/211:20:012
3837681993529,3cyclictest0-21swapper/210:36:072
3837681993528,4cyclictest3912129-21unixbench_singl10:11:122
3837681993527,5cyclictest3883083-21latency_hist09:25:022
3837681993527,5cyclictest3872202-21systemd-run09:05:012
3837681993527,5cyclictest3872202-21systemd-run09:05:012
3837681993527,5cyclictest3842102-21unixbench_singl07:21:132
3837681993527,5cyclictest1002-21mta-sts-daemon11:05:012
3837680993526,6cyclictest3983997-21ssh12:10:501
3837680993523,8cyclictest3873677-21idleruntime-cro09:10:001
3837680993523,8cyclictest3873677-21idleruntime-cro09:10:001
383768099352,27cyclictest3887111-21ssh09:30:591
3837680993516,11cyclictest271ktimers/108:35:591
3837679993529,4cyclictest3893302-21ntp_kernel_pll_09:41:010
3837679993529,3cyclictest1001-21mta-sts-daemon10:46:060
3837679993528,4cyclictest4001654-21ssh12:40:010
3837679993527,5cyclictest420-21avahi-daemon11:12:570
3837679993521,10cyclictest12650irq/32-eth010:06:040
3837681993427,4cyclictest3901208-21cron09:55:012
3837681993426,5cyclictest421-21dbus-daemon12:35:022
3837681993425,6cyclictest3032755-21cupsd12:10:032
383768099344,18cyclictest428-21mta-sts-daemon12:26:051
383768099343,8cyclictest171rcu_preempt10:11:051
3837680993427,4cyclictest3982085-21unixbench-2d12:06:131
3837680993425,6cyclictest3860062-21tail08:21:111
383768099341,26cyclictest3996749-21ssh12:31:011
3837679993428,4cyclictest3939323-21timerandwakeup10:56:150
3837679993428,4cyclictest3845695-21fschecks_time07:35:540
3837679993427,4cyclictest3907354-21latency_hist10:05:020
3837679993426,5cyclictest421-21dbus-daemon09:45:440
3837681993325,5cyclictest3848635-21cpuspeed07:45:532
3837681993325,5cyclictest3846656-21idleruntime-cro07:40:002
3837681993320,5cyclictest1003-21mta-sts-daemon09:25:342
3837680993327,3cyclictest0-21swapper/109:00:001
3837680993324,4cyclictest0-21swapper/112:36:061
3837680993322,7cyclictest420-21avahi-daemon07:21:081
3837679993325,6cyclictest3980972-21ssh12:05:510
3837679993325,5cyclictest420-21avahi-daemon11:23:470
3837679993325,5cyclictest151ktimers/009:01:050
3837679993325,5cyclictest151ktimers/009:01:050
3837681993225,4cyclictest3863154-21cron08:35:012
3837681993225,4cyclictest1001-21mta-sts-daemon10:43:232
3837681993224,5cyclictest3922506-21cron10:30:002
3837681993224,5cyclictest3922506-21cron10:30:002
3837681993224,5cyclictest3890602-21timerandwakeup09:36:122
3837681993223,7cyclictest3896755-21timerandwakeup09:46:132
3837681993223,6cyclictest3032758-21cups-browsed08:59:082
3837680993224,5cyclictest1000-21mta-sts-daemon10:51:161
3837679993226,4cyclictest0-21swapper/010:11:010
3837679993226,3cyclictest3866150-21kworker/u8:1-xprtiod09:00:010
3837679993226,3cyclictest0-21swapper/008:31:140
3837679993225,4cyclictest3990566-21missed_timers12:20:570
3837679993225,4cyclictest3868184-21fschecks_count08:50:530
3837679993223,4cyclictest0-21swapper/009:36:030
383767999322,25cyclictest3929844-21ssh10:41:030
3837679993220,9cyclictest0-21swapper/012:15:570
3837679993220,6cyclictest3878518-21unixbench_multi09:16:160
3837681993124,4cyclictest487-21rs:main2
3837681993123,5cyclictest3886434-21ssh09:30:492
3837681993123,5cyclictest3032758-21cups-browsed10:48:512
3837681993122,7cyclictest3840433-21timerwakeupswit07:16:122
3837681993122,6cyclictest421-21dbus-daemon11:11:292
3837681993120,8cyclictest3990953-21sendmail_mailtr12:21:102
3837680993123,5cyclictest3893878-21unixbench_singl09:41:121
3837680993121,5cyclictest0-21swapper/110:32:101
3837680993121,5cyclictest0-21swapper/110:32:101
383768099312,13cyclictest566-21gmain10:28:131
383768099312,13cyclictest566-21gmain10:28:131
3837679993125,3cyclictest0-21swapper/010:35:010
3837679993125,3cyclictest0-21swapper/010:35:010
3837679993123,5cyclictest3883368-21/usr/sbin/munin09:26:110
3837679993123,5cyclictest0-21swapper/011:45:010
3837679993122,6cyclictest1002-21mta-sts-daemon11:46:330
3837679993110,16cyclictest3972535-21ntpq11:51:060
3837681993025,2cyclictest0-21swapper/210:10:012
3837681993024,3cyclictest0-21swapper/210:51:102
3837681993024,3cyclictest0-21swapper/209:10:002
3837681993024,3cyclictest0-21swapper/209:10:002
3837681993023,3cyclictest0-21swapper/212:26:012
3837681993023,3cyclictest0-21swapper/209:11:032
3837681993022,4cyclictest0-21swapper/211:46:082
383768099304,17cyclictest428-21mta-sts-daemon07:16:021
3837680993022,5cyclictest3856179-21fschecks_count08:10:531
3837680993019,8cyclictest3838563-21ntp_kernel_pll_07:11:011
3837679993023,3cyclictest0-21swapper/009:06:000
3837679993023,3cyclictest0-21swapper/009:06:000
3837679993021,7cyclictest3899703-21switchtime09:51:110
3837679993021,7cyclictest12650irq/32-eth010:21:010
3837679993021,6cyclictest3997203-21timerandwakeup12:31:130
3837679993020,4cyclictest3959078-21grep11:30:010
383767999301,24cyclictest3995210-21ssh12:28:520
3837681992922,4cyclictest0-21swapper/211:50:502
3837681992921,3cyclictest0-21swapper/211:56:122
383768099295,9cyclictest171rcu_preempt10:21:051
3837680992921,4cyclictest0-21swapper/111:25:011
3837679992923,4cyclictest3032758-21cups-browsed08:11:030
3837679992923,4cyclictest0-21swapper/010:40:000
3837679992922,4cyclictest3876728-21ssh09:15:000
3837679992922,3cyclictest0-21swapper/012:01:050
3837679992921,5cyclictest420-21avahi-daemon08:01:080
3837679992921,3cyclictest0-21swapper/010:25:220
3837679992921,3cyclictest0-21swapper/010:25:220
3837681992820,5cyclictest485-21in:imuxsock08:49:492
3837681992820,5cyclictest3957695-21timerwakeupswit11:26:112
3837679992822,4cyclictest3986481-21grep12:15:020
3837679992821,3cyclictest0-21swapper/011:36:080
3837681992721,3cyclictest0-21swapper/208:20:012
3837681992721,3cyclictest0-21swapper/207:15:012
3837680992721,3cyclictest0-21swapper/108:30:011
3837679992721,4cyclictest3849610-21unixbench_singl07:46:130
3837679992721,4cyclictest1003-21mta-sts-daemon11:31:550
3837679992721,4cyclictest1003-21mta-sts-daemon11:31:550
3837679992721,3cyclictest1003-21mta-sts-daemon08:30:010
3837679992720,4cyclictest3840665-21sendmail07:20:010
3837679992716,6cyclictest3839156-21cron07:15:010
3837681992619,4cyclictest1001-21mta-sts-daemon12:15:182
3837681992618,5cyclictest3861857-21grep08:30:032
3837681992618,5cyclictest3853205-21fschecks_time08:00:542
3837681992610,12cyclictest1001-21mta-sts-daemon07:10:012
3837680992617,6cyclictest3837690-21systemd-run07:10:011
3837679992618,5cyclictest3032758-21cups-browsed07:54:320
383768199259,13cyclictest351ktimers/208:36:012
3837681992519,3cyclictest0-21swapper/208:20:592
3837681992518,4cyclictest3854687-21fschecks_count08:05:542
3837680992520,2cyclictest0-21swapper/107:30:011
3837680992520,2cyclictest0-21swapper/107:30:011
3837679992517,6cyclictest3843677-21cut07:30:010
3837679992517,6cyclictest3843677-21cut07:30:010
3837679992511,10cyclictest265-21systemd-journal07:44:490
383768199248,13cyclictest3850890-21sed07:51:102
383768199248,13cyclictest3844934-21sed07:31:122
383768099242,16cyclictest3032755-21cupsd08:31:041
3837679992419,3cyclictest0-21swapper/008:50:010
3837679992418,4cyclictest3032758-21cups-browsed08:07:120
3837679992416,5cyclictest3032758-21cups-browsed07:23:220
383768299233,9cyclictest0-21swapper/309:16:073
3837681992310,10cyclictest627-21lldpd07:43:562
3837679992315,6cyclictest3858662-21/usr/sbin/munin08:20:010
383768299223,8cyclictest0-21swapper/309:26:023
383767999221,18cyclictest3852136-21ntpq07:56:070
383767999221,16cyclictest3844621-21ntp_states07:31:060
383767999221,10cyclictest1-21systemd07:10:010
383768299214,8cyclictest0-21swapper/308:43:503
383768099212,14cyclictest3858040-21ntpq08:16:021
383768299193,7cyclictest0-21swapper/310:31:023
383768299193,7cyclictest0-21swapper/310:31:023
383768299183,9cyclictest0-21swapper/311:21:023
383768299183,8cyclictest0-21swapper/310:41:013
383768299173,7cyclictest0-21swapper/312:20:583
383768299172,9cyclictest0-21swapper/311:06:043
383768299172,9cyclictest0-21swapper/309:56:003
383768299172,9cyclictest0-21swapper/309:20:593
383768299172,8cyclictest0-21swapper/310:00:583
383768299172,7cyclictest0-21swapper/311:30:523
383768299172,7cyclictest0-21swapper/311:30:523
383768299172,7cyclictest0-21swapper/310:15:593
383768299172,7cyclictest0-21swapper/307:40:583
383768299164,7cyclictest0-21swapper/310:11:043
383768299163,8cyclictest0-21swapper/311:01:063
383768299163,8cyclictest0-21swapper/310:46:063
383768299163,8cyclictest0-21swapper/309:46:013
383768299163,7cyclictest0-21swapper/312:06:073
383768299162,9cyclictest0-21swapper/310:21:023
383768299162,8cyclictest0-21swapper/312:25:533
383768299162,8cyclictest0-21swapper/310:06:003
383768299154,8cyclictest0-21swapper/311:46:043
383768299153,8cyclictest0-21swapper/312:31:053
383768299153,7cyclictest0-21swapper/309:35:513
383768299153,6cyclictest0-21swapper/307:46:033
383768299152,9cyclictest0-21swapper/312:01:023
383768299152,8cyclictest0-21swapper/311:41:013
383768299152,8cyclictest0-21swapper/309:30:583
383768299152,8cyclictest0-21swapper/308:21:003
383768299152,7cyclictest0-21swapper/309:51:003
383768299152,6cyclictest0-21swapper/308:31:013
383768299151,9cyclictest0-21swapper/309:41:063
383768299151,7cyclictest0-21swapper/308:16:013
383768299144,5cyclictest0-21swapper/307:21:043
383768299143,7cyclictest0-21swapper/312:11:023
383768299143,7cyclictest0-21swapper/311:51:063
383768299143,7cyclictest0-21swapper/310:56:033
383768299143,4cyclictest0-21swapper/307:56:003
383768299142,9cyclictest0-21swapper/309:10:493
383768299142,8cyclictest0-21swapper/312:16:033
383768299142,8cyclictest0-21swapper/311:56:013
383768299142,8cyclictest0-21swapper/311:25:593
383768299142,8cyclictest0-21swapper/310:51:023
383768299142,7cyclictest0-21swapper/308:26:013
383768299142,5cyclictest0-21swapper/307:35:593
383768299141,8cyclictest0-21swapper/310:25:563
383768299141,8cyclictest0-21swapper/310:25:563
383768299141,7cyclictest0-21swapper/308:11:023
383768299134,6cyclictest0-21swapper/307:11:053
383768299134,4cyclictest0-21swapper/307:51:013
383768299133,7cyclictest0-21swapper/307:31:013
383768299133,6cyclictest0-21swapper/310:35:533
383768299132,8cyclictest0-21swapper/311:11:023
383768299132,7cyclictest0-21swapper/312:35:583
383768299132,7cyclictest0-21swapper/311:35:513
383768299132,7cyclictest0-21swapper/308:46:083
383768299132,6cyclictest0-21swapper/309:06:003
383768299132,6cyclictest0-21swapper/309:06:003
383768299132,6cyclictest0-21swapper/308:00:593
383768299131,6cyclictest0-21swapper/309:00:593
383768299131,6cyclictest0-21swapper/309:00:593
383768299125,5cyclictest0-21swapper/308:56:013
383768299122,6cyclictest0-21swapper/311:15:493
383768299122,6cyclictest0-21swapper/308:51:053
383768299111,7cyclictest0-21swapper/308:06:003
383768299111,7cyclictest0-21swapper/307:26:043
383768299111,7cyclictest0-21swapper/307:26:043
383768299111,6cyclictest0-21swapper/308:35:593
383768299101,5cyclictest0-21swapper/307:15:513
38376829981,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional