You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 21:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Aug 16, 2026 12:43:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3129585995343,7cyclictest3239970-21sshd11:11:012
3129585995343,7cyclictest3239970-21sshd11:11:012
3129585995239,8cyclictest3281577-21date12:20:032
3129582995243,6cyclictest421-21dbus-daemon07:20:010
3129582994839,6cyclictest2341350-21cups-browsed09:47:230
3129582994835,10cyclictest3166844-21sshd09:11:010
3129585994738,4cyclictest0-21swapper/211:25:592
3129582994738,5cyclictest3162587-21systemd-run09:00:010
3129582994737,8cyclictest0-21swapper/010:41:010
3129585994633,10cyclictest3267495-21mailstats11:56:062
3129582994628,9cyclictest12650irq/32-eth011:46:010
3129582994627,13cyclictest12650irq/32-eth009:40:570
312958299454,32cyclictest2341350-21cups-browsed11:00:580
3129582994538,4cyclictest3183997-21systemd09:40:010
3129582994533,10cyclictest3282789-21mailstats12:21:100
3129583994434,6cyclictest265-21systemd-journal12:04:591
3129583994434,6cyclictest265-21systemd-journal12:04:591
3129582994334,6cyclictest2341350-21cups-browsed10:19:080
3129582994334,6cyclictest2341350-21cups-browsed10:19:080
312958299424,17cyclictest3226579-21kworker/u8:0+rpciod10:50:560
3129582994231,8cyclictest3272189-21cron12:05:000
3129582994231,8cyclictest3272189-21cron12:05:000
3129582994219,14cyclictest151ktimers/009:51:030
312958399414,27cyclictest3255144-21ssh11:36:001
3129583994133,5cyclictest3269173-21systemd-run12:00:011
3129583994127,4cyclictest171rcu_preempt08:05:011
3129582994134,5cyclictest3156651-21latency_hist08:40:020
3129582994124,13cyclictest151ktimers/008:41:000
312958299412,34cyclictest3267234-21ssh11:55:570
312958599403,33cyclictest3273451-21rm12:06:012
312958599403,33cyclictest3273451-21rm12:06:012
3129585994031,6cyclictest421-21dbus-daemon07:20:012
3129583994033,4cyclictest1000-21mta-sts-daemon09:22:431
3129583994032,4cyclictest0-21swapper/107:20:011
3129583994030,6cyclictest421-21dbus-daemon10:59:411
3129582994025,12cyclictest3197865-21unixbench_multi10:01:100
3129585993933,3cyclictest0-21swapper/212:35:012
3129585993931,5cyclictest3146080-21idleruntime-cro08:05:012
3129583993931,4cyclictest1000-21mta-sts-daemon11:42:191
3129583993923,5cyclictest171rcu_preempt11:15:581
3129583993923,3cyclictest171rcu_preempt09:36:011
3129582993934,2cyclictest0-21swapper/009:20:010
3129582993932,4cyclictest3232677-21systemd-run11:00:020
3129582993926,10cyclictest3200801-21timerandwakeup10:06:100
312958299392,32cyclictest3236976-21ssh11:06:020
3129585993831,4cyclictest3281333-21ssh12:20:012
3129585993830,5cyclictest3144556-21cut08:00:002
3129585993830,5cyclictest3138595-21systemd-run07:40:012
312958399383,28cyclictest3169767-21rm09:15:561
3129583993831,4cyclictest3176536-21grep09:26:101
3129583993831,4cyclictest3176536-21grep09:26:101
3129583993830,5cyclictest3150559-21munin-run08:20:001
3129583993830,5cyclictest3150559-21munin-run08:20:001
3129583993826,9cyclictest2440529-21munin-node11:31:151
3129582993832,4cyclictest3131176-21latency_hist07:15:020
3129585993725,9cyclictest3285930-21timerandwakeup12:26:082
3129585993725,8cyclictest3173360-21timerwakeupswit09:21:082
312958399377,20cyclictest3209561-21ssh10:21:021
3129583993718,10cyclictest0-21swapper/109:50:021
312958299376,23cyclictest428-21mta-sts-daemon08:16:000
312958299376,23cyclictest428-21mta-sts-daemon08:16:000
3129582993730,5cyclictest12650irq/32-eth011:10:590
3129582993730,5cyclictest12650irq/32-eth011:10:590
3129582993722,10cyclictest12650irq/32-eth008:45:560
3129585993628,5cyclictest3224878-21ps10:46:072
3129585993624,8cyclictest420-21avahi-daemon10:51:152
3129583993629,4cyclictest3290421-21latency_hist12:35:011
3129583993629,4cyclictest3143051-21cron07:55:001
3129583993628,5cyclictest3275226-21sh12:10:001
3129583993628,5cyclictest3275226-21sh12:10:001
3129583993628,5cyclictest3266211-21latency_hist11:55:021
3129583993628,5cyclictest3191841-21unixbench_singl09:51:111
3129585993528,4cyclictest3164015-21unixbench_singl09:01:092
3129585993526,5cyclictest3246249-21mailstats11:21:052
3129585993525,6cyclictest3223513-21cron10:45:012
3129585993524,8cyclictest3209275-21latency10:20:542
3129585993524,6cyclictest1001-21mta-sts-daemon08:10:592
3129585993524,6cyclictest1001-21mta-sts-daemon08:10:592
312958399353,27cyclictest3276528-21ssh12:11:021
312958399353,27cyclictest3276528-21ssh12:11:021
312958399353,24cyclictest3215719-21ssh10:31:031
3129583993530,2cyclictest0-21swapper/111:01:011
3129583993529,3cyclictest0-21swapper/111:50:011
3129583993528,4cyclictest3205248-21cron10:15:011
3129583993528,4cyclictest3144597-21kworker/u8:11-xprtiod09:10:011
3129583993528,3cyclictest0-21swapper/111:21:011
3129583993527,5cyclictest3219124-21unixbench-2d10:36:101
3129583993524,7cyclictest3284923-21aten_r2power_fr12:25:471
3129583993518,7cyclictest171rcu_preempt07:46:111
3129583993518,7cyclictest171rcu_preempt07:46:111
3129583993517,2cyclictest171rcu_preempt10:16:051
3129583993517,2cyclictest171rcu_preempt10:16:051
3129582993520,12cyclictest3247894-21systemd-run11:25:010
3129585993428,3cyclictest0-21swapper/210:25:482
3129585993427,5cyclictest3129581-21cyclictest11:51:082
3129585993427,3cyclictest0-21swapper/209:51:092
3129585993426,5cyclictest3233619-21ssh11:00:542
3129585993426,5cyclictest3232631-21munin-run11:00:012
3129585993426,5cyclictest3165553-21cron09:10:012
3129585993426,5cyclictest265-21systemd-journal10:35:002
3129585993425,6cyclictest3196139-21idleruntime-cro10:00:012
3129585993425,6cyclictest1003-21mta-sts-daemon11:48:362
3129583993427,4cyclictest3152095-21kworker/u8:1008:25:011
3129583993426,5cyclictest3240421-21unixbench-2d11:11:101
3129583993426,5cyclictest3240421-21unixbench-2d11:11:101
3129583993426,5cyclictest3144553-21munin-plugin-st08:00:001
312958399342,27cyclictest3166788-21ssh09:10:591
3129583993416,3cyclictest171rcu_preempt12:16:041
312958299346,26cyclictest0-21swapper/007:40:590
312958299346,26cyclictest0-21swapper/007:40:590
3129582993423,8cyclictest3175721-21latency09:25:530
3129582993423,8cyclictest3175721-21latency09:25:530
312958299342,25cyclictest3264263-21ssh11:51:000
3129582993417,13cyclictest151ktimers/010:11:030
3129582993417,11cyclictest265-21systemd-journal11:31:010
3129585993325,5cyclictest3252703-21sed11:31:132
3129585993325,5cyclictest3200938-21grep10:06:112
3129585993325,5cyclictest3150580-21idleruntime-cro08:20:002
3129585993325,5cyclictest3150580-21idleruntime-cro08:20:002
3129583993325,5cyclictest3133114-21fschecks_time07:20:501
3129583993325,4cyclictest0-21swapper/109:56:001
3129583993324,6cyclictest1002-21mta-sts-daemon11:07:081
3129583993323,7cyclictest3146914-21ntp_kernel_err08:05:551
3129583993323,6cyclictest2341350-21cups-browsed10:45:561
3129583993313,13cyclictest421-21dbus-daemon10:40:571
3129582993328,2cyclictest0-21swapper/008:10:000
3129582993326,3cyclictest0-21swapper/011:40:480
3129582993326,3cyclictest0-21swapper/010:35:530
3129582993325,5cyclictest265-21systemd-journal09:33:010
3129582993325,5cyclictest265-21systemd-journal09:33:010
3129582993324,6cyclictest3286098-21tail12:26:090
3129582993323,7cyclictest3288368-21latency12:30:520
3129582993323,7cyclictest0-21swapper/012:36:090
3129582993322,8cyclictest3209940-21timerandwakeup10:21:100
3129582993322,8cyclictest0-21swapper/012:20:010
3129585993226,3cyclictest0-21swapper/212:40:012
3129585993226,3cyclictest0-21swapper/209:45:552
3129585993225,4cyclictest3165563-21diskmemload12:10:532
3129585993225,4cyclictest3165563-21diskmemload12:10:532
3129585993224,5cyclictest3218471-21missed_timers10:35:552
3129585993222,3cyclictest3186595-21ssh09:43:442
3129585993220,3cyclictest171rcu_preempt08:25:012
3129585993219,3cyclictest3238109-21ssh11:08:042
312958399323,23cyclictest3197351-21ssh10:01:001
3129583993226,3cyclictest3247897-21kworker/u8:14-xprtiod11:30:011
3129583993225,4cyclictest3185012-21wc09:40:541
3129583993224,4cyclictest271ktimers/109:34:291
3129583993224,4cyclictest271ktimers/109:34:291
3129583993214,5cyclictest28-21ksoftirqd/108:15:011
3129583993214,5cyclictest28-21ksoftirqd/108:15:011
3129583993212,8cyclictest28-21ksoftirqd/107:40:011
3129582993227,3cyclictest3160974-21unixbench_singl08:51:090
312958299322,21cyclictest3255216-21ssh11:36:030
3129582993219,9cyclictest3212521-21ssh10:25:570
3129585993124,4cyclictest0-21swapper/211:20:012
3129585993121,6cyclictest3147589-21systemd-run08:10:012
3129585993119,5cyclictest3259754-21rm11:44:122
3129585993119,3cyclictest3180740-21ssh09:34:352
3129585993119,3cyclictest3180740-21ssh09:34:352
312958399316,11cyclictest28-21ksoftirqd/108:36:011
3129583993124,4cyclictest3131065-21munin-plugin-st07:15:011
3129583993123,5cyclictest3227107-21aten_r2power_en10:50:471
3129583993123,5cyclictest3155057-21cron08:35:011
3129582993126,3cyclictest3152069-21cron08:25:000
3129582993126,2cyclictest0-21swapper/010:00:010
3129582993125,3cyclictest1001-21mta-sts-daemon11:27:240
312958299312,23cyclictest3276395-21ssh12:10:570
312958299312,23cyclictest3276395-21ssh12:10:570
312958599309,11cyclictest485-21in:imuxsock07:55:002
3129585993023,4cyclictest3197568-21mailstats10:01:052
3129585993023,4cyclictest3171691-21ssh09:20:012
3129585993022,5cyclictest2341350-21cups-browsed08:27:342
3129585993019,8cyclictest3136408-21ntp_kernel_err07:30:552
3129585993011,2cyclictest171rcu_preempt07:21:002
312958599300,11cyclictest3154513-21ntpq08:31:042
312958399303,11cyclictest28-21ksoftirqd/110:05:241
3129583993025,2cyclictest0-21swapper/109:05:011
3129583993020,6cyclictest3291741-21sshd12:36:031
3129582993023,5cyclictest2341350-21cups-browsed10:32:530
312958599299,6cyclictest3175248-21/usr/sbin/munin09:25:532
312958599299,6cyclictest3175248-21/usr/sbin/munin09:25:532
3129585992923,2cyclictest0-21swapper/211:36:012
3129585992922,3cyclictest0-21swapper/212:01:032
3129585992922,3cyclictest0-21swapper/212:01:032
3129585992921,5cyclictest0-21swapper/207:45:002
3129585992921,5cyclictest0-21swapper/207:45:002
3129585992916,4cyclictest3160405-21ntpq08:50:562
3129585992912,3cyclictest171rcu_preempt09:14:002
312958399299,5cyclictest28-21ksoftirqd/108:55:011
312958399298,7cyclictest28-21ksoftirqd/107:35:011
312958399296,7cyclictest28-21ksoftirqd/108:55:521
312958399296,5cyclictest28-21ksoftirqd/107:25:591
3129583992922,4cyclictest261rcuc/107:41:071
3129583992922,4cyclictest261rcuc/107:41:071
3129583992921,4cyclictest271ktimers/108:41:041
3129583992919,7cyclictest2341347-21cupsd10:25:531
3129583992919,2cyclictest171rcu_preempt08:26:011
3129582992919,7cyclictest3136464-21ntp_kernel_pll_07:30:580
312958599287,3cyclictest171rcu_preempt08:36:022
3129585992821,4cyclictest1000-21mta-sts-daemon08:40:472
3129585992821,3cyclictest0-21swapper/209:35:592
3129585992820,5cyclictest3162754-21date09:00:032
312958599281,21cyclictest3203342-21ssh10:10:552
3129583992820,5cyclictest271ktimers/112:21:001
3129582992823,2cyclictest0-21swapper/010:45:530
3129582992822,3cyclictest0-21swapper/007:40:010
3129582992821,5cyclictest443-21gpm07:46:010
3129582992821,5cyclictest443-21gpm07:46:010
3129582992817,8cyclictest0-21swapper/011:15:540
3129582992814,9cyclictest151ktimers/008:30:590
312958299281,22cyclictest3144110-21ps07:56:040
312958299281,19cyclictest3172185-21/usr/sbin/munin09:21:080
3129585992718,6cyclictest439-21systemd-logind07:15:012
3129585992716,3cyclictest171rcu_preempt10:15:592
3129585992716,3cyclictest171rcu_preempt10:15:592
3129582992722,3cyclictest151ktimers/009:05:590
3129582992721,3cyclictest0-21swapper/012:06:090
3129582992721,3cyclictest0-21swapper/012:06:090
3129582992717,7cyclictest0-21swapper/008:15:000
3129582992717,7cyclictest0-21swapper/008:15:000
3129582992716,9cyclictest3152603-21fschecks_time08:25:500
3129582992714,11cyclictest151ktimers/008:00:550
3129585992614,3cyclictest171rcu_preempt08:46:012
312958399269,3cyclictest171rcu_preempt08:45:581
3129583992617,6cyclictest3129535-21/usr/sbin/munin07:10:011
3129582992618,6cyclictest3141862-21aten_r2power_en07:50:460
3129582992617,5cyclictest151ktimers/007:21:020
3129585992519,3cyclictest0-21swapper/207:46:092
3129585992519,3cyclictest0-21swapper/207:46:092
312958599244,9cyclictest36-21ksoftirqd/207:26:032
312958299242,17cyclictest3163521-21sshd09:01:000
3129582992217,3cyclictest0-21swapper/007:25:420
3129585992110,3cyclictest171rcu_preempt07:10:022
312958699194,7cyclictest0-21swapper/309:21:013
312958699184,8cyclictest0-21swapper/311:31:003
312958699184,7cyclictest0-21swapper/310:51:003
312958699183,7cyclictest0-21swapper/309:46:003
312958699181,8cyclictest0-21swapper/310:55:573
312958299181,7cyclictest3129595-21systemd-run07:10:020
312958699173,8cyclictest0-21swapper/311:10:573
312958699173,8cyclictest0-21swapper/311:10:573
312958699173,8cyclictest0-21swapper/310:01:033
312958699173,10cyclictest0-21swapper/311:21:053
312958699172,8cyclictest0-21swapper/310:30:593
312958699164,7cyclictest0-21swapper/311:01:043
312958699163,8cyclictest0-21swapper/311:16:053
312958699163,7cyclictest0-21swapper/312:36:013
312958699163,7cyclictest0-21swapper/312:16:013
312958699163,6cyclictest0-21swapper/310:06:003
312958699163,6cyclictest0-21swapper/309:31:003
312958699163,6cyclictest0-21swapper/309:31:003
312958699162,7cyclictest0-21swapper/311:35:583
312958699161,8cyclictest0-21swapper/307:20:563
312958699153,8cyclictest0-21swapper/312:21:063
312958699153,7cyclictest0-21swapper/309:35:563
312958699153,7cyclictest0-21swapper/308:16:023
312958699153,7cyclictest0-21swapper/308:16:023
312958699153,6cyclictest0-21swapper/308:10:583
312958699153,6cyclictest0-21swapper/308:10:583
312958699152,9cyclictest0-21swapper/310:00:023
312958699152,8cyclictest0-21swapper/312:30:583
312958699152,8cyclictest0-21swapper/311:46:053
312958699152,8cyclictest0-21swapper/310:10:493
312958699152,7cyclictest0-21swapper/309:25:583
312958699152,7cyclictest0-21swapper/309:25:583
312958699152,6cyclictest0-21swapper/311:25:573
312958699151,8cyclictest0-21swapper/311:06:023
312958699151,7cyclictest0-21swapper/312:10:573
312958699151,7cyclictest0-21swapper/312:10:573
312958699151,7cyclictest0-21swapper/308:21:013
312958699151,7cyclictest0-21swapper/307:36:003
312958699143,7cyclictest0-21swapper/312:00:593
312958699143,7cyclictest0-21swapper/312:00:593
312958699143,6cyclictest0-21swapper/311:55:573
312958699143,5cyclictest0-21swapper/308:31:013
312958699142,8cyclictest0-21swapper/312:06:063
312958699142,8cyclictest0-21swapper/312:06:063
312958699142,7cyclictest0-21swapper/310:45:503
312958699142,7cyclictest0-21swapper/310:26:043
312958699142,7cyclictest0-21swapper/310:20:473
312958699142,7cyclictest0-21swapper/307:31:013
312958699141,9cyclictest0-21swapper/310:16:013
312958699141,9cyclictest0-21swapper/310:16:013
312958699141,8cyclictest0-21swapper/311:51:003
312958699141,8cyclictest0-21swapper/311:40:503
312958699141,8cyclictest0-21swapper/310:35:573
312958699141,8cyclictest0-21swapper/309:51:083
312958699141,7cyclictest0-21swapper/308:26:023
312958699141,6cyclictest0-21swapper/307:25:583
312958699141,6cyclictest0-21swapper/307:10:563
312958699135,6cyclictest0-21swapper/309:15:573
312958699135,5cyclictest0-21swapper/312:26:003
312958699133,6cyclictest0-21swapper/310:40:573
312958699133,6cyclictest0-21swapper/309:01:013
312958699133,6cyclictest0-21swapper/308:40:023
312958699132,7cyclictest0-21swapper/309:41:043
312958699132,6cyclictest0-21swapper/309:10:473
312958699132,6cyclictest0-21swapper/307:50:593
312958699131,6cyclictest0-21swapper/307:55:563
312958699131,6cyclictest0-21swapper/307:45:593
312958699131,6cyclictest0-21swapper/307:45:593
312958699131,6cyclictest0-21swapper/307:40:563
312958699131,6cyclictest0-21swapper/307:40:563
312958699131,6cyclictest0-21swapper/307:15:563
312958699123,5cyclictest0-21swapper/308:55:013
312958699122,6cyclictest0-21swapper/308:40:593
312958699121,6cyclictest0-21swapper/308:45:563
312958699112,6cyclictest0-21swapper/309:10:013
312958699111,6cyclictest0-21swapper/308:55:583
312958699111,6cyclictest0-21swapper/308:05:483
312958699101,6cyclictest0-21swapper/308:00:503
31295869972,4cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional