You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 00:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Aug 19, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3314101995349,3cyclictest3585518-21kworker/u8:0+flush-259:011:38:053
3314091995044,4cyclictest3606580-21apt-get11:50:090
3314091994932,3cyclictest335557-21systemd-journal07:30:240
3314101994844,3cyclictest3481733-21kworker/u8:2+flush-259:010:00:003
3314094994743,2cyclictest3637453-21kworker/u8:3+flush-259:012:40:001
3314094994337,4cyclictest3606320-21latency_hist11:50:011
3314091994239,2cyclictest3382112-21kworker/u8:1+flush-259:008:40:010
3314101994134,5cyclictest3632553-21latency_hist12:15:013
3314099994136,3cyclictest3340277-21latency_hist07:35:012
3314091994136,3cyclictest3533567-21base6410:40:010
3314094993933,4cyclictest3595757-21mandb11:38:071
3314094993933,4cyclictest3382758-21apt-get08:15:121
3314099993832,4cyclictest335557-21systemd-journal08:23:242
3314101993729,5cyclictest3497217-21latency_hist10:05:023
3314099993734,2cyclictest3528424-21kworker/u8:4+events_unbound11:15:182
3314099993733,3cyclictest3429690-21/usr/sbin/munin09:00:142
3314094993726,9cyclictest3376480-21ntpq08:05:221
3314099993634,1cyclictest3528198-21latency_hist10:35:012
3314101993531,3cyclictest3522890-21kworker/u8:2+flush-259:010:35:243
3314101993529,4cyclictest3548973-21kworker/u8:0+flush-259:011:00:183
3314099993531,3cyclictest3329539-21kworker/u8:3+flush-259:008:10:002
3314094993529,4cyclictest3319476-21apt-get07:15:111
3314091993531,3cyclictest3512980-21kworker/u8:3+flush-259:010:25:110
3314101993427,5cyclictest3621943-21base6412:05:013
3314099993429,3cyclictest3507575-21latency_hist10:15:012
3314094993428,4cyclictest509-21rs:main1
3314101993328,3cyclictest335557-21systemd-journal10:07:243
3314091993329,2cyclictest3308935-21kworker/u8:0+flush-259:007:20:170
3314101993228,3cyclictest3471636-21apt-get09:40:113
3314101993226,4cyclictest3512980-21kworker/u8:3+flush-259:010:40:163
3314101993224,3cyclictest3403177-21kworker/u8:0+flush-259:008:45:243
3314094993227,3cyclictest3611715-21apt-get11:55:121
3314099993125,4cyclictest3476798-21apt-get09:45:142
3314094993127,3cyclictest3329539-21kworker/u8:3+flush-259:007:30:011
3314094993126,4cyclictest3522890-21kworker/u8:2+flush-259:010:30:161
3314094993126,3cyclictest3518142-21/usr/sbin/munin10:25:231
3314091993125,4cyclictest3351448-21diskstats07:45:130
3314091993124,5cyclictest335557-21systemd-journal08:14:590
3314099993026,3cyclictest3559679-21kworker/u8:1+flush-259:012:10:242
3314094992924,4cyclictest335557-21systemd-journal10:40:001
3314091992924,4cyclictest3622411-21cp12:05:000
3314101992824,3cyclictest3528424-21kworker/u8:4+flush-259:011:10:173
3314101992823,3cyclictest3366560-21kworker/u8:0+events_unbound08:05:183
3314099992824,3cyclictest3533600-21store10:40:002
3314099992822,4cyclictest3314005-21latency_hist07:10:012
3314094992821,4cyclictest3590686-21latency_hist11:35:011
3314091992824,3cyclictest3429477-21kworker/u8:5+flush-259:009:40:000
3314091992823,4cyclictest3632819-21apt-get12:15:130
3314101992722,4cyclictest3429477-21kworker/u8:5+flush-259:009:35:173
3314094992722,3cyclictest3611680-21kworker/u8:3+flush-259:012:05:161
3314094992621,3cyclictest335557-21systemd-journal10:01:241
3314091992620,4cyclictest3492069-21latency_hist10:00:020
3314101992522,2cyclictest3308935-21kworker/u8:0+flush-259:007:40:253
3314101992521,3cyclictest3302478-21kworker/u8:1+events_unbound07:30:223
3314099992522,2cyclictest3429477-21kworker/u8:5+flush-259:009:45:002
3314099992521,3cyclictest3366560-21kworker/u8:0+flush-259:008:20:002
3314099992520,3cyclictest335557-21systemd-journal10:40:022
3314091992521,3cyclictest3465798-21apt-get09:35:010
3314091992521,3cyclictest3403177-21kworker/u8:0+flush-259:008:45:000
3314091992519,5cyclictest3318304-21ntpq07:10:220
3314091992519,4cyclictest3460573-21latency_hist09:30:010
3314101992419,4cyclictest3570259-21apt-get11:15:123
3314101992419,4cyclictest3366560-21kworker/u8:0+events_unbound08:00:183
3314099992421,2cyclictest3403177-21kworker/u8:0+flush-259:009:00:002
3314091992420,3cyclictest3481972-21apt-get09:50:090
3314091992420,3cyclictest3356462-21apt-get07:50:120
3314101992319,3cyclictest3585518-21kworker/u8:0+flush-259:011:40:163
3314101992318,3cyclictest335557-21systemd-journal08:13:133
3314099992316,5cyclictest3324378-21latency_hist07:20:022
3314094992319,3cyclictest3329539-21kworker/u8:3+flush-259:007:45:241
3314094992318,3cyclictest3398012-21apt-get08:30:001
3314101992219,2cyclictest3429476-21kworker/u8:3+flush-259:009:20:003
3314101992218,3cyclictest3528424-21kworker/u8:4+events_unbound10:50:223
3314101992218,3cyclictest3497264-21kworker/u8:0+events_unbound10:10:213
3314101992217,3cyclictest3440076-21apt-get09:10:123
3314101992214,5cyclictest3423422-21ntpq08:50:223
3314099992218,3cyclictest3559679-21kworker/u8:1+flush-259:011:55:512
3314094992218,3cyclictest3534059-21/usr/sbin/munin10:40:201
3314091992219,2cyclictest3429476-21kworker/u8:3+flush-259:009:05:010
3314091992219,2cyclictest3308935-21kworker/u8:0+events_unbound07:25:170
3314101992118,2cyclictest2971-21cron11:55:013
3314101992117,3cyclictest3528424-21kworker/u8:4+flush-259:011:05:203
3314101992116,3cyclictest3637453-21kworker/u8:3+flush-259:012:25:013
3314101992114,5cyclictest3580344-21latency_hist11:25:013
3314099992117,3cyclictest3559679-21kworker/u8:1+flush-259:011:50:192
3314099992117,3cyclictest3429477-21kworker/u8:5+flush-259:009:50:212
3314094992118,2cyclictest3528424-21kworker/u8:4+flush-259:010:50:211
3314101992016,3cyclictest3637453-21kworker/u8:3+flush-259:012:35:203
3314101992016,3cyclictest3627371-21kworker/u8:0+flush-259:012:15:273
3314101992016,3cyclictest3601159-21kworker/u8:2+events_unbound11:55:243
3314101992016,3cyclictest3429477-21kworker/u8:5+events_unbound09:30:223
3314101992016,2cyclictest3429444-21awk09:00:013
3314101992015,4cyclictest3512980-21kworker/u8:3+events_unbound10:25:123
3314101992015,4cyclictest3308935-21kworker/u8:0+events_unbound07:20:193
3314101992014,4cyclictest3387650-21latency_hist08:20:003
3314099992014,4cyclictest335557-21systemd-journal12:18:262
3314094992014,4cyclictest335557-21systemd-journal07:40:001
3314091992014,4cyclictest3371902-21cat08:05:000
3314101991916,2cyclictest3429477-21kworker/u8:5+events_unbound09:45:253
3314101991915,3cyclictest3559679-21kworker/u8:1+events_unbound11:25:223
3314101991914,3cyclictest3329539-21kworker/u8:3+events_unbound07:45:153
3314101991911,6cyclictest335557-21systemd-journal09:05:003
3314099991915,3cyclictest3559679-21kworker/u8:1+flush-259:011:40:202
3314094991914,4cyclictest3559926-21apt-get11:05:121
3314094991914,3cyclictest3554747-21/usr/sbin/munin11:00:141
3314094991914,3cyclictest3329539-21kworker/u8:3+flush-259:008:10:101
3314091991916,2cyclictest3382112-21kworker/u8:1+flush-259:008:15:190
3314091991915,3cyclictest3414183-21apt-get08:45:120
3314101991815,2cyclictest509-21rs:main3
3314101991815,2cyclictest3329539-21kworker/u8:3+events_unbound07:52:263
3314101991815,2cyclictest3319221-21latency_hist07:15:003
3314101991814,2cyclictest3382112-21kworker/u8:1+events_unbound08:20:213
3314101991811,5cyclictest335557-21systemd-journal10:45:253
3314099991816,1cyclictest0-21swapper/211:04:592
3314099991816,1cyclictest0-21swapper/207:54:592
3314099991813,4cyclictest3585764-21apt-get11:30:122
3314099991812,4cyclictest3648061-21latency_hist12:29:592
3314094991814,3cyclictest3497456-21/usr/sbin/munin10:05:201
3314094991814,3cyclictest3302478-21kworker/u8:1+flush-259:007:10:161
3314091991815,2cyclictest3345444-21idleruntime-cro07:40:000
3314091991814,3cyclictest3512980-21kworker/u8:3+flush-259:010:45:000
3314091991813,1cyclictest60350irq/132-enp0s29f1:rx-011:45:130
3314101991711,4cyclictest3471394-21kworker/u8:1+events_unbound10:15:193
3314099991714,2cyclictest3544141-21idleruntime-cro10:49:592
3314099991714,2cyclictest3429476-21kworker/u8:3+flush-259:009:20:262
3314099991713,3cyclictest3449284-21ntp_kernel_err09:15:212
3314094991713,3cyclictest3465823-21gpgv09:35:011
3314094991711,4cyclictest3428572-21ntpq08:55:221
3314091991714,2cyclictest3403177-21kworker/u8:0+flush-259:009:15:000
3314091991711,4cyclictest3579452-21ntpq11:20:200
331409199170,15cyclictest335557-21systemd-journal10:06:230
3314101991613,2cyclictest2971-21cron12:30:003
3314101991612,3cyclictest3429476-21kworker/u8:3+flush-259:009:20:213
3314101991611,3cyclictest335557-21systemd-journal07:58:103
3314101991611,3cyclictest0-21swapper/312:10:013
3314101991610,5cyclictest335557-21systemd-journal09:10:013
3314099991612,3cyclictest509-21rs:main2
3314099991611,4cyclictest3559679-21kworker/u8:1+flush-259:012:00:322
3314099991611,3cyclictest335557-21systemd-journal09:06:252
3314094991613,2cyclictest3360365-21needreboot07:50:191
3314094991612,3cyclictest3429477-21kworker/u8:5+flush-259:009:25:001
3314094991610,4cyclictest3398525-21cat08:30:001
3314094991610,4cyclictest3370845-21ntpq08:00:221
3314091991613,2cyclictest60350irq/132-enp0s29f1:rx-012:25:210
3314091991613,2cyclictest60350irq/132-enp0s29f1:rx-010:45:200
3314091991613,2cyclictest60350irq/132-enp0s29f1:rx-009:10:000
3314091991612,3cyclictest500-21systemd-logind07:59:590
3314091991612,3cyclictest3549620-21apt-get10:55:130
3314091991612,3cyclictest3372404-21apt-get08:05:120
3314091991612,3cyclictest335557-21systemd-journal11:30:240
3314101991512,2cyclictest3460554-21cron09:30:003
3314101991510,4cyclictest3308935-21kworker/u8:0+events_unbound07:38:073
3314099991512,2cyclictest3512740-21idleruntime-cro10:20:002
3314099991512,2cyclictest3403177-21kworker/u8:0+flush-259:008:50:002
3314099991510,4cyclictest335557-21systemd-journal11:08:252
331409499153,7cyclictest3329532-21munin-run07:25:001
3314094991512,1cyclictest0-21swapper/107:45:021
3314094991511,3cyclictest3627371-21kworker/u8:0+flush-259:012:15:181
3314094991511,3cyclictest3621782-21apt-get12:05:011
3314094991511,3cyclictest3512980-21kworker/u8:3+flush-259:010:45:171
3314094991510,4cyclictest335557-21systemd-journal12:28:261
3314091991513,2cyclictest60350irq/132-enp0s29f1:rx-010:18:220
3314091991513,1cyclictest60350irq/132-enp0s29f1:rx-010:10:120
3314091991512,2cyclictest3523502-21fschecks_count10:30:130
3314091991512,2cyclictest3349709-21ntp_kernel_pll_07:40:210
3314091991512,2cyclictest3329539-21kworker/u8:3+flush-259:008:25:010
3314091991510,3cyclictest509-21rs:main0
331409199150,14cyclictest3575186-21turbostat11:20:000
331410199149,4cyclictest3548973-21kworker/u8:0+events_unbound10:58:253
331410199148,4cyclictest3559679-21kworker/u8:1+flush-259:011:45:103
3314101991410,3cyclictest3559679-21kworker/u8:1+events_unbound11:34:253
3314101991410,3cyclictest3403177-21kworker/u8:0+flush-259:008:43:363
3314101991410,3cyclictest3302478-21kworker/u8:1+events_unbound07:15:253
3314099991410,3cyclictest3585518-21kworker/u8:0+flush-259:011:45:232
3314099991410,3cyclictest3466012-21gpgv09:35:012
331409499149,4cyclictest3403177-21kworker/u8:0+flush-259:008:54:551
331409499148,1cyclictest0-21swapper/109:50:011
3314094991411,2cyclictest3408301-21unixbench_singl08:35:251
3314094991411,2cyclictest0-21swapper/109:55:001
3314094991410,3cyclictest3580611-21apt-get11:25:111
331409199148,4cyclictest3323544-21ntpq07:15:220
3314091991413,1cyclictest60350irq/132-enp0s29f1:rx-011:55:160
3314091991413,1cyclictest60350irq/132-enp0s29f1:rx-011:13:050
3314091991413,1cyclictest60350irq/132-enp0s29f1:rx-010:00:170
3314091991412,2cyclictest61150irq/133-enp0s29f1:tx-012:00:190
3314091991412,2cyclictest60350irq/132-enp0s29f1:rx-008:58:410
3314091991412,1cyclictest60350irq/132-enp0s29f1:rx-011:35:270
3314091991412,1cyclictest60350irq/132-enp0s29f1:rx-011:00:240
3314091991412,1cyclictest60350irq/132-enp0s29f1:rx-010:50:330
3314091991412,1cyclictest60350irq/132-enp0s29f1:rx-009:15:210
3314091991411,2cyclictest61150irq/133-enp0s29f1:tx-011:26:230
3314091991411,2cyclictest60350irq/132-enp0s29f1:rx-009:23:240
3314091991411,2cyclictest60350irq/132-enp0s29f1:rx-008:30:000
3314091991410,3cyclictest3528424-21kworker/u8:4+flush-259:011:05:250
331410199139,3cyclictest3403177-21kworker/u8:0+flush-259:008:33:243
331410199136,5cyclictest3397147-21ntpq08:25:243
331409999139,3cyclictest3403177-21kworker/u8:0+flush-259:008:35:052
331409999139,3cyclictest3302478-21kworker/u8:1+flush-259:007:21:292
331409999137,4cyclictest335557-21systemd-journal07:25:242
3314099991310,2cyclictest3457969-21kthreadcore09:25:182
331409499139,3cyclictest3308935-21kworker/u8:0+flush-259:007:31:291
331409499139,2cyclictest1-21systemd10:13:241
331409499138,4cyclictest3548973-21kworker/u8:0+flush-259:010:55:111
331409499138,4cyclictest3403177-21kworker/u8:0+flush-259:009:10:181
3314094991310,2cyclictest0-21swapper/109:05:191
3314091991312,1cyclictest60350irq/132-enp0s29f1:rx-009:40:060
3314091991312,1cyclictest60350irq/132-enp0s29f1:rx-008:50:240
3314091991310,2cyclictest60350irq/132-enp0s29f1:rx-012:35:110
3314091991310,2cyclictest60350irq/132-enp0s29f1:rx-011:40:170
3314091991310,2cyclictest60350irq/132-enp0s29f1:rx-008:25:120
331410199128,3cyclictest3308935-21kworker/u8:0+events_unbound07:25:233
331409999129,2cyclictest3626953-21unixbench_singl12:05:252
331409999129,2cyclictest3580750-21cut11:25:142
331409999129,2cyclictest3518235-21cpuspeed_turbos10:25:122
331409999129,2cyclictest3491585-21unixbench_multi09:55:262
331409999129,2cyclictest3395391-21kthreadcore08:25:192
331409999129,2cyclictest3387676-21kworker/u8:2+flush-259:008:30:152
331409999128,3cyclictest3377776-21fschecks_count08:10:142
331409999127,4cyclictest3595735-21dpkg11:38:062
331409999121,3cyclictest3653183-21awk12:35:002
3314099991210,1cyclictest3654939-21kthreadcore12:35:182
331409499129,2cyclictest3409100-21cpuspeed_turbos08:40:131
3314094991210,1cyclictest0-21swapper/112:35:001
3314094991210,1cyclictest0-21swapper/109:15:171
331409199129,2cyclictest3477056-21fschecks_time09:45:170
331409199128,2cyclictest3627642-21apt-get12:10:120
331410199118,2cyclictest3523371-21cpuspeed_turbos10:30:133
331409999119,1cyclictest0-21swapper/208:55:012
331409999117,3cyclictest3575614-21cut11:20:132
331409999117,3cyclictest3554533-21pluginstate11:00:012
331409999117,3cyclictest3497264-21kworker/u8:0+flush-259:010:20:162
331409999117,2cyclictest0-21swapper/207:35:122
331409999115,5cyclictest335557-21systemd-journal10:09:592
331409999115,4cyclictest3409001-21apt-get08:40:132
331409499119,1cyclictest0-21swapper/109:35:231
331409199118,2cyclictest0-21swapper/007:10:010
331409499107,2cyclictest0-21swapper/112:10:221
33140949997,1cyclictest0-21swapper/111:10:241
33140949996,2cyclictest311ktimers/109:25:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional