You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 19:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun May 10, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3567912994138,2cyclictest3849662-21kworker/u8:1+events_unbound11:50:003
3567907994139,1cyclictest3624682-21fwupd08:02:202
3567907994134,5cyclictest3891575-21gpgv12:20:002
3567907994134,5cyclictest3891575-21gpgv12:20:002
3567912993937,1cyclictest3744950-21kworker/u8:1+flush-259:010:40:253
3567912993835,2cyclictest3630107-21kworker/u8:3+events_unbound08:30:013
3567912993832,4cyclictest3776920-21cp10:30:013
3567907993834,3cyclictest3897019-21apt-get12:24:592
3567912993633,2cyclictest3508238-21kworker/u8:4+events_unbound07:45:223
3567912993631,4cyclictest3802453-21latency_hist10:55:003
3567907993633,2cyclictest3593838-21kworker/u8:0+events_unbound09:14:232
3567912993431,2cyclictest3875542-21kworker/u8:0+flush-259:012:15:163
3567912993431,2cyclictest3875542-21kworker/u8:0+flush-259:012:15:163
3567912993431,2cyclictest3744950-21kworker/u8:1+events_unbound10:10:143
3567912993427,2cyclictest3630107-21kworker/u8:3+events_unbound08:35:253
3567904993430,2cyclictest3744950-21kworker/u8:1+events_unbound10:40:191
3567901993430,3cyclictest3833893-21gpgv11:25:010
3567901993428,4cyclictest3625168-21/usr/sbin/munin08:05:150
3567912993330,2cyclictest3593838-21kworker/u8:0+events_unbound08:50:273
3567904993330,2cyclictest3573221-21kworker/u8:3+events_unbound07:15:251
3567904993329,3cyclictest3744950-21kworker/u8:1+flush-259:010:35:251
3567901993327,4cyclictest3708393-21apt-get09:24:590
3567901993326,5cyclictest3874653-21ntpq12:00:190
3567912993230,2cyclictest3671455-21kworker/u8:1+events_unbound09:10:003
3567912993229,2cyclictest3849662-21kworker/u8:1+events_unbound12:00:123
3567912993229,2cyclictest3792382-21kworker/u8:0+flush-259:011:40:233
3567912993229,2cyclictest3755292-21kworker/u8:3+events_unbound10:20:153
3567912993227,3cyclictest3823042-21latency_hist11:15:013
3567904993226,4cyclictest3908416-21apt-get12:35:111
3567901993229,2cyclictest3792382-21kworker/u8:0+flush-259:010:50:000
3567912993128,2cyclictest3562718-21kworker/u8:0+flush-259:007:24:023
3567904993128,2cyclictest3744950-21kworker/u8:1+flush-259:010:35:011
3567904993128,2cyclictest3562718-21kworker/u8:0+flush-259:007:10:201
356790199311,28cyclictest1-21systemd09:05:010
3567912993024,4cyclictest252-21systemd-journal08:48:243
3567904993025,3cyclictest3817663-21kworker/u8:3+flush-259:011:10:211
3567901993026,3cyclictest3792382-21kworker/u8:0+flush-259:011:05:170
3567901993024,4cyclictest3802477-21kworker/u8:2+flush-259:010:55:190
3567912992926,2cyclictest3583364-21kworker/u8:1+events_unbound07:40:213
3567912992925,3cyclictest3902611-21apt-get12:30:023
3567912992925,3cyclictest3817663-21kworker/u8:3+events_unbound11:15:243
3567912992925,3cyclictest3604355-21kworker/u8:2+events_unbound08:15:213
3567912992924,4cyclictest3885901-21gpgv12:15:003
3567901992925,3cyclictest3573256-21apt-get07:15:120
3567912992825,2cyclictest3849662-21kworker/u8:1+events_unbound11:54:573
3567912992825,2cyclictest3713972-21kworker/u8:2+events_unbound09:30:223
3567912992825,2cyclictest3508238-21kworker/u8:4+events_unbound07:52:253
3567907992821,5cyclictest3599202-21latency_hist07:40:012
3567907992821,5cyclictest3599202-21latency_hist07:40:002
3567904992824,3cyclictest3583364-21kworker/u8:1+flush-259:007:30:251
3567904992822,4cyclictest3729196-21kworker/u8:0+events_unbound09:50:121
3567901992823,4cyclictest3593838-21kworker/u8:0+flush-259:007:35:170
3567901992823,4cyclictest3593838-21kworker/u8:0+flush-259:007:35:170
3567901992822,4cyclictest3792149-21latency_hist10:45:010
3567912992724,2cyclictest3802477-21kworker/u8:2+events_unbound11:22:263
3567912992724,2cyclictest3671455-21kworker/u8:1+events_unbound09:13:273
3567912992722,2cyclictest3729196-21kworker/u8:0+events_unbound10:05:233
3567904992722,4cyclictest3661343-21apt-get08:40:111
3567901992720,5cyclictest3754399-21ntpq10:05:220
3567901992719,6cyclictest3724711-21diskstats09:40:140
3567912992624,2cyclictest3744950-21kworker/u8:1+events_unbound10:38:273
3567912992623,2cyclictest3744950-21kworker/u8:1+events_unbound10:49:273
3567912992623,2cyclictest3681679-21kworker/u8:3+events_unbound09:40:203
3567912992623,2cyclictest3593838-21kworker/u8:0+events_unbound08:55:153
3567912992621,2cyclictest3891678-21kworker/u8:2+events_unbound12:20:153
3567912992621,2cyclictest3792382-21kworker/u8:0+events_unbound10:59:283
3567904992623,2cyclictest3713972-21kworker/u8:2+flush-259:009:55:231
3567904992623,2cyclictest3562718-21kworker/u8:0+events_unbound07:20:231
3567904992622,3cyclictest3573221-21kworker/u8:3+events_unbound07:29:261
3567912992521,3cyclictest3671455-21kworker/u8:1+events_unbound09:04:273
3567907992520,4cyclictest252-21systemd-journal08:00:012
3567904992522,2cyclictest3844281-21kworker/u8:3+events_unbound11:34:531
3567904992522,2cyclictest3671455-21kworker/u8:1+events_unbound09:20:261
3567904992521,3cyclictest3792382-21kworker/u8:0+events_unbound11:00:201
3567904992521,3cyclictest3593838-21kworker/u8:0+flush-259:008:20:161
3567901992521,3cyclictest3713972-21kworker/u8:2+flush-259:009:35:000
3567901992520,4cyclictest3630324-21/usr/sbin/munin08:10:160
356790199250,23cyclictest1-21systemd10:33:260
3567912992421,2cyclictest3792382-21kworker/u8:0+events_unbound11:25:253
3567912992421,2cyclictest3681679-21kworker/u8:3+flush-259:009:25:213
3567912992420,3cyclictest3792382-21kworker/u8:0+events_unbound11:38:273
3567912992420,2cyclictest3593838-21kworker/u8:0+events_unbound08:30:173
3567904992421,2cyclictest3792382-21kworker/u8:0+events_unbound10:54:441
3567904992421,2cyclictest3593838-21kworker/u8:0+flush-259:008:30:241
3567904992420,3cyclictest3860040-21kworker/u8:0+events_unbound11:54:281
3567904992420,3cyclictest3849662-21kworker/u8:1+flush-259:011:44:271
3567904992420,3cyclictest3744950-21kworker/u8:1+events_unbound10:45:131
356790499241,14cyclictest3734603-21awk09:50:001
3567912992320,2cyclictest3729648-21kworker/u8:4+events_unbound10:04:343
3567912992319,3cyclictest3875542-21kworker/u8:0+events_unbound12:08:273
3567912992317,4cyclictest3724293-21latency_hist09:40:013
3567912992317,2cyclictest3713972-21kworker/u8:2+events_unbound09:45:213
3567907992319,3cyclictest3776288-21apt-get10:30:012
3567904992319,3cyclictest3844281-21kworker/u8:3+events_unbound11:35:191
3567904992319,3cyclictest3744950-21kworker/u8:1+events_unbound10:10:211
3567904992319,3cyclictest3671455-21kworker/u8:1+events_unbound09:05:141
3567904992319,2cyclictest3849662-21kworker/u8:1+events_unbound11:45:181
3567904992317,4cyclictest252-21systemd-journal11:20:261
3567901992320,2cyclictest3593838-21kworker/u8:0+events_unbound08:00:000
3567901992319,3cyclictest3583364-21kworker/u8:1+flush-259:007:30:170
3567901992318,4cyclictest3891678-21kworker/u8:2+flush-259:012:25:160
3567901992318,4cyclictest3771388-21apt-get10:25:130
3567912992219,2cyclictest3755292-21kworker/u8:3+events_unbound10:29:273
3567912992219,2cyclictest3729196-21kworker/u8:0+flush-259:009:55:273
3567912992219,2cyclictest3593838-21kworker/u8:0+events_unbound08:10:203
3567912992219,2cyclictest3593838-21kworker/u8:0+events_unbound08:00:153
3567912992219,2cyclictest3593838-21kworker/u8:0+events_unbound07:35:133
3567912992219,2cyclictest3593838-21kworker/u8:0+events_unbound07:35:123
3567912992217,4cyclictest3572991-21latency_hist07:15:003
3567904992218,3cyclictest3671455-21kworker/u8:1+events_unbound09:04:221
3567904992216,4cyclictest252-21systemd-journal09:45:001
3567901992218,3cyclictest3697798-21cp09:15:000
3567901992216,4cyclictest3886106-21gpgv12:15:000
3567912992117,3cyclictest3891678-21kworker/u8:2+flush-259:012:25:193
3567912992117,3cyclictest3583364-21kworker/u8:1+events_unbound07:34:043
3567907992119,2cyclictest3661104-21kworker/u8:2+events_unbound08:48:212
3567907992116,4cyclictest3834289-21cat11:25:012
3567904992118,2cyclictest3875542-21kworker/u8:0+events_unbound12:05:211
3567904992118,2cyclictest3744950-21kworker/u8:1+events_unbound10:27:271
3567904992116,3cyclictest3886536-21/usr/sbin/munin12:15:141
3567904992116,3cyclictest3886536-21/usr/sbin/munin12:15:141
3567904992116,3cyclictest3713972-21kworker/u8:2+flush-259:009:35:201
3567904992115,4cyclictest3702349-21ntpq09:15:221
3567904992114,5cyclictest3594211-21cat07:35:001
3567904992114,5cyclictest3594211-21cat07:35:001
3567901992115,4cyclictest3860012-21latency_hist11:50:000
3567901992115,4cyclictest3765589-21apt-get10:20:010
3567912992017,2cyclictest3849662-21kworker/u8:1+events_unbound11:55:213
3567912992017,2cyclictest3593838-21kworker/u8:0+events_unbound09:15:263
3567907992018,2cyclictest3604355-21kworker/u8:2+events_unbound08:14:562
3567904992018,1cyclictest3854593-21kworker/u8:4+events_unbound12:30:191
3567904992017,2cyclictest3891678-21kworker/u8:2+events_unbound12:29:271
3567904992014,4cyclictest3875533-21latency_hist12:05:011
3567901992018,1cyclictest3750132-21latency_hist10:05:000
3567901992015,3cyclictest3744920-21sh10:00:000
3567912991917,2cyclictest3671455-21kworker/u8:1+flush-259:009:20:243
3567907991916,2cyclictest3817663-21kworker/u8:3+flush-259:011:15:152
3567904991916,2cyclictest3755292-21kworker/u8:3+events_unbound10:20:131
3567904991916,2cyclictest3508238-21kworker/u8:4+flush-259:007:54:051
3567904991915,3cyclictest3630107-21kworker/u8:3+events_unbound08:15:151
3567904991915,3cyclictest3604355-21kworker/u8:2+events_unbound07:55:151
3567904991914,4cyclictest3670542-21ntpq08:45:201
3567904991913,5cyclictest3816973-21ntpq11:05:211
3567901991916,2cyclictest3744950-21kworker/u8:1+flush-259:010:10:250
3567901991914,4cyclictest3709009-21base6409:25:010
3567907991816,2cyclictest3744950-21kworker/u8:1+events_unbound10:42:002
3567907991816,2cyclictest3630107-21kworker/u8:3+events_unbound08:30:022
3567907991815,2cyclictest0-21swapper/212:30:002
3567904991815,2cyclictest3849662-21kworker/u8:1+events_unbound12:20:231
3567904991815,2cyclictest3817663-21kworker/u8:3+events_unbound11:16:251
3567904991815,2cyclictest3661104-21kworker/u8:2+events_unbound08:55:241
3567904991814,3cyclictest3865419-21apt-get11:55:111
3567904991814,3cyclictest3729196-21kworker/u8:0+events_unbound10:03:281
3567904991813,3cyclictest3508238-21kworker/u8:4+flush-259:007:45:171
3567901991816,1cyclictest3593838-21kworker/u8:0+events_unbound08:25:010
3567912991714,2cyclictest3792382-21kworker/u8:0+events_unbound11:30:353
3567912991714,2cyclictest3744950-21kworker/u8:1+events_unbound10:15:213
3567912991714,2cyclictest3729196-21kworker/u8:0+flush-259:009:50:183
3567912991714,2cyclictest3593838-21kworker/u8:0+events_unbound08:08:263
3567912991713,3cyclictest3802477-21kworker/u8:2+events_unbound11:09:203
356790799175,3cyclictest41-21ksoftirqd/209:00:002
356790799173,2cyclictest3786993-21latency_hist10:40:002
3567907991715,2cyclictest3849662-21kworker/u8:1+events_unbound12:00:072
3567907991714,2cyclictest3865836-21irqcore11:55:162
3567907991714,2cyclictest3593838-21kworker/u8:0+events_unbound09:17:342
3567907991713,2cyclictest503-21tpm2-abrmd09:44:592
3567907991712,4cyclictest3766255-21/usr/sbin/munin10:20:132
3567907991711,4cyclictest252-21systemd-journal09:28:252
356790799170,11cyclictest3645625-21munin-run08:25:002
3567904991714,2cyclictest3802477-21kworker/u8:2+flush-259:010:55:211
3567904991714,2cyclictest3713972-21kworker/u8:2+events_unbound09:30:161
3567904991714,2cyclictest3671455-21kworker/u8:1+flush-259:009:10:111
3567904991713,3cyclictest3593838-21kworker/u8:0+events_unbound08:54:171
3567904991710,2cyclictest3508238-21kworker/u8:4+flush-259:007:40:161
3567901991714,2cyclictest60350irq/132-enp0s29f1:rx-009:35:380
3567901991714,2cyclictest3573221-21kworker/u8:3+events_unbound07:30:000
3567901991713,3cyclictest60350irq/132-enp0s29f1:rx-012:20:200
3567901991711,4cyclictest3903300-21apt-get12:30:110
3567912991613,2cyclictest3573221-21kworker/u8:3+events_unbound07:29:103
3567907991615,1cyclictest3729196-21kworker/u8:0+events_unbound09:59:552
3567907991614,2cyclictest3671455-21kworker/u8:1+events_unbound09:04:252
3567907991614,2cyclictest3604355-21kworker/u8:2+events_unbound08:16:282
3567907991613,2cyclictest3881417-21irqcore12:10:142
3567907991613,2cyclictest3802477-21kworker/u8:2+events_unbound11:01:552
3567907991613,2cyclictest3671455-21kworker/u8:1+events_unbound09:07:192
3567907991613,2cyclictest3593838-21kworker/u8:0+events_unbound09:21:042
3567907991613,2cyclictest3593838-21kworker/u8:0+events_unbound08:07:222
3567907991611,1cyclictest191rcu_preempt08:42:402
3567907991610,4cyclictest3780988-21ntpq10:30:222
3567904991613,2cyclictest3744950-21kworker/u8:1+events_unbound10:15:171
3567904991612,3cyclictest3817663-21kworker/u8:3+events_unbound11:25:041
3567904991612,3cyclictest3593838-21kworker/u8:0+flush-259:008:29:131
3567904991612,3cyclictest3593838-21kworker/u8:0+events_unbound08:05:181
3567901991614,1cyclictest60350irq/132-enp0s29f1:rx-009:15:240
3567901991614,1cyclictest60350irq/132-enp0s29f1:rx-008:45:240
3567901991612,3cyclictest3875542-21kworker/u8:0+events_unbound12:15:200
3567901991612,3cyclictest3875542-21kworker/u8:0+events_unbound12:15:200
3567901991612,3cyclictest3604355-21kworker/u8:2+flush-259:008:15:250
3567901991612,3cyclictest3562718-21kworker/u8:0+events_unbound07:10:150
3567901991612,2cyclictest60350irq/132-enp0s29f1:rx-011:15:120
3567901991611,4cyclictest3620065-21apt-get08:00:110
3567901991610,5cyclictest3734830-21/usr/sbin/munin09:50:200
3567901991610,4cyclictest252-21systemd-journal11:31:270
3567912991512,2cyclictest3807976-21cpuspeed_turbos11:00:133
3567912991512,2cyclictest3562718-21kworker/u8:0+events_unbound07:17:423
356790799158,1cyclictest191rcu_preempt07:45:162
356790799156,4cyclictest0-21swapper/211:06:332
356790799156,1cyclictest191rcu_preempt12:40:002
356790799152,3cyclictest41-21ksoftirqd/210:10:212
356790799151,3cyclictest41-21ksoftirqd/208:35:232
3567907991513,2cyclictest3729648-21kworker/u8:4+events_unbound10:04:302
3567907991512,2cyclictest3854593-21kworker/u8:4+events_unbound11:46:402
3567907991512,2cyclictest3849662-21kworker/u8:1+events_unbound11:51:152
3567907991512,2cyclictest3817663-21kworker/u8:3+events_unbound11:13:172
3567907991512,2cyclictest3802477-21kworker/u8:2+events_unbound11:21:492
3567907991512,2cyclictest3713972-21kworker/u8:2+events_unbound09:45:032
3567907991512,2cyclictest3713972-21kworker/u8:2+events_unbound09:33:232
3567907991512,2cyclictest3583364-21kworker/u8:1+events_unbound07:40:032
3567907991512,2cyclictest3508238-21kworker/u8:4+events_unbound07:52:022
3567907991512,2cyclictest3508238-21kworker/u8:4+events_unbound07:31:372
3567907991511,3cyclictest3880297-21unixbench_singl12:05:242
3567907991511,2cyclictest3803741-21kthreadcore10:55:202
3567907991510,1cyclictest191rcu_preempt10:20:012
3567904991513,2cyclictest3630107-21kworker/u8:3+events_unbound08:35:161
3567904991512,2cyclictest3593838-21kworker/u8:0+flush-259:008:10:201
3567904991511,3cyclictest3729196-21kworker/u8:0+events_unbound10:07:271
356790199159,4cyclictest3687388-21cat09:05:010
3567901991513,2cyclictest60350irq/132-enp0s29f1:rx-011:28:220
3567901991513,1cyclictest60350irq/132-enp0s29f1:rx-011:55:130
3567901991513,1cyclictest60350irq/132-enp0s29f1:rx-011:40:160
3567901991513,1cyclictest60350irq/132-enp0s29f1:rx-010:50:090
3567901991513,1cyclictest60350irq/132-enp0s29f1:rx-008:55:180
3567901991513,1cyclictest0-21swapper/008:55:000
3567901991512,2cyclictest60350irq/132-enp0s29f1:rx-010:15:140
3567901991510,4cyclictest3661084-21latency_hist08:40:010
3567901991510,3cyclictest3802477-21kworker/u8:2+flush-259:011:00:170
3567912991412,2cyclictest3593838-21kworker/u8:0+events_unbound08:40:103
356790799148,4cyclictest3849662-21kworker/u8:1+flush-259:011:43:572
356790799145,2cyclictest41-21ksoftirqd/207:10:012
356790799144,6cyclictest41-21ksoftirqd/208:50:122
356790799141,7cyclictest41-21ksoftirqd/207:20:252
356790799141,3cyclictest3650758-21awk08:30:012
3567907991411,2cyclictest3844281-21kworker/u8:3+events_unbound11:36:362
3567907991411,2cyclictest3792382-21kworker/u8:0+events_unbound11:32:332
3567907991411,2cyclictest3792382-21kworker/u8:0+events_unbound10:54:532
3567907991411,2cyclictest3729196-21kworker/u8:0+events_unbound10:06:222
3567907991411,2cyclictest3573665-21irqcore07:15:162
3567907991411,1cyclictest191rcu_preempt09:54:342
3567907991411,1cyclictest0-21swapper/209:35:252
3567904991412,1cyclictest0-21swapper/112:14:591
356790199148,4cyclictest252-21systemd-journal09:45:260
3567901991412,1cyclictest60350irq/132-enp0s29f1:rx-011:36:470
3567901991412,1cyclictest60350irq/132-enp0s29f1:rx-007:50:230
3567901991411,2cyclictest60350irq/132-enp0s29f1:rx-008:26:250
3567901991411,2cyclictest60350irq/132-enp0s29f1:rx-007:45:210
3567901991411,2cyclictest3593838-21kworker/u8:0+events_unbound08:40:150
3567912991310,2cyclictest3604355-21kworker/u8:2+events_unbound07:55:063
356790799132,6cyclictest41-21ksoftirqd/212:30:192
3567907991310,2cyclictest3744950-21kworker/u8:1+events_unbound10:47:002
356790499139,3cyclictest3593838-21kworker/u8:0+flush-259:008:00:201
3567901991311,2cyclictest60350irq/132-enp0s29f1:rx-007:20:210
3567901991311,1cyclictest60350irq/132-enp0s29f1:rx-012:37:520
356791299129,2cyclictest3630107-21kworker/u8:3+events_unbound08:21:203
356791299122,1cyclictest191rcu_preempt12:35:233
356790199129,2cyclictest61150irq/133-enp0s29f1:tx-012:05:170
356790199129,2cyclictest3744950-21kworker/u8:1+events_unbound10:35:150
356790199119,1cyclictest60350irq/132-enp0s29f1:rx-011:10:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional