You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 21:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Thu Jul 30, 2026 12:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
346063899661,63cyclictest1-21systemd09:44:520
35559542580,2chrt451irq_work/308:43:043
3460638995852,4cyclictest3445872-21kworker/u8:3+events_unbound09:05:000
35657172570,3chrt0-21swapper/208:53:062
3460638995546,6cyclictest3482335-21latency_hist07:30:010
3460638995447,5cyclictest3626169-21latency_hist09:55:010
37045122520,2sleep20-21swapper/211:13:042
3460638994638,6cyclictest3591290-21latency_hist09:20:000
3460641994536,6cyclictest3636037-21latency_hist10:05:021
3460641994438,4cyclictest3685616-21kworker/u8:3+events_unbound11:38:111
3460643994238,3cyclictest3537025-21kworker/u8:1+events_unbound09:08:072
3460643994137,3cyclictest3685616-21kworker/u8:3+events_unbound11:48:112
3460643994137,3cyclictest3660786-21kworker/u8:2+events_unbound10:37:592
3460641993933,4cyclictest3750483-21apt-get12:02:531
3460641993833,4cyclictest3537025-21kworker/u8:1+events_unbound09:41:281
3460641993833,4cyclictest3218001-21kworker/u8:2+flush-8:009:03:111
3460641993829,7cyclictest3621253-21kworker/u8:2+flush-8:010:13:061
3460641993825,9cyclictest275-21systemd-journal09:53:001
3460638993831,3cyclictest3705495-21sh11:15:000
346063899381,31cyclictest3460524-21kworker/0:1+events08:45:020
3460641993729,6cyclictest3537025-21kworker/u8:1+flush-8:008:37:591
3460641993728,5cyclictest3759879-21apt-get12:10:001
3460638993732,4cyclictest3218001-21kworker/u8:2+events_unbound07:48:020
3460638993726,8cyclictest275-21systemd-journal10:00:000
3460641993628,6cyclictest3472914-21apt-get07:22:531
346063899362,29cyclictest3460524-21kworker/0:1+events07:56:240
36557912350,6sleep03655796-21unixbench_multi10:23:110
3460643993531,3cyclictest3537025-21kworker/u8:1+events_unbound09:13:052
3460643993524,10cyclictest3745063-21idleruntime-cro11:55:002
3460641993531,3cyclictest3537025-21kworker/u8:1+events_unbound08:42:571
3460641993525,7cyclictest3651518-21/usr/sbin/munin10:23:031
346063899352,28cyclictest3725320-21kworker/0:0+events12:22:020
3460641993430,2cyclictest3690478-21unixbench_multi10:58:151
3460641993424,8cyclictest3585685-21ntpq09:13:071
3460638993427,5cyclictest3562146-21/usr/sbin/munin08:53:130
3460638993427,5cyclictest3537025-21kworker/u8:1+flush-8:009:07:560
346063899342,27cyclictest3636070-21kworker/0:1+events10:37:040
346063899342,26cyclictest3460524-21kworker/0:1+events08:06:380
346063899341,28cyclictest3460524-21kworker/0:1+events07:36:020
34559752343,6sleep10-21swapper/107:06:461
3460641993327,4cyclictest3537025-21kworker/u8:1+events_unbound10:00:011
3460641993326,4cyclictest3561697-21latency_hist08:50:011
3460638993329,3cyclictest275-21systemd-journal09:22:520
3460638993325,6cyclictest3720764-21/usr/sbin/munin11:33:100
3460638993325,6cyclictest3720764-21/usr/sbin/munin11:33:100
3460638993324,7cyclictest3780363-21/usr/sbin/munin12:33:030
3460638993324,7cyclictest3780363-21/usr/sbin/munin12:33:020
346063899332,30cyclictest3460524-21kworker/0:1+events07:16:080
3460638993322,9cyclictest3526883-21apt-get08:15:000
346063899332,26cyclictest3725320-21kworker/0:0+events12:01:300
346063899332,26cyclictest3636070-21kworker/0:1+events11:00:480
3460643993227,4cyclictest3537025-21kworker/u8:1+events_unbound09:00:022
3460641993227,4cyclictest3218001-21kworker/u8:2+flush-8:007:27:581
3460641993227,3cyclictest3564331-21kthreadcore08:53:051
3460641993224,6cyclictest3487738-21/usr/sbin/munin07:38:071
3460641993223,7cyclictest3705503-21latency_hist11:15:011
3460638993225,4cyclictest0-21swapper/007:32:530
346063899322,26cyclictest3636070-21kworker/0:1+events10:55:180
346063899322,26cyclictest3460524-21kworker/0:1+events07:40:520
346063899322,25cyclictest3460524-21kworker/0:1+events08:16:360
346063899320,27cyclictest3725320-21kworker/0:0+events12:15:140
346063899320,27cyclictest3460524-21kworker/0:1+events08:22:040
34558592324,9sleep20-21swapper/207:05:152
34660382310,2sleep3575-21kworker/3:2+events07:15:033
3460643993127,3cyclictest3665950-21kworker/u8:1+events_unbound10:53:122
3460641993127,2cyclictest3696437-21cut11:07:561
3460641993123,6cyclictest3676631-21cut10:47:551
3460638993125,4cyclictest3635831-21unixbench_singl10:03:130
346063899312,28cyclictest3460524-21kworker/0:1+events08:44:330
346063899312,25cyclictest3626189-21kworker/0:2+events10:07:400
346063899312,24cyclictest3596283-21kworker/0:3+events09:30:020
346063899312,24cyclictest3460524-21kworker/0:1+events07:11:200
346063899311,24cyclictest3636070-21kworker/0:1+events10:50:200
346063899310,26cyclictest3636070-21kworker/0:1+events11:35:550
3460643993026,3cyclictest3754763-21kworker/u8:1+events_unbound12:13:162
3460643993024,4cyclictest3680582-21unixbench_multi10:48:112
3460641993020,7cyclictest3532055-21latency_hist08:20:001
346063899302,27cyclictest3460524-21kworker/0:1+events07:21:180
346063899302,23cyclictest3725320-21kworker/0:0+events11:46:220
346063899302,23cyclictest3636070-21kworker/0:1+events11:07:120
346063899302,23cyclictest3636070-21kworker/0:1+events10:10:400
346063899301,28cyclictest3725320-21kworker/0:0+events12:06:020
346063899301,24cyclictest3636070-21kworker/0:1+events10:40:320
346063899300,29cyclictest3636070-21kworker/0:1+events10:16:460
34607142290,3sleep3575-21kworker/3:2+events07:10:003
346064399294,9cyclictest3724730-21cut11:33:102
346064399294,9cyclictest3724730-21cut11:33:092
3460643992925,3cyclictest3497666-21kworker/u8:0+events_unbound08:18:012
3460643992925,3cyclictest3491311-21kworker/u8:1+events_unbound07:41:022
3460641992924,4cyclictest3497666-21kworker/u8:0+flush-8:007:48:061
3460641992922,5cyclictest3651307-21kworker/u8:0+events_unbound11:58:111
3460641992921,4cyclictest0-21swapper/107:15:021
346063899297,17cyclictest3460524-21kworker/0:1+events08:00:220
346063899293,25cyclictest3636070-21kworker/0:1+events11:22:100
346063899292,26cyclictest3725320-21kworker/0:0+events11:56:160
346063899292,26cyclictest3725320-21kworker/0:0+events11:41:160
346063899292,26cyclictest3636070-21kworker/0:1+events11:29:090
346063899292,26cyclictest3606103-21kworker/0:0+events09:48:300
346063899290,28cyclictest3460524-21kworker/0:1+events08:31:320
3460643992825,2cyclictest3536227-21grep08:23:042
3460643992824,3cyclictest3651307-21kworker/u8:0+events_unbound11:28:102
3460643992820,2cyclictest161rcu_preempt11:58:012
3460643992820,2cyclictest161rcu_preempt10:07:582
3460641992824,2cyclictest0-21swapper/109:29:591
3460641992823,4cyclictest3497666-21kworker/u8:0+flush-8:008:07:581
3460641992823,3cyclictest3567161-21aten2_r5power_e08:57:561
3460641992820,6cyclictest3740131-21latency_hist11:50:011
346063899282,25cyclictest3725320-21kworker/0:0+events12:13:000
346063899282,25cyclictest3460524-21kworker/0:1+events09:13:240
346063899282,25cyclictest3460524-21kworker/0:1+events08:55:420
346063899282,25cyclictest3460524-21kworker/0:1+events07:53:140
346063899281,26cyclictest3460524-21kworker/0:1+events08:28:380
37018012270,3chrt575-21kworker/3:2+events11:13:013
346064399278,10cyclictest0-21swapper/212:27:522
3460643992724,2cyclictest3690459-21unixbench_multi10:58:152
3460643992723,3cyclictest3725302-21kworker/u8:2+events_unbound12:15:052
3460643992723,3cyclictest3218001-21kworker/u8:2+events_unbound08:13:132
3460643992723,2cyclictest3789598-21unixbench_singl12:38:142
3460643992723,2cyclictest3542471-21aten2_r5power_e08:32:542
3460643992719,2cyclictest161rcu_preempt09:43:012
346064399271,13cyclictest39-21ksoftirqd/209:53:032
346064199278,3cyclictest35956592open_inodes09:23:091
3460641992724,2cyclictest600-21sshd09:47:541
3460641992723,3cyclictest3536830-21unixbench_singl08:23:121
3460641992722,3cyclictest3711241-21cut11:22:571
3460641992720,5cyclictest3770896-21cut12:22:571
346063899272,21cyclictest3725320-21kworker/0:0+events12:35:200
346063899272,21cyclictest3596283-21kworker/0:3+events09:25:260
3460638992718,6cyclictest3611240-21latency_hist09:40:010
346063899270,26cyclictest3725320-21kworker/0:0+events11:50:180
3460643992622,2cyclictest3606919-21cpuspeed_turbos09:37:582
3460643992621,4cyclictest3506909-21unixbench_multi07:53:122
3460643992621,3cyclictest3592038-21fschecks_count09:22:572
3460643992618,2cyclictest161rcu_preempt11:07:592
3460641992622,3cyclictest3651307-21kworker/u8:0+events_unbound11:02:571
3460641992622,2cyclictest0-21swapper/109:08:081
3460641992621,4cyclictest3482355-21kworker/u8:0+events_unbound07:33:021
3460641992620,4cyclictest3730726-21apt-get11:42:511
3460641992619,4cyclictest3725302-21kworker/u8:2+flush-8:012:18:031
346063899263,22cyclictest3725320-21kworker/0:0+events12:28:120
346063899262,23cyclictest3636070-21kworker/0:1+events11:18:380
346063899262,23cyclictest3636070-21kworker/0:1+events10:32:540
346063899262,21cyclictest3460524-21kworker/0:1+events08:35:140
346063899260,25cyclictest3636070-21kworker/0:1+events10:28:580
346063899260,25cyclictest3636070-21kworker/0:1+events10:28:580
346064399252,7cyclictest36494652kthreadcore10:18:042
3460643992522,2cyclictest3571350-21unixbench_singl08:58:142
3460643992521,3cyclictest2650250-21kworker/u8:1+events_unbound07:23:152
3460643992521,3cyclictest2650250-21kworker/u8:1+events_unbound07:13:252
3460643992521,2cyclictest3488030-21fschecks_count07:37:572
3460643992519,4cyclictest3556561-21unixbench_singl08:43:132
346064399251,7cyclictest39-21ksoftirqd/207:20:002
346064399250,14cyclictest3512826-21runrttasks08:02:572
346064399250,12cyclictest3616747-21apt-get09:47:552
3460641992521,3cyclictest3497666-21kworker/u8:0+flush-8:007:53:001
3460641992520,2cyclictest0-21swapper/110:53:121
3460641992519,5cyclictest3725302-21kworker/u8:2+events_unbound12:37:561
3460641992519,4cyclictest3660786-21kworker/u8:2+flush-8:010:42:581
3460641992519,4cyclictest3537025-21kworker/u8:1+events_unbound09:18:121
3460641992518,5cyclictest3660936-21cron10:30:001
3460641992518,5cyclictest3660936-21cron10:30:001
3460641992515,5cyclictest0-21swapper/110:17:551
346063899250,24cyclictest3636070-21kworker/0:1+events10:45:040
346064399244,6cyclictest39-21ksoftirqd/212:10:012
346064399244,6cyclictest0-21swapper/211:15:582
346064399242,10cyclictest0-21swapper/211:23:012
3460643992411,1cyclictest161rcu_preempt07:48:282
3460643992410,2cyclictest161rcu_preempt12:23:382
3460641992420,3cyclictest3665745-21unixbench_singl10:33:111
3460641992420,2cyclictest0-21swapper/112:13:001
3460641992419,4cyclictest3537025-21kworker/u8:1+events_unbound09:37:571
3460641992419,3cyclictest1-21systemd08:30:011
3460641992419,2cyclictest0-21swapper/111:32:551
3460641992419,2cyclictest0-21swapper/111:32:551
3460641992418,5cyclictest3606020-21unixbench_singl09:33:131
3460641992418,3cyclictest0-21swapper/111:28:111
3460641992416,6cyclictest3512577-21apt-get08:02:531
3460641992416,4cyclictest0-21swapper/108:13:001
346064399239,2cyclictest161rcu_preempt09:35:012
3460643992320,2cyclictest3631855-21fschecks_count10:02:582
3460643992319,3cyclictest371rcuc/210:35:002
3460643992319,2cyclictest3557322-21aten2_r5power_p08:47:562
346064399231,11cyclictest3541248-21ntpq08:28:072
3460643992310,2cyclictest161rcu_preempt10:44:122
3460641992318,3cyclictest3754763-21kworker/u8:1-ext4-rsv-conversion12:35:001
3460641992318,3cyclictest3754763-21kworker/u8:1-ext4-rsv-conversion12:35:001
3460641992317,4cyclictest3651307-21kworker/u8:0+flush-8:011:53:051
3460641992315,6cyclictest3512087-21latency_hist08:00:011
3460641992315,5cyclictest3537025-21kworker/u8:1+flush-8:008:32:571
35857592220,3sleep3451irq_work/309:13:093
346064399229,2cyclictest161rcu_preempt10:20:252
346064399228,3cyclictest35203992kthreadcore08:08:032
346064399224,5cyclictest0-21swapper/209:15:162
3460643992217,4cyclictest20823-21cron12:35:002
3460643992217,4cyclictest20823-21cron12:35:002
346064399220,8cyclictest3481507-21sed07:28:042
346064399220,7cyclictest414-21systemd-logind11:40:002
346064199228,5cyclictest0-21swapper/111:16:301
3460641992215,4cyclictest0-21swapper/112:27:551
3460641992214,5cyclictest0-21swapper/110:07:541
34560112222,3sleep00-21swapper/007:07:130
346064399219,1cyclictest161rcu_preempt09:29:582
346064399216,3cyclictest36610212grep10:30:002
346064399216,3cyclictest36610212grep10:30:002
346064399215,7cyclictest0-21swapper/211:01:192
346064399215,5cyclictest0-21swapper/207:59:302
346064399210,8cyclictest3486468-21missed_timers07:33:042
346064399210,6cyclictest3645876-21cron10:15:002
346064399210,5cyclictest3754928-21idleruntime-cro12:05:002
3460641992115,4cyclictest3461897-21irqrtprio07:13:011
3460641992114,4cyclictest0-21swapper/107:42:581
346064399209,2cyclictest161rcu_preempt09:56:182
346064399204,6cyclictest0-21swapper/211:40:262
346064399200,10cyclictest3551809-21sendmail08:40:002
3460641992014,4cyclictest0-21swapper/110:38:041
36610182190,2sleep3471rcuc/310:30:003
36610182190,2sleep3471rcuc/310:30:003
35193072190,2sleep3451irq_work/308:08:023
34944612143,7sleep30-21swapper/307:43:033
37549862132,6sleep30-21swapper/312:05:023
37499822131,8sleep30-21swapper/312:00:003
36750272132,7sleep30-21swapper/310:43:053
36361772132,6sleep30-21swapper/310:05:023
36212262132,6sleep30-21swapper/309:50:023
35351642130,2sleep3481ktimers/308:23:023
35081452132,7sleep30-21swapper/307:58:003
34774252132,6sleep30-21swapper/307:25:023
34728872132,7sleep30-21swapper/307:22:533
37892652122,6sleep30-21swapper/312:38:123
37803622122,6sleep30-21swapper/312:32:503
37803622122,6sleep30-21swapper/312:32:503
37689872122,6sleep30-21swapper/312:18:063
37650602122,5sleep30-21swapper/312:15:013
37447632122,6sleep30-21swapper/311:53:103
37355992123,5sleep30-21swapper/311:47:243
37312442122,6sleep30-21swapper/311:42:593
37259942123,5sleep30-21swapper/311:37:583
37249852122,6sleep30-21swapper/311:33:123
37249852122,6sleep30-21swapper/311:33:123
37064102122,6sleep30-21swapper/311:18:003
36920752122,6sleep30-21swapper/311:02:593
36860332122,6sleep30-21swapper/310:55:533
36812592122,6sleep30-21swapper/310:52:523
36458742122,5sleep30-21swapper/310:15:013
36419342122,6sleep30-21swapper/310:13:003
36266412122,6sleep30-21swapper/309:57:563
36053672122,6sleep30-21swapper/309:33:043
35926352122,6sleep30-21swapper/309:23:023
35870972122,6sleep30-21swapper/309:18:003
35808892122,6sleep30-21swapper/309:08:103
35729602122,6sleep30-21swapper/309:03:033
35675922122,6sleep30-21swapper/308:58:023
35660072122,6sleep30-21swapper/308:53:123
35568282122,6sleep30-21swapper/308:45:023
35510772122,6sleep30-21swapper/308:38:083
35428812123,5sleep30-21swapper/308:32:583
35419282122,5sleep30-21swapper/308:30:013
35275622122,6sleep30-21swapper/308:17:543
35165692122,6sleep30-21swapper/308:03:133
35022652122,5sleep30-21swapper/307:50:013
34979732121,7sleep30-21swapper/307:47:583
34920832122,6sleep30-21swapper/307:38:143
34872802122,6sleep30-21swapper/307:35:023
34613762122,6sleep30-21swapper/307:12:583
37754222112,6sleep30-21swapper/312:27:283
37710002112,5sleep30-21swapper/312:22:593
37504552112,5sleep30-21swapper/312:02:533
37156342112,5sleep30-21swapper/311:25:023
37150752112,5sleep30-21swapper/311:23:133
36960102112,6sleep30-21swapper/311:06:443
36759272112,5sleep30-21swapper/310:45:023
36664052112,5sleep30-21swapper/310:37:563
36610182112,5sleep30-21swapper/310:30:013
36313872112,6sleep30-21swapper/310:00:363
36203452112,5sleep30-21swapper/309:48:053
36121132112,5sleep30-21swapper/309:42:593
36105492112,5sleep30-21swapper/309:38:113
35970102112,5sleep30-21swapper/309:27:593
35260072112,6sleep30-21swapper/308:13:043
36513082102,5sleep30-21swapper/310:20:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional