You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 12:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Jul 26, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3148813995246,4cyclictest3248383-21gpgv20:45:001
3148819995146,3cyclictest3159082-21latency_hist19:20:013
3148809994943,4cyclictest3196288-21cat19:55:000
3148815994337,4cyclictest335557-21systemd-journal22:55:012
3148809994134,5cyclictest3457224-21cp00:04:590
3148815994036,3cyclictest3322292-21kworker/u8:4+flush-259:023:20:012
3148809994036,3cyclictest3159083-21apt-get19:20:000
3148815993934,3cyclictest3269443-21kworker/u8:2+flush-259:021:15:202
3148813993833,4cyclictest3269729-21apt-get21:05:001
3148813993729,5cyclictest335557-21systemd-journal19:34:281
3148813993631,4cyclictest335557-21systemd-journal20:37:041
3148813993528,5cyclictest3337732-21latency_hist22:10:011
3148809993529,4cyclictest335557-21systemd-journal21:59:590
3148815993429,4cyclictest3343573-21/usr/sbin/munin22:15:152
3148815993329,3cyclictest3179196-21kworker/u8:2+flush-259:019:50:162
3148819993229,2cyclictest3399688-21kworker/u8:0+events_unbound23:30:143
3148809993226,4cyclictest3148616-21kworker/u8:1+flush-259:020:15:000
3148809993225,5cyclictest3358813-21latency_hist22:30:000
3148815993125,5cyclictest3322292-21kworker/u8:4+flush-259:021:55:242
3148813993125,4cyclictest3196811-21apt-get19:55:141
3148809993124,5cyclictest3274421-21ntpq21:05:210
3148809993123,5cyclictest3222402-21latency_hist20:20:000
3148819993026,3cyclictest3105060-21kworker/u8:3+flush-259:019:39:593
3148815993026,3cyclictest3232517-21kworker/u8:3+flush-259:020:50:192
3148809993027,2cyclictest3322292-21kworker/u8:4+flush-259:022:30:210
3148819992924,3cyclictest3472997-21gpgv00:20:003
3148813992923,4cyclictest3440643-21ntpq23:45:231
3148813992922,6cyclictest3285693-21gpgv21:20:001
3148809992923,4cyclictest3336842-21ntpq22:05:240
3148809992923,4cyclictest3153903-21latency_hist19:15:020
3148819992822,4cyclictest3441500-21latency_hist23:50:013
3148815992822,4cyclictest3248268-21apt-get20:45:002
3148809992824,3cyclictest3364236-21apt-get22:35:130
3148809992823,4cyclictest3399688-21kworker/u8:0+flush-259:000:05:200
3148809992823,3cyclictest3196057-21kworker/u8:0+flush-259:021:25:010
3148813992722,3cyclictest3222671-21apt-get20:20:131
3148813992722,3cyclictest3222671-21apt-get20:20:121
3148819992623,2cyclictest3478497-21kworker/u8:3+flush-259:000:25:163
3148819992623,2cyclictest3322292-21kworker/u8:4+flush-259:022:40:003
3148819992622,3cyclictest3201766-21kworker/u8:3+flush-259:020:07:293
3148819992620,4cyclictest509-21rs:main3
3148815992619,5cyclictest3290411-21ntpq21:20:222
3148813992623,2cyclictest3148616-21kworker/u8:1+flush-259:019:49:591
3148813992622,3cyclictest3343360-21kworker/u8:2+flush-259:022:35:001
3148813992619,4cyclictest3206901-21latency_hist20:05:001
3148809992620,4cyclictest335557-21systemd-journal00:20:020
3148809992619,5cyclictest3414818-21ntpq23:20:220
3148819992521,3cyclictest3148616-21kworker/u8:1+events_unbound19:30:023
3148819992520,4cyclictest3337733-21apt-get22:09:593
3148819992519,4cyclictest509-21rs:main3
3148819992518,5cyclictest3264362-21cp21:00:003
3148815992520,4cyclictest3138154-21kworker/u8:0+flush-259:019:15:002
3148815992519,4cyclictest3472969-21latency_hist00:20:002
3148809992521,3cyclictest3322231-21idleruntime-cro21:55:000
3148809992520,3cyclictest3338296-21gpgv22:10:000
3148819992421,2cyclictest3300659-21kworker/u8:1+events_unbound21:40:163
3148819992420,3cyclictest3285412-21kworker/u8:1+flush-259:021:20:253
3148815992420,3cyclictest3148616-21kworker/u8:1+flush-259:020:15:282
3148813992419,3cyclictest3441766-21apt-get23:50:121
3148813992418,4cyclictest3217467-21/usr/sbin/munin20:15:151
3148809992417,5cyclictest3196057-21kworker/u8:0+flush-259:020:40:170
3148819992319,3cyclictest3322292-21kworker/u8:4+events_unbound23:12:023
3148819992319,3cyclictest3196057-21kworker/u8:0+events_unbound21:30:153
3148819992317,4cyclictest335557-21systemd-journal21:16:293
3148815992320,2cyclictest3291314-21kworker/u8:3+events_unbound21:35:252
3148815992318,4cyclictest3426265-21/usr/sbin/munin23:35:152
3148815992318,3cyclictest3259261-21/usr/sbin/munin20:55:152
3148815992317,4cyclictest335557-21systemd-journal23:55:002
3148813992320,2cyclictest335557-21systemd-journal22:16:501
3148813992317,4cyclictest335557-21systemd-journal19:43:281
314881399230,13cyclictest3306631-21modprobe21:38:281
3148809992320,2cyclictest3264201-21gpgv21:00:010
3148819992218,3cyclictest3436355-21kworker/u8:1+events_unbound23:55:163
3148819992218,3cyclictest3222641-21kworker/u8:2+events_unbound20:20:243
3148819992218,3cyclictest3222641-21kworker/u8:2+events_unbound20:20:233
3148819992217,3cyclictest3148616-21kworker/u8:1+events_unbound19:11:203
3148819992217,3cyclictest3105060-21kworker/u8:3+events_unbound19:41:293
3148819992216,4cyclictest3148616-21kworker/u8:1+events_unbound20:15:193
3148815992219,2cyclictest3306794-21kworker/u8:0+events_unbound22:07:132
3148813992217,4cyclictest3159083-21apt-get19:20:021
3148809992215,5cyclictest3269733-21latency_hist21:05:000
3148809992215,5cyclictest3190106-21ntpq19:45:230
3148819992117,3cyclictest3322292-21kworker/u8:4+events_unbound23:05:243
3148819992117,3cyclictest3196057-21kworker/u8:0+flush-259:020:05:013
3148819992116,4cyclictest3196057-21kworker/u8:0+events_unbound20:25:013
3148819992116,3cyclictest3457704-21/usr/sbin/munin00:05:183
3148819992115,3cyclictest3196729-21base6419:55:003
3148815992118,2cyclictest3322292-21kworker/u8:4+flush-259:022:20:162
3148815992118,2cyclictest3196057-21kworker/u8:0+flush-259:021:05:252
3148815992117,3cyclictest3399688-21kworker/u8:0+events_unbound00:00:242
3148815992117,3cyclictest3196057-21kworker/u8:0+events_unbound20:30:252
3148815992116,4cyclictest3186028-21/usr/sbin/munin19:45:162
3148813992116,4cyclictest507-21in:imuxsock00:10:001
3148819992017,2cyclictest3337491-21kworker/u8:1+flush-259:022:20:253
3148819992017,2cyclictest3196057-21kworker/u8:0+events_unbound20:40:293
3148819992016,3cyclictest3399688-21kworker/u8:0+events_unbound23:25:213
3148819992016,3cyclictest3343360-21kworker/u8:2+flush-259:022:45:253
3148819992016,2cyclictest3322292-21kworker/u8:4+events_unbound22:30:213
3148819992016,2cyclictest0-21swapper/321:40:003
3148819992015,4cyclictest3452265-21diskstats00:00:163
3148819992015,4cyclictest3232517-21kworker/u8:3+events_unbound20:46:293
3148819992015,4cyclictest3196057-21kworker/u8:0+events_unbound20:38:313
3148819992015,4cyclictest3148616-21kworker/u8:1+flush-259:019:45:193
3148819992014,4cyclictest3322292-21kworker/u8:4+events_unbound21:55:153
3148815992017,2cyclictest3148616-21kworker/u8:1+flush-259:019:35:172
3148815992016,3cyclictest3399688-21kworker/u8:0+events_unbound23:20:152
3148815992016,2cyclictest3384652-21kworker/u8:5+flush-259:023:45:012
3148815992015,3cyclictest335557-21systemd-journal22:10:012
314881399209,9cyclictest3227801-21apt-get20:25:121
3148813992015,3cyclictest3291553-21apt-get21:25:121
3148813992014,4cyclictest3154139-21/usr/sbin/munin19:15:151
3148819991915,3cyclictest3478497-21kworker/u8:3+events_unbound00:30:253
3148819991915,3cyclictest3196057-21kworker/u8:0+events_unbound21:05:143
3148819991915,3cyclictest3148616-21kworker/u8:1+events_unbound20:14:293
3148815991916,3cyclictest3196057-21kworker/u8:0+events_unbound21:25:252
3148815991916,2cyclictest3436355-21kworker/u8:1+flush-259:023:55:252
3148815991916,2cyclictest3322292-21kworker/u8:4+events_unbound22:30:252
3148815991916,2cyclictest3222641-21kworker/u8:2+events_unbound20:25:232
3148815991915,3cyclictest3399688-21kworker/u8:0+events_unbound00:35:172
3148815991915,3cyclictest3196057-21kworker/u8:0+flush-259:020:45:262
3148815991915,3cyclictest3153924-21kworker/u8:2+events_unbound19:25:012
3148813991911,6cyclictest335557-21systemd-journal20:09:591
3148813991910,7cyclictest0-21swapper/122:35:191
314881399190,4cyclictest3426967-21sed23:35:171
314881399190,10cyclictest832-21runrttasks20:56:471
3148809991916,2cyclictest3416363-21irqcore23:25:160
3148809991915,3cyclictest3374587-21apt-get22:45:130
3148809991915,3cyclictest3322292-21kworker/u8:4+flush-259:022:55:250
3148809991914,3cyclictest3159742-21/usr/sbin/munin19:20:190
3148809991913,4cyclictest3468058-21apt-get00:15:130
3148809991912,5cyclictest3179321-21ntpq19:35:210
3148819991815,2cyclictest3269443-21kworker/u8:2+events_unbound21:27:093
3148819991815,1cyclictest335557-21systemd-journal22:52:153
3148819991814,3cyclictest3317313-21kworker/u8:2+events_unbound22:00:233
3148819991813,3cyclictest3337491-21kworker/u8:1+events_unbound22:15:263
3148819991812,4cyclictest3399688-21kworker/u8:0+flush-259:023:50:193
3148815991815,2cyclictest3148616-21kworker/u8:1+events_unbound19:25:232
3148815991814,3cyclictest3399688-21kworker/u8:0+events_unbound23:25:162
3148815991814,3cyclictest3148616-21kworker/u8:1+flush-259:019:40:312
3148815991811,5cyclictest3242164-21ntpq20:35:222
314881399180,10cyclictest3425881-21runrttasks23:33:251
3148819991713,3cyclictest3232517-21kworker/u8:3+events_unbound20:30:143
3148819991712,3cyclictest3322292-21kworker/u8:4+events_unbound23:15:233
314881399171,7cyclictest2109693-21snmpd23:06:281
3148813991712,3cyclictest3254158-21apt-get20:50:131
3148813991711,4cyclictest335557-21systemd-journal19:26:551
314881399171,14cyclictest3301604-21munin-run21:35:001
3148813991710,6cyclictest3317758-21irqcore21:50:181
3148813991710,5cyclictest0-21swapper/122:59:591
314881399170,9cyclictest3478510-21chrt00:24:431
314881399170,9cyclictest3472615-21taskset00:15:511
3148809991714,2cyclictest60350irq/132-enp0s29f1:rx-021:25:250
3148809991714,2cyclictest60350irq/132-enp0s29f1:rx-020:29:520
3148809991713,3cyclictest3322292-21kworker/u8:4+flush-259:022:45:000
3148809991710,5cyclictest3285663-21apt-get21:20:000
3148819991613,2cyclictest3306794-21kworker/u8:0+events_unbound21:52:293
3148819991613,2cyclictest3300659-21kworker/u8:1+events_unbound21:49:303
3148819991613,2cyclictest3153924-21kworker/u8:2+flush-259:019:20:243
3148819991612,3cyclictest3337491-21kworker/u8:1+events_unbound22:25:273
3148815991612,3cyclictest3399688-21kworker/u8:0+flush-259:023:45:162
314881399168,6cyclictest0-21swapper/122:00:181
3148813991613,2cyclictest3395038-21cron23:05:001
3148813991612,3cyclictest3196057-21kworker/u8:0+flush-259:020:45:171
314881399161,13cyclictest335557-21systemd-journal00:15:001
314881399161,13cyclictest3275314-21cron21:10:001
314881399161,11cyclictest3494072-21latency_hist00:40:001
314881399160,9cyclictest832-21runrttasks22:45:521
314881399160,8cyclictest3217025-21taskset20:12:401
314881399160,14cyclictest3233406-21ps20:30:181
314881399160,13cyclictest3379825-21cpuspeed_turbos22:50:151
314881399160,13cyclictest3348920-21tr22:20:151
314881399160,13cyclictest3291441-21latency_hist21:25:001
3148809991614,2cyclictest60350irq/132-enp0s29f1:rx-021:39:070
3148809991614,1cyclictest60350irq/132-enp0s29f1:rx-019:55:160
3148809991613,2cyclictest60350irq/132-enp0s29f1:rx-021:44:590
3148809991613,2cyclictest60350irq/132-enp0s29f1:rx-020:05:160
3148809991613,2cyclictest3253890-21sh20:50:000
3148809991612,2cyclictest60350irq/132-enp0s29f1:rx-000:30:000
3148809991611,4cyclictest509-21rs:main0
3148809991610,4cyclictest335557-21systemd-journal21:12:280
314881999159,4cyclictest3399688-21kworker/u8:0+events_unbound00:35:213
3148819991511,3cyclictest3343360-21kworker/u8:2+events_unbound22:42:293
3148819991511,3cyclictest3232517-21kworker/u8:3+flush-259:020:50:163
3148819991511,3cyclictest3179196-21kworker/u8:2+events_unbound19:51:233
314881599159,4cyclictest3159185-21cp19:20:012
3148815991511,3cyclictest3201766-21kworker/u8:3+flush-259:020:02:282
314881399154,9cyclictest3420068-21proc_pri23:25:221
314881399151,6cyclictest832-21runrttasks21:14:371
314881399151,12cyclictest3451817-21cron00:00:011
314881399151,12cyclictest3404478-21ntpq23:10:211
314881399151,10cyclictest3175343-21apt-get19:35:121
314881399150,14cyclictest335557-21systemd-journal19:54:591
314881399150,13cyclictest3415224-21tail23:20:251
314881399150,13cyclictest3373860-21grep22:40:251
314881399150,13cyclictest3153465-21sed19:10:251
314881399150,12cyclictest3484422-21ps00:30:181
314881399150,12cyclictest3405623-21apt-get23:15:141
314881399150,12cyclictest3312343-21cpuspeed_turbos21:45:141
3148809991513,1cyclictest60350irq/132-enp0s29f1:rx-023:15:000
3148809991513,1cyclictest60350irq/132-enp0s29f1:rx-022:50:290
3148809991513,1cyclictest60350irq/132-enp0s29f1:rx-019:29:230
3148809991513,1cyclictest60350irq/132-enp0s29f1:rx-000:10:230
3148809991512,2cyclictest61150irq/133-enp0s29f1:tx-020:20:150
3148809991512,2cyclictest61150irq/133-enp0s29f1:tx-020:20:140
3148809991512,2cyclictest60350irq/132-enp0s29f1:rx-023:06:030
3148809991512,2cyclictest60350irq/132-enp0s29f1:rx-022:15:130
3148809991512,2cyclictest60350irq/132-enp0s29f1:rx-020:30:270
3148809991512,2cyclictest3488925-21cron00:35:000
3148809991511,3cyclictest60350irq/132-enp0s29f1:rx-021:45:220
3148819991412,2cyclictest3196057-21kworker/u8:0+events_unbound21:10:153
3148819991411,2cyclictest3148616-21kworker/u8:1+events_unbound19:29:293
3148819991410,3cyclictest3431586-21fschecks_count23:40:133
3148815991412,2cyclictest3337491-21kworker/u8:1+events_unbound22:25:172
3148815991411,2cyclictest61750irq/145-enp0s29f1:tx-621:40:162
3148815991411,2cyclictest3399688-21kworker/u8:0+flush-259:023:30:172
3148815991411,2cyclictest3196057-21kworker/u8:0+events_unbound21:00:172
3148815991411,2cyclictest3164934-21kworker/u8:0+events_unbound19:30:242
314881399141,10cyclictest3478908-21grep00:25:141
314881399140,5cyclictest3342287-21/usr/sbin/munin22:10:191
314881399140,13cyclictest3452228-21grep00:00:151
314881399140,12cyclictest3431613-21grep23:40:151
314881399140,12cyclictest3354037-21cut22:25:131
3148809991413,1cyclictest60350irq/132-enp0s29f1:rx-023:52:580
3148809991413,1cyclictest60350irq/132-enp0s29f1:rx-019:40:340
3148809991412,2cyclictest60350irq/132-enp0s29f1:rx-020:54:400
3148809991412,2cyclictest60350irq/132-enp0s29f1:rx-000:36:520
3148809991412,1cyclictest60350irq/132-enp0s29f1:rx-023:03:330
3148809991412,1cyclictest60350irq/132-enp0s29f1:rx-022:00:260
3148809991412,1cyclictest60350irq/132-enp0s29f1:rx-020:00:140
3148809991410,3cyclictest3322292-21kworker/u8:4+flush-259:022:20:180
3148809991410,3cyclictest3170209-21cpuspeed_turbos19:30:120
314881999139,3cyclictest3384652-21kworker/u8:5+events_unbound23:21:303
3148819991310,2cyclictest3426357-21cpuspeed_turbos23:35:143
3148815991310,2cyclictest3322292-21kworker/u8:4+events_unbound22:55:202
3148815991310,2cyclictest3217239-21idleruntime-cro20:15:012
3148809991312,1cyclictest60350irq/132-enp0s29f1:rx-023:55:130
3148809991310,2cyclictest500-21systemd-logind23:50:000
314881599129,2cyclictest3199111-21kthreadcore19:55:222
314881599129,1cyclictest0-21swapper/200:10:172
314881599128,3cyclictest3478497-21kworker/u8:3+flush-259:000:30:182
314881599128,3cyclictest3478497-21kworker/u8:3+flush-259:000:25:262
314881599127,4cyclictest3473659-21apt-get00:20:122
3148815991210,1cyclictest0-21swapper/221:49:592
314880999121,9cyclictest3405623-21apt-get23:15:150
3148809991210,1cyclictest60350irq/132-enp0s29f1:rx-023:40:500
314880999120,10cyclictest3425806-21sshd23:32:280
314881999117,2cyclictest3388953-21ntp_kernel_pll_22:55:213
314881599117,3cyclictest509-21rs:main2
314881599117,3cyclictest3390181-21cpuspeed_turbos23:00:122
314881599117,3cyclictest3322292-21kworker/u8:4+events_unbound22:40:132
314880999119,1cyclictest60350irq/132-enp0s29f1:rx-021:30:130
314881999107,2cyclictest3322292-21kworker/u8:4+flush-259:023:02:023
314881599108,1cyclictest0-21swapper/223:15:002
314881599108,1cyclictest0-21swapper/222:00:132
31488199997,1cyclictest0-21swapper/300:20:243
31488199985,2cyclictest0-21swapper/300:10:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional