You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 06:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Tue Apr 07, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3237932995445,8cyclictest3486721-21kworker/u8:4+flush-259:023:10:250
3237932994338,4cyclictest3326524-21apt-get20:35:110
3237937993633,2cyclictest3486721-21kworker/u8:4+events_unbound23:30:132
3237937993529,4cyclictest3372642-21kworker/u8:3+flush-259:021:45:152
3237932993529,4cyclictest252-21systemd-journal20:45:160
3237942993429,4cyclictest3264734-21/usr/sbin/munin19:35:223
3237934993327,4cyclictest3388118-21latency_hist21:35:011
3237932993125,4cyclictest252-21systemd-journal21:43:180
3237934993025,3cyclictest3321385-21apt-get20:30:111
3237932993024,4cyclictest252-21systemd-journal21:40:000
3237932993023,5cyclictest3574272-21ntpq00:30:190
3237942992924,3cyclictest3248169-21apt-get19:20:003
3237934992922,5cyclictest252-21systemd-journal22:40:001
3237932992925,3cyclictest3362543-21kworker/u8:1+flush-259:022:00:250
3237932992924,4cyclictest3274836-21kworker/u8:2+flush-259:020:15:210
3237932992824,3cyclictest3280199-21kworker/u8:1+flush-259:020:00:250
3237932992822,4cyclictest3247293-21ntpq19:15:210
3237942992723,3cyclictest3352226-21/usr/sbin/munin21:00:153
3237942992722,3cyclictest252-21systemd-journal22:47:233
3237937992723,3cyclictest3398913-21kworker/u8:2+flush-259:021:55:002
3237937992723,3cyclictest3315865-21kworker/u8:1+flush-259:020:50:232
3237937992722,4cyclictest3254013-21apt-get19:25:102
3237934992723,3cyclictest3538748-21gpgv00:00:001
323793299270,26cyclictest1-21systemd22:07:220
3237942992620,4cyclictest3559685-21latency_hist00:20:013
3237937992623,2cyclictest3398913-21kworker/u8:2+flush-259:023:20:182
3237937992622,3cyclictest3486095-21kworker/u8:3+flush-259:023:08:312
3237937992621,4cyclictest3174349-21kworker/u8:4+flush-259:019:45:212
3237934992620,4cyclictest3290484-21/usr/sbin/munin20:00:141
3237934992620,4cyclictest252-21systemd-journal21:10:001
3237932992622,3cyclictest3274836-21kworker/u8:2+flush-259:020:15:010
3237937992521,3cyclictest3253980-21kworker/u8:2+flush-259:019:35:312
3237937992519,4cyclictest3274836-21kworker/u8:2+events_unbound19:52:142
3237934992520,3cyclictest252-21systemd-journal21:05:001
3237932992521,3cyclictest3362543-21kworker/u8:1+flush-259:022:25:000
3237942992420,3cyclictest3174349-21kworker/u8:4+flush-259:019:20:163
3237942992317,4cyclictest3414402-21latency_hist22:00:013
3237937992320,2cyclictest3570033-21kworker/u8:1+flush-259:000:35:182
3237937992320,2cyclictest3513007-21kworker/u8:2+events_unbound00:25:162
3237937992320,2cyclictest3326262-21turbostat.cron20:35:002
3237937992319,3cyclictest3398913-21kworker/u8:2+flush-259:022:10:242
3237937992317,4cyclictest3486721-21kworker/u8:4+flush-259:023:50:162
3237934992319,3cyclictest3513007-21kworker/u8:2+flush-259:000:15:151
3237932992319,3cyclictest3486721-21kworker/u8:4+flush-259:023:15:260
3237932992319,3cyclictest3274836-21kworker/u8:2+flush-259:020:25:240
3237932992318,4cyclictest3549510-21kworker/u8:3+flush-259:000:20:040
3237932992317,4cyclictest3352029-21kworker/u8:0+flush-259:021:10:160
3237932992317,4cyclictest252-21systemd-journal19:51:120
323793299230,22cyclictest1-21systemd21:16:170
3237942992216,4cyclictest3549280-21latency_hist00:10:003
3237937992219,2cyclictest3513007-21kworker/u8:2+flush-259:000:15:242
3237937992219,2cyclictest3398913-21kworker/u8:2+events_unbound22:05:232
3237937992219,2cyclictest3372848-21kworker/u8:4+flush-259:021:25:182
3237937992218,3cyclictest3372642-21kworker/u8:3+events_unbound21:20:232
3237937992218,3cyclictest3372642-21kworker/u8:3+events_unbound21:20:232
3237937992217,4cyclictest3398913-21kworker/u8:2+events_unbound22:45:242
3237934992218,3cyclictest3248788-21base6419:20:011
3237942992118,2cyclictest252-21systemd-journal20:39:173
3237942992116,4cyclictest3419585-21latency_hist22:05:013
3237937992118,2cyclictest3274836-21kworker/u8:2+flush-259:020:15:162
3237937992117,3cyclictest3398913-21kworker/u8:2+events_unbound22:15:192
3237937992117,3cyclictest3253980-21kworker/u8:2+flush-259:019:30:212
3237932992117,3cyclictest3497013-21kworker/u8:0+flush-259:023:35:250
3237932992117,3cyclictest3372642-21kworker/u8:3+flush-259:021:25:330
3237932992117,3cyclictest3269584-21kworker/u8:0+flush-259:019:40:170
3237932992114,5cyclictest3480434-21ntpq23:00:210
3237932992114,5cyclictest3470146-21ntpq22:50:220
323793299211,18cyclictest1-21systemd23:54:230
3237942992016,3cyclictest3331659-21/usr/sbin/munin20:40:193
3237942992016,3cyclictest3227269-21kworker/u8:1+flush-259:019:25:163
3237937992016,3cyclictest3486721-21kworker/u8:4+events_unbound23:40:242
3237937992016,3cyclictest3372642-21kworker/u8:3+events_unbound21:30:182
3237937992016,3cyclictest3280199-21kworker/u8:1+events_unbound20:04:152
3237937992016,3cyclictest3280199-21kworker/u8:1+events_unbound19:55:222
3237937992015,4cyclictest3362543-21kworker/u8:1+flush-259:023:10:232
323793799200,18cyclictest1-21systemd21:14:172
3237934992016,3cyclictest3362543-21kworker/u8:1+flush-259:022:30:171
3237934992012,7cyclictest0-21swapper/121:00:001
3237932992017,2cyclictest3549510-21kworker/u8:3+flush-259:000:10:260
3237932992015,4cyclictest3513007-21kworker/u8:2+flush-259:000:25:270
3237932992015,4cyclictest3295670-21apt-get20:05:110
3237932992015,3cyclictest3259149-21/usr/sbin/munin19:30:140
3237942991915,3cyclictest3393311-21apt-get21:40:023
3237942991914,4cyclictest3280199-21kworker/u8:1+flush-259:020:10:253
3237942991913,4cyclictest252-21systemd-journal21:37:183
3237937991915,3cyclictest3362543-21kworker/u8:1+events_unbound22:55:122
3237937991915,3cyclictest3174349-21kworker/u8:4+events_unbound19:40:132
3237937991914,4cyclictest3362543-21kworker/u8:1+events_unbound22:42:242
3237937991913,4cyclictest3321119-21latency_hist20:30:002
323793499197,10cyclictest3471014-21munin-plugin-st22:54:591
3237934991915,3cyclictest3253980-21kworker/u8:2+flush-259:019:30:161
3237934991915,2cyclictest3382937-21kworker/u8:2+flush-259:021:30:001
3237934991914,4cyclictest3367740-21apt-get21:15:111
3237934991914,3cyclictest3372859-21/usr/sbin/munin21:20:161
3237934991914,3cyclictest3372859-21/usr/sbin/munin21:20:161
3237932991915,2cyclictest3174349-21kworker/u8:4+flush-259:019:20:180
3237937991815,2cyclictest3442771-21kthreadcore22:25:172
3237937991814,3cyclictest3497013-21kworker/u8:0+events_unbound23:47:242
3237937991814,3cyclictest3362543-21kworker/u8:1+flush-259:022:35:172
3237937991814,3cyclictest3362543-21kworker/u8:1+events_unbound22:30:192
3237937991814,3cyclictest3362543-21kworker/u8:1+events_unbound21:57:452
3237937991814,3cyclictest3326283-21kworker/u8:0+events_unbound20:37:162
3237937991814,3cyclictest3290261-21kworker/u8:0+events_unbound20:06:162
3237937991814,3cyclictest3237838-21kworker/u8:2+flush-259:019:10:232
3237934991814,3cyclictest3549510-21kworker/u8:3+flush-259:000:10:181
3237934991814,3cyclictest3419844-21apt-get22:05:131
3237934991814,3cyclictest3275058-21apt-get19:45:111
3237934991812,4cyclictest252-21systemd-journal22:50:001
3237932991815,2cyclictest3538818-21sort00:00:010
3237932991814,2cyclictest0-21swapper/020:45:000
3237932991812,4cyclictest3507911-21apt-get23:30:130
3237932991812,4cyclictest3460727-21latency_hist22:45:010
3237942991714,2cyclictest3491719-21apt-key23:15:003
3237942991713,3cyclictest0-21swapper/321:25:003
3237942991713,3cyclictest0-21swapper/321:25:003
3237937991714,2cyclictest3497013-21kworker/u8:0+events_unbound00:08:352
3237937991714,2cyclictest3335577-21kworker/u8:2+flush-259:021:10:002
3237937991714,1cyclictest0-21swapper/221:45:012
3237937991713,3cyclictest3398913-21kworker/u8:2+flush-259:023:00:162
3237937991713,3cyclictest3398913-21kworker/u8:2+flush-259:022:00:212
3237937991712,4cyclictest3362543-21kworker/u8:1+events_unbound22:52:502
323793499179,7cyclictest0-21swapper/119:29:591
323793499177,9cyclictest0-21swapper/122:40:111
323793499175,10cyclictest3575192-21latency_hist00:35:001
323793499174,4cyclictest3487445-21sed23:10:151
323793499172,3cyclictest3245112-21cut19:15:171
3237934991714,2cyclictest3513007-21kworker/u8:2+flush-259:000:25:171
3237934991713,3cyclictest3513042-21apt-get23:35:121
3237934991711,4cyclictest3497507-21/usr/sbin/munin23:20:211
3237932991714,2cyclictest60350irq/132-enp0s29f1:rx-019:25:160
3237932991712,4cyclictest3440631-21diskstats22:25:130
3237942991611,4cyclictest3502465-21kworker/u8:1+flush-259:023:32:593
323793799161,5cyclictest3517936-21cron23:40:002
3237937991613,2cyclictest3570033-21kworker/u8:1+events_unbound00:34:112
3237937991612,3cyclictest3549510-21kworker/u8:3+flush-259:000:20:312
3237937991612,3cyclictest3497013-21kworker/u8:0+events_unbound23:19:242
3237937991612,3cyclictest3362543-21kworker/u8:1+events_unbound22:24:242
3237937991612,3cyclictest3227269-21kworker/u8:1+events_unbound19:15:222
3237937991611,4cyclictest3335577-21kworker/u8:2+events_unbound20:55:132
323793499167,8cyclictest3399409-21fschecks_time21:45:121
323793499161,4cyclictest3416782-21sshd22:00:181
323793499161,2cyclictest3265176-21cut19:35:161
323793499161,1cyclictest3496341-21ntpq23:15:201
3237934991610,4cyclictest252-21systemd-journal20:11:171
323793499160,2cyclictest3528249-21idleruntime-cro23:50:011
3237932991614,1cyclictest60350irq/132-enp0s29f1:rx-021:50:230
3237932991613,2cyclictest60350irq/132-enp0s29f1:rx-023:05:230
3237932991613,2cyclictest60350irq/132-enp0s29f1:rx-021:55:250
3237932991610,4cyclictest3497693-21diskstats23:20:120
323794299159,4cyclictest3311251-21diskstats20:20:153
3237942991512,1cyclictest0-21swapper/322:54:593
3237942991511,3cyclictest3507168-21unixbench_multi23:25:253
3237937991512,2cyclictest3335577-21kworker/u8:2+events_unbound21:00:282
3237937991511,3cyclictest3388603-21fschecks_time21:35:112
3237937991510,3cyclictest3310566-21kworker/u8:3+flush-259:020:20:252
323793499154,9cyclictest3434175-21grep22:15:201
323793499154,3cyclictest3444407-21cut22:25:191
323793499151,3cyclictest3345880-21cut20:50:191
323793499151,2cyclictest3486493-21turbostat.cron23:09:591
323793499151,2cyclictest3429864-21cron22:15:001
323793499151,2cyclictest3398882-21idleruntime-cro21:45:001
323793499151,2cyclictest3285116-21turbostat.cron19:55:001
323793499151,2cyclictest0-21swapper/120:20:161
323793499151,1cyclictest3309953-21ntpq20:15:211
323793499150,4cyclictest3533365-21latency_hist23:55:001
323793499150,4cyclictest3388791-21irqcore21:35:131
323793499150,3cyclictest3549265-21cron00:10:001
323793499150,2cyclictest3321109-21cron20:30:001
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-023:42:260
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-022:55:150
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-022:15:130
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-019:45:230
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:160
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-000:08:030
3237932991513,1cyclictest60350irq/132-enp0s29f1:rx-000:00:190
3237932991512,2cyclictest60350irq/132-enp0s29f1:rx-023:45:230
3237932991512,2cyclictest60350irq/132-enp0s29f1:rx-022:39:500
3237932991512,2cyclictest60350irq/132-enp0s29f1:rx-019:35:230
3237932991512,2cyclictest3346857-21turbostat20:55:000
3237932991511,2cyclictest60350irq/132-enp0s29f1:rx-022:30:210
323793299150,13cyclictest3316015-21turbostat20:25:010
323794299149,3cyclictest3497013-21kworker/u8:0+flush-259:000:00:163
3237942991412,1cyclictest0-21swapper/319:55:013
3237942991411,2cyclictest3481354-21latency_hist23:04:593
3237937991410,3cyclictest3315865-21kworker/u8:1+events_unbound20:45:192
3237937991410,3cyclictest3249052-21fschecks_count19:20:142
323793499141,3cyclictest3337002-21fschecks_count20:45:151
323793499141,3cyclictest3299483-21expr20:05:191
323793499141,1cyclictest3502713-21apt-get23:25:121
323793499140,4cyclictest3577155-21grep00:35:161
323793499140,4cyclictest3408805-21grep21:50:251
323793499140,3cyclictest3562832-21grep00:20:171
323793499140,3cyclictest3334320-21sshd20:40:171
323793499140,2cyclictest3435753-21sed22:20:181
323793499140,2cyclictest3410481-21cut21:55:181
323793299148,4cyclictest3424935-21/usr/sbin/munin22:10:210
3237932991413,1cyclictest60350irq/132-enp0s29f1:rx-022:46:330
3237932991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:440
3237932991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:150
3237932991412,1cyclictest0-21swapper/021:00:000
3237932991410,3cyclictest3237838-21kworker/u8:2+flush-259:019:10:160
323794299139,3cyclictest3549510-21kworker/u8:3+flush-259:000:14:043
323794299139,3cyclictest3398913-21kworker/u8:2+flush-259:022:17:443
323794299139,3cyclictest3174349-21kworker/u8:4+flush-259:019:45:263
323794299137,4cyclictest3274836-21kworker/u8:2+flush-259:020:15:173
323794299137,4cyclictest252-21systemd-journal23:54:243
323794299136,4cyclictest252-21systemd-journal22:22:463
323793799139,3cyclictest3513007-21kworker/u8:2+events_unbound00:15:002
323793799138,4cyclictest3310566-21kworker/u8:3+flush-259:020:40:102
323793499132,1cyclictest3471270-21apt-get22:55:111
323793499130,1cyclictest3237941-21sed19:10:011
3237932991312,1cyclictest60350irq/132-enp0s29f1:rx-020:30:480
3237932991311,1cyclictest60350irq/132-enp0s29f1:rx-021:00:170
3237932991310,2cyclictest60350irq/132-enp0s29f1:rx-021:31:380
3237932991310,2cyclictest60350irq/132-enp0s29f1:rx-019:55:120
323794299129,2cyclictest3440536-21cpuspeed_turbos22:25:143
323794299129,2cyclictest3294918-21unixbench_multi20:00:253
323794299129,2cyclictest0-21swapper/322:10:263
323794299129,2cyclictest0-21swapper/319:55:233
323794299129,2cyclictest0-21swapper/300:40:003
323794299129,1cyclictest0-21swapper/320:05:173
323794299128,3cyclictest3372642-21kworker/u8:3+flush-259:021:51:493
323794299121,2cyclictest3264071-21apt19:34:593
3237942991210,1cyclictest0-21swapper/321:45:113
323793799129,2cyclictest3543646-21unixbench_multi00:00:252
323793299128,3cyclictest60350irq/132-enp0s29f1:rx-023:25:230
3237932991210,1cyclictest60350irq/132-enp0s29f1:rx-021:20:150
3237932991210,1cyclictest60350irq/132-enp0s29f1:rx-021:20:150
323794299119,1cyclictest0-21swapper/300:00:003
323794299118,2cyclictest3445696-21cpuspeed_turbos22:30:133
323794299118,2cyclictest3362749-21fschecks_count21:10:143
323794299117,2cyclictest3325403-21ntpq20:30:203
323794299116,4cyclictest3523346-21apt-get23:45:113
32379429994,3cyclictest3242256-21/usr/sbin/munin19:10:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional