You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 19:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Nov 23, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2906097994541,3cyclictest3211518-21kworker/u8:2+events_unbound12:30:033
2906095994138,2cyclictest3051636-21kworker/u8:2+flush-259:010:05:152
2906095994138,2cyclictest2931899-21kworker/u8:2+events_unbound07:40:202
2906097994037,2cyclictest2900766-21kworker/u8:1+flush-259:009:03:463
2906087993936,2cyclictest2947376-21kworker/u8:0+events_unbound08:05:000
2906091993731,4cyclictest2994268-21kworker/u8:2+flush-259:008:45:001
2906091993328,3cyclictest3108526-21apt-get10:25:001
2906087993327,4cyclictest3155339-21latency_hist11:10:010
2906097993128,2cyclictest3211518-21kworker/u8:2+events_unbound12:15:013
2906091993126,3cyclictest3238090-21latency_hist12:30:021
2906087993128,2cyclictest211rcuc/012:40:000
2906097993027,2cyclictest3227545-21kworker/u8:1+flush-259:012:20:223
2906097993027,2cyclictest3149110-21kworker/u8:2+flush-259:011:25:213
2906097992926,2cyclictest3051636-21kworker/u8:2+flush-259:009:50:013
2906097992926,2cyclictest2900766-21kworker/u8:1+flush-259:007:25:003
2906095992926,2cyclictest3046269-21kworker/u8:3+flush-259:009:55:142
2906087992925,3cyclictest3025483-21apt-get09:05:120
2906097992825,2cyclictest3181042-21kworker/u8:4+flush-259:012:35:173
2906097992825,2cyclictest3181042-21kworker/u8:4+flush-259:011:50:003
2906091992824,3cyclictest3211518-21kworker/u8:2+flush-259:012:10:271
2906091992824,3cyclictest3181042-21kworker/u8:4+events_unbound11:45:001
2906091992823,4cyclictest3211518-21kworker/u8:2+flush-259:012:30:191
2906087992825,2cyclictest3119302-21kworker/u8:1+flush-259:011:15:140
2906087992822,4cyclictest2983084-21ntpq08:20:230
2906095992721,4cyclictest2947361-21latency_hist07:49:592
2906091992723,3cyclictest3109382-21kworker/u8:2+flush-259:010:50:001
2906091992721,4cyclictest3109382-21kworker/u8:2+flush-259:010:30:161
2906087992725,1cyclictest3046269-21kworker/u8:3+flush-259:009:50:300
2906095992622,3cyclictest2921593-21kworker/u8:0+flush-259:007:30:202
2906095992622,3cyclictest2900766-21kworker/u8:1+flush-259:009:11:152
2906095992621,4cyclictest2698871-21systemd-journal08:16:052
2906091992620,4cyclictest3119527-21/usr/sbin/munin10:35:211
2906091992620,4cyclictest2698871-21systemd-journal08:27:071
2906087992623,2cyclictest2900766-21kworker/u8:1+events_unbound08:35:160
2906095992520,4cyclictest2698871-21systemd-journal10:53:112
2906095992519,5cyclictest2973188-21apt-get08:15:002
2906091992521,3cyclictest3000063-21rs:main1
2906087992520,3cyclictest2698871-21systemd-journal09:59:590
2906097992422,1cyclictest2952730-21kworker/u8:3+events_unbound08:01:193
2906097992420,3cyclictest3212528-21cat12:05:023
2906097992419,4cyclictest3009758-21latency_hist08:50:013
2906097992419,3cyclictest2698871-21systemd-journal09:10:003
2906095992421,2cyclictest2911466-21kworker/u8:4+flush-259:007:15:262
2906095992420,3cyclictest3109383-21kworker/u8:4+flush-259:010:30:242
2906087992421,2cyclictest3211518-21kworker/u8:2+flush-259:012:20:000
2906087992421,2cyclictest2947376-21kworker/u8:0+flush-259:008:55:180
2906087992420,3cyclictest3227545-21kworker/u8:1+flush-259:012:25:110
2906087992418,4cyclictest2921572-21latency_hist07:25:010
2906095992320,2cyclictest3139926-21kworker/u8:0+flush-259:011:00:262
2906095992320,2cyclictest3109383-21kworker/u8:4+events_unbound10:35:232
2906095992319,3cyclictest3181042-21kworker/u8:4+events_unbound11:55:232
2906095992319,3cyclictest2900766-21kworker/u8:1+flush-259:008:50:212
2906091992319,3cyclictest2952730-21kworker/u8:3+flush-259:007:59:591
2906091992316,5cyclictest3092189-21ntpq10:05:201
2906087992320,2cyclictest3175832-21kworker/u8:0+flush-259:011:50:010
2906087992319,3cyclictest3181042-21kworker/u8:4+events_unbound12:10:150
2906087992319,3cyclictest3175832-21kworker/u8:0+events_unbound11:55:110
2906087992318,3cyclictest3176120-21/usr/sbin/munin11:30:150
2906087992317,5cyclictest3160433-21latency_hist11:15:000
2906097992219,2cyclictest3046269-21kworker/u8:3+events_unbound09:55:163
2906097992219,2cyclictest2900766-21kworker/u8:1+events_unbound08:25:203
2906095992219,2cyclictest3139926-21kworker/u8:0+events_unbound11:10:252
2906095992219,2cyclictest3056543-21idleruntime-cro09:35:002
2906095992219,2cyclictest3051636-21kworker/u8:2+flush-259:009:40:142
2906095992218,3cyclictest3227545-21kworker/u8:1+events_unbound12:20:152
2906091992218,3cyclictest3046269-21kworker/u8:3+flush-259:009:50:261
2906091992216,4cyclictest2698871-21systemd-journal07:52:501
2906087992220,2cyclictest2900766-21kworker/u8:1+events_unbound09:04:100
2906087992220,1cyclictest3109383-21kworker/u8:4+flush-259:010:40:000
2906087992219,2cyclictest3109383-21kworker/u8:4+flush-259:010:50:180
2906087992219,2cyclictest2868472-21kworker/u8:2+events_unbound07:10:240
2906097992118,2cyclictest3181042-21kworker/u8:4+flush-259:011:40:543
2906097992118,2cyclictest3046269-21kworker/u8:3+flush-259:009:55:013
2906097992118,2cyclictest3000063-21rs:main3
2906097992118,2cyclictest2937066-21idleruntime-cro07:40:013
2906097992118,2cyclictest2900766-21kworker/u8:1+flush-259:008:30:193
2906095992118,2cyclictest3181042-21kworker/u8:4+events_unbound11:50:242
2906095992118,2cyclictest2947376-21kworker/u8:0+events_unbound09:18:092
2906095992115,4cyclictest3133807-21ntpq10:45:192
2906091992114,5cyclictest2967147-21ntpq08:05:211
2906087992114,5cyclictest3008862-21ntpq08:45:220
2906097992017,2cyclictest2947376-21kworker/u8:0+events_unbound09:16:103
2906097992016,3cyclictest3119302-21kworker/u8:1+events_unbound10:40:163
2906097992015,3cyclictest2698871-21systemd-journal11:20:133
2906097992014,4cyclictest3050487-21ntpq09:25:233
2906095992017,2cyclictest3212408-21base6412:05:012
2906095992017,2cyclictest3175832-21kworker/u8:0+events_unbound11:40:122
2906095992017,2cyclictest2926743-21idleruntime-cro07:30:012
2906095992016,3cyclictest3119302-21kworker/u8:1+events_unbound11:39:572
2906095992016,3cyclictest2900766-21kworker/u8:1+events_unbound09:20:142
2906091992016,3cyclictest2994268-21kworker/u8:2+flush-259:008:55:001
2906091992016,2cyclictest0-21swapper/111:25:011
2906091992015,4cyclictest3030623-21apt-get09:10:111
2906091992014,4cyclictest2698871-21systemd-journal09:38:091
2906087992017,2cyclictest3139926-21kworker/u8:0+flush-259:011:00:000
2906087992017,2cyclictest3082544-21kworker/u8:0+events_unbound10:15:280
2906087992017,2cyclictest2900766-21kworker/u8:1+flush-259:008:29:330
2906087992016,3cyclictest3109383-21kworker/u8:4+events_unbound10:29:100
2906087992015,4cyclictest2942447-21/usr/sbin/munin07:45:150
2906087992015,3cyclictest2698871-21systemd-journal11:25:130
290608799200,19cyclictest1-21systemd10:11:100
2906097991917,2cyclictest2900766-21kworker/u8:1+flush-259:008:15:213
2906097991916,2cyclictest3051636-21kworker/u8:2+flush-259:010:00:223
2906097991916,2cyclictest2947376-21kworker/u8:0+events_unbound09:30:023
2906097991916,2cyclictest2900766-21kworker/u8:1+flush-259:009:20:153
2906095991916,2cyclictest3051636-21kworker/u8:2+events_unbound10:00:112
2906095991916,2cyclictest2900766-21kworker/u8:1+events_unbound08:21:082
2906095991916,2cyclictest0-21swapper/212:15:002
2906095991915,3cyclictest2900766-21kworker/u8:1+events_unbound08:30:182
2906091991915,3cyclictest2900766-21kworker/u8:1+flush-259:008:10:241
2906091991913,4cyclictest2698871-21systemd-journal08:34:071
2906087991916,2cyclictest3082544-21kworker/u8:0+flush-259:010:20:210
2906087991916,2cyclictest3061706-21kworker/u8:0+flush-259:009:45:210
2906097991816,2cyclictest2900766-21kworker/u8:1+flush-259:008:05:153
2906097991816,2cyclictest2900766-21kworker/u8:1+events_unbound08:35:263
2906097991815,2cyclictest3181042-21kworker/u8:4+flush-259:012:15:213
2906097991815,2cyclictest3109383-21kworker/u8:4+events_unbound11:00:133
2906097991815,2cyclictest3109382-21kworker/u8:2+events_unbound10:30:223
2906097991815,2cyclictest2952730-21kworker/u8:3+flush-259:007:55:273
2906097991815,2cyclictest2900766-21kworker/u8:1+flush-259:007:45:193
2906097991814,3cyclictest3181042-21kworker/u8:4+flush-259:011:50:213
2906097991814,3cyclictest3165298-21kworker/u8:3+events_unbound11:30:193
2906097991814,3cyclictest2868472-21kworker/u8:2+events_unbound07:14:183
2906095991815,2cyclictest3149110-21kworker/u8:2+events_unbound11:20:192
2906095991815,2cyclictest3082544-21kworker/u8:0+events_unbound10:10:192
2906095991815,2cyclictest2900766-21kworker/u8:1+events_unbound07:25:012
2906095991814,3cyclictest3211891-21apt-get12:05:012
2906095991814,3cyclictest3149110-21kworker/u8:2+flush-259:011:25:222
2906095991814,3cyclictest2947376-21kworker/u8:0+flush-259:007:55:202
2906091991814,3cyclictest3181042-21kworker/u8:4+flush-259:012:15:261
2906087991816,2cyclictest3051636-21kworker/u8:2+events_unbound10:00:000
2906087991815,2cyclictest3211518-21kworker/u8:2+events_unbound12:05:160
2906087991815,2cyclictest3134714-21apt10:50:000
2906087991815,2cyclictest2900766-21kworker/u8:1+events_unbound08:10:180
2906087991815,2cyclictest2900766-21kworker/u8:1+events_unbound07:35:180
2906087991813,4cyclictest3119269-21latency_hist10:35:000
2906087991813,4cyclictest2957651-21latency_hist08:00:000
2906097991716,1cyclictest2900766-21kworker/u8:1+events_unbound08:10:233
2906097991714,2cyclictest3139926-21kworker/u8:0+events_unbound11:05:133
2906097991714,2cyclictest3119302-21kworker/u8:1+flush-259:011:15:183
2906097991714,2cyclictest3109382-21kworker/u8:2+events_unbound10:45:153
2906097991714,2cyclictest3103378-21idleruntime-cro10:20:003
2906097991714,2cyclictest3051636-21kworker/u8:2+events_unbound10:05:253
2906097991714,2cyclictest2911466-21kworker/u8:4+events_unbound07:25:193
2906097991713,3cyclictest2698871-21systemd-journal10:40:003
2906095991714,2cyclictest3181042-21kworker/u8:4+events_unbound12:39:562
2906095991714,2cyclictest3139926-21kworker/u8:0+flush-259:011:05:192
2906095991714,2cyclictest3109382-21kworker/u8:2+flush-259:010:30:002
2906095991714,2cyclictest3098425-21kworker/u8:3+flush-259:010:19:562
2906095991713,3cyclictest3119302-21kworker/u8:1+flush-259:010:40:202
2906095991713,3cyclictest2994268-21kworker/u8:2+events_unbound08:45:232
2906091991714,2cyclictest3181042-21kworker/u8:4+flush-259:011:50:171
2906091991714,2cyclictest3098877-21irqcore10:15:191
2906091991713,3cyclictest3000061-21in:imuxsock11:00:001
2906087991715,2cyclictest3046269-21kworker/u8:3+events_unbound09:30:160
2906087991715,2cyclictest3046269-21kworker/u8:3+events_unbound09:25:260
2906087991714,2cyclictest3227545-21kworker/u8:1+events_unbound12:20:170
2906087991714,2cyclictest3139926-21kworker/u8:0+flush-259:011:00:250
2906087991713,3cyclictest3051636-21kworker/u8:2+flush-259:009:40:450
2906087991713,3cyclictest2900766-21kworker/u8:1+flush-259:007:50:250
2906087991712,3cyclictest3149110-21kworker/u8:2+events_unbound11:20:150
2906097991614,2cyclictest3051636-21kworker/u8:2+flush-259:009:40:243
2906097991613,2cyclictest3181042-21kworker/u8:4+flush-259:011:55:213
2906097991613,2cyclictest3175832-21kworker/u8:0+events_unbound12:01:143
2906097991613,2cyclictest3119302-21kworker/u8:1+events_unbound11:10:133
2906097991613,2cyclictest2931899-21kworker/u8:2+flush-259:007:40:213
2906097991613,2cyclictest2900766-21kworker/u8:1+flush-259:008:50:163
2906095991613,2cyclictest3227545-21kworker/u8:1+flush-259:012:25:172
2906095991613,2cyclictest3046269-21kworker/u8:3+events_unbound09:48:092
2906095991613,2cyclictest2947376-21kworker/u8:0+flush-259:008:00:232
2906095991613,2cyclictest2900766-21kworker/u8:1+flush-259:008:25:182
2906091991612,3cyclictest3181042-21kworker/u8:4+flush-259:012:00:261
2906091991612,3cyclictest3097873-21kworker/u8:1+flush-259:010:15:001
2906091991612,3cyclictest301rcuc/111:30:001
2906091991610,4cyclictest2698871-21systemd-journal09:35:001
2906087991613,3cyclictest2931899-21kworker/u8:2+flush-259:007:40:200
2906087991613,2cyclictest3211518-21kworker/u8:2+events_unbound12:30:200
2906087991613,2cyclictest2911466-21kworker/u8:4+flush-259:007:15:210
2906087991611,4cyclictest2989382-21apt-get08:30:120
2906097991513,2cyclictest3119302-21kworker/u8:1+events_unbound11:35:103
2906097991513,1cyclictest0-21swapper/307:20:013
2906097991512,2cyclictest3109382-21kworker/u8:2+events_unbound10:26:113
2906097991512,2cyclictest2921593-21kworker/u8:0+events_unbound07:34:363
2906095991512,2cyclictest3181042-21kworker/u8:4+flush-259:012:34:452
2906095991512,2cyclictest3149110-21kworker/u8:2+events_unbound11:18:142
2906095991512,2cyclictest3109383-21kworker/u8:4+events_unbound10:55:042
2906095991512,2cyclictest3108561-21gpgv10:25:002
2906095991512,2cyclictest2900766-21kworker/u8:1+flush-259:009:00:192
2906095991511,3cyclictest2947376-21kworker/u8:0+events_unbound08:55:162
290609199159,4cyclictest2921807-21/usr/sbin/munin07:25:191
2906091991512,2cyclictest3051412-21sh09:30:001
2906087991513,2cyclictest3051636-21kworker/u8:2+events_unbound10:09:570
2906087991513,2cyclictest2900766-21kworker/u8:1+flush-259:008:05:250
2906087991513,2cyclictest2900766-21kworker/u8:1+events_unbound07:29:480
2906087991513,1cyclictest60650irq/133-enp0s29f1:rx-010:40:260
2906087991512,2cyclictest60650irq/133-enp0s29f1:rx-011:50:250
2906087991512,2cyclictest2900766-21kworker/u8:1+events_unbound08:40:170
2906087991511,3cyclictest3181042-21kworker/u8:4+events_unbound11:38:130
2906097991412,2cyclictest2994268-21kworker/u8:2+events_unbound08:58:093
2906097991411,2cyclictest3109383-21kworker/u8:4+flush-259:010:50:213
2906097991411,2cyclictest3109383-21kworker/u8:4+events_unbound10:55:153
2906097991411,2cyclictest3082544-21kworker/u8:0+flush-259:010:20:153
290609799140,12cyclictest2698871-21systemd-journal12:25:123
2906095991410,3cyclictest3165298-21kworker/u8:3+events_unbound11:30:172
2906095991410,3cyclictest3000061-21in:imuxsock09:55:002
290609199149,4cyclictest2900766-21kworker/u8:1+flush-259:008:37:121
290609199149,3cyclictest2973718-21base6408:15:011
290609199147,5cyclictest2698871-21systemd-journal10:45:011
2906091991411,2cyclictest3177168-21kthreadcore11:30:171
2906091991411,2cyclictest3156037-21irqcore11:10:171
2906091991411,1cyclictest0-21swapper/107:30:161
2906091991410,3cyclictest3119302-21kworker/u8:1+flush-259:011:15:241
2906091991410,3cyclictest3061706-21kworker/u8:0+flush-259:009:45:121
290608799149,4cyclictest60650irq/133-enp0s29f1:rx-009:20:210
2906087991412,1cyclictest60650irq/133-enp0s29f1:rx-011:44:030
2906087991412,1cyclictest60650irq/133-enp0s29f1:rx-009:15:180
2906087991411,2cyclictest2900766-21kworker/u8:1+flush-259:008:15:050
290609799139,3cyclictest2900766-21kworker/u8:1+events_unbound09:10:233
2906097991311,2cyclictest3082544-21kworker/u8:0+events_unbound10:10:173
2906097991310,2cyclictest2983101-21ntp_offset08:20:233
290609599139,3cyclictest2947376-21kworker/u8:0+events_unbound07:50:272
2906095991311,2cyclictest3046269-21kworker/u8:3+flush-259:009:25:242
2906095991310,2cyclictest3227545-21kworker/u8:1+events_unbound12:18:152
2906095991310,2cyclictest2947376-21kworker/u8:0+flush-259:009:07:142
2906095991310,2cyclictest2900766-21kworker/u8:1+flush-259:008:05:242
2906095991310,2cyclictest2900766-21kworker/u8:1+events_unbound08:40:032
290609199138,3cyclictest2698871-21systemd-journal07:41:491
2906091991310,2cyclictest3181699-21sed11:35:181
2906087991312,1cyclictest60650irq/133-enp0s29f1:rx-012:05:010
290609599129,2cyclictest2868472-21kworker/u8:2+events_unbound07:10:262
290609599129,2cyclictest0-21swapper/207:35:192
2906095991210,1cyclictest0-21swapper/211:45:162
290609199129,2cyclictest3213006-21irqcore12:05:161
290609199129,2cyclictest0-21swapper/112:40:001
290609199128,3cyclictest2994268-21kworker/u8:2+flush-259:008:45:261
290609199128,3cyclictest2947376-21kworker/u8:0+flush-259:008:55:171
290608799129,2cyclictest2900766-21kworker/u8:1+events_unbound07:33:490
290609199116,3cyclictest3206739-21cron12:00:001
290609199115,4cyclictest2698871-21systemd-journal11:48:141
290609199108,1cyclictest0-21swapper/111:05:131
290609199107,2cyclictest0-21swapper/107:35:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional