You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 17:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Jan 01, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
956453994840,5cyclictest1246995-21latency_hist11:50:001
956457994643,2cyclictest987953-21kworker/u8:3+flush-259:007:50:232
956457994238,3cyclictest1008353-21kworker/u8:3+flush-259:008:40:192
956457994237,3cyclictest1288299-21gpgv12:30:012
956449994036,3cyclictest1112166-21kworker/u8:0+flush-259:010:05:000
956449993932,5cyclictest1277854-21latency_hist12:20:000
956453993832,4cyclictest1206080-21apt-get11:10:111
956457993734,2cyclictest1008353-21kworker/u8:3+events_unbound08:55:242
956457993734,2cyclictest1003108-21kworker/u8:1+flush-259:009:20:162
956453993530,3cyclictest1185383-21/usr/sbin/munin10:50:151
956459993430,3cyclictest1226474-21kworker/u8:0+flush-259:012:40:003
956457993430,3cyclictest1008353-21kworker/u8:3+events_unbound08:25:212
956453993429,4cyclictest1003108-21kworker/u8:1+flush-259:008:20:001
956453993428,4cyclictest1205324-21kworker/u8:3+flush-259:011:30:251
956457993330,2cyclictest1008353-21kworker/u8:3+events_unbound10:00:252
956453993327,4cyclictest252-21systemd-journal08:25:001
956449993329,3cyclictest1096310-21apt-get09:25:000
956457993229,2cyclictest1200674-21kworker/u8:2+events_unbound11:05:222
956457993229,2cyclictest1144732-21kworker/u8:1+flush-259:010:25:222
956457993229,2cyclictest1008353-21kworker/u8:3+events_unbound09:15:242
956457993228,3cyclictest1008353-21kworker/u8:3+flush-259:009:50:222
956453993227,4cyclictest1008353-21kworker/u8:3+flush-259:008:40:161
956453993226,4cyclictest956640-21/usr/sbin/munin07:10:141
95644999320,30cyclictest1-21systemd11:03:410
956459993127,3cyclictest1216395-21kworker/u8:1+flush-259:011:25:173
956459993127,3cyclictest1216395-21kworker/u8:1+flush-259:011:25:173
956457993129,1cyclictest1003108-21kworker/u8:1+flush-259:008:35:262
956457993128,2cyclictest1205324-21kworker/u8:3+flush-259:011:35:222
956457993128,2cyclictest1144732-21kworker/u8:1+flush-259:010:40:222
956457993126,4cyclictest1159114-21cp10:25:012
956457993027,2cyclictest1153636-21kworker/u8:3+events_unbound10:30:202
956453993026,3cyclictest1294408-21apt-get12:35:121
956453993026,3cyclictest1008353-21kworker/u8:3+flush-259:010:05:151
956453993024,4cyclictest1090716-21latency_hist09:20:001
956457992926,2cyclictest1008353-21kworker/u8:3+flush-259:010:05:212
956457992925,3cyclictest1112166-21kworker/u8:0+flush-259:010:10:222
956457992925,2cyclictest967217-21kworker/u8:2+events_unbound07:45:012
956459992824,3cyclictest1236579-21kworker/u8:2+flush-259:011:50:283
956457992825,2cyclictest1003108-21kworker/u8:1+flush-259:008:50:222
956457992825,2cyclictest1003108-21kworker/u8:1+events_unbound08:20:202
956453992823,4cyclictest252-21systemd-journal10:40:421
956449992822,4cyclictest1013888-21diskstats08:05:130
956457992723,3cyclictest1180005-21kworker/u8:0+flush-259:010:45:252
956457992723,3cyclictest1003108-21kworker/u8:1+flush-259:008:45:192
956457992721,5cyclictest1266640-21ntpq12:05:212
956459992622,3cyclictest1008353-21kworker/u8:3+events_unbound09:55:233
956459992622,2cyclictest1003108-21kworker/u8:1+flush-259:008:10:223
956459992620,4cyclictest1221304-21latency_hist11:25:003
956459992620,4cyclictest1221304-21latency_hist11:25:003
956459992619,5cyclictest1246085-21ntpq11:45:223
956457992622,3cyclictest923303-21kworker/u8:0+flush-259:007:30:302
956453992621,3cyclictest967413-21/usr/sbin/munin07:20:141
956453992621,3cyclictest1148923-21apt-get10:15:111
956453992620,4cyclictest1200634-21latency_hist11:05:001
956449992622,3cyclictest1144732-21kworker/u8:1+flush-259:010:40:120
956449992619,5cyclictest1251213-21ntpq11:50:250
956459992521,3cyclictest1185028-21kworker/u8:4+events_unbound10:50:233
956459992521,3cyclictest1144732-21kworker/u8:1+flush-259:010:10:253
956459992521,3cyclictest1144732-21kworker/u8:1+events_unbound10:35:163
956459992521,3cyclictest1003108-21kworker/u8:1+events_unbound09:30:153
956457992522,2cyclictest987953-21kworker/u8:3+events_unbound07:45:152
956457992522,2cyclictest1236579-21kworker/u8:2+events_unbound12:10:262
956457992522,2cyclictest1236579-21kworker/u8:2+events_unbound11:45:182
956457992520,3cyclictest1236579-21kworker/u8:2+flush-259:012:15:242
956453992521,3cyclictest923303-21kworker/u8:0+flush-259:007:35:261
956453992521,3cyclictest1190351-21kworker/u8:0+flush-259:010:55:261
956453992521,2cyclictest1262668-21apt-get12:05:111
956453992519,4cyclictest1048612-21ntpq08:35:241
956453992518,5cyclictest1096310-21apt-get09:25:011
956459992420,3cyclictest1226474-21kworker/u8:0+events_unbound12:30:253
956459992420,3cyclictest1008353-21kworker/u8:3+flush-259:009:00:223
956457992418,4cyclictest961580-21apt-get07:15:002
956457992418,4cyclictest1102106-21/usr/sbin/munin09:30:162
956453992420,3cyclictest1200895-21apt-get11:05:111
956459992320,2cyclictest1236579-21kworker/u8:2+events_unbound12:15:223
956459992320,2cyclictest1144732-21kworker/u8:1+flush-259:010:15:253
956459992320,2cyclictest1008353-21kworker/u8:3+events_unbound09:35:253
956459992319,3cyclictest944873-21kworker/u8:1+events_unbound07:15:213
956459992319,3cyclictest923303-21kworker/u8:0+events_unbound07:25:243
956459992319,3cyclictest1144732-21kworker/u8:1+flush-259:010:25:253
956459992318,3cyclictest252-21systemd-journal11:40:563
956459992317,4cyclictest1190567-21/usr/sbin/munin10:55:143
956457992319,3cyclictest1003108-21kworker/u8:1+events_unbound09:05:272
956453992318,4cyclictest1044316-21latency_hist08:35:001
956453992316,5cyclictest1168895-21ntpq10:30:241
956449992320,2cyclictest1236579-21kworker/u8:2+events_unbound12:00:240
956449992319,3cyclictest509-21rs:main0
956449992319,3cyclictest1200674-21kworker/u8:2+events_unbound11:05:250
956459992219,2cyclictest1236579-21kworker/u8:2+events_unbound12:05:143
956459992219,2cyclictest1003108-21kworker/u8:1+flush-259:008:45:233
956459992218,3cyclictest1236579-21kworker/u8:2+flush-259:012:00:283
956459992218,3cyclictest1236579-21kworker/u8:2+flush-259:011:55:203
956459992218,3cyclictest1008353-21kworker/u8:3+flush-259:009:15:113
956459992218,3cyclictest1008353-21kworker/u8:3+events_unbound09:05:213
956459992218,3cyclictest1003108-21kworker/u8:1+flush-259:008:38:123
956459992218,3cyclictest1003108-21kworker/u8:1+events_unbound08:20:223
956457992218,3cyclictest923303-21kworker/u8:0+events_unbound07:15:222
956457992218,3cyclictest1003108-21kworker/u8:1+flush-259:008:15:182
956457992218,3cyclictest1003108-21kworker/u8:1+events_unbound09:40:222
956457992218,3cyclictest1003108-21kworker/u8:1+events_unbound09:00:292
956457992216,4cyclictest252-21systemd-journal11:13:422
956453992217,4cyclictest1132306-21ntpq09:55:221
956453992216,4cyclictest252-21systemd-journal07:30:011
956453992216,4cyclictest1008353-21kworker/u8:3+flush-259:009:20:251
956449992219,2cyclictest1153636-21kworker/u8:3+events_unbound10:35:200
956449992219,2cyclictest1123572-21kworker/u8:2+flush-259:010:00:000
956449992219,2cyclictest1003108-21kworker/u8:1+flush-259:009:10:000
956449992218,3cyclictest1153636-21kworker/u8:3+events_unbound10:30:230
956449992217,3cyclictest1039446-21apt-get08:30:130
956459992118,2cyclictest923303-21kworker/u8:0+events_unbound07:30:183
956459992118,2cyclictest1112166-21kworker/u8:0+flush-259:009:50:003
956459992118,2cyclictest1008353-21kworker/u8:3+events_unbound09:50:243
956459992117,3cyclictest1008353-21kworker/u8:3+flush-259:010:05:223
956459992116,4cyclictest1034306-21/usr/sbin/munin08:25:163
956457992117,3cyclictest1205324-21kworker/u8:3+flush-259:011:40:202
956457992117,2cyclictest923303-21kworker/u8:0+events_unbound07:25:162
956457992115,4cyclictest1216403-21/usr/sbin/munin11:20:152
956457992115,4cyclictest1216403-21/usr/sbin/munin11:20:142
956453992118,2cyclictest1236579-21kworker/u8:2+flush-259:011:50:191
956449992117,3cyclictest1236579-21kworker/u8:2+events_unbound11:55:230
956449992117,3cyclictest1008353-21kworker/u8:3+events_unbound09:50:220
956459992017,2cyclictest1003108-21kworker/u8:1+events_unbound09:40:263
956459992015,4cyclictest1144732-21kworker/u8:1+flush-259:010:25:003
956457992017,2cyclictest1277668-21kworker/u8:3+flush-259:012:35:012
956457992017,2cyclictest1003108-21kworker/u8:1+flush-259:008:30:172
956457992016,3cyclictest1190351-21kworker/u8:0+flush-259:010:55:152
956457992016,3cyclictest1003108-21kworker/u8:1+events_unbound07:55:102
956453992017,2cyclictest1003406-21apt-get07:55:081
956453992016,3cyclictest1003108-21kworker/u8:1+flush-259:008:10:241
956449992017,2cyclictest923303-21kworker/u8:0+events_unbound07:30:160
956449992017,2cyclictest1205324-21kworker/u8:3+flush-259:011:40:230
956449992016,3cyclictest987953-21kworker/u8:3+flush-259:007:45:170
956449992016,3cyclictest923303-21kworker/u8:0+flush-259:007:10:170
956449992016,3cyclictest1277668-21kworker/u8:3+flush-259:012:35:170
956449992016,3cyclictest1008353-21kworker/u8:3+flush-259:009:45:220
956449992013,5cyclictest252-21systemd-journal12:30:000
956459991915,3cyclictest1123572-21kworker/u8:2+events_unbound10:00:293
956459991914,4cyclictest1190559-21kworker/u8:1+events_unbound11:05:183
956457991915,3cyclictest1205324-21kworker/u8:3+events_unbound11:15:172
956457991915,3cyclictest1153636-21kworker/u8:3+flush-259:010:50:252
956457991914,3cyclictest982997-21diskstats07:35:142
956457991913,2cyclictest1226474-21kworker/u8:0+events_unbound11:30:192
956453991915,3cyclictest1003108-21kworker/u8:1+flush-259:008:50:171
956453991914,3cyclictest1128027-21idleruntime-cro09:55:001
956453991912,5cyclictest1236699-21latency_hist11:40:001
956453991911,6cyclictest252-21systemd-journal11:22:411
956453991911,6cyclictest252-21systemd-journal11:22:411
956449991917,1cyclictest1008353-21kworker/u8:3+flush-259:008:25:150
956449991916,2cyclictest1112166-21kworker/u8:0+flush-259:010:10:170
956449991914,4cyclictest1159697-21apt-get10:25:130
956449991912,5cyclictest1116423-21ntpq09:40:220
956459991815,2cyclictest1277668-21kworker/u8:3+events_unbound12:25:203
956459991814,3cyclictest1190351-21kworker/u8:0+events_unbound11:10:263
956459991813,3cyclictest1064918-21turbostat08:55:003
956457991815,2cyclictest1226474-21kworker/u8:0+events_unbound12:20:152
956457991814,3cyclictest1226474-21kworker/u8:0+events_unbound12:30:042
956453991814,3cyclictest1159620-21base6410:25:021
956453991814,3cyclictest1003108-21kworker/u8:1+flush-259:009:05:161
956449991815,2cyclictest1217450-21kthreadcore11:20:180
956449991815,2cyclictest1217450-21kthreadcore11:20:170
956449991814,3cyclictest1003108-21kworker/u8:1+flush-259:008:50:220
956449991814,3cyclictest1003108-21kworker/u8:1+flush-259:008:10:230
956459991715,1cyclictest1205324-21kworker/u8:3+events_unbound11:30:293
956459991714,2cyclictest1008353-21kworker/u8:3+events_unbound09:25:143
956459991714,2cyclictest1003108-21kworker/u8:1+flush-259:007:55:213
956459991713,3cyclictest1236579-21kworker/u8:2+flush-259:012:10:153
956459991713,3cyclictest1003108-21kworker/u8:1+flush-259:008:00:183
956459991710,5cyclictest1184321-21ntpq10:45:253
956457991714,2cyclictest1205324-21kworker/u8:3+flush-259:011:25:182
956457991714,2cyclictest1205324-21kworker/u8:3+flush-259:011:25:182
956457991714,2cyclictest1144732-21kworker/u8:1+events_unbound10:15:122
956453991713,3cyclictest1236579-21kworker/u8:2+events_unbound12:15:171
956453991713,3cyclictest1205324-21kworker/u8:3+flush-259:011:20:001
956453991712,4cyclictest1293895-21cat12:35:001
956449991713,3cyclictest1190351-21kworker/u8:0+flush-259:011:10:220
956449991713,3cyclictest1144732-21kworker/u8:1+events_unbound10:25:010
956449991713,3cyclictest1008353-21kworker/u8:3+events_unbound08:40:130
956449991713,3cyclictest1003108-21kworker/u8:1+flush-259:009:15:140
956459991612,3cyclictest944873-21kworker/u8:1+flush-259:007:20:193
956459991612,3cyclictest1003108-21kworker/u8:1+flush-259:007:55:013
956459991612,2cyclictest1003108-21kworker/u8:1+flush-259:008:40:253
956457991614,2cyclictest967217-21kworker/u8:2+events_unbound08:05:232
956457991613,2cyclictest1236579-21kworker/u8:2+flush-259:011:50:222
956457991612,3cyclictest1226474-21kworker/u8:0+events_unbound12:03:192
956457991612,3cyclictest1144732-21kworker/u8:1+events_unbound10:35:242
956457991611,3cyclictest0-21swapper/208:05:012
956453991612,3cyclictest1226474-21kworker/u8:0+flush-259:011:40:181
956449991614,1cyclictest60350irq/132-enp0s29f1:rx-007:55:170
956449991613,2cyclictest60350irq/132-enp0s29f1:rx-009:37:320
956449991613,2cyclictest1190313-21cron10:54:590
956449991612,2cyclictest0-21swapper/012:35:000
95645999159,4cyclictest1090717-21apt-get09:20:013
956459991512,2cyclictest923303-21kworker/u8:0+flush-259:007:10:023
956459991511,3cyclictest1190559-21kworker/u8:1+flush-259:011:00:473
956457991512,2cyclictest1003108-21kworker/u8:1+events_unbound09:35:412
956457991511,3cyclictest500-21systemd-logind09:15:002
956457991511,3cyclictest1236579-21kworker/u8:2+events_unbound11:59:432
956457991510,3cyclictest252-21systemd-journal09:25:282
956449991513,2cyclictest60350irq/132-enp0s29f1:rx-010:45:310
956449991513,1cyclictest60350irq/132-enp0s29f1:rx-011:48:140
956449991513,1cyclictest60350irq/132-enp0s29f1:rx-011:15:000
956449991513,1cyclictest60350irq/132-enp0s29f1:rx-010:59:180
956449991513,1cyclictest60350irq/132-enp0s29f1:rx-007:20:240
956449991512,2cyclictest60350irq/132-enp0s29f1:rx-011:36:420
956449991512,2cyclictest60350irq/132-enp0s29f1:rx-008:55:120
956449991511,3cyclictest972860-21fschecks_time07:25:140
956449991511,3cyclictest1008353-21kworker/u8:3+events_unbound09:10:170
956449991510,4cyclictest987953-21kworker/u8:3+events_unbound07:50:210
956459991412,1cyclictest1153636-21kworker/u8:3+events_unbound10:30:213
956459991411,2cyclictest1153636-21kworker/u8:3+events_unbound10:40:203
956459991410,3cyclictest987924-21/usr/sbin/munin07:40:133
956459991410,3cyclictest923303-21kworker/u8:0+flush-259:007:35:063
95645799149,4cyclictest1190559-21kworker/u8:1+events_unbound11:04:052
956457991410,3cyclictest1003108-21kworker/u8:1+events_unbound09:49:422
956457991410,3cyclictest1003108-21kworker/u8:1+events_unbound08:13:432
95645399149,4cyclictest967217-21kworker/u8:2+flush-259:008:05:121
956453991411,2cyclictest1065599-21irqcore08:55:171
956453991411,2cyclictest0-21swapper/108:05:011
956453991410,3cyclictest1117867-21gpgv09:45:011
956449991412,1cyclictest60350irq/132-enp0s29f1:rx-010:15:260
956449991412,1cyclictest60350irq/132-enp0s29f1:rx-009:00:280
956449991412,1cyclictest60350irq/132-enp0s29f1:rx-008:35:150
956449991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:210
956449991412,1cyclictest60350irq/132-enp0s29f1:rx-007:16:570
956449991411,2cyclictest60350irq/132-enp0s29f1:rx-012:11:040
956449991411,2cyclictest60350irq/132-enp0s29f1:rx-010:05:210
956449991411,2cyclictest60350irq/132-enp0s29f1:rx-009:30:230
956449991410,3cyclictest923303-21kworker/u8:0+flush-259:007:35:170
95645999139,3cyclictest1205324-21kworker/u8:3+flush-259:011:37:163
95645999137,4cyclictest252-21systemd-journal07:50:003
95645999137,4cyclictest1084632-21ntpq09:10:203
956459991310,2cyclictest1277668-21kworker/u8:3+flush-259:012:20:163
956459991310,2cyclictest1014142-21irqcore08:05:153
95645399139,3cyclictest1153636-21kworker/u8:3+flush-259:010:50:001
956453991311,1cyclictest1102197-21cpuspeed_turbos09:30:141
95644999139,3cyclictest1003108-21kworker/u8:1+flush-259:008:00:120
95644999135,3cyclictest1028884-21munin-plugin-st08:20:000
956449991311,2cyclictest60350irq/132-enp0s29f1:rx-009:27:460
95645999129,2cyclictest1008353-21kworker/u8:3+events_unbound08:55:203
95645999128,3cyclictest1003108-21kworker/u8:1+flush-259:008:15:163
95645799129,2cyclictest944873-21kworker/u8:1+events_unbound07:24:592
95645399129,2cyclictest0-21swapper/108:50:001
95645399127,3cyclictest997992-21cron07:50:011
95644999129,2cyclictest60350irq/132-enp0s29f1:rx-008:45:160
95644999129,2cyclictest1216395-21kworker/u8:1+events_unbound11:25:140
95644999129,2cyclictest1216395-21kworker/u8:1+events_unbound11:25:130
956449991211,1cyclictest60350irq/132-enp0s29f1:rx-012:24:000
95645399119,1cyclictest0-21swapper/112:00:011
95645399118,1cyclictest0-21swapper/112:15:011
95645399117,3cyclictest1216395-21kworker/u8:1+flush-259:011:30:001
95645399117,3cyclictest1216395-21kworker/u8:1+flush-259:011:30:001
95645399114,5cyclictest1075226-21apt09:05:001
95644999119,1cyclictest60350irq/132-enp0s29f1:rx-012:05:210
95645399108,1cyclictest0-21swapper/109:40:001
95645399107,2cyclictest1278185-21cpuspeed_turbos12:20:131
9564539996,2cyclictest0-21swapper/110:35:191
9564539995,3cyclictest1123572-21kworker/u8:2+flush-259:010:00:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional