You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 23:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon Aug 17, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1102450995648,6cyclictest335557-21systemd-journal09:15:002
1102445994741,4cyclictest1404029-21latency_hist12:00:001
1102450994640,4cyclictest1118084-21gpgv07:25:002
1102438994640,4cyclictest1196261-21latency_hist08:40:010
1102450994238,3cyclictest1414732-21kworker/u8:2+flush-259:012:25:002
1102445994237,3cyclictest1388808-21apt-get11:45:131
1102455994137,3cyclictest1112774-21kworker/u8:0+flush-259:007:45:253
1102455994136,3cyclictest1170438-21latency_hist08:15:013
1102455994034,4cyclictest1342071-21/usr/sbin/munin11:00:143
1102445993934,3cyclictest1320966-21/usr/sbin/munin10:40:161
1102450993832,4cyclictest1243158-21latency_hist09:25:002
1102445993833,3cyclictest1440564-21/usr/sbin/munin12:35:131
1102438993835,2cyclictest1212137-21kworker/u8:2+flush-259:009:05:150
1102455993731,4cyclictest1284494-21latency_hist10:05:013
1102438993734,2cyclictest1243390-21kworker/u8:4+events_unbound11:00:250
1102438993633,2cyclictest1258964-21apt-get09:40:150
1102450993523,10cyclictest1331721-21/usr/sbin/munin10:50:162
1102438993530,4cyclictest1216416-21ntpq08:55:200
1102438993528,5cyclictest1418787-21ntpq12:10:220
1102455993429,3cyclictest1388565-21kworker/u8:1+events_unbound12:30:173
1102450993428,4cyclictest1440330-21latency_hist12:35:012
1102445993429,4cyclictest1096749-21kworker/u8:2+flush-259:007:35:001
1102445993429,4cyclictest1096749-21kworker/u8:2+flush-259:007:35:001
1102455993328,2cyclictest1289691-21kworker/u8:2+events_unbound10:15:003
1102455993327,4cyclictest335557-21systemd-journal10:51:233
1102445993327,4cyclictest1310374-21latency_hist10:30:001
1102445993320,4cyclictest1233105-21apt-get09:15:111
1102455993229,2cyclictest1212137-21kworker/u8:2+flush-259:009:10:193
1102438993229,2cyclictest1325741-21kworker/u8:0+flush-259:010:55:010
1102438993229,2cyclictest1243390-21kworker/u8:4+flush-259:009:35:160
1102438993228,3cyclictest1430233-21apt-get12:25:130
1102438993227,4cyclictest335557-21systemd-journal08:10:000
1102455993125,4cyclictest1289664-21latency_hist10:10:003
1102450993118,11cyclictest1128684-21latency_hist07:35:012
1102450993118,11cyclictest1128684-21latency_hist07:35:012
1102438993128,2cyclictest1112774-21kworker/u8:0+events_unbound07:45:270
1102455993025,4cyclictest1362598-21latency_hist11:20:013
1102445993026,3cyclictest1253482-21kworker/u8:1+flush-259:011:10:241
1102438993024,5cyclictest1331083-21gpgv10:50:010
1102455992926,2cyclictest1212137-21kworker/u8:2+flush-259:009:15:163
1102445992925,3cyclictest1435432-21apt-get12:30:111
1102438992926,2cyclictest1382992-21kworker/u8:0+flush-259:011:50:240
1102438992926,2cyclictest1325741-21kworker/u8:0+events_unbound11:15:240
1102438992925,3cyclictest1238261-21apt-get09:20:100
1102438992925,3cyclictest1081144-21kworker/u8:1+events_unbound07:10:230
1102438992924,4cyclictest1378136-21latency_hist11:35:010
1102455992824,3cyclictest1144629-21kworker/u8:3+events_unbound09:20:143
1102455992823,4cyclictest1414732-21kworker/u8:2+flush-259:012:15:243
1102450992824,3cyclictest1096749-21kworker/u8:2+flush-259:007:35:162
1102445992825,2cyclictest1253482-21kworker/u8:1+flush-259:010:40:011
1102445992825,2cyclictest1243390-21kworker/u8:4+flush-259:011:40:161
1102445992825,2cyclictest1144629-21kworker/u8:3+events_unbound09:30:131
1102438992825,2cyclictest1159902-21kworker/u8:1+events_unbound08:05:010
1102438992825,2cyclictest1112774-21kworker/u8:0+flush-259:007:50:270
1102438992824,3cyclictest1325741-21kworker/u8:0+events_unbound10:55:230
1102438992824,3cyclictest1284752-21apt-get10:05:130
1102438992824,3cyclictest1253482-21kworker/u8:1+events_unbound09:45:140
1102455992724,2cyclictest1253482-21kworker/u8:1+flush-259:011:10:263
1102455992723,3cyclictest1118574-21apt-get07:25:123
1102455992723,3cyclictest1118574-21apt-get07:25:123
1102455992723,2cyclictest1112774-21kworker/u8:0+events_unbound07:40:193
1102455992721,4cyclictest335557-21systemd-journal07:38:523
1102450992720,5cyclictest335557-21systemd-journal08:40:012
1102438992723,3cyclictest335557-21systemd-journal08:43:520
1102438992723,3cyclictest1382992-21kworker/u8:0+events_unbound11:40:150
1102438992723,3cyclictest1212137-21kworker/u8:2+flush-259:009:15:210
1102455992622,3cyclictest335557-21systemd-journal08:07:523
1102455992621,4cyclictest335557-21systemd-journal12:40:003
1102450992621,4cyclictest335557-21systemd-journal09:50:002
1102445992621,3cyclictest1149789-21latency_hist07:55:011
1102438992622,3cyclictest1414732-21kworker/u8:2+events_unbound12:35:510
1102438992622,3cyclictest1289691-21kworker/u8:2+flush-259:010:10:280
1102438992622,3cyclictest1159902-21kworker/u8:1+events_unbound08:22:530
1102455992522,2cyclictest1159902-21kworker/u8:1+flush-259:009:05:213
1102455992520,4cyclictest1435172-21latency_hist12:30:003
1102450992520,3cyclictest1144629-21kworker/u8:3+flush-259:008:30:152
1102445992521,3cyclictest1382992-21kworker/u8:0+flush-259:011:50:211
1102438992521,3cyclictest1404262-21kworker/u8:3+events_unbound12:02:240
1102438992521,3cyclictest1388565-21kworker/u8:1+events_unbound11:45:240
1102438992521,3cyclictest1243390-21kworker/u8:4+events_unbound10:30:300
1102438992521,3cyclictest1112774-21kworker/u8:0+events_unbound07:56:530
1102450992418,4cyclictest1248790-21diskstats09:30:152
1102445992420,3cyclictest1253482-21kworker/u8:1+flush-259:009:50:141
1102445992419,3cyclictest1196508-21/usr/sbin/munin08:40:151
1102438992421,2cyclictest1253482-21kworker/u8:1+events_unbound11:10:220
1102438992420,3cyclictest1159902-21kworker/u8:1+flush-259:009:00:150
1102438992420,3cyclictest1112774-21kworker/u8:0+events_unbound07:27:530
1102438992420,3cyclictest1112774-21kworker/u8:0+events_unbound07:27:530
110243899240,23cyclictest1-21systemd09:52:140
1102455992320,2cyclictest1112774-21kworker/u8:0+events_unbound07:50:203
1102455992319,3cyclictest1253482-21kworker/u8:1+events_unbound10:48:243
1102455992319,3cyclictest1243390-21kworker/u8:4+flush-259:009:30:173
1102445992318,4cyclictest1404262-21kworker/u8:3+flush-259:012:00:271
1102438992320,2cyclictest1243390-21kworker/u8:4+flush-259:010:25:240
1102438992320,2cyclictest1159902-21kworker/u8:1+events_unbound08:30:110
1102438992319,3cyclictest1081144-21kworker/u8:1+events_unbound07:20:190
1102438992318,4cyclictest1413590-21ntpq12:05:220
1102455992219,2cyclictest1243390-21kworker/u8:4+flush-259:010:20:203
1102455992218,3cyclictest1362626-21kworker/u8:5+events_unbound11:35:213
1102455992218,3cyclictest1159902-21kworker/u8:1+flush-259:008:55:163
1102455992217,4cyclictest1362838-21/usr/sbin/munin11:20:193
1102455992217,3cyclictest1382992-21kworker/u8:0+events_unbound11:50:183
1102450992217,3cyclictest1107815-21apt-get07:15:122
1102445992215,6cyclictest1133866-21latency_hist07:40:011
1102445992214,6cyclictest335557-21systemd-journal11:30:001
1102438992218,3cyclictest1382992-21kworker/u8:0+events_unbound11:55:230
1102438992217,3cyclictest1133862-21apt-get07:40:000
1102455992118,2cyclictest1243390-21kworker/u8:4+flush-259:010:20:003
1102455992117,3cyclictest1253482-21kworker/u8:1+events_unbound09:40:143
1102455992117,3cyclictest1144629-21kworker/u8:3+events_unbound08:41:523
1102455992116,3cyclictest1352217-21latency_hist11:10:003
1102450992117,3cyclictest335557-21systemd-journal12:26:222
1102450992117,3cyclictest1201994-21gpgv08:45:012
1102450992115,4cyclictest335557-21systemd-journal10:01:132
1102450992114,6cyclictest1424317-21unixbench_multi12:15:252
1102445992118,2cyclictest335557-21systemd-journal09:00:001
1102445992117,3cyclictest1274179-21kworker/u8:0+flush-259:009:55:161
1102445992116,4cyclictest1243390-21kworker/u8:4+flush-259:009:35:251
1102445992116,4cyclictest1170683-21/usr/sbin/munin08:15:241
1102438992118,3cyclictest1243390-21kworker/u8:4+flush-259:011:35:230
1102438992118,2cyclictest1362624-21kworker/u8:2+events_unbound11:25:440
1102438992117,3cyclictest1253482-21kworker/u8:1+events_unbound09:55:210
1102438992117,3cyclictest1144629-21kworker/u8:3+events_unbound09:30:010
1102438992117,3cyclictest1081144-21kworker/u8:1+events_unbound07:15:230
1102455992016,3cyclictest1388565-21kworker/u8:1+flush-259:012:04:593
1102455992016,3cyclictest1253482-21kworker/u8:1+flush-259:009:50:153
1102455992016,3cyclictest1243390-21kworker/u8:4+events_unbound11:30:143
1102455992016,3cyclictest1081144-21kworker/u8:1+flush-259:007:10:203
1102455992014,4cyclictest1180753-21latency_hist08:25:013
1102445992016,3cyclictest1253482-21kworker/u8:1+flush-259:010:45:261
1102445992015,4cyclictest1357634-21/usr/sbin/munin11:15:191
1102438992016,3cyclictest1159902-21kworker/u8:1+flush-259:008:25:170
1102438992016,3cyclictest1112774-21kworker/u8:0+events_unbound07:33:540
1102438992016,3cyclictest1112774-21kworker/u8:0+events_unbound07:33:540
1102455991916,2cyclictest1325741-21kworker/u8:0+flush-259:010:55:203
1102455991916,2cyclictest1243390-21kworker/u8:4+events_unbound11:40:183
1102455991916,2cyclictest1243390-21kworker/u8:4+events_unbound10:43:253
1102455991916,2cyclictest1111801-21ntp_kernel_err07:15:213
1102455991915,3cyclictest1274179-21kworker/u8:0+flush-259:009:55:263
110245099198,9cyclictest0-21swapper/208:45:002
110245099197,11cyclictest1243629-21fschecks_count09:25:142
1102450991915,3cyclictest1207236-21/usr/sbin/munin08:50:192
1102450991915,3cyclictest1159902-21kworker/u8:1+flush-259:008:55:052
1102445991916,2cyclictest1192374-21kthreadcore08:35:171
1102445991915,3cyclictest1159902-21kworker/u8:1+flush-259:009:01:431
1102445991914,4cyclictest1243390-21kworker/u8:4+flush-259:010:00:251
1102445991912,5cyclictest1377216-21ntpq11:30:211
1102438991916,2cyclictest1144629-21kworker/u8:3+events_unbound08:45:220
1102455991815,2cyclictest1144629-21kworker/u8:3+events_unbound08:30:533
1102455991814,3cyclictest1388565-21kworker/u8:1+events_unbound12:05:483
1102450991814,3cyclictest1243390-21kworker/u8:4+flush-259:011:35:222
1102450991813,3cyclictest335557-21systemd-journal07:57:522
110245099180,16cyclictest1307997-21kthreadcore10:25:182
110244599182,8cyclictest1367813-21sh11:25:011
1102445991814,3cyclictest1336907-21/usr/sbin/munin10:55:131
1102445991813,3cyclictest335557-21systemd-journal11:05:001
1102438991814,3cyclictest1112774-21kworker/u8:0+flush-259:007:44:520
1102438991813,4cyclictest1295105-21apt-get10:15:110
1102455991714,2cyclictest1314570-21needreboot10:30:213
1102455991713,3cyclictest1159902-21kworker/u8:1+events_unbound08:15:093
1102455991713,3cyclictest1144629-21kworker/u8:3+flush-259:009:00:243
110245099175,1cyclictest0-21swapper/212:00:002
110245099175,1cyclictest0-21swapper/207:40:182
110245099171,3cyclictest1258678-21cron09:39:592
110245099171,3cyclictest1149775-21idleruntime-cro07:55:002
1102450991712,3cyclictest335557-21systemd-journal09:55:002
1102450991711,4cyclictest335557-21systemd-journal10:00:012
1102450991711,4cyclictest1144599-21latency_hist07:50:012
1102445991712,3cyclictest335557-21systemd-journal07:24:301
1102445991710,5cyclictest335557-21systemd-journal10:35:001
1102438991714,2cyclictest1325741-21kworker/u8:0+flush-259:011:05:200
1102438991713,3cyclictest1414732-21kworker/u8:2+flush-259:012:20:180
1102438991713,3cyclictest1159902-21kworker/u8:1+flush-259:008:10:180
1102455991613,2cyclictest1180997-21/usr/sbin/munin08:25:213
1102455991613,2cyclictest1081144-21kworker/u8:1+events_unbound07:22:533
1102455991612,3cyclictest1159902-21kworker/u8:1+flush-259:008:39:033
110245099165,1cyclictest0-21swapper/211:25:012
110245099164,3cyclictest1122316-21cut07:25:192
110245099164,3cyclictest1122316-21cut07:25:192
110245099161,4cyclictest1352898-21sed11:10:172
110245099161,3cyclictest1288690-21needreboot10:05:202
1102450991613,2cyclictest1243390-21kworker/u8:4+flush-259:011:30:232
1102450991612,2cyclictest0-21swapper/210:50:002
1102450991610,4cyclictest335557-21systemd-journal09:43:122
110245099160,4cyclictest1290050-21tr10:10:172
110245099160,4cyclictest1225075-21kthreadcore09:05:172
110245099160,3cyclictest335557-21systemd-journal12:40:002
1102438991613,2cyclictest60350irq/132-enp0s29f1:rx-011:20:180
1102438991613,2cyclictest60350irq/132-enp0s29f1:rx-008:15:160
1102438991612,3cyclictest1388565-21kworker/u8:1+events_unbound12:30:270
1102438991612,3cyclictest1274179-21kworker/u8:0+events_unbound10:00:170
1102455991512,2cyclictest1253482-21kworker/u8:1+flush-259:009:45:193
1102455991511,3cyclictest1425053-21apt-get12:20:123
1102455991511,3cyclictest1388565-21kworker/u8:1+events_unbound11:46:233
110245099155,1cyclictest0-21swapper/212:05:012
110245099154,1cyclictest0-21swapper/211:05:002
110245099152,3cyclictest1165222-21cron08:10:002
110245099151,4cyclictest1383685-21tr11:40:122
110245099151,3cyclictest1180754-21turbostat.cron08:25:002
1102450991513,1cyclictest0-21swapper/212:14:592
110245099151,2cyclictest1352214-21cron11:10:012
1102450991512,2cyclictest1159902-21kworker/u8:1+flush-259:008:10:162
1102450991511,3cyclictest1388565-21kworker/u8:1+flush-259:012:05:262
1102450991511,3cyclictest1159902-21kworker/u8:1+flush-259:008:25:272
110245099150,4cyclictest1299512-21grep10:15:242
110245099150,3cyclictest1107096-21tail07:10:262
1102445991512,2cyclictest1118995-21irqcore07:25:171
1102445991512,2cyclictest1118995-21irqcore07:25:171
1102445991511,2cyclictest0-21swapper/109:50:001
1102438991512,2cyclictest60350irq/132-enp0s29f1:rx-010:20:150
1102438991511,3cyclictest509-21rs:main0
1102438991511,3cyclictest1388565-21kworker/u8:1+events_unbound12:19:240
1102438991511,3cyclictest1289691-21kworker/u8:2+flush-259:010:40:260
1102438991511,3cyclictest1159902-21kworker/u8:1+flush-259:008:50:160
110245599149,4cyclictest1123749-21/usr/sbin/munin07:30:143
110245599149,4cyclictest1123749-21/usr/sbin/munin07:30:143
1102455991412,1cyclictest1159902-21kworker/u8:1+events_unbound08:50:533
1102455991411,2cyclictest1382992-21kworker/u8:0+flush-259:011:55:233
110245099141,3cyclictest1362627-21systemd-run11:20:002
1102450991410,2cyclictest0-21swapper/210:40:002
110245099140,4cyclictest500-21systemd-logind10:35:002
110245099140,4cyclictest1398887-21idleruntime-cro11:55:002
110245099140,4cyclictest1372215-21/usr/sbin/munin11:25:232
110245099140,4cyclictest1337154-21/usr/sbin/munin10:55:132
110245099140,4cyclictest1236740-21grep09:15:192
110245099140,3cyclictest1320713-21cron10:40:012
110245099140,3cyclictest1160082-21sh08:05:002
110244599148,4cyclictest335557-21systemd-journal11:40:001
110244599148,4cyclictest1112774-21kworker/u8:0+flush-259:007:45:041
1102445991411,2cyclictest1243168-21munin-plugin-st09:24:591
1102445991410,3cyclictest1144629-21kworker/u8:3+flush-259:008:55:001
1102438991411,2cyclictest1316212-21irqcore10:35:170
110245599139,3cyclictest1243390-21kworker/u8:4+events_unbound11:29:243
110245599139,3cyclictest1112774-21kworker/u8:0+events_unbound08:00:163
110245099131,3cyclictest1302632-21grep10:20:172
110245099130,3cyclictest1389104-21grep11:45:152
110245099130,3cyclictest1221133-21grep09:00:192
110244599138,3cyclictest1334325-21cut10:50:201
110245599128,3cyclictest1243390-21kworker/u8:4+flush-259:010:35:163
110245599128,3cyclictest1243390-21kworker/u8:4+events_unbound09:36:143
110245599126,4cyclictest1206981-21cron08:50:003
110245099120,5cyclictest1174752-21ntpq08:15:232
110244599129,2cyclictest1295186-21cpuspeed_turbos10:15:121
110244599127,3cyclictest1202098-21/usr/sbin/munin08:45:261
110244599126,4cyclictest1-21systemd12:15:001
110244599125,4cyclictest1421741-21cut12:15:181
110245599118,2cyclictest1415176-21irqcore12:10:163
110244599119,1cyclictest0-21swapper/110:10:181
110244599119,1cyclictest0-21swapper/110:05:181
110244599119,1cyclictest0-21swapper/108:05:241
110244599117,2cyclictest0-21swapper/111:05:211
110244599114,6cyclictest335557-21systemd-journal10:20:261
110244599107,2cyclictest1222828-21cpuspeed_turbos09:05:121
110244599107,2cyclictest0-21swapper/109:10:211
110244599107,2cyclictest0-21swapper/108:25:151
110244599107,2cyclictest0-21swapper/108:00:171
110244599106,2cyclictest1179830-21ntpq08:20:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional