You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 20:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed Dec 03, 2025 12:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
130872921390,2sleep3586-21kworker/3:2+events08:31:113
1226668997263,7cyclictest1180255-21kworker/u8:3+flush-8:007:26:450
1226668996961,7cyclictest1476887-21kworker/u8:1+flush-8:012:11:210
1226670996761,4cyclictest1522444-21kworker/u8:2+flush-8:012:36:241
1226668996655,8cyclictest1243398-21latency_hist07:25:010
1226670996156,3cyclictest1476887-21kworker/u8:1+events_unbound11:51:111
1226670996151,4cyclictest1457848-21kworker/u8:0+flush-8:011:11:231
1226670996054,5cyclictest1268449-21kworker/u8:3+events_unbound09:11:181
1226670996053,5cyclictest1313777-21kworker/u8:0+events_unbound09:16:171
1226670995955,3cyclictest1180255-21kworker/u8:3+events_unbound07:16:171
1226670995954,4cyclictest1253184-21kworker/u8:2+flush-8:008:01:271
1226670995953,5cyclictest1476887-21kworker/u8:1+events_unbound11:46:261
1226670995953,5cyclictest1412847-21kworker/u8:1+events_unbound10:26:191
1226670995953,5cyclictest1397796-21kworker/u8:2+flush-8:010:56:201
1226668995952,5cyclictest1262905-21kworker/u8:0+flush-8:007:56:200
1226673995850,5cyclictest282-21systemd-journal08:40:012
1226668995848,6cyclictest1551856-21latency_hist12:35:010
12887412570,4chrt0-21swapper/208:11:072
1226670995752,4cyclictest1476887-21kworker/u8:1+events_unbound12:11:121
1226670995752,4cyclictest1253184-21kworker/u8:2+events_unbound07:36:201
1226670995750,5cyclictest1180255-21kworker/u8:3+flush-8:007:11:191
1226670995649,6cyclictest1313777-21kworker/u8:0+events_unbound08:41:171
1226670995649,5cyclictest1507540-21kworker/u8:0+flush-8:012:16:181
1226670995550,4cyclictest1378206-21kworker/u8:4+flush-8:009:41:181
1226670995550,4cyclictest1268449-21kworker/u8:3+events_unbound10:06:191
1226670995550,3cyclictest1268449-21kworker/u8:3+flush-8:009:21:191
1226670995548,5cyclictest1476887-21kworker/u8:1+events_unbound11:31:091
1226668995549,4cyclictest1253184-21kworker/u8:2+flush-8:007:46:180
1226670995448,5cyclictest1313777-21kworker/u8:0+events_unbound08:56:291
1226670995448,4cyclictest1397796-21kworker/u8:2+flush-8:011:16:161
1226670995448,4cyclictest1397796-21kworker/u8:2+flush-8:011:16:161
1226670995447,5cyclictest1253184-21kworker/u8:2+flush-8:008:06:201
1226668995448,4cyclictest1313777-21kworker/u8:0+flush-8:008:36:210
1226673995347,4cyclictest1457848-21kworker/u8:0+flush-8:011:01:202
1226670995349,3cyclictest1313777-21kworker/u8:0+events_unbound08:46:221
1226670995348,3cyclictest1298399-21kworker/u8:4+events_unbound08:31:171
1226670995347,5cyclictest1397796-21kworker/u8:2+events_unbound10:41:271
1226668995347,4cyclictest1268449-21kworker/u8:3+events_unbound09:21:170
1226668995345,6cyclictest1253184-21kworker/u8:2+flush-8:007:51:170
1226670995246,5cyclictest1471598-21kworker/u8:3+flush-8:011:26:271
1226670995246,5cyclictest1471598-21kworker/u8:3+flush-8:011:26:271
1226670995246,5cyclictest1268449-21kworker/u8:3+flush-8:009:06:081
1226670995246,4cyclictest1522444-21kworker/u8:2+events_unbound12:31:081
1226670995246,4cyclictest1412847-21kworker/u8:1+events_unbound10:16:251
1226670995246,4cyclictest1313777-21kworker/u8:0+events_unbound09:01:121
1226668995247,4cyclictest1298399-21kworker/u8:4+flush-8:008:21:250
1226668995246,4cyclictest1378206-21kworker/u8:4+flush-8:009:46:260
1226673995145,4cyclictest1497115-21kworker/u8:2+flush-8:011:41:222
1226670995146,4cyclictest1397796-21kworker/u8:2+events_unbound10:46:211
1226670995146,3cyclictest1507540-21kworker/u8:0+events_unbound12:01:241
1226670995145,5cyclictest1397796-21kworker/u8:2+events_unbound10:36:271
1226668995139,10cyclictest1412847-21kworker/u8:1+flush-8:010:31:210
1226670995046,3cyclictest1412847-21kworker/u8:1+events_unbound10:21:191
1226670995044,5cyclictest1537293-21kworker/u8:3+flush-8:012:21:191
1226670995044,5cyclictest1412847-21kworker/u8:1+events_unbound10:51:221
1226670995044,5cyclictest1268449-21kworker/u8:3+flush-8:008:36:191
1226670995043,5cyclictest1457848-21kworker/u8:0+events_unbound11:01:221
1226668995045,4cyclictest1507540-21kworker/u8:0+flush-8:012:16:210
1226670994942,5cyclictest1476887-21kworker/u8:1+flush-8:012:06:221
1226668994944,3cyclictest1268449-21kworker/u8:3+events_unbound09:06:210
1226668994942,5cyclictest1507540-21kworker/u8:0+flush-8:011:56:180
1226668994939,8cyclictest282-21systemd-journal10:30:010
1226670994844,3cyclictest1283491-21kworker/u8:1+events_unbound08:11:201
1226670994842,5cyclictest1372983-21kworker/u8:0+events_unbound09:36:111
1226670994837,9cyclictest282-21systemd-journal08:00:001
1226668994843,3cyclictest1397796-21kworker/u8:2+flush-8:010:51:180
1226673994736,9cyclictest282-21systemd-journal08:42:162
1226670994742,4cyclictest1253184-21kworker/u8:2+flush-8:008:51:111
1226670994742,3cyclictest1476887-21kworker/u8:1+flush-8:012:26:211
1226670994741,5cyclictest1457848-21kworker/u8:0+events_unbound11:36:181
1226670994741,5cyclictest1233746-21kworker/u8:0+events_unbound07:21:081
1226668994743,3cyclictest1372983-21kworker/u8:0+flush-8:009:36:260
1226668994742,4cyclictest1233746-21kworker/u8:0+events_unbound07:17:000
1226670994640,4cyclictest1293382-21kworker/u8:0+flush-8:008:21:141
1226668994642,3cyclictest1537293-21kworker/u8:3+flush-8:012:21:250
1226668994641,3cyclictest1442516-21kworker/u8:3+flush-8:010:46:150
1226668994640,4cyclictest1497115-21kworker/u8:2+events_unbound11:41:180
1226668994636,8cyclictest1319080-21diskstats08:41:070
1226673994539,4cyclictest1476887-21kworker/u8:1+flush-8:011:51:182
1226670994540,3cyclictest1293382-21kworker/u8:0+events_unbound08:26:241
1226668994534,4cyclictest1268449-21kworker/u8:3+flush-8:008:16:160
1226670994440,3cyclictest1268449-21kworker/u8:3+events_unbound09:51:091
1226668994440,3cyclictest1333600-21kworker/u8:1+flush-8:009:26:090
1226668994439,4cyclictest1397796-21kworker/u8:2+flush-8:011:16:210
1226668994439,4cyclictest1397796-21kworker/u8:2+flush-8:011:16:210
1226668994438,5cyclictest1313777-21kworker/u8:0+flush-8:008:46:180
1226668994438,4cyclictest1397796-21kworker/u8:2+flush-8:010:11:200
1226668994437,5cyclictest1412847-21kworker/u8:1+events_unbound10:41:190
1226668994435,6cyclictest1522181-21latency_hist12:05:010
1226670994337,4cyclictest1507540-21kworker/u8:0+events_unbound11:56:111
1226670994337,4cyclictest1457848-21kworker/u8:0+events_unbound11:06:211
1226670994336,5cyclictest282-21systemd-journal10:35:011
1226670994336,5cyclictest1333600-21kworker/u8:1+flush-8:009:31:241
1226668994338,3cyclictest1457848-21kworker/u8:0+flush-8:011:06:160
1226668994338,3cyclictest1412847-21kworker/u8:1+flush-8:010:21:170
1226668994338,3cyclictest1378206-21kworker/u8:4+events_unbound09:45:000
1226670994238,3cyclictest1246138-21kworker/u8:1+flush-8:007:31:241
1226668994233,7cyclictest1308892-21diskstats08:31:120
1226668994232,8cyclictest282-21systemd-journal12:40:000
1226670994136,4cyclictest1378206-21kworker/u8:4+events_unbound09:46:141
1226670994136,4cyclictest1293382-21kworker/u8:0+events_unbound08:16:221
1226668994134,5cyclictest1397796-21kworker/u8:2+events_unbound10:16:230
1226670994034,4cyclictest1180255-21kworker/u8:3+flush-8:007:26:411
1226668994036,3cyclictest1397796-21kworker/u8:2+flush-8:010:01:350
1226668994036,3cyclictest1397796-21kworker/u8:2+flush-8:010:01:340
1226668994035,3cyclictest1471598-21kworker/u8:3+events_unbound11:26:220
1226668994035,3cyclictest1471598-21kworker/u8:3+events_unbound11:26:220
1226668994031,2cyclictest161rcu_preempt11:05:000
1226668993934,3cyclictest1337460-21ntpq08:56:270
1226673993833,4cyclictest1333600-21kworker/u8:1+flush-8:009:26:202
1226673993830,7cyclictest1537293-21kworker/u8:3+events_unbound12:26:122
1226670993832,5cyclictest1253184-21kworker/u8:2+events_unbound07:46:071
1226668993834,3cyclictest1537293-21kworker/u8:3+events_unbound12:26:180
1226668993834,3cyclictest1268449-21kworker/u8:3+events_unbound09:01:260
1226673993732,4cyclictest1246138-21kworker/u8:1+flush-8:007:31:222
1226668993733,3cyclictest1283491-21kworker/u8:1+flush-8:008:11:210
1226668993733,3cyclictest1253184-21kworker/u8:2+events_unbound08:06:170
1226668993731,4cyclictest1497115-21kworker/u8:2+events_unbound11:46:160
1226670993632,3cyclictest1397796-21kworker/u8:2+flush-8:010:11:141
1226670993632,3cyclictest1180255-21kworker/u8:3+events_unbound07:41:331
1226670993631,4cyclictest1497115-21kworker/u8:2+flush-8:011:41:251
1226668993632,3cyclictest1293382-21kworker/u8:0+flush-8:008:26:230
1226668993631,4cyclictest1262905-21kworker/u8:0+events_unbound07:41:430
1226668993631,3cyclictest1180255-21kworker/u8:3+events_unbound07:11:120
1226670993530,4cyclictest1471598-21kworker/u8:3+events_unbound11:21:231
1226670993530,4cyclictest1471598-21kworker/u8:3+events_unbound11:21:221
1226670993527,6cyclictest1262905-21kworker/u8:0+events_unbound07:51:071
1226668993531,3cyclictest1268449-21kworker/u8:3+events_unbound10:06:280
1226668993530,4cyclictest1388085-21kworker/u8:0+events_unbound09:56:310
1226668993530,4cyclictest1388085-21kworker/u8:0+events_unbound09:56:310
12264322354,9sleep20-21swapper/207:09:122
1226670993427,5cyclictest1388085-21kworker/u8:0+flush-8:010:01:101
1226670993427,5cyclictest1388085-21kworker/u8:0+flush-8:010:01:101
1226668993430,3cyclictest1476887-21kworker/u8:1+events_unbound11:31:170
1226668993429,3cyclictest1268449-21kworker/u8:3+events_unbound09:51:230
1226673993329,3cyclictest1372983-21kworker/u8:0+flush-8:009:36:202
1226668993327,4cyclictest1278696-21grep08:01:120
1226673993228,3cyclictest1180255-21kworker/u8:3+events_unbound07:41:372
1226673993227,4cyclictest1537293-21kworker/u8:3+events_unbound12:21:152
1226673993227,4cyclictest1333600-21kworker/u8:1+flush-8:009:31:232
1226670993227,4cyclictest1388085-21kworker/u8:0+flush-8:009:56:271
1226670993227,4cyclictest1388085-21kworker/u8:0+flush-8:009:56:271
1226668993228,3cyclictest1436368-21tail10:36:170
1226668993227,3cyclictest1472112-21unixbench_singl11:11:280
1226673993127,3cyclictest1522444-21kworker/u8:2+events_unbound12:31:232
1226673993125,4cyclictest1476887-21kworker/u8:1+events_unbound12:06:262
1226673993124,5cyclictest1333365-21cron08:55:002
1226668993126,4cyclictest1476887-21kworker/u8:1+events_unbound11:21:190
1226668993126,4cyclictest1476887-21kworker/u8:1+events_unbound11:21:190
1226668993126,4cyclictest1457848-21kworker/u8:0+flush-8:011:36:190
1226668993126,4cyclictest1333600-21kworker/u8:1+flush-8:009:31:200
1226668993124,6cyclictest1452786-21aten2_r5power_c10:56:050
12434592300,5chrt451irq_work/307:25:013
1226673993025,4cyclictest1397796-21kworker/u8:2+events_unbound10:11:222
1226673993025,4cyclictest1298399-21kworker/u8:4+flush-8:008:31:232
1226673993024,4cyclictest1457359-21apt-get11:00:002
1226673993024,4cyclictest1268449-21kworker/u8:3+events_unbound10:01:162
1226673993024,4cyclictest1268449-21kworker/u8:3+events_unbound10:01:162
1226673993020,7cyclictest1387868-21awk09:50:002
1226668993027,2cyclictest1349505-21kthreadcore09:11:140
1226668993025,4cyclictest1268449-21kworker/u8:3+events_unbound08:51:270
122666899301,27cyclictest610-21sshd07:32:160
1226673992924,3cyclictest1516816-21unixbench_multi11:56:242
1226670992924,3cyclictest1333600-21kworker/u8:1+events_unbound09:26:091
1226668992924,4cyclictest1476887-21kworker/u8:1+flush-8:012:06:230
1226673992824,3cyclictest1298399-21kworker/u8:4+events_unbound08:26:162
1226673992822,4cyclictest1477261-21unixbench_multi11:16:312
1226673992822,4cyclictest1477261-21unixbench_multi11:16:312
1226668992825,2cyclictest1253184-21kworker/u8:2+events_unbound07:36:130
1226673992722,3cyclictest1358500-21aten2_r5power_c09:21:062
1226673992719,6cyclictest282-21systemd-journal09:55:002
1226668992725,1cyclictest1268449-21kworker/u8:3+events_unbound09:15:180
1226673992622,3cyclictest0-21swapper/211:40:012
1226673992621,4cyclictest1268449-21kworker/u8:3+flush-8:009:01:212
1226673992617,7cyclictest282-21systemd-journal08:16:162
1226673992617,7cyclictest282-21systemd-journal07:15:142
1226673992617,7cyclictest1488152-21fschecks_count11:31:082
1226673992616,8cyclictest282-21systemd-journal12:39:252
1226673992522,2cyclictest1427416-21ls10:30:012
1226673992520,4cyclictest1397796-21kworker/u8:2+flush-8:010:36:252
1226673992519,4cyclictest1283652-21aten2_r5power_e08:06:062
1226673992517,6cyclictest1443183-21fschecks_time10:46:102
1226673992517,6cyclictest1348893-21fschecks_time09:11:092
1226673992516,7cyclictest1263851-21fschecks_time07:46:092
1226673992421,2cyclictest1413182-21cpuspeed_turbos10:16:112
1226673992420,3cyclictest1246138-21kworker/u8:1+events_unbound07:26:462
1226673992419,3cyclictest1527606-21aten2_r5power_v12:11:092
1226673992418,4cyclictest1478525-21irqcore11:21:132
1226673992418,4cyclictest1478525-21irqcore11:21:132
1226673992416,7cyclictest1239143-21fschecks_time07:21:102
1226668992420,3cyclictest1497115-21kworker/u8:2+events_unbound11:50:030
12219712245,12sleep10-21swapper/107:06:061
15316842230,4sleep3481ktimers/312:11:273
1226673992320,2cyclictest1403324-21fschecks_count10:06:092
1226673992318,4cyclictest1253184-21kworker/u8:2+events_unbound07:36:262
1226673992318,3cyclictest1507307-21turbostat.cron11:50:012
1226673992318,3cyclictest1393181-21aten2_r5power_c09:56:092
1226673992318,3cyclictest1393181-21aten2_r5power_c09:56:082
1226673992318,3cyclictest0-21swapper/208:05:012
1226673992317,4cyclictest1521797-21unixbench_multi12:01:262
1226673992317,4cyclictest1427987-21cpuspeed_turbos10:31:082
1226673992317,4cyclictest1348232-21cron09:10:002
1226673992216,5cyclictest1327497-21needreboot08:46:162
1226673992214,6cyclictest1448176-21fschecks_time10:51:122
1226673992210,4cyclictest0-21swapper/211:11:232
122667399212,4cyclictest0-21swapper/209:41:152
1226673992118,2cyclictest1358113-21idleruntime-cro09:20:002
1226673992117,3cyclictest1298759-21fschecks_count08:21:092
1226673992113,7cyclictest1462965-21aten2_r5power_c11:06:062
1226673992112,8cyclictest1277137-21needreboot07:56:162
1226673992112,7cyclictest1442153-21unixbench_singl10:41:262
1226673992015,3cyclictest0-21swapper/212:16:252
122667399199,8cyclictest282-21systemd-journal08:56:152
1226673991915,2cyclictest0-21swapper/210:21:122
1226673991914,3cyclictest1268744-21cpuspeed_turbos07:51:072
12219152183,8sleep30-21swapper/307:05:433
1226673991712,3cyclictest1483307-21fschecks_time11:26:132
1226673991712,3cyclictest1483307-21fschecks_time11:26:132
1226673991711,4cyclictest831-21mta-sts-daemon07:11:162
122666899172,10cyclictest1226514-21latency_hist07:10:010
14826182150,2sleep3481ktimers/311:25:003
14826182150,2sleep3481ktimers/311:24:593
15419702132,7sleep30-21swapper/312:25:003
15225222132,7sleep30-21swapper/312:06:093
13977822132,6sleep30-21swapper/310:00:013
13977822132,6sleep30-21swapper/310:00:013
13729502132,6sleep30-21swapper/309:35:003
13634522132,7sleep30-21swapper/309:26:063
13441552132,6sleep30-21swapper/309:06:113
13383642132,6sleep30-21swapper/309:00:013
12532272132,6sleep30-21swapper/307:35:013
15491672123,5sleep30-21swapper/312:31:143
15423512122,7sleep30-21swapper/312:26:093
15211722122,6sleep30-21swapper/312:01:173
15126042122,6sleep30-21swapper/311:56:073
15076452122,6sleep30-21swapper/311:51:043
15019452122,6sleep30-21swapper/311:41:263
14968132122,6sleep30-21swapper/311:36:243
14878662122,6sleep30-21swapper/311:31:073
14826182122,6sleep30-21swapper/311:25:013
14826182122,6sleep30-21swapper/311:25:013
14719302122,6sleep30-21swapper/311:11:283
14619722122,6sleep30-21swapper/311:01:253
14527282122,6sleep30-21swapper/310:56:033
14381202122,6sleep30-21swapper/310:41:103
14327742122,6sleep30-21swapper/310:36:093
14277562122,6sleep30-21swapper/310:31:073
14266542122,6sleep30-21swapper/310:26:203
14123252122,6sleep30-21swapper/310:11:283
14067972122,6sleep30-21swapper/310:06:193
13879802122,6sleep30-21swapper/309:50:023
13848892122,6sleep30-21swapper/309:46:163
13782802122,6sleep30-21swapper/309:41:073
13736712122,6sleep30-21swapper/309:36:133
13591802122,7sleep30-21swapper/309:21:123
13517762122,6sleep30-21swapper/309:11:173
13352832122,6sleep30-21swapper/308:56:233
13330732122,6sleep30-21swapper/308:51:263
13240492122,6sleep30-21swapper/308:46:093
13147132122,6sleep30-21swapper/308:36:133
13038202122,6sleep30-21swapper/308:26:103
12887992123,6sleep30-21swapper/308:11:093
12780962122,5sleep30-21swapper/308:00:023
12685742122,6sleep30-21swapper/307:51:063
12641702122,6sleep30-21swapper/307:46:133
12612552122,6sleep30-21swapper/307:41:313
12442602122,6sleep30-21swapper/307:26:393
12336712122,6sleep30-21swapper/307:15:073
15526032112,5sleep30-21swapper/312:36:123
15323602112,6sleep30-21swapper/312:15:533
15026392112,6sleep30-21swapper/311:45:153
14730642112,5sleep30-21swapper/311:16:143
14730642112,5sleep30-21swapper/311:16:133
14633422112,5sleep30-21swapper/311:06:113
14477642112,5sleep30-21swapper/310:50:413
14432592112,5sleep30-21swapper/310:46:123
14217052112,5sleep30-21swapper/310:21:203
14129852112,5sleep30-21swapper/310:16:103
13931242112,5sleep30-21swapper/309:56:083
13931242112,5sleep30-21swapper/309:56:083
13539162112,5sleep30-21swapper/309:16:103
13228762112,5sleep30-21swapper/308:41:223
12988592112,5sleep30-21swapper/308:21:123
12939222112,5sleep30-21swapper/308:16:113
12882062112,6sleep30-21swapper/308:10:013
12733802112,5sleep30-21swapper/307:55:163
12486702112,5sleep30-21swapper/307:31:083
12271572112,5sleep30-21swapper/307:11:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional