You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Mon Nov 24, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12752662860,3chrt586-21kworker/3:2-events19:56:203
14677562720,3sleep20-21swapper/223:11:182
14677562720,3sleep20-21swapper/223:11:182
12651562640,6chrt0-21swapper/219:46:202
1226706995953,4cyclictest1253576-21kworker/u8:4+flush-8:019:41:340
1226711995851,5cyclictest1238262-21kworker/u8:2+flush-8:021:10:001
1226706995645,8cyclictest282-21systemd-journal22:39:590
1226711995441,10cyclictest1466844-21latency_hist23:10:011
1226711995437,15cyclictest1437295-21/usr/sbin/munin22:41:211
1226711995344,6cyclictest1287958-21latency_hist20:10:011
1226706995345,6cyclictest1303038-21kworker/u8:1+flush-8:021:51:270
1226706995343,7cyclictest1416335-21ntpq22:16:230
1226711995233,16cyclictest1257492-21ntpq19:36:231
1226706995247,4cyclictest1387318-21kworker/u8:0+flush-8:022:31:270
1226706995246,4cyclictest1238262-21kworker/u8:2+flush-8:022:51:260
1226711995141,8cyclictest1417538-21/usr/sbin/munin22:21:211
1226706994938,9cyclictest1282224-21ntpq20:01:240
1226706994840,6cyclictest1326930-21ntpq20:46:250
1226715994741,4cyclictest1303038-21kworker/u8:1+flush-8:021:46:272
1226711994729,15cyclictest1352809-21apt-get21:16:081
1226706994737,7cyclictest282-21systemd-journal22:56:210
1226706994736,8cyclictest1401357-21ntpq22:01:270
1226711994637,7cyclictest1268539-21/usr/sbin/munin19:51:281
1226711994636,8cyclictest1248718-21apt-get19:31:111
1226711994636,7cyclictest1520418-21ntpq00:01:241
1226706994639,5cyclictest1247502-21ntpq19:26:240
1226711994528,15cyclictest1338005-21apt-get21:01:121
1226711994528,15cyclictest1338005-21apt-get21:01:121
1226711994525,17cyclictest282-21systemd-journal23:24:211
1226706994435,6cyclictest1382133-21latency_hist21:45:010
1226706994434,8cyclictest1242621-21ntpq19:21:260
1226711994336,5cyclictest1278075-21latency_hist20:00:011
1226711994333,8cyclictest1366463-21ntpq21:26:251
1226706994338,4cyclictest383363-21kworker/u8:1+flush-8:019:11:290
1226706994335,6cyclictest1253346-21latency_hist19:35:010
1226706994333,8cyclictest1292134-21ntpq20:11:250
1226706994333,7cyclictest282-21systemd-journal00:01:220
12453662420,4sleep1281rcuc/119:26:191
1226711994234,5cyclictest1491554-21latency_hist23:35:011
1226711994224,15cyclictest282-21systemd-journal00:30:421
1226706994238,3cyclictest1303038-21kworker/u8:1+events_unbound21:56:250
1226706994237,3cyclictest1238262-21kworker/u8:2+flush-8:020:56:260
1226706994234,6cyclictest1311837-21ntpq20:31:220
1226706994233,7cyclictest1382679-21/usr/sbin/munin21:46:220
1226711994124,14cyclictest1382136-21apt-get21:45:011
1226706994131,8cyclictest1545139-21ntpq00:26:240
1226706994130,9cyclictest1238262-21kworker/u8:2+flush-8:021:26:320
1226711994032,6cyclictest1267412-21ntpq19:46:251
1226711994029,7cyclictest282-21systemd-journal20:23:501
1226706994035,4cyclictest1387318-21kworker/u8:0+flush-8:022:11:310
12257302403,30sleep01225759-21ntpq19:06:250
12257302403,30sleep01225759-21ntpq19:06:250
14333502390,5sleep30-21swapper/322:36:203
13418452390,4sleep2371rcuc/221:01:282
13418452390,4sleep2371rcuc/221:01:282
1226715993934,4cyclictest1238262-21kworker/u8:2+flush-8:020:46:152
1226715993933,4cyclictest1407107-21gpgv22:10:002
1226711993929,7cyclictest1372593-21/usr/sbin/munin21:36:231
1226711993927,10cyclictest1525287-21needreboot00:06:251
1226711993832,4cyclictest1313040-21apt-get20:36:111
1226706993833,3cyclictest1238262-21kworker/u8:2+flush-8:021:21:280
1226711993727,8cyclictest1328405-21fschecks_time20:51:151
1226711993727,8cyclictest1282681-21unixbench_multi20:01:301
1226711993719,16cyclictest1391909-21unixbench_multi21:51:301
1226706993733,3cyclictest1238262-21kworker/u8:2+events_unbound21:16:290
1226706993731,4cyclictest1253576-21kworker/u8:4+events_unbound19:56:140
1226706993730,5cyclictest1475941-21kworker/u8:0+flush-8:000:06:320
1226706993727,8cyclictest1550057-21ntpq00:31:240
1226715993631,4cyclictest1530647-21kworker/u8:2+events_unbound00:21:262
1226715993627,6cyclictest282-21systemd-journal22:25:012
1226715993624,9cyclictest1407496-21cat22:10:012
1226711993629,5cyclictest1237571-21ntpq19:16:221
1226711993623,11cyclictest1327779-21systemd-run20:49:591
1226711993623,11cyclictest0-21swapper/119:41:321
1226706993631,4cyclictest1238262-21kworker/u8:2+flush-8:022:06:250
1226706993630,4cyclictest1387318-21kworker/u8:0+flush-8:022:21:280
122671199354,5cyclictest0-21swapper/123:41:101
1226711993526,7cyclictest1230736-21ntpq19:11:251
1226711993517,16cyclictest0-21swapper/122:56:201
1226706993527,6cyclictest1475941-21kworker/u8:0+events_unbound23:41:310
1226706993525,8cyclictest1303038-21kworker/u8:1+flush-8:021:36:340
12219342353,27sleep10-21swapper/119:05:421
12219342353,27sleep10-21swapper/119:05:421
1226711993423,9cyclictest1312397-21unixbench_singl20:31:291
1226711993419,13cyclictest1462031-21latency_hist23:05:011
1226711993417,15cyclictest1321733-21ntp_kernel_err20:41:231
1226706993430,3cyclictest1303038-21kworker/u8:1+flush-8:020:26:150
1226706993429,4cyclictest1436701-21kworker/u8:3+events_unbound23:11:290
1226706993429,4cyclictest1436701-21kworker/u8:3+events_unbound23:11:290
1226706993429,4cyclictest1253576-21kworker/u8:4+flush-8:020:21:290
1226706993428,4cyclictest1238262-21kworker/u8:2+flush-8:020:16:210
1226706993425,6cyclictest1556057-21latency_hist00:40:010
1226706993425,6cyclictest1238262-21kworker/u8:2+flush-8:019:46:240
1226711993319,12cyclictest1372303-21systemd-run21:35:011
1226706993328,4cyclictest1169336-21kworker/u8:3+events_unbound19:16:270
1226706993327,4cyclictest1446004-21ntpq22:46:210
1226706993325,6cyclictest1436701-21kworker/u8:3+events_unbound23:16:210
1226715993228,3cyclictest1530647-21kworker/u8:2+flush-8:000:26:302
1226715993228,3cyclictest1238262-21kworker/u8:2+flush-8:021:31:282
1226715993224,6cyclictest1476927-21cat23:20:002
1226711993225,5cyclictest1238779-21apt-get19:21:101
1226711993224,6cyclictest1526533-21apt-get00:11:141
1226711993222,8cyclictest1386438-21ntpq21:46:231
122671199322,11cyclictest1501780-21apt-get23:46:091
1226711993220,10cyclictest1352105-21unixbench_multi21:11:301
1226711993217,14cyclictest1511346-21latency_hist23:55:011
1226711993217,13cyclictest0-21swapper/122:06:151
1226711993212,4cyclictest0-21swapper/100:29:421
1226711993210,4cyclictest0-21swapper/121:56:181
1226706993227,4cyclictest1238262-21kworker/u8:2+flush-8:020:06:150
1226706993226,4cyclictest1226576-21kworker/u8:0+events_unbound21:01:140
1226706993226,4cyclictest1226576-21kworker/u8:0+events_unbound21:01:130
12263392323,10sleep20-21swapper/219:07:402
12263392323,10sleep20-21swapper/219:07:402
1226711993125,4cyclictest1238262-21kworker/u8:2+flush-8:020:26:201
1226711993124,5cyclictest1442277-21apt-get22:46:081
122671199312,12cyclictest1293256-21apt-get20:16:121
1226711993116,13cyclictest1476788-21latency_hist23:20:001
1226711993114,15cyclictest1531695-21tr00:16:171
1226711993111,5cyclictest0-21swapper/122:11:301
1226711993111,4cyclictest0-21swapper/122:16:151
1226706993125,4cyclictest1436701-21kworker/u8:3+events_unbound22:41:120
1226706993124,5cyclictest1322613-21latency_hist20:45:000
1226706993120,7cyclictest1516272-40logrotate00:00:000
1226711993025,3cyclictest1423221-21irqrtprio22:26:201
1226711993012,6cyclictest1432578-21cpuspeed_turbos22:36:141
1226711993011,4cyclictest0-21swapper/122:51:121
122671199301,10cyclictest587-21runrttasks20:56:241
1226711993010,5cyclictest0-21swapper/123:36:131
1226706993026,3cyclictest1303038-21kworker/u8:1+events_unbound23:06:290
1226706993025,4cyclictest1238262-21kworker/u8:2+flush-8:022:26:500
1226706993024,4cyclictest1526762-21cpuspeed_turbos00:11:170
1226706993020,8cyclictest1495610-21ntpq23:36:210
1226715992923,4cyclictest1238262-21kworker/u8:2+flush-8:019:35:012
122671199298,6cyclictest282-21systemd-journal23:10:201
122671199298,6cyclictest282-21systemd-journal23:10:201
122671199298,4cyclictest0-21swapper/122:01:151
1226711992912,2cyclictest1288464-21cpuspeed20:11:141
1226711992910,3cyclictest0-21swapper/100:36:121
1226706992924,3cyclictest1226576-21kworker/u8:0+flush-8:020:51:210
1226706992919,7cyclictest1541014-21latency_hist00:25:010
14476652280,6sleep3451irq_work/322:51:163
1226715992824,3cyclictest1382136-21apt-get21:45:002
1226715992823,3cyclictest1432635-21fschecks_count22:36:162
122671199289,2cyclictest0-21swapper/123:30:011
122671199285,5cyclictest0-21swapper/122:31:111
122671199282,9cyclictest1361631-21ntpq21:21:271
1226711992810,2cyclictest0-21swapper/100:25:001
1226706992824,3cyclictest1514017-21kworker/u8:3+flush-8:000:16:320
1226706992823,4cyclictest1367535-21kworker/u8:3+events_unbound21:31:130
122670699282,25cyclictest1216800-21kworker/0:1+events23:24:100
1226706992822,4cyclictest1303038-21kworker/u8:1+events_unbound23:26:240
122670699282,21cyclictest1216800-21kworker/0:1+events23:00:460
1226715992723,3cyclictest1303038-21kworker/u8:1+flush-8:021:11:242
1226715992722,3cyclictest1510985-21unixbench_singl23:51:312
1226715992722,3cyclictest0-21swapper/223:40:002
1226715992721,4cyclictest1457998-21kthreadcore23:01:182
1226706992723,3cyclictest1238262-21kworker/u8:2+flush-8:019:52:170
1226706992722,3cyclictest1238262-21kworker/u8:2+events_unbound23:46:290
122670699272,20cyclictest1216800-21kworker/0:1+events23:30:260
122670699272,20cyclictest1216800-21kworker/0:1+events21:05:380
1226706992721,5cyclictest1303038-21kworker/u8:1+events_unbound21:11:160
1226715992623,2cyclictest1397813-21fschecks_count22:01:162
1226715992623,2cyclictest1303435-21fschecks_count20:26:142
1226715992621,3cyclictest1262931-21unixbench_multi19:41:362
1226715992620,4cyclictest1297520-21unixbench_singl20:16:312
1226706992616,5cyclictest1303038-21kworker/u8:1+flush-8:023:51:290
1226715992522,2cyclictest1521634-21aten2_r5power_e00:06:152
1226715992521,2cyclictest1422612-21aten2_r5power_v22:26:152
1226715992520,4cyclictest1169336-21kworker/u8:3+flush-8:019:21:262
1226715992513,9cyclictest1521243-21awk00:05:002
122671199259,5cyclictest1513582-21cut23:56:181
1226715992420,2cyclictest1442494-21cpuspeed_turbos22:46:112
1226715992420,2cyclictest1287577-21unixbench_multi20:06:302
1226715992419,3cyclictest1546527-21cpuspeed_turbos00:31:132
1226715992419,3cyclictest1497065-21cpuspeed_turbos23:41:132
1226715992419,3cyclictest1462533-21fschecks_count23:06:142
1226715992418,4cyclictest1357071-21unixbench_singl21:16:302
1226706992419,4cyclictest1-21systemd20:36:510
1226706992418,4cyclictest1257969-21unixbench_multi19:36:290
122671599238,3cyclictest13762342missed_timers21:36:222
1226715992320,2cyclictest1313126-21aten2_r5power_p20:36:132
1226715992319,2cyclictest1526592-21aten2_r5power_e00:11:162
1226715992319,2cyclictest0-21swapper/220:04:592
1226715992318,3cyclictest1451451-21unixbench_singl22:51:312
1226715992318,3cyclictest1328811-21cut20:51:182
122671599230,11cyclictest1237532-21/usr/sbin/munin19:16:212
122671599221,7cyclictest13884581irqrtprio21:51:162
1226715992217,4cyclictest1515330-21needreboot23:56:222
1226715992217,4cyclictest1238262-21kworker/u8:2+events_unbound21:26:342
1226715992217,3cyclictest0-21swapper/221:00:002
1226715992213,6cyclictest1441060-21awk22:41:202
1226715992116,4cyclictest1303038-21kworker/u8:1+flush-8:023:31:272
1226715992113,6cyclictest0-21swapper/223:30:002
122671599202,4cyclictest0-21swapper/219:56:262
122671599202,10cyclictest0-21swapper/221:06:212
1226715992016,2cyclictest1298539-21fschecks_time20:21:162
1226715992015,4cyclictest1231585-21awk19:15:002
1226715992013,5cyclictest0-21swapper/220:11:172
122671599192,12cyclictest1392596-21apt-get21:56:122
1226715991914,3cyclictest0-21swapper/200:40:002
1226715991911,6cyclictest1534131-21cut00:16:232
1226715991910,7cyclictest282-21systemd-journal20:30:512
1226715991910,7cyclictest1413274-21sed22:16:182
1226715991910,7cyclictest1245981-21cut19:26:212
1226715991814,2cyclictest1253939-21fschecks_count19:36:142
1226715991812,4cyclictest0-21swapper/219:51:202
1226715991811,5cyclictest1320736-21tr20:41:202
12223742183,8sleep30-21swapper/319:06:163
12223742183,8sleep30-21swapper/319:06:163
122671599179,6cyclictest1477849-21sed23:21:212
122671599179,6cyclictest1454824-21cut22:56:202
122671599179,6cyclictest1427644-21sort22:31:122
122671599178,7cyclictest1360114-21tr21:21:202
1226715991713,2cyclictest1505077-21kthreadcore23:46:192
12596062168,4sleep30-21swapper/319:41:203
14671792152,8sleep30-21swapper/323:11:123
14671792152,8sleep30-21swapper/323:11:123
13280662152,9sleep30-21swapper/320:51:113
15459982143,6sleep30-21swapper/300:30:013
13925992142,8sleep30-21swapper/321:56:123
13225992142,7sleep30-21swapper/320:45:013
15154712132,6sleep30-21swapper/323:56:273
14212802132,7sleep30-21swapper/322:21:243
14048432133,6sleep30-21swapper/322:06:193
13914812132,7sleep30-21swapper/321:51:263
13623982132,6sleep30-21swapper/321:25:023
13168122132,7sleep30-21swapper/320:36:253
15407152122,6sleep30-21swapper/300:21:313
15212512122,6sleep30-21swapper/300:05:013
15188412123,5sleep30-21swapper/300:01:193
15063652122,6sleep30-21swapper/323:49:593
14973252122,6sleep30-21swapper/323:41:183
14958902122,6sleep30-21swapper/323:36:283
14824142122,6sleep30-21swapper/323:26:173
14785932121,6sleep30-21swapper/323:21:233
14717982122,5sleep30-21swapper/323:15:013
14624812122,6sleep30-21swapper/323:06:143
14520462123,5sleep30-21swapper/322:55:313
14415952122,6sleep30-21swapper/322:41:303
14161112121,7sleep30-21swapper/322:16:223
14079952122,6sleep30-21swapper/322:11:153
13979222122,6sleep30-21swapper/322:01:173
13775402122,6sleep30-21swapper/321:41:113
13726232122,6sleep30-21swapper/321:36:133
13696062123,5sleep30-21swapper/321:31:203
13570252122,6sleep30-21swapper/321:16:313
13475232122,5sleep30-21swapper/321:10:013
13466012122,6sleep30-21swapper/321:06:233
13335242122,6sleep30-21swapper/320:56:163
13126362122,6sleep30-21swapper/320:34:213
13020582122,6sleep30-21swapper/320:21:293
12934172120,8sleep30-21swapper/320:16:153
12545612122,6sleep30-21swapper/319:36:183
12490922122,6sleep30-21swapper/319:31:163
12473512122,6sleep30-21swapper/319:26:233
12384662122,5sleep30-21swapper/319:20:013
15513442112,6sleep30-21swapper/300:35:073
15416032112,5sleep30-21swapper/300:26:163
15352732112,5sleep30-21swapper/300:16:293
15264582112,6sleep30-21swapper/300:10:353
15067192112,6sleep30-21swapper/323:51:133
14869322112,5sleep30-21swapper/323:31:103
14572252112,5sleep30-21swapper/323:01:123
14421722112,5sleep30-21swapper/322:45:073
14311342112,5sleep30-21swapper/322:31:233
14263382113,5sleep30-21swapper/322:26:293
13827242112,5sleep30-21swapper/321:46:123
13581412112,5sleep30-21swapper/321:21:153
13377752112,5sleep30-21swapper/321:00:013
13377752112,5sleep30-21swapper/321:00:013
13253412111,6sleep30-21swapper/320:46:213
13031202112,5sleep30-21swapper/320:26:103
12879752112,5sleep30-21swapper/320:10:013
12833902112,5sleep30-21swapper/320:06:123
12783722112,5sleep30-21swapper/320:00:463
12685072112,6sleep30-21swapper/319:50:473
12635682112,5sleep30-21swapper/319:45:413
12338772112,5sleep30-21swapper/319:16:113
12269432101,5sleep30-21swapper/319:11:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional