You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 14:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Tue Sep 16, 2025 12:43:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
141961725948,8sleep50-21swapper/515:25:545
141958025444,7sleep40-21swapper/415:25:214
141964925040,7sleep60-21swapper/615:26:196
1420043994543,2cyclictest1696068-21kworker/u16:4+flush-8:017:45:136
1420043994038,2cyclictest1758055-21kworker/u16:6+flush-8:018:17:186
14111122404,30sleep70-21swapper/715:23:167
1420043993937,2cyclictest1794395-21kworker/u16:3+events_unbound19:06:166
1420043993937,2cyclictest1794395-21kworker/u16:3+events_unbound18:50:186
1420040993935,3cyclictest1778165-21cat18:28:005
1420040993834,3cyclictest1696068-21kworker/u16:4+flush-8:018:08:255
1420043993735,2cyclictest1696068-21kworker/u16:4+events_unbound18:24:176
1420043993634,2cyclictest694594-21kworker/u16:3+events_unbound15:31:146
1420043993634,2cyclictest1696068-21kworker/u16:4+flush-8:018:34:166
1420043993634,2cyclictest1571385-21kworker/u16:0+flush-8:016:44:176
1420043993634,2cyclictest1571385-21kworker/u16:0+events_unbound17:51:176
1420043993634,2cyclictest1443367-21kworker/u16:2+events_unbound16:54:466
1420047993433,0cyclictest0-21swapper/719:48:267
1420043993431,2cyclictest1443367-21kworker/u16:2+flush-8:016:33:096
1420040993432,2cyclictest1876227-21kworker/u16:2+flush-8:019:23:285
1420038993431,2cyclictest1964662-21kworker/u16:0+flush-8:020:07:594
1420038993431,2cyclictest1826963-21kworker/u16:5+flush-8:019:08:244
1420043993331,2cyclictest1531456-21kworker/u16:3+flush-8:016:22:186
1420038993331,2cyclictest1964662-21kworker/u16:0+events_unbound20:13:294
1420043993230,2cyclictest1876227-21kworker/u16:2+events_unbound19:30:286
1420043993230,2cyclictest1826963-21kworker/u16:5+events_unbound19:00:166
1420043993230,2cyclictest1611239-21kworker/u16:2+flush-8:017:55:176
1420043993230,2cyclictest1419919-21kworker/u16:0+flush-8:016:17:406
1420043993230,2cyclictest1419919-21kworker/u16:0+events_unbound15:45:166
1420043993229,2cyclictest1826963-21kworker/u16:5+events_unbound18:53:026
1420043993129,2cyclictest1696068-21kworker/u16:4+events_unbound18:05:156
1420043993129,2cyclictest1531456-21kworker/u16:3+events_unbound16:26:106
1420040993127,3cyclictest1571562-21latency_hist16:42:595
1420047993029,0cyclictest0-21swapper/716:48:247
1420047993028,1cyclictest0-21swapper/719:33:227
1420043993028,2cyclictest1876227-21kworker/u16:2+flush-8:019:16:106
1420043993028,2cyclictest1758055-21kworker/u16:6+events_unbound19:40:286
1420043993028,2cyclictest1758055-21kworker/u16:6+events_unbound19:40:286
1420043993028,2cyclictest1696068-21kworker/u16:4+events_unbound18:30:166
1420043993028,2cyclictest1611239-21kworker/u16:2+flush-8:018:19:096
1420043993028,2cyclictest1443367-21kworker/u16:2+flush-8:016:39:106
1420040993027,3cyclictest1855826-21kworker/u16:0+flush-8:019:33:145
1420047992928,0cyclictest0-21swapper/720:18:127
1420043992927,2cyclictest1876227-21kworker/u16:2+flush-8:019:21:286
1420043992927,2cyclictest1876227-21kworker/u16:2+events_unbound20:09:096
1420043992927,2cyclictest1876227-21kworker/u16:2+events_unbound20:00:576
1420043992927,2cyclictest1611239-21kworker/u16:2+events_unbound17:13:106
1420043992927,2cyclictest1571385-21kworker/u16:0+events_unbound17:21:296
1420043992927,2cyclictest1531456-21kworker/u16:3+flush-8:017:11:056
1420043992927,2cyclictest1531456-21kworker/u16:3+events_unbound16:59:186
1420043992927,2cyclictest1443367-21kworker/u16:2+flush-8:015:48:106
1420043992927,2cyclictest1409730-21kworker/u16:1+events_unbound15:37:176
1420043992826,2cyclictest1660465-21kworker/u16:4+flush-8:017:33:266
1420043992826,2cyclictest1419919-21kworker/u16:0+events_unbound15:57:156
1420040992825,2cyclictest1660465-21kworker/u16:4+events_unbound17:28:175
1420040992825,2cyclictest1660465-21kworker/u16:4+events_unbound17:28:175
1420038992826,2cyclictest1531456-21kworker/u16:3+flush-8:017:03:244
1420038992825,2cyclictest1371176-21kworker/u16:4+flush-8:015:28:244
1420043992725,2cyclictest1984549-21kworker/u16:3+flush-8:020:53:166
1420043992725,2cyclictest1600764-21kworker/u16:5+flush-8:017:04:586
1420043992725,2cyclictest1571385-21kworker/u16:0+flush-8:017:29:106
1420043992725,2cyclictest1571385-21kworker/u16:0+flush-8:017:29:106
1420043992725,2cyclictest1571385-21kworker/u16:0+events_unbound18:01:176
1420043992725,2cyclictest1371176-21kworker/u16:4+events_unbound15:40:156
1420038992725,2cyclictest1826963-21kworker/u16:5+events_unbound19:48:274
1420047992620,5cyclictest589235-21systemd-journal17:42:597
1420043992624,2cyclictest1984549-21kworker/u16:3+events_unbound20:15:586
1420043992624,2cyclictest1876227-21kworker/u16:2+events_unbound19:27:286
1420043992624,2cyclictest1571385-21kworker/u16:0+flush-8:017:39:096
1420043992623,2cyclictest1689548-21kworker/u16:3+events_unbound18:08:136
1420040992623,3cyclictest1964662-21kworker/u16:0+flush-8:020:08:275
1420038992624,2cyclictest1876227-21kworker/u16:2+flush-8:020:38:164
1420038992624,2cyclictest1409730-21kworker/u16:1+events_unbound15:34:294
20101022250,0chrt0-21swapper/620:23:216
1420047992524,0cyclictest0-21swapper/716:13:217
1420043992523,2cyclictest1794395-21kworker/u16:3+flush-8:018:40:156
1420043992523,2cyclictest1758055-21kworker/u16:6+flush-8:020:18:586
1420043992522,2cyclictest1443367-21kworker/u16:2+events_unbound15:58:106
1420040992523,2cyclictest1571385-21kworker/u16:0+flush-8:018:43:345
1420038992523,2cyclictest1826963-21kworker/u16:5+events_unbound19:03:284
1420038992523,2cyclictest1581285-21kworker/u16:4+flush-8:016:48:284
1420038992523,2cyclictest1443367-21kworker/u16:2+flush-8:016:23:174
1420038992522,2cyclictest1443367-21kworker/u16:2+events_unbound15:58:144
1420043992422,2cyclictest1964662-21kworker/u16:0+events_unbound20:30:266
1420043992422,2cyclictest1758055-21kworker/u16:6+events_unbound18:47:526
1420043992422,2cyclictest1531456-21kworker/u16:3+events_unbound17:27:056
1420043992422,2cyclictest1531456-21kworker/u16:3+events_unbound17:27:056
1420043992422,2cyclictest1419919-21kworker/u16:0+events_unbound16:04:546
1420040992420,3cyclictest1512633-21kworker/u16:1+flush-8:016:13:315
1420038992422,2cyclictest1826360-21kworker/u16:2+events_unbound18:53:144
1420047992321,1cyclictest0-21swapper/720:28:217
1420043992321,2cyclictest1906036-21kworker/u16:1+events_unbound19:35:426
1420043992321,2cyclictest1758055-21kworker/u16:6+events_unbound20:41:346
1420040992319,3cyclictest1571385-21kworker/u16:0+flush-8:017:48:175
1420038992321,2cyclictest1876227-21kworker/u16:2+events_unbound19:58:214
1420038992321,2cyclictest1758055-21kworker/u16:6+events_unbound18:58:274
1420038992319,3cyclictest2043228-21kworker/u16:4+flush-8:020:48:334
1420047992221,0cyclictest0-21swapper/720:53:247
1420047992221,0cyclictest0-21swapper/718:18:277
1420038992220,2cyclictest1718154-21kworker/u16:1+events_unbound18:13:284
1420038992220,2cyclictest1611239-21kworker/u16:2+events_unbound17:38:134
1420038992220,2cyclictest1571385-21kworker/u16:0+events_unbound16:58:254
1420038992220,2cyclictest1371176-21kworker/u16:4+flush-8:015:38:284
1420047992120,0cyclictest0-21swapper/715:58:237
1420047992116,4cyclictest589235-21systemd-journal18:02:167
1420043992119,2cyclictest1906036-21kworker/u16:1+events_unbound19:50:596
1420038992117,3cyclictest1827288-21cat18:52:534
1420047992019,0cyclictest0-21swapper/720:38:277
1420047992017,2cyclictest1866640-21turbostat19:12:597
1420047992015,4cyclictest589235-21systemd-journal19:07:597
1420043992018,2cyclictest1531456-21kworker/u16:3+flush-8:016:49:016
1420043992018,2cyclictest1429501-21kworker/u16:5+flush-8:016:10:306
1420040992018,2cyclictest1807384-21kworker/u16:1+flush-8:019:12:595
1420038992018,2cyclictest1689548-21kworker/u16:3+flush-8:017:58:204
1420038992017,2cyclictest1758055-21kworker/u16:6+events_unbound18:38:134
142004799199,9cyclictest0-21swapper/720:03:257
1420047991919,0cyclictest0-21swapper/716:38:237
1420047991918,1cyclictest0-21swapper/718:03:247
1420047991917,1cyclictest0-21swapper/718:42:597
1420043991917,2cyclictest2043228-21kworker/u16:4+flush-8:020:43:456
1420043991917,2cyclictest1758055-21kworker/u16:6+flush-8:019:10:366
1420038991917,2cyclictest1409730-21kworker/u16:1+events_unbound15:48:254
1420038991916,2cyclictest1855826-21kworker/u16:0+events_unbound19:17:594
1420038991916,2cyclictest1611239-21kworker/u16:2+flush-8:017:08:214
1420038991916,2cyclictest1429501-21kworker/u16:5+flush-8:016:13:194
1420043991816,2cyclictest1984549-21kworker/u16:3+flush-8:020:48:526
1420040991815,2cyclictest1768216-21kworker/u16:1+events_unbound18:33:005
1420040991814,3cyclictest1660063-21latency_hist17:27:595
1420040991814,3cyclictest1660063-21latency_hist17:27:595
1420038991816,2cyclictest1531456-21kworker/u16:3+events_unbound17:18:274
1420038991814,3cyclictest1984549-21kworker/u16:3+events_unbound20:43:214
1420038991814,3cyclictest1483063-21latency_hist15:58:004
142004799176,9cyclictest27642-21gmain20:13:207
1420047991714,2cyclictest1313-21snmpd16:53:067
1420043991715,2cyclictest1758055-21kworker/u16:6+flush-8:020:03:236
1420038991715,2cyclictest1876227-21kworker/u16:2+flush-8:019:28:144
1420038991715,2cyclictest1571385-21kworker/u16:0+events_unbound18:43:324
1420047991615,0cyclictest0-21swapper/717:28:327
1420047991615,0cyclictest0-21swapper/717:28:327
1420047991614,1cyclictest1719071-21turbostat17:57:597
1420047991614,1cyclictest0-21swapper/718:13:147
1420047991614,1cyclictest0-21swapper/716:58:197
142004799160,16cyclictest0-21swapper/715:34:397
142004799160,14cyclictest0-21swapper/720:48:247
1420043991613,2cyclictest1758055-21kworker/u16:6+events_unbound19:43:066
1420040991614,2cyclictest1826963-21kworker/u16:5+flush-8:019:48:195
1420038991614,2cyclictest1443367-21kworker/u16:2+flush-8:016:53:154
142004799157,7cyclictest0-21swapper/718:48:107
1420047991513,1cyclictest1886346-21turbostat19:22:597
142004799150,14cyclictest0-21swapper/719:53:237
142004799150,13cyclictest0-21swapper/720:23:257
1420043991513,2cyclictest1876227-21kworker/u16:2+events_unbound20:36:256
1420043991512,3cyclictest1906036-21kworker/u16:1+flush-8:019:53:286
1420040991511,3cyclictest1876227-21kworker/u16:2+flush-8:019:18:155
1420038991513,2cyclictest1611239-21kworker/u16:2+events_unbound17:28:134
1420038991513,2cyclictest1611239-21kworker/u16:2+events_unbound17:28:134
1420038991513,2cyclictest1512633-21kworker/u16:1+flush-8:017:13:294
1420038991511,3cyclictest1419919-21kworker/u16:0+flush-8:016:03:314
142004799149,4cyclictest589235-21systemd-journal20:45:257
1420047991411,2cyclictest1313-21snmpd17:10:337
142004799140,1cyclictest0-21swapper/716:43:247
142004099149,4cyclictest2033744-21cat20:37:585
1420040991413,1cyclictest0-21swapper/516:28:195
1420040991411,2cyclictest1680082-21cut17:38:005
142004099140,1cyclictest0-21swapper/515:55:035
1420038991412,2cyclictest1906036-21kworker/u16:1+events_unbound19:33:264
1420038991412,2cyclictest1826963-21kworker/u16:5+flush-8:019:38:264
1420038991412,2cyclictest1826963-21kworker/u16:5+flush-8:019:38:264
1420038991412,2cyclictest1826963-21kworker/u16:5+flush-8:019:23:224
1420038991411,2cyclictest1876227-21kworker/u16:2+events_unbound19:18:214
142004799139,3cyclictest1552318-21/usr/sbin/munin16:33:167
142004799139,3cyclictest1313-21snmpd16:28:087
142004799136,6cyclictest27642-21gmain17:33:207
142004799136,6cyclictest0-21swapper/718:58:157
1420047991312,0cyclictest0-21swapper/718:43:167
142004799131,11cyclictest1796821-21ntpq18:33:267
1420047991311,1cyclictest0-21swapper/717:23:017
1420047991311,1cyclictest0-21swapper/717:23:017
142004799130,2cyclictest1420032-21cyclictest18:28:237
142004799130,12cyclictest1502813-21turbostat16:08:007
142004099139,3cyclictest1758055-21kworker/u16:6+flush-8:019:43:275
142004099138,4cyclictest1641519-21cat17:18:165
1420040991312,1cyclictest0-21swapper/516:33:315
1420040991311,2cyclictest1443367-21kworker/u16:2+flush-8:016:24:575
142004099130,13cyclictest0-21swapper/518:07:095
1420038991311,2cyclictest1571385-21kworker/u16:0+events_unbound17:43:144
142004799129,3cyclictest96-21ksoftirqd/719:23:287
142004799129,2cyclictest1827701-21/usr/sbin/munin18:53:167
142004799129,2cyclictest1313-21snmpd19:40:437
142004799129,2cyclictest1313-21snmpd19:40:437
142004799129,2cyclictest1313-21snmpd16:20:477
142004799128,3cyclictest1313-21snmpd20:09:237
142004799128,3cyclictest1313-21snmpd15:48:347
142004799127,4cyclictest589235-21systemd-journal15:53:557
142004799124,7cyclictest0-21swapper/717:43:167
142004799122,8cyclictest0-21swapper/719:28:197
142004799120,11cyclictest1313-21snmpd16:08:327
142004099128,3cyclictest1876227-21kworker/u16:2+flush-8:019:28:175
142004099127,4cyclictest1503724-21diskstats16:08:135
1420040991211,1cyclictest0-21swapper/519:53:065
1420040991210,2cyclictest1611239-21kworker/u16:2+events_unbound17:13:165
142004099120,12cyclictest0-21swapper/515:52:225
142004099120,11cyclictest0-21swapper/519:38:075
142004099120,11cyclictest0-21swapper/519:38:075
1420038991210,2cyclictest1758055-21kworker/u16:6+events_unbound19:43:264
1420038991210,2cyclictest1611239-21kworker/u16:2+events_unbound18:18:134
142003899120,1cyclictest0-21swapper/420:20:584
142003899120,1cyclictest0-21swapper/417:26:274
142003899120,1cyclictest0-21swapper/417:26:274
142004799118,2cyclictest1313-21snmpd20:00:117
142004799118,2cyclictest1313-21snmpd17:19:037
142004799118,2cyclictest1313-21snmpd17:13:577
142004799117,3cyclictest1313-21snmpd20:36:477
142004799117,3cyclictest1313-21snmpd18:24:507
142004799117,3cyclictest1313-21snmpd15:28:247
142004799116,4cyclictest1533219-21tune2fs16:23:147
142004799115,5cyclictest27763-21gvfs-afc-volume15:38:197
142004799114,6cyclictest27763-21gvfs-afc-volume15:43:207
1420043991110,1cyclictest0-21swapper/616:32:316
1420040991110,1cyclictest0-21swapper/520:44:585
1420040991110,1cyclictest0-21swapper/520:39:555
1420040991110,1cyclictest0-21swapper/520:29:505
1420040991110,1cyclictest0-21swapper/518:39:105
1420040991110,1cyclictest0-21swapper/518:34:545
1420040991110,1cyclictest0-21swapper/517:46:215
1420040991110,1cyclictest0-21swapper/517:40:315
1420040991110,1cyclictest0-21swapper/515:32:495
142004099110,1cyclictest0-21swapper/516:43:555
142004099110,11cyclictest0-21swapper/520:55:035
142004099110,11cyclictest0-21swapper/517:53:555
142004099110,11cyclictest0-21swapper/516:06:335
142003899119,2cyclictest1758055-21kworker/u16:6+events_unbound18:28:254
1420038991110,1cyclictest0-21swapper/415:43:214
142003899110,11cyclictest0-21swapper/420:53:454
142003899110,11cyclictest0-21swapper/416:42:474
142004799107,2cyclictest1313-21snmpd17:06:217
142004799106,3cyclictest1313-21snmpd17:50:347
142004799104,5cyclictest589235-21systemd-journal19:43:267
142004799104,5cyclictest1867016-21cron19:12:597
142004799100,8cyclictest1739112-21sshd18:08:047
142004099109,1cyclictest0-21swapper/517:10:245
142004099109,1cyclictest0-21swapper/516:19:395
142004099108,2cyclictest1758055-21kworker/u16:6+flush-8:019:13:285
142004099106,3cyclictest1610854-21latency_hist17:02:595
142004099100,0cyclictest0-21swapper/519:03:455
142004099100,0cyclictest0-21swapper/518:00:325
142003899109,1cyclictest0-21swapper/420:36:064
142003899109,1cyclictest0-21swapper/417:33:234
142003899109,1cyclictest0-21swapper/416:47:594
142003899108,2cyclictest1611239-21kworker/u16:2+flush-8:018:23:344
142003899100,9cyclictest0-21swapper/420:28:154
14200409998,1cyclictest0-21swapper/520:24:475
14200409998,1cyclictest0-21swapper/520:19:445
14200409998,1cyclictest0-21swapper/516:54:105
14200409998,1cyclictest0-21swapper/516:02:095
14200409998,1cyclictest0-21swapper/515:40:525
14200409995,3cyclictest1434867-21cat15:33:195
14200409994,4cyclictest1966085-21cat20:03:145
14200389998,1cyclictest0-21swapper/417:50:154
14200389998,1cyclictest0-21swapper/416:33:194
14200389995,3cyclictest1954120-21awk19:53:244
14200389994,4cyclictest2005392-21cat20:23:164
14200389994,4cyclictest1739899-21cat18:08:134
14200409987,1cyclictest0-21swapper/520:50:005
14200409987,1cyclictest0-21swapper/519:58:565
14200409986,1cyclictest0-21swapper/517:03:205
14200409986,1cyclictest0-21swapper/515:43:175
14200409983,4cyclictest1822027-21cut18:48:185
14200409980,5cyclictest1994406-21idleruntime-cro20:17:595
14200389987,1cyclictest0-21swapper/420:12:364
14200389986,1cyclictest0-21swapper/418:03:224
14200389985,2cyclictest0-21swapper/416:18:184
14200389984,3cyclictest1505741-21cut16:08:184
14200389983,4cyclictest1717955-21missed_timers17:53:224
14200409976,1cyclictest0-21swapper/518:54:275
14200409975,1cyclictest0-21swapper/516:48:145
14200409973,3cyclictest1841391-21grep18:58:195
14200409973,3cyclictest1759463-21tune2fs18:18:155
14200409973,3cyclictest1749420-21df_inode18:13:125
14200389973,3cyclictest1790276-21cut18:33:174
14200389972,4cyclictest1542481-21grep16:28:134
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional