You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 22:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot2.osadl.org (updated Sun Jul 13, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15378352950,1sleep71537830-21munin-run19:57:207
98876499350,33cyclictest643-21systemd-journal17:02:167
98876499350,33cyclictest1208184-21latency_hist17:17:217
98876499350,33cyclictest1074452-21latency_hist16:12:217
11464562340,0sleep50-21swapper/516:47:215
988750993230,2cyclictest1167769-21kworker/u16:4+events_unbound17:25:554
988750993230,2cyclictest1167769-21kworker/u16:4+events_unbound17:25:554
988755993128,2cyclictest1332518-21kworker/u16:3+events_unbound18:47:305
988750993128,2cyclictest1589073-21kworker/u16:2+flush-8:020:47:304
988764993029,0cyclictest0-21swapper/716:27:407
988755993028,2cyclictest1465639-21kworker/u16:3+events_unbound19:33:305
988755993028,2cyclictest1369252-21kworker/u16:2+events_unbound18:46:305
988755993028,2cyclictest1332518-21kworker/u16:3+events_unbound19:07:035
988755993027,2cyclictest1392538-21kworker/u16:5+events_unbound18:57:305
988755993027,2cyclictest1392538-21kworker/u16:5+events_unbound18:57:305
988750993028,2cyclictest1578575-21kworker/u16:3+events_unbound20:26:554
988750993028,2cyclictest1392538-21kworker/u16:5+events_unbound18:44:324
988750993028,2cyclictest1239807-21kworker/u16:1+events_unbound18:05:554
988750993027,2cyclictest935803-21kworker/u16:4+flush-8:015:47:314
988750993027,2cyclictest1605108-21kworker/u16:4+flush-8:020:37:304
988750993027,2cyclictest1465639-21kworker/u16:3+flush-8:019:22:314
988750993027,2cyclictest1167769-21kworker/u16:4+flush-8:017:57:324
988763992927,2cyclictest1167769-21kworker/u16:4+events_unbound18:12:556
988755992927,2cyclictest1605108-21kworker/u16:4+events_unbound20:54:555
988755992927,2cyclictest1413807-21kworker/u16:0+events_unbound19:13:305
988755992927,2cyclictest1115223-21kworker/u16:2+events_unbound17:10:485
988755992927,2cyclictest1115223-21kworker/u16:2+events_unbound17:01:565
988750992927,2cyclictest1332518-21kworker/u16:3+events_unbound19:02:554
988750992927,2cyclictest1101533-21kworker/u16:0+flush-8:016:46:324
98876499280,26cyclictest643-21systemd-journal16:35:517
98876499280,26cyclictest1197916-21latency_hist17:12:217
988755992826,2cyclictest1474990-21kworker/u16:5+events_unbound20:01:035
988750992826,2cyclictest988932-21kworker/u16:3+events_unbound15:40:314
988750992826,2cyclictest1640716-21kworker/u16:0+events_unbound20:56:554
988750992826,2cyclictest1369252-21kworker/u16:2+events_unbound19:45:314
988750992826,2cyclictest1115223-21kworker/u16:2+events_unbound17:37:504
988755992725,2cyclictest957774-21kworker/u16:2+events_unbound15:43:325
988755992725,2cyclictest1629255-21kworker/u16:3+events_unbound20:38:545
988755992725,2cyclictest1506552-21kworker/u16:4+events_unbound20:16:555
988755992725,2cyclictest1435448-21kworker/u16:1+flush-8:020:26:105
988755992725,2cyclictest1392538-21kworker/u16:5+flush-8:018:52:305
988755992725,2cyclictest1289618-21kworker/u16:0+flush-8:018:02:185
988755992725,2cyclictest1145491-21kworker/u16:5+events_unbound17:14:565
988750992725,2cyclictest1413807-21kworker/u16:0+events_unbound19:14:324
988750992725,2cyclictest1145491-21kworker/u16:5+events_unbound17:07:504
988750992725,2cyclictest1023730-21kworker/u16:1+flush-8:016:14:474
988755992624,2cyclictest988932-21kworker/u16:3+events_unbound15:41:515
988755992624,2cyclictest1605108-21kworker/u16:4+events_unbound20:31:305
988755992624,2cyclictest1589073-21kworker/u16:2+events_unbound20:58:305
988755992624,2cyclictest1506552-21kworker/u16:4+events_unbound19:38:565
988755992624,2cyclictest1424091-21kworker/u16:4+events_unbound19:10:305
988755992624,2cyclictest1101533-21kworker/u16:0+flush-8:016:50:585
988750992624,2cyclictest1063202-21kworker/u16:4+flush-8:016:42:214
988750992624,2cyclictest1063202-21kworker/u16:4+flush-8:016:42:214
988750992623,3cyclictest1435448-21kworker/u16:1+flush-8:019:41:004
988750992623,2cyclictest1167769-21kworker/u16:4+flush-8:017:47:314
988755992523,2cyclictest988932-21kworker/u16:3+flush-8:015:49:125
988755992523,2cyclictest957774-21kworker/u16:2+flush-8:016:11:435
988755992523,2cyclictest1589073-21kworker/u16:2+flush-8:020:44:265
988755992523,2cyclictest1289618-21kworker/u16:0+events_unbound18:10:595
988755992523,2cyclictest1115223-21kworker/u16:2+events_unbound17:52:185
988755992523,2cyclictest1101533-21kworker/u16:0+events_unbound16:41:315
988755992523,2cyclictest1101533-21kworker/u16:0+events_unbound16:41:315
988755992523,2cyclictest1042631-21kworker/u16:5+events_unbound16:01:435
988750992523,2cyclictest1435448-21kworker/u16:1+events_unbound19:50:304
988750992523,2cyclictest1289618-21kworker/u16:0+flush-8:018:11:314
988750992523,2cyclictest1167769-21kworker/u16:4+flush-8:017:28:314
988750992523,2cyclictest1084233-21kworker/u16:3+events_unbound17:01:474
988750992522,2cyclictest1023730-21kworker/u16:1+flush-8:015:52:314
988755992422,2cyclictest1578575-21kworker/u16:3+events_unbound20:21:225
988755992422,2cyclictest1506552-21kworker/u16:4+events_unbound19:46:225
988755992422,2cyclictest1435448-21kworker/u16:1+flush-8:019:27:075
988755992422,2cyclictest1435448-21kworker/u16:1+events_unbound20:03:265
988755992422,2cyclictest1332518-21kworker/u16:3+events_unbound18:24:035
988755992422,2cyclictest1115223-21kworker/u16:2+events_unbound17:41:235
988750992422,2cyclictest988932-21kworker/u16:3+events_unbound15:46:324
988750992422,2cyclictest1435448-21kworker/u16:1+events_unbound20:13:314
988750992422,2cyclictest1435448-21kworker/u16:1+events_unbound20:09:564
988750992422,2cyclictest1424091-21kworker/u16:4+events_unbound18:58:154
988750992422,2cyclictest1424091-21kworker/u16:4+events_unbound18:58:154
988750992422,2cyclictest1101533-21kworker/u16:0+events_unbound16:49:304
988750992422,2cyclictest1023730-21kworker/u16:1+events_unbound16:00:314
98876499230,21cyclictest1372864-21latency_hist18:37:217
988755992321,2cyclictest957774-21kworker/u16:2+events_unbound15:55:005
988755992321,2cyclictest1474990-21kworker/u16:5+events_unbound19:51:305
988755992321,2cyclictest1332518-21kworker/u16:3+events_unbound18:20:545
988755992321,2cyclictest1300479-21kworker/u16:6+events_unbound18:06:305
988755992321,2cyclictest1289618-21kworker/u16:0+events_unbound18:31:525
988755992321,2cyclictest1207938-21kworker/u16:0+flush-8:017:30:035
988755992321,2cyclictest1167769-21kworker/u16:4+events_unbound17:45:435
988755992321,2cyclictest1167769-21kworker/u16:4+events_unbound17:34:525
988755992321,2cyclictest1084233-21kworker/u16:3+flush-8:016:23:235
988755992321,2cyclictest1011597-21kworker/u16:0+events_unbound16:06:405
988750992321,2cyclictest1413807-21kworker/u16:0+events_unbound19:17:304
988750992321,2cyclictest1239807-21kworker/u16:1+events_unbound18:16:004
988750992321,2cyclictest1115223-21kworker/u16:2+events_unbound17:06:324
988750992321,2cyclictest1023730-21kworker/u16:1+flush-8:016:03:324
988750992320,2cyclictest1474990-21kworker/u16:5+events_unbound19:32:514
98876499220,21cyclictest921-21dbus-daemon17:07:217
98876499220,20cyclictest1053877-21latency_hist16:02:217
988755992220,2cyclictest957774-21kworker/u16:2+events_unbound16:16:035
988755992220,2cyclictest1605108-21kworker/u16:4+events_unbound20:51:075
988755992220,2cyclictest1605108-21kworker/u16:4+events_unbound20:36:065
988755992220,2cyclictest1605108-21kworker/u16:4+events_unbound20:36:065
988755992220,2cyclictest1506552-21kworker/u16:4+events_unbound19:53:595
988755992220,2cyclictest1369252-21kworker/u16:2+events_unbound19:18:355
988755992220,2cyclictest1239807-21kworker/u16:1+events_unbound18:17:035
988755992220,2cyclictest1217518-21kworker/u16:3+flush-8:017:23:545
988755992220,2cyclictest1217518-21kworker/u16:3+flush-8:017:23:545
988755992220,2cyclictest1145491-21kworker/u16:5+flush-8:016:54:035
988755992220,2cyclictest1063202-21kworker/u16:4+flush-8:016:20:365
988750992220,2cyclictest935803-21kworker/u16:4+flush-8:015:34:514
988750992220,2cyclictest1352098-21kworker/u16:1+events_unbound18:33:304
988763992118,2cyclictest1167769-21kworker/u16:4+events_unbound18:07:506
988763992117,3cyclictest1413807-21kworker/u16:0+events_unbound19:22:396
988755992119,2cyclictest1465639-21kworker/u16:3+events_unbound19:29:395
988755992119,2cyclictest1289618-21kworker/u16:0+events_unbound18:36:105
988755992119,2cyclictest1217518-21kworker/u16:3+events_unbound17:56:005
988755992119,2cyclictest1207938-21kworker/u16:0+events_unbound17:20:225
988755992119,2cyclictest1063202-21kworker/u16:4+events_unbound16:30:215
988755992118,2cyclictest1084233-21kworker/u16:3+events_unbound17:02:305
988750992119,2cyclictest1589073-21kworker/u16:2+events_unbound20:44:514
988750992119,2cyclictest1474990-21kworker/u16:5+events_unbound19:56:314
988750992119,2cyclictest1435448-21kworker/u16:1+events_unbound19:58:324
988750992119,2cyclictest1369252-21kworker/u16:2+events_unbound18:38:314
988750992119,2cyclictest1352098-21kworker/u16:1+events_unbound18:53:224
988750992119,2cyclictest1145491-21kworker/u16:5+flush-8:017:16:554
988750992119,2cyclictest1063202-21kworker/u16:4+flush-8:016:20:314
988764992019,0cyclictest0-21swapper/719:57:517
988763992018,2cyclictest957774-21kworker/u16:2+events_unbound16:07:526
988764991916,2cyclictest1043575-21munin-run15:57:217
988763991917,2cyclictest1537508-21kworker/u16:0+flush-8:020:22:206
988755991917,2cyclictest935803-21kworker/u16:4+flush-8:015:33:025
988750991917,2cyclictest1435448-21kworker/u16:1+flush-8:020:02:364
988750991917,2cyclictest1217518-21kworker/u16:3+flush-8:017:37:164
988764991816,1cyclictest0-21swapper/717:42:367
98876499180,17cyclictest1021-21rs:main7
98876499180,16cyclictest1609715-21latency_hist20:32:207
988750991816,2cyclictest957774-21kworker/u16:2+events_unbound16:08:474
988750991816,2cyclictest1640716-21kworker/u16:0+flush-8:020:59:074
988750991816,2cyclictest1207938-21kworker/u16:0+events_unbound17:22:204
988750991816,2cyclictest1084233-21kworker/u16:3+events_unbound16:26:244
988764991716,0cyclictest0-21swapper/718:02:427
98876499170,15cyclictest1313-21snmpd17:58:337
988763991715,2cyclictest957774-21kworker/u16:2+flush-8:015:57:216
988763991715,2cyclictest1167769-21kworker/u16:4+flush-8:017:22:516
988763991715,2cyclictest1167769-21kworker/u16:4+flush-8:017:22:516
988763991714,3cyclictest1435448-21kworker/u16:1+events_unbound19:39:196
988755991715,2cyclictest1537508-21kworker/u16:0+flush-8:020:09:065
988755991715,2cyclictest1101533-21kworker/u16:0+events_unbound16:33:285
988750991715,2cyclictest1537508-21kworker/u16:0+flush-8:020:21:034
988750991715,2cyclictest1352098-21kworker/u16:1+events_unbound18:49:224
988750991715,2cyclictest1289618-21kworker/u16:0+flush-8:018:21:564
988764991614,1cyclictest1620000-21munin-run20:37:207
988764991614,1cyclictest1620000-21munin-run20:37:207
988763991614,2cyclictest1167769-21kworker/u16:4+flush-8:018:32:366
988755991614,2cyclictest1352098-21kworker/u16:1+flush-8:018:41:395
988750991614,2cyclictest1474990-21kworker/u16:5+events_unbound19:30:514
988750991614,2cyclictest1332518-21kworker/u16:3+flush-8:018:23:594
988750991614,2cyclictest1289618-21kworker/u16:0+events_unbound18:30:074
988750991614,2cyclictest1289618-21kworker/u16:0+events_unbound17:53:304
988750991614,2cyclictest1145491-21kworker/u16:5+flush-8:016:54:444
988764991513,1cyclictest1157134-21latency_hist16:52:217
988764991511,2cyclictest1496476-21munin-run19:37:207
98876499150,14cyclictest27944-21gnome-software19:38:467
98876499150,13cyclictest1280300-21latency_hist17:52:217
988763991512,2cyclictest1436125-21cut19:07:396
988763991511,3cyclictest1264832-21kthreadcore17:42:406
98876399150,14cyclictest0-21swapper/616:52:456
988750991513,2cyclictest1435448-21kworker/u16:1+events_unbound20:28:324
988750991513,2cyclictest1101533-21kworker/u16:0+events_unbound16:32:364
988750991513,2cyclictest1084233-21kworker/u16:3+flush-8:016:29:284
98876499148,5cyclictest1064145-21turbostat16:07:217
98876499145,8cyclictest1313-21snmpd19:11:167
988764991411,2cyclictest1413201-21ntpq18:52:487
98876499140,1cyclictest0-21swapper/719:27:457
98876499140,1cyclictest0-21swapper/718:47:487
98876499140,13cyclictest1424404-21turbostat19:02:207
98876499140,13cyclictest1424404-21turbostat19:02:207
98876499140,13cyclictest1313-21snmpd15:39:157
98876499140,12cyclictest1084721-21latency_hist16:17:227
988763991412,2cyclictest1435448-21kworker/u16:1+events_unbound19:27:366
988750991411,2cyclictest1217518-21kworker/u16:3+flush-8:017:46:164
988764991311,1cyclictest1033377-21cut15:52:217
98876499130,6cyclictest1259706-21sshd-session17:41:567
98876499130,3cyclictest1639574-21ntpq20:42:467
98876499130,1cyclictest0-21swapper/719:12:457
98876499130,12cyclictest1313-21snmpd20:04:207
98876399137,6cyclictest0-21swapper/620:05:416
988750991312,1cyclictest0-21swapper/419:07:464
98876499126,5cyclictest27763-21gvfs-afc-volume15:42:507
98876499125,6cyclictest27924-21gmain17:22:347
98876499125,6cyclictest27924-21gmain17:22:347
988764991211,0cyclictest0-21swapper/715:32:267
988764991210,1cyclictest1167052-21awk16:57:217
98876499120,2cyclictest1651554-21/usr/sbin/munin20:52:447
98876499120,11cyclictest1313-21snmpd17:55:337
98876499120,10cyclictest1464625-21ntpq19:17:457
98876499120,10cyclictest1313-21snmpd20:48:247
98876399129,2cyclictest1590446-21fschecks_count20:22:366
98876399122,9cyclictest0-21swapper/618:02:376
98876399120,1cyclictest0-21swapper/618:57:246
98876399120,1cyclictest0-21swapper/618:57:246
98876499116,4cyclictest27763-21gvfs-afc-volume20:57:407
98876499115,5cyclictest27763-21gvfs-afc-volume17:27:457
98876499114,6cyclictest27924-21gmain19:22:507
988764991111,0cyclictest0-21swapper/716:24:187
988764991110,0cyclictest0-21swapper/719:43:437
98876499110,9cyclictest1560620-21kthreadcore20:07:397
98876499110,9cyclictest1313-21snmpd18:12:307
98876499110,3cyclictest1318762-21cut18:07:437
98876499110,2cyclictest1621872-21sed20:37:407
98876499110,2cyclictest1590722-21runrttasks20:22:377
98876499110,1cyclictest0-21swapper/719:47:477
98876499110,10cyclictest1313-21snmpd18:28:077
98876399119,2cyclictest1289618-21kworker/u16:0+events_unbound17:57:526
98876399117,3cyclictest1250928-21cat17:37:396
988763991110,1cyclictest0-21swapper/620:45:516
988763991110,1cyclictest0-21swapper/620:40:206
988763991110,1cyclictest0-21swapper/620:34:496
988763991110,1cyclictest0-21swapper/620:34:496
988763991110,1cyclictest0-21swapper/620:14:296
988763991110,1cyclictest0-21swapper/618:56:276
988763991110,1cyclictest0-21swapper/617:56:426
988763991110,1cyclictest0-21swapper/617:22:016
988763991110,1cyclictest0-21swapper/617:05:096
988763991110,1cyclictest0-21swapper/616:49:236
988763991110,1cyclictest0-21swapper/615:58:576
988763991110,1cyclictest0-21swapper/615:36:436
98876399110,1cyclictest0-21swapper/621:00:416
98876399110,1cyclictest0-21swapper/618:38:576
98876399110,1cyclictest0-21swapper/617:37:006
98876399110,1cyclictest0-21swapper/617:02:196
98876399110,11cyclictest0-21swapper/620:53:166
98876399110,11cyclictest0-21swapper/620:51:236
98876399110,11cyclictest0-21swapper/619:56:506
98876399110,11cyclictest0-21swapper/619:36:206
98876399110,11cyclictest0-21swapper/619:15:036
98876399110,11cyclictest0-21swapper/618:43:316
98876399110,11cyclictest0-21swapper/616:35:226
98875099119,2cyclictest1605108-21kworker/u16:4+events_unbound20:33:204
98875099119,2cyclictest1605108-21kworker/u16:4+events_unbound20:33:204
11267652110,1sleep60-21swapper/616:37:366
11267652110,1sleep60-21swapper/616:37:366
98876499107,3cyclictest27763-21gvfs-afc-volume17:32:537
98876499107,2cyclictest27642-21gmain16:42:227
98876499106,3cyclictest1205-21containerd20:12:457
98876499100,9cyclictest1393144-21sshd-session18:42:567
98876499100,9cyclictest1313-21snmpd19:02:377
98876499100,9cyclictest1313-21snmpd18:18:407
98876499100,9cyclictest1313-21snmpd16:37:287
98876499100,9cyclictest1313-21snmpd16:37:287
98876499100,2cyclictest1313-21snmpd20:17:227
98876499100,1cyclictest0-21swapper/718:22:467
98876399109,1cyclictest0-21swapper/619:42:576
98876399109,1cyclictest0-21swapper/618:49:036
98876399108,2cyclictest1063202-21kworker/u16:4+events_unbound16:42:396
9887649990,8cyclictest1313-21snmpd16:18:557
9887649990,2cyclictest1218153-21sshd-session17:17:567
9887639998,1cyclictest0-21swapper/620:29:186
9887639998,1cyclictest0-21swapper/620:08:016
9887639998,1cyclictest0-21swapper/619:50:226
9887639998,1cyclictest0-21swapper/617:08:566
9887639998,1cyclictest0-21swapper/616:12:306
9887639998,1cyclictest0-21swapper/616:06:406
9887639997,2cyclictest931559-21kworker/u16:1+flush-8:015:37:256
9887639995,3cyclictest1086237-21cat16:17:376
9887639987,1cyclictest0-21swapper/615:42:336
9887639986,1cyclictest0-21swapper/618:27:416
9887639986,1cyclictest0-21swapper/617:27:416
9887639986,1cyclictest0-21swapper/616:27:416
9887639984,3cyclictest1024531-21cat15:47:396
9887639983,4cyclictest1457908-21cat19:17:396
9887639983,4cyclictest1426595-21cut19:02:386
9887639983,4cyclictest1342908-21cpuspeed18:22:366
9887639980,7cyclictest0-21swapper/618:17:416
9887639980,7cyclictest0-21swapper/616:22:406
9887639974,2cyclictest0-21swapper/619:57:376
9887639973,3cyclictest1277573-21cut17:47:436
9887639972,4cyclictest1203690-21cut17:12:446
9887649960,5cyclictest988804-21cat15:32:237
9887559940,1cyclictest991rcuop/715:32:235
9887509920,1cyclictest0-21swapper/415:32:234
9887639910,1cyclictest0-21swapper/615:32:226
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional