You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 05:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Sun Jul 26, 2026 00:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29693826250,8sleep50-21swapper/519:05:505
29711225217,30sleep70-21swapper/719:08:167
297476994640,5cyclictest557654-21latency_hist21:19:137
297476994541,2cyclictest727694-21munin-run22:29:127
297471994239,3cyclictest821290-21kworker/u16:2+flush-8:023:44:126
29747699410,39cyclictest320659-21latency_hist19:19:137
297458994139,2cyclictest224055-21kworker/u16:1+flush-8:019:19:134
29691023928,8sleep60-21swapper/619:05:266
29688023726,8sleep40-21swapper/419:05:004
297471993431,2cyclictest378764-21kworker/u16:4+flush-8:020:29:136
297466993228,3cyclictest934015-21kworker/u16:4+flush-8:000:09:135
297476993125,5cyclictest648-21systemd-journal19:24:137
297458993128,2cyclictest821290-21kworker/u16:2+flush-8:023:14:134
8619792300,1chrt0-21swapper/523:24:235
297471992927,2cyclictest320662-21kworker/u16:2+events_unbound19:24:436
297471992927,2cyclictest320662-21kworker/u16:2+events_unbound19:24:436
297466992927,2cyclictest436857-21kworker/u16:1+events_unbound22:04:315
297458992926,2cyclictest702270-21kworker/u16:3+flush-8:023:34:364
7530512270,0sleep50-21swapper/522:39:285
297476992726,0cyclictest0-21swapper/723:09:227
297476992621,4cyclictest861318-21latency_hist23:24:137
297476992621,4cyclictest581968-21latency_hist21:29:127
297458992623,2cyclictest378764-21kworker/u16:4+events_unbound21:19:304
297476992520,4cyclictest824808-21latency_hist23:09:127
297471992523,2cyclictest436857-21kworker/u16:1+flush-8:021:39:286
297471992522,2cyclictest378764-21kworker/u16:4+flush-8:021:14:186
297466992521,3cyclictest378764-21kworker/u16:4+flush-8:022:34:185
297458992523,2cyclictest436857-21kworker/u16:1+events_unbound21:09:364
297476992420,3cyclictest125-21kswapd000:34:127
297458992320,2cyclictest378764-21kworker/u16:4+flush-8:021:54:454
297476992220,1cyclictest533346-21diskmemload21:34:417
297458992220,2cyclictest436857-21kworker/u16:1+flush-8:020:34:394
297458992220,2cyclictest378764-21kworker/u16:4+flush-8:020:24:414
297458992220,2cyclictest378764-21kworker/u16:4+events_unbound21:29:424
29747699210,19cyclictest921848-21latency_hist23:49:127
297471992119,2cyclictest378764-21kworker/u16:4+events_unbound21:24:276
297476992019,0cyclictest0-21swapper/719:24:307
297476992019,0cyclictest0-21swapper/719:24:307
297476992014,5cyclictest648-21systemd-journal21:44:127
297471992017,2cyclictest297244-21kworker/u16:3+flush-8:019:29:396
297476991918,0cyclictest0-21swapper/720:34:317
29747699190,17cyclictest928599-21cut23:49:357
297458991917,2cyclictest205779-21kworker/u16:0+flush-8:019:39:494
297458991916,3cyclictest297244-21kworker/u16:3+flush-8:019:29:314
297476991814,3cyclictest885496-21latency_hist23:34:127
297476991813,4cyclictest982697-21latency_hist00:14:127
29747699180,16cyclictest388384-21sed19:54:137
297471991816,2cyclictest436857-21kworker/u16:1+events_unbound21:44:376
297471991816,2cyclictest436857-21kworker/u16:1+events_unbound20:19:366
297471991815,2cyclictest378763-21kworker/u16:3+flush-8:020:09:316
297471991815,2cyclictest378763-21kworker/u16:3+flush-8:020:09:316
297458991815,2cyclictest670624-21kworker/u16:0+flush-8:022:19:124
297476991713,3cyclictest1302-21snmpd23:01:277
297476991713,3cyclictest1302-21snmpd23:01:277
297476991712,4cyclictest691263-21latency_hist22:14:127
29747699170,1cyclictest0-21swapper/720:09:397
29747699170,1cyclictest0-21swapper/720:09:397
29747699170,15cyclictest870447-21sshd-session23:24:417
29747699170,15cyclictest514103-21cat20:59:137
297471991716,1cyclictest0-21swapper/620:32:026
297471991715,2cyclictest378764-21kworker/u16:4+flush-8:020:44:406
297471991715,2cyclictest378763-21kworker/u16:3+flush-8:020:14:186
297471991715,2cyclictest359664-21kworker/u16:2+flush-8:019:49:136
297471991715,2cyclictest339650-21kworker/u16:1+events_unbound19:39:546
297466991713,3cyclictest968280-21kworker/u16:0+flush-8:000:34:485
29747699160,14cyclictest1767-21dockerd19:09:377
297471991613,2cyclictest763084-21kworker/u16:1+flush-8:022:54:416
297458991613,3cyclictest378764-21kworker/u16:4+events_unbound21:24:394
29747699159,5cyclictest349642-21latency_hist19:34:137
297476991513,1cyclictest1016814-21ssh00:24:537
29747699150,13cyclictest1302-21snmpd21:53:287
297471991513,2cyclictest436857-21kworker/u16:1+flush-8:020:34:286
297471991513,2cyclictest378764-21kworker/u16:4+flush-8:019:54:446
297458991513,2cyclictest702270-21kworker/u16:3+flush-8:023:19:434
297458991513,2cyclictest569870-21kworker/u16:2+events_unbound21:39:424
29745899150,14cyclictest0-21swapper/423:54:214
297476991413,0cyclictest0-21swapper/720:24:377
29747699140,13cyclictest0-21swapper/721:44:397
29747199140,14cyclictest0-21swapper/600:29:106
297466991413,1cyclictest0-21swapper/523:59:295
297466991413,1cyclictest0-21swapper/521:42:485
297466991413,1cyclictest0-21swapper/521:09:225
29746699140,1cyclictest0-21swapper/521:25:465
29746699140,14cyclictest0-21swapper/522:31:595
29746699140,14cyclictest0-21swapper/522:20:005
29746699140,14cyclictest0-21swapper/521:59:395
29746699140,14cyclictest0-21swapper/500:19:025
297458991413,1cyclictest0-21swapper/423:40:194
297458991413,1cyclictest0-21swapper/423:31:294
297458991413,1cyclictest0-21swapper/421:16:124
297458991413,1cyclictest0-21swapper/400:23:504
297458991413,1cyclictest0-21swapper/400:23:504
297458991410,3cyclictest1031373-21kworker/u16:3+flush-8:000:34:174
29745899140,1cyclictest0-21swapper/423:15:434
29745899140,1cyclictest0-21swapper/423:02:574
29745899140,1cyclictest0-21swapper/423:02:574
29745899140,1cyclictest0-21swapper/422:38:494
29745899140,14cyclictest0-21swapper/422:40:154
29745899140,14cyclictest0-21swapper/422:26:224
29745899140,14cyclictest0-21swapper/421:51:504
29745899140,14cyclictest0-21swapper/400:11:224
29747699139,3cyclictest1302-21snmpd21:06:517
29747699139,3cyclictest1302-21snmpd20:42:357
29747699139,3cyclictest1302-21snmpd20:04:017
29747699138,4cyclictest886806-21cat23:34:297
29747699137,4cyclictest692573-21cat22:14:307
297476991310,2cyclictest1302-21snmpd00:20:327
297476991310,2cyclictest1302-21snmpd00:20:327
297476991310,2cyclictest0-21swapper/722:19:417
29747699130,3cyclictest677319-21ssh22:05:577
29747699130,2cyclictest959421-21df_inode00:04:297
29747699130,1cyclictest0-21swapper/720:59:387
29747699130,1cyclictest0-21swapper/720:59:387
297471991312,1cyclictest0-21swapper/600:24:076
297471991312,1cyclictest0-21swapper/600:24:076
297471991311,2cyclictest821290-21kworker/u16:2+flush-8:023:29:386
297471991310,3cyclictest504482-21kworker/u16:0+flush-8:021:04:386
29747199130,1cyclictest0-21swapper/623:22:006
29747199130,1cyclictest0-21swapper/622:36:076
29747199130,13cyclictest0-21swapper/623:36:586
29747199130,13cyclictest0-21swapper/622:30:176
29747199130,13cyclictest0-21swapper/622:16:156
297466991312,1cyclictest0-21swapper/523:53:585
297466991312,1cyclictest0-21swapper/523:44:215
297466991312,1cyclictest0-21swapper/523:03:225
297466991312,1cyclictest0-21swapper/523:03:225
297466991312,1cyclictest0-21swapper/521:51:095
297466991312,1cyclictest0-21swapper/520:59:265
297466991312,1cyclictest0-21swapper/520:59:265
297466991312,1cyclictest0-21swapper/519:32:435
297466991312,1cyclictest0-21swapper/500:25:305
297466991312,1cyclictest0-21swapper/500:10:315
297458991312,1cyclictest0-21swapper/423:48:504
297458991312,1cyclictest0-21swapper/422:50:484
297458991310,2cyclictest378764-21kworker/u16:4+flush-8:020:59:274
297458991310,2cyclictest378764-21kworker/u16:4+flush-8:020:59:274
29745899130,12cyclictest0-21swapper/419:19:404
29747699129,2cyclictest1302-21snmpd23:16:187
29747699129,2cyclictest0-21swapper/722:39:137
29747699128,3cyclictest1302-21snmpd21:22:247
29747699128,3cyclictest1302-21snmpd00:02:287
297476991212,0cyclictest0-21swapper/723:58:057
297476991211,0cyclictest0-21swapper/720:49:317
297476991210,1cyclictest751904-21awk22:39:137
297476991210,1cyclictest0-21swapper/723:39:377
297476991210,1cyclictest0-21swapper/719:44:387
29747699120,2cyclictest652666-21ssh21:55:247
29747699120,10cyclictest1302-21snmpd19:39:557
29747199129,2cyclictest702270-21kworker/u16:3+flush-8:022:24:366
297471991211,1cyclictest0-21swapper/623:10:586
297471991211,1cyclictest0-21swapper/622:52:126
297471991211,1cyclictest0-21swapper/600:19:046
29747199120,4cyclictest410067-21kthreadcore20:04:316
29747199120,12cyclictest0-21swapper/623:19:006
29747199120,11cyclictest0-21swapper/619:19:216
29747199120,0cyclictest0-21swapper/622:04:446
297466991211,1cyclictest0-21swapper/523:41:215
297466991211,1cyclictest0-21swapper/523:22:075
297466991211,1cyclictest0-21swapper/523:15:115
297466991211,1cyclictest0-21swapper/523:08:155
297466991211,1cyclictest0-21swapper/521:36:485
297466991211,1cyclictest0-21swapper/521:32:425
297466991211,1cyclictest0-21swapper/520:33:255
297466991211,1cyclictest0-21swapper/500:19:315
297466991211,1cyclictest0-21swapper/500:19:315
29746699120,1cyclictest0-21swapper/520:05:315
29746699120,12cyclictest0-21swapper/520:57:045
29745899129,2cyclictest378764-21kworker/u16:4+flush-8:022:44:564
29745899125,7cyclictest0-21swapper/420:39:284
297458991211,1cyclictest0-21swapper/423:59:524
297458991211,1cyclictest0-21swapper/420:10:184
297458991211,1cyclictest0-21swapper/420:10:184
297458991211,1cyclictest0-21swapper/419:25:134
297458991211,1cyclictest0-21swapper/419:25:134
297458991211,1cyclictest0-21swapper/400:14:504
29747699118,2cyclictest1302-21snmpd22:03:087
29747699118,2cyclictest1302-21snmpd21:09:247
29747699118,2cyclictest0-21swapper/722:54:277
29747699118,2cyclictest0-21swapper/700:34:427
29747699117,3cyclictest5965-21gmain20:44:347
29747699117,3cyclictest5965-21gmain19:54:147
29747699117,3cyclictest1302-21snmpd20:14:477
29747699110,10cyclictest774387-21ssh22:45:577
29747699110,10cyclictest594024-21ssh21:34:047
29747699110,10cyclictest1302-21snmpd20:31:087
29747699110,10cyclictest1302-21snmpd00:15:267
29747199119,2cyclictest968280-21kworker/u16:0+flush-8:000:29:186
29747199119,2cyclictest504482-21kworker/u16:0+events_unbound20:54:376
29747199119,2cyclictest378764-21kworker/u16:4+flush-8:019:59:296
29747199118,2cyclictest378764-21kworker/u16:4+flush-8:020:59:286
29747199118,2cyclictest378764-21kworker/u16:4+flush-8:020:59:286
29747199116,4cyclictest595189-21tune2fs21:34:286
297471991110,1cyclictest0-21swapper/623:49:356
297471991110,1cyclictest0-21swapper/623:01:026
297471991110,1cyclictest0-21swapper/623:01:026
29747199110,11cyclictest0-21swapper/619:16:506
29747199110,10cyclictest352310-21kthreadcore19:34:316
29747199110,0cyclictest0-21swapper/620:42:556
29746699119,2cyclictest0-21swapper/520:23:105
297466991110,1cyclictest0-21swapper/523:11:435
297466991110,1cyclictest0-21swapper/522:51:425
29745899117,3cyclictest619649-21cat21:44:294
297458991110,1cyclictest0-21swapper/421:37:584
297458991110,1cyclictest0-21swapper/420:01:384
297458991110,1cyclictest0-21swapper/419:55:014
297458991110,1cyclictest0-21swapper/419:36:534
297458991110,1cyclictest0-21swapper/400:24:184
297458991110,1cyclictest0-21swapper/400:08:234
29747699107,2cyclictest1302-21snmpd22:49:217
29747699107,2cyclictest1302-21snmpd22:29:177
29747699107,2cyclictest1302-21snmpd20:07:557
29747699106,3cyclictest1302-21snmpd19:34:367
29747199109,1cyclictest0-21swapper/622:42:256
29747199109,1cyclictest0-21swapper/619:50:446
29747199108,2cyclictest86-21ksoftirqd/621:20:446
29747199106,3cyclictest631815-21cat21:49:296
29747199100,9cyclictest589777-21kthreadcore21:29:346
29747199100,3cyclictest300471-21cut19:09:316
29746699109,1cyclictest0-21swapper/523:37:255
29746699109,1cyclictest0-21swapper/523:33:285
29746699109,1cyclictest0-21swapper/519:53:515
29746699109,1cyclictest0-21swapper/519:48:485
29746699107,2cyclictest495043-21/usr/sbin/munin20:49:385
29745899109,1cyclictest0-21swapper/423:27:324
29745899109,1cyclictest0-21swapper/420:17:054
29745899109,1cyclictest0-21swapper/400:33:464
2974769998,0cyclictest0-21swapper/720:19:347
2974719998,1cyclictest0-21swapper/622:23:306
2974719997,2cyclictest86-21ksoftirqd/623:59:526
2974719997,2cyclictest86-21ksoftirqd/622:44:446
2974719997,2cyclictest86-21ksoftirqd/622:12:566
2974719995,3cyclictest534278-21expr21:09:266
2974719995,3cyclictest495824-21cat20:49:296
2974719993,5cyclictest0-21swapper/621:54:306
2974669998,1cyclictest2909475-21kworker/5:0+events_power_efficient19:23:065
2974669998,1cyclictest0-21swapper/523:54:275
2974669998,1cyclictest0-21swapper/521:04:385
2974669998,1cyclictest0-21swapper/520:42:155
2974669998,1cyclictest0-21swapper/520:17:205
2974669998,1cyclictest0-21swapper/520:10:055
2974669998,1cyclictest0-21swapper/520:10:055
2974669998,1cyclictest0-21swapper/519:41:145
2974669996,3cyclictest436857-21kworker/u16:1+flush-8:021:49:125
2974669996,2cyclictest312224-21irqcore19:14:305
2974669994,4cyclictest0-21swapper/522:45:205
2974669990,8cyclictest0-21swapper/519:09:295
2974589998,1cyclictest0-21swapper/422:57:074
2974589995,3cyclictest673153-21kthreadcore22:04:354
2974719987,1cyclictest0-21swapper/600:09:566
2974719986,2cyclictest968280-21kworker/u16:0+flush-8:000:34:186
2974719986,2cyclictest86-21ksoftirqd/623:59:106
2974719986,2cyclictest86-21ksoftirqd/623:24:566
2974719986,2cyclictest86-21ksoftirqd/623:04:396
2974719986,2cyclictest378764-21kworker/u16:4+flush-8:021:59:246
2974669987,1cyclictest0-21swapper/522:58:005
2974669984,3cyclictest644159-21irqcore21:54:295
2974669983,4cyclictest559707-21cut21:19:315
2974669983,4cyclictest552712-21grep21:14:365
2974669983,4cyclictest485427-21awk20:44:255
2974669983,4cyclictest393272-21kthreadcore19:54:345
2974669983,4cyclictest338802-21sed19:24:355
2974669983,4cyclictest338802-21sed19:24:355
2974669980,1cyclictest0-21swapper/519:34:315
2974589987,1cyclictest0-21swapper/423:49:184
2974589987,1cyclictest0-21swapper/423:04:224
2974589986,2cyclictest652972-21kworker/u16:2+events_unbound21:59:384
2974589986,1cyclictest0-21swapper/421:04:314
2974589986,1cyclictest0-21swapper/420:54:294
2974589986,1cyclictest0-21swapper/420:49:334
2974589986,1cyclictest0-21swapper/420:04:324
2974589986,1cyclictest0-21swapper/419:49:254
2974589986,1cyclictest0-21swapper/419:09:334
2974589985,2cyclictest378763-21kworker/u16:3+flush-8:020:19:314
2974589984,3cyclictest1-21systemd20:49:124
2974589983,4cyclictest460295-21tr20:29:334
2974589983,4cyclictest373387-21grep19:44:334
2974719971,6cyclictest0-21swapper/623:44:266
2974719970,7cyclictest0-21swapper/600:04:496
2974669973,3cyclictest447468-21sed20:24:285
2974669972,4cyclictest473782-21kthreadcore20:34:355
2974669972,4cyclictest399422-21cut19:59:315
2974669970,5cyclictest0-21swapper/522:25:595
2974669970,3cyclictest0-21swapper/522:09:315
2974589975,2cyclictest670624-21kworker/u16:0+flush-8:022:19:564
2974589973,1cyclictest681rcuog/422:30:354
2974589960,4cyclictest0-21swapper/422:11:584
2974669950,5cyclictest0-21swapper/522:14:315
2974669950,4cyclictest1019590-21chrt00:29:235
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional