You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-17 - 17:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Wed Sep 17, 2025 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12499025520,31sleep70-21swapper/703:26:167
12486025343,7sleep40-21swapper/403:24:294
125381994340,2cyclictest3623888-21kworker/u16:2+events_unbound04:15:015
125381994139,2cyclictest3623888-21kworker/u16:2+events_unbound04:10:005
125381994038,2cyclictest522015-21kworker/u16:5+events_unbound07:03:255
12495224030,7sleep60-21swapper/603:25:436
125381993836,2cyclictest463078-21kworker/u16:0+events_unbound07:20:275
125389993735,1cyclictest0-21swapper/708:07:577
125389993735,1cyclictest0-21swapper/708:07:577
125381993735,2cyclictest394374-21kworker/u16:1+events_unbound06:17:355
12492023625,8sleep50-21swapper/503:25:185
125389993534,0cyclictest0-21swapper/706:30:277
125381993432,2cyclictest364890-21kworker/u16:2+flush-8:005:29:275
125381993432,2cyclictest334661-21kworker/u16:3+events_unbound05:17:245
125384993230,2cyclictest473328-21kworker/u16:1+events_unbound06:38:126
125384993230,2cyclictest3623888-21kworker/u16:2+flush-8:003:53:256
125381993230,2cyclictest639759-21kworker/u16:5+events_unbound07:49:075
125381993230,2cyclictest473328-21kworker/u16:1+flush-8:006:36:265
125381993230,2cyclictest344427-21kworker/u16:4+events_unbound05:21:555
125381993227,4cyclictest689756-21latency_hist08:17:575
125384993129,2cyclictest209206-21kworker/u16:0+events_unbound05:03:126
125381993129,2cyclictest611065-21kworker/u16:4+events_unbound08:34:275
125389993028,1cyclictest0-21swapper/703:42:587
125381993028,2cyclictest394374-21kworker/u16:1+flush-8:006:01:265
125381993028,2cyclictest344427-21kworker/u16:4+events_unbound05:36:565
125381993028,2cyclictest257285-21kworker/u16:3+events_unbound04:34:595
125379993026,3cyclictest669880-21kworker/u16:0+flush-8:008:38:264
125381992927,2cyclictest542260-21kworker/u16:1+events_unbound08:04:275
125381992927,2cyclictest542260-21kworker/u16:1+events_unbound08:04:275
125381992927,2cyclictest256812-21kworker/u16:2+events_unbound05:11:555
125381992926,2cyclictest256812-21kworker/u16:2+events_unbound04:52:585
125384992826,2cyclictest738684-21kworker/u16:5+flush-8:008:43:236
125384992826,2cyclictest542260-21kworker/u16:1+flush-8:008:12:576
125389992722,3cyclictest325208-21timerandwakeup05:03:267
125384992725,2cyclictest95470-21kworker/u16:1+flush-8:003:58:236
125384992725,2cyclictest364890-21kworker/u16:2+events_unbound06:03:276
125384992725,2cyclictest3623888-21kworker/u16:2+flush-8:004:03:116
125381992725,2cyclictest669880-21kworker/u16:0+events_unbound08:49:285
125381992725,2cyclictest493708-21kworker/u16:4+events_unbound06:41:085
125381992725,2cyclictest114397-21kworker/u16:0+flush-8:003:59:575
125379992722,4cyclictest758649-21awk08:52:574
125389992626,0cyclictest0-21swapper/707:17:277
125384992624,2cyclictest95470-21kworker/u16:1+flush-8:004:28:216
125384992624,2cyclictest463078-21kworker/u16:0+flush-8:007:08:216
125384992624,2cyclictest256812-21kworker/u16:2+flush-8:005:08:236
125384992623,2cyclictest463078-21kworker/u16:0+flush-8:006:58:196
125381992624,2cyclictest601150-21kworker/u16:2+events_unbound07:31:265
125381992624,2cyclictest209206-21kworker/u16:0+events_unbound04:50:245
125384992523,2cyclictest443205-21kworker/u16:4+events_unbound06:14:436
125381992523,2cyclictest749166-21kworker/u16:1+events_unbound08:55:265
125381992523,2cyclictest276959-21kworker/u16:4+flush-8:004:43:225
125381992523,2cyclictest159431-21kworker/u16:3+events_unbound03:46:255
125381992522,2cyclictest209206-21kworker/u16:0+flush-8:004:38:225
125389992421,2cyclictest234242-21kthreadcore04:18:197
125384992422,2cyclictest209206-21kworker/u16:0+flush-8:004:38:236
125384992419,4cyclictest296297-21latency_hist04:52:576
125381992422,2cyclictest394881-21kworker/u16:3+events_unbound06:20:275
125381992422,2cyclictest364890-21kworker/u16:2+events_unbound06:07:225
125381992422,2cyclictest256812-21kworker/u16:2+events_unbound05:05:115
125381992421,2cyclictest344427-21kworker/u16:4+flush-8:005:38:075
125381992421,2cyclictest344427-21kworker/u16:4+flush-8:005:38:075
125389992322,0cyclictest0-21swapper/707:38:537
125389992321,1cyclictest0-21swapper/704:03:207
125389992315,6cyclictest589235-21systemd-journal08:22:577
125384992321,2cyclictest3623888-21kworker/u16:2+flush-8:004:08:256
125389992221,0cyclictest0-21swapper/708:13:117
125384992220,2cyclictest601150-21kworker/u16:2+flush-8:007:33:246
125384992220,2cyclictest542260-21kworker/u16:1+flush-8:007:28:276
125384992220,2cyclictest334661-21kworker/u16:3+events_unbound05:14:226
125381992220,2cyclictest669880-21kworker/u16:0+events_unbound08:25:265
125381992220,2cyclictest639759-21kworker/u16:5+events_unbound07:55:475
125381992220,2cyclictest542260-21kworker/u16:1+events_unbound07:10:275
125381992220,2cyclictest522015-21kworker/u16:5+flush-8:007:24:085
125381992219,3cyclictest679857-21kworker/u16:2+flush-8:008:30:565
125379992218,3cyclictest611065-21kworker/u16:4+flush-8:008:27:574
125389992120,0cyclictest0-21swapper/708:56:267
125389992120,0cyclictest0-21swapper/708:48:267
125389992120,0cyclictest0-21swapper/705:57:267
12538999210,20cyclictest522812-21turbostat06:52:567
12538999210,19cyclictest394490-21turbostat05:42:577
12538999210,19cyclictest394490-21turbostat05:42:577
125384992119,2cyclictest95470-21kworker/u16:1+flush-8:005:23:226
125384992119,2cyclictest95470-21kworker/u16:1+flush-8:004:18:126
125384992119,2cyclictest690154-21kworker/u16:3+flush-8:008:18:116
125384992119,2cyclictest639759-21kworker/u16:5+flush-8:008:03:126
125384992119,2cyclictest639759-21kworker/u16:5+flush-8:008:03:126
125384992119,2cyclictest590486-21kworker/u16:3+flush-8:007:23:196
125384992119,2cyclictest473328-21kworker/u16:1+events_unbound06:33:116
125384992119,2cyclictest114397-21kworker/u16:0+flush-8:003:28:236
125384992118,2cyclictest611065-21kworker/u16:4+events_unbound07:48:166
125381992119,2cyclictest611065-21kworker/u16:4+events_unbound08:43:135
125381992119,2cyclictest542260-21kworker/u16:1+events_unbound08:19:035
125381992119,2cyclictest522015-21kworker/u16:5+events_unbound06:51:265
125381992119,2cyclictest364890-21kworker/u16:2+flush-8:005:43:595
125389992019,0cyclictest0-21swapper/706:43:417
125384992018,2cyclictest749166-21kworker/u16:1+flush-8:008:48:236
125384992018,2cyclictest611065-21kworker/u16:4+flush-8:008:23:276
125384992018,2cyclictest611065-21kworker/u16:4+events_unbound07:53:276
125384992018,2cyclictest611065-21kworker/u16:4+events_unbound07:45:346
125384992018,2cyclictest463078-21kworker/u16:0+flush-8:006:18:236
125384992018,2cyclictest394881-21kworker/u16:3+events_unbound05:48:546
125384992018,2cyclictest3623888-21kworker/u16:2+flush-8:004:13:186
125384992018,2cyclictest139948-21kworker/u16:4+events_unbound03:33:166
125381992018,2cyclictest3623888-21kworker/u16:2+events_unbound03:29:085
3189662190,1sleep40-21swapper/405:03:164
125389991918,0cyclictest0-21swapper/706:34:597
125389991916,2cyclictest306154-21turbostat04:57:577
12538999190,17cyclictest0-21swapper/706:53:237
125384991917,2cyclictest611065-21kworker/u16:4+events_unbound07:38:556
125384991917,2cyclictest542260-21kworker/u16:1+flush-8:008:13:196
125384991917,2cyclictest463078-21kworker/u16:0+events_unbound07:14:106
125384991917,2cyclictest394881-21kworker/u16:3+events_unbound05:58:246
125381991917,2cyclictest95470-21kworker/u16:1+flush-8:004:07:075
125381991917,2cyclictest542260-21kworker/u16:1+flush-8:007:58:325
125381991917,2cyclictest542260-21kworker/u16:1+flush-8:007:58:325
125381991917,2cyclictest522015-21kworker/u16:5+events_unbound07:16:255
125381991917,2cyclictest463078-21kworker/u16:0+events_unbound06:55:075
125389991817,0cyclictest0-21swapper/708:34:537
125389991817,0cyclictest0-21swapper/705:20:147
125389991815,2cyclictest1313-21snmpd06:18:247
125384991816,2cyclictest611065-21kworker/u16:4+events_unbound08:42:556
125384991816,2cyclictest473328-21kworker/u16:1+events_unbound06:25:266
125384991816,2cyclictest463078-21kworker/u16:0+events_unbound06:53:136
125384991816,2cyclictest394881-21kworker/u16:3+flush-8:006:08:196
125384991816,2cyclictest276959-21kworker/u16:4+flush-8:004:43:186
125381991816,2cyclictest95470-21kworker/u16:1+events_unbound03:34:595
125381991816,2cyclictest639759-21kworker/u16:5+events_unbound08:09:265
125381991816,2cyclictest611065-21kworker/u16:4+flush-8:007:43:075
125381991816,2cyclictest463078-21kworker/u16:0+flush-8:006:58:275
125379991814,3cyclictest114397-21kworker/u16:0+flush-8:003:28:274
125389991713,2cyclictest0-21swapper/703:58:217
12538999170,15cyclictest0-21swapper/704:48:117
125384991715,2cyclictest95470-21kworker/u16:1+flush-8:004:53:186
125384991715,2cyclictest95470-21kworker/u16:1+flush-8:003:42:356
125384991715,2cyclictest95470-21kworker/u16:1+events_unbound04:23:376
125384991715,2cyclictest679857-21kworker/u16:2+flush-8:008:34:326
125384991715,2cyclictest522015-21kworker/u16:5+events_unbound07:21:376
125384991715,2cyclictest463078-21kworker/u16:0+events_unbound06:48:206
125384991715,2cyclictest209206-21kworker/u16:0+events_unbound05:47:046
125384991715,2cyclictest114397-21kworker/u16:0+events_unbound03:46:596
125381991715,2cyclictest590486-21kworker/u16:3+flush-8:007:41:145
125381991715,2cyclictest473328-21kworker/u16:1+events_unbound06:30:005
125381991715,2cyclictest209206-21kworker/u16:0+flush-8:004:27:075
125379991715,2cyclictest364890-21kworker/u16:2+events_unbound06:08:124
125379991712,4cyclictest188199-21latency_hist03:57:584
12537999170,1cyclictest0-21swapper/403:44:294
12538999169,6cyclictest384725-21turbostat05:37:577
125389991614,1cyclictest0-21swapper/706:26:277
125389991612,3cyclictest1313-21snmpd05:17:477
125384991614,2cyclictest749166-21kworker/u16:1+flush-8:008:53:246
125384991614,2cyclictest394881-21kworker/u16:3+events_unbound05:57:576
125384991614,2cyclictest364890-21kworker/u16:2+events_unbound07:07:006
125384991613,2cyclictest473328-21kworker/u16:1+flush-8:006:28:136
125381991614,2cyclictest611065-21kworker/u16:4+events_unbound07:36:075
125381991613,2cyclictest3623888-21kworker/u16:2+flush-8:003:53:265
12537999165,10cyclictest0-21swapper/404:53:114
12537999160,1cyclictest0-21swapper/403:49:324
12537999160,1cyclictest0-21swapper/403:42:264
12537999160,1cyclictest0-21swapper/403:33:214
125389991513,1cyclictest0-21swapper/707:58:287
125389991513,1cyclictest0-21swapper/707:58:287
125389991513,1cyclictest0-21swapper/704:23:267
12538999150,14cyclictest0-21swapper/704:33:267
12538999150,13cyclictest589235-21systemd-journal08:47:577
125384991513,2cyclictest639759-21kworker/u16:5+events_unbound07:58:246
125384991513,2cyclictest639759-21kworker/u16:5+events_unbound07:58:246
125384991513,2cyclictest542260-21kworker/u16:1+events_unbound08:31:576
125384991513,2cyclictest364890-21kworker/u16:2+events_unbound05:33:296
125384991513,2cyclictest209206-21kworker/u16:0+flush-8:005:18:506
125384991513,2cyclictest209206-21kworker/u16:0+flush-8:004:58:296
125384991512,2cyclictest169706-21hddtemp_smartct03:48:166
125381991513,2cyclictest394881-21kworker/u16:3+flush-8:006:08:235
125381991513,2cyclictest394374-21kworker/u16:1+events_unbound05:56:075
125381991513,2cyclictest3623888-21kworker/u16:2+flush-8:004:21:585
125381991513,2cyclictest209206-21kworker/u16:0+events_unbound05:01:265
125379991512,2cyclictest542260-21kworker/u16:1+flush-8:007:18:224
125389991413,0cyclictest0-21swapper/707:00:077
125389991411,2cyclictest1313-21snmpd07:18:477
12538999140,12cyclictest0-21swapper/707:08:227
125384991412,2cyclictest364890-21kworker/u16:2+events_unbound05:29:036
125384991412,2cyclictest344427-21kworker/u16:4+events_unbound05:42:136
125384991412,2cyclictest344427-21kworker/u16:4+events_unbound05:42:136
125384991412,2cyclictest257285-21kworker/u16:3+events_unbound04:34:466
125381991412,2cyclictest738684-21kworker/u16:5+flush-8:008:42:525
125381991412,2cyclictest463078-21kworker/u16:0+events_unbound06:23:415
125381991412,2cyclictest3623888-21kworker/u16:2+events_unbound03:41:585
125381991412,2cyclictest209206-21kworker/u16:0+events_unbound05:48:265
12538199140,1cyclictest0-21swapper/503:48:295
12537999149,4cyclictest335592-21latency_hist05:12:574
125379991413,1cyclictest0-21swapper/407:28:244
125379991412,2cyclictest0-21swapper/405:36:094
125379991410,3cyclictest355370-21cat05:22:574
12537999140,1cyclictest0-21swapper/407:00:484
12537999140,1cyclictest0-21swapper/405:57:574
12537999140,10cyclictest0-21swapper/404:22:054
12538999136,6cyclictest27763-21gvfs-afc-volume04:58:167
12538999135,7cyclictest0-21swapper/703:29:257
12538999134,7cyclictest0-21swapper/703:48:247
125389991312,0cyclictest0-21swapper/703:57:587
125384991311,2cyclictest493708-21kworker/u16:4+flush-8:006:45:596
12537999133,4cyclictest492570-21cat06:33:264
125379991312,1cyclictest0-21swapper/406:24:524
125379991311,2cyclictest209206-21kworker/u16:0+flush-8:004:43:504
125379991311,2cyclictest0-21swapper/408:10:484
12537999130,13cyclictest0-21swapper/406:41:254
12538999129,2cyclictest277083-21/usr/sbin/munin04:43:237
12538999129,2cyclictest1313-21snmpd07:56:357
12538999129,2cyclictest1313-21snmpd05:47:187
12538999128,3cyclictest542909-21/usr/sbin/munin07:03:167
12538999128,3cyclictest1313-21snmpd06:39:497
12538999127,4cyclictest27763-21gvfs-afc-volume03:43:087
12538999122,9cyclictest0-21swapper/707:28:257
12538999121,1cyclictest1127-21ntpd05:58:187
125389991210,1cyclictest364986-21turbostat05:27:577
12538199127,4cyclictest513211-21cat06:47:585
125381991210,2cyclictest209206-21kworker/u16:0+events_unbound04:31:265
12538199120,1cyclictest0-21swapper/505:26:325
12537999128,3cyclictest543704-21cat07:03:164
12537999128,3cyclictest364890-21kworker/u16:2+flush-8:006:53:274
12537999123,5cyclictest394499-21perf05:42:574
12537999123,5cyclictest394499-21perf05:42:574
12537999123,4cyclictest455540-21cut06:13:154
12537999123,4cyclictest342903-21grep05:13:194
12537999122,4cyclictest739356-21grep08:43:114
12537999122,4cyclictest663561-21kthreadcore08:03:134
12537999122,4cyclictest663561-21kthreadcore08:03:134
125379991211,1cyclictest0-21swapper/408:28:374
125379991211,1cyclictest0-21swapper/406:32:074
125379991210,2cyclictest0-21swapper/403:59:034
12537999120,9cyclictest365009-21idleruntime-cro05:27:574
12537999120,1cyclictest0-21swapper/404:16:234
12537999120,1cyclictest0-21swapper/404:09:554
12537999120,12cyclictest0-21swapper/407:54:434
12537999120,12cyclictest0-21swapper/407:35:294
12538999119,1cyclictest0-21swapper/708:28:137
12538999118,2cyclictest27204-21gmain04:08:157
12538999118,2cyclictest1313-21snmpd05:49:097
12538999118,2cyclictest1313-21snmpd05:28:297
12538999118,2cyclictest0-21swapper/704:28:227
12538999117,3cyclictest27763-21gvfs-afc-volume06:08:177
12538999115,5cyclictest347170-21sendmail08:38:267
12538999113,7cyclictest27642-21gmain06:03:227
125389991110,1cyclictest0-21swapper/707:33:197
12538999110,2cyclictest0-21swapper/708:23:227
12538999110,2cyclictest0-21swapper/706:13:227
12538999110,10cyclictest590957-21awk07:23:237
12538999110,10cyclictest1313-21snmpd08:08:187
12538999110,10cyclictest1313-21snmpd04:38:017
12538999110,10cyclictest1313-21snmpd04:15:157
12538999110,10cyclictest0-21swapper/703:33:257
12537999114,6cyclictest763030-21kthreadcore08:53:174
12537999114,6cyclictest413142-21nfs4_client05:48:214
12537999114,6cyclictest199447-21irqcore04:03:144
12537999113,2cyclictest395723-21irqcore05:43:184
12537999112,4cyclictest725875-21cut08:33:184
12537999112,4cyclictest698242-21cut08:18:194
12537999112,4cyclictest651860-21sed07:58:174
12537999112,4cyclictest651860-21sed07:58:174
12537999112,4cyclictest557760-21grep07:08:174
12537999112,3cyclictest635383-21kthreadcore07:48:184
125379991110,1cyclictest0-21swapper/408:15:514
125379991110,1cyclictest0-21swapper/406:21:434
125379991110,1cyclictest0-21swapper/405:00:504
125379991110,1cyclictest0-21swapper/404:49:484
12537999110,11cyclictest0-21swapper/407:24:184
12537999110,0cyclictest0-21swapper/407:16:064
125389991010,0cyclictest0-21swapper/705:12:537
12538999100,9cyclictest1313-21snmpd07:46:057
12538999100,2cyclictest0-21swapper/707:48:137
12537999109,1cyclictest0-21swapper/405:58:044
12537999105,4cyclictest0-21swapper/406:03:184
12537999103,2cyclictest611811-21cpuspeed_turbos07:38:104
12537999103,2cyclictest367126-21kthreadcore05:28:154
12537999103,2cyclictest254933-21kthreadcore04:28:194
12537999102,4cyclictest242148-21kthreadcore04:23:214
12537999102,3cyclictest504890-21cat06:43:174
12537999100,9cyclictest0-21swapper/404:38:134
12537999100,4cyclictest623220-21cut07:43:154
12537999100,4cyclictest522372-21/usr/sbin/munin06:48:254
1253799990,3cyclictest257586-21cpuspeed_turbos04:33:124
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional