You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-19 - 15:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri Apr 19, 2024 12:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
303897025520,30sleep70-21swapper/707:05:577
303898823323,7sleep50-21swapper/507:06:115
302970323321,8sleep40-21swapper/407:04:094
3039489993028,1cyclictest0-21swapper/712:14:257
3039480992826,2cyclictest3575640-21kworker/u16:2+flush-8:011:49:594
3039489992726,0cyclictest0-21swapper/711:24:177
3039489992726,0cyclictest0-21swapper/711:19:307
3039489992726,0cyclictest0-21swapper/708:29:197
3039489992726,0cyclictest0-21swapper/707:49:217
3039480992725,2cyclictest3535568-21kworker/u16:2+events_unbound11:24:074
3039489992625,0cyclictest0-21swapper/708:39:427
3039480992621,4cyclictest3447673-21latency_hist10:39:064
3039489992524,0cyclictest0-21swapper/707:24:307
3039480992522,3cyclictest3019244-21kworker/u16:4+flush-8:008:14:344
3039480992520,4cyclictest3348503-21latency_hist09:44:074
3039489992423,0cyclictest0-21swapper/710:19:227
3039489992423,0cyclictest0-21swapper/709:49:307
3039486992420,3cyclictest3408698-21kworker/u16:0+flush-8:010:24:376
30376152243,5sleep60-21swapper/607:04:356
3039489992322,0cyclictest0-21swapper/712:30:207
3039489992322,0cyclictest0-21swapper/711:34:297
3039489992221,0cyclictest0-21swapper/711:54:197
3039489992221,0cyclictest0-21swapper/707:09:087
3039489992220,1cyclictest0-21swapper/707:54:267
3039480992220,2cyclictest3408698-21kworker/u16:0+flush-8:010:29:034
3039480992219,2cyclictest3019203-21kworker/u16:0+flush-8:007:19:074
3039489992120,0cyclictest0-21swapper/710:24:257
3039482992119,2cyclictest3239901-21kworker/u16:4+flush-8:011:44:255
3039489992019,0cyclictest0-21swapper/712:04:407
3039489992019,0cyclictest0-21swapper/707:49:037
3039486992016,3cyclictest3546257-21latency_hist11:29:076
3039480992018,2cyclictest3210293-21kworker/u16:3+events_unbound08:34:354
3039480992018,2cyclictest3019203-21kworker/u16:0+flush-8:007:14:034
3039489991918,0cyclictest0-21swapper/712:20:267
3039489991918,0cyclictest0-21swapper/707:19:367
3039489991917,1cyclictest0-21swapper/710:44:237
3039482991917,2cyclictest3019244-21kworker/u16:4+flush-8:007:39:335
3039480991917,2cyclictest3019244-21kworker/u16:4+flush-8:007:54:264
3039489991817,0cyclictest0-21swapper/712:34:237
3039489991817,0cyclictest0-21swapper/709:41:007
3039489991817,0cyclictest0-21swapper/708:09:347
3039489991816,1cyclictest0-21swapper/708:24:007
303948699181,2cyclictest3304251-21kthreadcore09:19:266
3039482991816,2cyclictest3170973-21kworker/u16:0+flush-8:008:24:325
303948299180,13cyclictest0-21swapper/510:28:325
3039480991816,2cyclictest3151232-21kworker/u16:3+events_unbound08:09:254
303948999174,6cyclictest0-21swapper/711:53:287
3039489991716,0cyclictest0-21swapper/712:24:197
3039489991716,0cyclictest0-21swapper/711:40:057
3039489991716,0cyclictest0-21swapper/711:30:037
3039489991716,0cyclictest0-21swapper/711:30:037
3039489991716,0cyclictest0-21swapper/710:58:007
3039489991716,0cyclictest0-21swapper/710:05:307
3039489991715,1cyclictest0-21swapper/709:19:367
3039486991715,2cyclictest3151232-21kworker/u16:3+events_unbound08:09:216
3039486991713,3cyclictest3338653-21latency_hist09:39:076
3039480991715,2cyclictest3604974-21kworker/u16:0+flush-8:012:34:374
3039480991715,2cyclictest3348401-21kworker/u16:0+flush-8:009:54:194
3039480991715,2cyclictest3274322-21kworker/u16:2+flush-8:009:04:284
3039480991715,2cyclictest3274322-21kworker/u16:2+flush-8:009:04:284
3039480991715,2cyclictest3170973-21kworker/u16:0+flush-8:008:33:004
3039480991715,2cyclictest3019244-21kworker/u16:4+flush-8:007:29:004
3039480991714,2cyclictest3239901-21kworker/u16:4+events_unbound10:29:284
3039480991714,2cyclictest1135-21snmpd11:27:224
303948099170,16cyclictest3115196-21cut07:44:244
303948999163,5cyclictest0-21swapper/710:49:307
3039489991615,0cyclictest0-21swapper/712:10:587
3039489991615,0cyclictest0-21swapper/711:03:547
3039489991615,0cyclictest0-21swapper/710:42:167
3039489991615,0cyclictest0-21swapper/709:44:377
3039489991615,0cyclictest0-21swapper/709:01:107
3039489991615,0cyclictest0-21swapper/708:53:527
3039489991615,0cyclictest0-21swapper/708:14:397
3039482991613,2cyclictest3170973-21kworker/u16:0+events_unbound08:44:205
3039480991614,2cyclictest3615282-21kworker/u16:3+flush-8:012:05:594
3039480991614,2cyclictest3604974-21kworker/u16:0+flush-8:012:19:324
3039480991614,2cyclictest3448020-21kworker/u16:1+events_unbound10:39:344
3039480991614,2cyclictest3337454-21kworker/u16:5+flush-8:010:19:234
3039480991614,2cyclictest3170973-21kworker/u16:0+events_unbound09:18:494
3039480991614,2cyclictest3131600-21kworker/u16:1+events_unbound09:24:294
3039480991614,2cyclictest3019244-21kworker/u16:4+flush-8:007:59:304
3039480991613,2cyclictest3239901-21kworker/u16:4+events_unbound11:59:264
3039480991613,2cyclictest3131600-21kworker/u16:1+flush-8:008:43:254
3039480991613,2cyclictest1135-21snmpd10:51:304
303948999156,8cyclictest0-21swapper/707:18:267
303948999151,6cyclictest0-21swapper/712:01:267
303948999151,6cyclictest0-21swapper/708:05:257
3039489991514,0cyclictest0-21swapper/710:35:597
3039489991514,0cyclictest0-21swapper/710:33:157
3039489991514,0cyclictest0-21swapper/710:17:067
3039489991514,0cyclictest0-21swapper/709:54:087
3039489991514,0cyclictest0-21swapper/709:09:207
3039489991514,0cyclictest0-21swapper/709:09:207
3039489991514,0cyclictest0-21swapper/708:37:067
3039489991514,0cyclictest0-21swapper/708:26:417
3039489991514,0cyclictest0-21swapper/707:59:367
3039489991513,1cyclictest0-21swapper/710:10:097
303948999150,14cyclictest0-21swapper/711:19:047
303948999150,14cyclictest0-21swapper/711:04:317
3039482991513,2cyclictest3131600-21kworker/u16:1+flush-8:009:14:245
3039482991512,2cyclictest3408698-21kworker/u16:0+flush-8:010:19:335
3039482991511,3cyclictest3467771-21kworker/u16:1+flush-8:010:49:375
3039480991513,2cyclictest3556338-21kworker/u16:1+events_unbound11:47:154
3039480991513,2cyclictest3457899-21kworker/u16:0+events_unbound11:09:224
3039480991513,2cyclictest3337454-21kworker/u16:5+events_unbound10:13:154
3039480991513,2cyclictest3239901-21kworker/u16:4+flush-8:011:01:594
3039480991513,2cyclictest3239901-21kworker/u16:4+flush-8:009:00:084
3039480991513,2cyclictest3239901-21kworker/u16:4+events_unbound11:54:294
3039480991513,2cyclictest3239901-21kworker/u16:4+events_unbound11:40:154
3039480991513,2cyclictest3170973-21kworker/u16:0+events_unbound08:26:364
3039489991413,0cyclictest0-21swapper/709:37:317
303948999140,2cyclictest0-21swapper/709:59:407
303948999140,13cyclictest0-21swapper/711:44:347
303948699149,4cyclictest3655629-21cat12:24:196
303948699140,1cyclictest0-21swapper/610:58:076
3039482991412,2cyclictest3507793-21kworker/u16:3+flush-8:011:19:315
3039482991411,2cyclictest3019203-21kworker/u16:0+events_unbound07:09:195
3039482991410,3cyclictest3131881-21latency_hist07:54:075
303948299140,14cyclictest0-21swapper/512:06:565
3039480991412,2cyclictest3556338-21kworker/u16:1+flush-8:011:34:304
3039480991412,2cyclictest3467771-21kworker/u16:1+events_unbound10:55:554
3039480991412,2cyclictest3250304-21kworker/u16:3+events_unbound08:58:414
3039480991412,2cyclictest3170973-21kworker/u16:0+flush-8:008:44:224
3039480991412,2cyclictest3111980-21kworker/u16:2+events_unbound07:42:014
3039480991412,2cyclictest3071796-21kworker/u16:1+events_unbound07:34:354
3039489991312,0cyclictest0-21swapper/711:13:027
3039489991312,0cyclictest0-21swapper/708:44:357
3039489991312,0cyclictest0-21swapper/707:34:027
3039489991310,2cyclictest0-21swapper/709:31:547
303948999130,6cyclictest0-21swapper/709:27:577
303948699136,7cyclictest0-21swapper/608:44:326
303948699136,7cyclictest0-21swapper/608:23:076
3039486991312,1cyclictest0-21swapper/612:09:236
3039486991312,1cyclictest0-21swapper/612:04:396
3039486991312,1cyclictest0-21swapper/611:19:336
3039486991312,1cyclictest0-21swapper/611:08:036
3039486991312,1cyclictest0-21swapper/610:11:086
3039486991310,2cyclictest1135-21snmpd07:53:516
303948699130,1cyclictest0-21swapper/609:29:206
303948699130,1cyclictest0-21swapper/607:57:466
303948699130,13cyclictest0-21swapper/611:45:156
303948699130,13cyclictest0-21swapper/610:29:546
303948299136,7cyclictest0-21swapper/510:10:505
303948299135,8cyclictest0-21swapper/511:26:165
3039482991312,1cyclictest0-21swapper/512:13:595
3039482991312,1cyclictest0-21swapper/507:15:145
3039482991311,2cyclictest3337454-21kworker/u16:5+events_unbound10:04:005
3039482991311,2cyclictest3019259-21kworker/u16:5+events_unbound07:33:505
3039482991311,2cyclictest3019244-21kworker/u16:4+events_unbound08:14:315
3039482991310,3cyclictest3337454-21kworker/u16:5+flush-8:009:46:045
3039482991310,2cyclictest3604974-21kworker/u16:0+flush-8:012:24:335
303948299130,1cyclictest0-21swapper/510:08:395
3039480991311,2cyclictest3507793-21kworker/u16:3+events_unbound11:31:024
3039480991311,2cyclictest3507793-21kworker/u16:3+events_unbound11:31:024
3039480991311,2cyclictest3337454-21kworker/u16:5+flush-8:010:47:594
3039480991311,2cyclictest3328434-21kworker/u16:3+events_unbound09:34:214
3039480991311,2cyclictest3299546-21kworker/u16:3+events_unbound09:23:254
3039480991311,2cyclictest3239901-21kworker/u16:4+flush-8:009:33:254
3039480991311,2cyclictest3239901-21kworker/u16:4+events_unbound12:29:354
3039480991311,2cyclictest3131600-21kworker/u16:1+events_unbound10:17:314
3039480991311,2cyclictest3131600-21kworker/u16:1+events_unbound10:08:594
3039480991311,2cyclictest3071796-21kworker/u16:1+flush-8:007:19:374
3039480991311,2cyclictest3019203-21kworker/u16:0+flush-8:007:49:374
3039489991211,0cyclictest0-21swapper/709:14:277
3039489991210,1cyclictest0-21swapper/707:44:087
3039489991210,1cyclictest0-21swapper/707:34:267
303948699129,2cyclictest1135-21snmpd09:55:496
3039486991211,1cyclictest0-21swapper/610:35:066
3039486991211,1cyclictest0-21swapper/610:05:566
3039486991211,1cyclictest0-21swapper/609:49:516
3039486991211,1cyclictest0-21swapper/608:18:446
3039486991211,1cyclictest0-21swapper/608:00:556
3039486991210,2cyclictest86-21ksoftirqd/609:14:316
3039486991210,2cyclictest3239901-21kworker/u16:4+flush-8:009:46:006
303948699120,12cyclictest0-21swapper/612:23:346
303948699120,12cyclictest0-21swapper/612:15:416
303948699120,12cyclictest0-21swapper/609:40:336
303948699120,12cyclictest0-21swapper/607:47:126
303948699120,12cyclictest0-21swapper/607:15:316
303948699120,0cyclictest0-21swapper/607:27:116
303948299129,2cyclictest3131600-21kworker/u16:1+events_unbound09:49:095
303948299129,2cyclictest1135-21snmpd09:29:455
303948299128,3cyclictest3260975-21cat08:59:225
3039482991211,1cyclictest0-21swapper/507:55:175
3039482991210,2cyclictest3239901-21kworker/u16:4+events_unbound10:29:325
303948299120,12cyclictest0-21swapper/510:36:345
303948299120,0cyclictest0-21swapper/510:43:585
3039480991210,2cyclictest3604974-21kworker/u16:0+events_unbound12:16:594
3039480991210,2cyclictest3378388-21kworker/u16:2+flush-8:009:59:214
3039480991210,2cyclictest3274322-21kworker/u16:2+flush-8:009:10:584
3039480991210,2cyclictest3274322-21kworker/u16:2+flush-8:009:10:584
3039480991210,2cyclictest3239901-21kworker/u16:4+flush-8:011:04:284
3039480991210,2cyclictest3239901-21kworker/u16:4+events_unbound09:44:534
3039480991210,2cyclictest3131600-21kworker/u16:1+flush-8:008:49:274
3039480991210,2cyclictest3019244-21kworker/u16:4+events_unbound08:19:304
3039480991210,1cyclictest0-21swapper/407:32:094
303948999119,1cyclictest0-21swapper/709:05:517
303948999119,1cyclictest0-21swapper/709:05:517
3039489991110,0cyclictest0-21swapper/708:54:187
303948699119,2cyclictest86-21ksoftirqd/608:49:296
303948699119,2cyclictest86-21ksoftirqd/607:34:306
303948699118,2cyclictest3261567-21grep08:59:256
303948699118,2cyclictest1135-21snmpd09:14:006
303948699118,2cyclictest1135-21snmpd09:14:006
303948699117,3cyclictest3606449-21cat11:59:216
3039486991110,1cyclictest0-21swapper/610:16:206
3039486991110,1cyclictest0-21swapper/608:06:456
303948699110,5cyclictest0-21swapper/612:31:276
303948299119,2cyclictest3633946-21kworker/u16:1+flush-8:012:19:355
303948299119,2cyclictest3467771-21kworker/u16:1+events_unbound11:15:005
303948299119,2cyclictest3239901-21kworker/u16:4+events_unbound12:30:165
303948299119,2cyclictest3019203-21kworker/u16:0+flush-8:007:24:285
303948299118,2cyclictest1135-21snmpd11:11:425
303948299118,2cyclictest1135-21snmpd08:37:355
303948299117,3cyclictest3633946-21kworker/u16:1+flush-8:012:14:185
3039482991110,1cyclictest0-21swapper/511:52:145
3039482991110,1cyclictest0-21swapper/511:39:005
3039482991110,1cyclictest0-21swapper/510:16:325
3039482991110,1cyclictest0-21swapper/509:24:405
3039482991110,1cyclictest0-21swapper/509:19:285
3039482991110,1cyclictest0-21swapper/509:12:135
3039482991110,1cyclictest0-21swapper/509:12:135
3039482991110,1cyclictest0-21swapper/507:48:215
3039482991110,1cyclictest0-21swapper/507:23:175
303948299110,3cyclictest0-21swapper/508:07:495
303948299110,1cyclictest0-21swapper/508:21:185
303948299110,11cyclictest0-21swapper/510:47:075
303948299110,11cyclictest0-21swapper/509:41:235
303948099119,2cyclictest3604974-21kworker/u16:0+flush-8:012:24:334
303948099119,2cyclictest3337454-21kworker/u16:5+flush-8:009:49:174
303948099110,11cyclictest0-21swapper/412:09:444
303948699109,1cyclictest0-21swapper/611:40:326
303948699109,1cyclictest0-21swapper/610:23:076
303948699108,1cyclictest3594716-21sed11:49:366
303948699107,2cyclictest851ktimers/611:29:326
303948699107,2cyclictest851ktimers/611:29:326
303948699107,2cyclictest3217845-21kthreadcore08:34:276
303948699107,2cyclictest3045461-21tr07:09:256
303948699101,8cyclictest0-21swapper/608:24:196
303948699100,9cyclictest3473908-21cut10:49:306
303948699100,5cyclictest0-21swapper/612:38:006
303948699100,4cyclictest3063353-21/usr/sbin/munin07:19:236
303948699100,3cyclictest3312807-21kthreadcore09:24:236
303948699100,3cyclictest3272336-21grep09:04:236
303948699100,3cyclictest3272336-21grep09:04:236
303948299109,1cyclictest0-21swapper/510:59:165
303948299108,2cyclictest3239901-21kworker/u16:4+flush-8:011:54:375
303948299108,2cyclictest3131600-21kworker/u16:1+flush-8:009:34:175
303948299108,2cyclictest3131600-21kworker/u16:1+events_unbound08:09:365
303948299108,2cyclictest3019244-21kworker/u16:4+flush-8:007:59:335
303948299107,2cyclictest929-21lldpd08:30:355
303948299107,2cyclictest3250304-21kworker/u16:3+events_unbound09:04:165
303948299107,2cyclictest3250304-21kworker/u16:3+events_unbound09:04:165
303948299105,4cyclictest3552693-21kthreadcore11:29:275
303948299105,4cyclictest3552693-21kthreadcore11:29:275
303948299100,10cyclictest0-21swapper/507:35:345
303948099109,1cyclictest0-21swapper/411:14:344
303948099109,1cyclictest0-21swapper/408:09:004
30394869997,1cyclictest3259189-21sed08:54:376
30394869997,1cyclictest0-21swapper/611:54:196
30394869997,1cyclictest0-21swapper/611:14:066
30394869996,2cyclictest1135-21snmpd11:16:406
30394869996,2cyclictest0-21swapper/609:59:206
30394869995,3cyclictest3103513-21cat07:39:206
30394869990,8cyclictest3556466-21/usr/sbin/munin11:34:186
30394869990,8cyclictest3493499-21kthreadcore10:59:286
30394869990,8cyclictest0-21swapper/608:29:286
30394869990,2cyclictest3220830-21/usr/sbin/munin08:39:436
30394829998,1cyclictest0-21swapper/511:40:345
30394829998,1cyclictest0-21swapper/510:57:415
30394829997,2cyclictest3337454-21kworker/u16:5+flush-8:009:54:245
30394829996,2cyclictest3131600-21kworker/u16:1+flush-8:008:49:375
30394829995,3cyclictest3675313-21cat12:34:225
30394829994,4cyclictest27052-21gnome-software08:57:335
30394829991,1cyclictest0-21swapper/512:01:535
30394829990,1cyclictest0-21swapper/511:06:365
30394869983,4cyclictest3458939-21cut10:44:216
30394869980,2cyclictest3448065-21/usr/sbin/munin10:39:236
30394869980,2cyclictest3083056-21/usr/sbin/munin07:29:346
30394829983,4cyclictest3225616-21cut08:39:255
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional