You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-25 - 14:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot7.osadl.org (updated Sun Feb 25, 2024 00:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39495322209202,3sleep80-21swapper/819:06:2914
39498232202185,9sleep50-21swapper/519:09:3811
39496822201195,3sleep120-21swapper/1219:07:534
39497172198179,11sleep100-21swapper/1019:08:072
39495392184177,3sleep130-21swapper/1319:06:345
39495222179114,55sleep150-21swapper/1519:06:217
39495512172153,11sleep60-21swapper/619:06:4312
39498102166145,11sleep90-21swapper/919:09:2615
39498312161141,10sleep110-21swapper/1119:09:443
39495792139118,11sleep70-21swapper/719:07:0013
39498222134116,10sleep40-21swapper/419:09:3710
39497642130113,9sleep30-21swapper/319:08:499
39496502127107,11sleep10-21swapper/119:07:261
19777201120,111pipewire0-21swapper/019:05:550
3949798210755,42sleep140-21swapper/1419:09:166
394972627453,12sleep20-21swapper/219:08:168
3950206996354,0cyclictest0-21swapper/321:30:399
395024999560,26cyclictest0-21swapper/1521:58:317
395024999530,23cyclictest0-21swapper/1523:35:187
395024999530,23cyclictest0-21swapper/1519:58:327
395024999500,20cyclictest0-21swapper/1521:54:597
395024999460,18cyclictest0-21swapper/1521:02:097
395024999460,17cyclictest0-21swapper/1520:48:417
395024999450,16cyclictest0-21swapper/1500:02:097
3950214994316,0cyclictest0-21swapper/521:24:1611
395024999420,12cyclictest0-21swapper/1520:19:207
395024999410,13cyclictest0-21swapper/1521:15:557
395024999371,8cyclictest0-21swapper/1521:20:147
143999370,0ntpd0-21swapper/1221:29:404
143999360,0ntpd0-21swapper/1220:29:414
395023199348,0cyclictest141rcu_preempt21:48:492
395022199349,0cyclictest56950irq/134-i91521:21:2113
593262330,0sleep80-21swapper/820:50:3514
395024999330,7cyclictest0-21swapper/1500:05:177
3950231993322,11cyclictest0-21swapper/1022:43:452
3950214993319,9cyclictest0-21swapper/521:27:4111
395024999320,3cyclictest0-21swapper/1523:51:437
3950195993232,0cyclictest0-21swapper/022:21:370
395024999310,3cyclictest0-21swapper/1500:27:267
395023199316,25cyclictest0-21swapper/1019:37:022
395019599312,3cyclictest141rcu_preempt21:19:040
395024599300,30cyclictest0-21swapper/1421:28:436
3950241993029,0cyclictest0-21swapper/1321:22:175
395022499300,1cyclictest74-21ksoftirqd/821:22:1714
395024999290,1cyclictest0-21swapper/1500:22:567
3950245992927,1cyclictest0-21swapper/1421:20:166
3950229992928,0cyclictest0-21swapper/921:20:2615
3950221992929,0cyclictest56950irq/134-i91500:31:3713
3950217992928,0cyclictest0-21swapper/621:21:4112
1976920290,0pipewire-pulse0-21swapper/1021:22:352
395024999280,0cyclictest0-21swapper/1520:58:327
3950221992828,0cyclictest56950irq/134-i91519:24:2413
395021499281,0cyclictest0-21swapper/500:27:1011
395020399283,25cyclictest715534-1kworker/u33:000:31:388
395024999270,0cyclictest0-21swapper/1522:58:327
395024999270,0cyclictest0-21swapper/1520:02:097
3950239992727,0cyclictest0-21swapper/1221:23:144
3950234992726,0cyclictest0-21swapper/1121:22:163
3950224992715,1cyclictest1976920pipewire-pulse23:19:4714
3950224992715,1cyclictest1976920pipewire-pulse23:19:4614
395021499270,27cyclictest0-21swapper/500:22:5611
395021099270,1cyclictest138212-21Isolated01:24:0710
395024999260,0cyclictest0-21swapper/1519:25:157
3950245992623,1cyclictest0-21swapper/1422:29:226
395024199260,25cyclictest61515-21WebExtensions21:28:355
395024199260,1cyclictest174626-21sed21:30:255
3950231992623,2cyclictest88-21ksoftirqd/1023:10:132
3950224992625,1cyclictest74-21ksoftirqd/821:27:1514
3950221992626,0cyclictest56950irq/134-i91521:54:5813
3950221992626,0cyclictest56950irq/134-i91519:35:2813
3950221992625,0cyclictest0-21swapper/721:28:3813
3950206992624,1cyclictest2752892-21Isolated01:24:079
395024999250,24cyclictest0-21swapper/1523:09:017
395024999250,24cyclictest0-21swapper/1523:09:007
395024999250,0cyclictest0-21swapper/1521:30:267
395024599250,15cyclictest0-21swapper/1423:28:286
395024199250,1cyclictest512424-21TaskCon~ller5
395023999250,1cyclictest393003-21iotop22:35:564
3950234992523,1cyclictest2752892-21Isolated01:24:073
395023499250,15cyclictest0-21swapper/1100:34:153
395023199250,25cyclictest0-21swapper/1023:49:392
395023199250,25cyclictest0-21swapper/1023:38:392
395023199250,25cyclictest0-21swapper/1021:59:492
395023199250,25cyclictest0-21swapper/1020:20:592
395023199250,25cyclictest0-21swapper/1020:09:592
395023199250,25cyclictest0-21swapper/1019:59:002
395023199250,25cyclictest0-21swapper/1000:11:362
395023199250,15cyclictest9734-21Xorg22:52:222
395023199250,15cyclictest0-21swapper/1000:36:512
395023199250,15cyclictest0-21swapper/1000:09:492
395023199250,0cyclictest0-21swapper/1022:21:472
395023199250,0cyclictest0-21swapper/1021:37:502
395023199250,0cyclictest0-21swapper/1021:26:522
395023199250,0cyclictest0-21swapper/1021:15:532
395023199250,0cyclictest0-21swapper/1021:04:532
395023199250,0cyclictest0-21swapper/1020:53:552
395023199250,0cyclictest0-21swapper/1020:42:572
395023199250,0cyclictest0-21swapper/1020:31:572
395023199250,0cyclictest0-21swapper/1019:48:022
395023199250,0cyclictest0-21swapper/1019:48:012
395023199250,0cyclictest0-21swapper/1019:15:052
395023199250,0cyclictest0-21swapper/1000:00:382
395022999250,15cyclictest0-21swapper/921:30:5015
395022999250,15cyclictest0-21swapper/900:37:1915
395022499250,15cyclictest0-21swapper/800:32:0414
3950221992525,0cyclictest56950irq/134-i91521:10:3013
395021799250,24cyclictest0-21swapper/621:30:2612
395021799250,0cyclictest0-21swapper/622:30:1612
395021799250,0cyclictest0-21swapper/622:24:2212
3950214992523,1cyclictest0-21swapper/521:30:2711
395021499250,15cyclictest235563-21thunderbird23:09:3111
395021499250,15cyclictest235563-21thunderbird23:09:3111
395021499250,15cyclictest0-21swapper/522:30:5611
395021099250,24cyclictest0-21swapper/421:28:3310
395021099250,24cyclictest0-21swapper/400:31:3510
395021099250,15cyclictest0-21swapper/422:25:3110
395020699250,25cyclictest0-21swapper/321:28:469
3950203992524,0cyclictest2752892-21Isolated01:24:078
395020399250,25cyclictest152776-21ssh21:21:428
395020399250,1cyclictest389981-21ssh22:30:388
395020399250,15cyclictest187282-21ssh21:31:568
395020099250,1cyclictest2752892-21Isolated01:24:071
395020099250,1cyclictest235703-21Renderer00:31:381
395020099250,15cyclictest375240-21ssh22:28:221
395020099250,15cyclictest0-21swapper/123:24:121
395019599250,24cyclictest2752892-21Isolated01:24:070
3950249992422,1cyclictest0-21swapper/1521:28:377
395024999240,24cyclictest0-21swapper/1522:24:067
395024999240,0cyclictest0-21swapper/1523:10:257
395024999240,0cyclictest0-21swapper/1522:32:447
3950245992423,0cyclictest0-21swapper/1423:11:086
3950245992422,1cyclictest0-21swapper/1423:09:076
3950245992422,1cyclictest0-21swapper/1423:09:066
395024199240,14cyclictest0-21swapper/1323:23:575
3950239992422,1cyclictest9734-21Xorg23:10:034
395023999240,24cyclictest0-21swapper/1222:24:054
395023999240,0cyclictest0-21swapper/1222:32:534
3950234992424,0cyclictest0-21swapper/1123:14:403
395023499240,14cyclictest0-21swapper/1121:31:133
395023499240,14cyclictest0-21swapper/1120:30:583
395023499240,0cyclictest0-21swapper/1122:35:483
395023199240,14cyclictest0-21swapper/1000:33:052
395023199240,0cyclictest0-21swapper/1023:08:562
395023199240,0cyclictest0-21swapper/1023:08:562
395023199240,0cyclictest0-21swapper/1000:22:362
3950224992422,1cyclictest0-21swapper/822:33:4614
395022499240,14cyclictest0-21swapper/823:11:0414
3950221992424,0cyclictest56950irq/134-i91500:22:3113
395022199240,24cyclictest0-21swapper/721:18:0913
395022199240,0cyclictest0-21swapper/723:10:0313
395022199240,0cyclictest0-21swapper/721:46:0413
395021799240,24cyclictest0-21swapper/622:35:3812
395021799240,24cyclictest0-21swapper/621:25:4912
3950214992424,0cyclictest0-21swapper/522:22:0611
395021099242,20cyclictest61515-21WebExtensions22:21:3710
3950210992419,2cyclictest1977720pipewire23:10:0310
395021099240,14cyclictest0-21swapper/423:27:3510
395021099240,14cyclictest0-21swapper/423:20:2210
395021099240,14cyclictest0-21swapper/421:31:3210
395021099240,14cyclictest0-21swapper/400:37:2110
3950206992424,0cyclictest0-21swapper/323:14:419
395020399240,14cyclictest0-21swapper/223:34:068
395020399240,14cyclictest0-21swapper/223:25:468
395020399240,14cyclictest0-21swapper/200:38:068
395020099240,23cyclictest61515-21WebExtensions21:25:351
395020099240,15cyclictest0-21swapper/100:35:301
395020099240,14cyclictest0-21swapper/122:21:461
3950195992424,0cyclictest594599-21nodedev-init23:30:370
395024599233,10cyclictest0-21swapper/1400:39:566
3950245992323,0cyclictest0-21swapper/1422:32:476
3950245992321,1cyclictest0-21swapper/1422:36:096
395024599230,4cyclictest0-21swapper/1423:35:036
3950241992323,0cyclictest0-21swapper/1323:09:305
3950241992323,0cyclictest0-21swapper/1323:09:295
395024199230,13cyclictest0-21swapper/1323:28:285
3950239992323,0cyclictest0-21swapper/1221:18:414
3950239992323,0cyclictest0-21swapper/1200:31:344
395023499230,23cyclictest0-21swapper/1122:29:213
3950231992323,0cyclictest0-21swapper/1022:35:492
395023199230,14cyclictest0-21swapper/1023:29:092
395022999230,23cyclictest9734-21Xorg22:36:0815
395022999230,23cyclictest171307-21ssh21:29:0515
395022999230,13cyclictest0-21swapper/922:26:5115
395022999230,0cyclictest0-21swapper/921:18:4115
395022499230,13cyclictest0-21swapper/823:27:3014
3950221992323,0cyclictest56950irq/134-i91521:02:0813
3950221992323,0cyclictest56950irq/134-i91500:06:1213
3950221992323,0cyclictest0-21swapper/722:34:2313
3950221992321,1cyclictest56950irq/134-i91523:09:0213
3950221992321,1cyclictest56950irq/134-i91523:09:0213
3950217992323,0cyclictest0-21swapper/623:14:5312
3950217992323,0cyclictest0-21swapper/621:18:0912
3950214992323,0cyclictest0-21swapper/523:14:0511
3950214992323,0cyclictest0-21swapper/522:35:5711
3950214992318,5cyclictest0-21swapper/521:18:3911
3950214992310,13cyclictest0-21swapper/500:38:3511
3950210992323,0cyclictest0-21swapper/421:18:0810
3950206992323,0cyclictest0-21swapper/300:31:389
3950206992313,10cyclictest61109-21firefox22:24:449
3950203992323,0cyclictest0-21swapper/223:10:328
395020399230,0cyclictest0-21swapper/221:18:218
395020099230,23cyclictest0-21swapper/121:18:211
3950195992323,0cyclictest235563-21thunderbird22:29:260
3950195992323,0cyclictest0-21swapper/023:35:100
395019599230,15cyclictest0-21swapper/023:25:040
395019599230,0cyclictest800983-21ssh00:31:380
395019599230,0cyclictest0-21swapper/021:27:410
3950249992222,0cyclictest0-21swapper/1500:32:137
3950245992222,0cyclictest0-21swapper/1422:21:376
395024599220,11cyclictest0-21swapper/1421:18:236
3950241992222,0cyclictest389980-21ssh22:30:385
3950241992222,0cyclictest0-21swapper/1322:24:215
3950241992217,4cyclictest109-21ksoftirqd/1321:19:485
3950241992212,10cyclictest2472987-21x0vncserver23:35:085
395024199220,12cyclictest0-21swapper/1300:37:255
3950239992212,10cyclictest9734-21Xorg22:44:354
395023999220,12cyclictest0-21swapper/1223:28:234
395023999220,12cyclictest0-21swapper/1223:21:194
3950234992222,0cyclictest235563-21thunderbird21:18:333
395023499220,12cyclictest0-21swapper/1123:23:113
395023499220,12cyclictest0-21swapper/1123:19:583
395023499220,12cyclictest0-21swapper/1123:19:583
3950231992222,0cyclictest0-21swapper/1022:34:222
3950231992221,1cyclictest0-21swapper/1021:30:252
395023199220,22cyclictest0-21swapper/1023:16:422
395023199220,22cyclictest0-21swapper/1023:16:422
395022999220,14cyclictest0-21swapper/923:35:1115
395022999220,13cyclictest0-21swapper/900:34:0815
395022999220,0cyclictest0-21swapper/922:31:0315
3950224992222,0cyclictest0-21swapper/821:30:3514
3950224992219,3cyclictest0-21swapper/821:18:0914
3950224992211,10cyclictest0-21swapper/800:35:2014
395022499220,13cyclictest0-21swapper/823:20:2614
395022499220,12cyclictest18509-21gnome-shell20:57:5914
3950221992222,0cyclictest56950irq/134-i91500:12:1513
395021799220,21cyclictest357281-21Isolated01:24:0712
395021799220,21cyclictest357281-21Isolated01:24:0712
395021499220,12cyclictest0-21swapper/523:20:4411
3950210992222,0cyclictest0-21swapper/422:32:4310
395021099220,12cyclictest0-21swapper/420:17:5910
395020699222,3cyclictest1976920pipewire-pulse23:09:589
395020699222,3cyclictest1976920pipewire-pulse23:09:579
3950206992222,0cyclictest0-21swapper/321:18:239
395020699220,13cyclictest0-21swapper/300:35:379
395020699220,12cyclictest0-21swapper/323:32:369
395020699220,12cyclictest0-21swapper/323:24:179
3950200992222,0cyclictest0-21swapper/123:10:171
3950200992222,0cyclictest0-21swapper/122:35:491
3950195992222,0cyclictest0-21swapper/023:10:170
3950195992222,0cyclictest0-21swapper/022:35:490
1976920220,0pipewire-pulse0-21swapper/1223:08:544
1976920220,0pipewire-pulse0-21swapper/1223:08:544
395024999210,11cyclictest0-21swapper/1523:33:137
395024999210,11cyclictest0-21swapper/1522:28:487
3950245992119,1cyclictest0-21swapper/1400:31:386
395024599210,11cyclictest0-21swapper/1423:34:026
395024599210,11cyclictest0-21swapper/1419:42:236
3950241992111,10cyclictest0-21swapper/1300:32:385
3950239992111,10cyclictest0-21swapper/1222:28:144
3950239992111,10cyclictest0-21swapper/1200:37:124
395023999210,20cyclictest0-21swapper/1222:53:124
395023499210,14cyclictest0-21swapper/1123:09:323
395023499210,14cyclictest0-21swapper/1123:09:323
395023499210,11cyclictest0-21swapper/1123:30:563
395023199218,0cyclictest0-21swapper/1019:26:032
3950231992111,10cyclictest0-21swapper/1022:25:452
3950231992111,0cyclictest0-21swapper/1023:32:272
395022999210,11cyclictest235703-21Renderer22:07:5415
3950224992111,10cyclictest0-21swapper/823:58:0714
395022499210,0cyclictest615947-21kworker/8:023:40:0614
3950221992119,1cyclictest171377-21TaskCon~ller13
395022199210,20cyclictest0-21swapper/722:35:5613
3950217992121,0cyclictest0-21swapper/623:19:4712
3950217992121,0cyclictest0-21swapper/623:19:4712
3950214992111,10cyclictest0-21swapper/523:30:1911
395021499210,17cyclictest0-21swapper/521:35:1511
3950210992111,10cyclictest0-21swapper/422:35:3910
3950210992111,10cyclictest0-21swapper/421:12:1910
3950206992121,0cyclictest0-21swapper/322:53:139
3950206992111,10cyclictest0-21swapper/322:48:279
395020699210,20cyclictest0-21swapper/323:19:349
395020699210,20cyclictest0-21swapper/323:19:339
395020699210,12cyclictest0-21swapper/323:26:469
3950203992111,10cyclictest0-21swapper/222:23:108
3950203992110,11cyclictest0-21swapper/222:10:598
395020399210,20cyclictest357302-21TaskCon~ller8
395020399210,20cyclictest357302-21TaskCon~ller8
395020399210,11cyclictest0-21swapper/223:16:598
395020399210,11cyclictest0-21swapper/223:16:588
3950200992111,10cyclictest0-21swapper/121:31:281
395020099210,11cyclictest0-21swapper/123:25:071
3950195992111,0cyclictest0-21swapper/019:34:280
81875022019,0sleep150-21swapper/1500:37:527
76693122019,0sleep140-21swapper/1400:21:506
64793622019,0sleep150-21swapper/1523:47:587
63169722019,0sleep140-21swapper/1423:44:126
49516722019,0sleep140-21swapper/1423:04:436
45985022019,0sleep140-21swapper/1422:52:266
395024999200,10cyclictest0-21swapper/1523:24:067
395024999200,10cyclictest0-21swapper/1522:18:377
395024999200,10cyclictest0-21swapper/1520:41:547
395024999200,10cyclictest0-21swapper/1520:11:297
3950241992010,10cyclictest0-21swapper/1323:30:525
395023999209,10cyclictest0-21swapper/1222:55:284
3950239992010,10cyclictest0-21swapper/1223:32:204
3950239992010,10cyclictest0-21swapper/1222:45:044
3950239992010,10cyclictest0-21swapper/1221:31:474
3950234992019,0cyclictest598993-21ssh23:35:113
3950234992010,10cyclictest0-21swapper/1122:30:113
3950234992010,10cyclictest0-21swapper/1121:43:323
3950234992010,10cyclictest0-21swapper/1121:43:323
3950234992010,10cyclictest0-21swapper/1100:36:133
395023499200,19cyclictest0-21swapper/1122:23:363
3950229992010,10cyclictest0-21swapper/923:29:0715
3950229992010,10cyclictest0-21swapper/919:52:5115
3950229992010,10cyclictest0-21swapper/919:30:5915
3950224992018,2cyclictest0-21swapper/800:17:1114
3950224992016,4cyclictest0-21swapper/822:29:2314
3950224992014,3cyclictest393003-21iotop23:01:0414
3950224992014,3cyclictest393003-21iotop22:40:4514
3950224992010,10cyclictest0-21swapper/822:58:3114
3950224992010,10cyclictest0-21swapper/800:05:0114
395022499200,1cyclictest61515-21WebExtensions22:23:2414
395022499200,18cyclictest0-21swapper/819:40:0814
395022499200,17cyclictest0-21swapper/820:36:1814
395022499200,13cyclictest0-21swapper/822:35:3614
3950221992020,0cyclictest56950irq/134-i91519:55:2213
3950214992010,10cyclictest0-21swapper/522:52:5411
3950214992010,10cyclictest0-21swapper/522:44:2011
3950214992010,10cyclictest0-21swapper/522:27:1211
3950214992010,10cyclictest0-21swapper/522:17:2611
3950210992019,0cyclictest0-21swapper/423:10:0010
3950210992019,0cyclictest0-21swapper/423:09:5910
3950210992010,10cyclictest9734-21Xorg19:18:1810
395021099200,1cyclictest0-21swapper/423:19:4710
395021099200,1cyclictest0-21swapper/423:19:4710
3950206992010,10cyclictest0-21swapper/322:34:169
3950203992013,7cyclictest393003-21iotop22:36:398
3950203992011,9cyclictest374001-21ssh22:26:158
3950203992010,10cyclictest0-21swapper/219:31:188
3950200992020,0cyclictest0-21swapper/122:53:191
3950200992010,10cyclictest0-21swapper/123:31:531
3950200992010,0cyclictest0-21swapper/122:33:391
395020099200,14cyclictest0-21swapper/123:09:581
395020099200,14cyclictest0-21swapper/123:09:581
3950195992020,0cyclictest0-21swapper/022:53:190
3950195992011,9cyclictest0-21swapper/023:24:560
3950195992010,10cyclictest0-21swapper/021:31:240
3950195992010,10cyclictest0-21swapper/000:38:280
395019599200,1cyclictest61109-21firefox23:08:560
395019599200,1cyclictest61109-21firefox23:08:550
32396522019,0sleep140-21swapper/1422:13:126
1976920200,9pipewire-pulse0-21swapper/1123:28:503
18704522019,0sleep140-21swapper/1421:31:336
418268021919,0sleep140-21swapper/1420:29:046
402605621918,0sleep140-21swapper/1419:31:256
401188221918,0sleep140-21swapper/1419:25:486
3950249991918,1cyclictest0-21swapper/1523:19:527
3950249991918,1cyclictest0-21swapper/1523:19:517
395024999190,9cyclictest0-21swapper/1523:29:367
395024999190,9cyclictest0-21swapper/1522:44:437
395024999190,9cyclictest0-21swapper/1522:36:227
3950245991918,0cyclictest0-21swapper/1400:27:106
395024599190,9cyclictest0-21swapper/1423:22:076
395024199190,19cyclictest0-21swapper/1323:19:525
395024199190,19cyclictest0-21swapper/1323:19:515
395024199190,0cyclictest0-21swapper/1322:35:575
395023999199,10cyclictest0-21swapper/1221:00:064
3950239991919,0cyclictest0-21swapper/1223:19:304
3950239991919,0cyclictest0-21swapper/1223:19:294
3950234991919,0cyclictest4110166-21nodedev-init20:00:333
3950234991910,9cyclictest0-21swapper/1122:42:303
395023199199,10cyclictest0-21swapper/1021:43:522
395023199199,10cyclictest0-21swapper/1021:43:522
395022999199,10cyclictest0-21swapper/921:08:2915
3950229991919,0cyclictest0-21swapper/923:10:2215
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional