You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 04:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Mon Nov 24, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29444099620,60cyclictest864380-21latency_hist08:03:247
29389225949,7sleep50-21swapper/503:20:445
29391225848,7sleep60-21swapper/603:21:016
294440995751,5cyclictest304361-21latency_hist03:28:257
29388025746,7sleep40-21swapper/403:20:354
294440994843,4cyclictest571162-21latency_hist05:38:257
294440994746,0cyclictest0-21swapper/706:26:477
294427994744,3cyclictest318182-21kworker/u16:0+flush-8:003:33:254
294427994543,2cyclictest318182-21kworker/u16:0+flush-8:004:03:264
2856582436,31sleep70-21swapper/703:18:457
294427994139,2cyclictest449719-21kworker/u16:0+flush-8:006:08:254
29444099400,38cyclictest480116-21latency_hist04:53:257
294440993835,2cyclictest677121-21kthreadcore06:28:467
294436993836,2cyclictest284383-21kworker/u16:3+events_unbound03:52:356
294436993835,2cyclictest500396-21kworker/u16:4+events_unbound05:23:346
294427993835,2cyclictest894631-21kworker/u16:6+flush-8:008:28:254
294436993735,2cyclictest712601-21kworker/u16:0+flush-8:007:28:516
294431993735,2cyclictest284383-21kworker/u16:3+events_unbound03:42:355
294436993634,2cyclictest303975-21kworker/u16:2+flush-8:003:43:516
294427993632,3cyclictest471109-21kworker/u16:2+events_unbound04:53:404
294436993533,2cyclictest651123-21kworker/u16:3+events_unbound06:38:586
294436993533,2cyclictest642908-21kworker/u16:1+flush-8:006:14:346
294436993533,2cyclictest500396-21kworker/u16:4+flush-8:006:36:346
294436993432,2cyclictest701717-21kworker/u16:2+flush-8:007:10:506
294436993432,2cyclictest284383-21kworker/u16:3+flush-8:003:38:546
294436993431,2cyclictest284383-21kworker/u16:3+flush-8:003:23:286
294431993432,2cyclictest687018-21kworker/u16:1+flush-8:006:46:465
294427993431,3cyclictest449719-21kworker/u16:0+flush-8:005:13:504
294440993332,0cyclictest0-21swapper/704:30:287
294436993331,2cyclictest914904-21kworker/u16:3+flush-8:008:27:106
294436993331,2cyclictest914904-21kworker/u16:3+events_unbound08:30:356
294436993331,2cyclictest687018-21kworker/u16:1+events_unbound06:50:496
294436993331,2cyclictest500396-21kworker/u16:4+events_unbound05:43:346
294436993331,2cyclictest449719-21kworker/u16:0+events_unbound06:12:566
294436993330,2cyclictest416272-21kworker/u16:3+flush-8:004:23:356
294436993230,2cyclictest712601-21kworker/u16:0+events_unbound07:06:236
294436993230,2cyclictest575866-21kworker/u16:3+flush-8:005:49:396
294436993230,2cyclictest489950-21kworker/u16:3+events_unbound04:57:076
294436993230,2cyclictest449719-21kworker/u16:0+flush-8:005:14:356
294436993229,2cyclictest687018-21kworker/u16:1+flush-8:006:53:266
294431993230,2cyclictest860151-21kworker/u16:2+flush-8:008:22:105
294431993230,2cyclictest687018-21kworker/u16:1+events_unbound07:09:395
294431993230,2cyclictest550569-21kworker/u16:2+flush-8:005:38:185
294431993230,2cyclictest550569-21kworker/u16:2+events_unbound05:40:385
294431993230,2cyclictest339042-21kworker/u16:1+flush-8:004:52:465
294427993230,2cyclictest773419-21kworker/u16:3+flush-8:008:08:244
294440993130,0cyclictest0-21swapper/708:12:507
29444099310,29cyclictest896908-21cut08:18:467
294436993129,2cyclictest873492-21kworker/u16:0+events_unbound08:14:556
294436993129,2cyclictest823852-21kworker/u16:1+flush-8:008:00:546
294436993129,2cyclictest701717-21kworker/u16:2+events_unbound07:26:476
294436993129,2cyclictest701717-21kworker/u16:2+events_unbound07:14:346
294431993129,2cyclictest823852-21kworker/u16:1+events_unbound07:56:495
294431993129,2cyclictest773419-21kworker/u16:3+events_unbound07:52:505
294431993129,2cyclictest550569-21kworker/u16:2+flush-8:005:44:195
294431993129,2cyclictest449719-21kworker/u16:0+events_unbound05:05:065
294431993129,2cyclictest339042-21kworker/u16:1+events_unbound04:34:195
294431993128,2cyclictest489950-21kworker/u16:3+events_unbound05:13:285
294440993027,2cyclictest722918-21perf06:53:257
294436993028,2cyclictest823852-21kworker/u16:1+events_unbound08:35:116
294436993028,2cyclictest632274-21kworker/u16:2+events_unbound06:21:506
294436993028,2cyclictest349152-21kworker/u16:4+events_unbound04:17:156
294436993028,2cyclictest284383-21kworker/u16:3+flush-8:003:58:596
294431993028,2cyclictest687018-21kworker/u16:1+events_unbound06:56:185
294431993028,2cyclictest500396-21kworker/u16:4+flush-8:006:31:155
294431993028,2cyclictest449719-21kworker/u16:0+events_unbound05:59:105
294431993028,2cyclictest318182-21kworker/u16:0+flush-8:003:51:405
294436992927,2cyclictest925152-21kworker/u16:2+flush-8:008:38:466
294436992927,2cyclictest873492-21kworker/u16:0+events_unbound08:22:466
294436992927,2cyclictest712601-21kworker/u16:0+events_unbound07:39:236
294436992927,2cyclictest701717-21kworker/u16:2+events_unbound06:59:316
294436992927,2cyclictest687018-21kworker/u16:1+events_unbound06:47:346
294436992927,2cyclictest500396-21kworker/u16:4+events_unbound05:34:486
294436992927,2cyclictest416272-21kworker/u16:3+events_unbound04:19:356
294436992927,2cyclictest339042-21kworker/u16:1+events_unbound04:48:196
294436992927,2cyclictest339042-21kworker/u16:1+events_unbound04:06:266
294431992927,2cyclictest925152-21kworker/u16:2+flush-8:008:48:555
294431992927,2cyclictest712601-21kworker/u16:0+events_unbound07:47:145
294431992927,2cyclictest701717-21kworker/u16:2+flush-8:006:59:515
294431992927,2cyclictest701717-21kworker/u16:2+events_unbound07:32:465
294431992927,2cyclictest651123-21kworker/u16:3+flush-8:006:26:385
294431992927,2cyclictest489950-21kworker/u16:3+flush-8:005:11:435
294431992926,2cyclictest339042-21kworker/u16:1+flush-8:004:17:465
294436992826,2cyclictest945024-21kworker/u16:0+events_unbound08:50:476
294436992826,2cyclictest945024-21kworker/u16:0+events_unbound08:46:486
294436992826,2cyclictest773419-21kworker/u16:3+flush-8:008:07:556
294436992826,2cyclictest449719-21kworker/u16:0+flush-8:004:38:516
294436992826,2cyclictest318182-21kworker/u16:0+events_unbound03:57:276
294436992826,2cyclictest318182-21kworker/u16:0+events_unbound03:37:276
294436992826,2cyclictest303975-21kworker/u16:2+events_unbound03:33:036
294436992825,2cyclictest500396-21kworker/u16:4+events_unbound06:03:266
294431992826,2cyclictest925152-21kworker/u16:2+flush-8:008:41:265
294431992826,2cyclictest914904-21kworker/u16:3+flush-8:008:33:305
294431992826,2cyclictest823852-21kworker/u16:1+flush-8:008:18:155
294431992826,2cyclictest823852-21kworker/u16:1+events_unbound08:00:225
294431992826,2cyclictest773419-21kworker/u16:3+events_unbound07:42:585
294431992826,2cyclictest712601-21kworker/u16:0+flush-8:007:36:265
294431992826,2cyclictest687018-21kworker/u16:1+events_unbound06:51:425
294431992826,2cyclictest489950-21kworker/u16:3+flush-8:004:56:595
294431992826,2cyclictest409129-21kworker/u16:2+events_unbound04:30:035
294431992825,2cyclictest632274-21kworker/u16:2+flush-8:006:18:305
294440992722,4cyclictest753183-21latency_hist07:08:247
294436992725,2cyclictest873492-21kworker/u16:0+flush-8:008:08:516
294436992725,2cyclictest701717-21kworker/u16:2+events_unbound07:21:036
294436992725,2cyclictest500396-21kworker/u16:4+events_unbound05:39:596
294436992725,2cyclictest489950-21kworker/u16:3+flush-8:005:09:506
294436992725,2cyclictest471109-21kworker/u16:2+flush-8:004:50:446
294436992725,2cyclictest339042-21kworker/u16:1+flush-8:004:08:356
294431992725,2cyclictest500396-21kworker/u16:4+flush-8:006:06:595
294431992725,2cyclictest500396-21kworker/u16:4+events_unbound05:20:075
294431992725,2cyclictest339042-21kworker/u16:1+events_unbound04:42:315
294431992725,2cyclictest318182-21kworker/u16:0+events_unbound04:11:315
294427992725,2cyclictest687018-21kworker/u16:1+events_unbound06:48:504
294440992625,0cyclictest0-21swapper/707:12:527
294440992621,4cyclictest935248-21latency_hist08:38:247
294436992624,2cyclictest773419-21kworker/u16:3+events_unbound07:35:326
294436992624,2cyclictest712601-21kworker/u16:0+events_unbound07:55:256
294436992624,2cyclictest500396-21kworker/u16:4+flush-8:005:57:356
294436992624,2cyclictest449719-21kworker/u16:0+events_unbound05:00:326
294431992624,2cyclictest945024-21kworker/u16:0+events_unbound08:47:435
294431992624,2cyclictest701717-21kworker/u16:2+flush-8:007:14:465
294431992624,2cyclictest632274-21kworker/u16:2+events_unbound06:17:425
294431992624,2cyclictest575866-21kworker/u16:3+flush-8:005:51:195
294431992624,2cyclictest550569-21kworker/u16:2+events_unbound05:26:085
294431992624,2cyclictest500396-21kworker/u16:4+events_unbound06:39:195
294431992624,2cyclictest416272-21kworker/u16:3+flush-8:004:47:235
294431992624,2cyclictest3796992-21kworker/u16:8+flush-8:003:23:575
294431992624,2cyclictest339042-21kworker/u16:1+events_unbound04:27:285
294431992624,2cyclictest284383-21kworker/u16:3+flush-8:004:04:035
294427992621,4cyclictest380114-21grep04:03:404
294440992524,0cyclictest0-21swapper/708:07:227
294440992524,0cyclictest0-21swapper/707:15:277
294440992524,0cyclictest0-21swapper/704:58:507
294440992520,4cyclictest589235-21systemd-journal04:24:147
294436992523,2cyclictest773419-21kworker/u16:3+events_unbound07:49:316
294436992523,2cyclictest712601-21kworker/u16:0+events_unbound07:45:586
294436992523,2cyclictest500396-21kworker/u16:4+flush-8:006:32:476
294436992523,2cyclictest489950-21kworker/u16:3+events_unbound05:05:316
294431992523,2cyclictest823852-21kworker/u16:1+flush-8:008:33:165
294431992523,2cyclictest773419-21kworker/u16:3+events_unbound08:07:595
294431992523,2cyclictest701717-21kworker/u16:2+events_unbound07:20:395
294431992523,2cyclictest550569-21kworker/u16:2+events_unbound05:54:075
294431992523,2cyclictest318182-21kworker/u16:0+events_unbound03:59:395
294431992522,2cyclictest449719-21kworker/u16:0+events_unbound05:28:295
294427992522,2cyclictest500046-21kworker/u16:1+flush-8:005:03:404
294427992521,3cyclictest303975-21kworker/u16:2+flush-8:003:47:114
294440992423,0cyclictest0-21swapper/708:51:347
294440992423,0cyclictest0-21swapper/708:42:467
294440992423,0cyclictest0-21swapper/703:55:357
294440992417,5cyclictest4184977-17auditd05:43:257
294436992422,2cyclictest610803-21kworker/u16:1+events_unbound06:00:596
294436992422,2cyclictest500046-21kworker/u16:1+events_unbound05:21:036
294436992422,2cyclictest449719-21kworker/u16:0+events_unbound05:29:516
294431992422,2cyclictest860151-21kworker/u16:2+events_unbound08:26:455
294431992422,2cyclictest712601-21kworker/u16:0+events_unbound07:05:075
294431992422,2cyclictest687018-21kworker/u16:1+events_unbound06:35:015
294431992422,2cyclictest449719-21kworker/u16:0+flush-8:006:09:145
294431992422,2cyclictest339042-21kworker/u16:1+events_unbound04:21:465
294431992422,2cyclictest284383-21kworker/u16:3+flush-8:003:32:275
294427992421,2cyclictest449719-21kworker/u16:0+events_unbound05:33:384
294427992420,3cyclictest318182-21kworker/u16:0+events_unbound04:13:384
294440992317,5cyclictest589235-21systemd-journal07:38:207
29444099230,17cyclictest4175659-21snmpd07:23:277
294436992321,2cyclictest339042-21kworker/u16:1+events_unbound04:31:256
294431992321,2cyclictest701717-21kworker/u16:2+events_unbound07:27:595
294431992321,2cyclictest500046-21kworker/u16:1+events_unbound05:01:385
294431992321,2cyclictest349152-21kworker/u16:4+events_unbound03:56:035
294427992321,2cyclictest925152-21kworker/u16:2+events_unbound08:48:484
294427992320,2cyclictest284383-21kworker/u16:3+flush-8:003:39:424
294440992221,0cyclictest0-21swapper/708:18:017
294440992221,0cyclictest0-21swapper/704:08:277
294436992220,2cyclictest651123-21kworker/u16:3+events_unbound06:27:366
294431992220,2cyclictest873493-21kworker/u16:4+flush-8:008:09:265
294427992219,2cyclictest661420-21kworker/u16:5+flush-8:006:23:464
294440992116,4cyclictest589235-21systemd-journal08:31:087
294440992115,5cyclictest338655-21latency_hist03:43:257
294436992119,2cyclictest339042-21kworker/u16:1+flush-8:004:36:326
294431992119,2cyclictest318182-21kworker/u16:0+flush-8:003:37:315
294431992119,2cyclictest318182-21kworker/u16:0+events_unbound03:44:335
294427992118,2cyclictest349152-21kworker/u16:4+flush-8:003:53:074
294427992117,3cyclictest409129-21kworker/u16:2+flush-8:004:23:574
294440992020,0cyclictest0-21swapper/707:56:347
294440992019,0cyclictest0-21swapper/703:45:417
294440991918,0cyclictest0-21swapper/706:03:407
294440991918,0cyclictest0-21swapper/704:18:497
29444099190,17cyclictest684141-21kthreadcore06:33:457
294427991917,2cyclictest773419-21kworker/u16:3+flush-8:007:23:414
294427991917,2cyclictest500396-21kworker/u16:4+events_unbound05:48:544
294427991917,2cyclictest303975-21kworker/u16:2+flush-8:003:33:544
294440991817,0cyclictest0-21swapper/707:00:347
294440991817,0cyclictest0-21swapper/705:53:347
29444099180,16cyclictest389088-21sshd-session04:06:407
294427991816,2cyclictest945024-21kworker/u16:0+flush-8:008:43:504
29444099170,16cyclictest520568-21latency_hist05:13:257
29444099170,16cyclictest4175659-21snmpd06:11:287
294427991714,2cyclictest823852-21kworker/u16:1+flush-8:007:47:204
294440991616,0cyclictest0-21swapper/704:48:107
294440991615,0cyclictest0-21swapper/707:43:407
294440991615,0cyclictest0-21swapper/705:28:437
29444099160,14cyclictest4175659-21snmpd04:34:357
29444099160,14cyclictest409426-21latency_hist04:18:257
294427991615,1cyclictest0-21swapper/403:25:004
294427991614,2cyclictest500046-21kworker/u16:1+flush-8:005:18:404
294427991613,2cyclictest823852-21kworker/u16:1+flush-8:007:57:204
294427991612,3cyclictest318182-21kworker/u16:0+flush-8:004:12:114
294440991512,2cyclictest4175659-21snmpd04:55:597
29444099150,13cyclictest844179-21sshd-session07:52:497
294427991513,2cyclictest500046-21kworker/u16:1+flush-8:005:23:484
294427991513,1cyclictest580632-21find05:38:554
294427991511,3cyclictest701717-21kworker/u16:2+flush-8:007:33:564
29444099149,4cyclictest955399-21latency_hist08:48:247
294440991413,0cyclictest0-21swapper/705:51:317
294427991412,2cyclictest914904-21kworker/u16:3+events_unbound08:28:524
294427991412,2cyclictest712601-21kworker/u16:0+flush-8:007:13:414
294427991410,3cyclictest500396-21kworker/u16:4+flush-8:006:47:194
29444099139,3cyclictest96-21ksoftirqd/706:18:397
29444099139,3cyclictest4175659-21snmpd05:24:577
29444099138,4cyclictest589235-21systemd-journal08:24:007
294440991312,0cyclictest0-21swapper/706:38:517
294440991312,0cyclictest0-21swapper/703:58:417
294440991312,0cyclictest0-21swapper/703:33:407
294440991310,2cyclictest0-21swapper/705:18:257
29444099130,12cyclictest733008-21turbostat06:58:257
29442799139,3cyclictest409129-21kworker/u16:2+flush-8:004:33:114
294427991312,1cyclictest0-21swapper/404:43:544
29442799130,13cyclictest0-21swapper/405:28:404
29444099129,2cyclictest4175659-21snmpd05:04:337
29444099127,4cyclictest582360-21tune2fs05:43:397
29444099125,6cyclictest0-21swapper/705:18:397
294440991211,0cyclictest0-21swapper/703:28:477
29444099120,3cyclictest613290-21kthreadcore05:58:437
29444099120,2cyclictest802792-21ntpq07:28:487
29444099120,2cyclictest0-21swapper/703:48:537
29444099120,10cyclictest459952-21latency_hist04:43:267
29442799129,2cyclictest914904-21kworker/u16:3+flush-8:008:38:254
29442799129,2cyclictest672354-21kworker/u16:0+flush-8:006:28:504
29442799128,3cyclictest416272-21kworker/u16:3+flush-8:004:18:574
294427991211,1cyclictest0-21swapper/405:54:504
294427991211,1cyclictest0-21swapper/405:12:454
294427991211,1cyclictest0-21swapper/404:52:254
294427991211,1cyclictest0-21swapper/404:33:484
29442799120,1cyclictest0-21swapper/407:49:174
29444099114,6cyclictest589322-21tuned06:13:417
29444099110,10cyclictest4175659-21snmpd06:46:047
29442799118,2cyclictest773419-21kworker/u16:3+flush-8:007:18:494
29442799118,2cyclictest724136-21irqcore06:53:414
29442799118,2cyclictest712601-21kworker/u16:0+flush-8:007:38:534
294427991110,1cyclictest0-21swapper/408:33:544
294427991110,1cyclictest0-21swapper/408:11:124
294427991110,1cyclictest0-21swapper/403:53:404
29442799110,11cyclictest0-21swapper/406:19:454
29442799110,11cyclictest0-21swapper/406:10:554
29442799110,0cyclictest0-21swapper/408:22:524
29444099105,5cyclictest27763-21gvfs-afc-volume07:38:537
29442799108,2cyclictest701717-21kworker/u16:2+flush-8:007:03:534
29442799108,2cyclictest651123-21kworker/u16:3+flush-8:006:38:454
29442799108,2cyclictest651123-21kworker/u16:3+events_unbound06:33:394
2944409998,1cyclictest0-21swapper/707:18:447
2944279998,1cyclictest0-21swapper/408:13:344
2944279994,4cyclictest451115-21sed04:38:424
2944279986,1cyclictest0-21swapper/405:58:374
2944279986,1cyclictest0-21swapper/404:58:434
2944279985,2cyclictest642908-21kworker/u16:1+flush-8:006:13:534
2944279984,3cyclictest854849-21tr07:58:364
2944279983,4cyclictest754184-21cut07:08:404
2944279983,4cyclictest736827-21cut06:58:424
2944279973,3cyclictest794375-21expr07:28:364
2944279963,2cyclictest581742-21cpuspeed05:43:364
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional