You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 20:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Wed Nov 26, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
196924826,8sleep60-21swapper/606:50:126
238652460,0sleep00-21swapper/010:45:170
218224625,8sleep50-21swapper/506:53:085
275012440,2sleep6244899cyclictest08:57:436
17182440,0sleep70-21swapper/710:57:017
214424229,9sleep70-21swapper/706:52:377
179902420,0sleep60-21swapper/610:24:326
46982410,0sleep00-21swapper/008:59:210
46982410,0sleep00-21swapper/008:59:210
3722410,0sleep20-21swapper/209:04:122
3722410,0sleep20-21swapper/209:04:122
31552410,0sleep70-21swapper/709:31:007
235212410,0sleep50-21swapper/510:45:145
228942410,1sleep2763-21runrttasks10:00:182
224124129,8sleep00-21swapper/006:53:510
213024129,8sleep20-21swapper/206:52:242
54242400,0sleep20-21swapper/211:42:212
196724028,9sleep40-21swapper/406:50:104
196624028,8sleep30-21swapper/306:50:093
251002390,0sleep70-21swapper/708:29:257
213823928,8sleep10-21swapper/106:52:311
79572380,0sleep30-21swapper/309:47:373
79572380,0sleep30-21swapper/309:47:373
128192350,0sleep20-21swapper/207:09:292
2447992816,5cyclictest57-21ksoftirqd/509:29:525
244999234,18cyclictest15573-21bash11:04:087
2447992317,1cyclictest57-21ksoftirqd/510:56:525
244999221,17cyclictest26671-21sshd12:03:587
2447992217,1cyclictest57-21ksoftirqd/509:41:565
2447992217,1cyclictest57-21ksoftirqd/509:41:565
244999213,10cyclictest25029-21sshd09:55:327
244999210,18cyclictest2062-21cp10:51:567
244999210,17cyclictest11734-21systemd-cgroups09:48:167
244999210,17cyclictest11734-21systemd-cgroups09:48:167
244999210,14cyclictest32320-21sshd08:58:347
2447992114,6cyclictest57-21ksoftirqd/511:18:305
2447992112,9cyclictest57-21ksoftirqd/510:11:465
2447992110,0cyclictest0-21swapper/508:22:325
244999208,11cyclictest9919-21sshd09:10:587
244999202,15cyclictest16989-21cut10:39:267
244999200,17cyclictest2276-21bash10:32:127
244799209,6cyclictest57-21ksoftirqd/510:27:525
2447992016,0cyclictest57-21ksoftirqd/510:39:385
2447992015,0cyclictest0-21swapper/511:09:165
2447992011,9cyclictest57-21ksoftirqd/512:23:505
244999192,16cyclictest29266-21cp11:35:567
244999192,16cyclictest21155-21bash11:39:327
244999191,17cyclictest9117-21sshd09:05:267
244799198,6cyclictest57-21ksoftirqd/508:48:325
244799198,0cyclictest0-21swapper/507:40:265
2447991914,5cyclictest57-21ksoftirqd/509:16:465
2447991914,0cyclictest0-21swapper/507:34:105
2447991913,1cyclictest57-21ksoftirqd/511:04:185
2447991913,1cyclictest57-21ksoftirqd/511:04:185
244999188,9cyclictest1682-21sshd10:02:107
244999182,15cyclictest711-21dbus-daemon09:18:507
244999182,12cyclictest10415-21sshd11:56:547
2449991815,1cyclictest28105-21nfsd07:56:007
244999180,17cyclictest27146-21bash09:03:127
244999180,17cyclictest27146-21bash09:03:127
2447991813,5cyclictest0-21swapper/508:31:325
2447991813,1cyclictest57-21ksoftirqd/509:27:005
2447991812,1cyclictest57-21ksoftirqd/511:29:105
244999175,11cyclictest1247-21sshd12:19:147
244999174,9cyclictest9166-21sshd11:52:107
244999171,9cyclictest14711-21sshd11:33:587
244999170,17cyclictest15433-21sshd11:14:147
244799177,5cyclictest0-21swapper/508:59:285
244799177,5cyclictest0-21swapper/508:59:285
244799176,5cyclictest0-21swapper/507:23:165
244799176,3cyclictest57-21ksoftirqd/510:03:405
244799174,6cyclictest57-21ksoftirqd/509:45:105
244799174,6cyclictest57-21ksoftirqd/509:45:105
2447991712,5cyclictest57-21ksoftirqd/511:00:185
2447991712,4cyclictest57-21ksoftirqd/510:50:145
244799171,16cyclictest0-21swapper/509:51:165
244799170,17cyclictest0-21swapper/512:09:365
244099170,14cyclictest12622-21sshd09:16:393
8902160,0sleep00-21swapper/009:56:550
244999168,8cyclictest0-21swapper/707:10:147
244999165,9cyclictest0-21swapper/707:14:287
244999165,11cyclictest0-21swapper/708:07:487
244999164,11cyclictest6244-21bash12:05:267
244999163,7cyclictest0-21swapper/708:44:107
244999161,11cyclictest8732-21sshd11:13:087
244999160,6cyclictest0-21swapper/708:14:227
244999160,5cyclictest0-21swapper/710:19:187
244999160,15cyclictest17991-21iostat_ios07:19:287
244999160,12cyclictest27930-21sshd10:11:147
244999160,10cyclictest0-21swapper/707:50:207
244799166,5cyclictest57-21ksoftirqd/511:34:105
244799165,5cyclictest0-21swapper/507:54:205
244799165,0cyclictest0-21swapper/508:17:105
2447991611,5cyclictest57-21ksoftirqd/512:07:105
2447991611,5cyclictest57-21ksoftirqd/510:29:305
2447991611,5cyclictest57-21ksoftirqd/509:38:325
2447991611,5cyclictest0-21swapper/510:36:105
2447991611,1cyclictest57-21ksoftirqd/511:34:245
2447991611,0cyclictest0-21swapper/507:49:565
2447991611,0cyclictest0-21swapper/506:58:185
2447991610,5cyclictest57-21ksoftirqd/509:19:205
2447991610,1cyclictest57-21ksoftirqd/510:19:165
244799160,16cyclictest0-21swapper/512:03:005
2446991614,0cyclictest481ktimersoftd/407:45:514
244999157,5cyclictest0-21swapper/709:49:247
244999156,5cyclictest0-21swapper/707:00:487
244999154,11cyclictest0-21swapper/709:29:007
244999154,11cyclictest0-21swapper/706:55:587
244999152,10cyclictest0-21swapper/708:44:507
244999152,10cyclictest0-21swapper/707:38:147
244999151,11cyclictest17099-21sshd10:34:277
244999150,14cyclictest2030-21cp11:21:507
244799159,1cyclictest57-21ksoftirqd/509:54:305
244799154,5cyclictest0-21swapper/508:37:405
244799154,0cyclictest0-21swapper/508:40:185
244799154,0cyclictest0-21swapper/508:06:225
2447991510,5cyclictest57-21ksoftirqd/511:43:025
2447991510,5cyclictest57-21ksoftirqd/508:52:565
2447991510,5cyclictest0-21swapper/510:07:025
2447991510,5cyclictest0-21swapper/509:13:085
2447991510,5cyclictest0-21swapper/508:59:065
2447991510,1cyclictest57-21ksoftirqd/509:04:285
244799150,15cyclictest0-21swapper/511:45:525
242099150,15cyclictest99-21kswapd009:57:071
197172150,0sleep10-21swapper/112:21:441
244999144,6cyclictest4222-21grepconf.sh10:27:327
244999144,10cyclictest28105-21nfsd11:27:287
244999144,10cyclictest0-21swapper/707:44:427
244999143,11cyclictest0-21swapper/712:10:167
244999143,10cyclictest21906-21grepconf.sh10:05:227
244999142,11cyclictest27794-21sshd11:45:247
2449991410,4cyclictest28105-21nfsd08:20:427
244999140,3cyclictest0-21swapper/709:40:047
244999140,3cyclictest0-21swapper/709:40:047
244899143,6cyclictest25281-21sshd09:55:346
244799149,5cyclictest57-21ksoftirqd/508:10:245
244799149,5cyclictest57-21ksoftirqd/508:03:365
244799149,5cyclictest57-21ksoftirqd/507:02:225
244799149,0cyclictest0-21swapper/507:25:405
244799148,5cyclictest0-21swapper/511:57:585
244799148,1cyclictest57-21ksoftirqd/507:04:345
244799144,5cyclictest561ktimersoftd/512:15:545
244799142,6cyclictest0-21swapper/507:11:165
244799142,12cyclictest0-21swapper/511:49:165
2447991411,3cyclictest57-21ksoftirqd/510:14:345
244999138,5cyclictest0-21swapper/710:46:187
244999138,5cyclictest0-21swapper/708:24:467
244999138,5cyclictest0-21swapper/708:00:207
244999135,5cyclictest668-21systemd-logind09:38:027
244999134,6cyclictest0-21swapper/707:24:467
244999133,9cyclictest18777-21sshd10:14:407
244999132,11cyclictest0-21swapper/709:19:447
244999132,11cyclictest0-21swapper/707:42:167
244999130,7cyclictest0-21swapper/711:08:527
244999130,7cyclictest0-21swapper/711:08:527
244999130,6cyclictest0-21swapper/707:04:507
244999130,12cyclictest32182-21grepconf.sh12:19:047
244899133,7cyclictest31240-21grepconf.sh10:06:546
244899130,9cyclictest0-21swapper/611:12:296
244799138,5cyclictest0-21swapper/507:45:285
244799138,0cyclictest0-21swapper/507:36:365
2447991311,1cyclictest57-21ksoftirqd/511:21:555
244799131,0cyclictest0-21swapper/507:14:205
244999127,5cyclictest0-21swapper/708:49:267
244999122,5cyclictest0-21swapper/707:32:327
244999121,11cyclictest0-21swapper/708:38:487
244899128,4cyclictest0-21swapper/609:04:286
244899128,3cyclictest0-21swapper/609:47:026
244899128,3cyclictest0-21swapper/609:47:026
244899125,6cyclictest24092-21grep10:10:366
244899124,7cyclictest711-21dbus-daemon11:16:296
244899124,7cyclictest30530-21sshd10:56:176
244899124,7cyclictest25596-21grepconf.sh10:35:406
244899124,7cyclictest25056-21sshd09:34:346
2448991210,1cyclictest0-21swapper/612:17:356
244899120,2cyclictest7917-21sshd09:15:526
244899120,2cyclictest4916-21grepconf.sh09:31:246
244899120,2cyclictest4871-21sshd09:52:286
244899120,2cyclictest32520-21grepconf.sh11:21:326
244899120,1cyclictest32073-21sshd12:04:316
244799120,6cyclictest0-21swapper/508:25:165
2440991211,1cyclictest41-21ksoftirqd/309:55:383
244099120,12cyclictest0-21swapper/310:12:523
244099120,11cyclictest859-21polkitd09:25:153
2420991210,1cyclictest0-21swapper/110:27:411
118442120,0sleep1241ktimersoftd/110:23:471
244899119,1cyclictest668-21systemd-logind10:59:346
244899119,1cyclictest1730-21sshd10:32:076
244899115,5cyclictest740-21gdbus09:41:096
244899115,5cyclictest740-21gdbus09:41:096
244899115,5cyclictest24693-21grep10:50:216
244899114,6cyclictest8223-21grepconf.sh11:32:566
244899114,6cyclictest8070-21bash09:26:386
244899114,6cyclictest2771-21bash11:36:516
244899114,6cyclictest22452-21sshd10:45:036
244899114,6cyclictest14586-21grepconf.sh10:14:196
244899114,6cyclictest10438-21grepconf.sh12:01:346
244899113,7cyclictest711-21dbus-daemon12:19:476
244899113,7cyclictest4393-21grep08:59:186
244899113,7cyclictest4393-21grep08:59:186
244899113,7cyclictest20701-21grep10:39:476
244899113,7cyclictest18341-21grep11:58:066
244899113,7cyclictest17457-21bash09:12:146
244899113,2cyclictest25876-21sshd10:00:486
244899110,2cyclictest9315-21grepconf.sh11:52:116
244899110,2cyclictest29268-21bash11:40:526
244899110,2cyclictest1-21systemd11:04:476
244899110,2cyclictest1-21systemd11:04:476
244899110,10cyclictest4959-21sshd10:22:406
244899110,1cyclictest668-21systemd-logind12:10:396
244899110,1cyclictest30800-21sensors_temp09:19:326
244899110,0cyclictest0-21swapper/611:27:466
244699119,1cyclictest711-21dbus-daemon10:24:014
244099119,1cyclictest0-21swapper/311:35:193
244099110,11cyclictest0-21swapper/311:43:203
244099110,1cyclictest0-21swapper/309:42:353
244099110,1cyclictest0-21swapper/309:42:353
243299119,1cyclictest33-21ksoftirqd/209:44:242
243299119,1cyclictest33-21ksoftirqd/209:44:242
243299111,4cyclictest20580-21sshd12:21:532
243299110,10cyclictest11392-21sshd10:38:382
241299119,1cyclictest9-21ksoftirqd/010:54:530
241299110,1cyclictest0-21swapper/012:11:100
224672110,2sleep3244099cyclictest10:50:003
244899103,6cyclictest15387-21cut07:14:306
244899102,2cyclictest757-21snmpd07:42:516
244899100,9cyclictest0-21swapper/607:44:286
244899100,1cyclictest19065-21systemd-cgroups11:44:236
244699109,1cyclictest49-21ksoftirqd/410:49:134
244699109,1cyclictest0-21swapper/410:09:594
244699108,1cyclictest49-21ksoftirqd/409:06:134
244699103,6cyclictest30148-21sshd09:19:294
244699103,6cyclictest14274-21sshd09:48:414
244699103,6cyclictest14274-21sshd09:48:414
244699101,4cyclictest3699-21sshd11:12:074
244699100,4cyclictest15764-21sshd11:53:094
244699100,10cyclictest859-21polkitd11:00:074
244699100,1cyclictest0-21swapper/411:42:304
244099109,1cyclictest0-21swapper/312:01:203
244099108,2cyclictest17781-21grepconf.sh09:06:573
244099108,1cyclictest0-21swapper/312:11:343
244099100,9cyclictest19086-21kworker/3:409:09:283
243299100,10cyclictest0-21swapper/211:04:582
243299100,10cyclictest0-21swapper/211:04:582
243299100,1cyclictest0-21swapper/212:05:492
242099109,1cyclictest0-21swapper/111:59:281
242099104,4cyclictest12868-21id11:08:451
242099104,4cyclictest12868-21id11:08:451
242099103,4cyclictest24539-21/usr/sbin/munin09:29:261
242099100,1cyclictest25474-21sshd10:30:421
241299109,1cyclictest0-21swapper/009:48:010
241299109,1cyclictest0-21swapper/009:48:010
241299109,1cyclictest0-21swapper/009:29:550
175912100,0sleep40-21swapper/412:16:514
157312100,0sleep10-21swapper/111:53:091
24499995,0cyclictest0-21swapper/708:10:247
24489993,5cyclictest757-21snmpd07:57:226
24489993,5cyclictest757-21snmpd07:29:456
24489993,5cyclictest0-21swapper/608:22:106
24489992,7cyclictest0-21swapper/606:55:236
24489992,6cyclictest0-21swapper/608:47:326
24489992,2cyclictest757-21snmpd08:49:236
24489992,1cyclictest18597-21ls07:19:326
24489991,1cyclictest0-21swapper/608:39:296
24489991,1cyclictest0-21swapper/608:09:316
24489991,1cyclictest0-21swapper/607:59:256
24489991,1cyclictest0-21swapper/607:34:286
24489991,1cyclictest0-21swapper/606:59:276
24489990,8cyclictest0-21swapper/607:24:286
24489990,2cyclictest9478-21cat07:04:316
24489990,2cyclictest29473-21sed08:34:336
24489990,2cyclictest16382-21cut08:14:256
24489990,2cyclictest11626-21expr07:09:236
24489990,2cyclictest11039-21cut08:04:286
24489990,1cyclictest24748-21/usr/sbin/munin08:29:286
24469999,0cyclictest0-21swapper/410:35:034
24469997,1cyclictest49-21ksoftirqd/411:35:314
24469997,1cyclictest49-21ksoftirqd/409:17:454
24469997,0cyclictest481ktimersoftd/411:26:404
24469993,6cyclictest0-21swapper/411:15:294
24469992,6cyclictest15785-21systemd-cgroups08:55:284
24469992,6cyclictest13545-21sshd09:53:574
24469990,0cyclictest0-21swapper/410:39:194
24469990,0cyclictest0-21swapper/410:29:324
24469990,0cyclictest0-21swapper/410:17:154
24469990,0cyclictest0-21swapper/409:00:094
24469990,0cyclictest0-21swapper/409:00:094
24409997,1cyclictest41-21ksoftirqd/309:38:103
24409995,1cyclictest680-21nscd12:21:353
24409994,3cyclictest24153-21kworker/3:211:25:563
24409994,2cyclictest0-21swapper/310:05:163
24409993,3cyclictest8358-21sshd09:53:033
24409991,4cyclictest668-21systemd-logind11:44:243
24409990,9cyclictest0-21swapper/310:28:093
24409990,9cyclictest0-21swapper/309:59:373
24409990,9cyclictest0-21swapper/309:20:563
24409990,8cyclictest24265-21kworker/3:212:04:473
24409990,0cyclictest0-21swapper/312:14:343
24409990,0cyclictest0-21swapper/310:22:383
24329999,0cyclictest0-21swapper/211:25:472
24329999,0cyclictest0-21swapper/211:16:002
24329991,3cyclictest30843-21sshd11:45:552
24329990,9cyclictest0-21swapper/209:57:202
24329990,9cyclictest0-21swapper/209:13:592
24329990,0cyclictest0-21swapper/210:58:112
24329990,0cyclictest0-21swapper/210:04:352
24209997,1cyclictest25-21ksoftirqd/110:41:531
24209993,5cyclictest1334-21grep09:09:321
24209993,4cyclictest27819-21sshd11:35:411
24209993,3cyclictest28317-21sshd11:15:551
24209993,3cyclictest0-21swapper/109:39:131
24209992,4cyclictest26842-21sshd11:54:321
24209992,4cyclictest14529-21grep10:59:001
24209992,3cyclictest711-21dbus-daemon10:34:281
24209991,4cyclictest12561-21sshd10:09:081
24209990,9cyclictest0-21swapper/112:11:081
24209990,9cyclictest0-21swapper/111:23:501
24209990,9cyclictest0-21swapper/109:45:091
24209990,9cyclictest0-21swapper/109:45:091
24209990,9cyclictest0-21swapper/109:03:221
24209990,9cyclictest0-21swapper/109:03:221
24209990,9cyclictest0-21swapper/108:54:231
24209990,0cyclictest0-21swapper/111:46:231
24209990,0cyclictest0-21swapper/111:33:371
24209990,0cyclictest0-21swapper/109:49:241
24209990,0cyclictest0-21swapper/109:39:381
24209990,0cyclictest0-21swapper/109:39:381
24129999,0cyclictest0-21swapper/012:08:100
24129999,0cyclictest0-21swapper/010:02:410
24129998,1cyclictest9-21ksoftirqd/011:35:190
24129998,1cyclictest9-21ksoftirqd/010:34:250
24129998,1cyclictest9-21ksoftirqd/010:04:310
24129997,1cyclictest0-21swapper/011:18:210
24129993,3cyclictest1-21systemd09:42:170
24129993,3cyclictest1-21systemd09:42:170
24129992,4cyclictest19017-21cp11:44:230
24129990,9cyclictest0-21swapper/011:56:400
24129990,9cyclictest0-21swapper/011:22:370
24129990,9cyclictest0-21swapper/011:12:500
24129990,8cyclictest6737-21kworker/0:109:17:100
24129990,0cyclictest0-21swapper/012:21:250
16280290,0sleep316282-21unixbench_singl07:14:353
30070280,1sleep20-21swapper/211:50:232
27660280,1sleep327724-21sshd10:36:003
24489980,2cyclictest1935-21expr07:49:256
24489980,1cyclictest21948-21cpuspeed_turbos08:24:276
24469988,0cyclictest0-21swapper/412:06:014
24469988,0cyclictest0-21swapper/411:06:544
24469988,0cyclictest0-21swapper/411:06:544
24469988,0cyclictest0-21swapper/409:28:224
24469986,1cyclictest49-21ksoftirqd/412:12:194
24469986,1cyclictest49-21ksoftirqd/412:01:564
24469985,2cyclictest27612-21sshd12:22:564
24469984,1cyclictest3089-21sshd09:09:384
24469982,5cyclictest0-21swapper/410:24:354
24469981,6cyclictest1-21systemd09:36:574
24469980,7cyclictest31497-21sshd10:56:264
24469980,4cyclictest31547-21sshd11:30:594
24469980,2cyclictest25896-21sshd11:20:224
24409988,0cyclictest0-21swapper/310:54:573
24409986,2cyclictest15752-21kworker/3:008:15:403
24409986,1cyclictest5126-21kworker/3:111:04:163
24409986,1cyclictest5126-21kworker/3:111:04:163
24409986,1cyclictest373-21kworker/3:311:30:043
24409986,1cyclictest28517-21kworker/3:011:23:123
24409986,1cyclictest25270-21kworker/3:110:32:563
24409986,1cyclictest23237-21kworker/3:309:33:283
24409986,1cyclictest2262-21kworker/3:207:04:303
24409986,1cyclictest10676-21kworker/3:308:57:343
24409984,1cyclictest2079-21sshd10:17:163
24409983,4cyclictest20688-21sshd10:59:363
24409983,1cyclictest1-21systemd11:54:153
24409982,3cyclictest18005-21sshd10:39:293
24409981,4cyclictest31421-21bash11:50:343
24409980,8cyclictest24153-21kworker/3:211:17:473
24329988,0cyclictest0-21swapper/212:10:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional