You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 21:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sun Nov 16, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27512820,0sleep1231rcuc/109:08:521
3009699740,73cyclictest21070-21sshd09:11:204
2985924926,8sleep50-21swapper/506:54:125
124352480,0sleep60-21swapper/612:13:336
271262460,1sleep233-21ksoftirqd/210:21:172
268532460,1sleep4763-21runrttasks10:54:104
2972224525,8sleep00-21swapper/006:52:230
15612450,0sleep70-21swapper/710:45:407
126992450,0sleep50-21swapper/509:28:585
200222440,2sleep53010499cyclictest11:58:265
4722420,0sleep10-21swapper/110:17:391
2963624122,15sleep10-21swapper/106:51:121
30352400,0sleep60-21swapper/607:59:466
2973424027,9sleep30-21swapper/306:52:343
2964224028,8sleep70-21swapper/706:51:187
171232400,0sleep00-21swapper/009:52:310
322132390,0sleep30-21swapper/312:11:543
318872390,0sleep30-21swapper/310:35:593
2985623928,8sleep20-21swapper/206:54:092
2972823928,8sleep60-21swapper/606:52:296
2735423926,9sleep40-21swapper/406:49:384
313452380,0sleep70-21swapper/710:03:527
271092370,0sleep30-21swapper/310:44:483
155822370,0sleep10-21swapper/111:41:481
188622360,0sleep00-21swapper/010:06:330
30104992413,5cyclictest0-21swapper/507:07:275
3010499238,9cyclictest0-21swapper/507:10:475
228092210,0sleep6641ktimersoftd/607:34:566
3010499208,6cyclictest0-21swapper/510:22:355
30088992019,1cyclictest17836-21kworker/3:011:01:353
3012199198,5cyclictest0-21swapper/710:55:337
3012199198,5cyclictest0-21swapper/708:48:097
3012199198,5cyclictest0-21swapper/708:37:177
3012199198,5cyclictest0-21swapper/708:31:197
3012199198,5cyclictest0-21swapper/708:20:017
3012199198,5cyclictest0-21swapper/707:56:137
3012199198,5cyclictest0-21swapper/707:20:357
3012199198,5cyclictest0-21swapper/707:13:517
3012199198,0cyclictest0-21swapper/707:37:197
30121991914,5cyclictest0-21swapper/709:13:177
3012199191,6cyclictest0-21swapper/707:42:497
3010499198,0cyclictest0-21swapper/510:34:455
3010499198,0cyclictest0-21swapper/509:38:035
30104991914,5cyclictest0-21swapper/511:59:515
30104991914,5cyclictest0-21swapper/511:59:515
30088991913,2cyclictest41-21ksoftirqd/312:21:463
3006499191,17cyclictest19812-21grepconf.sh11:33:470
3012199187,5cyclictest0-21swapper/709:03:037
3012199187,5cyclictest0-21swapper/708:54:437
3012199187,5cyclictest0-21swapper/707:50:577
3012199187,5cyclictest0-21swapper/707:47:017
3012199187,0cyclictest0-21swapper/708:09:597
3012199187,0cyclictest0-21swapper/708:04:277
30121991813,5cyclictest0-21swapper/707:16:557
30121991813,0cyclictest0-21swapper/708:26:497
3012199180,6cyclictest0-21swapper/710:16:517
3010499187,5cyclictest0-21swapper/512:15:595
3010499187,5cyclictest0-21swapper/510:44:575
3010499187,11cyclictest0-21swapper/506:58:235
3010499187,0cyclictest0-21swapper/511:33:295
3010499187,0cyclictest0-21swapper/511:27:595
3010499187,0cyclictest0-21swapper/510:58:535
3010499187,0cyclictest0-21swapper/510:40:415
3010499187,0cyclictest0-21swapper/509:57:435
3010499187,0cyclictest0-21swapper/508:40:215
3010499187,0cyclictest0-21swapper/507:44:595
3010499187,0cyclictest0-21swapper/507:39:475
3010499187,0cyclictest0-21swapper/507:35:535
3010499187,0cyclictest0-21swapper/507:18:195
3010499186,6cyclictest0-21swapper/509:39:515
3010499186,6cyclictest0-21swapper/508:52:175
3010499186,6cyclictest0-21swapper/508:10:555
3010499186,6cyclictest0-21swapper/507:29:135
3010499186,0cyclictest0-21swapper/508:46:235
3010499180,6cyclictest0-21swapper/508:17:275
3010499180,0cyclictest0-21swapper/508:25:475
271312180,0sleep2763-21runrttasks10:35:102
3012199177,4cyclictest0-21swapper/712:21:517
3012199176,0cyclictest0-21swapper/707:04:417
3010499176,5cyclictest0-21swapper/507:51:015
3010499176,5cyclictest0-21swapper/507:02:535
3010499176,0cyclictest0-21swapper/510:52:595
3010499176,0cyclictest0-21swapper/508:37:435
3010499176,0cyclictest0-21swapper/508:22:415
3010499175,12cyclictest0-21swapper/512:21:595
3010499175,0cyclictest0-21swapper/507:59:075
30088991716,1cyclictest26685-21kworker/3:310:44:053
30088991715,1cyclictest41-21ksoftirqd/309:15:133
30088991712,4cyclictest41-21ksoftirqd/311:11:043
297152170,1sleep2763-21runrttasks10:59:052
277012170,0sleep10-21swapper/111:39:161
3012199168,5cyclictest0-21swapper/712:09:457
3012199165,5cyclictest0-21swapper/712:00:517
3012199165,5cyclictest0-21swapper/712:00:517
3012199165,5cyclictest0-21swapper/710:37:397
3012199165,5cyclictest0-21swapper/708:43:457
3012199165,5cyclictest0-21swapper/707:27:177
3012199165,0cyclictest0-21swapper/710:32:057
30121991611,0cyclictest0-21swapper/707:02:417
3010499168,8cyclictest0-21swapper/511:38:595
3010499168,2cyclictest0-21swapper/511:09:375
3010499168,2cyclictest0-21swapper/510:31:095
3010499168,2cyclictest0-21swapper/510:05:175
3010499165,0cyclictest0-21swapper/509:24:255
30088991615,1cyclictest41-21ksoftirqd/310:29:553
30088991614,2cyclictest41-21ksoftirqd/310:58:003
30088991614,1cyclictest41-21ksoftirqd/310:09:543
30088991614,1cyclictest41-21ksoftirqd/309:07:303
30088991613,3cyclictest41-21ksoftirqd/311:41:053
30088991612,1cyclictest41-21ksoftirqd/310:54:283
148322160,0sleep10-21swapper/109:29:181
3012199154,5cyclictest0-21swapper/711:38:437
3012199154,5cyclictest0-21swapper/710:21:477
3012199154,0cyclictest0-21swapper/711:31:237
3012199154,0cyclictest0-21swapper/710:10:477
3012199154,0cyclictest0-21swapper/708:17:417
30121991510,0cyclictest0-21swapper/710:49:577
30121991510,0cyclictest0-21swapper/708:52:057
3012199150,9cyclictest0-21swapper/712:09:097
3012199150,9cyclictest0-21swapper/711:19:457
3012199150,9cyclictest0-21swapper/711:15:237
3012199150,9cyclictest0-21swapper/710:25:197
3012199150,9cyclictest0-21swapper/709:25:097
3010499157,2cyclictest0-21swapper/509:02:555
3010499154,0cyclictest0-21swapper/510:59:535
3010499153,12cyclictest0-21swapper/511:47:195
3010499150,9cyclictest0-21swapper/510:18:475
30096991512,1cyclictest481ktimersoftd/412:22:554
3008899152,1cyclictest131rcu_sched09:09:493
30088991515,0cyclictest14219-21kworker/3:008:22:273
30088991514,1cyclictest41-21ksoftirqd/308:55:133
30088991513,1cyclictest41-21ksoftirqd/311:49:533
30088991513,1cyclictest41-21ksoftirqd/311:17:073
30088991513,1cyclictest41-21ksoftirqd/310:15:093
30088991513,1cyclictest41-21ksoftirqd/309:44:593
30088991513,1cyclictest41-21ksoftirqd/309:44:063
30088991512,3cyclictest41-21ksoftirqd/309:51:103
30088991512,2cyclictest41-21ksoftirqd/310:09:343
30088991511,4cyclictest41-21ksoftirqd/311:55:003
30088991511,3cyclictest41-21ksoftirqd/309:30:193
30088991511,1cyclictest41-21ksoftirqd/310:00:443
30088991510,3cyclictest41-21ksoftirqd/311:05:193
30088991510,3cyclictest41-21ksoftirqd/311:05:193
30080991514,1cyclictest99-21kswapd007:50:022
30080991512,2cyclictest861-21gdbus10:18:102
3007299150,12cyclictest23767-21latency12:14:521
171672150,0sleep20-21swapper/208:56:432
149582150,0sleep70-21swapper/709:33:487
3012199148,1cyclictest0-21swapper/711:50:177
3012199147,6cyclictest28654-21sshd09:58:557
3012199143,9cyclictest0-21swapper/709:21:397
3012199143,5cyclictest0-21swapper/707:31:557
3012199143,0cyclictest0-21swapper/709:05:097
3012199142,6cyclictest0-21swapper/711:07:237
3012199142,6cyclictest0-21swapper/711:07:237
3012199142,6cyclictest0-21swapper/711:01:497
3012199142,6cyclictest0-21swapper/709:39:237
3010499148,5cyclictest0-21swapper/509:50:015
3010499148,5cyclictest0-21swapper/509:12:575
3010499148,1cyclictest758-21sshd11:05:195
3010499148,1cyclictest758-21sshd11:05:195
3010499148,0cyclictest0-21swapper/510:10:355
3010499147,5cyclictest0-21swapper/509:09:055
3010499147,2cyclictest0-21swapper/511:51:095
3010499146,5cyclictest0-21swapper/511:20:395
3010499143,5cyclictest0-21swapper/507:21:195
30088991414,0cyclictest32536-21kworker/3:207:17:593
30088991414,0cyclictest21306-21kworker/3:108:15:313
30088991413,1cyclictest5831-21kworker/3:110:22:033
30088991413,1cyclictest41-21ksoftirqd/312:15:263
30088991413,1cyclictest41-21ksoftirqd/311:37:423
30088991413,1cyclictest32536-21kworker/3:207:14:313
30088991412,1cyclictest41-21ksoftirqd/312:04:483
30088991412,1cyclictest41-21ksoftirqd/309:55:543
30088991412,1cyclictest41-21ksoftirqd/309:38:503
30088991411,1cyclictest41-21ksoftirqd/311:30:163
183992140,0sleep00-21swapper/008:24:540
80972130,0sleep60-21swapper/610:55:486
3012199138,5cyclictest0-21swapper/711:55:097
3012199138,5cyclictest0-21swapper/711:44:357
3012199138,5cyclictest0-21swapper/711:42:237
3012199138,5cyclictest0-21swapper/711:25:257
3012199138,5cyclictest0-21swapper/709:50:537
3012199138,5cyclictest0-21swapper/709:45:297
3012199138,5cyclictest0-21swapper/709:16:317
3012199138,5cyclictest0-21swapper/708:07:137
3012199138,5cyclictest0-21swapper/706:54:497
3012199138,1cyclictest669-21avahi-daemon10:06:437
3012199138,0cyclictest0-21swapper/712:15:257
3012199132,5cyclictest0-21swapper/711:13:557
3012199131,0cyclictest0-21swapper/710:39:397
30110991312,0cyclictest85350irq/25-em1-rx-009:01:316
3010499138,5cyclictest0-21swapper/512:11:095
3010499138,5cyclictest0-21swapper/509:59:455
3010499138,5cyclictest0-21swapper/509:30:115
3010499138,5cyclictest0-21swapper/508:32:055
3010499138,5cyclictest0-21swapper/508:04:555
3010499138,5cyclictest0-21swapper/507:59:375
3010499138,5cyclictest0-21swapper/507:29:535
3010499138,0cyclictest0-21swapper/512:05:095
3010499138,0cyclictest0-21swapper/511:40:295
3010499138,0cyclictest0-21swapper/510:26:375
3010499138,0cyclictest0-21swapper/509:45:115
3010499138,0cyclictest0-21swapper/508:55:595
3010499132,6cyclictest12576-21sshd11:14:455
3010499130,13cyclictest0-21swapper/509:14:575
3008899139,1cyclictest41-21ksoftirqd/311:46:233
30088991312,1cyclictest32536-21kworker/3:207:06:113
30088991312,1cyclictest21306-21kworker/3:107:56:073
30088991312,1cyclictest14219-21kworker/3:008:46:333
30088991312,1cyclictest14219-21kworker/3:008:36:093
30088991311,1cyclictest41-21ksoftirqd/311:24:483
30088991311,1cyclictest41-21ksoftirqd/310:26:233
30088991311,1cyclictest41-21ksoftirqd/309:29:013
30088991311,1cyclictest41-21ksoftirqd/309:23:543
30088991310,1cyclictest41-21ksoftirqd/312:00:533
30088991310,1cyclictest41-21ksoftirqd/312:00:533
295112130,0sleep00-21swapper/010:59:030
225632130,0sleep60-21swapper/610:48:526
163082130,0sleep20-21swapper/209:15:092
3012199120,1cyclictest0-21swapper/709:39:517
30088991212,0cyclictest21306-21kworker/3:108:06:513
30088991212,0cyclictest14219-21kworker/3:008:51:453
30088991211,1cyclictest32536-21kworker/3:207:28:233
30088991211,1cyclictest32536-21kworker/3:207:02:133
30088991210,1cyclictest41-21ksoftirqd/311:20:093
30088991210,1cyclictest41-21ksoftirqd/308:59:483
3006499129,2cyclictest668-21systemd-logind09:40:440
121992120,0sleep00-21swapper/009:10:030
3011099110,10cyclictest711-21dbus-daemon11:08:196
3011099110,10cyclictest711-21dbus-daemon11:08:196
3011099110,10cyclictest4590-21systemd-cgroups12:00:216
3011099110,10cyclictest4590-21systemd-cgroups12:00:216
30088991111,0cyclictest21306-21kworker/3:107:49:313
3008099119,1cyclictest711-21dbus-daemon09:33:242
3008099119,1cyclictest15324-21memory09:19:512
3008099113,4cyclictest24597-21sshd11:42:592
3008099112,3cyclictest24728-21sshd12:10:542
3008099110,2cyclictest16717-21bash10:29:302
3008099110,10cyclictest0-21swapper/210:32:122
3008099110,10cyclictest0-21swapper/209:37:112
3008099110,1cyclictest1-21systemd09:39:432
3008099110,1cyclictest0-21swapper/209:44:452
3007299111,4cyclictest3121-21sshd11:27:031
3007299111,4cyclictest11697-21sshd09:09:571
3007299110,10cyclictest12239-21id10:05:321
3007299110,1cyclictest711-21dbus-daemon10:20:401
3006499111,4cyclictest32399-21sshd11:55:480
55852100,2sleep63011099cyclictest09:55:266
3011099109,1cyclictest0-21swapper/611:02:296
3011099108,1cyclictest711-21dbus-daemon10:17:156
3011099103,6cyclictest13235-21sshd12:17:376
3011099100,10cyclictest0-21swapper/610:03:226
3011099100,1cyclictest0-21swapper/612:21:006
3011099100,1cyclictest0-21swapper/608:59:316
3009699109,1cyclictest49-21ksoftirqd/412:11:514
3009699109,1cyclictest49-21ksoftirqd/409:50:504
3009699109,1cyclictest0-21swapper/411:24:514
3009699108,1cyclictest668-21systemd-logind11:21:044
3009699108,1cyclictest49-21ksoftirqd/412:19:304
3009699100,9cyclictest13781-21sshd11:45:304
3009699100,10cyclictest0-21swapper/410:19:064
3008899109,1cyclictest41-21ksoftirqd/307:49:493
3008899109,1cyclictest26960-21kworker/3:106:55:373
3008899109,1cyclictest14219-21kworker/3:008:30:473
30088991010,0cyclictest21306-21kworker/3:107:37:033
3008099109,1cyclictest0-21swapper/212:07:262
3008099109,1cyclictest0-21swapper/208:14:532
3008099101,4cyclictest0-21swapper/212:00:552
3008099101,4cyclictest0-21swapper/212:00:552
3008099100,4cyclictest14199-21cp11:19:332
3008099100,10cyclictest29562-21kworker/2:210:00:112
3007299109,1cyclictest0-21swapper/111:16:471
3007299108,1cyclictest861-21gdbus12:21:441
3007299107,1cyclictest4733-21kworker/1:109:40:111
3007299107,1cyclictest13125-21kworker/1:011:58:551
3007299103,6cyclictest17074-21sshd11:06:091
3007299103,6cyclictest17074-21sshd11:06:091
3007299101,4cyclictest0-21swapper/109:49:211
3006499108,1cyclictest9-21ksoftirqd/010:47:260
3006499103,6cyclictest31565-21sshd11:03:500
3006499101,4cyclictest28516-21sshd11:34:490
3006499101,4cyclictest26092-21sshd10:39:530
3006499101,4cyclictest0-21swapper/010:24:140
3006499100,4cyclictest7538-21grep10:37:140
3006499100,4cyclictest4581-21sshd12:04:340
3006499100,4cyclictest4581-21sshd12:04:340
3006499100,4cyclictest32463-21systemd-cgroups09:36:130
3006499100,4cyclictest32043-21sshd09:54:480
168732100,1sleep0711-21dbus-daemon09:15:150
301109999,0cyclictest0-21swapper/611:25:496
301109999,0cyclictest0-21swapper/610:52:056
301109999,0cyclictest0-21swapper/609:54:336
301109997,1cyclictest65-21ksoftirqd/609:07:376
301109990,9cyclictest0-21swapper/612:05:436
301109990,9cyclictest0-21swapper/611:54:596
301109990,9cyclictest0-21swapper/611:41:166
301109990,0cyclictest0-21swapper/610:12:126
300969998,1cyclictest49-21ksoftirqd/410:29:254
300969997,1cyclictest0-21swapper/411:29:534
300969994,3cyclictest5451-21sshd11:52:354
300969993,3cyclictest711-21dbus-daemon09:04:304
300969993,3cyclictest30702-21bash11:08:114
300969993,3cyclictest30702-21bash11:08:114
300969993,3cyclictest2374-21grepconf.sh10:54:584
300969992,4cyclictest27971-21cp09:40:054
300969992,3cyclictest861-21gdbus11:11:564
300889999,0cyclictest32536-21kworker/3:207:32:113
300889999,0cyclictest32536-21kworker/3:207:23:113
300889998,1cyclictest41-21ksoftirqd/308:24:493
300889998,1cyclictest21306-21kworker/3:108:09:593
300809999,0cyclictest0-21swapper/211:33:512
300809999,0cyclictest0-21swapper/211:07:122
300809999,0cyclictest0-21swapper/211:07:122
300809999,0cyclictest0-21swapper/209:12:362
300809994,4cyclictest14182-21rm09:05:452
300809993,6cyclictest0-21swapper/209:26:022
300809990,9cyclictest0-21swapper/209:55:382
300809990,8cyclictest27070-21kworker/2:010:44:362
300809990,4cyclictest25447-21sshd10:39:512
300809990,4cyclictest0-21swapper/212:15:402
300809990,4cyclictest0-21swapper/208:59:572
300809990,0cyclictest0-21swapper/211:13:022
300729997,1cyclictest8006-21id10:46:401
300729997,1cyclictest25-21ksoftirqd/112:00:531
300729997,1cyclictest25-21ksoftirqd/112:00:531
300729997,1cyclictest25-21ksoftirqd/110:36:501
300729997,1cyclictest25-21ksoftirqd/110:29:471
300729996,1cyclictest6766-21kworker/1:210:40:291
300729993,6cyclictest0-21swapper/109:31:351
300729992,6cyclictest19710-21sshd10:11:141
300729992,6cyclictest1495-21sshd09:54:551
300729990,9cyclictest0-21swapper/111:10:481
300649999,0cyclictest0-21swapper/011:07:460
300649999,0cyclictest0-21swapper/011:07:460
300649999,0cyclictest0-21swapper/010:25:120
300649999,0cyclictest0-21swapper/009:04:190
300649999,0cyclictest0-21swapper/008:57:320
300649992,1cyclictest1-21systemd10:14:540
300649991,3cyclictest29441-21sshd09:31:090
300649990,9cyclictest0-21swapper/012:17:270
300649990,9cyclictest0-21swapper/011:47:200
300649990,9cyclictest0-21swapper/007:34:560
300649990,4cyclictest28701-21sshd12:23:360
300649990,4cyclictest0-21swapper/012:09:360
300649990,0cyclictest0-21swapper/011:22:440
21453297,1sleep449-21ksoftirqd/408:57:264
13802290,0sleep10-21swapper/111:01:121
11827290,0sleep60-21swapper/611:45:146
301109987,1cyclictest0-21swapper/610:41:416
301109986,1cyclictest0-21swapper/610:36:456
301109984,3cyclictest9457-21bash11:18:496
301109983,5cyclictest1334-21sshd09:36:256
301109983,4cyclictest32132-21grepconf.sh09:40:446
301109983,2cyclictest0-21swapper/608:16:366
301109983,2cyclictest0-21swapper/608:14:276
301109982,5cyclictest2248-21grepconf.sh09:22:536
301109981,6cyclictest0-21swapper/611:34:436
301109981,6cyclictest0-21swapper/609:24:386
301109980,8cyclictest0-21swapper/611:14:096
301109980,7cyclictest861-21gdbus11:34:116
301109980,7cyclictest13005-21sshd10:28:546
301109980,4cyclictest2157-21cp10:08:546
300969987,1cyclictest49-21ksoftirqd/410:59:524
300969987,1cyclictest49-21ksoftirqd/409:59:444
300969987,1cyclictest0-21swapper/410:44:574
300969986,1cyclictest49-21ksoftirqd/411:15:154
300969986,1cyclictest49-21ksoftirqd/410:39:234
300969986,1cyclictest49-21ksoftirqd/410:30:364
300969986,1cyclictest49-21ksoftirqd/409:56:124
300969986,1cyclictest49-21ksoftirqd/409:45:384
300969985,2cyclictest25516-21sshd09:35:054
300969985,2cyclictest16538-21grep11:37:414
300969984,3cyclictest24236-21bash12:02:584
300969984,3cyclictest24236-21bash12:02:584
300969983,3cyclictest8169-21/usr/sbin/munin09:04:554
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional