You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 07:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Wed Dec 03, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
708226156,2sleep50-21swapper/518:52:565
76042530,0sleep40-21swapper/423:44:524
171332510,0sleep30-21swapper/323:10:233
710425025,9sleep00-21swapper/018:53:150
77262490,0sleep70-21swapper/700:13:387
256332490,1sleep6641ktimersoftd/623:52:396
689524825,9sleep60-21swapper/618:50:256
111822480,0sleep30-21swapper/321:57:353
698424635,8sleep20-21swapper/218:51:332
715124533,8sleep70-21swapper/718:53:557
223242450,0sleep30-21swapper/323:42:353
709824433,8sleep30-21swapper/318:53:103
173302430,1sleep517333-21proc_pri21:04:195
141172430,0sleep20-21swapper/221:21:542
40312420,0sleep40-21swapper/422:14:264
65702410,0sleep50-21swapper/523:04:245
41542410,0sleep50-21swapper/500:13:045
39922410,0sleep13991-21idleruntime23:04:121
242482410,0sleep60-21swapper/600:20:346
186892410,0sleep10-21swapper/120:09:141
34422400,0sleep00-21swapper/023:54:090
140382400,2sleep1732199cyclictest23:41:201
122672400,0sleep00-21swapper/023:36:390
695723927,8sleep10-21swapper/118:51:161
213232390,0sleep00-21swapper/000:10:430
686223826,9sleep40-21swapper/418:49:594
260482350,0sleep40-21swapper/421:50:314
7336992613,6cyclictest57-21ksoftirqd/522:49:035
7336992318,5cyclictest57-21ksoftirqd/523:56:015
7336992318,5cyclictest57-21ksoftirqd/522:39:075
7336992115,1cyclictest57-21ksoftirqd/521:20:455
7336992110,11cyclictest57-21ksoftirqd/523:37:575
733899208,5cyclictest27872-21sshd23:11:577
733699208,7cyclictest57-21ksoftirqd/521:39:395
733699208,7cyclictest57-21ksoftirqd/521:39:395
7336992015,5cyclictest57-21ksoftirqd/500:16:315
7336992014,6cyclictest57-21ksoftirqd/523:51:155
7336992014,6cyclictest57-21ksoftirqd/522:09:055
7336992010,9cyclictest57-21ksoftirqd/521:15:335
733899198,5cyclictest0-21swapper/721:17:277
733899198,0cyclictest0-21swapper/722:04:157
733899192,11cyclictest0-21swapper/723:21:557
7336991913,1cyclictest57-21ksoftirqd/500:07:235
7336991912,6cyclictest57-21ksoftirqd/523:15:475
733699191,12cyclictest57-21ksoftirqd/519:48:185
7338991812,5cyclictest9554-21bash22:47:037
733699189,9cyclictest57-21ksoftirqd/520:30:225
733699187,0cyclictest0-21swapper/519:19:545
733699184,11cyclictest57-21ksoftirqd/523:39:175
7336991814,0cyclictest0-21swapper/519:08:065
7336991813,3cyclictest57-21ksoftirqd/522:19:475
7336991813,3cyclictest57-21ksoftirqd/520:54:045
7330991812,4cyclictest49-21ksoftirqd/421:48:334
733099181,2cyclictest49-21ksoftirqd/422:41:054
7327991814,2cyclictest10170-21latency_hist22:28:593
733899177,5cyclictest73-21ksoftirqd/723:51:397
733899176,5cyclictest0-21swapper/722:34:337
733899176,0cyclictest0-21swapper/721:00:277
7338991713,4cyclictest21788-21httpd22:18:337
733899170,16cyclictest25570-21sshd21:41:317
733899170,16cyclictest25570-21sshd21:41:317
733699178,9cyclictest57-21ksoftirqd/521:50:575
733699176,0cyclictest16517-21kworker/5:022:01:595
733699176,0cyclictest0-21swapper/520:42:385
7336991713,1cyclictest57-21ksoftirqd/521:27:065
7336991712,5cyclictest57-21ksoftirqd/523:09:255
7336991712,5cyclictest57-21ksoftirqd/521:02:455
7336991712,5cyclictest57-21ksoftirqd/519:00:005
7336991712,1cyclictest57-21ksoftirqd/522:55:415
7336991711,1cyclictest57-21ksoftirqd/523:23:155
733699171,10cyclictest57-21ksoftirqd/522:44:495
733099172,3cyclictest49-21ksoftirqd/400:23:074
7330991714,3cyclictest49-21ksoftirqd/422:04:154
7330991714,3cyclictest49-21ksoftirqd/421:19:184
7330991711,4cyclictest49-21ksoftirqd/422:58:144
7326991715,1cyclictest27714-21sshd22:35:502
733899165,5cyclictest28103-21nfsd22:30:537
733899165,5cyclictest0-21swapper/720:09:547
733899165,5cyclictest0-21swapper/719:48:107
733799161,2cyclictest0-21swapper/600:09:356
733799161,1cyclictest0-21swapper/623:35:536
733799161,1cyclictest0-21swapper/622:07:486
733799160,15cyclictest28700-21sshd22:54:146
733699168,8cyclictest57-21ksoftirqd/521:47:415
733699167,4cyclictest57-21ksoftirqd/519:49:205
733699165,6cyclictest57-21ksoftirqd/521:58:135
733699165,11cyclictest0-21swapper/520:46:295
733699165,11cyclictest0-21swapper/520:46:295
7336991615,1cyclictest57-21ksoftirqd/521:34:195
7336991614,1cyclictest57-21ksoftirqd/521:33:065
7336991612,0cyclictest12741-21kworker/5:321:09:075
7336991611,5cyclictest57-21ksoftirqd/523:28:555
7336991611,5cyclictest57-21ksoftirqd/522:38:515
7336991611,5cyclictest57-21ksoftirqd/522:14:275
7336991611,5cyclictest57-21ksoftirqd/520:51:195
7336991611,5cyclictest57-21ksoftirqd/500:21:575
7336991611,5cyclictest57-21ksoftirqd/500:01:075
7336991611,1cyclictest57-21ksoftirqd/523:47:495
7336991611,0cyclictest57-21ksoftirqd/523:00:455
7336991611,0cyclictest57-21ksoftirqd/523:00:455
7336991610,6cyclictest57-21ksoftirqd/522:07:035
733099167,2cyclictest49-21ksoftirqd/423:22:314
733099167,2cyclictest49-21ksoftirqd/422:46:164
733099167,1cyclictest49-21ksoftirqd/423:24:464
733099166,2cyclictest49-21ksoftirqd/400:11:154
733099166,1cyclictest49-21ksoftirqd/421:37:004
7330991614,1cyclictest49-21ksoftirqd/421:11:424
7330991614,1cyclictest49-21ksoftirqd/400:14:114
7330991612,1cyclictest49-21ksoftirqd/423:10:444
7330991611,3cyclictest49-21ksoftirqd/423:37:404
7330991611,3cyclictest49-21ksoftirqd/421:00:074
7330991611,3cyclictest49-21ksoftirqd/420:56:064
732199160,15cyclictest16456-21sshd23:01:271
732199160,15cyclictest16456-21sshd23:01:271
733899154,5cyclictest0-21swapper/722:56:277
733899154,5cyclictest0-21swapper/722:09:137
733899154,0cyclictest0-21swapper/720:44:137
733899154,0cyclictest0-21swapper/720:44:137
733899154,0cyclictest0-21swapper/719:02:187
733899153,11cyclictest22361-21bash23:38:077
733899150,9cyclictest0-21swapper/723:40:497
733899150,9cyclictest0-21swapper/721:25:477
733899150,9cyclictest0-21swapper/720:24:327
733899150,9cyclictest0-21swapper/719:59:287
733899150,9cyclictest0-21swapper/719:31:427
733899150,9cyclictest0-21swapper/719:25:187
733699159,1cyclictest57-21ksoftirqd/519:11:345
733699155,5cyclictest57-21ksoftirqd/523:29:355
7336991513,1cyclictest57-21ksoftirqd/522:31:035
7336991510,5cyclictest57-21ksoftirqd/520:26:085
7336991510,5cyclictest57-21ksoftirqd/520:08:345
733099156,2cyclictest49-21ksoftirqd/422:10:534
733099156,2cyclictest49-21ksoftirqd/421:54:234
733099155,3cyclictest49-21ksoftirqd/423:40:184
733099154,3cyclictest49-21ksoftirqd/423:03:294
733099154,3cyclictest49-21ksoftirqd/423:03:294
733099153,2cyclictest49-21ksoftirqd/421:31:424
733099153,1cyclictest49-21ksoftirqd/423:04:154
733099152,1cyclictest49-21ksoftirqd/422:50:334
7330991514,1cyclictest49-21ksoftirqd/422:30:364
7330991514,1cyclictest49-21ksoftirqd/421:14:144
7330991513,2cyclictest49-21ksoftirqd/421:59:384
7330991513,1cyclictest49-21ksoftirqd/423:14:024
7330991513,1cyclictest49-21ksoftirqd/421:24:164
7330991513,1cyclictest49-21ksoftirqd/421:04:134
733099151,1cyclictest131rcu_sched00:04:354
733099150,14cyclictest16981-21kworker/4:223:32:574
733899149,1cyclictest13480-21sshd23:32:237
733899148,6cyclictest73-21ksoftirqd/722:20:277
733899148,5cyclictest12188-21bash22:42:557
733899148,5cyclictest0-21swapper/722:49:257
733899148,1cyclictest73-21ksoftirqd/700:14:057
733899148,1cyclictest0-21swapper/722:03:017
733899143,5cyclictest0-21swapper/720:23:467
733899143,0cyclictest0-21swapper/720:50:477
733899140,8cyclictest0-21swapper/720:38:527
733899140,8cyclictest0-21swapper/719:54:367
733899140,8cyclictest0-21swapper/719:43:407
733899140,8cyclictest0-21swapper/719:35:567
733899140,8cyclictest0-21swapper/719:35:567
733899140,8cyclictest0-21swapper/719:09:327
7337991412,0cyclictest85350irq/25-em1-rx-021:06:116
733799141,6cyclictest30387-21sshd21:51:116
733799141,2cyclictest0-21swapper/622:45:046
733799141,1cyclictest0-21swapper/623:44:126
733799141,1cyclictest0-21swapper/622:25:366
733799141,1cyclictest0-21swapper/621:24:126
733799141,1cyclictest0-21swapper/600:00:116
733799140,5cyclictest6399-21sshd23:31:206
733699149,5cyclictest57-21ksoftirqd/520:17:225
733699148,1cyclictest57-21ksoftirqd/519:42:145
733699142,6cyclictest0-21swapper/518:55:105
7336991410,0cyclictest9840-21kworker/5:019:57:005
733699140,8cyclictest0-21swapper/519:31:505
733099149,3cyclictest49-21ksoftirqd/423:53:384
733099145,3cyclictest49-21ksoftirqd/422:35:144
7330991412,1cyclictest49-21ksoftirqd/423:59:104
7330991412,1cyclictest49-21ksoftirqd/422:24:004
7330991412,1cyclictest49-21ksoftirqd/421:39:144
7330991412,1cyclictest49-21ksoftirqd/421:39:144
7330991411,2cyclictest49-21ksoftirqd/423:54:134
7330991411,2cyclictest49-21ksoftirqd/422:19:174
732199148,4cyclictest1647-21kworker/1:323:34:031
7321991413,1cyclictest25-21ksoftirqd/100:08:351
7321991410,3cyclictest6725-21bash21:52:281
199862140,2sleep1732199cyclictest21:27:141
733899138,5cyclictest0-21swapper/700:04:117
733899138,5cyclictest0-21swapper/700:01:477
733899138,1cyclictest1-21systemd22:24:497
733899138,0cyclictest0-21swapper/723:57:457
733899138,0cyclictest0-21swapper/721:48:477
733899138,0cyclictest0-21swapper/718:57:547
733899132,5cyclictest0-21swapper/721:55:017
733899131,6cyclictest0-21swapper/721:09:437
733899131,6cyclictest0-21swapper/719:14:087
733799139,3cyclictest13665-21bash21:21:516
733799131,3cyclictest0-21swapper/622:40:516
733799131,2cyclictest0-21swapper/623:43:156
733799131,1cyclictest0-21swapper/622:59:396
733799131,1cyclictest0-21swapper/622:59:396
733799131,1cyclictest0-21swapper/622:09:166
733799131,1cyclictest0-21swapper/622:01:226
733799131,1cyclictest0-21swapper/621:00:246
733799130,5cyclictest668-21systemd-logind20:56:216
733799130,5cyclictest538-21bash00:17:126
733799130,5cyclictest15890-21bash23:10:126
733699138,5cyclictest0-21swapper/520:09:305
733699138,0cyclictest0-21swapper/520:23:345
733699138,0cyclictest0-21swapper/519:59:445
733699137,1cyclictest57-21ksoftirqd/522:28:415
733699132,5cyclictest0-21swapper/520:37:355
733699131,0cyclictest0-21swapper/519:24:545
732799130,12cyclictest0-21swapper/322:54:263
44132130,0sleep04410-21sshd22:41:450
240252130,0sleep70-21swapper/721:32:167
18212130,0sleep1231rcuc/100:17:241
733899127,5cyclictest0-21swapper/723:14:137
733899127,5cyclictest0-21swapper/720:41:547
733899127,5cyclictest0-21swapper/720:14:087
733899127,0cyclictest0-21swapper/721:22:577
733899127,0cyclictest0-21swapper/719:50:067
733899127,0cyclictest0-21swapper/719:06:347
733899126,5cyclictest668-21systemd-logind21:34:217
733899126,5cyclictest17946-21unixbench-2d21:04:217
7338991211,1cyclictest73-21ksoftirqd/723:00:257
7338991211,1cyclictest73-21ksoftirqd/723:00:257
733899121,6cyclictest0-21swapper/723:44:357
733899121,5cyclictest0-21swapper/720:55:497
733899121,0cyclictest0-21swapper/720:29:307
733899120,6cyclictest0-21swapper/719:19:147
733899120,6cyclictest0-21swapper/700:23:317
733899120,0cyclictest0-21swapper/723:06:117
733899120,0cyclictest0-21swapper/720:05:227
733799122,4cyclictest9390-21bash00:04:166
733799121,3cyclictest0-21swapper/621:11:176
733799121,2cyclictest0-21swapper/621:47:256
733799121,1cyclictest0-21swapper/623:07:446
733799120,5cyclictest4400-21bash21:38:406
733799120,5cyclictest19037-21sshd21:31:336
733799120,11cyclictest24834-21sshd23:15:536
733699127,5cyclictest0-21swapper/519:37:025
733699127,5cyclictest0-21swapper/519:37:025
7330991211,1cyclictest49-21ksoftirqd/420:34:184
32952120,1sleep3391rcuc/300:08:133
250842120,0sleep30-21swapper/320:19:173
733899110,0cyclictest0-21swapper/721:49:277
733799112,1cyclictest0-21swapper/623:23:086
733799111,1cyclictest0-21swapper/622:49:456
733799111,1cyclictest0-21swapper/621:39:326
733799111,1cyclictest0-21swapper/621:39:326
733799110,5cyclictest10656-21sshd23:27:296
733799110,3cyclictest711-21dbus-daemon21:54:576
733799110,10cyclictest27709-21sshd22:31:186
733799110,10cyclictest0-21swapper/622:35:376
733699116,5cyclictest9840-21kworker/5:019:14:305
733099119,1cyclictest49-21ksoftirqd/420:53:594
733099119,1cyclictest49-21ksoftirqd/420:19:134
733099119,1cyclictest49-21ksoftirqd/420:09:134
7330991110,1cyclictest49-21ksoftirqd/420:39:194
7330991110,1cyclictest49-21ksoftirqd/420:14:124
732799118,1cyclictest11214-21kworker/u16:223:21:583
732799110,0cyclictest0-21swapper/320:09:163
732699110,1cyclictest0-21swapper/223:57:352
732199110,1cyclictest26885-21cpuspeed_turbos20:24:101
732199110,1cyclictest0-21swapper/123:12:521
730999113,4cyclictest0-21swapper/022:53:410
730999111,4cyclictest0-21swapper/023:08:360
730999110,2cyclictest19874-21sshd22:44:000
730999110,11cyclictest0-21swapper/023:19:190
730999110,10cyclictest0-21swapper/022:09:570
50982110,0sleep20-21swapper/221:02:482
733899103,6cyclictest27827-21grepconf.sh23:25:137
733799101,1cyclictest0-21swapper/623:54:346
733799101,1cyclictest0-21swapper/621:15:036
733799101,1cyclictest0-21swapper/619:54:206
733099109,1cyclictest49-21ksoftirqd/420:29:164
733099109,1cyclictest49-21ksoftirqd/419:34:154
733099109,1cyclictest49-21ksoftirqd/419:34:154
733099109,1cyclictest49-21ksoftirqd/419:14:184
733099108,1cyclictest49-21ksoftirqd/420:04:134
733099108,1cyclictest49-21ksoftirqd/419:49:004
733099104,6cyclictest49-21ksoftirqd/419:51:384
732799109,1cyclictest0-21swapper/320:58:103
732699107,2cyclictest31771-21bash22:18:352
732699102,3cyclictest0-21swapper/223:52:252
732699102,2cyclictest1-21systemd23:40:152
732699100,10cyclictest0-21swapper/223:01:182
732699100,10cyclictest0-21swapper/223:01:182
732699100,1cyclictest0-21swapper/222:24:252
732199109,1cyclictest0-21swapper/122:32:581
732199108,1cyclictest0-21swapper/121:37:291
732199106,3cyclictest27373-21kworker/1:021:43:591
732199106,3cyclictest27373-21kworker/1:021:43:591
732199103,6cyclictest21282-21sshd00:01:181
732199103,1cyclictest0-21swapper/123:15:471
732199100,10cyclictest0-21swapper/123:45:111
732199100,10cyclictest0-21swapper/122:10:071
732199100,10cyclictest0-21swapper/122:06:481
732199100,1cyclictest0-21swapper/121:54:491
730999108,1cyclictest711-21dbus-daemon23:45:200
730999108,1cyclictest21970-21bash20:55:230
730999103,6cyclictest25034-21sshd22:17:350
730999103,6cyclictest20975-21sshd22:03:290
730999103,6cyclictest20644-21sshd21:40:490
730999103,6cyclictest20644-21sshd21:40:490
730999103,6cyclictest13160-21sshd22:29:150
730999100,4cyclictest0-21swapper/023:02:430
730999100,4cyclictest0-21swapper/023:02:430
730999100,1cyclictest9902-21sshd21:30:130
73379992,1cyclictest0-21swapper/619:34:136
73379992,1cyclictest0-21swapper/619:34:136
73379991,1cyclictest0-21swapper/622:19:076
73379991,1cyclictest0-21swapper/620:14:166
73379991,1cyclictest0-21swapper/618:54:126
73379991,0cyclictest0-21swapper/622:14:386
73379990,1cyclictest0-21swapper/619:44:176
73309998,1cyclictest49-21ksoftirqd/419:59:004
73309998,1cyclictest49-21ksoftirqd/419:28:594
73309998,1cyclictest49-21ksoftirqd/419:19:424
73309997,1cyclictest49-21ksoftirqd/420:24:174
73309997,1cyclictest49-21ksoftirqd/419:29:174
73309994,5cyclictest49-21ksoftirqd/418:58:244
73309993,6cyclictest49-21ksoftirqd/419:08:424
73279999,0cyclictest0-21swapper/322:14:183
73279999,0cyclictest0-21swapper/321:20:233
73279997,2cyclictest711-21dbus-daemon23:18:133
73279997,1cyclictest41-21ksoftirqd/323:24:163
73279997,1cyclictest41-21ksoftirqd/321:34:213
73279997,1cyclictest41-21ksoftirqd/321:33:593
73279994,3cyclictest5959-21nscd23:33:113
73279994,3cyclictest12008-21sshd23:36:363
73279994,3cyclictest0-21swapper/300:02:273
73279994,1cyclictest689-21nscd22:40:073
73279993,1cyclictest28634-21sshd22:49:303
73279990,9cyclictest0-21swapper/323:55:503
73279990,1cyclictest23300-21sshd22:44:183
73279990,0cyclictest0-21swapper/321:27:293
73269999,0cyclictest0-21swapper/223:27:382
73269999,0cyclictest0-21swapper/223:05:522
73269999,0cyclictest0-21swapper/200:05:372
73269998,1cyclictest0-21swapper/222:02:012
73269997,1cyclictest1-21systemd22:55:282
73269992,6cyclictest0-21swapper/221:42:152
73269992,6cyclictest0-21swapper/221:42:152
73269990,9cyclictest0-21swapper/222:44:162
73269990,9cyclictest0-21swapper/221:51:372
73269990,4cyclictest0-21swapper/222:53:272
73219999,0cyclictest0-21swapper/121:33:421
73219998,1cyclictest0-21swapper/122:26:131
73219996,3cyclictest27595-21kworker/1:121:45:411
73219990,9cyclictest0-21swapper/121:07:131
73099999,0cyclictest0-21swapper/022:06:380
73099997,1cyclictest9-21ksoftirqd/021:06:580
73099995,1cyclictest1271-21sshd21:20:010
73099990,9cyclictest0-21swapper/023:18:030
73099990,9cyclictest0-21swapper/021:17:080
73099990,9cyclictest0-21swapper/021:11:270
73099990,4cyclictest9606-21sshd22:19:440
73099990,4cyclictest29249-21sshd21:55:280
19189290,0sleep20-21swapper/221:58:462
73379981,1cyclictest0-21swapper/620:29:136
73379981,1cyclictest0-21swapper/620:00:296
73379981,1cyclictest0-21swapper/619:50:436
73379981,1cyclictest0-21swapper/619:14:116
73379981,1cyclictest0-21swapper/619:04:466
73379981,0cyclictest0-21swapper/620:23:596
73379980,1cyclictest0-21swapper/620:34:176
73379980,1cyclictest0-21swapper/619:28:596
73379980,1cyclictest0-21swapper/619:19:186
73309987,1cyclictest49-21ksoftirqd/420:44:184
73309987,1cyclictest49-21ksoftirqd/420:44:184
73309986,1cyclictest49-21ksoftirqd/419:59:154
73309986,1cyclictest49-21ksoftirqd/419:39:114
73279988,0cyclictest0-21swapper/323:44:483
73279988,0cyclictest0-21swapper/322:59:333
73279988,0cyclictest0-21swapper/322:59:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional