You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 05:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Mon Dec 01, 2025 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14772910,2sleep30-21swapper/318:54:033
302802860,2sleep2164199cyclictest22:19:022
132025025,8sleep40-21swapper/418:52:054
233012490,0sleep40-21swapper/422:18:004
223232490,0sleep60-21swapper/600:05:586
156372490,2sleep1163399cyclictest00:04:571
113724844,1sleep00-21swapper/018:49:370
209112460,0sleep10-21swapper/122:31:031
118812460,0sleep50-21swapper/521:22:275
28442450,0sleep10-21swapper/123:40:301
205872430,0sleep10-21swapper/121:32:401
282872420,1sleep341-21ksoftirqd/323:03:483
128624229,9sleep60-21swapper/618:51:356
8772410,0sleep30-21swapper/323:58:423
86362410,0sleep60-21swapper/623:50:366
241072400,0sleep30-21swapper/322:58:443
7022390,0sleep00-21swapper/023:49:260
194792390,0sleep10-21swapper/121:45:541
190402390,0sleep70-21swapper/700:10:007
128523928,8sleep50-21swapper/518:51:345
126323927,9sleep10-21swapper/118:51:141
118923927,9sleep20-21swapper/218:50:192
117223926,9sleep70-21swapper/718:50:087
230512380,0sleep30-21swapper/322:54:033
164802360,0sleep00-21swapper/022:53:040
166499214,5cyclictest0-21swapper/520:42:485
168099198,5cyclictest0-21swapper/720:56:447
168099198,0cyclictest0-21swapper/720:45:507
168099198,0cyclictest0-21swapper/720:31:277
168099197,2cyclictest7948-21sshd22:24:367
166499198,5cyclictest0-21swapper/523:40:485
166499198,5cyclictest0-21swapper/523:30:265
166499198,5cyclictest0-21swapper/521:51:585
166499198,5cyclictest0-21swapper/519:31:175
166499198,0cyclictest0-21swapper/519:02:595
166499197,12cyclictest0-21swapper/522:30:525
166499197,12cyclictest0-21swapper/519:40:455
1641991915,3cyclictest33-21ksoftirqd/200:19:032
272052180,0sleep60-21swapper/622:36:286
168099187,5cyclictest0-21swapper/723:29:287
168099187,5cyclictest0-21swapper/721:39:067
168099187,5cyclictest0-21swapper/719:59:457
168099187,1cyclictest0-21swapper/721:45:427
168099187,0cyclictest0-21swapper/722:03:467
1680991813,5cyclictest0-21swapper/723:46:067
168099181,5cyclictest0-21swapper/723:21:187
168099180,6cyclictest0-21swapper/722:07:007
168099180,6cyclictest0-21swapper/720:43:407
168099180,17cyclictest3632-21sshd00:07:557
166499188,0cyclictest0-21swapper/521:16:145
166499187,5cyclictest0-21swapper/523:10:325
166499187,5cyclictest0-21swapper/521:59:345
166499187,5cyclictest0-21swapper/520:47:545
166499187,5cyclictest0-21swapper/519:23:255
166499187,0cyclictest0-21swapper/519:14:175
166499187,0cyclictest0-21swapper/519:09:495
166499187,0cyclictest0-21swapper/500:02:125
164199188,1cyclictest33-21ksoftirqd/221:48:022
97972170,0sleep00-21swapper/000:04:160
168099178,5cyclictest0-21swapper/721:10:167
168099176,5cyclictest0-21swapper/722:32:327
168099176,5cyclictest0-21swapper/700:15:347
168099176,0cyclictest0-21swapper/723:55:227
168099171,6cyclictest0-21swapper/722:09:267
168099170,15cyclictest26271-21sshd23:43:587
166499177,0cyclictest0-21swapper/519:37:195
166499176,5cyclictest0-21swapper/521:56:305
166499176,5cyclictest0-21swapper/521:37:245
166499176,5cyclictest0-21swapper/519:59:055
166499176,0cyclictest0-21swapper/521:42:125
166499176,0cyclictest0-21swapper/521:31:205
166499176,0cyclictest0-21swapper/520:57:425
166499176,0cyclictest0-21swapper/520:38:405
1664991712,5cyclictest0-21swapper/521:26:065
164899170,16cyclictest99-21kswapd021:47:543
1641991713,3cyclictest33-21ksoftirqd/221:20:582
1641991712,3cyclictest33-21ksoftirqd/221:27:002
187652160,0sleep50-21swapper/500:14:465
168099165,5cyclictest0-21swapper/723:59:107
168099165,5cyclictest0-21swapper/722:34:447
168099165,5cyclictest0-21swapper/720:10:157
168099165,0cyclictest0-21swapper/719:17:537
168099160,13cyclictest13907-21sshd00:24:027
168099160,12cyclictest0-21swapper/722:45:327
166499168,5cyclictest0-21swapper/521:45:125
166499168,5cyclictest0-21swapper/519:48:015
166499168,5cyclictest0-21swapper/519:28:575
166499168,2cyclictest0-21swapper/523:55:185
166499168,2cyclictest0-21swapper/520:19:215
166499165,11cyclictest0-21swapper/523:53:285
166499165,0cyclictest0-21swapper/523:28:585
166499165,0cyclictest0-21swapper/523:17:185
166499165,0cyclictest0-21swapper/521:10:365
166499165,0cyclictest0-21swapper/500:21:545
166499164,11cyclictest2140-21sshd22:05:585
1641991614,1cyclictest33-21ksoftirqd/221:29:242
1641991613,3cyclictest33-21ksoftirqd/223:18:332
1641991611,3cyclictest33-21ksoftirqd/223:41:372
80972150,0sleep78096-21cut21:04:037
168099155,0cyclictest0-21swapper/723:14:387
168099154,5cyclictest0-21swapper/721:29:187
168099152,12cyclictest9683-21bash22:52:027
1680991510,0cyclictest0-21swapper/719:49:597
168099150,9cyclictest0-21swapper/719:48:097
168099150,9cyclictest0-21swapper/719:24:337
168099150,9cyclictest0-21swapper/719:09:077
168099150,4cyclictest0-21swapper/720:34:317
168099150,11cyclictest23626-21grep21:28:407
166499158,0cyclictest0-21swapper/519:51:535
166499155,0cyclictest0-21swapper/519:59:035
166499154,5cyclictest0-21swapper/522:46:105
166499154,0cyclictest0-21swapper/522:53:305
166499154,0cyclictest0-21swapper/522:35:285
166499154,0cyclictest0-21swapper/520:54:025
166499154,0cyclictest0-21swapper/520:26:315
166499154,0cyclictest0-21swapper/520:26:315
166499154,0cyclictest0-21swapper/520:12:155
166499154,0cyclictest0-21swapper/520:05:435
166499154,0cyclictest0-21swapper/519:07:595
166499154,0cyclictest0-21swapper/518:54:495
166499150,9cyclictest0-21swapper/522:44:005
166499150,14cyclictest711-21dbus-daemon22:21:345
164199153,1cyclictest33-21ksoftirqd/222:26:182
1641991514,1cyclictest33-21ksoftirqd/200:09:252
1641991513,1cyclictest33-21ksoftirqd/221:02:022
1641991511,3cyclictest33-21ksoftirqd/221:08:132
1641991511,1cyclictest33-21ksoftirqd/200:06:502
89572140,0sleep20-21swapper/222:47:292
168099148,6cyclictest0-21swapper/723:13:287
168099148,6cyclictest0-21swapper/723:01:027
168099143,5cyclictest0-21swapper/723:34:187
168099142,6cyclictest0-21swapper/719:35:037
168099142,12cyclictest15847-21sshd21:04:547
168099142,11cyclictest2854-21sshd21:21:087
168099142,0cyclictest0-21swapper/720:50:527
168099140,9cyclictest0-21swapper/723:49:247
168099140,9cyclictest0-21swapper/723:25:347
168099140,9cyclictest0-21swapper/723:08:227
168099140,9cyclictest0-21swapper/723:08:227
166499143,11cyclictest0-21swapper/522:13:185
166499142,6cyclictest0-21swapper/500:13:045
166499140,8cyclictest0-21swapper/523:47:225
166499140,2cyclictest0-21swapper/521:05:065
164199149,3cyclictest33-21ksoftirqd/222:00:472
164199145,1cyclictest33-21ksoftirqd/222:08:162
164199144,2cyclictest33-21ksoftirqd/223:06:132
164199144,2cyclictest33-21ksoftirqd/223:06:132
1641991413,1cyclictest33-21ksoftirqd/222:42:042
1641991412,1cyclictest33-21ksoftirqd/223:19:042
1641991412,1cyclictest33-21ksoftirqd/223:13:042
1641991412,1cyclictest33-21ksoftirqd/222:57:292
1641991412,1cyclictest33-21ksoftirqd/221:34:572
1641991412,1cyclictest33-21ksoftirqd/221:09:482
1641991411,3cyclictest33-21ksoftirqd/223:47:432
1641991411,3cyclictest33-21ksoftirqd/222:20:212
1641991411,2cyclictest33-21ksoftirqd/221:43:532
1641991410,3cyclictest33-21ksoftirqd/220:57:102
168099138,5cyclictest0-21swapper/722:54:427
168099138,5cyclictest0-21swapper/722:41:347
168099138,5cyclictest0-21swapper/722:19:267
168099138,5cyclictest0-21swapper/722:14:107
168099138,5cyclictest0-21swapper/721:51:007
168099138,5cyclictest0-21swapper/721:16:367
168099138,5cyclictest0-21swapper/720:24:257
168099138,5cyclictest0-21swapper/720:24:257
168099138,5cyclictest0-21swapper/719:04:217
168099138,5cyclictest0-21swapper/718:55:197
168099138,0cyclictest0-21swapper/720:21:537
168099138,0cyclictest0-21swapper/720:05:397
168099138,0cyclictest0-21swapper/719:29:537
168099138,0cyclictest0-21swapper/719:19:577
168099138,0cyclictest0-21swapper/718:59:517
168099131,6cyclictest0-21swapper/719:54:497
168099131,6cyclictest0-21swapper/719:39:457
166499138,5cyclictest0-21swapper/523:35:225
166499138,5cyclictest0-21swapper/523:05:185
166499138,5cyclictest0-21swapper/523:05:185
166499138,5cyclictest0-21swapper/523:02:085
166499138,5cyclictest0-21swapper/522:24:205
166499138,5cyclictest0-21swapper/520:32:265
166499138,5cyclictest0-21swapper/500:07:085
166499138,0cyclictest0-21swapper/523:23:125
166499138,0cyclictest0-21swapper/520:14:195
166499138,0cyclictest0-21swapper/520:14:195
166499137,5cyclictest7164-21sshd22:15:365
166499132,0cyclictest0-21swapper/522:55:565
164199139,3cyclictest33-21ksoftirqd/223:00:302
164199139,3cyclictest33-21ksoftirqd/222:37:302
1641991311,1cyclictest33-21ksoftirqd/223:57:052
1641991311,1cyclictest33-21ksoftirqd/223:26:522
1641991311,1cyclictest33-21ksoftirqd/222:31:092
1641991311,1cyclictest33-21ksoftirqd/221:50:482
1641991311,1cyclictest33-21ksoftirqd/221:18:062
1641991311,1cyclictest33-21ksoftirqd/200:20:132
1641991310,1cyclictest33-21ksoftirqd/223:33:112
164199130,1cyclictest131rcu_sched23:38:032
164199130,1cyclictest131rcu_sched22:09:202
164199130,1cyclictest121rcu_preempt21:55:432
277252120,0sleep60-21swapper/600:21:046
168099128,0cyclictest0-21swapper/721:54:527
168099127,0cyclictest0-21swapper/720:15:197
168099127,0cyclictest0-21swapper/720:15:197
168099120,6cyclictest0-21swapper/721:35:507
167299122,4cyclictest10606-21idleruntime21:04:176
166499128,0cyclictest0-21swapper/520:59:145
1656991210,1cyclictest49-21ksoftirqd/421:33:474
164899120,1cyclictest0-21swapper/323:06:353
164899120,1cyclictest0-21swapper/323:06:353
1641991211,1cyclictest33-21ksoftirqd/223:49:052
1641991210,1cyclictest33-21ksoftirqd/220:29:162
164199121,3cyclictest33-21ksoftirqd/222:49:202
163399128,2cyclictest28908-21kworker/1:421:36:581
1633991210,1cyclictest0-21swapper/122:53:531
162499129,2cyclictest30158-21sshd21:02:330
162499122,4cyclictest19998-21sshd22:39:500
162499120,2cyclictest740-21gdbus23:27:170
162499120,11cyclictest20125-21grep21:05:330
150972120,0sleep60-21swapper/623:19:376
196242110,0sleep40-21swapper/422:39:474
167299119,1cyclictest7889-21sshd22:42:516
167299118,1cyclictest85350irq/25-em1-rx-022:56:356
167299112,4cyclictest4547-21sshd23:40:456
167299112,4cyclictest10450-21sshd21:13:206
167299111,4cyclictest0-21swapper/623:15:166
167299110,10cyclictest5777-21kworker/6:221:19:066
165699112,4cyclictest19806-21sshd23:24:474
165699110,11cyclictest0-21swapper/422:48:104
165699110,10cyclictest732-21NetworkManager22:03:334
165699110,10cyclictest25195-21sshd23:57:324
164899119,1cyclictest861-21gdbus00:13:233
164899112,5cyclictest19395-21sshd21:19:043
1648991110,1cyclictest0-21swapper/323:39:483
164899110,10cyclictest18963-21sshd23:24:403
164899110,1cyclictest13033-21sshd22:39:063
164199115,3cyclictest33-21ksoftirqd/223:59:422
1641991110,1cyclictest33-21ksoftirqd/220:49:042
1641991110,1cyclictest33-21ksoftirqd/220:19:032
1641991110,1cyclictest33-21ksoftirqd/220:19:032
1633991110,1cyclictest0-21swapper/100:21:451
162499119,1cyclictest9-21ksoftirqd/021:15:420
162499119,1cyclictest0-21swapper/021:41:390
162499110,10cyclictest25972-21cp23:48:240
162499110,1cyclictest0-21swapper/000:03:420
312112100,0sleep40-21swapper/421:52:054
255222100,0sleep30-21swapper/323:16:463
167299108,1cyclictest0-21swapper/623:01:366
167299103,6cyclictest24841-21sshd22:27:086
167299103,1cyclictest711-21dbus-daemon21:34:236
167299100,4cyclictest6401-21grepconf.sh21:57:416
167299100,4cyclictest29840-21sshd23:58:136
167299100,4cyclictest15388-21sshd21:45:176
167299100,4cyclictest14945-21sshd22:52:506
167299100,10cyclictest0-21swapper/620:09:176
165699109,1cyclictest0-21swapper/423:31:414
165699109,1cyclictest0-21swapper/423:01:054
165699103,6cyclictest0-21swapper/423:13:584
165699102,2cyclictest0-21swapper/421:38:514
165699101,4cyclictest11500-21sshd21:26:514
165699100,10cyclictest15897-21sshd22:21:224
165699100,1cyclictest22980-21sshd22:13:294
164899108,1cyclictest41-21ksoftirqd/322:14:403
164899108,1cyclictest41-21ksoftirqd/321:36:103
164899108,1cyclictest41-21ksoftirqd/321:05:403
164899108,1cyclictest22813-21sshd21:55:173
164899108,1cyclictest1601-21sshd21:16:303
164899107,2cyclictest41-21ksoftirqd/321:12:013
164199109,1cyclictest33-21ksoftirqd/220:24:172
164199109,1cyclictest33-21ksoftirqd/220:24:172
164199108,1cyclictest33-21ksoftirqd/220:09:162
164199108,1cyclictest33-21ksoftirqd/220:04:032
163399109,1cyclictest25-21ksoftirqd/123:26:211
163399109,1cyclictest0-21swapper/123:20:161
163399109,1cyclictest0-21swapper/122:00:011
163399107,2cyclictest7325-21sshd21:12:531
163399104,3cyclictest10595-21sshd21:40:061
163399100,1cyclictest0-21swapper/121:54:391
162499108,1cyclictest9-21ksoftirqd/023:16:520
162499105,1cyclictest732-21NetworkManager21:50:150
162499103,6cyclictest9297-21sshd22:07:000
162499103,6cyclictest19566-21sshd21:45:540
162499100,9cyclictest12089-21sshd23:37:240
162499100,10cyclictest15787-21kworker/0:021:26:240
162499100,10cyclictest0-21swapper/022:16:170
6538290,0sleep60-21swapper/600:13:116
28663297,1sleep449-21ksoftirqd/422:50:054
16729997,1cyclictest65-21ksoftirqd/620:09:036
16729993,2cyclictest27238-21grepconf.sh21:33:396
16729993,1cyclictest7319-21sshd22:02:176
16729992,1cyclictest24619-21bash22:44:596
16729991,3cyclictest15982-21sshd23:10:536
16729991,3cyclictest0-21swapper/622:19:596
16729990,9cyclictest0-21swapper/600:18:046
16729990,0cyclictest0-21swapper/622:29:176
16729990,0cyclictest0-21swapper/620:54:236
16569999,0cyclictest0-21swapper/400:09:504
16569998,1cyclictest49-21ksoftirqd/421:00:374
16569997,1cyclictest732-21NetworkManager21:06:264
16569997,1cyclictest481ktimersoftd/422:25:084
16569993,6cyclictest0-21swapper/423:14:154
16569990,9cyclictest0-21swapper/421:23:114
16569990,9cyclictest0-21swapper/420:56:044
16569990,9cyclictest0-21swapper/400:14:054
16569990,9cyclictest0-21swapper/400:03:034
16569990,0cyclictest0-21swapper/422:32:054
16489999,0cyclictest0-21swapper/300:16:233
16489998,1cyclictest2281-21kworker/3:000:06:023
16489998,1cyclictest0-21swapper/322:27:363
16489997,2cyclictest25992-21kworker/3:423:19:063
16489997,1cyclictest41-21ksoftirqd/322:19:163
16489997,1cyclictest41-21ksoftirqd/322:10:173
16489996,2cyclictest6555-21kworker/3:321:00:043
16489994,4cyclictest9577-21proc_pri22:29:233
16489992,4cyclictest1-21systemd23:31:453
16489992,3cyclictest12078-21sshd23:51:073
16489990,9cyclictest0-21swapper/322:44:283
16419998,1cyclictest33-21ksoftirqd/220:34:172
16419998,1cyclictest33-21ksoftirqd/218:59:222
16419998,1cyclictest33-21ksoftirqd/218:59:032
16419997,1cyclictest33-21ksoftirqd/219:24:042
16339999,0cyclictest0-21swapper/123:53:131
16339998,1cyclictest25-21ksoftirqd/100:00:461
16339997,1cyclictest25-21ksoftirqd/122:27:411
16339997,1cyclictest25-21ksoftirqd/122:08:151
16339995,3cyclictest26670-21sshd22:36:241
16339994,3cyclictest8466-21sshd22:20:151
16339994,3cyclictest732-21NetworkManager22:41:291
16339994,3cyclictest15000-21sshd21:27:231
16339994,2cyclictest18048-21sshd22:48:501
16249999,0cyclictest0-21swapper/023:05:030
16249999,0cyclictest0-21swapper/023:05:030
16249997,1cyclictest9-21ksoftirqd/022:33:510
16249993,5cyclictest21061-21bash23:20:320
16249993,5cyclictest12960-21sshd22:03:070
16249992,6cyclictest3810-21bash00:12:460
16249992,6cyclictest27083-21sshd00:16:070
16249992,6cyclictest23362-21sshd21:33:050
16249991,3cyclictest28094-21sshd22:45:320
16249991,2cyclictest24027-21grep22:54:100
16249990,9cyclictest0-21swapper/023:39:530
16249990,9cyclictest0-21swapper/020:54:410
16249990,4cyclictest21179-21sshd00:19:560
16729987,1cyclictest0-21swapper/620:14:256
16729987,1cyclictest0-21swapper/620:14:256
16729987,1cyclictest0-21swapper/619:39:206
16729986,1cyclictest65-21ksoftirqd/621:25:406
16729986,1cyclictest18983-21systemd-cgroups00:01:036
16729985,2cyclictest711-21dbus-daemon21:49:186
16729985,2cyclictest27847-21bash23:34:566
16729984,3cyclictest14403-21grep23:33:176
16729984,1cyclictest0-21swapper/621:02:136
16729983,2cyclictest28394-21grepconf.sh22:14:136
16729981,3cyclictest22569-21sshd22:04:196
16729980,7cyclictest711-21dbus-daemon21:18:166
16729980,7cyclictest0-21swapper/623:25:336
16729980,0cyclictest0-21swapper/620:44:276
16569988,0cyclictest0-21swapper/423:47:454
16569986,1cyclictest49-21ksoftirqd/423:36:334
16569985,2cyclictest6527-21grepconf.sh23:41:034
16569984,1cyclictest13299-21grepconf.sh21:54:104
16569983,5cyclictest24670-21grepconf.sh00:20:324
16569983,5cyclictest0-21swapper/423:22:544
16569983,5cyclictest0-21swapper/422:56:534
16569983,2cyclictest0-21swapper/419:02:044
16569981,1cyclictest26306-21sshd21:46:544
16569980,8cyclictest0-21swapper/419:10:414
16569980,2cyclictest0-21swapper/423:06:274
16569980,2cyclictest0-21swapper/423:06:274
16569980,0cyclictest0-21swapper/422:34:084
16489988,0cyclictest0-21swapper/320:34:243
16489988,0cyclictest0-21swapper/319:44:163
16489988,0cyclictest0-21swapper/300:19:223
16489987,1cyclictest12044-21kworker/3:100:01:243
16489986,1cyclictest30040-21kworker/3:121:51:283
16489986,1cyclictest14788-21kworker/3:221:26:303
16489985,2cyclictest21889-21cp23:34:173
16489983,3cyclictest17604-21sshd23:11:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional