You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 12:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sat Nov 22, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
327632900,0sleep10-21swapper/123:13:311
891425328,9sleep70-21swapper/718:50:157
188552530,2sleep4943899cyclictest23:11:334
313822510,0sleep40-21swapper/422:00:374
248172500,0sleep60-21swapper/623:38:186
907924926,8sleep20-21swapper/218:52:262
189862490,1sleep665-21ksoftirqd/623:33:146
312882480,0sleep00-21swapper/023:09:050
72092460,0sleep6641ktimersoftd/619:44:426
13682430,0sleep50-21swapper/522:09:355
35562420,0sleep50-21swapper/522:05:275
9302410,0sleep10-21swapper/123:04:501
189102410,2sleep0940899cyclictest23:54:540
96102400,0sleep3763-21runrttasks21:28:173
891124027,9sleep40-21swapper/418:50:124
889224027,9sleep30-21swapper/318:49:553
285432400,0sleep00-21swapper/022:08:580
903123926,9sleep10-21swapper/118:51:451
893823926,9sleep50-21swapper/518:50:295
890423928,8sleep60-21swapper/618:50:066
888923926,9sleep00-21swapper/018:49:520
102772380,0sleep40-21swapper/423:23:274
285922370,0sleep20-21swapper/223:08:432
129862360,0sleep20-21swapper/221:45:302
275902350,0sleep10-21swapper/122:08:511
945999258,0cyclictest0-21swapper/720:45:127
9459992514,0cyclictest0-21swapper/720:20:187
945999225,5cyclictest0-21swapper/722:09:077
945999214,5cyclictest0-21swapper/720:12:067
944999210,0cyclictest0-21swapper/519:43:465
945999208,0cyclictest0-21swapper/719:43:527
945999203,5cyclictest0-21swapper/720:39:547
9430992017,1cyclictest41-21ksoftirqd/300:20:193
307322200,0sleep30-21swapper/322:30:163
945999198,5cyclictest20681-21sshd00:19:017
945999198,5cyclictest0-21swapper/723:41:437
945999198,5cyclictest0-21swapper/723:41:437
945999198,5cyclictest0-21swapper/721:28:507
945999198,5cyclictest0-21swapper/721:22:507
945999198,5cyclictest0-21swapper/721:17:247
945999198,5cyclictest0-21swapper/719:58:487
945999198,5cyclictest0-21swapper/719:27:547
945999198,5cyclictest0-21swapper/719:20:427
945999198,5cyclictest0-21swapper/719:15:447
945999198,5cyclictest0-21swapper/718:59:207
945999198,5cyclictest0-21swapper/700:11:237
945999198,5cyclictest0-21swapper/700:08:237
945999198,0cyclictest0-21swapper/723:55:437
945999198,0cyclictest0-21swapper/722:58:397
945999198,0cyclictest0-21swapper/721:44:257
945999198,0cyclictest0-21swapper/720:50:127
945999198,0cyclictest0-21swapper/720:35:067
945999198,0cyclictest0-21swapper/720:29:347
945999198,0cyclictest0-21swapper/720:26:087
945999198,0cyclictest0-21swapper/720:05:227
945999198,0cyclictest0-21swapper/719:37:567
945999198,0cyclictest0-21swapper/719:12:487
945999198,0cyclictest0-21swapper/719:05:107
945999197,6cyclictest0-21swapper/723:53:217
9459991914,0cyclictest0-21swapper/720:00:587
945999191,0cyclictest0-21swapper/722:09:537
9430991914,3cyclictest41-21ksoftirqd/323:17:443
940899190,18cyclictest16461-21systemd-cgroups00:22:580
945999188,0cyclictest0-21swapper/722:35:297
945999187,5cyclictest0-21swapper/723:31:297
945999187,5cyclictest0-21swapper/723:13:217
945999187,5cyclictest0-21swapper/722:48:017
945999187,5cyclictest0-21swapper/722:21:337
945999187,5cyclictest0-21swapper/722:15:057
945999187,5cyclictest0-21swapper/721:04:347
945999187,5cyclictest0-21swapper/720:17:427
945999187,5cyclictest0-21swapper/719:30:227
945999187,0cyclictest0-21swapper/723:22:277
945999187,0cyclictest0-21swapper/723:15:057
945999187,0cyclictest0-21swapper/721:34:587
945999187,0cyclictest0-21swapper/721:32:587
944999180,0cyclictest0-21swapper/521:42:105
944999180,0cyclictest0-21swapper/519:02:225
9430991817,1cyclictest8738-21kworker/3:321:21:573
9430991815,3cyclictest41-21ksoftirqd/323:02:483
9430991814,1cyclictest41-21ksoftirqd/320:57:183
39222180,0sleep40-21swapper/421:06:154
945999177,0cyclictest0-21swapper/721:39:257
945999176,0cyclictest0-21swapper/723:45:177
945999176,0cyclictest0-21swapper/719:45:287
945999176,0cyclictest0-21swapper/700:02:277
9459991712,4cyclictest29544-21bash22:00:217
945999170,5cyclictest0-21swapper/721:10:127
944999176,5cyclictest21698-21sshd23:16:115
9430991716,1cyclictest18894-21kworker/3:122:53:493
9430991716,1cyclictest16801-21kworker/3:123:31:573
9430991715,1cyclictest41-21ksoftirqd/300:00:173
945999165,5cyclictest0-21swapper/723:28:017
945999165,0cyclictest0-21swapper/721:59:197
945999165,0cyclictest0-21swapper/719:53:527
945999160,15cyclictest32631-21sshd23:04:437
943099162,1cyclictest41-21ksoftirqd/322:44:223
9430991614,1cyclictest41-21ksoftirqd/300:13:193
9430991613,3cyclictest41-21ksoftirqd/300:04:393
9430991613,2cyclictest41-21ksoftirqd/323:28:043
9430991611,3cyclictest41-21ksoftirqd/321:13:593
9415991615,1cyclictest2353-21kworker/1:119:39:421
945999157,5cyclictest0-21swapper/720:54:547
945999154,5cyclictest0-21swapper/722:31:237
945999154,0cyclictest0-21swapper/723:34:437
944999150,3cyclictest0-21swapper/522:56:155
9430991515,0cyclictest32174-21kworker/3:019:39:423
9430991514,1cyclictest41-21ksoftirqd/323:39:293
9430991514,1cyclictest41-21ksoftirqd/323:39:293
9430991514,1cyclictest41-21ksoftirqd/322:58:333
9430991514,1cyclictest41-21ksoftirqd/322:35:203
9430991514,1cyclictest41-21ksoftirqd/321:59:383
9430991514,1cyclictest18505-21kworker/3:120:24:363
9430991514,1cyclictest16801-21kworker/3:123:35:253
9430991513,1cyclictest41-21ksoftirqd/323:54:543
9430991513,1cyclictest41-21ksoftirqd/323:05:003
9430991513,1cyclictest41-21ksoftirqd/322:26:563
9430991513,1cyclictest41-21ksoftirqd/322:09:373
9430991513,1cyclictest41-21ksoftirqd/321:54:113
9430991513,1cyclictest41-21ksoftirqd/321:45:133
9430991511,4cyclictest41-21ksoftirqd/321:40:193
940899150,12cyclictest99-21kswapd019:07:090
945999148,5cyclictest0-21swapper/721:00:187
945999142,0cyclictest0-21swapper/722:29:157
945999140,2cyclictest0-21swapper/700:21:017
9430991413,1cyclictest41-21ksoftirqd/323:19:573
9430991413,1cyclictest41-21ksoftirqd/321:54:313
9430991413,1cyclictest41-21ksoftirqd/320:54:233
9430991413,1cyclictest12805-21kworker/3:100:18:353
9430991412,1cyclictest41-21ksoftirqd/322:44:503
9430991412,1cyclictest41-21ksoftirqd/321:32:353
9430991412,1cyclictest41-21ksoftirqd/321:17:133
9430991412,1cyclictest41-21ksoftirqd/321:02:243
9430991411,1cyclictest41-21ksoftirqd/323:45:353
9430991410,3cyclictest41-21ksoftirqd/323:49:353
9430991410,3cyclictest41-21ksoftirqd/323:14:233
166782140,0sleep10-21swapper/123:45:421
945999138,5cyclictest27940-21sshd21:51:457
945999138,5cyclictest0-21swapper/723:00:517
945999138,5cyclictest0-21swapper/722:39:237
945999138,0cyclictest0-21swapper/718:59:447
945999132,0cyclictest0-21swapper/722:50:177
945799132,5cyclictest22797-21cat00:09:376
944999136,5cyclictest0-21swapper/523:27:035
944999132,8cyclictest0-21swapper/523:38:315
944999130,7cyclictest0-21swapper/521:24:155
944999130,13cyclictest0-21swapper/523:02:315
943099139,3cyclictest41-21ksoftirqd/321:06:553
9430991312,1cyclictest41-21ksoftirqd/321:36:473
9430991312,1cyclictest18505-21kworker/3:120:19:443
9430991311,1cyclictest41-21ksoftirqd/322:19:403
9430991311,1cyclictest41-21ksoftirqd/322:14:343
9430991311,1cyclictest41-21ksoftirqd/322:06:243
9430991310,3cyclictest41-21ksoftirqd/318:55:183
942299130,12cyclictest3868-21kworker/2:223:29:192
9415991311,1cyclictest25-21ksoftirqd/121:29:431
941599130,11cyclictest0-21swapper/121:40:421
236872130,0sleep50-21swapper/522:20:575
945799122,4cyclictest0-21swapper/621:38:486
944999126,5cyclictest0-21swapper/523:54:035
944999126,5cyclictest0-21swapper/520:44:445
944999125,0cyclictest0-21swapper/522:33:455
944999120,7cyclictest0-21swapper/520:27:405
944999120,7cyclictest0-21swapper/500:15:135
944999120,6cyclictest0-21swapper/520:59:345
944999120,0cyclictest0-21swapper/523:59:355
944999120,0cyclictest0-21swapper/523:57:375
944999120,0cyclictest0-21swapper/523:12:475
944999120,0cyclictest0-21swapper/522:53:135
944999120,0cyclictest0-21swapper/522:38:195
944999120,0cyclictest0-21swapper/522:18:235
944999120,0cyclictest0-21swapper/521:52:395
944999120,0cyclictest0-21swapper/521:27:195
944999120,0cyclictest0-21swapper/520:38:345
944999120,0cyclictest0-21swapper/520:32:385
944999120,0cyclictest0-21swapper/520:04:085
944999120,0cyclictest0-21swapper/519:56:105
944999120,0cyclictest0-21swapper/519:53:285
944999120,0cyclictest0-21swapper/519:45:025
944999120,0cyclictest0-21swapper/519:37:205
944999120,0cyclictest0-21swapper/519:13:025
944999120,0cyclictest0-21swapper/518:57:385
944999120,0cyclictest0-21swapper/500:04:235
9430991212,0cyclictest32174-21kworker/3:019:37:583
9422991211,1cyclictest33-21ksoftirqd/222:28:312
941599120,2cyclictest14528-21sshd21:16:071
940899120,11cyclictest14727-21grepconf.sh23:02:330
40882120,0sleep00-21swapper/021:23:170
33052120,0sleep20-21swapper/219:39:342
203152120,0sleep20-21swapper/221:21:032
945799119,1cyclictest2764-21systemd-cgroups23:57:186
945799112,4cyclictest6940-21sshd21:15:026
945799112,4cyclictest0-21swapper/623:24:086
945799111,4cyclictest13881-21sshd22:49:366
945799110,10cyclictest23035-21sshd22:59:266
945799110,10cyclictest0-21swapper/622:21:276
944999116,5cyclictest14549-21taskset22:45:195
944999116,5cyclictest0-21swapper/523:46:495
944999116,5cyclictest0-21swapper/523:30:135
944999116,5cyclictest0-21swapper/522:40:035
944999116,5cyclictest0-21swapper/522:00:175
944999116,5cyclictest0-21swapper/521:56:475
944999116,5cyclictest0-21swapper/521:44:235
944999116,5cyclictest0-21swapper/521:37:145
944999116,5cyclictest0-21swapper/521:04:265
944999116,5cyclictest0-21swapper/520:55:025
944999116,5cyclictest0-21swapper/520:40:545
944999116,5cyclictest0-21swapper/520:21:445
944999116,5cyclictest0-21swapper/520:10:305
944999116,5cyclictest0-21swapper/520:04:285
944999116,5cyclictest0-21swapper/519:29:325
944999116,5cyclictest0-21swapper/519:20:465
944999116,5cyclictest0-21swapper/519:14:405
944999116,5cyclictest0-21swapper/500:21:115
944999116,0cyclictest0-21swapper/523:21:255
944999116,0cyclictest0-21swapper/523:05:315
944999116,0cyclictest0-21swapper/521:12:025
944999116,0cyclictest0-21swapper/519:26:165
944999116,0cyclictest0-21swapper/519:04:485
944999116,0cyclictest0-21swapper/500:10:375
944999110,6cyclictest0-21swapper/520:14:425
944999110,5cyclictest0-21swapper/520:54:245
944999110,1cyclictest9964-21sshd21:15:275
944999110,0cyclictest0-21swapper/523:39:235
944999110,0cyclictest0-21swapper/523:39:235
944999110,0cyclictest0-21swapper/521:29:385
943899119,1cyclictest49-21ksoftirqd/423:03:134
943899110,1cyclictest11699-21sshd22:53:424
9430991111,0cyclictest18505-21kworker/3:120:17:503
9430991110,1cyclictest18505-21kworker/3:120:46:583
942299116,4cyclictest740-21gdbus22:49:552
9422991110,1cyclictest33-21ksoftirqd/223:35:022
9422991110,1cyclictest33-21ksoftirqd/221:10:582
942299110,11cyclictest0-21swapper/222:59:142
940899110,1cyclictest711-21dbus-daemon21:26:320
940899110,1cyclictest0-21swapper/023:47:480
93162110,0sleep00-21swapper/022:36:050
259382110,0sleep10-21swapper/123:29:401
98952100,1sleep4943899cyclictest00:21:574
945799109,1cyclictest0-21swapper/621:13:206
945799108,1cyclictest65-21ksoftirqd/623:24:376
945799108,1cyclictest2320-21sshd22:39:346
945799103,6cyclictest4306-21sshd22:09:476
945799101,9cyclictest85350irq/25-em1-rx-022:57:116
945799101,4cyclictest31250-21sshd00:06:046
945799101,4cyclictest12520-21sshd22:45:026
945799100,4cyclictest16954-21cp23:50:036
945799100,4cyclictest0-21swapper/621:39:376
944999106,4cyclictest0-21swapper/522:24:415
943899109,1cyclictest0-21swapper/422:29:354
943899109,1cyclictest0-21swapper/422:13:584
943899102,3cyclictest26736-21sshd22:08:444
943899100,4cyclictest690-21nscd23:57:264
943899100,4cyclictest0-21swapper/400:02:184
943899100,1cyclictest6680-21sshd21:19:254
943099109,1cyclictest29290-21kworker/3:119:32:463
943099109,1cyclictest18505-21kworker/3:120:10:543
943099108,1cyclictest41-21ksoftirqd/320:39:443
943099108,1cyclictest41-21ksoftirqd/319:14:373
9430991010,0cyclictest32174-21kworker/3:019:45:043
942299109,1cyclictest33-21ksoftirqd/223:14:412
942299109,1cyclictest33-21ksoftirqd/221:27:282
942299109,1cyclictest33-21ksoftirqd/221:14:452
942299108,1cyclictest33-21ksoftirqd/223:54:432
942299108,1cyclictest33-21ksoftirqd/222:35:582
942299102,2cyclictest24247-21bash21:55:272
942299100,9cyclictest14968-21grep00:03:532
941599106,2cyclictest25-21ksoftirqd/121:10:231
941599105,2cyclictest26147-21grepconf.sh22:34:131
941599100,9cyclictest15959-21/usr/sbin/munin21:54:411
940899108,1cyclictest9-21ksoftirqd/019:49:230
940899103,4cyclictest12510-21sshd22:10:580
940899101,4cyclictest0-21swapper/000:10:530
940899100,10cyclictest0-21swapper/000:08:460
940899100,1cyclictest0-21swapper/022:50:530
94579999,0cyclictest0-21swapper/621:21:136
94579998,1cyclictest65-21ksoftirqd/600:20:326
94579997,1cyclictest65-21ksoftirqd/621:29:336
94579996,2cyclictest27663-21httpd00:18:456
94579990,9cyclictest0-21swapper/622:30:076
94579990,9cyclictest0-21swapper/620:56:196
94579990,9cyclictest0-21swapper/620:34:336
94579990,2cyclictest14407-21sshd00:03:476
94389999,0cyclictest0-21swapper/422:15:424
94389997,1cyclictest30106-21sshd22:39:024
94389997,1cyclictest16252-21bash22:24:234
94389993,1cyclictest19519-21sshd21:25:084
94389990,9cyclictest0-21swapper/423:48:244
94389990,9cyclictest0-21swapper/423:35:194
94389990,9cyclictest0-21swapper/423:07:064
94389990,9cyclictest0-21swapper/421:51:184
94389990,9cyclictest0-21swapper/420:54:314
94389990,9cyclictest0-21swapper/400:17:444
94389990,0cyclictest15419-21sshd21:41:394
94389990,0cyclictest0-21swapper/423:24:274
94309998,1cyclictest18505-21kworker/3:120:35:003
94309995,4cyclictest41-21ksoftirqd/319:19:363
94229999,0cyclictest0-21swapper/200:22:002
94229998,1cyclictest33-21ksoftirqd/221:30:342
94229998,1cyclictest0-21swapper/221:01:192
94229997,1cyclictest33-21ksoftirqd/223:20:072
94229997,1cyclictest33-21ksoftirqd/219:14:222
94229993,3cyclictest2666-21sshd23:09:332
94229990,9cyclictest0-21swapper/222:30:142
94229990,9cyclictest0-21swapper/222:11:572
94159997,1cyclictest25-21ksoftirqd/121:29:221
94159997,1cyclictest0-21swapper/100:22:181
94159996,3cyclictest13051-21kworker/1:120:55:561
94159994,4cyclictest17203-21sshd23:50:051
94159994,1cyclictest0-21swapper/121:03:031
94159993,3cyclictest5677-21bash22:57:041
94159993,2cyclictest758-21sshd22:24:231
94159992,4cyclictest711-21dbus-daemon23:28:231
94159990,9cyclictest0-21swapper/123:00:101
94159990,8cyclictest26224-21kworker/1:222:34:261
94159990,0cyclictest0-21swapper/100:12:031
94089999,0cyclictest0-21swapper/023:30:460
94089999,0cyclictest0-21swapper/021:34:540
94089993,4cyclictest1-21systemd23:50:110
94089993,3cyclictest0-21swapper/020:15:200
94089992,4cyclictest32173-21grep00:01:280
94089992,4cyclictest20814-21grep23:24:420
94089990,9cyclictest0-21swapper/023:14:410
94089990,9cyclictest0-21swapper/021:47:210
94089990,8cyclictest21535-21bash21:59:310
94089990,4cyclictest5277-21sshd22:44:200
94089990,0cyclictest0-21swapper/022:26:170
94089990,0cyclictest0-21swapper/020:55:290
30311290,0sleep130302-21sshd22:17:401
94579988,0cyclictest0-21swapper/622:34:516
94579987,1cyclictest0-21swapper/622:05:506
94579986,1cyclictest65-21ksoftirqd/623:39:466
94579986,1cyclictest65-21ksoftirqd/623:39:466
94579985,2cyclictest9627-21bash22:14:476
94579985,2cyclictest23448-21grep23:46:406
94579985,2cyclictest2189-21grepconf.sh22:26:396
94579985,2cyclictest18910-21grep21:46:206
94579982,6cyclictest0-21swapper/621:57:566
94579982,6cyclictest0-21swapper/621:03:426
94579981,1cyclictest31828-21sshd22:00:416
94579980,7cyclictest16406-21grepconf.sh21:50:096
94389988,0cyclictest0-21swapper/400:09:234
94389987,1cyclictest0-21swapper/422:43:034
94389986,1cyclictest49-21ksoftirqd/422:27:184
94389986,1cyclictest49-21ksoftirqd/421:11:074
94389986,1cyclictest49-21ksoftirqd/420:14:224
94389985,2cyclictest26452-21bash22:47:014
94389983,5cyclictest0-21swapper/421:47:104
94389983,2cyclictest0-21swapper/423:51:094
94389983,2cyclictest0-21swapper/420:05:384
94389982,1cyclictest23282-21sshd23:33:504
94389980,8cyclictest0-21swapper/421:29:314
94389980,4cyclictest6221-21sshd21:02:234
94309988,0cyclictest9194-21kworker/3:019:10:243
94309988,0cyclictest9194-21kworker/3:019:03:283
94309987,1cyclictest9194-21kworker/3:019:05:123
94309987,1cyclictest18505-21kworker/3:120:29:483
94229988,0cyclictest0-21swapper/223:31:522
94229988,0cyclictest0-21swapper/220:55:102
94229987,1cyclictest33-21ksoftirqd/223:51:252
94229987,1cyclictest33-21ksoftirqd/222:45:082
94229987,1cyclictest33-21ksoftirqd/222:16:052
94229987,1cyclictest33-21ksoftirqd/222:03:332
94229987,1cyclictest33-21ksoftirqd/221:35:102
94229986,1cyclictest33-21ksoftirqd/223:47:102
94229986,1cyclictest33-21ksoftirqd/222:04:532
94229986,1cyclictest33-21ksoftirqd/200:11:292
94229984,3cyclictest31294-21bash00:06:042
94229984,1cyclictest711-21dbus-daemon22:19:532
94229982,4cyclictest23044-21munin-run00:19:222
94229982,3cyclictest32298-21sshd21:39:392
94229982,3cyclictest26208-21id21:04:492
94229982,2cyclictest0-21swapper/221:49:312
94159986,1cyclictest7066-21kworker/1:022:24:251
94159986,1cyclictest32530-21kworker/1:021:04:441
94159986,1cyclictest3219-21kworker/1:023:37:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional