You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 19:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sat Dec 06, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
314932750,2sleep7639299cyclictest09:07:367
53162620,1sleep55317-21proc_pri06:49:135
617225228,9sleep40-21swapper/406:53:384
132502520,0sleep40-21swapper/411:49:354
592625026,8sleep00-21swapper/006:50:130
599224925,9sleep70-21swapper/706:51:087
143902490,0sleep00-21swapper/010:34:110
281942440,0sleep60-21swapper/609:46:196
466624322,17sleep10-21swapper/106:49:101
587024229,9sleep30-21swapper/306:49:283
36852420,0sleep50-21swapper/512:05:455
221742420,0sleep40-21swapper/412:21:224
213332420,1sleep7711-21dbus-daemon10:49:397
80642410,0sleep10-21swapper/110:28:031
205872410,0sleep70-21swapper/711:33:047
130842410,0sleep30-21swapper/311:45:173
67162400,0sleep60-21swapper/611:21:506
595423926,9sleep60-21swapper/606:50:356
372323927,8sleep20-21swapper/206:49:032
285572390,0sleep50-21swapper/510:46:015
41262380,0sleep40-21swapper/408:39:144
34172380,0sleep00-21swapper/008:39:100
639299203,16cyclictest31763-21grepconf.sh08:57:117
638499207,2cyclictest65-21ksoftirqd/609:57:076
6334992019,1cyclictest9-21ksoftirqd/010:52:210
639299180,10cyclictest29781-21grep09:12:417
639299180,10cyclictest15131-21grepconf.sh11:49:517
639299180,10cyclictest11809-21sshd10:48:307
638499181,3cyclictest65-21ksoftirqd/611:01:066
637699187,5cyclictest0-21swapper/510:01:295
637699180,2cyclictest0-21swapper/511:46:275
635399183,14cyclictest27352-21sshd11:24:372
633499186,1cyclictest9-21ksoftirqd/009:55:450
6334991812,4cyclictest9-21ksoftirqd/012:23:350
639299173,6cyclictest680-21nscd12:07:137
639299172,14cyclictest16791-21sshd08:59:477
639299170,8cyclictest26302-21grep11:55:437
6384991714,1cyclictest65-21ksoftirqd/612:20:576
637699176,5cyclictest0-21swapper/508:40:095
634299179,3cyclictest25-21ksoftirqd/112:23:371
633499178,1cyclictest121rcu_preempt11:24:220
633499174,3cyclictest9-21ksoftirqd/011:42:510
6334991712,3cyclictest9-21ksoftirqd/011:59:410
6334991712,3cyclictest9-21ksoftirqd/010:45:020
639299160,9cyclictest8532-21bash11:44:387
639299160,9cyclictest25084-21grep09:31:227
639299160,9cyclictest11894-21systemd-cgroups10:28:537
639299160,8cyclictest18457-21sshd11:59:057
639299160,6cyclictest1324-21grepconf.sh09:52:017
638499165,2cyclictest65-21ksoftirqd/610:53:226
637699168,0cyclictest0-21swapper/510:14:175
637699165,5cyclictest0-21swapper/507:56:275
637699160,15cyclictest12151-21sshd11:27:135
635399160,15cyclictest711-21dbus-daemon11:59:022
633499169,4cyclictest9-21ksoftirqd/010:09:160
633499166,3cyclictest9-21ksoftirqd/010:14:340
633499166,1cyclictest9-21ksoftirqd/009:33:470
633499166,1cyclictest131rcu_sched11:13:390
633499165,3cyclictest9-21ksoftirqd/012:12:360
633499164,1cyclictest9-21ksoftirqd/012:14:530
633499164,1cyclictest9-21ksoftirqd/012:14:530
6334991612,1cyclictest9-21ksoftirqd/011:03:310
228322160,1sleep773-21ksoftirqd/711:24:077
639299155,9cyclictest9573-21sshd11:12:257
639299152,10cyclictest27123-21nfsd408:24:097
639299151,5cyclictest2790-21rm09:42:337
639299150,9cyclictest9054-21bash11:02:367
639299150,7cyclictest0-21swapper/710:35:427
639299150,6cyclictest711-21dbus-daemon11:05:017
639299150,6cyclictest0-21swapper/710:56:397
639299150,6cyclictest0-21swapper/710:23:007
639299150,6cyclictest0-21swapper/709:28:377
639299150,14cyclictest2272-21sshd11:39:237
638499158,1cyclictest65-21ksoftirqd/610:36:056
638499155,2cyclictest65-21ksoftirqd/612:06:056
6384991513,1cyclictest65-21ksoftirqd/609:31:566
6384991512,3cyclictest65-21ksoftirqd/610:58:046
638499151,3cyclictest65-21ksoftirqd/609:37:596
638499151,1cyclictest131rcu_sched10:41:376
637699158,6cyclictest17871-21bash10:04:195
637699158,6cyclictest17871-21bash10:04:195
637699158,5cyclictest0-21swapper/508:59:215
637699158,5cyclictest0-21swapper/508:56:155
637699158,5cyclictest0-21swapper/508:29:055
637699158,0cyclictest0-21swapper/507:27:035
637699157,0cyclictest0-21swapper/508:02:555
634299155,3cyclictest25-21ksoftirqd/110:04:571
634299155,3cyclictest25-21ksoftirqd/110:04:571
634299155,3cyclictest25-21ksoftirqd/109:25:441
6342991510,3cyclictest25-21ksoftirqd/109:30:071
634299151,3cyclictest25-21ksoftirqd/111:14:101
634299151,2cyclictest25-21ksoftirqd/112:04:151
634299151,2cyclictest25-21ksoftirqd/111:19:221
634299150,3cyclictest16479-21strings10:09:101
634299150,3cyclictest0-21swapper/112:09:071
634299150,3cyclictest0-21swapper/110:29:071
633499155,2cyclictest9-21ksoftirqd/010:32:530
633499154,1cyclictest9-21ksoftirqd/010:08:080
633499154,1cyclictest9-21ksoftirqd/010:08:080
633499153,2cyclictest9-21ksoftirqd/011:51:560
6334991513,1cyclictest9-21ksoftirqd/012:04:030
306432150,0sleep00-21swapper/009:17:380
639299149,0cyclictest0-21swapper/710:38:597
639299147,7cyclictest0-21swapper/709:54:317
639299142,11cyclictest15979-21iostat10:24:097
639299142,11cyclictest1-21systemd12:12:237
639299140,8cyclictest0-21swapper/709:36:077
639299140,7cyclictest0-21swapper/709:17:147
639299140,6cyclictest19425-21grepconf.sh10:14:187
639299140,6cyclictest0-21swapper/711:19:367
639299140,6cyclictest0-21swapper/710:04:127
639299140,6cyclictest0-21swapper/710:04:127
639299140,13cyclictest10900-21sshd12:15:277
639299140,13cyclictest10900-21sshd12:15:277
639299140,0cyclictest0-21swapper/710:11:567
638499144,2cyclictest65-21ksoftirqd/611:46:306
638499142,3cyclictest65-21ksoftirqd/608:55:006
638499142,1cyclictest65-21ksoftirqd/611:43:026
6384991413,1cyclictest65-21ksoftirqd/610:05:306
6384991413,1cyclictest65-21ksoftirqd/610:05:306
6384991412,1cyclictest65-21ksoftirqd/611:04:036
6384991412,1cyclictest65-21ksoftirqd/610:09:076
6384991412,1cyclictest65-21ksoftirqd/609:39:066
6384991412,0cyclictest65-21ksoftirqd/611:30:046
6384991411,3cyclictest65-21ksoftirqd/610:23:286
6384991411,3cyclictest65-21ksoftirqd/609:23:356
638499140,2cyclictest65-21ksoftirqd/612:01:346
637699149,5cyclictest0-21swapper/509:23:555
637699148,6cyclictest0-21swapper/511:55:375
637699148,5cyclictest861-21gdbus09:49:215
637699148,5cyclictest25317-21sshd10:30:435
637699148,5cyclictest0-21swapper/507:33:275
637699147,5cyclictest0-21swapper/510:55:555
637699147,5cyclictest0-21swapper/510:41:555
637699147,5cyclictest0-21swapper/509:37:475
637699147,5cyclictest0-21swapper/509:32:275
637699147,5cyclictest0-21swapper/506:54:155
637699147,0cyclictest0-21swapper/507:45:055
637699146,5cyclictest0-21swapper/509:19:475
637699146,0cyclictest0-21swapper/511:06:495
637699146,0cyclictest0-21swapper/510:09:535
637699146,0cyclictest0-21swapper/506:59:375
634299149,3cyclictest25-21ksoftirqd/109:04:041
634299146,3cyclictest25-21ksoftirqd/109:43:431
634299146,3cyclictest25-21ksoftirqd/109:14:041
634299146,3cyclictest25-21ksoftirqd/108:59:121
634299146,2cyclictest25-21ksoftirqd/111:46:501
634299146,1cyclictest25-21ksoftirqd/109:34:221
634299145,3cyclictest25-21ksoftirqd/111:34:051
634299145,3cyclictest25-21ksoftirqd/110:23:131
634299145,2cyclictest25-21ksoftirqd/109:44:441
634299145,1cyclictest25-21ksoftirqd/111:08:261
6342991413,1cyclictest25-21ksoftirqd/111:38:531
6342991413,1cyclictest25-21ksoftirqd/110:33:531
634299141,3cyclictest25-21ksoftirqd/111:29:101
634299141,3cyclictest25-21ksoftirqd/110:51:071
634299141,3cyclictest25-21ksoftirqd/109:59:121
634299141,2cyclictest25-21ksoftirqd/108:54:121
634299141,2cyclictest0-21swapper/111:13:331
634299141,1cyclictest0-21swapper/110:15:551
634299140,4cyclictest0-21swapper/111:24:361
634299140,3cyclictest0-21swapper/111:02:231
634299140,1cyclictest25-21ksoftirqd/110:44:071
634299140,1cyclictest0-21swapper/112:17:471
634299140,1cyclictest0-21swapper/112:17:471
633499149,2cyclictest9-21ksoftirqd/008:55:480
633499146,3cyclictest9-21ksoftirqd/011:22:530
633499144,3cyclictest9-21ksoftirqd/009:44:220
633499144,2cyclictest9-21ksoftirqd/009:42:470
633499144,2cyclictest9-21ksoftirqd/009:27:040
633499142,1cyclictest9-21ksoftirqd/010:01:320
6334991413,1cyclictest9-21ksoftirqd/009:49:040
6334991413,1cyclictest10961-21kworker/0:108:17:450
6334991412,1cyclictest9-21ksoftirqd/011:44:140
6334991412,1cyclictest9-21ksoftirqd/011:14:070
6334991412,1cyclictest9-21ksoftirqd/011:04:060
6334991412,1cyclictest9-21ksoftirqd/010:43:510
6334991412,1cyclictest9-21ksoftirqd/010:24:150
6334991411,1cyclictest9-21ksoftirqd/008:58:550
260632140,0sleep50-21swapper/509:16:555
639299130,7cyclictest0-21swapper/711:16:097
639299130,13cyclictest0-21swapper/711:36:517
639299130,13cyclictest0-21swapper/709:23:477
639299130,13cyclictest0-21swapper/707:59:077
638499139,1cyclictest65-21ksoftirqd/612:17:306
638499139,1cyclictest65-21ksoftirqd/612:17:306
638499138,2cyclictest121rcu_preempt10:28:536
638499134,2cyclictest65-21ksoftirqd/608:59:266
638499133,2cyclictest65-21ksoftirqd/609:05:166
6384991312,1cyclictest65-21ksoftirqd/611:24:086
6384991312,1cyclictest65-21ksoftirqd/609:09:086
6384991311,1cyclictest65-21ksoftirqd/609:24:106
6384991310,3cyclictest65-21ksoftirqd/612:09:446
6384991310,3cyclictest65-21ksoftirqd/611:50:176
6384991310,3cyclictest65-21ksoftirqd/609:14:566
638499131,1cyclictest65-21ksoftirqd/610:03:106
638499130,2cyclictest131rcu_sched11:14:276
637699138,5cyclictest0-21swapper/512:22:555
637699138,5cyclictest0-21swapper/512:14:255
637699138,5cyclictest0-21swapper/512:14:255
637699138,5cyclictest0-21swapper/512:12:095
637699138,5cyclictest0-21swapper/512:00:255
637699138,5cyclictest0-21swapper/511:49:155
637699138,5cyclictest0-21swapper/511:43:075
637699138,5cyclictest0-21swapper/511:22:095
637699138,5cyclictest0-21swapper/511:18:175
637699138,5cyclictest0-21swapper/511:01:075
637699138,5cyclictest0-21swapper/510:36:095
637699138,5cyclictest0-21swapper/509:54:075
637699138,5cyclictest0-21swapper/509:40:495
637699138,5cyclictest0-21swapper/509:13:135
637699138,5cyclictest0-21swapper/509:03:595
637699138,5cyclictest0-21swapper/508:51:035
637699138,5cyclictest0-21swapper/508:19:075
637699138,5cyclictest0-21swapper/507:42:095
637699138,0cyclictest0-21swapper/511:35:195
637699138,0cyclictest0-21swapper/511:30:275
637699137,5cyclictest32126-21cp10:51:215
637699137,5cyclictest0-21swapper/507:11:315
637699136,5cyclictest0-21swapper/509:44:495
637699136,5cyclictest0-21swapper/508:43:535
637699136,5cyclictest0-21swapper/508:33:535
637699136,5cyclictest0-21swapper/507:49:235
637699136,5cyclictest0-21swapper/507:16:255
637699136,5cyclictest0-21swapper/507:03:595
634299138,3cyclictest25-21ksoftirqd/109:56:191
634299135,3cyclictest25-21ksoftirqd/111:55:371
634299135,3cyclictest25-21ksoftirqd/111:50:521
634299135,3cyclictest25-21ksoftirqd/110:54:571
634299134,3cyclictest25-21ksoftirqd/112:01:251
634299134,3cyclictest25-21ksoftirqd/110:40:481
634299131,3cyclictest25-21ksoftirqd/109:09:431
634299130,2cyclictest0-21swapper/109:50:511
633499139,2cyclictest9-21ksoftirqd/010:23:020
633499138,2cyclictest9-21ksoftirqd/010:54:570
633499134,3cyclictest9-21ksoftirqd/011:35:520
633499134,1cyclictest9-21ksoftirqd/011:30:310
633499133,2cyclictest9-21ksoftirqd/009:38:520
633499133,1cyclictest9-21ksoftirqd/011:54:370
6334991311,1cyclictest9-21ksoftirqd/008:33:510
6334991311,1cyclictest9-21ksoftirqd/007:38:520
633499130,12cyclictest8985-21grepconf.sh09:09:040
296152130,0sleep20-21swapper/210:55:512
131962130,0sleep30-21swapper/311:22:513
639299125,6cyclictest29624-21grep09:46:337
639299124,5cyclictest0-21swapper/708:46:477
639299123,9cyclictest0-21swapper/707:38:577
639299123,9cyclictest0-21swapper/707:31:197
639299122,9cyclictest6119-21grep12:23:457
639299120,8cyclictest0-21swapper/707:13:297
639299120,12cyclictest0-21swapper/708:38:077
639299120,12cyclictest0-21swapper/708:19:237
639299120,12cyclictest0-21swapper/708:06:557
639299120,12cyclictest0-21swapper/707:08:097
639299120,0cyclictest0-21swapper/709:59:397
638499127,3cyclictest65-21ksoftirqd/611:38:146
638499122,3cyclictest65-21ksoftirqd/610:26:106
6384991211,1cyclictest65-21ksoftirqd/611:12:146
6384991210,1cyclictest65-21ksoftirqd/610:47:476
6384991210,1cyclictest65-21ksoftirqd/609:49:116
637699127,5cyclictest0-21swapper/511:10:415
637699127,5cyclictest0-21swapper/510:24:195
637699127,5cyclictest0-21swapper/508:05:395
637699127,5cyclictest0-21swapper/507:34:235
637699127,5cyclictest0-21swapper/507:19:435
637699127,0cyclictest0-21swapper/510:20:095
636899129,3cyclictest49-21ksoftirqd/410:16:084
6368991210,1cyclictest49-21ksoftirqd/411:34:044
6342991211,1cyclictest25-21ksoftirqd/109:19:061
6342991211,1cyclictest25-21ksoftirqd/108:04:091
6342991211,1cyclictest25-21ksoftirqd/107:56:131
6342991211,1cyclictest25-21ksoftirqd/107:44:101
633499124,8cyclictest9-21ksoftirqd/007:11:270
633499122,3cyclictest9-21ksoftirqd/009:05:450
6334991211,1cyclictest9-21ksoftirqd/009:19:150
639299116,5cyclictest0-21swapper/707:38:037
639299113,5cyclictest0-21swapper/707:53:137
639299111,10cyclictest0-21swapper/707:55:197
639299110,11cyclictest0-21swapper/708:29:057
639299110,11cyclictest0-21swapper/708:14:037
639299110,11cyclictest0-21swapper/708:13:357
639299110,11cyclictest0-21swapper/707:45:117
639299110,11cyclictest0-21swapper/707:01:297
639299110,11cyclictest0-21swapper/706:54:157
638499119,1cyclictest65-21ksoftirqd/611:54:076
638499119,1cyclictest65-21ksoftirqd/610:16:036
637699116,5cyclictest0-21swapper/508:09:355
637699116,0cyclictest0-21swapper/508:26:355
637699116,0cyclictest0-21swapper/508:15:295
636899119,1cyclictest49-21ksoftirqd/410:53:524
636899119,1cyclictest49-21ksoftirqd/409:20:194
636899116,3cyclictest21669-21sshd10:44:524
6368991110,1cyclictest49-21ksoftirqd/410:31:294
636199111,4cyclictest16654-21sshd11:32:283
635399119,1cyclictest33-21ksoftirqd/212:18:382
635399119,1cyclictest33-21ksoftirqd/212:18:382
6353991110,1cyclictest33-21ksoftirqd/210:09:092
634299118,1cyclictest25-21ksoftirqd/107:29:081
634299114,2cyclictest25-21ksoftirqd/106:58:541
6342991110,1cyclictest25-21ksoftirqd/107:59:091
6342991110,1cyclictest25-21ksoftirqd/107:49:051
634299110,3cyclictest25-21ksoftirqd/108:39:111
634299110,10cyclictest17690-21/usr/sbin/munin08:09:061
634299110,0cyclictest121rcu_preempt08:34:121
633499119,1cyclictest9-21ksoftirqd/008:03:520
6992100,0sleep21-21systemd11:34:352
639299105,5cyclictest0-21swapper/708:50:237
639299100,6cyclictest0-21swapper/707:22:357
639299100,10cyclictest0-21swapper/708:40:157
639299100,10cyclictest0-21swapper/707:26:097
639299100,10cyclictest0-21swapper/707:15:437
638499109,1cyclictest65-21ksoftirqd/608:24:136
638499108,1cyclictest65-21ksoftirqd/608:48:526
638499108,1cyclictest65-21ksoftirqd/607:33:526
636899109,1cyclictest49-21ksoftirqd/411:59:094
636899109,1cyclictest49-21ksoftirqd/411:21:454
636899109,1cyclictest49-21ksoftirqd/409:45:584
636899108,1cyclictest49-21ksoftirqd/412:07:154
636899108,1cyclictest49-21ksoftirqd/409:34:144
636899103,4cyclictest17909-21sshd11:32:394
636899100,10cyclictest30022-21kworker/4:009:26:174
636199103,1cyclictest17645-21sshd09:10:123
636199100,9cyclictest0-21swapper/311:02:183
636199100,10cyclictest0-21swapper/312:06:473
635399109,1cyclictest33-21ksoftirqd/211:42:112
635399109,0cyclictest0-21swapper/207:19:092
635399108,1cyclictest33-21ksoftirqd/209:59:572
635399101,6cyclictest0-21swapper/209:07:212
634299109,1cyclictest25-21ksoftirqd/108:13:531
634299109,1cyclictest25-21ksoftirqd/107:39:051
634299105,3cyclictest25-21ksoftirqd/108:24:051
634299105,3cyclictest25-21ksoftirqd/107:09:081
634299104,3cyclictest25-21ksoftirqd/107:19:091
634299101,2cyclictest25-21ksoftirqd/108:19:051
634299101,2cyclictest25-21ksoftirqd/107:04:091
633499108,1cyclictest9-21ksoftirqd/008:13:520
633499108,1cyclictest9-21ksoftirqd/007:43:520
633499106,1cyclictest25458-21awk08:23:510
633499105,1cyclictest9-21ksoftirqd/007:53:520
63849998,0cyclictest65-21ksoftirqd/607:13:526
63849997,1cyclictest65-21ksoftirqd/608:49:106
63849997,1cyclictest65-21ksoftirqd/607:39:156
63849997,1cyclictest65-21ksoftirqd/606:59:036
63689998,1cyclictest49-21ksoftirqd/410:24:554
63689998,1cyclictest49-21ksoftirqd/410:03:124
63689997,1cyclictest49-21ksoftirqd/410:10:524
63689997,1cyclictest49-21ksoftirqd/406:54:334
63689994,3cyclictest30577-21sshd08:56:594
63689994,3cyclictest18371-21sshd11:04:004
63689994,3cyclictest0-21swapper/409:17:154
63689993,3cyclictest22911-21sshd10:05:074
63689993,3cyclictest22911-21sshd10:05:074
63689993,3cyclictest16084-21sshd12:16:124
63689993,3cyclictest16084-21sshd12:16:124
63689993,3cyclictest15184-21sshd09:39:214
63689993,2cyclictest0-21swapper/409:00:114
63689990,8cyclictest3864-21kworker/4:209:56:404
63619999,0cyclictest31807-21sshd10:46:323
63619999,0cyclictest0-21swapper/310:49:513
63619998,1cyclictest41-21ksoftirqd/310:55:203
63619998,1cyclictest21557-21grepconf.sh12:21:173
63619998,1cyclictest0-21swapper/311:59:323
63619997,1cyclictest41-21ksoftirqd/311:42:583
63619997,1cyclictest41-21ksoftirqd/310:03:583
63619997,1cyclictest41-21ksoftirqd/310:03:583
63619997,1cyclictest41-21ksoftirqd/309:59:093
63619992,6cyclictest3731-21sshd10:27:213
63619992,6cyclictest0-21swapper/311:27:143
63619990,9cyclictest0-21swapper/310:13:543
63619990,0cyclictest0-21swapper/309:06:443
63539999,0cyclictest0-21swapper/211:09:392
63539999,0cyclictest0-21swapper/210:59:142
63539998,1cyclictest33-21ksoftirqd/209:41:582
63539998,1cyclictest0-21swapper/211:15:292
63539993,4cyclictest4879-21sshd09:28:302
63539992,3cyclictest20829-21sshd11:33:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional