You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 13:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Sun Dec 14, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
409266921280,0sleep6841rcuc/612:00:026
352338325319,30sleep70-21swapper/707:06:237
352362725040,7sleep40-21swapper/407:09:494
352341824232,7sleep60-21swapper/607:06:546
3523940993529,5cyclictest3907220-21latency_hist10:24:517
352335023020,7sleep50-21swapper/507:05:575
38271862290,0sleep40-21swapper/409:40:114
3523936992624,2cyclictest3819657-21kworker/u16:0+events_unbound09:39:226
3523940992521,3cyclictest3732201-21awk08:54:517
3523940992422,1cyclictest3848881-21basename09:54:507
3523940992422,1cyclictest3654403-21idleruntime-cro08:14:507
3523936992422,2cyclictest3917047-21kworker/u16:0+events_unbound10:31:216
3523928992422,2cyclictest2784844-21kworker/u16:1+flush-8:007:35:204
3523940992316,5cyclictest650-21systemd-journal07:59:527
3523936992321,2cyclictest3760138-21kworker/u16:4+events_unbound09:50:346
3523936992321,2cyclictest3673845-21kworker/u16:2+events_unbound08:38:236
3523936992321,2cyclictest3566259-21kworker/u16:0+events_unbound07:30:116
3523936992321,2cyclictest2784844-21kworker/u16:1+events_unbound07:42:226
3523936992321,2cyclictest2784844-21kworker/u16:1+events_unbound07:42:226
3523928992321,2cyclictest3595801-21kworker/u16:0+events_unbound07:45:044
3523928992320,2cyclictest3663906-21kworker/u16:1+flush-8:008:35:094
3523936992220,2cyclictest4023030-21kworker/u16:0+events_unbound12:06:336
3523936992220,2cyclictest3965795-21kworker/u16:1+events_unbound12:29:336
3523936992220,2cyclictest3663906-21kworker/u16:1+events_unbound08:28:226
3523936992220,2cyclictest3644303-21kworker/u16:0+events_unbound09:29:226
3523936992220,2cyclictest3625910-21kworker/u16:2+events_unbound08:11:396
3523936992220,2cyclictest2784844-21kworker/u16:1+events_unbound07:59:226
3523936992220,2cyclictest2784844-21kworker/u16:1+events_unbound07:15:166
3523928992220,2cyclictest3965795-21kworker/u16:1+flush-8:011:20:214
3523928992220,2cyclictest3760138-21kworker/u16:4+flush-8:009:19:464
3523928992220,2cyclictest3760138-21kworker/u16:4+flush-8:009:19:464
3523940992116,4cyclictest650-21systemd-journal09:34:517
3523936992119,2cyclictest3868347-21kworker/u16:1+events_unbound10:27:216
3523936992119,2cyclictest3731872-21kworker/u16:3+events_unbound09:05:276
3523936992118,2cyclictest3965795-21kworker/u16:1+events_unbound11:54:526
3523928992119,2cyclictest3965795-21kworker/u16:1+flush-8:012:30:134
3523928992119,2cyclictest3965795-21kworker/u16:1+flush-8:012:30:134
3523928992119,2cyclictest3848983-21kworker/u16:3+events_unbound10:29:224
3523928992119,2cyclictest3839529-21kworker/u16:2+events_unbound10:00:384
3523928992119,2cyclictest3731872-21kworker/u16:3+events_unbound08:52:024
3523940992014,5cyclictest3887837-21latency_hist10:14:517
3523936992018,2cyclictest3819657-21kworker/u16:0+flush-8:010:10:176
3523936992018,2cyclictest3819657-21kworker/u16:0+flush-8:009:59:516
3523936992018,2cyclictest3819657-21kworker/u16:0+events_unbound09:57:346
3523936992018,2cyclictest3595801-21kworker/u16:0+events_unbound07:51:106
3523936992017,2cyclictest3582640-21kworker/u16:3+flush-8:007:59:546
3523928992018,2cyclictest3848983-21kworker/u16:3+events_unbound11:04:524
3523928992018,2cyclictest3839529-21kworker/u16:2+events_unbound09:52:224
3523936991917,2cyclictest4023030-21kworker/u16:0+events_unbound11:51:346
3523936991917,2cyclictest4013848-21kworker/u16:5+events_unbound11:22:496
3523936991917,2cyclictest3965795-21kworker/u16:1+events_unbound11:40:536
3523936991917,2cyclictest3965795-21kworker/u16:1+events_unbound11:40:536
3523936991917,2cyclictest3673845-21kworker/u16:2+flush-8:008:30:126
3523936991917,2cyclictest3673845-21kworker/u16:2+flush-8:008:30:126
3523936991917,2cyclictest3644303-21kworker/u16:0+events_unbound09:00:546
3523936991917,2cyclictest3644303-21kworker/u16:0+events_unbound08:44:316
3523936991917,2cyclictest3484892-21kworker/u16:4+flush-8:007:45:196
3523928991918,1cyclictest3985102-21kworker/u16:2+flush-8:011:30:154
3523928991917,2cyclictest4091010-21kworker/u16:3+events_unbound11:59:454
3523928991917,2cyclictest3819657-21kworker/u16:0+events_unbound10:08:294
3523928991917,2cyclictest3819657-21kworker/u16:0+events_unbound10:08:294
3523928991917,2cyclictest3644303-21kworker/u16:0+events_unbound08:26:264
3523928991917,2cyclictest2784844-21kworker/u16:1+flush-8:008:00:044
3523940991813,4cyclictest650-21systemd-journal07:39:517
3523936991816,2cyclictest4111831-21kworker/u16:4+flush-8:012:30:186
3523936991816,2cyclictest4111831-21kworker/u16:4+flush-8:012:30:186
3523936991816,2cyclictest3790709-21kworker/u16:1+flush-8:009:43:256
3523936991816,2cyclictest3760138-21kworker/u16:4+events_unbound09:47:506
3523936991816,2cyclictest3731872-21kworker/u16:3+events_unbound08:54:506
3523936991815,2cyclictest3582640-21kworker/u16:3+events_unbound08:05:206
3523928991816,2cyclictest4111831-21kworker/u16:4+events_unbound12:11:214
3523928991816,2cyclictest4023030-21kworker/u16:0+events_unbound12:16:334
3523928991816,2cyclictest3955689-21kworker/u16:0+events_unbound10:53:024
3523928991816,2cyclictest3760138-21kworker/u16:4+events_unbound09:36:014
3523928991816,2cyclictest3484892-21kworker/u16:4+flush-8:007:10:124
3523928991816,2cyclictest2784844-21kworker/u16:1+events_unbound07:41:394
3523928991816,2cyclictest2784844-21kworker/u16:1+events_unbound07:41:394
3523928991815,2cyclictest3254067-21kworker/u16:2+events_unbound07:19:584
3523940991711,5cyclictest4033676-21latency_hist11:29:507
352394099170,15cyclictest1267-21snmpd10:46:137
3523936991715,2cyclictest4023030-21kworker/u16:0+events_unbound11:37:346
3523936991715,2cyclictest3965795-21kworker/u16:1+flush-8:012:18:236
3523936991715,2cyclictest3965795-21kworker/u16:1+events_unbound12:24:466
3523936991715,2cyclictest3868347-21kworker/u16:1+events_unbound10:24:486
3523936991715,2cyclictest3848983-21kworker/u16:3+events_unbound11:01:336
3523936991715,2cyclictest3780792-21kworker/u16:2+events_unbound09:21:276
3523936991715,2cyclictest3731872-21kworker/u16:3+events_unbound08:59:476
3523928991715,2cyclictest3965795-21kworker/u16:1+events_unbound11:17:014
3523928991715,2cyclictest3917047-21kworker/u16:0+events_unbound10:30:164
3523928991715,2cyclictest3731872-21kworker/u16:3+flush-8:009:05:174
3523928991715,2cyclictest3625910-21kworker/u16:2+flush-8:008:05:334
3523928991715,2cyclictest3595801-21kworker/u16:0+events_unbound07:55:144
3523928991715,2cyclictest3582640-21kworker/u16:3+events_unbound08:20:134
3523928991714,2cyclictest3731872-21kworker/u16:3+events_unbound08:55:044
352394099168,7cyclictest5944-21gmain09:45:177
3523940991613,2cyclictest0-21swapper/708:39:517
3523936991614,2cyclictest4159951-21kworker/u16:0+events_unbound12:37:186
3523936991614,2cyclictest4111831-21kworker/u16:4+events_unbound12:13:226
3523936991614,2cyclictest4043128-21kworker/u16:3+flush-8:011:34:416
3523936991614,2cyclictest3868347-21kworker/u16:1+events_unbound10:46:536
3523936991614,2cyclictest3848983-21kworker/u16:3+events_unbound11:06:066
3523936991614,2cyclictest3819657-21kworker/u16:0+events_unbound10:08:236
3523936991614,2cyclictest3819657-21kworker/u16:0+events_unbound10:08:236
352393199160,15cyclictest4122235-21fschecks_time12:15:045
3523928991614,2cyclictest3644303-21kworker/u16:0+flush-8:008:45:224
3523928991612,3cyclictest3673845-21kworker/u16:2+flush-8:008:40:114
352394099159,4cyclictest1267-21snmpd08:58:597
3523940991511,3cyclictest4159225-21ntpq12:30:127
3523940991511,3cyclictest4159225-21ntpq12:30:127
352394099150,14cyclictest0-21swapper/707:10:107
3523936991513,2cyclictest3848983-21kworker/u16:3+events_unbound11:12:336
3523936991513,2cyclictest3582640-21kworker/u16:3+events_unbound07:35:346
3523936991513,2cyclictest3254067-21kworker/u16:2+flush-8:007:14:016
3523928991513,2cyclictest4091010-21kworker/u16:3+flush-8:012:05:254
3523928991513,2cyclictest4023030-21kworker/u16:0+events_unbound12:03:494
3523928991513,2cyclictest3848983-21kworker/u16:3+flush-8:009:55:124
3523928991512,3cyclictest0-21swapper/411:50:104
3523928991512,2cyclictest3917047-21kworker/u16:0+events_unbound10:34:524
3523928991512,2cyclictest3868347-21kworker/u16:1+flush-8:010:09:524
352394099148,5cyclictest3556786-21latency_hist07:24:527
352394099147,6cyclictest6045-21gmain10:05:057
352394099147,6cyclictest6045-21gmain10:05:057
352394099146,7cyclictest6450-21gmain09:25:177
352394099146,7cyclictest0-21swapper/711:15:517
3523940991410,3cyclictest1267-21snmpd08:17:377
352394099140,4cyclictest4125120-21kthreadcore12:15:097
352394099140,1cyclictest0-21swapper/712:00:137
352394099140,1cyclictest0-21swapper/708:25:167
352394099140,12cyclictest0-21swapper/707:50:157
3523936991413,1cyclictest0-21swapper/610:35:046
3523936991412,2cyclictest3955689-21kworker/u16:0+events_unbound10:51:296
3523936991412,2cyclictest3887680-21kworker/u16:2+events_unbound10:40:306
3523936991412,2cyclictest3801947-21kworker/u16:3+events_unbound09:32:286
352393199140,1cyclictest0-21swapper/509:29:515
352393199140,14cyclictest0-21swapper/511:40:155
352393199140,14cyclictest0-21swapper/511:40:155
3523928991412,2cyclictest4139496-21kworker/u16:5+events_unbound12:25:504
3523928991412,2cyclictest3965795-21kworker/u16:1+flush-8:011:00:154
3523928991412,2cyclictest3965795-21kworker/u16:1+events_unbound10:55:044
3523928991412,2cyclictest3868347-21kworker/u16:1+events_unbound10:42:234
3523928991412,2cyclictest3848983-21kworker/u16:3+events_unbound10:20:074
3523928991412,2cyclictest3731872-21kworker/u16:3+events_unbound09:03:224
3523928991412,2cyclictest3663906-21kworker/u16:1+flush-8:008:18:264
352392899140,14cyclictest0-21swapper/409:20:054
352394099139,4cyclictest96-21ksoftirqd/710:00:077
352394099139,3cyclictest4111839-21/usr/sbin/munin12:10:117
352394099139,3cyclictest4071626-21ntp_states11:45:147
352394099139,3cyclictest1267-21snmpd12:24:527
352394099139,3cyclictest1267-21snmpd10:58:087
352394099139,3cyclictest1267-21snmpd09:19:157
352394099139,3cyclictest1267-21snmpd09:19:157
352394099139,3cyclictest1267-21snmpd08:23:297
352394099139,3cyclictest1267-21snmpd07:17:187
352394099139,3cyclictest0-21swapper/711:00:157
352394099138,4cyclictest4082300-21latency_hist11:54:507
352394099137,5cyclictest3946061-21latency_hist10:44:507
352394099136,6cyclictest6505-21gmain07:59:457
352394099130,5cyclictest3824376-21kthreadcore09:40:097
3523936991311,2cyclictest4013848-21kworker/u16:5+events_unbound11:17:226
3523936991311,2cyclictest3868347-21kworker/u16:1+events_unbound10:18:436
3523936991311,2cyclictest3663906-21kworker/u16:1+flush-8:008:20:086
3523936991311,2cyclictest3644303-21kworker/u16:0+events_unbound09:13:506
3523936991311,2cyclictest3644303-21kworker/u16:0+events_unbound08:18:026
352393199138,4cyclictest3772338-21cat09:15:065
352393199138,4cyclictest3772338-21cat09:15:065
3523931991312,1cyclictest0-21swapper/508:35:285
3523931991311,2cyclictest3965795-21kworker/u16:1+flush-8:010:55:165
352393199130,13cyclictest0-21swapper/512:28:105
3523928991311,2cyclictest3965795-21kworker/u16:1+flush-8:011:44:514
3523928991311,2cyclictest3965795-21kworker/u16:1+flush-8:011:25:144
3523928991311,2cyclictest3965795-21kworker/u16:1+events_unbound12:22:074
3523928991311,2cyclictest3673845-21kworker/u16:2+flush-8:008:30:064
3523928991311,2cyclictest3673845-21kworker/u16:2+flush-8:008:30:064
3523928991311,2cyclictest3625910-21kworker/u16:2+events_unbound08:14:394
352394099129,2cyclictest1267-21snmpd11:42:517
352394099129,2cyclictest1267-21snmpd11:42:517
352394099129,2cyclictest1267-21snmpd10:14:527
352394099128,3cyclictest1267-21snmpd12:24:197
352394099128,3cyclictest1267-21snmpd11:24:087
352394099128,3cyclictest1267-21snmpd11:10:387
352394099128,3cyclictest1267-21snmpd11:07:327
352394099128,3cyclictest1267-21snmpd07:27:157
352394099127,4cyclictest6184-21gvfs-afc-volume12:05:097
352394099127,4cyclictest6184-21gvfs-afc-volume08:05:027
352394099127,4cyclictest3761499-21latency_hist09:09:507
352394099126,5cyclictest4034788-21tune2fs11:30:037
352394099124,7cyclictest3916858-21sshd-session10:29:217
352394099121,3cyclictest3693636-21/usr/sbin/munin08:35:057
352394099120,11cyclictest3692284-21ntpq08:30:147
352394099120,11cyclictest3692284-21ntpq08:30:147
352394099120,11cyclictest3566902-21/usr/sbin/munin07:30:077
3523936991211,1cyclictest0-21swapper/609:17:016
3523936991211,1cyclictest0-21swapper/609:17:016
3523936991210,2cyclictest3965795-21kworker/u16:1+flush-8:011:48:416
3523936991210,2cyclictest3566259-21kworker/u16:0+events_unbound07:28:246
352393199129,2cyclictest4068615-21kthreadcore11:45:095
352393199129,2cyclictest3847823-21needreboot09:50:125
352393199129,2cyclictest3635690-21cstates08:05:025
3523931991211,1cyclictest0-21swapper/509:38:325
3523931991211,1cyclictest0-21swapper/507:15:045
352393199120,1cyclictest0-21swapper/508:13:525
352392899129,2cyclictest3965795-21kworker/u16:1+flush-8:011:10:214
352392899128,3cyclictest3567683-21cat07:30:064
3523928991210,2cyclictest3965795-21kworker/u16:1+flush-8:011:40:144
3523928991210,2cyclictest3965795-21kworker/u16:1+flush-8:011:40:144
3523928991210,2cyclictest3760138-21kworker/u16:4+flush-8:009:10:214
352392899120,12cyclictest0-21swapper/410:47:064
352394099118,2cyclictest0-21swapper/710:50:197
352394099118,2cyclictest0-21swapper/709:10:117
352394099117,3cyclictest4091981-21sshd-session11:59:217
352394099117,3cyclictest1267-21snmpd11:37:547
352394099117,3cyclictest1267-21snmpd09:55:427
352394099116,4cyclictest4169599-21sshd-session12:35:217
352394099115,5cyclictest0-21swapper/707:44:337
352394099115,5cyclictest0-21swapper/707:44:337
352394099110,9cyclictest0-21swapper/709:20:047
352394099110,3cyclictest3722674-21latency_hist08:49:517
352394099110,2cyclictest3927021-21/usr/sbin/munin10:35:127
352394099110,10cyclictest3751621-21latency_hist09:04:507
352394099110,10cyclictest0-21swapper/709:30:097
352393699119,2cyclictest3965795-21kworker/u16:1+events_unbound11:28:256
352393699119,2cyclictest3965795-21kworker/u16:1+events_unbound10:57:266
352393699119,2cyclictest2784844-21kworker/u16:1+events_unbound07:23:236
352393699110,11cyclictest0-21swapper/608:46:366
352393199117,3cyclictest1-21systemd09:29:515
3523931991110,1cyclictest0-21swapper/511:55:325
3523931991110,1cyclictest0-21swapper/510:52:285
3523931991110,1cyclictest0-21swapper/510:37:105
3523931991110,1cyclictest0-21swapper/509:23:235
3523931991110,1cyclictest0-21swapper/507:32:345
352393199110,1cyclictest0-21swapper/510:04:425
352393199110,1cyclictest0-21swapper/509:03:515
352393199110,1cyclictest0-21swapper/507:23:535
352393199110,11cyclictest0-21swapper/511:16:265
352393199110,11cyclictest0-21swapper/511:01:085
352393199110,11cyclictest0-21swapper/508:52:585
352393199110,11cyclictest0-21swapper/508:26:485
352393199110,11cyclictest0-21swapper/507:52:075
352393199110,0cyclictest0-21swapper/512:39:025
352392899119,2cyclictest3790709-21kworker/u16:1+flush-8:009:45:124
352392899119,2cyclictest3566259-21kworker/u16:0+events_unbound07:28:024
3523928991110,1cyclictest0-21swapper/409:33:104
352394099106,3cyclictest1267-21snmpd07:45:107
352394099104,5cyclictest847-21systemd-oomd10:30:147
352393199109,1cyclictest0-21swapper/512:30:135
352393199109,1cyclictest0-21swapper/512:30:135
352393199109,1cyclictest0-21swapper/510:26:185
352393199109,1cyclictest0-21swapper/510:15:255
352393199109,1cyclictest0-21swapper/509:47:125
352393199109,1cyclictest0-21swapper/509:05:535
352393199109,1cyclictest0-21swapper/508:15:555
352393199109,1cyclictest0-21swapper/507:41:145
352393199109,1cyclictest0-21swapper/507:41:145
352393199108,2cyclictest4111831-21kworker/u16:4+events_unbound12:10:065
352393199106,3cyclictest3849818-21tune2fs09:55:045
352393199106,3cyclictest3616632-21irqcore07:55:065
352393199106,3cyclictest1-21systemd10:49:505
352393199100,10cyclictest0-21swapper/508:44:185
352392899108,2cyclictest4051862-21kworker/u16:4+events_unbound11:35:154
352392899108,2cyclictest3595801-21kworker/u16:0+flush-8:007:54:224
35239319995,3cyclictest3684710-21cat08:30:055
35239319995,3cyclictest3684710-21cat08:30:055
35239319994,4cyclictest3869481-21cat10:05:055
35239319994,4cyclictest3869481-21cat10:05:055
35239289998,1cyclictest0-21swapper/412:35:534
35239289995,3cyclictest3538518-21cat07:15:064
35239289994,4cyclictest3896591-21ntp_states10:15:154
35239319987,1cyclictest0-21swapper/512:21:325
35239319986,1cyclictest0-21swapper/511:50:095
35239319986,1cyclictest0-21swapper/511:10:085
35239319985,2cyclictest3764752-21cut09:10:095
35239319984,3cyclictest3820727-21tune2fs09:40:045
35239319984,3cyclictest3739767-21kthreadcore08:55:115
35239319984,3cyclictest3721313-21/usr/sbin/munin08:45:125
35239319983,4cyclictest4092850-21expr12:00:025
35239319983,4cyclictest4048895-21cut11:35:095
35239319983,4cyclictest3986272-21sed11:05:065
35239319983,4cyclictest3903082-21cut10:20:115
35239319983,4cyclictest3667865-21cat08:20:095
35239319983,4cyclictest3604905-21wc07:45:175
35239319983,4cyclictest3565358-21cat07:25:165
35239289986,1cyclictest0-21swapper/409:25:094
35239319976,1cyclictest0-21swapper/511:31:345
35239319975,1cyclictest0-21swapper/510:40:085
35239319975,1cyclictest0-21swapper/507:35:055
35239319974,2cyclictest3921261-21kthreadcore10:30:095
35239319973,3cyclictest4102611-21expr12:05:035
35239319973,3cyclictest4015314-21sh11:20:055
35239319972,4cyclictest3624914-21gdbus07:59:595
35239319972,4cyclictest3524516-21cat07:09:595
35239319963,2cyclictest3878690-21cpuspeed_turbos10:10:025
35239319962,3cyclictest4025178-21sed11:25:065
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional