You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 20:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Fri Dec 26, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8857822940,1sleep7885783-21cat09:19:157
7145952650,1sleep60-21swapper/607:49:246
61980626048,8sleep50-21swapper/507:04:125
62907625948,8sleep40-21swapper/407:09:004
61980725645,7sleep60-21swapper/607:04:136
62881025317,31sleep70-21swapper/707:05:187
629374994741,5cyclictest1117071-21latency_hist11:19:037
629365994340,2cyclictest1166137-21kworker/u16:2+events_unbound11:54:055
629374994234,7cyclictest855890-21latency_hist09:04:037
629374994034,5cyclictest1262024-21latency_hist12:34:037
62937499400,38cyclictest1223421-21latency_hist12:14:037
629365993936,2cyclictest1043036-21kworker/u16:1+events_unbound10:54:185
629365993835,2cyclictest1246487-21kworker/u16:4+events_unbound12:29:215
629365993634,2cyclictest953186-21kworker/u16:2+flush-8:009:54:185
629365993634,2cyclictest953186-21kworker/u16:2+flush-8:009:54:185
629374993530,4cyclictest1175058-21latency_hist11:49:037
629365993533,2cyclictest681996-21kworker/u16:1+events_unbound08:53:145
629365993533,2cyclictest609737-21kworker/u16:0+events_unbound07:42:145
629365993432,2cyclictest681996-21kworker/u16:1+events_unbound08:01:155
629365993431,2cyclictest681996-21kworker/u16:1+events_unbound08:14:215
629365993431,2cyclictest1088503-21kworker/u16:3+events_unbound11:11:135
629365993431,2cyclictest1001490-21kworker/u16:0+events_unbound10:24:055
629365993331,2cyclictest923935-21kworker/u16:3+events_unbound09:44:195
629365993331,2cyclictest730192-21kworker/u16:4+events_unbound08:05:145
629365993331,2cyclictest681996-21kworker/u16:1+events_unbound07:48:145
629365993330,2cyclictest1088503-21kworker/u16:3+events_unbound11:04:165
629365993230,2cyclictest1185849-21kworker/u16:1+events_unbound12:17:225
629365993229,2cyclictest730192-21kworker/u16:4+events_unbound08:24:215
629365993129,2cyclictest903436-21kworker/u16:1+flush-8:009:39:195
629365993129,2cyclictest865598-21kworker/u16:0+events_unbound09:35:195
629365993129,2cyclictest798107-21kworker/u16:2+flush-8:008:38:145
629365993129,2cyclictest1059130-21kworker/u16:2+events_unbound10:49:335
629365993128,2cyclictest681996-21kworker/u16:1+events_unbound08:11:275
629365993128,2cyclictest662435-21kworker/u16:3+flush-8:007:29:345
629365993028,2cyclictest865686-21kworker/u16:3+events_unbound09:12:145
629365993028,2cyclictest730192-21kworker/u16:4+flush-8:008:19:335
629365993028,2cyclictest730192-21kworker/u16:4+flush-8:008:19:335
629365993028,2cyclictest1242744-21kworker/u16:0+events_unbound12:27:225
629365993028,2cyclictest1204179-21kworker/u16:3+events_unbound12:23:145
629365993028,2cyclictest1204179-21kworker/u16:3+events_unbound12:23:145
629365993028,2cyclictest1185849-21kworker/u16:1+events_unbound12:10:225
629365993028,2cyclictest1125800-21kworker/u16:0+events_unbound11:27:145
629365993028,2cyclictest1043036-21kworker/u16:1+events_unbound11:44:055
629365993028,2cyclictest1043036-21kworker/u16:1+events_unbound10:43:145
629365993026,3cyclictest662435-21kworker/u16:3+events_unbound07:49:265
629365992927,2cyclictest730192-21kworker/u16:4+events_unbound10:15:155
629365992927,2cyclictest681996-21kworker/u16:1+events_unbound08:29:345
629365992926,2cyclictest1166137-21kworker/u16:2+flush-8:011:49:205
629369992825,2cyclictest865598-21kworker/u16:0+events_unbound09:29:166
629369992825,2cyclictest845235-21kworker/u16:0+events_unbound08:59:166
629369992825,2cyclictest609737-21kworker/u16:0+events_unbound08:09:186
629369992823,2cyclictest1043036-21kworker/u16:1+events_unbound10:49:196
629365992826,2cyclictest923935-21kworker/u16:3+events_unbound10:38:155
629365992826,2cyclictest923935-21kworker/u16:3+events_unbound09:59:315
629365992826,2cyclictest845235-21kworker/u16:0+events_unbound08:59:225
629365992826,2cyclictest1204179-21kworker/u16:3+events_unbound12:04:245
629365992826,2cyclictest1185849-21kworker/u16:1+events_unbound12:01:135
629365992826,2cyclictest1001490-21kworker/u16:0+events_unbound10:22:235
629365992825,2cyclictest609737-21kworker/u16:0+events_unbound07:19:225
629365992725,2cyclictest798107-21kworker/u16:2+events_unbound09:19:395
629365992725,2cyclictest609737-21kworker/u16:0+events_unbound08:39:265
629365992725,2cyclictest609737-21kworker/u16:0+events_unbound07:10:155
629365992725,2cyclictest1059130-21kworker/u16:2+events_unbound11:24:015
629365992724,2cyclictest953186-21kworker/u16:2+events_unbound10:10:145
629365992624,2cyclictest730192-21kworker/u16:4+events_unbound09:26:225
629365992624,2cyclictest619493-21kworker/u16:1+events_unbound07:17:155
11180662260,1chrt1118067-21fschecks_count11:19:177
629365992523,2cyclictest865598-21kworker/u16:0+events_unbound09:53:515
629369992422,2cyclictest1001490-21kworker/u16:0+events_unbound11:04:306
629369992421,2cyclictest1125800-21kworker/u16:0+events_unbound11:59:156
629365992422,2cyclictest855881-21kworker/u16:1+events_unbound09:05:235
629365992422,2cyclictest730192-21kworker/u16:4+events_unbound10:04:265
629365992422,2cyclictest662435-21kworker/u16:3+flush-8:007:54:245
629365992422,2cyclictest609737-21kworker/u16:0+events_unbound07:27:235
629365992422,2cyclictest1125800-21kworker/u16:0+flush-8:011:39:055
629369992320,2cyclictest1126923-21kworker/u16:4+events_unbound11:54:186
629369992220,2cyclictest865598-21kworker/u16:0+flush-8:009:53:586
7852022210,0sleep7785204-21kthreadcore08:24:267
629374992115,5cyclictest904376-21latency_hist09:29:047
629369992119,2cyclictest923935-21kworker/u16:3+events_unbound09:54:346
629369992119,2cyclictest923935-21kworker/u16:3+events_unbound09:54:346
629369992119,2cyclictest923479-21kworker/u16:2+flush-8:009:38:596
629365992119,2cyclictest923935-21kworker/u16:3+events_unbound10:44:155
629365992119,2cyclictest923935-21kworker/u16:3+events_unbound10:44:155
629365992119,2cyclictest681996-21kworker/u16:1+events_unbound08:54:175
629365992119,2cyclictest1125800-21kworker/u16:0+events_unbound11:38:035
629365992118,2cyclictest865598-21kworker/u16:0+events_unbound09:32:235
629362992118,2cyclictest730192-21kworker/u16:4+flush-8:010:04:294
629369992018,2cyclictest1166137-21kworker/u16:2+events_unbound11:48:586
629365992018,2cyclictest923935-21kworker/u16:3+flush-8:010:33:545
629365992018,2cyclictest1125800-21kworker/u16:0+events_unbound11:29:315
629362992017,3cyclictest798107-21kworker/u16:2+flush-8:008:34:224
629374991915,3cyclictest1267-21snmpd11:56:157
629369991917,2cyclictest953186-21kworker/u16:2+flush-8:010:08:586
629365991917,2cyclictest798107-21kworker/u16:2+events_unbound08:45:235
629365991917,2cyclictest1204179-21kworker/u16:3+flush-8:012:34:235
629365991917,2cyclictest1043036-21kworker/u16:1+events_unbound11:14:345
629369991816,2cyclictest798107-21kworker/u16:2+flush-8:008:58:586
629369991816,2cyclictest1204179-21kworker/u16:3+events_unbound12:24:186
629369991816,2cyclictest1043036-21kworker/u16:1+events_unbound10:44:226
629369991816,2cyclictest1043036-21kworker/u16:1+events_unbound10:44:226
629365991815,2cyclictest865686-21kworker/u16:3+events_unbound09:14:055
62937499170,15cyclictest1267-21snmpd09:55:307
62937499170,15cyclictest1267-21snmpd09:55:307
629369991715,2cyclictest923935-21kworker/u16:3+flush-8:010:24:316
629369991715,2cyclictest609737-21kworker/u16:0+events_unbound08:00:236
629369991715,2cyclictest609737-21kworker/u16:0+events_unbound07:48:236
629369991715,2cyclictest1204179-21kworker/u16:3+flush-8:012:38:586
629369991714,2cyclictest730192-21kworker/u16:4+events_unbound08:39:066
629365991715,2cyclictest1001490-21kworker/u16:0+events_unbound11:03:025
62937499162,4cyclictest1076748-21grep10:54:257
629374991610,5cyclictest650-21systemd-journal11:24:037
629374991610,5cyclictest650-21systemd-journal08:54:037
629369991614,2cyclictest953186-21kworker/u16:2+events_unbound10:19:186
629369991614,2cyclictest865598-21kworker/u16:0+events_unbound09:40:246
629369991614,2cyclictest681996-21kworker/u16:1+events_unbound08:33:186
629369991614,2cyclictest662435-21kworker/u16:3+events_unbound07:39:226
629369991614,2cyclictest619493-21kworker/u16:1+events_unbound07:14:196
629369991614,2cyclictest1059130-21kworker/u16:2+events_unbound11:16:236
629369991614,2cyclictest1059130-21kworker/u16:2+events_unbound10:59:236
629362991614,2cyclictest609737-21kworker/u16:0+flush-8:007:24:344
62937499158,6cyclictest1138956-21kthreadcore11:29:207
629374991511,3cyclictest951844-21ntpq09:49:287
629369991513,2cyclictest798107-21kworker/u16:2+events_unbound08:36:146
629369991513,2cyclictest730192-21kworker/u16:4+events_unbound10:01:156
629369991513,2cyclictest730192-21kworker/u16:4+events_unbound09:22:186
629369991513,2cyclictest730192-21kworker/u16:4+events_unbound08:17:236
629369991513,2cyclictest662435-21kworker/u16:3+events_unbound07:29:036
629369991513,2cyclictest1166137-21kworker/u16:2+events_unbound11:49:376
629365991513,2cyclictest681996-21kworker/u16:1+events_unbound07:38:475
62937499149,4cyclictest869898-21kthreadcore09:09:217
62937499149,4cyclictest663401-21cat07:24:197
62937499149,4cyclictest0-21swapper/710:59:177
62937499149,3cyclictest738731-21ntpq07:59:307
62937499148,5cyclictest981801-21latency_hist10:09:037
62937499147,6cyclictest816142-21ntpq08:39:327
62937499147,6cyclictest750542-21cat08:09:187
62937499145,7cyclictest0-21swapper/711:49:047
629374991411,2cyclictest835512-21ntpq08:49:297
629374991411,2cyclictest0-21swapper/710:10:017
629374991410,3cyclictest1096874-21ntpq11:04:277
62937499140,12cyclictest1267-21snmpd10:26:197
629369991412,2cyclictest865598-21kworker/u16:0+events_unbound09:44:346
629369991412,2cyclictest609737-21kworker/u16:0+flush-8:007:34:236
629369991412,2cyclictest609737-21kworker/u16:0+events_unbound07:19:126
629369991412,2cyclictest1125800-21kworker/u16:0+flush-8:011:28:586
629369991412,2cyclictest1001490-21kworker/u16:0+events_unbound10:55:236
629362991413,1cyclictest0-21swapper/409:19:184
62936299140,14cyclictest0-21swapper/412:29:354
62936299140,14cyclictest0-21swapper/410:59:244
62936299140,14cyclictest0-21swapper/409:09:314
62937499139,3cyclictest864787-21smart_sda09:04:337
62937499138,4cyclictest876450-21cat09:14:207
62937499138,4cyclictest674412-21grep07:29:207
62937499138,4cyclictest1234232-21cat12:19:187
62937499138,4cyclictest1234232-21cat12:19:187
62937499138,4cyclictest0-21swapper/711:59:187
62937499137,5cyclictest729935-21sshd-session07:58:357
62937499137,5cyclictest1267-21snmpd12:14:247
62937499137,5cyclictest1262414-21/usr/sbin/munin12:34:177
62937499136,6cyclictest1148510-21cut11:34:227
62937499136,6cyclictest1002277-21cat10:19:207
62937499136,6cyclictest0-21swapper/708:05:227
62937499134,8cyclictest1243911-21iostat_ios12:24:207
62937499134,7cyclictest923788-21cat09:39:047
62937499131,5cyclictest5944-21gmain10:00:057
629374991311,1cyclictest0-21swapper/710:14:247
629374991311,1cyclictest0-21swapper/707:44:237
629374991310,2cyclictest825848-21ntpq08:44:297
629374991310,2cyclictest796772-21ntpq08:29:297
62937499130,2cyclictest767777-21ntpq08:14:307
62937499130,11cyclictest714662-21tr07:49:247
62937499130,11cyclictest682781-21cat07:34:217
62937499130,11cyclictest1211872-21kthreadcore12:04:257
629369991312,1cyclictest0-21swapper/612:23:166
629369991312,1cyclictest0-21swapper/612:23:166
629369991311,2cyclictest798107-21kworker/u16:2+events_unbound08:45:196
629369991311,2cyclictest730192-21kworker/u16:4+flush-8:010:13:256
629369991311,2cyclictest730192-21kworker/u16:4+events_unbound09:07:196
629369991311,2cyclictest629573-21kworker/u16:2+events_unbound07:29:146
629369991311,2cyclictest1185849-21kworker/u16:1+events_unbound12:14:086
629369991311,2cyclictest1185849-21kworker/u16:1+events_unbound12:09:196
629369991311,2cyclictest1125800-21kworker/u16:0+events_unbound11:34:016
629369991311,2cyclictest1043036-21kworker/u16:1+events_unbound10:41:216
629369991310,2cyclictest1204179-21kworker/u16:3+events_unbound12:04:066
629362991312,1cyclictest0-21swapper/407:54:194
629362991310,2cyclictest730192-21kworker/u16:4+flush-8:010:09:294
62937499128,3cyclictest933796-21/usr/sbin/munin09:44:327
62937499128,3cyclictest1267-21snmpd10:34:167
62937499128,3cyclictest1267-21snmpd09:29:237
62937499128,3cyclictest1059744-21expr10:49:167
62937499126,5cyclictest1059283-21cat10:49:037
62937499126,5cyclictest1059283-21cat10:49:037
62937499124,7cyclictest692650-21sed07:39:207
62937499124,7cyclictest6045-21gmain08:19:057
62937499124,7cyclictest6045-21gmain08:19:057
62937499124,7cyclictest1026955-21cut10:29:257
62937499124,6cyclictest924107-21grep09:39:177
62937499120,8cyclictest0-21swapper/710:39:127
62937499120,5cyclictest649078-21taskset07:14:257
62937499120,4cyclictest1161157-21cut11:39:277
62937499120,10cyclictest1098173-21/usr/sbin/munin11:09:177
62936999129,2cyclictest619493-21kworker/u16:1+events_unbound07:09:086
629369991211,1cyclictest0-21swapper/611:23:216
629369991211,1cyclictest0-21swapper/610:14:566
629369991211,1cyclictest0-21swapper/609:13:466
629369991211,1cyclictest0-21swapper/607:57:566
629369991210,2cyclictest953186-21kworker/u16:2+flush-8:010:29:096
629369991210,2cyclictest953186-21kworker/u16:2+events_unbound10:34:326
629369991210,2cyclictest730192-21kworker/u16:4+events_unbound09:25:106
629369991210,2cyclictest1125800-21kworker/u16:0+events_unbound11:34:326
629369991210,2cyclictest1059130-21kworker/u16:2+events_unbound11:09:196
62936999120,1cyclictest0-21swapper/611:39:556
629362991211,1cyclictest0-21swapper/411:34:244
629362991211,1cyclictest0-21swapper/408:08:214
62936299120,1cyclictest0-21swapper/412:19:114
62936299120,1cyclictest0-21swapper/412:19:114
62936299120,1cyclictest0-21swapper/412:08:564
62936299120,1cyclictest0-21swapper/412:01:594
62936299120,12cyclictest0-21swapper/408:27:264
62937499117,3cyclictest798553-21cat08:34:177
62937499115,5cyclictest1267-21snmpd07:10:297
62937499114,6cyclictest653424-21cat07:19:177
62936999119,2cyclictest681996-21kworker/u16:1+events_unbound08:50:136
62936999119,2cyclictest609737-21kworker/u16:0+events_unbound08:24:276
62936999110,11cyclictest0-21swapper/608:04:436
629362991110,1cyclictest0-21swapper/412:12:144
629362991110,1cyclictest0-21swapper/411:55:034
629362991110,1cyclictest0-21swapper/411:41:204
629362991110,1cyclictest0-21swapper/410:55:474
629362991110,1cyclictest0-21swapper/410:30:244
629362991110,1cyclictest0-21swapper/409:48:564
629362991110,1cyclictest0-21swapper/409:39:004
629362991110,1cyclictest0-21swapper/409:29:044
629362991110,1cyclictest0-21swapper/408:56:454
629362991110,1cyclictest0-21swapper/408:17:494
629362991110,1cyclictest0-21swapper/407:43:554
629362991110,1cyclictest0-21swapper/407:36:494
629362991110,1cyclictest0-21swapper/407:20:354
62936299110,1cyclictest0-21swapper/407:50:514
62936299110,11cyclictest0-21swapper/411:51:454
62936299110,11cyclictest0-21swapper/411:44:484
62936299110,11cyclictest0-21swapper/409:55:434
62936299110,11cyclictest0-21swapper/409:55:434
62936299110,11cyclictest0-21swapper/408:30:354
62936299110,0cyclictest0-21swapper/409:42:194
62936999109,1cyclictest0-21swapper/612:33:126
62936999108,2cyclictest730192-21kworker/u16:4+events_unbound08:22:516
62936999108,2cyclictest730192-21kworker/u16:4+events_unbound08:22:516
62936299109,1cyclictest0-21swapper/410:52:094
62936299109,1cyclictest0-21swapper/410:41:164
62936299109,1cyclictest0-21swapper/409:52:154
62936299107,2cyclictest619493-21kworker/u16:1+flush-8:007:14:304
6293699997,2cyclictest865686-21kworker/u16:3+events_unbound09:14:296
6293629998,1cyclictest0-21swapper/412:36:124
6293629998,1cyclictest0-21swapper/412:26:074
6293629998,1cyclictest0-21swapper/411:30:564
6293629998,1cyclictest0-21swapper/411:06:304
6293629998,1cyclictest0-21swapper/410:16:224
6293629998,1cyclictest0-21swapper/408:53:264
6293629987,1cyclictest0-21swapper/411:20:324
6293629987,1cyclictest0-21swapper/409:15:594
6293629987,1cyclictest0-21swapper/408:01:254
6293629976,1cyclictest0-21swapper/411:09:584
6293629976,1cyclictest0-21swapper/410:26:464
6293629976,1cyclictest0-21swapper/410:02:394
6293629976,1cyclictest0-21swapper/408:43:404
6293629976,1cyclictest0-21swapper/408:11:214
6293629976,1cyclictest0-21swapper/407:29:534
6293629972,4cyclictest630351-21if_err_p4p107:09:184
6293629970,6cyclictest0-21swapper/407:45:244
6293629965,1cyclictest0-21swapper/412:15:334
6293629965,1cyclictest0-21swapper/411:27:284
6293629965,1cyclictest0-21swapper/411:17:044
6293629965,1cyclictest0-21swapper/410:44:444
6293629965,1cyclictest0-21swapper/410:44:444
6293629965,1cyclictest0-21swapper/410:23:184
6293629965,1cyclictest0-21swapper/408:20:484
6293629965,1cyclictest0-21swapper/408:20:484
6293629964,2cyclictest0-21swapper/409:02:534
6293629963,3cyclictest923935-21kworker/u16:3+flush-8:010:34:504
6293629960,2cyclictest0-21swapper/409:04:454
6293629951,1cyclictest201rcu_preempt08:44:254
6293629950,4cyclictest904064-21chrt09:24:494
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional