You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 12:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sun Dec 28, 2025 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5899521180,102pltrace0-21swapper/519:10:015
3115521070,98pltrace0-21swapper/319:05:193
896552990,3pltrace0-21swapper/221:07:362
2292752980,2pltrace0-21swapper/121:32:361
589952960,82pltrace0-21swapper/119:08:071
311552930,62pltrace0-21swapper/719:05:257
311552860,77pltrace0-21swapper/619:05:406
311552750,58pltrace0-21swapper/219:07:242
578925945,10sleep40-21swapper/419:06:214
575825945,10sleep00-21swapper/019:05:530
2571252330,0pltrace0-21swapper/721:40:137
6269992421,2cyclictest32247-21ls21:50:157
6269992421,2cyclictest32247-21ls21:50:157
624399240,23cyclictest0-21swapper/223:38:542
626099230,15cyclictest0-21swapper/519:47:345
624099230,22cyclictest0-21swapper/119:36:161
6255992120,1cyclictest0-21swapper/400:06:404
167952200,4pltrace0-21swapper/721:55:197
167952200,4pltrace0-21swapper/721:55:197
6243991914,4cyclictest0-21swapper/200:06:402
3129052190,13pltrace1286-21sed21:50:240
3129052190,13pltrace1286-21sed21:50:240
626499188,4cyclictest1146-21snmpd22:28:396
626499180,11cyclictest0-21swapper/620:00:536
626999171,12cyclictest191rcu_preempt22:27:007
626499179,4cyclictest70-21ksoftirqd/619:45:156
626499175,5cyclictest0-21swapper/600:30:006
626499172,3cyclictest0-21swapper/619:12:246
6260991714,2cyclictest0-21swapper/519:19:125
6260991711,3cyclictest0-21swapper/521:37:365
626099171,8cyclictest0-21swapper/523:56:545
626099171,4cyclictest0-21swapper/521:51:055
626099171,4cyclictest0-21swapper/521:51:055
6255991711,5cyclictest0-21swapper/420:06:084
625599171,14cyclictest0-21swapper/422:20:004
6240991712,3cyclictest0-21swapper/119:16:141
624099171,15cyclictest0-21swapper/119:14:391
2228952170,10pltrace23057-21cpuspeed_turbos23:30:130
1285752170,0pltrace0-21swapper/722:14:367
626999161,8cyclictest0-21swapper/722:32:107
626999161,7cyclictest0-21swapper/723:54:577
626499169,2cyclictest70-21ksoftirqd/623:10:156
626499168,4cyclictest70-21ksoftirqd/622:56:016
6264991613,2cyclictest28754-21fschecks_time20:45:156
6264991613,2cyclictest23872-21taskset22:31:026
6264991610,3cyclictest70-21ksoftirqd/600:30:176
626499160,15cyclictest0-21swapper/622:47:346
626499160,12cyclictest70-21ksoftirqd/623:06:036
6260991610,3cyclictest0-21swapper/519:28:575
626099160,15cyclictest0-21swapper/522:51:105
6255991610,4cyclictest0-21swapper/419:41:464
625599161,8cyclictest0-21swapper/419:35:404
6250991611,3cyclictest0-21swapper/321:39:143
6250991610,5cyclictest0-21swapper/320:11:113
6250991610,3cyclictest0-21swapper/319:37:313
6243991611,3cyclictest0-21swapper/220:26:472
6240991613,2cyclictest0-21swapper/122:20:251
6240991611,3cyclictest0-21swapper/122:53:221
6240991610,3cyclictest0-21swapper/100:33:561
624099161,4cyclictest0-21swapper/121:37:531
624099161,14cyclictest0-21swapper/123:50:151
624099160,15cyclictest0-21swapper/122:26:181
623799161,8cyclictest0-21swapper/023:37:270
623799161,8cyclictest0-21swapper/020:44:310
623799161,14cyclictest0-21swapper/023:21:440
275252160,4pltrace0-21swapper/722:55:337
2571252160,4pltrace0-21swapper/721:40:007
2507552160,0pltrace0-21swapper/723:37:217
2461052160,12pltrace54-21ksoftirqd/420:40:164
835752150,11pltrace0-21swapper/023:03:040
626999157,4cyclictest78-21ksoftirqd/723:15:147
626999150,8cyclictest0-21swapper/720:14:297
626999150,4cyclictest0-21swapper/720:57:007
626999150,14cyclictest0-21swapper/721:00:307
626999150,14cyclictest0-21swapper/720:00:247
626999150,14cyclictest0-21swapper/700:11:047
626499159,3cyclictest70-21ksoftirqd/620:35:166
626499159,3cyclictest70-21ksoftirqd/619:20:046
626499159,2cyclictest70-21ksoftirqd/623:30:176
626499159,2cyclictest32595-21perl21:50:166
626499159,2cyclictest32595-21perl21:50:166
626499159,2cyclictest0-21swapper/600:35:236
626499155,6cyclictest70-21ksoftirqd/622:05:166
626499155,4cyclictest31248-21cut23:45:146
626499154,6cyclictest70-21ksoftirqd/623:55:186
626499154,6cyclictest2554-21head23:50:136
6264991513,1cyclictest20557-21taskset00:22:276
6264991513,1cyclictest10867-21fschecks_time19:15:156
6264991512,2cyclictest27656-21grep22:40:136
6264991512,2cyclictest26756-21sensors20:40:206
6264991512,2cyclictest15740-21timerandwakeup20:20:216
6264991512,2cyclictest12896-21awk21:15:136
6264991512,2cyclictest1178-21runrttasks22:14:516
6264991512,1cyclictest0-21swapper/623:25:246
6264991510,3cyclictest0-21swapper/620:33:426
626499150,4cyclictest0-21swapper/620:53:516
626499150,14cyclictest0-21swapper/619:37:346
6260991512,1cyclictest0-21swapper/519:10:215
6260991511,1cyclictest0-21swapper/521:24:105
6260991510,4cyclictest0-21swapper/520:30:005
626099151,12cyclictest0-21swapper/523:42:365
626099150,8cyclictest0-21swapper/523:00:015
626099150,8cyclictest0-21swapper/522:37:345
626099150,7cyclictest0-21swapper/520:41:325
626099150,7cyclictest0-21swapper/500:28:195
626099150,3cyclictest0-21swapper/521:11:015
626099150,14cyclictest0-21swapper/523:15:555
626099150,14cyclictest0-21swapper/520:51:415
6255991511,2cyclictest0-21swapper/422:14:014
6255991510,3cyclictest0-21swapper/422:01:214
625599151,3cyclictest0-21swapper/423:28:534
625599151,3cyclictest0-21swapper/421:47:054
625599150,9cyclictest0-21swapper/423:13:144
625599150,7cyclictest0-21swapper/420:31:384
625599150,7cyclictest0-21swapper/400:39:564
625599150,14cyclictest0-21swapper/423:04:384
625599150,14cyclictest0-21swapper/422:05:314
625599150,14cyclictest0-21swapper/421:59:244
625599150,14cyclictest0-21swapper/421:59:244
625599150,14cyclictest0-21swapper/419:16:124
6250991510,3cyclictest0-21swapper/323:16:173
625099150,8cyclictest0-21swapper/322:00:423
625099150,7cyclictest0-21swapper/323:58:283
625099150,14cyclictest0-21swapper/300:32:503
625099150,13cyclictest0-21swapper/322:46:223
6243991510,3cyclictest0-21swapper/219:42:162
624399150,8cyclictest0-21swapper/200:24:212
624399150,4cyclictest0-21swapper/220:34:302
624399150,4cyclictest0-21swapper/220:11:052
624399150,14cyclictest0-21swapper/223:55:302
624399150,14cyclictest0-21swapper/220:41:212
624399150,14cyclictest0-21swapper/220:16:052
624399150,14cyclictest0-21swapper/200:14:492
6240991510,3cyclictest0-21swapper/120:23:081
624099150,8cyclictest7530-21ntpq23:00:161
624099150,8cyclictest0-21swapper/119:44:131
624099150,14cyclictest0-21swapper/122:08:501
6237991513,1cyclictest0-21swapper/019:15:140
623799150,8cyclictest0-21swapper/000:26:220
623799150,8cyclictest0-21swapper/000:24:490
623799150,3cyclictest0-21swapper/022:19:000
623799150,14cyclictest0-21swapper/023:53:030
623799150,14cyclictest0-21swapper/023:06:500
623799150,14cyclictest0-21swapper/000:12:310
617152150,11pltrace0-21swapper/021:04:220
3129052150,12pltrace54-21ksoftirqd/421:50:204
3129052150,12pltrace54-21ksoftirqd/421:50:204
2740052150,2pltrace0-21swapper/720:45:137
2461052150,10pltrace27240-21taskset20:40:371
2228952150,2pltrace23391-21diskstats23:30:141
2013952150,3pltrace54-21ksoftirqd/421:29:524
876252140,3pltrace8774-21chrt19:12:400
626999149,4cyclictest0-21swapper/722:20:217
626999149,2cyclictest78-21ksoftirqd/700:30:187
626999148,3cyclictest0-21swapper/723:33:427
626999148,2cyclictest78-21ksoftirqd/723:40:147
626999148,2cyclictest78-21ksoftirqd/720:40:207
626999147,6cyclictest0-21swapper/719:33:377
626999146,7cyclictest10285-21cpuspeed_turbos00:05:147
626999145,5cyclictest15550-21processes20:20:197
626999144,6cyclictest78-21ksoftirqd/700:00:167
626999143,7cyclictest78-21ksoftirqd/700:22:277
626999140,6cyclictest0-21swapper/723:50:007
626499149,2cyclictest70-21ksoftirqd/620:15:116
626499149,2cyclictest70-21ksoftirqd/600:10:176
626499148,3cyclictest70-21ksoftirqd/619:40:146
626499148,3cyclictest70-21ksoftirqd/600:16:446
626499148,3cyclictest191rcu_preempt23:20:436
626499148,3cyclictest191rcu_preempt20:28:056
626499148,2cyclictest70-21ksoftirqd/622:50:136
626499144,6cyclictest70-21ksoftirqd/621:00:516
6264991412,1cyclictest0-21swapper/623:40:126
6264991412,1cyclictest0-21swapper/600:10:006
6264991411,2cyclictest6766-21irqstats_TLB20:05:196
6264991411,2cyclictest30109-21expr19:50:136
6264991411,2cyclictest2874-21sort21:55:156
6264991411,2cyclictest2874-21sort21:55:156
6264991411,2cyclictest26471-21fschecks_time23:35:146
6264991410,3cyclictest0-21swapper/600:00:056
626099149,3cyclictest0-21swapper/523:24:375
626099149,3cyclictest0-21swapper/500:17:225
626099148,3cyclictest0-21swapper/522:23:065
626099140,8cyclictest0-21swapper/522:00:315
626099140,8cyclictest0-21swapper/521:43:355
626099140,8cyclictest0-21swapper/521:15:435
626099140,12cyclictest0-21swapper/522:14:425
625599140,2cyclictest0-21swapper/423:35:124
625599140,13cyclictest0-21swapper/420:55:144
625599140,13cyclictest0-21swapper/420:45:024
625599140,13cyclictest0-21swapper/400:18:064
625099149,4cyclictest0-21swapper/321:56:173
625099149,4cyclictest0-21swapper/321:56:173
625099148,5cyclictest0-21swapper/321:52:383
625099148,5cyclictest0-21swapper/321:52:383
625099140,8cyclictest0-21swapper/300:01:493
625099140,14cyclictest0-21swapper/322:55:223
625099140,14cyclictest0-21swapper/319:10:193
624399149,3cyclictest0-21swapper/222:40:252
624399148,3cyclictest0-21swapper/223:20:022
6243991412,1cyclictest0-21swapper/223:05:132
624399141,12cyclictest0-21swapper/223:03:032
624399140,7cyclictest0-21swapper/219:23:332
624399140,13cyclictest0-21swapper/219:35:232
624099149,3cyclictest0-21swapper/121:53:021
624099149,3cyclictest0-21swapper/121:53:021
6240991412,1cyclictest0-21swapper/120:50:141
6240991411,2cyclictest0-21swapper/119:49:291
624099141,12cyclictest0-21swapper/121:13:191
624099140,7cyclictest0-21swapper/121:26:191
624099140,1cyclictest0-21swapper/120:15:161
623799140,8cyclictest0-21swapper/019:53:450
623799140,6cyclictest0-21swapper/020:20:410
623799140,1cyclictest0-21swapper/020:15:180
382852140,10pltrace54-21ksoftirqd/423:54:524
2400252140,6pltrace0-21swapper/522:34:565
2228952140,9pltrace23857-21idleruntime23:30:153
2228952140,7pltrace23952-21irqstats_TLB23:30:162
2059052140,9pltrace54-21ksoftirqd/400:26:524
626999138,2cyclictest78-21ksoftirqd/720:30:247
626999137,5cyclictest0-21swapper/720:17:147
626999137,2cyclictest78-21ksoftirqd/719:56:517
626999136,6cyclictest0-21swapper/722:44:457
626999136,3cyclictest78-21ksoftirqd/722:05:137
626999136,1cyclictest191rcu_preempt00:29:297
626999134,5cyclictest78-21ksoftirqd/721:25:157
626999133,6cyclictest78-21ksoftirqd/722:50:177
626999133,6cyclictest78-21ksoftirqd/721:47:307
626999133,6cyclictest78-21ksoftirqd/720:50:147
626999132,7cyclictest78-21ksoftirqd/723:25:247
6269991311,1cyclictest8723-21unixbench_singl21:05:227
6269991310,2cyclictest12285-21seq19:16:267
626999130,13cyclictest0-21swapper/719:45:347
626999130,12cyclictest0-21swapper/722:36:287
626999130,12cyclictest0-21swapper/720:25:057
626999130,12cyclictest0-21swapper/700:35:047
626499137,2cyclictest70-21ksoftirqd/622:20:016
626499137,2cyclictest70-21ksoftirqd/621:30:126
626499136,4cyclictest70-21ksoftirqd/621:23:256
626499134,4cyclictest26707-21cat21:40:126
626499133,7cyclictest70-21ksoftirqd/623:15:206
626499133,7cyclictest70-21ksoftirqd/623:00:116
626499132,7cyclictest70-21ksoftirqd/622:00:156
626499132,7cyclictest70-21ksoftirqd/621:45:186
626499132,5cyclictest70-21ksoftirqd/622:35:136
6264991310,2cyclictest18521-21tune2fs21:25:146
6264991310,2cyclictest1200-21cat19:55:176
6264991310,2cyclictest11104-21grep21:10:196
626099139,3cyclictest1988-21grep19:55:225
626099138,3cyclictest0-21swapper/522:43:285
626099136,6cyclictest0-21swapper/523:27:235
6260991312,1cyclictest0-21swapper/522:15:115
6260991310,2cyclictest11203-21cat00:05:185
626099130,12cyclictest0-21swapper/521:04:015
626099130,12cyclictest0-21swapper/500:04:355
625599138,3cyclictest0-21swapper/421:36:144
6255991312,1cyclictest0-21swapper/419:45:174
625599130,12cyclictest0-21swapper/419:12:064
625599130,1cyclictest0-21swapper/423:20:184
625599130,1cyclictest0-21swapper/423:16:054
625099130,12cyclictest0-21swapper/300:37:383
624399137,3cyclictest0-21swapper/220:21:352
6243991312,1cyclictest0-21swapper/221:34:332
624399130,12cyclictest0-21swapper/219:25:152
6240991311,1cyclictest0-21swapper/121:02:311
624099130,12cyclictest0-21swapper/123:22:351
623799138,3cyclictest0-21swapper/019:46:060
623799137,5cyclictest0-21swapper/019:32:580
623799133,2cyclictest0-21swapper/021:57:350
623799133,2cyclictest0-21swapper/021:57:350
6237991310,2cyclictest1146-21snmpd21:17:250
623799130,13cyclictest0-21swapper/021:37:330
617152130,6pltrace54-21ksoftirqd/421:07:164
51052130,7pltrace0-21swapper/020:54:310
3129052130,2pltrace1486-21users21:50:252
3129052130,2pltrace1486-21users21:50:252
2400252130,4pltrace0-21swapper/222:34:582
2337652130,8pltrace54-21ksoftirqd/400:30:414
1780452130,6pltrace0-21swapper/200:19:112
1392952130,5pltrace0-21swapper/123:16:481
98552120,0pltrace0-21swapper/223:52:292
944652120,7pltrace0-21swapper/100:07:221
728352120,8pltrace7838-21/usr/sbin/munin22:05:143
728352120,4pltrace0-21swapper/222:05:322
626999129,2cyclictest6732-21if_eth020:05:177
626999129,2cyclictest6724-21fschecks_count23:00:127
626999129,2cyclictest5025-21/usr/sbin/munin22:00:237
626999129,2cyclictest30861-21sed19:50:167
626999129,2cyclictest25587-21ps19:40:217
626999129,2cyclictest23475-21taskset19:37:047
626999129,2cyclictest13174-21fschecks_time21:15:137
626999129,2cyclictest10650-21irqstats_LOC21:10:157
626999126,5cyclictest0-21swapper/721:30:137
626999124,5cyclictest78-21ksoftirqd/719:25:137
626999123,5cyclictest78-21ksoftirqd/723:20:117
626999122,7cyclictest78-21ksoftirqd/722:45:107
626999122,7cyclictest78-21ksoftirqd/700:15:117
6269991211,1cyclictest0-21swapper/723:05:027
626999120,7cyclictest4981-21tune2fs23:55:147
626499129,2cyclictest9873-21ntp_offset20:10:196
626499129,2cyclictest6758-21wget21:05:116
626499129,2cyclictest23837-21cstates21:35:146
626499129,2cyclictest18973-21cstates19:30:136
626499129,2cyclictest16673-21cut19:25:146
626499123,4cyclictest16697-21cstates22:20:166
6264991210,1cyclictest2938-21sed20:55:236
626099127,2cyclictest0-21swapper/523:33:185
626099123,8cyclictest0-21swapper/520:38:595
625599126,2cyclictest54-21ksoftirqd/420:25:484
6255991211,1cyclictest0-21swapper/420:04:334
625599120,12cyclictest0-21swapper/422:22:204
625099128,2cyclictest0-21swapper/319:42:463
6250991211,1cyclictest0-21swapper/322:38:323
6250991211,1cyclictest0-21swapper/300:26:413
625099120,12cyclictest0-21swapper/319:15:253
625099120,11cyclictest0-21swapper/323:25:103
625099120,1cyclictest0-21swapper/323:06:043
625099120,1cyclictest0-21swapper/319:21:303
624399127,2cyclictest0-21swapper/219:11:482
624399126,5cyclictest0-21swapper/220:52:182
624399126,5cyclictest0-21swapper/219:30:042
624399120,5cyclictest0-21swapper/219:52:292
624399120,11cyclictest0-21swapper/221:25:092
624099126,3cyclictest0-21swapper/122:58:281
623799128,3cyclictest1146-21snmpd19:58:400
623799127,4cyclictest0-21swapper/022:21:390
623799126,5cyclictest0-21swapper/023:49:480
623799126,5cyclictest0-21swapper/022:12:360
623799122,8cyclictest0-21swapper/020:03:500
6237991211,1cyclictest0-21swapper/022:51:320
6237991210,1cyclictest0-21swapper/000:19:270
623799120,11cyclictest0-21swapper/021:44:110
623799120,1cyclictest0-21swapper/019:36:460
3236452120,5pltrace0-21swapper/022:49:150
3065352120,5pltrace0-21swapper/323:42:413
2632352120,1pltrace0-21swapper/019:42:430
2337652120,1pltrace0-21swapper/000:30:580
1285752120,0pltrace0-21swapper/722:15:527
626999118,2cyclictest6703-21cat19:10:147
626999118,2cyclictest15483-21cstates21:20:167
626999118,2cyclictest13636-21fschecks_count19:20:157
626999118,2cyclictest11689-21taskset23:10:027
626999116,1cyclictest191rcu_preempt20:35:027
626099119,1cyclictest0-21swapper/520:55:185
626099115,5cyclictest0-21swapper/500:13:165
626099115,3cyclictest0-21swapper/523:37:425
6260991110,1cyclictest0-21swapper/523:07:095
626099110,9cyclictest0-21swapper/519:39:465
626099110,11cyclictest0-21swapper/521:05:045
626099110,11cyclictest0-21swapper/520:01:375
626099110,11cyclictest0-21swapper/519:33:075
626099110,1cyclictest0-21swapper/523:00:415
626099110,1cyclictest0-21swapper/522:09:515
626099110,1cyclictest0-21swapper/521:56:245
626099110,1cyclictest0-21swapper/521:56:245
625599118,2cyclictest1055-17auditd23:10:004
625599117,3cyclictest0-21swapper/423:47:154
625599117,3cyclictest0-21swapper/419:55:554
6255991110,1cyclictest0-21swapper/422:53:354
6255991110,1cyclictest0-21swapper/422:36:444
6255991110,1cyclictest0-21swapper/420:36:174
6255991110,1cyclictest0-21swapper/419:31:124
625599110,11cyclictest0-21swapper/422:58:554
625599110,11cyclictest0-21swapper/422:44:024
625599110,11cyclictest0-21swapper/421:14:004
625599110,11cyclictest0-21swapper/420:15:434
625599110,1cyclictest0-21swapper/420:54:064
625099119,1cyclictest0-21swapper/320:05:223
625099115,5cyclictest0-21swapper/321:15:343
625099112,8cyclictest0-21swapper/321:40:313
6250991110,1cyclictest0-21swapper/323:12:323
6250991110,1cyclictest0-21swapper/321:20:303
6250991110,1cyclictest0-21swapper/319:30:053
6250991110,1cyclictest0-21swapper/319:29:163
625099110,11cyclictest0-21swapper/323:35:023
625099110,11cyclictest0-21swapper/321:27:373
625099110,11cyclictest0-21swapper/319:57:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional