You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 10:24
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Wed May 06, 2026 00:44:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13929521270,122pltrace0-21swapper/119:06:371
13929521230,119pltrace0-21swapper/519:06:455
13929521120,108pltrace0-21swapper/219:06:122
11123521070,103pltrace0-21swapper/319:05:143
11123521050,89pltrace0-21swapper/619:05:206
168082900,1sleep00-21swapper/021:05:230
168082900,1sleep00-21swapper/021:05:230
1400827344,10sleep40-21swapper/419:06:304
1112352730,56pltrace0-21swapper/719:05:037
1413526046,10sleep00-21swapper/019:08:180
2472752320,5pltrace0-21swapper/723:20:187
2815652250,1pltrace78-21ksoftirqd/721:30:007
1076952240,0pltrace78-21ksoftirqd/722:50:287
2922952220,5pltrace0-21swapper/722:30:127
2024152200,15pltrace20341-21taskset00:06:492
1591352190,13pltrace17706-21interrupts20:10:165
14469991914,4cyclictest0-21swapper/723:45:247
626652180,13pltrace54-21ksoftirqd/423:41:254
2922952180,5pltrace0-21swapper/722:28:357
2303052180,0pltrace0-21swapper/700:15:197
1446999182,9cyclictest13510-21ntpq21:00:177
1446299181,9cyclictest0-21swapper/519:15:415
1444199188,4cyclictest15-21ksoftirqd/022:58:490
14441991812,3cyclictest15-21ksoftirqd/022:05:180
2706352170,7pltrace0-21swapper/720:30:027
177652170,0pltrace0-21swapper/700:35:127
177652170,0pltrace0-21swapper/700:35:127
1446299170,2cyclictest0-21swapper/522:10:135
1446299170,12cyclictest0-21swapper/500:21:505
14458991711,5cyclictest0-21swapper/400:27:114
14458991711,5cyclictest0-21swapper/400:27:114
1445399170,1cyclictest0-21swapper/300:31:413
1445399170,1cyclictest0-21swapper/300:31:413
1444599171,2cyclictest0-21swapper/119:15:141
14441991712,2cyclictest15-21ksoftirqd/019:20:010
14441991711,2cyclictest2673-21ls00:35:130
14441991711,2cyclictest2673-21ls00:35:130
14441991710,4cyclictest15-21ksoftirqd/023:15:120
14441991710,4cyclictest15-21ksoftirqd/020:50:190
1184152170,11pltrace54-21ksoftirqd/423:54:194
626652160,3pltrace0-21swapper/723:41:127
2815652160,11pltrace54-21ksoftirqd/421:28:114
2706352160,6pltrace0-21swapper/120:27:501
2320152160,2pltrace0-21swapper/719:25:017
2258152160,12pltrace54-21ksoftirqd/421:16:454
2258152160,12pltrace54-21ksoftirqd/421:16:454
1678752160,11pltrace0-21swapper/619:13:026
1591352160,7pltrace0-21swapper/620:10:136
1446999167,8cyclictest1146-21snmpd19:31:487
1446999167,8cyclictest1146-21snmpd19:15:037
1446999160,11cyclictest0-21swapper/723:12:427
1446999160,1cyclictest0-21swapper/721:16:177
1446999160,1cyclictest0-21swapper/721:16:177
1446799160,8cyclictest0-21swapper/600:20:236
1446799160,15cyclictest0-21swapper/621:48:366
1446799160,15cyclictest0-21swapper/620:15:536
14462991610,5cyclictest0-21swapper/522:57:325
14462991610,5cyclictest0-21swapper/522:01:225
1446299161,3cyclictest0-21swapper/522:30:145
14458991611,3cyclictest0-21swapper/422:29:514
1445899161,4cyclictest0-21swapper/420:33:564
1445399160,15cyclictest0-21swapper/321:27:373
1445399160,15cyclictest0-21swapper/320:14:153
14450991610,3cyclictest0-21swapper/222:27:442
1445099160,8cyclictest0-21swapper/219:14:022
1444599165,5cyclictest439-21perl00:30:141
1444599165,5cyclictest439-21perl00:30:131
14445991611,4cyclictest3027-21hwlatdetect21:40:151
1444199169,4cyclictest15-21ksoftirqd/021:00:120
1444199169,4cyclictest15-21ksoftirqd/019:45:210
1444199167,6cyclictest15-21ksoftirqd/022:30:140
1444199167,6cyclictest15-21ksoftirqd/021:50:200
14441991610,2cyclictest26716-21cpuspeed_turbos00:20:140
1444199161,3cyclictest0-21swapper/020:58:590
1355252160,11pltrace54-21ksoftirqd/422:55:294
1698852150,9pltrace0-21swapper/621:06:216
1698852150,9pltrace0-21swapper/621:06:216
1591352150,4pltrace17376-21fschecks_time20:10:142
1446999156,8cyclictest1146-21snmpd20:20:017
14469991510,4cyclictest0-21swapper/723:37:477
14469991510,4cyclictest0-21swapper/723:18:127
1446999150,14cyclictest0-21swapper/720:17:217
1446999150,1cyclictest0-21swapper/722:20:437
14467991510,3cyclictest0-21swapper/621:58:526
1446799150,4cyclictest0-21swapper/622:12:286
1446799150,14cyclictest0-21swapper/622:02:436
1446299151,13cyclictest0-21swapper/519:33:175
1446299150,8cyclictest0-21swapper/520:44:215
1446299150,14cyclictest0-21swapper/523:36:395
1446299150,14cyclictest0-21swapper/522:17:255
1445899159,5cyclictest0-21swapper/421:45:254
1445899151,3cyclictest0-21swapper/422:48:004
1445899150,7cyclictest0-21swapper/421:54:054
1445899150,14cyclictest0-21swapper/422:04:504
1445899150,14cyclictest0-21swapper/421:07:034
1445899150,14cyclictest0-21swapper/421:07:034
1445899150,14cyclictest0-21swapper/419:53:184
1445899150,14cyclictest0-21swapper/419:12:534
1445899150,14cyclictest0-21swapper/400:01:464
1445899150,1cyclictest0-21swapper/422:19:344
1445399157,7cyclictest1146-21snmpd20:24:223
14453991510,4cyclictest0-21swapper/323:06:323
14453991510,4cyclictest0-21swapper/322:14:143
14453991510,4cyclictest0-21swapper/319:50:073
14453991510,4cyclictest0-21swapper/300:07:533
1445399150,1cyclictest0-21swapper/323:54:463
1445399150,1cyclictest0-21swapper/320:44:323
1445399150,1cyclictest0-21swapper/319:43:193
1445399150,1cyclictest0-21swapper/319:26:393
1445099150,7cyclictest0-21swapper/222:50:172
1445099150,7cyclictest0-21swapper/222:31:532
1445099150,14cyclictest0-21swapper/223:35:552
1445099150,14cyclictest0-21swapper/222:02:222
1445099150,14cyclictest0-21swapper/220:31:582
1444599155,4cyclictest23229-21sed22:15:211
1444599154,10cyclictest0-21swapper/122:39:211
14445991513,1cyclictest0-21swapper/123:35:161
1444599150,8cyclictest0-21swapper/122:49:201
1444599150,7cyclictest0-21swapper/123:19:231
1444599150,7cyclictest0-21swapper/121:07:141
1444599150,7cyclictest0-21swapper/121:07:141
1444599150,4cyclictest0-21swapper/121:59:211
1444599150,14cyclictest0-21swapper/123:30:141
1444599150,14cyclictest0-21swapper/120:36:521
1444599150,14cyclictest0-21swapper/119:31:411
1444199159,4cyclictest15-21ksoftirqd/020:20:120
1444199159,4cyclictest15-21ksoftirqd/000:10:220
1444199159,3cyclictest15-21ksoftirqd/020:00:110
1444199159,2cyclictest5423-21memory23:40:140
1444199158,4cyclictest15-21ksoftirqd/021:25:160
14441991512,2cyclictest914-21taskset23:34:220
14441991512,2cyclictest1146-21snmpd23:53:430
14441991511,2cyclictest15-21ksoftirqd/019:25:010
14441991510,3cyclictest0-21swapper/020:07:280
14441991510,2cyclictest15-21ksoftirqd/021:41:490
14441991510,2cyclictest15-21ksoftirqd/021:15:210
14441991510,2cyclictest15-21ksoftirqd/021:15:210
1444199151,7cyclictest0-21swapper/019:20:280
1444199150,8cyclictest0-21swapper/022:22:220
1444199150,8cyclictest0-21swapper/019:50:120
1444199150,4cyclictest0-21swapper/022:25:280
1444199150,14cyclictest0-21swapper/020:15:170
133182150,1sleep4521rcuc/423:55:124
131552150,7pltrace1664-21munin-run21:40:006
1138652150,0pltrace0-21swapper/720:56:537
3201552140,11pltrace54-21ksoftirqd/422:30:304
2258152140,8pltrace22686-21runrttasks21:16:522
2258152140,8pltrace22686-21runrttasks21:16:522
1870252140,2pltrace0-21swapper/720:10:547
177652140,2pltrace1146-21snmpd00:30:575
177652140,2pltrace1146-21snmpd00:30:575
175842140,0sleep50-21swapper/523:05:145
1446999149,4cyclictest0-21swapper/722:37:347
1446999148,5cyclictest0-21swapper/700:33:487
1446999148,5cyclictest0-21swapper/700:33:487
1446999144,9cyclictest1064-13audispd21:44:377
1446999143,10cyclictest29328-21hostname00:25:117
1446999143,10cyclictest29328-21hostname00:25:117
1446999142,10cyclictest7756-21sed22:45:217
1446999140,4cyclictest1146-21snmpd19:25:017
1446999140,13cyclictest0-21swapper/721:20:207
1446999140,13cyclictest0-21swapper/719:11:387
1446999140,12cyclictest17075-21/usr/sbin/munin23:05:207
14467991410,3cyclictest0-21swapper/622:52:086
1446799140,7cyclictest0-21swapper/623:22:546
1446799140,7cyclictest0-21swapper/623:14:306
1446799140,7cyclictest0-21swapper/620:23:206
1446799140,3cyclictest0-21swapper/623:45:246
1446799140,2cyclictest0-21swapper/619:42:496
14462991410,3cyclictest0-21swapper/521:20:575
1446299141,12cyclictest0-21swapper/519:20:145
1446299140,12cyclictest0-21swapper/523:45:015
1445899149,3cyclictest0-21swapper/419:55:364
1445899140,3cyclictest0-21swapper/421:12:524
1445899140,3cyclictest0-21swapper/421:12:524
1445899140,13cyclictest0-21swapper/423:30:544
1445899140,13cyclictest0-21swapper/420:15:124
1445399149,4cyclictest0-21swapper/322:39:393
1445399149,4cyclictest0-21swapper/319:46:433
1445399149,4cyclictest0-21swapper/300:15:493
1445399146,7cyclictest1146-21snmpd20:39:473
1445399146,7cyclictest1146-21snmpd19:32:443
1445399145,8cyclictest1146-21snmpd23:42:463
14453991410,4cyclictest0-21swapper/323:31:183
14453991410,4cyclictest0-21swapper/321:04:183
1445399140,12cyclictest17903-21cpuspeed_turbos21:10:123
1445399140,12cyclictest17903-21cpuspeed_turbos21:10:123
1445399140,12cyclictest11715-21grep22:55:123
1445399140,1cyclictest0-21swapper/322:08:453
14450991411,2cyclictest1055-17auditd23:30:012
1444599148,5cyclictest0-21swapper/100:05:231
1444599145,4cyclictest2133-21sort20:40:171
1444599142,7cyclictest0-21swapper/122:55:391
14445991410,3cyclictest0-21swapper/120:55:141
1444599141,12cyclictest0-21swapper/123:10:291
1444599141,12cyclictest0-21swapper/121:18:061
1444599141,12cyclictest0-21swapper/121:18:061
1444599140,7cyclictest0-21swapper/121:27:181
1444599140,13cyclictest0-21swapper/123:56:551
1444199149,3cyclictest0-21swapper/023:00:020
1444199149,2cyclictest15-21ksoftirqd/023:10:170
1444199149,2cyclictest15-21ksoftirqd/022:10:120
1444199149,2cyclictest15-21ksoftirqd/000:16:270
1444199148,4cyclictest15-21ksoftirqd/023:05:280
1444199146,5cyclictest15-21ksoftirqd/019:10:220
1444199144,6cyclictest15-21ksoftirqd/022:15:210
1444199142,5cyclictest15-21ksoftirqd/000:00:130
1444199142,4cyclictest15-21ksoftirqd/019:55:070
14441991411,2cyclictest6827-21sed21:45:220
14441991411,2cyclictest4279-21diskstats20:45:130
14441991411,2cyclictest29392-21/usr/sbin/munin00:25:130
14441991411,2cyclictest29392-21/usr/sbin/munin00:25:130
14441991410,2cyclictest15-21ksoftirqd/019:44:270
14441991410,1cyclictest266641switchtime20:25:200
131552140,3pltrace1582-21taskset21:39:035
1247752140,1pltrace0-21swapper/522:00:005
1184152140,5pltrace0-21swapper/723:53:597
690252130,5pltrace0-21swapper/521:49:165
2598652130,5pltrace0-21swapper/219:27:002
2581552130,5pltrace0-21swapper/600:18:406
2581552130,5pltrace0-21swapper/200:18:562
2581552130,0pltrace0-21swapper/700:20:147
2365552130,3pltrace24626-21cut22:20:112
1915452130,1pltrace0-21swapper/623:07:276
1446999138,4cyclictest0-21swapper/700:00:577
1446999135,7cyclictest1178-21runrttasks22:11:547
1446999135,7cyclictest1146-21snmpd00:14:047
1446999135,6cyclictest0-21swapper/720:38:597
1446999134,8cyclictest9436-21cat19:55:177
1446999134,8cyclictest1478-21lldpd22:09:207
1446999132,9cyclictest13965-21awk23:55:167
1446999131,11cyclictest0-21swapper/723:25:127
1446999131,11cyclictest0-21swapper/723:00:117
1446999130,5cyclictest2075-21chrt19:42:157
1446999130,12cyclictest28862-21/usr/sbin/munin21:30:147
1446999130,11cyclictest888-21/usr/sbin/munin20:40:207
14467991312,1cyclictest0-21swapper/622:45:136
1446799130,13cyclictest0-21swapper/620:30:216
1446799130,13cyclictest0-21swapper/600:05:126
1446799130,12cyclictest0-21swapper/623:44:376
1446799130,12cyclictest0-21swapper/619:36:186
1446299139,3cyclictest0-21swapper/523:01:265
1446299137,4cyclictest0-21swapper/523:14:065
1446299130,12cyclictest0-21swapper/523:32:245
1446299130,12cyclictest0-21swapper/521:30:035
1446299130,1cyclictest0-21swapper/522:25:175
14458991312,1cyclictest0-21swapper/423:39:424
14458991312,1cyclictest0-21swapper/422:44:404
1445899130,13cyclictest0-21swapper/423:45:514
1445899130,13cyclictest0-21swapper/422:10:144
1445899130,13cyclictest0-21swapper/419:41:034
1445899130,13cyclictest0-21swapper/419:37:394
1445899130,1cyclictest0-21swapper/421:58:414
1445399135,7cyclictest1178-21runrttasks19:24:363
1445399133,8cyclictest20606-21cat20:15:153
1445399132,9cyclictest16890-21processes00:00:203
1445399132,10cyclictest15114-21expr21:05:143
1445399132,10cyclictest15114-21expr21:05:143
1445399132,10cyclictest13782-21latency23:55:153
14453991312,1cyclictest0-21swapper/319:38:383
14453991312,1cyclictest0-21swapper/319:14:593
1445399130,8cyclictest1178-21runrttasks22:43:303
1445399130,8cyclictest0-21swapper/320:55:203
1445399130,5cyclictest31025-21threads00:25:223
1445399130,5cyclictest31025-21threads00:25:223
1445399130,12cyclictest1283-21pltrace-run00:35:293
1445399130,12cyclictest1283-21pltrace-run00:35:293
1445399130,12cyclictest1054-21irqbalance23:11:153
1445399130,11cyclictest7946-21expr23:45:153
1445399130,11cyclictest6678-21sed21:45:233
1445399130,11cyclictest32069-21expr21:35:143
1445399130,11cyclictest2788-21fschecks_time21:40:153
1445399130,11cyclictest25201-21unixbench-2d21:20:223
1445399130,11cyclictest17521-21munin-node19:15:013
1445399130,11cyclictest14863-21expr23:00:143
1445399130,11cyclictest11474-21expr20:00:123
1445099139,3cyclictest0-21swapper/223:33:222
1445099138,3cyclictest0-21swapper/220:44:562
1445099137,5cyclictest0-21swapper/223:20:222
1445099130,2cyclictest0-21swapper/221:40:462
1445099130,12cyclictest0-21swapper/221:56:282
1445099130,12cyclictest0-21swapper/219:20:352
1444599138,3cyclictest0-21swapper/121:30:151
1444599136,5cyclictest4049-21cpuspeed_turbos20:45:121
1444599136,5cyclictest23727-21chrt00:15:051
1444599136,5cyclictest14256-21cat22:00:151
1444599136,2cyclictest27877-21cut22:25:141
1444599135,6cyclictest9055-21expr22:50:121
1444599135,6cyclictest7035-21expr20:50:121
1444599135,6cyclictest5656-21pltrace_jitter23:40:171
1444599135,6cyclictest30812-21cat19:35:171
1444599135,6cyclictest29178-21/usr/sbin/munin20:30:171
1444599135,2cyclictest16316-21cpuspeed_turbos22:05:111
1444599133,5cyclictest0-21swapper/123:54:341
14445991311,1cyclictest0-21swapper/119:42:461
14445991310,2cyclictest14535-21latency_hist21:05:001
1444599130,12cyclictest0-21swapper/100:13:001
1444199139,2cyclictest15-21ksoftirqd/022:35:190
1444199138,2cyclictest15-21ksoftirqd/021:35:130
1444199138,2cyclictest15-21ksoftirqd/021:32:150
1444199137,5cyclictest0-21swapper/023:28:480
1444199135,5cyclictest15-21ksoftirqd/021:20:190
1444199135,5cyclictest15-21ksoftirqd/020:35:190
14441991310,2cyclictest29556-21aten_r4power_po19:35:110
14441991310,2cyclictest2531-21sensors20:40:200
14441991310,2cyclictest19266-21irqstats_TLB00:05:160
14441991310,2cyclictest18016-21nfs4_client20:10:170
14441991310,2cyclictest17737-21aten_r4power_cu21:10:100
14441991310,2cyclictest17737-21aten_r4power_cu21:10:100
14441991310,2cyclictest13668-21cat22:00:130
14441991310,2cyclictest1146-21snmpd20:30:140
1444199130,7cyclictest0-21swapper/022:47:520
1444199130,13cyclictest0-21swapper/022:50:130
1444199130,12cyclictest0-21swapper/023:22:120
752252120,4pltrace0-21swapper/519:51:135
581052120,4pltrace0-21swapper/620:47:266
3263552120,4pltrace0-21swapper/520:38:215
2815652120,5pltrace0-21swapper/521:28:165
2581552120,5pltrace0-21swapper/500:18:405
2365552120,1pltrace0-21swapper/622:20:196
194452120,5pltrace0-21swapper/519:40:305
1635252120,10pltrace0-21swapper/223:00:352
1446999126,5cyclictest21573-21/usr/sbin/munin22:15:167
1446999123,8cyclictest7874-21cut21:50:127
1446999123,8cyclictest11079-21basename20:00:127
1446999122,9cyclictest6899-21expr20:50:127
1446999121,10cyclictest0-21swapper/719:45:057
1446999120,5cyclictest622-21sh21:35:197
1446999120,5cyclictest5937-21cat19:50:147
1446999120,5cyclictest3758-21tr20:45:117
1446999120,5cyclictest3526-21cat22:40:117
1446999120,5cyclictest31234-21cut23:30:107
1446999120,5cyclictest18598-21fw_packets21:10:127
1446999120,5cyclictest18598-21fw_packets21:10:127
1446999120,5cyclictest18181-21hostname00:05:127
1446999120,5cyclictest15215-21cpuspeed_turbos21:05:137
1446999120,5cyclictest15215-21cpuspeed_turbos21:05:137
1446999120,5cyclictest13908-21cut22:00:147
1446999120,5cyclictest1178-21runrttasks22:55:097
1446999120,5cyclictest1146-21snmpd21:45:097
1446999120,5cyclictest1146-21snmpd20:25:107
1446999120,5cyclictest1146-21snmpd20:05:017
1446999120,5cyclictest1146-21snmpd19:35:037
1446999120,5cyclictest11037-21uniq21:55:137
1446799128,2cyclictest0-21swapper/620:42:326
1446799126,5cyclictest0-21swapper/622:43:506
1446799120,11cyclictest0-21swapper/619:19:366
1446299120,5cyclictest0-21swapper/520:46:515
1446299120,11cyclictest0-21swapper/522:41:025
1446299120,1cyclictest0-21swapper/500:10:315
1445899129,2cyclictest1178-21runrttasks19:19:524
1445899126,5cyclictest0-21swapper/423:01:444
1445899120,12cyclictest0-21swapper/421:20:384
1445899120,12cyclictest0-21swapper/420:11:484
1445899120,11cyclictest0-21swapper/400:11:354
1445899120,1cyclictest0-21swapper/423:26:544
1445399124,7cyclictest5001-21missed_timers20:45:173
1445399123,8cyclictest6402-21expr22:45:133
1445399123,8cyclictest3115-21timerandwakeup23:35:213
1445399122,9cyclictest8683-21expr19:55:123
1445399122,8cyclictest20510-21cut21:15:123
1445399122,8cyclictest20510-21cut21:15:123
1445399121,9cyclictest6932-21cat20:50:123
1445399121,10cyclictest30409-21expr22:30:133
1445399120,5cyclictest9227-21df22:50:143
1445399120,5cyclictest7817-21tr21:50:133
1445399120,5cyclictest28058-21tr20:30:113
1445399120,5cyclictest27918-21cut22:25:153
1445399120,5cyclictest26328-21tail22:20:223
1445399120,5cyclictest22036-21runrttasks00:10:153
1445399120,5cyclictest13393-21grep22:00:143
1445399120,5cyclictest1146-21snmpd22:15:343
1445399120,5cyclictest1081-21crond20:27:003
1445399120,4cyclictest29153-21gltestperf23:25:153
1445399120,11cyclictest26618-21latency23:20:163
1445399120,11cyclictest24244-21sensors23:15:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional