You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 17:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Thu Jul 23, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23028521390,126pltrace0-21swapper/507:05:205
23028521350,131pltrace0-21swapper/207:05:172
23028521250,116pltrace0-21swapper/307:05:063
23028521160,112pltrace0-21swapper/107:07:011
2302852950,67pltrace0-21swapper/607:05:136
2302852860,77pltrace0-21swapper/707:05:057
229242800,0sleep00-21swapper/011:25:210
2572625844,10sleep00-21swapper/007:08:270
2307925742,10sleep40-21swapper/407:05:014
2393852520,9pltrace78-21ksoftirqd/712:39:587
663852450,6pltrace78-21ksoftirqd/709:19:597
62322430,0sleep60-21swapper/611:43:056
582352430,13pltrace78-21ksoftirqd/712:22:287
2889352430,6pltrace78-21ksoftirqd/709:13:437
799152410,6pltrace78-21ksoftirqd/711:13:317
663852410,7pltrace78-21ksoftirqd/709:22:537
40452410,1sleep00-21swapper/012:15:220
995252400,7pltrace78-21ksoftirqd/710:01:357
995252400,7pltrace78-21ksoftirqd/710:01:357
1627952400,5pltrace78-21ksoftirqd/711:19:387
490552380,6pltrace78-21ksoftirqd/710:31:247
2928052370,1pltrace29284-21pltrace-run12:09:271
1883552370,9pltrace78-21ksoftirqd/710:13:307
2010552350,8pltrace78-21ksoftirqd/709:34:507
382952340,1pltrace6985-21ssh11:06:367
2010552340,8pltrace78-21ksoftirqd/709:38:127
1577852340,8pltrace78-21ksoftirqd/711:55:477
26115993129,1cyclictest78-21ksoftirqd/710:15:017
1031352310,6pltrace78-21ksoftirqd/712:25:397
221152300,6pltrace78-21ksoftirqd/711:41:437
1129152300,5pltrace78-21ksoftirqd/711:49:587
1479452290,5pltrace78-21ksoftirqd/712:34:117
2817352280,6pltrace78-21ksoftirqd/710:22:117
1123252280,6pltrace78-21ksoftirqd/709:25:287
126262260,2sleep454-21ksoftirqd/409:25:134
174862210,0sleep50-21swapper/511:20:135
26083992014,5cyclictest0-21swapper/109:29:441
1627952200,0pltrace0-21swapper/311:19:273
553252190,0pltrace0-21swapper/709:54:327
2610199190,12cyclictest25195-21sh09:40:065
1627952190,5pltrace0-21swapper/711:20:067
2610199180,17cyclictest0-21swapper/507:20:215
2610199180,13cyclictest0-21swapper/507:45:125
26092991815,2cyclictest1178-21runrttasks10:33:493
26092991813,2cyclictest46-21ksoftirqd/311:08:523
1129152180,0pltrace0-21swapper/711:50:127
382952170,12pltrace0-21swapper/211:07:332
26110991714,2cyclictest0-21swapper/609:18:226
2611099171,4cyclictest14891-21ssh12:29:406
26098991711,5cyclictest0-21swapper/409:32:164
26092991715,1cyclictest3122-21taskset11:02:343
26092991714,2cyclictest1283-21pltrace-run08:54:263
26092991714,2cyclictest1178-21runrttasks08:49:253
26092991714,2cyclictest1146-21snmpd12:13:343
26092991714,2cyclictest1146-21snmpd08:37:533
26092991713,2cyclictest46-21ksoftirqd/308:05:113
26092991711,5cyclictest0-21swapper/307:43:353
2609299171,8cyclictest0-21swapper/312:30:103
2609299171,8cyclictest0-21swapper/308:43:583
2608899171,8cyclictest0-21swapper/209:44:002
2608899170,12cyclictest0-21swapper/211:22:302
26083991711,4cyclictest0-21swapper/108:54:191
2608399171,15cyclictest0-21swapper/107:22:031
2607999171,5cyclictest0-21swapper/010:32:150
2607999170,15cyclictest0-21swapper/009:09:400
2476752170,12pltrace28562-21ssh12:07:462
1565752170,10pltrace19008-21ssh09:31:482
26115991611,4cyclictest0-21swapper/711:38:357
26110991611,2cyclictest0-21swapper/608:32:056
2611099161,14cyclictest0-21swapper/612:13:366
2610199164,11cyclictest0-21swapper/511:40:575
2610199160,15cyclictest0-21swapper/508:13:445
2610199160,15cyclictest0-21swapper/507:38:015
2610199160,1cyclictest0-21swapper/512:35:115
2609899161,4cyclictest0-21swapper/409:57:034
2609899161,4cyclictest0-21swapper/409:57:034
2609899161,14cyclictest0-21swapper/411:48:284
2609899160,15cyclictest0-21swapper/411:34:214
26092991614,1cyclictest1146-21snmpd09:33:033
26092991613,2cyclictest1178-21runrttasks12:02:473
26092991613,2cyclictest1146-21snmpd09:46:183
26092991613,2cyclictest10290-21expr10:35:123
26092991612,2cyclictest46-21ksoftirqd/310:48:233
26092991612,2cyclictest46-21ksoftirqd/309:10:233
26092991611,3cyclictest0-21swapper/309:05:193
26092991610,5cyclictest0-21swapper/307:16:433
26092991610,4cyclictest46-21ksoftirqd/311:25:183
2609299161,8cyclictest0-21swapper/308:19:423
2609299160,15cyclictest0-21swapper/308:56:343
2608899161,14cyclictest0-21swapper/211:01:352
2608899161,14cyclictest0-21swapper/208:03:322
2608899160,4cyclictest0-21swapper/211:54:432
2608899160,15cyclictest0-21swapper/210:56:102
2608399161,7cyclictest0-21swapper/110:53:381
2608399160,15cyclictest0-21swapper/107:45:161
2607999161,14cyclictest0-21swapper/011:13:190
102752160,12pltrace54-21ksoftirqd/409:50:444
269852150,10pltrace78-21ksoftirqd/707:25:197
2611599159,5cyclictest0-21swapper/707:43:197
2611599150,13cyclictest0-21swapper/712:01:457
2611099159,5cyclictest0-21swapper/608:28:236
2611099150,7cyclictest0-21swapper/609:14:556
2611099150,4cyclictest0-21swapper/611:39:566
2611099150,14cyclictest0-21swapper/610:19:136
2610199151,7cyclictest0-21swapper/508:53:155
2610199150,8cyclictest0-21swapper/509:56:565
2610199150,8cyclictest0-21swapper/509:56:565
2610199150,5cyclictest0-21swapper/512:02:125
2610199150,11cyclictest0-21swapper/511:35:575
2609899151,7cyclictest0-21swapper/412:20:344
2609899151,7cyclictest0-21swapper/408:01:534
2609899151,3cyclictest0-21swapper/412:15:574
2609899150,8cyclictest0-21swapper/411:09:114
2609899150,7cyclictest0-21swapper/410:05:274
2609899150,14cyclictest0-21swapper/412:27:584
2609899150,14cyclictest0-21swapper/407:45:224
2609899150,14cyclictest0-21swapper/407:31:014
2609899150,14cyclictest0-21swapper/407:10:284
26092991513,1cyclictest28821-21cut08:10:143
26092991513,1cyclictest15242-21cpuspeed_turbos10:05:103
26092991512,2cyclictest8368-21perl11:45:153
26092991512,2cyclictest6487-21/usr/sbin/munin12:20:113
26092991512,2cyclictest5358-21taskset11:40:593
26092991512,2cyclictest20468-21expr12:35:103
26092991512,2cyclictest19738-21cpuspeed_turbos10:10:123
26092991512,2cyclictest18300-21missed_timers07:50:173
26092991512,2cyclictest15876-21rdmsr07:45:233
26092991512,2cyclictest13967-21unixbench_singl09:25:233
26092991512,2cyclictest1178-21runrttasks09:57:353
26092991512,2cyclictest1178-21runrttasks09:57:353
26092991512,2cyclictest11077-21cpu12:25:113
26092991512,2cyclictest10142-21awk07:35:193
2609299150,14cyclictest0-21swapper/309:23:563
2609299150,14cyclictest0-21swapper/308:28:183
26088991510,3cyclictest0-21swapper/207:11:132
2608899151,8cyclictest0-21swapper/211:55:312
2608899151,8cyclictest0-21swapper/209:55:512
2608899151,8cyclictest0-21swapper/209:55:512
2608899150,4cyclictest0-21swapper/209:48:512
2608899150,4cyclictest0-21swapper/208:08:522
2608899150,14cyclictest0-21swapper/208:27:092
2608899150,14cyclictest0-21swapper/207:45:132
2608899150,14cyclictest0-21swapper/207:40:472
2608899150,14cyclictest0-21swapper/207:35:142
2608399159,5cyclictest0-21swapper/110:16:431
26083991511,3cyclictest0-21swapper/112:27:561
2608399151,3cyclictest0-21swapper/110:40:551
2608399150,7cyclictest0-21swapper/111:24:241
2608399150,7cyclictest0-21swapper/110:04:291
2608399150,7cyclictest0-21swapper/110:04:291
2608399150,7cyclictest0-21swapper/108:05:051
2608399150,2cyclictest0-21swapper/112:35:101
2608399150,14cyclictest0-21swapper/112:13:041
2608399150,14cyclictest0-21swapper/108:20:211
2608399150,14cyclictest0-21swapper/107:33:081
2607999150,8cyclictest0-21swapper/007:57:460
2607999150,7cyclictest0-21swapper/010:01:350
2607999150,7cyclictest0-21swapper/010:01:350
2607999150,2cyclictest0-21swapper/012:35:110
2607999150,14cyclictest0-21swapper/010:45:370
2607999150,14cyclictest0-21swapper/009:37:090
2607999150,14cyclictest0-21swapper/007:43:210
2607999150,13cyclictest0-21swapper/007:30:060
2476752150,2pltrace28558-21ssh12:07:446
1491952150,13pltrace54-21ksoftirqd/408:45:034
1439752150,4pltrace0-21swapper/010:09:220
1104952150,12pltrace54-21ksoftirqd/407:41:064
59092140,0sleep50-21swapper/509:18:095
2611599148,5cyclictest0-21swapper/707:58:197
26115991413,1cyclictest0-21swapper/710:05:207
26115991411,2cyclictest1064-13audispd10:45:007
26115991410,3cyclictest0-21swapper/708:30:357
2611599140,13cyclictest0-21swapper/707:45:207
2611599140,1cyclictest0-21swapper/711:01:517
2611599140,1cyclictest0-21swapper/710:45:207
2611599140,1cyclictest0-21swapper/708:50:237
2611099149,3cyclictest0-21swapper/612:35:106
26110991413,1cyclictest0-21swapper/609:58:206
26110991413,1cyclictest0-21swapper/609:58:206
26110991412,1cyclictest0-21swapper/610:46:256
2611099141,12cyclictest0-21swapper/612:20:196
2611099140,7cyclictest0-21swapper/611:19:296
2611099140,14cyclictest0-21swapper/612:16:166
2611099140,1cyclictest0-21swapper/611:59:366
2611099140,1cyclictest0-21swapper/611:14:066
26101991411,2cyclictest0-21swapper/508:26:225
2610199140,13cyclictest0-21swapper/511:28:415
2610199140,13cyclictest0-21swapper/509:35:145
2610199140,1cyclictest0-21swapper/511:10:175
26098991412,1cyclictest0-21swapper/412:04:494
26098991412,1cyclictest0-21swapper/411:44:064
26098991412,1cyclictest0-21swapper/410:10:034
26098991412,1cyclictest0-21swapper/410:04:134
26098991412,1cyclictest0-21swapper/410:04:134
26098991412,1cyclictest0-21swapper/409:14:104
2609899141,12cyclictest0-21swapper/410:19:374
2609899141,12cyclictest0-21swapper/409:24:374
2609899140,13cyclictest0-21swapper/410:54:444
2609899140,1cyclictest0-21swapper/412:09:124
2609899140,1cyclictest0-21swapper/411:20:484
2609899140,1cyclictest0-21swapper/410:42:064
2609899140,1cyclictest0-21swapper/409:45:354
2609299146,5cyclictest4124-21idleruntime07:25:153
26092991411,2cyclictest8401-21/usr/sbin/munin08:30:183
26092991411,2cyclictest6894-21wget07:30:153
26092991411,2cyclictest29309-21expr10:20:113
26092991411,2cyclictest27297-21unixbench_singl09:40:233
26092991411,2cyclictest25960-21unixbench_multi10:15:233
26092991411,2cyclictest2426-21taskset07:20:493
26092991411,2cyclictest1178-21runrttasks08:23:163
26092991411,2cyclictest1146-21snmpd08:02:563
26092991411,2cyclictest1028-21sed09:50:303
26092991410,2cyclictest46-21ksoftirqd/311:55:573
26092991410,2cyclictest46-21ksoftirqd/311:35:453
26088991413,1cyclictest0-21swapper/212:25:422
26088991413,1cyclictest0-21swapper/211:25:282
26088991413,1cyclictest0-21swapper/209:15:182
2608899140,3cyclictest0-21swapper/207:33:522
2608899140,14cyclictest0-21swapper/212:38:202
2608399149,4cyclictest1076-21systemd-logind09:10:011
2608399148,5cyclictest0-21swapper/110:22:461
26083991413,1cyclictest0-21swapper/112:23:521
26083991413,1cyclictest0-21swapper/109:20:161
26083991412,1cyclictest0-21swapper/112:17:231
26083991412,1cyclictest0-21swapper/111:41:261
2608399140,8cyclictest0-21swapper/111:58:251
2608399140,7cyclictest0-21swapper/109:59:561
2608399140,7cyclictest0-21swapper/109:59:551
2608399140,14cyclictest0-21swapper/110:05:161
2608399140,13cyclictest0-21swapper/111:12:181
2608399140,1cyclictest0-21swapper/112:31:481
26079991412,1cyclictest0-21swapper/008:30:190
26079991411,2cyclictest0-21swapper/009:20:140
26079991410,3cyclictest0-21swapper/012:23:380
2607999140,7cyclictest0-21swapper/011:45:270
2607999140,13cyclictest0-21swapper/011:07:510
2607999140,13cyclictest0-21swapper/007:45:140
2607999140,1cyclictest0-21swapper/012:05:280
2607999140,1cyclictest0-21swapper/011:37:290
2607999140,1cyclictest0-21swapper/009:54:500
2328452140,1pltrace0-21swapper/009:00:000
582352130,5pltrace0-21swapper/512:22:505
382952130,5pltrace0-21swapper/611:07:216
377352130,7pltrace54-21ksoftirqd/408:25:494
32852130,6pltrace0-21swapper/010:28:550
32852130,3pltrace0-21swapper/110:28:501
2611599139,2cyclictest0-21swapper/709:07:457
2611599137,5cyclictest0-21swapper/708:14:237
2611599130,12cyclictest0-21swapper/710:29:567
2611599130,1cyclictest0-21swapper/711:34:147
2611599130,1cyclictest0-21swapper/709:58:237
2611599130,1cyclictest0-21swapper/709:58:237
2611099137,5cyclictest0-21swapper/607:12:096
26110991312,1cyclictest0-21swapper/610:57:356
26110991312,1cyclictest0-21swapper/609:48:276
26110991311,1cyclictest0-21swapper/607:30:016
2610199139,3cyclictest1146-21snmpd09:05:275
2610199131,8cyclictest0-21swapper/510:31:575
2610199130,12cyclictest0-21swapper/511:16:115
2609899137,4cyclictest0-21swapper/411:36:594
26098991312,1cyclictest0-21swapper/412:35:544
26098991312,1cyclictest0-21swapper/410:32:334
26098991312,1cyclictest0-21swapper/410:24:274
26098991312,1cyclictest0-21swapper/409:19:124
26098991311,1cyclictest0-21swapper/411:25:104
2609899130,13cyclictest0-21swapper/408:10:244
2609899130,12cyclictest0-21swapper/409:44:054
2609899130,1cyclictest0-21swapper/410:36:364
26092991312,1cyclictest0-21swapper/311:50:193
26092991311,1cyclictest27660-21/usr/sbin/munin07:10:173
26092991311,1cyclictest23946-21cstates10:50:123
26092991311,1cyclictest0-21swapper/311:20:143
26092991311,1cyclictest0-21swapper/309:35:213
26092991310,2cyclictest8973-21cstates11:10:123
26092991310,2cyclictest27002-21proc_pri12:05:213
26092991310,2cyclictest24520-21fschecks_time09:00:143
26092991310,2cyclictest20259-21cstates07:55:143
26092991310,2cyclictest12161-21processes10:00:203
26092991310,2cyclictest12161-21processes10:00:203
26092991310,2cyclictest1178-21runrttasks09:16:523
2609299130,1cyclictest0-21swapper/312:17:073
2609299130,1cyclictest0-21swapper/310:42:543
2608899139,3cyclictest19343-21latency_hist10:10:012
26088991312,1cyclictest0-21swapper/212:21:002
26088991312,1cyclictest0-21swapper/212:14:322
26088991312,1cyclictest0-21swapper/211:35:402
26088991312,1cyclictest0-21swapper/211:32:452
26088991312,1cyclictest0-21swapper/209:38:372
26088991312,1cyclictest0-21swapper/209:27:262
26088991311,1cyclictest0-21swapper/210:21:502
2608899130,7cyclictest0-21swapper/210:11:572
2608899130,4cyclictest0-21swapper/210:04:572
2608899130,4cyclictest0-21swapper/210:04:572
2608899130,13cyclictest0-21swapper/210:52:262
2608899130,13cyclictest0-21swapper/209:23:532
2608899130,12cyclictest0-21swapper/210:45:282
2608899130,1cyclictest0-21swapper/211:40:022
26083991312,1cyclictest0-21swapper/112:00:421
26083991312,1cyclictest0-21swapper/111:26:521
26083991312,1cyclictest0-21swapper/110:59:491
26083991312,1cyclictest0-21swapper/109:41:091
26083991311,1cyclictest0-21swapper/110:32:281
26083991310,2cyclictest0-21swapper/111:38:321
2608399130,13cyclictest0-21swapper/110:37:581
2608399130,13cyclictest0-21swapper/109:45:121
2608399130,1cyclictest0-21swapper/108:25:031
2608399130,1cyclictest0-21swapper/107:59:571
2607999138,4cyclictest0-21swapper/010:15:330
2607999137,5cyclictest0-21swapper/010:22:060
26079991312,1cyclictest0-21swapper/012:31:150
26079991312,1cyclictest0-21swapper/009:49:200
26079991312,1cyclictest0-21swapper/009:15:580
2607999130,13cyclictest0-21swapper/009:31:220
2607999130,1cyclictest0-21swapper/011:43:180
2500852130,5pltrace0-21swapper/608:07:466
2328452130,6pltrace0-21swapper/009:00:150
2328452130,5pltrace0-21swapper/609:00:116
2328452130,5pltrace0-21swapper/109:00:161
2049652130,6pltrace54-21ksoftirqd/408:55:144
1565752130,1pltrace0-21swapper/609:31:456
134352130,4pltrace2561-21diskstats12:15:125
1129152130,3pltrace15164-21ssh11:53:016
102752130,5pltrace0-21swapper/509:50:545
102752130,5pltrace0-21swapper/209:50:572
377352120,1pltrace0-21swapper/008:24:590
32852120,5pltrace0-21swapper/510:28:555
2611599126,5cyclictest0-21swapper/709:02:187
2611599125,6cyclictest0-21swapper/712:05:147
26115991211,1cyclictest0-21swapper/710:37:347
26115991211,1cyclictest0-21swapper/709:45:067
2611599120,11cyclictest0-21swapper/712:12:097
2611099129,2cyclictest23092-21ssh10:14:026
2611099129,2cyclictest0-21swapper/608:15:126
26110991211,1cyclictest0-21swapper/610:36:526
26110991211,1cyclictest0-21swapper/610:31:216
26110991211,1cyclictest0-21swapper/608:50:006
2611099120,11cyclictest0-21swapper/609:43:256
2610199127,4cyclictest0-21swapper/508:34:595
2610199126,2cyclictest0-21swapper/511:48:035
2610199126,2cyclictest0-21swapper/507:16:165
2610199125,6cyclictest1055-13auditd12:35:005
26101991211,1cyclictest0-21swapper/511:55:175
2610199120,1cyclictest0-21swapper/511:00:145
2610199120,1cyclictest0-21swapper/510:00:305
2610199120,1cyclictest0-21swapper/510:00:305
2610199120,1cyclictest0-21swapper/509:47:335
2609899127,2cyclictest0-21swapper/410:57:234
26098991211,1cyclictest0-21swapper/411:01:034
26098991211,1cyclictest0-21swapper/410:46:094
26098991211,1cyclictest0-21swapper/409:37:294
2609299129,2cyclictest26681-21proc_pri11:30:193
2609299129,2cyclictest1871-21fschecks_time10:25:153
26092991210,1cyclictest0-21swapper/310:58:533
2608899127,3cyclictest0-21swapper/212:16:352
2608899127,3cyclictest0-21swapper/211:47:002
2608899127,3cyclictest0-21swapper/210:33:162
26088991211,1cyclictest0-21swapper/210:15:122
26088991211,1cyclictest0-21swapper/207:24:532
26088991210,1cyclictest0-21swapper/208:30:462
2608899120,11cyclictest0-21swapper/208:44:512
2608399127,3cyclictest0-21swapper/108:00:021
2608399127,2cyclictest0-21swapper/108:36:031
2608399126,5cyclictest0-21swapper/111:09:171
2608399126,3cyclictest0-21swapper/111:03:341
26083991211,1cyclictest0-21swapper/111:19:351
26083991211,1cyclictest0-21swapper/109:37:061
2608399120,12cyclictest0-21swapper/107:15:441
2608399120,11cyclictest0-21swapper/108:42:221
2608399120,11cyclictest0-21swapper/107:27:241
2607999127,3cyclictest0-21swapper/010:56:490
2607999127,3cyclictest0-21swapper/008:12:290
2607999127,2cyclictest0-21swapper/009:40:480
2607999126,5cyclictest0-21swapper/008:16:110
2607999122,8cyclictest0-21swapper/011:30:260
26079991211,1cyclictest0-21swapper/012:26:330
26079991211,1cyclictest0-21swapper/009:56:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional