You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 11:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sun Mar 15, 2026 00:44:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12067521160,112pltrace0-21swapper/619:05:226
14852521110,97pltrace0-21swapper/319:10:013
12067521050,101pltrace0-21swapper/519:05:115
1485252970,90pltrace0-21swapper/219:10:002
1488727245,10sleep10-21swapper/119:07:311
253322680,0sleep60-21swapper/623:15:246
1497626349,10sleep70-21swapper/719:08:507
1491325945,10sleep00-21swapper/019:07:540
1476925844,10sleep40-21swapper/419:06:144
3230552290,2pltrace0-21swapper/700:30:147
1061852290,0pltrace0-21swapper/721:55:217
1524599230,1cyclictest0-21swapper/522:00:205
1522199230,22cyclictest0-21swapper/022:22:430
611052220,7pltrace0-21swapper/722:46:087
504352190,2pltrace5903-21tr21:45:117
565852180,0pltrace0-21swapper/719:51:337
1728752180,6pltrace78-21ksoftirqd/723:03:187
1525399180,17cyclictest0-21swapper/720:40:167
1524099180,14cyclictest0-21swapper/420:29:064
844852170,0pltrace0-21swapper/719:55:287
2970152170,2pltrace0-21swapper/719:35:147
2286052170,14pltrace54-21ksoftirqd/423:15:314
1728752170,12pltrace0-21swapper/523:04:465
1622152170,4pltrace0-21swapper/722:03:497
1622152170,0pltrace0-21swapper/722:05:167
1525399179,7cyclictest1064-13audispd19:25:017
1525399170,15cyclictest673-21chrt22:34:477
1524999171,15cyclictest0-21swapper/622:30:196
1524099179,6cyclictest1055-13auditd00:00:004
1524099179,6cyclictest1055-13auditd00:00:004
1524099170,2cyclictest0-21swapper/422:39:104
1523599171,9cyclictest0-21swapper/300:04:153
15228991712,2cyclictest38-21ksoftirqd/223:58:012
15228991712,2cyclictest38-21ksoftirqd/223:58:012
1522899171,15cyclictest0-21swapper/221:10:522
1522199171,15cyclictest0-21swapper/023:28:140
3121752160,0pltrace0-21swapper/723:28:147
2691252160,4pltrace28913-21ntpdc19:30:190
1683852160,12pltrace54-21ksoftirqd/420:07:504
15253991611,3cyclictest0-21swapper/720:45:087
1525399160,9cyclictest10741-21taskset21:53:477
1525399160,5cyclictest0-21swapper/723:35:127
1525399160,15cyclictest0-21swapper/722:22:037
1525399160,15cyclictest0-21swapper/721:30:537
1525399160,15cyclictest0-21swapper/721:00:597
1525399160,15cyclictest0-21swapper/721:00:587
1525399160,10cyclictest19555-21idleruntime21:10:147
1524999169,6cyclictest0-21swapper/619:27:506
15249991611,2cyclictest0-21swapper/619:15:046
1524999161,7cyclictest0-21swapper/622:57:286
1524999161,2cyclictest0-21swapper/619:32:236
15245991611,3cyclictest0-21swapper/519:27:085
1524599161,8cyclictest0-21swapper/523:51:585
1524599160,8cyclictest0-21swapper/521:58:225
1524599160,4cyclictest0-21swapper/500:00:045
1524599160,15cyclictest0-21swapper/519:50:005
15240991611,3cyclictest0-21swapper/422:11:494
15240991610,5cyclictest0-21swapper/422:04:074
1524099160,15cyclictest0-21swapper/423:33:384
15235991610,5cyclictest0-21swapper/321:47:083
1522899169,6cyclictest0-21swapper/223:50:002
1522899167,4cyclictest38-21ksoftirqd/200:10:432
1522899167,4cyclictest38-21ksoftirqd/200:10:432
15228991613,2cyclictest22794-21perl19:20:132
15228991613,2cyclictest17522-21/usr/sbin/munin00:00:172
15228991612,2cyclictest38-21ksoftirqd/200:32:482
1522899161,8cyclictest0-21swapper/222:38:382
1522899161,8cyclictest0-21swapper/220:23:122
1522899161,14cyclictest0-21swapper/222:14:162
1522899160,15cyclictest0-21swapper/221:52:292
1522899160,15cyclictest0-21swapper/221:19:532
1522499161,8cyclictest0-21swapper/122:34:371
1522499160,8cyclictest0-21swapper/123:50:061
1522199161,7cyclictest0-21swapper/020:40:010
1522199161,14cyclictest0-21swapper/000:39:010
1522199160,8cyclictest0-21swapper/023:31:090
2691252150,3pltrace29556-21awk19:30:231
1525399159,5cyclictest0-21swapper/719:19:327
1525399159,3cyclictest0-21swapper/721:27:327
1525399154,6cyclictest25081-21switchtime23:15:257
15253991510,4cyclictest0-21swapper/721:43:427
15253991510,4cyclictest0-21swapper/720:33:467
15253991510,3cyclictest0-21swapper/722:18:277
1525399150,3cyclictest0-21swapper/723:44:337
1525399150,14cyclictest0-21swapper/723:45:227
1525399150,14cyclictest0-21swapper/721:15:527
1525399150,14cyclictest0-21swapper/719:47:217
1525399150,13cyclictest28284-21grep00:20:147
1525399150,13cyclictest10115-21/usr/sbin/munin20:55:197
1525399150,13cyclictest10115-21/usr/sbin/munin20:55:197
1525399150,13cyclictest0-21swapper/700:12:307
1525399150,13cyclictest0-21swapper/700:12:307
1525399150,1cyclictest0-21swapper/719:30:487
1524999150,8cyclictest0-21swapper/620:01:096
1524999150,14cyclictest0-21swapper/623:20:016
1524599159,4cyclictest31649-21gltestperf22:30:125
15245991510,3cyclictest0-21swapper/522:46:585
1524599150,2cyclictest0-21swapper/519:50:155
1524599150,13cyclictest0-21swapper/519:10:045
15240991510,3cyclictest0-21swapper/400:19:324
1524099150,14cyclictest0-21swapper/422:15:494
1524099150,14cyclictest0-21swapper/421:55:514
1524099150,14cyclictest0-21swapper/421:14:384
1524099150,14cyclictest0-21swapper/421:02:104
1524099150,14cyclictest0-21swapper/421:02:104
1524099150,14cyclictest0-21swapper/420:46:524
1524099150,14cyclictest0-21swapper/420:21:184
1524099150,14cyclictest0-21swapper/419:39:024
1524099150,14cyclictest0-21swapper/419:18:194
1524099150,14cyclictest0-21swapper/419:14:084
1524099150,13cyclictest0-21swapper/422:42:014
1524099150,13cyclictest0-21swapper/419:29:324
1524099150,13cyclictest0-21swapper/419:24:444
1524099150,13cyclictest0-21swapper/400:09:244
1524099150,13cyclictest0-21swapper/400:09:244
1523599151,7cyclictest0-21swapper/323:37:383
1523599150,8cyclictest0-21swapper/322:12:433
1523599150,8cyclictest0-21swapper/321:16:413
1523599150,14cyclictest0-21swapper/321:24:023
15228991512,2cyclictest8633-21sed22:45:212
15228991512,2cyclictest710-21/usr/sbin/munin19:40:112
15228991512,2cyclictest549-21cat23:30:162
15228991512,2cyclictest30956-21/usr/sbin/munin21:30:152
15228991512,2cyclictest30815-21cat23:25:202
15228991512,2cyclictest30645-21cstates19:35:122
15228991512,2cyclictest29081-21sed00:20:212
15228991512,2cyclictest28275-21netstat21:25:172
15228991512,2cyclictest25303-21sed23:15:242
15228991512,2cyclictest23316-21gltestperf22:15:162
15228991512,2cyclictest21091-21cstates23:10:132
15228991512,2cyclictest18073-21grep20:10:122
15228991512,2cyclictest13992-21/usr/sbin/munin22:00:142
15228991512,2cyclictest12165-21head21:55:142
15228991511,2cyclictest38-21ksoftirqd/220:30:352
1522899151,3cyclictest0-21swapper/221:48:442
1522899150,2cyclictest0-21swapper/223:35:212
1522899150,2cyclictest0-21swapper/219:50:152
15224991510,2cyclictest0-21swapper/121:56:261
1522499151,7cyclictest0-21swapper/120:18:581
1522499150,7cyclictest0-21swapper/123:44:171
1522499150,14cyclictest0-21swapper/120:39:521
15221991512,2cyclictest16609-21iostat_ios19:10:160
15221991511,3cyclictest0-21swapper/021:46:080
1522199150,8cyclictest0-21swapper/022:08:300
1522199150,2cyclictest0-21swapper/023:35:140
1522199150,2cyclictest0-21swapper/019:50:140
1522199150,14cyclictest0-21swapper/020:00:010
132942150,1sleep00-21swapper/023:55:000
844852140,2pltrace8662-21latency_hist19:55:011
15253991410,3cyclictest0-21swapper/723:14:537
1525399140,8cyclictest29885-21kworker/u16:2+ext4-rsv-conversion@
ext4_end_io_rsv_work
22:40:007
1525399140,7cyclictest0-21swapper/723:20:147
1525399140,13cyclictest0-21swapper/721:05:117
1525399140,12cyclictest617-21/usr/sbin/munin19:40:187
1525399140,12cyclictest571-21cut21:35:127
1525399140,12cyclictest26299-21timerandwakeup00:15:217
1525399140,12cyclictest16937-21ntpq19:10:187
1525399140,12cyclictest1146-21snmpd23:05:227
1525399140,1cyclictest0-21swapper/723:58:167
1525399140,1cyclictest0-21swapper/723:58:167
1525399140,1cyclictest0-21swapper/721:24:177
1525399140,1cyclictest0-21swapper/720:08:127
1525399140,1cyclictest0-21swapper/720:00:267
15249991410,3cyclictest0-21swapper/621:22:546
15249991410,2cyclictest0-21swapper/623:33:576
1524999140,13cyclictest0-21swapper/620:30:396
1524999140,13cyclictest0-21swapper/619:10:026
1524999140,1cyclictest0-21swapper/623:35:156
1524599140,13cyclictest0-21swapper/522:10:155
1524099140,7cyclictest0-21swapper/420:40:484
1524099140,7cyclictest0-21swapper/400:30:324
1524099140,13cyclictest0-21swapper/421:30:004
1524099140,13cyclictest0-21swapper/420:50:204
1524099140,13cyclictest0-21swapper/420:50:204
15235991412,1cyclictest0-21swapper/319:40:123
15235991410,3cyclictest0-21swapper/319:11:383
1523599140,14cyclictest0-21swapper/320:10:003
15228991412,2cyclictest19729-21cat23:05:212
15228991412,1cyclictest6959-21sed23:40:212
15228991412,1cyclictest18793-21/usr/sbin/munin22:05:222
15228991412,1cyclictest14049-21timerandwakeup22:55:192
15228991411,2cyclictest3455-21grep21:40:122
15228991411,2cyclictest27371-21latency23:20:142
15228991411,2cyclictest25750-21switchtime21:20:172
15228991411,2cyclictest19143-21/usr/sbin/munin19:15:172
15228991411,2cyclictest17368-21timerwakeupswit19:10:212
15228991411,2cyclictest15832-21sort20:05:142
1522899140,7cyclictest0-21swapper/200:27:312
1522899140,12cyclictest0-21swapper/200:09:002
1522899140,12cyclictest0-21swapper/200:09:002
1522499140,3cyclictest0-21swapper/121:50:001
1522199149,3cyclictest0-21swapper/022:02:180
1522199149,3cyclictest0-21swapper/021:03:370
1522199149,3cyclictest0-21swapper/021:03:370
1522199148,5cyclictest0-21swapper/023:23:510
1522199141,6cyclictest0-21swapper/020:18:130
1522199141,12cyclictest0-21swapper/021:22:230
1522199140,7cyclictest0-21swapper/022:12:330
1340452140,2pltrace15992-21unixbench_singl22:00:216
1275452140,10pltrace54-21ksoftirqd/423:54:144
3121752130,5pltrace0-21swapper/323:29:123
2798952130,6pltrace1178-21runrttasks20:31:111
1837252130,7pltrace1178-21runrttasks00:06:520
1837252130,7pltrace1178-21runrttasks00:06:520
1728752130,5pltrace0-21swapper/123:04:481
1683852130,2pltrace16901-21taskset20:07:550
1525399133,9cyclictest19613-21cut22:10:107
1525399133,9cyclictest12729-21chrt23:52:047
1525399131,7cyclictest0-21swapper/720:35:127
1525399130,11cyclictest7689-21seq20:50:127
1525399130,11cyclictest7689-21seq20:50:127
1525399130,11cyclictest5208-21missed_timers22:40:147
1525399130,11cyclictest30508-21cstates00:25:127
1525399130,11cyclictest22458-21chrt20:17:407
1525399130,11cyclictest19609-21grep00:05:147
1525399130,11cyclictest19609-21grep00:05:147
1525399130,11cyclictest16440-21tr00:00:137
1525399130,1cyclictest0-21swapper/720:25:167
15249991312,1cyclictest0-21swapper/622:05:236
15249991311,1cyclictest0-21swapper/622:11:136
15249991311,1cyclictest0-21swapper/621:42:436
15249991310,2cyclictest0-21swapper/600:38:166
1524599130,7cyclictest0-21swapper/521:00:425
1524599130,7cyclictest0-21swapper/521:00:425
1524599130,12cyclictest0-21swapper/523:35:445
1524599130,12cyclictest0-21swapper/521:23:125
1524599130,1cyclictest0-21swapper/521:30:205
1524099130,1cyclictest0-21swapper/419:50:194
15235991312,1cyclictest0-21swapper/320:55:103
15235991312,1cyclictest0-21swapper/320:55:103
15235991311,1cyclictest0-21swapper/323:55:223
15235991311,1cyclictest0-21swapper/323:55:223
1523599130,12cyclictest0-21swapper/319:29:473
15228991311,1cyclictest25072-21cstates19:25:132
15228991311,1cyclictest0-21swapper/223:00:212
15228991311,1cyclictest0-21swapper/222:25:122
15228991310,2cyclictest3516-21cpuspeed_turbos00:35:112
15228991310,2cyclictest3503-21aten_r4power_en19:45:112
15228991310,2cyclictest27661-21cpuspeed_turbos19:30:122
15228991310,2cyclictest26893-21irqstats_LOC20:25:152
15228991310,2cyclictest16330-21cut21:05:122
15228991310,2cyclictest1439-21runrttasks21:35:162
15228991310,2cyclictest11992-21ntp_states23:50:182
15228991310,2cyclictest10903-21ntp_kernel_pll_22:50:192
15228991310,2cyclictest10758-21threads19:55:202
1522899130,7cyclictest14216-21missed_timers21:00:162
1522899130,7cyclictest14216-21missed_timers21:00:162
1522899130,12cyclictest0-21swapper/220:15:032
1522499139,3cyclictest3692-21proc_pri23:35:191
1522499138,2cyclictest0-21swapper/121:37:531
15224991311,1cyclictest0-21swapper/121:10:121
15224991310,2cyclictest15869-21cpu21:05:111
15224991310,2cyclictest0-21swapper/121:54:131
1522499130,12cyclictest0-21swapper/122:25:031
1522499130,12cyclictest0-21swapper/119:28:391
1522199138,3cyclictest0-21swapper/021:50:530
15221991311,1cyclictest0-21swapper/021:13:190
15221991310,2cyclictest0-21swapper/023:47:160
1522199131,11cyclictest0-21swapper/023:16:440
1522199131,11cyclictest0-21swapper/019:42:090
1522199130,12cyclictest0-21swapper/020:50:200
1522199130,12cyclictest0-21swapper/020:50:200
1522199130,10cyclictest9119-21awk22:49:590
1340452130,4pltrace14646-21fschecks_count22:00:143
1340452130,2pltrace15455-21open_inodes22:00:181
844852120,4pltrace0-21swapper/119:55:061
3230552120,4pltrace0-21swapper/300:31:253
3186952120,4pltrace0-21swapper/621:33:386
2736552120,5pltrace0-21swapper/622:26:316
2672952120,5pltrace0-21swapper/600:19:346
2286052120,0pltrace0-21swapper/523:16:535
2179352120,7pltrace1178-21runrttasks22:13:381
1525399121,10cyclictest4311-21basename00:35:137
1525399121,10cyclictest28057-21cut22:25:117
1525399120,11cyclictest9747-21tr22:50:127
1525399120,11cyclictest32261-21cstates23:30:137
1525399120,11cyclictest24773-21basename19:25:137
1525399120,11cyclictest12456-21cat22:55:097
1525399120,10cyclictest17435-21basename20:10:117
1525399120,1cyclictest0-21swapper/720:20:197
1524999129,2cyclictest1146-21snmpd19:21:386
1524999126,5cyclictest0-21swapper/620:06:336
15249991211,1cyclictest0-21swapper/622:47:486
15249991211,1cyclictest0-21swapper/622:23:416
15249991211,1cyclictest0-21swapper/622:17:026
15249991211,1cyclictest0-21swapper/620:38:576
15249991211,1cyclictest0-21swapper/600:31:546
15249991210,1cyclictest0-21swapper/623:55:106
15249991210,1cyclictest0-21swapper/623:55:106
1524999120,12cyclictest0-21swapper/623:50:386
1524999120,12cyclictest0-21swapper/622:41:586
1524999120,12cyclictest0-21swapper/621:57:276
1524999120,12cyclictest0-21swapper/621:28:196
1524999120,12cyclictest0-21swapper/621:09:236
1524999120,12cyclictest0-21swapper/619:50:316
1524999120,12cyclictest0-21swapper/600:08:276
1524999120,12cyclictest0-21swapper/600:08:276
1524999120,1cyclictest0-21swapper/623:28:176
1524999120,1cyclictest0-21swapper/623:11:466
1524999120,1cyclictest0-21swapper/622:54:266
1524999120,1cyclictest0-21swapper/621:13:066
1524999120,1cyclictest0-21swapper/620:53:116
1524999120,1cyclictest0-21swapper/620:53:116
1524999120,1cyclictest0-21swapper/620:16:256
1524999120,1cyclictest0-21swapper/620:14:486
1524999120,1cyclictest0-21swapper/600:01:496
1524599129,2cyclictest16339-21fschecks_count21:05:135
1524599127,2cyclictest0-21swapper/500:15:175
15245991211,1cyclictest0-21swapper/500:40:015
1524599120,11cyclictest0-21swapper/520:53:415
1524599120,11cyclictest0-21swapper/520:53:405
1524099129,2cyclictest5255-21crond21:45:014
1524099126,5cyclictest0-21swapper/423:35:594
1524099126,5cyclictest0-21swapper/420:35:064
1524099122,8cyclictest0-21swapper/423:47:154
1524099122,8cyclictest0-21swapper/422:29:434
1524099122,8cyclictest0-21swapper/400:39:384
1524099120,11cyclictest0-21swapper/422:48:574
1524099120,11cyclictest0-21swapper/400:02:514
1523599128,3cyclictest768-21timerwakeupswit20:35:213
1523599127,4cyclictest26885-21threads22:20:203
1523599127,3cyclictest0-21swapper/322:35:563
1523599127,3cyclictest0-21swapper/319:19:553
1523599123,8cyclictest0-21swapper/322:33:023
1523599123,8cyclictest0-21swapper/322:05:143
15235991211,1cyclictest0-21swapper/323:20:243
15235991211,1cyclictest0-21swapper/319:30:393
15235991210,1cyclictest0-21swapper/322:53:133
15235991210,1cyclictest0-21swapper/320:16:383
1523599120,12cyclictest0-21swapper/321:30:163
1523599120,11cyclictest0-21swapper/321:14:413
1523599120,1cyclictest0-21swapper/300:19:103
1522899129,2cyclictest5291-21netstat22:40:152
1522899129,2cyclictest4572-21aten_r4power_en20:45:102
1522899129,2cyclictest26263-21irqstats22:20:152
1522899129,2cyclictest24649-21cpu00:15:122
1522899129,2cyclictest1474-21lldpd20:50:092
1522899129,2cyclictest1474-21lldpd20:50:092
1522899129,2cyclictest13443-21swap20:00:192
1522899129,2cyclictest11111-21hwlatdetect20:55:132
1522899129,2cyclictest11111-21hwlatdetect20:55:132
15228991211,1cyclictest0-21swapper/222:30:212
15228991210,1cyclictest0-21swapper/220:40:112
1522499126,5cyclictest0-21swapper/122:18:361
1522499120,11cyclictest0-21swapper/120:55:081
1522499120,11cyclictest0-21swapper/120:55:081
1522499120,1cyclictest0-21swapper/121:15:131
1522199129,2cyclictest0-21swapper/020:43:330
1522199126,5cyclictest0-21swapper/021:28:290
15221991211,1cyclictest0-21swapper/023:00:210
15221991210,1cyclictest0-21swapper/020:05:010
1522199120,11cyclictest0-21swapper/023:06:110
1522199120,11cyclictest0-21swapper/021:57:020
1168552120,1pltrace0-21swapper/022:52:180
114052120,4pltrace0-21swapper/620:42:076
782752110,6pltrace9979-21sshd21:50:204
3248552110,8pltrace2134-21awk19:40:186
2457752110,8pltrace54-21ksoftirqd/422:20:164
2007152110,8pltrace54-21ksoftirqd/423:10:164
1791352110,7pltrace0-21swapper/321:08:043
1524999117,3cyclictest1146-21snmpd20:23:486
1524999115,5cyclictest0-21swapper/623:04:256
15249991110,1cyclictest0-21swapper/621:37:426
15249991110,1cyclictest0-21swapper/620:59:506
15249991110,1cyclictest0-21swapper/620:59:506
15249991110,1cyclictest0-21swapper/620:48:106
1524999110,11cyclictest0-21swapper/622:39:046
1524999110,11cyclictest0-21swapper/619:35:576
1524999110,11cyclictest0-21swapper/600:25:466
1524599118,2cyclictest32725-21tr21:35:125
1524599118,2cyclictest2693-21awk19:40:215
1524599116,4cyclictest0-21swapper/519:30:265
15245991110,1cyclictest0-21swapper/522:17:225
15245991110,1cyclictest0-21swapper/520:04:465
1524599110,11cyclictest0-21swapper/523:41:435
1524599110,11cyclictest0-21swapper/522:27:245
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional