You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 07:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sun May 24, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14597521110,107pltrace0-21swapper/519:05:155
14597521060,101pltrace0-21swapper/119:06:081
66712930,0sleep40-21swapper/421:27:384
62962920,0sleep30-21swapper/322:35:083
2644352920,0pltrace0-21swapper/122:21:561
1738552890,86pltrace0-21swapper/619:07:076
208942800,0sleep10-21swapper/123:22:341
1738552800,72pltrace0-21swapper/719:07:277
1557152790,1pltrace15575-21pltrace-run23:16:570
2061652740,2pltrace0-21swapper/423:21:564
1733427446,10sleep20-21swapper/219:06:282
1755727042,10sleep30-21swapper/319:09:173
636752630,0pltrace0-21swapper/321:26:563
2922452630,12pltrace78-21ksoftirqd/721:20:117
1732725844,10sleep40-21swapper/419:06:224
1727625844,10sleep00-21swapper/019:05:370
2644352550,9pltrace78-21ksoftirqd/722:25:157
180552480,6pltrace78-21ksoftirqd/721:25:427
310552470,5pltrace78-21ksoftirqd/722:00:397
1672352460,9pltrace78-21ksoftirqd/723:54:057
1639652460,6pltrace78-21ksoftirqd/700:30:187
935952450,9pltrace78-21ksoftirqd/722:37:087
727352450,7pltrace78-21ksoftirqd/700:18:147
1672352450,7pltrace78-21ksoftirqd/723:56:367
204252420,6pltrace78-21ksoftirqd/723:40:557
26802410,0sleep10-21swapper/100:11:291
16462400,1sleep60-21swapper/623:05:006
2346852390,7pltrace78-21ksoftirqd/722:54:587
1557152390,7pltrace78-21ksoftirqd/723:21:517
1276052380,5pltrace78-21ksoftirqd/722:07:517
1557152370,7pltrace78-21ksoftirqd/723:18:017
1142652370,6pltrace78-21ksoftirqd/723:48:357
2984452360,0pltrace0-21swapper/723:33:117
648652350,0pltrace0-21swapper/720:45:117
2922452350,7pltrace78-21ksoftirqd/721:19:367
1731952340,8pltrace0-21swapper/722:13:327
2346852330,6pltrace78-21ksoftirqd/722:55:177
3133652320,6pltrace78-21ksoftirqd/722:30:057
2396252320,0pltrace0-21swapper/721:12:097
1545452320,8pltrace78-21ksoftirqd/721:37:117
1545452320,8pltrace78-21ksoftirqd/721:37:117
636752310,6pltrace78-21ksoftirqd/721:31:027
636752310,6pltrace78-21ksoftirqd/721:31:027
3103652310,5pltrace78-21ksoftirqd/700:07:097
2492552300,0pltrace0-21swapper/721:47:067
185532290,1sleep3441rcuc/320:05:173
153832260,0sleep50-21swapper/522:44:165
2000052250,2pltrace0-21swapper/721:42:057
1777999251,23cyclictest0-21swapper/421:20:014
2203452240,0pltrace0-21swapper/700:00:247
1777499230,3cyclictest0-21swapper/321:39:223
1777499230,3cyclictest0-21swapper/321:39:223
1893052220,0pltrace0-21swapper/722:46:577
17763992119,1cyclictest0-21swapper/123:33:241
17793992019,1cyclictest0-21swapper/622:10:196
1779399201,4cyclictest0-21swapper/620:24:056
1779399200,4cyclictest0-21swapper/600:22:326
1779399199,9cyclictest11149-21sh22:40:016
1779399191,12cyclictest0-21swapper/622:59:566
17774991812,5cyclictest0-21swapper/320:15:323
1777499180,2cyclictest0-21swapper/319:27:423
648652170,2pltrace0-21swapper/720:44:227
2628052170,14pltrace0-21swapper/600:39:306
17793991711,5cyclictest0-21swapper/621:14:036
1779399170,12cyclictest0-21swapper/621:28:086
1779399170,1cyclictest0-21swapper/623:42:026
17779991711,5cyclictest0-21swapper/400:39:304
1777999171,8cyclictest0-21swapper/421:19:104
1777999171,15cyclictest0-21swapper/422:37:044
17774991711,5cyclictest0-21swapper/300:10:263
1777499171,8cyclictest0-21swapper/320:43:573
1776399170,16cyclictest0-21swapper/122:17:461
1775799170,1cyclictest0-21swapper/022:59:530
1438452170,11pltrace54-21ksoftirqd/422:45:004
287152160,11pltrace0-21swapper/700:12:427
2203452160,10pltrace54-21ksoftirqd/400:00:134
1779799161,14cyclictest0-21swapper/723:04:237
1779399166,3cyclictest17753-21sendmail_mailst22:45:226
1779399161,3cyclictest0-21swapper/622:50:236
1779399160,15cyclictest0-21swapper/620:30:206
1779399160,15cyclictest0-21swapper/619:38:426
17787991614,1cyclictest0-21swapper/521:17:485
17779991614,1cyclictest0-21swapper/419:34:494
17779991611,3cyclictest0-21swapper/422:16:224
1777999161,14cyclictest0-21swapper/400:05:554
1777999160,14cyclictest0-21swapper/423:30:284
17774991611,3cyclictest0-21swapper/321:33:533
17774991611,3cyclictest0-21swapper/321:33:533
17774991610,5cyclictest0-21swapper/322:45:263
1776999161,7cyclictest0-21swapper/222:13:242
1776999160,4cyclictest0-21swapper/223:21:542
1776999160,15cyclictest4587-21strings19:40:182
1776999160,15cyclictest0-21swapper/219:21:192
17757991613,2cyclictest0-21swapper/022:30:260
17757991610,5cyclictest0-21swapper/000:20:080
1775799161,3cyclictest0-21swapper/000:04:540
85752150,10pltrace0-21swapper/020:32:340
49952150,1sleep130-21ksoftirqd/100:15:141
2104152150,9pltrace25942-21rm00:36:232
2104152150,11pltrace26044-21ssh00:36:340
2000052150,3pltrace22856-21gltestperf21:45:173
17797991511,2cyclictest0-21swapper/719:35:497
17797991510,3cyclictest0-21swapper/722:40:257
17793991511,3cyclictest0-21swapper/600:09:336
17793991510,3cyclictest0-21swapper/619:58:406
17793991510,1cyclictest0-21swapper/621:33:586
17793991510,1cyclictest0-21swapper/621:33:586
17793991510,1cyclictest0-21swapper/619:34:096
17779991513,1cyclictest0-21swapper/423:03:364
1777999151,8cyclictest0-21swapper/420:02:374
1777999150,8cyclictest0-21swapper/419:38:184
1777999150,7cyclictest0-21swapper/421:34:474
1777999150,7cyclictest0-21swapper/421:34:474
1777999150,3cyclictest0-21swapper/421:43:524
1777999150,3cyclictest0-21swapper/419:25:184
1777999150,14cyclictest0-21swapper/422:25:234
1777999150,14cyclictest0-21swapper/422:13:304
1777999150,14cyclictest0-21swapper/420:46:394
1777999150,14cyclictest0-21swapper/419:50:104
1777999150,14cyclictest0-21swapper/419:16:544
1777999150,14cyclictest0-21swapper/400:23:064
17774991511,3cyclictest0-21swapper/300:26:323
1777499150,7cyclictest0-21swapper/300:18:053
1777499150,14cyclictest0-21swapper/322:51:043
1777499150,14cyclictest0-21swapper/300:32:063
17769991510,4cyclictest1055-17auditd20:10:002
1776999150,8cyclictest0-21swapper/223:43:492
1776999150,7cyclictest0-21swapper/223:30:122
1776999150,14cyclictest21423-21ssh23:55:552
1776999150,14cyclictest0-21swapper/223:29:062
1776999150,14cyclictest0-21swapper/222:58:502
1776999150,13cyclictest0-21swapper/222:01:452
1776999150,13cyclictest0-21swapper/221:26:232
1776399158,4cyclictest30-21ksoftirqd/121:55:121
17763991512,2cyclictest1178-21runrttasks23:42:261
17763991510,2cyclictest30-21ksoftirqd/123:51:071
17763991510,2cyclictest30-21ksoftirqd/121:05:001
17763991510,2cyclictest30-21ksoftirqd/119:14:061
1775799150,14cyclictest0-21swapper/023:21:560
1775799150,14cyclictest0-21swapper/022:07:360
1775799150,14cyclictest0-21swapper/021:27:370
1485252150,3pltrace0-21swapper/720:57:547
1438452150,11pltrace8657-21kworker/u16:3+xprtiod@
xs_stream_data_receive_workfn
12:02:110
663852140,5pltrace11277-21ssh23:46:420
3020752140,5pltrace54-21ksoftirqd/421:55:304
2922452140,1pltrace30108-21ssh21:18:272
2801052140,8pltrace32329-21chrt23:01:250
2801052140,6pltrace54-21ksoftirqd/422:59:284
2774052140,11pltrace0-21swapper/220:22:292
2203452140,3pltrace26066-21sed00:00:213
2189452140,4pltrace0-21swapper/722:20:387
2104152140,6pltrace54-21ksoftirqd/400:34:474
2000052140,9pltrace21916-21ssh21:45:142
2000052140,8pltrace0-21swapper/621:45:256
1779799140,7cyclictest0-21swapper/720:29:267
1779799140,6cyclictest0-21swapper/719:40:227
1779799140,13cyclictest78-21ksoftirqd/700:35:387
1779799140,13cyclictest0-21swapper/720:35:367
1779799140,1cyclictest0-21swapper/723:25:127
1779799140,1cyclictest0-21swapper/721:53:147
1779399149,1cyclictest0-21swapper/620:02:396
17793991411,3cyclictest0-21swapper/622:24:006
17793991410,1cyclictest0-21swapper/620:47:496
1779399140,7cyclictest0-21swapper/623:11:546
1779399140,13cyclictest0-21swapper/623:26:546
1779399140,1cyclictest0-21swapper/622:31:266
1779399140,1cyclictest0-21swapper/622:05:326
1779399140,1cyclictest0-21swapper/621:23:176
1778799149,4cyclictest23043-21iostat_ios21:45:195
1778799149,1cyclictest191rcu_preempt20:00:225
17787991413,1cyclictest0-21swapper/500:20:155
17787991411,2cyclictest1146-21snmpd22:17:465
1778799140,3cyclictest0-21swapper/521:27:145
1778799140,14cyclictest0-21swapper/523:21:195
1778799140,13cyclictest0-21swapper/520:15:225
1777999149,4cyclictest0-21swapper/422:33:134
17779991413,1cyclictest0-21swapper/420:10:124
17779991412,1cyclictest0-21swapper/400:25:024
17779991410,3cyclictest0-21swapper/420:28:084
1777999140,8cyclictest0-21swapper/422:24:524
1777999140,1cyclictest0-21swapper/423:50:334
1777499141,12cyclictest0-21swapper/321:55:133
1777499140,7cyclictest0-21swapper/322:13:223
1777499140,7cyclictest0-21swapper/319:48:213
1777499140,14cyclictest0-21swapper/322:42:193
1777499140,14cyclictest0-21swapper/322:24:313
1777499140,13cyclictest0-21swapper/322:17:333
1777499140,13cyclictest0-21swapper/321:20:243
1777499140,1cyclictest0-21swapper/323:23:563
17769991412,1cyclictest0-21swapper/200:28:002
17769991410,2cyclictest0-21swapper/223:50:012
1776999140,7cyclictest0-21swapper/220:55:032
1776999140,7cyclictest0-21swapper/219:51:512
1776999140,6cyclictest0-21swapper/223:36:542
1776999140,3cyclictest0-21swapper/221:04:432
1776999140,14cyclictest0-21swapper/223:16:262
1776999140,14cyclictest0-21swapper/223:02:412
1776999140,13cyclictest0-21swapper/223:11:422
1776999140,13cyclictest0-21swapper/220:36:032
1776999140,1cyclictest0-21swapper/221:34:362
1776999140,1cyclictest0-21swapper/221:34:362
1776399149,2cyclictest30-21ksoftirqd/122:45:161
1776399149,2cyclictest30-21ksoftirqd/122:40:111
1776399149,2cyclictest30-21ksoftirqd/121:40:141
1776399149,2cyclictest30-21ksoftirqd/121:05:221
1776399149,2cyclictest30-21ksoftirqd/120:58:381
1776399149,2cyclictest30-21ksoftirqd/120:50:211
1776399149,2cyclictest30-21ksoftirqd/120:17:251
1776399149,2cyclictest30-21ksoftirqd/120:06:431
1776399149,2cyclictest30-21ksoftirqd/119:55:121
1776399148,4cyclictest30-21ksoftirqd/122:55:131
1776399148,2cyclictest1178-21runrttasks21:53:311
1776399147,4cyclictest30-21ksoftirqd/123:05:111
1776399147,4cyclictest30-21ksoftirqd/122:50:141
1776399147,4cyclictest30-21ksoftirqd/122:30:151
1776399146,5cyclictest30-21ksoftirqd/119:15:031
1776399145,6cyclictest30-21ksoftirqd/122:00:151
17763991411,2cyclictest8789-21cat20:45:211
17763991411,2cyclictest19245-21if_eth000:30:151
17763991411,2cyclictest13860-21missed_timers23:15:181
17763991411,2cyclictest0-21swapper/100:08:261
17763991411,1cyclictest191rcu_preempt19:46:251
17763991410,1cyclictest191rcu_preempt23:04:091
1776399140,13cyclictest0-21swapper/123:29:541
1775799149,3cyclictest0-21swapper/019:50:340
17757991413,1cyclictest0-21swapper/021:31:110
17757991413,1cyclictest0-21swapper/021:31:110
17757991413,1cyclictest0-21swapper/020:06:200
17757991410,3cyclictest1065-17sedispatch00:00:000
1775799140,8cyclictest0-21swapper/020:51:520
1775799140,7cyclictest0-21swapper/000:07:430
1775799140,3cyclictest0-21swapper/023:12:250
1775799140,13cyclictest0-21swapper/023:37:180
1775799140,13cyclictest0-21swapper/022:02:360
1775799140,1cyclictest0-21swapper/023:05:240
1438452140,5pltrace54-21ksoftirqd/422:45:144
1183952140,4pltrace12957-21ssh00:24:312
1098352140,5pltrace54-21ksoftirqd/419:55:154
663852130,5pltrace54-21ksoftirqd/423:45:014
663852130,5pltrace54-21ksoftirqd/423:44:304
3052452130,5pltrace54-21ksoftirqd/420:31:254
287152130,6pltrace0-21swapper/000:12:400
2801052130,5pltrace0-21swapper/523:01:285
2216752130,4pltrace23139-21expr20:15:142
1779799138,2cyclictest17750-21cyclictest00:24:327
17797991312,1cyclictest0-21swapper/723:37:307
17797991312,1cyclictest0-21swapper/719:15:137
1779799130,12cyclictest0-21swapper/700:28:387
17793991312,1cyclictest0-21swapper/623:33:276
17793991312,1cyclictest0-21swapper/623:16:466
17793991310,2cyclictest0-21swapper/623:37:596
1779399130,13cyclictest0-21swapper/623:51:356
1778799138,2cyclictest17970-21taskset21:40:135
1778799137,3cyclictest2454-21ssh23:37:525
17787991312,1cyclictest0-21swapper/523:55:585
17787991312,1cyclictest0-21swapper/522:59:085
17787991312,1cyclictest0-21swapper/522:29:025
1778799130,13cyclictest0-21swapper/522:47:195
1778799130,13cyclictest0-21swapper/522:12:215
1778799130,13cyclictest0-21swapper/521:12:075
1778799130,13cyclictest0-21swapper/500:00:205
1778799130,1cyclictest0-21swapper/523:41:535
1778799130,1cyclictest0-21swapper/522:39:335
1778799130,1cyclictest0-21swapper/522:04:065
1778799130,1cyclictest0-21swapper/521:24:265
1778799130,1cyclictest0-21swapper/500:19:185
17779991312,1cyclictest0-21swapper/423:56:034
17779991312,1cyclictest0-21swapper/421:36:594
17779991312,1cyclictest0-21swapper/421:36:594
1777999130,13cyclictest0-21swapper/423:35:304
1777999130,13cyclictest0-21swapper/422:05:584
1777999130,13cyclictest0-21swapper/421:50:544
1777999130,13cyclictest0-21swapper/420:55:514
1777999130,12cyclictest0-21swapper/420:20:214
1777999130,1cyclictest0-21swapper/419:21:374
17774991312,1cyclictest0-21swapper/323:31:423
17774991312,1cyclictest0-21swapper/322:02:493
1777499130,13cyclictest0-21swapper/323:51:183
1777499130,13cyclictest0-21swapper/323:41:153
1777499130,12cyclictest0-21swapper/319:33:333
1777499130,12cyclictest0-21swapper/300:20:223
1777499130,1cyclictest0-21swapper/323:13:243
1776999135,7cyclictest0-21swapper/221:55:002
17769991312,1cyclictest0-21swapper/221:22:282
17769991311,1cyclictest0-21swapper/223:07:132
17769991310,2cyclictest24443-21ssh21:12:452
1776999130,13cyclictest0-21swapper/222:45:502
1776999130,13cyclictest0-21swapper/222:32:532
1776999130,13cyclictest0-21swapper/222:08:172
1776399139,1cyclictest191rcu_preempt21:25:121
1776399138,2cyclictest30-21ksoftirqd/123:55:171
1776399138,2cyclictest30-21ksoftirqd/122:05:141
1776399138,2cyclictest30-21ksoftirqd/121:20:251
1776399138,2cyclictest30-21ksoftirqd/100:22:051
1776399137,4cyclictest30-21ksoftirqd/119:37:061
1776399136,4cyclictest30-21ksoftirqd/121:11:451
1776399135,4cyclictest30-21ksoftirqd/121:35:131
1776399135,4cyclictest30-21ksoftirqd/121:35:131
1776399134,6cyclictest30-21ksoftirqd/122:10:141
1776399134,6cyclictest30-21ksoftirqd/121:30:171
1776399134,6cyclictest30-21ksoftirqd/121:30:171
17763991310,2cyclictest9870-21ntpdc23:45:191
17763991310,2cyclictest24097-21fschecks_time00:35:151
17763991310,2cyclictest2147-21fschecks_count20:35:121
17763991310,2cyclictest14755-21cut00:25:161
17763991310,2cyclictest1178-21runrttasks19:50:041
17763991310,2cyclictest1146-21snmpd22:37:341
1776399130,5cyclictest191rcu_preempt20:23:511
1776399130,5cyclictest16303-21sed20:00:221
1776399130,12cyclictest0-21swapper/123:39:481
17757991312,1cyclictest0-21swapper/023:52:220
17757991312,1cyclictest0-21swapper/023:32:460
1775799131,11cyclictest0-21swapper/000:16:320
1775799130,13cyclictest0-21swapper/022:28:200
1775799130,12cyclictest0-21swapper/020:41:330
1775799130,1cyclictest0-21swapper/019:30:240
1766152130,5pltrace0-21swapper/621:03:246
1766152130,10pltrace0-21swapper/021:05:460
1091452130,10pltrace12778-21cpu21:35:116
1091452130,10pltrace12778-21cpu21:35:116
1029852130,5pltrace54-21ksoftirqd/423:15:314
1029852130,5pltrace54-21ksoftirqd/423:12:134
819252120,5pltrace0-21swapper/619:50:126
3223552120,4pltrace0-21swapper/319:36:523
3020752120,1pltrace0-21swapper/021:56:520
2922452120,7pltrace0-21swapper/621:18:316
2922452120,4pltrace0-21swapper/321:18:323
2216752120,5pltrace0-21swapper/620:13:486
2045552120,8pltrace54-21ksoftirqd/421:10:174
1779799129,2cyclictest1478-21lldpd20:30:247
1779799126,5cyclictest0-21swapper/723:10:217
17797991211,1cyclictest0-21swapper/723:05:567
1779799120,12cyclictest0-21swapper/719:13:077
1779399126,5cyclictest0-21swapper/621:51:546
1779399126,5cyclictest0-21swapper/600:11:476
17793991211,1cyclictest0-21swapper/622:27:436
17793991211,1cyclictest0-21swapper/600:34:296
1779399120,12cyclictest0-21swapper/623:08:316
1778799128,1cyclictest191rcu_preempt19:25:145
1778799127,4cyclictest4970-21fschecks_count20:40:135
1778799127,1cyclictest191rcu_preempt21:00:205
1778799126,2cyclictest1146-21snmpd21:30:175
1778799126,2cyclictest1146-21snmpd21:30:175
1778799125,2cyclictest23874-21/usr/sbin/munin23:25:175
17787991211,1cyclictest0-21swapper/523:46:255
17787991211,1cyclictest0-21swapper/523:13:235
17787991211,1cyclictest0-21swapper/520:21:565
1778799120,1cyclictest0-21swapper/500:37:545
1777999127,4cyclictest0-21swapper/420:37:304
1777999126,5cyclictest0-21swapper/423:10:004
1777999120,12cyclictest0-21swapper/419:13:224
1777999120,1cyclictest0-21swapper/421:01:414
1777999120,1cyclictest0-21swapper/419:42:404
1777499129,2cyclictest1047-21NetworkManager23:00:003
1777499128,3cyclictest0-21swapper/321:06:403
1777499126,5cyclictest26926-21ssh00:38:033
1777499126,5cyclictest0-21swapper/323:06:563
17774991211,1cyclictest0-21swapper/323:19:433
17774991211,1cyclictest0-21swapper/322:08:193
1777499120,11cyclictest0-21swapper/323:46:013
1776999129,2cyclictest11941-21cstates22:40:112
1776999126,5cyclictest0-21swapper/222:18:432
1776999126,5cyclictest0-21swapper/221:42:352
17769991211,1cyclictest0-21swapper/222:52:002
17769991211,1cyclictest0-21swapper/222:37:352
17769991211,1cyclictest0-21swapper/221:39:372
17769991211,1cyclictest0-21swapper/221:39:372
17769991210,2cyclictest0-21swapper/200:15:552
1776999120,11cyclictest0-21swapper/219:29:392
1776399129,2cyclictest8987-21grep23:10:211
1776399129,2cyclictest3763-21sh19:40:141
1776399129,2cyclictest28423-21wget20:25:111
1776399129,2cyclictest281rcuc/120:30:111
1776399129,2cyclictest26876-21cut21:15:141
1776399129,2cyclictest25272-21basename00:00:151
1776399129,2cyclictest25161-21taskset21:47:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional