You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 18:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sun Mar 15, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10937521310,122pltrace0-21swapper/107:05:211
10937521050,97pltrace0-21swapper/207:05:272
10937521020,93pltrace0-21swapper/607:05:566
13722521010,93pltrace0-21swapper/507:07:275
391152990,2pltrace0-21swapper/209:42:252
2685552800,2pltrace0-21swapper/708:27:247
1385726652,10sleep30-21swapper/307:09:173
1382526147,10sleep70-21swapper/707:08:497
1363426046,10sleep00-21swapper/007:06:250
1386925945,10sleep40-21swapper/407:09:264
1678152350,1pltrace16782-21sed09:07:245
2299652320,4pltrace0-21swapper/707:25:137
3181052250,2pltrace0-21swapper/710:35:237
1407899230,22cyclictest0-21swapper/611:56:216
1406199230,22cyclictest0-21swapper/210:06:472
14078992219,2cyclictest25746-21uname08:25:166
1405899210,20cyclictest0-21swapper/112:07:341
731552200,7pltrace0-21swapper/707:53:107
2066452190,1pltrace21231-21/usr/sbin/munin10:15:147
14078991916,2cyclictest1178-21runrttasks10:42:156
731552180,13pltrace8815-21gltestperf07:55:141
14078991816,1cyclictest0-21swapper/607:15:066
14078991815,2cyclictest24726-21fschecks_time10:20:136
14078991815,2cyclictest24499-21taskset11:17:196
14054991812,3cyclictest0-21swapper/009:19:430
669652170,14pltrace54-21ksoftirqd/409:52:034
3117552170,4pltrace0-21swapper/712:31:287
2685552170,11pltrace54-21ksoftirqd/408:31:254
1658252170,11pltrace0-21swapper/207:15:022
1408299170,1cyclictest0-21swapper/709:56:227
14078991714,2cyclictest7883-21/usr/sbin/munin07:55:126
14078991714,2cyclictest21700-21/usr/sbin/munin09:15:196
14078991714,2cyclictest15825-21open_inodes07:10:186
14078991714,2cyclictest13950-21uname10:00:156
14078991714,2cyclictest13950-21uname10:00:156
14078991711,5cyclictest0-21swapper/610:59:076
14078991711,5cyclictest0-21swapper/610:17:306
14078991710,3cyclictest16916-21unixbench-2d12:00:206
1407399177,5cyclictest3458-21grep10:40:135
1407399177,5cyclictest18212-21tr07:15:125
14054991711,5cyclictest0-21swapper/007:40:020
731552160,11pltrace0-21swapper/007:55:140
669652160,0pltrace0-21swapper/709:49:497
2794952160,0pltrace0-21swapper/709:29:187
219552160,11pltrace0-21swapper/210:40:202
1658252160,10pltrace19468-21cat07:15:181
1508952160,11pltrace1146-21snmpd10:05:534
1408299160,9cyclictest9525-21fschecks_count08:55:147
1408299160,9cyclictest30797-21switchtime07:35:207
1408299160,11cyclictest0-21swapper/711:54:147
1408299160,1cyclictest0-21swapper/712:37:387
14078991615,1cyclictest0-21swapper/611:50:146
14078991614,1cyclictest9354-21idleruntime10:50:166
14078991614,1cyclictest1146-21snmpd09:02:136
14078991613,2cyclictest15657-21/usr/sbin/munin10:05:106
14078991613,2cyclictest14930-21idleruntime11:00:136
14078991613,2cyclictest1178-21runrttasks08:45:476
14078991613,2cyclictest1146-21snmpd12:27:536
14078991613,2cyclictest1146-21snmpd09:45:276
14078991613,2cyclictest1146-21snmpd08:12:426
14078991613,2cyclictest1146-21snmpd08:09:236
14078991610,5cyclictest0-21swapper/612:38:166
1407899161,8cyclictest0-21swapper/610:48:446
1407399169,4cyclictest62-21ksoftirqd/510:10:205
1407399169,4cyclictest62-21ksoftirqd/509:05:005
1407399169,4cyclictest62-21ksoftirqd/507:40:205
1407399164,6cyclictest62-21ksoftirqd/507:25:155
14073991611,2cyclictest62-21ksoftirqd/510:20:195
1407099169,6cyclictest0-21swapper/411:43:094
14070991613,2cyclictest0-21swapper/408:55:224
1407099160,2cyclictest0-21swapper/408:25:204
14061991611,3cyclictest0-21swapper/207:41:102
1406199161,8cyclictest0-21swapper/207:51:282
1406199160,14cyclictest0-21swapper/209:55:122
1405899168,1cyclictest191rcu_preempt11:10:201
14054991613,2cyclictest0-21swapper/011:35:120
1405499161,8cyclictest0-21swapper/008:49:510
1405499161,14cyclictest0-21swapper/012:21:430
1405499160,8cyclictest0-21swapper/012:26:460
1405499160,2cyclictest0-21swapper/008:25:180
1163852160,7pltrace0-21swapper/211:55:212
731552150,9pltrace8065-21cat07:55:122
606352150,1pltrace1283-21pltrace-run11:42:252
391152150,2pltrace0-21swapper/709:43:117
2685552150,3pltrace0-21swapper/708:30:587
219552150,9pltrace4493-21wc10:40:210
2002952150,8pltrace54-21ksoftirqd/412:10:134
1678152150,10pltrace54-21ksoftirqd/409:07:294
14082991510,4cyclictest0-21swapper/709:05:327
1408299150,9cyclictest1178-21runrttasks09:35:377
1408299150,1cyclictest0-21swapper/712:04:467
1408299150,1cyclictest0-21swapper/711:47:077
1408299150,1cyclictest0-21swapper/711:04:427
1408299150,1cyclictest0-21swapper/707:19:297
14078991514,1cyclictest0-21swapper/608:50:216
14078991513,1cyclictest25325-21timerandwakeup07:25:206
14078991513,1cyclictest11433-21cut08:00:156
14078991513,1cyclictest0-21swapper/607:40:156
14078991512,2cyclictest19334-21gltestperf10:10:136
14078991512,2cyclictest1249-21missed_timers10:35:166
1407899150,14cyclictest0-21swapper/611:11:566
1407899150,14cyclictest0-21swapper/609:05:066
1407399159,3cyclictest62-21ksoftirqd/510:55:205
1407399159,3cyclictest62-21ksoftirqd/508:35:135
1407399159,2cyclictest62-21ksoftirqd/509:50:145
1407399159,2cyclictest62-21ksoftirqd/509:10:175
1407399159,2cyclictest62-21ksoftirqd/507:20:115
1407399158,4cyclictest62-21ksoftirqd/509:40:215
1407399158,4cyclictest62-21ksoftirqd/509:35:115
1407399158,4cyclictest62-21ksoftirqd/508:15:245
1407399158,2cyclictest4632-21cut11:40:135
1407399158,2cyclictest3729-21smart_sda12:35:205
1407399158,1cyclictest191rcu_preempt10:45:155
1407399157,4cyclictest62-21ksoftirqd/508:02:595
1407399157,4cyclictest62-21ksoftirqd/507:35:215
1407399157,2cyclictest191rcu_preempt11:30:175
1407399155,7cyclictest62-21ksoftirqd/512:14:415
1407399155,6cyclictest62-21ksoftirqd/510:05:125
1407399154,5cyclictest62-21ksoftirqd/510:31:245
14073991511,2cyclictest62-21ksoftirqd/511:56:435
14073991511,1cyclictest191rcu_preempt07:49:135
14073991510,2cyclictest62-21ksoftirqd/509:31:385
1407399150,1cyclictest0-21swapper/511:50:015
1407099156,8cyclictest0-21swapper/411:20:114
1407099150,8cyclictest0-21swapper/407:32:254
1407099150,3cyclictest0-21swapper/409:28:014
1407099150,14cyclictest0-21swapper/409:55:154
1406599150,7cyclictest0-21swapper/308:34:293
1406199153,6cyclictest1055-13auditd08:10:002
14061991512,2cyclictest0-21swapper/207:10:182
1406199150,8cyclictest0-21swapper/209:13:042
1406199150,8cyclictest0-21swapper/208:12:592
1406199150,14cyclictest0-21swapper/210:38:012
1405499159,5cyclictest0-21swapper/010:26:510
1405499150,14cyclictest0-21swapper/009:30:460
1405499150,14cyclictest0-21swapper/008:15:290
1291052150,6pltrace78-21ksoftirqd/708:07:117
3135252140,10pltrace54-21ksoftirqd/407:38:494
2839052140,8pltrace54-21ksoftirqd/412:24:134
2731752140,10pltrace78-21ksoftirqd/711:25:147
2578052140,1pltrace0-21swapper/207:30:012
2407052140,9pltrace54-21ksoftirqd/408:22:404
2002952140,9pltrace22222-21grep12:10:200
1658252140,5pltrace0-21swapper/007:15:020
1408299149,4cyclictest0-21swapper/708:47:437
1408299148,5cyclictest0-21swapper/712:26:077
1408299146,7cyclictest1146-21snmpd09:53:417
1408299142,10cyclictest1561-21sed08:40:137
1408299140,8cyclictest2856852pltrace07:32:247
1408299140,8cyclictest2560252pltrace12:17:257
1408299140,7cyclictest28404-21smart_sda10:25:217
1408299140,7cyclictest1787752pltrace10:07:267
1408299140,13cyclictest1146-21snmpd12:12:497
1408299140,12cyclictest30693-21seq09:32:067
1408299140,12cyclictest24019-21/usr/sbin/munin10:20:157
1408299140,12cyclictest20191-21chrt12:09:407
1408299140,1cyclictest0-21swapper/710:54:107
1408299140,1cyclictest0-21swapper/708:00:017
1407899147,4cyclictest0-21swapper/609:10:116
14078991412,1cyclictest899-21cstates08:40:126
14078991412,1cyclictest7265-21fschecks_count11:45:116
14078991412,1cyclictest5706-21timerwakeupswit11:40:226
14078991412,1cyclictest26815-21timerandwakeup11:20:186
14078991412,1cyclictest24195-21fschecks_time12:15:136
14078991412,1cyclictest0-21swapper/609:30:186
14078991411,2cyclictest31612-21gltestperf11:30:156
1407899140,7cyclictest0-21swapper/609:50:086
1407899140,13cyclictest0-21swapper/608:15:506
1407399149,1cyclictest731552pltrace07:55:125
1407399148,3cyclictest62-21ksoftirqd/512:15:145
1407399148,3cyclictest62-21ksoftirqd/511:20:115
1407399148,3cyclictest62-21ksoftirqd/508:09:225
1407399148,2cyclictest62-21ksoftirqd/512:20:165
1407399148,2cyclictest62-21ksoftirqd/509:25:205
1407399148,1cyclictest191rcu_preempt11:45:095
1407399147,4cyclictest62-21ksoftirqd/511:12:215
1407399147,2cyclictest21607-21cstates10:15:125
1407399145,5cyclictest0-21swapper/507:30:135
1407399144,7cyclictest62-21ksoftirqd/508:33:405
14073991411,2cyclictest0-21swapper/509:46:535
14073991410,2cyclictest191rcu_preempt07:14:475
14073991410,1cyclictest191rcu_preempt10:28:355
1407099149,2cyclictest0-21swapper/410:47:404
1407099147,6cyclictest0-21swapper/409:42:254
1407099140,8cyclictest0-21swapper/408:42:304
1407099140,3cyclictest0-21swapper/409:15:164
1407099140,13cyclictest0-21swapper/409:00:174
14065991413,1cyclictest0-21swapper/307:10:133
1406599140,14cyclictest0-21swapper/307:45:113
1405899148,1cyclictest191rcu_preempt10:17:251
1405899140,3cyclictest0-21swapper/110:11:181
14054991413,1cyclictest0-21swapper/010:35:170
14054991412,1cyclictest0-21swapper/011:30:010
1405499140,3cyclictest0-21swapper/011:28:340
1405499140,13cyclictest0-21swapper/009:55:160
1405499140,1cyclictest0-21swapper/009:50:180
1163852140,1pltrace0-21swapper/011:55:000
669652130,3pltrace38-21ksoftirqd/209:52:222
2623852130,3pltrace1178-21runrttasks10:23:032
2516352130,4pltrace0-21swapper/709:22:377
2516352130,3pltrace25243-21taskset09:23:290
1849752130,3pltrace18609-21taskset08:13:550
156252130,6pltrace0-21swapper/012:36:430
1408299131,10cyclictest23550-21missed_timers11:15:167
1408299130,8cyclictest5553-21/usr/sbin/munin10:45:117
1408299130,8cyclictest13034-21ls10:00:127
1408299130,8cyclictest13034-21ls10:00:117
1408299130,8cyclictest0-21swapper/711:22:397
1408299130,13cyclictest0-21swapper/712:20:187
1408299130,11cyclictest6261-21aten_r4power_vo08:50:127
1408299130,11cyclictest32652-21tune2fs07:40:127
1408299130,11cyclictest31594-21grep11:30:157
1408299130,11cyclictest19794-21cut08:15:157
1408299130,11cyclictest19642-21cpu11:10:127
1408299130,11cyclictest17428-21sort08:10:177
1408299130,11cyclictest14609-21grep07:10:127
1407899132,7cyclictest70-21ksoftirqd/612:20:176
14078991312,1cyclictest0-21swapper/612:05:106
14078991312,1cyclictest0-21swapper/609:40:136
14078991312,1cyclictest0-21swapper/608:36:116
14078991312,1cyclictest0-21swapper/608:30:116
14078991312,1cyclictest0-21swapper/607:45:136
14078991312,1cyclictest0-21swapper/607:30:126
14078991311,2cyclictest31307-21threads10:30:206
14078991311,1cyclictest0-21swapper/609:25:116
14078991311,1cyclictest0-21swapper/609:00:006
14078991311,1cyclictest0-21swapper/607:50:196
14078991311,1cyclictest0-21swapper/607:35:216
14078991310,2cyclictest462-21switchtime09:35:196
14078991310,2cyclictest32088-21cstates12:30:126
14078991310,2cyclictest10055-21/usr/sbin/munin09:55:176
1407899130,8cyclictest24397-21open_inodes09:20:176
1407899130,7cyclictest21228-21cstates07:20:116
1407899130,12cyclictest0-21swapper/608:21:566
1407399139,2cyclictest191rcu_preempt08:20:505
1407399138,3cyclictest191rcu_preempt12:25:145
1407399138,2cyclictest62-21ksoftirqd/510:39:025
1407399138,1cyclictest191rcu_preempt11:25:145
1407399137,2cyclictest62-21ksoftirqd/508:55:125
1407399137,2cyclictest62-21ksoftirqd/508:45:105
1407399137,2cyclictest62-21ksoftirqd/508:10:135
1407399136,4cyclictest62-21ksoftirqd/512:08:545
1407399136,4cyclictest0-21swapper/509:15:215
1407399135,5cyclictest62-21ksoftirqd/509:22:225
1407399134,6cyclictest62-21ksoftirqd/508:40:135
1407399133,6cyclictest62-21ksoftirqd/509:55:155
1407399133,6cyclictest13141-21cpuspeed_turbos10:00:125
1407399133,6cyclictest13141-21cpuspeed_turbos10:00:125
14073991312,1cyclictest0-21swapper/511:00:205
14073991310,1cyclictest191rcu_preempt12:00:375
14073991310,1cyclictest191rcu_preempt11:35:155
1407099137,5cyclictest0-21swapper/409:38:104
1407099136,2cyclictest1146-21snmpd11:18:594
1407099130,13cyclictest0-21swapper/410:31:224
1407099130,13cyclictest0-21swapper/408:47:274
1407099130,12cyclictest0-21swapper/411:32:274
1407099130,12cyclictest0-21swapper/411:05:034
14065991312,1cyclictest0-21swapper/311:20:023
14065991312,1cyclictest0-21swapper/308:20:143
14065991312,1cyclictest0-21swapper/307:25:173
1406599130,6cyclictest0-21swapper/309:39:013
1406599130,13cyclictest0-21swapper/309:52:273
1406599130,1cyclictest0-21swapper/308:42:113
1406199138,4cyclictest0-21swapper/212:02:302
1406199138,3cyclictest0-21swapper/210:54:102
14061991312,1cyclictest0-21swapper/208:53:452
1406199130,13cyclictest0-21swapper/209:22:262
1406199130,1cyclictest0-21swapper/208:25:202
1405899139,1cyclictest191rcu_preempt09:55:211
1405899137,4cyclictest520-21missed_timers12:30:161
1405899137,2cyclictest25883-21/usr/sbin/munin09:25:111
1405899130,6cyclictest0-21swapper/111:01:331
1405899130,5cyclictest14956-21expr09:05:121
1405899130,13cyclictest0-21swapper/111:20:181
1405899130,12cyclictest0-21swapper/108:47:001
14054991312,1cyclictest0-21swapper/008:35:190
14054991310,2cyclictest19408-21smart_sda12:05:190
1405499131,11cyclictest0-21swapper/007:47:560
1405499130,12cyclictest0-21swapper/007:21:310
1163852130,8pltrace14174-21sed11:55:220
108752130,5pltrace0-21swapper/209:38:312
1056052130,5pltrace0-21swapper/210:57:222
839152120,6pltrace0-21swapper/008:55:460
3242752120,7pltrace1478-21lldpd08:38:304
2685552120,5pltrace0-21swapper/008:31:380
2623852120,5pltrace0-21swapper/010:23:060
2516352120,3pltrace54-21ksoftirqd/409:23:174
2407052120,5pltrace0-21swapper/008:22:410
219552120,5pltrace0-21swapper/110:40:201
1895452120,7pltrace54-21ksoftirqd/411:10:214
1658252120,6pltrace0-21swapper/007:14:580
156252120,5pltrace0-21swapper/212:37:062
1508952120,5pltrace0-21swapper/010:07:090
1408299122,8cyclictest18368-21sed11:05:187
1408299120,7cyclictest0-21swapper/710:40:127
1408299120,11cyclictest31904-21chrt10:33:427
1408299120,11cyclictest3074-21tune2fs07:45:137
1408299120,11cyclictest22733-21grep08:20:137
1408299120,11cyclictest21877-21irqstats_LOC07:20:147
1408299120,11cyclictest17353-21chrt09:10:027
1408299120,11cyclictest14412-21chrt11:57:237
1408299120,11cyclictest11957-21cut09:00:117
1408299120,11cyclictest0-21swapper/711:35:147
1408299120,10cyclictest7994-21cpu07:55:137
1408299120,10cyclictest30297-21basename08:35:107
1408299120,10cyclictest20379-21cpuspeed_turbos09:15:117
1408299120,10cyclictest18522-21cut10:10:117
1408299120,10cyclictest11164-21cut10:55:127
1407899129,2cyclictest20755-21cpuspeed_turbos12:10:116
1407899129,2cyclictest18092-21strings11:05:156
14078991211,1cyclictest0-21swapper/611:35:116
14078991211,1cyclictest0-21swapper/611:25:176
14078991210,1cyclictest0-21swapper/610:25:216
1407399129,2cyclictest5320-21expr07:50:115
1407399129,2cyclictest24953-21cpuspeed_turbos08:25:135
1407399129,2cyclictest22320-21aten_r4power_cu11:15:125
1407399129,2cyclictest17279-21sh11:05:115
1407399126,2cyclictest0-21swapper/512:30:005
1407399124,4cyclictest62-21ksoftirqd/508:50:215
14073991210,1cyclictest0-21swapper/510:50:125
14070991211,1cyclictest0-21swapper/411:50:244
14070991210,1cyclictest0-21swapper/411:02:084
1407099120,11cyclictest0-21swapper/411:36:244
1407099120,11cyclictest0-21swapper/408:51:484
1407099120,11cyclictest0-21swapper/408:00:094
1406599129,2cyclictest20943-21grep11:10:183
1406599127,4cyclictest14093-21cat11:55:223
1406599124,4cyclictest0-21swapper/310:11:183
14065991211,1cyclictest441rcuc/309:05:313
14065991211,1cyclictest0-21swapper/312:26:253
14065991211,1cyclictest0-21swapper/312:20:163
14065991211,1cyclictest0-21swapper/311:04:593
14065991211,1cyclictest0-21swapper/309:25:343
14065991211,1cyclictest0-21swapper/309:23:283
14065991211,1cyclictest0-21swapper/308:46:333
14065991211,1cyclictest0-21swapper/308:05:133
14065991211,1cyclictest0-21swapper/307:16:223
14065991210,1cyclictest0-21swapper/309:55:143
1406599120,12cyclictest0-21swapper/307:40:303
1406599120,1cyclictest0-21swapper/310:20:573
1406599120,1cyclictest0-21swapper/308:27:563
1406199129,2cyclictest1478-21lldpd09:46:022
1406199127,3cyclictest0-21swapper/211:05:462
1406199126,3cyclictest0-21swapper/211:46:152
14061991211,1cyclictest0-21swapper/207:39:562
1406199120,11cyclictest0-21swapper/209:27:282
1405899129,2cyclictest28134-21fschecks_time08:30:121
1405899129,2cyclictest22767-21unixbench_singl07:20:211
1405899129,1cyclictest191rcu_preempt09:35:201
1405899128,1cyclictest191rcu_preempt07:30:171
1405899120,6cyclictest2019-21cpuspeed_turbos09:40:111
1405899120,6cyclictest0-21swapper/110:34:321
1405899120,12cyclictest0-21swapper/108:05:221
1405499126,5cyclictest0-21swapper/009:25:430
14054991211,1cyclictest0-21swapper/011:10:160
14054991211,1cyclictest0-21swapper/007:42:230
1405499120,11cyclictest0-21swapper/011:22:200
108752120,6pltrace0-21swapper/009:38:140
281752110,4pltrace0-21swapper/208:44:122
2731752110,4pltrace0-21swapper/111:26:251
2731752110,1pltrace30000-21taskset11:26:172
2578052110,4pltrace27032-21cut07:30:140
2578052110,3pltrace0-21swapper/207:30:092
1678152110,1pltrace0-21swapper/009:07:350
1658252110,4pltrace0-21swapper/107:14:511
1408299110,10cyclictest4097-21cpuspeed_turbos11:40:127
1407099117,3cyclictest1146-21snmpd10:01:384
1407099117,3cyclictest1146-21snmpd10:01:384
1407099117,2cyclictest0-21swapper/409:32:284
1407099115,5cyclictest0-21swapper/407:46:104
14070991110,1cyclictest0-21swapper/412:19:224
1407099110,1cyclictest0-21swapper/410:26:214
1407099110,1cyclictest0-21swapper/407:17:444
1406599119,1cyclictest0-21swapper/311:40:223
1406599119,1cyclictest0-21swapper/311:25:223
1406599118,2cyclictest0-21swapper/310:55:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional