You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 13:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Mar 31, 2026 00:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
322629912464,53cyclictest300-21rs:main0
32262991063,30cyclictest10812-21unin-run23:54:110
32262991054,9cyclictest0-21swapper/022:29:250
32262991045,9cyclictest0-21swapper/022:04:270
32262991023,32cyclictest19734-21cstates23:14:240
32262991022,32cyclictest87-21kswapd019:04:140
32262991013,34cyclictest19608-21sh21:24:410
32262991004,8cyclictest0-21swapper/020:49:210
32262991004,8cyclictest0-21swapper/020:49:210
32262991004,89cyclictest0-21swapper/020:29:230
32262991004,85cyclictest18879-21latency_hist21:24:130
32262991003,8cyclictest0-21swapper/019:39:260
32262991003,8cyclictest0-21swapper/019:39:260
32262991003,28cyclictest14493-21ssh23:59:430
32262991003,28cyclictest14493-21ssh23:59:430
32262991002,93cyclictest0-21swapper/000:24:240
3226299994,67cyclictest5028-21runrttasks22:49:320
3226299993,27cyclictest990-21ssh23:37:190
3226299983,11cyclictest0-21swapper/020:19:240
3226299974,8cyclictest0-21swapper/022:34:210
3226299974,7cyclictest0-21swapper/023:54:210
3226299974,7cyclictest0-21swapper/023:54:210
32262999743,47cyclictest0-21swapper/021:59:390
32262999743,47cyclictest0-21swapper/021:59:390
32262999742,47cyclictest0-21swapper/019:34:100
3226299973,8cyclictest0-21swapper/023:04:220
3226299972,27cyclictest5547-21taskset23:44:400
3226299964,84cyclictest0-21swapper/023:19:230
3226299964,84cyclictest0-21swapper/023:19:230
3226299964,80cyclictest16518-21ssh22:14:210
3226299964,80cyclictest16518-21ssh22:14:210
3226299955,8cyclictest0-21swapper/021:44:240
3226299954,9cyclictest0-21swapper/021:39:240
3226299954,9cyclictest0-21swapper/021:39:240
3226299954,9cyclictest0-21swapper/020:44:230
3226299954,9cyclictest0-21swapper/020:44:230
3226299954,83cyclictest7065-21ssh21:04:270
3226299944,9cyclictest0-21swapper/019:14:210
3226299944,78cyclictest0-21swapper/000:09:250
3226299943,33cyclictest13388-21modprobe23:04:090
3226299943,33cyclictest13388-21modprobe23:04:090
3226299943,27cyclictest22525-21taskset20:11:200
3226299943,26cyclictest14120-21awk22:09:390
3226299942,13cyclictest4790-21ssh21:54:410
3226299937,77cyclictest0-21swapper/020:59:280
3226299937,77cyclictest0-21swapper/020:59:280
3226299937,77cyclictest0-21swapper/020:59:280
3226299935,8cyclictest0-21swapper/019:54:220
3226299934,29cyclictest19726-21fschecks_count22:19:320
3226299933,9cyclictest0-21swapper/019:44:220
3226299933,9cyclictest0-21swapper/019:44:220
3226299933,33cyclictest352-21runrttasks21:36:550
3226299933,32cyclictest22405-21/usr/sbin/munin22:24:320
3226299933,32cyclictest22405-21/usr/sbin/munin22:24:320
3226299933,32cyclictest16869-21sh23:09:310
3226299933,32cyclictest16869-21sh23:09:310
32277999262,23cyclictest23025-21kworker/u17:2+events_unbound20:19:092
3226299924,8cyclictest0-21swapper/020:39:240
3226299924,8cyclictest0-21swapper/020:39:240
3226299923,24cyclictest793-21ssh22:42:090
32269999167,18cyclictest16852-21fschecks_count23:09:311
32269999167,18cyclictest16852-21fschecks_count23:09:311
3226299914,31cyclictest2172-21sh23:39:310
3226299914,31cyclictest2172-21sh23:39:310
3226299914,28cyclictest2248-21sh00:34:120
3226299913,9cyclictest0-21swapper/019:49:220
3226299913,30cyclictest23988-21head20:14:460
3226299913,26cyclictest20458-21/usr/sbin/munin20:04:430
32262999132,51cyclictest1429-21acpi21:49:220
3226299913,22cyclictest6818-21latency_hist21:04:120
3226299912,31cyclictest23783-21ssh00:14:430
3226299912,31cyclictest23783-21ssh00:14:430
3226299904,30cyclictest87-21kswapd022:44:260
3226299904,30cyclictest87-21kswapd022:44:260
3226299904,25cyclictest11060-21unixbench_multi21:09:490
3226299903,30cyclictest26690-21/usr/sbin/munin00:19:420
3226299903,27cyclictest5541-21grep19:19:320
3226299903,11cyclictest0-21swapper/019:09:250
3226299903,11cyclictest0-21swapper/019:09:250
3226299902,30cyclictest13018-21/usr/sbin/munin21:14:390
3226299902,29cyclictest8694-21/usr/sbin/munin22:54:450
3226299902,29cyclictest18451-21grep19:59:320
3226299902,26cyclictest22983-21unixbench_multi21:29:470
3226299902,26cyclictest22983-21unixbench_multi21:29:470
32269998968,12cyclictest0-21swapper/123:34:391
3226299895,73cyclictest0-21swapper/020:34:230
3226299893,27cyclictest25502-21/usr/sbin/munin23:24:260
3226299886,10cyclictest0-21swapper/023:29:350
3226299886,10cyclictest0-21swapper/023:29:350
3226299884,31cyclictest7886-21vmstat19:24:490
3226299883,28cyclictest19913-21ssh00:09:120
3226299883,28cyclictest11479-21munin-plugin-st19:39:100
32262998736,43cyclictest0-21swapper/019:04:360
32269998662,17cyclictest3541-21apt-key20:54:141
32269998662,17cyclictest3541-21apt-key20:54:141
32269998646,33cyclictest0-21swapper/100:04:251
3227799853,54cyclictest8121-21apt-get23:49:402
32269998567,11cyclictest0-21swapper/119:34:321
32277998462,14cyclictest2952-21systemd-journal22:01:342
32277998462,14cyclictest2952-21systemd-journal22:01:342
32277998367,10cyclictest18575-21kworker/u17:2-rpciod21:34:352
32277998358,19cyclictest277-21dbus-daemon23:14:092
32277998358,19cyclictest277-21dbus-daemon23:14:092
3226999833,52cyclictest8266-21ssh23:49:481
32277998266,10cyclictest21653-21latency_hist21:29:142
32269998241,32cyclictest0-21swapper/119:54:211
32277998120,26cyclictest0-21swapper/222:34:122
32269998158,12cyclictest0-21swapper/120:29:131
32269998140,31cyclictest16508-21ssh22:14:201
32269998140,31cyclictest16508-21ssh22:14:201
32269998137,27cyclictest0-21swapper/121:44:241
32269998058,12cyclictest0-21swapper/120:44:331
32269998058,12cyclictest0-21swapper/120:44:331
32269998057,13cyclictest0-21swapper/121:30:291
32269998057,13cyclictest0-21swapper/121:30:291
32277997950,20cyclictest31308-21fschecks_count20:39:312
32277997950,20cyclictest31308-21fschecks_count20:39:312
32269997957,13cyclictest0-21swapper/119:49:121
32269997957,13cyclictest0-21swapper/119:49:121
32269997956,12cyclictest0-21swapper/120:44:131
32269997956,12cyclictest0-21swapper/120:44:131
32269997954,13cyclictest0-21swapper/120:19:331
32269997951,17cyclictest277-21dbus-daemon19:09:121
32269997929,27cyclictest6637-21munin-plugin-st19:24:101
32277997862,11cyclictest32253-21cyclictest19:59:322
32277997859,11cyclictest13754-21cpu23:04:272
3227799783,48cyclictest3750-21apt-get20:54:222
3227799783,48cyclictest3750-21apt-get20:54:222
3227799783,48cyclictest3750-21apt-get20:54:222
32269997855,12cyclictest0-21swapper/123:19:121
32269997854,14cyclictest0-21swapper/119:09:341
32269997854,14cyclictest0-21swapper/119:09:341
32277997758,13cyclictest6702-21diskmemload22:45:352
32277997758,13cyclictest6702-21diskmemload22:45:352
32277997736,36cyclictest0-21swapper/200:24:192
32277997733,32cyclictest0-21swapper/220:19:242
32277997728,26cyclictest0-21swapper/223:24:262
32277997722,35cyclictest2992-21grep23:39:472
32277997722,35cyclictest2992-21grep23:39:472
32269997762,9cyclictest0-21swapper/119:24:331
32269997758,10cyclictest0-21swapper/121:49:481
32269997757,10cyclictest0-21swapper/120:04:131
32269997753,13cyclictest0-21swapper/121:24:321
32269997751,20cyclictest16203-21sh00:02:221
32269997751,20cyclictest16203-21sh00:02:221
32269997742,31cyclictest0-21swapper/123:39:251
32269997742,31cyclictest0-21swapper/123:39:251
32269997738,27cyclictest0-21swapper/122:09:231
32269997728,35cyclictest0-21swapper/119:29:211
321972778,44sleep232208-21latency_hist19:04:142
3081627711,41sleep130851-21grep18:59:281
32277997661,11cyclictest32253-21cyclictest20:49:122
32277997661,11cyclictest32253-21cyclictest20:49:122
32277997655,12cyclictest11273-21ssh22:04:462
32277997651,19cyclictest22233-21fschecks_time21:29:322
32277997651,19cyclictest22233-21fschecks_time21:29:322
32269997656,13cyclictest0-21swapper/122:49:121
32269997656,13cyclictest0-21swapper/122:49:121
32269997653,13cyclictest0-21swapper/119:39:471
32269997653,13cyclictest0-21swapper/119:39:471
32269997645,25cyclictest29140-21mailstats22:34:441
32269997641,21cyclictest5212-21apt-get20:59:291
32269997641,21cyclictest5212-21apt-get20:59:291
32269997641,21cyclictest5212-21apt-get20:59:291
32269997637,28cyclictest0-21swapper/122:29:231
32277997554,13cyclictest0-21swapper/222:24:342
32277997554,13cyclictest0-21swapper/222:24:342
32277997552,16cyclictest7734-21sh22:54:192
32277997545,19cyclictest22953-21latency_hist20:14:132
32277997534,29cyclictest0-21swapper/219:54:222
32269997560,9cyclictest0-21swapper/122:54:451
32269997558,10cyclictest0-21swapper/100:29:441
32269997555,11cyclictest0-21swapper/122:09:121
32269997539,31cyclictest0-21swapper/100:09:221
32269997538,29cyclictest0-21swapper/119:14:211
3226999753,47cyclictest4675-21apt-get21:54:281
3109327522,32sleep30-21swapper/318:59:383
32277997456,11cyclictest0-21swapper/221:19:402
32277997451,14cyclictest0-21swapper/221:00:462
32269997458,11cyclictest32253-21cyclictest21:19:101
32269997453,13cyclictest0-21swapper/120:04:311
32269997431,7cyclictest181rcu_preempt23:54:531
32269997431,7cyclictest181rcu_preempt23:54:531
32269997423,20cyclictest6702-21diskmemload23:29:191
32269997423,20cyclictest6702-21diskmemload23:29:191
32277997356,9cyclictest0-21swapper/219:39:472
32277997356,9cyclictest0-21swapper/219:39:472
32269997360,7cyclictest0-21swapper/121:09:481
32269997358,11cyclictest32253-21cyclictest22:39:351
32269997357,8cyclictest0-21swapper/123:24:281
32269997353,8cyclictest0-21swapper/119:54:121
32269997353,12cyclictest0-21swapper/120:29:321
32269997351,16cyclictest11735-21unixbench_multi22:59:471
32269997351,16cyclictest11735-21unixbench_multi22:59:471
32269997341,20cyclictest29509-21apt-get20:34:261
32269997337,26cyclictest0-21swapper/121:39:241
32269997337,26cyclictest0-21swapper/121:39:241
32277997256,11cyclictest7683-21apt-key23:49:122
32277997252,14cyclictest9772-21sh21:09:122
32277997251,15cyclictest19676-21snmpd21:54:102
32277997248,14cyclictest0-21swapper/219:24:322
32277997238,27cyclictest17177-21ssh00:04:252
32277997231,32cyclictest15675-21cron21:19:102
32269997239,21cyclictest7032-21apt-get21:04:221
32277997154,11cyclictest2952-21systemd-journal22:51:372
32277997150,13cyclictest2265-21ssh00:34:122
32277997148,12cyclictest0-21swapper/220:49:462
32277997148,12cyclictest0-21swapper/220:49:462
32277997146,19cyclictest4694-21kworker/u18:2+events_unbound23:19:092
32277997132,34cyclictest0-21swapper/223:34:222
32269997140,23cyclictest23080-21ssh00:14:211
32269997140,23cyclictest23080-21ssh00:14:211
32269997134,8cyclictest181rcu_preempt23:49:091
32277997052,12cyclictest22736-21latency_hist00:14:122
32277997051,14cyclictest11966-21sh23:54:532
32277997051,14cyclictest11966-21sh23:54:532
32277997050,15cyclictest4298-21sendmail_mailqu19:14:422
32277997048,15cyclictest32253-21cyclictest22:59:302
32277997048,15cyclictest32253-21cyclictest22:59:302
32277997048,15cyclictest0-21swapper/221:14:092
32277997044,18cyclictest31839-21timerwakeupswit21:44:462
32277997042,12cyclictest2047-21apt-get19:09:252
32277997042,12cyclictest2047-21apt-get19:09:252
32277997033,28cyclictest10135-21apt-get19:34:262
32277997031,25cyclictest5292-21apt-get19:19:222
32277997028,31cyclictest25738-21modprobe00:19:092
32277997028,31cyclictest25738-21modprobe00:19:092
32277997023,32cyclictest16839-21/usr/sbin/munin22:14:322
32277997023,32cyclictest16839-21/usr/sbin/munin22:14:322
32269997053,8cyclictest0-21swapper/122:24:361
32269997053,8cyclictest0-21swapper/122:24:361
32269997038,8cyclictest181rcu_preempt22:49:361
32269997037,26cyclictest2952-21systemd-journal23:08:371
32269997034,28cyclictest26237-21seq00:19:311
32277996951,12cyclictest22135-21systemd-run22:24:102
3227799692,57cyclictest4675-21apt-get21:54:272
32269996938,24cyclictest21419-21cpu20:09:261
32269996934,24cyclictest25301-21grep23:24:121
32269996934,24cyclictest25301-21grep23:24:121
32277996849,12cyclictest297-21systemd-logind20:29:102
32277996848,11cyclictest0-21swapper/223:59:472
32277996848,11cyclictest0-21swapper/223:59:472
32269996836,20cyclictest7661-21ssh21:59:261
32269996836,20cyclictest7661-21ssh21:59:261
32269996835,24cyclictest29298-21ssh00:24:311
32269996835,23cyclictest15980-21/usr/sbin/munin21:19:491
32269996833,26cyclictest23499-21taskset20:14:361
3228899673,51cyclictest11219-21taskset22:04:433
32277996751,11cyclictest32253-21cyclictest20:09:102
32277996750,12cyclictest352-21runrttasks22:09:462
32277996749,10cyclictest0-21swapper/223:19:402
32277996749,10cyclictest0-21swapper/223:19:402
32277996748,10cyclictest0-21swapper/221:39:452
32277996748,10cyclictest0-21swapper/221:39:452
32277996743,18cyclictest15716-21timerandwakeup19:49:462
32277996726,10cyclictest0-21swapper/219:09:132
32269996739,20cyclictest19946-21/usr/sbin/munin22:19:361
32269996733,25cyclictest24964-21apt-get21:34:221
3227799664,49cyclictest29509-21apt-get20:34:222
32277996630,23cyclictest0-21swapper/219:29:212
32269996631,28cyclictest5648-21chrt20:59:591
62942650,7sleep30-21swapper/323:45:343
32277996523,31cyclictest31616-21fschecks_time22:39:312
3228899644,45cyclictest25367-21taskset00:18:123
3228899644,45cyclictest25367-21taskset00:18:123
3228899643,48cyclictest10818-21taskset19:34:443
3228899643,47cyclictest17178-21chrt00:04:253
3228899643,47cyclictest15385-21chrt22:11:573
95832630,5sleep30-21swapper/322:02:163
95832630,5sleep30-21swapper/322:02:163
3228899633,47cyclictest1169-21chrt20:44:463
3228899633,47cyclictest1169-21chrt20:44:463
3228899633,46cyclictest27249-21chrt22:32:003
3228899633,45cyclictest19767-21chrt23:14:243
3228899632,46cyclictest27924-21chrt23:28:313
3228899632,46cyclictest13057-21taskset21:14:243
32277996350,10cyclictest32253-21cyclictest23:30:082
32277996350,10cyclictest32253-21cyclictest23:30:082
32277996346,7cyclictest0-21swapper/200:22:482
32277996331,21cyclictest13345-21apt-get19:44:252
32277996331,21cyclictest13345-21apt-get19:44:252
32277996325,27cyclictest31007-21ls22:39:092
3228899623,46cyclictest12865-21chrt23:02:453
3228899623,46cyclictest12865-21chrt23:02:453
3228899623,45cyclictest32535-21chrt00:29:393
3228899623,44cyclictest31515-21chrt22:39:303
3228899622,47cyclictest22455-21chrt22:24:263
3228899622,47cyclictest22455-21chrt22:24:263
3228899622,47cyclictest21207-21taskset21:27:583
32277996228,26cyclictest0-21swapper/220:29:232
3228899614,42cyclictest3351-21taskset23:40:353
3228899614,42cyclictest3351-21taskset23:40:353
3228899613,46cyclictest21548-21taskset22:22:353
3228899613,45cyclictest5166-21chrt19:19:133
3228899613,45cyclictest31224-21chrt21:44:323
3228899613,45cyclictest28494-21taskset22:34:273
3228899613,45cyclictest2832-21taskset21:50:363
3228899613,45cyclictest11001-21chrt23:54:143
3228899613,44cyclictest16472-21taskset00:03:083
3228899613,44cyclictest16472-21taskset00:03:083
3228899613,43cyclictest3153-21chrt22:45:263
3228899613,43cyclictest3153-21chrt22:45:263
3228899613,42cyclictest26071-21taskset00:19:253
3228899612,46cyclictest30348-21chrt23:32:003
3228899612,46cyclictest30348-21chrt23:32:003
3228899612,44cyclictest24481-21chrt23:22:073
3228899612,44cyclictest24481-21chrt23:22:073
3228899603,45cyclictest11277-21taskset21:10:153
3228899603,44cyclictest9676-21chrt22:57:053
3228899603,44cyclictest29753-21chrt21:41:493
3228899603,44cyclictest29753-21chrt21:41:493
3228899603,44cyclictest21615-21chrt00:11:023
3228899603,44cyclictest14344-21chrt23:04:443
3228899603,43cyclictest18545-21taskset19:59:343
3228899603,43cyclictest17540-21chrt21:21:013
3228899602,46cyclictest32621-21chrt19:04:303
3228899602,44cyclictest27313-21chrt20:25:313
3228899602,44cyclictest16509-21chrt22:14:203
3228899602,44cyclictest16509-21chrt22:14:203
3228899593,44cyclictest7384-21chrt22:54:053
3228899593,44cyclictest30610-21chrt00:26:153
3228899593,44cyclictest2328-21taskset20:49:363
3228899593,44cyclictest2328-21taskset20:49:363
3228899593,43cyclictest30698-21chrt20:37:253
3228899593,43cyclictest26475-21chrt21:35:573
3228899593,43cyclictest22236-21chrt21:29:323
3228899593,43cyclictest22236-21chrt21:29:323
3228899593,43cyclictest16228-21taskset19:53:313
3228899593,42cyclictest8904-21chrt21:06:583
3228899593,42cyclictest4968-21chrt20:57:343
3228899593,42cyclictest4968-21chrt20:57:343
3228899593,42cyclictest4968-21chrt20:57:343
3228899593,42cyclictest25243-21chrt20:19:433
3228899592,45cyclictest631-21chrt23:36:183
3228899592,45cyclictest31118-21chrt20:39:243
3228899592,45cyclictest31118-21chrt20:39:243
3228899592,43cyclictest21570-21chrt20:09:313
3228899583,42cyclictest9748-21chrt19:33:103
3228899583,42cyclictest5299-21chrt19:19:253
3228899583,42cyclictest13886-21chrt19:44:403
3228899583,42cyclictest13886-21chrt19:44:403
3228899583,41cyclictest14002-21chrt23:59:123
3228899583,41cyclictest14002-21chrt23:59:123
3228899582,43cyclictest6668-21chrt21:57:213
3228899574,42cyclictest12947-21chrt19:42:373
3228899574,42cyclictest12947-21chrt19:42:373
3228899573,43cyclictest1831-21chrt19:09:203
3228899573,43cyclictest1831-21chrt19:09:203
3228899573,42cyclictest7914-21chrt19:25:123
3228899573,42cyclictest28977-21chrt20:31:183
3228899573,41cyclictest17583-21chrt19:54:533
3228899572,43cyclictest17986-21chrt23:10:313
3228899572,43cyclictest17986-21chrt23:10:313
3228899572,42cyclictest24243-21chrt20:17:253
3228899563,42cyclictest6651-21chrt21:03:463
3228899552,41cyclictest20860-21chrt20:05:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional