You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 20:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Mar 28, 2026 12:44:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31925218814,62sleep031928-21cron07:04:190
31265212613,78sleep219676-21snmpd07:00:012
320949911660,18cyclictest32078-21cyclictest11:59:421
320949911660,18cyclictest32078-21cyclictest11:59:421
320879911453,25cyclictest24063-21ntp_kernel_pll_12:14:490
32102991132,87cyclictest29232-21ssh11:29:372
32102991002,71cyclictest22272-21apt-get09:29:482
32087999472,15cyclictest1398-21munin-node11:16:040
32102999175,10cyclictest10612-21modprobe10:59:182
32102999067,17cyclictest6371-21sh11:44:512
3208799896,68cyclictest29295-21ssh11:29:490
32094998864,16cyclictest2572-21latency_hist11:39:231
32094998855,10cyclictest181rcu_preempt11:09:321
32087998742,19cyclictest13104-21sh09:14:320
3209499863,44cyclictest31-21ksoftirqd/109:24:351
32087998658,20cyclictest20360-21latency_hist12:09:220
3208799864,43cyclictest5497-21awk09:54:530
32094998456,19cyclictest14606-21apt-key07:49:231
32094998410,42cyclictest29189-21apt-get11:29:371
32087998457,20cyclictest32078-21cyclictest11:24:360
32087998457,18cyclictest2952-21systemd-journal10:28:300
32102998368,11cyclictest32078-21cyclictest08:49:422
32102998359,18cyclictest21902-21latency_hist09:29:222
32094998348,12cyclictest181rcu_preempt07:29:501
32087998337,33cyclictest21177-21modprobe08:09:190
317812839,50sleep131783-21sshd07:02:251
32087998256,17cyclictest14316-21latency_hist11:59:210
32087998253,22cyclictest17164-21sh09:19:550
32087998253,22cyclictest17164-21sh09:19:550
32094998153,10cyclictest17202-21ssh10:14:491
32094998131,28cyclictest13645-21ntp_kernel_err09:14:481
32087998154,19cyclictest1753-21latency_hist10:44:210
32094998065,9cyclictest58-21kcompactd009:47:141
32094998063,11cyclictest29570-21unixbench_multi10:34:581
32094998063,11cyclictest29570-21unixbench_multi10:34:581
32094998051,9cyclictest181rcu_preempt08:44:361
32087998063,10cyclictest16773-21kworker/u17:1+xprtiod09:24:460
32087998060,11cyclictest352-21runrttasks10:54:310
32087998056,14cyclictest17763-21diskmemload11:09:590
32094997954,17cyclictest23260-21fschecks_count11:19:411
32094997954,17cyclictest23260-21fschecks_count11:19:411
32094997949,13cyclictest10110-21rm10:57:541
32087997958,14cyclictest17452-21ssh10:14:560
32102997861,11cyclictest5398-21acpi08:59:302
32102997859,13cyclictest1307-21sh11:36:182
3210299784,65cyclictest20077-21apt-get11:14:342
32094997856,8cyclictest31-21ksoftirqd/108:36:291
32094997856,8cyclictest31-21ksoftirqd/108:36:291
3209499782,45cyclictest2350-21ssh09:49:461
32102997749,18cyclictest20409-21sh12:09:222
32094997756,8cyclictest181rcu_preempt09:59:361
32094997752,17cyclictest32078-21cyclictest11:04:411
3209499773,24cyclictest10940-21apt-get10:59:311
3208799779,39cyclictest1-21systemd10:39:200
3208799779,39cyclictest1-21systemd10:39:200
32087997758,11cyclictest0-21swapper/008:35:040
32087997758,11cyclictest0-21swapper/008:35:040
32087997757,14cyclictest18450-21kworker/u17:1+xprtiod10:20:430
3208799775,38cyclictest8848-21fschecks_count11:49:400
3208799775,38cyclictest8848-21fschecks_count11:49:400
3208799774,37cyclictest2201-21df_inode10:44:390
32087997720,27cyclictest0-21swapper/012:29:300
173582770,5sleep2371rcuc/209:20:262
173582770,5sleep2371rcuc/209:20:262
32102997658,12cyclictest2952-21systemd-journal12:12:302
32102997658,12cyclictest2952-21systemd-journal12:12:302
32102997654,15cyclictest13099-21latency_hist07:44:232
3210299764,34cyclictest1030-21/usr/sbin/munin08:44:472
3210299762,59cyclictest24657-21kworker/u19:1+ext4-rsv-conversion07:49:472
32094997645,14cyclictest181rcu_preempt07:19:361
3209499762,29cyclictest22725-21ssh10:24:371
3208799767,59cyclictest28193-21munin-run10:34:200
3208799764,41cyclictest30005-21ntp_states12:24:500
3208799764,41cyclictest30005-21ntp_states12:24:500
32087997633,36cyclictest0-21swapper/009:34:520
32087997633,36cyclictest0-21swapper/009:34:520
32087997619,35cyclictest0-21swapper/009:39:400
32102997555,14cyclictest28639-21latency_hist11:29:232
32102997553,16cyclictest1-21systemd10:15:252
32094997551,8cyclictest181rcu_preempt11:44:221
32094997546,9cyclictest181rcu_preempt09:19:331
32094997546,9cyclictest181rcu_preempt09:19:331
3209499753,48cyclictest4431-21unixbench_singl07:14:571
3209499753,40cyclictest2581-21latency08:49:461
3209499753,24cyclictest24917-21grep08:19:391
32087997554,14cyclictest16773-21kworker/u17:1+xprtiod09:44:520
32087997550,15cyclictest16858-21fschecks_time07:54:420
32102997457,12cyclictest32078-21cyclictest07:24:482
32102997454,14cyclictest10392-21fschecks_time09:09:442
32102997433,31cyclictest20293-21wc10:19:522
32094997440,7cyclictest181rcu_preempt07:44:221
3209499742,52cyclictest21965-21sshd10:23:301
3209499742,48cyclictest23342-21fschecks_time08:14:421
3209499742,23cyclictest10479-21ssh11:52:071
3209499742,23cyclictest10479-21ssh11:52:071
3208799744,57cyclictest10706-21acpi10:04:300
3208799743,40cyclictest2029-21sh09:49:390
32102997356,11cyclictest12159-21ntp_kernel_pll_11:54:452
32102997355,10cyclictest0-21swapper/210:29:192
3210299734,34cyclictest27103-21timerandwakeup08:24:562
32102997338,28cyclictest0-21swapper/209:54:532
32094997349,9cyclictest181rcu_preempt09:34:501
32094997349,9cyclictest181rcu_preempt09:34:501
32094997344,13cyclictest21998-21kworker/u17:3+xprtiod12:29:551
32094997342,9cyclictest181rcu_preempt10:09:341
32094997342,9cyclictest181rcu_preempt10:09:341
3209499733,34cyclictest663-21grep07:04:551
3208799735,53cyclictest21738-21apt-get09:29:350
3208799734,41cyclictest6552-21latency_hist07:24:220
32102997255,11cyclictest16773-21kworker/u17:1-nfsiod10:04:362
3210299723,59cyclictest29787-21apt-get08:34:352
3210299723,59cyclictest29787-21apt-get08:34:352
3210299723,47cyclictest9562-21kworker/u17:2+xprtiod10:02:082
3209499723,23cyclictest27287-21sshd11:25:291
3209499722,43cyclictest29720-21ssh12:24:421
3209499722,43cyclictest29720-21ssh12:24:421
3209499722,29cyclictest27916-21apt-get08:29:301
32087997245,17cyclictest23336-21latency_hist12:14:220
32087997245,17cyclictest23336-21latency_hist12:14:220
32087997245,17cyclictest12910-21systemd-run07:44:210
32087997241,20cyclictest10097-21ssh09:09:360
3208799722,37cyclictest27102-21/usr/sbin/munin12:19:520
3208799722,36cyclictest7211-21ntpq07:24:500
3110927224,32sleep30-21swapper/306:59:543
32102997138,24cyclictest32308-21tail08:39:552
32102997126,28cyclictest39-21ksoftirqd/208:00:222
32102997113,14cyclictest39-21ksoftirqd/207:04:432
32094997156,10cyclictest8303-21ssh11:49:191
3209499712,54cyclictest2192-21kworker/u18:0+ext4-rsv-conversion11:14:401
3209499712,24cyclictest22336-21ssh09:29:401
3208799715,29cyclictest0-21swapper/010:00:080
32087997147,15cyclictest9874-21ssh09:09:220
32087997145,18cyclictest23169-21sh11:19:370
32087997145,18cyclictest23169-21sh11:19:370
3208799714,47cyclictest14596-21cpu11:59:350
3208799714,47cyclictest14596-21cpu11:59:350
32087997117,24cyclictest0-21swapper/008:40:370
32102997052,13cyclictest26112-21idleruntime-cro12:19:182
32102997052,12cyclictest4177-21latency08:54:452
32102997048,15cyclictest30348-21timerwakeupswit12:24:562
32102997048,15cyclictest30348-21timerwakeupswit12:24:562
3210299702,53cyclictest7702-21/usr/sbin/munin09:04:502
3209499704,35cyclictest26326-21latency_hist12:19:221
3209499703,45cyclictest17589-21cpu12:04:341
32094997028,13cyclictest181rcu_preempt08:59:521
3209499702,55cyclictest10097-21ssh09:09:361
32094997019,18cyclictest0-21swapper/107:39:221
3210299698,26cyclictest39-21ksoftirqd/211:13:202
32102996952,11cyclictest23579-21ntp_kernel_err08:14:482
32102996951,11cyclictest0-21swapper/210:59:372
3210299694,28cyclictest39-21ksoftirqd/209:35:082
3210299694,28cyclictest39-21ksoftirqd/209:35:082
3210299693,37cyclictest13757-21cpuspeed10:09:352
3210299693,37cyclictest13757-21cpuspeed10:09:352
3210299692,34cyclictest39-21ksoftirqd/208:33:242
3210299692,29cyclictest985-21awk12:29:572
32094996945,16cyclictest277-21dbus-daemon10:54:191
32094996932,8cyclictest181rcu_preempt10:44:351
32087996949,14cyclictest6649-21unixbench_multi11:44:550
3208799694,44cyclictest352-21runrttasks07:59:570
3208799693,47cyclictest22944-21latency_hist08:14:220
3208799692,29cyclictest0-21swapper/007:09:480
32108996855,9cyclictest451rcuc/310:19:253
32102996851,11cyclictest352-21runrttasks12:19:422
32102996849,15cyclictest31071-21kworker/u18:0+events_unbound09:49:192
32102996847,14cyclictest0-21swapper/207:34:232
3210299684,37cyclictest25841-21fschecks_time10:29:412
32102996841,16cyclictest15199-21journal-offline11:05:322
3210299683,43cyclictest21731-21fschecks_time08:09:422
3210299683,26cyclictest39-21ksoftirqd/210:50:402
32094996846,9cyclictest58-21kcompactd008:13:441
32094996845,16cyclictest32078-21cyclictest10:39:431
32094996833,12cyclictest181rcu_preempt08:54:421
32094996831,14cyclictest181rcu_preempt11:58:401
3209499682,33cyclictest26525-21fschecks_count08:24:431
3209499682,27cyclictest7552-21ssh09:04:451
32094996821,21cyclictest19764-21latency_hist08:04:231
3208799685,28cyclictest6896-21grep09:04:230
32087996849,13cyclictest4824-21idleruntime-cro07:19:190
32087996839,15cyclictest0-21swapper/010:49:530
32087996836,23cyclictest0-21swapper/007:50:290
3208799683,44cyclictest26034-21modprobe08:24:190
3208799682,30cyclictest15747-21ssh10:12:430
3208799682,30cyclictest15747-21ssh10:12:430
3208799682,29cyclictest9920-21/usr/sbin/munin07:34:530
32102996751,11cyclictest32078-21cyclictest07:19:522
32102996749,14cyclictest32078-21cyclictest10:34:532
32102996749,14cyclictest32078-21cyclictest10:34:532
3210299674,24cyclictest39-21ksoftirqd/209:17:382
3210299673,31cyclictest6766-21diskmemload09:44:222
32094996746,13cyclictest1856-21cpu07:09:381
3209499673,38cyclictest15019-21modprobe07:49:471
32094996717,29cyclictest0-21swapper/107:54:351
32087996746,14cyclictest14765-21apt-key07:49:240
3208799673,39cyclictest14156-21fschecks_count11:04:410
3208799673,37cyclictest26985-21grep08:24:540
3208799672,52cyclictest9252-21unixbench_singl07:29:590
3208799672,28cyclictest0-21swapper/011:03:220
3210299663,46cyclictest2668-21proc_pri10:44:522
3210299662,37cyclictest25806-21latency_hist11:24:222
3210299662,37cyclictest25806-21latency_hist11:24:222
32102996612,30cyclictest39-21ksoftirqd/210:40:212
32094996627,13cyclictest181rcu_preempt12:09:571
32094996627,13cyclictest181rcu_preempt12:09:571
3209499662,42cyclictest27446-21tail12:19:561
3209499662,41cyclictest21175-21cron08:09:191
3208799663,41cyclictest28823-21tail08:29:560
32087996613,19cyclictest0-21swapper/011:37:250
3210299658,27cyclictest39-21ksoftirqd/211:40:342
32102996541,14cyclictest0-21swapper/207:14:222
3210299653,45cyclictest3690-21fschecks_count07:14:422
3210299653,33cyclictest14671-21ssh11:59:362
3210299653,33cyclictest14671-21ssh11:59:362
3210299652,24cyclictest39-21ksoftirqd/208:04:462
3210299651,33cyclictest6766-21diskmemload11:49:552
3210299651,33cyclictest6766-21diskmemload11:49:552
32102996512,12cyclictest2893-21seq09:49:592
32094996536,8cyclictest7025-21ssh09:58:131
32094996528,18cyclictest11158-21grep10:04:451
32102996439,14cyclictest0-21swapper/207:44:422
3210299643,41cyclictest10510-21proc_pri07:34:512
3210299643,35cyclictest25422-21switchtime08:19:512
3209499644,33cyclictest28787-21latency09:39:471
3209499642,36cyclictest374-21sshd08:40:161
3208799643,47cyclictest32274-21cpu07:04:360
3208799643,47cyclictest23083-21/usr/sbin/munin08:14:440
3208799643,39cyclictest5342-21latency_hist11:44:220
32108996346,13cyclictest451rcuc/307:04:353
3210899634,53cyclictest0-21swapper/307:49:473
3209499633,25cyclictest26572-21awk10:29:561
3208799633,33cyclictest3781-21/usr/sbin/munin08:54:490
3208799631,30cyclictest7182-1kworker/0:2H+mmc_complete08:45:160
32108996249,9cyclictest451rcuc/311:29:323
3210299623,38cyclictest17207-21grep07:54:532
3209499625,10cyclictest7145-21memory07:24:481
3208799623,20cyclictest2511-21irqstats08:49:450
32108996047,9cyclictest451rcuc/308:54:383
32108996039,13cyclictest451rcuc/309:14:113
3210899597,31cyclictest0-21swapper/311:14:403
32108995744,8cyclictest451rcuc/308:08:093
32108995712,27cyclictest0-21swapper/309:29:483
32108995636,15cyclictest451rcuc/307:26:093
32108995636,12cyclictest451rcuc/311:10:063
3210899552,42cyclictest25123-21taskset11:22:463
3210899552,42cyclictest25123-21taskset11:22:463
298452550,5sleep30-21swapper/309:41:143
3210899544,36cyclictest8588-21chrt07:29:453
3210899543,38cyclictest23416-21chrt10:24:553
32108995239,9cyclictest451rcuc/312:26:573
32108995239,9cyclictest451rcuc/312:26:573
32108995237,10cyclictest451rcuc/310:39:353
32108995237,10cyclictest451rcuc/310:39:353
3210899523,35cyclictest30815-21chrt08:35:303
3210899523,35cyclictest30815-21chrt08:35:303
3210899523,35cyclictest17788-21chrt12:04:403
32108995232,12cyclictest451rcuc/312:30:503
32108995230,15cyclictest451rcuc/308:40:133
3210899522,38cyclictest15746-21chrt12:00:103
3210899522,38cyclictest15746-21chrt12:00:103
3210899514,35cyclictest2117-21chrt08:49:233
3210899513,38cyclictest9511-21chrt10:01:493
3210899513,38cyclictest4786-21chrt10:49:223
3210899513,37cyclictest10729-21taskset11:52:493
3210899513,37cyclictest10729-21taskset11:52:493
3210899513,35cyclictest22678-21chrt12:13:033
3210899513,35cyclictest22678-21chrt12:13:033
3210899513,34cyclictest6650-21chrt11:44:553
32108995038,8cyclictest451rcuc/307:09:293
3210899503,37cyclictest17869-21chrt10:15:153
3210899503,37cyclictest10878-21chrt10:04:373
3210899503,36cyclictest22021-21chrt09:29:223
3210899503,36cyclictest14753-21chrt11:04:543
3210899503,35cyclictest27693-21chrt10:32:573
3210899503,35cyclictest27192-21chrt08:24:573
3210899503,34cyclictest11303-21chrt07:39:213
3210899503,34cyclictest1007-21chrt11:35:273
3210899502,37cyclictest27980-21chrt12:21:133
3210899502,36cyclictest29314-21chrt08:34:203
3210899502,35cyclictest666-21chrt09:47:063
3210899502,34cyclictest4218-21chrt07:14:553
3210899502,32cyclictest5538-21taskset11:44:223
162702504,9sleep30-21swapper/309:19:393
162702504,9sleep30-21swapper/309:19:393
32108994933,9cyclictest451rcuc/308:11:453
3210899493,35cyclictest14532-21chrt07:49:213
3210899493,35cyclictest14379-21chrt10:09:523
3210899493,35cyclictest14379-21chrt10:09:523
3210899493,34cyclictest8713-21chrt10:54:533
3210899493,32cyclictest26404-21chrt11:24:443
3210899484,31cyclictest13024-21chrt07:44:233
32108994834,9cyclictest451rcuc/311:55:253
3210899483,36cyclictest26437-21chrt09:35:023
3210899483,36cyclictest26437-21chrt09:35:023
3210899483,32cyclictest12098-21chrt11:00:083
3210899482,32cyclictest32678-21chrt10:40:113
3210899473,34cyclictest5705-21chrt08:59:443
3210899472,33cyclictest7537-21chrt09:59:223
3210899463,35cyclictest3682-21taskset09:52:133
3210899463,30cyclictest7885-21chrt09:04:543
3210899463,29cyclictest13634-21chrt09:14:473
3210899443,31cyclictest17458-21chrt07:54:563
3210899442,33cyclictest5984-21chrt07:19:573
3210899442,29cyclictest23866-21chrt12:14:453
3210899433,27cyclictest24157-21chrt08:15:503
32108994327,9cyclictest451rcuc/308:50:003
3210899432,29cyclictest6091-21chrt10:49:593
32108994229,9cyclictest451rcuc/308:22:593
3210899413,29cyclictest18673-21chrt07:59:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional