You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 19:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Apr 13, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191009911158,45cyclictest0-21swapper/210:28:182
191009910038,8cyclictest0-21swapper/210:28:532
191009910038,8cyclictest0-21swapper/210:28:532
1863921006,15sleep10-21swapper/107:01:041
1863921006,15sleep10-21swapper/107:01:041
212232990,8sleep30-21swapper/309:48:423
1910099992,75cyclictest3957-21cstates11:08:542
19100999832,13cyclictest0-21swapper/211:01:122
19100999772,18cyclictest1-21systemd09:33:162
19100999728,12cyclictest0-21swapper/210:53:332
19100999725,11cyclictest0-21swapper/207:13:542
19100999725,11cyclictest0-21swapper/207:13:542
1910099972,32cyclictest21067-21latency_hist08:48:182
19100999646,44cyclictest3357-21latency_hist11:08:192
19100999636,8cyclictest0-21swapper/212:31:092
19100999634,8cyclictest0-21swapper/212:03:412
19100999628,10cyclictest0-21swapper/209:48:522
1910099954,33cyclictest21811-21ls07:08:442
19100999435,10cyclictest0-21swapper/210:48:522
1910099943,31cyclictest8131-21latency_hist08:08:172
1910099943,31cyclictest8131-21latency_hist08:08:172
19100999433,12cyclictest0-21swapper/211:25:292
1910099943,30cyclictest32068-21grep07:38:532
19100999428,10cyclictest0-21swapper/211:33:562
1910099933,32cyclictest20707-21latency_hist10:43:192
19100999333,12cyclictest0-21swapper/210:08:402
19100999324,29cyclictest10073-21latency_hist12:13:192
1910099932,38cyclictest8811-21ntp_kernel_err08:08:442
1910099932,34cyclictest6622-21grep10:18:412
19090999373,13cyclictest26598-21runrttasks09:58:191
19090999373,13cyclictest26598-21runrttasks09:58:191
185302936,67sleep00-21swapper/006:59:360
185302936,67sleep00-21swapper/006:59:360
1910099924,33cyclictest352-21runrttasks09:39:242
1910099923,41cyclictest11931-21uname08:18:402
1910099923,40cyclictest2757-21latency_hist09:18:192
1910099923,36cyclictest18240-21latency_hist11:33:182
1910099923,35cyclictest28896-21munin-run09:08:182
1910099923,34cyclictest29400-21ssh09:08:312
1910099923,32cyclictest26566-21idleruntime07:23:382
1910099923,32cyclictest26566-21idleruntime07:23:382
19100999231,55cyclictest0-21swapper/208:23:452
19100999225,11cyclictest0-21swapper/207:28:372
1910099922,36cyclictest4928-21latency_hist07:58:172
1910099922,36cyclictest4928-21latency_hist07:58:172
1910099922,32cyclictest5386-21fschecks_time07:58:382
1910099922,32cyclictest5386-21fschecks_time07:58:382
1910099913,37cyclictest18989-21ssh10:38:532
1910099913,37cyclictest18989-21ssh10:38:532
1910099913,35cyclictest10065-21irqstats11:18:402
1910099913,35cyclictest10065-21irqstats11:18:402
1910099913,32cyclictest15733-21unixbench_singl08:28:522
1910099913,31cyclictest25242-21ps07:18:482
1910099913,31cyclictest25242-21ps07:18:482
1910099913,29cyclictest14197-21unixbench_singl12:18:532
1910099913,28cyclictest2061-21fschecks_count07:48:382
19100999131,9cyclictest0-21swapper/210:33:542
1910099912,35cyclictest30604-21latency_hist11:53:192
19090999161,19cyclictest22537-21unixbench_multi11:38:521
1910099903,37cyclictest21809-21fschecks_count11:38:372
1910099903,37cyclictest16666-21fschecks_time08:33:372
1910099903,36cyclictest29361-21latency_hist10:03:172
1910099903,33cyclictest11818-21/usr/sbin/munin09:33:422
1910099903,32cyclictest27525-21latency_hist11:48:182
1910099903,32cyclictest25407-21unixbench_multi08:58:522
19100999027,11cyclictest0-21swapper/211:18:172
1910099902,37cyclictest714-21needreboot07:43:432
1910099902,36cyclictest1979-21cut11:58:472
1910099902,30cyclictest23494-21grep08:53:472
1910099893,31cyclictest21536-21ls08:48:402
1910099893,31cyclictest21536-21ls08:48:402
1910099893,29cyclictest18877-21vmstat09:43:542
19100998931,11cyclictest0-21swapper/207:33:372
1910099892,37cyclictest24001-21fschecks_time09:53:372
1910099892,37cyclictest24001-21fschecks_time09:53:372
1910099892,33cyclictest29933-21grep10:03:382
19100998922,14cyclictest0-21swapper/210:13:402
19100998922,14cyclictest0-21swapper/210:13:402
1910099883,33cyclictest9952-21/usr/sbin/munin08:13:262
1910099883,31cyclictest6007-21cstates09:23:342
1910099882,35cyclictest18695-21mailstats08:38:472
1910099882,32cyclictest12907-21latency_hist12:18:182
19078998867,15cyclictest544-21cron07:33:160
1910099873,29cyclictest957-21latency_hist11:58:172
19090998761,15cyclictest8275-21kworker/u17:1-rpciod10:33:521
1909099873,68cyclictest19378-21apt-get07:03:451
1910099863,34cyclictest19842-21latency07:04:172
1910099862,34cyclictest2893-21/usr/sbin/munin09:18:332
19090998661,17cyclictest32335-21latency_hist10:08:181
19078998661,17cyclictest32052-21sendmail09:13:150
19078998646,29cyclictest9067-21kworker/u17:1+nfsiod12:12:360
19078998630,10cyclictest181rcu_preempt10:08:390
1910099853,50cyclictest16323-21apt-get12:23:312
1909099853,72cyclictest24775-21ssh11:43:331
19090998514,61cyclictest16634-21cat12:24:081
19078998451,17cyclictest181rcu_preempt10:28:530
19078998451,17cyclictest181rcu_preempt10:28:530
1911799833,52cyclictest24767-21chrt11:43:313
19090998365,11cyclictest26000-21sh07:23:151
19090998365,11cyclictest26000-21sh07:23:151
19078998341,11cyclictest181rcu_preempt08:48:400
19078998341,11cyclictest181rcu_preempt08:48:400
19078998330,10cyclictest181rcu_preempt07:08:440
1910099822,37cyclictest19253-21sed07:03:222
1910099822,37cyclictest19253-21sed07:03:222
19090998257,16cyclictest10347-21latency08:13:411
1909099822,45cyclictest31-21ksoftirqd/108:24:111
1907899813,58cyclictest15773-21apt-get12:23:290
19090998054,17cyclictest25847-21diskmemload10:56:261
19078998049,26cyclictest3188-21systemd-run11:08:160
19090997959,14cyclictest12463-21latency_hist11:23:181
19090997959,14cyclictest12463-21latency_hist11:23:181
19078997853,10cyclictest181rcu_preempt11:32:030
1907899783,41cyclictest28866-21systemd-run09:08:170
19090997760,11cyclictest25847-21diskmemload09:27:181
19090997755,14cyclictest14480-21kworker/u17:1-rpciod10:12:201
19090997753,18cyclictest9567-21kernelversion10:23:431
19090997733,20cyclictest11976-21ssh12:15:561
19078997758,13cyclictest17766-21idleruntime-cro08:38:150
19078997745,9cyclictest181rcu_preempt11:56:410
19078997742,11cyclictest181rcu_preempt11:11:210
1909099765,51cyclictest31-21ksoftirqd/109:49:351
19078997655,7cyclictest181rcu_preempt10:23:530
19078997649,19cyclictest437-21fschecks_count07:43:370
19078997639,16cyclictest181rcu_preempt08:07:200
19078997639,16cyclictest181rcu_preempt08:07:200
19078997639,16cyclictest181rcu_preempt07:53:490
19078997639,16cyclictest181rcu_preempt07:53:490
19078997635,10cyclictest181rcu_preempt10:53:190
19078997634,17cyclictest181rcu_preempt10:43:400
19090997556,12cyclictest352-21runrttasks12:28:441
19090997552,15cyclictest11816-21sh09:33:271
19090997548,9cyclictest181rcu_preempt11:33:271
19090997515,13cyclictest0-21swapper/111:58:291
19078997561,9cyclictest58-21kcompactd010:56:180
19078997558,11cyclictest5100-21/usr/sbin/munin07:58:470
19078997558,11cyclictest5100-21/usr/sbin/munin07:58:470
19078997556,14cyclictest23409-21kworker/u19:0+events_unbound09:58:150
19078997556,14cyclictest23409-21kworker/u19:0+events_unbound09:58:150
19078997552,13cyclictest13143-21ssh11:23:430
19078997551,9cyclictest181rcu_preempt12:07:160
19078997534,17cyclictest181rcu_preempt11:38:330
19078997515,28cyclictest27433-21systemd11:48:170
19090997458,10cyclictest3158-21apt-get11:08:211
19090997458,10cyclictest2952-21systemd-journal11:03:161
19090997456,12cyclictest29124-21latency_hist09:08:191
1909099743,53cyclictest28830-21apt-get09:08:311
19078997464,6cyclictest0-21swapper/009:43:260
19078997451,7cyclictest181rcu_preempt07:23:390
19078997451,7cyclictest181rcu_preempt07:23:390
19078997445,8cyclictest181rcu_preempt12:28:260
19078997444,21cyclictest3799-21ssh12:03:050
19078997444,21cyclictest11723-21latency_hist09:33:190
1907899744,35cyclictest13103-21/usr/sbin/munin12:18:320
19078997442,14cyclictest19676-21snmpd08:26:100
19078997436,20cyclictest181rcu_preempt12:13:440
19090997357,10cyclictest374-21sshd08:09:201
19090997352,16cyclictest19059-21cyclictest07:48:151
19090997352,14cyclictest16152-21latency_hist12:23:191
19090997349,15cyclictest2952-21systemd-journal12:06:201
1909099733,26cyclictest31-21ksoftirqd/111:53:531
19078997356,11cyclictest298-21in:imuxsock07:53:170
19078997354,13cyclictest7349-21sh09:24:570
19078997348,17cyclictest9660-21cron08:13:150
19078997347,11cyclictest181rcu_preempt11:01:060
19078997336,12cyclictest181rcu_preempt08:46:430
19078997333,12cyclictest181rcu_preempt11:16:010
1909099726,25cyclictest31-21ksoftirqd/108:18:261
19090997243,20cyclictest10047-21sh12:13:191
19090997224,9cyclictest291rcuc/107:58:311
19090997224,9cyclictest291rcuc/107:58:311
19078997254,13cyclictest15027-21fschecks_count09:38:380
19078997248,10cyclictest181rcu_preempt09:48:230
19078997246,9cyclictest181rcu_preempt10:38:490
19078997246,9cyclictest181rcu_preempt10:38:490
19078997229,13cyclictest181rcu_preempt11:35:470
1907899722,55cyclictest3712-21switchtime09:18:520
19090997153,11cyclictest297-21systemd-logind08:08:161
19090997153,11cyclictest297-21systemd-logind08:08:161
19090997145,7cyclictest181rcu_preempt08:40:171
1909099714,56cyclictest15597-21ssh11:28:281
19090997120,41cyclictest24318-21cut10:48:401
19078997157,9cyclictest19059-21cyclictest07:18:430
19078997157,9cyclictest19059-21cyclictest07:18:430
19078997142,14cyclictest181rcu_preempt09:34:230
19078997135,13cyclictest181rcu_preempt07:38:400
1907899712,29cyclictest459-21needreboot09:13:440
1909099702,27cyclictest31-21ksoftirqd/109:39:211
19090997022,23cyclictest31-21ksoftirqd/108:33:321
19078997046,9cyclictest181rcu_preempt07:13:290
19078997046,9cyclictest181rcu_preempt07:13:290
19078997035,11cyclictest181rcu_preempt10:00:200
184112705,46sleep30-21swapper/306:59:023
184112705,46sleep30-21swapper/306:59:023
1911799697,41cyclictest0-21swapper/307:04:033
1909099695,31cyclictest31-21ksoftirqd/107:34:411
19090996950,13cyclictest9702-21sh09:28:511
19078996938,14cyclictest181rcu_preempt10:33:380
1907899693,54cyclictest352-21runrttasks10:14:350
1907899693,54cyclictest352-21runrttasks10:14:350
1907899693,33cyclictest30391-21sendmail-msp11:53:150
1907899692,43cyclictest15-21ksoftirqd/008:58:520
19117996812,43cyclictest0-21swapper/312:24:103
1909099685,41cyclictest24951-21netstat08:58:431
1909099684,52cyclictest3230-21ssh10:13:301
1909099684,52cyclictest3230-21ssh10:13:301
19090996827,19cyclictest31-21ksoftirqd/109:14:321
1909099682,42cyclictest31-21ksoftirqd/110:38:361
1909099682,42cyclictest31-21ksoftirqd/110:38:361
1907899683,49cyclictest12302-21cron11:23:160
1907899683,49cyclictest12302-21cron11:23:160
1909099675,41cyclictest1002-21latency11:03:421
19090996749,11cyclictest277-21dbus-daemon11:28:161
19090996743,8cyclictest181rcu_preempt10:46:311
19090996734,23cyclictest7072-21sort10:18:511
19090996652,10cyclictest19059-21cyclictest10:30:421
19090996652,10cyclictest19059-21cyclictest10:30:421
19090996649,11cyclictest7312-21ssh11:13:441
19090996645,14cyclictest0-21swapper/108:33:191
19090996644,8cyclictest181rcu_preempt07:27:021
19090996644,8cyclictest181rcu_preempt07:27:021
1909099664,35cyclictest22890-21cpu08:53:311
19090996642,7cyclictest181rcu_preempt10:00:221
19090996637,20cyclictest5543-21modprobe09:23:151
1909099663,36cyclictest31-21ksoftirqd/107:39:351
19090996633,14cyclictest181rcu_preempt09:45:091
19090996632,8cyclictest181rcu_preempt11:50:041
1909099662,27cyclictest31-21ksoftirqd/108:43:441
19078996648,13cyclictest11927-21kernelversion08:18:400
19078996628,8cyclictest0-21swapper/010:18:440
1907899662,44cyclictest19534-21latency_hist08:43:190
1907899662,32cyclictest10168-21fschecks_count08:13:370
1909099653,35cyclictest23042-21/usr/sbin/munin07:13:381
1909099653,35cyclictest23042-21/usr/sbin/munin07:13:381
19090996514,12cyclictest301ktimers/107:53:331
19090996514,12cyclictest301ktimers/107:53:331
19078996551,9cyclictest19059-21cyclictest07:04:180
19078996542,10cyclictest181rcu_preempt08:53:380
1907899652,31cyclictest30743-21grep10:03:540
19090996440,16cyclictest29456-21latency_hist07:33:191
19090996440,13cyclictest0-21swapper/107:48:451
1909099643,34cyclictest22612-21readlink08:53:161
1909099643,34cyclictest22612-21readlink08:53:161
1909099633,39cyclictest21692-21fschecks_time07:08:411
19078996322,11cyclictest181rcu_preempt08:28:510
19117996047,8cyclictest451rcuc/311:26:423
1907899603,38cyclictest29489-21/usr/sbin/munin07:33:350
19117995946,9cyclictest451rcuc/311:40:153
19117995845,9cyclictest451rcuc/312:06:133
19117995844,9cyclictest451rcuc/310:18:213
19117995843,10cyclictest451rcuc/309:26:453
1911799573,40cyclictest25626-21taskset07:18:523
1911799573,40cyclictest25626-21taskset07:18:523
112252570,7sleep30-21swapper/311:20:123
112252570,7sleep30-21swapper/311:20:123
19117995643,9cyclictest451rcuc/307:34:163
19117995441,9cyclictest451rcuc/309:23:083
19117995441,8cyclictest451rcuc/308:21:473
19117995440,9cyclictest451rcuc/307:29:553
1911799543,37cyclictest7331-21chrt08:03:483
1911799543,37cyclictest7331-21chrt08:03:483
19117995432,14cyclictest451rcuc/311:35:013
1911799542,36cyclictest3205-21taskset10:13:253
1911799542,36cyclictest3205-21taskset10:13:253
1911799534,39cyclictest28708-21chrt10:55:403
19117995340,9cyclictest451rcuc/309:55:073
19117995340,9cyclictest451rcuc/309:55:073
19117995339,9cyclictest451rcuc/308:35:113
19117995338,8cyclictest451rcuc/309:12:013
1911799533,38cyclictest1024-21chrt10:08:523
1911799533,36cyclictest32575-21taskset07:43:183
1911799533,36cyclictest10923-21chrt08:13:523
1911799524,36cyclictest27110-21taskset07:23:503
1911799524,36cyclictest27110-21taskset07:23:503
19117995239,9cyclictest451rcuc/308:25:363
1911799523,37cyclictest17155-21chrt11:30:133
19117995213,26cyclictest0-21swapper/311:08:353
19117995139,8cyclictest451rcuc/310:03:463
19117995138,9cyclictest451rcuc/311:53:123
19117995138,9cyclictest451rcuc/311:03:353
1911799512,35cyclictest5254-21chrt07:58:333
1911799512,35cyclictest5254-21chrt07:58:333
1911799512,29cyclictest14488-21taskset12:19:373
19117995038,8cyclictest451rcuc/308:28:393
19117995037,9cyclictest451rcuc/309:58:313
1911799503,35cyclictest14741-21chrt10:33:153
1911799503,35cyclictest14741-21chrt10:33:153
1911799503,34cyclictest8447-21chrt08:08:333
1911799503,34cyclictest32574-21chrt09:13:353
1911799503,34cyclictest18697-21chrt08:38:473
1911799503,33cyclictest21266-21chrt10:43:363
1911799503,31cyclictest24454-21chrt08:58:193
1911799502,37cyclictest26666-21taskset09:03:423
1911799502,37cyclictest25366-21chrt10:49:403
19117994936,9cyclictest451rcuc/312:32:353
19117994936,8cyclictest451rcuc/307:46:523
1911799493,39cyclictest917-21kworker/3:1+events09:28:283
1911799493,37cyclictest13117-21taskset09:34:233
1911799493,36cyclictest20274-21chrt10:42:023
1911799493,36cyclictest20274-21chrt10:42:023
1911799493,36cyclictest17000-21chrt10:36:113
1911799493,33cyclictest20159-21chrt08:43:453
1911799492,36cyclictest3377-21chrt07:53:193
19117994835,9cyclictest451rcuc/309:01:323
1911799483,35cyclictest20338-21chrt09:48:053
1911799483,31cyclictest10284-21chrt12:13:343
1911799482,34cyclictest9595-21chrt12:12:403
1911799474,31cyclictest15453-21chrt09:38:493
19117994734,9cyclictest451rcuc/307:55:023
19117994734,9cyclictest451rcuc/307:55:023
19117994734,8cyclictest451rcuc/307:11:323
1911799473,35cyclictest30609-21chrt10:58:483
19117994633,9cyclictest451rcuc/311:58:393
1911799462,35cyclictest9848-21chrt10:23:513
1911799462,33cyclictest32644-21chrt11:56:153
1911799454,30cyclictest24603-21taskset07:18:203
1911799454,30cyclictest24603-21taskset07:18:203
1911799452,31cyclictest7484-21taskset11:13:483
1911799422,27cyclictest22229-21chrt08:48:553
1911799422,27cyclictest22229-21chrt08:48:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional