You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 16:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Apr 06, 2026 12:44:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3329910556,14cyclictest181rcu_preempt11:53:580
3559910477,19cyclictest18225-21sh12:04:052
326372976,18sleep20-21swapper/207:03:292
355999463,23cyclictest16359-21sh09:18:462
355999361,12cyclictest181rcu_preempt11:24:042
355999042,16cyclictest435-21ssh12:28:502
355999028,20cyclictest0-21swapper/208:35:142
340999066,17cyclictest10307-21awk09:08:441
340999066,17cyclictest10307-21awk09:08:441
340998863,19cyclictest7284-21latency_hist07:23:471
355998661,18cyclictest323-21cyclictest07:34:062
355998661,18cyclictest323-21cyclictest07:34:062
332998657,21cyclictest24269-21kworker/u19:1+events_unbound08:33:430
355998554,22cyclictest20765-21sh11:14:052
313472858,52sleep131366-21cpu06:59:031
340998443,30cyclictest3164-21processes07:09:181
332998445,31cyclictest994-21threads11:34:180
355998346,12cyclictest181rcu_preempt07:18:592
340998364,13cyclictest2952-21systemd-journal10:14:411
355998251,22cyclictest4313-21/usr/sbin/munin08:54:152
340998256,18cyclictest26380-21sh10:29:031
332998258,15cyclictest1874-21systemd-run09:48:440
355998155,14cyclictest0-21swapper/210:48:542
355998154,20cyclictest8850-21latency_hist07:28:462
355998146,8cyclictest181rcu_preempt11:34:162
340998158,15cyclictest20280-21kworker/u17:1-xprtiod10:44:111
340998151,21cyclictest5202-21apt-config09:53:471
340998144,28cyclictest28043-21grep12:19:221
340998144,28cyclictest28043-21grep12:19:221
340998111,57cyclictest23772-21apt-get11:18:561
332998162,13cyclictest23991-21/usr/sbin/munin12:13:540
332998146,12cyclictest181rcu_preempt08:03:070
332998146,12cyclictest181rcu_preempt08:03:070
332998134,36cyclictest20659-21cpuspeed08:04:000
355998060,12cyclictest10815-21ssh11:51:112
355998050,10cyclictest181rcu_preempt09:33:562
340998055,18cyclictest15301-21fschecks_count11:59:051
332998041,9cyclictest0-21swapper/007:44:180
355997956,12cyclictest0-21swapper/210:54:002
355997956,12cyclictest0-21swapper/210:54:002
355997952,18cyclictest12912-21kworker/u17:3-rpciod10:44:182
355997943,16cyclictest181rcu_preempt11:58:462
355997943,16cyclictest181rcu_preempt10:01:472
355997930,25cyclictest19527-21latency_hist09:23:472
355997930,25cyclictest19527-21latency_hist09:23:472
340997956,16cyclictest24650-21timerwakeupswit08:14:191
355997854,15cyclictest9292-21fschecks_time07:29:062
355997853,13cyclictest0-21swapper/207:18:472
355997850,12cyclictest58-21kcompactd009:42:172
340997858,14cyclictest3145-21latency_hist12:33:471
340997836,12cyclictest181rcu_preempt07:08:441
340997836,12cyclictest181rcu_preempt07:08:441
33299783,60cyclictest87-21kswapd009:54:050
33299783,51cyclictest15720-21apt-get07:49:130
33299783,51cyclictest15720-21apt-get07:49:130
3162727825,28sleep30-21swapper/306:59:143
297912780,15sleep3451rcuc/309:39:333
355997748,18cyclictest17848-21ssh10:14:122
34099779,31cyclictest0-21swapper/109:29:211
33299777,32cyclictest15-21ksoftirqd/008:44:050
332997744,8cyclictest0-21swapper/010:28:470
332997738,12cyclictest181rcu_preempt09:52:350
332997733,25cyclictest181rcu_preempt09:34:200
355997650,20cyclictest5188-21apt-key09:53:482
355997648,20cyclictest21335-21threads08:04:182
355997643,12cyclictest181rcu_preempt10:08:042
355997643,12cyclictest181rcu_preempt10:08:042
355997631,10cyclictest181rcu_preempt10:38:432
340997660,10cyclictest20227-21ntp_offset09:24:131
340997651,17cyclictest11895-21fschecks_time10:59:051
340997640,28cyclictest31831-21sh10:38:431
332997644,10cyclictest181rcu_preempt08:33:560
332997643,12cyclictest181rcu_preempt11:47:460
332997641,11cyclictest16317-21latency_hist09:18:470
332997620,24cyclictest181rcu_preempt11:03:460
35999753,57cyclictest5670-21chrt08:58:433
355997549,16cyclictest0-21swapper/211:23:202
355997543,8cyclictest39-21ksoftirqd/211:03:432
355997534,19cyclictest0-21swapper/207:49:202
355997534,19cyclictest0-21swapper/207:49:202
355997529,26cyclictest20853-21ntp_offset10:19:132
340997554,15cyclictest7301-21diskmemload10:07:381
340997554,15cyclictest7301-21diskmemload10:07:381
340997552,13cyclictest28646-21sh09:39:001
340997512,23cyclictest11302-21latency_hist10:58:461
340997512,23cyclictest11302-21latency_hist10:58:461
332997556,10cyclictest31822-21cron10:38:430
33299755,48cyclictest23772-21apt-get11:19:060
332997544,10cyclictest181rcu_preempt11:04:000
332997544,10cyclictest181rcu_preempt11:04:000
332997535,12cyclictest181rcu_preempt07:08:510
332997534,21cyclictest323-21cyclictest11:28:470
332997533,17cyclictest181rcu_preempt11:39:050
332997533,17cyclictest181rcu_preempt11:39:050
35999743,57cyclictest9165-21taskset10:54:103
35999743,57cyclictest9165-21taskset10:54:103
35999743,54cyclictest13228-21taskset11:54:303
35999742,37cyclictest28938-21chrt11:27:473
355997459,9cyclictest13280-21sh09:13:432
35599745,38cyclictest16989-21awk10:13:442
332997453,14cyclictest2039-21latency_hist07:08:470
332997453,14cyclictest2039-21latency_hist07:08:470
332997438,11cyclictest28000-21unixbench_multi12:19:210
332997438,11cyclictest28000-21unixbench_multi12:19:210
332997434,13cyclictest181rcu_preempt07:21:150
332997433,14cyclictest181rcu_preempt08:18:460
355997359,10cyclictest8448-21kworker/u18:1-rpciod11:30:332
355997351,8cyclictest181rcu_preempt08:29:102
355997349,16cyclictest5990-21latency_hist11:43:472
355997349,16cyclictest5990-21latency_hist11:43:472
340997356,12cyclictest323-21cyclictest10:22:151
340997351,12cyclictest22356-21diskmemload12:10:031
340997351,12cyclictest22356-21diskmemload12:10:031
332997355,12cyclictest277-21dbus-daemon09:13:440
332997339,7cyclictest181rcu_preempt10:04:050
332997339,7cyclictest181rcu_preempt10:04:050
33299733,18cyclictest10358-1kworker/0:0H+mmc_complete07:03:470
35999723,54cyclictest9416-21taskset09:06:173
35999723,54cyclictest9416-21taskset09:06:173
35999723,54cyclictest18427-21taskset09:21:173
35999723,54cyclictest18427-21taskset09:21:173
35999723,36cyclictest14632-21chrt11:03:593
35999723,36cyclictest14632-21chrt11:03:593
35999723,35cyclictest536-21chrt11:34:073
35999723,34cyclictest5956-21chrt08:58:593
355997258,10cyclictest323-21cyclictest09:25:302
355997251,11cyclictest0-21swapper/209:44:172
355997250,12cyclictest0-21swapper/207:44:062
355997249,12cyclictest0-21swapper/211:08:472
355997249,12cyclictest0-21swapper/211:08:472
35599723,45cyclictest5425-21apt-get09:54:002
355997223,23cyclictest0-21swapper/212:10:282
355997223,23cyclictest0-21swapper/212:10:282
332997230,7cyclictest181rcu_preempt12:31:370
332997213,7cyclictest0-21swapper/011:48:590
35999713,55cyclictest15764-21chrt07:49:173
35999713,55cyclictest15764-21chrt07:49:173
35999712,38cyclictest18420-21chrt11:09:193
35999712,38cyclictest18420-21chrt11:09:193
355997150,14cyclictest323-21cyclictest10:28:462
355997148,11cyclictest0-21swapper/208:39:102
355997148,11cyclictest0-21swapper/207:58:462
355997148,11cyclictest0-21swapper/207:58:462
355997147,14cyclictest0-21swapper/210:43:462
34099712,53cyclictest11007-21apt10:03:431
33299713,55cyclictest18469-21timerandwakeup11:09:190
33299713,55cyclictest18469-21timerandwakeup11:09:190
332997134,8cyclictest181rcu_preempt12:24:060
332997129,12cyclictest181rcu_preempt09:22:510
332997129,12cyclictest181rcu_preempt09:22:510
35999702,55cyclictest20230-21chrt12:07:283
35999702,53cyclictest17304-21chrt10:13:573
355997048,13cyclictest0-21swapper/208:19:162
355997044,20cyclictest352-21runrttasks08:44:182
35599703,40cyclictest6305-21grep11:44:042
34099707,36cyclictest5425-21apt-get09:54:091
340997051,14cyclictest323-21cyclictest10:50:091
340997045,16cyclictest0-21swapper/109:44:091
340997045,15cyclictest20749-21fschecks_count08:04:041
340997013,19cyclictest19676-21snmpd11:27:481
33299704,34cyclictest5335-21latency_hist10:48:470
332997038,8cyclictest181rcu_preempt10:00:220
332997036,13cyclictest181rcu_preempt10:50:050
35999694,52cyclictest6306-21chrt09:54:393
35999694,51cyclictest20232-21chrt09:24:133
35999693,55cyclictest6356-21chrt11:44:053
35999693,52cyclictest5327-21taskset10:48:433
35999693,33cyclictest6804-21chrt07:19:213
35999692,37cyclictest16201-21taskset11:59:293
35999692,34cyclictest24087-21chrt10:24:193
35999692,34cyclictest1407-21chrt09:47:323
35599698,43cyclictest27122-21ssh12:19:022
35599698,43cyclictest27122-21ssh12:19:022
355996952,10cyclictest0-21swapper/211:59:112
355996949,10cyclictest0-21swapper/207:59:102
355996949,10cyclictest0-21swapper/207:59:102
355996946,12cyclictest0-21swapper/208:18:472
35599693,46cyclictest30774-21switchtime12:24:172
355996914,23cyclictest2769-21fschecks_count07:09:072
340996952,11cyclictest277-21dbus-daemon12:18:441
340996945,16cyclictest365-21ntpd08:58:461
34099693,28cyclictest13299-21modprobe09:13:431
340996924,7cyclictest301ktimers/108:59:071
33299696,33cyclictest19952-21fschecks_time09:24:060
33299694,41cyclictest27131-21grep10:29:200
332996939,11cyclictest8448-21kworker/u18:1+events_unbound08:58:430
332996933,8cyclictest181rcu_preempt08:28:480
33299693,33cyclictest21236-21fschecks_count12:09:060
33299693,33cyclictest21236-21fschecks_count12:09:060
35999683,52cyclictest12034-21chrt07:38:443
35999683,52cyclictest12034-21chrt07:38:443
35999683,34cyclictest23056-21chrt11:18:183
35999682,51cyclictest13645-21taskset11:01:573
355996850,10cyclictest0-21swapper/207:39:012
355996850,10cyclictest0-21swapper/207:39:012
355996835,23cyclictest4164-21latency_hist08:53:462
355996835,23cyclictest4164-21latency_hist08:53:462
355996828,10cyclictest181rcu_preempt09:03:542
355996828,10cyclictest181rcu_preempt09:03:542
355996820,24cyclictest7494-21ssh09:03:472
340996845,15cyclictest12571-21idleruntime07:39:071
340996845,15cyclictest12571-21idleruntime07:39:071
340996842,18cyclictest9513-21ntp_kernel_err07:29:111
34099683,43cyclictest32655-21cat08:39:191
34099683,40cyclictest18155-21sort07:55:011
34099683,40cyclictest18155-21sort07:55:011
34099683,21cyclictest9504-21kernelversion11:49:101
34099682,44cyclictest11468-21cat07:34:191
34099682,44cyclictest11468-21cat07:34:191
332996829,10cyclictest181rcu_preempt11:18:460
332996828,12cyclictest181rcu_preempt10:22:000
21152680,8sleep30-21swapper/309:48:503
35999673,50cyclictest13087-21chrt10:06:153
35999673,50cyclictest13087-21chrt10:06:153
35999672,53cyclictest22994-21chrt09:29:073
35999672,51cyclictest30358-21chrt11:29:173
35999672,35cyclictest23561-21taskset12:13:273
35999672,35cyclictest23561-21taskset12:13:273
35999672,34cyclictest14976-21chrt09:15:003
35999672,32cyclictest18980-21taskset07:58:573
35999672,32cyclictest18980-21taskset07:58:573
35599673,43cyclictest24723-21find12:14:172
355996733,12cyclictest181rcu_preempt11:11:032
355996733,12cyclictest181rcu_preempt11:11:032
340996754,9cyclictest323-21cyclictest10:39:151
340996733,7cyclictest291rcuc/111:08:561
340996733,7cyclictest291rcuc/111:08:561
34099672,45cyclictest3246-21ssh11:39:011
34099672,45cyclictest3246-21ssh11:39:011
332996750,11cyclictest30414-21ssh09:41:180
33299674,50cyclictest23864-21ssh09:29:460
33299674,47cyclictest31970-21cpu08:39:000
332996714,20cyclictest11462-21/usr/sbin/munin07:34:190
332996714,20cyclictest11462-21/usr/sbin/munin07:34:190
35999663,49cyclictest2707-21chrt08:48:593
35999663,49cyclictest2707-21chrt08:48:593
35999663,28cyclictest31692-21chrt10:38:213
35999662,50cyclictest6776-21taskset10:49:243
35999662,50cyclictest12674-21chrt09:12:003
35999662,33cyclictest12283-21chrt07:38:573
35999662,33cyclictest12283-21chrt07:38:573
355996644,16cyclictest28859-21sh10:33:442
355996634,22cyclictest729-21cut07:04:052
355996634,22cyclictest729-21cut07:04:052
34099666,43cyclictest12049-21/usr/sbin/munin11:54:071
340996650,9cyclictest0-21swapper/111:18:441
34099664,48cyclictest2877-21fschecks_count08:49:061
34099664,48cyclictest2877-21fschecks_count08:49:061
34099664,47cyclictest32254-21munin-run11:33:441
340996641,14cyclictest0-21swapper/109:34:051
34099663,43cyclictest16764-21cstates09:19:021
34099663,43cyclictest16764-21cstates09:19:021
340996631,8cyclictest181rcu_preempt07:48:471
33299664,31cyclictest11310-21idleruntime-cro10:58:440
33299664,31cyclictest11310-21idleruntime-cro10:58:440
332996629,15cyclictest181rcu_preempt09:06:000
332996629,15cyclictest181rcu_preempt09:06:000
33299662,32cyclictest17548-21fschecks_count10:14:050
332996614,12cyclictest0-21swapper/007:14:180
35999654,49cyclictest25224-21taskset08:18:463
35999654,49cyclictest22166-21chrt08:08:543
35999653,48cyclictest32430-21chrt12:28:153
35999653,30cyclictest7436-21chrt07:23:583
35999652,32cyclictest4779-21taskset11:40:563
35999652,32cyclictest4779-21taskset11:40:563
35999652,32cyclictest30330-21taskset08:33:573
35999652,30cyclictest26194-21chrt08:19:193
35999652,30cyclictest15171-21chrt07:48:163
35999652,27cyclictest32540-21chrt10:39:113
355996540,18cyclictest0-21swapper/208:09:002
355996523,11cyclictest181rcu_preempt09:29:422
34099654,40cyclictest20863-21latency_hist12:08:461
340996542,16cyclictest15214-21sh11:04:151
340996542,16cyclictest15214-21sh11:04:151
34099653,48cyclictest17070-21ls10:13:461
34099653,35cyclictest31868-21latency_hist08:38:471
340996527,26cyclictest19948-21grep07:59:211
340996527,26cyclictest19948-21grep07:59:211
34099652,41cyclictest8934-21latency_hist07:28:471
34099652,41cyclictest23686-21latency_hist08:13:461
33299654,40cyclictest18400-21if_eth012:04:070
332996540,9cyclictest111-21hwrng08:52:550
332996540,9cyclictest111-21hwrng08:52:550
33299653,50cyclictest30585-21grep11:29:200
33299653,31cyclictest14270-21/usr/sbin/munin10:09:290
332996526,9cyclictest181rcu_preempt10:39:200
35999644,47cyclictest0-21swapper/309:58:573
35999643,48cyclictest807-21chrt07:04:063
35999643,48cyclictest807-21chrt07:04:063
35999642,50cyclictest27826-21chrt12:19:193
35999642,50cyclictest27826-21chrt12:19:193
35999642,49cyclictest27323-21taskset09:36:003
35999642,33cyclictest18744-21chrt07:58:453
35999642,33cyclictest18744-21chrt07:58:453
35999642,31cyclictest16820-21chrt10:13:243
35999642,29cyclictest3053-21chrt12:33:303
35599643,47cyclictest28418-21munin-run08:28:462
34099647,40cyclictest2799-21modprobe11:38:431
34099644,38cyclictest16538-21switchtime07:50:061
34099644,38cyclictest16538-21switchtime07:50:061
340996442,16cyclictest300-21rs:main1
340996442,12cyclictest0-21swapper/112:23:591
34099643,38cyclictest2529-21latency_hist08:48:461
340996426,31cyclictest29189-21ntp_kernel_pll_08:29:101
34099642,46cyclictest352-21runrttasks09:16:191
332996452,8cyclictest323-21cyclictest07:29:110
332996446,12cyclictest25434-21apt-get08:18:580
33299644,30cyclictest23550-21idleruntime-cro08:13:440
35999634,47cyclictest5340-21chrt07:16:043
35999633,49cyclictest23798-21chrt11:18:593
35999633,47cyclictest0-21swapper/308:23:593
35999632,30cyclictest628-21chrt08:41:223
35999632,30cyclictest10381-21taskset11:49:563
35999632,28cyclictest2484-21chrt07:08:563
35999632,27cyclictest20284-21chrt10:19:003
340996348,8cyclictest0-21swapper/111:48:441
34099633,43cyclictest26860-21latency_hist08:23:461
340996321,10cyclictest301ktimers/110:25:281
33299633,46cyclictest16166-21awk11:59:240
33299633,41cyclictest7410-21/usr/sbin/munin07:24:200
35999623,47cyclictest25660-21taskset12:15:553
35999623,29cyclictest9121-21chrt07:29:003
35999623,26cyclictest21563-21chrt08:04:203
35999622,29cyclictest2395-21chrt08:48:263
35999622,28cyclictest27322-21chrt10:29:263
340996225,25cyclictest10585-1kworker/1:0H+kblockd08:28:471
33299623,48cyclictest18697-21modprobe07:58:430
33299623,48cyclictest18697-21modprobe07:58:430
332996233,10cyclictest58-21kcompactd007:39:010
332996233,10cyclictest58-21kcompactd007:39:010
33299622,30cyclictest7292-21latency_hist09:03:470
35999602,30cyclictest30003-21taskset08:33:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional