You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-20 - 02:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Thu Nov 19, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31866212115,77sleep10-21swapper/107:04:131
541991153,80cyclictest4583-21apt-get12:04:030
541991153,80cyclictest4583-21apt-get12:04:030
541991123,88cyclictest19633-21modprobe08:04:010
1655721070,12sleep0211rcuc/010:14:560
3207421048,68sleep232103-21processes07:04:232
541991023,69cyclictest11876-21ssh09:14:240
3242121010,8sleep00-21swapper/011:48:510
541999737,28cyclictest181rcu_preempt11:39:000
54199955,73cyclictest29952-21ssh09:43:570
315192938,71sleep00-21swapper/007:03:560
56299914,65cyclictest11680-21apt-get09:14:252
541998864,16cyclictest203-21systemd-journal09:28:230
541998863,9cyclictest181rcu_preempt08:09:170
541998850,13cyclictest181rcu_preempt09:34:090
541998850,13cyclictest181rcu_preempt09:34:090
562998770,11cyclictest10787-21cron07:38:412
541998757,9cyclictest181rcu_preempt11:44:080
56299862,65cyclictest15692-21apt-get10:14:052
56299843,67cyclictest29375-21apt-get09:43:572
541998460,10cyclictest4904-21/usr/sbin/munin12:04:540
541998460,10cyclictest4904-21/usr/sbin/munin12:04:540
541998451,24cyclictest12498-21latency_hist07:43:440
541998451,12cyclictest181rcu_preempt12:08:560
541998451,12cyclictest181rcu_preempt12:08:560
541998449,13cyclictest181rcu_preempt12:18:580
541998446,26cyclictest27505-21latency_hist08:28:430
562998354,20cyclictest357-21runrttasks09:39:082
554998363,14cyclictest1321-21grep09:49:141
55499833,55cyclictest16126-21apt-get07:53:551
55499824,67cyclictest7832-21apt-get07:28:581
55499823,66cyclictest11781-21apt-get12:24:131
541998258,18cyclictest24105-21latency_hist11:23:440
541998251,9cyclictest181rcu_preempt07:53:450
541998240,32cyclictest0-21swapper/009:19:130
541998236,12cyclictest181rcu_preempt09:08:590
54199823,43cyclictest15-21ksoftirqd/010:07:440
554998147,25cyclictest30554-21modprobe11:43:411
541998161,15cyclictest532-21cyclictest11:14:020
541998146,16cyclictest181rcu_preempt09:54:530
562998060,14cyclictest11401-21apt-key12:23:432
562998050,20cyclictest19658-21fschecks_count11:14:012
55499804,62cyclictest4923-21apt-get12:04:071
55499804,62cyclictest4923-21apt-get12:04:071
541998030,11cyclictest181rcu_preempt08:14:150
562997963,9cyclictest0-21swapper/211:04:302
562997960,13cyclictest18098-21kernelversion09:24:052
562997929,21cyclictest39-21ksoftirqd/210:40:292
554997955,18cyclictest12268-21latency_hist10:08:431
55499787,61cyclictest13460-21apt-get11:03:561
554997859,13cyclictest1-21systemd10:19:221
554997859,13cyclictest1-21systemd10:19:221
541997851,19cyclictest29382-21cstates08:33:570
541997850,12cyclictest181rcu_preempt09:53:450
541997847,9cyclictest181rcu_preempt07:08:560
541997847,9cyclictest181rcu_preempt07:08:560
541997840,16cyclictest181rcu_preempt07:34:170
541997839,13cyclictest181rcu_preempt10:09:190
54199783,28cyclictest12919-21apt-get11:03:510
554997751,19cyclictest19489-21latency_hist08:03:451
554997751,19cyclictest19489-21latency_hist08:03:451
55499773,54cyclictest5166-21apt-get08:58:561
55499773,54cyclictest15142-21apt-get10:13:561
541997756,15cyclictest13250-21latency_hist12:28:430
541997750,18cyclictest5728-21sh07:23:410
541997737,8cyclictest181rcu_preempt10:44:160
541997733,17cyclictest181rcu_preempt11:34:040
562997660,10cyclictest6677-21/usr/sbin/munin12:09:092
562997660,10cyclictest6677-21/usr/sbin/munin12:09:092
562997658,13cyclictest2050-21kernelversion08:49:052
562997658,13cyclictest2050-21kernelversion08:49:052
562997657,14cyclictest1469-21fschecks_count10:44:022
554997656,14cyclictest30672-21kernelversion09:45:021
554997652,17cyclictest0-21swapper/111:48:441
554997651,19cyclictest7430-21timerwakeupswit09:59:171
541997648,20cyclictest27445-21latency_hist11:33:440
562997559,10cyclictest357-21runrttasks10:08:402
554997558,12cyclictest11252-21apt-key09:13:441
554997557,12cyclictest24351-21/usr/sbin/munin11:24:081
554997553,16cyclictest611-21cron12:33:411
554997550,16cyclictest21169-21latency_hist08:08:451
541997552,13cyclictest0-21swapper/011:08:540
541997550,11cyclictest181rcu_preempt10:48:580
541997535,30cyclictest0-21swapper/010:24:180
541997522,15cyclictest7665-21apt-config07:28:450
541997522,15cyclictest7665-21apt-config07:28:450
562997437,9cyclictest532-21cyclictest09:04:502
562997437,9cyclictest532-21cyclictest09:04:502
562997431,27cyclictest39-21ksoftirqd/210:19:352
562997431,27cyclictest39-21ksoftirqd/210:19:352
554997458,10cyclictest1-21systemd08:48:421
541997450,15cyclictest26511-21ssh10:31:190
541997448,8cyclictest181rcu_preempt10:18:580
541997448,8cyclictest181rcu_preempt10:18:580
541997440,10cyclictest181rcu_preempt10:55:430
541997440,10cyclictest181rcu_preempt10:55:430
54199743,57cyclictest357-21runrttasks10:39:130
541997432,11cyclictest181rcu_preempt08:54:150
541997429,15cyclictest181rcu_preempt08:03:420
541997429,15cyclictest181rcu_preempt08:03:420
54199742,51cyclictest1607-21cpu11:53:560
562997349,7cyclictest181rcu_preempt09:38:032
562997349,7cyclictest181rcu_preempt09:38:032
554997352,15cyclictest611-21cron07:53:411
554997352,15cyclictest1800-21fschecks_count11:54:021
541997348,9cyclictest181rcu_preempt08:48:500
541997348,9cyclictest181rcu_preempt08:48:500
541997341,11cyclictest181rcu_preempt07:14:030
541997336,13cyclictest181rcu_preempt11:24:120
541997332,14cyclictest181rcu_preempt08:43:560
541997315,21cyclictest15-21ksoftirqd/009:29:050
562997256,10cyclictest12877-21fschecks_count07:44:032
562997254,13cyclictest3951-21switchtime11:59:152
562997254,13cyclictest3951-21switchtime11:59:152
56299723,28cyclictest6106-21unixbench_multi08:59:302
554997256,10cyclictest28941-21idleruntime-cro11:38:411
554997251,14cyclictest4582-21latency_hist12:04:051
554997248,15cyclictest315-21kernelversion11:49:051
554997246,18cyclictest19676-21fschecks_count11:14:011
554997242,11cyclictest181rcu_preempt07:23:451
55499723,47cyclictest27484-21latency_hist11:33:441
554997226,32cyclictest21309-21awk08:08:571
541997226,28cyclictest181rcu_preempt09:41:250
541997222,11cyclictest181rcu_preempt07:55:110
54199722,16cyclictest27944-21/usr/sbin/munin08:29:020
57199717,53cyclictest13507-21chrt11:03:583
562997153,12cyclictest532-21cyclictest09:59:052
56299714,29cyclictest39-21ksoftirqd/207:19:012
554997152,13cyclictest611-21cron12:38:411
554997151,14cyclictest532-21cyclictest07:24:131
554997151,14cyclictest532-21cyclictest07:24:131
554997150,15cyclictest532-21cyclictest08:41:231
554997147,15cyclictest24541-21diskmemload09:34:151
554997147,15cyclictest24541-21diskmemload09:34:151
541997143,21cyclictest7037-21ntp_kernel_err09:59:080
562997045,7cyclictest181rcu_preempt10:59:502
554997054,11cyclictest8137-21diskmemload10:24:011
55499703,40cyclictest17387-21munin-run09:23:411
554997012,27cyclictest357-21runrttasks09:39:561
541997033,10cyclictest181rcu_preempt09:04:060
541997033,10cyclictest181rcu_preempt09:04:060
54199702,42cyclictest31826-21tail08:39:170
562996954,9cyclictest27988-21sh10:34:032
562996954,9cyclictest27988-21sh10:34:032
562996953,11cyclictest374-21snmpd09:19:112
562996944,11cyclictest181rcu_preempt09:29:162
56299693,44cyclictest12047-21fschecks_count12:24:452
562996933,15cyclictest181rcu_preempt09:53:192
562996931,31cyclictest15280-21kworker/u19:2+nfsiod10:31:102
562996923,30cyclictest39-21ksoftirqd/212:18:282
554996954,9cyclictest4640-21cron08:58:411
554996949,11cyclictest12770-21timerandwakeup09:14:511
554996945,13cyclictest0-21swapper/107:43:421
55499694,42cyclictest357-21runrttasks08:34:401
55499693,48cyclictest4923-21apt-get12:03:561
55499693,46cyclictest24392-21latency_hist08:18:441
55499693,44cyclictest6292-21latency_hist09:58:431
541996944,10cyclictest181rcu_preempt12:28:520
541996944,10cyclictest181rcu_preempt10:59:370
541996931,17cyclictest181rcu_preempt08:23:440
562996850,10cyclictest0-21swapper/210:13:462
562996846,15cyclictest0-21swapper/212:29:142
56299684,35cyclictest8136-21fschecks_time07:29:592
56299684,35cyclictest3976-21mailstats08:54:142
562996825,27cyclictest15998-21date07:53:452
562996824,28cyclictest39-21ksoftirqd/207:56:302
554996849,13cyclictest28207-21netstat10:34:081
554996849,13cyclictest28207-21netstat10:34:081
554996848,12cyclictest0-21swapper/110:39:161
55499684,35cyclictest20661-21/usr/sbin/munin09:28:501
55499683,43cyclictest28640-21unixbench_singl08:29:161
541996854,9cyclictest532-21cyclictest10:38:450
541996854,9cyclictest532-21cyclictest10:38:450
541996835,11cyclictest181rcu_preempt08:59:260
541996827,12cyclictest181rcu_preempt07:32:090
56299679,25cyclictest39-21ksoftirqd/209:54:462
562996750,12cyclictest532-21cyclictest07:09:162
562996750,12cyclictest532-21cyclictest07:09:162
562996747,13cyclictest293-21systemd-logind12:08:412
562996747,13cyclictest293-21systemd-logind12:08:412
562996744,8cyclictest181rcu_preempt10:49:142
56299673,44cyclictest32176-21munin-run11:48:412
56299673,30cyclictest17441-21/usr/sbin/munin11:09:172
562996730,18cyclictest39-21ksoftirqd/210:54:242
562996730,18cyclictest39-21ksoftirqd/210:54:242
554996751,10cyclictest18900-21ssh11:13:061
554996748,11cyclictest0-21swapper/107:48:421
554996747,13cyclictest1-21systemd11:22:221
55499673,46cyclictest11455-21/usr/sbin/munin10:59:391
55499673,37cyclictest9648-21fschecks_count07:34:021
55499672,47cyclictest18365-21ps09:24:131
541996733,12cyclictest181rcu_preempt07:47:130
562996648,11cyclictest297-21rs:main2
56299664,27cyclictest39-21ksoftirqd/211:33:542
562996640,17cyclictest532-21cyclictest11:38:572
562996636,21cyclictest32263-21modprobe08:43:412
56299663,40cyclictest4332-21latency_hist07:18:452
562996632,7cyclictest181rcu_preempt08:37:142
554996648,11cyclictest0-21swapper/112:13:561
55499664,46cyclictest2603-21cpu07:13:511
55499663,45cyclictest1191-21ntp_kernel_err07:09:081
55499663,45cyclictest1191-21ntp_kernel_err07:09:081
541996648,13cyclictest8395-21cstates12:13:570
56299654,28cyclictest39-21ksoftirqd/208:13:512
562996541,13cyclictest0-21swapper/211:53:582
562996539,16cyclictest0-21swapper/208:19:152
562996537,19cyclictest29022-21modprobe08:33:412
56299653,37cyclictest22768-21unixbench_singl10:24:162
562996517,21cyclictest371rcuc/208:24:012
554996545,14cyclictest1209-21/usr/sbin/munin10:44:151
55499654,43cyclictest26662-21wc08:24:121
55499654,35cyclictest7895-21ps10:54:141
55499654,35cyclictest7895-21ps10:54:141
55499653,42cyclictest357-21runrttasks08:20:251
541996539,8cyclictest181rcu_preempt12:33:510
562996445,13cyclictest0-21swapper/208:44:022
562996433,22cyclictest19143-21apt-key08:03:432
562996433,22cyclictest19143-21apt-key08:03:432
56299642,24cyclictest21421-21fschecks_count08:09:022
554996448,9cyclictest0-21swapper/110:49:161
554996439,12cyclictest0-21swapper/112:13:431
554996439,12cyclictest0-21swapper/112:13:431
55499643,35cyclictest27349-21ssh10:33:401
554996420,34cyclictest15326-21readlink10:13:441
57199633,42cyclictest16134-21chrt07:53:553
562996341,11cyclictest0-21swapper/211:49:032
562996316,18cyclictest371rcuc/207:24:112
562996316,18cyclictest371rcuc/207:24:112
55499633,31cyclictest7007-21grep09:04:021
55499633,31cyclictest7007-21grep09:04:021
57199623,48cyclictest4934-21chrt12:03:563
57199623,48cyclictest4934-21chrt12:03:563
57199623,44cyclictest23506-21chrt10:26:173
562996245,9cyclictest0-21swapper/211:19:162
562996241,7cyclictest181rcu_preempt08:08:092
56299623,45cyclictest357-21runrttasks11:26:312
56299623,32cyclictest15199-21grep12:34:022
554996239,9cyclictest0-21swapper/108:49:021
554996239,9cyclictest0-21swapper/108:49:021
125152620,8sleep30-21swapper/309:14:473
562996142,13cyclictest11086-21cpu07:38:572
562996113,20cyclictest39-21ksoftirqd/209:11:022
57199604,43cyclictest32380-21chrt10:40:563
57199603,39cyclictest23311-21taskset11:19:143
554995935,17cyclictest611-21cron12:23:411
57199578,30cyclictest0-21swapper/308:58:573
57199574,36cyclictest2482-21taskset10:44:413
57199572,39cyclictest2140-21chrt08:49:073
57199572,39cyclictest2140-21chrt08:49:073
57199556,30cyclictest0-21swapper/307:29:043
57199553,40cyclictest8208-21taskset12:13:433
57199553,40cyclictest8208-21taskset12:13:433
57199553,37cyclictest23162-21chrt08:14:043
57199553,36cyclictest7261-21chrt10:53:573
57199553,36cyclictest7261-21chrt10:53:573
57199553,36cyclictest10368-21taskset10:04:173
57199543,39cyclictest11571-21taskset07:39:083
57199543,38cyclictest19827-21taskset09:27:123
57199543,34cyclictest27943-21chrt08:29:023
57199543,34cyclictest11594-21chrt09:13:463
57199543,32cyclictest19954-21chrt08:04:203
57199542,43cyclictest25954-21taskset10:29:463
57199542,39cyclictest4940-21chrt09:55:213
57199542,33cyclictest3005-21chrt07:14:063
57199536,27cyclictest0-21swapper/312:23:553
57199535,35cyclictest12759-21taskset10:09:033
57199533,37cyclictest29150-21chrt09:43:053
57199533,36cyclictest4378-21chrt07:18:513
57199533,34cyclictest4245-21chrt08:54:173
57199532,41cyclictest13088-21chrt07:44:073
57199532,38cyclictest8656-21chrt10:02:183
57199532,30cyclictest5460-21chrt10:49:403
57199524,31cyclictest23588-21taskset09:33:453
57199523,40cyclictest12962-21chrt11:03:423
57199523,36cyclictest14666-21chrt09:18:493
57199523,35cyclictest31594-21taskset08:39:153
571995215,30cyclictest0-21swapper/310:13:563
57199515,29cyclictest0-21swapper/312:04:073
57199515,29cyclictest0-21swapper/312:04:073
57199512,35cyclictest25519-21chrt11:26:233
57199512,35cyclictest24401-21taskset09:34:133
57199512,35cyclictest24401-21taskset09:34:133
57199507,28cyclictest0-21swapper/309:43:563
57199503,31cyclictest25108-21chrt08:19:143
57199502,31cyclictest18288-21chrt07:59:093
57199502,31cyclictest18288-21chrt07:59:093
57199493,32cyclictest711-21chrt09:49:013
57199493,31cyclictest21555-21chrt11:16:593
57199482,35cyclictest10928-21chrt12:19:183
571994813,25cyclictest2073-21taskset07:13:413
571994813,25cyclictest2073-21taskset07:13:413
57199473,35cyclictest30290-21chrt10:38:333
57199473,35cyclictest30290-21chrt10:38:333
57199472,35cyclictest15739-21chrt07:53:423
57199472,33cyclictest2980-21chrt11:58:413
57199472,33cyclictest1422-21chrt11:53:433
571994715,21cyclictest727-21chrt08:44:133
571994710,21cyclictest29902-21chrt08:34:133
571994611,21cyclictest26085-21taskset08:23:523
57199453,29cyclictest22201-21chrt08:09:183
57199452,29cyclictest16484-21taskset11:08:573
57199448,23cyclictest9417-21taskset07:33:463
57199444,30cyclictest7470-21chrt07:28:443
57199444,30cyclictest7470-21chrt07:28:443
57199443,28cyclictest31734-21chrt11:44:163
318392447,21sleep30-21swapper/307:04:113
57199438,24cyclictest30386-21taskset11:41:333
57199433,31cyclictest21134-21chrt10:23:013
57199433,31cyclictest21134-21chrt10:23:013
57199432,28cyclictest14870-21taskset12:33:433
571994210,21cyclictest7142-21chrt09:04:053
571994210,21cyclictest7142-21chrt09:04:053
57199419,23cyclictest16033-21taskset12:34:533
57199413,23cyclictest27092-21chrt11:30:523
57199402,29cyclictest28165-21chrt11:34:123
57199392,27cyclictest9702-21chrt12:18:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional