You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 09:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Fri Aug 21, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
62859912340,76cyclictest650-21cron22:09:530
6285991143,30cyclictest2380-21sed00:15:220
6285991103,28cyclictest3719-21/usr/sbin/munin21:46:230
62859910927,73cyclictest0-21swapper/023:00:050
6285991085,31cyclictest26544-21runrttasks20:10:140
6285991082,27cyclictest23895-21ntp_kernel_pll_21:25:240
6285991082,27cyclictest23895-21ntp_kernel_pll_21:25:240
62859910652,49cyclictest16603-21kworker/u17:2+xprtiod21:16:300
62859910539,59cyclictest26670-21sh21:30:150
586321059,70sleep15864-21users19:06:191
6285991044,37cyclictest31942-21sort20:25:350
62859910433,64cyclictest0-21swapper/021:20:020
62859910239,53cyclictest0-21swapper/023:50:150
6285991022,32cyclictest14559-21sh23:43:340
6285991022,32cyclictest14559-21sh23:43:340
6285991013,29cyclictest351-21runrttasks21:56:340
6285991013,29cyclictest351-21runrttasks21:56:340
6285991013,25cyclictest351-21runrttasks21:10:140
6285991013,19cyclictest13920-21/usr/sbin/munin22:00:250
6285991012,31cyclictest27943-21/usr/sbin/munin00:05:160
6285991012,31cyclictest27943-21/usr/sbin/munin00:05:160
6285991012,30cyclictest767-21uname21:40:180
6285991012,29cyclictest2487-21/usr/sbin/munin23:25:120
6285991012,22cyclictest9500-21/usr/sbin/munin23:35:260
6285991004,31cyclictest15035-21taskset19:35:070
62859910030,61cyclictest0-21swapper/020:50:070
6285991002,30cyclictest32370-21ssh21:39:440
6285991002,20cyclictest8597-21date23:34:540
628599994,86cyclictest25299-21sed00:00:180
628599994,86cyclictest25299-21sed00:00:180
628599994,27cyclictest351-21runrttasks19:45:070
628599994,27cyclictest351-21runrttasks19:45:070
628599993,84cyclictest6160-21apt-get22:40:040
628599992,29cyclictest20639-21unixbench_multi22:10:300
628599992,29cyclictest20639-21unixbench_multi22:10:300
628599992,28cyclictest11810-21uname00:30:180
628599992,28cyclictest11810-21uname00:30:180
6289999849,37cyclictest22965-21sh21:24:531
628599983,45cyclictest16913-21ssh00:38:530
628599983,35cyclictest202-21systemd-journal22:17:560
628599983,30cyclictest8501-21/usr/sbin/munin19:15:180
628599983,30cyclictest12073-21/usr/sbin/munin21:05:140
628599983,21cyclictest26399-21cut22:20:190
628599982,29cyclictest29817-21apt-config20:19:570
628599982,27cyclictest481-21/usr/sbin/munin22:30:250
628599973,29cyclictest351-21runrttasks23:46:260
628599973,26cyclictest25554-21kworker/u17:4+xprtiod23:24:540
628599973,25cyclictest10140-21/usr/sbin/munin19:20:250
6285999712,24cyclictest26010-21rm23:11:080
506229710,58sleep05134-21fschecks_time19:05:580
6290999675,15cyclictest23830-21sed22:15:292
628599964,40cyclictest7869-21timerwakeupswit21:50:300
628599964,28cyclictest13069-21/usr/sbin/munin22:50:190
628599963,38cyclictest2988-21timerwakeupswit00:15:340
628599963,31cyclictest16594-21/usr/sbin/munin19:40:130
628599963,28cyclictest2958-21/usr/sbin/munin22:35:160
628599963,28cyclictest2958-21/usr/sbin/munin22:35:160
628599963,22cyclictest495-21cron20:29:520
628599963,22cyclictest495-21cron20:29:520
628599962,32cyclictest7836-21idleruntime-cro19:14:530
628599962,32cyclictest7836-21idleruntime-cro19:14:530
628599962,28cyclictest1677-21grep20:30:290
628599962,22cyclictest21915-21ssh23:55:140
628599962,21cyclictest23736-21ssh23:06:560
628599953,26cyclictest23049-21/usr/sbin/munin20:00:030
628599952,31cyclictest12493-21wc19:25:280
628599952,29cyclictest26302-21apt-config20:09:560
628599952,29cyclictest26302-21apt-config20:09:560
628599952,27cyclictest29933-21ssh22:25:280
628599943,38cyclictest22781-21cron19:59:520
628599943,18cyclictest2380-21apt-get20:35:040
6285999413,21cyclictest21383-21date19:54:560
6285999413,21cyclictest21383-21date19:54:560
628599933,29cyclictest351-21runrttasks22:49:030
628599933,27cyclictest11237-21timerandwakeup21:00:270
628599933,27cyclictest11237-21timerandwakeup21:00:270
628599933,26cyclictest5555-21latency_hist20:44:560
628599932,31cyclictest9459-21find20:55:250
628599932,31cyclictest9459-21find20:55:250
628599923,27cyclictest28584-21/usr/sbin/munin23:15:280
628599922,33cyclictest18639-21latency_hist22:59:550
628599922,21cyclictest5135-21apt-get00:20:040
628599913,31cyclictest14199-21timerwakeupswit19:30:270
628599913,30cyclictest351-21runrttasks20:45:260
628599912,25cyclictest8245-21/usr/sbin/munin00:25:300
61942909,55sleep26211-21unin-run19:09:552
6290998860,7cyclictest181rcu_preempt19:19:562
6290998636,14cyclictest181rcu_preempt22:57:102
6290998561,9cyclictest181rcu_preempt23:54:332
629099855,43cyclictest39-21ksoftirqd/221:59:552
629099855,43cyclictest39-21ksoftirqd/221:59:552
6290998458,9cyclictest181rcu_preempt21:20:182
629099839,51cyclictest12807-21diskstats23:41:092
629099839,51cyclictest12807-21diskstats23:41:092
6290998333,14cyclictest181rcu_preempt23:01:352
6290998252,11cyclictest181rcu_preempt00:09:152
6290998252,11cyclictest181rcu_preempt00:09:152
6290998248,9cyclictest181rcu_preempt23:45:202
6290998233,12cyclictest181rcu_preempt00:39:172
6290998161,15cyclictest26554-21sh23:11:362
629099814,30cyclictest39-21ksoftirqd/200:28:542
628999819,17cyclictest0-21swapper/119:15:111
6289998158,15cyclictest19942-21ssh23:51:141
6289998139,30cyclictest4432-21sort21:46:221
628999813,48cyclictest6936-21awk22:40:251
6290998065,9cyclictest30209-21/usr/sbin/munin20:20:442
6290998060,14cyclictest351-21runrttasks00:15:252
6289998061,13cyclictest26299-21apt-key20:09:571
6289998061,13cyclictest26299-21apt-key20:09:571
6290997957,16cyclictest10421-21fschecks_time19:20:142
6290997956,8cyclictest181rcu_preempt20:45:142
6290997955,9cyclictest181rcu_preempt20:05:322
6290997955,9cyclictest181rcu_preempt20:05:322
6290997939,13cyclictest181rcu_preempt22:24:192
6290997933,14cyclictest181rcu_preempt19:31:472
6289997962,10cyclictest23314-21ssh23:56:531
6290997862,10cyclictest13987-21ssh22:00:272
6290997833,23cyclictest181rcu_preempt22:30:452
6289997833,31cyclictest10696-21cut21:00:141
6289997833,31cyclictest10696-21cut21:00:141
6290997752,10cyclictest181rcu_preempt21:15:322
6290997739,10cyclictest181rcu_preempt21:36:332
6290997729,13cyclictest181rcu_preempt23:57:032
6289997760,11cyclictest6549-21threads23:30:281
6289997753,14cyclictest25789-21ssh00:00:281
6289997753,14cyclictest25789-21ssh00:00:281
6290997661,10cyclictest12249-21sh00:30:292
6290997661,10cyclictest12249-21sh00:30:292
6290997655,13cyclictest58-21kcompactd021:26:162
6290997655,13cyclictest58-21kcompactd021:26:162
6290997654,16cyclictest16357-21idleruntime-cro19:39:532
6290997654,14cyclictest202-21systemd-journal23:28:432
6290997637,12cyclictest181rcu_preempt00:18:472
629099762,42cyclictest39-21ksoftirqd/223:07:392
6290997622,31cyclictest0-21swapper/220:15:082
6289997658,13cyclictest58-21kcompactd022:34:101
6289997652,9cyclictest181rcu_preempt22:25:171
6289997619,51cyclictest301ktimers/120:00:081
6290997557,12cyclictest29686-21ssh22:25:212
6290997555,13cyclictest280-21systemd-logind22:09:532
6290997545,8cyclictest181rcu_preempt21:05:062
6290997538,13cyclictest181rcu_preempt22:54:422
629099752,41cyclictest39-21ksoftirqd/223:15:032
629099752,35cyclictest39-21ksoftirqd/200:04:322
629099752,35cyclictest39-21ksoftirqd/200:04:322
629099752,24cyclictest39-21ksoftirqd/221:40:102
262472750,9sleep30-21swapper/323:11:273
6290997441,14cyclictest181rcu_preempt20:52:552
6290997429,16cyclictest181rcu_preempt21:12:142
629099742,44cyclictest39-21ksoftirqd/220:56:072
629099742,44cyclictest39-21ksoftirqd/220:56:072
629099742,31cyclictest39-21ksoftirqd/222:48:382
6289997455,13cyclictest32438-21sh23:20:231
6289997445,22cyclictest2136-21sh21:42:131
6290997332,12cyclictest181rcu_preempt20:14:542
6290997317,30cyclictest39-21ksoftirqd/223:35:052
6289997329,31cyclictest29044-21grep20:15:281
6290997254,13cyclictest8231-21kworker/u18:2-events_unbound21:45:342
6290997252,15cyclictest6279-21cyclictest00:20:042
6290997239,12cyclictest181rcu_preempt22:42:332
6290997237,13cyclictest181rcu_preempt22:10:172
6290997237,13cyclictest181rcu_preempt22:10:172
6290997232,13cyclictest181rcu_preempt22:36:322
6290997232,13cyclictest181rcu_preempt22:36:322
6290997227,8cyclictest181rcu_preempt20:30:272
6289997252,13cyclictest19945-21sed23:00:301
6289997251,15cyclictest15239-21fschecks_time19:35:131
6289997244,19cyclictest18547-21idleruntime-cro22:59:531
6289997221,24cyclictest19261-21ssh22:09:521
6290997145,11cyclictest181rcu_preempt20:29:322
6290997145,11cyclictest181rcu_preempt20:29:322
6290997131,15cyclictest181rcu_preempt21:54:212
6289997153,12cyclictest19889-21cpuspeed19:50:091
6289997153,12cyclictest19889-21cpuspeed19:50:091
6289997150,15cyclictest3723-21sh20:39:531
6289997150,15cyclictest13440-21diskmemload22:20:221
6290997052,12cyclictest280-21systemd-logind19:59:542
6290997047,15cyclictest6279-21cyclictest19:25:152
6290997035,16cyclictest181rcu_preempt23:30:022
629099703,46cyclictest3175-21ls20:35:282
6290997034,12cyclictest181rcu_preempt21:01:292
6290997034,12cyclictest181rcu_preempt21:01:292
6290997030,13cyclictest181rcu_preempt20:41:442
6289997055,11cyclictest6279-21cyclictest00:30:031
6289997055,11cyclictest6279-21cyclictest00:30:031
6289997053,10cyclictest0-21swapper/123:49:511
6289997049,14cyclictest0-21swapper/119:55:251
6290996945,9cyclictest181rcu_preempt23:22:052
6290996944,11cyclictest181rcu_preempt20:00:292
629099693,39cyclictest16872-21cut19:40:142
6290996920,36cyclictest26795-21idleruntime21:30:162
6289996951,13cyclictest22627-21ntp_kernel_err23:05:201
6289996948,15cyclictest19155-21unixbench_singl19:45:281
6289996948,15cyclictest19155-21unixbench_singl19:45:281
6289996939,21cyclictest18792-21sh21:17:221
6289996922,15cyclictest181rcu_preempt23:35:071
483426912,29sleep30-21swapper/319:05:153
6290996849,15cyclictest32579-21kworker/u18:0+events_unbound19:49:522
6290996849,15cyclictest32579-21kworker/u18:0+events_unbound19:49:522
6290996832,12cyclictest181rcu_preempt19:50:232
6290996832,12cyclictest181rcu_preempt19:50:232
6289996848,14cyclictest32444-21idleruntime-cro21:39:531
6289996846,16cyclictest3079-21ntp_kernel_pll_23:25:201
6289996844,16cyclictest9641-21cut22:45:151
6289996842,19cyclictest1931-21ssh00:15:111
6289996830,13cyclictest181rcu_preempt20:54:561
6289996751,9cyclictest0-21swapper/120:25:301
6289996750,10cyclictest257-21dbus-daemon20:29:531
6289996750,10cyclictest257-21dbus-daemon20:29:531
6289996749,11cyclictest29286-21ssh21:34:521
6289996746,15cyclictest22429-21idleruntime-cro22:14:531
6289996746,15cyclictest22429-21idleruntime-cro22:14:531
6289996746,14cyclictest0-21swapper/120:34:521
628999671,29cyclictest4780-21latency_hist00:19:551
6290996648,11cyclictest6868-21latency19:10:172
6290996648,11cyclictest6868-21latency19:10:172
6289996653,9cyclictest58-21kcompactd020:11:281
6289996650,9cyclictest0-21swapper/119:25:151
6289996648,12cyclictest8062-21sh00:24:551
6289996642,11cyclictest181rcu_preempt23:15:051
628999662,30cyclictest15247-21mailstats00:35:261
6289996551,10cyclictest6279-21cyclictest22:50:161
6289996548,10cyclictest0-21swapper/121:59:531
6289996548,10cyclictest0-21swapper/121:59:531
6289996545,14cyclictest5831-21idleruntime-cro22:39:531
6289996545,14cyclictest5831-21idleruntime-cro22:39:531
6289996540,16cyclictest15065-21cron23:44:521
6289996540,16cyclictest15065-21cron23:44:521
628999653,42cyclictest6958-21modprobe20:49:531
628999653,34cyclictest351-21runrttasks21:13:021
629199649,35cyclictest0-21swapper/300:10:263
6289996448,9cyclictest280-21systemd-logind21:54:541
6289996447,9cyclictest0-21swapper/123:11:411
6289996446,8cyclictest0-21swapper/100:25:161
6289996443,15cyclictest12293-21fschecks_count21:05:151
6289996442,12cyclictest0-21swapper/119:44:531
6289996332,21cyclictest5347-21modprobe20:44:531
628999624,38cyclictest13942-21ps19:30:231
6289996238,15cyclictest0-21swapper/100:09:511
6289996238,15cyclictest0-21swapper/100:09:511
6289996217,34cyclictest15147-21ssh22:02:281
6289996122,12cyclictest23566-21fschecks_count21:25:171
6289996122,12cyclictest23566-21fschecks_count21:25:171
628999604,11cyclictest10159-21apt-get19:20:051
6289996018,31cyclictest23611-21threads22:15:271
6291995946,9cyclictest451rcuc/323:16:513
6291995940,13cyclictest451rcuc/322:32:143
628999584,38cyclictest6724-21fschecks_time19:10:141
628999584,38cyclictest6724-21fschecks_time19:10:141
628999582,36cyclictest9313-21netstat20:55:201
628999582,36cyclictest9313-21netstat20:55:201
6291995742,9cyclictest451rcuc/322:08:233
6291995739,11cyclictest451rcuc/322:45:143
629199573,41cyclictest22005-21taskset23:04:583
629199572,39cyclictest10023-21taskset19:19:553
6291995643,9cyclictest451rcuc/320:12:173
6291995643,7cyclictest451rcuc/322:42:063
6291995642,9cyclictest451rcuc/323:57:513
629199563,41cyclictest7366-21taskset19:10:283
629199563,41cyclictest7366-21taskset19:10:283
629199563,36cyclictest28396-21chrt00:05:203
629199563,36cyclictest28396-21chrt00:05:203
629199553,38cyclictest11375-21chrt00:30:033
629199553,38cyclictest11375-21chrt00:30:033
6291995525,22cyclictest14511-21chrt00:35:063
629199552,39cyclictest24889-21taskset21:26:453
629199552,39cyclictest24889-21taskset21:26:453
629199543,38cyclictest2847-21chrt21:44:033
629199543,38cyclictest13584-21chrt22:00:163
629199543,35cyclictest11213-21chrt00:29:553
629199535,29cyclictest0-21swapper/323:40:543
629199535,29cyclictest0-21swapper/323:40:543
629199533,40cyclictest20407-21taskset23:01:303
629199533,37cyclictest2926-21taskset20:35:243
629199533,36cyclictest20126-21taskset22:10:213
629199533,36cyclictest20126-21taskset22:10:213
629199533,35cyclictest26967-21taskset22:20:303
629199533,35cyclictest21472-21chrt23:54:543
629199532,39cyclictest13920-21chrt22:51:093
629199524,35cyclictest8596-21chrt21:51:573
629199523,38cyclictest10556-21taskset21:55:183
629199523,38cyclictest10556-21taskset21:55:183
629199523,37cyclictest30507-21taskset22:26:133
629199523,36cyclictest27109-21chrt00:03:333
629199523,36cyclictest27109-21chrt00:03:333
629199514,35cyclictest29867-21chrt20:19:573
629199513,39cyclictest16000-21chrt22:55:143
629199513,38cyclictest18503-21chrt19:45:143
629199513,38cyclictest18503-21chrt19:45:143
629199513,36cyclictest17620-21taskset21:15:263
629199513,36cyclictest15812-21chrt23:45:183
629199513,35cyclictest28038-21taskset21:31:433
629199513,35cyclictest10429-21chrt23:36:433
629199513,34cyclictest15459-21taskset19:35:183
629199513,32cyclictest8752-21chrt20:54:563
629199513,32cyclictest13232-21chrt19:29:563
6291995133,11cyclictest451rcuc/319:54:063
6291995133,11cyclictest451rcuc/319:54:063
629199512,37cyclictest23999-21chrt20:00:293
629199508,29cyclictest0-21swapper/321:45:183
629199504,30cyclictest3089-21chrt22:35:093
629199504,30cyclictest3089-21chrt22:35:093
6291995036,9cyclictest451rcuc/319:34:423
629199503,33cyclictest22777-21chrt21:24:263
629199503,31cyclictest4624-21chrt00:19:303
629199502,38cyclictest5924-21chrt20:45:163
629199502,37cyclictest1456-21taskset23:23:023
629199502,35cyclictest11836-21chrt21:04:523
629199502,35cyclictest11836-21chrt21:04:523
629199502,33cyclictest29891-21chrt21:35:163
629199494,32cyclictest26454-21chrt20:09:573
629199494,32cyclictest26454-21chrt20:09:573
629199493,32cyclictest21655-21chrt19:55:153
629199492,34cyclictest5981-21chrt23:30:153
629199483,34cyclictest31397-21chrt20:24:593
629199473,35cyclictest2644-21taskset23:25:083
629199473,34cyclictest9351-21chrt20:55:213
629199473,34cyclictest9351-21chrt20:55:213
6291994723,16cyclictest166702sleep319:40:093
6291994634,8cyclictest451rcuc/320:34:363
629199454,30cyclictest4108-21chrt20:40:103
629199453,31cyclictest24813-21taskset22:17:553
629199453,30cyclictest16439-21chrt21:14:283
629199443,30cyclictest13501-21chrt21:09:543
6291994331,8cyclictest451rcuc/319:21:063
6291994220,15cyclictest451rcuc/300:21:053
6291994025,10cyclictest451rcuc/320:26:593
6291994025,10cyclictest451rcuc/320:26:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional