You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 19:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Fri Mar 06, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2632121709,49sleep226405-21grep07:02:252
2535121695,19sleep00-21swapper/007:01:000
268769910553,43cyclictest17400-21latency_hist10:25:492
26026210118,47sleep10-21swapper/107:02:191
2687699964,8cyclictest0-21swapper/210:16:102
2687699964,8cyclictest0-21swapper/210:16:102
26876999640,47cyclictest0-21swapper/207:36:222
26876999633,12cyclictest0-21swapper/209:41:332
26876999432,57cyclictest0-21swapper/210:26:062
26866999477,11cyclictest22448-21systemd-run08:30:480
26876999343,42cyclictest0-21swapper/211:51:102
26876999343,42cyclictest0-21swapper/211:51:102
26876999267,15cyclictest18175-21cron11:20:462
2687699923,79cyclictest13006-21apt-get08:01:052
26876999237,47cyclictest0-21swapper/209:15:502
26876999237,47cyclictest0-21swapper/209:15:502
26876999236,48cyclictest26566-21netstat09:46:142
2687699923,35cyclictest14370-21fschecks_time09:26:102
2687699923,33cyclictest6210-21ssh10:06:192
2687699923,33cyclictest6210-21ssh10:06:192
26876999233,10cyclictest0-21swapper/207:46:232
2687699922,30cyclictest18444-21grep08:16:192
2687699922,30cyclictest18444-21grep08:16:192
26876999154,16cyclictest26857-21cyclictest07:15:512
2687699913,32cyclictest18814-21latency_hist12:15:502
2687699913,32cyclictest18814-21latency_hist12:15:502
26876999127,12cyclictest0-21swapper/211:36:122
2687699912,33cyclictest2072-21grep10:52:432
2687699912,33cyclictest2072-21grep10:52:432
26876999122,14cyclictest0-21swapper/211:15:502
26876999039,45cyclictest0-21swapper/211:46:092
26876999034,7cyclictest0-21swapper/211:31:232
2687699903,39cyclictest20147-21cpuspeed09:36:042
26876999033,9cyclictest0-21swapper/210:46:122
2687699903,33cyclictest6519-21/usr/sbin/munin11:01:112
2687699903,27cyclictest11314-21fschecks_count09:21:092
2687699903,27cyclictest11314-21fschecks_count09:21:092
26876998947,36cyclictest23190-21ntp_kernel_pll_08:31:152
2687699893,38cyclictest25655-21modprobe08:40:472
2687699893,36cyclictest29004-21/usr/sbin/munin09:51:222
2687699893,36cyclictest29004-21/usr/sbin/munin09:51:222
26876998930,14cyclictest26521-21latency_hist10:40:492
26876998929,8cyclictest0-21swapper/212:31:142
26876998928,8cyclictest0-21swapper/209:31:242
26876998926,9cyclictest0-21swapper/211:30:472
2687699892,35cyclictest7214-21sort11:56:092
2687699892,34cyclictest27602-21runrttasks12:30:472
2687699892,34cyclictest27602-21runrttasks12:30:472
2687699892,33cyclictest5419-21latency_hist10:05:502
2687699892,31cyclictest2536-21cut09:06:142
26876998918,13cyclictest0-21swapper/208:55:512
2687699883,35cyclictest29587-21fschecks_count08:51:082
2687699883,34cyclictest25910-21/usr/sbin/munin08:41:222
2687699883,34cyclictest14855-21fschecks_time08:06:092
2687699883,32cyclictest20876-21latency_hist08:25:492
2687699883,32cyclictest20876-21latency_hist08:25:492
2687699883,27cyclictest29655-21latency_hist10:45:502
26876998829,12cyclictest0-21swapper/212:00:572
26876998829,12cyclictest0-21swapper/212:00:572
26876998823,12cyclictest0-21swapper/207:31:052
2687699882,30cyclictest28039-21fschecks_count08:46:472
26876998815,14cyclictest0-21swapper/212:21:112
26876998761,17cyclictest32693-21latency07:21:122
26876998761,17cyclictest32693-21latency07:21:122
2687699873,32cyclictest352-21runrttasks09:58:422
2687699873,32cyclictest18780-21fschecks_time11:21:092
2687699873,32cyclictest11031-21latency_hist07:55:512
2687699872,33cyclictest7741-21latency_hist07:45:502
26869998765,16cyclictest544-21cron07:20:481
2687699864,37cyclictest12921-21/usr/sbin/munin12:06:202
2687699863,31cyclictest27132-21cstates07:06:052
2687699863,26cyclictest9225-21processes10:11:192
2687699863,26cyclictest9225-21processes10:11:192
26876998629,10cyclictest0-21swapper/209:16:122
26866998664,15cyclictest1773-21diskmemload09:55:510
26866998664,15cyclictest1773-21diskmemload09:55:510
2687699853,35cyclictest30808-21cstates07:16:042
2687699853,34cyclictest2815-21apt07:30:472
2687699853,34cyclictest2815-21apt07:30:472
2687699853,31cyclictest18939-21/usr/sbin/munin12:15:572
26876998525,12cyclictest0-21swapper/210:56:252
26876998520,12cyclictest0-21swapper/208:26:142
26869998565,14cyclictest26857-21cyclictest10:27:021
26866998563,14cyclictest3638-21unixbench_singl10:01:250
2687699842,34cyclictest11912-21sshd11:10:022
2687699842,31cyclictest22144-21unixbench_multi10:32:232
26866998469,8cyclictest0-21swapper/009:05:470
2687699839,23cyclictest30446-21sort11:40:572
2687699839,23cyclictest30446-21sort11:40:572
2687699833,35cyclictest1101-21sendmail_mailqu09:02:132
2687699833,32cyclictest16321-21cstates08:11:042
26869998358,16cyclictest28766-21unixbench_singl08:47:021
2686699834,44cyclictest15-21ksoftirqd/011:37:420
26866998217,41cyclictest15-21ksoftirqd/011:42:030
26866998217,41cyclictest15-21ksoftirqd/011:42:030
169232820,10sleep116928-21sed12:11:231
169232820,10sleep116928-21sed12:11:231
2687699812,27cyclictest11360-21fschecks_count07:56:102
26869998160,14cyclictest1773-21diskmemload11:48:321
26869998045,8cyclictest181rcu_preempt12:16:011
2686699808,50cyclictest15-21ksoftirqd/008:50:580
26866998043,29cyclictest0-21swapper/010:00:500
26869997960,15cyclictest26857-21cyclictest12:09:011
26869997958,14cyclictest352-21runrttasks11:11:121
2686999795,56cyclictest540-21apt-get10:51:081
2686999795,56cyclictest540-21apt-get10:51:081
26866997958,15cyclictest21312-21latency_hist11:25:490
2686699794,60cyclictest17800-21ssh10:26:090
26866997831,28cyclictest15-21ksoftirqd/007:56:100
26869997758,12cyclictest9876-21kworker/u17:2-rpciod10:15:471
26869997758,12cyclictest9876-21kworker/u17:2-rpciod10:15:471
26869997756,15cyclictest2221-21threads07:26:231
26869997756,15cyclictest2221-21threads07:26:231
26869997754,14cyclictest352-21runrttasks08:31:241
2686699776,37cyclictest24606-21modprobe12:25:470
26866997757,14cyclictest27483-21latency_hist08:45:500
26866997756,14cyclictest26518-21ssh10:40:490
2686699773,42cyclictest352-21runrttasks09:17:210
26866997729,17cyclictest181rcu_preempt12:19:360
2686699772,47cyclictest15-21ksoftirqd/007:21:190
2686699772,47cyclictest15-21ksoftirqd/007:21:190
26869997657,13cyclictest19676-21snmpd09:26:261
26866997658,12cyclictest30683-21latency_hist12:35:490
26866997656,13cyclictest32644-21kworker/u18:1-rpciod10:41:420
26866997654,16cyclictest24442-21/usr/sbin/munin11:31:110
26866997651,10cyclictest181rcu_preempt10:58:150
2686699764,40cyclictest24363-21latency_hist11:30:500
2686699763,35cyclictest15504-21/usr/sbin/munin11:16:070
26866997621,30cyclictest15-21ksoftirqd/010:35:350
26866997618,37cyclictest15-21ksoftirqd/010:46:010
304412750,5sleep30-21swapper/309:52:173
304412750,5sleep30-21swapper/309:52:173
26869997553,12cyclictest0-21swapper/112:26:241
26869997553,12cyclictest0-21swapper/112:26:241
26869997552,15cyclictest26857-21cyclictest08:16:041
26869997552,15cyclictest26857-21cyclictest08:16:041
2686699758,36cyclictest352-21runrttasks11:05:420
26866997541,14cyclictest181rcu_preempt10:51:590
26866997541,14cyclictest181rcu_preempt10:51:590
2686699753,40cyclictest32677-21dpkg09:00:500
26866997533,11cyclictest181rcu_preempt11:57:160
26869997458,10cyclictest300-21rs:main1
26869997458,10cyclictest300-21rs:main1
26869997442,8cyclictest181rcu_preempt07:11:171
26869997432,8cyclictest181rcu_preempt11:56:201
26866997457,10cyclictest501-21kworker/u17:1+rpciod11:06:150
26866997452,15cyclictest28623-21munin-node07:15:490
26866997444,21cyclictest32043-21modprobe07:20:480
2686699744,40cyclictest15-21ksoftirqd/009:41:210
2686699744,38cyclictest20189-21cstates09:36:050
26866997441,23cyclictest277-21dbus-daemon09:50:490
2686699743,32cyclictest18083-21unixbench_multi09:31:240
2686699742,41cyclictest4526-21netstat11:51:150
2686699742,41cyclictest4526-21netstat11:51:150
26869997353,14cyclictest17827-21latency_hist08:15:511
26869997353,12cyclictest6271-21ssh11:00:491
26869997352,15cyclictest3407-21sh09:07:021
2686999734,38cyclictest19469-21apt-get08:21:021
2686999734,38cyclictest19469-21apt-get08:21:021
26869997337,8cyclictest181rcu_preempt10:24:021
26869997329,9cyclictest0-21swapper/107:21:091
26869997329,9cyclictest0-21swapper/107:21:091
26866997336,13cyclictest0-21swapper/012:30:500
26866997336,13cyclictest0-21swapper/012:30:500
2686699733,56cyclictest14835-21fschecks_count08:06:090
26869997254,12cyclictest28358-21cron07:10:471
26869997251,11cyclictest26599-21ssh10:40:491
2686699725,34cyclictest14226-21latency_hist10:20:500
2686699725,34cyclictest14226-21latency_hist10:20:500
26866997252,13cyclictest2057-21unixbench_singl11:46:240
26866997246,11cyclictest181rcu_preempt09:11:180
26866997246,11cyclictest181rcu_preempt09:11:180
26866997238,12cyclictest181rcu_preempt08:05:500
2686699723,33cyclictest5622-21grep07:36:220
26866997231,16cyclictest0-21swapper/010:06:120
26866997231,16cyclictest0-21swapper/010:06:120
26869997149,16cyclictest27340-21netstat10:41:141
26869997142,19cyclictest20577-21kworker/u17:0+events_unbound08:40:471
26869997127,34cyclictest31747-21cat08:56:231
26866997149,15cyclictest18723-21latency_hist12:15:490
26866997149,15cyclictest18723-21latency_hist12:15:490
2686699714,28cyclictest15124-21/usr/sbin/munin09:26:260
26866997141,8cyclictest181rcu_preempt11:13:230
2686699713,54cyclictest28366-21modprobe07:10:470
2688199703,35cyclictest0-21swapper/310:51:493
2688199703,35cyclictest0-21swapper/310:51:493
26869997054,10cyclictest22034-21/usr/sbin/munin11:26:141
26869997053,9cyclictest0-21swapper/110:20:501
26869997053,9cyclictest0-21swapper/110:20:501
26869997050,14cyclictest24834-21latency_hist12:25:511
26869997050,14cyclictest13132-21cstates08:01:211
26869997046,10cyclictest181rcu_preempt07:36:131
26866997054,9cyclictest0-21swapper/010:25:470
2686699703,36cyclictest15-21ksoftirqd/008:36:110
2686699703,34cyclictest2073-21/usr/sbin/munin09:06:280
26869996953,10cyclictest18182-21ssh11:20:461
26869996948,15cyclictest352-21runrttasks07:51:221
2686699697,45cyclictest15-21ksoftirqd/012:07:050
2686699695,45cyclictest24266-21latency_hist08:35:510
26866996943,11cyclictest181rcu_preempt08:15:510
26866996941,12cyclictest181rcu_preempt07:35:500
26866996931,23cyclictest181rcu_preempt07:26:130
26866996931,23cyclictest181rcu_preempt07:26:130
254082696,16sleep30-21swapper/307:01:243
26869996851,11cyclictest277-21dbus-daemon08:45:491
26869996846,9cyclictest31-21ksoftirqd/111:38:101
26869996833,10cyclictest181rcu_preempt11:41:041
26869996833,10cyclictest181rcu_preempt11:41:041
2686999683,22cyclictest3402-21switchtime10:01:221
2686999682,40cyclictest9210-21latency_hist11:05:501
26869996823,11cyclictest5782-21uniq10:06:081
26869996823,11cyclictest5782-21uniq10:06:081
26866996843,10cyclictest181rcu_preempt07:41:060
2686699683,34cyclictest11300-21ssh09:21:090
2686699683,34cyclictest11300-21ssh09:21:090
85072670,4sleep18503-21ssh09:16:111
26869996750,11cyclictest4978-21ssh11:51:241
26869996750,11cyclictest4978-21ssh11:51:241
26869996749,13cyclictest10175-21open_files11:06:191
26869996722,7cyclictest181rcu_preempt10:32:541
2686699674,35cyclictest8835-21grep07:46:230
2686699673,42cyclictest9507-21/usr/sbin/munin07:51:080
26866996729,12cyclictest181rcu_preempt08:16:230
26866996729,12cyclictest181rcu_preempt08:16:230
2686699672,46cyclictest19451-21/usr/sbin/munin08:21:120
2686699672,46cyclictest19451-21/usr/sbin/munin08:21:120
26866996718,40cyclictest27301-21gpgv08:45:520
105932673,11sleep010612-21ssh12:01:180
105932673,11sleep010612-21ssh12:01:180
2686699664,32cyclictest352-21runrttasks10:13:350
2686699664,32cyclictest352-21runrttasks10:13:350
26869996538,13cyclictest0-21swapper/107:46:091
26869996536,22cyclictest17663-21sh09:31:181
2686999653,43cyclictest29782-21/usr/sbin/munin10:46:221
26869996531,9cyclictest181rcu_preempt07:56:201
2686999647,8cyclictest21079-21apt-get08:26:041
26869996450,10cyclictest26857-21cyclictest09:50:511
26869996450,10cyclictest26857-21cyclictest09:50:511
26869996439,19cyclictest0-21swapper/109:46:041
26869996425,9cyclictest181rcu_preempt12:05:381
26869996425,9cyclictest181rcu_preempt12:05:381
2686999642,42cyclictest28457-21sed12:31:141
26869996336,21cyclictest0-21swapper/109:21:161
26869996336,21cyclictest0-21swapper/109:21:161
26869996310,26cyclictest31-21ksoftirqd/108:09:211
26869996238,15cyclictest0-21swapper/111:35:501
26869996237,18cyclictest19112-21kworker/u18:2+nfsiod09:39:241
26869996227,10cyclictest181rcu_preempt09:55:581
2686999622,33cyclictest3232-21/usr/sbin/munin07:31:071
2688199613,37cyclictest15213-21taskset09:26:313
26869996144,13cyclictest17206-21kworker/u17:2+events_unbound11:25:471
26869996142,10cyclictest0-21swapper/107:41:241
26869996131,21cyclictest569-21sort09:01:571
26869996041,11cyclictest0-21swapper/108:51:031
2686999603,44cyclictest23353-21ssh09:41:091
2688199582,35cyclictest0-21swapper/310:31:193
2688199574,37cyclictest24449-21chrt10:36:183
2688199574,36cyclictest9801-21chrt11:06:093
2688199563,36cyclictest24214-21chrt08:35:503
2688199563,35cyclictest20100-21taskset09:36:043
2688199562,32cyclictest0-21swapper/308:01:083
2688199553,39cyclictest29021-21taskset10:44:333
2688199552,40cyclictest30248-21chrt10:46:153
2688199544,37cyclictest8739-21chrt11:58:123
2688199543,38cyclictest24421-21chrt12:25:193
2688199542,40cyclictest12471-21taskset12:05:213
2688199542,40cyclictest12471-21taskset12:05:213
2688199542,35cyclictest27185-21taskset12:29:413
2688199542,35cyclictest27185-21taskset12:29:413
2688199533,38cyclictest2664-21chrt09:06:163
2688199533,37cyclictest9672-21chrt09:17:433
2688199533,36cyclictest10629-21taskset10:14:093
2688199533,36cyclictest10629-21taskset10:14:093
2688199533,35cyclictest3179-21chrt10:01:163
2688199532,37cyclictest25628-21chrt11:31:343
2688199532,37cyclictest21805-21chrt08:26:233
2688199532,36cyclictest1152-21chrt11:46:043
2688199532,34cyclictest4876-21taskset11:51:233
2688199532,34cyclictest4876-21taskset11:51:233
2688199532,34cyclictest27354-21taskset07:06:283
2688199532,34cyclictest13066-21chrt11:11:153
2688199522,38cyclictest15153-21chrt12:09:313
2688199522,36cyclictest28900-21chrt12:31:233
2688199522,35cyclictest5353-21chrt09:11:113
2688199522,35cyclictest5353-21chrt09:11:113
2688199522,35cyclictest12167-21chrt10:16:163
2688199522,35cyclictest12167-21chrt10:16:163
2688199522,34cyclictest2943-21chrt07:30:503
2688199522,34cyclictest2943-21chrt07:30:503
2688199522,32cyclictest0-21swapper/309:01:073
2688199522,31cyclictest27944-21taskset11:36:123
2688199514,35cyclictest17031-21chrt09:30:573
2688199513,35cyclictest26867-21chrt09:46:213
2688199513,35cyclictest12510-21chrt08:00:473
2688199513,30cyclictest917-21kworker/3:1+events07:46:023
2688199512,37cyclictest3763-21chrt10:56:083
2688199512,37cyclictest10047-21chrt07:51:153
2688199512,35cyclictest17356-21chrt12:12:183
2688199512,35cyclictest17356-21chrt12:12:183
2688199503,34cyclictest6767-21chrt07:41:153
2688199503,34cyclictest18635-21chrt11:21:043
2688199503,30cyclictest32519-21chrt09:00:493
2688199503,30cyclictest18191-21taskset10:26:173
2688199502,35cyclictest30663-21chrt07:15:513
2688199493,33cyclictest23075-21chrt09:41:023
2688199492,32cyclictest30146-21chrt08:51:223
2688199483,34cyclictest20977-21taskset08:25:503
2688199483,34cyclictest20977-21taskset08:25:503
2688199483,33cyclictest467-21chrt09:56:223
2688199483,33cyclictest15414-21chrt08:06:223
2688199482,33cyclictest16277-21chrt10:23:093
2688199482,33cyclictest11494-21chrt09:21:123
2688199482,33cyclictest11494-21chrt09:21:123
2688199473,32cyclictest7438-21taskset11:01:233
2688199473,30cyclictest7423-21chrt10:08:343
2688199473,30cyclictest7423-21chrt10:08:343
2688199472,33cyclictest19358-21chrt12:16:113
2688199463,33cyclictest3862-21chrt07:31:233
2688199462,29cyclictest16399-21chrt08:11:083
2688199453,31cyclictest27569-21chrt08:45:513
2688199443,31cyclictest18036-21chrt11:20:233
2688199443,30cyclictest32277-21chrt11:43:253
2688199443,30cyclictest32277-21chrt11:43:253
2688199442,28cyclictest0-21swapper/308:46:053
2688199423,29cyclictest25695-21chrt08:40:483
2688199423,27cyclictest18677-21chrt08:16:223
2688199423,27cyclictest18677-21chrt08:16:223
2688199422,28cyclictest22171-21chrt11:26:193
320832415,25sleep30-21swapper/307:20:513
2688199412,27cyclictest32320-21chrt07:21:023
2688199412,27cyclictest32320-21chrt07:21:023
2688199382,25cyclictest5243-21taskset07:36:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional