You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 16:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Apr 20, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224069912229,82cyclictest0-21swapper/009:48:160
224069911832,11cyclictest0-21swapper/008:13:160
224069911534,13cyclictest0-21swapper/010:18:140
22406991146,27cyclictest26293-21timerandwakeup11:38:240
21131211112,63sleep121207-21tune2fs06:58:151
224069911041,14cyclictest2481-21ntp_kernel_pll_11:53:170
224069911041,14cyclictest2481-21ntp_kernel_pll_11:53:170
22406991094,35cyclictest14891-21latency_hist09:32:500
22406991093,37cyclictest24004-21cmp09:47:500
22406991093,28cyclictest16642-21/usr/sbin/munin08:23:250
224069910818,25cyclictest16100-21modprobe11:22:470
224069910734,16cyclictest0-21swapper/010:28:250
22406991072,23cyclictest11403-21grep12:08:100
224069910653,46cyclictest2374-21systemd-journal12:07:480
224069910633,11cyclictest0-21swapper/007:38:200
22406991054,33cyclictest26126-21latency_hist07:12:510
22406991054,33cyclictest26126-21latency_hist07:12:510
224069910435,13cyclictest1-21systemd11:02:480
224069910434,9cyclictest0-21swapper/007:32:510
22406991034,39cyclictest6708-21sh11:59:450
22406991033,37cyclictest22114-21ssh11:32:470
224069910248,47cyclictest8025-21ssh09:20:320
22406991023,38cyclictest27822-21latency_hist08:57:500
22406991023,38cyclictest13263-21ssh10:23:080
224069910233,10cyclictest0-21swapper/009:07:510
22339210218,61sleep022285-21/usr/sbin/munin07:02:510
22407991003,76cyclictest24313-21apt-get09:48:161
22406991003,41cyclictest3655-21grep09:13:150
22406991003,36cyclictest4194-21wc07:43:470
22406991003,36cyclictest4194-21wc07:43:470
22406991002,35cyclictest4646-21grep10:08:160
22406991002,35cyclictest30871-21latency_hist10:52:510
22406999937,10cyclictest18745-21ssh09:38:220
2240699993,39cyclictest19235-21latency_hist11:27:500
2240699993,27cyclictest6245-21tail07:48:250
2240699992,39cyclictest1691-21sh10:03:190
2240699992,39cyclictest1691-21sh10:03:190
22407999848,44cyclictest4420-21kworker/u17:3+events_unbound11:52:471
2240699983,38cyclictest352-21runrttasks11:51:390
2240699983,36cyclictest8213-21ssh11:08:220
2240699983,36cyclictest8213-21ssh11:08:220
2240699983,31cyclictest27525-21/usr/sbin/munin09:53:510
2240699982,37cyclictest16653-21rm12:17:300
2240699982,35cyclictest18248-21/usr/sbin/munin08:28:210
2240699982,25cyclictest9746-21ntp_kernel_pll_09:23:170
2240699982,25cyclictest9746-21ntp_kernel_pll_09:23:170
22406999737,8cyclictest0-21swapper/012:27:590
22406999737,8cyclictest0-21swapper/012:27:590
2240699973,35cyclictest23941-21latency_hist07:07:500
2240699973,21cyclictest28988-21/usr/sbin/munin11:43:170
2240699972,47cyclictest24800-21latency_hist10:42:500
2240699972,47cyclictest24800-21latency_hist10:42:500
2240699972,39cyclictest4462-21/usr/sbin/munin11:03:110
2240699972,39cyclictest4462-21/usr/sbin/munin11:03:110
2240699972,37cyclictest28957-21unixbench_singl08:58:250
2240699972,37cyclictest28957-21unixbench_singl08:58:250
2240699963,37cyclictest9293-21cat07:58:230
2240699962,40cyclictest29700-21fschecks_count07:23:100
2240699962,38cyclictest352-21runrttasks09:12:100
2240699943,35cyclictest25871-21ssh10:43:230
2240699943,34cyclictest24547-21latency_hist08:47:490
2240699943,34cyclictest24547-21latency_hist08:47:490
2240699942,39cyclictest19903-21/usr/sbin/munin10:33:230
2240799934,47cyclictest29332-21diskmemload11:58:131
22407999337,49cyclictest29332-21diskmemload11:12:471
22407999337,49cyclictest29332-21diskmemload11:12:471
2240699932,35cyclictest31334-21sh09:58:230
2240799903,25cyclictest3733-21sshd07:42:591
2240799903,25cyclictest3733-21sshd07:42:591
2240699903,40cyclictest17793-21modprobe09:37:470
22407998938,44cyclictest30298-21sh09:03:201
2240799893,31cyclictest23362-21rm12:28:111
2240799893,31cyclictest23362-21rm12:28:111
22407998850,32cyclictest16163-21ssh12:16:061
2240799884,31cyclictest1063-21apt-get10:02:591
2240799884,31cyclictest1063-21apt-get10:02:591
22407998833,48cyclictest21393-21latency_hist08:37:511
2240799882,30cyclictest25777-21ssh10:43:221
22408998766,14cyclictest300-21rs:main2
22407998763,14cyclictest12672-21sh11:16:311
22407998721,23cyclictest16416-21cat11:22:571
22408998644,13cyclictest181rcu_preempt09:17:512
22407998662,18cyclictest10937-21latency_hist12:07:501
22407998658,21cyclictest6879-21latency_hist10:12:501
2240799862,44cyclictest20083-21grep10:33:251
22408998561,16cyclictest352-21runrttasks07:23:112
2240799854,31cyclictest6884-21processes09:18:211
22407998540,38cyclictest0-21swapper/107:38:161
2240799853,41cyclictest31613-21unixbench_singl09:58:261
2240799853,31cyclictest15988-21/usr/sbin/munin10:28:171
2240799853,24cyclictest352-21runrttasks09:57:251
22407998531,46cyclictest0-21swapper/111:47:491
22408998465,13cyclictest2374-21systemd-journal10:18:592
22408998465,11cyclictest0-21swapper/209:28:232
22407998460,17cyclictest4878-21apt-key11:57:511
22407998460,17cyclictest4878-21apt-key11:57:511
22407998438,38cyclictest0-21swapper/108:42:511
2130528411,41sleep20-21swapper/206:58:202
22408998363,13cyclictest14115-21proc_pri11:18:192
22407998355,20cyclictest13723-21fschecks_time08:13:101
22407998334,40cyclictest0-21swapper/110:58:171
2240799833,32cyclictest20508-21kernelversion12:23:131
2240799833,29cyclictest5535-21grep11:03:251
2240799833,29cyclictest5535-21grep11:03:251
2240799833,29cyclictest13352-21ssh10:23:091
22407998332,44cyclictest0-21swapper/112:18:021
22407998330,46cyclictest7079-21cstates07:53:051
22407998330,46cyclictest0-21swapper/109:28:171
22407998310,31cyclictest29266-21cat07:22:491
22408998251,14cyclictest181rcu_preempt11:03:232
22408998251,14cyclictest181rcu_preempt11:03:232
22407998234,41cyclictest0-21swapper/110:38:591
22407998234,41cyclictest0-21swapper/110:38:591
2240799823,37cyclictest9275-21cstates09:23:051
2240799823,37cyclictest9275-21cstates09:23:051
2240799823,31cyclictest3847-21open_files09:13:211
2240799823,27cyclictest8572-21/usr/sbin/munin07:58:221
2240799822,34cyclictest28145-21ssh11:42:491
2240799822,30cyclictest15426-21fschecks_time09:33:101
2240799822,27cyclictest24107-21apt-config09:47:511
2240999812,34cyclictest4783-21chrt11:57:493
2240999812,34cyclictest4783-21chrt11:57:493
2240899818,61cyclictest5119-21apt-get11:58:012
22408998160,13cyclictest277-21dbus-daemon11:57:482
22408998160,13cyclictest277-21dbus-daemon11:57:482
22408998156,17cyclictest22157-21sed09:43:252
22408998148,13cyclictest181rcu_preempt08:37:592
22408998130,14cyclictest181rcu_preempt11:10:432
22408998130,14cyclictest181rcu_preempt11:10:432
2240799813,39cyclictest11527-21uname12:08:131
2240799813,30cyclictest31402-21latency10:53:101
2240799813,27cyclictest24727-21cpuspeed07:08:071
2240799813,27cyclictest24727-21cpuspeed07:08:071
2240799812,29cyclictest28924-21head08:58:251
2240799812,29cyclictest28924-21head08:58:251
2240799812,28cyclictest27879-21latency_hist08:57:501
2240799812,28cyclictest13417-21/usr/sbin/munin11:18:201
22407998117,27cyclictest1708-21diskmemload09:09:371
2240999803,30cyclictest30525-21chrt09:57:593
22408998060,12cyclictest0-21swapper/211:37:502
22408998060,12cyclictest0-21swapper/211:37:502
22408998037,8cyclictest181rcu_preempt07:08:062
22408998037,8cyclictest181rcu_preempt07:08:062
2240799803,28cyclictest352-21runrttasks08:08:401
2240799803,28cyclictest352-21runrttasks08:08:401
2240799803,24cyclictest902-21/usr/sbin/munin07:33:221
22407998025,47cyclictest0-21swapper/110:18:211
22407998010,28cyclictest6258-21unixbench_multi07:48:251
2231928019,41sleep30-21swapper/307:02:503
2240799798,24cyclictest18308-21diskmemload09:38:101
2240799794,27cyclictest25347-21/usr/sbin/munin08:48:201
2240799794,23cyclictest24142-21latency_hist07:07:511
2240799793,30cyclictest2374-21systemd-journal11:28:581
2240799793,27cyclictest352-21runrttasks07:29:121
2240799793,24cyclictest22389-21/usr/sbin/munin11:33:201
2240799793,24cyclictest22389-21/usr/sbin/munin11:33:201
2240799792,34cyclictest30778-21latency_hist10:52:501
2240999783,63cyclictest24377-21taskset09:48:103
22408997863,11cyclictest22404-21cyclictest12:18:222
22408997861,12cyclictest58-21kcompactd011:02:132
22408997815,52cyclictest27548-21apt-get09:53:062
2240799784,29cyclictest7841-21/usr/sbin/munin10:13:211
2240799784,29cyclictest7841-21/usr/sbin/munin10:13:211
2240799784,27cyclictest11005-21timerandwakeup08:03:221
2240799783,30cyclictest26462-21fschecks_count07:13:091
2240799783,30cyclictest18371-21cstates08:28:061
2240799783,29cyclictest17154-21ntp_kernel_err08:23:151
2240799783,28cyclictest31040-21latency_hist07:27:511
22408997763,10cyclictest22404-21cyclictest07:36:472
22408997759,11cyclictest1042-21/usr/sbin/munin10:02:582
22408997759,11cyclictest1042-21/usr/sbin/munin10:02:582
22408997755,16cyclictest22404-21cyclictest10:38:092
22408997755,16cyclictest22404-21cyclictest10:38:092
22408997751,9cyclictest181rcu_preempt10:13:152
22408997751,9cyclictest181rcu_preempt10:13:152
2240999762,31cyclictest7941-21chrt10:13:233
2240999762,31cyclictest7941-21chrt10:13:233
2240899767,44cyclictest39-21ksoftirqd/210:37:502
22408997660,10cyclictest20666-21ntp_offset12:23:172
22408997654,15cyclictest13263-21ssh10:23:082
22408997647,11cyclictest181rcu_preempt07:18:172
22408997638,8cyclictest181rcu_preempt08:43:232
22408997638,8cyclictest181rcu_preempt08:43:232
22408997633,13cyclictest181rcu_preempt11:42:482
2240799763,25cyclictest24470-21idleruntime-cro08:47:471
2240799763,25cyclictest24470-21idleruntime-cro08:47:471
2240799762,30cyclictest15722-21mailstats08:18:211
22408997559,10cyclictest3231-21apt-get07:43:022
22408997559,10cyclictest3231-21apt-get07:43:022
22408997536,15cyclictest181rcu_preempt07:53:102
22408997527,12cyclictest181rcu_preempt11:43:232
2240999742,36cyclictest17952-21chrt10:30:493
22408997456,12cyclictest352-21runrttasks10:28:062
22408997455,13cyclictest22404-21cyclictest07:58:222
22408997454,10cyclictest0-21swapper/210:00:302
22408997446,11cyclictest181rcu_preempt09:38:422
22408997445,12cyclictest181rcu_preempt11:52:142
22408997440,14cyclictest8405-21ssh12:03:102
2240999732,27cyclictest16838-21chrt09:35:093
22408997357,10cyclictest15643-21processes08:18:202
22408997355,12cyclictest25806-21switchtime10:43:232
22408997349,8cyclictest5283-21ssh10:08:482
22408997348,9cyclictest181rcu_preempt11:26:222
2240899734,56cyclictest9176-21ssh09:23:032
2240899734,56cyclictest9176-21ssh09:23:032
22408997344,11cyclictest181rcu_preempt07:02:592
22406997357,10cyclictest297-21systemd-logind08:42:480
22408997247,11cyclictest181rcu_preempt08:10:502
22408997247,11cyclictest181rcu_preempt08:10:502
22408997243,13cyclictest181rcu_preempt10:52:222
22406997253,13cyclictest23399-21diskmemload11:33:240
22406997253,13cyclictest23399-21diskmemload11:33:240
2240999712,30cyclictest6928-21chrt07:52:513
2240999712,27cyclictest18109-21taskset09:38:003
22408997153,11cyclictest26620-21grep08:53:072
22408997151,14cyclictest22404-21cyclictest09:52:492
22408997147,11cyclictest181rcu_preempt07:39:352
22408997136,12cyclictest181rcu_preempt09:06:282
22408997126,12cyclictest181rcu_preempt10:53:352
2240999703,27cyclictest27704-21chrt08:57:473
22408997055,10cyclictest58-21kcompactd008:03:152
22408997043,9cyclictest181rcu_preempt08:28:122
22408997037,9cyclictest181rcu_preempt11:13:102
2240999692,27cyclictest27979-21taskset10:47:593
22408996954,9cyclictest58-21kcompactd009:09:572
22408996941,11cyclictest19402-21apt-get11:27:582
22408996939,11cyclictest181rcu_preempt08:33:222
22408996936,13cyclictest181rcu_preempt12:13:192
2240899692,45cyclictest29513-21wc09:02:512
2240899692,45cyclictest29513-21wc09:02:512
2240899692,30cyclictest25493-21awk08:48:222
22408996920,27cyclictest13547-21cstates08:13:042
22406996954,10cyclictest277-21dbus-daemon12:27:480
22406996945,16cyclictest352-21runrttasks08:08:120
22406996945,16cyclictest352-21runrttasks08:08:120
22406996940,22cyclictest0-21swapper/007:57:500
22406996912,30cyclictest7712-21pluginstate10:13:200
22406996912,30cyclictest7712-21pluginstate10:13:200
45362680,6sleep30-21swapper/307:43:563
45362680,6sleep30-21swapper/307:43:563
22408996847,16cyclictest22404-21cyclictest12:32:522
22408996847,16cyclictest22404-21cyclictest12:32:522
2240899684,42cyclictest352-21runrttasks07:31:582
22408996836,9cyclictest181rcu_preempt09:19:492
22406996842,20cyclictest15477-21kworker/u17:3+nfsiod12:21:550
2240699682,48cyclictest26090-21run-parts08:52:470
22408996747,11cyclictest381ktimers/207:47:592
22408996738,11cyclictest181rcu_preempt12:11:562
2240899673,47cyclictest26484-21fschecks_count07:13:092
22406996728,28cyclictest16403-21modprobe08:22:470
2240699672,32cyclictest26457-21fschecks_count07:13:090
37402660,7sleep30-21swapper/311:01:373
22408996635,8cyclictest181rcu_preempt08:23:132
2240699667,17cyclictest15-21ksoftirqd/007:18:140
22406996652,8cyclictest32637-21ssh10:54:590
2240999653,22cyclictest3413-21chrt07:42:503
22409996517,25cyclictest0-21swapper/311:58:093
22406996545,14cyclictest19914-21cpu08:33:050
22406996446,13cyclictest352-21runrttasks11:13:270
2240999634,26cyclictest11957-21chrt12:08:233
2240699633,43cyclictest11536-21latency_hist08:07:500
2240999593,40cyclictest22687-21chrt11:33:103
2240999593,40cyclictest22687-21chrt11:33:103
2240999583,41cyclictest5435-21chrt10:09:133
2240999583,41cyclictest13238-21taskset10:23:073
2240699572,34cyclictest1170-21grep07:33:250
2240999562,41cyclictest10153-21taskset10:18:053
2240999543,36cyclictest10675-21chrt11:13:083
2240999543,34cyclictest6350-21chrt09:18:063
2240999533,38cyclictest26138-21chrt08:52:493
2240999533,32cyclictest19414-21taskset11:27:593
2240999532,35cyclictest14893-21chrt08:17:503
2240999532,34cyclictest30182-21chrt07:23:223
2240999532,34cyclictest26031-21chrt12:32:513
2240999532,34cyclictest26031-21chrt12:32:513
2240999523,33cyclictest17327-21chrt08:23:213
2240999522,37cyclictest18446-21taskset08:28:113
2240999522,35cyclictest23535-21chrt10:39:373
2240999522,35cyclictest23535-21chrt10:39:373
2240999522,32cyclictest22273-21taskset08:38:233
2240999522,32cyclictest1549-21chrt10:03:143
2240999522,32cyclictest1549-21chrt10:03:143
2240999514,36cyclictest7827-21chrt07:53:243
2240999514,33cyclictest28834-21chrt07:18:243
2240999513,35cyclictest25454-21chrt10:43:133
2240999512,34cyclictest10983-21chrt12:07:503
22409995021,20cyclictest10948-21chrt09:25:243
22409995021,20cyclictest10948-21chrt09:25:243
22409995021,13cyclictest0-21swapper/311:03:213
22409995021,13cyclictest0-21swapper/311:03:213
2240999493,34cyclictest29077-21chrt09:54:343
2240999493,33cyclictest23227-21chrt08:43:053
2240999493,33cyclictest23227-21chrt08:43:053
2240999493,33cyclictest22598-21chrt09:44:333
2240999493,31cyclictest14880-21chrt11:19:203
2240999492,36cyclictest25624-21chrt07:08:283
2240999492,36cyclictest25624-21chrt07:08:283
2240999492,33cyclictest8766-21chrt11:08:593
2240999492,33cyclictest8766-21chrt11:08:593
2240999482,29cyclictest19288-21chrt11:27:513
2240999472,32cyclictest32180-21chrt10:53:433
2240999472,31cyclictest10150-21chrt08:02:513
2240999464,29cyclictest16420-21chrt08:22:483
2240999462,31cyclictest31588-21taskset09:05:453
2240999453,32cyclictest30944-21chrt11:47:323
2240999453,29cyclictest17164-21taskset12:18:043
2240999452,32cyclictest12005-21chrt08:08:103
2240999452,32cyclictest12005-21chrt08:08:103
2240999452,31cyclictest26413-21chrt11:38:253
2240999443,28cyclictest570-21chrt09:08:133
2240999443,28cyclictest28311-21chrt08:58:123
2240999443,28cyclictest28311-21chrt08:58:123
2240999443,28cyclictest14109-21taskset12:12:573
2240999443,28cyclictest12375-21chrt09:28:113
2240999432,30cyclictest24145-21taskset07:07:513
2240999432,30cyclictest19019-21taskset10:33:003
2240999432,29cyclictest737-21chrt11:50:283
2240999432,28cyclictest3224-21taskset09:13:033
2240999423,27cyclictest657-21taskset07:33:153
2240999423,27cyclictest21374-21taskset08:37:503
2240999423,27cyclictest20040-21taskset12:22:583
2240999412,27cyclictest26542-21taskset07:13:103
2240999392,26cyclictest31104-21chrt07:28:003
2240999372,26cyclictest10358-21taskset08:03:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional