You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 13:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat Apr 25, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
71749912156,58cyclictest277-21dbus-daemon23:57:300
7174991213,34cyclictest5749-21/usr/sbin/munin00:23:050
7174991113,50cyclictest3280-21ssh22:30:490
7174991113,50cyclictest3280-21ssh22:30:490
7174991113,48cyclictest1944-21/usr/sbin/munin23:22:520
7174991113,48cyclictest1944-21/usr/sbin/munin23:22:520
7174991062,41cyclictest12928-21latency_hist22:47:310
71749910451,12cyclictest19676-21snmpd23:13:060
7174991024,29cyclictest4855-21ntp_states22:33:000
71809910178,15cyclictest2374-21systemd-journal19:27:301
71749910139,10cyclictest0-21swapper/019:47:310
71749910139,10cyclictest0-21swapper/019:47:310
7174991003,31cyclictest15665-21systemctl21:03:020
1622621000,5sleep30-21swapper/321:57:563
717499993,43cyclictest25048-21cut19:52:540
717499983,44cyclictest7287-21ntp_kernel_pll_21:42:590
717499983,44cyclictest7287-21ntp_kernel_pll_21:42:590
717499983,33cyclictest20022-21switchtime22:58:040
717499973,29cyclictest15180-21sshd20:26:570
7180999654,35cyclictest11049-21gpgv19:12:331
7180999652,37cyclictest25001-21sh23:07:321
718399932,44cyclictest39-21ksoftirqd/221:32:442
658029312,54sleep26662-21unixbench_singl18:58:172
7180999039,42cyclictest32711-21latency_hist20:17:331
718099903,73cyclictest19902-21sh23:52:331
718099903,73cyclictest19902-21sh23:52:331
7180998946,35cyclictest0-21swapper/121:37:301
718099894,39cyclictest6110-21grep20:33:051
718099894,39cyclictest6110-21grep20:33:051
718099892,30cyclictest5749-21/usr/sbin/munin00:22:561
7180998741,39cyclictest0-21swapper/119:07:321
7180998740,38cyclictest0-21swapper/100:12:411
718099872,54cyclictest29135-21apt-get20:06:121
718099872,28cyclictest8325-21latency_hist20:42:321
718099872,28cyclictest8325-21latency_hist20:42:321
7183998669,12cyclictest7168-21cyclictest19:28:032
7183998659,20cyclictest25841-21sh00:02:312
7180998661,17cyclictest0-21swapper/119:34:191
7180998661,17cyclictest0-21swapper/119:34:191
718099864,39cyclictest22414-21users19:43:071
718099864,39cyclictest22414-21users19:43:071
718099863,24cyclictest26093-21/usr/sbin/munin00:02:451
7180998631,46cyclictest0-21swapper/119:28:011
7180998625,47cyclictest0-21swapper/100:20:181
718099853,31cyclictest15892-21latency_hist22:52:581
718099852,28cyclictest10372-21cstates22:42:471
7180998519,57cyclictest18678-21/usr/sbin/munin22:02:451
7180998519,57cyclictest18678-21/usr/sbin/munin22:02:451
7183998440,12cyclictest181rcu_preempt23:08:042
7183998440,12cyclictest181rcu_preempt23:08:042
7180998436,40cyclictest0-21swapper/122:57:321
7180998435,10cyclictest181rcu_preempt22:37:541
7180998434,43cyclictest0-21swapper/121:02:301
7180998434,43cyclictest0-21swapper/121:02:301
718099843,35cyclictest12060-21grep20:52:541
718099843,31cyclictest744-21seq20:17:561
718099843,31cyclictest15635-21latency_hist21:57:331
718099843,29cyclictest18606-21latency_hist22:02:331
718099843,27cyclictest2569-21/usr/sbin/munin23:22:571
718099843,27cyclictest2569-21/usr/sbin/munin23:22:571
718099843,26cyclictest352-21runrttasks22:11:071
7180998430,47cyclictest0-21swapper/121:18:041
718099842,31cyclictest30541-21latency_hist22:22:321
718099842,31cyclictest30541-21latency_hist22:22:321
7174998462,16cyclictest31756-21munin-plugin-st00:12:320
7174998459,8cyclictest181rcu_preempt21:51:510
7174998449,26cyclictest0-21swapper/000:02:290
7183998341,12cyclictest181rcu_preempt22:15:202
7180998357,18cyclictest3565-21cpu20:27:451
7180998336,40cyclictest6477-21latency_hist21:42:311
718099833,40cyclictest31817-21if_eth023:17:531
718099833,33cyclictest30104-21grep00:08:071
718099833,31cyclictest8256-21fschecks_count23:32:521
718099833,30cyclictest15706-21ssh21:03:031
718099833,28cyclictest6974-21ssh23:30:381
718099833,17cyclictest1244-21ssh22:27:451
718099833,17cyclictest1244-21ssh22:27:451
7180998329,46cyclictest0-21swapper/121:13:051
7180998318,28cyclictest32260-21sh21:31:351
7180998316,29cyclictest4258-21sh22:32:451
7180998313,31cyclictest9043-21grep00:27:511
7174998351,10cyclictest29048-21kworker/u19:1-events_unbound19:27:450
7183998259,8cyclictest181rcu_preempt00:24:002
7183998234,18cyclictest181rcu_preempt23:42:292
7180998234,40cyclictest0-21swapper/121:43:081
7180998234,40cyclictest0-21swapper/121:43:081
718099823,30cyclictest18457-21/usr/sbin/munin21:07:571
718099823,28cyclictest31351-21ps22:23:031
718099823,27cyclictest23897-21timerwakeupswit19:48:051
718099823,26cyclictest23116-21/usr/sbin/munin23:58:041
718099823,26cyclictest13948-21/usr/sbin/munin23:42:521
718099822,31cyclictest12454-21timerwakeupswit19:14:081
718099822,28cyclictest26003-21runrttasks22:13:391
718099822,27cyclictest9784-21cpuspeed21:47:461
7180998222,51cyclictest14612-21diskmemload23:14:591
718099813,25cyclictest11006-21ssh23:37:451
7180998131,41cyclictest0-21swapper/119:53:131
718099812,31cyclictest9355-21grep20:43:051
718099812,31cyclictest9355-21grep20:43:051
7183998061,13cyclictest7168-21cyclictest23:28:052
7183998057,16cyclictest14612-21diskmemload21:27:512
7183998046,12cyclictest181rcu_preempt00:13:032
7183998014,32cyclictest39-21ksoftirqd/219:17:362
718099804,27cyclictest19192-21/usr/sbin/munin22:58:011
718099803,35cyclictest29749-21fschecks_count20:07:521
718099803,28cyclictest26800-21ntp_kernel_pll_19:57:581
718099802,25cyclictest26093-21grep23:09:031
718099802,25cyclictest26093-21grep23:09:031
717499803,35cyclictest18678-21/usr/sbin/munin22:03:060
717499803,35cyclictest18678-21/usr/sbin/munin22:03:060
718399794,48cyclictest15881-21apt22:52:292
718099793,29cyclictest26914-21/usr/sbin/munin21:22:511
718099793,26cyclictest21356-21latency23:55:001
717499793,39cyclictest25905-21cstates23:08:470
717499793,39cyclictest25905-21cstates23:08:470
719199783,38cyclictest19907-21taskset23:52:333
719199783,38cyclictest19907-21taskset23:52:333
7183997859,13cyclictest18974-21latency_hist22:57:312
7183997858,15cyclictest16078-21kworker/u18:0+events_unbound23:52:292
7183997858,15cyclictest16078-21kworker/u18:0+events_unbound23:52:292
7183997845,9cyclictest181rcu_preempt21:58:042
7183997841,14cyclictest181rcu_preempt19:52:572
7183997828,13cyclictest181rcu_preempt00:02:542
718099783,28cyclictest20732-21grep19:38:061
718099783,28cyclictest20732-21grep19:38:061
718099783,27cyclictest14483-21munin-run19:22:301
718099783,23cyclictest2110-21/usr/sbin/munin20:22:511
7180997832,39cyclictest0-21swapper/120:48:021
7174997831,27cyclictest12373-21ssh23:38:560
7183997762,9cyclictest8709-21apt-get19:07:442
7183997755,15cyclictest19720-21latency_hist19:37:322
7183997755,15cyclictest19720-21latency_hist19:37:322
7183997732,13cyclictest181rcu_preempt22:22:312
7183997732,13cyclictest181rcu_preempt22:22:312
7174997759,12cyclictest19676-21snmpd22:08:060
719199762,34cyclictest30992-21chrt23:17:103
7183997649,22cyclictest7168-21cyclictest22:58:012
718399762,37cyclictest31046-21grep20:12:322
7174997652,16cyclictest5594-21sh23:28:010
7174997650,9cyclictest181rcu_preempt19:02:330
7174997626,29cyclictest17939-21ssh22:01:160
709827611,41sleep17105-21munin-run19:02:321
719199752,49cyclictest0-21swapper/323:08:023
719199752,49cyclictest0-21swapper/323:08:023
7183997548,10cyclictest181rcu_preempt19:42:402
7183997548,10cyclictest181rcu_preempt19:42:402
7183997547,12cyclictest181rcu_preempt22:10:122
718399754,49cyclictest3563-21sh00:18:042
7183997544,22cyclictest11666-21cpu19:13:482
7174997554,15cyclictest6074-21unixbench_multi20:33:040
7174997554,15cyclictest6074-21unixbench_multi20:33:040
717499754,56cyclictest20147-21latency_hist23:52:350
717499754,56cyclictest20147-21latency_hist23:52:350
719199742,32cyclictest32435-21chrt21:32:053
7183997460,10cyclictest7168-21cyclictest21:43:142
7183997460,10cyclictest7168-21cyclictest21:43:142
7183997455,12cyclictest3796-21kworker/u17:4-rpciod21:39:062
7183997453,8cyclictest181rcu_preempt20:56:152
7183997436,13cyclictest181rcu_preempt20:12:452
7183997434,12cyclictest181rcu_preempt23:23:032
7183997434,12cyclictest181rcu_preempt23:23:032
719199733,35cyclictest9956-21chrt22:42:313
7183997351,8cyclictest181rcu_preempt23:02:532
7183997349,8cyclictest181rcu_preempt20:30:112
7183997349,15cyclictest2976-21sh22:29:582
7183997349,15cyclictest2976-21sh22:29:582
7183997349,10cyclictest181rcu_preempt23:36:412
7183997345,8cyclictest181rcu_preempt23:42:542
7183997337,18cyclictest31193-21kworker/u19:2+rpciod22:33:072
7183997335,12cyclictest181rcu_preempt20:48:042
718399733,43cyclictest23219-21fschecks_count19:47:512
7174997346,21cyclictest14947-21fschecks_count19:22:510
7183997256,10cyclictest31833-21readlink00:12:312
7183997249,10cyclictest181rcu_preempt20:42:132
7183997249,10cyclictest181rcu_preempt20:42:132
7183997247,8cyclictest181rcu_preempt20:43:032
7183997247,8cyclictest181rcu_preempt20:43:032
7183997238,13cyclictest181rcu_preempt19:22:572
718399722,40cyclictest1795-21grep23:22:322
7174997255,11cyclictest277-21dbus-daemon19:12:310
7174997252,14cyclictest8163-21timerandwakeup19:03:040
7174997237,28cyclictest0-21swapper/021:28:030
717499723,47cyclictest11524-21apt-get19:12:440
7183997156,9cyclictest17598-21kworker/u17:1-rpciod21:08:142
7183997154,10cyclictest12638-21ssh22:46:512
7183997148,9cyclictest181rcu_preempt20:19:132
7183997130,11cyclictest181rcu_preempt21:02:472
7183997129,13cyclictest181rcu_preempt20:07:302
7183997114,28cyclictest21168-21apt19:42:292
7183997114,28cyclictest21168-21apt19:42:292
7174997154,10cyclictest28769-21latency_hist00:07:310
7174997149,16cyclictest18962-21unixbench_multi21:08:070
7174997145,18cyclictest27748-21latency_hist20:02:320
717499713,37cyclictest22905-21cut23:03:000
7183997054,10cyclictest58-21kcompactd021:18:032
7183997050,14cyclictest27041-21threads19:58:042
7183997044,10cyclictest181rcu_preempt21:49:212
7183997037,13cyclictest181rcu_preempt21:17:312
7183997031,8cyclictest181rcu_preempt23:53:312
7183997029,12cyclictest181rcu_preempt21:52:512
7183997028,13cyclictest181rcu_preempt20:57:512
7183997028,13cyclictest181rcu_preempt20:57:512
7174997050,14cyclictest26613-21idleruntime-cro21:22:300
7174997049,12cyclictest0-21swapper/020:13:010
717499704,56cyclictest3596-21swap00:18:040
717499703,36cyclictest537-21ntpq00:13:150
717499703,23cyclictest30679-21apt-get22:22:460
7183996956,9cyclictest58-21kcompactd022:04:092
7183996956,9cyclictest58-21kcompactd022:04:092
7183996954,10cyclictest58-21kcompactd023:12:402
7183996944,9cyclictest181rcu_preempt21:27:332
7183996944,18cyclictest5411-21fschecks_time20:32:502
7183996944,18cyclictest5411-21fschecks_time20:32:502
7183996924,11cyclictest181rcu_preempt22:22:582
7174996955,8cyclictest0-21swapper/022:42:290
7174996949,15cyclictest19676-21snmpd20:48:040
7174996948,15cyclictest544-21cron22:22:290
7174996948,15cyclictest544-21cron22:22:290
7174996943,19cyclictest15937-21kworker/u17:1+nfsiod23:44:530
717499692,26cyclictest9046-21ssh00:27:510
717499692,25cyclictest8527-21netstat23:32:570
7174996914,36cyclictest13714-21needreboot20:57:550
7174996914,36cyclictest13714-21needreboot20:57:550
719199683,25cyclictest5770-21chrt22:34:203
719199682,28cyclictest32558-21chrt20:17:303
7191996818,40cyclictest272542chrt19:58:063
7174996849,11cyclictest2964-21ssh21:36:590
717499683,52cyclictest23141-21cstates19:47:460
7174996834,25cyclictest1673-21modprobe20:22:290
717499683,28cyclictest4089-21cpu21:38:440
719199672,25cyclictest16857-21taskset21:05:263
7183996745,8cyclictest181rcu_preempt22:40:012
7174996753,8cyclictest23968-21kworker/u17:2+xprtiod21:53:040
7174996749,11cyclictest2374-21systemd-journal20:32:290
7174996747,13cyclictest9749-21idleruntime-cro20:47:290
7174996747,13cyclictest9749-21idleruntime-cro20:47:290
717499673,35cyclictest29647-21ssh21:27:310
7174996729,23cyclictest30353-21kworker/u17:2+xprtiod22:47:490
719199663,26cyclictest18190-21taskset19:32:353
719199663,26cyclictest18190-21taskset19:32:353
719199662,26cyclictest11013-21chrt19:12:303
719199662,24cyclictest7531-21chrt19:02:493
719199662,23cyclictest29551-21chrt20:07:423
719199662,22cyclictest32337-21taskset00:12:443
7183996643,9cyclictest181rcu_preempt19:03:032
718399663,35cyclictest2429-21ntpq20:22:582
717499662,54cyclictest20403-21proc_pri19:38:020
717499662,54cyclictest20403-21proc_pri19:38:020
719199653,25cyclictest15041-21taskset21:56:303
719199653,23cyclictest25381-21chrt21:19:043
719199653,23cyclictest13238-21chrt20:57:363
719199653,23cyclictest13238-21chrt20:57:363
719199653,22cyclictest29331-21chrt21:26:463
719199653,21cyclictest18866-21chrt22:02:463
719199653,21cyclictest18866-21chrt22:02:463
719199653,21cyclictest12621-21taskset21:52:333
719199652,25cyclictest8914-21taskset21:46:123
719199652,25cyclictest8914-21taskset21:46:123
719199652,24cyclictest8410-21chrt20:42:413
719199652,24cyclictest8410-21chrt20:42:413
719199652,24cyclictest22210-21chrt22:07:573
719199652,24cyclictest11007-21chrt23:37:453
719199652,23cyclictest12088-21taskset19:14:003
719199652,22cyclictest31712-21taskset00:12:273
7183996528,9cyclictest181rcu_preempt00:27:542
7174996538,21cyclictest15937-21kworker/u17:1+nfsiod23:19:230
717499653,34cyclictest21194-21fschecks_count21:12:490
62042658,33sleep30-21swapper/318:58:063
719199643,25cyclictest5470-21taskset00:22:293
719199643,23cyclictest7776-21chrt20:38:083
719199643,23cyclictest7776-21chrt20:38:083
719199643,23cyclictest15789-21chrt22:52:163
719199643,22cyclictest25220-21chrt22:12:563
719199643,22cyclictest19837-21chrt19:37:453
719199643,22cyclictest19837-21chrt19:37:453
719199642,24cyclictest10155-21chrt20:47:483
719199642,23cyclictest703-21chrt22:26:593
719199642,23cyclictest5338-21chrt21:39:363
719199642,23cyclictest27767-21taskset00:04:413
719199642,22cyclictest24254-21chrt23:05:463
719199642,22cyclictest23470-21chrt23:57:523
719199642,22cyclictest19546-21chrt22:57:523
717499643,32cyclictest18216-21apt-get19:32:450
717499643,32cyclictest18216-21apt-get19:32:450
7174996432,8cyclictest181rcu_preempt19:18:010
717499642,43cyclictest352-21runrttasks20:07:470
719199634,24cyclictest3199-21chrt23:23:263
719199634,24cyclictest3199-21chrt23:23:263
719199633,26cyclictest643-21chrt23:19:413
719199633,22cyclictest9600-21taskset23:34:333
719199633,22cyclictest864-21chrt20:17:593
719199633,22cyclictest29959-21chrt22:21:233
719199633,22cyclictest29959-21chrt22:21:233
719199633,22cyclictest2620-21chrt22:28:573
719199633,22cyclictest2620-21chrt22:28:573
719199632,26cyclictest7450-21chrt00:24:443
719199632,23cyclictest20515-21chrt23:53:383
719199632,23cyclictest18415-21chrt22:56:093
719199632,22cyclictest20388-21chrt21:11:543
719199632,22cyclictest14597-21chrt23:43:003
719199632,21cyclictest23550-21chrt19:47:583
717499633,41cyclictest8342-21latency_hist20:42:320
717499633,41cyclictest8342-21latency_hist20:42:320
719199623,21cyclictest10915-21chrt00:31:093
719199622,24cyclictest4613-21chrt20:29:003
719199622,24cyclictest12727-21chrt20:54:123
719199622,23cyclictest27875-21chrt20:02:403
719199622,23cyclictest14235-21chrt19:19:353
717499623,43cyclictest24927-21fschecks_count22:12:500
719199613,20cyclictest2727-21chrt21:36:193
719199612,23cyclictest22485-21chrt19:43:573
719199612,23cyclictest22485-21chrt19:43:573
719199612,23cyclictest17404-21chrt19:28:583
719199612,22cyclictest25823-21chrt19:55:063
719199612,22cyclictest11422-21chrt22:43:253
719199612,21cyclictest15281-21chrt19:22:593
717499613,29cyclictest11630-21/usr/sbin/munin20:52:460
717499613,27cyclictest16520-21fschecks_count22:53:010
717499612,36cyclictest29485-21latency_hist20:07:330
719199602,23cyclictest6073-21chrt23:28:083
719199593,20cyclictest6327-21chrt20:35:043
719199593,20cyclictest6327-21chrt20:35:043
719199593,20cyclictest22960-21chrt21:15:393
719199592,22cyclictest3033-21chrt20:24:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional