You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 14:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Mon Dec 15, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
789824332,0sleep00-21swapper/007:07:230
779623926,0sleep10-21swapper/107:06:121
777523929,0sleep20-21swapper/207:05:572
804923728,0sleep30-21swapper/307:09:103
82119960,0cyclictest0-21swapper/111:43:531
82109960,0cyclictest0-21swapper/012:08:140
4160,0ktimersoftd/026864-21port_sshpriv09:45:330
339960,0migration/32990-21sh12:11:053
339960,0migration/326775-21sh09:11:533
339960,0migration/323299-21sh10:01:543
339960,0migration/314666-21sh10:34:013
29815060,0irq/29-eth0-tx-8466-21cut11:40:312
29805060,0irq/28-eth0-rx-17624-21grep12:20:303
29805060,0irq/28-eth0-rx-11809-21irqcore11:25:283
29805060,0irq/28-eth0-rx-0-21swapper/311:40:313
149960,0migration/0892-21sh10:08:430
149960,0migration/023751-21kthreadcore10:55:320
149960,0migration/021329-21needreboot11:10:320
8305050,0irq/26-0000:00:8824-21cut08:40:132
8305050,0irq/26-0000:00:25912-21unixbench_multi12:05:402
8305050,0irq/26-0000:00:0-21swapper/212:20:202
82129955,0cyclictest0-21swapper/209:25:332
82119955,0cyclictest0-21swapper/109:19:001
82119954,0cyclictest3524-21snmpd08:37:201
82119954,0cyclictest3524-21snmpd08:33:191
82119954,0cyclictest3524-21snmpd08:13:191
82119950,0cyclictest0-21swapper/112:12:441
82119950,0cyclictest0-21swapper/109:47:511
82109955,0cyclictest0-21swapper/012:18:290
82109955,0cyclictest0-21swapper/010:17:240
82109955,0cyclictest0-21swapper/009:57:320
82109955,0cyclictest0-21swapper/007:10:160
82109950,0cyclictest11207-21basename09:40:140
82109950,0cyclictest0-21swapper/011:37:490
339950,0migration/39283-21munin-node08:40:263
339950,0migration/36108-21fschecks_time12:15:293
339950,0migration/33906-21munin-node08:10:313
339950,0migration/332477-21needreboot07:45:303
339950,0migration/331769-21kthreadcore10:25:313
339950,0migration/328915-21ksmtuned09:15:243
339950,0migration/328645-21sh10:23:583
339950,0migration/32752-21kthreadcore07:25:313
339950,0migration/32501-21diskmemload10:45:293
339950,0migration/325008-21gltestperf11:50:283
339950,0migration/322948-21taskset11:12:373
339950,0migration/316978-21irqcore07:15:303
339950,0migration/316078-21aten_r1power_en08:45:253
339950,0migration/315811-21kthreadcore12:00:323
339950,0migration/314661-21taskset09:58:123
339950,0migration/313463-21gltestperf09:05:283
339950,0migration/311972-21taskset08:18:323
329950,0watchdog/33186-21kthreadcore08:55:323
329950,0watchdog/328771-21sh11:35:163
329950,0watchdog/324831-21ssh09:30:163
329950,0watchdog/323087-21kthreadcore10:55:323
329950,0watchdog/315639-21fschecks_time11:45:283
329950,0watchdog/30-21swapper/310:05:163
29815050,0irq/29-eth0-tx-30914-21cut10:40:352
29815050,0irq/29-eth0-tx-30315-21unixbench_singl12:25:392
29815050,0irq/29-eth0-tx-29544-21munin-node10:25:292
29815050,0irq/29-eth0-tx-22110-21grep11:30:302
29815050,0irq/29-eth0-tx-21273-21perl10:20:262
29815050,0irq/29-eth0-tx-13681-21ssh09:56:062
29815050,0irq/29-eth0-tx-13533-21ssh09:22:132
29815050,0irq/29-eth0-tx-11694-21grep10:30:322
29815050,0irq/29-eth0-tx-0-21swapper/211:55:262
29815050,0irq/29-eth0-tx-0-21swapper/210:51:252
29815050,0irq/29-eth0-tx-0-21swapper/008:37:190
29815050,0irq/29-eth0-tx-0-21swapper/007:45:260
29805050,0irq/28-eth0-rx-3997-21cut11:20:313
29805050,0irq/28-eth0-rx-3598-21kthreadcore10:10:313
29805050,0irq/28-eth0-rx-32607-21cat09:50:293
29805050,0irq/28-eth0-rx-31481-21ssh12:28:013
29805050,0irq/28-eth0-rx-30403-21iostat_ios11:00:293
29805050,0irq/28-eth0-rx-28452-21tr10:40:323
29805050,0irq/28-eth0-rx-19264-21ssh12:05:203
29805050,0irq/28-eth0-rx-19061-21cut08:00:313
29805050,0irq/28-eth0-rx-18928-21awk07:38:223
29805050,0irq/28-eth0-rx-15679-21cpu09:25:253
29805050,0irq/28-eth0-rx-11914-21df_inode09:40:253
29805050,0irq/28-eth0-rx-0-21swapper/312:35:273
29805050,0irq/28-eth0-rx-0-21swapper/312:30:323
29805050,0irq/28-eth0-rx-0-21swapper/311:18:463
29805050,0irq/28-eth0-rx-0-21swapper/311:05:233
29805050,0irq/28-eth0-rx-0-21swapper/310:35:543
29805050,0irq/28-eth0-rx-0-21swapper/310:16:213
29805050,0irq/28-eth0-rx-0-21swapper/309:48:463
29805050,0irq/28-eth0-rx-0-21swapper/307:30:383
259950,0migration/29975-21sh09:37:482
259950,0migration/23536-21sh09:16:092
259950,0migration/227348-21kthreadcore11:15:322
259950,0migration/225495-21kthreadcore10:55:332
259950,0migration/222497-21sh10:36:072
259950,0migration/221355-21timerandwakeup11:45:402
259950,0migration/219383-21aten_r1power_en07:40:242
259950,0migration/218686-21timerandwakeup08:20:372
259950,0migration/21818-21kthreadcore11:35:332
259950,0migration/215836-21ntp_offset12:35:342
259950,0migration/21455-21sh09:34:512
259950,0migration/21337-21latency_hist10:45:142
259950,0migration/211382-21aten_r1power_vo10:15:252
259950,0migration/210341-21kthreadcore07:55:302
249950,0watchdog/226817-21cpuspeed_turbos07:45:232
179950,0migration/17367-21sh12:30:471
179950,0migration/17025-21munin-node09:20:261
179950,0migration/14375-21switchtime08:55:381
179950,0migration/12749-21rt-features11:35:361
179950,0migration/125693-21kthreadcore12:25:311
179950,0migration/112429-21kthreadcore12:00:311
179950,0migration/110255-21sh09:38:221
169950,0watchdog/126818-21grep07:45:231
169950,0watchdog/10-21swapper/107:35:311
149950,0migration/09526-21sh10:12:150
149950,0migration/08733-21sh11:22:080
149950,0migration/07369-21kthreadcore09:00:310
149950,0migration/07211-21latency07:50:300
149950,0migration/05183-21kthreadcore10:45:310
149950,0migration/04862-21sh09:51:550
149950,0migration/030173-21irqcore09:15:290
149950,0migration/030167-21latency_hist08:10:130
149950,0migration/026530-21sh09:11:020
149950,0migration/026347-21kthreadcore07:20:290
149950,0migration/024579-21aten_r1power_po11:15:260
149950,0migration/024410-21kthreadcore07:40:310
149950,0migration/024200-21aten_r1power_vo11:50:260
149950,0migration/023778-21fschecks_count08:50:260
149950,0migration/022332-21taskset11:47:150
149950,0migration/021307-21gltestperf10:20:270
149950,0migration/020722-21kthreadcore07:15:320
149950,0migration/019090-21taskset08:22:210
149950,0migration/018222-21kthreadcore08:00:310
149950,0migration/017478-21kthreadcore09:25:300
149950,0migration/016533-21kthreadcore10:50:340
149950,0migration/015054-21taskset10:34:500
149950,0migration/013457-21kthreadcore08:40:310
149950,0migration/012210-21kthreadcore12:00:310
149950,0migration/011359-21kthreadcore11:40:320
149950,0migration/011254-21fschecks_time11:25:250
82129943,0cyclictest3524-21snmpd10:05:212
82129943,0cyclictest3524-21snmpd08:17:492
82129943,0cyclictest3524-21snmpd08:00:342
82129943,0cyclictest3250-21irqbalance10:10:192
82129943,0cyclictest29584-21awk12:10:302
82129943,0cyclictest2688-21awk08:35:282
82129943,0cyclictest20817-21awk09:10:282
82129943,0cyclictest16488-21awk11:10:292
82119944,0cyclictest0-21swapper/111:05:251
82119944,0cyclictest0-21swapper/110:36:151
82119944,0cyclictest0-21swapper/110:07:241
82119943,0cyclictest9836-21awk07:55:291
82119943,0cyclictest3571-21lldpd10:50:281
82119943,0cyclictest3571-21lldpd09:55:571
82119943,0cyclictest3571-21lldpd09:25:261
82119943,0cyclictest3571-21lldpd07:10:241
82119943,0cyclictest3524-21snmpd08:50:201
82119943,0cyclictest3250-21irqbalance09:00:291
82119943,0cyclictest3250-21irqbalance08:47:091
82119943,0cyclictest3250-21irqbalance07:50:191
82119943,0cyclictest2628-21awk10:45:291
82119943,0cyclictest24110-21awk08:05:291
82119943,0cyclictest23949-21awk07:20:271
82119943,0cyclictest1860-21awk11:55:301
82119943,0cyclictest16516-21irqcore11:10:301
82119943,0cyclictest15694-21awk12:20:291
82109943,0cyclictest31077-21awk07:25:290
82109943,0cyclictest30541-21kworker/0:211:05:320
82109943,0cyclictest17218-21awk08:45:300
82109943,0cyclictest17105-21awk10:00:290
82109943,0cyclictest1596-21awk12:30:300
82109943,0cyclictest12623-21awk09:40:280
82109940,0cyclictest12112-21munin-node07:35:280
339940,0migration/34907-21kthreadcore07:50:303
329940,0watchdog/325876-21grep08:05:313
329940,0watchdog/323950-21if_eth1.307:20:273
329940,0watchdog/30-21swapper/311:30:243
329940,0watchdog/30-21swapper/309:04:243
329940,0watchdog/30-21swapper/308:30:243
29815040,0irq/29-eth0-tx-5248-21ssh12:15:212
29815040,0irq/29-eth0-tx-32762-21ssh12:30:172
29815040,0irq/29-eth0-tx-12491-21ssh11:05:382
29815040,0irq/29-eth0-tx-1171-21munin-node11:20:272
29815040,0irq/29-eth0-tx-0-21swapper/211:50:312
29815040,0irq/29-eth0-tx-0-21swapper/211:25:142
29815040,0irq/29-eth0-tx-0-21swapper/211:00:252
29815040,0irq/29-eth0-tx-0-21swapper/209:50:282
29815040,0irq/29-eth0-tx-0-21swapper/209:40:542
29815040,0irq/29-eth0-tx-0-21swapper/008:55:250
29805040,0irq/28-eth0-rx-25262-21kthreadcore08:50:303
29805040,0irq/28-eth0-rx-20786-21ps08:25:293
29805040,0irq/28-eth0-rx-13407-21kthreadcore10:50:333
29805040,0irq/28-eth0-rx-0-21swapper/311:55:303
29805040,0irq/28-eth0-rx-0-21swapper/309:35:223
29805040,0irq/28-eth0-rx-0-21swapper/309:20:293
29805040,0irq/28-eth0-rx-0-21swapper/308:20:243
29805040,0irq/28-eth0-rx-0-21swapper/307:10:273
259940,0migration/27520-21kthreadcore07:30:302
259940,0migration/230946-21munin-node07:25:282
259940,0migration/221857-21latency10:00:312
259940,0migration/218724-21sh09:05:362
259940,0migration/213469-21kthreadcore07:10:272
249940,0watchdog/28456-21kthreadcore09:00:322
249940,0watchdog/226828-21aten_r1power_cu08:30:242
249940,0watchdog/223969-21awk07:20:272
249940,0watchdog/223678-21fschecks_count08:05:272
249940,0watchdog/220746-21ssh09:45:242
249940,0watchdog/220247-21kthreadcore08:45:322
249940,0watchdog/20-21swapper/212:00:282
179940,0migration/19082-21kthreadcore10:30:321
179940,0migration/132337-21kthreadcore09:50:291
179940,0migration/130225-21fschecks_time10:25:271
179940,0migration/129221-21kthreadcore11:50:321
179940,0migration/122289-21unixbench-2d08:00:391
179940,0migration/121774-21kthreadcore09:10:301
179940,0migration/120556-21cpu10:20:251
179940,0migration/118671-21switchtime08:20:371
179940,0migration/117259-21kthreadcore09:05:321
179940,0migration/116594-21kthreadcore11:45:321
179940,0migration/115593-21taskset07:15:211
179940,0migration/114407-21kthreadcore09:40:301
179940,0migration/113088-21kthreadcore10:15:301
169940,0watchdog/18831-21sshd11:22:201
169940,0watchdog/1684-21ssh11:19:561
169940,0watchdog/16554-21irqcore12:15:321
169940,0watchdog/13370-21kthreadcore07:25:311
169940,0watchdog/127814-21kthreadcore10:40:321
169940,0watchdog/127072-21ssh11:32:081
169940,0watchdog/115921-21munin-node10:00:321
169940,0watchdog/110450-21ssh11:25:201
169940,0watchdog/10-21swapper/112:39:361
169940,0watchdog/10-21swapper/112:05:161
169940,0watchdog/10-21swapper/110:55:321
149940,0migration/0849-21aten_r1power_fr11:55:260
149940,0migration/07625-21irqrtprio09:20:290
149940,0migration/032389-21kthreadcore08:10:290
149940,0migration/027956-21sh11:34:010
149940,0migration/024823-21if_eth1.212:25:290
149940,0migration/024789-21aten_r1power_en10:40:250
149940,0migration/018728-21runrttasks09:05:370
149940,0migration/016096-21fschecks_count10:35:270
82129932,0cyclictest24271-21awk08:50:282
82129931,0cyclictest0-21swapper/207:50:292
82129930,0cyclictest3875-21netstat08:10:312
82129930,0cyclictest20316-21perl08:25:262
82119932,0cyclictest6283-21awk08:15:281
82119932,0cyclictest3524-21snmpd08:26:191
82119932,0cyclictest20365-21awk07:40:281
82119932,0cyclictest10018-21awk08:40:291
82109933,0cyclictest32515-21kworker/0:009:30:200
82109933,0cyclictest32515-21kworker/0:008:28:020
82109933,0cyclictest30541-21kworker/0:212:20:270
82109933,0cyclictest30541-21kworker/0:212:11:430
82109933,0cyclictest10814-21awk12:35:290
82109932,0cyclictest5958-21awk07:30:280
82109932,0cyclictest5571-21diskstats08:15:250
82109932,0cyclictest31502-21kthreadcore11:00:300
82109932,0cyclictest27955-21awk08:30:280
82109930,0cyclictest8660-21munin-node07:55:300
82109930,0cyclictest29978-21diskstats10:25:260
329930,0watchdog/30-21swapper/308:35:323
329930,0watchdog/30-21swapper/307:55:433
29805030,0irq/28-eth0-rx-0-21swapper/307:40:233
249930,0watchdog/20-21swapper/208:55:322
249930,0watchdog/20-21swapper/207:35:312
179930,0migration/17292-21kthreadcore07:30:301
169930,0watchdog/10-21swapper/111:00:321
169930,0watchdog/10-21swapper/110:12:241
169930,0watchdog/10-21swapper/109:30:161
249920,0watchdog/20-21swapper/207:15:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional