You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 14:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Nov 13, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
519623625,0sleep20-21swapper/206:19:472
502823625,0sleep00-21swapper/006:17:490
3061923524,0sleep10-21swapper/106:15:301
522623026,0sleep30-21swapper/306:20:083
329970,0watchdog/329293-21taskset09:03:453
329970,0watchdog/317191-21cut10:55:373
329970,0watchdog/30-21swapper/311:05:533
329970,0watchdog/30-21swapper/309:23:493
329970,0watchdog/30-21swapper/306:40:413
53199960,0cyclictest1154-21kthreadcore10:05:452
53189960,0cyclictest0-21swapper/109:47:431
329960,0watchdog/3164062chrt06:49:173
329960,0watchdog/312577-21timerandwakeup07:05:493
329960,0watchdog/30-21swapper/311:37:133
329960,0watchdog/30-21swapper/311:27:093
329960,0watchdog/30-21swapper/311:18:453
329960,0watchdog/30-21swapper/311:10:373
329960,0watchdog/30-21swapper/311:01:053
329960,0watchdog/30-21swapper/310:53:173
329960,0watchdog/30-21swapper/310:40:453
329960,0watchdog/30-21swapper/310:20:493
329960,0watchdog/30-21swapper/310:10:413
329960,0watchdog/30-21swapper/309:55:453
329960,0watchdog/30-21swapper/309:50:173
329960,0watchdog/30-21swapper/309:41:053
329960,0watchdog/30-21swapper/309:15:173
329960,0watchdog/30-21swapper/308:58:253
329960,0watchdog/30-21swapper/308:54:333
329960,0watchdog/30-21swapper/308:43:053
329960,0watchdog/30-21swapper/308:35:453
29925060,0irq/32-eth0-rx-8705-21perl09:30:413
29925060,0irq/32-eth0-rx-8705-21perl09:30:413
29925060,0irq/32-eth0-rx-19221-21if_err_eth1.308:20:413
29925060,0irq/32-eth0-rx-0-21swapper/311:33:593
29925060,0irq/32-eth0-rx-0-21swapper/310:45:433
29925060,0irq/32-eth0-rx-0-21swapper/310:35:283
29925060,0irq/32-eth0-rx-0-21swapper/310:27:593
29925060,0irq/32-eth0-rx-0-21swapper/309:29:373
29925060,0irq/32-eth0-rx-0-21swapper/308:47:183
29925060,0irq/32-eth0-rx-0-21swapper/308:32:233
149960,0migration/019728-21latency_hist10:00:250
149960,0migration/012039-21kthreadcore09:10:430
149960,0migration/010377-21cpuspeed_turbos07:30:370
8315050,0irq/26-0000:00:21316-21cut08:00:252
53199955,0cyclictest0-21swapper/211:19:042
53199955,0cyclictest0-21swapper/209:43:032
53199955,0cyclictest0-21swapper/208:30:412
53199950,0cyclictest0-21swapper/210:52:352
53199950,0cyclictest0-21swapper/210:17:252
53199950,0cyclictest0-21swapper/209:27:172
53189955,0cyclictest23871-21snmpd08:28:171
53189955,0cyclictest0-21swapper/111:07:391
53189955,0cyclictest0-21swapper/110:12:281
53189950,0cyclictest0-21swapper/109:08:271
53189950,0cyclictest0-21swapper/108:32:591
53179955,0cyclictest0-21swapper/011:35:490
53179955,0cyclictest0-21swapper/009:45:460
339950,0migration/37281-21munin-node07:50:403
339950,0migration/326230-21kthreadcore08:00:433
339950,0migration/320516-21fschecks_count10:00:393
339950,0migration/317420-21kthreadcore09:15:443
339950,0migration/316641-21cpuspeed_turbos09:35:373
329950,0watchdog/331277-21awk09:05:413
329950,0watchdog/329818-21iostat_ios08:05:413
329950,0watchdog/30-21swapper/311:45:293
329950,0watchdog/30-21swapper/311:40:373
329950,0watchdog/30-21swapper/311:21:293
329950,0watchdog/30-21swapper/310:15:253
329950,0watchdog/30-21swapper/310:05:253
329950,0watchdog/30-21swapper/308:25:253
329950,0watchdog/30-21swapper/308:10:453
329950,0watchdog/30-21swapper/307:57:013
329950,0watchdog/30-21swapper/307:45:493
329950,0watchdog/30-21swapper/307:36:013
329950,0watchdog/30-21swapper/307:30:373
329950,0watchdog/30-21swapper/307:25:253
329950,0watchdog/30-21swapper/307:20:493
329950,0watchdog/30-21swapper/307:16:213
329950,0watchdog/30-21swapper/307:10:333
329950,0watchdog/30-21swapper/307:00:373
329950,0watchdog/30-21swapper/306:55:453
329950,0watchdog/30-21swapper/306:50:293
329950,0watchdog/30-21swapper/306:35:453
329950,0watchdog/30-21swapper/306:31:533
329950,0watchdog/30-21swapper/306:27:133
329950,0watchdog/30-21swapper/306:20:293
29935050,0irq/33-eth0-tx-8759-21grep10:10:452
29935050,0irq/33-eth0-tx-6294-21munin-node09:10:412
29935050,0irq/33-eth0-tx-5541-21munin-node06:20:530
29935050,0irq/33-eth0-tx-17152-21munin-node10:55:382
29935050,0irq/33-eth0-tx-0-21swapper/209:24:182
29935050,0irq/33-eth0-tx-0-21swapper/208:22:162
29935050,0irq/33-eth0-tx-0-21swapper/006:36:440
29925050,0irq/32-eth0-rx-30855-21sensors07:40:493
29925050,0irq/32-eth0-rx-0-21swapper/310:30:413
29925050,0irq/32-eth0-rx-0-21swapper/309:50:353
259950,0migration/27291-21munin-node07:50:402
259950,0migration/26825-21taskset06:40:422
259950,0migration/222383-21kthreadcore06:30:432
259950,0migration/210773-21munin-node08:50:462
249950,0watchdog/226591-21kthreadcore10:20:452
19150,0ktimersoftd/10-21swapper/108:36:121
179950,0migration/15902-21taskset07:05:141
179950,0migration/13246-21proc_pri11:40:461
179950,0migration/13038-21taskset10:06:151
179950,0migration/116678-21latency_hist06:50:251
179950,0migration/116010-21taskset10:52:551
179950,0migration/115943-21latency07:30:441
179950,0migration/110303-21kthreadcore09:10:431
169950,0watchdog/12922-21tune2fs06:40:371
149950,0migration/09590-21sh09:50:350
149950,0migration/08604-21kthreadcore10:10:450
149950,0migration/08283-21kthreadcore11:45:440
149950,0migration/06749-21latency_hist07:50:250
149950,0migration/05683-21pmu-power07:45:470
149950,0migration/05332-21missed_timers09:25:480
149950,0migration/04380-21sh10:10:080
149950,0migration/03534-21timerandwakeup11:40:500
149950,0migration/032513-21kthreadcore10:25:420
149950,0migration/028736-21taskset10:21:030
149950,0migration/027455-21kthreadcore11:00:420
149950,0migration/026829-21kthreadcore10:40:430
149950,0migration/025456-21kthreadcore08:00:430
149950,0migration/02387-21kthreadcore07:00:440
149950,0migration/023084-21switchtime06:50:510
149950,0migration/022982-21timerwakeupswit09:35:510
149950,0migration/022792-21sh09:19:590
149950,0migration/022247-21irqrtprio07:15:420
149950,0migration/022105-21kthreadcore08:40:470
149950,0migration/020089-21aten_r1power_cu06:30:370
149950,0migration/017762-21kthreadcore07:10:440
149950,0migration/017013-21kthreadcore07:55:450
149950,0migration/015944-21kthreadcore06:25:440
149950,0migration/014800-21kthreadcore08:35:450
149950,0migration/013458-21sh10:30:590
149950,0migration/013245-21if_eth111:30:380
149950,0migration/011046-21taskset08:50:490
149950,0migration/010462-21gltestperf06:45:400
149950,0migration/010012-21timerwakeupswit08:10:530
53199944,0cyclictest24374-21crond07:40:252
53199943,0cyclictest9879-21sadc07:30:252
53199943,0cyclictest4709-21awk08:10:432
53199943,0cyclictest428-21awk07:45:422
53199943,0cyclictest3599-21lldpd07:42:262
53199943,0cyclictest28505-21awk11:20:412
53199943,0cyclictest28396-21awk06:35:382
53199943,0cyclictest23871-21snmpd07:33:102
53199943,0cyclictest23871-21snmpd06:27:552
53199943,0cyclictest2038-21awk09:45:402
53199943,0cyclictest14546-21awk07:10:412
53199943,0cyclictest12511-21awk09:55:412
53199943,0cyclictest11905-21awk08:15:422
53199940,0cyclictest31731-21kthreadcore08:25:442
53199940,0cyclictest22402-21perl09:00:392
53199940,0cyclictest19994-21munin-node10:00:432
53189944,0cyclictest0-21swapper/109:20:261
53189944,0cyclictest0-21swapper/108:43:421
53189943,0cyclictest9439-21grep09:30:441
53189943,0cyclictest9439-21grep09:30:441
53189943,0cyclictest476-21awk07:45:421
53189943,0cyclictest4636-21awk08:10:431
53189943,0cyclictest3599-21lldpd08:56:021
53189943,0cyclictest32752-21awk09:25:441
53189943,0cyclictest31551-21awk10:25:401
53189943,0cyclictest26831-21awk11:00:411
53189943,0cyclictest25153-21awk06:55:411
53189943,0cyclictest23871-21snmpd07:20:101
53189943,0cyclictest23871-21snmpd07:07:251
53189943,0cyclictest18419-21awk07:35:421
53189943,0cyclictest18354-21awk10:55:421
53189943,0cyclictest16331-21awk10:35:441
53189943,0cyclictest15194-21awk07:55:421
53189943,0cyclictest13994-21awk06:25:421
53189943,0cyclictest12032-21awk08:15:431
53189943,0cyclictest0-21swapper/110:44:561
53179944,0cyclictest0-21swapper/008:31:500
53179943,0cyclictest8999-21awk09:30:420
53179943,0cyclictest8999-21awk09:30:420
53179943,0cyclictest29539-21awk08:45:420
53179943,0cyclictest26072-21awk09:40:430
53179943,0cyclictest25771-21awk07:40:430
53179943,0cyclictest25065-21awk06:55:410
53179943,0cyclictest20081-21awk11:15:420
53179940,0cyclictest0-21swapper/010:58:270
53179940,0cyclictest0-21swapper/010:37:480
53179940,0cyclictest0-21swapper/008:05:460
4140,0ktimersoftd/00-21swapper/009:07:090
329940,0watchdog/30-21swapper/308:17:093
29935040,0irq/33-eth0-tx-6839-21tail10:30:372
29935040,0irq/33-eth0-tx-6315-21iceccd08:45:272
29935040,0irq/33-eth0-tx-30419-21ssh10:25:252
29935040,0irq/33-eth0-tx-0-21swapper/211:45:312
29935040,0irq/33-eth0-tx-0-21swapper/211:25:392
29935040,0irq/33-eth0-tx-0-21swapper/211:00:302
29935040,0irq/33-eth0-tx-0-21swapper/210:45:312
29935040,0irq/33-eth0-tx-0-21swapper/210:40:372
29935040,0irq/33-eth0-tx-0-21swapper/209:35:502
29935040,0irq/33-eth0-tx-0-21swapper/209:17:212
29935040,0irq/33-eth0-tx-0-21swapper/208:55:282
29935040,0irq/33-eth0-tx-0-21swapper/208:40:262
29935040,0irq/33-eth0-tx-0-21swapper/008:15:380
29935040,0irq/33-eth0-tx-0-21swapper/007:35:430
29935040,0irq/33-eth0-tx-0-21swapper/007:20:360
259940,0migration/29441-21latency_hist06:45:252
259940,0migration/229618-21cpuspeed_turbos11:40:372
259940,0migration/218942-21kthreadcore10:35:462
259940,0migration/217589-21fschecks_time06:50:402
259940,0migration/216758-21kthreadcore11:30:402
259940,0migration/211277-21fschecks_time11:10:382
249940,0watchdog/28650-21unixbench_singl11:05:532
249940,0watchdog/26026-21fschecks_count06:20:412
249940,0watchdog/23599-21lldpd07:56:252
249940,0watchdog/230056-21ntp_offset06:55:452
249940,0watchdog/221921-21iostat07:15:412
249940,0watchdog/212433-21grep09:30:452
249940,0watchdog/212433-21grep09:30:452
249940,0watchdog/20-21swapper/211:35:452
249940,0watchdog/20-21swapper/209:51:292
249940,0watchdog/20-21swapper/209:05:412
249940,0watchdog/20-21swapper/208:35:292
179940,0migration/19590-21aten2.4-expect06:45:371
179940,0migration/19174-21switchtime07:25:511
179940,0migration/18069-21kthreadcore06:20:451
179940,0migration/16721-21kthreadcore11:25:461
179940,0migration/13410-21latency_hist08:10:251
179940,0migration/130730-21kthreadcore07:20:451
179940,0migration/130534-21kthreadcore11:20:441
179940,0migration/128706-21cpuspeed_turbos08:45:391
179940,0migration/127487-21kthreadcore07:40:451
179940,0migration/125467-21kthreadcore08:00:431
179940,0migration/123587-21kthreadcore10:20:441
179940,0migration/117623-21munin-node09:35:421
179940,0migration/115202-21kthreadcore07:10:431
179940,0migration/112959-21fschecks_time11:30:361
179940,0migration/112819-21ntp_offset10:30:451
169940,0watchdog/18855-21kthreadcore08:50:451
169940,0watchdog/125872-21hddtemp_smartct09:40:411
169940,0watchdog/125401-21ssh08:22:571
169940,0watchdog/120737-21ssh11:35:331
169940,0watchdog/118864-21ssh11:15:251
169940,0watchdog/10-21swapper/111:45:411
169940,0watchdog/10-21swapper/109:03:451
149940,0migration/07563-21irqcore07:05:410
149940,0migration/029761-21kthreadcore11:20:430
149940,0migration/027438-21fschecks_time08:25:400
149940,0migration/023340-21aten_r1power_po09:20:370
149940,0migration/020446-21sh08:57:340
149940,0migration/014150-21kthreadcore10:50:430
53199932,0cyclictest32319-21awk07:00:412
53199932,0cyclictest29078-21awk07:20:432
53199930,0cyclictest29351-21perl08:05:382
53189932,0cyclictest951-21awk10:45:401
53189932,0cyclictest3599-21lldpd06:30:441
53189930,0cyclictest22066-21kthreadcore10:00:431
53189930,0cyclictest11363-21perl11:10:381
53179933,0cyclictest5173-21kworker/0:207:25:290
53179933,0cyclictest19108-21kworker/0:111:10:370
53179933,0cyclictest19108-21kworker/0:111:05:390
53179933,0cyclictest19108-21kworker/0:110:45:450
53179933,0cyclictest19108-21kworker/0:109:00:330
53179932,0cyclictest5173-21kworker/0:208:20:370
53179932,0cyclictest4584-21awk11:25:440
53179932,0cyclictest25018-21grep10:00:440
53179932,0cyclictest14499-21awk10:15:430
53179931,0cyclictest0-21swapper/006:40:350
249930,0watchdog/20-21swapper/207:05:492
179930,0migration/115279-21fschecks_count09:15:401
169930,0watchdog/117872-21diskmemload09:56:091
169930,0watchdog/110048-21ssh09:52:051
169930,0watchdog/10-21swapper/110:15:451
169930,0watchdog/10-21swapper/107:50:411
53189921,0cyclictest0-21swapper/106:35:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional