You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 16:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Mon Dec 01, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
540224635,0sleep20-21swapper/206:16:022
471924126,0sleep00-21swapper/006:14:440
533823728,0sleep30-21swapper/306:15:193
553923526,0sleep10-21swapper/106:17:371
5763991010,0cyclictest24810-21awk11:34:372
57619970,0cyclictest6929-21cut08:09:370
29925070,0irq/32-eth0-rx-0-21swapper/311:17:493
57639966,0cyclictest17248-21diskmemload11:18:332
57629966,0cyclictest0-21swapper/107:24:401
4160,0ktimersoftd/00-21swapper/009:39:370
339960,0migration/37559-21munin-node11:24:373
339960,0migration/35854-21unixbench_singl07:44:523
329960,0watchdog/35296-21ssh09:26:483
329960,0watchdog/3299250irq/32-eth0-rx-11:44:323
329960,0watchdog/3299250irq/32-eth0-rx-10:09:323
329960,0watchdog/3299250irq/32-eth0-rx-09:42:363
329960,0watchdog/327858-21sed09:19:403
329960,0watchdog/310695-21munin-node11:04:403
329960,0watchdog/30-21swapper/310:59:203
329960,0watchdog/30-21swapper/310:58:003
329960,0watchdog/30-21swapper/310:40:083
329960,0watchdog/30-21swapper/310:27:443
329960,0watchdog/30-21swapper/310:01:563
329960,0watchdog/30-21swapper/309:51:163
329960,0watchdog/30-21swapper/309:47:523
329960,0watchdog/30-21swapper/309:38:323
329960,0watchdog/30-21swapper/308:49:363
329960,0watchdog/30-21swapper/308:31:083
329960,0watchdog/30-21swapper/308:22:243
329960,0watchdog/30-21swapper/307:19:403
329960,0watchdog/30-21swapper/306:54:203
329960,0watchdog/30-21swapper/306:34:403
29935060,0irq/33-eth0-tx-25584-21perl09:59:342
29935060,0irq/33-eth0-tx-0-21swapper/008:14:380
29935060,0irq/33-eth0-tx-0-21swapper/007:37:180
29935060,0irq/33-eth0-tx-0-21swapper/007:09:120
29925060,0irq/32-eth0-rx-558-21perl11:39:343
29925060,0irq/32-eth0-rx-2300-21tr11:19:383
29925060,0irq/32-eth0-rx-20936-21grep11:29:383
29925060,0irq/32-eth0-rx-0-21swapper/311:34:383
29925060,0irq/32-eth0-rx-0-21swapper/310:51:523
29925060,0irq/32-eth0-rx-0-21swapper/310:45:033
29925060,0irq/32-eth0-rx-0-21swapper/310:35:423
29925060,0irq/32-eth0-rx-0-21swapper/310:17:143
29925060,0irq/32-eth0-rx-0-21swapper/309:30:093
29925060,0irq/32-eth0-rx-0-21swapper/309:01:553
29925060,0irq/32-eth0-rx-0-21swapper/308:55:123
29925060,0irq/32-eth0-rx-0-21swapper/308:45:363
149960,0migration/08955-21sh08:31:230
149960,0migration/032088-21latency_hist11:39:190
149960,0migration/025449-21hwlatdetect07:39:350
8315050,0irq/26-0000:00:4940-21awk09:09:192
57639955,0cyclictest0-21swapper/210:47:112
57639950,0cyclictest0-21swapper/209:26:562
57629955,0cyclictest0-21swapper/111:34:331
57629955,0cyclictest0-21swapper/111:19:441
57629955,0cyclictest0-21swapper/111:09:291
57629955,0cyclictest0-21swapper/108:19:411
57629950,0cyclictest0-21swapper/110:46:001
57629950,0cyclictest0-21swapper/110:29:551
57629950,0cyclictest0-21swapper/110:23:361
57619955,0cyclictest25103-21kworker/0:011:31:180
57619950,0cyclictest0-21swapper/011:14:240
57619950,0cyclictest0-21swapper/010:05:590
339950,0migration/320677-21munin-node07:14:363
339950,0migration/31492-21kthreadcore09:04:383
339950,0migration/311335-21kthreadcore07:49:403
329950,0watchdog/321909-21irqstats06:29:363
329950,0watchdog/30-21swapper/311:09:283
329950,0watchdog/30-21swapper/310:20:403
329950,0watchdog/30-21swapper/310:04:243
329950,0watchdog/30-21swapper/309:55:443
329950,0watchdog/30-21swapper/309:15:243
329950,0watchdog/30-21swapper/309:09:483
329950,0watchdog/30-21swapper/308:34:243
329950,0watchdog/30-21swapper/308:24:443
329950,0watchdog/30-21swapper/308:14:483
329950,0watchdog/30-21swapper/308:10:043
329950,0watchdog/30-21swapper/308:05:123
329950,0watchdog/30-21swapper/308:03:043
329950,0watchdog/30-21swapper/307:54:323
329950,0watchdog/30-21swapper/307:40:403
329950,0watchdog/30-21swapper/307:34:443
329950,0watchdog/30-21swapper/307:30:403
329950,0watchdog/30-21swapper/307:24:363
329950,0watchdog/30-21swapper/307:10:523
329950,0watchdog/30-21swapper/307:04:563
329950,0watchdog/30-21swapper/306:59:283
329950,0watchdog/30-21swapper/306:49:203
329950,0watchdog/30-21swapper/306:44:323
329950,0watchdog/30-21swapper/306:39:243
329950,0watchdog/30-21swapper/306:24:283
329950,0watchdog/30-21swapper/306:20:043
29935050,0irq/33-eth0-tx-6377-21munin-node09:29:462
29935050,0irq/33-eth0-tx-29488-21kthreadcore10:19:382
29935050,0irq/33-eth0-tx-25653-21munin-node08:24:552
29935050,0irq/33-eth0-tx-22690-21fschecks_time09:19:352
29935050,0irq/33-eth0-tx-20380-21ssh11:11:592
29935050,0irq/33-eth0-tx-1608-21munin-node08:29:382
29935050,0irq/33-eth0-tx-16006-21chrt09:34:372
29935050,0irq/33-eth0-tx-13241-21ssh11:25:132
29935050,0irq/33-eth0-tx-10467-21munin-node10:29:382
29935050,0irq/33-eth0-tx-0-21swapper/211:04:382
29935050,0irq/33-eth0-tx-0-21swapper/210:25:522
29935050,0irq/33-eth0-tx-0-21swapper/209:54:062
29935050,0irq/33-eth0-tx-0-21swapper/209:08:022
29935050,0irq/33-eth0-tx-0-21swapper/208:51:032
29935050,0irq/33-eth0-tx-0-21swapper/208:39:552
29935050,0irq/33-eth0-tx-0-21swapper/008:20:390
29935050,0irq/33-eth0-tx-0-21swapper/007:32:520
29935050,0irq/33-eth0-tx-0-21swapper/006:34:420
29925050,0irq/32-eth0-rx-0-21swapper/310:29:303
29925050,0irq/32-eth0-rx-0-21swapper/308:39:533
259950,0migration/23425-21gltestperf06:39:372
259950,0migration/216312-21sh11:48:002
259950,0migration/214514-21idleruntime09:14:382
249950,0watchdog/29732-21fschecks_count10:09:362
249950,0watchdog/216607-21yum07:29:482
179950,0migration/126433-21kthreadcore09:59:371
179950,0migration/124212-21taskset09:58:201
179950,0migration/123973-21latency_hist06:54:201
179950,0migration/123324-21kthreadcore07:14:421
179950,0migration/119748-21kthreadcore07:34:371
179950,0migration/119528-21fschecks_time08:39:341
179950,0migration/113642-21munin-node09:14:351
179950,0migration/11217-21timerandwakeup08:44:471
149950,0migration/07868-21kthreadcore10:44:370
149950,0migration/05241-21smart_nvme0n106:59:430
149950,0migration/04694-21sh11:20:230
149950,0migration/030518-21cpuspeed_turbos09:24:320
149950,0migration/030254-21proc_pri06:54:480
149950,0migration/029336-21fschecks_time10:59:370
149950,0migration/027499-21taskset07:18:130
149950,0migration/027338-21taskset10:56:010
149950,0migration/02626-21sh08:48:010
149950,0migration/025753-21taskset10:35:140
149950,0migration/023669-21kthreadcore08:59:370
149950,0migration/023222-21kthreadcore07:59:370
149950,0migration/021520-21sh09:19:150
149950,0migration/020068-21timerandwakeup07:54:490
149950,0migration/018055-21kthreadcore07:09:370
149950,0migration/017968-21kthreadcore06:24:390
149950,0migration/017805-21munin-node06:49:340
149950,0migration/016431-21missed_timers10:29:380
149950,0migration/015205-21kthreadcore10:49:400
149950,0migration/013822-21taskset09:32:000
149950,0migration/013397-21taskset07:52:510
149950,0migration/010698-21hwlatdetect06:44:360
149950,0migration/010541-21kthreadcore09:49:420
57639944,0cyclictest0-21swapper/208:09:312
57639943,0cyclictest7325-21awk07:04:372
57639943,0cyclictest3599-21lldpd11:19:502
57639943,0cyclictest3599-21lldpd07:42:452
57639943,0cyclictest29777-21awk10:59:402
57639943,0cyclictest28558-21awk10:39:372
57639943,0cyclictest23871-21snmpd07:53:112
57639943,0cyclictest23871-21snmpd07:19:412
57639943,0cyclictest23871-21snmpd06:19:412
57639943,0cyclictest1831-21awk10:04:372
57639943,0cyclictest18008-21awk06:49:352
57639943,0cyclictest16082-21awk11:29:352
57629944,0cyclictest0-21swapper/109:29:321
57629944,0cyclictest0-21swapper/109:05:251
57629943,0cyclictest4636-21awk08:49:371
57629943,0cyclictest3599-21lldpd09:39:341
57629943,0cyclictest32693-21awk07:44:391
57629943,0cyclictest3262-21irqbalance06:54:341
57629943,0cyclictest3262-21irqbalance06:21:441
57629943,0cyclictest29226-21awk08:04:351
57629943,0cyclictest23871-21snmpd11:04:571
57629943,0cyclictest23871-21snmpd11:00:421
57629943,0cyclictest23871-21snmpd10:55:571
57629943,0cyclictest23871-21snmpd10:34:421
57629943,0cyclictest23871-21snmpd07:59:261
57629943,0cyclictest23871-21snmpd07:04:261
57629943,0cyclictest18646-21awk10:14:371
57629943,0cyclictest11125-21awk07:29:371
57619944,0cyclictest17248-21diskmemload11:26:230
57619944,0cyclictest0-21swapper/010:02:300
57619944,0cyclictest0-21swapper/008:43:310
57619943,0cyclictest7075-21awk06:19:370
57619943,0cyclictest4522-21awk08:49:360
57619943,0cyclictest25103-21kworker/0:011:04:320
57619943,0cyclictest23871-21snmpd10:41:270
57619943,0cyclictest23871-21snmpd10:15:270
57619943,0cyclictest23871-21snmpd07:45:260
57619940,0cyclictest2423-21munin-node06:39:450
29935040,0irq/33-eth0-tx-23345-21ssh09:39:272
29935040,0irq/33-eth0-tx-19211-21ssh10:51:402
29935040,0irq/33-eth0-tx-0-21swapper/211:39:282
29935040,0irq/33-eth0-tx-0-21swapper/210:54:372
29935040,0irq/33-eth0-tx-0-21swapper/210:34:202
29935040,0irq/33-eth0-tx-0-21swapper/210:14:382
29935040,0irq/33-eth0-tx-0-21swapper/209:54:262
29935040,0irq/33-eth0-tx-0-21swapper/209:44:332
29935040,0irq/33-eth0-tx-0-21swapper/208:54:452
259940,0migration/24498-21kthreadcore06:59:392
259940,0migration/231850-21kthreadcore06:34:402
259940,0migration/229072-21kthreadcore08:44:392
259940,0migration/221944-21irqcore08:59:362
259940,0migration/221915-21kthreadcore08:19:372
259940,0migration/217109-21kthreadcore06:24:382
259940,0migration/215766-21kthreadcore08:34:432
259940,0migration/213487-21kthreadcore06:44:392
259940,0migration/211315-21idleruntime08:14:372
249940,0watchdog/22360-21chrt08:05:082
249940,0watchdog/20-21swapper/207:09:362
179940,0migration/19455-21kthreadcore11:24:391
179940,0migration/18098-21kthreadcore09:09:381
179940,0migration/16760-21munin-node07:49:341
179940,0migration/15905-21taskset07:03:161
179940,0migration/1581-21kthreadcore09:24:371
179940,0migration/15314-21kthreadcore08:09:371
179940,0migration/13874-21kthreadcore09:44:391
179940,0migration/129782-21kthreadcore10:39:391
179940,0migration/124366-21kthreadcore11:14:381
179940,0migration/123638-21kthreadcore09:19:381
179940,0migration/120668-21cpuspeed_turbos06:29:321
179940,0migration/118804-21kthreadcore06:24:391
179940,0migration/117904-21missed_timers10:49:411
179940,0migration/11707-21munin-node07:19:461
179940,0migration/115275-21irqrtprio07:54:411
179940,0migration/115037-21aten_r1power_en11:29:301
179940,0migration/113032-21kthreadcore08:34:431
179940,0migration/112623-21fschecks_count08:54:351
179940,0migration/112200-21kthreadcore06:44:381
179940,0migration/111110-21kthreadcore11:44:381
169940,0watchdog/19731-21fschecks_count10:09:361
169940,0watchdog/19183-21ssh10:26:121
169940,0watchdog/13157-21perl06:39:361
169940,0watchdog/128286-40yum-updatesd-he09:03:201
169940,0watchdog/125700-21munin-node08:24:401
169940,0watchdog/115783-21kthreadcore07:09:361
169940,0watchdog/10-21swapper/109:51:081
169940,0watchdog/10-21swapper/109:34:401
169940,0watchdog/10-21swapper/108:29:361
169940,0watchdog/10-21swapper/108:14:481
149940,0migration/031508-21kthreadcore08:04:370
149940,0migration/025651-21runrttasks08:24:310
149940,0migration/021923-21kernelversion06:29:370
149940,0migration/020032-21diskmemload08:58:370
57639933,0cyclictest0-21swapper/207:24:352
57639932,0cyclictest3599-21lldpd07:59:542
57639932,0cyclictest3599-21lldpd07:54:542
57639932,0cyclictest3599-21lldpd07:34:442
57639932,0cyclictest20633-21grep06:29:322
57639930,0cyclictest29981-21munin-node06:54:462
57639930,0cyclictest0-21swapper/207:14:392
57629932,0cyclictest25617-21awk07:39:361
57629932,0cyclictest1007-21awk11:39:361
57619933,0cyclictest28133-21kworker/0:210:09:260
57619933,0cyclictest28133-21kworker/0:209:54:200
57619933,0cyclictest28133-21kworker/0:209:44:220
57619933,0cyclictest28133-21kworker/0:209:34:200
57619933,0cyclictest28133-21kworker/0:209:19:200
57619933,0cyclictest28133-21kworker/0:209:09:300
57619933,0cyclictest28133-21kworker/0:209:04:260
57619933,0cyclictest25103-21kworker/0:011:09:200
57619933,0cyclictest25103-21kworker/0:010:24:200
57619933,0cyclictest25103-21kworker/0:010:19:220
57619933,0cyclictest24842-21kworker/0:008:34:200
57619933,0cyclictest23100-21kworker/0:207:24:200
57619933,0cyclictest23100-21kworker/0:207:19:210
57619933,0cyclictest23047-21kworker/0:211:44:220
57619933,0cyclictest23047-21kworker/0:211:39:240
179930,0migration/14410-21kthreadcore10:04:391
169930,0watchdog/10-21swapper/106:34:481
57639920,0cyclictest0-21swapper/207:44:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional