You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 22:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Thu Dec 11, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1500523826,0sleep10-21swapper/106:17:521
1478423728,0sleep30-21swapper/306:15:183
1475223725,0sleep00-21swapper/006:14:540
1472923727,0sleep20-21swapper/206:14:362
329970,0watchdog/331061-21iostat_ios06:29:003
329970,0watchdog/30-21swapper/308:33:203
29925070,0irq/32-eth0-rx-2668-21irqstats06:54:013
151559977,0cyclictest2546-21nvme11:33:252
4160,0ktimersoftd/00-21swapper/006:23:440
339960,0migration/39830-21sh11:37:073
339960,0migration/318116-21sh11:40:443
339960,0migration/312621-21sh08:46:393
329960,0watchdog/330533-21kthreadcore07:34:043
329960,0watchdog/311138-21sh10:03:523
29935060,0irq/33-eth0-tx-5574-21munin-node10:38:592
29935060,0irq/33-eth0-tx-0-21swapper/007:03:570
29925060,0irq/32-eth0-rx-14398-21rt-features09:24:063
29925060,0irq/32-eth0-rx-10025-21grep09:43:573
29925060,0irq/32-eth0-rx-0-21swapper/311:48:083
29925060,0irq/32-eth0-rx-0-21swapper/311:24:013
29925060,0irq/32-eth0-rx-0-21swapper/311:12:333
29925060,0irq/32-eth0-rx-0-21swapper/310:29:013
29925060,0irq/32-eth0-rx-0-21swapper/310:24:433
29925060,0irq/32-eth0-rx-0-21swapper/310:08:573
29925060,0irq/32-eth0-rx-0-21swapper/309:54:023
29925060,0irq/32-eth0-rx-0-21swapper/309:38:493
29925060,0irq/32-eth0-rx-0-21swapper/309:34:303
29925060,0irq/32-eth0-rx-0-21swapper/309:28:493
29925060,0irq/32-eth0-rx-0-21swapper/309:19:023
29925060,0irq/32-eth0-rx-0-21swapper/309:16:433
29925060,0irq/32-eth0-rx-0-21swapper/309:03:543
29925060,0irq/32-eth0-rx-0-21swapper/308:39:043
169960,0watchdog/187642chrt11:34:191
151559966,0cyclictest2546-21nvme11:11:432
151559966,0cyclictest2546-21nvme11:03:342
151559966,0cyclictest2546-21nvme10:51:592
151559966,0cyclictest2546-21nvme10:35:532
151559966,0cyclictest2546-21nvme10:00:132
151559966,0cyclictest2546-21nvme09:57:152
151559966,0cyclictest2546-21nvme09:53:002
151559966,0cyclictest2546-21nvme09:42:072
151559966,0cyclictest2546-21nvme09:36:122
151559966,0cyclictest2546-21nvme09:31:532
151559966,0cyclictest2546-21nvme09:18:522
151559966,0cyclictest2546-21nvme09:17:052
151559966,0cyclictest2546-21nvme09:08:592
151559966,0cyclictest2546-21nvme09:07:522
151559966,0cyclictest2546-21nvme09:03:162
151559966,0cyclictest2546-21nvme08:56:552
151559966,0cyclictest2546-21nvme08:52:412
151559966,0cyclictest2546-21nvme08:46:392
151559966,0cyclictest2546-21nvme08:42:382
151559966,0cyclictest2546-21nvme08:33:202
151559966,0cyclictest2546-21nvme08:22:012
151559966,0cyclictest2546-21nvme08:17:252
151559966,0cyclictest2546-21nvme08:06:542
151559966,0cyclictest2546-21nvme07:36:272
151559966,0cyclictest2546-21nvme06:33:562
149960,0migration/016759-21missed_timers11:39:020
4150,0ktimersoftd/00-21swapper/010:20:080
4150,0ktimersoftd/00-21swapper/009:37:240
339950,0migration/39065-21cpuspeed_turbos07:43:583
339950,0migration/33533-21kthreadcore10:54:043
339950,0migration/331887-21kernelversion07:59:013
339950,0migration/33032-21missed_timers10:34:043
339950,0migration/330114-21aten_r1power_fr07:13:533
339950,0migration/330114-21aten_r1power_fr07:13:533
339950,0migration/323787-21runrttasks08:34:023
339950,0migration/321622-21sendmail_mailqu06:19:123
339950,0migration/318593-21kthreadcore07:49:033
339950,0migration/317381-21kthreadcore08:09:023
339950,0migration/316554-21idleruntime11:04:003
339950,0migration/31602-21munin-node11:14:033
339950,0migration/314771-21kthreadcore07:24:023
339950,0migration/314769-21fschecks_time10:44:023
339950,0migration/31388-21latency_hist07:38:443
339950,0migration/311979-21idleruntime-cro07:23:443
329950,0watchdog/35781-21if_err_eth1.306:34:003
329950,0watchdog/328153-21kthreadcore07:54:043
329950,0watchdog/314375-21ssh11:01:123
329950,0watchdog/30-21swapper/308:55:403
29935050,0irq/33-eth0-tx-8418-21munin-node06:59:000
29935050,0irq/33-eth0-tx-5947-21if_err_eth007:19:030
29935050,0irq/33-eth0-tx-4328-21munin-node10:19:002
29935050,0irq/33-eth0-tx-30913-21munin-node10:54:002
29935050,0irq/33-eth0-tx-23871-21snmpd06:39:080
29935050,0irq/33-eth0-tx-20168-21ssh10:27:202
29935050,0irq/33-eth0-tx-0-21swapper/211:36:552
29935050,0irq/33-eth0-tx-0-21swapper/210:03:552
29935050,0irq/33-eth0-tx-0-21swapper/007:59:110
29935050,0irq/33-eth0-tx-0-21swapper/007:54:010
29935050,0irq/33-eth0-tx-0-21swapper/007:45:120
29935050,0irq/33-eth0-tx-0-21swapper/007:34:060
29935050,0irq/33-eth0-tx-0-21swapper/006:36:340
29925050,0irq/32-eth0-rx-7072-21kthreadcore10:19:033
29925050,0irq/32-eth0-rx-20370-21ssh08:49:193
29925050,0irq/32-eth0-rx-0-21swapper/311:28:583
29925050,0irq/32-eth0-rx-0-21swapper/311:19:113
29925050,0irq/32-eth0-rx-0-21swapper/310:49:043
29925050,0irq/32-eth0-rx-0-21swapper/310:14:013
29925050,0irq/32-eth0-rx-0-21swapper/309:59:013
29925050,0irq/32-eth0-rx-0-21swapper/309:48:523
29925050,0irq/32-eth0-rx-0-21swapper/309:10:083
29925050,0irq/32-eth0-rx-0-21swapper/308:59:213
29925050,0irq/32-eth0-rx-0-21swapper/308:26:313
29925050,0irq/32-eth0-rx-0-21swapper/308:18:463
29925050,0irq/32-eth0-rx-0-21swapper/307:04:033
29925050,0irq/32-eth0-rx-0-21swapper/306:43:563
29925050,0irq/32-eth0-rx-0-21swapper/306:40:063
259950,0migration/224540-21kthreadcore07:09:022
259950,0migration/219248-21latency_hist07:28:442
259950,0migration/217270-21kthreadcore07:04:032
259950,0migration/210477-21kthreadcore09:24:022
249950,0watchdog/212732-21tr08:08:562
19150,0ktimersoftd/10-21swapper/108:52:051
179950,0migration/18647-21latency_hist07:43:441
179950,0migration/17397-21kthreadcore10:19:031
179950,0migration/1445-21kthreadcore10:54:031
179950,0migration/131427-21timerandwakeup09:34:121
179950,0migration/129624-21taskset07:11:361
179950,0migration/129006-21kthreadcore07:34:041
179950,0migration/128954-21sh10:33:081
179950,0migration/12828-21kthreadcore11:14:031
179950,0migration/126777-21kthreadcore06:24:031
179950,0migration/125436-21kthreadcore08:54:041
179950,0migration/124732-21unixbench-2d09:49:091
179950,0migration/122949-21latency11:24:021
179950,0migration/121237-21kthreadcore08:14:051
179950,0migration/120305-21kthreadcore09:29:031
179950,0migration/116664-21runrttasks09:45:031
169950,0watchdog/126586-21cut06:48:561
169950,0watchdog/117940-21ssh10:04:321
169950,0watchdog/112733-21cut08:08:561
151559955,0cyclictest2546-21nvme11:48:162
151559955,0cyclictest2546-21nvme10:43:512
151559955,0cyclictest2546-21nvme10:30:372
151559955,0cyclictest2546-21nvme07:43:182
151559955,0cyclictest2546-21nvme07:31:242
151559955,0cyclictest2546-21nvme07:22:132
151559955,0cyclictest2546-21nvme07:15:222
151559955,0cyclictest2546-21nvme07:15:222
151559955,0cyclictest2546-21nvme07:03:142
151559955,0cyclictest2546-21nvme06:53:352
151559955,0cyclictest2546-21nvme06:44:172
151559955,0cyclictest2546-21nvme06:40:522
151559955,0cyclictest2546-21nvme06:21:072
151549955,0cyclictest0-21swapper/110:26:121
151549950,0cyclictest0-21swapper/110:46:321
151549950,0cyclictest0-21swapper/109:05:191
151549950,0cyclictest0-21swapper/108:28:261
149950,0migration/06679-21sh09:03:550
149950,0migration/0655-21kthreadcore07:14:000
149950,0migration/0655-21kthreadcore07:14:000
149950,0migration/05571-21sh09:01:210
149950,0migration/02990-21kthreadcore11:14:030
149950,0migration/029889-21munin-node08:38:590
149950,0migration/028480-21cpuspeed_turbos10:13:570
149950,0migration/028457-21kthreadcore06:49:020
149950,0migration/028081-21munin-node08:19:020
149950,0migration/027923-21kthreadcore06:24:040
149950,0migration/027865-21if_eth1.309:54:000
149950,0migration/027485-21kthreadcore09:14:030
149950,0migration/025657-21kthreadcore11:09:030
149950,0migration/02523-21munin-node07:38:590
149950,0migration/025013-21kthreadcore11:44:040
149950,0migration/023538-21timerwakeupswit11:24:090
149950,0migration/022583-21taskset07:50:060
149950,0migration/022293-21sh11:04:280
149950,0migration/021954-21kthreadcore08:14:050
149950,0migration/019377-21sh10:24:430
149950,0migration/019325-21kthreadcore09:29:020
149950,0migration/018890-21munin-node08:09:090
149950,0migration/017903-21kthreadcore10:44:060
149950,0migration/016790-21latency_hist11:23:430
149950,0migration/016598-21kthreadcore10:04:010
149950,0migration/01443-21sh11:33:260
8315040,0irq/26-0000:00:30249-21wc07:58:432
339940,0migration/319596-21latency_hist08:13:433
329940,0watchdog/30-21swapper/306:48:563
29935040,0irq/33-eth0-tx-29955-21tail06:28:560
29935040,0irq/33-eth0-tx-0-21swapper/211:19:142
29935040,0irq/33-eth0-tx-0-21swapper/210:08:472
29935040,0irq/33-eth0-tx-0-21swapper/209:45:392
29935040,0irq/33-eth0-tx-0-21swapper/208:33:532
29935040,0irq/33-eth0-tx-0-21swapper/208:24:102
29935040,0irq/33-eth0-tx-0-21swapper/008:03:590
29935040,0irq/33-eth0-tx-0-21swapper/007:28:540
29925040,0irq/32-eth0-rx-15152-21cyclictest08:05:013
29925040,0irq/32-eth0-rx-0-21swapper/310:39:223
29925040,0irq/32-eth0-rx-0-21swapper/307:28:563
29925040,0irq/32-eth0-rx-0-21swapper/307:08:573
29925040,0irq/32-eth0-rx-0-21swapper/306:58:563
249940,0watchdog/21825-21ssh10:14:042
249940,0watchdog/211969-21iostat_ios11:38:592
249940,0watchdog/20-21swapper/211:03:562
21310240,0sleep00-21swapper/008:52:220
179940,0migration/17180-21kthreadcore08:04:051
179940,0migration/17076-21ksmtuned09:21:281
179940,0migration/131627-21kthreadcore09:54:031
179940,0migration/12934-21kthreadcore08:59:021
179940,0migration/127594-21kthreadcore07:54:041
179940,0migration/125645-21munin-node11:44:051
179940,0migration/123491-21taskset09:13:471
179940,0migration/12273-21if_eth1.309:39:001
179940,0migration/119514-21kthreadcore07:04:041
179940,0migration/113848-21kthreadcore06:59:031
169940,0watchdog/129942-21ls08:39:001
169940,0watchdog/122774-21df_abs10:49:001
169940,0watchdog/10-21swapper/110:40:441
169940,0watchdog/10-21swapper/110:36:321
169940,0watchdog/10-21swapper/108:28:441
169940,0watchdog/10-21swapper/108:18:561
169940,0watchdog/10-21swapper/106:34:001
151559943,0cyclictest3599-21lldpd07:51:042
151559943,0cyclictest23871-21snmpd11:24:052
151559943,0cyclictest23742-21awk06:24:012
151559943,0cyclictest1213-21awk11:14:022
151549944,0cyclictest0-21swapper/107:58:481
151549943,0cyclictest9663-21awk11:19:001
151549943,0cyclictest9234-21awk09:24:001
151549943,0cyclictest5824-21grep07:19:021
151549943,0cyclictest3262-21irqbalance11:08:471
151549943,0cyclictest3262-21irqbalance10:10:181
151549943,0cyclictest3262-21irqbalance07:43:481
151549943,0cyclictest3262-21irqbalance06:18:481
151549943,0cyclictest31218-21awk07:13:581
151549943,0cyclictest31218-21awk07:13:581
151549943,0cyclictest29296-21awk10:14:001
151549943,0cyclictest26735-21awk11:28:591
151549943,0cyclictest23871-21snmpd11:06:051
151549943,0cyclictest23871-21snmpd10:59:201
151549943,0cyclictest23871-21snmpd07:49:201
151549943,0cyclictest23871-21snmpd06:28:491
151549943,0cyclictest2270-21awk06:54:001
151549943,0cyclictest20475-21awk07:28:581
151549943,0cyclictest20321-21awk06:44:011
151549943,0cyclictest13346-21awk07:24:011
151549943,0cyclictest13020-21awk06:38:591
151539944,0cyclictest0-21swapper/010:34:480
151539944,0cyclictest0-21swapper/009:12:390
151539944,0cyclictest0-21swapper/008:29:460
151539943,0cyclictest9192-21awk09:24:000
151539943,0cyclictest8937-21ps09:59:050
151539943,0cyclictest3262-21irqbalance10:55:070
151539943,0cyclictest30207-21sadc08:58:430
151539943,0cyclictest23571-21awk10:49:030
151539943,0cyclictest20297-21awk06:44:010
151539943,0cyclictest15445-21kworker/0:207:09:000
151539943,0cyclictest11220-21kworker/0:108:34:340
151539943,0cyclictest10890-21awk09:44:000
149940,0migration/07580-21kthreadcore10:39:020
149940,0migration/03501-21fschecks_count08:23:580
149940,0migration/02886-21sh11:34:010
149940,0migration/017985-21kthreadcore07:24:030
149940,0migration/011921-21sh08:44:450
8315030,0irq/26-0000:00:1101-21runrttasks06:53:522
29925030,0irq/32-eth0-rx-0-21swapper/306:23:573
249930,0watchdog/228154-21kthreadcore07:54:042
249930,0watchdog/20-21swapper/207:44:042
169930,0watchdog/10-21swapper/109:58:521
169930,0watchdog/10-21swapper/109:09:281
169930,0watchdog/10-21swapper/108:43:481
169930,0watchdog/10-21swapper/108:33:481
151559932,0cyclictest29864-21munin-node06:29:002
151549932,0cyclictest107012chrt11:38:481
151539933,0cyclictest9098-21kworker/0:010:58:460
151539933,0cyclictest9098-21kworker/0:010:28:460
151539933,0cyclictest9098-21kworker/0:010:08:520
151539933,0cyclictest9098-21kworker/0:009:48:440
151539933,0cyclictest9098-21kworker/0:009:38:440
151539933,0cyclictest21394-21kworker/0:209:18:500
151539933,0cyclictest15445-21kworker/0:206:53:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional