You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-11 - 22:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Tue Nov 11, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2496726258,0sleep20-21swapper/206:19:002
1900026059,0sleep00-21swapper/006:15:500
25176995858,0cyclictest10331-21md0_resync08:26:041
25175994646,0cyclictest10330-21md1_resync08:26:040
25177993939,0cyclictest83150irq/26-0000:00:08:26:042
2495223228,0sleep30-21swapper/306:18:493
2473322920,0sleep10-21swapper/106:16:191
25177991616,0cyclictest83150irq/26-0000:00:11:47:402
25177991616,0cyclictest83150irq/26-0000:00:10:06:072
25177991515,0cyclictest83150irq/26-0000:00:10:39:262
25177991414,0cyclictest83150irq/26-0000:00:09:34:102
25177991111,0cyclictest83150irq/26-0000:00:10:32:482
25177991010,0cyclictest83150irq/26-0000:00:09:29:322
251779999,0cyclictest83150irq/26-0000:00:08:55:572
251779988,0cyclictest83150irq/26-0000:00:11:43:402
251779988,0cyclictest83150irq/26-0000:00:11:35:432
251779988,0cyclictest83150irq/26-0000:00:11:09:352
251779988,0cyclictest83150irq/26-0000:00:10:42:502
251779988,0cyclictest83150irq/26-0000:00:10:29:012
251779988,0cyclictest83150irq/26-0000:00:10:04:222
251779988,0cyclictest83150irq/26-0000:00:08:39:422
329970,0watchdog/30-21swapper/309:25:563
29925070,0irq/32-eth0-rx-0-21swapper/308:37:293
251779977,0cyclictest83150irq/26-0000:00:11:10:442
251779977,0cyclictest83150irq/26-0000:00:11:02:272
251779977,0cyclictest83150irq/26-0000:00:10:21:052
251779977,0cyclictest83150irq/26-0000:00:10:18:112
251779977,0cyclictest83150irq/26-0000:00:09:20:372
251779977,0cyclictest83150irq/26-0000:00:08:21:562
251759970,0cyclictest0-21swapper/009:51:500
8315060,0irq/26-0000:00:426-21port_sshpriv09:50:552
8315060,0irq/26-0000:00:2261-21aten2.4-expect08:20:463
8315060,0irq/26-0000:00:17004-21python07:20:563
8315060,0irq/26-0000:00:0-21swapper/307:15:513
8315060,0irq/26-0000:00:0-21swapper/306:45:483
8315060,0irq/26-0000:00:0-21swapper/306:30:483
339960,0migration/37001-21munin-node06:55:493
339960,0migration/328116-21aten_r1power_cu07:10:443
339960,0migration/318349-21sh10:01:533
339960,0migration/317692-21gltestperf11:00:463
339960,0migration/312404-21aten_r1power_en08:05:443
329960,0watchdog/36976-21ssh09:19:283
329960,0watchdog/329816-21grep11:25:523
329960,0watchdog/30-21swapper/311:40:563
329960,0watchdog/30-21swapper/311:39:563
329960,0watchdog/30-21swapper/311:22:043
329960,0watchdog/30-21swapper/311:14:483
329960,0watchdog/30-21swapper/310:47:043
329960,0watchdog/30-21swapper/310:25:363
329960,0watchdog/30-21swapper/310:06:523
329960,0watchdog/30-21swapper/309:57:243
329960,0watchdog/30-21swapper/309:54:403
329960,0watchdog/30-21swapper/309:37:203
329960,0watchdog/30-21swapper/309:12:083
329960,0watchdog/30-21swapper/308:55:483
329960,0watchdog/30-21swapper/308:51:203
329960,0watchdog/30-21swapper/308:47:363
329960,0watchdog/30-21swapper/308:00:363
329960,0watchdog/30-21swapper/307:55:523
329960,0watchdog/30-21swapper/307:25:483
329960,0watchdog/30-21swapper/306:30:053
29935060,0irq/33-eth0-tx-0-21swapper/008:05:460
29925060,0irq/32-eth0-rx-15639-21iostat09:05:553
29925060,0irq/32-eth0-rx-13248-21tr09:40:513
29925060,0irq/32-eth0-rx-0-21swapper/310:53:273
29925060,0irq/32-eth0-rx-0-21swapper/310:39:473
29925060,0irq/32-eth0-rx-0-21swapper/310:35:093
29925060,0irq/32-eth0-rx-0-21swapper/310:16:263
29925060,0irq/32-eth0-rx-0-21swapper/309:30:413
29925060,0irq/32-eth0-rx-0-21swapper/308:41:123
29925060,0irq/32-eth0-rx-0-21swapper/306:40:523
27160,0ktimersoftd/20-21swapper/207:52:282
27160,0ktimersoftd/20-21swapper/206:57:082
251779966,0cyclictest83150irq/26-0000:00:11:22:592
251779966,0cyclictest83150irq/26-0000:00:10:58:442
251779966,0cyclictest83150irq/26-0000:00:10:50:302
251779966,0cyclictest83150irq/26-0000:00:09:36:112
251779966,0cyclictest83150irq/26-0000:00:08:55:042
251779960,0cyclictest0-21swapper/209:59:232
19160,0ktimersoftd/10-21swapper/106:57:081
149960,0migration/030545-21kthreadcore11:05:520
149960,0migration/027698-3199-raid-check07:08:370
149960,0migration/01875-21ntp_offset07:10:560
8315050,0irq/26-0000:00:8948-21cpu06:35:553
8315050,0irq/26-0000:00:5534-21python09:15:572
8315050,0irq/26-0000:00:4621-21missed_timers06:50:553
8315050,0irq/26-0000:00:4088-21ssh08:40:382
8315050,0irq/26-0000:00:32401-21expr10:50:472
8315050,0irq/26-0000:00:21286-21cut07:05:523
8315050,0irq/26-0000:00:1888-21ssh11:26:472
8315050,0irq/26-0000:00:12587-21sed09:01:032
8315050,0irq/26-0000:00:10667-21diskstats11:15:452
8315050,0irq/26-0000:00:10448-21awk09:40:492
8315050,0irq/26-0000:00:0-21swapper/307:50:393
8315050,0irq/26-0000:00:0-21swapper/307:40:513
8315050,0irq/26-0000:00:0-21swapper/307:35:473
8315050,0irq/26-0000:00:0-21swapper/306:20:503
339950,0migration/312372-21kthreadcore09:20:533
329950,0watchdog/383150irq/26-0000:00:07:00:333
329950,0watchdog/30-21swapper/311:45:363
329950,0watchdog/30-21swapper/311:30:363
329950,0watchdog/30-21swapper/311:15:483
329950,0watchdog/30-21swapper/311:06:003
329950,0watchdog/30-21swapper/310:55:403
329950,0watchdog/30-21swapper/310:20:403
329950,0watchdog/30-21swapper/309:45:323
329950,0watchdog/30-21swapper/308:10:323
329950,0watchdog/30-21swapper/307:45:323
329950,0watchdog/30-21swapper/307:30:363
29935050,0irq/33-eth0-tx-25386-21munin-node07:30:470
29935050,0irq/33-eth0-tx-24522-21kthreadcore09:10:522
29935050,0irq/33-eth0-tx-2101-21munin-node08:20:470
29935050,0irq/33-eth0-tx-20115-21ssh08:47:282
29935050,0irq/33-eth0-tx-10592-21munin-node07:20:470
29935050,0irq/33-eth0-tx-0-21swapper/008:10:450
29925050,0irq/32-eth0-rx-20528-21irqcore08:30:503
29925050,0irq/32-eth0-rx-0-21swapper/310:40:383
29925050,0irq/32-eth0-rx-0-21swapper/310:10:573
29925050,0irq/32-eth0-rx-0-21swapper/309:00:463
29925050,0irq/32-eth0-rx-0-21swapper/308:25:393
29925050,0irq/32-eth0-rx-0-21swapper/308:15:453
251779955,0cyclictest83150irq/26-0000:00:10:12:572
251779955,0cyclictest0-21swapper/209:45:492
251779955,0cyclictest0-21swapper/208:30:372
251779955,0cyclictest0-21swapper/207:44:352
251779950,0cyclictest2092-21md1_raid108:08:042
251779950,0cyclictest0-21swapper/208:03:112
251769955,0cyclictest0-21swapper/111:07:321
251769955,0cyclictest0-21swapper/110:56:111
251769955,0cyclictest0-21swapper/110:52:241
251769955,0cyclictest0-21swapper/108:40:361
251769955,0cyclictest0-21swapper/107:03:481
251769950,0cyclictest0-21swapper/111:11:201
251769950,0cyclictest0-21swapper/107:05:321
251759955,0cyclictest23515-21kworker/0:108:44:540
251759955,0cyclictest0-21swapper/007:00:460
251759955,0cyclictest0-21swapper/006:36:110
251759954,0cyclictest23515-21kworker/0:109:46:300
251759950,0cyclictest0-21swapper/010:58:220
251759950,0cyclictest0-21swapper/009:37:380
251759950,0cyclictest0-21swapper/008:52:330
251759950,0cyclictest0-21swapper/008:01:470
259950,0migration/28772-21kthreadcore11:30:512
259950,0migration/231080-21munin-node07:55:552
259950,0migration/214319-21latency_hist09:05:322
249950,0watchdog/227176-21aten_r1power_fr08:15:442
179950,0migration/15161-21kthreadcore06:30:521
179950,0migration/128493-21latency07:50:511
179950,0migration/12644-21taskset07:12:501
179950,0migration/124133-21taskset11:02:421
179950,0migration/119742-21munin-node11:20:481
179950,0migration/117523-21taskset07:22:511
169950,0watchdog/122124-21memory06:40:531
169950,0watchdog/120585-21ssh11:40:081
169950,0watchdog/116366-21ps08:45:521
149950,0migration/08545-21kthreadcore11:30:510
149950,0migration/06385-21latency07:35:530
149950,0migration/04194-21idleruntime07:15:480
149950,0migration/03392-21kthreadcore08:55:510
149950,0migration/032237-21kthreadcore11:45:510
149950,0migration/03153-21kthreadcore06:50:540
149950,0migration/030989-21kthreadcore11:25:520
149950,0migration/030139-21unixbench_multi10:46:030
149950,0migration/029797-21taskset09:11:060
149950,0migration/029400-21irqrtprio08:35:520
149950,0migration/028593-21kthreadcore06:20:500
149950,0migration/027904-21proc_pri10:25:560
149950,0migration/023534-21kthreadcore10:06:000
149950,0migration/022641-21kthreadcore11:00:510
149950,0migration/021664-21timerandwakeup07:45:580
149950,0migration/021585-21kthreadcore07:25:510
149950,0migration/021350-21kthreadcore08:30:510
149950,0migration/015542-21kthreadcore10:20:500
149950,0migration/013766-21taskset09:04:120
149950,0migration/013324-21kthreadcore09:40:510
149950,0migration/013077-21kthreadcore10:00:510
149950,0migration/013007-21cpuspeed_turbos08:45:470
149950,0migration/010853-21kthreadcore07:40:510
29935040,0irq/33-eth0-tx-0-21swapper/006:42:580
251779944,0cyclictest0-21swapper/206:33:002
251769944,0cyclictest0-21swapper/111:46:291
251769944,0cyclictest0-21swapper/110:35:041
251769944,0cyclictest0-21swapper/109:55:581
251769944,0cyclictest0-21swapper/106:36:321
251769943,0cyclictest8733-21awk09:20:501
251769943,0cyclictest7454-21awk10:35:531
251769943,0cyclictest4222-21awk07:15:491
251769943,0cyclictest3599-21lldpd10:08:371
251769943,0cyclictest3599-21lldpd08:53:261
251769943,0cyclictest3599-21lldpd06:26:391
251769943,0cyclictest30867-21awk08:55:481
251769943,0cyclictest24581-21awk06:45:491
251769943,0cyclictest24372-21awk10:45:521
251769943,0cyclictest23871-21snmpd08:10:401
251769943,0cyclictest23871-21snmpd08:03:551
251769943,0cyclictest22662-21awk10:25:501
251769943,0cyclictest19134-21awk07:25:491
251769943,0cyclictest13413-21awk08:05:481
251759944,0cyclictest9696-21kworker/0:006:31:350
251759944,0cyclictest9696-21kworker/0:006:26:090
251759944,0cyclictest23515-21kworker/0:111:37:200
251759944,0cyclictest23515-21kworker/0:111:13:340
251759944,0cyclictest23515-21kworker/0:110:51:220
251759944,0cyclictest23515-21kworker/0:110:36:220
251759944,0cyclictest23515-21kworker/0:109:30:400
251759944,0cyclictest23515-21kworker/0:109:05:340
251759944,0cyclictest23515-21kworker/0:107:55:370
251759944,0cyclictest22499-21kworker/0:207:50:370
251759944,0cyclictest22499-21kworker/0:206:47:110
251759943,0cyclictest7109-21awk06:55:500
251759943,0cyclictest3960-21awk09:55:510
251759943,0cyclictest3262-21irqbalance10:30:360
251759943,0cyclictest3262-21irqbalance10:15:360
251759943,0cyclictest3262-21irqbalance10:11:560
251759943,0cyclictest3262-21irqbalance08:15:460
259940,0migration/232226-21kthreadcore07:10:532
259940,0migration/225878-21munin-node08:10:532
259940,0migration/225605-21kthreadcore06:45:512
259940,0migration/225504-21aten_r1power_po06:20:442
259940,0migration/225318-21latency_hist07:30:322
259940,0migration/220195-21taskset07:02:502
259940,0migration/215689-21fschecks_count07:45:462
249940,0watchdog/223079-31sleep06:43:372
249940,0watchdog/21891-21kthreadcore07:35:522
249940,0watchdog/20-21swapper/206:35:372
179940,0migration/18363-21fschecks_count07:40:461
179940,0migration/1648-21missed_timers08:15:521
179940,0migration/1318-21irqcore09:15:511
179940,0migration/12968-21kthreadcore06:50:541
179940,0migration/128857-21kthreadcore06:20:501
179940,0migration/118856-21kthreadcore07:45:511
179940,0migration/118697-21fschecks_time09:45:461
179940,0migration/115898-21latency_hist09:25:321
179940,0migration/114534-21kthreadcore11:15:501
179940,0migration/113361-21kthreadcore10:00:511
179940,0migration/112485-21kthreadcore09:40:511
179940,0migration/111945-21runrttasks10:17:231
169940,0watchdog/16789-21cut08:20:521
169940,0watchdog/13217-21ssh10:12:241
169940,0watchdog/130904-21grep09:50:521
169940,0watchdog/130703-21memory09:30:521
169940,0watchdog/119527-21python10:20:561
169940,0watchdog/10-21swapper/111:25:521
169940,0watchdog/10-21swapper/109:10:481
169940,0watchdog/10-21swapper/109:05:521
169940,0watchdog/10-21swapper/107:55:521
169940,0watchdog/10-21swapper/107:35:481
149940,0migration/09842-21kthreadcore09:20:520
149940,0migration/027931-21ksmtuned11:41:030
149940,0migration/022044-21kthreadcore11:20:500
149940,0migration/017651-21kthreadcore09:25:530
149940,0migration/016478-21kthreadcore10:40:510
149940,0migration/014408-21kthreadcore11:15:500
149940,0migration/01343-21kthreadcore09:15:510
251779933,0cyclictest3599-21lldpd06:26:042
251779932,0cyclictest3599-21lldpd07:26:532
251779932,0cyclictest31961-21awk06:50:512
251769933,0cyclictest1899-21awk09:35:501
251769932,0cyclictest7042-21awk09:00:531
251769932,0cyclictest4711-21awk11:30:491
251769932,0cyclictest3599-21lldpd11:40:421
259930,0migration/225264-21kthreadcore07:05:572
259930,0migration/212597-21kthreadcore07:20:512
249930,0watchdog/24205-21basename07:15:482
169930,0watchdog/11828-21diskmemload08:35:401
169930,0watchdog/10-21swapper/110:40:441
169930,0watchdog/10-21swapper/108:32:081
169930,0watchdog/10-21swapper/107:30:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional