You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-11 - 02:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Tue Feb 11, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351480,0ktimersoftd/315715-21perf18:36:553
2246023826,0sleep10-21swapper/118:32:501
2255023625,0sleep00-21swapper/018:33:530
2246623425,0sleep20-21swapper/218:32:552
329980,0watchdog/324662-21cut20:47:253
329970,0watchdog/323738-21cpu20:07:173
329970,0watchdog/30-21swapper/321:33:453
29925070,0irq/32-eth0-rx-2501-21cut21:22:043
29925070,0irq/32-eth0-rx-0-21swapper/323:17:263
29925070,0irq/32-eth0-rx-0-21swapper/322:17:193
29925070,0irq/32-eth0-rx-0-21swapper/322:11:403
339960,0migration/329622-21kthreadcore22:47:243
339960,0migration/328841-21sh20:53:163
329960,0watchdog/30-21swapper/323:57:213
329960,0watchdog/30-21swapper/323:47:333
329960,0watchdog/30-21swapper/323:37:093
329960,0watchdog/30-21swapper/323:22:173
329960,0watchdog/30-21swapper/323:13:053
329960,0watchdog/30-21swapper/323:07:213
329960,0watchdog/30-21swapper/322:58:333
329960,0watchdog/30-21swapper/322:52:053
329960,0watchdog/30-21swapper/322:37:053
329960,0watchdog/30-21swapper/322:30:293
329960,0watchdog/30-21swapper/321:42:213
329960,0watchdog/30-21swapper/321:02:053
29925060,0irq/32-eth0-rx-20545-21grep23:27:253
29925060,0irq/32-eth0-rx-0-21swapper/323:52:153
29925060,0irq/32-eth0-rx-0-21swapper/323:42:263
29925060,0irq/32-eth0-rx-0-21swapper/323:33:103
29925060,0irq/32-eth0-rx-0-21swapper/322:05:393
29925060,0irq/32-eth0-rx-0-21swapper/321:27:213
29925060,0irq/32-eth0-rx-0-21swapper/320:42:583
29925060,0irq/32-eth0-rx-0-21swapper/300:04:093
259960,0migration/213601-21sh21:49:012
228949960,0cyclictest18994-21processes20:22:272
228949960,0cyclictest0-21swapper/222:42:372
149960,0migration/06864-21idleruntime-cro21:02:050
149960,0migration/04667-21sh20:12:300
149960,0migration/04363-21smart_sda19:27:270
149960,0migration/022776-21taskset23:31:450
149960,0migration/019883-21sh23:57:110
149960,0migration/018626-21sh23:53:570
149960,0migration/016127-21sh21:55:590
4150,0ktimersoftd/00-21swapper/020:38:430
339950,0migration/331556-21hddtemp_smartct20:57:173
339950,0migration/331266-21hddtemp_smartct18:42:213
339950,0migration/329306-21kthreadcore19:47:243
339950,0migration/327319-21gltestperf21:17:143
339950,0migration/32640-21munin-node19:07:213
339950,0migration/310074-21gltestperf19:57:223
329950,0watchdog/315210-21runrttasks19:37:253
329950,0watchdog/30-21swapper/323:02:173
329950,0watchdog/30-21swapper/322:13:453
329950,0watchdog/30-21swapper/321:57:053
329950,0watchdog/30-21swapper/321:52:373
329950,0watchdog/30-21swapper/321:37:293
329950,0watchdog/30-21swapper/321:12:053
329950,0watchdog/30-21swapper/321:07:053
329950,0watchdog/30-21swapper/320:37:133
329950,0watchdog/30-21swapper/320:33:333
329950,0watchdog/30-21swapper/320:27:213
329950,0watchdog/30-21swapper/320:22:253
329950,0watchdog/30-21swapper/320:17:453
329950,0watchdog/30-21swapper/320:12:093
329950,0watchdog/30-21swapper/320:03:013
329950,0watchdog/30-21swapper/319:43:453
329950,0watchdog/30-21swapper/319:27:453
329950,0watchdog/30-21swapper/319:25:253
329950,0watchdog/30-21swapper/319:17:533
329950,0watchdog/30-21swapper/319:12:213
329950,0watchdog/30-21swapper/319:02:413
329950,0watchdog/30-21swapper/318:57:333
329950,0watchdog/30-21swapper/318:55:293
329950,0watchdog/30-21swapper/318:50:253
329950,0watchdog/30-21swapper/318:37:133
29935050,0irq/33-eth0-tx-5367-21munin-node19:32:220
29935050,0irq/33-eth0-tx-32500-21ssh22:03:582
29935050,0irq/33-eth0-tx-30182-21diskstats23:17:172
29935050,0irq/33-eth0-tx-29106-21ssh20:54:012
29935050,0irq/33-eth0-tx-2752-21chrt00:02:482
29935050,0irq/33-eth0-tx-16840-21ntpdc23:02:262
29935050,0irq/33-eth0-tx-16372-21munin-node20:02:220
29935050,0irq/33-eth0-tx-1137-21munin-node23:42:262
29935050,0irq/33-eth0-tx-11297-21grep21:02:242
29935050,0irq/33-eth0-tx-0-21swapper/223:55:502
29935050,0irq/33-eth0-tx-0-21swapper/222:33:522
29935050,0irq/33-eth0-tx-0-21swapper/221:56:322
29935050,0irq/33-eth0-tx-0-21swapper/221:33:222
29935050,0irq/33-eth0-tx-0-21swapper/220:48:392
29935050,0irq/33-eth0-tx-0-21swapper/019:43:160
29935050,0irq/33-eth0-tx-0-21swapper/019:18:090
29925050,0irq/32-eth0-rx-0-21swapper/322:42:153
29925050,0irq/32-eth0-rx-0-21swapper/322:32:073
29925050,0irq/32-eth0-rx-0-21swapper/322:23:593
29925050,0irq/32-eth0-rx-0-21swapper/321:47:193
29925050,0irq/32-eth0-rx-0-21swapper/319:52:223
29925050,0irq/32-eth0-rx-0-21swapper/319:32:163
259950,0migration/2428-21sh23:40:552
259950,0migration/231098-21kthreadcore22:27:242
259950,0migration/223813-21sh21:11:142
259950,0migration/222354-21sh23:30:312
259950,0migration/21811-21munin-node22:07:182
259950,0migration/21793-21latency_hist19:52:052
249950,0watchdog/20-21swapper/220:17:252
228949955,0cyclictest0-21swapper/223:13:032
228949955,0cyclictest0-21swapper/222:55:232
228939955,0cyclictest0-21swapper/121:24:021
228939950,0cyclictest0-21swapper/123:26:131
228939950,0cyclictest0-21swapper/122:14:581
228939950,0cyclictest0-21swapper/121:41:511
228939950,0cyclictest0-21swapper/121:36:201
228929954,0cyclictest17590-21perl21:57:140
228929954,0cyclictest11016-21munin-run21:27:040
228929950,0cyclictest0-21swapper/022:56:120
228929950,0cyclictest0-21swapper/022:29:140
228929950,0cyclictest0-21swapper/022:23:050
228929950,0cyclictest0-21swapper/021:43:030
228929950,0cyclictest0-21swapper/021:11:310
228929950,0cyclictest0-21swapper/019:22:060
179950,0migration/16836-21kthreadcore22:57:181
179950,0migration/15939-21fschecks_time18:47:191
179950,0migration/130383-21fschecks_count23:17:181
179950,0migration/118793-21timerandwakeup18:52:311
179950,0migration/113990-21sh21:50:111
169950,0watchdog/10-21swapper/119:37:251
149950,0migration/09174-21ksmtuned23:46:300
149950,0migration/0810-21kthreadcore18:42:250
149950,0migration/03806-21kthreadcore19:07:230
149950,0migration/027689-21latency_hist23:12:040
149950,0migration/027346-21sh23:11:330
149950,0migration/026118-21kthreadcore20:07:220
149950,0migration/026001-21fschecks_time22:02:180
149950,0migration/025672-21kernelversion22:47:230
149950,0migration/025325-21kthreadcore18:37:230
149950,0migration/02348-21cpuspeed_turbos22:57:130
149950,0migration/023283-21kthreadcore20:27:230
149950,0migration/022625-21gltestperf21:37:170
149950,0migration/019200-21taskset22:19:130
149950,0migration/018407-21sh21:28:320
149950,0migration/017168-21taskset20:42:300
149950,0migration/015638-21diskmemload21:07:050
149950,0migration/012094-21needreboot23:22:270
149950,0migration/010933-21sh22:15:070
8315040,0irq/26-0000:00:0-21swapper/223:07:042
29935040,0irq/33-eth0-tx-30662-21hostname20:57:142
29935040,0irq/33-eth0-tx-26815-21ssh22:22:292
29935040,0irq/33-eth0-tx-0-21swapper/223:32:202
29935040,0irq/33-eth0-tx-0-21swapper/223:22:492
29935040,0irq/33-eth0-tx-0-21swapper/222:57:122
29935040,0irq/33-eth0-tx-0-21swapper/222:12:522
29935040,0irq/33-eth0-tx-0-21swapper/221:57:172
29935040,0irq/33-eth0-tx-0-21swapper/221:42:242
29935040,0irq/33-eth0-tx-0-21swapper/221:38:222
29935040,0irq/33-eth0-tx-0-21swapper/221:27:182
29935040,0irq/33-eth0-tx-0-21swapper/221:17:282
29935040,0irq/33-eth0-tx-0-21swapper/020:17:200
29935040,0irq/33-eth0-tx-0-21swapper/019:58:190
29935040,0irq/33-eth0-tx-0-21swapper/018:58:540
259940,0migration/28065-21kthreadcore21:22:242
259940,0migration/27772-21kthreadcore19:32:222
259940,0migration/27155-21aten_r1power_po22:37:122
259940,0migration/25722-21kthreadcore19:07:242
259940,0migration/24995-21ksmtuned19:30:282
259940,0migration/224257-21aten_r1power_vo22:47:182
259940,0migration/223259-21latency_hist19:22:052
259940,0migration/222767-21runrttasks19:18:352
259940,0migration/222497-21kthreadcore23:57:172
259940,0migration/22224-21kthreadcore18:42:252
259940,0migration/213354-21irqrtprio22:17:172
259940,0migration/211054-21cpuspeed_turbos20:42:182
259940,0migration/210861-21kthreadcore19:57:252
249940,0watchdog/26386-21runrttasks19:37:252
249940,0watchdog/230817-21kthreadcore20:32:252
249940,0watchdog/229559-21munin-node20:07:252
249940,0watchdog/20-21swapper/221:13:012
249940,0watchdog/20-21swapper/220:37:292
249940,0watchdog/20-21swapper/220:12:252
228949943,0cyclictest9966-21awk19:12:212
228949943,0cyclictest27889-21awk19:02:212
228949943,0cyclictest24238-21awk18:37:212
228949943,0cyclictest23871-21snmpd19:42:232
228949943,0cyclictest23871-21snmpd18:52:232
228949943,0cyclictest21181-21awk20:27:212
228949943,0cyclictest20722-21awk18:57:252
228949943,0cyclictest10859-21grep23:47:232
228939944,0cyclictest0-21swapper/122:40:021
228939944,0cyclictest0-21swapper/122:19:511
228939943,0cyclictest3599-21lldpd23:59:171
228939943,0cyclictest3599-21lldpd22:45:411
228939943,0cyclictest3599-21lldpd22:23:081
228939943,0cyclictest3599-21lldpd22:09:061
228939943,0cyclictest3599-21lldpd21:27:571
228939943,0cyclictest3262-21irqbalance23:07:081
228939943,0cyclictest3262-21irqbalance21:52:191
228939943,0cyclictest3262-21irqbalance21:42:091
228939943,0cyclictest3262-21irqbalance21:13:191
228939943,0cyclictest3262-21irqbalance21:07:191
228939943,0cyclictest3262-21irqbalance19:32:191
228939943,0cyclictest3262-21irqbalance19:23:291
228939943,0cyclictest3262-21irqbalance19:12:091
228939943,0cyclictest3262-21irqbalance19:07:291
228939943,0cyclictest3262-21irqbalance19:02:091
228939943,0cyclictest3262-21irqbalance18:38:491
228939943,0cyclictest31551-21awk19:27:201
228939943,0cyclictest31370-21awk18:42:221
228939943,0cyclictest26534-21awk22:02:201
228939943,0cyclictest23871-21snmpd20:29:531
228939943,0cyclictest20737-21awk18:57:251
228939943,0cyclictest17881-21awk21:57:161
228939943,0cyclictest15804-21awk23:27:221
228929944,0cyclictest0-21swapper/021:49:490
228929943,0cyclictest3599-21lldpd21:18:570
228929943,0cyclictest29471-21awk00:02:220
228929943,0cyclictest26278-21awk23:37:170
228929940,0cyclictest0-21swapper/022:45:480
179940,0migration/18321-21if_eth1.221:02:221
179940,0migration/18140-21munin-node19:52:261
179940,0migration/15274-21sh20:58:091
179940,0migration/14803-21sh22:31:511
179940,0migration/129034-21kthreadcore21:17:181
179940,0migration/125821-21kthreadcore20:07:221
179940,0migration/125801-21fschecks_time23:37:151
179940,0migration/122611-21munin-node20:02:301
179940,0migration/119966-21munin-node19:42:171
179940,0migration/119656-21munin-node20:47:171
179940,0migration/118120-21kthreadcore19:17:241
179940,0migration/115865-21proc_pri23:47:281
179940,0migration/112606-21kthreadcore23:02:231
169940,0watchdog/19902-21ssh20:40:211
169940,0watchdog/17243-21kthreadcore20:17:251
169940,0watchdog/13275-21cut20:12:251
169940,0watchdog/130417-21python22:47:291
169940,0watchdog/130168-21ssh20:56:411
169940,0watchdog/127054-21munin-node19:47:211
169940,0watchdog/117794-21ssh20:43:171
169940,0watchdog/113879-21hwlatdetect20:22:211
149940,0migration/09452-21kthreadcore22:37:170
149940,0migration/024637-21sh23:36:320
149940,0migration/021536-21latency_hist21:37:040
149940,0migration/015641-21kthreadcore23:02:240
149940,0migration/015560-21kthreadcore18:52:240
149940,0migration/014349-21irqrtprio20:22:230
29935030,0irq/33-eth0-tx-27474-21aten2.4-expect20:32:190
249930,0watchdog/20-21swapper/219:47:252
228949932,0cyclictest3599-21lldpd18:51:242
228949932,0cyclictest21025-21kthreadcore20:02:232
228939932,0cyclictest3599-21lldpd23:14:151
228939932,0cyclictest3599-21lldpd19:57:241
228939932,0cyclictest30308-21grep00:02:231
228939932,0cyclictest1870-21diskmemload22:35:311
228929933,0cyclictest4221-21kworker/0:219:12:190
228929933,0cyclictest4221-21kworker/0:219:02:090
228929933,0cyclictest4221-21kworker/0:218:47:170
228929933,0cyclictest29802-21kworker/0:122:33:510
228929933,0cyclictest29802-21kworker/0:121:14:350
228929933,0cyclictest29802-21kworker/0:120:52:290
228929932,0cyclictest6739-21kthreadcore19:52:220
228929932,0cyclictest3599-21lldpd19:47:230
228929932,0cyclictest29997-21cpuspeed_turbos23:17:170
228929932,0cyclictest29802-21kworker/0:122:07:190
228929932,0cyclictest22410-21grep20:47:240
228929930,0cyclictest12608-21munin-node19:37:230
169930,0watchdog/124757-21rm23:36:531
169930,0watchdog/10-21swapper/123:44:131
169930,0watchdog/10-21swapper/122:52:211
169930,0watchdog/10-21swapper/120:32:251
149930,0migration/016699-21sh23:48:370
169920,0watchdog/10-21swapper/123:52:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional