You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-26 - 01:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2s.osadl.org (updated Tue Jun 25, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
716423625,0sleep10-21swapper/106:56:011
709123526,0sleep20-21swapper/206:55:102
291350350,0irq/25-eth10-21swapper/306:53:493
709423321,0sleep00-21swapper/006:55:120
7447991010,0cyclictest28342-21sensors10:54:020
339980,0migration/321461-21munin-node09:08:513
28415070,0irq/27-eth0-tx-24945-21munin-node08:49:220
28415070,0irq/27-eth0-tx-0-21swapper/009:03:400
8225060,0irq/24-0000:00:8363-21latency_hist07:23:312
74489960,0cyclictest0-21swapper/110:55:211
74479960,0cyclictest0-21swapper/011:22:040
74479960,0cyclictest0-21swapper/011:02:030
339960,0migration/39256-21sh10:03:063
339960,0migration/33388-21sh11:20:183
339960,0migration/326345-21sh12:19:133
339960,0migration/32194-21sh12:03:103
339960,0migration/321594-21cpuspeed_turbos11:13:453
339960,0migration/321193-21sh10:50:133
339960,0migration/319463-21sh11:31:413
339960,0migration/310395-21sh11:46:053
28415060,0irq/27-eth0-tx-30301-21ssh10:34:212
28415060,0irq/27-eth0-tx-27718-21telnet07:13:460
28415060,0irq/27-eth0-tx-0-21swapper/212:24:362
28415060,0irq/27-eth0-tx-0-21swapper/212:13:032
28415060,0irq/27-eth0-tx-0-21swapper/211:46:522
28415060,0irq/27-eth0-tx-0-21swapper/211:29:302
28415060,0irq/27-eth0-tx-0-21swapper/211:25:372
28415060,0irq/27-eth0-tx-0-21swapper/211:18:442
28415060,0irq/27-eth0-tx-0-21swapper/211:15:582
28415060,0irq/27-eth0-tx-0-21swapper/211:01:592
28415060,0irq/27-eth0-tx-0-21swapper/210:22:232
28415060,0irq/27-eth0-tx-0-21swapper/210:09:572
28415060,0irq/27-eth0-tx-0-21swapper/209:52:442
28415060,0irq/27-eth0-tx-0-21swapper/209:44:262
28415060,0irq/27-eth0-tx-0-21swapper/209:36:432
28415060,0irq/27-eth0-tx-0-21swapper/209:13:552
28415060,0irq/27-eth0-tx-0-21swapper/009:02:110
28415060,0irq/27-eth0-tx-0-21swapper/008:20:130
28415060,0irq/27-eth0-tx-0-21swapper/007:38:460
28415060,0irq/27-eth0-tx-0-21swapper/007:03:430
28405060,0irq/26-eth0-rx-27434-21iostat09:53:593
28405060,0irq/26-eth0-rx-13326-21idleruntime-cro11:08:303
249960,0watchdog/232311-21grep07:39:012
249960,0watchdog/2138742chrt07:01:132
249960,0watchdog/210328-21sh09:42:492
249960,0watchdog/20-21swapper/210:53:052
249960,0watchdog/20-21swapper/209:33:052
149960,0migration/05632-21sh10:39:340
149960,0migration/03272-21sh09:16:280
149960,0migration/017340-21mdstat09:24:010
74499955,0cyclictest0-21swapper/208:58:492
74499950,0cyclictest0-21swapper/209:09:422
74489955,0cyclictest0-21swapper/108:21:191
74489953,0cyclictest5060-21awk09:18:531
74489953,0cyclictest3425-21lldpd08:11:561
74489950,0cyclictest0-21swapper/112:02:401
74489950,0cyclictest0-21swapper/111:04:581
74489950,0cyclictest0-21swapper/111:01:111
74489950,0cyclictest0-21swapper/110:14:121
74489950,0cyclictest0-21swapper/110:01:361
74479955,0cyclictest0-21swapper/011:23:480
74479955,0cyclictest0-21swapper/010:12:140
74479955,0cyclictest0-21swapper/008:34:000
74479950,0cyclictest0-21swapper/011:07:340
339950,0migration/33783-21rt-features10:59:123
339950,0migration/326639-21sendmail_mailtr07:09:093
339950,0migration/325753-21kthreadcore08:23:593
339950,0migration/325381-21kthreadcore07:59:013
339950,0migration/324561-21taskset08:46:563
339950,0migration/323455-21kthreadcore09:29:053
339950,0migration/322134-21sendmail_mailqu10:29:143
339950,0migration/316889-21kthreadcore07:04:023
339950,0migration/316491-21kthreadcore11:09:073
339950,0migration/315764-21kthreadcore08:13:543
339950,0migration/314879-21kthreadcore08:38:593
339950,0migration/313706-21kthreadcore11:48:533
339950,0migration/313014-21missed_timers06:58:553
339950,0migration/31274-21fschecks_time10:18:483
339950,0migration/310629-21aten_r1power_en12:08:453
28415050,0irq/27-eth0-tx-8218-21ps10:44:012
28415050,0irq/27-eth0-tx-6372-21iceccd08:13:570
28415050,0irq/27-eth0-tx-6372-21iceccd07:29:090
28415050,0irq/27-eth0-tx-26646-21ssh11:34:432
28415050,0irq/27-eth0-tx-2085-21ssh12:02:492
28415050,0irq/27-eth0-tx-16321-21munin-node10:29:042
28415050,0irq/27-eth0-tx-0-21swapper/212:15:092
28415050,0irq/27-eth0-tx-0-21swapper/212:04:432
28415050,0irq/27-eth0-tx-0-21swapper/211:54:442
28415050,0irq/27-eth0-tx-0-21swapper/211:52:092
28415050,0irq/27-eth0-tx-0-21swapper/211:09:442
28415050,0irq/27-eth0-tx-0-21swapper/210:40:072
28415050,0irq/27-eth0-tx-0-21swapper/210:17:352
28415050,0irq/27-eth0-tx-0-21swapper/210:07:142
28415050,0irq/27-eth0-tx-0-21swapper/209:54:232
28415050,0irq/27-eth0-tx-0-21swapper/008:53:420
28415050,0irq/27-eth0-tx-0-21swapper/008:09:160
28415050,0irq/27-eth0-tx-0-21swapper/007:54:010
28415050,0irq/27-eth0-tx-0-21swapper/007:33:470
28405050,0irq/26-eth0-rx-7635-21kthreadcore09:39:053
28405050,0irq/26-eth0-rx-5689-21idleruntime-cro08:58:303
28405050,0irq/26-eth0-rx-32642-21if_eth1.210:38:593
28405050,0irq/26-eth0-rx-24168-21cut10:53:533
28405050,0irq/26-eth0-rx-23771-21ssh10:09:523
28405050,0irq/26-eth0-rx-224652sleep307:56:593
28405050,0irq/26-eth0-rx-19145-21kthreadcore08:19:023
28405050,0irq/26-eth0-rx-0-21swapper/311:43:293
28405050,0irq/26-eth0-rx-0-21swapper/310:13:433
28405050,0irq/26-eth0-rx-0-21swapper/309:33:533
28405050,0irq/26-eth0-rx-0-21swapper/309:19:373
27150,0ktimersoftd/20-21swapper/210:54:532
27150,0ktimersoftd/20-21swapper/209:21:422
27150,0ktimersoftd/20-21swapper/208:05:422
259950,0migration/232299-21kthreadcore11:39:002
259950,0migration/21571-21kthreadcore08:53:552
249950,0watchdog/20-21swapper/210:23:492
249950,0watchdog/20-21swapper/209:24:172
249950,0watchdog/20-21swapper/208:13:572
249950,0watchdog/20-21swapper/208:09:292
249950,0watchdog/20-21swapper/207:23:492
179950,0migration/1835-21sendmail_mailqu07:14:091
179950,0migration/13994-21munin-node08:29:191
169950,0watchdog/132304-21fschecks_time08:53:451
149950,0migration/09355-21dump-pmu-power10:03:250
149950,0migration/06454-21kthreadcore09:18:590
149950,0migration/032586-21kthreadcore09:34:060
149950,0migration/031068-21switchtime10:14:030
149950,0migration/029511-21latency_hist08:03:300
149950,0migration/028914-21kthreadcore10:34:030
149950,0migration/025635-21smart_sdb09:29:200
149950,0migration/024193-21kthreadcore11:34:010
149950,0migration/019224-21fschecks_time09:48:500
149950,0migration/019121-21kthreadcore10:29:010
149950,0migration/018432-21aten_r1power_po08:43:450
149950,0migration/017775-21mdstat12:14:020
149950,0migration/014921-21sh10:24:510
149950,0migration/014596-21kthreadcore07:48:570
149950,0migration/012534-21kthreadcore10:04:010
149950,0migration/010845-21aten_r1power_cu09:43:420
74499943,0cyclictest6445-21snmpd09:03:442
74499943,0cyclictest6445-21snmpd08:38:442
74499943,0cyclictest6445-21snmpd08:23:442
74499943,0cyclictest31598-21awk08:28:592
74499943,0cyclictest3110-21irqbalance07:43:352
74499943,0cyclictest25807-21awk08:48:552
74499943,0cyclictest22758-21awk07:33:582
74489944,0cyclictest27082-21kworker/1:007:48:571
74489944,0cyclictest0-21swapper/111:14:351
74489943,0cyclictest3425-21lldpd11:45:541
74489943,0cyclictest3425-21lldpd11:08:341
74489943,0cyclictest3425-21lldpd09:06:191
74489943,0cyclictest3425-21lldpd07:23:491
74489943,0cyclictest3110-21irqbalance12:23:451
74489943,0cyclictest3110-21irqbalance11:57:551
74489943,0cyclictest3110-21irqbalance10:24:051
74489943,0cyclictest3110-21irqbalance07:35:151
74489943,0cyclictest3110-21irqbalance07:28:351
74489943,0cyclictest3110-21irqbalance07:18:451
74489943,0cyclictest25942-21awk08:49:021
74489943,0cyclictest24943-21awk08:23:551
74489943,0cyclictest21816-21awk07:08:531
74489943,0cyclictest17053-21awk07:53:571
74489943,0cyclictest1630-21awk10:18:521
74489943,0cyclictest12394-21awk11:48:511
74489940,0cyclictest0-21swapper/109:24:331
74479944,0cyclictest0-21swapper/012:27:200
74479944,0cyclictest0-21swapper/012:06:510
74479943,0cyclictest8578-21awk06:58:500
74479943,0cyclictest4562-21awk11:43:520
74479943,0cyclictest3425-21lldpd08:30:590
74479943,0cyclictest20227-21awk11:53:540
74479943,0cyclictest13405-21awk11:28:570
339940,0migration/39515-21taskset07:48:363
339940,0migration/33482-21kthreadcore08:29:043
339940,0migration/330829-21taskset10:36:363
339940,0migration/328570-21aten_r1power_en07:38:443
339940,0migration/319295-21aten_r1power_fr11:53:423
28415040,0irq/27-eth0-tx-1863-21munin-node07:18:470
28415040,0irq/27-eth0-tx-11621-21aten2.4-expect08:38:430
28415040,0irq/27-eth0-tx-0-21swapper/211:03:422
28415040,0irq/27-eth0-tx-0-21swapper/008:25:270
28405040,0irq/26-eth0-rx-0-21swapper/312:25:493
28405040,0irq/26-eth0-rx-0-21swapper/312:13:553
28405040,0irq/26-eth0-rx-0-21swapper/312:06:023
28405040,0irq/26-eth0-rx-0-21swapper/311:24:183
28405040,0irq/26-eth0-rx-0-21swapper/310:47:043
28405040,0irq/26-eth0-rx-0-21swapper/310:23:503
28405040,0irq/26-eth0-rx-0-21swapper/310:03:373
28405040,0irq/26-eth0-rx-0-21swapper/309:48:393
28405040,0irq/26-eth0-rx-0-21swapper/309:44:353
28405040,0irq/26-eth0-rx-0-21swapper/309:24:223
28405040,0irq/26-eth0-rx-0-21swapper/309:17:053
28405040,0irq/26-eth0-rx-0-21swapper/309:00:003
28405040,0irq/26-eth0-rx-0-21swapper/307:20:163
259940,0migration/225991-21switchtime12:19:072
259940,0migration/222393-21runrttasks07:56:082
259940,0migration/215593-21runrttasks07:49:392
249940,0watchdog/225361-21kthreadcore07:59:012
249940,0watchdog/223546-21grep07:09:012
249940,0watchdog/219632-21kthreadcore08:44:012
249940,0watchdog/218003-21which08:18:492
249940,0watchdog/20-21swapper/209:59:012
249940,0watchdog/20-21swapper/208:33:412
249940,0watchdog/20-21swapper/207:28:412
249940,0watchdog/20-21swapper/207:13:452
249940,0watchdog/20-21swapper/207:03:452
22631240,0sleep30-21swapper/311:34:013
179940,0migration/19702-21kthreadcore11:24:021
179940,0migration/19246-21kthreadcore08:59:011
179940,0migration/18956-21kthreadcore10:44:021
179940,0migration/17309-21kthreadcore08:34:001
179940,0migration/16873-21kthreadcore09:39:051
179940,0migration/1459-21irqrtprio10:39:021
179940,0migration/130128-21kthreadcore09:34:041
179940,0migration/128441-21cpuspeed_turbos11:38:481
179940,0migration/118416-21kthreadcore10:29:011
179940,0migration/116001-21irqrtprio10:48:521
179940,0migration/114881-21kthreadcore12:14:001
179940,0migration/114418-21kthreadcore08:13:541
179940,0migration/110770-21irqcore10:03:591
169940,0watchdog/15685-21diskmemload12:12:331
169940,0watchdog/120171-21ssh12:18:451
169940,0watchdog/118645-21irqrtprio10:08:531
169940,0watchdog/111973-21irqcore09:44:011
169940,0watchdog/10-21swapper/109:53:531
169940,0watchdog/10-21swapper/107:39:011
169940,0watchdog/10-21swapper/107:03:451
149940,0migration/029037-21runrttasks07:59:440
149940,0migration/01951-21runrttasks11:39:160
149940,0migration/011887-21kthreadcore07:24:080
74489933,0cyclictest3562-21awk12:03:481
74489932,0cyclictest3425-21lldpd09:09:201
74489932,0cyclictest3425-21lldpd08:03:561
74489932,0cyclictest3425-21lldpd07:43:511
74489932,0cyclictest23832-21awk07:58:551
74489932,0cyclictest21153-21awk11:33:541
74489932,0cyclictest20726-21awk09:28:581
74489931,0cyclictest12913-21kthreadcore08:38:591
74479933,0cyclictest3425-21lldpd12:09:330
74479932,0cyclictest8558-21sensors07:44:160
74479932,0cyclictest22236-21awk11:13:520
74479932,0cyclictest21890-21awk07:08:560
74479932,0cyclictest21110-21awk12:18:530
74479932,0cyclictest18924-21crond11:53:300
74479932,0cyclictest1513-21awk10:18:500
74479930,0cyclictest20643-21ssh10:49:120
339930,0migration/314881-21runrttasks07:27:233
28405030,0irq/26-eth0-rx-0-21swapper/309:03:433
28405030,0irq/26-eth0-rx-0-21swapper/308:49:053
28405030,0irq/26-eth0-rx-0-21swapper/308:33:463
28405030,0irq/26-eth0-rx-0-21swapper/308:04:303
28405030,0irq/26-eth0-rx-0-21swapper/307:43:433
28405030,0irq/26-eth0-rx-0-21swapper/307:33:593
28405030,0irq/26-eth0-rx-0-21swapper/307:28:453
28405030,0irq/26-eth0-rx-0-21swapper/307:13:463
179930,0migration/130058-21irqrtprio09:13:581
169930,0watchdog/10-21swapper/111:28:531
169930,0watchdog/10-21swapper/111:19:051
169930,0watchdog/10-21swapper/110:34:211
169930,0watchdog/10-21swapper/109:48:491
169930,0watchdog/10-21swapper/108:44:011
169930,0watchdog/10-21swapper/106:58:491
149930,0migration/020273-21sh11:10:390
74479921,0cyclictest26654-21cut09:53:480
74479921,0cyclictest0-21swapper/010:43:310
74479921,0cyclictest0-21swapper/009:08:400
74479920,0cyclictest0-21swapper/011:58:310
74479920,0cyclictest0-21swapper/009:38:310
28405020,0irq/26-eth0-rx-0-21swapper/308:09:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional