You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 17:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 25, 2024 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350440,0irq/25-eth10-21swapper/306:59:473
2874123525,0sleep20-21swapper/206:58:452
2886023228,0sleep00-21swapper/007:00:110
2865422723,0sleep10-21swapper/106:57:441
289869999,0cyclictest15098-21sensors11:36:122
28415070,0irq/27-eth0-tx-0-21swapper/211:02:432
8225060,0irq/24-0000:00:24468-21basename12:05:452
4160,0ktimersoftd/00-21swapper/011:41:410
339960,0migration/39161-21sh11:15:313
339960,0migration/37299-21kthreadcore07:56:033
339960,0migration/36913-21sh12:12:143
339960,0migration/331706-21sh11:47:113
339960,0migration/326714-21kthreadcore07:46:033
339960,0migration/319874-21sh10:21:013
339960,0migration/318399-21sh09:36:183
289869966,0cyclictest0-21swapper/208:41:062
289869960,0cyclictest0-21swapper/211:51:152
28415060,0irq/27-eth0-tx-0-21swapper/212:25:012
28415060,0irq/27-eth0-tx-0-21swapper/212:18:472
28415060,0irq/27-eth0-tx-0-21swapper/211:41:542
28415060,0irq/27-eth0-tx-0-21swapper/211:26:512
28415060,0irq/27-eth0-tx-0-21swapper/211:21:472
28415060,0irq/27-eth0-tx-0-21swapper/211:13:152
28415060,0irq/27-eth0-tx-0-21swapper/210:58:112
28415060,0irq/27-eth0-tx-0-21swapper/210:50:072
28415060,0irq/27-eth0-tx-0-21swapper/210:26:582
28415060,0irq/27-eth0-tx-0-21swapper/210:06:042
28415060,0irq/27-eth0-tx-0-21swapper/209:43:552
28415060,0irq/27-eth0-tx-0-21swapper/209:38:392
28415060,0irq/27-eth0-tx-0-21swapper/209:33:232
28415060,0irq/27-eth0-tx-0-21swapper/209:05:572
28415060,0irq/27-eth0-tx-0-21swapper/007:46:050
28405060,0irq/26-eth0-rx-9421-21python09:51:123
28405060,0irq/26-eth0-rx-30843-21kthreadcore09:46:083
28405060,0irq/26-eth0-rx-15168-21awk10:56:123
28405060,0irq/26-eth0-rx-0-21swapper/311:43:303
249960,0watchdog/231161-21kthreadcore07:01:042
249960,0watchdog/216736-21sed10:16:122
249960,0watchdog/20-21swapper/211:31:122
249960,0watchdog/20-21swapper/208:46:042
249960,0watchdog/20-21swapper/208:35:562
8225050,0irq/24-0000:00:17532-21latency_hist07:40:452
339950,0migration/39850-21kthreadcore12:16:063
339950,0migration/36623-21munin-node11:51:103
339950,0migration/34826-21aten_r1power_cu09:10:543
339950,0migration/34432-21kthreadcore10:31:033
339950,0migration/332540-21kthreadcore07:51:063
339950,0migration/329578-21latency12:26:073
339950,0migration/327972-21taskset09:02:533
339950,0migration/327444-21kthreadcore11:06:063
339950,0migration/325073-21runrttasks08:09:593
339950,0migration/324694-21sh12:05:493
339950,0migration/322008-21munin-node09:21:033
339950,0migration/32089-21latency_hist10:30:463
339950,0migration/320273-21fschecks_time08:30:563
339950,0migration/317835-21aten_r1power_fr07:40:553
339950,0migration/317599-21idleruntime12:01:033
339950,0migration/31702-21kthreadcore07:26:053
339950,0migration/315125-21kthreadcore07:36:073
339950,0migration/313936-21kthreadcore07:11:063
339950,0migration/311496-21kthreadcore08:46:043
339950,0migration/310253-21latency_hist10:35:463
289869955,0cyclictest0-21swapper/210:11:172
289859955,0cyclictest0-21swapper/111:01:111
289859954,0cyclictest4538-21munin-run09:10:451
289859950,0cyclictest0-21swapper/110:07:531
289859950,0cyclictest0-21swapper/109:29:161
289859950,0cyclictest0-21swapper/109:16:481
289849955,0cyclictest0-21swapper/011:20:050
289849955,0cyclictest0-21swapper/009:48:480
289849954,0cyclictest6445-21snmpd10:42:250
289849954,0cyclictest4798-21ps09:51:030
289849950,0cyclictest0-21swapper/011:27:110
289849950,0cyclictest0-21swapper/010:55:010
289849950,0cyclictest0-21swapper/009:06:330
28415050,0irq/27-eth0-tx-2247-21munin-node10:31:002
28415050,0irq/27-eth0-tx-0-21swapper/212:26:002
28415050,0irq/27-eth0-tx-0-21swapper/212:11:402
28415050,0irq/27-eth0-tx-0-21swapper/212:05:552
28415050,0irq/27-eth0-tx-0-21swapper/211:47:102
28415050,0irq/27-eth0-tx-0-21swapper/211:06:282
28415050,0irq/27-eth0-tx-0-21swapper/210:52:252
28415050,0irq/27-eth0-tx-0-21swapper/210:42:292
28415050,0irq/27-eth0-tx-0-21swapper/210:21:142
28415050,0irq/27-eth0-tx-0-21swapper/209:55:552
28415050,0irq/27-eth0-tx-0-21swapper/209:51:442
28415050,0irq/27-eth0-tx-0-21swapper/209:48:542
28415050,0irq/27-eth0-tx-0-21swapper/209:30:002
28415050,0irq/27-eth0-tx-0-21swapper/209:20:592
28415050,0irq/27-eth0-tx-0-21swapper/209:16:552
28415050,0irq/27-eth0-tx-0-21swapper/209:11:042
28405050,0irq/26-eth0-rx-9739-21diskstats11:35:593
28405050,0irq/26-eth0-rx-5037-21perl09:31:013
28405050,0irq/26-eth0-rx-3831-21ssh09:08:433
28405050,0irq/26-eth0-rx-0-21swapper/311:56:113
28405050,0irq/26-eth0-rx-0-21swapper/311:31:033
28405050,0irq/26-eth0-rx-0-21swapper/311:28:063
28405050,0irq/26-eth0-rx-0-21swapper/311:03:023
28405050,0irq/26-eth0-rx-0-21swapper/310:16:483
28405050,0irq/26-eth0-rx-0-21swapper/310:11:003
28405050,0irq/26-eth0-rx-0-21swapper/310:01:433
28405050,0irq/26-eth0-rx-0-21swapper/309:42:223
28405050,0irq/26-eth0-rx-0-21swapper/309:18:033
259950,0migration/29631-21idleruntime11:55:592
259950,0migration/212946-21latency_hist08:25:452
259950,0migration/210361-21sh11:16:002
249950,0watchdog/216531-21cut08:56:042
249950,0watchdog/20-21swapper/210:37:402
249950,0watchdog/20-21swapper/210:03:362
249950,0watchdog/20-21swapper/209:02:522
249950,0watchdog/20-21swapper/207:50:482
249950,0watchdog/20-21swapper/207:41:082
249950,0watchdog/20-21swapper/207:21:002
249950,0watchdog/20-21swapper/207:15:482
19150,0ktimersoftd/10-21swapper/111:41:411
17853250,0sleep018154-21kthreadcore07:16:080
179950,0migration/17943-21taskset11:53:581
179950,0migration/17269-21kthreadcore09:11:051
179950,0migration/127120-21runrttasks09:43:281
179950,0migration/122522-21switchtime07:16:151
179950,0migration/118436-21kthreadcore12:21:041
179950,0migration/116512-21kthreadcore08:56:041
169950,0watchdog/19001-21ssh10:54:481
169950,0watchdog/123159-21ssh11:21:081
149950,0migration/09679-21sh10:13:170
149950,0migration/05628-21idleruntime09:11:000
149950,0migration/0410-21latency_hist08:40:450
149950,0migration/032465-21sendmail_mailtr10:26:090
149950,0migration/029407-21unixbench_multi07:21:140
149950,0migration/02312-21timerwakeupswit09:26:190
149950,0migration/017424-21fschecks_time12:01:010
149950,0migration/012659-21irqcore09:56:040
149950,0migration/012305-21aten_r1power_en09:35:550
339940,0migration/35220-21kthreadcore08:41:073
339940,0migration/316548-21kthreadcore08:01:073
339940,0migration/315880-21irqrtprio08:56:043
289869943,0cyclictest6224-21awk07:56:002
289869940,0cyclictest0-21swapper/207:14:242
289859944,0cyclictest0-21swapper/111:56:591
289859943,0cyclictest6445-21snmpd12:28:411
289859943,0cyclictest6445-21snmpd10:41:101
289859943,0cyclictest6445-21snmpd10:26:101
289859943,0cyclictest6445-21snmpd10:22:551
289859943,0cyclictest5251-21awk07:30:591
289859943,0cyclictest4661-21tr09:51:031
289859943,0cyclictest388-21awk08:16:011
289859943,0cyclictest3170-21awk10:31:001
289859943,0cyclictest29214-21kworker/1:010:16:031
289859943,0cyclictest27219-21awk10:45:591
289859943,0cyclictest26628-21awk11:26:051
289859943,0cyclictest2028-21kworker/1:112:08:311
289859943,0cyclictest13967-21awk08:26:041
289859943,0cyclictest12557-21awk09:56:031
289859943,0cyclictest1197-21awk12:11:041
289849944,0cyclictest0-21swapper/011:53:300
289849944,0cyclictest0-21swapper/008:50:470
289849943,0cyclictest6445-21snmpd12:25:560
289849943,0cyclictest6445-21snmpd12:17:400
289849943,0cyclictest6445-21snmpd11:06:250
289849943,0cyclictest6445-21snmpd10:33:100
289849943,0cyclictest6445-21snmpd10:16:100
289849943,0cyclictest6445-21snmpd09:16:240
289849943,0cyclictest6445-21snmpd08:04:230
289849943,0cyclictest6445-21snmpd07:03:530
289849943,0cyclictest6305-21awk07:56:010
289849943,0cyclictest3425-21lldpd11:56:300
289849943,0cyclictest3425-21lldpd10:36:530
289849943,0cyclictest3425-21lldpd07:44:140
289849943,0cyclictest27117-21awk10:45:590
289849943,0cyclictest26151-21awk11:46:000
289849943,0cyclictest21796-21awk09:21:020
289849943,0cyclictest2170-21awk11:31:040
289849943,0cyclictest17296-21awk12:21:020
289849943,0cyclictest14634-21munin-node08:55:580
289849940,0cyclictest31043-21munin-node07:51:050
289849940,0cyclictest17373-21idleruntime-cro11:00:460
28415040,0irq/27-eth0-tx-7438-21telnet08:45:560
28415040,0irq/27-eth0-tx-6445-21snmpd08:08:390
28415040,0irq/27-eth0-tx-0-21swapper/007:11:110
28405040,0irq/26-eth0-rx-27686-40yum-updatesd-he07:21:063
28405040,0irq/26-eth0-rx-20030-40yum-updatesd-he11:21:073
28405040,0irq/26-eth0-rx-0-21swapper/312:21:053
28405040,0irq/26-eth0-rx-0-21swapper/311:16:053
28405040,0irq/26-eth0-rx-0-21swapper/310:53:073
28405040,0irq/26-eth0-rx-0-21swapper/310:45:493
28405040,0irq/26-eth0-rx-0-21swapper/310:42:083
28405040,0irq/26-eth0-rx-0-21swapper/310:07:083
28405040,0irq/26-eth0-rx-0-21swapper/309:56:133
28405040,0irq/26-eth0-rx-0-21swapper/308:26:063
28405040,0irq/26-eth0-rx-0-21swapper/308:15:313
259940,0migration/26640-21kthreadcore07:31:032
259940,0migration/223530-21kthreadcore08:31:012
249940,0watchdog/24159-21perl07:06:002
249940,0watchdog/20-21swapper/208:21:082
249940,0watchdog/20-21swapper/208:15:562
249940,0watchdog/20-21swapper/208:10:562
249940,0watchdog/20-21swapper/208:06:122
249940,0watchdog/20-21swapper/208:02:402
249940,0watchdog/20-21swapper/207:46:042
249940,0watchdog/20-21swapper/207:30:322
179940,0migration/18012-21fschecks_time08:45:591
179940,0migration/17428-21unixbench_multi11:31:161
179940,0migration/1681-21munin-node08:40:591
179940,0migration/16626-21cpuspeed_turbos08:20:561
179940,0migration/126721-21timerandwakeup09:21:121
179940,0migration/121216-21latency_hist09:00:461
179940,0migration/117570-21hwlatdetect12:01:031
179940,0migration/114666-21kthreadcore09:36:041
179940,0migration/114579-21taskset12:16:241
179940,0migration/114105-21kthreadcore10:36:061
179940,0migration/110689-21kthreadcore11:36:051
169940,0watchdog/16547-21runrttasks07:13:521
169940,0watchdog/16366-21kthreadcore10:11:041
169940,0watchdog/131162-21kthreadcore07:01:041
169940,0watchdog/120557-21cut10:01:041
169940,0watchdog/120141-21fschecks_count08:30:561
169940,0watchdog/111341-21kthreadcore11:16:041
169940,0watchdog/10-21swapper/111:46:041
169940,0watchdog/10-21swapper/111:10:561
169940,0watchdog/10-21swapper/111:09:081
169940,0watchdog/10-21swapper/110:56:121
169940,0watchdog/10-21swapper/108:35:561
169940,0watchdog/10-21swapper/107:40:561
169940,0watchdog/10-21swapper/107:26:041
149940,0migration/05621-21kthreadcore11:11:020
149940,0migration/029576-21kthreadcore12:06:090
149940,0migration/025830-21sh09:41:080
149940,0migration/024259-21kthreadcore08:31:010
149940,0migration/023107-21kthreadcore09:01:040
149940,0migration/019789-21kthreadcore11:01:040
149940,0migration/016644-21kthreadcore08:26:070
149940,0migration/013699-21kthreadcore11:36:060
339930,0migration/39705-21runrttasks07:08:173
289869930,0cyclictest13991-21runrttasks08:51:182
289859933,0cyclictest29214-21kworker/1:009:30:511
289859933,0cyclictest29214-21kworker/1:008:51:191
289859933,0cyclictest29214-21kworker/1:008:10:571
289859933,0cyclictest29214-21kworker/1:007:52:331
289859933,0cyclictest29214-21kworker/1:007:06:091
289859933,0cyclictest12089-21awk07:36:041
289859932,0cyclictest6209-21awk07:56:001
289859932,0cyclictest24432-21awk07:21:021
289859932,0cyclictest19612-21awk08:06:021
289859932,0cyclictest15171-21kthreadcore08:01:061
289849932,0cyclictest825-21grep12:11:020
289849932,0cyclictest7225-21awk08:21:000
289849932,0cyclictest4409-21awk07:06:040
289849932,0cyclictest419-21awk08:16:010
289849932,0cyclictest32741-21cat07:26:050
289849932,0cyclictest26224-21awk08:11:020
289849932,0cyclictest21455-21kthreadcore11:21:060
289849932,0cyclictest20693-21irqrtprio10:01:040
289849932,0cyclictest19418-21head10:20:570
289849932,0cyclictest1550-21awk08:41:040
289849930,0cyclictest4757-21munin-node09:30:580
289849930,0cyclictest28357-21interrupts10:06:020
289849930,0cyclictest11103-21munin-node07:36:050
28405030,0irq/26-eth0-rx-0-21swapper/309:25:553
28405030,0irq/26-eth0-rx-0-21swapper/308:53:473
28405030,0irq/26-eth0-rx-0-21swapper/308:20:543
28405030,0irq/26-eth0-rx-0-21swapper/307:33:223
28405030,0irq/26-eth0-rx-0-21swapper/307:15:583
169930,0watchdog/130346-21kthreadcore09:46:081
169930,0watchdog/10-21swapper/107:45:561
289849920,0cyclictest4409-21taskset07:30:520
28405020,0irq/26-eth0-rx-0-21swapper/308:35:593
28405020,0irq/26-eth0-rx-0-21swapper/308:15:583
28405020,0irq/26-eth0-rx-0-21swapper/307:00:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional