You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-25 - 07:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sat Jan 25, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382424544,0sleep00-21swapper/018:50:220
291350440,0irq/25-eth10-21swapper/318:49:193
369623626,0sleep20-21swapper/218:48:562
366523425,0sleep10-21swapper/118:48:311
39229965,0cyclictest16372-21irqcore20:40:552
39229960,0cyclictest0-21swapper/221:45:222
39209960,0cyclictest0-21swapper/021:49:250
339960,0migration/38867-21kthreadcore23:00:583
339960,0migration/327462-21sh22:27:293
339960,0migration/324918-21sh23:40:343
28415060,0irq/27-eth0-tx-7126-21munin-node20:06:000
28415060,0irq/27-eth0-tx-26123-21munin-node19:55:570
28415060,0irq/27-eth0-tx-18598-21telnet19:25:520
28415060,0irq/27-eth0-tx-0-21swapper/223:55:522
28415060,0irq/27-eth0-tx-0-21swapper/223:36:422
28415060,0irq/27-eth0-tx-0-21swapper/223:32:212
28415060,0irq/27-eth0-tx-0-21swapper/223:21:392
28415060,0irq/27-eth0-tx-0-21swapper/223:05:472
28415060,0irq/27-eth0-tx-0-21swapper/223:04:342
28415060,0irq/27-eth0-tx-0-21swapper/222:44:212
28415060,0irq/27-eth0-tx-0-21swapper/222:30:562
28415060,0irq/27-eth0-tx-0-21swapper/222:13:432
28415060,0irq/27-eth0-tx-0-21swapper/222:06:382
28415060,0irq/27-eth0-tx-0-21swapper/221:47:362
28415060,0irq/27-eth0-tx-0-21swapper/221:37:172
28415060,0irq/27-eth0-tx-0-21swapper/221:19:002
28415060,0irq/27-eth0-tx-0-21swapper/221:09:502
28415060,0irq/27-eth0-tx-0-21swapper/220:57:102
28415060,0irq/27-eth0-tx-0-21swapper/200:16:522
28415060,0irq/27-eth0-tx-0-21swapper/200:13:302
28415060,0irq/27-eth0-tx-0-21swapper/020:35:080
28415060,0irq/27-eth0-tx-0-21swapper/019:44:590
28415060,0irq/27-eth0-tx-0-21swapper/018:59:510
28405060,0irq/26-eth0-rx-3762-21cut23:20:583
28405060,0irq/26-eth0-rx-24623-21port_sshpriv22:21:003
249960,0watchdog/2170692sleep218:58:162
249960,0watchdog/20-21swapper/222:48:592
249960,0watchdog/20-21swapper/221:27:192
149960,0migration/031456-21sh23:41:200
149960,0migration/010839-21sh20:56:490
39229955,0cyclictest28462-21diskmemload22:02:242
39229955,0cyclictest0-21swapper/222:25:352
39229955,0cyclictest0-21swapper/200:02:322
39219955,0cyclictest3062-21kworker/1:122:56:501
39219955,0cyclictest0-21swapper/123:27:161
39219955,0cyclictest0-21swapper/123:10:151
39219955,0cyclictest0-21swapper/122:52:161
39219954,0cyclictest3062-21kworker/1:123:17:121
39219954,0cyclictest3062-21kworker/1:121:54:441
39209955,0cyclictest0-21swapper/022:21:160
39209955,0cyclictest0-21swapper/020:55:010
39209950,0cyclictest0-21swapper/023:38:590
39209950,0cyclictest0-21swapper/022:37:390
39209950,0cyclictest0-21swapper/022:28:210
39209950,0cyclictest0-21swapper/021:40:260
39209950,0cyclictest0-21swapper/000:18:340
35150,0ktimersoftd/30-21swapper/300:08:323
339950,0migration/36523-21kthreadcore18:50:563
339950,0migration/34319-21latency_hist22:35:403
339950,0migration/34183-21switchtime19:11:083
339950,0migration/33211-21aten_r1power_en00:15:513
339950,0migration/331097-21kthreadcore00:10:543
339950,0migration/328819-21taskset22:52:163
339950,0migration/328203-21taskset20:47:503
339950,0migration/327186-21kthreadcore22:01:013
339950,0migration/326823-21gltestperf19:55:553
339950,0migration/326446-21munin-node20:16:073
339950,0migration/326260-21munin-node23:40:593
339950,0migration/324851-21needreboot19:51:043
339950,0migration/324851-21needreboot19:51:043
339950,0migration/32449-21kthreadcore22:55:583
339950,0migration/323899-21missed_timers00:00:593
339950,0migration/32367-21sh23:50:353
339950,0migration/323391-21unixbench-2d19:01:053
339950,0migration/321101-21taskset21:07:323
339950,0migration/317978-21pmu-power22:40:593
339950,0migration/317906-21munin-node21:01:003
339950,0migration/316174-21idleruntime20:40:543
339950,0migration/314472-21taskset21:26:063
339950,0migration/312048-21munin-node19:20:573
339950,0migration/310830-21kthreadcore20:05:573
28415050,0irq/27-eth0-tx-6568-21cpu21:20:522
28415050,0irq/27-eth0-tx-2672-21munin-node23:50:582
28415050,0irq/27-eth0-tx-23767-21munin-node23:15:582
28415050,0irq/27-eth0-tx-17723-21unixbench_multi19:21:080
28415050,0irq/27-eth0-tx-0-21swapper/223:11:182
28415050,0irq/27-eth0-tx-0-21swapper/222:59:192
28415050,0irq/27-eth0-tx-0-21swapper/222:37:202
28415050,0irq/27-eth0-tx-0-21swapper/221:56:292
28415050,0irq/27-eth0-tx-0-21swapper/221:35:052
28415050,0irq/27-eth0-tx-0-21swapper/200:06:012
28415050,0irq/27-eth0-tx-0-21swapper/020:15:590
28415050,0irq/27-eth0-tx-0-21swapper/020:00:550
28405050,0irq/26-eth0-rx-8806-21aten_r1power_fr20:35:553
28405050,0irq/26-eth0-rx-6445-21snmpd23:30:533
28405050,0irq/26-eth0-rx-5820-21unixbench-2d21:41:053
28405050,0irq/26-eth0-rx-31665-21munin-node21:15:553
28405050,0irq/26-eth0-rx-29971-21sensors21:36:023
28405050,0irq/26-eth0-rx-22810-21ps21:32:153
28405050,0irq/26-eth0-rx-16623-21unixbench_multi23:56:033
28405050,0irq/26-eth0-rx-12890-40yum-updatesd-he21:22:023
28405050,0irq/26-eth0-rx-0-21swapper/323:50:513
28405050,0irq/26-eth0-rx-0-21swapper/323:08:053
28405050,0irq/26-eth0-rx-0-21swapper/321:13:243
28405050,0irq/26-eth0-rx-0-21swapper/320:51:163
27150,0ktimersoftd/20-21swapper/220:05:442
259950,0migration/27749-21sh23:26:512
259950,0migration/25107-21kthreadcore20:00:592
259950,0migration/225660-21latency20:16:002
259950,0migration/225205-21kthreadcore21:10:542
259950,0migration/224918-21ntp_kernel_pll_19:51:052
259950,0migration/224918-21ntp_kernel_pll_19:51:052
249950,0watchdog/20-21swapper/223:47:112
249950,0watchdog/20-21swapper/223:40:512
249950,0watchdog/20-21swapper/222:52:592
249950,0watchdog/20-21swapper/222:26:272
249950,0watchdog/20-21swapper/222:17:432
249950,0watchdog/20-21swapper/221:51:232
249950,0watchdog/20-21swapper/220:52:312
249950,0watchdog/20-21swapper/220:29:552
249950,0watchdog/20-21swapper/220:21:152
249950,0watchdog/20-21swapper/219:57:392
249950,0watchdog/20-21swapper/219:40:152
249950,0watchdog/20-21swapper/219:22:122
249950,0watchdog/20-21swapper/219:15:522
249950,0watchdog/20-21swapper/219:14:562
249950,0watchdog/20-21swapper/219:00:522
249950,0watchdog/20-21swapper/218:52:122
179950,0migration/14949-21fschecks_time20:55:531
179950,0migration/132309-21sh22:02:551
179950,0migration/1320-21kthreadcore19:11:021
179950,0migration/130616-21sh21:15:451
179950,0migration/13036-21latency_hist00:15:411
179950,0migration/121480-21kthreadcore21:55:571
179950,0migration/117709-21kthreadcore21:00:591
169950,0watchdog/120750-21cut19:01:001
149950,0migration/07842-21taskset23:27:090
149950,0migration/07227-21idleruntime21:20:550
149950,0migration/05965-21sh23:22:150
149950,0migration/05557-21fschecks_count19:15:520
149950,0migration/031452-21fschecks_time19:10:590
149950,0migration/029419-21munin-node23:16:020
149950,0migration/027367-21sh00:09:400
149950,0migration/025071-21if_eth019:05:550
149950,0migration/025037-21latency_hist19:30:410
149950,0migration/024047-21kthreadcore20:45:580
149950,0migration/023239-21pmu-power21:56:000
149950,0migration/022580-21sh23:13:000
149950,0migration/021028-21sh22:47:560
149950,0migration/013726-21kthreadcore22:15:570
149950,0migration/01366-21taskset20:26:180
149950,0migration/013655-21kthreadcore23:30:550
149950,0migration/010130-21kthreadcore20:36:000
39229944,0cyclictest9856-21kworker/2:120:45:412
39229944,0cyclictest9856-21kworker/2:120:35:412
39229944,0cyclictest9856-21kworker/2:119:42:132
39229944,0cyclictest9856-21kworker/2:119:05:572
39229943,0cyclictest6445-21snmpd21:00:522
39219944,0cyclictest3062-21kworker/1:123:45:581
39219944,0cyclictest3062-21kworker/1:123:00:421
39219944,0cyclictest3062-21kworker/1:122:45:541
39219944,0cyclictest3062-21kworker/1:122:30:481
39219944,0cyclictest3062-21kworker/1:122:27:161
39219944,0cyclictest3062-21kworker/1:122:17:561
39219944,0cyclictest3062-21kworker/1:121:47:501
39219944,0cyclictest3062-21kworker/1:121:31:011
39219944,0cyclictest3062-21kworker/1:120:36:011
39219944,0cyclictest3062-21kworker/1:120:27:451
39219944,0cyclictest3062-21kworker/1:120:22:011
39219944,0cyclictest3062-21kworker/1:120:00:531
39219944,0cyclictest3062-21kworker/1:119:25:551
39219944,0cyclictest3062-21kworker/1:119:16:311
39219944,0cyclictest3062-21kworker/1:118:56:351
39219944,0cyclictest3062-21kworker/1:118:50:551
39219944,0cyclictest19255-21kworker/1:200:11:201
39219944,0cyclictest19255-21kworker/1:200:10:081
39219944,0cyclictest0-21swapper/121:26:121
39219943,0cyclictest6445-21snmpd23:35:531
39219943,0cyclictest6445-21snmpd23:31:081
39219943,0cyclictest6445-21snmpd21:21:071
39219943,0cyclictest6445-21snmpd21:06:071
39219943,0cyclictest32635-21awk19:35:571
39219943,0cyclictest3062-21kworker/1:122:37:321
39219943,0cyclictest3062-21kworker/1:122:13:341
39219943,0cyclictest3062-21kworker/1:119:56:031
39219943,0cyclictest20291-21awk19:51:001
39219943,0cyclictest20291-21awk19:51:001
39219943,0cyclictest13240-21awk22:40:541
39219943,0cyclictest12680-21awk19:21:001
39219942,0cyclictest3062-21kworker/1:121:11:011
39209944,0cyclictest0-21swapper/023:55:040
39209944,0cyclictest0-21swapper/022:13:510
39209944,0cyclictest0-21swapper/022:06:080
39209944,0cyclictest0-21swapper/021:31:080
39209944,0cyclictest0-21swapper/021:20:150
39209944,0cyclictest0-21swapper/000:02:480
39209943,0cyclictest6445-21snmpd21:29:520
39209943,0cyclictest6445-21snmpd21:01:070
39209943,0cyclictest6445-21snmpd20:21:070
39209943,0cyclictest4966-21awk18:50:530
39209943,0cyclictest29763-21awk22:30:520
39209943,0cyclictest23078-21awk22:50:510
39209943,0cyclictest16232-21awk20:40:540
39209943,0cyclictest16147-21ntpdc23:55:590
39209940,0cyclictest30377-21idleruntime-cro21:15:400
339940,0migration/35821-21kthreadcore20:30:533
339940,0migration/327504-21kthreadcore19:05:573
339940,0migration/315834-21aten_r1power_fr23:10:483
339940,0migration/311497-21fschecks_time18:55:543
28415040,0irq/27-eth0-tx-0-21swapper/020:10:560
28415040,0irq/27-eth0-tx-0-21swapper/019:35:540
28405040,0irq/26-eth0-rx-10308-21fschecks_time21:50:503
28405040,0irq/26-eth0-rx-0-21swapper/323:27:503
28405040,0irq/26-eth0-rx-0-21swapper/323:17:353
28405040,0irq/26-eth0-rx-0-21swapper/322:46:183
28405040,0irq/26-eth0-rx-0-21swapper/322:32:083
28405040,0irq/26-eth0-rx-0-21swapper/322:15:423
28405040,0irq/26-eth0-rx-0-21swapper/322:11:153
28405040,0irq/26-eth0-rx-0-21swapper/322:07:133
28405040,0irq/26-eth0-rx-0-21swapper/321:56:173
28405040,0irq/26-eth0-rx-0-21swapper/321:46:243
28405040,0irq/26-eth0-rx-0-21swapper/320:55:563
249940,0watchdog/20-21swapper/220:30:472
249940,0watchdog/20-21swapper/220:10:432
249940,0watchdog/20-21swapper/219:45:552
249940,0watchdog/20-21swapper/219:25:562
179940,0migration/18890-21kthreadcore19:41:021
179940,0migration/15092-21proc_pri23:21:011
179940,0migration/131652-21kthreadcore20:51:021
179940,0migration/13133-21fschecks_count23:50:501
179940,0migration/129187-21kthreadcore19:05:581
179940,0migration/128116-21kthreadcore21:35:581
179940,0migration/127335-21runrttasks20:45:591
179940,0migration/12541-21fschecks_time20:30:501
179940,0migration/124575-21kthreadcore20:16:001
179940,0migration/123878-21missed_timers00:00:591
179940,0migration/119163-21runrttasks22:20:511
179940,0migration/114460-21kthreadcore19:45:551
179940,0migration/113548-21kthreadcore23:55:571
169940,0watchdog/18334-21cut20:05:551
169940,0watchdog/129507-21cut19:31:001
169940,0watchdog/126366-21irqcore23:40:591
169940,0watchdog/122219-21ssh23:11:551
169940,0watchdog/12154-21cut21:40:591
169940,0watchdog/120893-21port_sshpriv20:40:591
169940,0watchdog/11542-21ssh22:07:111
169940,0watchdog/10-21swapper/120:10:591
149940,0migration/02116-21sh00:12:510
149940,0migration/019845-21kthreadcore19:00:590
149940,0migration/013043-21sh23:04:080
39229933,0cyclictest9856-21kworker/2:119:30:432
39209933,0cyclictest0-21swapper/021:40:450
39209933,0cyclictest0-21swapper/021:08:440
39209932,0cyclictest6445-21snmpd23:07:230
39209932,0cyclictest26465-21awk22:00:590
39209932,0cyclictest15496-21ps22:40:560
39209932,0cyclictest10527-21awk21:50:510
28415030,0irq/27-eth0-tx-19509-21aten2.4-expect19:50:560
28415030,0irq/27-eth0-tx-19509-21aten2.4-expect19:50:560
28405030,0irq/26-eth0-rx-0-21swapper/320:21:353
28405030,0irq/26-eth0-rx-0-21swapper/320:10:573
28405030,0irq/26-eth0-rx-0-21swapper/320:04:423
28405030,0irq/26-eth0-rx-0-21swapper/319:40:563
28405030,0irq/26-eth0-rx-0-21swapper/319:36:063
28405030,0irq/26-eth0-rx-0-21swapper/319:25:473
28405030,0irq/26-eth0-rx-0-21swapper/319:15:413
149930,0migration/04616-21runrttasks22:57:540
39209921,0cyclictest0-21swapper/023:45:410
39209920,0cyclictest0-21swapper/019:45:410
28405020,0irq/26-eth0-rx-0-21swapper/320:26:033
28405020,0irq/26-eth0-rx-0-21swapper/319:45:503
28405020,0irq/26-eth0-rx-0-21swapper/319:31:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional