You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-10 - 02:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Mar 10, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:57:313
2417923523,0sleep00-21swapper/018:59:010
1807123526,0sleep20-21swapper/218:56:552
2346523425,0sleep10-21swapper/118:57:101
244909977,0cyclictest20907-21sensors20:32:110
4160,0ktimersoftd/00-21swapper/021:04:310
339960,0migration/35563-21sh22:10:173
339960,0migration/35335-21sh22:30:453
339960,0migration/322367-21sh21:40:323
339960,0migration/315217-21aten_r1power_vo21:17:013
339960,0migration/313305-21sh22:35:293
339960,0migration/312399-21sh23:15:063
29725060,0irq/27-eth0-tx-1779-21munin-node20:22:130
29715060,0irq/26-eth0-rx-6541-21ssh21:31:053
29715060,0irq/26-eth0-rx-3737-21users22:47:133
29715060,0irq/26-eth0-rx-19928-21ntp_offset22:17:093
27160,0ktimersoftd/20-21swapper/221:25:292
244919960,0cyclictest0-21swapper/119:37:061
244909965,0cyclictest30348-21python19:02:130
244909965,0cyclictest14295-21ps23:17:040
149960,0migration/05715-21sh21:09:360
149960,0migration/029384-21sh22:24:460
149960,0migration/021550-21sh22:41:220
149960,0migration/012687-21idleruntime-cro23:36:500
4150,0ktimersoftd/00-21swapper/000:00:200
35150,0ktimersoftd/30-21swapper/322:39:353
35150,0ktimersoftd/30-21swapper/322:01:443
339950,0migration/39366-21kthreadcore22:52:073
339950,0migration/38595-21aten_r1power_en20:27:023
339950,0migration/36996-21fschecks_time22:12:033
339950,0migration/35008-21kthreadcore19:32:073
339950,0migration/34252-21kthreadcore20:47:063
339950,0migration/332263-21if_eth119:07:053
339950,0migration/326940-21kthreadcore21:02:083
339950,0migration/32585-21munin-node23:07:073
339950,0migration/323380-21munin-node21:22:013
339950,0migration/323374-21kthreadcore20:12:073
339950,0migration/321506-21munin-node23:42:023
339950,0migration/320842-21munin-node19:47:053
339950,0migration/31910-21munin-node20:22:013
339950,0migration/319058-21munin-node23:17:113
339950,0migration/313992-21taskset20:06:333
339950,0migration/313902-21taskset21:14:303
339950,0migration/312332-21aten_r1power_cu19:17:003
339950,0migration/312276-21runrttasks23:56:293
339950,0migration/31039-21kthreadcore20:42:083
29725050,0irq/27-eth0-tx-4557-21ssh23:51:522
29725050,0irq/27-eth0-tx-31331-21munin-node19:07:070
29725050,0irq/27-eth0-tx-28758-21munin-node00:07:052
29725050,0irq/27-eth0-tx-26256-21grep00:02:082
29725050,0irq/27-eth0-tx-23308-21cat22:22:042
29725050,0irq/27-eth0-tx-12168-21ssh22:12:122
29725050,0irq/27-eth0-tx-0-21swapper/223:58:072
29725050,0irq/27-eth0-tx-0-21swapper/223:50:092
29725050,0irq/27-eth0-tx-0-21swapper/222:35:162
29725050,0irq/27-eth0-tx-0-21swapper/221:38:092
29725050,0irq/27-eth0-tx-0-21swapper/221:33:132
29725050,0irq/27-eth0-tx-0-21swapper/221:12:332
29725050,0irq/27-eth0-tx-0-21swapper/200:25:012
29725050,0irq/27-eth0-tx-0-21swapper/020:57:020
29725050,0irq/27-eth0-tx-0-21swapper/020:47:030
29725050,0irq/27-eth0-tx-0-21swapper/019:40:020
29715050,0irq/26-eth0-rx-5986-21ssh21:10:183
29715050,0irq/26-eth0-rx-4253-21ntp_offset21:47:093
29715050,0irq/26-eth0-rx-29894-21if_eth023:47:043
29715050,0irq/26-eth0-rx-20918-21python20:32:113
29715050,0irq/26-eth0-rx-11075-21cut21:52:073
29715050,0irq/26-eth0-rx-0-21swapper/323:36:513
29715050,0irq/26-eth0-rx-0-21swapper/323:24:523
29715050,0irq/26-eth0-rx-0-21swapper/323:04:333
29715050,0irq/26-eth0-rx-0-21swapper/322:43:573
29715050,0irq/26-eth0-rx-0-21swapper/322:02:323
29715050,0irq/26-eth0-rx-0-21swapper/320:17:083
29715050,0irq/26-eth0-rx-0-21swapper/319:52:063
29715050,0irq/26-eth0-rx-0-21swapper/300:30:403
29715050,0irq/26-eth0-rx-0-21swapper/300:03:163
259950,0migration/27455-21latency_hist20:01:502
259950,0migration/24132-21sendmail_mailqu22:07:112
259950,0migration/232600-21cpuspeed_turbos19:32:012
259950,0migration/232369-21munin-node21:27:052
259950,0migration/231651-21munin-node22:27:062
259950,0migration/230923-21latency_hist21:06:502
259950,0migration/220832-21sh22:18:102
259950,0migration/220017-21sh23:39:542
259950,0migration/213721-21fschecks_count23:17:022
244919955,0cyclictest0-21swapper/123:06:381
244919950,0cyclictest0-21swapper/123:58:301
244919950,0cyclictest0-21swapper/122:36:591
244919950,0cyclictest0-21swapper/122:08:371
244919950,0cyclictest0-21swapper/121:45:361
244919950,0cyclictest0-21swapper/121:22:161
244919950,0cyclictest0-21swapper/121:19:351
244919950,0cyclictest0-21swapper/121:08:511
244919950,0cyclictest0-21swapper/100:26:341
244919950,0cyclictest0-21swapper/100:20:061
244909955,0cyclictest20147-21sensors22:17:110
244909955,0cyclictest0-21swapper/022:58:230
244909950,0cyclictest0-21swapper/000:18:570
249950,0watchdog/22429-21ls23:27:072
249950,0watchdog/221866-21if_eth023:42:032
249950,0watchdog/220120-21missed_timers21:57:082
179950,0migration/13805-21switchtime22:27:131
179950,0migration/127765-21irqrtprio19:52:051
179950,0migration/121999-21irqrtprio00:02:051
179950,0migration/121746-21munin-node23:42:021
179950,0migration/11988-21kthreadcore21:27:071
179950,0migration/11920-21munin-node23:47:071
179950,0migration/118399-21missed_timers23:37:081
179950,0migration/115635-21fschecks_count20:32:031
179950,0migration/113597-21munin-node20:02:131
179950,0migration/111128-21munin-node19:12:081
149950,0migration/09556-21kthreadcore21:32:070
149950,0migration/09490-21kthreadcore23:52:060
149950,0migration/07597-21kthreadcore22:52:070
149950,0migration/032010-21kthreadcore20:17:080
149950,0migration/031649-21kthreadcore00:07:060
149950,0migration/030507-21aten_r1power_vo22:27:010
149950,0migration/029726-21gltestperf23:47:030
149950,0migration/028238-21latency_hist00:26:490
149950,0migration/028223-21kthreadcore19:52:060
149950,0migration/027606-21timerwakeupswit23:02:130
149950,0migration/021311-21taskset21:37:270
149950,0migration/020733-21kthreadcore19:22:060
149950,0migration/017579-21kthreadcore21:17:060
149950,0migration/015644-21kthreadcore20:07:090
149950,0migration/014920-21latency_hist21:16:500
149950,0migration/01470-21fschecks_time19:57:030
339940,0migration/331094-21kthreadcore23:27:043
339940,0migration/326715-21kthreadcore19:02:073
339940,0migration/322345-21diskmemload22:21:583
339940,0migration/321737-21kthreadcore19:22:063
339940,0migration/318388-21kthreadcore20:07:103
339940,0migration/316963-21gltestperf20:57:043
339940,0migration/314270-21kthreadcore00:17:053
339940,0migration/310659-21diskmemload00:12:343
339940,0migration/310491-21kthreadcore19:12:063
29725040,0irq/27-eth0-tx-29964-21munin-node22:47:092
29725040,0irq/27-eth0-tx-12364-21ssh00:16:592
29725040,0irq/27-eth0-tx-0-21swapper/223:12:222
29725040,0irq/27-eth0-tx-0-21swapper/223:07:572
29725040,0irq/27-eth0-tx-0-21swapper/222:36:552
29725040,0irq/27-eth0-tx-0-21swapper/222:03:062
29725040,0irq/27-eth0-tx-0-21swapper/221:52:072
29725040,0irq/27-eth0-tx-0-21swapper/200:26:512
29725040,0irq/27-eth0-tx-0-21swapper/020:51:590
29715040,0irq/26-eth0-rx-0-21swapper/323:56:503
29715040,0irq/26-eth0-rx-0-21swapper/323:32:273
29715040,0irq/26-eth0-rx-0-21swapper/322:56:563
29715040,0irq/26-eth0-rx-0-21swapper/321:42:103
29715040,0irq/26-eth0-rx-0-21swapper/321:32:363
29715040,0irq/26-eth0-rx-0-21swapper/300:21:523
29715040,0irq/26-eth0-rx-0-21swapper/300:07:383
259940,0migration/28130-21kthreadcore19:12:052
259940,0migration/27493-21kthreadcore23:32:062
259940,0migration/24496-21aten_r1power_po00:11:592
259940,0migration/225706-21sh19:02:062
259940,0migration/221145-21unixbench-2d20:32:122
259940,0migration/21580-21kthreadcore19:07:072
259940,0migration/215304-21taskset21:17:012
244929944,0cyclictest0-21swapper/221:48:052
244929943,0cyclictest7494-21awk19:37:052
244929943,0cyclictest3556-21lldpd20:47:022
244929943,0cyclictest3514-21snmpd19:49:162
244929943,0cyclictest3514-21snmpd19:24:152
244929943,0cyclictest3514-21snmpd19:18:452
244929943,0cyclictest3241-21irqbalance23:22:182
244929943,0cyclictest3241-21irqbalance23:02:182
244929943,0cyclictest3241-21irqbalance22:58:382
244929943,0cyclictest3241-21irqbalance21:41:582
244929943,0cyclictest28601-21basename20:42:012
244929943,0cyclictest26497-21awk19:27:032
244929943,0cyclictest22888-21awk22:42:042
244929943,0cyclictest17068-21awk20:57:052
244929943,0cyclictest15237-21awk20:07:082
244929940,0cyclictest14217-21munin-node19:42:062
244919943,0cyclictest6978-21awk22:32:031
244919943,0cyclictest3514-21snmpd23:22:031
244919943,0cyclictest3514-21snmpd22:18:481
244919943,0cyclictest3514-21snmpd20:20:161
244919943,0cyclictest3514-21snmpd20:16:161
244919943,0cyclictest3241-21irqbalance21:02:181
244919943,0cyclictest25637-21awk19:02:061
244919943,0cyclictest23262-21awk22:22:041
244919943,0cyclictest20843-21awk19:47:051
244909944,0cyclictest0-21swapper/023:30:330
244909943,0cyclictest6094-21sadc22:11:500
244909943,0cyclictest5304-21awk00:12:030
244909943,0cyclictest3556-21lldpd19:18:030
244909943,0cyclictest3514-21snmpd23:43:490
244909943,0cyclictest3514-21snmpd23:25:480
244909943,0cyclictest3514-21snmpd20:38:010
244909943,0cyclictest3241-21irqbalance22:47:380
244909943,0cyclictest3241-21irqbalance21:58:280
244909943,0cyclictest3241-21irqbalance20:27:080
244909943,0cyclictest3241-21irqbalance00:22:070
244909943,0cyclictest3241-21irqbalance00:02:570
244909943,0cyclictest23960-21awk21:42:040
244909940,0cyclictest0-21swapper/020:12:320
244909940,0cyclictest0-21swapper/019:27:010
249940,0watchdog/228116-21perl20:17:042
249940,0watchdog/226602-21switchtime20:12:122
249940,0watchdog/22624-21if_err_eth020:22:042
249940,0watchdog/223492-21perl21:02:042
249940,0watchdog/20-21swapper/222:52:082
249940,0watchdog/20-21swapper/220:52:042
249940,0watchdog/20-21swapper/220:37:042
179940,0migration/19574-21kthreadcore23:32:071
179940,0migration/18560-21aten_r1power_cu20:27:021
179940,0migration/16802-21kthreadcore20:47:071
179940,0migration/15229-21taskset23:12:001
179940,0migration/14575-21timerandwakeup21:47:121
179940,0migration/130651-21kthreadcore20:42:071
179940,0migration/129962-21df_abs23:07:011
179940,0migration/123614-21kthreadcore22:42:061
179940,0migration/120272-21taskset22:58:411
179940,0migration/115876-21kthreadcore19:17:071
179940,0migration/115076-21kthreadcore23:17:051
179940,0migration/114995-21cpuspeed_turbos21:57:011
179940,0migration/111645-21kthreadcore20:52:041
169940,0watchdog/17913-21ssh21:32:041
169940,0watchdog/14599-21ssh23:51:591
169940,0watchdog/127693-21kthreadcore22:02:081
169940,0watchdog/117417-21kthreadcore19:42:081
169940,0watchdog/11711-21sshd00:07:071
169940,0watchdog/113050-21ssh22:13:481
169940,0watchdog/10-21swapper/123:27:071
169940,0watchdog/10-21swapper/121:37:081
169940,0watchdog/10-21swapper/120:57:041
169940,0watchdog/10-21swapper/120:37:081
149940,0migration/09555-21kthreadcore22:32:050
149940,0migration/025904-21kthreadcore22:42:070
149940,0migration/025693-21kthreadcore21:22:060
29715030,0irq/26-eth0-rx-0-21swapper/320:51:583
29715030,0irq/26-eth0-rx-0-21swapper/320:40:413
29715030,0irq/26-eth0-rx-0-21swapper/319:57:023
29715030,0irq/26-eth0-rx-0-21swapper/319:42:023
29715030,0irq/26-eth0-rx-0-21swapper/319:37:393
29715030,0irq/26-eth0-rx-0-21swapper/319:27:003
244929933,0cyclictest0-21swapper/219:52:032
244929932,0cyclictest8656-21aten_r1power_po20:27:022
244929932,0cyclictest7685-21hostname20:02:012
244919933,0cyclictest26542-21awk19:27:041
244919932,0cyclictest563-21latency_hist19:56:501
244919932,0cyclictest5526-21cut22:51:501
244919932,0cyclictest4625-21sensors19:07:121
244919932,0cyclictest3514-21snmpd20:21:571
244919932,0cyclictest30028-21aten_r1power_en22:47:001
244919930,0cyclictest2189-21grep19:32:061
244909933,0cyclictest27872-21kworker/0:119:41:590
244909933,0cyclictest27872-21kworker/0:119:11:540
244909933,0cyclictest20908-21kworker/0:123:37:010
244909933,0cyclictest14228-21kworker/0:223:07:570
244909933,0cyclictest14228-21kworker/0:222:12:110
244909933,0cyclictest14228-21kworker/0:220:41:550
244909932,0cyclictest8536-21awk20:02:050
244909932,0cyclictest829-21awk19:32:040
244909932,0cyclictest6196-21awk23:12:050
244909932,0cyclictest4805-21sensors21:27:110
244909932,0cyclictest2825-21cut21:47:070
244909932,0cyclictest25245-21tr22:02:070
244909932,0cyclictest24489-21cyclictest21:51:500
244909932,0cyclictest0-21swapper/019:46:520
179930,0migration/115095-21if_err_eth020:07:071
169930,0watchdog/118977-21munin-node19:22:001
169930,0watchdog/112924-21unixbench_multi21:12:121
169930,0watchdog/10-21swapper/121:52:121
169930,0watchdog/10-21swapper/100:27:071
169930,0watchdog/10-21swapper/100:12:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional