You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 11:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Mar 31, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
311523625,0sleep10-21swapper/118:55:461
333423524,0sleep00-21swapper/018:58:100
304450350,0irq/25-eth10-21swapper/318:55:003
2948122622,0sleep20-21swapper/218:54:552
8215060,0irq/24-0000:00:14006-21idleruntime-cro00:09:512
35739965,0cyclictest9862-21munin-plugin-st21:04:511
35739960,0cyclictest0-21swapper/121:10:531
339960,0migration/38187-21sh21:00:173
339960,0migration/3765-21sh23:00:083
339960,0migration/37485-21sh22:43:053
339960,0migration/37389-21munin-node21:40:103
339960,0migration/35435-21sh00:24:403
339960,0migration/331940-21sh22:38:263
339960,0migration/318327-21aten_r1power_en21:10:033
339960,0migration/315160-21sh22:47:143
339960,0migration/314495-21sh23:28:263
29725060,0irq/27-eth0-tx-23784-21munin-node19:15:070
29715060,0irq/26-eth0-rx-14937-21grep00:10:053
27160,0ktimersoftd/20-21swapper/221:55:092
149960,0migration/08205-21taskset22:23:510
149960,0migration/022712-21sh22:51:040
149960,0migration/016864-21sh21:06:500
149960,0migration/015833-21ntp_offset21:25:110
149960,0migration/013787-21taskset23:47:490
149960,0migration/011886-21ntp_offset00:05:110
8215050,0irq/24-0000:00:724-21latency_hist20:34:512
35749955,0cyclictest0-21swapper/222:26:032
35749955,0cyclictest0-21swapper/222:22:072
35749955,0cyclictest0-21swapper/222:09:302
35749955,0cyclictest0-21swapper/200:19:062
35749950,0cyclictest0-21swapper/223:47:432
35749950,0cyclictest0-21swapper/221:32:272
35739955,0cyclictest0-21swapper/123:05:301
35739950,0cyclictest0-21swapper/123:42:331
35729955,0cyclictest1739-21diskmemload22:10:440
339950,0migration/39824-21kthreadcore20:15:103
339950,0migration/39698-21kthreadcore23:45:113
339950,0migration/37402-21kthreadcore19:50:073
339950,0migration/3735-21gltestperf20:10:053
339950,0migration/332047-21pmu-power21:15:123
339950,0migration/33158-21if_eth1.321:20:073
339950,0migration/329827-21kthreadcore19:40:093
339950,0migration/325566-21taskset21:33:463
339950,0migration/324831-21kthreadcore23:35:083
339950,0migration/323148-21taskset19:10:143
339950,0migration/322248-21diskmemload23:54:163
339950,0migration/315830-21ntp_offset21:25:113
339950,0migration/314983-21kthreadcore19:30:093
339950,0migration/312955-21munin-node21:45:103
29725050,0irq/27-eth0-tx-16895-21aten_r1power_vo22:30:032
29725050,0irq/27-eth0-tx-0-21swapper/223:42:482
29725050,0irq/27-eth0-tx-0-21swapper/223:22:512
29725050,0irq/27-eth0-tx-0-21swapper/222:42:242
29725050,0irq/27-eth0-tx-0-21swapper/222:02:342
29725050,0irq/27-eth0-tx-0-21swapper/221:35:592
29725050,0irq/27-eth0-tx-0-21swapper/221:21:152
29725050,0irq/27-eth0-tx-0-21swapper/020:55:150
29725050,0irq/27-eth0-tx-0-21swapper/020:00:030
29725050,0irq/27-eth0-tx-0-21swapper/019:59:490
29725050,0irq/27-eth0-tx-0-21swapper/019:05:110
29715050,0irq/26-eth0-rx-7834-21cut00:25:093
29715050,0irq/26-eth0-rx-7834-21cut00:25:093
29715050,0irq/26-eth0-rx-7796-21ssh22:01:013
29715050,0irq/26-eth0-rx-22222-21kthreadcore22:10:083
29715050,0irq/26-eth0-rx-0-21swapper/322:55:203
29715050,0irq/26-eth0-rx-0-21swapper/322:53:053
29715050,0irq/26-eth0-rx-0-21swapper/322:31:293
29715050,0irq/26-eth0-rx-0-21swapper/322:21:503
29715050,0irq/26-eth0-rx-0-21swapper/322:15:333
29715050,0irq/26-eth0-rx-0-21swapper/321:50:043
29715050,0irq/26-eth0-rx-0-21swapper/321:38:063
29715050,0irq/26-eth0-rx-0-21swapper/321:07:423
29715050,0irq/26-eth0-rx-0-21swapper/319:15:103
29715050,0irq/26-eth0-rx-0-21swapper/300:15:063
29715050,0irq/26-eth0-rx-0-21swapper/300:09:093
29715050,0irq/26-eth0-rx-0-21swapper/300:03:023
259950,0migration/27523-21kthreadcore00:25:092
259950,0migration/27523-21kthreadcore00:25:092
259950,0migration/232466-21switchtime20:30:142
259950,0migration/226781-21munin-node20:05:072
259950,0migration/22515-21kthreadcore23:00:092
259950,0migration/224209-21kthreadcore20:50:082
259950,0migration/224201-21kthreadcore20:00:092
259950,0migration/223394-21vmstat22:10:142
259950,0migration/219746-21kthreadcore23:30:092
259950,0migration/219288-21kthreadcore22:50:082
259950,0migration/21850-21latency_hist20:59:512
259950,0migration/214231-21switchtime22:45:142
259950,0migration/211355-21latency_hist19:29:512
179950,0migration/131724-21munin-node19:45:051
179950,0migration/130131-21latency_hist00:19:511
179950,0migration/123667-21sh22:32:351
179950,0migration/120984-21kthreadcore22:10:081
179950,0migration/118787-21needreboot20:20:101
179950,0migration/117717-21kthreadcore23:50:081
179950,0migration/113324-21sh00:29:241
179950,0migration/113324-21sh00:29:241
169950,0watchdog/17935-21perl23:25:051
169950,0watchdog/119232-21kthreadcore00:10:091
149950,0migration/08784-21kthreadcore00:25:090
149950,0migration/08784-21kthreadcore00:25:090
149950,0migration/07880-21pmu-power21:20:120
149950,0migration/07543-21kthreadcore19:25:090
149950,0migration/07521-21kthreadcore19:50:070
149950,0migration/0709-21dump-pmu-power20:34:460
149950,0migration/06381-21sh23:01:320
149950,0migration/06313-21kthreadcore21:00:100
149950,0migration/0575-21kthreadcore00:20:060
149950,0migration/04673-21switchtime23:40:130
149950,0migration/032617-21kthreadcore19:20:090
149950,0migration/028665-21kthreadcore21:35:080
149950,0migration/027043-21fschecks_time21:15:050
149950,0migration/026978-21kthreadcore22:35:080
149950,0migration/025396-21kthreadcore20:50:090
149950,0migration/022824-21kthreadcore19:35:100
149950,0migration/020435-21kthreadcore21:50:080
149950,0migration/017974-21irqrtprio22:30:080
149950,0migration/015932-21users21:45:170
149950,0migration/014063-21taskset20:43:300
149950,0migration/012721-21missed_timers23:25:100
149950,0migration/01152-21kthreadcore19:45:090
149950,0migration/010281-21kthreadcore22:45:090
149950,0migration/010028-21taskset19:02:390
35749943,0cyclictest26207-21awk22:15:072
35749943,0cyclictest24744-21awk19:15:072
35749943,0cyclictest20908-21awk20:25:072
35749943,0cyclictest18962-21awk19:35:072
35749943,0cyclictest15276-21awk20:45:062
35749943,0cyclictest11416-21awk19:05:072
35739944,0cyclictest0-21swapper/120:55:071
35739943,0cyclictest9256-21awk22:45:071
35739943,0cyclictest7204-21awk00:05:051
35739943,0cyclictest3556-21lldpd23:45:401
35739943,0cyclictest3556-21lldpd23:15:101
35739943,0cyclictest3514-21snmpd23:55:341
35739943,0cyclictest3514-21snmpd23:10:041
35739943,0cyclictest3514-21snmpd21:56:191
35739943,0cyclictest3514-21snmpd21:47:191
35739943,0cyclictest3514-21snmpd21:40:341
35739943,0cyclictest3514-21snmpd20:35:191
35739943,0cyclictest3514-21snmpd20:34:191
35739943,0cyclictest25581-21awk19:40:051
35739943,0cyclictest25557-21rm22:35:051
35739943,0cyclictest25117-21awk22:55:061
35739943,0cyclictest21903-21awk20:50:061
35739943,0cyclictest19983-21awk20:00:061
35739943,0cyclictest16114-21awk23:30:061
35739943,0cyclictest12351-21awk19:30:071
35739943,0cyclictest11186-21grep21:05:081
35729944,0cyclictest0-21swapper/022:26:110
35729944,0cyclictest0-21swapper/021:43:270
35729943,0cyclictest3556-21lldpd19:44:390
35729943,0cyclictest3514-21snmpd19:33:330
35729943,0cyclictest3241-21irqbalance23:55:190
35729943,0cyclictest3241-21irqbalance23:51:290
35729943,0cyclictest3241-21irqbalance23:19:290
35729943,0cyclictest3241-21irqbalance23:11:090
35729943,0cyclictest3241-21irqbalance22:55:090
35729943,0cyclictest3241-21irqbalance00:17:390
35729943,0cyclictest1939-21awk20:35:060
35729943,0cyclictest19299-21awk21:10:080
35729943,0cyclictest19117-21awk21:30:060
35729943,0cyclictest16245-21awk23:30:060
35729942,0cyclictest6134-21sensors22:40:130
339940,0migration/39361-21fschecks_count22:25:053
339940,0migration/38175-21df_abs23:05:033
339940,0migration/38079-21kthreadcore19:00:093
339940,0migration/331138-21diskmemload23:40:013
339940,0migration/326915-21kthreadcore23:15:103
339940,0migration/322583-21kthreadcore20:25:093
339940,0migration/321160-21kthreadcore19:35:093
339940,0migration/319832-21ntp_kernel_pll_20:45:113
339940,0migration/318624-21taskset19:57:023
29725040,0irq/27-eth0-tx-26375-21ssh21:14:532
29725040,0irq/27-eth0-tx-13412-21munin-node20:20:050
29725040,0irq/27-eth0-tx-11784-21cut23:25:092
29725040,0irq/27-eth0-tx-0-21swapper/223:35:062
29725040,0irq/27-eth0-tx-0-21swapper/223:10:072
29725040,0irq/27-eth0-tx-0-21swapper/223:05:012
29725040,0irq/27-eth0-tx-0-21swapper/222:55:032
29725040,0irq/27-eth0-tx-0-21swapper/222:35:072
29725040,0irq/27-eth0-tx-0-21swapper/221:45:042
29725040,0irq/27-eth0-tx-0-21swapper/221:40:022
29725040,0irq/27-eth0-tx-0-21swapper/221:09:522
29725040,0irq/27-eth0-tx-0-21swapper/221:05:002
29725040,0irq/27-eth0-tx-0-21swapper/020:15:070
29725040,0irq/27-eth0-tx-0-21swapper/020:11:180
29715040,0irq/26-eth0-rx-0-21swapper/323:55:213
29715040,0irq/26-eth0-rx-0-21swapper/323:30:583
29715040,0irq/26-eth0-rx-0-21swapper/323:20:093
29715040,0irq/26-eth0-rx-0-21swapper/323:10:053
29715040,0irq/26-eth0-rx-0-21swapper/322:05:023
29715040,0irq/26-eth0-rx-0-21swapper/321:55:033
259940,0migration/27411-21kthreadcore19:50:072
259940,0migration/26182-21kthreadcore21:00:102
259940,0migration/232487-21kthreadcore00:00:082
259940,0migration/22844-21kthreadcore00:20:072
259940,0migration/22657-21kthreadcore19:20:092
259940,0migration/226256-21kthreadcore23:55:112
259940,0migration/224416-21gltestperf23:15:062
259940,0migration/222429-21kthreadcore21:50:092
259940,0migration/218087-21munin-node19:10:072
259940,0migration/217220-21kthreadcore23:50:082
259940,0migration/213529-21aten_r1power_po20:20:032
259940,0migration/210264-21kthreadcore20:40:092
259940,0migration/210238-21latency_hist21:24:512
249940,0watchdog/232022-21fschecks_time19:45:062
249940,0watchdog/20-21swapper/200:10:092
179940,0migration/18421-21kthreadcore19:00:091
179940,0migration/17019-21sh22:00:141
179940,0migration/16759-21taskset20:14:531
179940,0migration/16092-21kthreadcore19:25:081
179940,0migration/126809-21kthreadcore00:15:081
179940,0migration/125739-21kthreadcore19:15:091
179940,0migration/123688-21missed_timers21:30:101
179940,0migration/123558-21taskset22:53:371
179940,0migration/122908-21kthreadcore19:35:101
179940,0migration/121555-21kthreadcore21:50:091
179940,0migration/11597-21kthreadcore20:10:081
179940,0migration/113989-21kthreadcore21:25:091
179940,0migration/112654-21kthreadcore22:25:101
179940,0migration/111733-21kthreadcore22:05:091
179940,0migration/111730-21kthreadcore20:40:091
179940,0migration/111235-21sh19:05:051
169940,0watchdog/15491-21kthreadcore22:20:091
169940,0watchdog/131483-21hostname00:00:051
169940,0watchdog/128183-21kthreadcore23:35:091
169940,0watchdog/123920-21cut20:25:101
169940,0watchdog/11739-21diskmemload23:00:091
169940,0watchdog/11739-21diskmemload21:15:101
169940,0watchdog/10-21swapper/122:40:091
169940,0watchdog/10-21swapper/122:15:131
169940,0watchdog/10-21swapper/120:45:061
149940,0migration/027420-21kthreadcore21:55:090
149940,0migration/026559-21kthreadcore23:35:090
149940,0migration/014290-21sh00:10:020
35749933,0cyclictest3556-21lldpd20:14:382
35749932,0cyclictest9459-21sensors19:00:132
35749932,0cyclictest7639-21awk20:15:082
35749932,0cyclictest5658-21awk19:25:072
35749932,0cyclictest13331-21awk19:55:062
35749930,0cyclictest0-21swapper/219:40:212
35739932,0cyclictest5830-21basename19:50:021
35739932,0cyclictest3514-21snmpd21:35:191
35739932,0cyclictest3514-21snmpd21:20:041
35739932,0cyclictest32590-21awk23:20:061
35739932,0cyclictest13346-21awk19:55:061
35729933,0cyclictest26563-21kworker/0:022:15:290
35729933,0cyclictest26563-21kworker/0:022:05:010
35729933,0cyclictest26563-21kworker/0:022:00:090
35729933,0cyclictest26563-21kworker/0:020:24:550
35729933,0cyclictest26563-21kworker/0:000:00:340
35729933,0cyclictest22847-21kworker/0:220:04:530
35729933,0cyclictest22847-21kworker/0:219:09:550
35729932,0cyclictest3241-21irqbalance23:19:590
35729932,0cyclictest13529-21sensors23:05:120
35729930,0cyclictest14368-21wc20:44:510
29715030,0irq/26-eth0-rx-0-21swapper/320:54:593
29715030,0irq/26-eth0-rx-0-21swapper/320:50:033
29715030,0irq/26-eth0-rx-0-21swapper/320:39:573
29715030,0irq/26-eth0-rx-0-21swapper/320:35:023
29715030,0irq/26-eth0-rx-0-21swapper/320:30:043
29715030,0irq/26-eth0-rx-0-21swapper/319:45:043
29715030,0irq/26-eth0-rx-0-21swapper/319:08:393
179930,0migration/125945-21cpu20:05:031
169930,0watchdog/10-21swapper/119:20:061
35739921,0cyclictest17027-21latency_hist19:09:521
35120,0ktimersoftd/30-21swapper/319:25:043
29715020,0irq/26-eth0-rx-0-21swapper/320:20:073
29715020,0irq/26-eth0-rx-0-21swapper/320:04:573
29715020,0irq/26-eth0-rx-0-21swapper/319:59:573
29715020,0irq/26-eth0-rx-0-21swapper/319:19:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional