You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 12:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Wed Apr 01, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/318:55:013
892324736,0sleep00-21swapper/018:57:550
884423324,0sleep10-21swapper/118:57:001
337523329,0sleep20-21swapper/218:55:002
91559970,0cyclictest27944-21cut20:50:050
339970,0migration/320422-21sh23:48:033
91559965,0cyclictest27254-21kworker/0:021:41:370
339960,0migration/36496-21sh00:04:313
339960,0migration/35388-21sh22:37:563
339960,0migration/330532-21sh21:53:153
339960,0migration/330283-21sh21:31:313
339960,0migration/314058-21munin-node23:04:563
29715060,0irq/26-eth0-rx-30174-21perl23:34:583
29715060,0irq/26-eth0-rx-26684-21memory23:50:033
149960,0migration/020970-21sh22:46:430
149960,0migration/020449-21switchtime22:25:070
149960,0migration/013054-21munin-node21:00:050
91579955,0cyclictest7333-21diskmemload22:30:362
91579955,0cyclictest0-21swapper/223:54:002
91579955,0cyclictest0-21swapper/223:49:452
91579955,0cyclictest0-21swapper/222:51:152
91579955,0cyclictest0-21swapper/222:37:512
91579955,0cyclictest0-21swapper/221:45:022
91579955,0cyclictest0-21swapper/200:21:352
91579954,0cyclictest3514-21snmpd23:00:422
91579950,0cyclictest0-21swapper/223:58:152
91579950,0cyclictest0-21swapper/223:34:182
91579950,0cyclictest0-21swapper/221:37:002
91579950,0cyclictest0-21swapper/220:45:172
91569950,0cyclictest0-21swapper/123:45:491
91569950,0cyclictest0-21swapper/119:14:581
91559955,0cyclictest7427-21kworker/0:000:04:300
35150,0ktimersoftd/30-21swapper/321:29:273
339950,0migration/39276-21kthreadcore20:35:023
339950,0migration/37978-21taskset20:59:593
339950,0migration/36267-21latency_hist00:24:453
339950,0migration/36214-21missed_timers20:55:023
339950,0migration/36171-21taskset20:33:043
339950,0migration/35756-21sh23:19:563
339950,0migration/35756-21sh23:19:563
339950,0migration/3482-21kthreadcore23:15:033
339950,0migration/332177-21munin-node20:50:093
339950,0migration/331571-21fschecks_time22:15:003
339950,0migration/329282-21diskmemload23:54:533
339950,0migration/326235-21kthreadcore19:35:023
339950,0migration/324794-21kthreadcore00:15:033
339950,0migration/324611-21kthreadcore22:30:013
339950,0migration/322244-21kthreadcore20:45:033
339950,0migration/321568-21munin-node19:05:073
339950,0migration/319616-21ksmtuned20:44:163
339950,0migration/318066-21kthreadcore21:05:033
339950,0migration/318007-21munin-node19:54:583
339950,0migration/314904-21diskmemload21:23:173
339950,0migration/31344-21kthreadcore20:05:033
29725050,0irq/27-eth0-tx-30250-21munin-node19:39:590
29725050,0irq/27-eth0-tx-19208-21ssh23:25:042
29725050,0irq/27-eth0-tx-1817-21kthreadcore23:15:032
29725050,0irq/27-eth0-tx-14310-21ssh21:02:142
29725050,0irq/27-eth0-tx-13888-21ssh21:41:192
29725050,0irq/27-eth0-tx-0-21swapper/222:16:252
29725050,0irq/27-eth0-tx-0-21swapper/222:10:052
29725050,0irq/27-eth0-tx-0-21swapper/020:39:150
29715050,0irq/26-eth0-rx-6559-21runrttasks22:23:553
29715050,0irq/26-eth0-rx-31189-21if_err_eth022:55:003
29715050,0irq/26-eth0-rx-27542-21kthreadcore22:50:043
29715050,0irq/26-eth0-rx-22986-21irqcore23:10:023
29715050,0irq/26-eth0-rx-17646-21kthreadcore22:05:003
29715050,0irq/26-eth0-rx-16840-21tr23:25:013
29715050,0irq/26-eth0-rx-16728-21cut00:10:013
29715050,0irq/26-eth0-rx-15052-21munin-node22:24:583
29715050,0irq/26-eth0-rx-1392-21grep21:15:023
29715050,0irq/26-eth0-rx-11126-21tr22:40:033
29715050,0irq/26-eth0-rx-0-21swapper/323:41:523
29715050,0irq/26-eth0-rx-0-21swapper/323:34:203
29715050,0irq/26-eth0-rx-0-21swapper/323:02:093
29715050,0irq/26-eth0-rx-0-21swapper/322:45:053
29715050,0irq/26-eth0-rx-0-21swapper/322:13:013
29715050,0irq/26-eth0-rx-0-21swapper/321:39:563
29715050,0irq/26-eth0-rx-0-21swapper/321:35:593
29715050,0irq/26-eth0-rx-0-21swapper/300:24:063
29715050,0irq/26-eth0-rx-0-21swapper/300:08:343
27150,0ktimersoftd/20-21swapper/221:29:272
27150,0ktimersoftd/20-21swapper/219:05:102
27150,0ktimersoftd/20-21swapper/200:09:302
259950,0migration/230354-21gltestperf23:34:582
19150,0ktimersoftd/10-21swapper/100:09:301
149950,0migration/06388-21taskset22:39:570
149950,0migration/06240-21if_eth1.223:39:590
149950,0migration/05817-21kthreadcore20:55:010
149950,0migration/04777-21sh22:36:080
149950,0migration/04617-21irqcore19:20:020
149950,0migration/04053-21sh23:16:030
149950,0migration/032633-21kthreadcore20:05:020
149950,0migration/028195-21kthreadcore19:35:030
149950,0migration/026876-21munin-node23:50:050
149950,0migration/025259-21kthreadcore21:50:020
149950,0migration/024548-21kthreadcore22:30:010
149950,0migration/024162-21kthreadcore23:30:040
149950,0migration/023273-21df_abs22:09:580
149950,0migration/022362-21munin-node23:09:590
149950,0migration/022338-21needreboot19:30:050
149950,0migration/021178-21kernelversion20:45:020
149950,0migration/020033-21kthreadcore21:05:030
149950,0migration/017298-21munin-node19:50:090
149950,0migration/015790-21munin-node00:09:590
149950,0migration/015776-21idleruntime22:04:580
149950,0migration/014243-21taskset22:03:260
149950,0migration/014081-21taskset21:20:580
149950,0migration/01298-21kthreadcore22:55:020
149950,0migration/012022-21kthreadcore19:25:030
149950,0migration/010272-21kthreadcore23:20:020
149950,0migration/010272-21kthreadcore23:20:020
91579943,0cyclictest6497-21awk20:10:012
91579943,0cyclictest3556-21lldpd22:00:392
91579943,0cyclictest3556-21lldpd21:15:102
91579943,0cyclictest3556-21lldpd19:51:092
91579943,0cyclictest3556-21lldpd19:28:092
91579943,0cyclictest3556-21lldpd00:00:402
91579943,0cyclictest3514-21snmpd19:17:422
91579943,0cyclictest3241-21irqbalance23:12:042
91579943,0cyclictest3241-21irqbalance20:19:542
91579943,0cyclictest3241-21irqbalance18:59:552
91579943,0cyclictest24473-21awk19:35:002
91579943,0cyclictest18809-21awk19:55:022
91579943,0cyclictest17456-21grep19:29:592
91579943,0cyclictest14909-21awk23:04:592
91569944,0cyclictest0-21swapper/122:14:321
91569944,0cyclictest0-21swapper/121:16:501
91569944,0cyclictest0-21swapper/119:21:161
91569943,0cyclictest8556-21awk21:20:001
91569943,0cyclictest8234-21awk21:40:011
91569943,0cyclictest7434-21awk20:35:001
91569943,0cyclictest7305-21awk00:24:591
91569943,0cyclictest7299-21awk22:40:001
91569943,0cyclictest6169-21awk23:39:591
91569943,0cyclictest3556-21lldpd22:29:401
91569943,0cyclictest3556-21lldpd21:46:401
91569943,0cyclictest3556-21lldpd21:28:391
91569943,0cyclictest3556-21lldpd20:23:091
91569943,0cyclictest3514-21snmpd21:59:271
91569943,0cyclictest3514-21snmpd21:37:421
91569943,0cyclictest3514-21snmpd19:59:571
91569943,0cyclictest3241-21irqbalance22:20:041
91569943,0cyclictest3241-21irqbalance20:59:541
91569943,0cyclictest3241-21irqbalance20:45:041
91569943,0cyclictest3241-21irqbalance20:10:141
91569943,0cyclictest3241-21irqbalance20:05:341
91569943,0cyclictest3241-21irqbalance19:45:041
91569943,0cyclictest3241-21irqbalance19:40:251
91569943,0cyclictest3241-21irqbalance19:34:551
91569943,0cyclictest3241-21irqbalance19:29:551
91569943,0cyclictest3241-21irqbalance19:12:251
91569943,0cyclictest30872-21awk23:15:001
91569943,0cyclictest30460-21awk23:34:591
91569943,0cyclictest26547-21awk20:25:001
91569943,0cyclictest24395-21kthreadcore00:15:031
91569943,0cyclictest22532-21awk23:30:021
91569943,0cyclictest16856-21awk19:05:011
91569943,0cyclictest15828-21awk22:04:581
91569943,0cyclictest0-21swapper/119:25:031
91569942,0cyclictest7333-21diskmemload00:23:241
91569942,0cyclictest6630-21sed00:04:451
91569942,0cyclictest6252-21ssh23:19:581
91569942,0cyclictest6252-21ssh23:19:581
91569942,0cyclictest3840-21ntp_kernel_pll_22:35:051
91569942,0cyclictest31066-21aten2.4-expect22:14:581
91569942,0cyclictest30256-21rm23:57:491
91569942,0cyclictest27711-21cut21:50:031
91569942,0cyclictest24691-21tr21:10:011
91569942,0cyclictest23655-21rm21:29:491
91569942,0cyclictest23423-21ntp_states19:55:071
91569942,0cyclictest23343-21irqcore22:50:031
91569942,0cyclictest22827-21if_err_eth123:10:011
91569942,0cyclictest22712-21aten2.4-expect22:29:551
91569942,0cyclictest20748-21rm22:46:031
91569942,0cyclictest18803-21processes20:40:061
91569942,0cyclictest15592-21hddtemp_smartct00:09:581
91569942,0cyclictest14345-21perl23:24:581
91569942,0cyclictest13591-21sh22:01:311
91569942,0cyclictest11818-21tune2fs19:49:591
91569941,0cyclictest5031-21kthreadcore20:55:011
91569941,0cyclictest27185-21perl20:50:011
91559944,0cyclictest7427-21kworker/0:023:39:410
91559944,0cyclictest7427-21kworker/0:023:05:450
91559944,0cyclictest7427-21kworker/0:022:59:490
91559944,0cyclictest7427-21kworker/0:022:21:470
91559944,0cyclictest7427-21kworker/0:000:26:140
91559944,0cyclictest7427-21kworker/0:000:04:580
91559944,0cyclictest27254-21kworker/0:021:55:430
91559944,0cyclictest27254-21kworker/0:021:46:510
91559944,0cyclictest27254-21kworker/0:021:30:010
91559944,0cyclictest27254-21kworker/0:021:18:210
91559944,0cyclictest27254-21kworker/0:020:39:490
91559944,0cyclictest27254-21kworker/0:020:30:110
91559944,0cyclictest27254-21kworker/0:020:25:070
91559944,0cyclictest27254-21kworker/0:020:19:470
91559944,0cyclictest27254-21kworker/0:020:09:490
91559944,0cyclictest27254-21kworker/0:019:54:550
91559944,0cyclictest27254-21kworker/0:019:44:550
91559944,0cyclictest27254-21kworker/0:019:15:200
91559944,0cyclictest27254-21kworker/0:019:09:520
91559944,0cyclictest27254-21kworker/0:018:59:540
91559944,0cyclictest23720-21kworker/0:222:14:490
91559943,0cyclictest31654-21awk00:20:000
91559943,0cyclictest16659-21awk21:25:010
91559943,0cyclictest14537-21awk23:24:590
91559943,0cyclictest14111-21sort23:45:000
91559943,0cyclictest13132-21awk20:15:020
91559940,0cyclictest0-21swapper/023:55:290
8215040,0irq/24-0000:00:30022-21latency_hist19:39:462
8215040,0irq/24-0000:00:24590-21cat19:59:462
339940,0migration/332626-21kthreadcore19:40:023
339940,0migration/330291-21interrupts19:15:023
339940,0migration/324744-21aten2.4-expect19:59:573
339940,0migration/317203-21kthreadcore20:15:053
339940,0migration/315455-21kthreadcore19:50:033
339940,0migration/315306-40yum-updatesd21:44:503
29725040,0irq/27-eth0-tx-7333-21diskmemload22:04:502
29725040,0irq/27-eth0-tx-7333-21diskmemload21:04:522
29725040,0irq/27-eth0-tx-6307-21munin-node22:39:582
29725040,0irq/27-eth0-tx-5578-21ssh23:19:522
29725040,0irq/27-eth0-tx-5578-21ssh23:19:522
29725040,0irq/27-eth0-tx-21267-21ssh22:26:352
29725040,0irq/27-eth0-tx-20593-21ssh22:45:352
29725040,0irq/27-eth0-tx-16018-21munin-node19:04:580
29725040,0irq/27-eth0-tx-13489-21ssh00:27:492
29725040,0irq/27-eth0-tx-0-21swapper/223:40:462
29725040,0irq/27-eth0-tx-0-21swapper/222:54:582
29725040,0irq/27-eth0-tx-0-21swapper/222:19:542
29725040,0irq/27-eth0-tx-0-21swapper/221:51:102
29725040,0irq/27-eth0-tx-0-21swapper/221:31:072
29725040,0irq/27-eth0-tx-0-21swapper/221:10:052
29725040,0irq/27-eth0-tx-0-21swapper/200:10:002
29715040,0irq/26-eth0-rx-27261-21kthreadcore19:10:053
29715040,0irq/26-eth0-rx-0-21swapper/322:01:213
29715040,0irq/26-eth0-rx-0-21swapper/321:55:243
29715040,0irq/26-eth0-rx-0-21swapper/321:09:573
29715040,0irq/26-eth0-rx-0-21swapper/319:19:583
259940,0migration/27872-21kernelversion20:35:022
259940,0migration/231441-21timerwakeupswit20:25:082
259940,0migration/228001-21kthreadcore20:50:052
259940,0migration/22161-21kthreadcore20:55:002
259940,0migration/21983-21kthreadcore20:05:032
259940,0migration/216224-21kthreadcore20:40:022
249940,0watchdog/28495-21if_err_eth021:20:002
249940,0watchdog/229463-21ssh00:16:472
249940,0watchdog/228188-21proc_pri19:10:082
249940,0watchdog/213598-21kthreadcore20:15:042
249940,0watchdog/20-21swapper/221:55:002
249940,0watchdog/20-21swapper/220:29:562
249940,0watchdog/20-21swapper/219:45:002
19140,0ktimersoftd/10-21swapper/123:04:241
149940,0migration/027213-21kthreadcore00:15:040
149940,0migration/02668-21kthreadcore21:35:030
149940,0migration/026635-21kthreadcore20:00:020
149940,0migration/025222-21kthreadcore21:10:020
91579932,0cyclictest4487-21awk19:20:012
91569932,0cyclictest32269-21aten2.4-expect20:29:561
91569931,0cyclictest9489-21aten2.4-expect18:59:591
91569931,0cyclictest30282-21ssh22:54:471
91569931,0cyclictest21264-21rm23:49:521
91569931,0cyclictest15505-21chrt21:04:481
91569931,0cyclictest13916-21ssh23:04:471
91569931,0cyclictest12223-21latency_hist20:14:461
91559933,0cyclictest7427-21kworker/0:022:49:550
29715030,0irq/26-eth0-rx-0-21swapper/320:24:563
29715030,0irq/26-eth0-rx-0-21swapper/320:19:563
29715030,0irq/26-eth0-rx-0-21swapper/320:09:583
29715030,0irq/26-eth0-rx-0-21swapper/319:48:533
29715030,0irq/26-eth0-rx-0-21swapper/319:29:573
29715030,0irq/26-eth0-rx-0-21swapper/319:29:023
29715030,0irq/26-eth0-rx-0-21swapper/319:00:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional