You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 03:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Jul 06, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
231924028,0sleep30-21swapper/318:46:003
301723626,0sleep10-21swapper/118:47:181
290923322,0sleep00-21swapper/018:46:050
293323026,0sleep20-21swapper/218:46:232
33919960,0cyclictest1554-21diskmemload22:02:492
33909960,0cyclictest0-21swapper/123:26:001
339960,0migration/39647-21sh21:55:003
339960,0migration/39312-21sh20:54:573
339960,0migration/38586-21sh22:53:163
339960,0migration/332718-21sh22:07:073
339960,0migration/331485-21sh19:35:513
29715060,0irq/26-eth0-rx-8477-21ssh21:11:283
29715060,0irq/26-eth0-rx-18313-21ssh21:40:513
29715060,0irq/26-eth0-rx-0-21swapper/322:47:423
27160,0ktimersoftd/27516-21ssh22:51:032
149960,0migration/0351-21sh23:28:470
149960,0migration/024706-21taskset21:02:220
4150,0ktimersoftd/00-21swapper/000:01:360
33919955,0cyclictest0-21swapper/223:46:522
33919955,0cyclictest0-21swapper/223:30:472
33919955,0cyclictest0-21swapper/223:25:542
33919955,0cyclictest0-21swapper/222:49:292
33919955,0cyclictest0-21swapper/221:01:002
33919955,0cyclictest0-21swapper/200:19:582
33919950,0cyclictest0-21swapper/222:35:272
33919950,0cyclictest0-21swapper/221:14:342
33909955,0cyclictest0-21swapper/123:05:581
33909955,0cyclictest0-21swapper/121:00:571
33909954,0cyclictest3514-21snmpd23:14:411
33909950,0cyclictest0-21swapper/121:55:111
33899955,0cyclictest0-21swapper/022:49:150
33899950,0cyclictest0-21swapper/022:32:030
33899950,0cyclictest0-21swapper/022:16:180
33899950,0cyclictest0-21swapper/021:23:100
33899950,0cyclictest0-21swapper/000:13:450
339950,0migration/38131-21kthreadcore20:05:563
339950,0migration/37720-21kthreadcore19:40:563
339950,0migration/34633-21munin-node18:50:543
339950,0migration/33214-21irqcore21:30:563
339950,0migration/329930-21latency19:30:583
339950,0migration/328455-21kthreadcore19:55:553
339950,0migration/328304-21kthreadcore23:25:553
339950,0migration/325510-21sh21:04:223
339950,0migration/324552-21latency_hist19:30:393
339950,0migration/322106-21kthreadcore22:40:563
339950,0migration/316976-21kthreadcore20:35:553
339950,0migration/316946-21sh23:19:263
339950,0migration/315441-21kthreadcore21:55:563
339950,0migration/31430-21kthreadcore20:00:573
339950,0migration/313021-21kthreadcore19:20:583
339950,0migration/312771-21kthreadcore21:15:563
29725050,0irq/27-eth0-tx-9375-21munin-node23:55:532
29725050,0irq/27-eth0-tx-7601-21munin-node20:30:540
29725050,0irq/27-eth0-tx-32579-21sshd19:10:560
29725050,0irq/27-eth0-tx-12747-21cut22:55:552
29725050,0irq/27-eth0-tx-10847-21iostat20:55:542
29725050,0irq/27-eth0-tx-10393-21munin-node21:15:582
29725050,0irq/27-eth0-tx-0-21swapper/223:43:542
29725050,0irq/27-eth0-tx-0-21swapper/223:06:512
29725050,0irq/27-eth0-tx-0-21swapper/222:45:322
29725050,0irq/27-eth0-tx-0-21swapper/222:07:392
29715050,0irq/26-eth0-rx-32037-21python21:06:013
29715050,0irq/26-eth0-rx-31273-21kthreadcore21:25:573
29715050,0irq/26-eth0-rx-21964-21cut22:00:563
29715050,0irq/26-eth0-rx-20899-21kthreadcore21:20:563
29715050,0irq/26-eth0-rx-1659-21munin-node22:30:503
29715050,0irq/26-eth0-rx-0-21swapper/323:58:433
29715050,0irq/26-eth0-rx-0-21swapper/323:51:133
29715050,0irq/26-eth0-rx-0-21swapper/323:20:523
29715050,0irq/26-eth0-rx-0-21swapper/323:05:583
29715050,0irq/26-eth0-rx-0-21swapper/323:00:543
29715050,0irq/26-eth0-rx-0-21swapper/322:36:573
29715050,0irq/26-eth0-rx-0-21swapper/322:25:423
29715050,0irq/26-eth0-rx-0-21swapper/322:21:313
29715050,0irq/26-eth0-rx-0-21swapper/321:46:403
29715050,0irq/26-eth0-rx-0-21swapper/300:15:553
29715050,0irq/26-eth0-rx-0-21swapper/300:05:413
259950,0migration/25505-21kthreadcore23:50:572
259950,0migration/231089-21kthreadcore21:25:572
259950,0migration/22793-21kthreadcore00:10:542
259950,0migration/224837-21kthreadcore20:40:572
259950,0migration/223616-21kthreadcore19:50:582
259950,0migration/223581-21sh00:01:032
259950,0migration/21413-21sh21:49:502
259950,0migration/211201-21irqrtprio22:15:532
179950,0migration/18405-21sh20:52:381
179950,0migration/17936-21munin-node22:31:041
179950,0migration/126777-21unixbench_multi20:41:031
179950,0migration/124864-21timerwakeupswit19:51:031
179950,0migration/121624-21kthreadcore23:00:581
179950,0migration/120509-21taskset20:39:021
179950,0migration/118418-21fschecks_time22:40:521
179950,0migration/118297-21aten_r1power_cu21:40:511
179950,0migration/117533-21latency_hist23:00:381
179950,0migration/114059-21kthreadcore21:35:561
179950,0migration/110077-21latency_hist21:55:381
149950,0migration/0829-21sh22:29:100
149950,0migration/07871-21kthreadcore19:40:560
149950,0migration/07610-21taskset23:51:250
149950,0migration/05789-21kthreadcore21:50:570
149950,0migration/0529-21taskset20:25:040
149950,0migration/0529-21taskset20:25:040
149950,0migration/04621-21kthreadcore22:50:580
149950,0migration/03570-21kthreadcore20:50:550
149950,0migration/03321-21irqrtprio21:30:560
149950,0migration/031759-21munin-node22:06:000
149950,0migration/030070-21kthreadcore21:05:560
149950,0migration/029962-21kthreadcore20:45:560
149950,0migration/029630-21kthreadcore21:25:560
149950,0migration/025871-21cpuspeed_turbos23:45:510
149950,0migration/024420-21hddtemp_smartct19:05:530
149950,0migration/023939-21kthreadcore20:40:570
149950,0migration/023469-21kthreadcore22:00:570
149950,0migration/023060-21taskset19:01:350
149950,0migration/02133-21kthreadcore20:00:580
149950,0migration/020302-21kthreadcore22:40:550
149950,0migration/01852-21df_abs23:10:500
149950,0migration/017423-21sh21:59:130
149950,0migration/017236-21kthreadcore19:45:560
149950,0migration/016852-21ntp_offset19:21:000
149950,0migration/014289-21kthreadcore21:15:560
149950,0migration/012523-21timerwakeupswit20:06:020
149950,0migration/012510-21kthreadcore20:55:560
149950,0migration/012105-21kthreadcore00:15:560
149950,0migration/011414-21kthreadcore22:35:550
149950,0migration/011349-21kthreadcore23:15:550
149950,0migration/010724-21taskset19:16:120
8215040,0irq/24-0000:00:0-21swapper/223:35:382
33919943,0cyclictest6567-21awk19:40:542
33919943,0cyclictest3556-21lldpd23:10:422
33919943,0cyclictest3556-21lldpd20:21:392
33919943,0cyclictest3556-21lldpd20:21:392
33919943,0cyclictest3556-21lldpd20:13:392
33919943,0cyclictest3556-21lldpd19:13:392
33919943,0cyclictest3556-21lldpd19:09:392
33919943,0cyclictest3556-21lldpd18:56:392
33919943,0cyclictest3514-21snmpd21:50:562
33919943,0cyclictest3514-21snmpd20:15:572
33919943,0cyclictest3241-21irqbalance19:26:382
33919943,0cyclictest3037-21awk21:30:552
33919943,0cyclictest28462-21awk20:45:552
33919943,0cyclictest27248-21awk21:05:542
33919943,0cyclictest18608-21awk23:20:542
33919943,0cyclictest13301-21awk19:45:542
33919943,0cyclictest10250-21munin-node21:36:022
33909943,0cyclictest3556-21lldpd23:33:111
33909943,0cyclictest3556-21lldpd22:54:411
33909943,0cyclictest3556-21lldpd22:25:111
33909943,0cyclictest3556-21lldpd22:02:111
33909943,0cyclictest3556-21lldpd20:48:401
33909943,0cyclictest3556-21lldpd20:15:091
33909943,0cyclictest3556-21lldpd18:59:391
33909943,0cyclictest3514-21snmpd19:17:561
33909943,0cyclictest26993-21awk22:25:531
33909943,0cyclictest2687-21awk22:10:531
33909943,0cyclictest24442-21sadc19:30:391
33909943,0cyclictest20787-21awk20:15:541
33909943,0cyclictest1844-21awk20:25:541
33909943,0cyclictest12224-21awk19:20:551
33909943,0cyclictest11244-21awk21:15:531
33909943,0cyclictest10844-21awk22:15:521
33899944,0cyclictest0-21swapper/000:06:200
33899943,0cyclictest3556-21lldpd23:10:110
33899943,0cyclictest3556-21lldpd20:30:390
33899943,0cyclictest3514-21snmpd22:13:110
33899943,0cyclictest3241-21irqbalance21:43:070
33899943,0cyclictest31097-21kworker/0:223:42:080
33899943,0cyclictest31097-21kworker/0:223:23:240
33899943,0cyclictest31097-21kworker/0:222:55:440
33899943,0cyclictest31097-21kworker/0:222:20:520
33899943,0cyclictest2950-21awk21:10:540
33899943,0cyclictest26601-21awk19:55:540
33899943,0cyclictest10278-21awk23:55:530
33899940,0cyclictest9227-21munin-run23:35:380
339940,0migration/37537-21kthreadcore19:15:573
339940,0migration/34219-21taskset19:13:183
339940,0migration/331765-21runrttasks23:46:073
339940,0migration/319524-21kthreadcore23:40:543
339940,0migration/313297-21kthreadcore21:35:553
339940,0migration/313059-21kthreadcore18:55:563
339940,0migration/312068-21kthreadcore20:30:573
29725040,0irq/27-eth0-tx-17988-21munin-node22:20:542
29725040,0irq/27-eth0-tx-14361-21kthreadcore18:55:560
29725040,0irq/27-eth0-tx-0-21swapper/223:15:542
29725040,0irq/27-eth0-tx-0-21swapper/223:01:272
29725040,0irq/27-eth0-tx-0-21swapper/222:36:172
29725040,0irq/27-eth0-tx-0-21swapper/222:10:502
29725040,0irq/27-eth0-tx-0-21swapper/221:55:422
29725040,0irq/27-eth0-tx-0-21swapper/221:40:512
29725040,0irq/27-eth0-tx-0-21swapper/221:21:532
29715040,0irq/26-eth0-rx-0-21swapper/323:36:253
29715040,0irq/26-eth0-rx-0-21swapper/323:31:013
29715040,0irq/26-eth0-rx-0-21swapper/323:13:133
29715040,0irq/26-eth0-rx-0-21swapper/322:55:553
29715040,0irq/26-eth0-rx-0-21swapper/322:16:343
29715040,0irq/26-eth0-rx-0-21swapper/322:12:373
29715040,0irq/26-eth0-rx-0-21swapper/320:56:533
29715040,0irq/26-eth0-rx-0-21swapper/320:15:593
29715040,0irq/26-eth0-rx-0-21swapper/320:10:533
29715040,0irq/26-eth0-rx-0-21swapper/300:10:493
29715040,0irq/26-eth0-rx-0-21swapper/300:00:503
259940,0migration/27519-21kthreadcore18:50:562
259940,0migration/26134-21kthreadcore20:25:562
259940,0migration/227722-21kthreadcore22:25:552
259940,0migration/22057-21cpuspeed_turbos20:50:512
259940,0migration/217650-21fschecks_time19:00:532
259940,0migration/21282-21kthreadcore20:00:572
249940,0watchdog/28178-21perl20:30:532
249940,0watchdog/228882-21kthreadcore00:05:562
249940,0watchdog/226558-21if_eth119:55:532
249940,0watchdog/220233-21vmstat20:36:012
249940,0watchdog/217417-21yum19:21:052
249940,0watchdog/20-21swapper/219:15:542
179940,0migration/19693-21kthreadcore20:30:571
179940,0migration/19402-21sh00:15:501
179940,0migration/132762-21fschecks_count20:00:541
179940,0migration/129788-21taskset19:07:121
179940,0migration/128415-21kthreadcore19:55:551
179940,0migration/128005-21kthreadcore21:45:581
179940,0migration/126994-21kthreadcore23:45:561
179940,0migration/126768-21munin-node22:45:531
179940,0migration/119106-21kthreadcore00:00:551
169940,0watchdog/19065-21ssh21:13:011
169940,0watchdog/14821-21cut21:30:571
169940,0watchdog/13683-21swap19:11:021
169940,0watchdog/131545-21grep21:25:571
169940,0watchdog/131220-21rt-features00:06:001
169940,0watchdog/127477-21if_err_eth1.320:20:531
169940,0watchdog/127477-21if_err_eth1.320:20:531
169940,0watchdog/121744-21ps23:20:571
169940,0watchdog/1169872chrt19:00:421
169940,0watchdog/115215-21memory22:35:571
169940,0watchdog/10-21swapper/123:56:121
169940,0watchdog/10-21swapper/123:16:011
169940,0watchdog/10-21swapper/121:20:571
149940,0migration/027484-21kthreadcore19:30:570
149940,0migration/021734-21kthreadcore20:15:560
149940,0migration/012216-21kthreadcore21:35:550
149940,0migration/010023-21fschecks_count23:35:510
33919932,0cyclictest3073-21kthreadcore19:35:562
33919932,0cyclictest30046-21expr19:30:582
33909932,0cyclictest7288-21cat19:40:561
33909932,0cyclictest4585-21awk18:50:541
33909932,0cyclictest3556-21lldpd00:10:411
33909932,0cyclictest25511-21awk19:30:541
33909932,0cyclictest10784-21munin-node23:35:541
33909931,0cyclictest23678-21vmstat23:41:011
33899933,0cyclictest31097-21kworker/0:223:00:500
33899933,0cyclictest31097-21kworker/0:221:45:440
33899933,0cyclictest31097-21kworker/0:220:35:460
33899933,0cyclictest31097-21kworker/0:219:50:430
33899933,0cyclictest31097-21kworker/0:219:36:010
33899933,0cyclictest31097-21kworker/0:219:25:430
33899933,0cyclictest15411-21kworker/0:118:50:510
339930,0migration/322796-21kthreadcore19:50:573
29715030,0irq/26-eth0-rx-0-21swapper/320:45:583
29715030,0irq/26-eth0-rx-0-21swapper/320:21:443
29715030,0irq/26-eth0-rx-0-21swapper/320:21:443
29715030,0irq/26-eth0-rx-0-21swapper/319:45:503
29715030,0irq/26-eth0-rx-0-21swapper/319:00:423
249930,0watchdog/20-21swapper/220:05:572
169930,0watchdog/115608-21sh20:56:011
169930,0watchdog/10-21swapper/123:50:521
169930,0watchdog/10-21swapper/122:05:571
169930,0watchdog/10-21swapper/121:05:571
169930,0watchdog/10-21swapper/120:05:571
169930,0watchdog/10-21swapper/119:35:571
149930,0migration/013312-21runrttasks20:10:400
35120,0ktimersoftd/323529-21kthreadcore20:40:573
33909920,0cyclictest0-21swapper/119:45:391
29715020,0irq/26-eth0-rx-0-21swapper/320:26:013
29715020,0irq/26-eth0-rx-0-21swapper/319:07:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional