You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 16:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun May 03, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191860,0ktimersoftd/114758-21perf06:56:321
2119723635,0sleep20-21swapper/206:55:122
304450350,0irq/25-eth10-21swapper/306:52:333
2095223329,0sleep00-21swapper/006:52:220
35160,0ktimersoftd/30-21swapper/310:03:253
35160,0ktimersoftd/30-21swapper/310:03:253
339960,0migration/324899-21munin-node11:17:043
339960,0migration/319949-21fschecks_time10:36:543
339960,0migration/318945-21sh09:34:223
29725060,0irq/27-eth0-tx-23503-21munin-node07:46:560
29725060,0irq/27-eth0-tx-15883-21munin-node07:17:060
29715060,0irq/26-eth0-rx-26232-21yum09:37:063
29715060,0irq/26-eth0-rx-18347-21ssh10:55:133
29715060,0irq/26-eth0-rx-13318-21kthreadcore09:51:593
29715060,0irq/26-eth0-rx-12327-21entropy09:11:563
214039960,0cyclictest0-21swapper/207:02:052
213959965,0cyclictest19444-21diskmemload11:01:231
213949965,0cyclictest25586-21ssh12:00:350
149960,0migration/09441-21sh11:08:200
339950,0migration/39810-21sh09:27:043
339950,0migration/38757-21sh11:27:153
339950,0migration/38365-21kthreadcore08:21:593
339950,0migration/37989-21kthreadcore10:06:583
339950,0migration/36011-21kthreadcore07:31:593
339950,0migration/34492-21gltestperf09:46:553
339950,0migration/32981-21sh10:46:433
339950,0migration/328659-21irqrtprio10:41:583
339950,0migration/328355-21kthreadcore08:36:593
339950,0migration/327904-21cpuspeed_turbos10:21:553
339950,0migration/327378-21gltestperf11:41:563
339950,0migration/325477-21kthreadcore07:21:583
339950,0migration/325448-21taskset12:21:013
339950,0migration/325444-21ntp_offset09:57:003
339950,0migration/325373-21munin-node08:11:593
339950,0migration/319241-21sh10:56:533
339950,0migration/318610-21munin-node11:36:523
339950,0migration/317688-21idleruntime07:41:573
339950,0migration/317600-21latency_hist08:06:423
339950,0migration/316740-21munin-node07:16:583
339950,0migration/314290-21kthreadcore10:12:003
339950,0migration/31359-21munin-node09:22:013
339950,0migration/311349-21fschecks_count11:11:533
339950,0migration/310556-21taskset07:57:413
29725050,0irq/27-eth0-tx-3620-21munin-node09:07:042
29725050,0irq/27-eth0-tx-2305-21ssh09:22:512
29725050,0irq/27-eth0-tx-14288-21kthreadcore10:51:592
29725050,0irq/27-eth0-tx-12950-21iostat_ios09:11:592
29725050,0irq/27-eth0-tx-11142-21munin-node08:02:080
29725050,0irq/27-eth0-tx-0-21swapper/211:16:582
29725050,0irq/27-eth0-tx-0-21swapper/210:08:102
29725050,0irq/27-eth0-tx-0-21swapper/007:56:520
29725050,0irq/27-eth0-tx-0-21swapper/007:06:580
29725050,0irq/27-eth0-tx-0-21swapper/007:03:390
29715050,0irq/26-eth0-rx-27462-21df11:01:533
29715050,0irq/26-eth0-rx-11411-21aten_r1power_en10:31:523
29715050,0irq/26-eth0-rx-10299-21taskset10:29:233
29715050,0irq/26-eth0-rx-0-21swapper/312:22:533
29715050,0irq/26-eth0-rx-0-21swapper/311:57:263
29715050,0irq/26-eth0-rx-0-21swapper/311:54:193
29715050,0irq/26-eth0-rx-0-21swapper/311:31:523
29715050,0irq/26-eth0-rx-0-21swapper/311:22:353
29715050,0irq/26-eth0-rx-0-21swapper/311:10:343
29715050,0irq/26-eth0-rx-0-21swapper/310:17:233
29715050,0irq/26-eth0-rx-0-21swapper/309:17:303
29715050,0irq/26-eth0-rx-0-21swapper/309:06:583
29715050,0irq/26-eth0-rx-0-21swapper/309:04:163
259950,0migration/27714-21rt-features12:07:032
259950,0migration/23854-21df_abs07:31:552
259950,0migration/23379-21latency_hist09:06:422
259950,0migration/226240-21cpuspeed_turbos12:21:542
259950,0migration/226202-21sh11:41:192
259950,0migration/212820-21gltestperf08:26:562
249950,0watchdog/215600-21cut11:31:592
249950,0watchdog/213486-21fschecks_count08:51:562
213959955,0cyclictest0-21swapper/111:58:281
213959955,0cyclictest0-21swapper/111:55:281
213959950,0cyclictest0-21swapper/112:09:211
213959950,0cyclictest0-21swapper/109:47:181
213959950,0cyclictest0-21swapper/107:15:561
213949955,0cyclictest18661-21kworker/0:009:31:590
213949955,0cyclictest0-21swapper/009:47:220
213949955,0cyclictest0-21swapper/009:39:570
213949954,0cyclictest3514-21snmpd08:12:000
213949954,0cyclictest17851-21awk07:41:580
213949950,0cyclictest0-21swapper/009:27:210
179950,0migration/14165-21idleruntime10:26:571
179950,0migration/14036-21munin-node11:06:571
179950,0migration/13475-21irqrtprio11:46:571
179950,0migration/132183-21kthreadcore07:26:591
179950,0migration/131674-21fschecks_time08:16:551
179950,0migration/127294-21taskset09:01:091
179950,0migration/126295-21sh11:41:351
179950,0migration/121153-21kthreadcore12:16:571
179950,0migration/12025-21munin-node09:22:061
179950,0migration/119852-21latency_hist09:36:421
179950,0migration/112872-21hddtemp_smartct08:26:561
179950,0migration/111187-21aten_r1power_en08:01:541
169950,0watchdog/125791-21kthreadcore08:12:001
149950,0migration/09784-21taskset10:48:320
149950,0migration/07382-21kthreadcore11:27:000
149950,0migration/04142-21kthreadcore11:46:580
149950,0migration/04012-21taskset07:54:230
149950,0migration/032664-21kthreadcore10:41:590
149950,0migration/031772-21fschecks_time08:16:550
149950,0migration/030795-21kthreadcore10:22:000
149950,0migration/029269-21kthreadcore09:01:580
149950,0migration/028970-21fschecks_time09:21:560
149950,0migration/027053-21fschecks_count11:41:550
149950,0migration/026386-21sh09:58:070
149950,0migration/024985-21taskset08:34:500
149950,0migration/023362-21kthreadcore08:56:580
149950,0migration/022214-21kthreadcore10:16:570
149950,0migration/02168-40yum-updatesd09:43:040
149950,0migration/019744-21munin-node11:16:560
149950,0migration/018456-21sh10:55:310
149950,0migration/017030-21sh12:14:520
149950,0migration/012962-21munin-node08:26:570
149950,0migration/012373-21kthreadcore11:31:580
149950,0migration/012171-21cpuspeed_turbos09:11:550
149950,0migration/011950-21aten_r1power_fr09:51:540
149950,0migration/01067-21sh12:05:140
339940,0migration/332322-21cpuspeed_turbos08:41:543
339940,0migration/33114-21irqrtprio12:06:583
339940,0migration/324892-21kthreadcore08:56:593
339940,0migration/323933-21fschecks_count07:46:553
339940,0migration/323491-21kthreadcore06:56:593
339940,0migration/315673-21latency_hist07:16:423
339940,0migration/315267-21kthreadcore07:36:593
29725040,0irq/27-eth0-tx-3776-21munin-node09:46:532
29725040,0irq/27-eth0-tx-2980-21ssh10:46:432
29725040,0irq/27-eth0-tx-20751-21if_err_eth1.209:56:552
29725040,0irq/27-eth0-tx-20077-21aten_r1power_en09:36:522
29725040,0irq/27-eth0-tx-18073-21ssh09:52:162
29725040,0irq/27-eth0-tx-0-21swapper/212:16:522
29725040,0irq/27-eth0-tx-0-21swapper/211:51:422
29725040,0irq/27-eth0-tx-0-21swapper/211:46:552
29725040,0irq/27-eth0-tx-0-21swapper/211:41:512
29725040,0irq/27-eth0-tx-0-21swapper/211:26:422
29725040,0irq/27-eth0-tx-0-21swapper/211:22:022
29725040,0irq/27-eth0-tx-0-21swapper/211:06:482
29725040,0irq/27-eth0-tx-0-21swapper/210:37:092
29725040,0irq/27-eth0-tx-0-21swapper/210:16:482
29725040,0irq/27-eth0-tx-0-21swapper/210:01:492
29725040,0irq/27-eth0-tx-0-21swapper/210:01:492
29725040,0irq/27-eth0-tx-0-21swapper/209:26:522
29715040,0irq/26-eth0-rx-0-21swapper/312:12:013
29715040,0irq/26-eth0-rx-0-21swapper/312:01:583
29715040,0irq/26-eth0-rx-0-21swapper/311:48:423
29715040,0irq/26-eth0-rx-0-21swapper/309:41:443
259940,0migration/29955-21kthreadcore08:21:592
259940,0migration/25218-21kthreadcore07:06:592
259940,0migration/25216-21sh07:56:552
259940,0migration/24045-21taskset07:54:492
259940,0migration/232173-21runrttasks08:41:532
259940,0migration/232074-21kthreadcore11:01:592
259940,0migration/230880-21kthreadcore12:01:582
259940,0migration/229935-21kthreadcore10:22:002
259940,0migration/228654-21kthreadcore08:36:592
259940,0migration/226480-21sh09:17:402
259940,0migration/225221-21munin-node08:57:002
259940,0migration/221838-21kthreadcore11:56:572
259940,0migration/221568-21sendmail_mailtr07:17:052
259940,0migration/219029-21latency_hist10:56:412
259940,0migration/215868-21kthreadcore08:02:002
259940,0migration/214160-21kthreadcore09:31:582
259940,0migration/210848-21kthreadcore08:46:592
259940,0migration/210259-21cpuspeed_turbos12:11:552
249940,0watchdog/20-21swapper/210:11:562
214039943,0cyclictest4212-21awk10:26:572
214039943,0cyclictest3556-21lldpd09:41:582
214039943,0cyclictest3556-21lldpd08:31:582
214039943,0cyclictest3556-21lldpd08:11:572
214039943,0cyclictest3556-21lldpd07:46:572
214039943,0cyclictest3556-21lldpd07:36:572
214039943,0cyclictest3556-21lldpd07:27:572
214039943,0cyclictest3556-21lldpd07:11:572
214039943,0cyclictest3514-21snmpd07:23:002
214039943,0cyclictest32069-21awk08:16:562
214039943,0cyclictest28373-21awk10:41:572
214039943,0cyclictest18712-21awk08:06:572
214039943,0cyclictest17740-21awk07:41:572
214039943,0cyclictest12255-21awk10:31:562
214039943,0cyclictest11816-21awk11:11:552
213959944,0cyclictest0-21swapper/112:04:271
213959944,0cyclictest0-21swapper/109:29:571
213959943,0cyclictest3556-21lldpd12:12:581
213959943,0cyclictest3556-21lldpd09:02:281
213959943,0cyclictest3514-21snmpd10:42:001
213959943,0cyclictest3514-21snmpd07:39:451
213959943,0cyclictest28791-21awk10:01:561
213959943,0cyclictest28791-21awk10:01:561
213959943,0cyclictest25911-21sed09:57:041
213959943,0cyclictest22495-21awk06:56:571
213959943,0cyclictest22233-21strings07:42:011
213949944,0cyclictest8356-21kworker/0:012:06:540
213949944,0cyclictest8356-21kworker/0:011:53:190
213949944,0cyclictest19991-21kworker/0:211:13:270
213949944,0cyclictest19991-21kworker/0:210:38:350
213949944,0cyclictest19991-21kworker/0:210:09:130
213949944,0cyclictest19991-21kworker/0:210:04:070
213949944,0cyclictest19991-21kworker/0:210:04:070
213949944,0cyclictest18661-21kworker/0:009:18:510
213949944,0cyclictest18661-21kworker/0:009:11:350
213949944,0cyclictest0-21swapper/012:22:020
213949944,0cyclictest0-21swapper/007:27:030
213949943,0cyclictest5434-21munin-node08:21:580
213949943,0cyclictest4459-21awk07:31:570
213949943,0cyclictest3556-21lldpd12:18:280
213949943,0cyclictest28001-21awk11:01:560
213949943,0cyclictest18736-21awk08:06:570
179940,0migration/18630-21taskset07:07:171
179940,0migration/129933-21kthreadcore11:21:581
179940,0migration/129326-21kthreadcore11:41:591
179940,0migration/127649-21aten_r1power_cu10:21:531
179940,0migration/125821-21taskset11:19:201
179940,0migration/120526-21fschecks_count09:36:541
179940,0migration/119311-21fschecks_time08:31:561
179940,0migration/119064-21irqrtprio08:06:581
179940,0migration/117491-21runrttasks11:13:291
179940,0migration/113563-21kthreadcore09:52:001
179940,0migration/113397-21kthreadcore10:11:591
179940,0migration/11329-21latency_hist12:26:411
179940,0migration/112276-21df_abs09:11:561
169940,0watchdog/13630-21aten_r1power_cu09:06:521
169940,0watchdog/129237-21sed07:47:041
169940,0watchdog/128919-21fschecks_time07:01:561
169940,0watchdog/126573-21ssh10:40:001
169940,0watchdog/11936-21ssh11:05:041
169940,0watchdog/116286-21diskstats07:16:561
169940,0watchdog/113485-21fschecks_count08:51:561
169940,0watchdog/111343-21hddtemp_smartct11:31:551
169940,0watchdog/11093-21missed_timers09:42:001
169940,0watchdog/10-21swapper/110:52:281
169940,0watchdog/10-21swapper/110:46:521
169940,0watchdog/10-21swapper/110:06:521
149940,0migration/04749-21kthreadcore10:26:590
149940,0migration/030450-21kthreadcore11:21:580
149940,0migration/022727-21cpu07:21:530
149940,0migration/011869-21cpuspeed_turbos10:11:550
339930,0migration/36514-21kthreadcore07:06:593
29725030,0irq/27-eth0-tx-0-21swapper/006:56:530
29715030,0irq/26-eth0-rx-0-21swapper/308:26:543
29715030,0irq/26-eth0-rx-0-21swapper/308:21:293
29715030,0irq/26-eth0-rx-0-21swapper/307:51:533
29715030,0irq/26-eth0-rx-0-21swapper/307:26:533
214039932,0cyclictest22569-21awk06:56:572
213959933,0cyclictest3556-21lldpd09:18:271
213959932,0cyclictest9442-21kthreadcore07:56:591
213959932,0cyclictest7276-21awk08:46:571
213959932,0cyclictest31370-21irqrtprio07:51:581
213959932,0cyclictest27829-21grep08:36:591
213959932,0cyclictest12272-21awk10:31:561
213959930,0cyclictest0-21swapper/108:43:071
213949933,0cyclictest8356-21kworker/0:011:36:490
213949933,0cyclictest19991-21kworker/0:210:56:430
213949933,0cyclictest18661-21kworker/0:008:51:570
213949933,0cyclictest18661-21kworker/0:008:46:550
213949933,0cyclictest0-21swapper/008:36:440
213949932,0cyclictest32605-21tune2fs08:41:550
213949932,0cyclictest3241-21irqbalance07:13:510
213949932,0cyclictest15871-21sensors07:37:030
213949932,0cyclictest12296-21awk10:31:560
169930,0watchdog/119444-21diskmemload10:16:521
169930,0watchdog/10-21swapper/111:26:591
169930,0watchdog/10-21swapper/107:21:561
35120,0ktimersoftd/318638-21rt-features08:52:043
329920,0watchdog/36418-21aten_r1power_fr08:46:533
29715020,0irq/26-eth0-rx-0-21swapper/308:32:163
29715020,0irq/26-eth0-rx-0-21swapper/308:07:023
29715020,0irq/26-eth0-rx-0-21swapper/307:01:453
213959921,0cyclictest3251-21latency_hist07:31:421
169920,0watchdog/10-21swapper/108:21:521
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional