You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 16:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Dec 15, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1748427372,0sleep00-21swapper/007:05:250
304450350,0irq/25-eth10-21swapper/307:07:293
1711423531,0sleep20-21swapper/207:05:072
1763122925,0sleep10-21swapper/107:07:061
17930991919,0cyclictest14453-21kworker/2:209:12:252
339960,0migration/38882-21timerwakeupswit11:50:083
339960,0migration/326089-21sh09:31:173
339960,0migration/32066-21irqrtprio08:10:013
29715060,0irq/26-eth0-rx-25319-21if_eth111:24:593
29715060,0irq/26-eth0-rx-19954-21cut07:10:003
27160,0ktimersoftd/20-21swapper/209:32:452
19160,0ktimersoftd/10-21swapper/109:32:451
8215050,0irq/24-0000:00:19788-21munin-plugin-st07:59:472
339950,0migration/39035-21fschecks_count08:40:003
339950,0migration/37881-21unixbench_multi08:35:093
339950,0migration/37131-21kthreadcore09:00:053
339950,0migration/35726-21kthreadcore11:30:023
339950,0migration/330253-21kthreadcore08:30:023
339950,0migration/329999-21kthreadcore08:55:033
339950,0migration/327490-21sh09:50:033
339950,0migration/321277-21irqrtprio10:45:033
339950,0migration/318198-21switchtime07:30:083
339950,0migration/317980-21taskset11:39:563
339950,0migration/31673-21kthreadcore12:05:043
339950,0migration/313135-21diskmemload12:14:003
339950,0migration/312230-21kthreadcore09:25:023
339950,0migration/311407-21sh10:55:453
339950,0migration/311341-21kthreadcore11:15:043
339950,0migration/310925-21kernelversion09:05:043
29725050,0irq/27-eth0-tx-32059-21ssh09:17:112
29725050,0irq/27-eth0-tx-22379-21ssh12:19:542
29725050,0irq/27-eth0-tx-21942-21munin-node08:49:590
29725050,0irq/27-eth0-tx-16063-21diskmemload10:41:172
29725050,0irq/27-eth0-tx-14680-21ssh10:03:352
29725050,0irq/27-eth0-tx-14479-21if_err_eth1.211:00:012
29725050,0irq/27-eth0-tx-12093-21ssh10:57:132
29725050,0irq/27-eth0-tx-0-21swapper/210:34:582
29725050,0irq/27-eth0-tx-0-21swapper/210:30:052
29715050,0irq/26-eth0-rx-28891-21cpuspeed_turbos10:50:003
29715050,0irq/26-eth0-rx-27877-21kernelversion11:45:033
29715050,0irq/26-eth0-rx-27220-21sh12:02:023
29715050,0irq/26-eth0-rx-20536-21grep10:25:033
29715050,0irq/26-eth0-rx-19742-21yum11:00:103
29715050,0irq/26-eth0-rx-18637-21kthreadcore11:20:033
29715050,0irq/26-eth0-rx-1861-21fschecks_count09:19:593
29715050,0irq/26-eth0-rx-1861-21fschecks_count09:19:593
29715050,0irq/26-eth0-rx-14498-21cut11:55:033
29715050,0irq/26-eth0-rx-0-21swapper/312:30:063
29715050,0irq/26-eth0-rx-0-21swapper/312:25:013
29715050,0irq/26-eth0-rx-0-21swapper/312:19:583
29715050,0irq/26-eth0-rx-0-21swapper/312:19:243
29715050,0irq/26-eth0-rx-0-21swapper/311:37:163
29715050,0irq/26-eth0-rx-0-21swapper/311:11:083
29715050,0irq/26-eth0-rx-0-21swapper/311:04:593
29715050,0irq/26-eth0-rx-0-21swapper/310:40:003
29715050,0irq/26-eth0-rx-0-21swapper/310:31:073
29715050,0irq/26-eth0-rx-0-21swapper/310:21:323
29715050,0irq/26-eth0-rx-0-21swapper/310:07:543
29715050,0irq/26-eth0-rx-0-21swapper/310:02:193
29715050,0irq/26-eth0-rx-0-21swapper/309:54:503
29715050,0irq/26-eth0-rx-0-21swapper/309:45:023
29715050,0irq/26-eth0-rx-0-21swapper/309:40:023
29715050,0irq/26-eth0-rx-0-21swapper/309:36:363
29715050,0irq/26-eth0-rx-0-21swapper/309:15:403
29715050,0irq/26-eth0-rx-0-21swapper/307:39:583
259950,0migration/28345-21fschecks_time08:14:592
259950,0migration/27019-21munin-node08:10:082
259950,0migration/24684-21timerwakeupswit12:25:082
259950,0migration/24086-21munin-node11:10:062
259950,0migration/2386-21kthreadcore12:05:032
259950,0migration/228521-21sshd11:05:292
259950,0migration/227515-21kthreadcore07:40:042
259950,0migration/223806-21munin-node07:10:042
259950,0migration/221744-21taskset10:05:142
259950,0migration/219892-21kthreadcore11:40:022
259950,0migration/218200-21kthreadcore12:35:022
259950,0migration/210801-21sh12:10:062
249950,0watchdog/225115-21ssh10:27:452
249950,0watchdog/217087-21ssh09:26:092
19150,0ktimersoftd/10-21swapper/112:32:231
179309955,0cyclictest0-21swapper/209:36:542
179309955,0cyclictest0-21swapper/208:05:092
179309950,0cyclictest0-21swapper/209:47:002
179309950,0cyclictest0-21swapper/207:50:102
179289953,0cyclictest14203-21awk07:55:011
179289950,0cyclictest0-21swapper/112:18:121
179279955,0cyclictest0-21swapper/011:05:010
179950,0migration/16472-21taskset11:14:051
179950,0migration/130540-21taskset12:24:211
179950,0migration/125532-21fschecks_time09:14:591
179950,0migration/124373-21aten_r1power_cu11:24:541
179950,0migration/122046-21sh10:05:521
179950,0migration/120471-21kthreadcore08:45:051
179950,0migration/117029-21kthreadcore08:20:031
179950,0migration/112858-21kthreadcore09:25:021
179950,0migration/110481-21kthreadcore10:20:021
179950,0migration/110344-21kthreadcore08:15:031
169950,0watchdog/1323-21diskstats07:44:571
169950,0watchdog/132172-21python10:30:091
149950,0migration/0698-21runrttasks07:44:580
149950,0migration/05451-21sh11:11:560
149950,0migration/05229-21kthreadcore08:35:030
149950,0migration/032402-21switchtime11:45:090
149950,0migration/030646-21kthreadcore09:35:040
149950,0migration/026454-21sh12:00:260
149950,0migration/022849-21kthreadcore11:40:030
149950,0migration/022396-21sh12:35:450
149950,0migration/022320-21kthreadcore08:25:030
149950,0migration/021754-21kthreadcore07:35:050
149950,0migration/020197-21munin-node09:30:000
149950,0migration/016839-21irqrtprio10:05:010
149950,0migration/016164-21kthreadcore12:15:050
149950,0migration/015506-21latency_hist11:19:460
149950,0migration/014258-21munin-node07:55:010
149950,0migration/013267-21sh10:00:340
149950,0migration/01240-21sh10:32:340
149950,0migration/01131-21gltestperf10:15:000
149950,0migration/01037-21kthreadcore12:05:030
339940,0migration/38785-21irqcore08:15:023
339940,0migration/38438-21diskmemload10:35:073
339940,0migration/33333-21kthreadcore07:45:023
339940,0migration/331785-21kthreadcore08:05:053
339940,0migration/32493-21kthreadcore07:20:023
339940,0migration/323839-21kthreadcore07:35:053
339940,0migration/321498-21kthreadcore08:00:043
339940,0migration/320374-21taskset08:20:453
339940,0migration/310463-21kthreadcore07:50:033
29725040,0irq/27-eth0-tx-9319-21ssh11:34:572
29725040,0irq/27-eth0-tx-0-21swapper/212:14:572
29725040,0irq/27-eth0-tx-0-21swapper/211:55:002
29725040,0irq/27-eth0-tx-0-21swapper/211:50:272
29725040,0irq/27-eth0-tx-0-21swapper/211:29:502
29725040,0irq/27-eth0-tx-0-21swapper/211:25:002
29725040,0irq/27-eth0-tx-0-21swapper/210:49:592
29725040,0irq/27-eth0-tx-0-21swapper/210:45:032
29725040,0irq/27-eth0-tx-0-21swapper/210:19:462
29725040,0irq/27-eth0-tx-0-21swapper/209:54:512
29725040,0irq/27-eth0-tx-0-21swapper/209:20:002
29725040,0irq/27-eth0-tx-0-21swapper/209:20:002
29725040,0irq/27-eth0-tx-0-21swapper/008:59:590
29715040,0irq/26-eth0-rx-6178-21diskstats07:24:593
29715040,0irq/26-eth0-rx-0-21swapper/312:35:023
29715040,0irq/26-eth0-rx-0-21swapper/310:14:573
29715040,0irq/26-eth0-rx-0-21swapper/310:10:193
29715040,0irq/26-eth0-rx-0-21swapper/309:11:433
259940,0migration/28120-21kthreadcore09:40:022
259940,0migration/27043-21kthreadcore09:00:052
259940,0migration/229212-21kthreadcore07:15:052
259940,0migration/228362-21gltestperf08:29:592
259940,0migration/228188-21kthreadcore11:45:032
259940,0migration/22756-21kthreadcore07:20:022
259940,0migration/226497-21kthreadcore10:10:032
259940,0migration/223623-21kthreadcore09:50:012
259940,0migration/220821-21fschecks_time11:59:592
259940,0migration/220029-21aten_r1power_cu07:59:582
259940,0migration/214776-21fschecks_count08:20:002
249940,0watchdog/27113-21ssh11:14:572
249940,0watchdog/25889-21sh10:15:052
249940,0watchdog/221864-21if_eth008:25:012
179309943,0cyclictest807-21awk07:44:592
179309943,0cyclictest3514-21snmpd12:29:532
179309943,0cyclictest3514-21snmpd09:06:082
179309943,0cyclictest3514-21snmpd08:53:382
179309943,0cyclictest3514-21snmpd08:39:532
179309943,0cyclictest3514-21snmpd08:38:532
179309943,0cyclictest3514-21snmpd07:25:382
179309943,0cyclictest29502-21awk08:55:012
179309943,0cyclictest20003-21awk07:35:032
179309943,0cyclictest13343-21awk07:30:012
179309940,0cyclictest15738-21munin-node11:20:092
179289944,0cyclictest0-21swapper/111:04:341
179289944,0cyclictest0-21swapper/109:50:001
179289944,0cyclictest0-21swapper/109:48:441
179289943,0cyclictest9562-21awk08:40:021
179289943,0cyclictest6789-21awk07:25:021
179289943,0cyclictest3883-21awk09:00:021
179289943,0cyclictest3514-21snmpd10:55:081
179289943,0cyclictest3514-21snmpd10:35:081
179289943,0cyclictest3514-21snmpd10:01:231
179289943,0cyclictest29456-21awk08:55:011
179289943,0cyclictest28447-21awk08:30:001
179289943,0cyclictest27530-21awk08:05:021
179289943,0cyclictest27379-21sadc08:29:471
179289943,0cyclictest25037-21awk10:10:011
179289943,0cyclictest20918-21awk08:00:021
179289943,0cyclictest20870-21awk10:45:011
179289943,0cyclictest10201-21awk11:35:011
179279943,0cyclictest5589-21awk09:40:000
179279943,0cyclictest3514-21snmpd11:51:080
179279943,0cyclictest3514-21snmpd08:06:230
179279943,0cyclictest3514-21snmpd07:51:530
179279943,0cyclictest29523-21awk10:50:030
179279943,0cyclictest25749-21awk09:15:000
179279943,0cyclictest25053-21awk10:10:010
179279943,0cyclictest18975-21awk07:09:580
179279943,0cyclictest18769-21awk10:25:010
179279943,0cyclictest1799-21awk08:10:000
179279943,0cyclictest16654-21awk11:20:010
179279943,0cyclictest15035-21grep08:20:000
179279943,0cyclictest1472-21awk11:30:000
179279943,0cyclictest14464-21awk11:00:000
179279943,0cyclictest14255-21awk09:45:000
179940,0migration/19549-21sh09:23:451
179940,0migration/19549-21sh09:23:451
179940,0migration/17404-21taskset08:14:271
179940,0migration/1600-21kthreadcore12:05:031
179940,0migration/15148-21kthreadcore11:50:021
179940,0migration/127423-21kthreadcore07:40:041
179940,0migration/127289-21gltestperf11:45:001
179940,0migration/12547-21fschecks_time08:34:591
179940,0migration/122465-21kthreadcore07:10:001
179940,0migration/12188-21kthreadcore10:15:031
179940,0migration/121274-21kthreadcore07:35:041
179940,0migration/120594-21kthreadcore12:35:031
179940,0migration/120139-21kthreadcore11:20:041
179940,0migration/117892-21aten_r1power_vo10:24:581
179940,0migration/117292-21kthreadcore07:30:031
179940,0migration/116196-21latency_hist09:09:471
179940,0migration/111563-21cpuspeed_turbos10:39:591
169940,0watchdog/15728-21iostat_ios09:40:011
169940,0watchdog/14738-21tail12:25:081
169940,0watchdog/1308862chrt07:15:491
169940,0watchdog/12492-1kworker/1:2H10:50:331
169940,0watchdog/118102-21ssh11:55:561
169940,0watchdog/114979-21ssh11:18:491
169940,0watchdog/10-21swapper/111:30:041
149940,0migration/09657-21runrttasks09:23:570
149940,0migration/09657-21runrttasks09:23:570
149940,0migration/08502-21sh10:35:080
149940,0migration/030945-21kthreadcore08:55:030
149940,0migration/030210-21kthreadcore07:40:050
149940,0migration/027787-21sh09:50:070
149940,0migration/027658-21aten_r1power_en08:29:570
149940,0migration/027375-21kthreadcore11:25:010
149940,0migration/01753-21kthreadcore07:20:020
149940,0migration/017371-21sh10:24:360
149940,0migration/017044-21missed_timers11:55:050
149940,0migration/015580-21unixbench_singl09:05:100
149940,0migration/012719-21kthreadcore10:40:030
149940,0migration/010688-21kthreadcore07:25:040
29725030,0irq/27-eth0-tx-0-21swapper/008:44:590
29715030,0irq/26-eth0-rx-0-21swapper/308:44:593
29715030,0irq/26-eth0-rx-0-21swapper/308:28:563
29715030,0irq/26-eth0-rx-0-21swapper/307:54:493
179309931,0cyclictest0-21swapper/208:44:472
179289933,0cyclictest3241-21irqbalance11:59:551
179289933,0cyclictest3241-21irqbalance08:50:461
179289933,0cyclictest3241-21irqbalance07:20:161
179289933,0cyclictest0-21swapper/111:04:531
179289932,0cyclictest28871-21perl09:35:001
179289932,0cyclictest23753-21sensors11:40:071
179289932,0cyclictest12559-21kthreadcore09:05:041
179289932,0cyclictest12353-21sensors07:50:071
179279933,0cyclictest13822-21kworker/0:112:09:450
179279932,0cyclictest9490-21awk08:40:020
179279932,0cyclictest8198-21awk12:30:000
179279932,0cyclictest3514-21snmpd07:32:230
179279932,0cyclictest25677-21awk07:15:020
179279932,0cyclictest22097-21sensors09:10:070
179279932,0cyclictest19996-21munin-node10:45:060
179279932,0cyclictest16207-21sensors09:25:070
179279932,0cyclictest10217-21awk11:35:010
179279930,0cyclictest9227-21cut08:15:030
179279930,0cyclictest5342-21diskstats10:55:000
179279930,0cyclictest31744-21perl12:24:590
179279930,0cyclictest0-21swapper/007:59:580
169930,0watchdog/10-21swapper/112:09:521
169930,0watchdog/10-21swapper/109:55:251
29715020,0irq/26-eth0-rx-0-21swapper/308:50:063
29715020,0irq/26-eth0-rx-0-21swapper/307:15:103
179279921,0cyclictest0-21swapper/012:19:460
179279921,0cyclictest0-21swapper/009:54:500
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional