You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 21:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Fri May 01, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191790,0ktimersoftd/129616-21perf06:56:441
304450350,0irq/25-eth10-21swapper/306:52:063
365023425,0sleep20-21swapper/206:54:382
384722723,0sleep00-21swapper/006:56:480
39249983,0cyclictest3356-21awk09:57:061
39249983,0cyclictest3163-21awk08:57:071
39249973,0cyclictest7811-21sadc08:36:541
39249973,0cyclictest14625-21awk08:17:071
39249972,0cyclictest2491-21awk10:57:061
39249970,0cyclictest23771-21kthreadcore09:07:091
39249970,0cyclictest12417-21needreboot08:12:111
4160,0ktimersoftd/02045-21diskmemload10:28:520
39269960,0cyclictest0-21swapper/211:45:242
39249962,0cyclictest3514-21snmpd09:05:281
39249962,0cyclictest3107-21awk10:17:051
39249962,0cyclictest27861-21awk09:12:071
39249962,0cyclictest25148-21irqcore07:12:091
39249962,0cyclictest1265-21awk08:07:081
39249962,0cyclictest11864-21awk09:22:071
39249961,0cyclictest9671-21awk12:22:061
39249961,0cyclictest1143-21munin-node11:37:031
39249961,0cyclictest10580-21awk11:02:071
39249960,0cyclictest9637-21ntp_offset06:57:131
39249960,0cyclictest6812-21cut10:37:071
39249960,0cyclictest29530-21grep09:32:111
39249960,0cyclictest29085-21kthreadcore11:12:081
39249960,0cyclictest28291-21grep08:52:061
39249960,0cyclictest27754-21kthreadcore08:02:101
39249960,0cyclictest2767-21munin-node09:17:131
39249960,0cyclictest27026-21perl10:32:051
39249960,0cyclictest26692-21fschecks_time10:52:051
39249960,0cyclictest26660-21kthreadcore11:52:091
39249960,0cyclictest26078-21latency_hist10:31:521
39249960,0cyclictest25508-21munin-run11:11:521
39249960,0cyclictest24763-21latency_hist12:11:521
39249960,0cyclictest23967-21ssh11:28:131
39249960,0cyclictest23759-21python10:47:141
39249960,0cyclictest23194-21kthreadcore08:47:101
39249960,0cyclictest22170-21kthreadcore11:47:091
39249960,0cyclictest20385-21cut09:47:081
39249960,0cyclictest19732-21irqcore10:07:091
39249960,0cyclictest19310-21munin-node07:32:081
39249960,0cyclictest19141-21basename09:27:041
39249960,0cyclictest18179-21tail07:07:051
39249960,0cyclictest17919-21ssh09:45:111
39249960,0cyclictest17252-21munin-node12:07:041
39249960,0cyclictest15791-21cut10:02:101
39249960,0cyclictest14852-21cut07:52:101
39249960,0cyclictest14790-21munin-node11:42:111
39249960,0cyclictest12016-21irqrtprio07:02:101
39249960,0cyclictest11777-21munin-node07:27:121
39249960,0cyclictest10980-21cut10:22:051
39249960,0cyclictest10705-21gltestperf10:42:051
39249960,0cyclictest10481-21sh09:41:521
39249960,0cyclictest0-21swapper/112:17:591
39249960,0cyclictest0-21swapper/111:58:071
39239966,0cyclictest0-21swapper/011:17:060
39239960,0cyclictest0-21swapper/011:42:290
339960,0migration/38328-21sh10:58:443
339960,0migration/31927-21sh09:35:423
339960,0migration/318056-21aten_r1power_en10:47:043
339960,0migration/317959-21sh10:26:393
339960,0migration/316621-21runrttasks11:04:273
339960,0migration/310597-21sh10:01:583
339960,0migration/310071-21munin-node11:42:063
29725060,0irq/27-eth0-tx-10847-21munin-node07:02:100
29715060,0irq/26-eth0-rx-319-21ssh11:56:153
29715060,0irq/26-eth0-rx-31261-21grep10:32:103
29715060,0irq/26-eth0-rx-12365-21grep09:22:093
29715060,0irq/26-eth0-rx-0-21swapper/310:52:363
29715060,0irq/26-eth0-rx-0-21swapper/310:11:353
149960,0migration/028477-21kthreadcore08:02:100
149960,0migration/024846-21sh11:10:010
149960,0migration/024494-21timerandwakeup10:07:150
149960,0migration/018057-21sh10:06:020
4150,0ktimersoftd/00-21swapper/009:59:510
39269955,0cyclictest0-21swapper/211:29:472
39269955,0cyclictest0-21swapper/211:02:122
39269955,0cyclictest0-21swapper/210:47:322
39269950,0cyclictest0-21swapper/212:04:192
39269950,0cyclictest0-21swapper/211:22:232
39269950,0cyclictest0-21swapper/210:54:292
39269950,0cyclictest0-21swapper/209:36:172
39249950,0cyclictest9426-21sh11:21:541
39249950,0cyclictest8970-21ssh12:01:531
39249950,0cyclictest8001-21ls08:36:541
39249950,0cyclictest8001-21ls08:36:541
39249950,0cyclictest5997-21cat07:46:541
39249950,0cyclictest5120-21munin-node07:22:031
39249950,0cyclictest31504-21latency_hist07:41:541
39249950,0cyclictest30527-21latency_hist07:16:541
39249950,0cyclictest27064-21basename08:27:041
39249950,0cyclictest26789-21rm09:51:551
39249950,0cyclictest26500-21grep10:11:531
39249950,0cyclictest25428-21cat11:31:521
39249950,0cyclictest24827-21latency_hist07:36:541
39249950,0cyclictest20325-21latency_hist08:21:541
39249950,0cyclictest19399-21cut07:57:041
39249950,0cyclictest14607-21date08:41:541
39249950,0cyclictest1274-21sed11:16:521
39239955,0cyclictest23303-21ssh12:07:430
39239955,0cyclictest0-21swapper/011:33:300
39239955,0cyclictest0-21swapper/010:19:150
39239955,0cyclictest0-21swapper/009:17:270
39239955,0cyclictest0-21swapper/008:52:130
39239950,0cyclictest0-21swapper/009:35:160
339950,0migration/38973-21taskset09:37:333
339950,0migration/38086-21kthreadcore09:17:103
339950,0migration/36859-21munin-node08:32:133
339950,0migration/36779-21kthreadcore10:37:073
339950,0migration/36059-21latency12:17:093
339950,0migration/328252-21munin-node09:12:083
339950,0migration/327117-21kthreadcore11:12:083
339950,0migration/325062-21runrttasks09:47:373
339950,0migration/322910-21timerandwakeup12:07:143
339950,0migration/32086-21kthreadcore07:17:103
339950,0migration/320844-21kthreadcore07:32:103
339950,0migration/319644-21munin-node10:27:083
339950,0migration/319079-21needreboot08:17:113
339950,0migration/319015-21irqrtprio11:07:083
339950,0migration/31485-21taskset10:15:333
339950,0migration/311481-21kthreadcore11:22:083
339950,0migration/310773-21munin-node07:22:133
339950,0migration/310119-21taskset10:42:033
29725050,0irq/27-eth0-tx-32409-21ntpdc09:52:122
29725050,0irq/27-eth0-tx-29998-21tr11:32:112
29725050,0irq/27-eth0-tx-20368-21munin-node08:22:050
29725050,0irq/27-eth0-tx-1877-21munin-node10:37:072
29725050,0irq/27-eth0-tx-0-21swapper/209:12:142
29725050,0irq/27-eth0-tx-0-21swapper/008:57:100
29725050,0irq/27-eth0-tx-0-21swapper/008:47:150
29715050,0irq/26-eth0-rx-7151-21yum11:57:163
29715050,0irq/26-eth0-rx-6415-21chrt08:07:393
29715050,0irq/26-eth0-rx-27053-21kthreadcore07:12:103
29715050,0irq/26-eth0-rx-21123-21grep09:27:093
29715050,0irq/26-eth0-rx-15470-21ssh12:03:133
29715050,0irq/26-eth0-rx-13637-21kthreadcore09:02:103
29715050,0irq/26-eth0-rx-11565-21grep12:22:083
29715050,0irq/26-eth0-rx-0-21swapper/311:37:033
29715050,0irq/26-eth0-rx-0-21swapper/311:32:113
29715050,0irq/26-eth0-rx-0-21swapper/311:31:333
29715050,0irq/26-eth0-rx-0-21swapper/311:17:363
29715050,0irq/26-eth0-rx-0-21swapper/310:17:043
29715050,0irq/26-eth0-rx-0-21swapper/309:46:123
29715050,0irq/26-eth0-rx-0-21swapper/309:07:473
29715050,0irq/26-eth0-rx-0-21swapper/309:00:193
29715050,0irq/26-eth0-rx-0-21swapper/308:42:043
29715050,0irq/26-eth0-rx-0-21swapper/307:11:543
149950,0migration/06518-21kthreadcore11:37:090
149950,0migration/06453-21rt-features12:17:120
149950,0migration/06248-21kthreadcore11:57:090
149950,0migration/03788-21kthreadcore10:57:080
149950,0migration/031058-21unixbench_singl07:37:150
149950,0migration/030063-21kthreadcore09:12:090
149950,0migration/027232-21kthreadcore11:12:080
149950,0migration/026560-21cpuspeed_turbos10:32:040
149950,0migration/024105-21latency_hist07:11:540
149950,0migration/018729-21irqrtprio07:07:080
149950,0migration/015105-21kthreadcore07:52:100
149950,0migration/014500-21latency_hist08:41:540
149950,0migration/014500-21latency_hist08:41:540
149950,0migration/014209-21kthreadcore11:02:110
149950,0migration/014016-21kthreadcore10:22:090
149950,0migration/012467-21taskset07:51:370
149950,0migration/011530-21kthreadcore11:22:080
149950,0migration/010786-21munin-node07:22:130
149950,0migration/010370-21taskset06:59:090
149950,0migration/010184-21sh09:41:030
149950,0migration/010089-21aten_r1power_po10:42:030
39269943,0cyclictest8288-21ntpq09:37:112
39269943,0cyclictest5924-21awk07:22:062
39269943,0cyclictest3556-21lldpd08:32:112
39269943,0cyclictest3556-21lldpd08:28:402
39269943,0cyclictest3514-21snmpd07:52:582
39269943,0cyclictest3514-21snmpd07:07:282
39269943,0cyclictest3241-21irqbalance11:47:012
39269943,0cyclictest3241-21irqbalance09:23:212
39269943,0cyclictest3241-21irqbalance09:07:022
39269943,0cyclictest3241-21irqbalance08:58:122
39269943,0cyclictest3241-21irqbalance08:52:022
39269943,0cyclictest3241-21irqbalance08:47:222
39269943,0cyclictest20263-21awk07:57:082
39269943,0cyclictest19525-21awk10:07:072
39269943,0cyclictest19167-21awk10:27:062
39269943,0cyclictest11715-21awk07:02:092
39269943,0cyclictest1169-21awk08:07:082
39269943,0cyclictest11487-21awk10:02:072
39269943,0cyclictest11375-21awk09:02:082
39239944,0cyclictest0-21swapper/010:51:340
39239943,0cyclictest7472-21kworker/0:111:52:080
39239943,0cyclictest3514-21snmpd12:04:430
39239943,0cyclictest3514-21snmpd10:55:430
39239943,0cyclictest30757-21munin-node07:17:130
39239943,0cyclictest2177-21awk08:32:080
39239943,0cyclictest19959-21awk09:27:070
39239941,0cyclictest0-21swapper/008:42:040
339940,0migration/322478-21kthreadcore08:22:103
339940,0migration/319370-21kthreadcore11:47:083
339940,0migration/312087-21kthreadcore08:12:103
29725040,0irq/27-eth0-tx-31308-21kthreadcore10:12:102
29725040,0irq/27-eth0-tx-17402-21ssh09:43:432
29725040,0irq/27-eth0-tx-1143-21munin-node11:37:042
29725040,0irq/27-eth0-tx-0-21swapper/212:24:072
29725040,0irq/27-eth0-tx-0-21swapper/212:12:042
29725040,0irq/27-eth0-tx-0-21swapper/212:08:002
29725040,0irq/27-eth0-tx-0-21swapper/211:52:052
29725040,0irq/27-eth0-tx-0-21swapper/211:11:572
29725040,0irq/27-eth0-tx-0-21swapper/211:07:042
29725040,0irq/27-eth0-tx-0-21swapper/210:58:212
29725040,0irq/27-eth0-tx-0-21swapper/210:42:092
29725040,0irq/27-eth0-tx-0-21swapper/210:32:242
29725040,0irq/27-eth0-tx-0-21swapper/210:22:012
29725040,0irq/27-eth0-tx-0-21swapper/210:17:152
29725040,0irq/27-eth0-tx-0-21swapper/209:57:022
29725040,0irq/27-eth0-tx-0-21swapper/209:28:372
29725040,0irq/27-eth0-tx-0-21swapper/209:17:182
29715040,0irq/26-eth0-rx-786-21taskset08:28:583
29715040,0irq/26-eth0-rx-0-21swapper/312:12:043
29715040,0irq/26-eth0-rx-0-21swapper/309:57:133
29715040,0irq/26-eth0-rx-0-21swapper/309:51:573
29715040,0irq/26-eth0-rx-0-21swapper/307:42:023
259940,0migration/27115-21kthreadcore06:57:102
259940,0migration/26878-21sh12:17:162
259940,0migration/23250-21kthreadcore11:17:062
259940,0migration/232259-21runrttasks08:04:362
259940,0migration/224194-21munin-node09:47:102
259940,0migration/22072-21munin-node11:57:082
259940,0migration/220522-21timerwakeupswit08:42:152
249940,0watchdog/229470-21tr07:12:122
249940,0watchdog/21305-21kthreadcore07:42:082
149940,0migration/024875-21open_files07:57:130
149940,0migration/020396-21kthreadcore07:32:100
149940,0migration/020034-21irqrtprio09:47:070
149940,0migration/019334-21rt-features08:17:130
149940,0migration/017808-21df11:27:040
149940,0migration/017070-21runrttasks09:42:430
149940,0migration/014390-21kthreadcore07:27:090
8215030,0irq/24-0000:00:0-21swapper/207:47:062
39269932,0cyclictest4093-21sensors07:17:152
39269932,0cyclictest3514-21snmpd07:38:432
39269932,0cyclictest21236-21awk08:22:072
39269932,0cyclictest14547-21awk08:17:072
39269932,0cyclictest12659-21awk07:27:072
39239933,0cyclictest7472-21kworker/0:112:22:020
39239933,0cyclictest7472-21kworker/0:112:11:540
39239933,0cyclictest15258-21kworker/0:010:11:540
39239933,0cyclictest15258-21kworker/0:009:52:020
39239933,0cyclictest15258-21kworker/0:009:22:020
39239933,0cyclictest15258-21kworker/0:009:07:050
39239933,0cyclictest15258-21kworker/0:009:02:070
39239933,0cyclictest15258-21kworker/0:008:27:090
39239933,0cyclictest15258-21kworker/0:008:07:110
39239932,0cyclictest3514-21snmpd11:46:580
39239930,0cyclictest7436-21diskstats08:12:050
39239930,0cyclictest2597-21perl10:37:030
339930,0migration/315390-21kthreadcore07:02:123
29725030,0irq/27-eth0-tx-0-21swapper/007:42:020
29715030,0irq/26-eth0-rx-0-21swapper/308:52:163
29715030,0irq/26-eth0-rx-0-21swapper/308:47:053
29715030,0irq/26-eth0-rx-0-21swapper/308:37:043
29715030,0irq/26-eth0-rx-0-21swapper/308:37:043
29715030,0irq/26-eth0-rx-0-21swapper/307:57:043
29715030,0irq/26-eth0-rx-0-21swapper/307:52:043
29715030,0irq/26-eth0-rx-0-21swapper/307:48:293
29715030,0irq/26-eth0-rx-0-21swapper/307:39:203
29715030,0irq/26-eth0-rx-0-21swapper/307:27:033
249930,0watchdog/20-21swapper/207:35:242
39269920,0cyclictest0-21swapper/208:11:542
339920,0migration/34532-21diskstats06:57:063
29715020,0irq/26-eth0-rx-0-21swapper/308:01:583
249920,0watchdog/20-21swapper/208:37:032
249920,0watchdog/20-21swapper/208:37:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional