You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 04:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Mar 20, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2426628278,0sleep10-21swapper/118:59:421
304450520,0irq/25-eth10-21swapper/318:57:123
2419423524,0sleep00-21swapper/018:58:530
1841723026,0sleep20-21swapper/218:56:072
339960,0migration/35913-21sh21:09:323
339960,0migration/34931-21sh21:46:283
339960,0migration/329794-21fschecks_count23:46:073
339960,0migration/314395-21sh21:55:323
339960,0migration/313231-21sh22:13:093
29725060,0irq/27-eth0-tx-15264-21munin-node20:31:060
29715060,0irq/26-eth0-rx-30865-21iostat_ios23:06:083
29715060,0irq/26-eth0-rx-29244-21ls00:06:053
29715060,0irq/26-eth0-rx-12728-21port_sshpriv21:31:133
244469965,0cyclictest7354-21ssh22:11:081
149960,0migration/029575-21sh22:44:550
149960,0migration/028908-21latency_hist00:05:530
149960,0migration/028736-21switchtime21:01:160
339950,0migration/36014-21idleruntime23:31:063
339950,0migration/35630-21fschecks_time23:51:073
339950,0migration/34225-21munin-node22:06:153
339950,0migration/33726-21pmu-power22:26:133
339950,0migration/332175-21latency_hist19:30:533
339950,0migration/331125-21latency_hist19:05:533
339950,0migration/32843-21munin-node20:21:103
339950,0migration/324882-21kthreadcore21:41:093
339950,0migration/324304-21kthreadcore21:01:103
339950,0migration/322471-21taskset20:58:333
339950,0migration/32118-21munin-node19:56:093
339950,0migration/320652-21munin-node21:36:133
339950,0migration/320531-21taskset22:57:553
339950,0migration/316630-21kthreadcore23:16:093
339950,0migration/314512-21munin-node20:26:163
339950,0migration/314438-21kthreadcore00:16:103
339950,0migration/312267-21taskset00:15:323
339950,0migration/310580-21munin-node20:51:073
339950,0migration/310281-21kthreadcore21:11:113
29725050,0irq/27-eth0-tx-3944-21ssh23:48:432
29725050,0irq/27-eth0-tx-29544-21munin-node23:26:082
29725050,0irq/27-eth0-tx-27733-21taskset00:03:012
29725050,0irq/27-eth0-tx-13505-21ssh22:34:482
29715050,0irq/26-eth0-rx-30190-21ssh22:46:023
29715050,0irq/26-eth0-rx-28215-21kthreadcore19:26:093
29715050,0irq/26-eth0-rx-22744-21rm22:01:023
29715050,0irq/26-eth0-rx-19590-21kthreadcore21:16:113
29715050,0irq/26-eth0-rx-17017-21kthreadcore22:36:093
29715050,0irq/26-eth0-rx-0-21swapper/323:42:493
29715050,0irq/26-eth0-rx-0-21swapper/323:37:293
29715050,0irq/26-eth0-rx-0-21swapper/323:21:093
29715050,0irq/26-eth0-rx-0-21swapper/323:01:103
29715050,0irq/26-eth0-rx-0-21swapper/322:42:003
29715050,0irq/26-eth0-rx-0-21swapper/321:56:053
29715050,0irq/26-eth0-rx-0-21swapper/321:26:093
29715050,0irq/26-eth0-rx-0-21swapper/300:27:043
244519955,0cyclictest0-21swapper/220:31:102
244519954,0cyclictest3514-21snmpd22:12:542
244519950,0cyclictest0-21swapper/223:36:052
244469955,0cyclictest0-21swapper/121:59:471
244469950,0cyclictest0-21swapper/122:58:261
244469950,0cyclictest0-21swapper/122:42:211
244469950,0cyclictest0-21swapper/122:23:351
244369950,0cyclictest21922-21munin-run22:40:530
244369950,0cyclictest0-21swapper/021:36:080
179950,0migration/14178-21kernelversion20:46:101
179950,0migration/13779-21sh23:06:361
179950,0migration/131089-21cpuspeed_turbos21:06:041
179950,0migration/129594-21sh21:42:241
179950,0migration/128587-21sh22:01:141
179950,0migration/12484-21fschecks_time20:21:071
179950,0migration/124626-21munin-node19:21:131
179950,0migration/120073-21timerandwakeup22:36:151
179950,0migration/119921-21munin-node20:06:151
179950,0migration/11868-21kthreadcore22:26:101
179950,0migration/110766-21munin-node23:31:101
149950,0migration/09856-21kthreadcore21:11:110
149950,0migration/09258-21kthreadcore23:51:110
149950,0migration/08382-21kthreadcore22:51:090
149950,0migration/07834-21irqcore21:51:090
149950,0migration/07011-21df_abs19:36:050
149950,0migration/06768-21timerwakeupswit19:56:150
149950,0migration/0636-21kthreadcore21:06:090
149950,0migration/06090-21taskset22:31:010
149950,0migration/05344-21latency_hist23:10:530
149950,0migration/03364-21kthreadcore19:31:090
149950,0migration/031918-21taskset19:29:360
149950,0migration/027221-21sh00:01:290
149950,0migration/027195-21taskset20:14:070
149950,0migration/026511-21kthreadcore22:01:100
149950,0migration/025991-21kthreadcore23:21:100
149950,0migration/024579-21kthreadcore22:21:090
149950,0migration/023469-21kthreadcore19:46:090
149950,0migration/021425-21taskset23:00:320
149950,0migration/020674-21timerandwakeup21:56:160
149950,0migration/020591-21latency_hist00:20:530
149950,0migration/018328-21taskset19:16:190
149950,0migration/01806-21latency_hist20:20:530
149950,0migration/017139-21kthreadcore21:16:110
149950,0migration/011366-21sh23:31:150
149950,0migration/011170-21runrttasks00:12:180
149950,0migration/010063-21kthreadcore20:01:110
8215040,0irq/24-0000:00:29921-21latency_hist22:45:532
8215040,0irq/24-0000:00:0-21swapper/221:50:532
339940,0migration/325216-21kthreadcore20:36:103
339940,0migration/324995-21kthreadcore20:11:113
339940,0migration/321151-21aten_r1power_vo00:01:033
339940,0migration/315999-21kthreadcore20:06:113
339940,0migration/313666-21taskset20:01:293
29725040,0irq/27-eth0-tx-7846-21hostname21:31:072
29725040,0irq/27-eth0-tx-5116-21ssh21:07:322
29725040,0irq/27-eth0-tx-22869-21perl22:41:052
29725040,0irq/27-eth0-tx-13558-21munin-node23:16:042
29725040,0irq/27-eth0-tx-10614-21open_files23:51:142
29725040,0irq/27-eth0-tx-0-21swapper/223:42:032
29725040,0irq/27-eth0-tx-0-21swapper/223:31:202
29725040,0irq/27-eth0-tx-0-21swapper/223:24:302
29725040,0irq/27-eth0-tx-0-21swapper/223:00:562
29725040,0irq/27-eth0-tx-0-21swapper/222:26:032
29725040,0irq/27-eth0-tx-0-21swapper/222:16:112
29725040,0irq/27-eth0-tx-0-21swapper/222:06:552
29725040,0irq/27-eth0-tx-0-21swapper/222:01:252
29725040,0irq/27-eth0-tx-0-21swapper/221:41:082
29725040,0irq/27-eth0-tx-0-21swapper/221:16:082
29725040,0irq/27-eth0-tx-0-21swapper/221:04:002
29725040,0irq/27-eth0-tx-0-21swapper/200:21:082
29725040,0irq/27-eth0-tx-0-21swapper/200:06:112
29725040,0irq/27-eth0-tx-0-21swapper/020:56:070
29725040,0irq/27-eth0-tx-0-21swapper/020:06:060
29725040,0irq/27-eth0-tx-0-21swapper/019:11:100
29715040,0irq/26-eth0-rx-28819-21ntpdc21:21:133
29715040,0irq/26-eth0-rx-0-21swapper/323:56:043
29715040,0irq/26-eth0-rx-0-21swapper/323:26:013
29715040,0irq/26-eth0-rx-0-21swapper/323:11:353
29715040,0irq/26-eth0-rx-0-21swapper/322:51:213
29715040,0irq/26-eth0-rx-0-21swapper/322:31:033
29715040,0irq/26-eth0-rx-0-21swapper/322:21:213
29715040,0irq/26-eth0-rx-0-21swapper/322:17:373
29715040,0irq/26-eth0-rx-0-21swapper/320:31:053
29715040,0irq/26-eth0-rx-0-21swapper/319:51:103
29715040,0irq/26-eth0-rx-0-21swapper/319:16:113
29715040,0irq/26-eth0-rx-0-21swapper/319:11:033
29715040,0irq/26-eth0-rx-0-21swapper/319:06:103
29715040,0irq/26-eth0-rx-0-21swapper/300:21:053
259940,0migration/232244-21idleruntime21:26:072
259940,0migration/228294-21gltestperf20:16:082
259940,0migration/215991-21kthreadcore22:36:092
259940,0migration/214044-21gltestperf19:41:062
244519943,0cyclictest8514-21awk20:01:092
244519943,0cyclictest7678-21awk21:11:082
244519943,0cyclictest6440-21awk23:11:062
244519943,0cyclictest3556-21lldpd23:57:142
244519943,0cyclictest3514-21snmpd20:56:382
244519943,0cyclictest3514-21snmpd20:36:232
244519943,0cyclictest3514-21snmpd20:21:082
244519943,0cyclictest3514-21snmpd19:20:382
244519943,0cyclictest3514-21snmpd00:26:392
244519943,0cyclictest3241-21irqbalance22:51:312
244519943,0cyclictest3241-21irqbalance21:57:112
244519943,0cyclictest3241-21irqbalance20:46:022
244519943,0cyclictest3241-21irqbalance20:26:022
244519943,0cyclictest3241-21irqbalance20:11:022
244519943,0cyclictest31563-21tr19:06:052
244519943,0cyclictest29405-21awk20:41:082
244519943,0cyclictest26518-21awk19:26:072
244519943,0cyclictest24288-21awk21:21:092
244519943,0cyclictest13513-21awk00:16:072
244519940,0cyclictest0-21swapper/222:21:052
244519940,0cyclictest0-21swapper/200:11:342
244469944,0cyclictest0-21swapper/123:03:281
244469944,0cyclictest0-21swapper/122:31:001
244469943,0cyclictest6841-21awk22:51:071
244469943,0cyclictest3514-21snmpd23:57:391
244469943,0cyclictest3514-21snmpd23:21:241
244469943,0cyclictest3514-21snmpd23:16:391
244469943,0cyclictest3514-21snmpd21:25:531
244469943,0cyclictest3514-21snmpd20:31:381
244469943,0cyclictest3241-21irqbalance23:51:111
244469943,0cyclictest3241-21irqbalance21:01:021
244469943,0cyclictest32150-21awk19:06:071
244469943,0cyclictest32036-21sadc19:30:531
244469943,0cyclictest31796-21awk22:06:081
244469943,0cyclictest30344-21awk23:46:091
244469943,0cyclictest15935-21awk21:16:081
244469943,0cyclictest14189-21awk23:36:081
244469940,0cyclictest0-21swapper/100:30:021
244369944,0cyclictest0-21swapper/022:26:070
244369943,0cyclictest3514-21snmpd23:41:090
244369943,0cyclictest3514-21snmpd19:04:230
244369943,0cyclictest30244-21awk23:46:090
244369943,0cyclictest23954-21awk21:41:070
244369943,0cyclictest22665-21grep20:36:060
244369943,0cyclictest14530-21awk23:16:070
244369940,0cyclictest0-21swapper/021:46:330
249940,0watchdog/231854-21smartctl21:46:072
249940,0watchdog/229895-21hostname23:06:032
249940,0watchdog/227096-21tune2fs19:51:082
249940,0watchdog/2250402sleep219:22:002
249940,0watchdog/20-21swapper/222:56:032
249940,0watchdog/20-21swapper/221:36:072
249940,0watchdog/20-21swapper/220:06:122
179940,0migration/17985-21kthreadcore19:36:091
179940,0migration/17631-21aten2.4-expect20:01:051
179940,0migration/129615-21aten_r1power_fr23:26:031
179940,0migration/127941-21runrttasks20:36:231
179940,0migration/127085-21kthreadcore21:21:121
179940,0migration/125911-21kthreadcore00:21:101
179940,0migration/123783-21kthreadcore00:01:081
179940,0migration/121873-21processes20:56:141
179940,0migration/120333-21timerandwakeup22:16:171
179940,0migration/11748-21kthreadcore19:31:091
179940,0migration/116831-21kthreadcore19:16:111
179940,0migration/116326-21irqrtprio21:36:091
179940,0migration/114416-21sh21:55:361
179940,0migration/11280-21kthreadcore22:46:101
169940,0watchdog/13380-21ssh00:08:071
169940,0watchdog/122241-21if_eth1.323:41:071
169940,0watchdog/114288-21iostat_ios19:41:081
169940,0watchdog/110593-21kthreadcore21:11:111
169940,0watchdog/10-21swapper/121:46:111
169940,0watchdog/10-21swapper/100:16:111
149940,0migration/09739-21kthreadcore21:31:090
149940,0migration/09244-21kthreadcore22:11:100
149940,0migration/028896-21runrttasks23:03:530
149940,0migration/026324-21kthreadcore21:21:110
149940,0migration/023538-21kthreadcore19:21:090
149940,0migration/01621-21idleruntime-cro20:20:530
149940,0migration/015195-21kthreadcore23:36:100
149940,0migration/01299-21kthreadcore21:26:100
29715030,0irq/26-eth0-rx-0-21swapper/320:46:043
29715030,0irq/26-eth0-rx-0-21swapper/320:41:053
29715030,0irq/26-eth0-rx-0-21swapper/319:49:213
29715030,0irq/26-eth0-rx-0-21swapper/319:41:043
29715030,0irq/26-eth0-rx-0-21swapper/319:21:033
244519933,0cyclictest3556-21lldpd19:39:142
244519932,0cyclictest9910-21df20:51:032
244519932,0cyclictest6624-21awk19:11:072
244519932,0cyclictest26888-21grep19:01:092
244519930,0cyclictest0-21swapper/219:45:552
244469933,0cyclictest28520-21awk20:16:091
244469932,0cyclictest6476-21awk19:11:071
244469932,0cyclictest3514-21snmpd00:11:391
244469932,0cyclictest20308-21df_inode19:46:051
244469932,0cyclictest15514-21tail20:51:141
244469931,0cyclictest32115-21kthreadcore20:41:111
244469930,0cyclictest27620-21cut19:01:101
244369933,0cyclictest7656-21kworker/0:120:50:550
244369933,0cyclictest7656-21kworker/0:119:41:030
244369933,0cyclictest5420-21kworker/0:000:26:020
244369933,0cyclictest5420-21kworker/0:000:15:580
244369933,0cyclictest32277-21awk19:06:080
244369933,0cyclictest15841-21kworker/0:223:56:060
244369933,0cyclictest15841-21kworker/0:223:25:580
244369933,0cyclictest15841-21kworker/0:223:10:540
244369933,0cyclictest15841-21kworker/0:222:15:540
244369932,0cyclictest30689-21munin-node22:46:060
244369932,0cyclictest29592-21irqcore20:41:090
244369930,0cyclictest8582-21munin-node20:26:040
244369930,0cyclictest6559-21runrttasks19:51:080
244369930,0cyclictest4292-21kthreadcore20:46:100
244369930,0cyclictest31196-21diskstats22:06:050
249930,0watchdog/20-21swapper/219:56:082
249930,0watchdog/20-21swapper/219:31:042
169930,0watchdog/122687-21diskmemload21:30:551
169930,0watchdog/120934-21munin-node20:11:041
169930,0watchdog/10-21swapper/123:11:031
169930,0watchdog/10-21swapper/120:26:161
169930,0watchdog/10-21swapper/119:56:081
169930,0watchdog/10-21swapper/119:51:081
29715020,0irq/26-eth0-rx-0-21swapper/320:16:013
29715020,0irq/26-eth0-rx-0-21swapper/319:36:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional