You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 01:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun May 10, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450750,0irq/25-eth10-21swapper/318:51:163
3177323422,0sleep00-21swapper/018:55:410
3175523424,0sleep10-21swapper/118:55:301
3168523127,0sleep20-21swapper/218:54:392
3188099130,0cyclictest0-21swapper/020:56:230
339960,0migration/34400-21sh22:38:163
339960,0migration/331740-21munin-node21:56:213
339960,0migration/331643-21latency19:16:223
339960,0migration/329953-21sh21:55:103
339960,0migration/329635-21sh21:33:393
339960,0migration/329199-21sh22:13:423
339960,0migration/313485-21latency_hist22:46:043
339960,0migration/310853-21sh00:02:393
318829966,0cyclictest30105-21diskmemload00:23:072
318829966,0cyclictest0-21swapper/221:01:282
318829965,0cyclictest19899-21pmu-power22:06:252
318829960,0cyclictest0-21swapper/221:14:332
318809966,0cyclictest30105-21diskmemload22:51:240
29715060,0irq/26-eth0-rx-0-21swapper/323:04:483
149960,0migration/06112-21latency_hist21:01:040
149960,0migration/03555-21sh23:38:570
149960,0migration/029615-21cpuspeed_turbos23:16:170
149960,0migration/029536-21aten2.4-expect20:31:170
149960,0migration/021901-21taskset21:29:220
149960,0migration/013888-21kthreadcore00:06:220
149960,0migration/011431-21sh23:43:260
149960,0migration/011429-21sh00:04:200
8215050,0irq/24-0000:00:0-21swapper/219:51:062
339950,0migration/39227-21latency_hist20:16:053
339950,0migration/35927-21kthreadcore20:36:223
339950,0migration/35498-21fschecks_count23:21:173
339950,0migration/34754-21latency_hist23:21:043
339950,0migration/330825-21aten_r1power_fr21:36:143
339950,0migration/328878-21cpuspeed_turbos23:56:153
339950,0migration/327197-21latency_hist19:41:063
339950,0migration/323306-21kthreadcore20:01:223
339950,0migration/323014-21cpuspeed_turbos21:11:153
339950,0migration/32179-21latency20:31:233
339950,0migration/320123-21sh23:50:213
339950,0migration/318528-21munin-node23:26:223
339950,0migration/314637-21taskset19:53:523
339950,0migration/314123-21sh21:45:423
339950,0migration/313893-21cpuspeed_turbos22:46:163
339950,0migration/312163-21munin-node22:21:273
318829955,0cyclictest30105-21diskmemload22:18:152
318829955,0cyclictest0-21swapper/223:59:182
318829955,0cyclictest0-21swapper/223:34:052
318829955,0cyclictest0-21swapper/222:38:542
318829955,0cyclictest0-21swapper/221:29:512
318829955,0cyclictest0-21swapper/200:04:302
318829950,0cyclictest0-21swapper/223:54:062
318829950,0cyclictest0-21swapper/222:59:142
318829950,0cyclictest0-21swapper/222:49:372
318819955,0cyclictest0-21swapper/120:51:171
318809950,0cyclictest0-21swapper/023:02:360
318809950,0cyclictest0-21swapper/021:38:530
318809950,0cyclictest0-21swapper/021:16:200
29725050,0irq/27-eth0-tx-8441-21munin-node19:51:160
29725050,0irq/27-eth0-tx-29379-21ssh21:32:552
29725050,0irq/27-eth0-tx-28790-21ssh22:12:322
29725050,0irq/27-eth0-tx-21670-21munin-node22:51:222
29725050,0irq/27-eth0-tx-20703-21munin-node19:36:280
29725050,0irq/27-eth0-tx-14524-21ssh21:46:112
29725050,0irq/27-eth0-tx-13637-21kthreadcore00:06:222
29725050,0irq/27-eth0-tx-0-21swapper/221:18:022
29725050,0irq/27-eth0-tx-0-21swapper/200:13:012
29715050,0irq/26-eth0-rx-3542-21lldpd23:11:223
29715050,0irq/26-eth0-rx-32236-21kthreadcore23:36:203
29715050,0irq/26-eth0-rx-30359-21aten_r1power_en22:16:153
29715050,0irq/26-eth0-rx-2822-21ntp_kernel_pll_22:56:233
29715050,0irq/26-eth0-rx-17873-21kthreadcore19:31:223
29715050,0irq/26-eth0-rx-17873-21kthreadcore19:31:223
29715050,0irq/26-eth0-rx-13028-21kthreadcore20:16:223
29715050,0irq/26-eth0-rx-10958-21rm00:24:043
29715050,0irq/26-eth0-rx-0-21swapper/323:41:213
29715050,0irq/26-eth0-rx-0-21swapper/323:32:283
29715050,0irq/26-eth0-rx-0-21swapper/322:10:083
29715050,0irq/26-eth0-rx-0-21swapper/322:01:213
29715050,0irq/26-eth0-rx-0-21swapper/321:28:263
29715050,0irq/26-eth0-rx-0-21swapper/321:01:183
29715050,0irq/26-eth0-rx-0-21swapper/319:06:163
29715050,0irq/26-eth0-rx-0-21swapper/300:16:553
29715050,0irq/26-eth0-rx-0-21swapper/300:12:203
259950,0migration/2944-21kthreadcore21:36:202
259950,0migration/219310-21sh23:27:032
179950,0migration/13575-21latency_hist20:36:051
179950,0migration/132724-21taskset19:20:171
179950,0migration/131072-21munin-node21:16:181
179950,0migration/129071-21aten_r1power_vo23:36:151
179950,0migration/125234-21kthreadcore20:26:221
179950,0migration/124475-21kthreadcore22:11:201
179950,0migration/120309-21munin-node21:26:261
179950,0migration/113168-21sh21:22:261
179950,0migration/112658-21taskset23:25:571
149950,0migration/08858-21kthreadcore21:41:210
149950,0migration/07142-21munin-node21:21:170
149950,0migration/06892-21kthreadcore20:36:220
149950,0migration/0531-21processes19:41:260
149950,0migration/05148-21cpu23:21:160
149950,0migration/03436-21kthreadcore18:56:280
149950,0migration/030565-21munin-node22:16:160
149950,0migration/029268-21munin-node20:06:190
149950,0migration/027980-21kthreadcore19:16:210
149950,0migration/025258-21kthreadcore23:51:210
149950,0migration/021459-21sh22:09:270
149950,0migration/021228-21cpuspeed_turbos23:31:160
149950,0migration/019850-21aten2.4-expect19:11:170
149950,0migration/018171-21kthreadcore21:06:230
149950,0migration/015147-21kthreadcore22:46:210
149950,0migration/014631-21kthreadcore23:26:190
149950,0migration/01172-21kthreadcore21:56:220
149950,0migration/010382-21aten_r1power_cu20:41:170
149950,0migration/010057-21kthreadcore19:26:220
4140,0ktimersoftd/00-21swapper/023:14:540
339940,0migration/39847-21kthreadcore19:01:233
339940,0migration/3720-21timerandwakeup19:41:283
339940,0migration/321155-21kthreadcore19:11:223
339940,0migration/319571-21kthreadcore19:56:213
339940,0migration/319247-21kthreadcore21:46:233
339940,0migration/318527-21kthreadcore20:21:213
339940,0migration/314065-21gltestperf23:06:193
318829943,0cyclictest7272-21awk19:01:212
318829943,0cyclictest6921-21awk22:21:192
318829943,0cyclictest3556-21lldpd20:39:222
318829943,0cyclictest3556-21lldpd19:53:522
318829943,0cyclictest3556-21lldpd19:27:212
318829943,0cyclictest3556-21lldpd19:06:212
318829943,0cyclictest29237-21awk20:06:192
318829943,0cyclictest28288-21awk19:41:202
318829943,0cyclictest22190-21awk23:11:202
318829943,0cyclictest15357-21awk21:06:202
318819943,0cyclictest7258-21awk21:01:211
318819943,0cyclictest6548-21awk22:41:201
318819943,0cyclictest3514-21snmpd23:51:301
318819943,0cyclictest3514-21snmpd23:46:451
318819943,0cyclictest3514-21snmpd23:41:451
318819943,0cyclictest3514-21snmpd23:32:301
318819943,0cyclictest3514-21snmpd23:26:301
318819943,0cyclictest3514-21snmpd21:51:451
318819943,0cyclictest3514-21snmpd21:11:291
318819943,0cyclictest3514-21snmpd20:33:291
318819943,0cyclictest3514-21snmpd20:08:291
318819943,0cyclictest3514-21snmpd20:05:291
318819943,0cyclictest3514-21snmpd19:57:291
318819943,0cyclictest3514-21snmpd19:39:291
318819943,0cyclictest3514-21snmpd19:22:141
318819943,0cyclictest3514-21snmpd19:09:291
318819943,0cyclictest3514-21snmpd00:01:301
318819943,0cyclictest3241-21irqbalance23:02:031
318819943,0cyclictest3241-21irqbalance00:12:231
318819943,0cyclictest30560-21awk22:56:181
318819943,0cyclictest15155-21awk22:06:191
318819943,0cyclictest14918-21awk19:31:201
318819943,0cyclictest14918-21awk19:31:201
318819943,0cyclictest0-21swapper/121:42:401
318819942,0cyclictest8421-21sensors20:11:261
318819942,0cyclictest32165-21grep21:56:221
318809944,0cyclictest5375-21kworker/0:100:19:530
318809944,0cyclictest0-21swapper/000:23:000
318809943,0cyclictest6645-21awk22:41:210
318809943,0cyclictest3556-21lldpd21:15:220
318809943,0cyclictest3514-21snmpd22:26:150
318809943,0cyclictest29510-21awk23:56:170
318809943,0cyclictest22654-21awk20:01:200
318809943,0cyclictest14285-21awk23:06:210
318809943,0cyclictest13765-21sadc22:26:040
318809940,0cyclictest0-21swapper/022:02:130
318809940,0cyclictest0-21swapper/021:53:330
29725040,0irq/27-eth0-tx-5646-21munin-node22:41:212
29725040,0irq/27-eth0-tx-22765-21telnet20:26:160
29725040,0irq/27-eth0-tx-22567-21munin-node21:51:182
29725040,0irq/27-eth0-tx-17167-21tail20:46:170
29725040,0irq/27-eth0-tx-12974-21ssh21:21:512
29725040,0irq/27-eth0-tx-0-21swapper/223:36:512
29725040,0irq/27-eth0-tx-0-21swapper/223:06:292
29725040,0irq/27-eth0-tx-0-21swapper/222:26:092
29725040,0irq/27-eth0-tx-0-21swapper/220:57:342
29725040,0irq/27-eth0-tx-0-21swapper/200:16:172
29725040,0irq/27-eth0-tx-0-21swapper/020:21:220
29725040,0irq/27-eth0-tx-0-21swapper/020:11:190
29725040,0irq/27-eth0-tx-0-21swapper/019:06:290
29725040,0irq/27-eth0-tx-0-21swapper/019:05:480
29715040,0irq/26-eth0-rx-0-21swapper/323:51:183
29715040,0irq/26-eth0-rx-0-21swapper/322:51:113
29715040,0irq/26-eth0-rx-0-21swapper/322:31:403
29715040,0irq/26-eth0-rx-0-21swapper/322:26:093
29715040,0irq/26-eth0-rx-0-21swapper/321:21:553
29715040,0irq/26-eth0-rx-0-21swapper/321:17:483
29715040,0irq/26-eth0-rx-0-21swapper/321:06:233
29715040,0irq/26-eth0-rx-0-21swapper/320:56:183
29715040,0irq/26-eth0-rx-0-21swapper/320:51:223
29715040,0irq/26-eth0-rx-0-21swapper/320:06:153
29715040,0irq/26-eth0-rx-0-21swapper/300:06:163
259940,0migration/28557-21timerandwakeup20:11:282
259940,0migration/27209-21kthreadcore23:21:212
259940,0migration/26306-21kthreadcore23:41:202
259940,0migration/225798-21kthreadcore20:51:222
259940,0migration/224840-21kthreadcore19:36:222
259940,0migration/223125-21kthreadcore20:01:222
259940,0migration/221579-21kthreadcore19:11:222
259940,0migration/22011-21kthreadcore20:31:232
259940,0migration/219874-21timerwakeupswit19:31:282
259940,0migration/219874-21timerwakeupswit19:31:282
259940,0migration/219713-21kthreadcore19:56:212
259940,0migration/219177-21kthreadcore20:21:212
259940,0migration/218471-21kthreadcore20:46:222
259940,0migration/210863-21munin-node23:01:232
249940,0watchdog/27374-21if_eth021:41:192
249940,0watchdog/230068-21gltestperf23:16:192
249940,0watchdog/22498-21kthreadcore18:56:282
249940,0watchdog/22448-21grep21:56:232
249940,0watchdog/222508-21fschecks_count22:31:192
249940,0watchdog/214196-21kthreadcore20:41:232
249940,0watchdog/20-21swapper/223:46:222
249940,0watchdog/20-21swapper/222:01:272
179940,0migration/19025-21kthreadcore22:01:221
179940,0migration/1483-21kthreadcore20:56:221
179940,0migration/131056-21kthreadcore00:16:211
179940,0migration/128048-21runrttasks22:52:101
179940,0migration/126183-21kthreadcore22:31:231
179940,0migration/125417-21kthreadcore23:11:221
179940,0migration/121775-21latency_hist22:31:041
179940,0migration/116843-21sh20:21:181
179940,0migration/116103-21timerandwakeup20:41:281
179940,0migration/112852-21kthreadcore19:51:221
179940,0migration/111163-21kthreadcore20:16:211
179940,0migration/110710-21kthreadcore19:26:221
169940,0watchdog/16559-21runrttasks00:00:221
169940,0watchdog/126956-21tr21:31:231
169940,0watchdog/1195602chrt21:46:231
169940,0watchdog/115084-21kthreadcore23:06:231
169940,0watchdog/110999-21ssh00:24:101
169940,0watchdog/10-21swapper/123:16:191
169940,0watchdog/10-21swapper/122:48:071
169940,0watchdog/10-21swapper/122:39:191
169940,0watchdog/10-21swapper/121:36:151
149940,0migration/026901-21kthreadcore21:31:230
149940,0migration/016269-21kthreadcore23:46:220
149940,0migration/015706-21kthreadcore19:31:220
149940,0migration/015706-21kthreadcore19:31:220
149940,0migration/015151-21fschecks_time21:46:190
149940,0migration/014230-21kthreadcore20:16:220
339930,0migration/323392-21taskset20:50:163
318829932,0cyclictest3556-21lldpd20:27:522
318829932,0cyclictest27089-21grep19:16:182
318829932,0cyclictest14562-21grep20:16:222
318819932,0cyclictest7435-21sensors19:46:261
318819932,0cyclictest7326-21awk19:01:211
318819932,0cyclictest3763-21sensors22:16:261
318819932,0cyclictest3514-21snmpd19:41:291
318819932,0cyclictest25353-21sensors19:11:261
318819932,0cyclictest20632-21grep20:46:221
318809933,0cyclictest5375-21kworker/0:100:11:060
318809933,0cyclictest3783-21kworker/0:022:56:160
318809933,0cyclictest3783-21kworker/0:022:36:220
318809933,0cyclictest3783-21kworker/0:022:31:120
318809933,0cyclictest31067-21kworker/0:020:51:060
318809933,0cyclictest31067-21kworker/0:019:56:130
318809933,0cyclictest31067-21kworker/0:019:46:090
318809933,0cyclictest31067-21kworker/0:019:21:130
318809932,0cyclictest22241-21latency_hist22:11:040
29715030,0irq/26-eth0-rx-0-21swapper/320:26:163
29715030,0irq/26-eth0-rx-0-21swapper/319:27:473
29715030,0irq/26-eth0-rx-0-21swapper/319:22:303
29715030,0irq/26-eth0-rx-0-21swapper/318:58:553
249930,0watchdog/20-21swapper/219:21:272
179930,0migration/112456-21sh22:22:161
169930,0watchdog/116124-21kthreadcore00:06:221
169930,0watchdog/10-21swapper/121:06:231
169930,0watchdog/10-21swapper/118:56:281
29715020,0irq/26-eth0-rx-0-21swapper/320:41:143
29715020,0irq/26-eth0-rx-0-21swapper/319:46:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional