You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-07 - 10:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Mar 06, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1086123622,0sleep00-21swapper/007:00:110
304450350,0irq/25-eth10-21swapper/306:57:403
504623228,0sleep10-21swapper/106:57:241
1071823228,0sleep20-21swapper/206:58:332
111059975,0cyclictest0-21swapper/210:54:572
111059975,0cyclictest0-21swapper/209:07:352
111039974,0cyclictest24118-21basename10:32:090
35160,0ktimersoftd/30-21swapper/311:02:403
339960,0migration/39048-21sh09:21:453
339960,0migration/36590-21sh10:59:323
339960,0migration/330594-21sh10:53:303
339960,0migration/328690-21munin-node11:52:323
339960,0migration/324831-21sh09:31:043
339960,0migration/323564-21sh09:48:263
339960,0migration/315470-21sh10:25:163
29715060,0irq/26-eth0-rx-25400-21iostat_ios10:32:243
29715060,0irq/26-eth0-rx-20563-21perl07:57:233
29715060,0irq/26-eth0-rx-15979-21ssh09:44:363
29715060,0irq/26-eth0-rx-14718-21ssh10:44:013
29715060,0irq/26-eth0-rx-10556-21iostat_ios09:02:253
19160,0ktimersoftd/19295-21diskmemload11:02:401
149960,0migration/03971-21timerwakeupswit12:17:320
149960,0migration/017793-21latency_hist07:07:100
111059965,0cyclictest0-21swapper/209:13:342
111059964,0cyclictest0-21swapper/210:33:492
111049966,0cyclictest9295-21diskmemload11:00:241
339950,0migration/38173-21kthreadcore12:02:243
339950,0migration/37266-21fschecks_time07:47:223
339950,0migration/36790-21sh12:22:233
339950,0migration/34522-21kthreadcore09:57:273
339950,0migration/33660-21idleruntime08:57:243
339950,0migration/331051-21runrttasks10:12:553
339950,0migration/326267-21munin-node09:32:233
339950,0migration/321689-21timerandwakeup11:07:323
339950,0migration/321238-21taskset08:42:343
339950,0migration/320642-21taskset12:07:483
339950,0migration/319210-21munin-node10:47:263
339950,0migration/31227-21irqrtprio10:37:243
29725050,0irq/27-eth0-tx-2799-21munin-node08:57:260
29725050,0irq/27-eth0-tx-13890-21sshd08:15:370
29715050,0irq/26-eth0-rx-8851-21cut10:02:213
29715050,0irq/26-eth0-rx-5296-21grep10:17:283
29715050,0irq/26-eth0-rx-31898-21cut11:37:223
29715050,0irq/26-eth0-rx-3181-21cut07:17:263
29715050,0irq/26-eth0-rx-30030-21kthreadcore09:12:263
29715050,0irq/26-eth0-rx-29110-21grep11:12:273
29715050,0irq/26-eth0-rx-27804-21munin-node12:12:273
29715050,0irq/26-eth0-rx-18255-21awk07:27:333
29715050,0irq/26-eth0-rx-15119-21awk08:17:233
29715050,0irq/26-eth0-rx-0-21swapper/311:47:273
29715050,0irq/26-eth0-rx-0-21swapper/311:32:293
29715050,0irq/26-eth0-rx-0-21swapper/311:22:363
29715050,0irq/26-eth0-rx-0-21swapper/311:17:263
29715050,0irq/26-eth0-rx-0-21swapper/310:29:513
29715050,0irq/26-eth0-rx-0-21swapper/309:24:143
29715050,0irq/26-eth0-rx-0-21swapper/309:11:063
27150,0ktimersoftd/20-21swapper/211:48:522
27150,0ktimersoftd/20-21swapper/211:10:242
27150,0ktimersoftd/20-21swapper/207:02:522
19150,0ktimersoftd/10-21swapper/112:23:581
179950,0migration/17759-21turbostat10:42:091
179950,0migration/1766-21latency_hist08:07:101
179950,0migration/131596-21runrttasks10:35:371
179950,0migration/131250-21munin-node09:12:281
179950,0migration/122713-21taskset11:09:451
179950,0migration/120741-21latency_hist08:22:091
169950,0watchdog/127774-21kthreadcore11:52:271
149950,0migration/09609-21kthreadcore12:22:260
149950,0migration/07149-21unixbench-2d09:37:320
149950,0migration/06128-21munin-node10:17:310
149950,0migration/05656-21aten_r1power_fr07:22:220
149950,0migration/03853-21kthreadcore09:17:270
149950,0migration/031877-21aten2.4-expect11:17:200
149950,0migration/030081-21munin-node07:12:280
149950,0migration/028792-21cpuspeed_turbos08:52:230
149950,0migration/028089-21fschecks_count08:27:220
149950,0migration/026829-21kthreadcore10:32:250
149950,0migration/026354-21irqrtprio09:52:250
149950,0migration/026165-21kthreadcore11:32:250
149950,0migration/025896-21munin-node09:32:220
149950,0migration/022149-21kthreadcore09:07:290
149950,0migration/020970-21kthreadcore07:32:250
149950,0migration/018112-21kthreadcore08:42:260
149950,0migration/017367-21kthreadcore11:47:260
149950,0migration/016276-21munin-node11:07:230
149950,0migration/013887-21kthreadcore07:27:260
149950,0migration/012973-21kthreadcore09:02:270
149950,0migration/012717-21sh12:02:520
149950,0migration/011710-21kthreadcore09:22:270
149950,0migration/010225-21kthreadcore10:22:260
111059955,0cyclictest0-21swapper/212:26:422
111059955,0cyclictest0-21swapper/212:17:242
111059955,0cyclictest0-21swapper/210:31:082
111059950,0cyclictest0-21swapper/211:55:012
111059950,0cyclictest0-21swapper/210:58:252
111049955,0cyclictest0-21swapper/112:20:111
111049955,0cyclictest0-21swapper/112:06:371
111049955,0cyclictest0-21swapper/110:53:281
111049955,0cyclictest0-21swapper/109:32:451
111049955,0cyclictest0-21swapper/109:19:581
111049950,0cyclictest0-21swapper/110:24:181
111039954,0cyclictest16048-21ssh10:47:140
111039950,0cyclictest0-21swapper/011:55:200
111039950,0cyclictest0-21swapper/010:43:460
339940,0migration/37035-21kthreadcore07:22:273
339940,0migration/330624-21kthreadcore08:27:263
339940,0migration/329533-21kthreadcore07:37:273
339940,0migration/324980-21kthreadcore08:47:263
339940,0migration/324576-21kthreadcore08:22:283
339940,0migration/320590-21kthreadcore07:32:253
339940,0migration/319458-21kthreadcore10:07:253
339940,0migration/319109-21timerandwakeup07:52:333
339940,0migration/314645-21fschecks_time12:27:223
339940,0migration/31304-21kthreadcore07:42:263
339940,0migration/312088-21kthreadcore11:42:263
29725040,0irq/27-eth0-tx-0-21swapper/212:31:192
29725040,0irq/27-eth0-tx-0-21swapper/210:47:322
29725040,0irq/27-eth0-tx-0-21swapper/209:42:192
29725040,0irq/27-eth0-tx-0-21swapper/007:37:320
29715040,0irq/26-eth0-rx-24771-21aten2.4-expect07:12:213
29715040,0irq/26-eth0-rx-1485-21forks09:37:223
29715040,0irq/26-eth0-rx-0-21swapper/312:17:573
29715040,0irq/26-eth0-rx-0-21swapper/311:57:133
29715040,0irq/26-eth0-rx-0-21swapper/311:27:473
29715040,0irq/26-eth0-rx-0-21swapper/309:53:013
29715040,0irq/26-eth0-rx-0-21swapper/307:02:303
249940,0watchdog/26708-21kthreadcore09:17:282
249940,0watchdog/20-21swapper/207:22:282
179940,0migration/19240-21aten_r1power_fr09:42:201
179940,0migration/131006-21runrttasks07:16:391
179940,0migration/129782-21munin-node08:52:271
179940,0migration/127878-21kthreadcore07:37:271
179940,0migration/125791-21kthreadcore11:32:251
179940,0migration/125126-21kthreadcore08:47:261
179940,0migration/12492-21fschecks_time08:32:221
179940,0migration/120965-21kthreadcore09:27:261
179940,0migration/120382-21kthreadcore10:27:281
179940,0migration/117917-21kthreadcore11:27:241
179940,0migration/114098-21idleruntime07:52:251
179940,0migration/112821-21kthreadcore08:37:281
179940,0migration/111098-21kthreadcore11:42:261
169940,0watchdog/17972-21ssh10:00:401
169940,0watchdog/17135-21unixbench-2d09:37:321
169940,0watchdog/16298-21gltestperf07:22:241
169940,0watchdog/1521-21netstat08:27:281
169940,0watchdog/125817-21kthreadcore08:22:281
169940,0watchdog/125673-21if_err_eth1.210:12:231
169940,0watchdog/124317-21ssh09:09:121
169940,0watchdog/121558-21ssh12:10:311
169940,0watchdog/121463-21memory10:47:271
169940,0watchdog/121215-21users08:42:321
169940,0watchdog/116973-21munin-node07:02:321
169940,0watchdog/114967-21ssh10:02:361
169940,0watchdog/113490-21kthreadcore09:22:281
169940,0watchdog/113370-21timerwakeupswit11:22:311
149940,0migration/04178-21kthreadcore08:32:250
149940,0migration/028945-21kthreadcore09:12:260
149940,0migration/026616-21kthreadcore11:12:260
149940,0migration/022333-21fschecks_count08:47:220
149940,0migration/022163-21irqrtprio08:22:260
149940,0migration/016931-21sh10:07:190
149940,0migration/015634-21kthreadcore07:52:270
149940,0migration/012583-21kthreadcore11:22:250
111059944,0cyclictest0-21swapper/208:32:252
111059943,0cyclictest9364-21awk10:22:252
111059943,0cyclictest8995-21awk10:42:262
111059943,0cyclictest3556-21lldpd11:58:212
111059943,0cyclictest3556-21lldpd11:32:202
111059943,0cyclictest3556-21lldpd11:04:212
111059943,0cyclictest3556-21lldpd10:04:512
111059943,0cyclictest3556-21lldpd09:57:512
111059943,0cyclictest3556-21lldpd09:53:512
111059943,0cyclictest3556-21lldpd09:50:212
111059943,0cyclictest3556-21lldpd09:34:212
111059943,0cyclictest3556-21lldpd09:27:512
111059943,0cyclictest3556-21lldpd07:59:212
111059943,0cyclictest3556-21lldpd07:48:512
111059943,0cyclictest3556-21lldpd07:07:502
111059943,0cyclictest3514-21snmpd08:49:172
111059943,0cyclictest3514-21snmpd08:43:022
111059943,0cyclictest3514-21snmpd08:08:172
111059943,0cyclictest3514-21snmpd08:04:162
111059943,0cyclictest3514-21snmpd07:56:162
111059943,0cyclictest3514-21snmpd07:33:162
111059943,0cyclictest3514-21snmpd07:19:162
111059943,0cyclictest3241-21irqbalance12:12:272
111059943,0cyclictest3241-21irqbalance11:45:572
111059943,0cyclictest3241-21irqbalance11:27:572
111059943,0cyclictest3241-21irqbalance11:22:172
111059943,0cyclictest3241-21irqbalance10:40:382
111059943,0cyclictest3241-21irqbalance10:17:382
111059943,0cyclictest3241-21irqbalance09:37:182
111059943,0cyclictest3241-21irqbalance09:02:382
111059943,0cyclictest3241-21irqbalance08:59:382
111059943,0cyclictest3241-21irqbalance08:53:082
111059943,0cyclictest32289-21awk11:37:242
111059943,0cyclictest28504-21awk08:27:242
111059943,0cyclictest25794-21awk10:12:242
111059943,0cyclictest15367-21awk12:07:232
111059943,0cyclictest15197-21awk08:17:242
111059943,0cyclictest14183-21munin-run12:07:092
111059943,0cyclictest13143-21awk07:27:242
111059943,0cyclictest10037-21cut09:22:232
111049944,0cyclictest0-21swapper/112:15:081
111049944,0cyclictest0-21swapper/111:39:391
111049944,0cyclictest0-21swapper/111:19:191
111049944,0cyclictest0-21swapper/111:12:231
111049943,0cyclictest695-21awk07:42:241
111049943,0cyclictest3727-21awk08:57:251
111049943,0cyclictest3556-21lldpd10:09:211
111049943,0cyclictest3556-21lldpd09:47:211
111049943,0cyclictest32161-21awk07:17:241
111049943,0cyclictest27501-21awk08:02:231
111049943,0cyclictest25964-21awk09:52:231
111049943,0cyclictest15904-21awk11:47:241
111049943,0cyclictest1217-21awk10:17:251
111039944,0cyclictest28848-21kworker/0:010:52:260
111039944,0cyclictest0-21swapper/012:08:070
111039944,0cyclictest0-21swapper/011:30:350
111039943,0cyclictest9517-21awk08:37:260
111039943,0cyclictest926-21awk10:37:230
111039943,0cyclictest3556-21lldpd09:29:210
111039943,0cyclictest3556-21lldpd07:47:210
111039943,0cyclictest3556-21lldpd07:18:210
111039943,0cyclictest3241-21irqbalance10:57:170
111039943,0cyclictest3241-21irqbalance10:02:180
111039943,0cyclictest3241-21irqbalance09:50:180
111039943,0cyclictest32313-21awk11:37:240
111039943,0cyclictest28848-21kworker/0:011:05:020
111039943,0cyclictest28848-21kworker/0:009:43:230
111039943,0cyclictest18946-21tail07:07:240
111039943,0cyclictest1650-21awk09:57:250
29715030,0irq/26-eth0-rx-0-21swapper/308:55:353
29715030,0irq/26-eth0-rx-0-21swapper/308:12:213
29715030,0irq/26-eth0-rx-0-21swapper/308:07:213
29715030,0irq/26-eth0-rx-0-21swapper/308:02:213
29715030,0irq/26-eth0-rx-0-21swapper/307:07:273
249930,0watchdog/29295-21diskmemload10:07:192
249930,0watchdog/20-21swapper/208:22:282
249930,0watchdog/20-21swapper/207:12:322
169930,0watchdog/119988-21munin-node07:57:201
169930,0watchdog/10-21swapper/111:58:031
169930,0watchdog/10-21swapper/109:03:441
111059932,0cyclictest7376-21latency_hist08:12:102
111059932,0cyclictest5160-21latency07:42:272
111059932,0cyclictest26432-21awk07:37:242
111059932,0cyclictest0-21swapper/211:17:162
111059930,0cyclictest0-21swapper/211:12:142
111049933,0cyclictest7428-21awk07:47:231
111049932,0cyclictest8490-21awk08:12:241
111049932,0cyclictest19849-21awk07:32:241
111049932,0cyclictest19713-21sensors12:27:301
111049932,0cyclictest13854-21sensors10:42:331
111049932,0cyclictest12545-21tr07:27:221
111039933,0cyclictest28848-21kworker/0:012:27:140
111039933,0cyclictest28848-21kworker/0:012:12:100
111039933,0cyclictest28848-21kworker/0:011:57:140
111039933,0cyclictest28848-21kworker/0:011:42:120
111039933,0cyclictest28848-21kworker/0:010:12:210
111039933,0cyclictest28848-21kworker/0:008:19:550
111039933,0cyclictest28848-21kworker/0:008:07:110
111039933,0cyclictest28848-21kworker/0:008:02:230
111039933,0cyclictest28848-21kworker/0:007:42:210
111039932,0cyclictest25663-21sensors07:57:300
329920,0watchdog/33624-21kthreadcore08:32:253
29715020,0irq/26-eth0-rx-0-21swapper/308:37:293
249920,0watchdog/20-21swapper/208:37:122
169920,0watchdog/10-21swapper/107:07:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional