You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 14:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun May 10, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1186223829,0sleep20-21swapper/206:52:422
304450360,0irq/25-eth10-21swapper/306:54:433
1205423622,0sleep00-21swapper/006:54:560
1210823324,0sleep10-21swapper/106:55:331
29715070,0irq/26-eth0-rx-28304-21irqstats09:06:193
35160,0ktimersoftd/37272-21grep08:26:193
35160,0ktimersoftd/30-21swapper/312:06:253
339960,0migration/39065-21sh10:54:243
339960,0migration/39065-21sh10:54:243
339960,0migration/38992-21sh11:55:463
339960,0migration/3817-21sh11:30:203
339960,0migration/34296-21munin-node08:51:153
339960,0migration/324366-21sh11:22:483
339960,0migration/318499-21sh09:19:523
149960,0migration/016731-21sh11:19:010
122229965,0cyclictest1985-21tr07:36:212
122189966,0cyclictest0-21swapper/111:36:181
8215050,0irq/24-0000:00:32153-21latency_hist07:11:022
339950,0migration/39767-21taskset09:13:213
339950,0migration/37912-21needreboot10:11:213
339950,0migration/35722-21kthreadcore07:36:233
339950,0migration/331456-21timerandwakeup11:46:223
339950,0migration/331369-21kthreadcore07:31:203
339950,0migration/330802-21kthreadcore07:06:193
339950,0migration/327833-21idleruntime09:46:163
339950,0migration/326306-21kthreadcore08:16:193
339950,0migration/323412-21munin-node11:41:243
339950,0migration/322251-21kthreadcore07:01:193
339950,0migration/32056-21ksmtuned10:31:103
339950,0migration/32032-21timerandwakeup08:21:263
339950,0migration/320103-21irqcore09:41:193
339950,0migration/319430-21cpuspeed_turbos09:21:143
339950,0migration/318068-21if_eth112:21:153
339950,0migration/317678-21cpuspeed_turbos12:01:133
339950,0migration/316123-21latency09:56:203
339950,0migration/312874-21kthreadcore10:16:193
339950,0migration/312505-21kthreadcore11:36:183
339950,0migration/310754-21idleruntime10:56:153
29725050,0irq/27-eth0-tx-945-21munin-run11:51:012
29725050,0irq/27-eth0-tx-25762-21munin-node07:06:140
29725050,0irq/27-eth0-tx-1952-21munin-node08:01:160
29725050,0irq/27-eth0-tx-19087-21munin-node07:01:240
29725050,0irq/27-eth0-tx-0-21swapper/212:03:522
29725050,0irq/27-eth0-tx-0-21swapper/210:24:132
29725050,0irq/27-eth0-tx-0-21swapper/209:50:202
29725050,0irq/27-eth0-tx-0-21swapper/209:11:572
29725050,0irq/27-eth0-tx-0-21swapper/008:24:560
29725050,0irq/27-eth0-tx-0-21swapper/007:52:550
29725050,0irq/27-eth0-tx-0-21swapper/007:47:250
29725050,0irq/27-eth0-tx-0-21swapper/007:44:250
29725050,0irq/27-eth0-tx-0-21swapper/007:26:140
29725050,0irq/27-eth0-tx-0-21swapper/007:11:130
29715050,0irq/26-eth0-rx-9454-21grep07:16:193
29715050,0irq/26-eth0-rx-8981-21sshd11:35:273
29715050,0irq/26-eth0-rx-27920-21irqrtprio10:06:173
29715050,0irq/26-eth0-rx-27649-21cat11:06:173
29715050,0irq/26-eth0-rx-25312-21ssh09:02:253
29715050,0irq/26-eth0-rx-24358-21ssh10:41:363
29715050,0irq/26-eth0-rx-167292sleep310:37:423
29715050,0irq/26-eth0-rx-16148-21smartctl08:11:163
29715050,0irq/26-eth0-rx-11802-21kthreadcore08:06:203
29715050,0irq/26-eth0-rx-0-21swapper/312:13:053
29715050,0irq/26-eth0-rx-0-21swapper/311:56:153
29715050,0irq/26-eth0-rx-0-21swapper/311:13:353
29715050,0irq/26-eth0-rx-0-21swapper/311:01:123
29715050,0irq/26-eth0-rx-0-21swapper/310:46:183
29715050,0irq/26-eth0-rx-0-21swapper/310:29:023
29715050,0irq/26-eth0-rx-0-21swapper/310:02:273
29715050,0irq/26-eth0-rx-0-21swapper/309:51:193
29715050,0irq/26-eth0-rx-0-21swapper/309:36:293
29715050,0irq/26-eth0-rx-0-21swapper/309:34:483
29715050,0irq/26-eth0-rx-0-21swapper/308:56:233
259950,0migration/225769-21sh09:03:342
259950,0migration/211212-21munin-node10:56:172
259950,0migration/210229-21gltestperf11:36:152
249950,0watchdog/225309-21kthreadcore08:41:202
249950,0watchdog/214750-21grep11:16:202
179950,0migration/19766-21aten_r1power_po08:31:131
179950,0migration/19732-21switchtime08:51:251
179950,0migration/19557-21taskset09:32:511
179950,0migration/127905-21kthreadcore11:06:171
179950,0migration/126773-21latency_hist09:46:021
179950,0migration/123826-21missed_timers10:21:211
179950,0migration/118355-21kthreadcore08:11:201
179950,0migration/116580-21needreboot09:16:201
179950,0migration/111798-21kthreadcore10:16:181
179950,0migration/110549-21idleruntime11:16:161
169950,0watchdog/131205-21python12:06:241
149950,0migration/09666-21taskset10:14:380
149950,0migration/07164-21taskset07:37:160
149950,0migration/05975-21kthreadcore09:11:190
149950,0migration/04755-21kthreadcore11:11:190
149950,0migration/031099-21processes12:06:220
149950,0migration/029849-21kthreadcore11:46:160
149950,0migration/027319-21kthreadcore11:06:160
149950,0migration/026890-21kthreadcore08:16:190
149950,0migration/026580-21latency_hist10:06:020
149950,0migration/020276-21kthreadcore08:36:180
149950,0migration/019038-21aten_r1power_cu09:41:130
149950,0migration/018934-21latency_hist09:21:020
149950,0migration/018843-21ntp_kernel_err07:21:220
149950,0migration/017763-21fschecks_time12:21:140
149950,0migration/012101-21kthreadcore12:16:190
122229955,0cyclictest0-21swapper/211:05:512
122229955,0cyclictest0-21swapper/210:54:492
122229955,0cyclictest0-21swapper/210:54:492
122229954,0cyclictest8148-21cut07:16:182
122229950,0cyclictest0-21swapper/211:33:552
122229950,0cyclictest0-21swapper/209:22:542
122229950,0cyclictest0-21swapper/209:06:302
122229950,0cyclictest0-21swapper/208:19:122
122189950,0cyclictest0-21swapper/110:05:201
122179950,0cyclictest0-21swapper/011:21:110
8215040,0irq/24-0000:00:0-21swapper/207:31:022
339940,0migration/324274-21kernelversion08:41:203
339940,0migration/317463-21latency_hist11:21:023
339940,0migration/316947-21kthreadcore07:21:213
339940,0migration/313599-21timerwakeupswit07:41:243
339940,0migration/312026-21kthreadcore08:31:183
29725040,0irq/27-eth0-tx-21531-21kthreadcore10:01:182
29725040,0irq/27-eth0-tx-0-21swapper/212:11:172
29725040,0irq/27-eth0-tx-0-21swapper/212:06:252
29725040,0irq/27-eth0-tx-0-21swapper/211:27:592
29725040,0irq/27-eth0-tx-0-21swapper/210:46:502
29725040,0irq/27-eth0-tx-0-21swapper/210:42:092
29725040,0irq/27-eth0-tx-0-21swapper/209:51:102
29725040,0irq/27-eth0-tx-0-21swapper/209:41:202
29725040,0irq/27-eth0-tx-0-21swapper/209:16:092
29725040,0irq/27-eth0-tx-0-21swapper/208:57:442
29725040,0irq/27-eth0-tx-0-21swapper/008:56:080
29725040,0irq/27-eth0-tx-0-21swapper/007:18:040
29715040,0irq/26-eth0-rx-0-21swapper/312:16:133
29715040,0irq/26-eth0-rx-0-21swapper/310:21:183
29715040,0irq/26-eth0-rx-0-21swapper/309:29:503
29715040,0irq/26-eth0-rx-0-21swapper/308:47:013
29715040,0irq/26-eth0-rx-0-21swapper/307:47:073
29715040,0irq/26-eth0-rx-0-21swapper/306:56:133
259940,0migration/25826-21kthreadcore11:51:182
259940,0migration/231377-21kthreadcore08:21:202
259940,0migration/230653-21kthreadcore09:26:192
259940,0migration/226512-21aten_r1power_en10:26:132
259940,0migration/222654-21runrttasks08:40:102
259940,0migration/217607-21fschecks_count12:21:132
259940,0migration/214407-21kthreadcore10:16:192
259940,0migration/213487-21aten_r1power_en07:21:152
259940,0migration/213058-21kthreadcore09:36:172
259940,0migration/212807-21kthreadcore09:56:182
249940,0watchdog/22314-21ssh10:11:122
249940,0watchdog/20-21swapper/212:16:192
249940,0watchdog/20-21swapper/210:36:122
179940,0migration/17233-21taskset11:11:211
179940,0migration/15315-21kthreadcore11:51:181
179940,0migration/15178-21kthreadcore08:26:191
179940,0migration/13812-21kthreadcore10:11:191
179940,0migration/132614-21runrttasks09:06:231
179940,0migration/131546-21kthreadcore10:06:191
179940,0migration/130310-21kthreadcore07:06:191
179940,0migration/130132-21kthreadcore08:21:201
179940,0migration/129604-21kthreadcore09:46:191
179940,0migration/129333-21kthreadcore10:46:181
179940,0migration/128693-21irqcore07:56:191
179940,0migration/128057-21idleruntime09:26:161
179940,0migration/126916-21irqcore11:26:181
179940,0migration/126339-21latency_hist10:26:021
179940,0migration/126230-21fschecks_time11:46:121
179940,0migration/12610-21kthreadcore07:36:221
179940,0migration/124255-21runrttasks12:03:291
179940,0migration/123612-21kthreadcore08:16:181
179940,0migration/121117-21irqcore07:26:171
179940,0migration/120769-21kthreadcore11:01:181
179940,0migration/119988-21kthreadcore11:41:181
179940,0migration/119792-21kthreadcore08:36:181
179940,0migration/115707-21kthreadcore07:46:181
179940,0migration/112645-21kthreadcore07:41:191
169940,0watchdog/16559-21runrttasks12:12:591
169940,0watchdog/116282-21grep08:56:201
169940,0watchdog/111243-21kthreadcore11:56:161
169940,0watchdog/110301-21aten_r1power_vo10:36:121
169940,0watchdog/10-21swapper/110:57:161
169940,0watchdog/10-21swapper/108:41:201
169940,0watchdog/10-21swapper/107:21:211
169940,0watchdog/10-21swapper/107:16:131
149940,0migration/06427-21kthreadcore08:26:190
149940,0migration/04512-21kthreadcore10:51:190
149940,0migration/04512-21kthreadcore10:51:190
149940,0migration/026846-21kthreadcore08:41:210
149940,0migration/02616-21irqrtprio11:31:180
149940,0migration/024163-21sendmail_mailqu10:21:240
149940,0migration/017986-21kthreadcore08:11:200
149940,0migration/01551-21latency_hist10:51:020
149940,0migration/01492-21runrttasks07:58:300
122229943,0cyclictest9422-21turbostat08:31:022
122229943,0cyclictest4900-21awk08:51:182
122229943,0cyclictest3556-21lldpd11:21:562
122229943,0cyclictest3556-21lldpd09:31:562
122229943,0cyclictest3556-21lldpd07:56:252
122229943,0cyclictest3514-21snmpd10:08:172
122229943,0cyclictest3514-21snmpd07:55:032
122229943,0cyclictest30629-21awk08:46:192
122229943,0cyclictest2956-21awk10:31:162
122229943,0cyclictest26960-21awk11:06:152
122229943,0cyclictest2525-21awk11:11:172
122229943,0cyclictest21802-21sadc08:16:022
122229943,0cyclictest19970-21awk07:01:172
122229943,0cyclictest18901-21tail11:41:182
122229943,0cyclictest15298-21awk07:46:172
122229943,0cyclictest13340-21awk06:56:172
122229943,0cyclictest10674-21tail08:31:182
122229943,0cyclictest10259-21awk11:56:142
122189944,0cyclictest0-21swapper/109:51:181
122189944,0cyclictest0-21swapper/109:11:151
122189943,0cyclictest3514-21snmpd08:10:031
122189943,0cyclictest3514-21snmpd08:02:181
122189943,0cyclictest3514-21snmpd07:11:481
122189943,0cyclictest3241-21irqbalance09:56:101
122189943,0cyclictest3241-21irqbalance09:38:401
122189943,0cyclictest2687-21awk10:51:161
122189943,0cyclictest2687-21awk10:51:161
122189943,0cyclictest20393-21kthreadcore07:01:181
122189943,0cyclictest19776-21awk09:01:161
122189943,0cyclictest18597-21awk11:21:161
122189943,0cyclictest17992-21awk12:21:151
122189943,0cyclictest10134-21awk12:16:171
122179944,0cyclictest0-21swapper/009:56:030
122179943,0cyclictest8706-21munin-node08:06:190
122179943,0cyclictest3581-21awk09:51:170
122179943,0cyclictest3514-21snmpd12:15:020
122179943,0cyclictest3514-21snmpd07:31:030
122179943,0cyclictest3241-21irqbalance11:42:100
122179943,0cyclictest3241-21irqbalance11:37:200
122179943,0cyclictest3241-21irqbalance11:01:100
122179943,0cyclictest3241-21irqbalance10:57:000
122179943,0cyclictest3241-21irqbalance09:47:300
122179943,0cyclictest3241-21irqbalance09:36:100
122179943,0cyclictest3241-21irqbalance09:31:500
122179943,0cyclictest3241-21irqbalance09:29:200
122179943,0cyclictest26734-21sadc09:06:020
122179943,0cyclictest2101-21awk11:51:150
122179942,0cyclictest3481-21kworker/0:011:57:010
8215030,0irq/24-0000:00:0-21swapper/208:06:022
29725030,0irq/27-eth0-tx-0-21swapper/006:56:130
29715030,0irq/26-eth0-rx-0-21swapper/308:36:173
29715030,0irq/26-eth0-rx-0-21swapper/308:01:153
29715030,0irq/26-eth0-rx-0-21swapper/307:56:143
29715030,0irq/26-eth0-rx-0-21swapper/307:53:483
29715030,0irq/26-eth0-rx-0-21swapper/307:26:133
29715030,0irq/26-eth0-rx-0-21swapper/307:11:133
249930,0watchdog/226645-21munin-node07:06:172
169930,0watchdog/17601-21ssh11:31:281
169930,0watchdog/124451-21ssh10:41:521
169930,0watchdog/10-21swapper/108:46:161
122229932,0cyclictest3556-21lldpd07:41:252
122229932,0cyclictest3067-21irqcore08:01:192
122229932,0cyclictest20918-21awk07:26:162
122189932,0cyclictest3110-21grep10:31:171
122189932,0cyclictest21964-21awk07:51:171
122189932,0cyclictest20168-21awk09:21:181
122189932,0cyclictest15517-21kthreadcore06:56:191
122179933,0cyclictest3481-21kworker/0:012:01:070
122179933,0cyclictest3481-21kworker/0:011:26:110
122179933,0cyclictest3481-21kworker/0:010:41:070
122179933,0cyclictest3481-21kworker/0:010:36:070
122179933,0cyclictest3481-21kworker/0:010:31:150
122179933,0cyclictest3481-21kworker/0:010:26:130
122179933,0cyclictest3481-21kworker/0:010:16:170
122179933,0cyclictest3481-21kworker/0:009:21:150
122179933,0cyclictest3481-21kworker/0:009:06:150
122179933,0cyclictest3481-21kworker/0:008:46:150
122179932,0cyclictest3241-21irqbalance08:51:200
122179932,0cyclictest3241-21irqbalance08:31:210
122179932,0cyclictest27703-21awk10:06:160
169920,0watchdog/10-21swapper/107:31:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional