You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 14:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 29, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
522224637,0sleep20-21swapper/206:59:462
304450360,0irq/25-eth10-21swapper/306:56:113
501023425,0sleep10-21swapper/106:57:171
492323422,0sleep00-21swapper/006:56:160
53189966,0cyclictest3428-21diskmemload10:07:252
53189964,0cyclictest0-21swapper/209:20:172
53189960,0cyclictest0-21swapper/209:18:142
53179965,0cyclictest0-21swapper/110:30:151
53179960,0cyclictest0-21swapper/111:12:201
53179960,0cyclictest0-21swapper/109:30:581
53169960,0cyclictest0-21swapper/010:05:110
35160,0ktimersoftd/30-21swapper/309:53:103
339960,0migration/38754-21sh11:02:193
339960,0migration/38072-21sh11:42:213
339960,0migration/3678-21latency_hist07:45:013
339960,0migration/3678-21latency_hist07:45:013
339960,0migration/332307-21sh11:15:583
339960,0migration/324888-21sh10:30:353
339960,0migration/324119-21sh11:10:273
339960,0migration/318178-21sh10:08:023
29715060,0irq/26-eth0-rx-7356-21kthreadcore09:40:173
29715060,0irq/26-eth0-rx-29616-21kthreadcore10:15:163
29715060,0irq/26-eth0-rx-27472-21tr12:15:173
29715060,0irq/26-eth0-rx-0-21swapper/309:31:093
149960,0migration/023915-21sh12:12:570
149960,0migration/02387-21sh10:19:590
8215050,0irq/24-0000:00:512-21munin-run11:59:592
8215050,0irq/24-0000:00:2401-21latency_hist10:19:592
53189955,0cyclictest3428-21diskmemload09:27:412
53189955,0cyclictest0-21swapper/212:06:462
53189950,0cyclictest0-21swapper/212:18:172
53179955,0cyclictest0-21swapper/109:10:001
53179955,0cyclictest0-21swapper/108:35:191
53179950,0cyclictest0-21swapper/112:26:361
53179950,0cyclictest0-21swapper/111:23:131
53179950,0cyclictest0-21swapper/111:04:181
53169955,0cyclictest9733-21kworker/0:109:19:410
339950,0migration/39250-21latency_hist08:39:593
339950,0migration/39156-21cpuspeed_turbos12:05:113
339950,0migration/38682-21cpuspeed_turbos08:15:123
339950,0migration/37783-21kthreadcore07:00:163
339950,0migration/35338-21kthreadcore10:20:163
339950,0migration/33444-21munin-node10:00:133
339950,0migration/332560-21sh11:58:513
339950,0migration/330170-21kthreadcore09:15:163
339950,0migration/329852-21kthreadcore07:15:163
339950,0migration/324448-21latency10:10:163
339950,0migration/321113-21munin-node07:35:183
339950,0migration/320375-21latency08:20:183
339950,0migration/315013-21latency11:25:183
339950,0migration/314226-21kthreadcore11:05:163
339950,0migration/312250-21kthreadcore10:25:163
339950,0migration/312007-21kthreadcore10:45:173
339950,0migration/310882-21kthreadcore11:45:173
29725050,0irq/27-eth0-tx-28379-21munin-node08:30:120
29725050,0irq/27-eth0-tx-27866-21munin-node09:35:132
29725050,0irq/27-eth0-tx-26334-21munin-node10:55:172
29725050,0irq/27-eth0-tx-0-21swapper/210:29:442
29725050,0irq/27-eth0-tx-0-21swapper/210:10:092
29725050,0irq/27-eth0-tx-0-21swapper/210:00:132
29725050,0irq/27-eth0-tx-0-21swapper/209:59:032
29725050,0irq/27-eth0-tx-0-21swapper/008:36:270
29725050,0irq/27-eth0-tx-0-21swapper/007:35:130
29715050,0irq/26-eth0-rx-6559-21runrttasks12:24:263
29715050,0irq/26-eth0-rx-6202-21kthreadcore09:20:163
29715050,0irq/26-eth0-rx-30687-21cut09:35:173
29715050,0irq/26-eth0-rx-14275-21kthreadcore09:05:163
29715050,0irq/26-eth0-rx-0-21swapper/311:54:593
29715050,0irq/26-eth0-rx-0-21swapper/311:20:513
29715050,0irq/26-eth0-rx-0-21swapper/310:53:453
29715050,0irq/26-eth0-rx-0-21swapper/310:40:173
29715050,0irq/26-eth0-rx-0-21swapper/309:45:103
29715050,0irq/26-eth0-rx-0-21swapper/309:10:183
29715050,0irq/26-eth0-rx-0-21swapper/309:03:283
29715050,0irq/26-eth0-rx-0-21swapper/308:40:163
29715050,0irq/26-eth0-rx-0-21swapper/307:50:193
27150,0ktimersoftd/20-21swapper/212:10:292
259950,0migration/24731-21kthreadcore12:00:162
259950,0migration/23491-21latency_hist08:59:592
259950,0migration/223268-21unixbench-2d11:50:202
259950,0migration/221586-21kthreadcore09:50:152
259950,0migration/21920-21gltestperf11:40:132
259950,0migration/211215-21kthreadcore11:25:162
259950,0migration/211018-21irqcore11:05:142
259950,0migration/210461-21aten_r1power_po10:45:122
259950,0migration/210217-21sh09:01:412
19150,0ktimersoftd/10-21swapper/111:41:021
179950,0migration/19209-21sh08:15:141
179950,0migration/18522-21kthreadcore09:00:181
179950,0migration/16778-21timerandwakeup12:00:221
179950,0migration/14164-21cpuspeed_turbos09:20:121
179950,0migration/131880-21kthreadcore08:55:171
179950,0migration/128704-21munin-node09:35:141
179950,0migration/120424-21kthreadcore08:45:171
179950,0migration/119773-21if_eth107:10:141
179950,0migration/117575-21sh10:48:361
179950,0migration/116294-21kthreadcore07:30:161
179950,0migration/113817-21kthreadcore10:25:161
179950,0migration/111619-21taskset07:01:071
169950,0watchdog/17667-21grep08:10:221
169950,0watchdog/125537-21fschecks_time12:15:131
149950,0migration/09958-21sh10:44:430
149950,0migration/09425-21aten_r1power_en08:40:110
149950,0migration/08521-21latency_hist12:24:590
149950,0migration/07520-21aten_r1power_cu07:50:120
149950,0migration/07457-21kthreadcore09:20:170
149950,0migration/04018-21irqrtprio10:20:150
149950,0migration/03803-21kthreadcore11:20:170
149950,0migration/03493-21latency_hist08:59:590
149950,0migration/03248-21kthreadcore08:10:160
149950,0migration/032315-21munin-node07:20:120
149950,0migration/031194-21kthreadcore11:15:180
149950,0migration/028383-21kthreadcore11:55:170
149950,0migration/028006-21taskset08:28:320
149950,0migration/026997-21sh09:12:570
149950,0migration/025766-21munin-node12:15:150
149950,0migration/025677-21taskset09:50:390
149950,0migration/021599-21kthreadcore11:10:160
149950,0migration/019477-21hddtemp_smartct10:30:140
149950,0migration/01942-21sh10:39:590
149950,0migration/019182-21aten_r1power_en10:10:090
149950,0migration/019030-21kthreadcore07:55:190
149950,0migration/018579-21taskset07:09:460
149950,0migration/018573-21kthreadcore08:45:160
149950,0migration/017430-21latency_hist11:29:590
149950,0migration/016763-21sh10:46:130
149950,0migration/016055-21unixbench_singl11:05:220
149950,0migration/014733-21missed_timers11:45:190
149950,0migration/012600-21taskset07:26:110
149950,0migration/01085-21fschecks_time12:20:120
8215040,0irq/24-0000:00:1608-21latency_hist10:59:592
53189943,0cyclictest9312-21awk08:15:152
53189943,0cyclictest591-21awk07:20:152
53189943,0cyclictest3556-21lldpd11:46:292
53189943,0cyclictest3556-21lldpd08:42:272
53189943,0cyclictest3556-21lldpd08:39:572
53189943,0cyclictest3556-21lldpd08:05:572
53189943,0cyclictest3556-21lldpd07:42:572
53189943,0cyclictest3556-21lldpd07:17:572
53189943,0cyclictest28797-21grep08:30:132
53189943,0cyclictest16030-21awk08:20:162
53189943,0cyclictest12626-21awk09:05:132
53179944,0cyclictest0-21swapper/110:01:241
53179943,0cyclictest9739-21awk12:05:131
53179943,0cyclictest7470-21awk07:25:171
53179943,0cyclictest3597-21awk10:20:131
53179943,0cyclictest3556-21lldpd11:53:591
53179943,0cyclictest3556-21lldpd11:45:291
53179943,0cyclictest3556-21lldpd11:26:291
53179943,0cyclictest3556-21lldpd09:07:571
53179943,0cyclictest3556-21lldpd08:27:571
53179943,0cyclictest3556-21lldpd07:47:571
53179943,0cyclictest3556-21lldpd07:47:571
53179943,0cyclictest3241-21irqbalance11:31:481
53179943,0cyclictest2728-21sadc09:59:591
53179943,0cyclictest15076-21awk07:55:171
53179943,0cyclictest12868-21awk09:25:161
53179943,0cyclictest12075-21awk10:05:141
53169943,0cyclictest9733-21kworker/0:110:01:210
53169943,0cyclictest9733-21kworker/0:109:27:110
53169943,0cyclictest3514-21snmpd09:37:420
53169943,0cyclictest3514-21snmpd09:09:570
53169943,0cyclictest2179-21awk11:40:150
53169943,0cyclictest1800-21awk12:00:140
53169943,0cyclictest12397-21awk09:45:130
53169943,0cyclictest11863-21kworker/0:010:27:270
4140,0ktimersoftd/00-21swapper/010:51:030
339940,0migration/33711-21kthreadcore07:20:183
339940,0migration/330251-21kthreadcore10:35:173
339940,0migration/330140-21kthreadcore08:30:163
339940,0migration/327475-21aten_r1power_cu09:55:113
339940,0migration/32015-21kthreadcore08:55:183
339940,0migration/319926-21kthreadcore12:10:153
339940,0migration/315358-21kthreadcore07:30:153
339940,0migration/314117-21kthreadcore07:05:163
339940,0migration/311859-21latency_hist09:24:593
339940,0migration/310416-21kthreadcore12:25:153
29725040,0irq/27-eth0-tx-24610-21ssh10:50:502
29725040,0irq/27-eth0-tx-21431-21cut10:30:172
29725040,0irq/27-eth0-tx-0-21swapper/211:35:472
29725040,0irq/27-eth0-tx-0-21swapper/211:20:132
29725040,0irq/27-eth0-tx-0-21swapper/211:15:212
29725040,0irq/27-eth0-tx-0-21swapper/211:10:032
29725040,0irq/27-eth0-tx-0-21swapper/210:36:312
29725040,0irq/27-eth0-tx-0-21swapper/209:41:292
29725040,0irq/27-eth0-tx-0-21swapper/209:10:342
29725040,0irq/27-eth0-tx-0-21swapper/008:50:120
29725040,0irq/27-eth0-tx-0-21swapper/007:15:190
29715040,0irq/26-eth0-rx-9091-21kthreadcore07:25:193
29715040,0irq/26-eth0-rx-25092-21ps07:12:503
29715040,0irq/26-eth0-rx-0-21swapper/312:04:553
29715040,0irq/26-eth0-rx-0-21swapper/311:35:073
29715040,0irq/26-eth0-rx-0-21swapper/311:31:283
29715040,0irq/26-eth0-rx-0-21swapper/310:55:173
29715040,0irq/26-eth0-rx-0-21swapper/308:50:343
29715040,0irq/26-eth0-rx-0-21swapper/307:55:183
259940,0migration/2996-21fschecks_count12:20:122
259940,0migration/24566-21kthreadcore10:40:162
259940,0migration/227235-21sh10:15:102
259940,0migration/226637-21kthreadcore08:50:162
259940,0migration/221223-21kthreadcore11:30:182
259940,0migration/211917-21fschecks_count09:45:112
249940,0watchdog/29452-21ssh12:25:132
249940,0watchdog/225273-21chrt07:14:552
249940,0watchdog/225224-21latency09:30:182
249940,0watchdog/224121-21kthreadcore08:25:182
249940,0watchdog/21688-21chrt08:10:102
249940,0watchdog/210573-21kthreadcore07:50:182
249940,0watchdog/20-21swapper/207:25:182
249940,0watchdog/20-21swapper/207:00:112
179940,0migration/17653-21kthreadcore09:40:171
179940,0migration/13421-21kthreadcore12:20:161
179940,0migration/132310-21aten_r1power_vo07:20:121
179940,0migration/132063-21kthreadcore10:15:161
179940,0migration/129255-21kthreadcore11:15:171
179940,0migration/128446-21kthreadcore11:35:171
179940,0migration/127239-21kthreadcore07:15:151
179940,0migration/127059-21gltestperf10:55:141
179940,0migration/126853-21runrttasks08:00:521
179940,0migration/12515-21cpuspeed_turbos10:40:111
179940,0migration/119005-21kthreadcore08:20:181
179940,0migration/118808-21fschecks_count10:50:131
179940,0migration/116717-21kthreadcore09:45:161
179940,0migration/111018-21kthreadcore07:50:181
169940,0watchdog/125828-21ssh09:51:021
169940,0watchdog/12483-21ssh09:17:181
169940,0watchdog/120244-21diskstats07:35:141
169940,0watchdog/116180-21ssh11:05:381
169940,0watchdog/11447-21ssh10:38:341
169940,0watchdog/10-21swapper/108:30:181
149940,0migration/08595-21kthreadcore09:00:180
149940,0migration/0826-21sh11:39:510
149940,0migration/07452-21kthreadcore09:40:170
149940,0migration/028793-21kthreadcore09:55:160
149940,0migration/027773-21kernelversion07:40:150
149940,0migration/026948-21sh09:33:080
149940,0migration/021800-21kthreadcore11:50:150
149940,0migration/021434-21kthreadcore07:10:160
8215030,0irq/24-0000:00:0-21swapper/208:00:452
8215030,0irq/24-0000:00:0-21swapper/207:35:052
53189932,0cyclictest16383-21df_abs08:45:122
53189932,0cyclictest14845-21grep07:55:152
53189932,0cyclictest14051-21awk07:30:132
53189930,0cyclictest0-21swapper/207:06:162
53179932,0cyclictest26230-21awk11:55:151
53169933,0cyclictest9733-21kworker/0:108:20:010
53169933,0cyclictest9733-21kworker/0:108:15:030
53169933,0cyclictest9733-21kworker/0:108:05:080
53169933,0cyclictest9733-21kworker/0:108:00:040
53169933,0cyclictest9733-21kworker/0:107:30:040
53169933,0cyclictest9733-21kworker/0:107:00:020
53169933,0cyclictest11863-21kworker/0:012:05:070
53169933,0cyclictest11863-21kworker/0:011:30:030
53169933,0cyclictest11863-21kworker/0:011:00:090
53169933,0cyclictest11863-21kworker/0:010:55:210
53169930,0cyclictest713-21munin-node07:45:110
53169930,0cyclictest713-21munin-node07:45:110
339930,0migration/327983-21fschecks_count08:05:133
29715030,0irq/26-eth0-rx-0-21swapper/308:45:113
29715030,0irq/26-eth0-rx-0-21swapper/308:25:133
29715030,0irq/26-eth0-rx-0-21swapper/308:10:113
29715030,0irq/26-eth0-rx-0-21swapper/308:00:353
249930,0watchdog/20-21swapper/207:45:222
249930,0watchdog/20-21swapper/207:45:222
19130,0ktimersoftd/10-21swapper/110:11:291
179930,0migration/115887-21kthreadcore07:05:171
169930,0watchdog/10-21swapper/112:10:371
169930,0watchdog/10-21swapper/108:40:261
169930,0watchdog/10-21swapper/108:05:221
169930,0watchdog/10-21swapper/107:40:101
35120,0ktimersoftd/326460-21latency_hist07:40:013
169920,0watchdog/10-21swapper/108:50:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional