You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 01:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Jan 02, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271650,0ktimersoftd/25907-21perf19:07:582
304450640,0irq/25-eth10-21swapper/319:06:063
1249523625,0sleep00-21swapper/019:08:020
1216823425,0sleep10-21swapper/119:04:151
339960,0migration/35303-21sh00:16:303
339960,0migration/322764-21sh22:28:553
339960,0migration/32010-21sh23:16:313
339960,0migration/31905-21sh23:33:343
339960,0migration/318767-21kthreadcore00:23:253
339960,0migration/315309-21sh22:26:083
29715060,0irq/26-eth0-rx-6948-21munin-node21:43:253
29715060,0irq/26-eth0-rx-20456-21sh21:31:343
29715060,0irq/26-eth0-rx-16709-21irqstats22:08:243
29715060,0irq/26-eth0-rx-0-21swapper/323:07:063
149960,0migration/016188-21latency_hist22:28:080
149960,0migration/012968-21sh21:45:290
149960,0migration/012229-21sh00:00:330
125749966,0cyclictest0-21swapper/120:48:311
125739966,0cyclictest10855-21diskmemload22:58:220
125739963,0cyclictest24535-21munin-run22:33:080
8215050,0irq/24-0000:00:3870-21munin-run23:38:082
8215050,0irq/24-0000:00:19238-21grep19:13:082
8215050,0irq/24-0000:00:10842-21latency_hist21:08:082
4150,0ktimersoftd/00-21swapper/000:36:260
339950,0migration/3913-21missed_timers21:18:263
339950,0migration/38928-21munin-node20:38:303
339950,0migration/38402-21munin-node23:18:263
339950,0migration/37046-21munin-node22:38:323
339950,0migration/36879-21runrttasks22:20:553
339950,0migration/35684-21kernelversion21:03:263
339950,0migration/33882-21taskset20:59:483
339950,0migration/33506-21kthreadcore19:23:253
339950,0migration/329445-21kthreadcore20:08:253
339950,0migration/328576-21diskmemload00:09:403
339950,0migration/328020-21if_eth119:43:253
339950,0migration/326051-21aten_r1power_vo22:53:203
339950,0migration/325897-21kthreadcore20:28:293
339950,0migration/325673-21kthreadcore20:03:263
339950,0migration/320096-21hwlatdetect21:13:213
339950,0migration/31811-21kthreadcore00:33:273
339950,0migration/317032-21runrttasks21:08:333
339950,0migration/314620-21kthreadcore23:23:293
29725050,0irq/27-eth0-tx-9256-21munin-node22:43:292
29725050,0irq/27-eth0-tx-32070-21munin-node22:18:232
29725050,0irq/27-eth0-tx-16428-21munin-node22:28:252
29725050,0irq/27-eth0-tx-0-21swapper/223:54:032
29725050,0irq/27-eth0-tx-0-21swapper/222:35:152
29725050,0irq/27-eth0-tx-0-21swapper/221:23:202
29725050,0irq/27-eth0-tx-0-21swapper/221:19:272
29725050,0irq/27-eth0-tx-0-21swapper/200:06:382
29715050,0irq/26-eth0-rx-5586-21kthreadcore20:13:273
29715050,0irq/26-eth0-rx-4984-21ssh21:23:233
29715050,0irq/26-eth0-rx-4923-21ssh21:41:233
29715050,0irq/26-eth0-rx-27379-21switchtime21:33:333
29715050,0irq/26-eth0-rx-26559-21kthreadcore00:28:263
29715050,0irq/26-eth0-rx-24233-21nfs_client23:08:283
29715050,0irq/26-eth0-rx-13280-21ssh00:20:303
29715050,0irq/26-eth0-rx-13280-21ssh00:20:303
29715050,0irq/26-eth0-rx-11732-21ssh23:59:183
29715050,0irq/26-eth0-rx-10130-21cut20:43:193
29715050,0irq/26-eth0-rx-0-21swapper/323:53:183
29715050,0irq/26-eth0-rx-0-21swapper/323:38:233
29715050,0irq/26-eth0-rx-0-21swapper/323:02:223
29715050,0irq/26-eth0-rx-0-21swapper/322:48:193
29715050,0irq/26-eth0-rx-0-21swapper/322:35:053
29715050,0irq/26-eth0-rx-0-21swapper/322:03:223
29715050,0irq/26-eth0-rx-0-21swapper/321:53:253
29715050,0irq/26-eth0-rx-0-21swapper/319:13:253
29715050,0irq/26-eth0-rx-0-21swapper/300:04:523
27150,0ktimersoftd/20-21swapper/223:13:172
259950,0migration/230126-21sh21:56:372
259950,0migration/226454-21kthreadcore20:03:272
249950,0watchdog/226874-21kthreadcore22:13:262
249950,0watchdog/20-21swapper/219:33:272
19150,0ktimersoftd/10-21swapper/122:47:141
179950,0migration/13778-21taskset20:58:471
179950,0migration/132687-21kthreadcore20:33:301
179950,0migration/127385-21kthreadcore22:33:271
179950,0migration/120894-21sh00:06:221
179950,0migration/115566-21idleruntime19:58:231
179950,0migration/114160-21unixbench_multi22:23:331
179950,0migration/113596-21kthreadcore20:18:261
169950,0watchdog/126873-21kthreadcore22:13:261
169950,0watchdog/114989-21ssh22:07:501
149950,0migration/08549-21taskset20:14:150
149950,0migration/07581-21fschecks_count19:28:210
149950,0migration/04234-21aten_r1power_en21:23:200
149950,0migration/031916-21timerandwakeup19:18:310
149950,0migration/031595-21kthreadcore20:08:260
149950,0migration/031311-21cpuspeed_turbos21:58:200
149950,0migration/031182-21irqcore20:58:280
149950,0migration/030790-21kthreadcore22:53:270
149950,0migration/027285-21cpuspeed_turbos23:13:210
149950,0migration/027052-21latency_hist19:43:080
149950,0migration/025260-21timerwakeupswit21:13:310
149950,0migration/024272-21fschecks_count22:13:220
149950,0migration/023902-21fschecks_count20:53:220
149950,0migration/023893-21kthreadcore21:33:270
149950,0migration/021080-21kernelversion23:28:250
149950,0migration/019598-21sh23:45:390
149950,0migration/014901-21aten_r1power_vo19:58:200
149950,0migration/013382-21fschecks_time21:28:210
149950,0migration/012839-21kthreadcore23:23:280
149950,0migration/012610-21kthreadcore20:18:260
149950,0migration/011076-21kthreadcore00:18:270
149950,0migration/011076-21kthreadcore00:18:270
125759954,0cyclictest3908-21awk23:18:232
125759950,0cyclictest0-21swapper/223:23:122
125759950,0cyclictest0-21swapper/222:50:442
125759950,0cyclictest0-21swapper/200:26:072
125749955,0cyclictest0-21swapper/120:23:271
125749950,0cyclictest0-21swapper/122:50:231
125739955,0cyclictest0-21swapper/023:33:510
125739955,0cyclictest0-21swapper/022:08:050
125739955,0cyclictest0-21swapper/021:13:030
125739955,0cyclictest0-21swapper/000:16:060
339940,0migration/319925-21kthreadcore20:23:283
339940,0migration/319541-21kthreadcore21:48:263
339940,0migration/316725-21kthreadcore19:08:263
339940,0migration/315789-21kthreadcore23:43:263
339940,0migration/311689-21kthreadcore19:53:263
29725040,0irq/27-eth0-tx-5221-21munin-node23:58:192
29725040,0irq/27-eth0-tx-21868-21ssh00:08:102
29725040,0irq/27-eth0-tx-12558-21munin-node23:43:232
29725040,0irq/27-eth0-tx-0-21swapper/223:48:112
29725040,0irq/27-eth0-tx-0-21swapper/223:38:242
29725040,0irq/27-eth0-tx-0-21swapper/223:28:232
29725040,0irq/27-eth0-tx-0-21swapper/223:09:112
29725040,0irq/27-eth0-tx-0-21swapper/223:03:322
29725040,0irq/27-eth0-tx-0-21swapper/222:58:552
29725040,0irq/27-eth0-tx-0-21swapper/222:53:362
29725040,0irq/27-eth0-tx-0-21swapper/222:08:262
29725040,0irq/27-eth0-tx-0-21swapper/221:58:222
29725040,0irq/27-eth0-tx-0-21swapper/221:13:192
29725040,0irq/27-eth0-tx-0-21swapper/220:43:202
29725040,0irq/27-eth0-tx-0-21swapper/200:33:312
29725040,0irq/27-eth0-tx-0-21swapper/200:28:102
29725040,0irq/27-eth0-tx-0-21swapper/200:19:222
29725040,0irq/27-eth0-tx-0-21swapper/200:19:222
29715040,0irq/26-eth0-rx-0-21swapper/323:48:593
29715040,0irq/26-eth0-rx-0-21swapper/323:28:203
29715040,0irq/26-eth0-rx-0-21swapper/322:43:193
29715040,0irq/26-eth0-rx-0-21swapper/322:14:313
29715040,0irq/26-eth0-rx-0-21swapper/321:58:483
29715040,0irq/26-eth0-rx-0-21swapper/319:48:303
259940,0migration/28110-21aten_r1power_cu22:23:202
259940,0migration/2452-21kthreadcore00:13:242
259940,0migration/232062-21kthreadcore20:33:302
259940,0migration/230468-21fschecks_count21:38:222
259940,0migration/227326-21cpuspeed_turbos19:43:222
259940,0migration/223729-21kthreadcore19:13:262
259940,0migration/221547-21aten_r1power_en21:33:212
259940,0migration/216290-21fschecks_count20:23:242
249940,0watchdog/29132-21hostname20:18:182
249940,0watchdog/27924-21tr21:43:262
249940,0watchdog/217711-21munin-node20:48:222
249940,0watchdog/212773-21ssh21:28:142
249940,0watchdog/20-21swapper/222:38:262
249940,0watchdog/20-21swapper/219:48:272
179940,0migration/1702-21taskset22:38:141
179940,0migration/1515-21munin-node19:43:341
179940,0migration/132650-21fschecks_time22:18:211
179940,0migration/132071-21kthreadcore21:38:261
179940,0migration/131405-21munin-node22:53:291
179940,0migration/130898-21kthreadcore19:18:251
179940,0migration/127607-21fschecks_time23:13:221
179940,0migration/125542-21kthreadcore20:53:261
179940,0migration/121403-21kthreadcore23:08:251
179940,0migration/121252-21kthreadcore21:13:251
179940,0migration/117630-21runrttasks23:06:221
179940,0migration/115142-21kthreadcore21:28:251
179940,0migration/110895-21hddtemp_smartct20:43:221
169940,0watchdog/15508-21ssh00:34:171
169940,0watchdog/14718-21kthreadcore20:38:261
169940,0watchdog/13765-21kthreadcore20:13:261
169940,0watchdog/13540-21cut19:48:271
169940,0watchdog/13343-21missed_timers00:13:261
169940,0watchdog/124356-21grep23:28:261
169940,0watchdog/123510-21rm22:30:421
169940,0watchdog/119015-21kthreadcore21:48:261
169940,0watchdog/10-21swapper/123:18:261
169940,0watchdog/10-21swapper/121:58:261
149940,0migration/07923-21runrttasks22:40:380
149940,0migration/028262-21entropy21:18:200
149940,0migration/025167-21kthreadcore19:38:260
149940,0migration/023871-21kthreadcore23:48:240
149940,0migration/020775-21runrttasks00:06:040
149940,0migration/015779-21sensors_fan20:43:310
125759943,0cyclictest8078-21awk22:03:272
125759943,0cyclictest3556-21lldpd20:58:352
125759943,0cyclictest3556-21lldpd20:56:052
125759943,0cyclictest3309-21awk20:13:252
125759943,0cyclictest3241-21irqbalance21:08:162
125759943,0cyclictest3241-21irqbalance20:38:472
125759943,0cyclictest3241-21irqbalance20:29:072
125759943,0cyclictest3241-21irqbalance19:59:172
125759943,0cyclictest3241-21irqbalance19:53:172
125759943,0cyclictest3241-21irqbalance19:39:272
125759943,0cyclictest29089-21awk20:08:242
125759943,0cyclictest26928-21awk19:18:222
125759943,0cyclictest15238-21awk21:48:232
125759943,0cyclictest1307-21awk19:23:232
125759940,0cyclictest7128-21munin-node19:28:202
125749944,0cyclictest0-21swapper/123:54:331
125749944,0cyclictest0-21swapper/121:03:301
125749944,0cyclictest0-21swapper/100:10:571
125749943,0cyclictest9088-21awk19:53:241
125749943,0cyclictest7981-21awk19:28:221
125749943,0cyclictest7183-21awk00:18:241
125749943,0cyclictest7183-21awk00:18:241
125749943,0cyclictest5044-21awk21:23:231
125749943,0cyclictest3514-21snmpd20:31:451
125749943,0cyclictest3514-21snmpd20:10:451
125749943,0cyclictest3241-21irqbalance23:42:061
125749943,0cyclictest3241-21irqbalance22:58:561
125749943,0cyclictest3241-21irqbalance21:44:061
125749943,0cyclictest3241-21irqbalance21:19:161
125749943,0cyclictest29163-21awk23:33:241
125749943,0cyclictest24008-21awk00:28:241
125749943,0cyclictest22756-21sadc00:28:081
125749943,0cyclictest22364-21awk20:03:231
125749943,0cyclictest21877-21awk23:48:231
125749943,0cyclictest21298-21awk19:38:221
125749943,0cyclictest14623-21awk19:33:241
125749943,0cyclictest13416-21awk23:43:231
125749943,0cyclictest1331-21awk19:23:231
125749943,0cyclictest11600-21grep21:08:221
125749940,0cyclictest0-21swapper/123:24:361
125739944,0cyclictest0-21swapper/023:21:520
125739944,0cyclictest0-21swapper/023:08:470
125739944,0cyclictest0-21swapper/022:50:580
125739944,0cyclictest0-21swapper/022:21:100
125739944,0cyclictest0-21swapper/021:56:530
125739944,0cyclictest0-21swapper/020:33:390
125739944,0cyclictest0-21swapper/000:29:490
125739943,0cyclictest5341-21awk21:03:250
125739943,0cyclictest4940-21sadc23:58:080
125739943,0cyclictest4052-21grep20:38:210
125739943,0cyclictest3556-21lldpd19:48:340
125739943,0cyclictest3514-21snmpd20:27:150
125739943,0cyclictest3241-21irqbalance23:40:060
125739943,0cyclictest3241-21irqbalance00:08:360
125739943,0cyclictest30836-21awk21:38:240
125739943,0cyclictest15223-21awk21:48:230
125739943,0cyclictest14693-21awk19:33:250
125739940,0cyclictest17321-21munin-run22:48:080
339930,0migration/39896-21kthreadcore19:28:253
29715030,0irq/26-eth0-rx-0-21swapper/320:54:433
29715030,0irq/26-eth0-rx-0-21swapper/320:48:273
29715030,0irq/26-eth0-rx-0-21swapper/320:33:293
29715030,0irq/26-eth0-rx-0-21swapper/320:18:193
29715030,0irq/26-eth0-rx-0-21swapper/319:38:193
29715030,0irq/26-eth0-rx-0-21swapper/319:33:213
29715030,0irq/26-eth0-rx-0-21swapper/319:19:273
169930,0watchdog/10-21swapper/121:34:501
125749932,0cyclictest7420-21kthreadcore23:58:271
125749932,0cyclictest23510-21cut21:53:231
125749932,0cyclictest21197-21sensors22:08:301
125739932,0cyclictest3514-21snmpd23:07:150
125739932,0cyclictest3514-21snmpd20:50:450
125739932,0cyclictest3514-21snmpd20:28:150
125739932,0cyclictest32764-21aten_r1power_cu19:23:190
125739932,0cyclictest22461-21awk20:03:240
29715020,0irq/26-eth0-rx-0-21swapper/319:58:273
169920,0watchdog/10-21swapper/119:13:151
125749920,0cyclictest0-21swapper/119:08:201
125739921,0cyclictest8046-21cat19:53:080
125739921,0cyclictest0-21swapper/022:33:110
125739920,0cyclictest19406-21latency_hist19:13:080
125739920,0cyclictest0-21swapper/022:08:120
125739920,0cyclictest0-21swapper/019:08:080
125739920,0cyclictest0-21swapper/000:23:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional