You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 11:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Jul 11, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3160423624,0sleep10-21swapper/118:45:521
304450350,0irq/25-eth10-21swapper/318:45:333
3180923423,0sleep00-21swapper/018:48:190
3180623026,0sleep20-21swapper/218:48:172
339960,0migration/34659-21sh20:52:373
339960,0migration/321834-21sh22:04:283
339960,0migration/320976-21sh00:03:523
339960,0migration/320584-21sh23:42:213
339960,0migration/311666-21switchtime23:15:323
320429965,0cyclictest12940-21ssh23:38:432
320429960,0cyclictest0-21swapper/222:02:422
320419960,0cyclictest0-21swapper/121:49:411
320419960,0cyclictest0-21swapper/121:15:281
29725060,0irq/27-eth0-tx-20854-21munin-node19:30:310
29715060,0irq/26-eth0-rx-6967-21iostat_ios22:55:263
27160,0ktimersoftd/20-21swapper/221:42:412
259960,0migration/228140-21proc_pri21:05:312
19160,0ktimersoftd/130132-21diskmemload22:39:491
149960,0migration/04326-21sh23:32:110
149960,0migration/022269-21kthreadcore19:05:280
149960,0migration/022039-21sh21:04:480
149960,0migration/011917-21sh23:56:020
149960,0migration/011450-21missed_timers22:35:280
8215050,0irq/24-0000:00:29913-21latency_hist22:50:092
339950,0migration/37048-21kthreadcore23:35:263
339950,0migration/36502-21kthreadcore20:05:273
339950,0migration/36459-21latency_hist21:15:113
339950,0migration/36457-21munin-node00:15:253
339950,0migration/33299-21pmu-power23:10:303
339950,0migration/331527-21munin-node21:10:273
339950,0migration/331224-21kthreadcore20:00:273
339950,0migration/331126-21fschecks_time22:10:233
339950,0migration/329748-21kthreadcore19:10:283
339950,0migration/327929-21kthreadcore20:20:283
339950,0migration/327819-21munin-node19:55:333
339950,0migration/326023-21kthreadcore21:25:263
339950,0migration/325507-21kthreadcore21:45:283
339950,0migration/323538-21irqrtprio22:05:233
339950,0migration/323467-21kthreadcore19:05:283
339950,0migration/321438-21needreboot20:15:283
339950,0migration/31710-21kthreadcore00:10:273
339950,0migration/316843-21kthreadcore23:20:273
339950,0migration/315492-21taskset20:11:003
339950,0migration/31416-21taskset19:39:023
339950,0migration/314155-21latency_hist22:20:093
339950,0migration/313807-21crond22:40:093
339950,0migration/313107-21taskset23:59:263
339950,0migration/312348-21timerandwakeup21:35:323
339950,0migration/312348-21timerandwakeup21:35:323
339950,0migration/310242-21latency_hist20:35:113
320429955,0cyclictest0-21swapper/223:26:062
320429955,0cyclictest0-21swapper/223:22:572
320429955,0cyclictest0-21swapper/221:20:272
320429955,0cyclictest0-21swapper/200:00:282
320429950,0cyclictest0-21swapper/222:15:192
320419955,0cyclictest0-21swapper/123:00:091
320419955,0cyclictest0-21swapper/122:50:231
320419955,0cyclictest0-21swapper/122:30:211
320419955,0cyclictest0-21swapper/122:19:291
320419955,0cyclictest0-21swapper/122:13:101
320419950,0cyclictest0-21swapper/123:11:211
320419950,0cyclictest0-21swapper/121:57:241
320419950,0cyclictest0-21swapper/120:51:211
320409955,0cyclictest0-21swapper/022:19:400
320409953,0cyclictest4454-21kworker/0:122:26:010
320409950,0cyclictest0-21swapper/023:22:530
320409950,0cyclictest0-21swapper/022:45:220
320409950,0cyclictest0-21swapper/019:36:480
29725050,0irq/27-eth0-tx-7036-21sh21:55:242
29725050,0irq/27-eth0-tx-3254-21open_files23:10:292
29725050,0irq/27-eth0-tx-30132-21diskmemload21:19:202
29725050,0irq/27-eth0-tx-19323-21nfs_client23:40:282
29725050,0irq/27-eth0-tx-15184-21munin-node19:50:290
29725050,0irq/27-eth0-tx-0-21swapper/223:31:262
29725050,0irq/27-eth0-tx-0-21swapper/223:00:532
29725050,0irq/27-eth0-tx-0-21swapper/222:57:082
29725050,0irq/27-eth0-tx-0-21swapper/222:34:042
29725050,0irq/27-eth0-tx-0-21swapper/222:10:122
29725050,0irq/27-eth0-tx-0-21swapper/222:05:352
29725050,0irq/27-eth0-tx-0-21swapper/221:40:102
29725050,0irq/27-eth0-tx-0-21swapper/221:40:102
29725050,0irq/27-eth0-tx-0-21swapper/221:00:202
29725050,0irq/27-eth0-tx-0-21swapper/220:52:182
29725050,0irq/27-eth0-tx-0-21swapper/020:50:200
29725050,0irq/27-eth0-tx-0-21swapper/020:45:310
29725050,0irq/27-eth0-tx-0-21swapper/020:15:280
29725050,0irq/27-eth0-tx-0-21swapper/020:10:210
29715050,0irq/26-eth0-rx-5150-21kthreadcore19:15:283
29715050,0irq/26-eth0-rx-32551-21kthreadcore20:25:283
29715050,0irq/26-eth0-rx-31132-21hwlatdetect22:30:243
29715050,0irq/26-eth0-rx-29056-21ssh21:06:503
29715050,0irq/26-eth0-rx-2841-21cut19:40:263
29715050,0irq/26-eth0-rx-22821-21if_eth123:25:243
29715050,0irq/26-eth0-rx-22803-21fschecks_time22:45:233
29715050,0irq/26-eth0-rx-22282-21ssh21:24:473
29715050,0irq/26-eth0-rx-2116-21cut23:30:273
29715050,0irq/26-eth0-rx-20785-21sh22:42:043
29715050,0irq/26-eth0-rx-19899-21proc_pri21:00:303
29715050,0irq/26-eth0-rx-1782-21kthreadcore23:50:253
29715050,0irq/26-eth0-rx-12489-21munin-node18:55:333
29715050,0irq/26-eth0-rx-0-21swapper/323:00:193
29715050,0irq/26-eth0-rx-0-21swapper/322:25:403
29715050,0irq/26-eth0-rx-0-21swapper/322:15:273
29715050,0irq/26-eth0-rx-0-21swapper/321:55:223
29715050,0irq/26-eth0-rx-0-21swapper/320:58:123
27150,0ktimersoftd/20-21swapper/223:57:382
259950,0migration/29383-21kthreadcore19:20:252
259950,0migration/26699-21fschecks_time22:35:232
259950,0migration/25652-21latency_hist23:15:092
259950,0migration/231207-21munin-node21:10:252
259950,0migration/21728-21munin-node20:00:322
259950,0migration/215176-21munin-node22:20:262
259950,0migration/212467-21sh00:17:482
179950,0migration/15731-21taskset23:15:121
179950,0migration/13750-21missed_timers21:10:311
179950,0migration/132623-21kthreadcore23:30:261
179950,0migration/12108-21taskset20:00:461
179950,0migration/115774-21munin-node20:35:281
179950,0migration/114562-21kthreadcore19:00:281
179950,0migration/113834-21missed_timers19:45:291
149950,0migration/0900-21kthreadcore21:10:290
149950,0migration/04002-21timerandwakeup22:10:320
149950,0migration/030145-21gltestperf20:25:240
149950,0migration/029160-21sh21:07:090
149950,0migration/028063-21sh23:45:470
149950,0migration/026426-21kthreadcore20:20:270
149950,0migration/02583-21idleruntime19:40:250
149950,0migration/024602-21kthreadcore23:05:250
149950,0migration/024369-21kthreadcore21:45:270
149950,0migration/023656-21kthreadcore00:05:260
149950,0migration/023234-21munin-node22:05:220
149950,0migration/021577-21turbostat23:45:090
149950,0migration/021300-21sh22:23:140
149950,0migration/02111-21kthreadcore18:50:260
149950,0migration/019853-21munin-node22:00:290
149950,0migration/015790-21irqcore21:20:250
149950,0migration/015004-21fschecks_count21:40:230
149950,0migration/013823-21taskset21:59:400
149950,0migration/012802-21ksmtuned18:57:460
149950,0migration/012264-21switchtime21:35:320
149950,0migration/012264-21switchtime21:35:320
149950,0migration/011911-21timerwakeupswit22:55:330
339940,0migration/332706-21kthreadcore22:50:263
339940,0migration/324119-21taskset20:45:233
339940,0migration/318069-21kthreadcore19:50:313
320429944,0cyclictest0-21swapper/219:19:502
320429943,0cyclictest9408-21awk19:45:252
320429943,0cyclictest3649-21awk20:05:242
320429943,0cyclictest3514-21snmpd20:38:382
320429943,0cyclictest3514-21snmpd20:33:382
320429943,0cyclictest3514-21snmpd20:26:382
320429943,0cyclictest3514-21snmpd20:23:382
320429943,0cyclictest3514-21snmpd20:19:382
320429943,0cyclictest3514-21snmpd19:58:232
320429943,0cyclictest3514-21snmpd19:51:382
320429943,0cyclictest3514-21snmpd19:25:382
320429943,0cyclictest3514-21snmpd18:51:382
320419943,0cyclictest7431-21awk21:35:241
320419943,0cyclictest7431-21awk21:35:241
320419943,0cyclictest6971-21irqcore20:55:271
320419943,0cyclictest3768-21awk20:05:251
320419943,0cyclictest3556-21lldpd21:30:121
320419943,0cyclictest3556-21lldpd19:43:411
320419943,0cyclictest3556-21lldpd00:00:421
320419943,0cyclictest3514-21snmpd20:42:231
320419943,0cyclictest3514-21snmpd18:57:381
320419943,0cyclictest3514-21snmpd00:17:391
320419943,0cyclictest3241-21irqbalance00:07:081
320419943,0cyclictest16115-21awk19:50:291
320419943,0cyclictest14975-21awk23:00:251
320419943,0cyclictest0-21swapper/123:35:091
320409944,0cyclictest0-21swapper/023:50:380
320409944,0cyclictest0-21swapper/000:19:290
320409943,0cyclictest8402-21awk19:20:230
320409943,0cyclictest4454-21kworker/0:122:44:030
320409943,0cyclictest3514-21snmpd21:15:380
320409940,0cyclictest6176-21awk21:35:100
29725040,0irq/27-eth0-tx-6804-21if_err_eth120:55:252
29725040,0irq/27-eth0-tx-29723-21ssh00:10:152
29725040,0irq/27-eth0-tx-27173-21needreboot00:05:282
29725040,0irq/27-eth0-tx-22562-21munin-node21:45:212
29725040,0irq/27-eth0-tx-0-21swapper/223:52:282
29725040,0irq/27-eth0-tx-0-21swapper/223:05:172
29725040,0irq/27-eth0-tx-0-21swapper/222:40:212
29725040,0irq/27-eth0-tx-0-21swapper/222:25:262
29725040,0irq/27-eth0-tx-0-21swapper/221:25:122
29725040,0irq/27-eth0-tx-0-21swapper/020:35:220
29725040,0irq/27-eth0-tx-0-21swapper/020:05:220
29725040,0irq/27-eth0-tx-0-21swapper/019:00:320
29715040,0irq/26-eth0-rx-0-21swapper/323:45:223
29715040,0irq/26-eth0-rx-0-21swapper/323:07:143
29715040,0irq/26-eth0-rx-0-21swapper/321:50:193
29715040,0irq/26-eth0-rx-0-21swapper/321:40:213
29715040,0irq/26-eth0-rx-0-21swapper/321:31:383
29715040,0irq/26-eth0-rx-0-21swapper/319:30:213
29715040,0irq/26-eth0-rx-0-21swapper/300:05:253
259940,0migration/28166-21kthreadcore18:55:272
259940,0migration/23424-21kthreadcore21:50:272
259940,0migration/229834-21kthreadcore19:35:272
259940,0migration/227735-21irqrtprio19:10:272
259940,0migration/227231-21kthreadcore22:45:272
259940,0migration/221864-21aten_r1power_cu23:45:212
259940,0migration/221561-21kthreadcore19:05:282
259940,0migration/218714-21kthreadcore20:40:272
249940,0watchdog/29478-21aten_r1power_cu20:10:212
249940,0watchdog/232036-21kthreadcore21:30:242
179940,0migration/132679-21kthreadcore00:10:271
179940,0migration/131763-21kthreadcore19:10:291
179940,0migration/131614-21irqcore21:50:251
179940,0migration/128951-21taskset20:22:521
179940,0migration/128600-21sh23:27:121
179940,0migration/125455-21proc_pri19:05:321
179940,0migration/12472-21pmu-power20:25:311
179940,0migration/124389-21kthreadcore22:45:261
179940,0migration/123503-21hwlatdetect21:25:231
179940,0migration/123022-21kthreadcore19:30:261
179940,0migration/122616-21fschecks_time23:05:211
179940,0migration/116009-21kthreadcore21:00:261
179940,0migration/115253-21kthreadcore23:20:261
179940,0migration/114528-21aten2.4-expect21:40:201
179940,0migration/114253-21aten_r1power_vo22:40:221
179940,0migration/113583-21taskset19:20:441
169940,0watchdog/127783-21latency21:05:281
169940,0watchdog/127687-21ssh22:25:281
169940,0watchdog/127195-21latency23:45:281
169940,0watchdog/125507-21sed22:05:241
169940,0watchdog/121652-21ssh22:03:561
169940,0watchdog/121133-21sh21:21:281
169940,0watchdog/120421-21ssh23:41:521
169940,0watchdog/112299-21ssh23:57:081
169940,0watchdog/10-21swapper/120:45:281
169940,0watchdog/10-21swapper/120:10:211
169940,0watchdog/10-21swapper/119:25:291
149940,0migration/07506-21kthreadcore20:55:270
149940,0migration/03637-21sensors_volt22:50:310
149940,0migration/02106-21kernelversion19:15:270
149940,0migration/01965-21kthreadcore00:10:270
149940,0migration/012684-21kthreadcore19:45:280
8215030,0irq/24-0000:00:0-21swapper/219:40:322
320429932,0cyclictest24579-21awk20:45:262
320429932,0cyclictest21832-21awk19:30:252
320419932,0cyclictest5650-21kthreadcore19:15:281
320419932,0cyclictest3556-21lldpd20:31:411
320419932,0cyclictest3241-21irqbalance18:50:291
320419932,0cyclictest30873-21awk23:50:231
320409933,0cyclictest6752-21awk23:15:250
320409933,0cyclictest4454-21kworker/0:122:30:130
320409933,0cyclictest4454-21kworker/0:121:50:150
320409933,0cyclictest4454-21kworker/0:121:25:160
320409933,0cyclictest4454-21kworker/0:120:40:240
320409933,0cyclictest4454-21kworker/0:120:30:220
320409933,0cyclictest4454-21kworker/0:120:00:120
320409933,0cyclictest4454-21kworker/0:119:55:220
320409933,0cyclictest4454-21kworker/0:119:25:240
320409933,0cyclictest4454-21kworker/0:119:10:200
320409933,0cyclictest30982-21kworker/0:023:35:170
320409933,0cyclictest30982-21kworker/0:023:25:110
320409933,0cyclictest30982-21kworker/0:023:10:110
320409933,0cyclictest30982-21kworker/0:023:00:310
320409933,0cyclictest30982-21kworker/0:000:00:110
29715030,0irq/26-eth0-rx-0-21swapper/320:35:333
29715030,0irq/26-eth0-rx-0-21swapper/319:45:223
29715030,0irq/26-eth0-rx-0-21swapper/319:25:343
29715030,0irq/26-eth0-rx-0-21swapper/318:50:213
249930,0watchdog/213903-21fschecks_time19:00:252
19130,0ktimersoftd/114245-21munin-node22:20:251
179930,0migration/131668-21kthreadcore19:35:281
179930,0migration/122281-21fschecks_count19:55:221
169930,0watchdog/10-21swapper/120:18:011
339920,0migration/38319-21hddtemp_smartct19:20:223
329920,0watchdog/322435-21netstat20:40:293
29715020,0irq/26-eth0-rx-0-21swapper/319:00:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional