You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 05:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed May 20, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271510,0ktimersoftd/229902-21perf18:54:582
393423627,0sleep30-21swapper/318:53:023
399623430,0sleep00-21swapper/018:53:470
397123425,0sleep10-21swapper/118:53:281
41909960,0cyclictest0-21swapper/223:30:362
41879960,0cyclictest0-21swapper/022:47:110
41879960,0cyclictest0-21swapper/022:47:110
339960,0migration/39989-21sh22:59:383
339960,0migration/39918-21sh21:37:293
339960,0migration/3689-21sh23:32:293
339960,0migration/34363-21kthreadcore23:35:253
339960,0migration/317860-21sh22:23:163
29725060,0irq/27-eth0-tx-3653-21ssh20:55:250
29725060,0irq/27-eth0-tx-18728-21munin-node19:30:310
149960,0migration/01313-21gltestperf20:05:210
8215050,0irq/24-0000:00:8552-21port_sshpriv21:55:282
8215050,0irq/24-0000:00:18812-21latency_hist21:05:082
41909955,0cyclictest0-21swapper/222:34:002
41909955,0cyclictest0-21swapper/221:50:102
41909955,0cyclictest0-21swapper/221:36:462
41899955,0cyclictest2321-21diskmemload23:50:141
41899955,0cyclictest0-21swapper/120:59:391
41899950,0cyclictest0-21swapper/121:56:531
41899950,0cyclictest0-21swapper/121:28:581
41879955,0cyclictest2321-21diskmemload00:03:300
41879955,0cyclictest0-21swapper/022:22:070
41879950,0cyclictest0-21swapper/023:38:070
41879950,0cyclictest0-21swapper/023:01:510
41879950,0cyclictest0-21swapper/022:52:040
41879950,0cyclictest0-21swapper/022:37:250
339950,0migration/37385-21kthreadcore22:15:253
339950,0migration/36789-21taskset20:06:163
339950,0migration/36452-21irqcore19:20:253
339950,0migration/33959-21kthreadcore20:30:243
339950,0migration/33914-21munin-node20:55:243
339950,0migration/32789-21taskset21:55:143
339950,0migration/326876-21runrttasks21:50:063
339950,0migration/324952-21runrttasks23:06:263
339950,0migration/320991-21kthreadcore22:45:233
339950,0migration/320991-21kthreadcore22:45:233
339950,0migration/318858-21munin-node19:30:203
339950,0migration/317325-21sh21:01:283
339950,0migration/313614-21munin-node20:35:263
339950,0migration/313027-21kthreadcore19:00:263
29725050,0irq/27-eth0-tx-7077-21nfs4_client00:15:282
29725050,0irq/27-eth0-tx-27676-21cpuspeed_turbos21:30:232
29725050,0irq/27-eth0-tx-27029-21munin-node22:10:212
29725050,0irq/27-eth0-tx-26184-21ssh21:48:062
29725050,0irq/27-eth0-tx-2575-21munin-node20:55:302
29725050,0irq/27-eth0-tx-20203-21kthreadcore00:05:272
29725050,0irq/27-eth0-tx-17395-21unixbench_singl21:20:312
29725050,0irq/27-eth0-tx-11528-21munin-node21:40:212
29725050,0irq/27-eth0-tx-0-21swapper/223:55:212
29725050,0irq/27-eth0-tx-0-21swapper/223:15:182
29725050,0irq/27-eth0-tx-0-21swapper/223:00:292
29725050,0irq/27-eth0-tx-0-21swapper/222:56:482
29725050,0irq/27-eth0-tx-0-21swapper/020:35:310
29725050,0irq/27-eth0-tx-0-21swapper/019:29:170
29715050,0irq/26-eth0-rx-8726-21ssh22:35:383
29715050,0irq/26-eth0-rx-29660-21kthreadcore21:50:263
29715050,0irq/26-eth0-rx-25904-21sshd21:07:093
29715050,0irq/26-eth0-rx-25868-21aten_r1power_cu00:10:183
29715050,0irq/26-eth0-rx-11727-21irqrtprio23:00:253
29715050,0irq/26-eth0-rx-0-21swapper/323:58:073
29715050,0irq/26-eth0-rx-0-21swapper/323:53:343
29715050,0irq/26-eth0-rx-0-21swapper/323:41:273
29715050,0irq/26-eth0-rx-0-21swapper/323:25:503
29715050,0irq/26-eth0-rx-0-21swapper/323:20:243
29715050,0irq/26-eth0-rx-0-21swapper/322:25:253
29715050,0irq/26-eth0-rx-0-21swapper/322:10:153
29715050,0irq/26-eth0-rx-0-21swapper/322:06:383
29715050,0irq/26-eth0-rx-0-21swapper/321:15:403
29715050,0irq/26-eth0-rx-0-21swapper/300:00:233
259950,0migration/28443-21kthreadcore19:45:252
259950,0migration/2389-21ntp_offset21:10:272
259950,0migration/231211-21kthreadcore22:50:242
259950,0migration/225120-21sh23:27:322
259950,0migration/224187-21kthreadcore21:05:262
259950,0migration/219334-21idleruntime23:05:232
259950,0migration/212362-21kthreadcore00:00:232
259950,0migration/211770-21idleruntime22:00:232
249950,0watchdog/231958-21grep23:10:262
249950,0watchdog/222708-21kthreadcore19:55:272
249950,0watchdog/20-21swapper/219:40:232
19150,0ktimersoftd/10-21swapper/121:40:391
179950,0migration/18211-21sh23:56:101
179950,0migration/18143-21timerwakeupswit23:15:301
179950,0migration/132109-21irqcore19:15:231
179950,0migration/131291-21kthreadcore22:10:251
179950,0migration/126698-21latency_hist22:30:081
179950,0migration/125500-21timerandwakeup19:55:331
179950,0migration/124571-21munin-node21:05:271
179950,0migration/121432-21latency_hist20:45:081
179950,0migration/120999-21munin-node00:05:271
179950,0migration/119211-21munin-node22:25:201
179950,0migration/114008-21munin-node20:35:301
179950,0migration/112652-21munin-node20:10:261
169950,0watchdog/116162-21cut19:00:271
149950,0migration/08901-21sh23:17:060
149950,0migration/08182-21sh22:55:280
149950,0migration/07629-21kthreadcore19:20:260
149950,0migration/07384-21irqcore19:45:240
149950,0migration/04924-21runrttasks19:17:360
149950,0migration/04395-21taskset18:55:160
149950,0migration/030575-21taskset19:12:090
149950,0migration/029125-21kthreadcore21:10:240
149950,0migration/028357-21aten_r1power_fr20:50:190
149950,0migration/028334-21kthreadcore23:30:250
149950,0migration/028214-21kthreadcore23:10:240
149950,0migration/026038-21latency_hist23:30:070
149950,0migration/021362-21fschecks_time20:20:210
149950,0migration/020820-21kthreadcore21:45:250
149950,0migration/020358-21kthreadcore22:25:240
149950,0migration/019175-21aten_r1power_fr21:05:190
149950,0migration/018450-21taskset21:44:020
149950,0migration/011929-21fschecks_time21:20:210
149950,0migration/01188-21sh23:54:290
149950,0migration/010971-21kthreadcore20:10:250
149950,0migration/010529-21latency_hist22:20:080
8215040,0irq/24-0000:00:19473-21crond19:55:082
41909943,0cyclictest8294-21awk20:10:232
41909943,0cyclictest6358-21awk19:20:252
41909943,0cyclictest3556-21lldpd18:56:482
41909943,0cyclictest3514-21snmpd22:40:142
41909943,0cyclictest3514-21snmpd22:15:582
41909943,0cyclictest3514-21snmpd21:25:132
41909943,0cyclictest32005-21awk19:15:222
41909943,0cyclictest25327-21awk19:10:232
41899944,0cyclictest0-21swapper/120:00:221
41899944,0cyclictest0-21swapper/119:46:491
41899943,0cyclictest3961-21awk21:35:231
41899943,0cyclictest3622-21awk22:15:221
41899943,0cyclictest3514-21snmpd23:20:141
41899943,0cyclictest3514-21snmpd22:20:131
41899943,0cyclictest3514-21snmpd22:00:131
41899943,0cyclictest3514-21snmpd19:30:581
41899943,0cyclictest3514-21snmpd00:11:291
41899943,0cyclictest3399-21awk22:35:261
41899943,0cyclictest2937-21awk23:35:241
41899943,0cyclictest28158-21awk20:25:221
41899943,0cyclictest27586-21awk22:50:221
41899943,0cyclictest19385-21awk23:05:231
41899943,0cyclictest10521-21awk00:20:231
41879944,0cyclictest0-21swapper/000:05:230
41879943,0cyclictest3514-21snmpd23:40:140
41879943,0cyclictest3514-21snmpd00:22:140
41879943,0cyclictest3241-21irqbalance23:57:260
41879943,0cyclictest3241-21irqbalance23:05:460
41879943,0cyclictest3241-21irqbalance20:00:270
41879943,0cyclictest3241-21irqbalance19:55:370
41879943,0cyclictest26713-21awk00:10:220
41879943,0cyclictest19933-21awk22:05:240
41879943,0cyclictest18606-21awk19:05:220
41879943,0cyclictest11773-21awk21:00:250
339940,0migration/37875-21fschecks_count20:10:213
339940,0migration/3449-21diskmemload23:11:213
339940,0migration/328389-21aten_r1power_po20:50:203
339940,0migration/327545-21kthreadcore19:10:253
339940,0migration/327429-21aten_r1power_po20:25:193
339940,0migration/324327-21kthreadcore20:20:243
339940,0migration/322891-21kernelversion20:45:243
339940,0migration/318494-21ntp_kernel_pll_19:50:283
339940,0migration/315246-21kthreadcore21:40:253
339940,0migration/314774-21kthreadcore22:40:243
339940,0migration/31312-21kthreadcore19:40:253
29725040,0irq/27-eth0-tx-8732-21ssh22:35:392
29725040,0irq/27-eth0-tx-3153-21sed23:35:252
29725040,0irq/27-eth0-tx-25862-21ssh00:10:182
29725040,0irq/27-eth0-tx-14551-21kthreadcore23:20:252
29725040,0irq/27-eth0-tx-0-21swapper/222:20:202
29725040,0irq/27-eth0-tx-0-21swapper/222:05:142
29725040,0irq/27-eth0-tx-0-21swapper/200:20:192
29725040,0irq/27-eth0-tx-0-21swapper/020:45:200
29725040,0irq/27-eth0-tx-0-21swapper/019:00:220
29715040,0irq/26-eth0-rx-29383-21kthreadcore21:10:253
29715040,0irq/26-eth0-rx-0-21swapper/323:45:263
29715040,0irq/26-eth0-rx-0-21swapper/323:15:383
29715040,0irq/26-eth0-rx-0-21swapper/322:50:393
29715040,0irq/26-eth0-rx-0-21swapper/322:32:323
29715040,0irq/26-eth0-rx-0-21swapper/322:00:313
29715040,0irq/26-eth0-rx-0-21swapper/321:30:123
29715040,0irq/26-eth0-rx-0-21swapper/321:25:223
29715040,0irq/26-eth0-rx-0-21swapper/321:20:273
29715040,0irq/26-eth0-rx-0-21swapper/300:22:033
29715040,0irq/26-eth0-rx-0-21swapper/300:15:213
29715040,0irq/26-eth0-rx-0-21swapper/300:05:133
259940,0migration/28270-21kthreadcore21:15:262
259940,0migration/231224-21kthreadcore20:50:252
259940,0migration/229153-21kthreadcore20:25:242
259940,0migration/224180-21latency22:25:252
259940,0migration/223627-21kthreadcore22:45:242
259940,0migration/223627-21kthreadcore22:45:242
259940,0migration/220993-21munin-node20:20:202
259940,0migration/218892-21cpu19:30:202
259940,0migration/216360-21kthreadcore20:40:242
259940,0migration/21632-21aten_r1power_en20:30:192
259940,0migration/214177-21kthreadcore19:25:252
249940,0watchdog/226597-21grep23:50:262
249940,0watchdog/219858-21kthreadcore23:45:262
249940,0watchdog/211727-21cat23:40:222
249940,0watchdog/20-21swapper/220:35:222
249940,0watchdog/20-21swapper/219:00:272
179940,0migration/17863-21kthreadcore19:20:261
179940,0migration/16178-21kthreadcore00:15:261
179940,0migration/13675-21kthreadcore19:40:251
179940,0migration/131635-21taskset19:38:191
179940,0migration/127508-21aten_r1power_vo21:10:191
179940,0migration/122861-21kthreadcore21:45:261
179940,0migration/12272-21runrttasks21:33:511
179940,0migration/119686-21fschecks_time22:05:221
179940,0migration/115416-21kthreadcore21:20:251
179940,0migration/115154-21kthreadcore22:40:241
179940,0migration/113702-21kthreadcore00:00:241
179940,0migration/111735-21irqrtprio23:00:251
179940,0migration/110787-21runrttasks18:58:501
169940,0watchdog/18593-21ssh22:55:381
169940,0watchdog/131960-21needreboot23:10:261
169940,0watchdog/130889-21tr21:50:261
169940,0watchdog/129805-21needreboot19:10:271
169940,0watchdog/129094-21idleruntime20:50:221
169940,0watchdog/124489-21ssh23:46:141
169940,0watchdog/117976-21kthreadcore19:50:271
169940,0watchdog/10-21swapper/123:34:101
169940,0watchdog/10-21swapper/122:46:261
169940,0watchdog/10-21swapper/122:46:261
169940,0watchdog/10-21swapper/121:15:261
149940,0migration/09484-21runrttasks21:15:560
149940,0migration/0435-21timerandwakeup22:10:300
149940,0migration/03972-21kthreadcore00:15:250
149940,0migration/027256-21latency_hist21:30:080
149940,0migration/019781-21kthreadcore20:40:250
149940,0migration/016642-21kthreadcore20:15:260
149940,0migration/01521-21sh22:33:240
8215030,0irq/24-0000:00:24216-21idleruntime-cro19:10:082
41909932,0cyclictest31031-21sensors19:35:302
41899932,0cyclictest3514-21snmpd23:25:141
41899932,0cyclictest3514-21snmpd20:48:131
41899932,0cyclictest3514-21snmpd20:15:431
41899932,0cyclictest20892-21cpuspeed_turbos20:20:191
41899932,0cyclictest13990-21kthreadcore23:40:221
41899930,0cyclictest23785-21users19:05:301
41879933,0cyclictest29053-21kworker/0:021:55:190
41879933,0cyclictest29053-21kworker/0:021:50:290
41879933,0cyclictest29053-21kworker/0:021:35:270
41879933,0cyclictest29053-21kworker/0:021:30:290
41879933,0cyclictest25326-21kworker/0:223:22:390
41879933,0cyclictest25326-21kworker/0:222:40:090
41879933,0cyclictest19652-21kworker/0:220:30:540
41879933,0cyclictest19652-21kworker/0:219:40:160
41879933,0cyclictest19652-21kworker/0:219:35:160
41879932,0cyclictest3241-21irqbalance20:25:270
41879932,0cyclictest13910-21awk19:50:240
41879932,0cyclictest12835-21grep22:00:260
41879931,0cyclictest22733-21kthreadcore23:45:270
339930,0migration/326115-21taskset20:00:043
29715030,0irq/26-eth0-rx-0-21swapper/320:40:253
29715030,0irq/26-eth0-rx-0-21swapper/320:00:133
29715030,0irq/26-eth0-rx-0-21swapper/319:45:193
29715030,0irq/26-eth0-rx-0-21swapper/319:25:293
29715030,0irq/26-eth0-rx-0-21swapper/319:05:253
29715030,0irq/26-eth0-rx-0-21swapper/318:55:223
259930,0migration/227629-21irqstats20:00:252
249930,0watchdog/219681-21smartctl20:15:312
249930,0watchdog/20-21swapper/220:49:102
179930,0migration/16720-21runrttasks20:05:391
169930,0watchdog/10-21swapper/121:00:261
169930,0watchdog/10-21swapper/120:30:261
169930,0watchdog/10-21swapper/119:25:311
29715020,0irq/26-eth0-rx-0-21swapper/320:15:383
29715020,0irq/26-eth0-rx-0-21swapper/319:35:493
29715020,0irq/26-eth0-rx-0-21swapper/319:15:253
249920,0watchdog/20-21swapper/220:05:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional