You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 10:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Mar 16, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
231124026,0sleep30-21swapper/319:00:343
234423627,0sleep20-21swapper/219:00:572
218023524,0sleep00-21swapper/018:59:030
204123228,0sleep10-21swapper/118:57:281
339960,0migration/38878-21sh21:46:213
339960,0migration/37622-21sh22:46:133
339960,0migration/35733-21sh00:04:453
339960,0migration/35479-21sh23:21:543
339960,0migration/33250-21sshd19:24:373
339960,0migration/332209-21sh21:18:413
339960,0migration/331104-21sh21:56:563
339960,0migration/330122-21sh22:56:503
339960,0migration/322798-21sh22:53:453
339960,0migration/314017-21sh23:07:323
30445060,0irq/25-eth10-21swapper/023:26:400
29715060,0irq/26-eth0-rx-5777-21sh23:43:573
29715060,0irq/26-eth0-rx-28790-21munin-node23:36:353
29715060,0irq/26-eth0-rx-27265-21perl20:56:283
24419960,0cyclictest575-21diskmemload23:15:061
24419960,0cyclictest0-21swapper/121:08:201
179960,0migration/116800-21turbostat19:36:161
149960,0migration/031961-21sh21:59:260
149960,0migration/021578-21latency22:31:330
8215050,0irq/24-0000:00:32159-21cat22:41:152
8215050,0irq/24-0000:00:22527-21idleruntime-cro19:16:162
339950,0migration/39885-21if_eth021:06:293
339950,0migration/38909-21if_eth1.322:46:313
339950,0migration/38848-21kthreadcore20:41:333
339950,0migration/36261-21kthreadcore21:01:333
339950,0migration/329506-21timerwakeupswit19:41:383
339950,0migration/325642-21kthreadcore00:16:313
339950,0migration/323520-21taskset21:32:043
339950,0migration/323484-21latency_hist23:16:153
339950,0migration/318685-21kthreadcore23:51:343
339950,0migration/318497-21kthreadcore23:11:323
339950,0migration/312658-21runrttasks00:27:183
339950,0migration/311692-21kthreadcore22:06:323
339950,0migration/310763-21kthreadcore19:06:323
29725050,0irq/27-eth0-tx-9104-21kthreadcore00:26:332
29725050,0irq/27-eth0-tx-6559-21runrttasks21:28:402
29725050,0irq/27-eth0-tx-31311-21munin-node20:11:310
29725050,0irq/27-eth0-tx-16279-21iostat_ios23:31:312
29725050,0irq/27-eth0-tx-16244-21ssh22:11:072
29725050,0irq/27-eth0-tx-16097-21munin-node19:11:340
29725050,0irq/27-eth0-tx-15578-21munin-node23:11:272
29725050,0irq/27-eth0-tx-10356-21munin-node19:31:320
29725050,0irq/27-eth0-tx-0-21swapper/223:56:192
29725050,0irq/27-eth0-tx-0-21swapper/223:06:482
29725050,0irq/27-eth0-tx-0-21swapper/222:01:362
29725050,0irq/27-eth0-tx-0-21swapper/221:56:452
29725050,0irq/27-eth0-tx-0-21swapper/221:49:112
29725050,0irq/27-eth0-tx-0-21swapper/221:25:392
29725050,0irq/27-eth0-tx-0-21swapper/221:12:102
29725050,0irq/27-eth0-tx-0-21swapper/221:09:302
29715050,0irq/26-eth0-rx-8189-21kernelversion23:46:313
29715050,0irq/26-eth0-rx-7362-21latency_hist23:06:153
29715050,0irq/26-eth0-rx-32227-21ssh21:39:033
29715050,0irq/26-eth0-rx-30630-21grep19:46:283
29715050,0irq/26-eth0-rx-17395-21idleruntime22:11:303
29715050,0irq/26-eth0-rx-16815-21ssh21:30:493
29715050,0irq/26-eth0-rx-16755-21perl19:11:283
29715050,0irq/26-eth0-rx-13708-21ssh23:27:333
29715050,0irq/26-eth0-rx-12328-21python00:06:373
29715050,0irq/26-eth0-rx-0-21swapper/323:56:323
29715050,0irq/26-eth0-rx-0-21swapper/322:36:273
29715050,0irq/26-eth0-rx-0-21swapper/322:33:503
29715050,0irq/26-eth0-rx-0-21swapper/322:21:333
29715050,0irq/26-eth0-rx-0-21swapper/322:03:113
29715050,0irq/26-eth0-rx-0-21swapper/321:12:153
29715050,0irq/26-eth0-rx-0-21swapper/300:24:333
259950,0migration/28020-21aten_r1power_po22:46:272
259950,0migration/27882-21munin-node23:46:312
259950,0migration/25254-21unixbench-2d23:21:372
259950,0migration/229380-21missed_timers22:56:352
259950,0migration/229322-21latency_hist19:21:162
259950,0migration/223426-21sh21:52:462
259950,0migration/223099-21taskset22:12:422
259950,0migration/222391-21latency_hist00:16:152
259950,0migration/214059-21kthreadcore19:31:332
259950,0migration/21272-21irqcore22:21:312
24439955,0cyclictest0-21swapper/222:36:492
24439950,0cyclictest0-21swapper/222:30:592
24439950,0cyclictest0-21swapper/221:37:422
24439950,0cyclictest0-21swapper/221:31:522
24439950,0cyclictest0-21swapper/200:20:412
24419955,0cyclictest0-21swapper/122:05:341
24419954,0cyclictest9297-21tr22:26:291
24419954,0cyclictest10703-21ps23:06:321
24419950,0cyclictest0-21swapper/121:20:001
24419950,0cyclictest0-21swapper/120:21:401
24419950,0cyclictest0-21swapper/100:19:541
24409950,0cyclictest0-21swapper/000:19:290
179950,0migration/19698-21kthreadcore20:41:331
179950,0migration/18468-21kthreadcore20:16:341
179950,0migration/131311-21fschecks_count00:01:271
179950,0migration/128482-21timerandwakeup19:16:381
179950,0migration/124676-21taskset21:35:261
179950,0migration/121803-21kthreadcore21:11:321
179950,0migration/121490-21kthreadcore20:51:301
179950,0migration/121093-21kthreadcore19:36:331
179950,0migration/118624-21kthreadcore21:51:331
179950,0migration/114560-21pmu-power21:46:381
179950,0migration/113750-21kthreadcore21:26:331
179950,0migration/113069-21sh00:28:321
169950,0watchdog/17700-21sort23:46:291
169950,0watchdog/126686-21aten2.4-expect20:56:261
169950,0watchdog/121724-21kthreadcore20:01:341
149950,0migration/09417-21latency_hist19:06:160
149950,0migration/08226-21sh21:24:040
149950,0migration/06939-21aten_r1power_en23:46:260
149950,0migration/0692-21fschecks_time22:21:290
149950,0migration/03810-21kthreadcore23:01:330
149950,0migration/031006-21fschecks_time19:46:290
149950,0migration/030316-21sh22:57:250
149950,0migration/027256-21fschecks_time20:56:280
149950,0migration/027058-21kthreadcore21:16:330
149950,0migration/026125-21munin-node21:36:300
149950,0migration/025406-21kthreadcore19:16:330
149950,0migration/023357-21kthreadcore20:26:330
149950,0migration/020992-21sendmail_mailqu23:31:360
149950,0migration/018710-21kthreadcore22:51:320
149950,0migration/018596-21fschecks_time20:01:300
149950,0migration/016957-21latency_hist19:36:160
149950,0migration/015370-21latency_hist23:11:150
149950,0migration/014335-21kthreadcore20:21:340
149950,0migration/014141-21taskset22:46:530
149950,0migration/01021-21irqcore20:36:320
339940,0migration/37450-21kthreadcore19:51:343
339940,0migration/3716-21kthreadcore20:11:333
339940,0migration/327875-21kthreadcore20:31:313
339940,0migration/32341-21kthreadcore21:41:323
339940,0migration/322496-21kthreadcore20:51:303
339940,0migration/319136-21timerwakeupswit20:46:363
29725040,0irq/27-eth0-tx-575-21diskmemload23:51:572
29725040,0irq/27-eth0-tx-17370-21cut22:51:312
29725040,0irq/27-eth0-tx-0-21swapper/223:41:182
29725040,0irq/27-eth0-tx-0-21swapper/223:16:322
29725040,0irq/27-eth0-tx-0-21swapper/222:16:322
29725040,0irq/27-eth0-tx-0-21swapper/221:41:252
29725040,0irq/27-eth0-tx-0-21swapper/221:16:342
29725040,0irq/27-eth0-tx-0-21swapper/200:21:202
29725040,0irq/27-eth0-tx-0-21swapper/200:06:162
29725040,0irq/27-eth0-tx-0-21swapper/021:02:190
29725040,0irq/27-eth0-tx-0-21swapper/020:45:270
29725040,0irq/27-eth0-tx-0-21swapper/019:41:340
29715040,0irq/26-eth0-rx-20350-21munin-node00:11:373
29715040,0irq/26-eth0-rx-15833-21grep23:31:283
29715040,0irq/26-eth0-rx-0-21swapper/322:26:233
29715040,0irq/26-eth0-rx-0-21swapper/322:16:393
29715040,0irq/26-eth0-rx-0-21swapper/321:51:163
29715040,0irq/26-eth0-rx-0-21swapper/321:21:303
29715040,0irq/26-eth0-rx-0-21swapper/320:26:283
29715040,0irq/26-eth0-rx-0-21swapper/319:56:273
259940,0migration/29511-21timerandwakeup19:26:392
259940,0migration/23342-21fschecks_time19:01:302
259940,0migration/230851-21aten_r1power_cu00:01:252
259940,0migration/230555-21cpu19:46:282
259940,0migration/224682-21munin-node20:06:272
259940,0migration/223475-21ntp_kernel_pll_20:01:362
259940,0migration/211199-21kthreadcore23:26:342
24439943,0cyclictest836-21awk20:36:322
24439943,0cyclictest5610-21awk19:51:322
24439943,0cyclictest3514-21snmpd20:16:392
24439943,0cyclictest3514-21snmpd19:42:392
24439943,0cyclictest3514-21snmpd19:06:542
24439943,0cyclictest3241-21irqbalance23:36:242
24439943,0cyclictest32298-21awk20:11:322
24439943,0cyclictest19966-21awk20:26:312
24439943,0cyclictest17111-21awk22:31:292
24419944,0cyclictest0-21swapper/123:35:451
24419943,0cyclictest3556-21lldpd23:26:211
24419943,0cyclictest3556-21lldpd22:36:221
24419943,0cyclictest3556-21lldpd22:14:511
24419943,0cyclictest3514-21snmpd19:10:541
24419943,0cyclictest3241-21irqbalance22:44:341
24419943,0cyclictest1775-21awk21:01:301
24419943,0cyclictest1732-21awk21:41:301
24409944,0cyclictest20384-21kworker/0:021:41:370
24409944,0cyclictest20384-21kworker/0:020:16:330
24409944,0cyclictest0-21swapper/023:53:280
24409943,0cyclictest3556-21lldpd23:11:210
24409943,0cyclictest3556-21lldpd21:49:220
24409943,0cyclictest3556-21lldpd21:09:520
24409943,0cyclictest3514-21snmpd23:20:390
24409943,0cyclictest3514-21snmpd20:52:540
24409943,0cyclictest3514-21snmpd19:55:540
24409943,0cyclictest3514-21snmpd19:01:240
24409943,0cyclictest3241-21irqbalance22:16:340
24409943,0cyclictest25115-21awk22:36:280
24409943,0cyclictest10124-21sadc19:31:160
24409940,0cyclictest0-21swapper/021:28:530
249940,0watchdog/232579-21grep20:56:382
249940,0watchdog/231157-21pmu-power20:31:342
249940,0watchdog/220356-21munin-node20:51:262
249940,0watchdog/20-21swapper/223:01:342
249940,0watchdog/20-21swapper/221:01:382
249940,0watchdog/20-21swapper/219:16:342
179940,0migration/19916-21kthreadcore00:06:321
179940,0migration/19489-21kthreadcore19:51:351
179940,0migration/17946-21aten_r1power_cu22:46:261
179940,0migration/15605-21kthreadcore19:26:331
179940,0migration/14501-21latency_hist19:51:161
179940,0migration/13615-21kthreadcore21:21:321
179940,0migration/1343-21kthreadcore20:11:331
179940,0migration/132641-21kthreadcore19:21:331
179940,0migration/132590-21fschecks_time23:01:291
179940,0migration/132105-21kthreadcore00:21:311
179940,0migration/130044-21taskset22:56:401
179940,0migration/124919-21kernelversion23:16:311
179940,0migration/117800-21kthreadcore19:11:311
179940,0migration/11730-21kthreadcore23:21:311
179940,0migration/110479-21kthreadcore22:06:311
169940,0watchdog/1985-21if_err_eth022:21:301
169940,0watchdog/16158-21ssh23:45:011
169940,0watchdog/132082-21ssh21:38:381
169940,0watchdog/130076-21rm23:38:451
169940,0watchdog/129058-21ssh23:56:491
169940,0watchdog/122535-21ssh22:52:581
169940,0watchdog/10-21swapper/120:31:341
149940,0migration/09686-21irqcore22:06:310
149940,0migration/0445-21runrttasks23:21:310
149940,0migration/027574-21kthreadcore20:31:310
149940,0migration/026719-21kthreadcore20:06:330
149940,0migration/022269-21kthreadcore21:11:330
149940,0migration/019690-21kthreadcore00:11:330
149940,0migration/018987-21kthreadcore21:31:320
149940,0migration/016490-21kthreadcore20:46:310
149940,0migration/011831-21fschecks_count19:56:290
29715030,0irq/26-eth0-rx-0-21swapper/320:40:163
29715030,0irq/26-eth0-rx-0-21swapper/320:10:223
29715030,0irq/26-eth0-rx-0-21swapper/320:01:283
29715030,0irq/26-eth0-rx-0-21swapper/319:36:273
29715030,0irq/26-eth0-rx-0-21swapper/319:31:273
29715030,0irq/26-eth0-rx-0-21swapper/319:28:083
29715030,0irq/26-eth0-rx-0-21swapper/319:16:443
29715030,0irq/26-eth0-rx-0-21swapper/319:03:323
259930,0migration/213901-21kthreadcore19:56:322
24439932,0cyclictest7544-21awk20:41:312
24439932,0cyclictest18767-21grep19:36:322
24439932,0cyclictest14162-21awk20:46:282
24419933,0cyclictest0-21swapper/119:42:341
24419932,0cyclictest3556-21lldpd20:06:221
24419932,0cyclictest3241-21irqbalance22:31:441
24419932,0cyclictest25762-21if_err_eth1.221:56:301
24419932,0cyclictest19918-21awk20:26:311
24419932,0cyclictest12268-21awk19:56:301
24419930,0cyclictest961-21cut20:36:321
24419930,0cyclictest7366-21kthreadcore19:01:341
24409933,0cyclictest23789-21awk23:56:300
24409932,0cyclictest9118-21awk22:26:280
24409932,0cyclictest696-21awk22:41:310
24409932,0cyclictest3556-21lldpd22:11:210
24409932,0cyclictest3514-21snmpd23:44:540
24409932,0cyclictest31771-21if_eth000:01:290
24409932,0cyclictest30220-21awk19:21:310
24409932,0cyclictest12243-21cat00:26:380
24409930,0cyclictest445-21grep00:21:310
24409930,0cyclictest16902-21runrttasks21:51:220
24409930,0cyclictest1152-21perl22:01:280
249930,0watchdog/20-21swapper/220:21:302
169930,0watchdog/1575-21diskmemload23:51:211
169930,0watchdog/124594-21munin-node22:16:341
169930,0watchdog/10-21swapper/100:12:491
149930,0migration/023231-21sh23:36:230
29715020,0irq/26-eth0-rx-0-21swapper/320:21:223
29715020,0irq/26-eth0-rx-0-21swapper/320:16:443
24419921,0cyclictest0-21swapper/120:46:161
24409920,0cyclictest0-21swapper/000:06:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional