You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 05:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Jun 06, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/318:48:423
643123422,0sleep00-21swapper/018:53:090
623023425,0sleep20-21swapper/218:50:512
615223324,0sleep10-21swapper/118:49:581
65339960,0cyclictest0-21swapper/223:03:502
65339960,0cyclictest0-21swapper/222:19:512
65319960,0cyclictest0-21swapper/120:59:141
4160,0ktimersoftd/00-21swapper/023:48:270
35160,0ktimersoftd/30-21swapper/323:48:273
339960,0migration/328010-21sh22:46:203
339960,0migration/327288-21sh00:06:343
339960,0migration/325617-21kthreadcore19:28:493
339960,0migration/319233-21sh22:59:503
339960,0migration/313075-21sh22:18:353
339960,0migration/312366-21sh21:36:053
339960,0migration/312174-21sh22:36:563
29715060,0irq/26-eth0-rx-28030-21sort19:33:443
29715060,0irq/26-eth0-rx-0-21swapper/321:00:433
19160,0ktimersoftd/10-21swapper/123:48:271
149960,0migration/028209-21sh22:46:540
8215050,0irq/24-0000:00:28439-21cat19:58:312
8215050,0irq/24-0000:00:2663-21perf20:03:312
8215050,0irq/24-0000:00:15088-21idleruntime-cro19:48:312
65339955,0cyclictest0-21swapper/221:13:572
65339955,0cyclictest0-21swapper/220:18:462
65339954,0cyclictest17175-21awk20:13:452
65339950,0cyclictest0-21swapper/222:38:462
65339950,0cyclictest0-21swapper/222:06:182
65339950,0cyclictest0-21swapper/220:57:432
65319955,0cyclictest0-21swapper/122:53:411
65319955,0cyclictest0-21swapper/122:20:351
65319955,0cyclictest0-21swapper/121:04:261
65319950,0cyclictest0-21swapper/121:29:211
65309955,0cyclictest0-21swapper/023:11:550
65309955,0cyclictest0-21swapper/023:01:020
65309950,0cyclictest0-21swapper/023:08:170
65309950,0cyclictest0-21swapper/021:05:090
4150,0ktimersoftd/00-21swapper/021:40:190
339950,0migration/38365-21kthreadcore23:53:473
339950,0migration/37720-21munin-node18:53:463
339950,0migration/37287-21kthreadcore21:13:473
339950,0migration/35764-21munin-node20:53:453
339950,0migration/34673-21hwlatdetect20:28:443
339950,0migration/33283-21taskset22:09:253
339950,0migration/332110-21munin-node19:08:513
339950,0migration/331843-21kthreadcore22:28:473
339950,0migration/330498-21latency_hist20:48:313
339950,0migration/329730-21kthreadcore23:48:463
339950,0migration/329678-21aten_r1power_fr20:23:423
339950,0migration/329385-21munin-node20:43:493
339950,0migration/321154-21munin-node23:23:423
339950,0migration/321089-21taskset21:22:403
339950,0migration/319415-21taskset18:58:583
339950,0migration/318211-21munin-node00:18:533
29725050,0irq/27-eth0-tx-8637-21munin-node19:43:500
29725050,0irq/27-eth0-tx-23895-21munin-node20:43:440
29725050,0irq/27-eth0-tx-16308-21munin-node20:13:490
29725050,0irq/27-eth0-tx-12200-21munin-node23:58:522
29725050,0irq/27-eth0-tx-0-21swapper/019:03:460
29715050,0irq/26-eth0-rx-23900-21cat22:23:483
29715050,0irq/26-eth0-rx-22792-21irqstats21:03:483
29715050,0irq/26-eth0-rx-20750-21ssh23:03:323
29715050,0irq/26-eth0-rx-1888-21kthreadcore21:08:483
29715050,0irq/26-eth0-rx-10636-21nfs_client22:13:513
29715050,0irq/26-eth0-rx-0-21swapper/323:58:323
29715050,0irq/26-eth0-rx-0-21swapper/323:15:103
29715050,0irq/26-eth0-rx-0-21swapper/323:09:083
29715050,0irq/26-eth0-rx-0-21swapper/322:50:443
29715050,0irq/26-eth0-rx-0-21swapper/321:58:463
29715050,0irq/26-eth0-rx-0-21swapper/321:53:473
29715050,0irq/26-eth0-rx-0-21swapper/321:48:103
29715050,0irq/26-eth0-rx-0-21swapper/321:28:543
29715050,0irq/26-eth0-rx-0-21swapper/321:25:293
29715050,0irq/26-eth0-rx-0-21swapper/319:43:443
29715050,0irq/26-eth0-rx-0-21swapper/300:09:513
27150,0ktimersoftd/20-21swapper/223:48:272
249950,0watchdog/27734-21aten_r1power_fr19:18:422
249950,0watchdog/216427-21kthreadcore21:58:492
179950,0migration/14799-21sh21:12:301
179950,0migration/14776-21cpuspeed_turbos23:13:431
179950,0migration/130075-21munin-node22:08:441
179950,0migration/122191-21munin-node22:03:451
179950,0migration/122027-21munin-node22:23:451
179950,0migration/121506-21munin-node23:23:441
179950,0migration/114967-21munin-node21:18:461
179950,0migration/113633-21kthreadcore23:38:471
179950,0migration/113593-21kthreadcore00:18:471
149950,0migration/0903-21kthreadcore21:08:480
149950,0migration/09027-21kthreadcore23:53:470
149950,0migration/08824-21irqcore19:18:460
149950,0migration/07502-21kthreadcore21:53:460
149950,0migration/06695-21kthreadcore20:53:470
149950,0migration/0641-21sa119:13:310
149950,0migration/04154-21latency_hist23:33:310
149950,0migration/04154-21latency_hist23:33:310
149950,0migration/03383-21kthreadcore19:13:480
149950,0migration/02998-21irqcore19:38:470
149950,0migration/027197-21munin-node21:43:500
149950,0migration/025726-21kthreadcore19:53:480
149950,0migration/025493-21kthreadcore00:03:470
149950,0migration/02334-21munin-node22:28:500
149950,0migration/02325-21sh23:49:280
149950,0migration/023152-21kthreadcore22:23:470
149950,0migration/021357-21aten_r1power_en22:03:410
149950,0migration/020990-21aten_r1power_en19:28:420
149950,0migration/019876-21sh22:00:100
149950,0migration/019751-21taskset00:23:190
149950,0migration/019168-21ntp_offset21:18:490
149950,0migration/01865-21kthreadcore21:48:490
149950,0migration/018636-21runrttasks23:18:580
149950,0migration/013793-21fschecks_time20:58:440
149950,0migration/013165-21aten_r1power_vo22:38:420
149950,0migration/012786-21kthreadcore20:08:470
149950,0migration/010409-21latency_hist20:33:310
8215040,0irq/24-0000:00:10572-21cat20:33:312
65339944,0cyclictest0-21swapper/222:54:042
65339943,0cyclictest6099-21awk21:53:442
65339943,0cyclictest5908-21awk22:13:462
65339943,0cyclictest3514-21snmpd20:30:012
65339943,0cyclictest3514-21snmpd19:39:012
65339943,0cyclictest3514-21snmpd19:38:312
65339943,0cyclictest3514-21snmpd18:55:162
65339943,0cyclictest3514-21snmpd00:10:012
65339943,0cyclictest3241-21irqbalance21:34:492
65339943,0cyclictest3241-21irqbalance21:31:092
65339943,0cyclictest14521-21awk21:18:442
65339943,0cyclictest14034-21awk20:58:452
65339943,0cyclictest13696-21awk22:58:462
65319944,0cyclictest0-21swapper/123:52:201
65319944,0cyclictest0-21swapper/119:33:471
65319943,0cyclictest6365-21awk21:33:461
65319943,0cyclictest5779-21awk22:33:451
65319943,0cyclictest5095-21awk23:33:451
65319943,0cyclictest5095-21awk23:33:451
65319943,0cyclictest3514-21snmpd23:11:011
65319943,0cyclictest3514-21snmpd22:59:011
65319943,0cyclictest3514-21snmpd22:44:011
65319943,0cyclictest3514-21snmpd21:49:011
65319943,0cyclictest3514-21snmpd20:45:011
65319943,0cyclictest3514-21snmpd20:43:011
65319943,0cyclictest3514-21snmpd19:18:011
65319943,0cyclictest3514-21snmpd19:12:011
65319943,0cyclictest3514-21snmpd00:07:161
65319943,0cyclictest3241-21irqbalance23:58:491
65319943,0cyclictest3241-21irqbalance22:39:391
65319943,0cyclictest30548-21awk20:23:461
65319943,0cyclictest30123-21awk22:28:451
65319943,0cyclictest29512-21awk23:28:451
65319943,0cyclictest17262-21awk20:13:461
65309944,0cyclictest0-21swapper/023:59:030
65309944,0cyclictest0-21swapper/023:31:090
65309944,0cyclictest0-21swapper/000:12:080
65309943,0cyclictest3824-21awk20:03:450
65309943,0cyclictest3241-21irqbalance23:23:590
65309943,0cyclictest3241-21irqbalance23:15:290
65309943,0cyclictest3241-21irqbalance22:19:290
65309943,0cyclictest3241-21irqbalance22:14:090
65309943,0cyclictest31546-21awk20:48:460
65309943,0cyclictest22562-21awk21:23:450
65309943,0cyclictest21724-21sadc19:53:310
65309943,0cyclictest14288-21awk18:58:460
65309943,0cyclictest11426-21awk20:33:450
65309940,0cyclictest0-21swapper/020:23:310
339940,0migration/33131-21cpuspeed_turbos20:03:433
339940,0migration/329463-21latency_hist21:48:313
339940,0migration/324444-21kthreadcore19:03:483
339940,0migration/324249-21kernelversion20:18:473
339940,0migration/319347-21kthreadcore20:38:493
339940,0migration/316056-21kthreadcore23:38:473
339940,0migration/315155-21if_err_eth119:23:443
29725040,0irq/27-eth0-tx-4670-21diskmemload23:54:052
29725040,0irq/27-eth0-tx-4670-21diskmemload21:03:462
29725040,0irq/27-eth0-tx-3210-21ssh22:09:142
29725040,0irq/27-eth0-tx-12297-21munin-node23:38:412
29725040,0irq/27-eth0-tx-11650-21ssh23:37:252
29725040,0irq/27-eth0-tx-11650-21ssh23:37:252
29725040,0irq/27-eth0-tx-0-21swapper/223:28:532
29725040,0irq/27-eth0-tx-0-21swapper/223:23:422
29725040,0irq/27-eth0-tx-0-21swapper/223:13:422
29725040,0irq/27-eth0-tx-0-21swapper/222:33:382
29725040,0irq/27-eth0-tx-0-21swapper/222:28:482
29725040,0irq/27-eth0-tx-0-21swapper/222:24:182
29725040,0irq/27-eth0-tx-0-21swapper/221:23:582
29725040,0irq/27-eth0-tx-0-21swapper/200:17:132
29725040,0irq/27-eth0-tx-0-21swapper/200:06:502
29715040,0irq/26-eth0-rx-0-21swapper/323:33:433
29715040,0irq/26-eth0-rx-0-21swapper/323:33:433
29715040,0irq/26-eth0-rx-0-21swapper/323:28:513
29715040,0irq/26-eth0-rx-0-21swapper/323:19:373
29715040,0irq/26-eth0-rx-0-21swapper/322:54:183
29715040,0irq/26-eth0-rx-0-21swapper/322:39:193
29715040,0irq/26-eth0-rx-0-21swapper/322:03:483
29715040,0irq/26-eth0-rx-0-21swapper/321:40:213
29715040,0irq/26-eth0-rx-0-21swapper/320:13:483
29715040,0irq/26-eth0-rx-0-21swapper/319:53:493
29715040,0irq/26-eth0-rx-0-21swapper/319:38:453
29715040,0irq/26-eth0-rx-0-21swapper/300:14:033
259940,0migration/2820-21kthreadcore23:08:472
259940,0migration/27279-21kthreadcore20:03:482
259940,0migration/232496-21kthreadcore21:48:482
259940,0migration/232109-21kthreadcore20:48:472
259940,0migration/230787-21kthreadcore23:48:462
259940,0migration/224702-21kthreadcore19:28:492
259940,0migration/22371-21sendmail_mailqu22:48:502
259940,0migration/223580-21kthreadcore19:03:482
259940,0migration/221004-21runrttasks22:43:372
259940,0migration/217483-21missed_timers00:18:482
259940,0migration/215147-21kthreadcore21:38:472
259940,0migration/212829-21df_abs23:18:412
259940,0migration/212417-21kthreadcore20:08:472
259940,0migration/21021-21kthreadcore20:23:482
249940,0watchdog/22616-21missed_timers21:08:492
249940,0watchdog/222452-21if_eth1.221:43:452
249940,0watchdog/20-21swapper/219:13:462
179940,0migration/17102-21kthreadcore20:03:481
179940,0migration/14225-21kthreadcore19:38:481
179940,0migration/13900-21latency_hist00:13:301
179940,0migration/131566-21kthreadcore19:58:481
179940,0migration/130946-21df_abs20:48:431
179940,0migration/130835-21kthreadcore00:08:471
179940,0migration/12873-21sh22:49:091
179940,0migration/128215-21taskset19:56:441
179940,0migration/124826-21kthreadcore19:28:491
179940,0migration/124403-21kthreadcore19:03:481
179940,0migration/117077-21kthreadcore19:23:461
179940,0migration/115884-21kthreadcore23:18:461
179940,0migration/114514-21irqcore21:38:461
179940,0migration/114131-21gltestperf18:58:451
169940,0watchdog/19717-21munin-node23:53:481
169940,0watchdog/15514-21munin-node21:13:531
169940,0watchdog/124327-21kthreadcore23:03:491
169940,0watchdog/120033-21ssh22:00:371
169940,0watchdog/113678-21kthreadcore19:43:491
169940,0watchdog/1130942sleep118:57:341
169940,0watchdog/112505-21ssh21:57:091
169940,0watchdog/10-21swapper/122:13:491
169940,0watchdog/10-21swapper/121:43:451
169940,0watchdog/10-21swapper/120:33:411
149940,0migration/032497-21kthreadcore21:28:480
149940,0migration/03068-21unixbench-2d20:23:540
149940,0migration/030297-21kthreadcore19:33:480
149940,0migration/02501-21ntp_states22:08:500
149940,0migration/011579-21runrttasks22:55:580
149940,0migration/010802-21kthreadcore18:53:490
65339932,0cyclictest3514-21snmpd19:03:312
65339932,0cyclictest16173-21awk19:48:452
65319932,0cyclictest3514-21snmpd21:26:011
65319932,0cyclictest23746-21hwlatdetect20:18:451
65319932,0cyclictest15315-21sensors20:08:521
65309933,0cyclictest5762-21awk22:33:450
65309933,0cyclictest3704-21kworker/0:222:49:000
65309933,0cyclictest3704-21kworker/0:221:34:000
65309933,0cyclictest29609-21kworker/0:023:38:420
65309933,0cyclictest10176-21kworker/0:019:23:430
65309932,0cyclictest8329-21grep00:13:470
65309932,0cyclictest6485-21awk21:13:450
65309932,0cyclictest29547-21awk19:58:460
65309930,0cyclictest17240-21munin-node20:38:450
339930,0migration/314278-21kthreadcore20:08:483
29715030,0irq/26-eth0-rx-0-21swapper/320:33:413
29715030,0irq/26-eth0-rx-0-21swapper/320:03:273
29715030,0irq/26-eth0-rx-0-21swapper/319:52:383
29715030,0irq/26-eth0-rx-0-21swapper/319:18:423
29715030,0irq/26-eth0-rx-0-21swapper/319:13:423
259930,0migration/226700-21aten_r1power_cu19:08:422
259930,0migration/219910-21kthreadcore20:38:492
249930,0watchdog/20-21swapper/220:43:452
249930,0watchdog/20-21swapper/219:23:462
179930,0migration/17755-21kthreadcore20:28:471
169930,0watchdog/111571-21ssh20:55:131
169930,0watchdog/10-21swapper/119:48:411
169930,0watchdog/10-21swapper/119:18:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional