You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 06:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 14, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2696324435,0sleep20-21swapper/218:55:292
2689223627,0sleep30-21swapper/318:54:333
2088523524,0sleep00-21swapper/018:53:380
2707123423,0sleep10-21swapper/118:56:401
339970,0migration/322640-21sh00:15:463
339960,0migration/39556-21sh21:27:393
339960,0migration/38884-21sh21:07:123
339960,0migration/3689-21sh22:02:243
339960,0migration/331838-21munin-node21:18:543
339960,0migration/325294-21sh21:16:573
339960,0migration/316735-21idleruntime-cro22:53:323
339960,0migration/316735-21idleruntime-cro22:53:323
339960,0migration/315722-21sh23:11:353
339960,0migration/315546-21sh23:52:373
29725060,0irq/27-eth0-tx-15499-21latency21:28:512
29725060,0irq/27-eth0-tx-15186-21munin-node19:13:520
29715060,0irq/26-eth0-rx-6366-21kthreadcore22:03:503
29715060,0irq/26-eth0-rx-31911-21ssh22:40:373
29715060,0irq/26-eth0-rx-31859-21chrt00:23:373
29715060,0irq/26-eth0-rx-30626-21ps22:58:533
29715060,0irq/26-eth0-rx-30020-21kthreadcore20:58:503
273079966,0cyclictest25434-21diskmemload23:25:212
273079965,0cyclictest21314-21tail21:33:492
273079965,0cyclictest17787-21crond20:28:322
273059965,0cyclictest8176-21sh23:07:521
273049966,0cyclictest25434-21diskmemload21:33:490
8215050,0irq/24-0000:00:12101-21latency_hist20:48:322
4150,0ktimersoftd/00-21swapper/021:41:310
339950,0migration/39612-21aten2.4-expect19:33:443
339950,0migration/38082-21munin-node00:08:443
339950,0migration/37662-21kthreadcore20:43:493
339950,0migration/36529-21kthreadcore19:03:513
339950,0migration/33540-21latency_hist19:53:323
339950,0migration/332064-21gltestperf20:38:463
339950,0migration/329092-21missed_timers20:08:523
339950,0migration/327626-21munin-node18:58:453
339950,0migration/327591-21kthreadcore23:18:493
339950,0migration/326844-21kthreadcore20:33:503
339950,0migration/325641-21kthreadcore19:43:493
339950,0migration/325531-21aten_r1power_cu22:18:433
339950,0migration/324733-21kthreadcore19:18:513
339950,0migration/319213-21kthreadcore21:53:483
339950,0migration/318893-21kthreadcore19:13:493
339950,0migration/318242-21munin-node20:28:443
339950,0migration/317304-21latency_hist22:13:323
339950,0migration/312411-21kthreadcore21:48:503
339950,0migration/312392-21kthreadcore21:08:503
339950,0migration/312133-21kthreadcore22:28:483
339950,0migration/311922-21kthreadcore19:08:483
29725050,0irq/27-eth0-tx-31354-21munin-node20:38:460
29725050,0irq/27-eth0-tx-23612-21python21:53:542
29725050,0irq/27-eth0-tx-13804-21kthreadcore22:48:502
29725050,0irq/27-eth0-tx-13804-21kthreadcore22:48:502
29725050,0irq/27-eth0-tx-10278-21diskstats21:48:462
29725050,0irq/27-eth0-tx-0-21swapper/223:50:392
29715050,0irq/26-eth0-rx-7513-21ssh23:05:583
29715050,0irq/26-eth0-rx-6559-21runrttasks21:41:543
29715050,0irq/26-eth0-rx-24085-21cpu00:18:443
29715050,0irq/26-eth0-rx-23708-21awk21:33:513
29715050,0irq/26-eth0-rx-18666-21irqrtprio22:13:483
29715050,0irq/26-eth0-rx-17216-21ssh21:31:343
29715050,0irq/26-eth0-rx-0-21swapper/323:53:553
29715050,0irq/26-eth0-rx-0-21swapper/323:23:493
29715050,0irq/26-eth0-rx-0-21swapper/323:15:013
29715050,0irq/26-eth0-rx-0-21swapper/322:43:453
29715050,0irq/26-eth0-rx-0-21swapper/322:25:423
29715050,0irq/26-eth0-rx-0-21swapper/321:43:443
29715050,0irq/26-eth0-rx-0-21swapper/300:03:443
273079955,0cyclictest0-21swapper/223:53:442
273079950,0cyclictest0-21swapper/222:46:152
273079950,0cyclictest0-21swapper/222:27:012
273079950,0cyclictest0-21swapper/222:02:442
273079950,0cyclictest0-21swapper/221:13:522
273059955,0cyclictest0-21swapper/120:23:551
273059950,0cyclictest25434-21diskmemload21:24:181
273059950,0cyclictest0-21swapper/123:51:331
273059950,0cyclictest0-21swapper/123:20:291
273059950,0cyclictest0-21swapper/121:38:571
273049955,0cyclictest0-21swapper/022:48:420
273049955,0cyclictest0-21swapper/022:48:420
273049950,0cyclictest0-21swapper/023:57:460
273049950,0cyclictest0-21swapper/022:13:240
27150,0ktimersoftd/20-21swapper/200:09:162
259950,0migration/226200-21aten_r1power_cu21:38:432
259950,0migration/225954-21sh22:38:462
259950,0migration/225560-21kthreadcore00:18:492
249950,0watchdog/21287-21expr23:03:462
19150,0ktimersoftd/10-21swapper/100:09:161
179950,0migration/13579-21kthreadcore22:23:511
179950,0migration/129498-21munin-node00:18:531
179950,0migration/125612-21aten_r1power_po22:18:441
179950,0migration/124727-21cpuspeed_turbos23:38:441
179950,0migration/116196-21cpuspeed_turbos00:13:431
179950,0migration/114172-21switchtime23:28:551
179950,0migration/113175-21kthreadcore19:58:471
179950,0migration/111369-21switchtime20:43:551
149950,0migration/09252-21idleruntime-cro22:08:320
149950,0migration/08211-21taskset21:44:330
149950,0migration/03698-21munin-node19:28:480
149950,0migration/03509-21munin-node20:13:530
149950,0migration/031908-21sh23:42:530
149950,0migration/031067-21ntp_offset22:18:520
149950,0migration/029115-21kthreadcore21:58:500
149950,0migration/028788-21munin-node18:58:490
149950,0migration/021326-21sh22:13:490
149950,0migration/017496-21irqcore23:33:500
149950,0migration/015564-21sh23:11:080
149950,0migration/013678-21kthreadcore21:48:500
339940,0migration/38701-21kthreadcore19:53:503
339940,0migration/33283-21latency20:13:503
339940,0migration/332687-21cpuspeed_turbos23:43:433
339940,0migration/330339-21diskmemload23:59:013
339940,0migration/323672-21diskmemload22:35:033
339940,0migration/323133-21diskmemload23:35:413
29725040,0irq/27-eth0-tx-9502-21ssh22:08:412
29725040,0irq/27-eth0-tx-7761-21ssh22:04:172
29725040,0irq/27-eth0-tx-2811-21ssh21:23:462
29725040,0irq/27-eth0-tx-24552-21ssh23:58:442
29725040,0irq/27-eth0-tx-18024-21munin-node20:28:460
29725040,0irq/27-eth0-tx-14020-21latency23:08:502
29725040,0irq/27-eth0-tx-0-21swapper/223:13:422
29725040,0irq/27-eth0-tx-0-21swapper/222:58:382
29725040,0irq/27-eth0-tx-0-21swapper/222:34:162
29725040,0irq/27-eth0-tx-0-21swapper/222:28:362
29725040,0irq/27-eth0-tx-0-21swapper/222:13:582
29725040,0irq/27-eth0-tx-0-21swapper/221:08:342
29725040,0irq/27-eth0-tx-0-21swapper/200:23:492
29725040,0irq/27-eth0-tx-0-21swapper/200:14:222
29725040,0irq/27-eth0-tx-0-21swapper/020:48:510
29725040,0irq/27-eth0-tx-0-21swapper/020:33:450
29725040,0irq/27-eth0-tx-0-21swapper/020:13:170
29725040,0irq/27-eth0-tx-0-21swapper/019:21:300
29715040,0irq/26-eth0-rx-0-21swapper/323:38:363
29715040,0irq/26-eth0-rx-0-21swapper/323:30:353
29715040,0irq/26-eth0-rx-0-21swapper/322:53:383
273079944,0cyclictest0-21swapper/220:12:322
273079944,0cyclictest0-21swapper/219:04:352
273079943,0cyclictest976-21awk23:43:462
273079943,0cyclictest9207-21awk23:28:472
273079943,0cyclictest557-21awk00:03:462
273079943,0cyclictest3556-21lldpd21:18:472
273079943,0cyclictest3556-21lldpd20:48:472
273079943,0cyclictest3556-21lldpd19:48:462
273079943,0cyclictest3556-21lldpd19:36:462
273079943,0cyclictest3556-21lldpd19:16:462
273079943,0cyclictest3556-21lldpd19:01:162
273079943,0cyclictest29377-21awk19:23:482
273079943,0cyclictest25668-21awk23:18:472
273079943,0cyclictest23674-21awk19:43:472
273079943,0cyclictest18979-21awk20:28:472
273059944,0cyclictest0-21swapper/122:56:031
273059944,0cyclictest0-21swapper/121:32:111
273059943,0cyclictest9918-21awk22:48:481
273059943,0cyclictest9918-21awk22:48:481
273059943,0cyclictest9807-21ps21:48:321
273059943,0cyclictest383-21awk00:23:471
273059943,0cyclictest3514-21snmpd23:15:551
273059943,0cyclictest3514-21snmpd22:03:401
273059943,0cyclictest3241-21irqbalance22:58:501
273059943,0cyclictest3241-21irqbalance21:05:411
273059943,0cyclictest3241-21irqbalance20:28:511
273059943,0cyclictest30240-21awk19:48:471
273059943,0cyclictest2653-21awk21:43:461
273059943,0cyclictest23635-21awk19:43:471
273059943,0cyclictest18796-21awk21:53:471
273059943,0cyclictest1764-21awk22:43:461
273059943,0cyclictest15884-21sadc19:38:321
273059943,0cyclictest10365-21awk22:08:461
273049944,0cyclictest0-21swapper/022:46:300
273049944,0cyclictest0-21swapper/000:10:410
273049943,0cyclictest7837-21sadc23:48:310
273049943,0cyclictest5436-21grep20:18:480
273049943,0cyclictest3514-21snmpd23:19:550
273049943,0cyclictest3514-21snmpd23:07:400
273049943,0cyclictest3514-21snmpd22:59:100
273049943,0cyclictest3241-21irqbalance23:30:000
273049943,0cyclictest3241-21irqbalance22:28:000
273049943,0cyclictest3241-21irqbalance21:04:210
273049943,0cyclictest3241-21irqbalance20:03:410
273049943,0cyclictest3241-21irqbalance19:58:410
273049943,0cyclictest3241-21irqbalance19:53:510
273049943,0cyclictest3241-21irqbalance19:48:410
273049943,0cyclictest3241-21irqbalance19:42:410
273049943,0cyclictest3241-21irqbalance19:09:010
273049943,0cyclictest3241-21irqbalance19:03:410
273049943,0cyclictest3241-21irqbalance00:24:100
273049943,0cyclictest3241-21irqbalance00:19:000
273049943,0cyclictest29425-21awk19:23:480
273049943,0cyclictest18097-21awk22:33:470
273049943,0cyclictest16827-21awk00:13:460
273049943,0cyclictest1115-21awk23:23:470
273049943,0cyclictest10052-21awk22:28:460
259940,0migration/25225-21runrttasks20:41:212
259940,0migration/23146-21kthreadcore21:43:482
259940,0migration/228672-21kthreadcore20:58:492
259940,0migration/224905-21fschecks_count23:38:452
259940,0migration/224263-21kthreadcore19:18:512
259940,0migration/219993-21if_eth1.320:53:472
259940,0migration/218716-21kthreadcore22:53:492
259940,0migration/210908-21runrttasks20:20:392
249940,0watchdog/29535-21iostat_ios19:08:472
249940,0watchdog/23793-21kthreadcore21:03:502
249940,0watchdog/20-21swapper/223:33:462
249940,0watchdog/20-21swapper/222:18:502
249940,0watchdog/20-21swapper/220:33:472
249940,0watchdog/20-21swapper/220:03:512
249940,0watchdog/20-21swapper/219:53:472
179940,0migration/19717-21irqrtprio19:08:471
179940,0migration/132033-21kthreadcore19:23:501
179940,0migration/132007-21kthreadcore18:58:511
179940,0migration/13169-21kthreadcore20:13:501
179940,0migration/130344-21kthreadcore21:18:511
179940,0migration/129338-21kthreadcore20:33:511
179940,0migration/128751-21kthreadcore20:58:491
179940,0migration/128610-21kthreadcore21:58:501
179940,0migration/125053-21latency_hist22:38:321
179940,0migration/121920-21munin-node23:53:531
179940,0migration/121328-21kthreadcore21:13:501
179940,0migration/118583-21kthreadcore19:13:491
179940,0migration/118230-21unixbench-2d20:48:561
179940,0migration/117501-21fschecks_count20:03:461
169940,0watchdog/125610-21ssh21:37:421
169940,0watchdog/122865-21memory22:13:501
169940,0watchdog/117425-21irqcore23:33:501
169940,0watchdog/115029-21timerandwakeup22:28:541
169940,0watchdog/10-21swapper/120:18:511
169940,0watchdog/10-21swapper/119:53:511
169940,0watchdog/10-21swapper/119:18:471
169940,0watchdog/10-21swapper/119:03:511
169940,0watchdog/10-21swapper/100:03:541
149940,0migration/0444-21gltestperf00:03:450
149940,0migration/031183-21unixbench-2d22:38:550
149940,0migration/026351-21kthreadcore23:58:480
149940,0migration/024381-21kthreadcore19:43:490
149940,0migration/022341-21latency22:53:500
149940,0migration/020451-21kthreadcore21:53:480
149940,0migration/020426-21kthreadcore23:13:490
149940,0migration/014838-21needreboot19:33:510
149940,0migration/013688-21ntp_states23:48:510
149940,0migration/011556-21kthreadcore21:08:490
8215030,0irq/24-0000:00:0-21swapper/219:28:332
339930,0migration/311343-21latency_hist20:23:323
29715030,0irq/26-eth0-rx-0-21swapper/320:53:523
29715030,0irq/26-eth0-rx-0-21swapper/320:48:563
29715030,0irq/26-eth0-rx-0-21swapper/320:03:453
29715030,0irq/26-eth0-rx-0-21swapper/319:58:473
29715030,0irq/26-eth0-rx-0-21swapper/319:38:443
29715030,0irq/26-eth0-rx-0-21swapper/319:28:443
29715030,0irq/26-eth0-rx-0-21swapper/319:23:443
273079933,0cyclictest3556-21lldpd19:40:452
273079932,0cyclictest16067-21sensors19:58:532
273059933,0cyclictest0-21swapper/122:33:591
273059932,0cyclictest3241-21irqbalance23:08:401
273059932,0cyclictest19950-21awk20:53:471
273059932,0cyclictest13518-21kthreadcore21:08:501
273049932,0cyclictest3514-21snmpd20:46:400
273049932,0cyclictest31708-21needreboot21:18:530
273049932,0cyclictest25674-21munin-node20:58:470
273049932,0cyclictest23946-21timerwakeupswit21:13:560
273049932,0cyclictest19327-21tr20:53:440
273049932,0cyclictest11010-21awk21:28:470
273049930,0cyclictest0-21swapper/020:23:450
27130,0ktimersoftd/20-21swapper/220:14:162
179930,0migration/11577-21kthreadcore20:38:501
169930,0watchdog/10-21swapper/123:23:541
169930,0watchdog/10-21swapper/120:08:511
339920,0migration/35102-21diskstats20:18:473
273059921,0cyclictest2670-21latency_hist19:28:331
273059920,0cyclictest0-21swapper/123:58:321
273059920,0cyclictest0-21swapper/123:43:321
273059920,0cyclictest0-21swapper/119:38:331
273049921,0cyclictest1694-21latency_hist21:23:320
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional