You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 16:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 02, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/306:54:463
191460,0ktimersoftd/125960-21perf06:59:281
3226823624,0sleep00-21swapper/006:55:360
3244123526,0sleep20-21swapper/206:57:392
339960,0migration/39220-21hwlatdetect11:49:503
339960,0migration/39090-21munin-node11:29:493
339960,0migration/38105-21aten_r1power_vo12:09:473
339960,0migration/331441-21sh12:02:543
339960,0migration/331346-21runrttasks11:41:183
339960,0migration/323358-21munin-node10:54:563
339960,0migration/31282-21sh09:40:303
339960,0migration/310292-21sh10:09:233
326839966,0cyclictest27111-21diskmemload10:04:291
326839966,0cyclictest0-21swapper/110:27:301
326839965,0cyclictest0-21swapper/109:49:491
326839960,0cyclictest0-21swapper/111:48:131
326839960,0cyclictest0-21swapper/109:32:281
29715060,0irq/26-eth0-rx-9277-21yum09:05:013
29715060,0irq/26-eth0-rx-6296-21processes08:39:573
29715060,0irq/26-eth0-rx-31988-21cut07:44:493
29715060,0irq/26-eth0-rx-25472-21users09:34:593
29715060,0irq/26-eth0-rx-2538-21irqrtprio11:04:513
29715060,0irq/26-eth0-rx-1516-21perl11:24:503
149960,0migration/07450-21sh12:08:540
149960,0migration/025589-21sh09:56:090
149960,0migration/017059-21sh09:49:590
339950,0migration/36232-21sh12:05:193
339950,0migration/3575-21latency_hist11:24:363
339950,0migration/33848-21kthreadcore10:24:533
339950,0migration/330693-21kthreadcore09:59:543
339950,0migration/329476-21taskset07:38:343
339950,0migration/327647-21munin-node08:34:533
339950,0migration/326090-21fschecks_time08:09:503
339950,0migration/323666-21kthreadcore09:54:543
339950,0migration/323495-21taskset11:57:523
339950,0migration/320542-21idleruntime08:29:513
339950,0migration/320392-21kthreadcore11:14:543
339950,0migration/317672-21kthreadcore07:24:493
339950,0migration/317195-21df_abs11:34:493
339950,0migration/315413-21taskset07:22:093
339950,0migration/314071-21kthreadcore09:09:543
339950,0migration/313081-21switchtime08:44:583
339950,0migration/312058-21kthreadcore11:09:533
339950,0migration/311348-21aten_r1power_cu09:29:473
326839955,0cyclictest0-21swapper/111:18:061
326839955,0cyclictest0-21swapper/110:30:111
326839955,0cyclictest0-21swapper/110:18:401
326839950,0cyclictest0-21swapper/112:16:551
326839950,0cyclictest0-21swapper/112:02:431
326839950,0cyclictest0-21swapper/111:52:471
326839950,0cyclictest0-21swapper/111:43:391
326839950,0cyclictest0-21swapper/109:48:141
326839950,0cyclictest0-21swapper/109:00:191
326829955,0cyclictest0-21swapper/012:22:030
326829955,0cyclictest0-21swapper/010:21:460
326829950,0cyclictest0-21swapper/012:02:490
326829950,0cyclictest0-21swapper/008:44:480
29725050,0irq/27-eth0-tx-9301-21ssh09:45:202
29725050,0irq/27-eth0-tx-8902-21ssh09:24:562
29725050,0irq/27-eth0-tx-570-21gltestperf12:04:472
29725050,0irq/27-eth0-tx-4030-21kthreadcore10:04:542
29725050,0irq/27-eth0-tx-32311-21df_inode12:24:492
29725050,0irq/27-eth0-tx-30771-21kthreadcore10:59:532
29725050,0irq/27-eth0-tx-27356-21munin-node09:39:532
29725050,0irq/27-eth0-tx-27111-21diskmemload10:27:402
29725050,0irq/27-eth0-tx-16608-21ssh10:30:412
29725050,0irq/27-eth0-tx-14017-21munin-node08:49:500
29725050,0irq/27-eth0-tx-0-21swapper/211:44:332
29725050,0irq/27-eth0-tx-0-21swapper/007:37:080
29725050,0irq/27-eth0-tx-0-21swapper/007:17:040
29715050,0irq/26-eth0-rx-7391-21taskset11:47:253
29715050,0irq/26-eth0-rx-4536-21cut09:24:533
29715050,0irq/26-eth0-rx-29736-21python12:19:583
29715050,0irq/26-eth0-rx-23996-21kthreadcore09:14:553
29715050,0irq/26-eth0-rx-19616-21munin-node10:14:523
29715050,0irq/26-eth0-rx-18826-21kthreadcore12:14:523
29715050,0irq/26-eth0-rx-11952-21irqstats10:09:523
29715050,0irq/26-eth0-rx-11952-21irqstats10:09:523
29715050,0irq/26-eth0-rx-11189-21if_eth1.210:29:513
29715050,0irq/26-eth0-rx-0-21swapper/310:49:503
29715050,0irq/26-eth0-rx-0-21swapper/310:44:503
29715050,0irq/26-eth0-rx-0-21swapper/310:39:483
29715050,0irq/26-eth0-rx-0-21swapper/310:34:593
259950,0migration/2543-21latency_hist08:39:382
259950,0migration/24830-21kthreadcore10:44:542
259950,0migration/23870-21kthreadcore11:04:522
259950,0migration/223105-21latency_hist07:34:382
259950,0migration/219505-21aten_r1power_vo09:34:482
259950,0migration/218687-21sh09:13:232
259950,0migration/212360-21runrttasks07:59:502
259950,0migration/211447-21kthreadcore07:14:512
249950,0watchdog/25453-21kthreadcore11:44:552
249950,0watchdog/20-21swapper/211:49:552
179950,0migration/19703-21munin-node11:29:511
179950,0migration/129161-21munin-node07:34:571
179950,0migration/116058-21munin-node10:09:541
179950,0migration/116058-21munin-node10:09:541
179950,0migration/113903-21latency_hist08:49:381
169950,0watchdog/132122-21ntp_states10:19:551
149950,0migration/09262-21irqstats12:09:520
149950,0migration/07536-21kthreadcore09:44:530
149950,0migration/05165-21kthreadcore11:24:540
149950,0migration/04314-21taskset07:47:010
149950,0migration/032047-21latency_hist08:14:380
149950,0migration/031329-21kthreadcore08:59:550
149950,0migration/030494-21kthreadcore10:39:540
149950,0migration/030118-21kthreadcore09:19:530
149950,0migration/028811-21kthreadcore11:39:530
149950,0migration/028281-21if_eth009:39:520
149950,0migration/02343-21kthreadcore11:44:540
149950,0migration/020766-21kthreadcore10:34:530
149950,0migration/020021-21kthreadcore07:24:500
149950,0migration/018579-21kthreadcore11:34:530
149950,0migration/017805-21kthreadcore12:14:520
149950,0migration/016373-21taskset10:51:070
149950,0migration/016228-21sh11:12:010
149950,0migration/014486-21unixbench_singl11:50:000
149950,0migration/014373-21kthreadcore09:09:540
149950,0migration/013586-21kthreadcore11:29:540
149950,0migration/013500-21kthreadcore07:59:540
149950,0migration/011048-21kthreadcore08:19:550
149950,0migration/010456-21sh09:27:520
8215040,0irq/24-0000:00:27108-21latency_hist08:59:382
339940,0migration/37487-21taskset07:07:213
326849943,0cyclictest7306-21awk08:19:522
326849943,0cyclictest3514-21snmpd12:14:512
326849943,0cyclictest3514-21snmpd09:55:052
326849943,0cyclictest3514-21snmpd08:32:202
326849943,0cyclictest3514-21snmpd07:50:192
326849943,0cyclictest3514-21snmpd07:45:342
326849943,0cyclictest3514-21snmpd07:42:342
326849943,0cyclictest3514-21snmpd07:30:042
326849943,0cyclictest3514-21snmpd07:29:342
326849943,0cyclictest3514-21snmpd07:13:342
326849943,0cyclictest27428-21awk10:19:502
326849943,0cyclictest18695-21awk10:54:512
326839944,0cyclictest27111-21diskmemload12:28:541
326839944,0cyclictest0-21swapper/108:19:571
326839943,0cyclictest9512-21awk07:14:491
326839943,0cyclictest8819-21awk12:09:501
326839943,0cyclictest737-21awk12:04:491
326839943,0cyclictest3975-21awk09:04:511
326839943,0cyclictest3514-21snmpd10:06:201
326839943,0cyclictest3514-21snmpd09:37:351
326839943,0cyclictest3514-21snmpd08:44:051
326839943,0cyclictest3514-21snmpd07:41:191
326839943,0cyclictest3514-21snmpd07:30:491
326839943,0cyclictest3514-21snmpd07:12:041
326839943,0cyclictest3514-21snmpd07:00:191
326839943,0cyclictest27359-21awk08:34:521
326839943,0cyclictest17320-21awk11:54:541
326839943,0cyclictest14054-21awk08:24:521
326839943,0cyclictest12200-21awk09:09:511
326829944,0cyclictest0-21swapper/009:04:590
326829943,0cyclictest3558-21awk10:04:510
326829943,0cyclictest3241-21irqbalance11:55:250
326829943,0cyclictest2938-21kworker/0:011:08:540
326829943,0cyclictest28032-21awk09:59:520
326829943,0cyclictest26278-21awk08:09:510
326829943,0cyclictest2105-21awk07:04:480
326829943,0cyclictest19632-21awk08:04:510
29725040,0irq/27-eth0-tx-17921-21ssh09:31:122
29725040,0irq/27-eth0-tx-0-21swapper/212:21:552
29725040,0irq/27-eth0-tx-0-21swapper/212:09:532
29725040,0irq/27-eth0-tx-0-21swapper/211:59:382
29725040,0irq/27-eth0-tx-0-21swapper/211:54:552
29725040,0irq/27-eth0-tx-0-21swapper/211:34:432
29725040,0irq/27-eth0-tx-0-21swapper/211:24:422
29725040,0irq/27-eth0-tx-0-21swapper/211:10:062
29725040,0irq/27-eth0-tx-0-21swapper/210:49:422
29725040,0irq/27-eth0-tx-0-21swapper/210:39:412
29725040,0irq/27-eth0-tx-0-21swapper/210:34:422
29725040,0irq/27-eth0-tx-0-21swapper/210:09:532
29725040,0irq/27-eth0-tx-0-21swapper/210:09:532
29725040,0irq/27-eth0-tx-0-21swapper/209:59:422
29725040,0irq/27-eth0-tx-0-21swapper/209:49:382
29725040,0irq/27-eth0-tx-0-21swapper/209:20:202
29725040,0irq/27-eth0-tx-0-21swapper/209:05:002
29725040,0irq/27-eth0-tx-0-21swapper/209:00:012
29725040,0irq/27-eth0-tx-0-21swapper/008:29:500
29725040,0irq/27-eth0-tx-0-21swapper/008:24:480
29725040,0irq/27-eth0-tx-0-21swapper/007:52:100
29715040,0irq/26-eth0-rx-0-21swapper/312:26:073
29715040,0irq/26-eth0-rx-0-21swapper/311:01:273
29715040,0irq/26-eth0-rx-0-21swapper/310:19:453
29715040,0irq/26-eth0-rx-0-21swapper/309:49:493
29715040,0irq/26-eth0-rx-0-21swapper/309:44:473
29715040,0irq/26-eth0-rx-0-21swapper/309:19:513
29715040,0irq/26-eth0-rx-0-21swapper/309:00:013
29715040,0irq/26-eth0-rx-0-21swapper/308:54:543
29715040,0irq/26-eth0-rx-0-21swapper/308:19:543
259940,0migration/29095-21kthreadcore07:54:502
259940,0migration/26310-21latency_hist08:19:382
259940,0migration/231856-21idleruntime-cro08:14:382
259940,0migration/230333-21kthreadcore11:19:542
259940,0migration/229414-21kthreadcore08:34:542
259940,0migration/225494-21timerwakeupswit09:15:002
259940,0migration/221556-21kthreadcore11:14:542
259940,0migration/221499-21kthreadcore10:14:532
259940,0migration/215152-21irqcore08:49:532
259940,0migration/212612-21kthreadcore11:29:542
249940,0watchdog/20-21swapper/208:04:482
179940,0migration/16190-21kthreadcore09:24:541
179940,0migration/14238-21kthreadcore07:04:501
179940,0migration/14129-21kthreadcore08:14:551
179940,0migration/13412-21kthreadcore11:24:541
179940,0migration/127793-21fschecks_count09:39:501
179940,0migration/127137-21kthreadcore10:59:521
179940,0migration/123509-21kthreadcore08:29:541
179940,0migration/123230-21kthreadcore08:54:541
179940,0migration/122183-21kthreadcore09:14:551
179940,0migration/118896-21fschecks_time10:34:491
179940,0migration/118495-21kthreadcore07:24:501
179940,0migration/11664-21cpuspeed_turbos11:04:481
169940,0watchdog/131937-21ssh11:21:311
169940,0watchdog/11748-21ssh10:44:431
169940,0watchdog/10-21swapper/111:35:431
169940,0watchdog/10-21swapper/108:04:481
149940,0migration/09978-21turbostat10:34:320
149940,0migration/05070-21kthreadcore08:39:540
149940,0migration/04472-21kthreadcore10:24:530
149940,0migration/031726-21runrttasks11:20:510
149940,0migration/027461-21kthreadcore10:59:520
149940,0migration/019400-21kthreadcore10:54:530
8215030,0irq/24-0000:00:0-21swapper/207:19:382
326849932,0cyclictest3514-21snmpd07:02:042
326849932,0cyclictest2056-21awk07:04:482
326849932,0cyclictest13936-21awk08:24:512
326839933,0cyclictest0-21swapper/107:49:411
326839932,0cyclictest6440-21awk07:54:481
326839932,0cyclictest3514-21snmpd07:22:051
326839932,0cyclictest31300-21awk07:44:471
326839932,0cyclictest27320-21irqcore10:39:521
326839932,0cyclictest26233-21awk08:09:511
326839932,0cyclictest16592-21cut08:49:541
326839932,0cyclictest12971-21awk07:59:521
326839930,0cyclictest23378-21ntp_states10:54:571
326829933,0cyclictest29418-21kworker/0:209:29:390
326829933,0cyclictest29418-21kworker/0:208:54:410
326829933,0cyclictest29418-21kworker/0:208:34:410
326829933,0cyclictest29418-21kworker/0:207:54:410
326829933,0cyclictest29418-21kworker/0:207:39:510
326829933,0cyclictest29418-21kworker/0:207:29:410
326829933,0cyclictest29418-21kworker/0:207:19:410
326829933,0cyclictest29418-21kworker/0:207:09:490
326829933,0cyclictest29418-21kworker/0:206:59:470
326829933,0cyclictest18790-21kworker/0:110:14:400
326829932,0cyclictest3879-21grep12:24:540
326829932,0cyclictest24935-21grep09:14:560
326829932,0cyclictest20307-21awk09:34:510
326829932,0cyclictest18731-21grep11:14:530
326829932,0cyclictest12125-21grep10:09:520
326829932,0cyclictest12125-21grep10:09:520
326829931,0cyclictest0-21swapper/010:44:540
29715030,0irq/26-eth0-rx-0-21swapper/308:24:433
29715030,0irq/26-eth0-rx-0-21swapper/308:14:583
29715030,0irq/26-eth0-rx-0-21swapper/308:03:033
29715030,0irq/26-eth0-rx-0-21swapper/307:50:283
29715030,0irq/26-eth0-rx-0-21swapper/307:41:103
29715030,0irq/26-eth0-rx-0-21swapper/307:31:333
29715030,0irq/26-eth0-rx-0-21swapper/307:19:263
249930,0watchdog/27380-40yum-updatesd-he08:44:482
169930,0watchdog/115434-21munin-node11:09:591
169930,0watchdog/10-21swapper/112:19:471
169930,0watchdog/10-21swapper/110:49:471
169930,0watchdog/10-21swapper/109:54:431
169930,0watchdog/10-21swapper/109:19:471
329920,0watchdog/36657-21cut07:54:493
329920,0watchdog/321182-21kthreadcore08:04:533
29715020,0irq/26-eth0-rx-0-21swapper/308:50:513
29715020,0irq/26-eth0-rx-0-21swapper/307:09:403
29715020,0irq/26-eth0-rx-0-21swapper/307:00:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional