You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 07:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Apr 03, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2960424637,0sleep10-21swapper/118:55:401
304450350,0irq/25-eth10-21swapper/318:54:353
2970623524,0sleep00-21swapper/018:56:510
2955023127,0sleep20-21swapper/218:55:052
149970,0migration/010052-21sh22:05:300
339960,0migration/37575-21needreboot23:44:523
339960,0migration/328275-21sh21:19:353
339960,0migration/32490-21kthreadcore20:14:513
339960,0migration/32013-21sh22:00:413
339960,0migration/319762-21sh21:13:373
339960,0migration/319245-21sh21:32:213
339960,0migration/317301-21sh23:11:183
339960,0migration/31233-21sh22:40:263
339960,0migration/310828-21sh22:28:543
300079960,0cyclictest0-21swapper/222:31:402
300079960,0cyclictest0-21swapper/221:13:282
29715060,0irq/26-eth0-rx-18743-21sh22:12:323
8215050,0irq/24-0000:00:9123-21latency_hist00:29:332
339950,0migration/39466-21taskset23:27:443
339950,0migration/38270-21ntp_offset23:04:543
339950,0migration/37760-21kthreadcore21:24:513
339950,0migration/36834-21kthreadcore22:44:513
339950,0migration/34488-21munin-node19:24:553
339950,0migration/332051-21kthreadcore18:59:503
339950,0migration/331595-21kthreadcore21:39:523
339950,0migration/330352-21timerwakeupswit19:19:563
339950,0migration/327485-21latency_hist22:19:353
339950,0migration/327348-21if_eth1.223:39:483
339950,0migration/327218-21irqcore23:59:513
339950,0migration/327213-21munin-node20:54:563
339950,0migration/327023-21munin-node22:59:453
339950,0migration/326135-21fschecks_time19:44:483
339950,0migration/323108-21missed_timers00:14:533
339950,0migration/322196-21kthreadcore22:34:523
339950,0migration/320902-21irqcore20:04:503
339950,0migration/319585-21fschecks_count22:54:473
339950,0migration/317744-21sh23:54:253
339950,0migration/317456-21idleruntime-cro00:14:333
339950,0migration/317386-210anacron22:50:343
339950,0migration/314605-21kthreadcore19:09:513
339950,0migration/312141-21munin-node19:54:563
339950,0migration/311670-21munin-node23:29:523
339950,0migration/311166-21munin-node19:29:573
300079955,0cyclictest0-21swapper/220:59:452
300079955,0cyclictest0-21swapper/219:22:382
300069955,0cyclictest0-21swapper/120:54:531
300069954,0cyclictest20567-21ps22:34:511
300059955,0cyclictest13068-21kworker/0:021:36:360
300059955,0cyclictest0-21swapper/023:09:330
300059954,0cyclictest13135-21python20:19:550
300059953,0cyclictest13068-21kworker/0:021:51:160
300059950,0cyclictest0-21swapper/021:56:050
29725050,0irq/27-eth0-tx-3684-21munin-node22:04:492
29725050,0irq/27-eth0-tx-29429-21kthreadcore23:59:522
29725050,0irq/27-eth0-tx-28749-21grep00:19:532
29725050,0irq/27-eth0-tx-28364-21ssh20:59:370
29725050,0irq/27-eth0-tx-27184-21ssh21:16:412
29725050,0irq/27-eth0-tx-25440-21sendmail_mailtr22:14:572
29725050,0irq/27-eth0-tx-19044-21munin-node22:54:562
29725050,0irq/27-eth0-tx-1515-21ssh23:02:192
29725050,0irq/27-eth0-tx-11883-21ps23:09:502
29725050,0irq/27-eth0-tx-11022-21munin-node22:49:512
29725050,0irq/27-eth0-tx-0-21swapper/223:47:132
29725050,0irq/27-eth0-tx-0-21swapper/222:44:512
29725050,0irq/27-eth0-tx-0-21swapper/200:11:362
29725050,0irq/27-eth0-tx-0-21swapper/019:19:520
29725050,0irq/27-eth0-tx-0-21swapper/019:09:030
29715050,0irq/26-eth0-rx-8500-21ssh00:06:503
29715050,0irq/26-eth0-rx-31543-21kthreadcore20:59:533
29715050,0irq/26-eth0-rx-2248-21if_eth1.300:24:483
29715050,0irq/26-eth0-rx-21834-21kthreadcore21:34:503
29715050,0irq/26-eth0-rx-1466-21ssh22:20:053
29715050,0irq/26-eth0-rx-12904-21if_eth021:49:493
29715050,0irq/26-eth0-rx-11693-21df_abs22:29:473
29715050,0irq/26-eth0-rx-0-21swapper/323:58:043
29715050,0irq/26-eth0-rx-0-21swapper/323:39:283
29715050,0irq/26-eth0-rx-0-21swapper/323:14:513
29715050,0irq/26-eth0-rx-0-21swapper/321:57:413
29715050,0irq/26-eth0-rx-0-21swapper/321:47:503
29715050,0irq/26-eth0-rx-0-21swapper/321:47:503
29715050,0irq/26-eth0-rx-0-21swapper/321:19:353
29715050,0irq/26-eth0-rx-0-21swapper/320:19:503
29715050,0irq/26-eth0-rx-0-21swapper/300:21:063
259950,0migration/28141-21sendmail_mailtr23:24:552
259950,0migration/28115-21kthreadcore19:29:512
259950,0migration/23314-21sh22:24:452
259950,0migration/230961-21kthreadcore21:39:522
259950,0migration/230172-21kthreadcore21:19:512
259950,0migration/215746-21kthreadcore20:24:502
259950,0migration/214348-21kthreadcore21:29:522
259950,0migration/210849-21sh21:47:042
259950,0migration/210849-21sh21:47:042
179950,0migration/15596-21timerwakeupswit19:49:561
179950,0migration/130241-21kthreadcore22:39:481
179950,0migration/127972-21turbostat.cron21:39:301
179950,0migration/123950-21kthreadcore19:39:521
179950,0migration/122387-21kthreadcore22:14:521
179950,0migration/122063-21kthreadcore21:54:511
179950,0migration/118324-21taskset23:14:181
179950,0migration/11787-21latency_hist23:44:341
179950,0migration/117805-21kthreadcore19:59:521
179950,0migration/114487-21kthreadcore19:34:501
179950,0migration/111917-21kthreadcore00:09:491
169950,0watchdog/15253-21sed21:04:531
149950,0migration/08682-21sh00:07:230
149950,0migration/08660-21kthreadcore19:54:520
149950,0migration/0561-21sh00:02:230
149950,0migration/05480-21kthreadcore21:04:530
149950,0migration/03791-21kthreadcore23:44:500
149950,0migration/032474-21sendmail_mailtr23:19:540
149950,0migration/025807-21taskset22:35:560
149950,0migration/025580-21sh23:17:090
149950,0migration/024415-21munin-node23:34:580
149950,0migration/021738-21kernelversion21:14:510
149950,0migration/021316-21kthreadcore22:14:520
149950,0migration/016313-21runrttasks20:49:490
149950,0migration/01569-21latency_hist20:39:350
149950,0migration/012754-21kthreadcore23:09:500
149950,0migration/011945-21latency_hist19:34:350
149950,0migration/011197-21taskset21:48:050
149950,0migration/011197-21taskset21:48:050
149950,0migration/010960-21latency_hist22:49:340
8215040,0irq/24-0000:00:18417-21latency_hist23:14:342
339940,0migration/3624-21if_eth1.219:49:483
339940,0migration/331955-21kthreadcore20:34:513
339940,0migration/326542-21aten_r1power_fr20:09:453
339940,0migration/322739-21kthreadcore20:29:513
339940,0migration/315024-21latency_hist20:49:353
339940,0migration/314793-21kthreadcore19:34:513
339940,0migration/310073-21kthreadcore20:44:503
300079944,0cyclictest0-21swapper/219:34:372
300079943,0cyclictest578-21awk19:49:482
300079943,0cyclictest5417-21awk19:04:492
300079943,0cyclictest3241-21irqbalance23:49:422
300079943,0cyclictest32109-21awk19:24:492
300079943,0cyclictest20889-21awk21:54:492
300079943,0cyclictest12172-21awk19:09:492
300079943,0cyclictest11229-21awk23:29:482
300079940,0cyclictest0-21swapper/220:42:152
300069943,0cyclictest8378-21awk20:19:481
300069943,0cyclictest3556-21lldpd23:04:401
300069943,0cyclictest3556-21lldpd22:27:401
300069943,0cyclictest3556-21lldpd21:17:101
300069943,0cyclictest3556-21lldpd20:50:101
300069943,0cyclictest3556-21lldpd20:39:401
300069943,0cyclictest3556-21lldpd19:55:091
300069943,0cyclictest3556-21lldpd19:28:071
300069943,0cyclictest3556-21lldpd19:15:081
300069943,0cyclictest3556-21lldpd19:09:071
300069943,0cyclictest3556-21lldpd00:16:101
300069943,0cyclictest3514-21snmpd23:58:251
300069943,0cyclictest3514-21snmpd22:46:101
300069943,0cyclictest3514-21snmpd22:29:551
300069943,0cyclictest3514-21snmpd00:20:101
300069943,0cyclictest29285-21awk20:59:501
300069943,0cyclictest26933-21awk23:59:491
300069943,0cyclictest2538-21awk00:04:481
300069943,0cyclictest2197-21awk00:24:481
300069943,0cyclictest19996-21awk22:54:491
300069943,0cyclictest19385-21awk23:34:501
300069943,0cyclictest1624-21awk20:14:501
300069943,0cyclictest13283-21awk21:29:501
300069943,0cyclictest12100-21awk19:09:481
300069940,0cyclictest0-21swapper/123:44:461
300069940,0cyclictest0-21swapper/123:25:461
300069940,0cyclictest0-21swapper/123:02:451
300069940,0cyclictest0-21swapper/122:02:221
300059944,0cyclictest0-21swapper/022:24:470
300059943,0cyclictest3556-21lldpd22:46:400
300059943,0cyclictest3556-21lldpd21:32:100
300059943,0cyclictest3514-21snmpd20:31:540
300059943,0cyclictest3241-21irqbalance23:01:020
300059943,0cyclictest3241-21irqbalance22:58:120
300059943,0cyclictest3241-21irqbalance22:30:030
300059943,0cyclictest3241-21irqbalance22:00:130
300059943,0cyclictest3241-21irqbalance21:09:430
300059943,0cyclictest3241-21irqbalance00:24:420
300059943,0cyclictest3241-21irqbalance00:22:420
300059943,0cyclictest29153-21awk21:39:500
300059943,0cyclictest28173-21awk22:39:450
300059943,0cyclictest26328-21awk19:44:490
300059943,0cyclictest19731-21awk19:39:500
300059943,0cyclictest1797-21kworker/0:120:00:440
300059943,0cyclictest11302-21awk23:29:490
29725040,0irq/27-eth0-tx-9331-21kthreadcore21:24:512
29725040,0irq/27-eth0-tx-692-21munin-node20:14:480
29725040,0irq/27-eth0-tx-31299-21kthreadcore22:19:532
29725040,0irq/27-eth0-tx-17745-21unixbench-2d22:09:592
29725040,0irq/27-eth0-tx-0-21swapper/223:54:512
29725040,0irq/27-eth0-tx-0-21swapper/223:34:522
29725040,0irq/27-eth0-tx-0-21swapper/223:19:582
29725040,0irq/27-eth0-tx-0-21swapper/222:39:492
29725040,0irq/27-eth0-tx-0-21swapper/221:34:462
29725040,0irq/27-eth0-tx-0-21swapper/200:04:442
29725040,0irq/27-eth0-tx-0-21swapper/020:59:230
29725040,0irq/27-eth0-tx-0-21swapper/020:39:500
29725040,0irq/27-eth0-tx-0-21swapper/018:59:480
29715040,0irq/26-eth0-rx-29835-21kthreadcore23:19:493
29715040,0irq/26-eth0-rx-0-21swapper/322:05:173
29715040,0irq/26-eth0-rx-0-21swapper/321:04:453
259940,0migration/27722-21irqrtprio19:54:512
259940,0migration/27684-21munin-node20:19:462
259940,0migration/231041-21proc_pri19:44:542
259940,0migration/228206-21kthreadcore20:09:502
259940,0migration/228110-21sh21:59:452
259940,0migration/227843-21munin-node20:34:462
259940,0migration/227568-21irqrtprio23:39:492
259940,0migration/225723-21unixbench_multi20:04:572
259940,0migration/222891-21irqcore20:54:522
259940,0migration/220484-21kthreadcore19:14:512
259940,0migration/216551-21kthreadcore20:49:492
259940,0migration/214412-21kthreadcore21:49:512
259940,0migration/21365-21kthreadcore18:59:512
249940,0watchdog/29454-21iostat_ios20:44:492
249940,0watchdog/25254-21grep21:04:532
249940,0watchdog/221647-21if_err_eth120:29:492
249940,0watchdog/221413-21kthreadcore00:14:522
249940,0watchdog/20-21swapper/223:04:482
249940,0watchdog/20-21swapper/222:34:482
179940,0migration/129208-21kthreadcore22:19:521
179940,0migration/128719-21kernelversion20:34:501
179940,0migration/127454-21gltestperf23:19:461
179940,0migration/126768-21cpuspeed_turbos20:09:461
179940,0migration/124607-21timerandwakeup23:14:571
179940,0migration/120762-21latency_hist20:29:351
179940,0migration/111253-21cpuspeed_turbos22:49:451
169940,0watchdog/132581-21kthreadcore21:39:531
169940,0watchdog/118630-21ssh21:10:331
169940,0watchdog/112312-21fschecks_time22:09:481
169940,0watchdog/110555-21ssh21:46:131
169940,0watchdog/110555-21ssh21:46:131
149940,0migration/05419-21irqcore21:24:490
149940,0migration/030241-21kthreadcore23:39:500
149940,0migration/028352-21kthreadcore20:09:500
149940,0migration/027695-21runrttasks22:19:460
149940,0migration/02385-21sh21:20:080
149940,0migration/020532-21fschecks_time20:04:470
149940,0migration/019649-21kthreadcore23:54:510
149940,0migration/018525-21fschecks_time19:14:480
149940,0migration/013734-21missed_timers20:44:510
149940,0migration/012576-21kthreadcore23:49:510
8215030,0irq/24-0000:00:0-21swapper/219:39:442
300079932,0cyclictest14039-21awk19:59:492
300069933,0cyclictest3556-21lldpd20:44:391
300069933,0cyclictest3556-21lldpd18:59:381
300069933,0cyclictest11351-21awk23:29:491
300069932,0cyclictest9795-21sh22:04:581
300069932,0cyclictest4247-21munin-node21:24:591
300069932,0cyclictest30126-21kthreadcore21:19:511
300069932,0cyclictest15593-21netstat23:49:531
300069932,0cyclictest12750-21hddtemp_smartct21:49:471
300059933,0cyclictest3388-21kworker/0:223:24:550
300059933,0cyclictest3388-21kworker/0:200:14:350
300059933,0cyclictest3388-21kworker/0:200:09:350
300059933,0cyclictest27720-21kworker/0:222:09:400
300059933,0cyclictest1797-21kworker/0:120:24:480
300059933,0cyclictest1797-21kworker/0:119:49:400
300059933,0cyclictest1797-21kworker/0:119:34:360
300059933,0cyclictest1797-21kworker/0:119:24:360
300059932,0cyclictest11217-21munin-node19:09:490
29715030,0irq/26-eth0-rx-0-21swapper/320:39:443
29715030,0irq/26-eth0-rx-0-21swapper/320:27:153
29715030,0irq/26-eth0-rx-0-21swapper/319:59:463
29715030,0irq/26-eth0-rx-0-21swapper/319:39:483
29715030,0irq/26-eth0-rx-0-21swapper/319:14:493
29715030,0irq/26-eth0-rx-0-21swapper/319:05:583
169930,0watchdog/120215-21ps20:25:371
169930,0watchdog/10-21swapper/119:44:451
169930,0watchdog/10-21swapper/119:29:531
169930,0watchdog/10-21swapper/119:19:451
300069920,0cyclictest0-21swapper/120:04:361
249920,0watchdog/20-21swapper/220:14:372
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional