You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 02:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Dec 28, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450710,0irq/25-eth10-21swapper/319:03:483
3170424131,0sleep10-21swapper/119:04:421
3168723723,0sleep00-21swapper/019:04:290
2568123725,0sleep20-21swapper/219:03:512
4160,0ktimersoftd/02534-21ssh22:07:200
339960,0migration/39248-21if_eth023:28:513
339960,0migration/34353-21sh22:45:413
339960,0migration/3339-21kthreadcore21:08:533
339960,0migration/330710-21sh21:25:413
339960,0migration/32079-21sh00:03:333
339960,0migration/312949-21sh21:14:503
339960,0migration/310158-21sh00:07:333
321139966,0cyclictest0-21swapper/200:24:452
321139960,0cyclictest0-21swapper/223:53:412
321139960,0cyclictest0-21swapper/221:24:132
321129960,0cyclictest0-21swapper/100:23:501
29725060,0irq/27-eth0-tx-18964-21munin-node21:59:022
29715060,0irq/26-eth0-rx-8870-21aten_r1power_en21:33:493
29715060,0irq/26-eth0-rx-7250-21irqstats22:48:543
29715060,0irq/26-eth0-rx-5547-21sed21:28:553
29715060,0irq/26-eth0-rx-3077-21munin-node21:48:543
29715060,0irq/26-eth0-rx-0-21swapper/321:38:553
8215050,0irq/24-0000:00:29267-21ntp_states21:03:562
339950,0migration/38701-21aten_r1power_en20:03:513
339950,0migration/36073-21kthreadcore00:23:543
339950,0migration/331949-21sh23:40:433
339950,0migration/331375-21kthreadcore19:53:543
339950,0migration/329015-21munin-node22:23:503
339950,0migration/326118-21kthreadcore20:38:533
339950,0migration/32355-21taskset20:20:293
339950,0migration/322102-21sh23:15:013
339950,0migration/321779-21kthreadcore21:58:563
339950,0migration/321523-21taskset20:11:323
339950,0migration/31773-21kthreadcore23:03:533
339950,0migration/317267-21munin-node20:33:533
339950,0migration/317239-21aten_r1power_fr20:58:483
321139955,0cyclictest0-21swapper/221:54:582
321139955,0cyclictest0-21swapper/221:45:592
321139955,0cyclictest0-21swapper/221:14:462
321139955,0cyclictest0-21swapper/221:12:432
321129955,0cyclictest0-21swapper/123:16:591
321129955,0cyclictest0-21swapper/119:18:561
321129950,0cyclictest0-21swapper/121:10:331
321109955,0cyclictest4328-21kworker/0:023:18:510
321109955,0cyclictest0-21swapper/021:58:580
321109955,0cyclictest0-21swapper/000:23:230
321109954,0cyclictest5590-21ssh23:05:400
321109950,0cyclictest0-21swapper/023:27:060
321109950,0cyclictest0-21swapper/000:09:210
29725050,0irq/27-eth0-tx-9753-21munin-node23:48:472
29725050,0irq/27-eth0-tx-9287-21if_eth1.223:28:522
29725050,0irq/27-eth0-tx-5027-21tr23:23:562
29725050,0irq/27-eth0-tx-2879-21munin-node20:24:002
29725050,0irq/27-eth0-tx-28042-21diskstats00:18:472
29725050,0irq/27-eth0-tx-27312-21if_eth1.223:58:512
29725050,0irq/27-eth0-tx-22892-21munin-node20:38:492
29725050,0irq/27-eth0-tx-21939-21munin-node20:14:012
29725050,0irq/27-eth0-tx-19964-21munin-node19:23:572
29725050,0irq/27-eth0-tx-16116-21iostat21:18:552
29725050,0irq/27-eth0-tx-13152-21rm22:33:482
29725050,0irq/27-eth0-tx-0-21swapper/222:58:322
29725050,0irq/27-eth0-tx-0-21swapper/221:52:232
29725050,0irq/27-eth0-tx-0-21swapper/221:34:012
29725050,0irq/27-eth0-tx-0-21swapper/221:29:112
29725050,0irq/27-eth0-tx-0-21swapper/200:05:332
29715050,0irq/26-eth0-rx-2536-21ssh22:07:203
29715050,0irq/26-eth0-rx-24010-21kthreadcore22:58:523
29715050,0irq/26-eth0-rx-21604-21ssh21:19:403
29715050,0irq/26-eth0-rx-19962-21proc_pri22:53:573
29715050,0irq/26-eth0-rx-14027-21cut00:28:533
29715050,0irq/26-eth0-rx-0-21swapper/323:58:103
29715050,0irq/26-eth0-rx-0-21swapper/323:51:343
29715050,0irq/26-eth0-rx-0-21swapper/323:46:383
29715050,0irq/26-eth0-rx-0-21swapper/323:33:523
29715050,0irq/26-eth0-rx-0-21swapper/323:23:543
29715050,0irq/26-eth0-rx-0-21swapper/323:08:373
29715050,0irq/26-eth0-rx-0-21swapper/322:42:553
29715050,0irq/26-eth0-rx-0-21swapper/322:33:413
29715050,0irq/26-eth0-rx-0-21swapper/322:28:573
29715050,0irq/26-eth0-rx-0-21swapper/322:20:353
29715050,0irq/26-eth0-rx-0-21swapper/322:17:503
29715050,0irq/26-eth0-rx-0-21swapper/322:12:303
29715050,0irq/26-eth0-rx-0-21swapper/321:53:583
29715050,0irq/26-eth0-rx-0-21swapper/300:34:283
259950,0migration/24496-21kthreadcore19:58:572
259950,0migration/23548-21kthreadcore23:43:552
259950,0migration/231313-21sh23:39:092
259950,0migration/224748-21sh23:38:272
259950,0migration/223886-21kthreadcore00:13:522
259950,0migration/21726-21sh22:05:252
259950,0migration/216952-21kthreadcore19:43:562
259950,0migration/213241-21sh22:51:432
249950,0watchdog/211914-21cat20:53:542
179950,0migration/15787-21kthreadcore20:48:541
179950,0migration/111977-21unixbench-2d22:49:011
179950,0migration/111517-21taskset22:30:451
149950,0migration/05865-21kthreadcore00:23:540
149950,0migration/05222-21kthreadcore21:48:560
149950,0migration/03132-21sh22:25:560
149950,0migration/02999-21irqcore19:58:560
149950,0migration/024893-21kthreadcore22:18:570
149950,0migration/022214-21kthreadcore21:38:530
149950,0migration/021975-21kthreadcore00:33:530
149950,0migration/019848-21fschecks_time00:13:480
149950,0migration/016485-21munin-node20:54:020
149950,0migration/016040-21sh23:32:470
149950,0migration/014806-21kthreadcore22:13:540
149950,0migration/013480-21taskset21:15:580
149950,0migration/010950-21taskset21:53:500
149950,0migration/010296-21gltestperf23:48:480
339940,0migration/332248-21ksmtuned21:44:073
339940,0migration/330558-21irqcore20:43:533
339940,0migration/323538-21kthreadcore19:48:543
339940,0migration/318647-21missed_timers19:18:563
339940,0migration/314476-21aten2.4-expect19:43:513
339940,0migration/311119-21kthreadcore19:13:563
321139944,0cyclictest0-21swapper/220:51:072
321139943,0cyclictest5572-21awk22:28:512
321139943,0cyclictest3556-21lldpd19:29:492
321139943,0cyclictest18004-21awk20:58:512
321139943,0cyclictest16174-21awk20:08:532
321129943,0cyclictest7549-21awk19:13:531
321129943,0cyclictest3907-21awk20:23:541
321129943,0cyclictest3556-21lldpd23:50:521
321129943,0cyclictest3556-21lldpd21:16:221
321129943,0cyclictest3514-21snmpd22:43:581
321129943,0cyclictest3514-21snmpd20:13:281
321129943,0cyclictest3514-21snmpd19:49:581
321129943,0cyclictest3514-21snmpd19:42:431
321129943,0cyclictest3514-21snmpd19:36:431
321129943,0cyclictest3514-21snmpd19:09:131
321129943,0cyclictest26115-21awk23:38:521
321129943,0cyclictest20871-21awk19:23:511
321129943,0cyclictest18850-21awk23:53:521
321129943,0cyclictest15296-21awk19:43:541
321129943,0cyclictest14026-21awk22:33:521
321109944,0cyclictest4328-21kworker/0:023:58:390
321109944,0cyclictest4328-21kworker/0:023:53:490
321109944,0cyclictest4328-21kworker/0:023:43:370
321109944,0cyclictest4328-21kworker/0:023:38:530
321109944,0cyclictest4328-21kworker/0:023:34:170
321109944,0cyclictest4328-21kworker/0:023:08:450
321109944,0cyclictest4328-21kworker/0:022:58:430
321109944,0cyclictest4328-21kworker/0:022:54:210
321109944,0cyclictest4328-21kworker/0:022:49:230
321109944,0cyclictest4328-21kworker/0:022:38:370
321109944,0cyclictest4328-21kworker/0:022:33:390
321109944,0cyclictest4328-21kworker/0:000:04:050
321109944,0cyclictest22391-21kworker/0:222:28:430
321109944,0cyclictest22391-21kworker/0:222:08:470
321109944,0cyclictest22391-21kworker/0:221:43:410
321109944,0cyclictest22391-21kworker/0:221:33:410
321109944,0cyclictest22391-21kworker/0:221:25:250
321109944,0cyclictest22391-21kworker/0:221:09:420
321109944,0cyclictest22391-21kworker/0:220:58:480
321109944,0cyclictest22391-21kworker/0:220:48:540
321109944,0cyclictest22391-21kworker/0:220:38:560
321109944,0cyclictest22391-21kworker/0:220:28:520
321109944,0cyclictest22391-21kworker/0:220:18:500
321109944,0cyclictest22391-21kworker/0:220:08:500
321109944,0cyclictest22391-21kworker/0:220:03:560
321109944,0cyclictest22391-21kworker/0:219:48:380
321109944,0cyclictest22391-21kworker/0:219:33:380
321109944,0cyclictest22391-21kworker/0:219:18:380
321109944,0cyclictest22391-21kworker/0:219:13:380
321109944,0cyclictest22391-21kworker/0:219:08:520
321109943,0cyclictest4328-21kworker/0:000:28:550
321109943,0cyclictest3883-21awk20:23:540
321109943,0cyclictest30730-21awk22:43:490
321109943,0cyclictest27434-21awk19:28:510
321109943,0cyclictest16552-21awk23:13:520
321109943,0cyclictest1609-21sadc19:58:360
321109943,0cyclictest15982-21awk21:18:540
29725040,0irq/27-eth0-tx-30247-21diskmemload22:43:392
29725040,0irq/27-eth0-tx-22668-21munin-node22:58:522
29725040,0irq/27-eth0-tx-18881-21kthreadcore21:38:532
29725040,0irq/27-eth0-tx-15511-21ssh23:13:422
29725040,0irq/27-eth0-tx-0-21swapper/223:19:212
29725040,0irq/27-eth0-tx-0-21swapper/223:03:532
29725040,0irq/27-eth0-tx-0-21swapper/222:38:482
29725040,0irq/27-eth0-tx-0-21swapper/222:18:442
29725040,0irq/27-eth0-tx-0-21swapper/222:13:502
29725040,0irq/27-eth0-tx-0-21swapper/222:08:422
29725040,0irq/27-eth0-tx-0-21swapper/220:04:002
29725040,0irq/27-eth0-tx-0-21swapper/219:13:502
29725040,0irq/27-eth0-tx-0-21swapper/200:34:172
29725040,0irq/27-eth0-tx-0-21swapper/200:08:442
29715040,0irq/26-eth0-rx-0-21swapper/323:18:533
29715040,0irq/26-eth0-rx-0-21swapper/319:28:543
29715040,0irq/26-eth0-rx-0-21swapper/300:18:393
29715040,0irq/26-eth0-rx-0-21swapper/300:13:363
29715040,0irq/26-eth0-rx-0-21swapper/300:08:403
27140,0ktimersoftd/20-21swapper/219:41:072
259940,0migration/2486-21fschecks_time19:08:522
259940,0migration/232008-21kthreadcore19:53:552
259940,0migration/222805-21kthreadcore19:48:542
259940,0migration/210700-21irqcore20:28:552
249940,0watchdog/27099-21ssh23:08:382
249940,0watchdog/229280-21perl22:23:502
249940,0watchdog/219015-21kthreadcore20:33:552
249940,0watchdog/216165-21kthreadcore19:18:552
249940,0watchdog/215463-21kthreadcore00:28:542
249940,0watchdog/20-21swapper/220:43:502
249940,0watchdog/20-21swapper/220:18:592
179940,0migration/1414-21runrttasks19:31:541
179940,0migration/129469-21munin-node21:03:581
179940,0migration/129127-21kthreadcore21:43:541
179940,0migration/128618-21sh22:23:471
179940,0migration/125809-21sh23:18:521
179940,0migration/125500-21kthreadcore20:13:541
179940,0migration/124915-21timerandwakeup21:59:031
179940,0migration/124383-21fschecks_time21:23:501
179940,0migration/12290-21taskset20:19:441
179940,0migration/121154-21kthreadcore21:38:531
179940,0migration/119430-21kthreadcore20:33:551
179940,0migration/117207-21aten_r1power_en20:58:481
179940,0migration/114560-21kthreadcore21:53:551
179940,0migration/114280-21kthreadcore22:13:531
179940,0migration/114200-21latency_hist22:53:361
179940,0migration/112953-21kthreadcore20:53:551
169940,0watchdog/18318-21munin-node23:28:501
169940,0watchdog/16960-21ssh23:26:061
169940,0watchdog/1589-21grep21:28:501
169940,0watchdog/15688-21ssh23:05:541
169940,0watchdog/132146-21cut19:53:551
169940,0watchdog/130247-21diskmemload00:15:141
169940,0watchdog/126892-21ssh22:20:141
169940,0watchdog/124135-21tr22:38:541
169940,0watchdog/123107-21ssh23:34:341
169940,0watchdog/12060-21if_eth1.323:43:541
169940,0watchdog/12049-21ssh00:20:341
169940,0watchdog/118087-21ssh00:11:261
169940,0watchdog/116027-21ssh21:36:101
169940,0watchdog/115465-21kthreadcore00:28:541
169940,0watchdog/114975-21ssh23:12:581
169940,0watchdog/10-21swapper/123:58:541
169940,0watchdog/10-21swapper/122:58:381
169940,0watchdog/10-21swapper/122:03:581
169940,0watchdog/10-21swapper/121:49:221
169940,0watchdog/10-21swapper/100:33:541
149940,0migration/03813-21kthreadcore21:28:540
149940,0migration/028013-21taskset20:14:230
149940,0migration/027183-21kthreadcore21:03:530
149940,0migration/021896-21kthreadcore19:23:530
339930,0migration/313695-21runrttasks19:39:083
321129932,0cyclictest7960-21uptime19:59:031
321129932,0cyclictest23744-21awk20:38:511
321109933,0cyclictest22391-21kworker/0:220:43:520
321109933,0cyclictest22391-21kworker/0:220:33:500
321109933,0cyclictest22391-21kworker/0:219:43:380
321109933,0cyclictest22391-21kworker/0:219:38:420
29715030,0irq/26-eth0-rx-0-21swapper/321:03:483
29715030,0irq/26-eth0-rx-0-21swapper/320:54:343
29715030,0irq/26-eth0-rx-0-21swapper/320:48:373
29715030,0irq/26-eth0-rx-0-21swapper/320:28:513
29715030,0irq/26-eth0-rx-0-21swapper/320:23:513
29715030,0irq/26-eth0-rx-0-21swapper/320:14:573
29715030,0irq/26-eth0-rx-0-21swapper/320:02:433
29715030,0irq/26-eth0-rx-0-21swapper/319:33:483
29715030,0irq/26-eth0-rx-0-21swapper/319:23:473
249930,0watchdog/26205-21mdstat19:33:552
169930,0watchdog/16559-21runrttasks20:28:511
169930,0watchdog/10-21swapper/122:08:541
169930,0watchdog/10-21swapper/121:18:501
169930,0watchdog/10-21swapper/120:43:501
169930,0watchdog/10-21swapper/120:03:511
169930,0watchdog/10-21swapper/100:03:421
29715020,0irq/26-eth0-rx-0-21swapper/319:08:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional