You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Jun 16, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
451523924,0sleep00-21swapper/006:44:550
442723728,0sleep30-21swapper/306:43:543
456123625,0sleep20-21swapper/206:45:252
449122925,0sleep10-21swapper/106:44:361
8225060,0irq/24-0000:00:0-21swapper/210:50:312
46439966,0cyclictest0-21swapper/211:33:172
46439960,0cyclictest1406-21kthreadcore07:30:482
46439960,0cyclictest0-21swapper/208:54:412
46429960,0cyclictest0-21swapper/109:55:471
339960,0migration/324381-21sh09:41:103
339960,0migration/318976-21sh08:56:143
339960,0migration/314652-21sh10:40:473
28415060,0irq/27-eth0-tx-0-21swapper/212:05:552
28415060,0irq/27-eth0-tx-0-21swapper/210:05:412
28415060,0irq/27-eth0-tx-0-21swapper/209:23:412
28405060,0irq/26-eth0-rx-8732-21vmstat11:35:583
28405060,0irq/26-eth0-rx-27613-21aten_r1power_fr11:10:443
28405060,0irq/26-eth0-rx-25313-21df_inode07:00:443
28405060,0irq/26-eth0-rx-12893-21cut09:35:523
249960,0watchdog/20-21swapper/211:07:462
19160,0ktimersoftd/129310-21diskmemload11:10:221
46439955,0cyclictest0-21swapper/210:35:542
46439955,0cyclictest0-21swapper/210:15:332
46439954,0cyclictest6445-21snmpd07:00:352
46439950,0cyclictest0-21swapper/209:53:382
46429955,0cyclictest0-21swapper/110:40:531
46429950,0cyclictest0-21swapper/111:04:131
46429950,0cyclictest0-21swapper/108:59:491
46429950,0cyclictest0-21swapper/108:36:011
46429950,0cyclictest0-21swapper/107:05:501
46419950,0cyclictest0-21swapper/010:42:060
339950,0migration/39434-21sh11:59:473
339950,0migration/38587-21latency_hist08:05:313
339950,0migration/35481-21kthreadcore09:10:503
339950,0migration/34827-21kthreadcore07:30:493
339950,0migration/34153-21latency_hist10:55:313
339950,0migration/33225-21kthreadcore10:30:523
339950,0migration/332226-21taskset07:29:583
339950,0migration/324581-21munin-node10:25:513
339950,0migration/323008-21kthreadcore09:00:513
339950,0migration/321149-21munin-node07:50:443
339950,0migration/319819-21munin-node11:25:523
339950,0migration/317838-21switchtime11:00:583
339950,0migration/315932-21fschecks_time08:10:493
339950,0migration/315837-21munin-node10:20:473
339950,0migration/312698-21kthreadcore12:00:513
28415050,0irq/27-eth0-tx-9250-21ssh11:37:262
28415050,0irq/27-eth0-tx-6445-21snmpd07:23:310
28415050,0irq/27-eth0-tx-31554-21munin-node07:05:550
28415050,0irq/27-eth0-tx-27545-21munin-node07:56:100
28415050,0irq/27-eth0-tx-20631-21ssh09:00:402
28415050,0irq/27-eth0-tx-0-21swapper/211:57:482
28415050,0irq/27-eth0-tx-0-21swapper/211:15:352
28415050,0irq/27-eth0-tx-0-21swapper/211:10:362
28415050,0irq/27-eth0-tx-0-21swapper/210:50:522
28415050,0irq/27-eth0-tx-0-21swapper/210:25:592
28415050,0irq/27-eth0-tx-0-21swapper/210:21:062
28415050,0irq/27-eth0-tx-0-21swapper/210:12:442
28415050,0irq/27-eth0-tx-0-21swapper/209:47:562
28415050,0irq/27-eth0-tx-0-21swapper/209:14:222
28415050,0irq/27-eth0-tx-0-21swapper/008:20:530
28405050,0irq/26-eth0-rx-7254-21kthreadcore11:15:543
28405050,0irq/26-eth0-rx-5807-21sshd06:45:473
28405050,0irq/26-eth0-rx-5807-21sshd06:45:473
28405050,0irq/26-eth0-rx-3378-21ssh10:53:403
28405050,0irq/26-eth0-rx-25792-21latency_hist12:10:313
28405050,0irq/26-eth0-rx-20652-21irqcore09:20:493
28405050,0irq/26-eth0-rx-1987-21ssh09:29:303
28405050,0irq/26-eth0-rx-1910-21munin-node09:50:473
28405050,0irq/26-eth0-rx-17643-21kthreadcore07:45:523
28405050,0irq/26-eth0-rx-17487-21fschecks_time10:00:493
28405050,0irq/26-eth0-rx-1186-21kthreadcore08:45:533
28405050,0irq/26-eth0-rx-0-21swapper/312:05:403
28405050,0irq/26-eth0-rx-0-21swapper/311:46:593
28405050,0irq/26-eth0-rx-0-21swapper/311:09:033
28405050,0irq/26-eth0-rx-0-21swapper/310:46:093
28405050,0irq/26-eth0-rx-0-21swapper/310:35:583
28405050,0irq/26-eth0-rx-0-21swapper/310:06:363
28405050,0irq/26-eth0-rx-0-21swapper/309:55:463
28405050,0irq/26-eth0-rx-0-21swapper/309:35:063
28405050,0irq/26-eth0-rx-0-21swapper/309:19:483
28405050,0irq/26-eth0-rx-0-21swapper/309:05:553
28405050,0irq/26-eth0-rx-0-21swapper/308:50:543
28405050,0irq/26-eth0-rx-0-21swapper/308:25:473
28405050,0irq/26-eth0-rx-0-21swapper/306:55:513
249950,0watchdog/224582-21ntpq11:25:582
249950,0watchdog/210975-21cut07:40:502
249950,0watchdog/20-21swapper/212:11:542
249950,0watchdog/20-21swapper/212:02:022
249950,0watchdog/20-21swapper/211:51:462
249950,0watchdog/20-21swapper/211:47:062
249950,0watchdog/20-21swapper/211:41:502
249950,0watchdog/20-21swapper/211:24:422
249950,0watchdog/20-21swapper/211:01:502
249950,0watchdog/20-21swapper/210:31:142
249950,0watchdog/20-21swapper/210:00:382
249950,0watchdog/20-21swapper/209:41:262
249950,0watchdog/20-21swapper/209:36:302
249950,0watchdog/20-21swapper/209:31:302
249950,0watchdog/20-21swapper/209:28:022
249950,0watchdog/20-21swapper/209:09:382
249950,0watchdog/20-21swapper/208:55:542
249950,0watchdog/20-21swapper/208:47:222
249950,0watchdog/20-21swapper/208:40:382
249950,0watchdog/20-21swapper/208:29:382
249950,0watchdog/20-21swapper/208:18:302
249950,0watchdog/20-21swapper/208:06:302
249950,0watchdog/20-21swapper/208:02:102
249950,0watchdog/20-21swapper/207:15:422
249950,0watchdog/20-21swapper/207:10:262
19150,0ktimersoftd/10-21swapper/111:58:461
179950,0migration/16213-21kthreadcore06:45:481
179950,0migration/16213-21kthreadcore06:45:481
179950,0migration/119673-21kthreadcore06:55:501
179950,0migration/114526-21cpuspeed_turbos07:45:471
179950,0migration/111007-21latency_hist11:20:311
149950,0migration/07702-21munin-node11:55:540
149950,0migration/026502-21cpuspeed_turbos11:50:480
149950,0migration/022185-21latency10:00:530
46439943,0cyclictest6445-21snmpd08:23:052
46439943,0cyclictest6445-21snmpd07:58:352
46439943,0cyclictest6445-21snmpd07:45:352
46439943,0cyclictest5229-21awk10:55:502
46439943,0cyclictest26694-21awk07:25:482
46439943,0cyclictest20086-21awk07:20:472
46439943,0cyclictest16268-21awk08:10:512
46429944,0cyclictest29233-21kworker/1:110:25:511
46429943,0cyclictest969-21awk07:30:461
46429943,0cyclictest6445-21snmpd11:46:361
46429943,0cyclictest6445-21snmpd11:32:211
46429943,0cyclictest6445-21snmpd09:26:361
46429943,0cyclictest6445-21snmpd07:43:351
46429943,0cyclictest3957-21awk11:15:511
46429943,0cyclictest3517-21awk11:35:501
46429943,0cyclictest29304-21sadc08:45:311
46429943,0cyclictest28396-21awk11:10:481
46429943,0cyclictest26899-21awk09:45:491
46429943,0cyclictest12432-21awk06:50:481
46419943,0cyclictest9547-21munin-node08:30:470
46419943,0cyclictest8356-21find11:35:560
46419943,0cyclictest6445-21snmpd12:14:360
46419943,0cyclictest6445-21snmpd09:25:360
46419943,0cyclictest6445-21snmpd07:45:500
46419943,0cyclictest6445-21snmpd07:25:500
46419943,0cyclictest3425-21lldpd09:21:120
46419943,0cyclictest3425-21lldpd08:01:210
46419943,0cyclictest28443-21awk11:10:480
46419943,0cyclictest27798-21awk11:30:500
46419943,0cyclictest24921-21munin-node07:00:490
46419943,0cyclictest23756-21awk08:40:500
46419943,0cyclictest17111-21awk08:35:520
46419943,0cyclictest12863-21awk11:00:490
46419943,0cyclictest11396-21awk11:40:490
46419943,0cyclictest10886-21awk12:00:490
339940,0migration/34990-21kthreadcore08:00:493
339940,0migration/329350-21hddtemp_smartct08:20:503
339940,0migration/325228-21taskset07:21:183
339940,0migration/324088-21kthreadcore08:15:503
339940,0migration/322770-21latency_hist08:40:313
339940,0migration/315289-21switchtime08:30:573
339940,0migration/310674-21kthreadcore07:10:493
28497240,0sleep00-21swapper/010:50:170
28415040,0irq/27-eth0-tx-19936-21ssh10:41:082
28415040,0irq/27-eth0-tx-15311-21munin-node08:10:570
28415040,0irq/27-eth0-tx-0-21swapper/209:16:112
28415040,0irq/27-eth0-tx-0-21swapper/008:45:350
28415040,0irq/27-eth0-tx-0-21swapper/008:25:560
28415040,0irq/27-eth0-tx-0-21swapper/006:55:450
28415040,0irq/27-eth0-tx-0-21swapper/006:50:550
28405040,0irq/26-eth0-rx-0-21swapper/311:53:583
28405040,0irq/26-eth0-rx-0-21swapper/311:44:033
28405040,0irq/26-eth0-rx-0-21swapper/311:30:323
28405040,0irq/26-eth0-rx-0-21swapper/311:20:543
28405040,0irq/26-eth0-rx-0-21swapper/310:11:263
28405040,0irq/26-eth0-rx-0-21swapper/309:49:183
28405040,0irq/26-eth0-rx-0-21swapper/307:43:333
259940,0migration/218542-21munin-node06:55:462
249940,0watchdog/219999-21kthreadcore08:35:542
249940,0watchdog/212539-21kthreadcore09:55:542
249940,0watchdog/211793-21unixbench_multi07:10:542
249940,0watchdog/20-21swapper/208:30:542
249940,0watchdog/20-21swapper/207:35:342
249940,0watchdog/20-21swapper/206:50:462
249940,0watchdog/20-21swapper/206:45:462
249940,0watchdog/20-21swapper/206:45:462
179940,0migration/17188-21taskset07:38:281
179940,0migration/14827-21kthreadcore08:00:491
179940,0migration/129258-21kthreadcore12:10:511
179940,0migration/128852-21munin-node07:56:061
179940,0migration/127581-21kthreadcore07:25:501
179940,0migration/126526-21diskmemload11:50:481
179940,0migration/123854-21kthreadcore10:45:541
179940,0migration/122649-21kthreadcore09:00:511
179940,0migration/122021-21kthreadcore10:00:531
179940,0migration/121698-21kthreadcore11:25:551
179940,0migration/121390-21kthreadcore12:05:521
179940,0migration/119980-21df_abs09:20:451
179940,0migration/118226-21kthreadcore08:10:521
179940,0migration/11709-21pmu-power09:05:571
179940,0migration/112792-21cpuspeed_turbos07:15:441
179940,0migration/112401-21kthreadcore09:35:521
179940,0migration/112298-21taskset08:55:031
179940,0migration/111870-21aten_r1power_po09:15:441
179940,0migration/111606-21kthreadcore08:30:511
179940,0migration/110159-21runrttasks09:10:581
169940,0watchdog/17811-21ssh09:51:461
169940,0watchdog/15696-21ssh10:34:301
169940,0watchdog/132525-21ssh10:10:501
169940,0watchdog/129310-21diskmemload09:41:581
169940,0watchdog/127462-21proc_pri08:15:541
169940,0watchdog/116107-21users12:00:581
169940,0watchdog/110703-21telnet11:40:461
169940,0watchdog/10-21swapper/110:50:461
169940,0watchdog/10-21swapper/110:15:501
149940,0migration/09849-21fschecks_time09:55:500
149940,0migration/07555-21kthreadcore10:55:520
149940,0migration/06746-21munin-node09:50:510
149940,0migration/04005-21if_eth011:15:510
149940,0migration/026655-21timerandwakeup07:50:540
149940,0migration/025592-21kthreadcore10:25:510
149940,0migration/023262-21kthreadcore08:15:500
149940,0migration/022330-21kthreadcore12:05:520
149940,0migration/02206-21kthreadcore10:10:550
149940,0migration/020526-21kthreadcore11:25:540
149940,0migration/019071-21fschecks_time11:45:490
149940,0migration/013426-21kthreadcore08:05:500
149940,0migration/012066-21kthreadcore07:40:500
149940,0migration/011014-21diskmemload09:13:440
149940,0migration/010968-21kthreadcore07:10:490
149940,0migration/010615-21kthreadcore10:15:500
46439932,0cyclictest21744-21awk07:50:462
46429933,0cyclictest29233-21kworker/1:110:35:391
46429933,0cyclictest14519-21kworker/1:008:45:551
46429933,0cyclictest14519-21kworker/1:008:20:331
46429933,0cyclictest14519-21kworker/1:008:05:331
46429933,0cyclictest14519-21kworker/1:007:50:411
46429933,0cyclictest14519-21kworker/1:007:20:351
46429933,0cyclictest14519-21kworker/1:007:10:351
46429933,0cyclictest14519-21kworker/1:007:00:411
46429932,0cyclictest29233-21kworker/1:110:05:351
46419932,0cyclictest9642-21tr10:35:530
46419932,0cyclictest886-21wc07:30:450
46419932,0cyclictest4852-21aten_r1power_cu06:45:420
46419932,0cyclictest4852-21aten_r1power_cu06:45:420
46419932,0cyclictest3425-21lldpd11:05:460
46419932,0cyclictest29332-21awk09:05:520
46419932,0cyclictest28627-21kthreadcore09:45:510
46419932,0cyclictest25414-21if_err_eth1.310:05:490
46419932,0cyclictest18181-21sensors07:15:530
46419932,0cyclictest17545-21sensors09:15:560
46419932,0cyclictest12481-21sadc08:55:310
46419932,0cyclictest11934-21awk11:20:510
46419932,0cyclictest103992sleep009:35:360
46419931,0cyclictest24774-21kthreadcore09:00:520
46419930,0cyclictest29575-21perl10:50:470
46419930,0cyclictest22083-21sh10:22:340
46419930,0cyclictest0-21swapper/010:30:380
339930,0migration/328554-21sh08:40:573
28405030,0irq/26-eth0-rx-0-21swapper/310:15:463
28405030,0irq/26-eth0-rx-0-21swapper/307:55:343
28405030,0irq/26-eth0-rx-0-21swapper/307:37:063
28405030,0irq/26-eth0-rx-0-21swapper/307:07:053
169930,0watchdog/10-21swapper/110:55:381
169930,0watchdog/10-21swapper/110:20:421
169930,0watchdog/10-21swapper/109:30:501
169930,0watchdog/10-21swapper/108:25:341
149930,0migration/025264-21sh09:44:050
149930,0migration/019461-21sh08:57:500
46419921,0cyclictest6761-21date07:35:310
46419920,0cyclictest0-21swapper/009:30:420
28405020,0irq/26-eth0-rx-0-21swapper/306:50:483
13547220,0sleep30-21swapper/307:15:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional