You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 00:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Dec 18, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3161824028,0sleep00-21swapper/007:05:010
3180023827,0sleep30-21swapper/307:07:083
3185823627,0sleep20-21swapper/207:07:472
3187923324,0sleep10-21swapper/107:08:021
32080992120,0cyclictest14879-21turbostat11:49:292
320719976,0cyclictest0-21swapper/010:04:120
339960,0migration/327896-21kthreadcore10:39:473
339960,0migration/314796-21sh10:32:023
320809966,0cyclictest0-21swapper/211:02:392
320739960,0cyclictest0-21swapper/110:36:451
320719963,0cyclictest17637-21kworker/0:010:51:550
29725060,0irq/27-eth0-tx-14209-21munin-node07:44:550
29715060,0irq/26-eth0-rx-30333-21if_err_eth1.308:44:443
29715060,0irq/26-eth0-rx-26169-21grep07:49:493
29715060,0irq/26-eth0-rx-15194-21open_files10:49:493
149960,0migration/06096-21sh09:13:570
149960,0migration/04670-21sh11:24:340
149960,0migration/022466-21sh10:19:240
149960,0migration/018346-21kthreadcore09:19:470
8215050,0irq/24-0000:00:7357-21munin-run07:39:292
8215050,0irq/24-0000:00:10201-21latency_hist08:54:292
339950,0migration/37951-21taskset12:04:003
339950,0migration/37920-21kthreadcore11:24:473
339950,0migration/33598-21latency_hist09:49:293
339950,0migration/331978-21kthreadcore11:39:473
339950,0migration/331668-21unixbench_singl12:14:533
339950,0migration/328350-21kthreadcore07:29:473
339950,0migration/325522-21kthreadcore11:34:473
339950,0migration/32484-21sh11:20:233
339950,0migration/324522-21proc_pri09:39:503
339950,0migration/321453-21kthreadcore11:14:463
339950,0migration/319332-21diskmemload09:56:423
339950,0migration/319192-21unixbench_multi07:19:523
339950,0migration/318415-21kthreadcore08:09:483
339950,0migration/316502-21taskset08:55:313
339950,0migration/31527-21fschecks_time09:29:433
339950,0migration/311303-21kthreadcore09:34:463
339950,0migration/310504-21aten_r1power_vo12:24:403
320809950,0cyclictest0-21swapper/211:25:402
320809950,0cyclictest0-21swapper/209:41:082
320809950,0cyclictest0-21swapper/209:10:242
320739955,0cyclictest0-21swapper/112:38:381
320739955,0cyclictest0-21swapper/112:14:401
320739955,0cyclictest0-21swapper/109:19:301
320739950,0cyclictest0-21swapper/110:08:321
320739950,0cyclictest0-21swapper/109:26:361
320719955,0cyclictest17637-21kworker/0:010:29:450
320719953,0cyclictest17637-21kworker/0:012:35:280
320719953,0cyclictest17637-21kworker/0:012:29:440
320719953,0cyclictest17637-21kworker/0:012:24:580
320719953,0cyclictest17637-21kworker/0:012:20:380
320719953,0cyclictest17637-21kworker/0:012:14:400
320719953,0cyclictest17637-21kworker/0:012:07:240
320719953,0cyclictest17637-21kworker/0:012:00:080
320719953,0cyclictest17637-21kworker/0:011:54:380
320719953,0cyclictest17637-21kworker/0:011:50:280
320719953,0cyclictest17637-21kworker/0:011:35:110
320719953,0cyclictest17637-21kworker/0:011:20:110
320719953,0cyclictest17637-21kworker/0:011:09:410
320719953,0cyclictest17637-21kworker/0:011:00:190
320719953,0cyclictest17637-21kworker/0:010:55:110
320719953,0cyclictest17637-21kworker/0:010:44:390
320719953,0cyclictest17637-21kworker/0:010:39:550
320719953,0cyclictest17637-21kworker/0:010:34:390
320719953,0cyclictest17637-21kworker/0:010:25:390
320719953,0cyclictest17637-21kworker/0:010:20:090
320719953,0cyclictest17637-21kworker/0:010:04:550
320719953,0cyclictest17637-21kworker/0:009:54:390
320719953,0cyclictest17637-21kworker/0:009:49:490
320719953,0cyclictest17637-21kworker/0:009:44:390
320719953,0cyclictest17637-21kworker/0:009:35:390
320719953,0cyclictest17637-21kworker/0:009:30:190
320719953,0cyclictest17637-21kworker/0:009:24:390
320719953,0cyclictest17637-21kworker/0:009:15:410
29725050,0irq/27-eth0-tx-8508-21munin-node08:04:430
29725050,0irq/27-eth0-tx-797-21munin-node07:34:470
29725050,0irq/27-eth0-tx-7799-21ssh09:31:012
29725050,0irq/27-eth0-tx-29856-21grep12:14:472
29725050,0irq/27-eth0-tx-20926-21munin-node09:59:492
29725050,0irq/27-eth0-tx-17214-21kthreadcore09:54:462
29725050,0irq/27-eth0-tx-0-21swapper/211:21:242
29725050,0irq/27-eth0-tx-0-21swapper/210:25:502
29725050,0irq/27-eth0-tx-0-21swapper/210:20:432
29725050,0irq/27-eth0-tx-0-21swapper/209:44:462
29725050,0irq/27-eth0-tx-0-21swapper/209:24:572
29725050,0irq/27-eth0-tx-0-21swapper/007:42:290
29725050,0irq/27-eth0-tx-0-21swapper/007:22:260
29715050,0irq/26-eth0-rx-6559-21runrttasks12:04:383
29715050,0irq/26-eth0-rx-6551-21ntp_offset10:44:493
29715050,0irq/26-eth0-rx-3205-21ssh10:04:523
29715050,0irq/26-eth0-rx-31550-21irqcore09:09:473
29715050,0irq/26-eth0-rx-29659-21ssh10:21:213
29715050,0irq/26-eth0-rx-23841-21kthreadcore09:59:463
29715050,0irq/26-eth0-rx-21705-21ssh11:50:473
29715050,0irq/26-eth0-rx-21265-21kthreadcore10:54:453
29715050,0irq/26-eth0-rx-16911-21cut10:14:453
29715050,0irq/26-eth0-rx-16433-21runrttasks09:19:453
29715050,0irq/26-eth0-rx-13981-21crond10:14:293
29715050,0irq/26-eth0-rx-12498-21munin-node11:09:463
29715050,0irq/26-eth0-rx-10357-21cut11:44:473
29715050,0irq/26-eth0-rx-0-21swapper/312:34:403
29715050,0irq/26-eth0-rx-0-21swapper/312:14:043
29715050,0irq/26-eth0-rx-0-21swapper/311:58:523
29715050,0irq/26-eth0-rx-0-21swapper/311:31:293
29715050,0irq/26-eth0-rx-0-21swapper/311:06:363
29715050,0irq/26-eth0-rx-0-21swapper/310:34:533
29715050,0irq/26-eth0-rx-0-21swapper/310:24:413
29715050,0irq/26-eth0-rx-0-21swapper/309:45:263
29715050,0irq/26-eth0-rx-0-21swapper/309:24:433
29715050,0irq/26-eth0-rx-0-21swapper/309:04:463
29715050,0irq/26-eth0-rx-0-21swapper/308:29:403
29715050,0irq/26-eth0-rx-0-21swapper/308:19:443
259950,0migration/28330-21sh12:04:292
259950,0migration/229470-21irqcore08:19:442
259950,0migration/222898-21irqcore11:34:452
259950,0migration/222766-21kthreadcore12:29:452
259950,0migration/218931-21kthreadcore08:09:482
259950,0migration/214942-21latency10:49:472
249950,0watchdog/223262-21sh11:54:192
179950,0migration/16602-21latency_hist07:14:311
179950,0migration/14190-21kthreadcore12:19:471
179950,0migration/14179-21kthreadcore07:59:461
179950,0migration/129415-21munin-node08:19:431
179950,0migration/127715-21runrttasks11:18:261
179950,0migration/125334-21kthreadcore11:54:451
179950,0migration/124479-21ntp_offset12:29:481
179950,0migration/124043-21timerandwakeup10:54:501
179950,0migration/123600-21idleruntime08:39:451
179950,0migration/123481-21kthreadcore11:34:461
179950,0migration/122928-21kthreadcore09:59:461
179950,0migration/115305-21taskset12:06:291
179950,0migration/114358-21aten_r1power_vo07:44:431
179950,0migration/111884-21kthreadcore10:49:461
179950,0migration/111396-21sh09:53:041
179950,0migration/110222-21missed_timers11:24:481
179950,0migration/110141-21fschecks_time08:29:431
169950,0watchdog/15336-21kthreadcore08:24:481
169950,0watchdog/132255-21python07:29:521
169950,0watchdog/111970-21kthreadcore10:29:481
149950,0migration/05987-21cpuspeed_turbos10:09:420
149950,0migration/032565-21kthreadcore11:39:470
149950,0migration/030141-21taskset09:09:140
149950,0migration/02833-21taskset11:04:390
149950,0migration/026948-21kthreadcore08:39:480
149950,0migration/026858-21cpuspeed_turbos07:29:420
149950,0migration/022286-21fschecks_count08:14:430
149950,0migration/021676-21kthreadcore11:14:460
149950,0migration/021609-21kthreadcore12:09:480
149950,0migration/019734-21kthreadcore08:09:480
149950,0migration/018975-21aten2.4-expect09:39:410
149950,0migration/016723-21kthreadcore11:29:470
149950,0migration/012809-21kthreadcore08:54:460
149950,0migration/010474-21kthreadcore11:44:470
149950,0migration/010306-21latency_hist08:54:290
8215040,0irq/24-0000:00:15389-21cat09:19:292
339940,0migration/39442-21kthreadcore07:39:473
339940,0migration/34075-21kthreadcore07:09:433
339940,0migration/33034-21cpuspeed_turbos08:24:433
339940,0migration/33026-21kthreadcore07:34:473
339940,0migration/32981-21irqrtprio07:59:453
339940,0migration/326233-21taskset07:28:183
320809944,0cyclictest0-21swapper/209:49:302
320809944,0cyclictest0-21swapper/208:39:302
320809943,0cyclictest691-21awk07:09:402
320809943,0cyclictest2733-21awk07:59:442
320809943,0cyclictest24640-21awk09:04:432
320809943,0cyclictest22668-21awk08:14:442
320809942,0cyclictest27451-21awk07:29:452
320739943,0cyclictest8514-21awk07:39:461
320739943,0cyclictest3556-21lldpd09:37:281
320739943,0cyclictest3514-21snmpd10:47:391
320739943,0cyclictest3514-21snmpd10:25:391
320739943,0cyclictest3514-21snmpd10:12:541
320739943,0cyclictest28405-21awk09:44:441
320739943,0cyclictest17989-21awk12:09:461
320739940,0cyclictest0-21swapper/107:58:281
320719943,0cyclictest20912-21awk07:24:460
320719943,0cyclictest10352-21awk08:29:440
29725040,0irq/27-eth0-tx-30181-21diskmemload11:54:382
29725040,0irq/27-eth0-tx-29923-21ssh10:04:412
29725040,0irq/27-eth0-tx-1654-21ssh12:19:352
29725040,0irq/27-eth0-tx-15076-21ssh10:14:422
29725040,0irq/27-eth0-tx-10496-21ssh12:24:402
29725040,0irq/27-eth0-tx-0-21swapper/212:34:382
29725040,0irq/27-eth0-tx-0-21swapper/211:15:312
29725040,0irq/27-eth0-tx-0-21swapper/211:04:412
29725040,0irq/27-eth0-tx-0-21swapper/210:54:442
29725040,0irq/27-eth0-tx-0-21swapper/210:45:102
29725040,0irq/27-eth0-tx-0-21swapper/210:39:382
29725040,0irq/27-eth0-tx-0-21swapper/210:34:322
29725040,0irq/27-eth0-tx-0-21swapper/210:29:472
29725040,0irq/27-eth0-tx-0-21swapper/210:09:402
29725040,0irq/27-eth0-tx-0-21swapper/008:59:400
29725040,0irq/27-eth0-tx-0-21swapper/008:44:490
29725040,0irq/27-eth0-tx-0-21swapper/008:34:510
29725040,0irq/27-eth0-tx-0-21swapper/007:09:390
29715040,0irq/26-eth0-rx-0-21swapper/312:30:013
29715040,0irq/26-eth0-rx-0-21swapper/312:19:483
29715040,0irq/26-eth0-rx-0-21swapper/310:59:493
29715040,0irq/26-eth0-rx-0-21swapper/309:15:283
29715040,0irq/26-eth0-rx-0-21swapper/308:59:443
259940,0migration/26743-21aten_r1power_en09:14:412
259940,0migration/227394-21latency_hist07:54:302
259940,0migration/220006-21aten_r1power_fr07:24:412
259940,0migration/218702-21kthreadcore08:34:482
259940,0migration/217912-21idleruntime08:59:442
259940,0migration/216300-21kthreadcore11:29:472
259940,0migration/214791-21kthreadcore09:34:472
259940,0migration/213811-21kthreadcore11:09:472
259940,0migration/211510-21kthreadcore08:04:452
259940,0migration/210850-21kthreadcore08:29:462
249940,0watchdog/23128-21diskstats08:24:442
249940,0watchdog/229458-21munin-node08:44:402
249940,0watchdog/223133-21yum12:09:552
249940,0watchdog/216189-21kthreadcore07:44:482
249940,0watchdog/20-21swapper/211:59:472
249940,0watchdog/20-21swapper/211:39:432
179940,0migration/19293-21hwlatdetect08:04:421
179940,0migration/14771-21kthreadcore07:09:431
179940,0migration/14263-21fschecks_count08:49:431
179940,0migration/128781-21kthreadcore10:59:461
179940,0migration/123529-21munin-run09:04:291
179940,0migration/120388-21kthreadcore11:49:471
179940,0migration/120077-21aten_r1power_vo07:24:421
179940,0migration/12007-21irqcore09:29:451
179940,0migration/119606-21sh09:39:431
179940,0migration/116740-21taskset08:58:181
179940,0migration/115881-21munin-node12:24:481
169940,0watchdog/17468-21tail09:14:441
169940,0watchdog/15706-21kthreadcore11:04:471
169940,0watchdog/13331-21ssh11:22:111
169940,0watchdog/131778-21kthreadcore09:09:481
169940,0watchdog/130181-21diskmemload11:45:551
169940,0watchdog/126502-21python07:49:521
169940,0watchdog/120405-21vmstat10:14:521
169940,0watchdog/119369-21kthreadcore08:34:481
169940,0watchdog/118523-21kthreadcore11:29:471
169940,0watchdog/112179-21perl11:09:431
169940,0watchdog/10-21swapper/111:39:431
169940,0watchdog/10-21swapper/110:41:001
169940,0watchdog/10-21swapper/110:22:281
169940,0watchdog/10-21swapper/109:55:361
320809932,0cyclictest4553-21sort08:49:442
320739933,0cyclictest0-21swapper/107:21:061
320739932,0cyclictest16138-21awk08:09:461
320739930,0cyclictest1225-21diskstats07:34:431
320719933,0cyclictest17637-21kworker/0:008:24:430
320719933,0cyclictest17637-21kworker/0:007:59:310
320719933,0cyclictest17637-21kworker/0:007:54:450
320719933,0cyclictest17637-21kworker/0:007:14:380
320719932,0cyclictest28461-21munin-node08:19:390
29725030,0irq/27-eth0-tx-0-21swapper/007:49:410
29715030,0irq/26-eth0-rx-0-21swapper/308:34:433
29715030,0irq/26-eth0-rx-0-21swapper/308:14:413
29715030,0irq/26-eth0-rx-0-21swapper/308:09:113
29715030,0irq/26-eth0-rx-0-21swapper/307:57:223
259930,0migration/221405-21fschecks_time07:49:432
259930,0migration/218614-21missed_timers07:19:482
249930,0watchdog/20-21swapper/207:39:482
249930,0watchdog/20-21swapper/207:14:442
169930,0watchdog/10-21swapper/111:59:471
169930,0watchdog/10-21swapper/109:04:401
169930,0watchdog/10-21swapper/108:44:401
320739921,0cyclictest21712-21latency_hist08:14:301
29715020,0irq/26-eth0-rx-0-21swapper/308:49:463
29715020,0irq/26-eth0-rx-0-21swapper/308:39:423
29715020,0irq/26-eth0-rx-0-21swapper/307:44:463
29715020,0irq/26-eth0-rx-0-21swapper/307:14:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional