You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 21:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Apr 03, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
861728180,0sleep10-21swapper/106:58:241
837323724,0sleep00-21swapper/006:55:350
304450350,0irq/25-eth10-21swapper/306:56:133
856123424,0sleep20-21swapper/206:57:452
87849966,0cyclictest3514-21snmpd07:36:171
87849965,0cyclictest29135-21ls11:54:311
87849960,0cyclictest12987-21sed08:14:471
87839960,0cyclictest0-21swapper/012:23:240
87839960,0cyclictest0-21swapper/011:44:180
339960,0migration/37865-21gltestperf09:59:463
339960,0migration/330000-21sh10:54:053
339960,0migration/329815-21sh09:10:353
339960,0migration/312101-21sh11:20:233
339960,0migration/311428-21runrttasks12:00:273
29725060,0irq/27-eth0-tx-9002-21munin-node06:59:480
29725060,0irq/27-eth0-tx-13281-21unixbench_singl08:59:550
29715060,0irq/26-eth0-rx-9166-21cut10:59:483
29715060,0irq/26-eth0-rx-31092-21perl10:54:433
29715060,0irq/26-eth0-rx-28101-21ssh11:30:223
149960,0migration/05946-21sh10:39:010
149960,0migration/030591-21latency_hist10:34:310
149960,0migration/020770-21taskset11:48:230
149960,0migration/019018-21sh12:25:270
149960,0migration/014405-21kthreadcore08:39:480
149960,0migration/014104-21sh10:44:250
87869955,0cyclictest0-21swapper/211:17:122
87869950,0cyclictest0-21swapper/212:10:202
87869950,0cyclictest0-21swapper/210:43:562
87869950,0cyclictest0-21swapper/210:15:432
87869950,0cyclictest0-21swapper/208:36:432
87839955,0cyclictest0-21swapper/011:37:310
87839955,0cyclictest0-21swapper/008:29:540
87839950,0cyclictest0-21swapper/011:08:490
87839950,0cyclictest0-21swapper/009:14:310
8215050,0irq/24-0000:00:7258-21cat09:39:312
8215050,0irq/24-0000:00:20429-21basename12:29:312
339950,0migration/39880-21kthreadcore10:39:483
339950,0migration/3381-21taskset08:54:393
339950,0migration/332183-21fschecks_count09:34:443
339950,0migration/330186-21munin-node11:34:423
339950,0migration/328417-21diskmemload12:13:303
339950,0migration/328064-21timerwakeupswit11:09:533
339950,0migration/327020-21kthreadcore10:09:483
339950,0migration/324815-21kthreadcore10:29:473
339950,0migration/324230-21idleruntime09:49:443
339950,0migration/323628-21kthreadcore07:09:473
339950,0migration/319981-21runrttasks12:07:163
339950,0migration/318371-21kthreadcore07:29:483
339950,0migration/318080-21kthreadcore11:04:473
339950,0migration/317316-21kthreadcore12:24:473
339950,0migration/315164-21df_abs10:24:443
339950,0migration/311029-21kthreadcore06:59:483
339950,0migration/310581-21kthreadcore09:19:493
29725050,0irq/27-eth0-tx-6696-21munin-node10:19:422
29725050,0irq/27-eth0-tx-5662-21cpuspeed_turbos11:39:432
29725050,0irq/27-eth0-tx-31592-21munin-node09:14:452
29725050,0irq/27-eth0-tx-29923-21munin-node07:39:520
29725050,0irq/27-eth0-tx-29787-21munin-node11:34:442
29725050,0irq/27-eth0-tx-25916-21ssh10:49:492
29725050,0irq/27-eth0-tx-23639-21ssh10:29:452
29725050,0irq/27-eth0-tx-15195-21ssh10:24:452
29725050,0irq/27-eth0-tx-13785-21munin-node11:24:512
29725050,0irq/27-eth0-tx-13094-21munin-node12:04:492
29725050,0irq/27-eth0-tx-11423-21ls11:19:512
29725050,0irq/27-eth0-tx-0-21swapper/212:19:432
29725050,0irq/27-eth0-tx-0-21swapper/210:06:232
29715050,0irq/26-eth0-rx-5694-21ssh09:15:433
29715050,0irq/26-eth0-rx-5449-21ssh11:39:423
29715050,0irq/26-eth0-rx-4993-21ssh10:15:243
29715050,0irq/26-eth0-rx-3612-21kthreadcore10:34:503
29715050,0irq/26-eth0-rx-28585-21kthreadcore09:29:483
29715050,0irq/26-eth0-rx-223702sleep309:47:183
29715050,0irq/26-eth0-rx-21523-21aten_r1power_fr11:49:413
29715050,0irq/26-eth0-rx-20193-21ssh11:25:443
29715050,0irq/26-eth0-rx-18028-21kthreadcore09:04:483
29715050,0irq/26-eth0-rx-14062-21tune2fs11:44:443
29715050,0irq/26-eth0-rx-12320-21munin-node10:19:493
29715050,0irq/26-eth0-rx-10345-21kthreadcore08:59:493
29715050,0irq/26-eth0-rx-0-21swapper/312:22:063
29715050,0irq/26-eth0-rx-0-21swapper/311:55:413
29715050,0irq/26-eth0-rx-0-21swapper/311:15:553
29715050,0irq/26-eth0-rx-0-21swapper/310:44:453
29715050,0irq/26-eth0-rx-0-21swapper/310:07:123
29715050,0irq/26-eth0-rx-0-21swapper/309:56:353
29715050,0irq/26-eth0-rx-0-21swapper/307:49:503
259950,0migration/27402-21latency_hist09:19:312
259950,0migration/23805-21munin-node10:34:502
259950,0migration/228245-21kthreadcore08:49:482
259950,0migration/227882-21munin-node11:09:522
259950,0migration/222244-21latency_hist07:09:322
259950,0migration/221196-21sh09:44:542
259950,0migration/213446-21sh11:03:302
249950,0watchdog/25940-21ssh09:36:182
249950,0watchdog/23270-21ssh12:16:212
249950,0watchdog/210950-21sort11:59:532
19150,0ktimersoftd/10-21swapper/111:25:141
179950,0migration/15812-21sh09:16:021
179950,0migration/15617-21latency_hist11:19:311
179950,0migration/127511-21kthreadcore07:59:491
179950,0migration/117783-21kthreadcore10:24:491
179950,0migration/116372-21irqcore09:04:471
179950,0migration/111824-21taskset12:22:401
169950,0watchdog/1364-21kthreadcore09:54:501
169950,0watchdog/113343-21ssh11:24:011
149950,0migration/08708-21kthreadcore09:59:490
149950,0migration/07839-21kthreadcore10:59:480
149950,0migration/05457-21taskset10:16:450
149950,0migration/030916-21taskset09:33:080
149950,0migration/029496-21fschecks_count12:14:440
149950,0migration/029385-21timerwakeupswit09:09:530
149950,0migration/029269-21latency_hist11:54:310
149950,0migration/029220-21latency_hist11:54:310
149950,0migration/028139-21unixbench-2d11:09:530
149950,0migration/027959-21runrttasks10:49:510
149950,0migration/024332-21irqcore07:34:470
149950,0migration/023450-21kthreadcore08:19:490
149950,0migration/023450-21kthreadcore08:19:490
149950,0migration/020335-21kthreadcore07:54:490
149950,0migration/018502-21missed_timers12:04:490
149950,0migration/018166-21kthreadcore10:24:490
149950,0migration/016773-21kthreadcore10:44:470
149950,0migration/016489-21irqrtprio09:04:470
149950,0migration/014401-21kthreadcore07:49:480
149950,0migration/013801-21aten_r1power_cu11:24:410
149950,0migration/013746-21taskset10:22:220
149950,0migration/01221-21kthreadcore10:54:470
87869943,0cyclictest3556-21lldpd08:39:432
87869943,0cyclictest3514-21snmpd08:45:122
87869943,0cyclictest32738-21awk08:29:462
87869943,0cyclictest3241-21irqbalance11:49:392
87869943,0cyclictest16686-21awk09:24:472
87869943,0cyclictest14404-21awk11:44:462
87869943,0cyclictest11774-21awk07:49:452
87869943,0cyclictest10884-21awk07:24:462
87849944,0cyclictest10313-21sort06:59:481
87849944,0cyclictest0-21swapper/110:01:021
87849943,0cyclictest8363-21awk09:19:461
87849943,0cyclictest3556-21lldpd12:04:441
87849943,0cyclictest3556-21lldpd11:01:151
87849943,0cyclictest3556-21lldpd10:56:151
87849943,0cyclictest3556-21lldpd10:48:151
87849943,0cyclictest3556-21lldpd10:19:451
87849943,0cyclictest349-21awk08:29:461
87849943,0cyclictest3241-21irqbalance11:46:191
87849943,0cyclictest31984-21awk10:14:461
87849943,0cyclictest29977-21awk12:14:461
87849943,0cyclictest24283-21awk09:49:451
87849943,0cyclictest23951-21awk10:09:451
87849943,0cyclictest20430-21awk08:44:451
87839944,0cyclictest0-21swapper/011:23:580
87839944,0cyclictest0-21swapper/010:12:040
87839944,0cyclictest0-21swapper/008:38:350
87839943,0cyclictest3514-21snmpd09:55:120
87839943,0cyclictest3514-21snmpd09:36:120
87839943,0cyclictest3514-21snmpd08:00:420
87839943,0cyclictest3514-21snmpd07:20:420
87839943,0cyclictest31048-21awk11:14:460
87839943,0cyclictest12907-21awk08:14:460
87839940,0cyclictest25201-21munin-node08:24:430
87839940,0cyclictest23429-21idleruntime-cro09:29:310
87839940,0cyclictest23149-21awk09:49:310
339940,0migration/39625-21missed_timers07:44:493
339940,0migration/37895-21kthreadcore08:09:483
339940,0migration/36083-21latency_hist08:34:323
339940,0migration/330340-21kthreadcore08:49:493
339940,0migration/329868-21munin-node07:59:533
339940,0migration/323800-21kthreadcore08:44:483
339940,0migration/318149-21fschecks_count07:54:443
339940,0migration/317054-21kthreadcore08:14:483
339940,0migration/315692-21kthreadcore08:39:483
339940,0migration/311708-21ntp_states08:34:493
29725040,0irq/27-eth0-tx-27763-21timerwakeupswit11:29:562
29725040,0irq/27-eth0-tx-27365-21kthreadcore10:09:482
29725040,0irq/27-eth0-tx-0-21swapper/211:54:362
29725040,0irq/27-eth0-tx-0-21swapper/211:04:422
29725040,0irq/27-eth0-tx-0-21swapper/210:54:412
29725040,0irq/27-eth0-tx-0-21swapper/209:54:442
29725040,0irq/27-eth0-tx-0-21swapper/209:09:322
29715040,0irq/26-eth0-rx-465-21kthreadcore07:14:483
29715040,0irq/26-eth0-rx-32210-21kthreadcore08:04:473
29715040,0irq/26-eth0-rx-0-21swapper/312:16:253
29715040,0irq/26-eth0-rx-0-21swapper/309:39:423
29715040,0irq/26-eth0-rx-0-21swapper/309:25:353
259940,0migration/29620-21missed_timers07:44:492
259940,0migration/25783-21latency08:54:492
259940,0migration/231696-21kthreadcore07:14:472
259940,0migration/228745-21kthreadcore09:49:482
259940,0migration/22818-21kthreadcore08:04:482
259940,0migration/22441-21kthreadcore07:39:482
259940,0migration/223999-21fschecks_time07:34:442
259940,0migration/223648-21sh09:29:352
259940,0migration/215290-21aten_r1power_en09:04:422
259940,0migration/214230-21kthreadcore06:59:492
249940,0watchdog/26976-21diskmemload09:59:382
249940,0watchdog/225259-21aten2.4-expect08:24:422
249940,0watchdog/214488-21cut10:44:412
249940,0watchdog/212615-21memory08:59:502
179940,0migration/18532-21kthreadcore07:44:481
179940,0migration/17827-21kthreadcore08:34:461
179940,0migration/17338-21aten_r1power_po08:59:431
179940,0migration/15370-21taskset10:37:201
179940,0migration/13824-21runrttasks07:42:171
179940,0migration/13220-21kthreadcore08:54:481
179940,0migration/130422-21fschecks_time11:34:431
179940,0migration/127487-21proc_pri11:29:541
179940,0migration/126990-21timerwakeupswit12:09:541
179940,0migration/124339-21kthreadcore10:49:481
179940,0migration/123772-21kthreadcore11:09:471
179940,0migration/122770-21runrttasks07:31:121
179940,0migration/122412-21kthreadcore07:54:501
179940,0migration/118286-21kthreadcore09:24:481
179940,0migration/115580-21latency_hist07:04:321
179940,0migration/112108-21kthreadcore07:24:481
179940,0migration/110831-21proc_pri08:09:511
179940,0migration/110500-21mdstat11:59:491
179940,0migration/110182-21kthreadcore10:39:481
169940,0watchdog/16976-21diskmemload10:31:101
169940,0watchdog/131698-21aten_r1power_en09:34:421
169940,0watchdog/130123-21ssh09:30:461
169940,0watchdog/119261-21ssh12:26:091
169940,0watchdog/1191982sleep108:43:101
169940,0watchdog/111966-21kthreadcore09:39:501
169940,0watchdog/10-21swapper/111:39:451
169940,0watchdog/10-21swapper/107:14:461
149940,0migration/09880-21kthreadcore09:19:480
149940,0migration/09199-21kthreadcore11:59:480
149940,0migration/019727-21kthreadcore07:29:490
149940,0migration/010817-21kthreadcore09:39:490
87869932,0cyclictest3514-21snmpd08:23:422
87869932,0cyclictest3514-21snmpd08:23:422
87869932,0cyclictest3514-21snmpd08:13:422
87869932,0cyclictest23277-21sensors07:54:532
87869932,0cyclictest17632-21awk07:29:472
87869932,0cyclictest16524-21awk07:04:452
87869930,0cyclictest0-21swapper/208:04:322
87849933,0cyclictest3556-21lldpd11:04:451
87849932,0cyclictest3556-21lldpd08:49:451
87849932,0cyclictest3556-21lldpd07:20:131
87849932,0cyclictest31787-21awk08:04:451
87849932,0cyclictest27979-21kthreadcore09:09:481
87849932,0cyclictest23154-21awk07:09:451
87839933,0cyclictest2987-21kworker/0:107:24:370
87839933,0cyclictest17607-21kworker/0:009:50:410
87839933,0cyclictest17607-21kworker/0:008:54:410
87839933,0cyclictest17607-21kworker/0:008:44:370
87839932,0cyclictest5243-21awk07:44:460
87839932,0cyclictest3514-21snmpd11:29:420
87839932,0cyclictest3514-21snmpd08:51:290
87839932,0cyclictest30907-21aten_r1power_cu08:04:410
87839932,0cyclictest26098-21grep07:09:480
87839930,0cyclictest21427-21diskstats12:09:440
87839930,0cyclictest0-21swapper/010:04:480
29725030,0irq/27-eth0-tx-0-21swapper/008:09:430
29725030,0irq/27-eth0-tx-0-21swapper/007:14:520
29725030,0irq/27-eth0-tx-0-21swapper/007:04:420
29715030,0irq/26-eth0-rx-0-21swapper/308:24:423
29715030,0irq/26-eth0-rx-0-21swapper/307:39:553
29715030,0irq/26-eth0-rx-0-21swapper/307:34:423
29715030,0irq/26-eth0-rx-0-21swapper/307:19:413
259930,0migration/27529-21kthreadcore07:19:472
249930,0watchdog/20-21swapper/208:14:462
169930,0watchdog/16976-21diskmemload10:06:341
169930,0watchdog/122111-21ssh09:46:341
169930,0watchdog/10-21swapper/111:54:371
169930,0watchdog/10-21swapper/108:24:421
87849921,0cyclictest10887-21wc07:49:321
339920,0migration/318955-21diskstats08:19:443
339920,0migration/318955-21diskstats08:19:443
29715020,0irq/26-eth0-rx-0-21swapper/307:24:323
29715020,0irq/26-eth0-rx-0-21swapper/307:04:503
169920,0watchdog/10-21swapper/108:19:341
169920,0watchdog/10-21swapper/108:19:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional