You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-20 - 00:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2s.osadl.org (updated Mon Oct 19, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1341628483,0sleep00-21swapper/006:38:080
1343825038,0sleep10-21swapper/106:38:211
1335423728,0sleep30-21swapper/306:37:233
1335322622,0sleep20-21swapper/206:37:222
137019999,0cyclictest3514-21snmpd09:53:391
339960,0migration/330378-21sh11:51:213
339960,0migration/324855-21munin-node10:06:003
339960,0migration/324855-21munin-node10:06:003
339960,0migration/323920-21sh11:07:363
339960,0migration/319347-21sh08:44:203
339960,0migration/314081-21munin-node11:40:573
339960,0migration/31112-21sh09:51:153
29725060,0irq/27-eth0-tx-31126-21munin-node08:30:500
29715060,0irq/26-eth0-rx-8087-21ssh10:57:443
29715060,0irq/26-eth0-rx-22862-21yum11:26:023
259960,0migration/212036-21sh09:00:372
149960,0migration/019309-21sh08:44:140
149960,0migration/01892-21sh09:32:050
149960,0migration/014415-21sh11:41:090
149960,0migration/012633-21irqrtprio09:40:520
149960,0migration/010079-21sh09:59:080
137019960,0cyclictest0-21swapper/112:05:551
137009960,0cyclictest0-21swapper/010:51:540
8215050,0irq/24-0000:00:8522-21latency_hist11:20:362
8215050,0irq/24-0000:00:27178-21cat09:50:372
4150,0ktimersoftd/00-21swapper/009:10:410
35150,0ktimersoftd/30-21swapper/310:27:323
35150,0ktimersoftd/30-21swapper/309:45:193
35150,0ktimersoftd/30-21swapper/308:35:563
339950,0migration/37419-21runrttasks11:17:213
339950,0migration/33596-21timerwakeupswit08:05:593
339950,0migration/331460-21kthreadcore10:10:543
339950,0migration/331445-21kthreadcore10:30:533
339950,0migration/329596-21taskset08:02:023
339950,0migration/32920-21runrttasks08:53:013
339950,0migration/329184-21munin-node09:10:503
339950,0migration/328117-21latency_hist07:15:373
339950,0migration/325352-21munin-node09:25:583
339950,0migration/323204-21kthreadcore08:45:543
339950,0migration/321868-21kthreadcore08:20:543
339950,0migration/317712-21latency_hist10:45:363
339950,0migration/316267-21sh10:41:453
339950,0migration/313232-21kthreadcore10:20:523
339950,0migration/312808-21fschecks_count09:00:503
339950,0migration/310937-21hwlatdetect07:50:533
29725050,0irq/27-eth0-tx-8190-21munin-node07:00:560
29725050,0irq/27-eth0-tx-28349-21munin-node09:10:482
29725050,0irq/27-eth0-tx-25992-21open_files09:05:572
29725050,0irq/27-eth0-tx-24721-21timerandwakeup10:05:592
29725050,0irq/27-eth0-tx-24721-21timerandwakeup10:05:592
29725050,0irq/27-eth0-tx-18334-21munin-node10:25:572
29725050,0irq/27-eth0-tx-16736-21munin-node11:25:502
29725050,0irq/27-eth0-tx-0-21swapper/211:33:162
29725050,0irq/27-eth0-tx-0-21swapper/210:00:532
29725050,0irq/27-eth0-tx-0-21swapper/008:20:550
29725050,0irq/27-eth0-tx-0-21swapper/008:10:590
29725050,0irq/27-eth0-tx-0-21swapper/008:05:560
29715050,0irq/26-eth0-rx-8654-21munin-node12:00:513
29715050,0irq/26-eth0-rx-8196-21python10:15:583
29715050,0irq/26-eth0-rx-32379-21df11:55:473
29715050,0irq/26-eth0-rx-29029-21cut09:30:523
29715050,0irq/26-eth0-rx-26516-21kthreadcore11:10:513
29715050,0irq/26-eth0-rx-24874-21ntp_offset09:45:573
29715050,0irq/26-eth0-rx-0-21swapper/311:30:523
29715050,0irq/26-eth0-rx-0-21swapper/311:22:083
29715050,0irq/26-eth0-rx-0-21swapper/311:04:333
29715050,0irq/26-eth0-rx-0-21swapper/310:50:543
29715050,0irq/26-eth0-rx-0-21swapper/310:00:493
29715050,0irq/26-eth0-rx-0-21swapper/309:36:383
29715050,0irq/26-eth0-rx-0-21swapper/309:20:443
29715050,0irq/26-eth0-rx-0-21swapper/309:05:473
29715050,0irq/26-eth0-rx-0-21swapper/307:45:483
259950,0migration/26932-21sh11:59:072
259950,0migration/26885-21kthreadcore09:55:562
259950,0migration/231385-21kthreadcore09:30:532
259950,0migration/23038-21kthreadcore06:55:532
259950,0migration/230143-21kthreadcore10:10:532
259950,0migration/227881-21kthreadcore08:00:552
259950,0migration/226560-21idleruntime10:50:512
259950,0migration/219168-21latency_hist09:45:372
259950,0migration/216739-21kthreadcore09:00:542
259950,0migration/216164-21df_abs07:30:522
259950,0migration/214949-21kthreadcore10:40:522
259950,0migration/213598-21munin-node11:40:532
259950,0migration/212121-21latency_hist08:40:372
249950,0watchdog/217589-21munin-node07:55:512
249950,0watchdog/213170-21missed_timers12:00:542
249950,0watchdog/20-21swapper/208:50:552
179950,0migration/19227-21taskset10:18:031
179950,0migration/126182-21munin-node06:45:571
179950,0migration/116408-21munin-node10:00:551
169950,0watchdog/16154-21perl08:35:511
169950,0watchdog/129307-21if_eth1.209:10:511
169950,0watchdog/10-21swapper/111:45:541
169950,0watchdog/10-21swapper/111:25:541
149950,0migration/07935-21munin-node10:35:590
149950,0migration/07231-21taskset12:00:030
149950,0migration/04861-21kthreadcore07:45:540
149950,0migration/031280-21kthreadcore08:50:540
149950,0migration/030958-21kthreadcore07:15:520
149950,0migration/030469-21taskset11:51:380
149950,0migration/029105-21kthreadcore11:10:520
149950,0migration/027643-21taskset08:49:310
149950,0migration/026857-21kthreadcore07:35:540
149950,0migration/025815-21munin-node09:05:550
149950,0migration/025350-21munin-node08:25:520
149950,0migration/025333-21kthreadcore07:10:550
149950,0migration/023541-21aten_r1power_fr08:00:490
149950,0migration/023025-21taskset07:58:150
149950,0migration/022265-21ntp_offset11:25:570
149950,0migration/020915-21sh09:45:540
149950,0migration/020767-21kthreadcore12:05:550
149950,0migration/019961-21irqcore10:05:520
149950,0migration/019961-21irqcore10:05:520
149950,0migration/018407-21kthreadcore06:40:540
149950,0migration/018156-21kthreadcore11:45:530
149950,0migration/014481-21missed_timers07:25:550
149950,0migration/012397-21kthreadcore10:00:530
149950,0migration/012333-21kthreadcore10:20:520
137029950,0cyclictest0-21swapper/210:36:052
137019955,0cyclictest18055-21sensors09:00:581
137019955,0cyclictest0-21swapper/109:44:491
137019955,0cyclictest0-21swapper/109:31:531
137019955,0cyclictest0-21swapper/107:20:521
137019950,0cyclictest0-21swapper/111:20:501
137009955,0cyclictest6559-21runrttasks11:34:190
137009955,0cyclictest0-21swapper/010:11:040
137009950,0cyclictest0-21swapper/011:21:230
339940,0migration/36923-21kthreadcore07:20:563
339940,0migration/36197-21kthreadcore09:55:563
339940,0migration/35673-21irqrtprio08:10:523
339940,0migration/331181-21kthreadcore07:40:533
339940,0migration/329936-21kthreadcore06:50:543
339940,0migration/326159-21kthreadcore08:25:543
339940,0migration/325096-21kthreadcore07:35:533
339940,0migration/323391-21kthreadcore06:45:533
339940,0migration/322517-21irqcore07:10:533
339940,0migration/316242-21kthreadcore06:40:533
29725040,0irq/27-eth0-tx-32704-21ssh10:31:392
29725040,0irq/27-eth0-tx-32622-21ssh11:35:472
29725040,0irq/27-eth0-tx-0-21swapper/211:50:432
29725040,0irq/27-eth0-tx-0-21swapper/211:15:462
29725040,0irq/27-eth0-tx-0-21swapper/210:55:382
29725040,0irq/27-eth0-tx-0-21swapper/210:47:452
29725040,0irq/27-eth0-tx-0-21swapper/210:20:572
29725040,0irq/27-eth0-tx-0-21swapper/209:26:592
29725040,0irq/27-eth0-tx-0-21swapper/209:15:532
29725040,0irq/27-eth0-tx-0-21swapper/208:45:482
29725040,0irq/27-eth0-tx-0-21swapper/008:38:240
29725040,0irq/27-eth0-tx-0-21swapper/007:30:580
29715040,0irq/26-eth0-rx-0-21swapper/312:06:313
29715040,0irq/26-eth0-rx-0-21swapper/311:45:543
29715040,0irq/26-eth0-rx-0-21swapper/311:37:003
29715040,0irq/26-eth0-rx-0-21swapper/310:35:543
29715040,0irq/26-eth0-rx-0-21swapper/309:15:503
29715040,0irq/26-eth0-rx-0-21swapper/308:56:073
29715040,0irq/26-eth0-rx-0-21swapper/307:55:543
259940,0migration/29255-21kthreadcore08:35:552
259940,0migration/25518-21kthreadcore09:35:522
259940,0migration/25096-21kthreadcore10:15:532
259940,0migration/230309-21cpu08:05:492
259940,0migration/229649-21kthreadcore08:25:562
259940,0migration/222910-21kthreadcore08:20:552
259940,0migration/218770-21kthreadcore11:45:532
259940,0migration/216405-21latency08:15:572
259940,0migration/21481-21kthreadcore07:40:542
259940,0migration/211289-21kthreadcore11:00:522
249940,0watchdog/23792-21kthreadcore07:20:552
249940,0watchdog/2313932sleep211:11:022
249940,0watchdog/220002-21perl09:45:512
249940,0watchdog/214516-21missed_timers07:25:552
249940,0watchdog/20-21swapper/212:05:542
249940,0watchdog/20-21swapper/211:05:582
249940,0watchdog/20-21swapper/209:20:552
179940,0migration/14688-21irqrtprio07:45:531
179940,0migration/14102-21kthreadcore11:35:541
179940,0migration/130849-21kthreadcore06:50:551
179940,0migration/130629-21taskset11:52:081
179940,0migration/129281-21kthreadcore10:50:541
179940,0migration/128205-21kthreadcore11:30:531
179940,0migration/124626-21aten_r1power_po08:25:491
179940,0migration/124293-21runrttasks11:08:441
179940,0migration/124061-21kthreadcore07:10:541
179940,0migration/122770-21kthreadcore09:05:541
179940,0migration/122542-21kthreadcore09:25:561
179940,0migration/120982-21kthreadcore10:05:531
179940,0migration/120982-21kthreadcore10:05:531
179940,0migration/120658-21sh09:45:541
179940,0migration/118396-21taskset08:41:271
179940,0migration/116005-21timerwakeupswit07:51:021
179940,0migration/115591-21gltestperf07:05:501
169940,0watchdog/18557-21munin-node12:00:501
169940,0watchdog/18327-21chrt11:20:061
169940,0watchdog/17348-21ls06:55:591
169940,0watchdog/14997-21hwlatdetect08:55:511
169940,0watchdog/117073-21kthreadcore09:20:551
169940,0watchdog/110065-21sh10:40:471
169940,0watchdog/10-21swapper/109:36:511
169940,0watchdog/10-21swapper/109:16:551
149940,0migration/06850-21kthreadcore07:20:560
149940,0migration/05092-21kthreadcore06:55:540
149940,0migration/04199-21kthreadcore10:15:530
149940,0migration/024530-21kthreadcore09:25:570
149940,0migration/022935-21kthreadcore10:25:530
149940,0migration/020683-21kthreadcore11:05:520
149940,0migration/012150-21kthreadcore07:50:560
137029943,0cyclictest3556-21lldpd08:13:082
137029943,0cyclictest3514-21snmpd07:55:092
137029943,0cyclictest3514-21snmpd07:46:092
137029943,0cyclictest3514-21snmpd07:00:392
137029943,0cyclictest28129-21awk06:50:522
137029943,0cyclictest23415-21awk07:35:512
137029943,0cyclictest21416-21awk06:45:512
137029943,0cyclictest15804-21awk07:05:522
137019943,0cyclictest3556-21lldpd07:39:381
137019943,0cyclictest3556-21lldpd07:29:381
137019943,0cyclictest3556-21lldpd07:00:381
137019943,0cyclictest3514-21snmpd11:55:551
137019943,0cyclictest3514-21snmpd10:45:401
137019943,0cyclictest3514-21snmpd10:31:401
137019943,0cyclictest3514-21snmpd10:00:251
137019943,0cyclictest3514-21snmpd08:31:251
137019943,0cyclictest3514-21snmpd08:08:391
137019943,0cyclictest3514-21snmpd08:01:541
137019943,0cyclictest3514-21snmpd07:43:391
137019943,0cyclictest3241-21irqbalance08:51:051
137019943,0cyclictest3241-21irqbalance08:20:551
137019943,0cyclictest27684-21awk10:10:511
137019943,0cyclictest26041-21sadc10:30:371
137019943,0cyclictest2189-21sadc07:20:371
137019943,0cyclictest2166-21awk10:55:521
137019943,0cyclictest20999-21awk08:45:521
137019943,0cyclictest12022-21awk08:15:531
137019943,0cyclictest10210-21awk11:00:501
137009944,0cyclictest0-21swapper/010:56:190
137009944,0cyclictest0-21swapper/010:49:420
137009944,0cyclictest0-21swapper/009:19:410
137009943,0cyclictest5099-21awk08:55:520
137009943,0cyclictest1161-21awk11:35:510
137009943,0cyclictest10772-21awk10:40:490
29715030,0irq/26-eth0-rx-0-21swapper/307:30:593
29715030,0irq/26-eth0-rx-0-21swapper/307:25:423
29715030,0irq/26-eth0-rx-0-21swapper/307:05:483
29715030,0irq/26-eth0-rx-0-21swapper/307:00:483
169930,0watchdog/10-21swapper/111:42:341
169930,0watchdog/10-21swapper/110:35:391
169930,0watchdog/10-21swapper/110:21:111
169930,0watchdog/10-21swapper/107:55:511
137029932,0cyclictest3514-21snmpd07:15:392
137029932,0cyclictest31269-21basename08:30:502
137029932,0cyclictest14750-21awk06:40:512
137029930,0cyclictest0-21swapper/207:10:552
137019932,0cyclictest3556-21lldpd08:12:381
137019932,0cyclictest3514-21snmpd07:30:391
137019932,0cyclictest25929-21grep11:10:481
137019932,0cyclictest14867-21awk06:40:511
137009933,0cyclictest3556-21lldpd07:05:370
137009933,0cyclictest20101-21kworker/0:011:15:400
137009933,0cyclictest20101-21kworker/0:009:35:400
137009933,0cyclictest10726-21kworker/0:109:00:440
137009933,0cyclictest10726-21kworker/0:108:15:460
137009933,0cyclictest10726-21kworker/0:107:40:420
137009933,0cyclictest10726-21kworker/0:106:45:510
137009933,0cyclictest0-21swapper/011:00:440
137009932,0cyclictest8604-21awk12:00:510
137009932,0cyclictest3556-21lldpd09:20:380
137009932,0cyclictest31563-21kthreadcore10:30:530
137009932,0cyclictest28250-21awk09:50:520
137009930,0cyclictest27221-21munin-node06:51:000
29715020,0irq/26-eth0-rx-0-21swapper/308:30:573
29715020,0irq/26-eth0-rx-0-21swapper/308:16:013
29715020,0irq/26-eth0-rx-0-21swapper/306:55:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional