You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 16:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Jun 12, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2735126463,0sleep00-21swapper/006:51:440
304450350,0irq/25-eth10-21swapper/306:48:313
2721223223,0sleep10-21swapper/106:50:051
2718122723,0sleep20-21swapper/206:49:462
339960,0migration/37990-21sh08:59:003
339960,0migration/37218-21sh12:22:303
339960,0migration/36156-21sh11:58:453
339960,0migration/329273-21munin-node09:33:133
339960,0migration/324432-21sh11:32:573
339960,0migration/323885-21sh11:11:103
339960,0migration/317599-21sh09:47:433
339960,0migration/316912-21sh09:05:423
339960,0migration/316695-21sh10:26:503
339960,0migration/313556-21missed_timers11:43:123
339960,0migration/313556-21missed_timers11:43:123
339960,0migration/312918-21kthreadcore11:03:113
29725060,0irq/27-eth0-tx-17381-21munin-node07:58:150
29715060,0irq/26-eth0-rx-23080-21sendmail_mailst10:28:143
29715060,0irq/26-eth0-rx-0-21swapper/311:54:283
29715060,0irq/26-eth0-rx-0-21swapper/309:38:543
275279960,0cyclictest0-21swapper/012:10:430
259960,0migration/232324-21sh09:13:192
149960,0migration/08176-21sh11:22:460
149960,0migration/03345-21kthreadcore06:58:120
149960,0migration/031330-21sh11:14:250
149960,0migration/030234-21sh12:13:430
149960,0migration/027287-21kernelversion09:53:080
149960,0migration/024334-21sh11:12:290
149960,0migration/016978-21sh09:45:550
149960,0migration/015910-21sh10:24:310
8215050,0irq/24-0000:00:9525-21cat10:02:542
8215050,0irq/24-0000:00:25845-21latency_hist09:32:542
339950,0migration/39593-21taskset07:03:093
339950,0migration/37506-21switchtime09:58:163
339950,0migration/34664-21fschecks_time07:48:093
339950,0migration/331728-21munin-node09:13:143
339950,0migration/328838-21timerandwakeup07:38:153
339950,0migration/328163-21kthreadcore08:53:103
339950,0migration/32785-21munin-node09:18:073
339950,0migration/325549-21latency_hist10:12:543
339950,0migration/323659-21open_files09:28:133
339950,0migration/320917-21kthreadcore07:33:133
339950,0migration/320421-21kthreadcore07:58:113
339950,0migration/317777-21kthreadcore08:43:133
339950,0migration/316849-21fschecks_time11:48:063
339950,0migration/315083-21taskset11:24:383
29725050,0irq/27-eth0-tx-9036-21munin-node10:43:152
29725050,0irq/27-eth0-tx-8143-21munin-node11:43:132
29725050,0irq/27-eth0-tx-8143-21munin-node11:43:132
29725050,0irq/27-eth0-tx-7258-21ssh10:38:342
29725050,0irq/27-eth0-tx-313-21munin-node11:18:082
29725050,0irq/27-eth0-tx-17152-21df_inode11:08:042
29725050,0irq/27-eth0-tx-1564-21ssh11:38:092
29725050,0irq/27-eth0-tx-1280-21munin-node10:18:122
29725050,0irq/27-eth0-tx-11732-21cat09:43:102
29725050,0irq/27-eth0-tx-10009-21munin-node09:23:082
29725050,0irq/27-eth0-tx-0-21swapper/212:18:082
29725050,0irq/27-eth0-tx-0-21swapper/211:23:122
29725050,0irq/27-eth0-tx-0-21swapper/211:01:182
29725050,0irq/27-eth0-tx-0-21swapper/210:48:082
29725050,0irq/27-eth0-tx-0-21swapper/209:41:002
29725050,0irq/27-eth0-tx-0-21swapper/008:38:050
29725050,0irq/27-eth0-tx-0-21swapper/008:10:190
29715050,0irq/26-eth0-rx-6280-21pmu-power10:58:123
29715050,0irq/26-eth0-rx-5295-21tr11:38:103
29715050,0irq/26-eth0-rx-29490-21hdparm07:18:093
29715050,0irq/26-eth0-rx-26752-21kthreadcore11:13:103
29715050,0irq/26-eth0-rx-20644-21kthreadcore12:08:103
29715050,0irq/26-eth0-rx-13279-21kthreadcore07:53:133
29715050,0irq/26-eth0-rx-0-21swapper/311:18:303
29715050,0irq/26-eth0-rx-0-21swapper/310:53:103
29715050,0irq/26-eth0-rx-0-21swapper/310:48:083
29715050,0irq/26-eth0-rx-0-21swapper/310:46:433
29715050,0irq/26-eth0-rx-0-21swapper/310:42:293
29715050,0irq/26-eth0-rx-0-21swapper/310:19:063
29715050,0irq/26-eth0-rx-0-21swapper/310:08:093
29715050,0irq/26-eth0-rx-0-21swapper/310:04:253
29715050,0irq/26-eth0-rx-0-21swapper/309:56:233
29715050,0irq/26-eth0-rx-0-21swapper/309:49:323
29715050,0irq/26-eth0-rx-0-21swapper/309:23:543
275299955,0cyclictest82150irq/24-0000:00:06:58:142
275299955,0cyclictest25749-21diskmemload10:54:372
275299954,0cyclictest3514-21snmpd08:50:502
275299954,0cyclictest3514-21snmpd08:43:502
275299954,0cyclictest3514-21snmpd07:21:502
275299950,0cyclictest0-21swapper/212:14:232
275299950,0cyclictest0-21swapper/209:09:372
275279955,0cyclictest0-21swapper/011:43:170
275279955,0cyclictest0-21swapper/011:43:170
275279955,0cyclictest0-21swapper/011:03:140
275279954,0cyclictest25067-21munin-run10:32:540
275279950,0cyclictest0-21swapper/009:04:590
275279950,0cyclictest0-21swapper/008:15:570
259950,0migration/27014-21missed_timers08:58:132
259950,0migration/231250-21sh11:35:062
259950,0migration/230581-21munin-node07:43:082
259950,0migration/224520-21sh09:29:092
259950,0migration/221456-21kthreadcore10:08:092
259950,0migration/219092-21kthreadcore11:28:102
259950,0migration/217061-21sh10:06:492
259950,0migration/213294-21kthreadcore07:03:132
259950,0migration/210010-21cpuspeed_turbos07:28:062
179950,0migration/1894-21sh09:36:191
179950,0migration/1837-21sh09:15:541
179950,0migration/17910-21taskset10:40:291
179950,0migration/14008-21latency_hist07:47:541
179950,0migration/13889-21kthreadcore11:18:101
179950,0migration/132329-21sh10:35:501
179950,0migration/128621-21munin-node06:53:081
179950,0migration/120851-21kthreadcore09:08:111
179950,0migration/118384-21fschecks_count09:48:061
169950,0watchdog/128224-21cut08:28:121
169950,0watchdog/123309-21irqstats07:13:131
149950,0migration/04636-21missed_timers08:33:130
149950,0migration/032214-21kthreadcore06:53:110
149950,0migration/03092-21munin-node07:23:080
149950,0migration/030715-21kthreadcore09:33:130
149950,0migration/030272-21kthreadcore10:33:120
149950,0migration/030117-21kthreadcore08:53:110
149950,0migration/02945-21kthreadcore08:58:110
149950,0migration/02924-21munin-node09:18:080
149950,0migration/029238-21taskset07:40:240
149950,0migration/028393-21kthreadcore08:28:120
149950,0migration/028304-21taskset07:15:540
149950,0migration/027804-21kthreadcore09:13:100
149950,0migration/027707-21kthreadcore10:13:090
149950,0migration/024777-21taskset09:09:590
149950,0migration/024586-21fschecks_time08:03:080
149950,0migration/02307-21kthreadcore11:38:090
149950,0migration/018641-21fschecks_time09:48:070
149950,0migration/018310-21munin-node10:48:070
149950,0migration/015168-21taskset07:28:150
149950,0migration/014026-21munin-node11:23:140
149950,0migration/011556-21idleruntime07:53:110
149950,0migration/011449-21kthreadcore10:43:120
8215040,0irq/24-0000:00:0-21swapper/208:32:542
339940,0migration/35621-21taskset08:37:063
339940,0migration/328518-21kthreadcore08:28:123
339940,0migration/326447-21kthreadcore12:13:093
339940,0migration/322079-21aten_r1power_en07:13:073
339940,0migration/317436-21kthreadcore07:08:143
339940,0migration/313409-21kthreadcore07:28:123
339940,0migration/310313-21kthreadcore08:38:103
29725040,0irq/27-eth0-tx-0-21swapper/212:08:532
29725040,0irq/27-eth0-tx-0-21swapper/211:58:072
29725040,0irq/27-eth0-tx-0-21swapper/211:48:452
29725040,0irq/27-eth0-tx-0-21swapper/211:13:052
29725040,0irq/27-eth0-tx-0-21swapper/210:13:102
29725040,0irq/27-eth0-tx-0-21swapper/209:53:362
29725040,0irq/27-eth0-tx-0-21swapper/209:48:142
29725040,0irq/27-eth0-tx-0-21swapper/007:48:130
29715040,0irq/26-eth0-rx-0-21swapper/312:03:573
29715040,0irq/26-eth0-rx-0-21swapper/311:28:063
29715040,0irq/26-eth0-rx-0-21swapper/310:36:183
29715040,0irq/26-eth0-rx-0-21swapper/309:08:123
29715040,0irq/26-eth0-rx-0-21swapper/308:03:123
275299943,0cyclictest8809-21awk12:03:082
275299943,0cyclictest4896-21awk07:48:102
275299943,0cyclictest3556-21lldpd08:34:142
275299943,0cyclictest3556-21lldpd07:57:132
275299943,0cyclictest3556-21lldpd06:56:132
275299943,0cyclictest3514-21snmpd08:40:502
275299943,0cyclictest26290-21awk10:33:092
275299943,0cyclictest25031-21awk11:53:062
275299943,0cyclictest18220-21awk10:28:072
275299943,0cyclictest10807-21awk09:03:102
275289943,0cyclictest3976-21awk07:23:121
275289943,0cyclictest3556-21lldpd10:21:151
275289943,0cyclictest3514-21snmpd12:05:201
275289943,0cyclictest3514-21snmpd11:40:201
275289943,0cyclictest3514-21snmpd11:28:201
275289943,0cyclictest3514-21snmpd11:16:201
275289943,0cyclictest3514-21snmpd11:08:201
275289943,0cyclictest3514-21snmpd09:44:201
275289943,0cyclictest3514-21snmpd09:18:201
275289943,0cyclictest3514-21snmpd07:08:201
275289943,0cyclictest23921-21awk07:38:071
275289943,0cyclictest11003-21tail10:03:091
275289943,0cyclictest10549-21awk07:28:091
275279944,0cyclictest31642-21kthreadcore07:43:100
275279944,0cyclictest0-21swapper/012:07:520
275279944,0cyclictest0-21swapper/010:05:030
275279943,0cyclictest391-21awk12:18:070
275279943,0cyclictest2726-21awk09:38:080
275279943,0cyclictest25938-21awk10:53:080
275279943,0cyclictest25153-21awk11:53:070
259940,0migration/29477-21gltestperf11:03:082
259940,0migration/26521-21kthreadcore09:18:102
259940,0migration/24258-21irqrtprio07:23:122
259940,0migration/229749-21kthreadcore08:53:112
259940,0migration/223032-21aten_r1power_cu07:38:032
259940,0migration/219116-21runrttasks08:23:092
249940,0watchdog/29300-21ssh10:22:562
249940,0watchdog/24041-21switchtime08:08:162
249940,0watchdog/20-21swapper/208:13:082
249940,0watchdog/20-21swapper/207:33:122
179940,0migration/18608-21kthreadcore08:38:101
179940,0migration/14062-21kthreadcore08:58:111
179940,0migration/131910-21runrttasks12:17:591
179940,0migration/130711-21irqcore07:43:091
179940,0migration/128884-21kthreadcore08:53:111
179940,0migration/127990-21kthreadcore11:53:091
179940,0migration/124891-21if_eth1.212:13:071
179940,0migration/124788-21hddtemp_smartct08:03:081
179940,0migration/123821-21kthreadcore08:48:121
179940,0migration/123071-21runrttasks12:11:141
179940,0migration/118871-21fschecks_count08:23:081
179940,0migration/118870-21kthreadcore10:28:091
179940,0migration/117934-21kthreadcore11:48:101
179940,0migration/116272-21runrttasks11:07:181
179940,0migration/115573-21sh11:47:011
179940,0migration/115573-21sh11:47:011
179940,0migration/114577-21kthreadcore08:43:121
179940,0migration/110519-21fschecks_count09:23:071
179940,0migration/110050-21kthreadcore07:03:121
169940,0watchdog/18055-21ssh11:01:521
169940,0watchdog/16512-21kthreadcore06:58:131
169940,0watchdog/15752-21cut08:13:081
169940,0watchdog/120428-21tr10:48:081
169940,0watchdog/111841-21kthreadcore11:23:121
169940,0watchdog/111740-21iostat_ios07:53:121
169940,0watchdog/10-21swapper/111:58:071
169940,0watchdog/10-21swapper/111:33:081
169940,0watchdog/10-21swapper/110:23:121
169940,0watchdog/10-21swapper/109:38:441
169940,0watchdog/10-21swapper/109:03:441
169940,0watchdog/10-21swapper/107:33:121
149940,0migration/0887-21kthreadcore07:18:140
149940,0migration/029440-21kthreadcore11:33:120
149940,0migration/019687-21kthreadcore09:28:100
149940,0migration/019623-21kthreadcore07:08:150
149940,0migration/018974-21irqrtprio10:08:080
149940,0migration/016580-21forks11:48:060
149940,0migration/012104-21kthreadcore07:03:120
339930,0migration/38411-21runrttasks07:02:173
29725030,0irq/27-eth0-tx-0-21swapper/007:33:060
29715030,0irq/26-eth0-rx-0-21swapper/308:48:063
29715030,0irq/26-eth0-rx-0-21swapper/308:23:063
29715030,0irq/26-eth0-rx-0-21swapper/308:18:053
29715030,0irq/26-eth0-rx-0-21swapper/308:12:563
29715030,0irq/26-eth0-rx-0-21swapper/308:08:053
29715030,0irq/26-eth0-rx-0-21swapper/307:46:083
29715030,0irq/26-eth0-rx-0-21swapper/307:27:323
29715030,0irq/26-eth0-rx-0-21swapper/306:53:143
275299933,0cyclictest3556-21lldpd07:09:132
275299932,0cyclictest3556-21lldpd07:13:132
275299932,0cyclictest24969-21awk08:03:102
275299932,0cyclictest18313-21awk07:58:092
275299932,0cyclictest12596-21awk08:18:092
275289932,0cyclictest31469-21sensors10:13:141
275289931,0cyclictest20534-21kthreadcore07:58:111
275289930,0cyclictest4703-21nfs4_client08:33:141
275289930,0cyclictest12409-21perl08:18:071
275279933,0cyclictest4698-21kworker/0:109:23:030
275279933,0cyclictest4698-21kworker/0:108:49:470
275279933,0cyclictest4698-21kworker/0:108:43:110
275279933,0cyclictest18871-21kworker/0:010:57:550
275279933,0cyclictest18871-21kworker/0:010:17:570
275279933,0cyclictest18871-21kworker/0:009:58:310
275279933,0cyclictest1161-21kworker/0:111:58:050
275279933,0cyclictest1161-21kworker/0:111:27:570
275279932,0cyclictest920-21munin-node10:38:090
275279930,0cyclictest18341-21munin-node08:23:080
275279930,0cyclictest0-21swapper/008:18:130
179930,0migration/127251-21irqrtprio09:53:081
179930,0migration/119055-21kthreadcore10:08:091
169930,0watchdog/10-21swapper/110:53:041
169930,0watchdog/10-21swapper/110:44:401
169930,0watchdog/10-21swapper/109:57:561
169930,0watchdog/10-21swapper/109:28:121
169930,0watchdog/10-21swapper/108:08:161
169930,0watchdog/10-21swapper/107:18:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional