You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 00:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Jul 23, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450660,0irq/25-eth10-21swapper/306:46:183
117723425,0sleep20-21swapper/206:47:332
113623422,0sleep10-21swapper/106:47:031
94422921,0sleep00-21swapper/006:44:540
35160,0ktimersoftd/30-21swapper/311:53:003
339960,0migration/332380-21gltestperf09:29:123
339960,0migration/331533-21sh09:08:043
339960,0migration/325298-21sh12:06:253
339960,0migration/322267-21missed_timers07:49:173
339960,0migration/321602-21sh09:40:293
339960,0migration/314548-21sh08:55:593
339960,0migration/314431-21sh09:15:373
339960,0migration/313707-21sh09:35:303
339960,0migration/313272-21munin-node08:34:173
29715060,0irq/26-eth0-rx-5681-21ssh09:52:233
29715060,0irq/26-eth0-rx-1678-21python11:09:203
149960,0migration/05427-21sh09:51:350
13779965,0cyclictest2295-21grep06:49:122
13769966,0cyclictest32024-21diskmemload10:19:271
13769966,0cyclictest32024-21diskmemload09:35:281
13769960,0cyclictest0-21swapper/108:39:201
4150,0ktimersoftd/00-21swapper/010:28:210
339950,0migration/38972-21taskset07:18:213
339950,0migration/36500-21munin-node08:29:143
339950,0migration/34587-21kthreadcore08:49:173
339950,0migration/33595-21munin-node11:54:133
339950,0migration/332658-21kthreadcore07:59:153
339950,0migration/332114-21kthreadcore10:09:153
339950,0migration/330210-21munin-node08:19:193
339950,0migration/325772-21kthreadcore07:04:163
339950,0migration/324092-21taskset08:16:173
339950,0migration/322586-21munin-node10:24:133
339950,0migration/322308-21taskset08:59:593
339950,0migration/321817-21munin-node09:19:223
339950,0migration/321077-21sh10:01:013
339950,0migration/320176-21irqcore11:44:153
339950,0migration/317294-21kthreadcore06:59:173
339950,0migration/316920-21rt-features11:39:203
339950,0migration/313445-21kthreadcore07:44:163
339950,0migration/310102-21taskset11:14:263
29725050,0irq/27-eth0-tx-2963-21kthreadcore08:49:160
29725050,0irq/27-eth0-tx-2549-21munin-node07:14:170
29725050,0irq/27-eth0-tx-23581-21munin-node07:54:200
29725050,0irq/27-eth0-tx-13000-21switchtime09:34:192
29725050,0irq/27-eth0-tx-0-21swapper/008:14:200
29725050,0irq/27-eth0-tx-0-21swapper/008:00:240
29725050,0irq/27-eth0-tx-0-21swapper/007:31:410
29715050,0irq/26-eth0-rx-4828-21kthreadcore07:39:143
29715050,0irq/26-eth0-rx-2801-21tr09:09:163
29715050,0irq/26-eth0-rx-25535-21munin-node11:24:203
29715050,0irq/26-eth0-rx-24462-21kthreadcore10:44:163
29715050,0irq/26-eth0-rx-23351-21cut08:39:163
29715050,0irq/26-eth0-rx-18778-21cut10:39:193
29715050,0irq/26-eth0-rx-11867-21fschecks_time08:09:123
29715050,0irq/26-eth0-rx-0-21swapper/311:05:403
29715050,0irq/26-eth0-rx-0-21swapper/310:56:033
29715050,0irq/26-eth0-rx-0-21swapper/310:04:403
29715050,0irq/26-eth0-rx-0-21swapper/308:44:163
29715050,0irq/26-eth0-rx-0-21swapper/308:04:163
259950,0migration/231633-21sh09:08:202
249950,0watchdog/2495-21kthreadcore11:29:162
19150,0ktimersoftd/10-21swapper/111:35:551
179950,0migration/17979-21timerwakeupswit12:14:191
179950,0migration/129088-21gltestperf11:09:111
179950,0migration/126443-21aten_r1power_en12:09:071
179950,0migration/12349-21latency_hist07:13:591
179950,0migration/12349-21latency_hist07:13:591
179950,0migration/123211-21kthreadcore10:44:161
179950,0migration/122631-21kthreadcore11:04:151
179950,0migration/118058-21kthreadcore09:19:151
169950,0watchdog/130810-21kthreadcore11:49:161
169950,0watchdog/12470-21sensors_volt07:34:211
169950,0watchdog/121125-21kthreadcore08:14:171
149950,0migration/07072-21kthreadcore08:29:150
149950,0migration/06584-21gltestperf10:14:110
149950,0migration/05423-21munin-node10:34:090
149950,0migration/04308-21taskset10:32:060
149950,0migration/0395-21sh12:10:140
149950,0migration/031489-21irqrtprio10:09:150
149950,0migration/028335-21fschecks_time11:29:120
149950,0migration/019145-21runrttasks11:00:560
149950,0migration/019118-21taskset11:22:530
149950,0migration/018304-21taskset08:34:390
149950,0migration/018140-21kthreadcore09:39:160
149950,0migration/017456-21kthreadcore06:59:170
149950,0migration/017400-21kthreadcore08:59:150
149950,0migration/016878-21latency_hist07:48:580
149950,0migration/016781-21sh11:59:290
149950,0migration/015168-21sh09:17:540
149950,0migration/015124-21timerwakeupswit07:19:210
149950,0migration/01499-21munin-node11:09:180
149950,0migration/012469-21timerwakeupswit09:54:230
149950,0migration/011137-21sh10:55:420
149950,0migration/010138-21taskset11:14:320
13769955,0cyclictest0-21swapper/106:59:041
13769950,0cyclictest0-21swapper/110:50:211
13769950,0cyclictest0-21swapper/109:53:451
13769950,0cyclictest0-21swapper/109:30:441
13749954,0cyclictest14850-21munin-run09:38:580
8215040,0irq/24-0000:00:0-21swapper/208:48:582
339940,0migration/331203-21kthreadcore07:09:163
339940,0migration/331203-21kthreadcore07:09:163
339940,0migration/330234-21iostat07:34:163
339940,0migration/329304-21kthreadcore11:29:153
339940,0migration/323733-21sh09:24:093
339940,0migration/315641-21taskset07:23:293
339940,0migration/314862-21latency_hist06:59:003
29725040,0irq/27-eth0-tx-32024-21diskmemload10:25:022
29725040,0irq/27-eth0-tx-32024-21diskmemload09:19:342
29725040,0irq/27-eth0-tx-26563-21ssh11:26:262
29725040,0irq/27-eth0-tx-0-21swapper/212:14:102
29725040,0irq/27-eth0-tx-0-21swapper/211:43:552
29725040,0irq/27-eth0-tx-0-21swapper/211:19:172
29725040,0irq/27-eth0-tx-0-21swapper/210:45:172
29725040,0irq/27-eth0-tx-0-21swapper/210:33:442
29725040,0irq/27-eth0-tx-0-21swapper/210:15:072
29725040,0irq/27-eth0-tx-0-21swapper/208:59:572
29715040,0irq/26-eth0-rx-0-21swapper/312:14:183
29715040,0irq/26-eth0-rx-0-21swapper/312:10:093
29715040,0irq/26-eth0-rx-0-21swapper/311:59:433
29715040,0irq/26-eth0-rx-0-21swapper/311:34:063
29715040,0irq/26-eth0-rx-0-21swapper/311:19:173
29715040,0irq/26-eth0-rx-0-21swapper/310:58:583
29715040,0irq/26-eth0-rx-0-21swapper/310:49:013
29715040,0irq/26-eth0-rx-0-21swapper/310:37:023
29715040,0irq/26-eth0-rx-0-21swapper/310:29:073
29715040,0irq/26-eth0-rx-0-21swapper/310:19:133
29715040,0irq/26-eth0-rx-0-21swapper/310:15:373
29715040,0irq/26-eth0-rx-0-21swapper/309:54:273
29715040,0irq/26-eth0-rx-0-21swapper/309:44:133
29715040,0irq/26-eth0-rx-0-21swapper/306:49:163
259940,0migration/27867-21aten_r1power_cu08:54:102
259940,0migration/2719-21kthreadcore09:29:152
259940,0migration/23690-21kthreadcore08:49:162
259940,0migration/231041-21kthreadcore10:49:142
259940,0migration/228242-21latency_hist07:09:002
259940,0migration/228242-21latency_hist07:09:002
259940,0migration/222208-21kthreadcore12:04:142
259940,0migration/219798-21munin-node09:59:162
259940,0migration/21802-21latency_hist12:13:582
259940,0migration/213510-21kthreadcore08:09:162
249940,0watchdog/28724-21idleruntime09:14:132
249940,0watchdog/225844-21kthreadcore09:24:172
249940,0watchdog/223064-21gltestperf10:04:132
249940,0watchdog/218241-21kthreadcore09:39:172
249940,0watchdog/216454-21fschecks_count07:24:132
249940,0watchdog/213523-21irqrtprio08:34:172
249940,0watchdog/211385-21hwlatdetect11:59:122
249940,0watchdog/20-21swapper/211:49:162
249940,0watchdog/20-21swapper/210:39:172
249940,0watchdog/20-21swapper/209:54:172
249940,0watchdog/20-21swapper/206:59:172
249940,0watchdog/20-21swapper/206:54:172
179940,0migration/19503-21cpuspeed_turbos07:19:111
179940,0migration/17432-21kthreadcore11:14:161
179940,0migration/16729-21kthreadcore08:04:161
179940,0migration/16329-21irqcore10:34:131
179940,0migration/15233-21taskset08:27:511
179940,0migration/13334-21kthreadcore09:09:161
179940,0migration/131486-21irqrtprio10:09:151
179940,0migration/126819-21kthreadcore08:19:151
179940,0migration/126629-21kthreadcore09:04:161
179940,0migration/124662-21kthreadcore10:24:161
179940,0migration/12108-21kthreadcore10:29:161
179940,0migration/118562-21aten_r1power_en12:04:081
179940,0migration/116182-21cpuspeed_turbos07:24:121
179940,0migration/115986-21taskset09:39:131
179940,0migration/110481-21kthreadcore08:29:161
169940,0watchdog/18759-21diskstats06:54:131
169940,0watchdog/1588-21hwlatdetect08:49:131
169940,0watchdog/125841-21kthreadcore09:24:171
169940,0watchdog/11908-21ssh11:30:521
169940,0watchdog/117866-21kthreadcore10:39:171
169940,0watchdog/117761-21munin-node07:49:131
169940,0watchdog/116556-21missed_timers08:09:171
169940,0watchdog/115900-21munin-node08:59:211
169940,0watchdog/115752-21tr11:19:161
169940,0watchdog/112617-21df_abs08:34:131
169940,0watchdog/112493-21munin-node10:59:121
169940,0watchdog/112189-21processes09:54:211
169940,0watchdog/10-21swapper/108:44:211
149940,0migration/04210-21kthreadcore06:49:160
149940,0migration/027906-21taskset10:46:290
149940,0migration/026906-21kthreadcore09:24:170
149940,0migration/026852-21kthreadcore08:19:150
149940,0migration/024799-21kthreadcore11:04:160
13779943,0cyclictest563-21awk09:09:142
13779943,0cyclictest3514-21snmpd08:29:332
13779943,0cyclictest3514-21snmpd08:28:182
13779943,0cyclictest3514-21snmpd08:19:032
13779943,0cyclictest3514-21snmpd08:15:032
13779943,0cyclictest3241-21irqbalance11:54:162
13779943,0cyclictest3241-21irqbalance11:34:062
13779943,0cyclictest3241-21irqbalance10:59:072
13779943,0cyclictest3241-21irqbalance10:34:072
13779943,0cyclictest3241-21irqbalance10:09:072
13779943,0cyclictest3241-21irqbalance07:19:272
13779943,0cyclictest3241-21irqbalance07:14:182
13779943,0cyclictest3241-21irqbalance07:06:182
13779943,0cyclictest29216-21awk11:09:132
13779943,0cyclictest23931-21awk09:44:112
13779943,0cyclictest21379-21awk11:04:142
13779943,0cyclictest20079-21awk11:44:142
13779943,0cyclictest17866-21awk07:49:142
13779943,0cyclictest0-21swapper/211:14:282
13779940,0cyclictest0-21swapper/210:21:062
13769944,0cyclictest0-21swapper/111:54:031
13769944,0cyclictest0-21swapper/110:08:341
13769944,0cyclictest0-21swapper/108:54:191
13769943,0cyclictest8655-21tail09:14:121
13769943,0cyclictest6696-21awk10:14:121
13769943,0cyclictest3556-21lldpd12:00:501
13769943,0cyclictest3556-21lldpd11:45:511
13769943,0cyclictest3556-21lldpd10:54:211
13769943,0cyclictest3556-21lldpd07:17:201
13769943,0cyclictest20731-21awk11:24:141
13769943,0cyclictest15314-21awk09:59:121
13769943,0cyclictest12087-21awk11:39:131
13749944,0cyclictest0-21swapper/012:16:200
13749944,0cyclictest0-21swapper/011:24:190
13749944,0cyclictest0-21swapper/010:08:290
13749943,0cyclictest9319-21awk06:54:160
13749943,0cyclictest5393-21awk08:04:140
13749943,0cyclictest4363-21awk07:39:120
13749943,0cyclictest3556-21lldpd10:19:500
13749943,0cyclictest3556-21lldpd09:05:500
13749943,0cyclictest3514-21snmpd08:55:030
13749943,0cyclictest3514-21snmpd08:11:180
13749943,0cyclictest3514-21snmpd07:36:180
13749943,0cyclictest3514-21snmpd07:26:480
13749943,0cyclictest3455-21awk11:54:120
13749943,0cyclictest28066-21sadc07:08:590
13749943,0cyclictest28032-21awk11:49:140
13749943,0cyclictest14072-21awk10:39:140
13749940,0cyclictest0-21swapper/009:11:430
29725030,0irq/27-eth0-tx-0-21swapper/008:44:160
29715030,0irq/26-eth0-rx-0-21swapper/308:24:173
29715030,0irq/26-eth0-rx-0-21swapper/307:54:103
29715030,0irq/26-eth0-rx-0-21swapper/307:33:553
29715030,0irq/26-eth0-rx-0-21swapper/307:27:163
249930,0watchdog/20-21swapper/208:39:132
249930,0watchdog/20-21swapper/207:59:172
169930,0watchdog/10-21swapper/109:44:131
169930,0watchdog/10-21swapper/107:44:171
169930,0watchdog/10-21swapper/107:29:211
149930,0migration/022112-21sh09:19:510
13779932,0cyclictest5433-21awk08:04:142
13779932,0cyclictest4456-21awk07:39:132
13779932,0cyclictest3514-21snmpd10:54:042
13779932,0cyclictest3514-21snmpd07:54:332
13779932,0cyclictest31873-21awk09:49:132
13779932,0cyclictest30833-21grep07:34:162
13779932,0cyclictest23745-21irqrtprio07:29:152
13779930,0cyclictest11234-21iostat_ios07:44:152
13769932,0cyclictest4378-21awk07:39:121
13769932,0cyclictest31147-21if_eth1.207:59:131
13769932,0cyclictest27293-21sensors07:04:201
13769932,0cyclictest24341-21tail07:54:121
13749932,0cyclictest3514-21snmpd11:42:270
13749932,0cyclictest3514-21snmpd11:35:190
13749932,0cyclictest3514-21snmpd08:26:510
13749932,0cyclictest32466-21awk09:29:130
13749932,0cyclictest30648-21sadc09:48:580
13749932,0cyclictest29987-21awk10:49:120
13749932,0cyclictest19233-21grep12:04:100
13749932,0cyclictest15070-21grep09:59:110
13749932,0cyclictest10232-21munin-node07:44:150
13749930,0cyclictest19458-21munin-node11:44:120
13769921,0cyclictest2008-21sort06:49:121
13749920,0cyclictest28070-21latency_hist07:09:000
13749920,0cyclictest28070-21latency_hist07:09:000
13749920,0cyclictest0-21swapper/008:38:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional