You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 08:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Fri Sep 04, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2325025554,0sleep20-21swapper/218:40:422
304450350,0irq/25-eth10-21swapper/318:40:353
2342623524,0sleep00-21swapper/018:42:440
2328623424,0sleep10-21swapper/118:41:091
23687991111,0cyclictest27758-21sensors20:45:182
339960,0migration/318259-21sh22:21:023
339960,0migration/318072-21sh22:41:593
339960,0migration/316765-21taskset23:43:193
339960,0migration/313312-21sh20:59:193
339960,0migration/311168-21sh22:18:283
29715060,0irq/26-eth0-rx-5949-21df_abs21:15:103
29715060,0irq/26-eth0-rx-25232-21df_inode00:10:093
29715060,0irq/26-eth0-rx-24875-21grep23:05:233
29715060,0irq/26-eth0-rx-0-21swapper/320:44:553
236879960,0cyclictest19414-21aten_r1power_fr22:45:102
236879960,0cyclictest0-21swapper/223:35:562
236859960,0cyclictest0-21swapper/122:35:181
19160,0ktimersoftd/10-21swapper/121:58:341
19160,0ktimersoftd/10-21swapper/121:49:351
149960,0migration/09858-21sh22:57:520
149960,0migration/05441-21kthreadcore22:15:140
149960,0migration/026330-21runrttasks23:09:310
149960,0migration/019500-21taskset22:24:480
8215050,0irq/24-0000:00:824-21cat23:54:552
8215050,0irq/24-0000:00:641-21perf23:54:552
8215050,0irq/24-0000:00:10797-21tr19:45:132
339950,0migration/39512-21fschecks_count00:00:083
339950,0migration/3898-21sh23:54:563
339950,0migration/33648-21hwlatdetect22:55:103
339950,0migration/33552-21kthreadcore19:40:153
339950,0migration/33013-21timerwakeupswit21:30:193
339950,0migration/329906-21latency_hist20:49:553
339950,0migration/329332-21taskset20:49:053
339950,0migration/327166-21kthreadcore23:50:133
339950,0migration/326895-21sh22:28:053
339950,0migration/325102-21kthreadcore21:05:183
339950,0migration/323993-21timerandwakeup19:05:203
339950,0migration/322333-21kernelversion20:20:123
339950,0migration/322248-21kthreadcore19:55:143
339950,0migration/321271-21munin-node21:45:113
339950,0migration/320440-21taskset20:15:503
339950,0migration/319881-21idleruntime-cro19:54:563
339950,0migration/319404-21cpuspeed_turbos19:30:103
339950,0migration/319222-21kthreadcore23:45:123
339950,0migration/315841-21kthreadcore19:00:143
339950,0migration/314386-21munin-node21:40:143
339950,0migration/314197-21munin-node21:00:103
339950,0migration/31072-21kthreadcore21:10:133
29725050,0irq/27-eth0-tx-5586-21munin-node21:15:142
29725050,0irq/27-eth0-tx-3533-21ssh21:52:462
29725050,0irq/27-eth0-tx-31251-21kthreadcore21:10:122
29725050,0irq/27-eth0-tx-20102-21munin-node19:55:160
29725050,0irq/27-eth0-tx-13515-21munin-node21:20:102
29725050,0irq/27-eth0-tx-12516-21munin-node22:00:192
29725050,0irq/27-eth0-tx-11665-21tail19:00:080
29725050,0irq/27-eth0-tx-0-21swapper/223:46:352
29725050,0irq/27-eth0-tx-0-21swapper/223:40:312
29725050,0irq/27-eth0-tx-0-21swapper/223:20:172
29725050,0irq/27-eth0-tx-0-21swapper/222:33:082
29725050,0irq/27-eth0-tx-0-21swapper/222:10:082
29725050,0irq/27-eth0-tx-0-21swapper/221:57:542
29725050,0irq/27-eth0-tx-0-21swapper/020:45:320
29715050,0irq/26-eth0-rx-7144-21kthreadcore19:20:143
29715050,0irq/26-eth0-rx-6559-21runrttasks21:53:183
29715050,0irq/26-eth0-rx-513-21missed_timers20:25:133
29715050,0irq/26-eth0-rx-2730-21ssh22:33:563
29715050,0irq/26-eth0-rx-19814-21fschecks_count22:45:123
29715050,0irq/26-eth0-rx-17831-21kthreadcore21:20:133
29715050,0irq/26-eth0-rx-17787-21ssh23:24:313
29715050,0irq/26-eth0-rx-1075-21kthreadcore22:10:143
29715050,0irq/26-eth0-rx-0-21swapper/323:30:143
29715050,0irq/26-eth0-rx-0-21swapper/323:16:443
29715050,0irq/26-eth0-rx-0-21swapper/323:13:223
29715050,0irq/26-eth0-rx-0-21swapper/322:51:583
29715050,0irq/26-eth0-rx-0-21swapper/322:05:433
29715050,0irq/26-eth0-rx-0-21swapper/321:38:083
27150,0ktimersoftd/20-21swapper/223:30:002
259950,0migration/226040-21kthreadcore21:25:162
259950,0migration/221462-21fschecks_count21:45:122
259950,0migration/221178-21kthreadcore19:05:142
259950,0migration/217947-21munin-node19:25:172
259950,0migration/217116-21sh23:00:502
249950,0watchdog/28487-21cut20:55:142
249950,0watchdog/24435-21ssh20:52:062
249950,0watchdog/231112-21sh00:10:532
249950,0watchdog/224483-21kthreadcore19:55:142
236879955,0cyclictest0-21swapper/223:19:322
236879955,0cyclictest0-21swapper/222:54:282
236879955,0cyclictest0-21swapper/222:26:052
236879955,0cyclictest0-21swapper/222:09:322
236879950,0cyclictest0-21swapper/221:05:412
236879950,0cyclictest0-21swapper/200:09:312
236859955,0cyclictest0-21swapper/123:20:041
236859955,0cyclictest0-21swapper/122:44:171
236859954,0cyclictest27781-21tr20:00:121
236859950,0cyclictest0-21swapper/123:18:391
236849955,0cyclictest10-21rcuc/023:03:150
236849955,0cyclictest0-21swapper/021:35:160
236849954,0cyclictest6807-21cut21:15:140
236849954,0cyclictest2785-21awk20:30:100
236849954,0cyclictest21582-21crond21:04:550
236849950,0cyclictest0-21swapper/023:40:080
179950,0migration/131472-21latency23:30:141
179950,0migration/127314-21ksmtuned22:29:191
179950,0migration/12547-21runrttasks22:33:231
179950,0migration/122872-21kthreadcore20:20:131
179950,0migration/12078-21users22:10:201
179950,0migration/111633-21switchtime20:55:211
149950,0migration/08858-21irqcore20:10:130
149950,0migration/05530-21ntp_offset19:40:170
149950,0migration/032536-21kthreadcore20:50:130
149950,0migration/031820-21kernelversion18:50:110
149950,0migration/030129-21fschecks_time21:30:100
149950,0migration/028622-21sh21:28:150
149950,0migration/028270-21latency_hist22:09:550
149950,0migration/027879-21kthreadcore00:10:140
149950,0migration/027091-21kthreadcore23:50:130
149950,0migration/026462-21fschecks_time19:35:090
149950,0migration/025637-21hwlatdetect19:10:090
149950,0migration/022272-21irqrtprio21:45:150
149950,0migration/021728-21kernelversion22:05:150
149950,0migration/021170-21kernelversion22:25:170
149950,0migration/019869-21kthreadcore23:45:120
149950,0migration/018927-21switchtime21:40:200
149950,0migration/018923-21taskset22:44:310
149950,0migration/018880-21ksmtuned19:29:180
149950,0migration/018198-21kernelversion00:05:140
149950,0migration/018123-21latency_hist19:04:560
149950,0migration/016727-21latency_hist00:04:550
149950,0migration/016244-21kthreadcore21:20:120
149950,0migration/01301-21kthreadcore19:15:140
149950,0migration/011968-21sh21:59:070
149950,0migration/010668-21kthreadcore20:35:140
339940,0migration/37933-21kthreadcore18:55:143
339940,0migration/37370-21fschecks_time19:45:093
339940,0migration/33829-21kthreadcore20:05:123
339940,0migration/331878-21runrttasks19:36:013
339940,0migration/316668-21kthreadcore22:00:143
339940,0migration/313963-21taskset20:12:493
339940,0migration/312159-21kthreadcore20:35:143
339940,0migration/310824-21diskmemload21:55:393
29725040,0irq/27-eth0-tx-8979-21ssh00:00:012
29725040,0irq/27-eth0-tx-0-21swapper/223:10:022
29725040,0irq/27-eth0-tx-0-21swapper/222:55:212
29725040,0irq/27-eth0-tx-0-21swapper/222:40:052
29725040,0irq/27-eth0-tx-0-21swapper/222:35:132
29725040,0irq/27-eth0-tx-0-21swapper/222:20:082
29725040,0irq/27-eth0-tx-0-21swapper/222:16:352
29725040,0irq/27-eth0-tx-0-21swapper/220:59:572
29725040,0irq/27-eth0-tx-0-21swapper/019:20:090
29715040,0irq/26-eth0-rx-0-21swapper/323:35:143
29715040,0irq/26-eth0-rx-0-21swapper/323:25:553
29715040,0irq/26-eth0-rx-0-21swapper/323:00:023
29715040,0irq/26-eth0-rx-0-21swapper/322:35:313
29715040,0irq/26-eth0-rx-0-21swapper/321:24:583
29715040,0irq/26-eth0-rx-0-21swapper/320:34:203
29715040,0irq/26-eth0-rx-0-21swapper/320:01:393
29715040,0irq/26-eth0-rx-0-21swapper/319:25:103
29715040,0irq/26-eth0-rx-0-21swapper/318:50:483
29715040,0irq/26-eth0-rx-0-21swapper/300:05:393
259940,0migration/28688-21munin-node20:35:082
259940,0migration/25917-21munin-node21:35:102
259940,0migration/23529-21kthreadcore20:05:122
259940,0migration/229984-21fschecks_count21:30:092
259940,0migration/225823-21aten_r1power_cu19:35:062
259940,0migration/221453-21kthreadcore23:05:172
259940,0migration/221344-21munin-node20:40:192
249940,0watchdog/225955-21irqcore19:10:102
249940,0watchdog/22338-21kthreadcore19:40:142
249940,0watchdog/212411-21kthreadcore20:10:142
249940,0watchdog/211531-21chrt18:59:582
236879943,0cyclictest3241-21irqbalance21:40:142
236879943,0cyclictest31502-21awk18:50:102
236879943,0cyclictest28619-21awk20:25:102
236879943,0cyclictest20081-21awk19:30:122
236879943,0cyclictest19147-21awk23:25:132
236859944,0cyclictest0-21swapper/123:39:181
236859944,0cyclictest0-21swapper/121:53:121
236859944,0cyclictest0-21swapper/121:13:101
236859943,0cyclictest886-21awk19:40:121
236859943,0cyclictest3556-21lldpd23:05:591
236859943,0cyclictest3556-21lldpd19:50:271
236859943,0cyclictest3556-21lldpd18:50:561
236859943,0cyclictest3514-21snmpd21:35:471
236859943,0cyclictest3514-21snmpd20:15:161
236859943,0cyclictest32441-21sadc00:14:551
236859943,0cyclictest3241-21irqbalance22:56:041
236859943,0cyclictest3241-21irqbalance00:04:031
236859943,0cyclictest1825-21awk23:55:091
236859943,0cyclictest17848-21awk00:05:131
236859943,0cyclictest13387-21awk22:00:111
236859943,0cyclictest12786-21awk22:20:091
236849944,0cyclictest0-21swapper/023:20:070
236849944,0cyclictest0-21swapper/022:00:110
236849943,0cyclictest3514-21snmpd22:50:020
236849943,0cyclictest3514-21snmpd20:55:010
236849943,0cyclictest3514-21snmpd19:30:300
236849943,0cyclictest3514-21snmpd18:47:450
236849943,0cyclictest3241-21irqbalance23:31:140
236849943,0cyclictest3241-21irqbalance22:48:040
236849943,0cyclictest3241-21irqbalance22:35:540
236849943,0cyclictest3241-21irqbalance22:30:040
236849943,0cyclictest3241-21irqbalance21:05:140
236849943,0cyclictest3241-21irqbalance20:00:240
236849943,0cyclictest21899-21awk20:20:110
236849943,0cyclictest1930-21awk20:05:100
179940,0migration/16807-21aten_r1power_en19:45:061
179940,0migration/15658-21latency_hist19:19:561
179940,0migration/14516-21kthreadcore20:30:131
179940,0migration/130386-21kthreadcore22:50:131
179940,0migration/127567-21idleruntime23:10:111
179940,0migration/125953-21aten_r1power_vo19:35:071
179940,0migration/124025-21kthreadcore22:05:161
179940,0migration/123642-21kthreadcore22:45:161
179940,0migration/121696-21kthreadcore19:55:131
179940,0migration/120875-21kthreadcore19:30:141
179940,0migration/120214-21kthreadcore23:25:151
179940,0migration/117711-21kthreadcore21:00:151
179940,0migration/117132-21kthreadcore21:40:151
179940,0migration/117008-21sh23:00:291
179940,0migration/116712-21kthreadcore20:40:121
179940,0migration/115668-21kthreadcore21:20:121
179940,0migration/112930-21kthreadcore23:40:131
169940,0watchdog/16559-21runrttasks20:09:341
169940,0watchdog/14373-21diskstats22:15:101
169940,0watchdog/123953-21ssh23:46:011
169940,0watchdog/122569-21fschecks_time20:45:101
169940,0watchdog/122514-21perl21:05:141
169940,0watchdog/112091-21ssh21:16:261
169940,0watchdog/10-21swapper/123:50:131
169940,0watchdog/10-21swapper/121:30:141
169940,0watchdog/10-21swapper/120:50:101
169940,0watchdog/10-21swapper/119:10:101
169940,0watchdog/10-21swapper/118:55:141
149940,0migration/06424-21kthreadcore18:55:130
149940,0migration/030790-21sh21:10:100
149940,0migration/02792-21irqcore23:35:120
149940,0migration/023217-21kthreadcore23:25:160
149940,0migration/014526-21aten_r1power_vo20:15:060
149940,0migration/010944-21kthreadcore19:45:130
29715030,0irq/26-eth0-rx-0-21swapper/319:15:083
29715030,0irq/26-eth0-rx-0-21swapper/319:10:093
29715030,0irq/26-eth0-rx-0-21swapper/318:46:283
27130,0ktimersoftd/20-21swapper/220:18:202
259930,0migration/227343-21kthreadcore18:45:132
249930,0watchdog/20-21swapper/220:00:142
249930,0watchdog/20-21swapper/219:20:142
236879933,0cyclictest2817-21awk20:30:112
236879932,0cyclictest5759-21awk18:55:122
236879932,0cyclictest26498-21ntp_kernel_pll_20:20:162
236879930,0cyclictest17673-21kthreadcore19:50:132
236859932,0cyclictest28693-21awk20:25:111
236859932,0cyclictest13315-21awk19:25:121
236859932,0cyclictest11867-21grep20:35:141
236859932,0cyclictest11308-21kthreadcore20:10:141
236859930,0cyclictest22113-21perl21:25:121
236849933,0cyclictest30498-21kworker/0:023:54:560
236849933,0cyclictest30498-21kworker/0:023:14:570
236849933,0cyclictest30498-21kworker/0:023:10:070
236849933,0cyclictest30498-21kworker/0:021:49:570
236849933,0cyclictest30498-21kworker/0:020:40:090
236849933,0cyclictest30498-21kworker/0:020:24:570
236849933,0cyclictest30498-21kworker/0:019:50:110
169930,0watchdog/10-21swapper/119:20:141
169930,0watchdog/10-21swapper/119:05:141
169930,0watchdog/10-21swapper/118:59:581
169930,0watchdog/10-21swapper/118:45:581
236879920,0cyclictest0-21swapper/219:14:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional