You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 19:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Jun 21, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1587225242,0sleep20-21swapper/206:48:212
304450350,0irq/25-eth10-21swapper/306:48:373
1614723425,0sleep10-21swapper/106:51:281
1578522622,0sleep00-21swapper/006:47:270
29715070,0irq/26-eth0-rx-5865-21ntpq08:17:243
29715070,0irq/26-eth0-rx-5865-21ntpq08:17:243
162659970,0cyclictest0-21swapper/209:07:232
339960,0migration/331000-21sh09:02:113
339960,0migration/328578-21sh10:57:283
339960,0migration/321495-21sh10:14:033
339960,0migration/321283-21sh10:33:563
339960,0migration/313020-21sh09:47:353
29725060,0irq/27-eth0-tx-29779-21munin-node07:02:160
29715060,0irq/26-eth0-rx-9558-21netstat07:07:203
29715060,0irq/26-eth0-rx-30530-21perl11:42:143
29715060,0irq/26-eth0-rx-0-21swapper/311:02:193
8215050,0irq/24-0000:00:24928-21munin-run07:47:032
339950,0migration/39730-21kthreadcore09:27:203
339950,0migration/38364-21munin-node08:22:183
339950,0migration/34527-21munin-node10:42:263
339950,0migration/332513-21kthreadcore12:02:183
339950,0migration/332077-21irqrtprio10:22:193
339950,0migration/331839-21aten_r1power_en07:52:133
339950,0migration/331100-21sh09:42:133
339950,0migration/325994-21kthreadcore08:57:183
339950,0migration/324580-21kthreadcore10:37:193
339950,0migration/323820-21missed_timers07:42:223
339950,0migration/323722-21taskset10:17:173
339950,0migration/323587-21fschecks_count06:57:173
339950,0migration/323550-21taskset07:18:083
339950,0migration/323378-21fschecks_count09:57:163
339950,0migration/322639-21taskset06:53:543
339950,0migration/321629-21aten_r1power_cu11:57:123
339950,0migration/318006-21kthreadcore08:02:213
339950,0migration/317281-21munin-node11:12:203
339950,0migration/316520-21kthreadcore10:52:183
339950,0migration/314255-21taskset08:49:063
339950,0migration/314079-21sh09:10:253
339950,0migration/312172-21taskset11:27:493
339950,0migration/310920-21timerandwakeup07:32:263
29725050,0irq/27-eth0-tx-6581-21munin-node10:27:252
29725050,0irq/27-eth0-tx-6287-21perl11:47:162
29725050,0irq/27-eth0-tx-6088-21munin-node11:07:162
29725050,0irq/27-eth0-tx-3653-21ssh12:03:072
29725050,0irq/27-eth0-tx-23150-21munin-node06:57:200
29725050,0irq/27-eth0-tx-13731-21ssh09:29:152
29725050,0irq/27-eth0-tx-11919-21ssh12:08:512
29725050,0irq/27-eth0-tx-0-21swapper/212:17:192
29725050,0irq/27-eth0-tx-0-21swapper/211:02:292
29725050,0irq/27-eth0-tx-0-21swapper/210:22:062
29725050,0irq/27-eth0-tx-0-21swapper/210:02:182
29725050,0irq/27-eth0-tx-0-21swapper/008:52:080
29725050,0irq/27-eth0-tx-0-21swapper/008:12:230
29715050,0irq/26-eth0-rx-27921-21munin-node11:17:233
29715050,0irq/26-eth0-rx-25599-21kthreadcore09:17:183
29715050,0irq/26-eth0-rx-22557-21perl11:37:133
29715050,0irq/26-eth0-rx-20900-21cat09:32:263
29715050,0irq/26-eth0-rx-19562-21kthreadcore09:12:203
29715050,0irq/26-eth0-rx-14686-21irqcore11:52:173
29715050,0irq/26-eth0-rx-12426-21sh11:49:243
29715050,0irq/26-eth0-rx-11016-21kthreadcore10:07:183
29715050,0irq/26-eth0-rx-0-21swapper/312:20:433
29715050,0irq/26-eth0-rx-0-21swapper/311:07:143
29715050,0irq/26-eth0-rx-0-21swapper/310:47:193
29715050,0irq/26-eth0-rx-0-21swapper/309:24:413
29715050,0irq/26-eth0-rx-0-21swapper/308:56:153
27150,0ktimersoftd/20-21swapper/209:20:562
259950,0migration/29945-21kthreadcore08:47:202
259950,0migration/24901-21runrttasks10:43:242
259950,0migration/24797-21sh11:24:452
259950,0migration/232614-21idleruntime07:52:162
259950,0migration/232587-21munin-node08:37:232
259950,0migration/223099-21kthreadcore11:57:182
259950,0migration/220655-21irqrtprio08:07:182
259950,0migration/215667-21munin-node08:52:162
249950,0watchdog/223845-21irqcore08:57:172
249950,0watchdog/223635-21fschecks_count06:57:172
249950,0watchdog/222786-21ssh09:16:332
249950,0watchdog/215360-21tune2fs09:52:162
249950,0watchdog/210643-21grep10:47:202
19150,0ktimersoftd/10-21swapper/109:11:591
179950,0migration/17346-21kthreadcore07:07:181
179950,0migration/132576-21kthreadcore11:02:181
179950,0migration/131400-21taskset07:50:131
179950,0migration/127560-21fschecks_count08:37:151
179950,0migration/124194-21munin-node09:17:161
179950,0migration/123087-21cpuspeed_turbos10:17:141
179950,0migration/123045-21kthreadcore12:17:171
179950,0migration/119855-21kthreadcore10:32:191
179950,0migration/113039-21irqrtprio07:37:171
179950,0migration/111169-21timerandwakeup12:07:241
179950,0migration/110919-21aten_r1power_po07:12:151
162659955,0cyclictest0-21swapper/212:12:282
162659955,0cyclictest0-21swapper/211:45:112
162659955,0cyclictest0-21swapper/210:12:102
162649954,0cyclictest22863-21awk11:37:151
162649950,0cyclictest0-21swapper/111:18:441
162639950,0cyclictest0-21swapper/011:32:450
149950,0migration/09488-21kthreadcore11:47:200
149950,0migration/09365-21kthreadcore10:07:170
149950,0migration/0919-21taskset08:40:390
149950,0migration/06514-21kthreadcore07:32:200
149950,0migration/04681-21kthreadcore07:52:200
149950,0migration/04620-21unixbench_singl10:22:260
149950,0migration/03379-21latency11:02:190
149950,0migration/03327-21kthreadcore09:22:200
149950,0migration/032412-21kthreadcore11:22:160
149950,0migration/031387-21cpuspeed_turbos09:42:150
149950,0migration/031329-21cpuspeed_turbos09:02:130
149950,0migration/031037-21irqcore11:42:170
149950,0migration/030189-21fschecks_count12:02:140
149950,0migration/030185-21sh10:00:350
149950,0migration/029613-21sh09:38:430
149950,0migration/028421-21taskset11:38:240
149950,0migration/023802-21fschecks_time09:17:140
149950,0migration/022663-21irqcore11:57:170
149950,0migration/02130-21kthreadcore08:17:200
149950,0migration/02130-21kthreadcore08:17:200
149950,0migration/020325-21kthreadcore09:12:200
149950,0migration/020030-21fschecks_time08:07:150
149950,0migration/017283-21idleruntime-cro07:17:030
149950,0migration/016976-21kthreadcore10:52:190
149950,0migration/014790-21latency_hist09:12:020
149950,0migration/013288-21sh10:29:050
149950,0migration/012255-21missed_timers09:47:230
149950,0migration/011917-21kthreadcore08:22:190
149950,0migration/010518-21kthreadcore11:07:190
4140,0ktimersoftd/00-21swapper/009:36:330
339940,0migration/35230-21sh12:07:033
339940,0migration/34722-21taskset07:30:383
339940,0migration/331940-21kthreadcore07:02:193
339940,0migration/331431-21irqrtprio11:22:153
339940,0migration/331257-21taskset07:48:273
339940,0migration/324692-21fschecks_time07:22:153
339940,0migration/321481-21irqcore08:32:183
339940,0migration/315445-21kthreadcore08:27:183
29725040,0irq/27-eth0-tx-6850-21munin-node09:47:262
29725040,0irq/27-eth0-tx-22846-21cpu10:37:142
29725040,0irq/27-eth0-tx-13901-21munin-node11:32:152
29725040,0irq/27-eth0-tx-12939-21ssh10:07:422
29725040,0irq/27-eth0-tx-0-21swapper/211:37:062
29725040,0irq/27-eth0-tx-0-21swapper/211:27:052
29725040,0irq/27-eth0-tx-0-21swapper/211:17:232
29725040,0irq/27-eth0-tx-0-21swapper/210:32:072
29725040,0irq/27-eth0-tx-0-21swapper/008:32:170
29715040,0irq/26-eth0-rx-0-21swapper/312:12:173
29715040,0irq/26-eth0-rx-0-21swapper/311:36:103
29715040,0irq/26-eth0-rx-0-21swapper/310:28:393
29715040,0irq/26-eth0-rx-0-21swapper/310:04:333
29715040,0irq/26-eth0-rx-0-21swapper/309:52:453
29715040,0irq/26-eth0-rx-0-21swapper/309:38:443
29715040,0irq/26-eth0-rx-0-21swapper/308:07:143
29715040,0irq/26-eth0-rx-0-21swapper/307:12:213
259940,0migration/29534-21needreboot07:07:202
259940,0migration/28990-21kthreadcore07:57:202
259940,0migration/232266-21if_eth1.309:22:172
259940,0migration/228992-21runrttasks10:17:402
259940,0migration/22684-21kthreadcore09:02:192
259940,0migration/224383-21kthreadcore09:57:192
259940,0migration/216476-21munin-node06:52:142
259940,0migration/214954-21kthreadcore08:02:202
259940,0migration/214330-21latency_hist10:52:022
259940,0migration/211870-21aten_r1power_fr07:37:122
249940,0watchdog/238212sleep207:06:172
249940,0watchdog/228597-21open_inodes09:37:202
249940,0watchdog/227122-21cut10:57:202
249940,0watchdog/216511-21python07:12:252
249940,0watchdog/20-21swapper/208:42:172
249940,0watchdog/20-21swapper/207:47:212
179940,0migration/19035-21kthreadcore08:22:181
179940,0migration/15859-21runrttasks07:56:381
179940,0migration/1534-21cpu08:17:161
179940,0migration/1534-21cpu08:17:161
179940,0migration/13912-21kthreadcore09:22:201
179940,0migration/122617-21taskset10:37:051
179940,0migration/119946-21kthreadcore07:17:191
179940,0migration/118591-21kthreadcore11:32:191
179940,0migration/118197-21kthreadcore06:52:191
179940,0migration/116498-21kthreadcore11:52:181
179940,0migration/114990-21runrttasks09:12:081
179940,0migration/11162-21kthreadcore07:27:181
162659944,0cyclictest0-21swapper/207:17:102
162659943,0cyclictest7948-21awk08:22:162
162659943,0cyclictest3556-21lldpd09:32:412
162659943,0cyclictest3514-21snmpd11:12:082
162659943,0cyclictest3514-21snmpd09:42:072
162659943,0cyclictest3514-21snmpd08:14:512
162659943,0cyclictest3514-21snmpd07:34:202
162659943,0cyclictest14751-21awk08:27:172
162659943,0cyclictest14551-21awk11:52:162
162659943,0cyclictest1251-21awk08:17:182
162659943,0cyclictest1251-21awk08:17:182
162649943,0cyclictest7571-21awk10:27:181
162649943,0cyclictest7365-21awk10:47:181
162649943,0cyclictest7117-21awk07:57:181
162649943,0cyclictest5993-21awk07:32:191
162649943,0cyclictest3556-21lldpd11:27:411
162649943,0cyclictest3556-21lldpd10:04:111
162649943,0cyclictest3556-21lldpd09:57:111
162649943,0cyclictest3556-21lldpd09:48:111
162649943,0cyclictest3556-21lldpd09:38:111
162649943,0cyclictest3556-21lldpd09:29:111
162649943,0cyclictest3514-21snmpd08:11:211
162649943,0cyclictest3241-21irqbalance11:11:101
162649943,0cyclictest31089-21awk11:22:141
162649943,0cyclictest25034-21awk07:22:171
162649943,0cyclictest23322-21awk10:57:171
162649943,0cyclictest19325-21awk07:42:181
162649943,0cyclictest15937-21awk09:32:181
162649943,0cyclictest15691-21awk08:52:161
162649943,0cyclictest15089-21awk11:12:181
162649943,0cyclictest13632-21awk08:02:181
162639944,0cyclictest14524-21diskmemload10:37:350
162639944,0cyclictest0-21swapper/008:27:210
162639943,0cyclictest7332-21awk10:47:170
162639943,0cyclictest3556-21lldpd12:18:110
162639943,0cyclictest3556-21lldpd11:17:410
162639943,0cyclictest3514-21snmpd07:23:200
162639943,0cyclictest31711-21awk10:42:180
162639943,0cyclictest26881-21kworker/0:110:15:070
162639943,0cyclictest23378-21awk10:57:180
162639943,0cyclictest13011-21sadc12:12:020
162639940,0cyclictest30799-21munin-node07:27:210
162639940,0cyclictest25151-21munin-node07:47:270
162639940,0cyclictest18457-21munin-node07:42:170
162639940,0cyclictest12796-21munin-node08:02:270
169940,0watchdog/14003-21ssh11:43:041
169940,0watchdog/128860-21ssh12:00:321
169940,0watchdog/123844-21irqcore08:57:171
169940,0watchdog/123637-21fschecks_count06:57:171
169940,0watchdog/120843-21ssh10:53:121
169940,0watchdog/120139-21ls10:12:201
169940,0watchdog/118235-21kthreadcore09:52:201
169940,0watchdog/11805-21kthreadcore09:42:201
169940,0watchdog/113039-21ssh11:51:121
169940,0watchdog/10-21swapper/112:02:241
169940,0watchdog/10-21swapper/110:22:401
169940,0watchdog/10-21swapper/109:02:331
169940,0watchdog/10-21swapper/108:47:211
169940,0watchdog/10-21swapper/108:42:171
169940,0watchdog/10-21swapper/108:32:171
149940,0migration/08607-21fschecks_count08:47:160
149940,0migration/029938-21sh10:20:240
149940,0migration/026163-21kthreadcore08:57:180
149940,0migration/021486-21kthreadcore06:52:200
149940,0migration/019184-21open_files11:52:210
149940,0migration/018353-21latency_hist07:42:030
149940,0migration/015239-21fschecks_time10:32:140
149940,0migration/014540-21irqrtprio12:12:180
29725030,0irq/27-eth0-tx-0-21swapper/007:57:150
29715030,0irq/26-eth0-rx-0-21swapper/308:42:153
29715030,0irq/26-eth0-rx-0-21swapper/308:37:483
29715030,0irq/26-eth0-rx-0-21swapper/308:12:143
29715030,0irq/26-eth0-rx-0-21swapper/307:57:253
29715030,0irq/26-eth0-rx-0-21swapper/307:37:123
249930,0watchdog/20-21swapper/207:27:132
162659932,0cyclictest28861-21grep07:22:192
162659932,0cyclictest20710-21tr08:32:142
162649933,0cyclictest0-21swapper/112:12:301
162649932,0cyclictest3556-21lldpd08:12:121
162649932,0cyclictest32232-21kthreadcore07:02:191
162639933,0cyclictest26881-21kworker/0:110:02:140
162639933,0cyclictest26881-21kworker/0:109:52:340
162639933,0cyclictest26881-21kworker/0:109:27:140
162639933,0cyclictest26881-21kworker/0:108:43:580
162639932,0cyclictest8398-21kthreadcore11:27:170
162639932,0cyclictest15140-21awk11:12:180
162639930,0cyclictest0-21swapper/007:17:160
162639930,0cyclictest0-21swapper/007:07:120
169930,0watchdog/10-21swapper/110:42:081
169930,0watchdog/10-21swapper/110:07:201
162649920,0cyclictest13699-21cat08:27:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional