You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 14:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Dec 21, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
366923626,0sleep20-21swapper/207:08:352
355423626,0sleep30-21swapper/307:07:163
331723624,0sleep00-21swapper/007:04:490
363923425,0sleep10-21swapper/107:08:141
38069960,0cyclictest1937-21diskmemload11:49:462
38059960,0cyclictest0-21swapper/108:09:321
38049965,0cyclictest4912-21ssh09:47:220
339960,0migration/38573-21sh10:45:383
339960,0migration/332356-21sh12:17:583
339960,0migration/32746-21sh09:26:313
339960,0migration/325512-21sh12:33:183
339960,0migration/314974-21sh10:12:353
339960,0migration/31447-21sh11:00:283
339960,0migration/312867-21sh11:45:223
29725060,0irq/27-eth0-tx-5977-21munin-node07:59:360
29715060,0irq/26-eth0-rx-24931-21cut11:34:303
29715060,0irq/26-eth0-rx-14204-21ssh09:54:123
149960,0migration/09022-21sh10:46:410
149960,0migration/030420-21sh11:56:550
149960,0migration/025827-21sh10:39:170
149960,0migration/018398-21if_eth010:34:260
8215050,0irq/24-0000:00:8727-21munin-run10:29:122
8215050,0irq/24-0000:00:21526-21latency_hist11:34:122
38069955,0cyclictest0-21swapper/210:45:172
38069950,0cyclictest0-21swapper/212:23:022
38069950,0cyclictest0-21swapper/210:40:142
38069950,0cyclictest0-21swapper/210:17:042
38059955,0cyclictest0-21swapper/111:28:391
38059955,0cyclictest0-21swapper/111:02:391
38059950,0cyclictest0-21swapper/112:16:541
38059950,0cyclictest0-21swapper/112:02:431
38049955,0cyclictest0-21swapper/010:16:130
38049955,0cyclictest0-21swapper/009:59:400
38049950,0cyclictest0-21swapper/009:39:190
35150,0ktimersoftd/30-21swapper/311:16:143
339950,0migration/36876-21runrttasks10:24:573
339950,0migration/35880-21munin-node11:24:283
339950,0migration/35236-21kthreadcore09:09:293
339950,0migration/35031-21kthreadcore08:44:313
339950,0migration/331720-21kthreadcore09:04:323
339950,0migration/331336-21taskset10:04:193
339950,0migration/329606-21kthreadcore08:39:303
339950,0migration/327875-21kthreadcore10:39:303
339950,0migration/326932-21kthreadcore09:59:313
339950,0migration/326771-21latency_hist08:39:123
339950,0migration/324374-21taskset10:19:163
339950,0migration/321406-21runrttasks09:56:223
339950,0migration/32129-21kthreadcore11:59:303
339950,0migration/32077-21irqrtprio12:19:283
339950,0migration/310670-21munin-node07:34:353
29725050,0irq/27-eth0-tx-28730-21ssh09:42:132
29725050,0irq/27-eth0-tx-27946-21munin-node09:04:270
29725050,0irq/27-eth0-tx-20314-21munin-node08:34:320
29725050,0irq/27-eth0-tx-0-21swapper/009:09:270
29715050,0irq/26-eth0-rx-9326-21aten_r1power_cu12:24:233
29715050,0irq/26-eth0-rx-7098-21irqcore07:59:283
29715050,0irq/26-eth0-rx-6559-21runrttasks11:19:143
29715050,0irq/26-eth0-rx-6559-21runrttasks09:29:263
29715050,0irq/26-eth0-rx-32006-21python12:34:343
29715050,0irq/26-eth0-rx-25493-21ssh09:19:373
29715050,0irq/26-eth0-rx-25266-21kthreadcore11:54:303
29715050,0irq/26-eth0-rx-22195-21ssh10:14:433
29715050,0irq/26-eth0-rx-20873-21kthreadcore10:54:283
29715050,0irq/26-eth0-rx-17415-21ssh10:51:323
29715050,0irq/26-eth0-rx-17091-21kthreadcore08:54:303
29715050,0irq/26-eth0-rx-15451-21sh10:30:123
29715050,0irq/26-eth0-rx-15152-21df_abs11:49:253
29715050,0irq/26-eth0-rx-0-21swapper/312:04:243
29715050,0irq/26-eth0-rx-0-21swapper/311:40:493
29715050,0irq/26-eth0-rx-0-21swapper/311:29:283
29715050,0irq/26-eth0-rx-0-21swapper/311:09:443
29715050,0irq/26-eth0-rx-0-21swapper/310:35:443
29715050,0irq/26-eth0-rx-0-21swapper/309:44:343
29715050,0irq/26-eth0-rx-0-21swapper/309:42:073
29715050,0irq/26-eth0-rx-0-21swapper/309:34:123
29715050,0irq/26-eth0-rx-0-21swapper/309:15:503
259950,0migration/29667-21kthreadcore12:04:292
259950,0migration/230731-21sh11:57:382
19150,0ktimersoftd/10-21swapper/109:28:221
179950,0migration/17184-21kthreadcore10:44:301
179950,0migration/129505-21taskset10:00:361
179950,0migration/121664-21kthreadcore12:09:291
179950,0migration/117242-21kthreadcore08:54:301
169950,0watchdog/110647-21cpuspeed_turbos10:49:271
149950,0migration/08623-21runrttasks12:23:180
149950,0migration/07732-21kthreadcore07:09:320
149950,0migration/06395-21kthreadcore11:04:300
149950,0migration/04801-21latency_hist11:24:120
149950,0migration/03465-21open_files11:39:320
149950,0migration/031428-21taskset12:15:480
149950,0migration/030032-21kthreadcore07:49:310
149950,0migration/030032-21kthreadcore07:49:310
149950,0migration/028054-21kthreadcore08:14:300
149950,0migration/026996-21sh12:34:260
149950,0migration/02504-21kthreadcore10:24:300
149950,0migration/024866-21timerwakeupswit10:54:340
149950,0migration/024719-21kthreadcore11:34:300
149950,0migration/022609-21sh11:53:220
149950,0migration/021435-21aten_r1power_vo08:59:250
149950,0migration/020617-21taskset11:32:060
149950,0migration/020146-21kthreadcore07:44:320
149950,0migration/018867-21irqcore12:29:280
149950,0migration/016443-21switchtime07:14:330
149950,0migration/015481-21if_eth109:54:300
149950,0migration/014747-21missed_timers12:24:300
149950,0migration/013837-21sh09:53:220
149950,0migration/013682-21kthreadcore07:39:300
149950,0migration/013156-21fschecks_count08:04:260
149950,0migration/01269-21kthreadcore07:54:290
149950,0migration/01101-21kthreadcore07:29:320
149950,0migration/010797-21kthreadcore08:49:300
149950,0migration/010100-21kthreadcore07:34:310
38069944,0cyclictest0-21swapper/211:11:562
38069944,0cyclictest0-21swapper/208:34:252
38069943,0cyclictest8853-21awk08:49:282
38069943,0cyclictest3556-21lldpd08:02:252
38069943,0cyclictest3514-21snmpd10:49:132
38069943,0cyclictest3514-21snmpd09:04:122
38069943,0cyclictest3514-21snmpd08:17:272
38069943,0cyclictest3514-21snmpd08:12:272
38069943,0cyclictest3514-21snmpd07:55:272
38069943,0cyclictest3241-21irqbalance12:14:202
38069943,0cyclictest3241-21irqbalance09:09:212
38069943,0cyclictest3241-21irqbalance07:20:322
38069943,0cyclictest3241-21irqbalance07:15:322
38069943,0cyclictest3241-21irqbalance07:09:222
38069943,0cyclictest31540-21awk07:29:302
38069943,0cyclictest27711-21tail08:39:272
38069943,0cyclictest21329-21awk11:14:292
38069943,0cyclictest19219-21awk07:44:302
38069943,0cyclictest14619-21awk08:29:292
38069943,0cyclictest1117-21awk08:19:292
38059944,0cyclictest0-21swapper/110:25:171
38059944,0cyclictest0-21swapper/109:22:041
38059943,0cyclictest4335-21awk11:04:281
38059943,0cyclictest3556-21lldpd10:13:561
38059943,0cyclictest3514-21snmpd12:04:281
38059943,0cyclictest3514-21snmpd11:40:281
38059943,0cyclictest30761-21awk09:44:291
38059943,0cyclictest21281-21awk11:14:281
38059943,0cyclictest2042-21awk08:44:281
38059943,0cyclictest16894-21awk10:14:301
38059943,0cyclictest12588-21awk07:39:291
38059943,0cyclictest11679-21awk07:14:261
38049944,0cyclictest0-21swapper/011:12:390
38049943,0cyclictest3556-21lldpd10:13:260
38049943,0cyclictest3241-21irqbalance10:49:510
38049943,0cyclictest3241-21irqbalance09:24:210
38049943,0cyclictest3241-21irqbalance09:20:310
38049943,0cyclictest27849-21awk08:39:280
38049943,0cyclictest11572-21awk09:14:290
339940,0migration/35736-21kthreadcore07:09:313
339940,0migration/34365-21kthreadcore08:19:313
339940,0migration/33173-21kthreadcore07:29:323
339940,0migration/325624-21kthreadcore07:24:313
339940,0migration/325205-21kthreadcore08:59:303
339940,0migration/319884-21fschecks_count08:09:253
339940,0migration/315396-21kthreadcore08:04:303
339940,0migration/311917-21kthreadcore08:49:303
29725040,0irq/27-eth0-tx-5705-21ssh11:59:582
29725040,0irq/27-eth0-tx-11476-21if_err_eth109:14:282
29725040,0irq/27-eth0-tx-10635-21ssh09:29:392
29725040,0irq/27-eth0-tx-10503-21kthreadcore09:49:302
29725040,0irq/27-eth0-tx-0-21swapper/212:35:562
29725040,0irq/27-eth0-tx-0-21swapper/212:09:252
29725040,0irq/27-eth0-tx-0-21swapper/211:39:442
29725040,0irq/27-eth0-tx-0-21swapper/211:29:312
29725040,0irq/27-eth0-tx-0-21swapper/211:06:192
29725040,0irq/27-eth0-tx-0-21swapper/210:59:282
29725040,0irq/27-eth0-tx-0-21swapper/210:54:332
29725040,0irq/27-eth0-tx-0-21swapper/209:59:372
29725040,0irq/27-eth0-tx-0-21swapper/209:21:442
29715040,0irq/26-eth0-rx-0-21swapper/312:09:143
29715040,0irq/26-eth0-rx-0-21swapper/311:04:253
29715040,0irq/26-eth0-rx-0-21swapper/307:54:253
259940,0migration/29917-21kthreadcore10:09:302
259940,0migration/29164-21latency_hist12:24:122
259940,0migration/27684-21kthreadcore11:44:282
259940,0migration/2425-21kthreadcore10:04:302
259940,0migration/230073-21runrttasks07:25:132
259940,0migration/228444-21sh09:24:252
259940,0migration/224950-21latency_hist07:49:132
259940,0migration/224950-21latency_hist07:49:132
259940,0migration/222836-21munin-node10:34:302
259940,0migration/221279-21runrttasks08:59:242
259940,0migration/217758-21kthreadcore09:34:302
259940,0migration/213475-21hddtemp_smartct08:04:272
249940,0watchdog/28218-21grep11:24:302
249940,0watchdog/27307-21cpuspeed_turbos08:24:232
249940,0watchdog/224469-21ssh12:30:542
249940,0watchdog/22195-21sensors_temp11:19:342
249940,0watchdog/211856-21kthreadcore10:29:312
249940,0watchdog/20-21swapper/210:19:272
249940,0watchdog/20-21swapper/209:54:152
249940,0watchdog/20-21swapper/209:44:312
249940,0watchdog/20-21swapper/208:54:352
249940,0watchdog/20-21swapper/208:44:312
179940,0migration/19014-21sh10:29:231
179940,0migration/16075-21munin-node09:49:241
179940,0migration/15369-21kthreadcore12:19:301
179940,0migration/128691-21gltestperf09:04:291
179940,0migration/128290-21kthreadcore08:14:301
179940,0migration/126916-21kthreadcore07:49:301
179940,0migration/126916-21kthreadcore07:49:301
179940,0migration/12652-21kthreadcore10:04:311
179940,0migration/124539-21kthreadcore08:59:301
179940,0migration/124306-21sh11:54:291
179940,0migration/122756-21latency_hist09:59:121
179940,0migration/119143-21cpuspeed_turbos10:54:241
179940,0migration/118268-21fschecks_count12:29:261
179940,0migration/11718-21taskset12:38:421
179940,0migration/115768-21runrttasks12:25:181
179940,0migration/114344-21runrttasks08:53:311
169940,0watchdog/15982-21aten_r1power_cu07:59:231
169940,0watchdog/14533-21ssh11:24:021
169940,0watchdog/130237-21ssh10:19:351
169940,0watchdog/128645-21cat10:39:311
169940,0watchdog/121757-21ssh11:34:141
169940,0watchdog/119606-21irqstats07:44:311
169940,0watchdog/118599-21cut08:29:311
169940,0watchdog/118471-21if_eth1.310:34:271
169940,0watchdog/110845-21ssh09:30:031
169940,0watchdog/10-21swapper/109:34:351
169940,0watchdog/10-21swapper/109:14:311
169940,0watchdog/10-21swapper/107:29:311
169940,0watchdog/10-21swapper/107:24:311
149940,0migration/07726-21kthreadcore09:29:310
149940,0migration/030698-21kthreadcore10:59:300
38069932,0cyclictest12539-21awk07:39:282
38069930,0cyclictest9886-21cut07:34:312
38059933,0cyclictest0-21swapper/111:47:251
38059932,0cyclictest4899-21awk07:09:291
38059932,0cyclictest3514-21snmpd11:49:131
38059932,0cyclictest3514-21snmpd11:09:281
38059932,0cyclictest3514-21snmpd09:41:281
38059932,0cyclictest3514-21snmpd08:39:421
38059932,0cyclictest3514-21snmpd07:34:271
38059932,0cyclictest32547-21awk07:54:281
38059932,0cyclictest13422-21grep08:04:271
38049933,0cyclictest17343-21kworker/0:010:19:300
38049932,0cyclictest79352sleep012:04:250
38049932,0cyclictest3556-21lldpd10:04:260
38049932,0cyclictest3514-21snmpd12:10:430
38049932,0cyclictest3514-21snmpd11:22:160
38049932,0cyclictest23420-21kthreadcore08:09:290
38049932,0cyclictest2088-21awk08:44:290
38049932,0cyclictest11509-21kthreadcore11:44:290
38049932,0cyclictest0-21swapper/008:33:400
38049930,0cyclictest26571-21munin-node11:14:390
38049930,0cyclictest0-21swapper/007:19:270
339930,0migration/332024-21munin-node08:14:383
29725030,0irq/27-eth0-tx-0-21swapper/008:54:240
29725030,0irq/27-eth0-tx-0-21swapper/008:24:220
29725030,0irq/27-eth0-tx-0-21swapper/007:24:260
29715030,0irq/26-eth0-rx-0-21swapper/308:24:233
29715030,0irq/26-eth0-rx-0-21swapper/307:49:243
29715030,0irq/26-eth0-rx-0-21swapper/307:49:243
29715030,0irq/26-eth0-rx-0-21swapper/307:14:223
169930,0watchdog/17306-21cpuspeed_turbos08:24:231
169930,0watchdog/10-21swapper/111:29:301
169930,0watchdog/10-21swapper/109:09:271
169930,0watchdog/10-21swapper/108:34:271
169930,0watchdog/10-21swapper/108:19:311
169930,0watchdog/10-21swapper/107:19:231
149930,0migration/016852-21diskmemload10:33:280
149930,0migration/012935-21sh09:34:130
38049921,0cyclictest3514-21snmpd11:59:190
38049921,0cyclictest0-21swapper/008:19:130
35120,0ktimersoftd/30-21swapper/307:19:263
329920,0watchdog/30-21swapper/307:44:323
29715020,0irq/26-eth0-rx-0-21swapper/308:29:133
29715020,0irq/26-eth0-rx-0-21swapper/307:39:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional