You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 07:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Dec 22, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450370,0irq/25-eth10-21swapper/319:04:363
259723723,0sleep00-21swapper/019:06:030
270023626,0sleep10-21swapper/119:07:081
250423526,0sleep20-21swapper/219:05:012
339960,0migration/313406-21sh23:48:543
29715060,0irq/26-eth0-rx-29235-21ntp_states22:19:303
29715060,0irq/26-eth0-rx-11516-21grep21:49:333
29519966,0cyclictest0-21swapper/221:38:532
29519960,0cyclictest0-21swapper/222:06:292
29509965,0cyclictest18150-21taskset21:19:071
8215050,0irq/24-0000:00:4138-21python23:59:332
8215050,0irq/24-0000:00:20679-21cat23:34:092
4150,0ktimersoftd/00-21swapper/023:55:190
4150,0ktimersoftd/00-21swapper/000:31:340
339950,0migration/38708-21kthreadcore20:49:283
339950,0migration/37966-21irqcore22:09:273
339950,0migration/35674-21kthreadcore21:29:273
339950,0migration/34711-21sh00:00:053
339950,0migration/33865-21kthreadcore22:24:283
339950,0migration/332215-21sh22:41:433
339950,0migration/330681-21kthreadcore23:19:253
339950,0migration/329054-21kthreadcore21:24:293
339950,0migration/328070-21if_eth121:04:273
339950,0migration/326424-21kthreadcore00:14:283
339950,0migration/326152-21sh21:22:393
339950,0migration/324149-21kthreadcore20:34:293
339950,0migration/323807-21kthreadcore23:54:253
339950,0migration/322462-21kthreadcore20:09:313
339950,0migration/322037-21missed_timers00:29:283
339950,0migration/320861-21munin-node20:59:243
339950,0migration/319314-21kthreadcore22:34:283
339950,0migration/314982-21kthreadcore19:14:293
339950,0migration/314909-21irqcore23:49:273
339950,0migration/313801-21kthreadcore22:49:263
339950,0migration/313342-21munin-node23:29:243
29725050,0irq/27-eth0-tx-9729-21kthreadcore21:49:282
29725050,0irq/27-eth0-tx-8247-21munin-node00:24:282
29725050,0irq/27-eth0-tx-5869-21chrt19:59:210
29725050,0irq/27-eth0-tx-27680-21grep00:34:272
29725050,0irq/27-eth0-tx-21801-21ssh00:10:252
29725050,0irq/27-eth0-tx-14137-21ssh00:07:192
29725050,0irq/27-eth0-tx-0-21swapper/223:14:262
29725050,0irq/27-eth0-tx-0-21swapper/223:11:372
29725050,0irq/27-eth0-tx-0-21swapper/222:55:332
29725050,0irq/27-eth0-tx-0-21swapper/222:03:452
29725050,0irq/27-eth0-tx-0-21swapper/221:55:102
29725050,0irq/27-eth0-tx-0-21swapper/221:41:532
29725050,0irq/27-eth0-tx-0-21swapper/221:33:182
29725050,0irq/27-eth0-tx-0-21swapper/020:34:330
29725050,0irq/27-eth0-tx-0-21swapper/019:48:540
29715050,0irq/26-eth0-rx-3953-21date23:24:093
29715050,0irq/26-eth0-rx-3487-21kthreadcore22:04:283
29715050,0irq/26-eth0-rx-32439-21df00:19:233
29715050,0irq/26-eth0-rx-29618-21ssh22:02:183
29715050,0irq/26-eth0-rx-27313-21kthreadcore20:14:293
29715050,0irq/26-eth0-rx-24076-21cut21:39:303
29715050,0irq/26-eth0-rx-2208-21ssh23:04:053
29715050,0irq/26-eth0-rx-18276-21cut22:54:233
29715050,0irq/26-eth0-rx-16653-21sort22:14:303
29715050,0irq/26-eth0-rx-14664-21ssh00:08:323
29715050,0irq/26-eth0-rx-11400-21df23:09:223
29715050,0irq/26-eth0-rx-0-21swapper/323:41:303
29715050,0irq/26-eth0-rx-0-21swapper/323:38:043
29715050,0irq/26-eth0-rx-0-21swapper/323:04:263
29715050,0irq/26-eth0-rx-0-21swapper/322:44:343
29715050,0irq/26-eth0-rx-0-21swapper/322:31:453
29715050,0irq/26-eth0-rx-0-21swapper/321:55:533
29715050,0irq/26-eth0-rx-0-21swapper/321:35:123
29715050,0irq/26-eth0-rx-0-21swapper/300:34:273
29715050,0irq/26-eth0-rx-0-21swapper/300:24:163
29519955,0cyclictest0-21swapper/223:50:232
29519955,0cyclictest0-21swapper/222:33:262
29519955,0cyclictest0-21swapper/222:10:442
29519955,0cyclictest0-21swapper/221:28:392
29519955,0cyclictest0-21swapper/221:20:462
29519954,0cyclictest3514-21snmpd20:03:162
29519950,0cyclictest0-21swapper/223:38:522
29509955,0cyclictest0-21swapper/100:06:091
29509950,0cyclictest0-21swapper/122:54:441
29509950,0cyclictest0-21swapper/122:17:221
29509950,0cyclictest0-21swapper/100:11:311
29489954,0cyclictest16444-21ps00:09:270
29489950,0cyclictest0-21swapper/022:49:430
29489950,0cyclictest0-21swapper/022:43:350
29489950,0cyclictest0-21swapper/022:24:490
29489950,0cyclictest0-21swapper/022:06:130
259950,0migration/28309-21taskset21:11:412
259950,0migration/27841-21sh22:45:362
259950,0migration/26785-21kthreadcore23:04:272
259950,0migration/24136-21sh21:46:572
259950,0migration/23448-21sh23:40:252
259950,0migration/232212-21kthreadcore23:19:262
259950,0migration/23001-21munin-node19:29:312
259950,0migration/227297-21kthreadcore22:19:272
259950,0migration/225478-21kthreadcore23:54:262
259950,0migration/22013-21sh23:03:382
259950,0migration/213321-21kthreadcore20:04:262
259950,0migration/210581-21sh23:25:282
249950,0watchdog/221152-21perl20:59:242
19150,0ktimersoftd/10-21swapper/122:21:471
179950,0migration/124021-21kthreadcore23:34:291
149950,0migration/05800-21taskset20:22:490
149950,0migration/029427-21sh22:01:510
149950,0migration/027796-21fschecks_time21:04:250
149950,0migration/023721-21aten_r1power_cu22:19:210
149950,0migration/019610-21kthreadcore22:14:300
149950,0migration/016318-21kthreadcore20:29:260
149950,0migration/016226-21sh22:33:490
149950,0migration/012755-21kthreadcore21:14:280
149950,0migration/012335-21sh21:51:170
8215040,0irq/24-0000:00:17313-21latency_hist19:44:102
339940,0migration/332362-21kthreadcore19:29:283
339940,0migration/330674-21kthreadcore20:39:293
339940,0migration/329406-21runrttasks19:26:333
339940,0migration/32483-21kthreadcore20:19:273
339940,0migration/323073-21kthreadcore23:14:263
339940,0migration/318849-21kthreadcore19:19:283
339940,0migration/315191-21diskmemload00:09:153
339940,0migration/313467-21kthreadcore19:39:263
339940,0migration/310717-21latency_hist19:39:103
29725040,0irq/27-eth0-tx-512-21munin-node22:24:262
29725040,0irq/27-eth0-tx-26121-21ssh22:39:252
29725040,0irq/27-eth0-tx-13804-21munin-node20:54:340
29725040,0irq/27-eth0-tx-0-21swapper/222:34:122
29725040,0irq/27-eth0-tx-0-21swapper/222:15:162
29725040,0irq/27-eth0-tx-0-21swapper/221:14:222
29725040,0irq/27-eth0-tx-0-21swapper/200:14:272
29725040,0irq/27-eth0-tx-0-21swapper/021:09:240
29725040,0irq/27-eth0-tx-0-21swapper/020:44:330
29725040,0irq/27-eth0-tx-0-21swapper/019:49:220
29725040,0irq/27-eth0-tx-0-21swapper/019:39:280
29715040,0irq/26-eth0-rx-0-21swapper/321:44:273
29715040,0irq/26-eth0-rx-0-21swapper/321:14:363
29715040,0irq/26-eth0-rx-0-21swapper/321:09:383
29715040,0irq/26-eth0-rx-0-21swapper/320:27:183
29715040,0irq/26-eth0-rx-0-21swapper/319:59:253
29519943,0cyclictest3556-21lldpd20:43:002
29519943,0cyclictest3556-21lldpd20:11:302
29519943,0cyclictest32691-21awk20:19:252
29519943,0cyclictest1390-21awk20:44:262
29519943,0cyclictest10773-21awk19:14:262
29509943,0cyclictest3556-21lldpd23:49:311
29509943,0cyclictest3556-21lldpd20:17:001
29509943,0cyclictest3514-21snmpd21:54:161
29509943,0cyclictest3514-21snmpd21:53:461
29509943,0cyclictest3514-21snmpd21:47:161
29509943,0cyclictest3514-21snmpd21:06:311
29509943,0cyclictest3514-21snmpd20:05:011
29509943,0cyclictest28797-21awk23:19:231
29509943,0cyclictest10664-21sadc19:39:101
29489944,0cyclictest0-21swapper/023:16:320
29489944,0cyclictest0-21swapper/023:01:140
29489944,0cyclictest0-21swapper/000:39:080
29489944,0cyclictest0-21swapper/000:00:120
29489943,0cyclictest9923-21munin-node19:14:250
29489943,0cyclictest4880-21awk23:24:240
29489943,0cyclictest3556-21lldpd23:05:010
29489943,0cyclictest3556-21lldpd22:35:310
29489943,0cyclictest3556-21lldpd21:56:000
29489943,0cyclictest3556-21lldpd21:35:010
29489943,0cyclictest3556-21lldpd21:34:010
29489943,0cyclictest3514-21snmpd23:44:020
29489943,0cyclictest3514-21snmpd20:50:310
29489943,0cyclictest3241-21irqbalance23:30:380
29489943,0cyclictest3241-21irqbalance20:59:280
29489943,0cyclictest3241-21irqbalance20:25:180
29489943,0cyclictest3241-21irqbalance19:54:190
29489943,0cyclictest3241-21irqbalance00:04:380
29489943,0cyclictest28828-21awk23:19:230
29489943,0cyclictest26105-21awk20:14:270
29489943,0cyclictest21888-21awk23:34:280
29489943,0cyclictest2130-21awk22:44:250
29489943,0cyclictest19470-21awk21:19:240
29489943,0cyclictest11957-21awk23:09:250
259940,0migration/231216-21df_abs19:54:242
259940,0migration/21914-21kthreadcore00:19:272
259940,0migration/216590-21taskset00:29:152
259940,0migration/216191-21kthreadcore20:29:262
249940,0watchdog/25490-21aten2.4-expect23:44:242
249940,0watchdog/20-21swapper/222:49:202
179940,0migration/17519-21munin-node20:49:231
179940,0migration/17500-21kthreadcore19:59:241
179940,0migration/16094-21munin-node20:44:321
179940,0migration/15208-21kthreadcore21:29:261
179940,0migration/14597-21kthreadcore19:09:271
179940,0migration/14335-21kthreadcore21:09:271
179940,0migration/129990-21kthreadcore21:24:291
179940,0migration/129126-21kthreadcore19:49:281
179940,0migration/128753-21kthreadcore22:39:261
179940,0migration/127777-21kthreadcore00:14:291
179940,0migration/125901-21kthreadcore21:59:281
179940,0migration/124839-21mdstat20:34:301
179940,0migration/123874-21fschecks_time19:24:251
179940,0migration/118526-21kthreadcore20:54:301
179940,0migration/114824-21kthreadcore23:29:261
179940,0migration/114765-21kthreadcore20:29:251
179940,0migration/112735-21kthreadcore23:09:281
179940,0migration/110908-21kthreadcore22:09:281
179940,0migration/110615-21kthreadcore00:24:281
169940,0watchdog/19803-21ssh23:06:561
169940,0watchdog/19671-21hostname22:49:201
169940,0watchdog/18209-21ssh22:46:281
169940,0watchdog/129662-21ssh23:57:311
169940,0watchdog/12603-21kthreadcore19:29:291
169940,0watchdog/125122-21grep20:59:281
169940,0watchdog/1184222sleep121:19:121
169940,0watchdog/117138-21fschecks_time19:19:251
169940,0watchdog/112993-21kthreadcore19:14:291
169940,0watchdog/10-21swapper/123:44:241
169940,0watchdog/10-21swapper/121:34:121
169940,0watchdog/10-21swapper/100:32:511
149940,0migration/08101-21latency_hist00:24:090
149940,0migration/04716-21kthreadcore00:19:280
149940,0migration/03986-21sh21:46:350
149940,0migration/024156-21fschecks_count00:14:250
149940,0migration/01672-21sh21:26:000
339930,0migration/330510-21taskset19:53:143
339930,0migration/317220-21needreboot20:04:283
29725030,0irq/27-eth0-tx-26318-21telnet20:39:230
29725030,0irq/27-eth0-tx-11955-21aten2.4-expect20:04:200
29725030,0irq/27-eth0-tx-0-21swapper/019:34:220
29725030,0irq/27-eth0-tx-0-21swapper/019:29:220
29715030,0irq/26-eth0-rx-0-21swapper/320:55:203
29715030,0irq/26-eth0-rx-0-21swapper/320:44:263
29715030,0irq/26-eth0-rx-0-21swapper/319:54:233
29715030,0irq/26-eth0-rx-0-21swapper/319:09:213
29519933,0cyclictest3556-21lldpd19:36:002
29519933,0cyclictest24059-21awk19:24:262
29519932,0cyclictest3556-21lldpd20:58:002
29519932,0cyclictest30469-21grep21:04:292
29519932,0cyclictest12432-21kthreadcore19:39:262
29519931,0cyclictest9940-21kthreadcore20:24:272
29509932,0cyclictest4913-21awk23:24:241
29509932,0cyclictest3556-21lldpd19:41:001
29509932,0cyclictest3514-21snmpd23:41:471
29509932,0cyclictest32660-21awk20:19:251
29509932,0cyclictest31741-21awk19:54:271
29509932,0cyclictest25908-21awk00:34:241
29509932,0cyclictest21736-21awk21:39:271
29509932,0cyclictest20233-21head23:14:231
29509932,0cyclictest19435-21awk20:09:291
29509930,0cyclictest8841-21perl22:29:231
29509930,0cyclictest17017-21df_abs22:34:221
29489933,0cyclictest29904-21kworker/0:021:39:290
29489932,0cyclictest6272-21awk23:44:270
29489932,0cyclictest4164-21awk19:09:260
29489932,0cyclictest3514-21snmpd23:50:260
29489932,0cyclictest3514-21snmpd19:27:450
29489932,0cyclictest17527-21awk19:19:270
29489931,0cyclictest19807-21kthreadcore20:09:310
29489931,0cyclictest18432-21expr22:54:240
249930,0watchdog/20-21swapper/220:49:242
249930,0watchdog/20-21swapper/220:14:242
249930,0watchdog/20-21swapper/219:19:252
249930,0watchdog/20-21swapper/219:09:332
179930,0migration/110580-21kthreadcore20:24:271
169930,0watchdog/132093-21ssh00:19:191
169930,0watchdog/11975-21grep23:59:271
169930,0watchdog/11781-21kthreadcore22:04:281
169930,0watchdog/10-21swapper/123:00:081
169930,0watchdog/10-21swapper/122:24:241
169930,0watchdog/10-21swapper/120:39:241
169930,0watchdog/10-21swapper/119:44:281
149930,0migration/08137-21irqrtprio22:09:270
339920,0migration/318550-21iostat_ios19:44:263
29715020,0irq/26-eth0-rx-0-21swapper/320:29:343
29519921,0cyclictest19493-21latency_hist20:34:102
249920,0watchdog/20-21swapper/219:49:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional