You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 13:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Mar 17, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3220623622,0sleep00-21swapper/018:57:580
3230523525,0sleep20-21swapper/218:59:082
304450350,0irq/25-eth10-21swapper/318:57:113
3210123026,0sleep10-21swapper/118:56:431
339960,0migration/35709-21sh21:24:083
339960,0migration/33859-21sh23:24:143
339960,0migration/32547-21sh00:23:163
339960,0migration/323432-21kthreadcore19:16:273
339960,0migration/312803-21sh23:10:533
339960,0migration/311940-21sh22:06:323
325559966,0cyclictest0-21swapper/222:31:292
29725060,0irq/27-eth0-tx-0-21swapper/019:41:320
29715060,0irq/26-eth0-rx-5575-21iostat23:46:243
29715060,0irq/26-eth0-rx-23170-21perl21:56:223
29715060,0irq/26-eth0-rx-17643-21ssh00:11:273
149960,0migration/03623-21kernelversion19:51:270
149960,0migration/029620-21sh21:18:090
149960,0migration/028408-21sh23:19:480
149960,0migration/022426-21timerandwakeup20:26:330
149960,0migration/020384-21sh23:35:370
149960,0migration/019447-21sh00:14:390
149960,0migration/014520-21aten_r1power_fr21:51:210
149960,0migration/011875-21sh23:49:550
149960,0migration/011339-21sh00:30:230
8215050,0irq/24-0000:00:15568-21cat20:01:102
35150,0ktimersoftd/30-21swapper/321:53:183
339950,0migration/3969-21missed_timers19:46:293
339950,0migration/38684-21taskset19:54:413
339950,0migration/35892-21kthreadcore00:06:283
339950,0migration/328670-21timerwakeupswit21:36:333
339950,0migration/326452-21timerandwakeup23:56:303
339950,0migration/323916-21kthreadcore23:36:283
339950,0migration/323834-21taskset20:53:183
339950,0migration/323153-21munin-node22:16:243
339950,0migration/321537-21timerwakeupswit20:01:333
339950,0migration/316741-21kthreadcore19:36:273
339950,0migration/314313-21taskset19:34:403
339950,0migration/314257-21kthreadcore23:51:263
339950,0migration/312117-21kthreadcore20:21:263
339950,0migration/31027-21taskset19:25:433
339950,0migration/310214-21kthreadcore21:26:273
339950,0migration/310100-21timerandwakeup20:41:333
325559955,0cyclictest0-21swapper/223:01:172
325559955,0cyclictest0-21swapper/221:44:492
325559955,0cyclictest0-21swapper/221:13:262
325559955,0cyclictest0-21swapper/200:09:522
325509955,0cyclictest0-21swapper/122:50:251
325509950,0cyclictest0-21swapper/123:21:381
325509950,0cyclictest0-21swapper/123:03:301
325509950,0cyclictest0-21swapper/122:53:531
325509950,0cyclictest0-21swapper/122:21:341
325499950,0cyclictest0-21swapper/023:37:050
325499950,0cyclictest0-21swapper/022:28:130
325499950,0cyclictest0-21swapper/022:22:230
325499950,0cyclictest0-21swapper/021:26:340
325499950,0cyclictest0-21swapper/000:10:410
29725050,0irq/27-eth0-tx-4044-21ssh23:24:462
29725050,0irq/27-eth0-tx-28300-21ssh22:58:252
29725050,0irq/27-eth0-tx-26475-21grep22:36:272
29725050,0irq/27-eth0-tx-21750-21if_err_eth1.223:36:262
29725050,0irq/27-eth0-tx-19409-21ssh00:14:322
29725050,0irq/27-eth0-tx-13715-21ssh21:29:082
29725050,0irq/27-eth0-tx-0-21swapper/223:53:422
29725050,0irq/27-eth0-tx-0-21swapper/223:33:362
29725050,0irq/27-eth0-tx-0-21swapper/221:06:262
29715050,0irq/26-eth0-rx-6812-21ssh22:26:263
29715050,0irq/26-eth0-rx-6559-21runrttasks23:35:063
29715050,0irq/26-eth0-rx-4757-21ssh23:26:123
29715050,0irq/26-eth0-rx-31137-21munin-node22:21:253
29715050,0irq/26-eth0-rx-28916-21aten_r1power_en23:41:213
29715050,0irq/26-eth0-rx-28701-21smartctl21:16:313
29715050,0irq/26-eth0-rx-28284-21taskset22:58:223
29715050,0irq/26-eth0-rx-27046-21python19:41:333
29715050,0irq/26-eth0-rx-19707-21ssh22:51:433
29715050,0irq/26-eth0-rx-1721-21cut22:01:273
29715050,0irq/26-eth0-rx-1160-21tr21:01:253
29715050,0irq/26-eth0-rx-0-21swapper/323:11:243
29715050,0irq/26-eth0-rx-0-21swapper/322:42:213
29715050,0irq/26-eth0-rx-0-21swapper/322:31:243
29715050,0irq/26-eth0-rx-0-21swapper/322:11:593
29715050,0irq/26-eth0-rx-0-21swapper/321:46:393
29715050,0irq/26-eth0-rx-0-21swapper/321:42:513
29715050,0irq/26-eth0-rx-0-21swapper/321:07:543
29715050,0irq/26-eth0-rx-0-21swapper/300:01:343
27150,0ktimersoftd/20-21swapper/223:47:292
259950,0migration/27773-21fschecks_time19:06:232
259950,0migration/230877-21latency_hist21:21:102
259950,0migration/213422-21cpuspeed_turbos23:11:212
259950,0migration/21278-21munin-node19:01:252
259950,0migration/211594-21switchtime22:26:322
259950,0migration/211525-21sh00:30:542
259950,0migration/210945-21munin-node23:26:342
249950,0watchdog/222040-21fschecks_time19:41:252
249950,0watchdog/216798-21timerwakeupswit20:46:322
179950,0migration/19192-21timerwakeupswit20:16:321
179950,0migration/130480-21aten_r1power_en22:01:201
179950,0migration/130446-21kthreadcore23:41:261
179950,0migration/127643-21munin-node22:56:331
179950,0migration/126298-21sendmail_mailqu23:56:281
179950,0migration/120993-21latency_hist23:16:101
179950,0migration/119937-21latency19:36:281
179950,0migration/118505-21timerwakeupswit23:51:321
179950,0migration/118448-21munin-node20:51:241
179950,0migration/115648-21latency_hist20:01:101
179950,0migration/113582-21latency_hist22:31:101
179950,0migration/110773-21kthreadcore21:46:261
179950,0migration/110418-21sh00:27:411
149950,0migration/09578-21kthreadcore19:31:270
149950,0migration/07665-21kthreadcore23:06:270
149950,0migration/05130-21taskset22:03:590
149950,0migration/032766-21kthreadcore23:41:270
149950,0migration/032732-21smart_sdb19:21:330
149950,0migration/03185-21kthreadcore19:01:270
149950,0migration/031361-21kthreadcore00:21:270
149950,0migration/03034-21kthreadcore21:41:270
149950,0migration/029695-21taskset20:35:420
149950,0migration/029243-21aten_r1power_fr23:21:210
149950,0migration/028854-21sh21:57:240
149950,0migration/028710-21fschecks_time19:46:250
149950,0migration/026163-21kthreadcore22:16:260
149950,0migration/025269-21kthreadcore20:06:290
149950,0migration/023861-21kthreadcore22:36:260
149950,0migration/023654-21kthreadcore23:56:240
149950,0migration/02233-21kthreadcore23:01:280
149950,0migration/019851-21taskset19:12:320
149950,0migration/019474-21taskset23:11:550
149950,0migration/018553-21kthreadcore22:31:300
149950,0migration/018240-21kthreadcore20:01:270
149950,0migration/016023-21kthreadcore22:11:250
149950,0migration/014882-21aten2.4-expect21:11:210
149950,0migration/014648-21kthreadcore23:51:260
149950,0migration/011722-21taskset22:46:480
8215040,0irq/24-0000:00:9813-21cat20:21:102
339940,0migration/36495-21kthreadcore19:26:283
339940,0migration/331136-21irqrtprio20:36:273
339940,0migration/327809-21sh20:06:323
339940,0migration/317570-21kthreadcore19:11:273
339940,0migration/311019-21kthreadcore19:56:253
325559943,0cyclictest3556-21lldpd20:55:292
325559943,0cyclictest30696-21awk22:41:242
325559943,0cyclictest29961-21awk20:11:262
325559943,0cyclictest29705-21awk23:41:242
325559943,0cyclictest29457-21awk00:01:242
325559943,0cyclictest29117-21awk00:21:242
325559943,0cyclictest24290-21awk20:31:262
325559943,0cyclictest21485-21awk23:56:222
325559943,0cyclictest21358-21awk19:16:252
325509944,0cyclictest0-21swapper/121:17:241
325509944,0cyclictest0-21swapper/120:46:111
325509944,0cyclictest0-21swapper/100:06:531
325509943,0cyclictest3241-21irqbalance21:22:191
325509943,0cyclictest23429-21awk21:56:241
325509943,0cyclictest22983-21awk22:16:231
325509943,0cyclictest22195-21awk19:41:261
325509943,0cyclictest15708-21awk21:11:241
325509943,0cyclictest12904-21sadc23:11:101
325499944,0cyclictest0-21swapper/022:42:150
325499944,0cyclictest0-21swapper/021:36:210
325499944,0cyclictest0-21swapper/020:41:290
325499943,0cyclictest8100-21awk19:06:250
325499943,0cyclictest3556-21lldpd19:20:300
325499943,0cyclictest3241-21irqbalance21:46:380
325499943,0cyclictest31825-21awk21:01:230
325499943,0cyclictest18556-21awk20:51:240
325499943,0cyclictest11906-21awk20:46:250
325499943,0cyclictest10206-21ps19:56:250
29725040,0irq/27-eth0-tx-30964-21munin-node21:01:312
29725040,0irq/27-eth0-tx-27511-21ssh23:17:112
29725040,0irq/27-eth0-tx-22663-21ssh21:36:132
29725040,0irq/27-eth0-tx-14408-21ssh21:51:132
29725040,0irq/27-eth0-tx-0-21swapper/222:46:282
29725040,0irq/27-eth0-tx-0-21swapper/222:11:242
29725040,0irq/27-eth0-tx-0-21swapper/222:06:122
29725040,0irq/27-eth0-tx-0-21swapper/222:01:142
29725040,0irq/27-eth0-tx-0-21swapper/221:56:222
29725040,0irq/27-eth0-tx-0-21swapper/221:47:512
29725040,0irq/27-eth0-tx-0-21swapper/221:16:192
29725040,0irq/27-eth0-tx-0-21swapper/020:16:300
29725040,0irq/27-eth0-tx-0-21swapper/020:16:030
29715040,0irq/26-eth0-rx-24764-21kthreadcore23:16:263
29715040,0irq/26-eth0-rx-0-21swapper/323:01:233
29715040,0irq/26-eth0-rx-0-21swapper/322:46:273
29715040,0irq/26-eth0-rx-0-21swapper/321:31:543
29715040,0irq/26-eth0-rx-0-21swapper/321:11:273
29715040,0irq/26-eth0-rx-0-21swapper/320:56:213
29715040,0irq/26-eth0-rx-0-21swapper/320:46:273
29715040,0irq/26-eth0-rx-0-21swapper/300:26:263
29715040,0irq/26-eth0-rx-0-21swapper/300:16:253
259940,0migration/28034-21kthreadcore20:41:272
259940,0migration/230191-21kthreadcore19:21:282
259940,0migration/225476-21irqcore20:56:262
259940,0migration/224912-21kthreadcore20:06:292
259940,0migration/224232-21kthreadcore00:16:292
259940,0migration/222084-21runrttasks20:05:342
259940,0migration/217322-21fschecks_time20:26:242
259940,0migration/216959-21kthreadcore22:51:282
259940,0migration/216062-21kthreadcore21:31:272
259940,0migration/212257-21kthreadcore19:31:282
259940,0migration/210631-21missed_timers23:06:292
249940,0watchdog/26688-21memory19:26:292
249940,0watchdog/23112-21memory22:21:282
249940,0watchdog/227646-21munin-node22:16:282
179940,0migration/17515-21munin-node19:06:231
179940,0migration/16973-21munin-node22:06:241
179940,0migration/15045-21cpuspeed_turbos23:26:221
179940,0migration/14362-21kthreadcore19:51:271
179940,0migration/129437-21irqrtprio00:21:261
179940,0migration/12885-21kthreadcore19:01:271
179940,0migration/128523-21sh00:01:131
179940,0migration/126503-21runrttasks00:17:001
179940,0migration/125250-21if_eth120:56:251
179940,0migration/125167-21kthreadcore20:31:271
179940,0migration/123338-21kthreadcore23:16:261
179940,0migration/121134-21cpuspeed_turbos23:36:231
179940,0migration/11734-21kthreadcore20:36:281
179940,0migration/116154-21kthreadcore21:31:271
179940,0migration/114079-21latency_hist22:11:101
179940,0migration/11376-21kthreadcore21:01:251
179940,0migration/112905-21fschecks_time00:11:221
179940,0migration/110905-21kthreadcore19:31:271
169940,0watchdog/16559-21runrttasks20:30:371
169940,0watchdog/15719-21ssh21:44:561
169940,0watchdog/125078-21kthreadcore20:06:291
169940,0watchdog/113466-21ssh21:28:241
169940,0watchdog/10-21swapper/119:21:291
149940,0migration/08972-21kthreadcore23:26:280
149940,0migration/031546-21kthreadcore00:01:260
149940,0migration/020485-21runrttasks22:53:580
149940,0migration/01920-21kthreadcore21:21:250
8215030,0irq/24-0000:00:0-21swapper/219:46:332
325559932,0cyclictest2361-21basename19:51:212
325559932,0cyclictest15715-21sed19:36:262
325559932,0cyclictest15686-21sensors20:21:312
325559932,0cyclictest14917-21sed19:11:262
325509933,0cyclictest0-21swapper/119:46:451
325509932,0cyclictest6669-21aten_r1power_cu21:06:201
325509932,0cyclictest5214-21awk20:41:261
325509932,0cyclictest30681-21awk22:41:241
325509932,0cyclictest29764-21tail20:11:241
325509932,0cyclictest23612-21awk21:36:251
325509930,0cyclictest0-21swapper/119:17:351
325499933,0cyclictest24250-21kworker/0:219:36:400
325499933,0cyclictest24250-21kworker/0:219:26:200
325499933,0cyclictest22311-21kworker/0:200:16:130
325499933,0cyclictest22133-21kworker/0:022:56:110
325499933,0cyclictest22133-21kworker/0:022:06:210
325499933,0cyclictest22133-21kworker/0:021:31:230
325499933,0cyclictest22133-21kworker/0:021:06:200
325499933,0cyclictest22133-21kworker/0:020:56:140
325499933,0cyclictest22133-21kworker/0:020:36:140
325499933,0cyclictest22133-21kworker/0:020:21:180
29715030,0irq/26-eth0-rx-0-21swapper/322:36:203
29715030,0irq/26-eth0-rx-0-21swapper/320:26:253
29715030,0irq/26-eth0-rx-0-21swapper/320:16:323
29715030,0irq/26-eth0-rx-0-21swapper/320:11:223
29715030,0irq/26-eth0-rx-0-21swapper/319:02:063
249930,0watchdog/20-21swapper/220:36:242
19130,0ktimersoftd/10-21swapper/120:23:291
179930,0migration/17655-21runrttasks19:29:231
179930,0migration/124044-21kthreadcore22:36:261
179930,0migration/111373-21sh23:48:271
169930,0watchdog/10-21swapper/123:31:241
169930,0watchdog/10-21swapper/122:31:201
169930,0watchdog/10-21swapper/121:52:561
169930,0watchdog/10-21swapper/119:56:291
325509920,0cyclictest14492-21gltestperf19:11:241
29715020,0irq/26-eth0-rx-0-21swapper/320:31:293
29715020,0irq/26-eth0-rx-0-21swapper/319:06:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional