You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 09:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Thu Mar 26, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271520,0ktimersoftd/211938-21perf19:00:102
1839724128,0sleep10-21swapper/118:59:131
1831923726,0sleep30-21swapper/318:58:153
1811323423,0sleep00-21swapper/018:55:520
35160,0ktimersoftd/30-21swapper/320:05:433
339960,0migration/38990-21aten_r1power_cu21:35:293
339960,0migration/37227-21sh21:30:573
339960,0migration/35343-21sh00:13:153
339960,0migration/324708-21sh21:05:133
339960,0migration/321238-21switchtime23:20:413
339960,0migration/320550-21sh00:20:503
339960,0migration/312252-21munin-node23:55:373
185719960,0cyclictest0-21swapper/000:12:070
149960,0migration/029657-21sh00:29:030
149960,0migration/023428-21aten_r1power_en23:25:290
8215050,0irq/24-0000:00:20493-21idleruntime-cro19:50:202
339950,0migration/39613-21munin-node22:15:343
339950,0migration/37119-21sh21:51:293
339950,0migration/35098-21munin-node23:10:393
339950,0migration/34437-21kthreadcore20:00:363
339950,0migration/330907-21ntp_offset21:25:393
339950,0migration/330254-21sh22:46:023
339950,0migration/3300-21kthreadcore23:30:353
339950,0migration/329675-21unixbench-2d23:05:413
339950,0migration/329172-21idleruntime20:20:343
339950,0migration/326479-21kthreadcore22:05:363
339950,0migration/326086-21munin-node19:50:383
339950,0migration/324806-21kthreadcore19:25:383
339950,0migration/324309-21runrttasks21:44:043
339950,0migration/323750-21fschecks_time23:45:313
339950,0migration/322636-21cpuspeed_turbos00:25:303
339950,0migration/321812-21munin-node22:40:413
339950,0migration/32112-21kthreadcore22:50:363
339950,0migration/321077-21taskset20:10:563
339950,0migration/317327-21aten_r1power_fr21:20:303
339950,0migration/316424-21taskset20:56:233
339950,0migration/316334-21kthreadcore19:45:373
29725050,0irq/27-eth0-tx-9508-21ssh00:15:362
29725050,0irq/27-eth0-tx-8702-21munin-node21:55:412
29725050,0irq/27-eth0-tx-31908-21ssh21:26:562
29725050,0irq/27-eth0-tx-30778-21ps21:05:412
29725050,0irq/27-eth0-tx-29406-21ssh00:28:172
29725050,0irq/27-eth0-tx-22563-21ssh23:02:482
29725050,0irq/27-eth0-tx-15015-21munin-node23:40:332
29725050,0irq/27-eth0-tx-14673-21ssh00:00:212
29725050,0irq/27-eth0-tx-0-21swapper/223:50:342
29725050,0irq/27-eth0-tx-0-21swapper/223:13:552
29725050,0irq/27-eth0-tx-0-21swapper/222:55:392
29725050,0irq/27-eth0-tx-0-21swapper/221:20:492
29725050,0irq/27-eth0-tx-0-21swapper/200:21:022
29725050,0irq/27-eth0-tx-0-21swapper/021:00:270
29725050,0irq/27-eth0-tx-0-21swapper/019:11:470
29715050,0irq/26-eth0-rx-9204-21irqrtprio22:35:363
29715050,0irq/26-eth0-rx-3090-21kthreadcore22:30:393
29715050,0irq/26-eth0-rx-302232sleep300:09:443
29715050,0irq/26-eth0-rx-2183-21cut22:10:343
29715050,0irq/26-eth0-rx-18142-21kthreadcore22:00:363
29715050,0irq/26-eth0-rx-1607-21hwlatdetect21:10:323
29715050,0irq/26-eth0-rx-13525-21sh23:36:453
29715050,0irq/26-eth0-rx-12468-21cut21:55:353
29715050,0irq/26-eth0-rx-10644-21cut23:15:363
29715050,0irq/26-eth0-rx-0-21swapper/323:50:353
29715050,0irq/26-eth0-rx-0-21swapper/323:26:233
29715050,0irq/26-eth0-rx-0-21swapper/323:04:223
29715050,0irq/26-eth0-rx-0-21swapper/322:55:213
29715050,0irq/26-eth0-rx-0-21swapper/322:25:523
29715050,0irq/26-eth0-rx-0-21swapper/300:15:273
259950,0migration/29206-21idleruntime20:05:352
259950,0migration/24433-21unixbench_singl00:10:422
259950,0migration/23447-21hwlatdetect20:25:332
259950,0migration/227920-21latency20:40:372
259950,0migration/221775-21sh23:21:252
259950,0migration/215915-21sh21:17:272
259950,0migration/215123-21latency_hist23:20:192
259950,0migration/211317-21kthreadcore20:30:352
185739954,0cyclictest24888-21awk22:45:332
185739953,0cyclictest31075-21sadc23:30:192
185739950,0cyclictest0-21swapper/221:45:312
185729950,0cyclictest0-21swapper/100:22:431
185719955,0cyclictest0-21swapper/022:51:330
185719955,0cyclictest0-21swapper/022:49:390
179950,0migration/19996-21munin-node21:55:341
179950,0migration/1929-21kthreadcore00:10:361
179950,0migration/17044-21fschecks_time19:15:321
179950,0migration/16352-21sh22:52:221
179950,0migration/13119-21kthreadcore20:00:361
179950,0migration/130162-21munin-node22:05:391
179950,0migration/129263-21kthreadcore21:45:371
179950,0migration/126396-21kthreadcore23:05:351
179950,0migration/124832-21idleruntime22:45:321
179950,0migration/115611-21taskset21:16:371
179950,0migration/113970-21proc_pri22:15:381
179950,0migration/113895-21munin-node19:20:341
169950,0watchdog/121952-21pmu-power22:20:371
169950,0watchdog/115592-21ssh22:39:571
149950,0migration/07885-21latency_hist22:35:190
149950,0migration/07197-21fschecks_count23:55:310
149950,0migration/05417-21kthreadcore19:35:360
149950,0migration/03747-21kthreadcore21:10:350
149950,0migration/03301-21kthreadcore21:30:350
149950,0migration/031834-21hddtemp_smartct23:50:310
149950,0migration/031630-21taskset19:07:500
149950,0migration/031267-21kthreadcore20:45:360
149950,0migration/030466-21latency21:05:380
149950,0migration/027712-21kthreadcore21:25:360
149950,0migration/027302-21kthreadcore20:40:360
149950,0migration/026809-21kthreadcore22:05:360
149950,0migration/025303-21cpuspeed_turbos21:45:320
149950,0migration/024699-21munin-node23:05:330
149950,0migration/023148-21kthreadcore19:00:370
149950,0migration/022627-21latency_hist00:05:190
149950,0migration/022122-21fschecks_count20:15:330
149950,0migration/020277-21fschecks_count19:25:330
149950,0migration/020152-21kthreadcore20:10:370
149950,0migration/019141-21munin-node22:40:350
149950,0migration/01859-21kthreadcore22:10:340
149950,0migration/017342-21kthreadcore23:20:350
149950,0migration/016978-21fschecks_time22:20:300
149950,0migration/015981-21runrttasks21:59:070
149950,0migration/015576-21munin-node00:20:350
149950,0migration/012582-21taskset19:17:380
339940,0migration/34019-21kthreadcore20:25:353
339940,0migration/330613-21kernelversion20:45:363
339940,0migration/328335-21kthreadcore19:05:373
339940,0migration/323941-21kthreadcore20:15:373
339940,0migration/319080-21kthreadcore00:00:373
339940,0migration/317314-21kthreadcore20:35:343
339940,0migration/315607-21kthreadcore19:20:363
339940,0migration/312301-21kthreadcore20:30:353
29725040,0irq/27-eth0-tx-22575-21ntp_kernel_pll_21:40:372
29725040,0irq/27-eth0-tx-1482-21munin-node22:10:332
29725040,0irq/27-eth0-tx-0-21swapper/223:55:492
29725040,0irq/27-eth0-tx-0-21swapper/223:35:202
29725040,0irq/27-eth0-tx-0-21swapper/223:30:292
29725040,0irq/27-eth0-tx-0-21swapper/222:51:092
29725040,0irq/27-eth0-tx-0-21swapper/222:40:292
29725040,0irq/27-eth0-tx-0-21swapper/222:35:372
29725040,0irq/27-eth0-tx-0-21swapper/222:30:342
29725040,0irq/27-eth0-tx-0-21swapper/222:05:202
29725040,0irq/27-eth0-tx-0-21swapper/221:50:582
29725040,0irq/27-eth0-tx-0-21swapper/221:35:392
29725040,0irq/27-eth0-tx-0-21swapper/221:30:292
29725040,0irq/27-eth0-tx-0-21swapper/221:10:222
29725040,0irq/27-eth0-tx-0-21swapper/221:01:302
29725040,0irq/27-eth0-tx-0-21swapper/020:54:150
29725040,0irq/27-eth0-tx-0-21swapper/020:20:310
29725040,0irq/27-eth0-tx-0-21swapper/019:46:410
29715040,0irq/26-eth0-rx-11605-21kthreadcore21:15:373
29715040,0irq/26-eth0-rx-0-21swapper/323:40:573
29715040,0irq/26-eth0-rx-0-21swapper/322:21:013
29715040,0irq/26-eth0-rx-0-21swapper/321:45:193
29715040,0irq/26-eth0-rx-0-21swapper/321:05:313
259940,0migration/229992-21fschecks_time20:45:332
259940,0migration/228046-21kthreadcore19:30:352
259940,0migration/226651-21kthreadcore00:05:362
259940,0migration/225226-21kthreadcore19:50:372
259940,0migration/223691-21sh23:05:262
259940,0migration/223441-21kthreadcore20:15:362
259940,0migration/223384-21cpuspeed_turbos23:45:302
259940,0migration/220542-21kthreadcore22:00:362
259940,0migration/212073-21kthreadcore22:15:352
249940,0watchdog/224326-21ssh22:25:252
249940,0watchdog/217582-21irqrtprio22:20:332
249940,0watchdog/217475-21kthreadcore20:35:342
249940,0watchdog/20-21swapper/219:20:382
185739943,0cyclictest8389-21awk19:40:342
185739943,0cyclictest7589-21cut19:15:352
185739943,0cyclictest4459-21awk20:50:332
185739943,0cyclictest3514-21snmpd19:07:482
185739943,0cyclictest20622-21awk19:25:352
185739943,0cyclictest1613-21awk19:35:342
185739943,0cyclictest16020-21awk20:10:352
185739943,0cyclictest11279-21awk20:55:342
185729943,0cyclictest7695-21awk23:55:331
185729943,0cyclictest3691-21grep20:25:351
185729943,0cyclictest3556-21lldpd00:25:251
185729943,0cyclictest3514-21snmpd23:41:331
185729943,0cyclictest3514-21snmpd21:28:031
185729943,0cyclictest3514-21snmpd21:24:331
185729943,0cyclictest3514-21snmpd20:35:331
185729943,0cyclictest3514-21snmpd19:05:481
185729943,0cyclictest3241-21irqbalance21:50:481
185729943,0cyclictest3241-21irqbalance20:10:381
185729943,0cyclictest17610-21awk22:00:341
185729943,0cyclictest15590-21awk00:00:341
185729943,0cyclictest0-21swapper/119:00:371
185729940,0cyclictest0-21swapper/123:46:081
185719944,0cyclictest0-21swapper/023:18:590
185719943,0cyclictest7257-21awk00:15:330
185719943,0cyclictest3514-21snmpd23:30:330
185719943,0cyclictest3241-21irqbalance22:16:170
185719943,0cyclictest28278-21awk19:55:340
185719943,0cyclictest16542-21awk23:00:330
185719943,0cyclictest15942-21awk23:40:330
185719943,0cyclictest11255-21awk20:55:330
185719940,0cyclictest0-21swapper/019:45:120
179940,0migration/18518-21kthreadcore23:35:351
179940,0migration/15572-21kthreadcore22:10:351
179940,0migration/13907-21kthreadcore23:30:361
179940,0migration/128634-21kthreadcore22:25:361
179940,0migration/12658-21kthreadcore19:35:351
179940,0migration/124890-21latency_hist21:45:191
179940,0migration/122011-21unixbench-2d22:40:411
179940,0migration/11974-21kthreadcore22:30:391
179940,0migration/115287-21latency_hist23:20:191
179940,0migration/113578-21sh23:15:581
179940,0migration/112199-21switchtime00:15:411
179940,0migration/112094-21kthreadcore21:35:351
179940,0migration/11130-21cpuspeed_turbos21:10:301
169940,0watchdog/16516-21awk21:30:371
169940,0watchdog/128753-21ssh23:25:371
169940,0watchdog/126441-21switchtime19:50:421
169940,0watchdog/121852sleep120:23:101
169940,0watchdog/120511-21smartctl19:25:341
169940,0watchdog/117153-21kthreadcore19:45:381
169940,0watchdog/115655-21ssh23:00:291
169940,0watchdog/10-21swapper/120:05:381
149940,0migration/028837-21sensors_temp23:45:400
149940,0migration/022926-21runrttasks22:01:050
149940,0migration/020771-21kthreadcore21:40:350
149940,0migration/019845-21kthreadcore20:35:340
149940,0migration/014398-21diskmemload22:57:300
29715030,0irq/26-eth0-rx-0-21swapper/320:53:463
29715030,0irq/26-eth0-rx-0-21swapper/320:43:113
29715030,0irq/26-eth0-rx-0-21swapper/319:43:333
29715030,0irq/26-eth0-rx-0-21swapper/319:37:513
29715030,0irq/26-eth0-rx-0-21swapper/319:15:303
185739932,0cyclictest3514-21snmpd19:10:482
185739932,0cyclictest3514-21snmpd19:00:482
185739932,0cyclictest29191-21awk20:20:342
185739932,0cyclictest2464-21awk20:00:342
185739930,0cyclictest27970-21perl19:55:332
185729933,0cyclictest3556-21lldpd23:50:231
185729933,0cyclictest26064-21awk21:05:351
185729933,0cyclictest10210-21awk20:30:331
185729932,0cyclictest8614-21awk22:55:341
185729932,0cyclictest8365-21awk19:40:341
185729932,0cyclictest507-21awk19:10:351
185729932,0cyclictest4577-21awk20:50:341
185729932,0cyclictest3241-21irqbalance20:15:381
185729932,0cyclictest18405-21kthreadcore21:00:351
185729932,0cyclictest11185-21awk20:55:331
185729930,0cyclictest27793-21diskstats19:55:321
185719933,0cyclictest31764-21kworker/0:023:35:300
185719933,0cyclictest31764-21kworker/0:022:30:380
185719933,0cyclictest31764-21kworker/0:022:25:540
185719933,0cyclictest31764-21kworker/0:021:50:390
185719933,0cyclictest21279-21kworker/0:200:00:360
185719933,0cyclictest14841-21kworker/0:221:35:250
185719933,0cyclictest14841-21kworker/0:221:15:270
185719933,0cyclictest14841-21kworker/0:220:25:230
185719933,0cyclictest14841-21kworker/0:220:05:250
185719933,0cyclictest14841-21kworker/0:220:00:250
185719933,0cyclictest14841-21kworker/0:219:50:230
185719933,0cyclictest10181-21kworker/0:119:30:270
185719933,0cyclictest10181-21kworker/0:119:20:210
185719933,0cyclictest0-21swapper/020:30:270
185719932,0cyclictest32662-21awk23:10:330
185719932,0cyclictest18116-21awk21:20:330
179930,0migration/13421-21latency_hist20:50:191
169930,0watchdog/10-21swapper/123:14:531
169930,0watchdog/10-21swapper/120:40:301
169930,0watchdog/10-21swapper/119:30:301
169930,0watchdog/10-21swapper/100:06:011
29715020,0irq/26-eth0-rx-0-21swapper/319:55:213
29715020,0irq/26-eth0-rx-0-21swapper/319:31:133
29715020,0irq/26-eth0-rx-0-21swapper/319:10:333
29715020,0irq/26-eth0-rx-0-21swapper/319:00:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional