You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 00:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jan 09, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2299923440,2sleep023002-21missed_timers07:05:370
1113021070,6sleep32424099cyclictest11:00:373
24238997912,14cyclictest23-21ksoftirqd/108:40:261
2423999742,28cyclictest7950irq/56-dwc_otg_08:20:262
186072740,3sleep2301ktimersoftd/211:20:412
2424099673,56cyclictest0-21swapper/307:10:183
2423899675,9cyclictest7750irq/56-dwc_otg07:50:261
2423799659,37cyclictest0-21swapper/007:10:180
2424099634,42cyclictest7950irq/56-dwc_otg_09:35:323
2424099631,6cyclictest28822-21fschecks_count11:50:313
2423799631,23cyclictest9063-21ls12:25:310
2424099614,43cyclictest7950irq/56-dwc_otg_10:25:303
2424099611,11cyclictest9450irq/48-DMA10:44:553
2424099605,42cyclictest7950irq/56-dwc_otg_09:30:163
2423899602,13cyclictest0-21swapper/110:45:271
2423799603,44cyclictest0-21swapper/010:20:280
2423799603,44cyclictest0-21swapper/010:20:280
2424099596,35cyclictest5139-21aten_rbpower_vo10:45:303
2424099592,10cyclictest7850irq/56-dwc_otg_11:45:283
2424099591,6cyclictest23225-21/usr/sbin/munin10:05:453
2423799592,32cyclictest25061-21wget11:40:300
24237995913,5cyclictest10338-21latency_hist11:00:160
2424099585,39cyclictest7950irq/56-dwc_otg_11:25:153
2424099583,4cyclictest6355-21apt-get07:50:263
2424099581,46cyclictest7950irq/56-dwc_otg_08:15:293
24237995825,7cyclictest0-21swapper/007:30:260
2423799581,30cyclictest21245-21fschecks_time08:30:320
2423899572,9cyclictest7950irq/56-dwc_otg_10:35:261
24237995724,6cyclictest0-21swapper/011:30:260
24237995715,20cyclictest28494-21fschecks_time08:50:310
24237995715,20cyclictest28494-21fschecks_time08:50:310
2424099561,5cyclictest0-21swapper/309:33:103
2424099561,4cyclictest7950irq/56-dwc_otg_07:50:103
24237995624,5cyclictest101ktimersoftd/009:50:280
2424099553,5cyclictest7850irq/56-dwc_otg_12:15:323
2424099551,43cyclictest0-21swapper/310:11:383
2423999557,10cyclictest22511-21latency_hist08:35:172
2423799552,31cyclictest24327-21apt08:40:160
2423799552,29cyclictest17041-21grep11:15:440
24240995413,38cyclictest3262-21sed12:10:163
2423999543,13cyclictest0-21swapper/207:10:182
2423799547,4cyclictest0-21swapper/012:20:270
24237995414,19cyclictest5876-21missed_timers12:15:360
2423799541,30cyclictest16507-21sed08:15:430
2424099531,3cyclictest0-21swapper/311:05:303
24240995310,40cyclictest435-21cron08:00:153
2423799536,6cyclictest0-21swapper/011:10:280
2423799536,6cyclictest0-21swapper/011:10:280
24237995332,6cyclictest7950irq/56-dwc_otg_07:35:290
24237995324,9cyclictest10973-21fschecks_count12:30:330
2423799532,39cyclictest0-21swapper/010:50:310
2423799532,28cyclictest29227-21ntp_kernel_err11:50:360
2423799531,29cyclictest24565-21cron10:10:160
2424099521,9cyclictest7850irq/56-dwc_otg_10:42:233
2424099521,5cyclictest0-21swapper/312:25:413
2424099520,8cyclictest7950irq/56-dwc_otg_10:00:403
24238995212,25cyclictest0-21swapper/109:40:291
24238995212,25cyclictest0-21swapper/109:40:291
2423799522,26cyclictest0-21swapper/009:15:270
2424099518,39cyclictest834-21timerwakeupswit12:00:423
2424099511,7cyclictest30075-21latency_hist10:25:173
2424099511,7cyclictest30075-21latency_hist10:25:173
2424099511,4cyclictest0-21swapper/308:40:403
2423799515,30cyclictest0-21swapper/008:25:220
24237995127,4cyclictest0-21swapper/008:00:400
2423799511,27cyclictest2985-21cron10:40:150
2423799511,25cyclictest23183-21aten_rbpower_en11:35:280
2424099509,37cyclictest5432chrt09:00:463
2424099507,39cyclictest15914-21aten_rbpower_cu11:15:293
2424099502,7cyclictest14597-21wc11:10:363
2424099502,7cyclictest14597-21wc11:10:363
2424099502,6cyclictest3291-21fschecks_time09:10:323
24240995019,29cyclictest23057-21fschecks_time08:35:303
2424099501,8cyclictest7749-21unixbench_singl09:20:433
2424099501,8cyclictest26945-21fschecks_time10:15:323
2424099501,6cyclictest18788-21awk08:25:163
2424099501,5cyclictest12517-21aten_rbpower_cu12:35:283
2424099501,5cyclictest12517-21aten_rbpower_cu12:35:283
2424099501,40cyclictest0-21swapper/308:45:483
2423799502,28cyclictest9100-21ntpq09:25:390
2423799502,27cyclictest6178-21latency_hist07:50:160
24237995014,20cyclictest28187-21cron10:20:160
2423799501,29cyclictest12618-21tr12:35:290
2423799501,29cyclictest12618-21tr12:35:290
2423799501,28cyclictest3028-21grep07:40:320
2423799501,28cyclictest3028-21grep07:40:320
2423799501,24cyclictest16329-21ntpq09:45:350
24237995011,21cyclictest387-21uname10:30:410
2424099492,8cyclictest5206-21ls12:15:173
2424099491,8cyclictest6892-21aten_rbpower_en10:50:303
2424099491,6cyclictest22438-21sed11:30:433
2424099491,6cyclictest19407-21fschecks_time08:25:333
2424099491,4cyclictest7850irq/56-dwc_otg_11:28:023
2424099491,40cyclictest0-21swapper/309:52:463
2423799491,27cyclictest1830-21ntpq09:05:380
2423799491,26cyclictest31078-21grep08:55:430
2424099484,33cyclictest7950irq/56-dwc_otg_07:40:163
2424099482,7cyclictest13921-21aten_rbpower_po09:40:303
2424099482,7cyclictest13921-21aten_rbpower_po09:40:303
2424099482,6cyclictest30819-21tail07:25:443
2424099481,8cyclictest9665-21wc10:55:433
24240994817,28cyclictest1729-21fschecks_time10:35:323
2424099481,6cyclictest6400-21latency_hist09:20:163
2424099481,6cyclictest25063-21cut11:40:303
2423999484,7cyclictest7750irq/56-dwc_otg12:35:262
2423999484,7cyclictest7750irq/56-dwc_otg12:35:262
24237994824,6cyclictest31228-21proc_pri11:55:400
24237994820,6cyclictest9296-21grep07:55:450
2423799481,27cyclictest0-21swapper/009:35:390
2424099472,6cyclictest0-21swapper/307:15:323
2424099471,7cyclictest15764-21grep09:45:283
2424099471,6cyclictest23221-21aten_rbpower_vo11:35:283
2424099471,6cyclictest11094-21grep08:00:423
2424099471,5cyclictest31627-21cut07:30:293
2424099471,5cyclictest2227-21grep09:05:443
2424099471,5cyclictest11829-21wget08:05:283
2424099471,5cyclictest11792-21tail12:30:443
24237994721,14cyclictest0-21swapper/011:05:300
24237994719,17cyclictest0-21swapper/008:45:250
24237994719,15cyclictest14329-21ps08:10:390
24237994718,9cyclictest0-21swapper/007:50:270
24237994718,3cyclictest0-21swapper/009:30:260
2423799471,24cyclictest2375-21/usr/sbin/munin12:05:390
2424099467,8cyclictest0-21swapper/308:30:263
2424099467,4cyclictest29514-21rm08:52:243
2424099467,4cyclictest29514-21rm08:52:243
2424099462,5cyclictest0-21swapper/309:55:323
2424099462,34cyclictest0-21swapper/312:24:093
2424099461,42cyclictest3796-21grep07:40:423
2424099461,42cyclictest3796-21grep07:40:423
2423899469,24cyclictest0-21swapper/107:25:291
24237994614,4cyclictest0-21swapper/009:10:260
24237994614,22cyclictest0-21swapper/010:10:500
2423799461,27cyclictest21164-21acpi10:00:250
2423799461,25cyclictest0-21swapper/009:40:380
2423799461,25cyclictest0-21swapper/009:40:380
2424099451,7cyclictest30715-21grep11:55:333
2424099451,7cyclictest29528-21grep07:25:163
2424099451,6cyclictest30034-21aten_rbpower_en08:55:293
2423899457,24cyclictest0-21swapper/111:00:281
2423899451,7cyclictest0-21swapper/110:50:281
24237994525,7cyclictest7950irq/56-dwc_otg_12:10:400
2423799452,24cyclictest0-21swapper/010:45:170
2423799451,27cyclictest575-21wc12:00:390
2424099442,38cyclictest0-21swapper/308:10:283
2423899444,8cyclictest7750irq/56-dwc_otg07:10:171
2423899442,5cyclictest0-21swapper/107:15:271
2423899441,8cyclictest0-21swapper/108:20:271
2423799442,27cyclictest0-21swapper/010:25:350
2423799442,22cyclictest0-21swapper/009:55:290
24237994421,7cyclictest12524-21grep08:05:380
24237994419,13cyclictest0-21swapper/011:45:350
24237994411,24cyclictest0-21swapper/008:40:290
2423999437,25cyclictest0-21swapper/211:50:282
2423999437,25cyclictest0-21swapper/209:25:302
2423899436,23cyclictest0-21swapper/109:10:281
2423799432,21cyclictest0-21swapper/009:20:420
24237994318,18cyclictest0-21swapper/008:24:250
24237994314,25cyclictest4804-21munin-run10:45:160
24237994312,18cyclictest28946-21unixbench-2d07:20:420
2423999427,25cyclictest0-21swapper/208:00:282
2423899427,25cyclictest0-21swapper/107:50:111
2423899427,24cyclictest0-21swapper/108:05:271
2423899426,24cyclictest0-21swapper/108:25:221
24238994221,10cyclictest30264-21basename10:25:291
2423799421,28cyclictest0-21swapper/011:03:280
2423999407,21cyclictest0-21swapper/211:55:302
2423999401,24cyclictest0-21swapper/210:55:292
2423899407,24cyclictest0-21swapper/110:30:291
2423899407,23cyclictest0-21swapper/109:15:281
2423899407,23cyclictest0-21swapper/108:55:291
2423899407,23cyclictest0-21swapper/108:15:291
24238994028,7cyclictest221ktimersoftd/109:00:271
2423799401,22cyclictest0-21swapper/011:29:150
2423799401,19cyclictest30374-21ntpq07:25:380
2423899398,7cyclictest0-21swapper/107:55:271
24238993915,5cyclictest21221-21latency_hist11:30:161
2423899391,4cyclictest7850irq/56-dwc_otg_08:05:161
24238993910,5cyclictest7750irq/56-dwc_otg11:45:431
24237993917,5cyclictest452-21unixbench_multi09:00:420
24237993912,20cyclictest0-21swapper/007:16:090
45302381,3sleep24558-21modprobe09:15:152
2423899383,27cyclictest7950irq/56-dwc_otg_10:20:261
2423899383,27cyclictest7950irq/56-dwc_otg_10:20:261
2423899383,15cyclictest7850irq/56-dwc_otg_12:00:161
24238993817,4cyclictest27881-21/usr/sbin/munin07:20:421
2423899381,5cyclictest0-21swapper/112:30:291
2423899381,27cyclictest23617-21switchtime08:35:381
2423899381,23cyclictest26693-21cut08:45:321
148522380,1sleep20-21swapper/209:40:422
148522380,1sleep20-21swapper/209:40:422
24238993719,4cyclictest23-21ksoftirqd/109:30:161
2423899371,25cyclictest12918-21ls09:35:411
2423999361,10cyclictest0-21swapper/208:50:312
2423999361,10cyclictest0-21swapper/208:50:312
2423899368,15cyclictest0-21swapper/110:00:271
2423899367,7cyclictest30296-21cut11:55:171
2423899361,5cyclictest1124-21snmpd11:18:311
24238993610,5cyclictest7950irq/56-dwc_otg_12:25:451
2423799361,22cyclictest1125-21runrttasks11:20:500
2423999357,18cyclictest0-21swapper/212:00:282
2423999356,15cyclictest7850irq/56-dwc_otg_09:50:312
2423999355,20cyclictest7850irq/56-dwc_otg_07:50:152
2423999352,20cyclictest7750irq/56-dwc_otg07:24:482
2423999351,26cyclictest24767-21ls08:40:302
2423899357,12cyclictest0-21swapper/109:05:361
2423899354,5cyclictest7850irq/56-dwc_otg_09:30:331
2423899354,10cyclictest7750irq/56-dwc_otg07:35:271
238482352,25sleep10-21swapper/107:08:461
2423999348,16cyclictest0-21swapper/210:35:092
2423999342,20cyclictest7750irq/56-dwc_otg07:37:162
2423899349,16cyclictest0-21swapper/112:20:271
2423899348,5cyclictest7950irq/56-dwc_otg_12:10:161
2423899347,7cyclictest9450irq/48-DMA10:44:551
2423899347,23cyclictest0-21swapper/111:30:291
2423899342,8cyclictest0-21swapper/109:55:261
2423899342,21cyclictest28537-21ls08:50:321
2423899342,21cyclictest28537-21ls08:50:321
2423899342,16cyclictest7750irq/56-dwc_otg12:35:261
2423899342,16cyclictest7750irq/56-dwc_otg12:35:261
24238993415,5cyclictest1383-21cron12:05:151
2423899341,4cyclictest7850irq/56-dwc_otg_08:30:411
2423999339,4cyclictest0-21swapper/210:35:422
2423999336,14cyclictest0-21swapper/211:35:272
2423999336,14cyclictest0-21swapper/207:15:282
2423999334,10cyclictest7850irq/56-dwc_otg_07:30:262
24239993315,5cyclictest31552-21latency_hist09:00:162
2423899338,14cyclictest0-21swapper/112:10:291
2423899337,14cyclictest0-21swapper/110:15:281
2423899335,13cyclictest7950irq/56-dwc_otg_07:40:421
2423899335,13cyclictest7950irq/56-dwc_otg_07:40:421
2423899332,14cyclictest7750irq/56-dwc_otg07:30:331
2423899331,24cyclictest24568-21latency_hist10:10:171
157032330,3sleep1221ktimersoftd/111:15:161
157032330,3sleep1221ktimersoftd/111:15:161
2423999327,15cyclictest0-21swapper/210:25:362
2423999325,8cyclictest7850irq/56-dwc_otg_11:05:302
2423999325,14cyclictest7950irq/56-dwc_otg_10:50:402
2423999325,12cyclictest7950irq/56-dwc_otg_08:15:442
2423999324,5cyclictest7850irq/56-dwc_otg_07:40:322
2423999324,5cyclictest7850irq/56-dwc_otg_07:40:322
24239993214,5cyclictest12443-21if_eth009:35:332
2423999321,14cyclictest7750irq/56-dwc_otg11:20:162
2423899325,16cyclictest0-21swapper/111:05:271
24238993224,6cyclictest24274-21head11:35:421
2423899321,22cyclictest5497-21cut12:15:321
2423899321,22cyclictest14594-21timerwakeupswit08:10:421
2423899321,17cyclictest0-21swapper/111:00:161
2423999319,6cyclictest0-21swapper/209:15:372
2423999317,21cyclictest0-21swapper/208:25:292
2423999315,4cyclictest7850irq/56-dwc_otg_12:25:332
2423999313,5cyclictest7850irq/56-dwc_otg_09:55:282
2423999313,20cyclictest0-21swapper/210:00:582
2423999312,6cyclictest5932-21ntpq12:15:372
24239993122,7cyclictest10538-21aten_rbpower_po11:00:292
2423999311,18cyclictest4828-21ls10:45:162
2423999311,14cyclictest7950irq/56-dwc_otg_08:45:422
24239993111,14cyclictest31-21ksoftirqd/210:20:282
24239993111,14cyclictest31-21ksoftirqd/210:20:282
2423899317,14cyclictest0-21swapper/111:20:281
2423899317,14cyclictest0-21swapper/109:50:221
2423899311,21cyclictest3491-21uniq10:40:321
240632316,13sleep224004-21ls07:10:172
2423999308,10cyclictest24958-21basename11:40:292
2423999306,10cyclictest9225-21unixbench-2d07:55:442
2423999303,18cyclictest7850irq/56-dwc_otg_11:46:392
2423999302,7cyclictest10810-21wget12:30:312
2423999302,4cyclictest7950irq/56-dwc_otg_12:10:272
2423999302,4cyclictest7850irq/56-dwc_otg_09:20:422
24239993024,4cyclictest8656-21munin-plugin-st12:25:162
2423999302,3cyclictest7950irq/56-dwc_otg_12:05:252
24239993015,5cyclictest1125-21runrttasks11:10:202
24239993015,5cyclictest1125-21runrttasks11:10:202
2423999301,20cyclictest22911-21ls08:35:272
2423999298,12cyclictest7750irq/56-dwc_otg09:05:392
2423999295,13cyclictest22882-21apt11:35:162
2423999294,6cyclictest7950irq/56-dwc_otg_10:50:162
2423999294,12cyclictest15638-21/usr/sbin/munin09:45:342
2423999292,20cyclictest0-21swapper/209:00:332
2423899292,7cyclictest24797-21/usr/sbin/munin10:10:321
2423899291,22cyclictest0-21swapper/111:40:271
2423999282,7cyclictest0-21swapper/207:25:302
2423999282,19cyclictest0-21swapper/210:10:122
2423999282,16cyclictest7850irq/56-dwc_otg_10:10:342
2423999282,15cyclictest7850irq/56-dwc_otg_07:54:412
24239992820,6cyclictest21075-21munin-plugin-st11:30:152
2423999281,5cyclictest7850irq/56-dwc_otg_10:15:422
2423999281,4cyclictest7850irq/56-dwc_otg_08:05:252
2423899287,5cyclictest7750irq/56-dwc_otg09:20:311
2423999275,5cyclictest7850irq/56-dwc_otg_08:10:372
2423999271,13cyclictest0-21swapper/209:32:272
237772262,14sleep30-21swapper/307:07:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional