You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 14:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Tue Dec 23, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2860499841,9cyclictest0-21swapper/021:05:170
28604998136,18cyclictest0-21swapper/019:10:090
2860499771,22cyclictest14354-21cron21:30:060
2860499741,43cyclictest0-21swapper/020:25:060
2861699665,44cyclictest1366-21cron19:25:063
28616996412,38cyclictest4628-21cat00:00:293
2860799644,28cyclictest7750irq/56-dwc_otg19:10:091
157942640,2sleep30-21swapper/300:30:313
2860499631,5cyclictest12571-21munin-run21:25:060
2860499631,37cyclictest1661-21aten_rbpower_cu19:25:200
2861699616,44cyclictest7950irq/56-dwc_otg_00:25:303
28616996110,5cyclictest17508-21unixbench_multi21:35:343
2861599613,16cyclictest7750irq/56-dwc_otg19:10:182
2860499612,10cyclictest17006-21grep20:05:330
2861699593,6cyclictest0-21swapper/319:10:093
2861599592,13cyclictest0-21swapper/223:50:192
2861599592,10cyclictest0-21swapper/223:20:212
2860799597,18cyclictest0-21swapper/100:20:211
28607995915,19cyclictest7850irq/56-dwc_otg_22:15:191
2860499596,8cyclictest0-21swapper/023:25:180
2861699588,47cyclictest22096-21fschecks_count21:50:263
2861699584,44cyclictest7950irq/56-dwc_otg_20:20:073
2860799583,5cyclictest0-21swapper/119:30:191
28604995816,25cyclictest25191-21idleruntime-cro22:00:070
2861599577,41cyclictest0-21swapper/222:30:202
28604995719,19cyclictest0-21swapper/000:10:060
2861699564,43cyclictest7850irq/56-dwc_otg_23:30:333
2861699564,43cyclictest1125-21runrttasks22:25:363
2861699561,46cyclictest23775-21cat20:25:233
2861699561,12cyclictest14893-21/usr/sbin/munin21:30:233
2861599562,11cyclictest0-21swapper/221:25:182
2860799567,17cyclictest0-21swapper/121:10:201
2860799563,10cyclictest0-21swapper/121:50:211
2861699556,4cyclictest7950irq/56-dwc_otg_22:30:303
2861699551,5cyclictest23830-21df_inode21:55:233
2861699551,45cyclictest7127-21/usr/sbin/munin19:40:283
2861699551,45cyclictest10110-21runrttasks22:45:333
2861699551,45cyclictest10110-21runrttasks22:45:333
2861699544,6cyclictest7950irq/56-dwc_otg_21:02:583
2861699543,5cyclictest11468-21cut00:20:223
2861699541,12cyclictest2699-21/usr/sbin/munin23:55:273
28604995410,11cyclictest0-21swapper/020:45:200
2861699538,41cyclictest0-21swapper/323:10:253
2861699532,9cyclictest0-21swapper/323:05:073
2861699531,4cyclictest7950irq/56-dwc_otg_23:24:593
28616995314,36cyclictest12778-21fschecks_count19:55:253
2861699531,42cyclictest0-21swapper/321:30:063
2861599537,26cyclictest0-21swapper/219:40:162
2860499539,5cyclictest9550irq/80-mmc023:15:060
28604995341,4cyclictest0-21swapper/022:30:120
28604995328,6cyclictest7850irq/56-dwc_otg_19:50:240
2860499531,31cyclictest19308-21unixbench_multi21:40:320
2860499531,30cyclictest0-21swapper/019:55:370
2861699523,4cyclictest7950irq/56-dwc_otg_20:25:063
2861699523,10cyclictest2210-21latency_hist23:55:073
2861699522,7cyclictest7950irq/56-dwc_otg_19:29:593
2861699521,7cyclictest8877-21grep00:10:353
28604995232,4cyclictest9550irq/80-mmc020:40:070
28604995232,4cyclictest9550irq/80-mmc020:40:070
28604995225,11cyclictest0-21swapper/022:55:070
2860499522,28cyclictest0-21swapper/020:55:070
2860499521,29cyclictest5761-21latency_hist00:05:070
2861699515,4cyclictest7950irq/56-dwc_otg_23:05:273
2861699512,6cyclictest0-21swapper/320:30:203
2861699511,42cyclictest0-21swapper/323:46:493
2861699511,3cyclictest7950irq/56-dwc_otg_19:30:213
2861699511,3cyclictest0-21swapper/321:20:243
2860499515,24cyclictest0-21swapper/023:00:060
28604995132,6cyclictest7950irq/56-dwc_otg_21:50:360
28604995129,10cyclictest0-21swapper/000:35:230
28604995126,11cyclictest7850irq/56-dwc_otg_21:00:310
28604995124,5cyclictest14415-21wget20:00:220
28604995124,5cyclictest14415-21wget20:00:220
28604995122,13cyclictest9-21ksoftirqd/000:25:200
28604995121,16cyclictest1125-21runrttasks21:45:320
2860499511,21cyclictest0-21swapper/020:15:340
2861699502,6cyclictest17944-21grep00:35:383
2861699501,9cyclictest31202-21ps19:15:313
2861699501,5cyclictest2992-21sed20:55:353
2861699501,4cyclictest7950irq/56-dwc_otg_21:05:173
28616995012,34cyclictest20916-21tr23:15:313
2861599503,9cyclictest0-21swapper/219:50:202
2861599503,24cyclictest0-21swapper/221:05:172
2860799502,7cyclictest0-21swapper/121:15:191
2860499502,23cyclictest0-21swapper/020:30:060
2861699494,4cyclictest7850irq/56-dwc_otg_22:03:473
2861699491,7cyclictest28999-21grep20:40:213
2861699491,6cyclictest9440-21fschecks_count21:15:233
2861699491,5cyclictest0-21swapper/321:46:563
2861699491,4cyclictest5923-21/usr/sbin/munin19:35:303
2861699491,3cyclictest7950irq/56-dwc_otg_22:40:183
2861699491,3cyclictest0-21swapper/322:59:523
2861699491,3cyclictest0-21swapper/322:37:183
2861699491,2cyclictest0-21swapper/319:55:073
2861699491,13cyclictest326432chrt22:20:143
2860799492,7cyclictest7750irq/56-dwc_otg23:45:181
2860799492,7cyclictest0-21swapper/100:30:181
2860799492,4cyclictest0-21swapper/121:20:181
2860499492,27cyclictest6331-21unixbench_multi19:35:360
2860499491,29cyclictest12448-21wc00:20:360
2861699483,6cyclictest28239-21unixbench_singl20:35:343
2861699483,6cyclictest28239-21unixbench_singl20:35:343
2861699482,8cyclictest17256-21unixbench_multi20:05:353
2861699482,37cyclictest0-21swapper/322:06:193
2861699482,37cyclictest0-21swapper/322:06:193
2861699481,8cyclictest8938-21cron21:15:073
2861699481,8cyclictest16094-21latency_hist20:05:073
2861699481,8cyclictest16094-21latency_hist20:05:073
2860799488,25cyclictest0-21swapper/120:30:121
2860499481,27cyclictest17955-21awk20:10:190
28604994810,24cyclictest0-21swapper/000:30:240
49962470,2sleep10-21swapper/100:00:351
2861699472,8cyclictest31150-21fschecks_time22:15:233
2861699472,7cyclictest30901-21idleruntime-cro23:45:063
2861699472,7cyclictest28504-21ls23:35:333
2861699472,7cyclictest28504-21ls23:35:333
2861699472,6cyclictest30604-21munin-run22:15:063
28616994711,33cyclictest9271-21fschecks_time19:45:233
28616994711,33cyclictest323002chrt20:49:253
2860799477,28cyclictest0-21swapper/122:30:131
2860799477,25cyclictest0-21swapper/120:10:211
2860799477,24cyclictest0-21swapper/123:05:211
2860799476,26cyclictest0-21swapper/122:05:211
2860799476,26cyclictest0-21swapper/122:05:211
2860499479,19cyclictest6409-21missed_timers22:35:260
2860499472,28cyclictest10710-21grep00:15:360
2860499472,24cyclictest0-21swapper/000:00:070
28604994710,6cyclictest0-21swapper/023:55:060
2861699467,7cyclictest7850irq/56-dwc_otg_20:54:573
2861699467,25cyclictest0-21swapper/322:50:213
2861699461,9cyclictest0-21swapper/321:40:273
2861699461,5cyclictest0-21swapper/320:10:323
2861599467,5cyclictest7950irq/56-dwc_otg_20:25:172
2861599467,24cyclictest0-21swapper/222:55:212
2861599464,10cyclictest7850irq/56-dwc_otg_20:00:192
2861599464,10cyclictest7850irq/56-dwc_otg_20:00:192
2860799467,27cyclictest0-21swapper/123:55:221
2860799466,25cyclictest0-21swapper/120:20:201
2860499466,26cyclictest0-21swapper/021:35:190
2860499462,23cyclictest0-21swapper/020:30:240
28604994614,22cyclictest0-21swapper/020:40:390
2860499461,33cyclictest26599-21grep23:30:330
2861699452,5cyclictest0-21swapper/300:20:023
2861699451,7cyclictest24233-21fschecks_time23:25:233
2861699451,11cyclictest0-21swapper/300:08:573
2861599453,6cyclictest0-21swapper/223:40:202
2860799457,25cyclictest0-21swapper/121:30:201
2860799457,23cyclictest0-21swapper/120:40:201
28607994511,24cyclictest0-21swapper/122:20:221
2860499451,5cyclictest7818-21ntp_kernel_pll_19:40:270
2860499451,27cyclictest10174-21/usr/sbin/munin22:45:350
2860499451,27cyclictest10174-21/usr/sbin/munin22:45:350
2860499451,23cyclictest30787-21fschecks_time19:15:240
28604994511,19cyclictest2345-21cat20:55:260
279712453,14sleep00-21swapper/019:06:410
2860799449,25cyclictest0-21swapper/121:05:191
2860499442,7cyclictest16595-21runrttasks23:05:070
28604994419,5cyclictest4516-21tail19:30:350
28604994417,14cyclictest1125-21runrttasks22:04:040
28604994414,14cyclictest7750irq/56-dwc_otg21:35:070
2860499441,22cyclictest13-21rcuc/022:40:240
28604994410,19cyclictest1125-21runrttasks23:41:500
2861599434,24cyclictest0-21swapper/222:00:212
2861599434,18cyclictest7950irq/56-dwc_otg_19:45:192
2860799437,23cyclictest0-21swapper/122:25:211
2860799436,27cyclictest0-21swapper/119:35:211
2860799434,6cyclictest0-21swapper/123:30:201
2860499432,31cyclictest0-21swapper/021:10:350
28604994320,15cyclictest0-21swapper/022:11:520
28604994319,13cyclictest0-21swapper/021:15:340
2861599427,23cyclictest0-21swapper/222:10:202
2860799424,7cyclictest7850irq/56-dwc_otg_23:20:201
2860499426,24cyclictest0-21swapper/023:45:200
28604994216,5cyclictest9273-21awk00:15:060
2860499421,29cyclictest0-21swapper/023:35:330
2860499421,29cyclictest0-21swapper/023:35:330
2861599417,22cyclictest0-21swapper/221:15:202
28607994117,5cyclictest21-21rcuc/123:02:231
28607994115,12cyclictest0-21swapper/120:45:121
28604994115,18cyclictest0-21swapper/023:07:240
2860499411,22cyclictest2017-21/usr/sbin/munin22:25:190
2860499411,21cyclictest0-21swapper/023:20:060
64112400,2sleep20-21swapper/200:05:272
2861599407,24cyclictest0-21swapper/222:20:222
28607994011,6cyclictest28600-21cyclictest19:25:061
28604994014,17cyclictest0-21swapper/022:20:360
283372400,5sleep30-21swapper/319:10:073
2861599396,17cyclictest7850irq/56-dwc_otg_19:40:072
2861599393,7cyclictest7850irq/56-dwc_otg_23:05:202
2860799393,14cyclictest221ktimersoftd/122:45:221
2860799393,14cyclictest221ktimersoftd/122:45:221
28607993919,4cyclictest722550wget19:40:191
28615993817,5cyclictest9545-21aten_rbpower_vo22:45:252
28615993817,5cyclictest9545-21aten_rbpower_vo22:45:252
2860799389,7cyclictest7850irq/56-dwc_otg_20:35:341
2860799389,7cyclictest7850irq/56-dwc_otg_20:35:341
28607993824,4cyclictest9342-21latency_hist00:15:071
2860499382,21cyclictest0-21swapper/022:15:440
2860499382,20cyclictest0-21swapper/022:05:430
2860499382,20cyclictest0-21swapper/022:05:430
28604993816,9cyclictest0-21swapper/023:20:470
2861599372,20cyclictest0-21swapper/200:35:172
2860799373,5cyclictest7850irq/56-dwc_otg_23:40:211
2860799371,7cyclictest21-21rcuc/121:05:071
28607993717,6cyclictest23-21ksoftirqd/119:45:241
28607993711,16cyclictest221ktimersoftd/122:55:221
28604993714,13cyclictest0-21swapper/019:45:310
28604993710,18cyclictest0-21swapper/019:22:330
2861599365,15cyclictest7950irq/56-dwc_otg_20:30:332
28615993615,6cyclictest239050wget23:55:232
28615993614,4cyclictest30960-21/usr/sbin/munin22:15:202
2861599361,25cyclictest18617-21ls23:10:232
2860799368,15cyclictest0-21swapper/100:25:211
2860799367,5cyclictest0-21swapper/120:05:211
2860799367,16cyclictest0-21swapper/100:35:231
2860799367,15cyclictest7850irq/56-dwc_otg_21:55:241
2860799366,7cyclictest23-21ksoftirqd/122:00:251
2860799363,4cyclictest7750irq/56-dwc_otg21:45:191
28607993627,7cyclictest3538-21sh21:00:061
2860799362,25cyclictest0-21swapper/122:40:201
2860799361,4cyclictest7950irq/56-dwc_otg_22:10:181
28607993612,11cyclictest7750irq/56-dwc_otg20:25:191
2860799361,12cyclictest221ktimersoftd/123:10:361
117292360,15sleep17750irq/56-dwc_otg22:50:271
2861599359,5cyclictest7950irq/56-dwc_otg_23:25:232
2861599358,4cyclictest7950irq/56-dwc_otg_19:20:362
2861599356,13cyclictest7850irq/56-dwc_otg_23:35:222
2861599356,13cyclictest7850irq/56-dwc_otg_23:35:222
28615993519,4cyclictest5811-21/usr/sbin/munin22:35:322
28615993517,5cyclictest18543-21fschecks_time21:40:222
2860799357,16cyclictest7950irq/56-dwc_otg_23:35:321
2860799357,16cyclictest7950irq/56-dwc_otg_23:35:321
2860799357,11cyclictest7850irq/56-dwc_otg_00:05:241
28607993516,5cyclictest913-21if_wlan023:50:261
28607993511,5cyclictest23-21ksoftirqd/100:20:061
2860799347,7cyclictest7750irq/56-dwc_otg21:35:231
2860799347,12cyclictest7850irq/56-dwc_otg_20:20:071
2860799346,19cyclictest24916-1kworker/1:2H22:35:181
2861599335,5cyclictest12976-21cron00:25:062
2861599334,4cyclictest7950irq/56-dwc_otg_23:50:072
2861599332,5cyclictest7850irq/56-dwc_otg_00:30:352
2861599332,3cyclictest0-21swapper/222:25:272
2861599331,20cyclictest20609-21ls20:15:322
2860799331,9cyclictest14120-21idleruntime-cro20:00:071
2860799331,7cyclictest1265-21unixbench_multi20:50:361
2860799331,18cyclictest21962-21ls23:20:071
2861599322,18cyclictest7374-21/usr/sbin/munin21:10:212
28615993213,5cyclictest15995-21unixbench_singl23:00:352
2860799322,5cyclictest13328-21latency21:25:271
2860799321,21cyclictest221ktimersoftd/119:30:071
28607993210,4cyclictest0-21swapper/123:25:171
2861599317,15cyclictest0-21swapper/222:40:202
2861599316,11cyclictest7850irq/56-dwc_otg_20:10:272
2861599313,5cyclictest7750irq/56-dwc_otg00:00:372
2861599313,15cyclictest7950irq/56-dwc_otg_19:28:552
28615993112,7cyclictest4934-21unixbench_singl21:00:352
2861599311,22cyclictest645-21latency20:50:282
2861599311,21cyclictest20364-21idleruntime21:45:252
2861599311,19cyclictest31677-21switchtime20:45:322
2860799315,4cyclictest7850irq/56-dwc_otg_20:00:321
2860799315,4cyclictest7850irq/56-dwc_otg_20:00:321
2860799313,13cyclictest7950irq/56-dwc_otg_19:50:301
2860799312,19cyclictest0-21swapper/119:15:151
2861599303,15cyclictest7950irq/56-dwc_otg_20:35:232
2861599303,15cyclictest7950irq/56-dwc_otg_20:35:232
2861599302,4cyclictest11020-21/usr/sbin/munin21:20:192
2861599301,6cyclictest28851-21latency_hist22:10:072
2861599301,6cyclictest28851-21latency_hist22:10:072
2861599301,23cyclictest21987-21ls21:50:232
2860799306,14cyclictest0-21swapper/121:40:181
2861599296,4cyclictest7750irq/56-dwc_otg20:20:172
2861599295,11cyclictest7750irq/56-dwc_otg20:05:322
2861599294,4cyclictest7950irq/56-dwc_otg_21:55:342
2861599294,13cyclictest7950irq/56-dwc_otg_20:40:342
2861599294,13cyclictest0-21swapper/219:30:302
2861599292,3cyclictest7750irq/56-dwc_otg21:40:062
28615992921,4cyclictest11212-21latency_hist00:20:062
2861599291,22cyclictest0-21swapper/200:10:332
2861599291,13cyclictest0-21swapper/221:00:072
2861599283,3cyclictest7750irq/56-dwc_otg00:30:072
2861599282,18cyclictest0-21swapper/221:31:532
2861599281,9cyclictest25798-21aten_rbpower_po23:30:222
272912286,10sleep20-21swapper/219:05:302
2861599274,11cyclictest7750irq/56-dwc_otg19:55:372
2861599272,5cyclictest11996-21proc_pri22:50:312
266722273,17sleep10-21swapper/119:05:091
2861599262,8cyclictest21919-21latency_hist23:20:072
28615992620,4cyclictest31132-21ntp_offset19:15:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional