You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 12:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Jun 29, 2026 00:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1922822400,2sleep119154-21ls19:10:101
2305221260,2sleep00-21swapper/021:39:190
2305221260,2sleep00-21swapper/021:39:190
19389991133,80cyclictest25620-21ssh23:00:293
19388991126,83cyclictest25620-21ssh23:00:282
2604421080,2sleep20-21swapper/200:19:002
19386991074,7cyclictest1188-21ssh23:10:210
1507321020,2sleep00-21swapper/023:25:400
1938799993,12cyclictest0-21swapper/123:00:281
1938699977,61cyclictest0-21swapper/021:30:240
19386999024,42cyclictest727-21runrttasks00:25:230
19386998911,32cyclictest7850irq/56-dwc_otg23:00:280
1938799874,8cyclictest0-21swapper/121:45:401
1938999862,78cyclictest29473-21ssh21:45:423
116232820,1sleep30-21swapper/322:43:243
68752810,1sleep10-21swapper/122:37:181
68752810,1sleep10-21swapper/122:37:181
19386998135,14cyclictest7950irq/56-dwc_otg_21:45:390
1938999803,70cyclictest13203-21ssh22:45:283
140192800,5sleep21938899cyclictest21:28:262
1938799793,12cyclictest0-21swapper/122:45:251
19386997961,10cyclictest0-21swapper/023:51:450
1938899773,50cyclictest29469-21users21:45:412
1938899763,66cyclictest13166-21ssh22:45:252
1938699763,38cyclictest13194-21diskstats22:45:270
19386997548,6cyclictest7850irq/56-dwc_otg21:50:210
239862740,5sleep21938899cyclictest22:58:162
319772720,1sleep00-21swapper/019:45:110
319772720,1sleep00-21swapper/019:45:110
1938899711,55cyclictest10219-21latency_hist20:15:102
282852680,2sleep10-21swapper/122:24:501
1938699688,45cyclictest30843-21aten_rbpower_vo21:10:240
1938699688,45cyclictest30843-21aten_rbpower_vo21:10:240
1938699684,9cyclictest727-21runrttasks22:35:230
1938699684,9cyclictest727-21runrttasks22:35:230
1938899671,53cyclictest917-21taskset19:47:062
1938899671,47cyclictest13391-21awk20:20:382
1938899671,47cyclictest13391-21awk20:20:382
1938699676,29cyclictest0-21swapper/022:20:230
1938999664,47cyclictest8050irq/56-dwc_otg_23:40:293
1938899661,43cyclictest8783-21/usr/sbin/munin22:40:322
1938699666,50cyclictest2980-21cron19:55:090
19386996613,49cyclictest101ktimersoftd/022:40:360
1938899652,44cyclictest0-21swapper/223:40:212
1938899651,51cyclictest30590-21ssh22:26:272
1938899651,49cyclictest4292-21grep19:55:372
1938899651,49cyclictest0-21swapper/223:56:542
1938899651,43cyclictest30689-21/usr/sbin/munin21:10:352
1938899651,43cyclictest30689-21/usr/sbin/munin21:10:352
1938899651,24cyclictest7850irq/56-dwc_otg22:20:232
1938699653,42cyclictest0-21swapper/020:06:220
19386996511,36cyclictest20237-21latency_hist22:15:100
1938899644,39cyclictest8050irq/56-dwc_otg_21:30:332
1938899641,51cyclictest21998-21sh00:14:132
1938999631,6cyclictest23853-21chrt23:36:223
1938999631,6cyclictest23853-21chrt23:36:223
1938999631,6cyclictest0-21swapper/319:20:213
1938899635,39cyclictest8050irq/56-dwc_otg_22:05:232
1938899632,46cyclictest8050irq/56-dwc_otg_19:25:262
1938899631,52cyclictest4680-21ssh22:35:222
1938899631,52cyclictest4680-21ssh22:35:222
1938899631,47cyclictest4775-21ssh21:55:332
1938699635,39cyclictest21584-21basename20:45:230
1938699631,29cyclictest0-21swapper/021:55:230
1938899622,42cyclictest8050irq/56-dwc_otg_21:19:212
1938899621,47cyclictest2893-21ssh22:32:332
1938899621,47cyclictest19172-21ssh22:13:182
1938699624,49cyclictest15841-21sh00:05:380
1938699624,49cyclictest15841-21sh00:05:380
1938699624,30cyclictest61-21kswapd000:35:230
1938699624,30cyclictest61-21kswapd000:35:230
1938999615,46cyclictest32645-21ssh00:25:583
1938999614,4cyclictest7950irq/56-dwc_otg_20:20:243
1938999614,4cyclictest7950irq/56-dwc_otg_20:20:243
1938999614,43cyclictest0-21swapper/323:32:253
1938999611,5cyclictest15006-21cron21:30:093
1938899613,47cyclictest18252-21fschecks_count23:30:272
1938899611,48cyclictest12928-21sh00:02:522
1938699615,43cyclictest0-21swapper/020:31:270
1938699615,43cyclictest0-21swapper/020:31:270
1938999607,41cyclictest8050irq/56-dwc_otg_22:15:393
1938999603,38cyclictest8050irq/56-dwc_otg_22:57:533
1938999601,48cyclictest0-21swapper/322:07:103
1938999601,44cyclictest8050irq/56-dwc_otg_23:20:323
1938999601,44cyclictest8050irq/56-dwc_otg_23:20:323
1938899607,31cyclictest0-21swapper/223:50:362
1938899602,46cyclictest0-21swapper/220:25:562
1938899601,4cyclictest8050irq/56-dwc_otg_21:50:312
1938899601,49cyclictest10072-21if_eth023:20:282
1938899601,49cyclictest10072-21if_eth023:20:282
1938899601,48cyclictest32002-21ssh23:46:082
1938899601,48cyclictest31027-21ssh23:06:022
1938899601,48cyclictest31027-21ssh23:06:022
1938899601,46cyclictest25881-21ssh21:41:482
1938899601,44cyclictest21732-21fschecks_count20:45:262
19387996018,5cyclictest3054250wget23:45:241
1938699602,38cyclictest0-21swapper/022:15:320
1938899595,39cyclictest8050irq/56-dwc_otg_21:21:482
1938899592,46cyclictest9216-21rm00:37:172
1938899592,46cyclictest9216-21rm00:37:172
1938899592,41cyclictest8050irq/56-dwc_otg_23:15:122
1938899591,48cyclictest29477-21ssh00:22:442
1938899591,46cyclictest31793-21memory00:25:312
1938899591,36cyclictest28267-21sshd21:02:582
1938699595,40cyclictest24498-21seq20:50:560
1938699592,37cyclictest0-21swapper/023:05:200
1938699592,37cyclictest0-21swapper/023:05:200
19386995911,24cyclictest7245-21idleruntime21:20:290
253162580,2sleep10-21swapper/121:40:491
1938999588,30cyclictest7850irq/56-dwc_otg21:35:303
1938999588,30cyclictest7850irq/56-dwc_otg21:35:303
1938999582,44cyclictest8050irq/56-dwc_otg_00:35:443
1938999582,44cyclictest8050irq/56-dwc_otg_00:35:443
1938999581,44cyclictest7950irq/56-dwc_otg_23:20:103
1938999581,43cyclictest8050irq/56-dwc_otg_23:12:043
1938899582,43cyclictest9284-21timerwakeupswit22:00:382
1938899581,47cyclictest22296-21runrttasks22:16:272
1938899581,43cyclictest4676-21ssh00:31:322
1938899581,42cyclictest16084-21sh00:06:042
1938899581,42cyclictest16084-21sh00:06:042
19386995831,8cyclictest0-21swapper/000:19:370
19386995819,25cyclictest5426-21/usr/sbin/munin23:15:290
19386995811,36cyclictest8190-21aten_rbpower_en22:00:250
19386995810,5cyclictest22544-21latency_hist00:15:090
1938999575,40cyclictest8050irq/56-dwc_otg_21:42:543
1938999574,41cyclictest8050irq/56-dwc_otg_22:55:103
1938999572,45cyclictest0-21swapper/300:09:313
1938999572,45cyclictest0-21swapper/300:09:313
1938999572,44cyclictest253862sleep300:17:493
1938999571,47cyclictest0-21swapper/322:02:443
1938899572,41cyclictest8050irq/56-dwc_otg_22:51:572
1938899571,43cyclictest0-21swapper/219:55:102
1938899571,41cyclictest22626-21ssh21:38:352
1938899571,41cyclictest22626-21ssh21:38:352
1938699576,31cyclictest24077-21ssh23:36:470
1938699576,31cyclictest24077-21ssh23:36:470
1938699574,44cyclictest31771-21rm23:45:430
19386995730,13cyclictest0-21swapper/021:44:130
1938699572,28cyclictest0-21swapper/022:55:090
19386995719,6cyclictest0-21swapper/020:35:200
19386995716,21cyclictest4329-21tail19:55:370
1938999566,33cyclictest0-21swapper/321:22:013
1938999561,45cyclictest0-21swapper/323:06:253
1938999561,45cyclictest0-21swapper/323:06:253
1938999561,44cyclictest17023-21ssh22:10:303
1938899562,42cyclictest16709-21chrt23:28:402
1938899561,44cyclictest3453-21ssh23:12:212
1938899561,44cyclictest26348-21seq20:56:322
1938899561,42cyclictest20722-21taskset19:12:032
1938899561,42cyclictest0-21swapper/221:05:182
1938699569,5cyclictest101ktimersoftd/020:00:380
1938699564,37cyclictest9903-21ssh23:59:540
19386995625,6cyclictest12901-21ssh22:05:320
1938699562,37cyclictest9779-21taskset23:20:240
1938699562,37cyclictest9779-21taskset23:20:240
1938699562,30cyclictest22312-21grep19:15:370
189282563,13sleep00-21swapper/019:07:430
1938999558,44cyclictest6629-21latency_hist21:20:103
1938999555,38cyclictest8050irq/56-dwc_otg_21:30:353
1938999552,42cyclictest8050irq/56-dwc_otg_22:35:243
1938999552,42cyclictest8050irq/56-dwc_otg_22:35:243
1938999552,36cyclictest7850irq/56-dwc_otg20:45:293
1938999551,5cyclictest1554-21ssh22:30:383
1938899557,26cyclictest7950irq/56-dwc_otg_19:21:422
1938899551,43cyclictest24312-21ssh23:37:132
1938899551,43cyclictest24312-21ssh23:37:132
1938899551,42cyclictest8050irq/56-dwc_otg_20:03:372
1938699552,33cyclictest521-21munin-node19:25:200
1938699551,45cyclictest0-21swapper/021:05:150
1938699551,40cyclictest0-21swapper/021:15:280
1938699551,33cyclictest0-21swapper/000:00:230
1938999546,36cyclictest8050irq/56-dwc_otg_19:25:273
1938999546,29cyclictest7950irq/56-dwc_otg_20:50:333
1938999542,4cyclictest0-21swapper/300:02:253
1938999542,31cyclictest0-21swapper/321:55:223
1938999541,6cyclictest31957-21tail21:10:393
1938999541,6cyclictest31957-21tail21:10:393
1938999541,4cyclictest16888-21rm23:29:003
1938999541,3cyclictest0-21swapper/323:46:083
1938999541,39cyclictest0-21swapper/323:56:203
1938899545,46cyclictest23658-21/usr/sbin/munin20:50:282
1938899543,37cyclictest8050irq/56-dwc_otg_20:31:322
1938899543,37cyclictest8050irq/56-dwc_otg_20:31:322
1938899541,42cyclictest22583-21taskset19:17:382
1938899541,41cyclictest0-21swapper/219:39:182
1938899541,32cyclictest7950irq/56-dwc_otg_19:31:122
1938699545,34cyclictest23538-21sh22:57:270
19386995430,11cyclictest0-21swapper/022:26:310
1938699542,32cyclictest26741-21ssh23:40:290
25912530,3sleep123-21ksoftirqd/123:50:291
1938999532,38cyclictest0-21swapper/319:15:233
19389995312,38cyclictest32690-21/usr/sbin/munin19:45:313
19386995315,27cyclictest101ktimersoftd/021:05:100
19386995312,5cyclictest13956-21cat20:25:100
19386995312,5cyclictest13956-21cat20:25:100
1938999525,38cyclictest8050irq/56-dwc_otg_20:17:383
1938999523,35cyclictest8050irq/56-dwc_otg_23:50:513
1938999521,4cyclictest0-21swapper/321:05:293
1938999521,2cyclictest0-21swapper/322:27:393
1938899525,36cyclictest117922chrt20:17:372
1938899522,35cyclictest8050irq/56-dwc_otg_20:06:262
1938899521,46cyclictest0-21swapper/219:40:202
1938899521,46cyclictest0-21swapper/219:40:202
1938699522,28cyclictest11839-21ssh21:25:330
19386995221,5cyclictest0-21swapper/022:30:240
1938999516,3cyclictest7950irq/56-dwc_otg_20:14:493
1938999516,37cyclictest0-21swapper/300:25:103
1938999513,44cyclictest0-21swapper/322:20:273
1938999512,5cyclictest0-21swapper/319:55:383
1938999511,5cyclictest20441-21ssh00:11:223
1938999511,4cyclictest8050irq/56-dwc_otg_00:30:343
1938999511,4cyclictest0-21swapper/321:50:133
1938999511,4cyclictest0-21swapper/320:05:243
19386995125,15cyclictest0-21swapper/019:49:480
1938699512,36cyclictest727-21runrttasks00:33:550
1938999509,37cyclictest28044-21unixbench_singl21:00:383
1938999501,41cyclictest0-21swapper/320:37:283
19386995029,11cyclictest0-21swapper/000:22:590
19386995023,6cyclictest7850irq/56-dwc_otg23:30:160
19386995021,15cyclictest23276-21sort19:20:270
1938999491,7cyclictest21341-21latency_hist20:45:103
1938999491,6cyclictest0-21swapper/319:30:213
1938899491,45cyclictest18027-21grep20:35:242
19386994922,5cyclictest0-21swapper/019:10:210
19386994921,6cyclictest29197-21proc_pri19:35:350
1938999489,36cyclictest31755-21run-parts19:44:353
1938999489,36cyclictest31755-21run-parts19:44:353
1938999482,5cyclictest17147-21grep20:30:373
1938999482,5cyclictest17147-21grep20:30:373
1938999481,5cyclictest15273-21grep20:25:383
1938899481,43cyclictest19677-21/usr/sbin/munin20:40:192
1938699482,31cyclictest0-21swapper/020:28:400
1938999472,6cyclictest19402-21/usr/sbin/munin19:10:353
1938999472,41cyclictest0-21swapper/320:00:233
1938999472,3cyclictest8050irq/56-dwc_otg_19:36:173
1938999471,37cyclictest0-21swapper/319:50:123
19386994727,10cyclictest9931-21chrt20:11:560
1938999461,5cyclictest26182-21grep20:55:383
19386994630,8cyclictest0-21swapper/019:30:470
19386994517,18cyclictest12048-21ls20:20:100
19386994517,14cyclictest727-21runrttasks20:40:190
19386994417,19cyclictest0-21swapper/020:57:310
186502420,9sleep3381ktimersoftd/319:05:383
1938799412,31cyclictest0-21swapper/119:45:211
58642400,1sleep10-21swapper/121:19:071
1938799406,19cyclictest8050irq/56-dwc_otg_23:15:101
1938799401,12cyclictest27013-21sh00:20:201
1938799394,12cyclictest9244-21kworker/1:223:25:201
1938799392,9cyclictest0-21swapper/120:00:231
209792380,6sleep11938799cyclictest23:33:421
19387993818,5cyclictest887250wget22:40:231
19387993727,6cyclictest3983-21ssh21:55:211
19387993710,2cyclictest0-21swapper/120:50:231
1938799366,4cyclictest7850irq/56-dwc_otg00:30:351
1938799365,12cyclictest27739-21unixbench_multi19:30:381
19387993611,14cyclictest0-21swapper/121:52:321
19387993610,16cyclictest0-21swapper/122:10:231
1938799358,17cyclictest0-21swapper/122:26:331
1938799356,13cyclictest8050irq/56-dwc_otg_19:40:101
1938799354,4cyclictest8050irq/56-dwc_otg_00:35:231
1938799354,4cyclictest8050irq/56-dwc_otg_00:35:231
19387993517,5cyclictest15819-21latency_hist20:30:101
1938799351,2cyclictest0-21swapper/120:45:211
19387993512,6cyclictest0-21swapper/123:20:221
19387993512,6cyclictest0-21swapper/123:20:221
1938799348,17cyclictest0-21swapper/100:00:281
1938799345,8cyclictest7850irq/56-dwc_otg21:15:101
1938799345,8cyclictest7850irq/56-dwc_otg21:15:101
1938799345,15cyclictest8050irq/56-dwc_otg_21:05:381
1938799344,4cyclictest0-21swapper/122:50:221
1938799342,4cyclictest8050irq/56-dwc_otg_00:10:231
19387993414,13cyclictest0-21swapper/119:40:271
19387993414,13cyclictest0-21swapper/119:40:271
19387993410,11cyclictest0-21swapper/121:35:301
19387993410,11cyclictest0-21swapper/121:35:301
1938799339,11cyclictest0-21swapper/122:55:211
1938799335,4cyclictest0-21swapper/119:10:201
1938799333,8cyclictest20539-21ssh22:15:211
1938799333,14cyclictest8050irq/56-dwc_otg_19:20:101
19387993314,12cyclictest529-21ssh22:30:231
1938799331,23cyclictest30625-21timerwakeupswit23:05:381
1938799331,23cyclictest30625-21timerwakeupswit23:05:381
1938799331,18cyclictest7850irq/56-dwc_otg23:57:501
1938799327,14cyclictest8050irq/56-dwc_otg_23:15:371
1938799324,16cyclictest21-21rcuc/122:00:391
1938799322,6cyclictest7950irq/56-dwc_otg_23:35:401
1938799322,6cyclictest7950irq/56-dwc_otg_23:35:401
1938799322,23cyclictest0-21swapper/121:20:311
19387993214,5cyclictest8397-21latency_hist20:10:101
19387993211,5cyclictest2694850wget21:00:241
19387993210,11cyclictest8050irq/56-dwc_otg_23:40:111
188982323,23sleep20-21swapper/219:07:202
1938799318,8cyclictest11450-21grep21:25:281
1938799318,3cyclictest7950irq/56-dwc_otg_21:30:231
1938799316,5cyclictest7950irq/56-dwc_otg_20:10:281
1938799316,13cyclictest8050irq/56-dwc_otg_20:15:291
1938799312,9cyclictest7950irq/56-dwc_otg_00:15:281
1938799311,5cyclictest8050irq/56-dwc_otg_20:20:311
1938799311,5cyclictest8050irq/56-dwc_otg_20:20:311
19387993110,10cyclictest0-21swapper/100:05:211
19387993110,10cyclictest0-21swapper/100:05:211
19387993023,5cyclictest26649-21idleruntime-cro21:00:091
1938799302,15cyclictest7850irq/56-dwc_otg22:05:291
1938799301,4cyclictest7950irq/56-dwc_otg_00:28:341
1938799298,4cyclictest0-21swapper/119:26:431
19387992912,6cyclictest9550irq/48-DMA15:26:471
1938799291,24cyclictest23235-21ls19:20:251
1938799284,19cyclictest19682-21runrttasks20:40:191
1938799283,4cyclictest0-21swapper/120:30:301
1938799283,4cyclictest0-21swapper/120:30:301
1938799282,3cyclictest0-21swapper/119:50:381
19387992812,4cyclictest17848-21taskset20:35:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional