You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 17:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Mar 23, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
787221050,2sleep20-21swapper/211:44:222
3113199933,68cyclictest27986-21ssh10:50:191
3113199751,44cyclictest27131-21cat10:10:091
3113099742,51cyclictest0-21swapper/011:20:200
167722740,2sleep30-21swapper/310:35:393
167722740,2sleep30-21swapper/310:35:393
31130997311,50cyclictest0-21swapper/008:20:210
3113199721,57cyclictest9647-21rm09:48:361
3113199721,49cyclictest2699-21ssh11:37:281
3113199713,39cyclictest0-21swapper/111:00:291
3113199683,49cyclictest7572-21basename09:05:081
31130996818,10cyclictest24016-21cpuspeed10:45:210
295512680,2sleep00-21swapper/012:10:250
21282680,2sleep23113299cyclictest11:36:252
263452670,1sleep20-21swapper/209:29:592
3113199664,48cyclictest7850irq/56-dwc_otg_08:15:361
3113199662,9cyclictest21785-21ntp_states12:00:321
3113099664,9cyclictest3159-21ssh10:20:210
3113099661,45cyclictest3502-21diskmemload11:00:080
3113199653,40cyclictest31715-21taskset10:15:251
3113199641,6cyclictest19669-21ssh12:37:161
3113199641,5cyclictest19688-21/usr/sbin/munin10:40:271
3113199641,5cyclictest11672-21ssh11:48:441
3113199641,52cyclictest9478-21diskmemload09:37:261
3113199632,51cyclictest18283-21ssh10:38:101
3113199632,51cyclictest18283-21ssh10:38:101
3113199631,7cyclictest20749-21sed12:00:081
3113199631,5cyclictest19293-21/usr/sbin/munin08:05:341
3113199631,5cyclictest13914-21rm09:53:491
3113199631,5cyclictest10056-21df_inode09:10:231
3113199631,5cyclictest10056-21df_inode09:10:231
3113199622,46cyclictest9478-21diskmemload11:33:131
3113199621,6cyclictest32104-21cat07:10:341
3113199621,6cyclictest20455-21aten_rbpower_po08:10:211
3113099622,26cyclictest0-21swapper/008:35:200
103912620,1sleep10-21swapper/107:40:331
3113199616,36cyclictest0-21swapper/110:14:401
3113199616,36cyclictest0-21swapper/110:14:401
3113199616,35cyclictest7750irq/56-dwc_otg11:20:231
3113199613,43cyclictest7950irq/56-dwc_otg_09:40:261
3113199611,5cyclictest5343-21grep12:20:241
3113199611,4cyclictest29224-21ssh12:10:201
31130996120,26cyclictest25149-21wget12:05:200
3113099611,50cyclictest21485-21unixbench_multi08:10:350
3113199605,39cyclictest21569-21ssh09:23:561
3113199603,3cyclictest0-21swapper/112:26:011
3113099604,38cyclictest0-21swapper/010:30:180
31130996014,25cyclictest908-21runrttasks10:25:280
31131995918,39cyclictest24946-21sh09:27:221
3113199591,6cyclictest20986-21run-parts10:01:441
31130995937,5cyclictest845-21fschecks_time07:15:260
31130995937,5cyclictest845-21fschecks_time07:15:260
31130995932,11cyclictest14481-21fschecks_time09:15:270
3113099591,41cyclictest0-21swapper/009:50:360
3113199581,6cyclictest8116-21df_inode09:05:251
3113199581,5cyclictest8738-21ssh11:05:331
31130995842,4cyclictest315782sleep009:35:290
31130995832,11cyclictest0-21swapper/008:50:250
3113099582,43cyclictest0-21swapper/009:00:210
3113099582,41cyclictest0-21swapper/011:20:090
3113099582,41cyclictest0-21swapper/011:20:090
3113099581,44cyclictest0-21swapper/011:41:240
3113199573,4cyclictest0-21swapper/111:29:011
3113199573,44cyclictest9478-21diskmemload10:27:181
3113199572,5cyclictest0-21swapper/110:46:271
3113199571,6cyclictest149-21systemd-journal11:54:521
3113199571,6cyclictest0-21swapper/108:46:441
3113199571,6cyclictest0-21swapper/108:46:441
31130995727,12cyclictest7850irq/56-dwc_otg_08:30:210
31130995718,6cyclictest0-21swapper/009:20:190
3113099571,48cyclictest0-21swapper/012:00:330
3113199561,48cyclictest0-21swapper/109:18:321
3113199561,40cyclictest7750irq/56-dwc_otg12:18:461
3113199561,40cyclictest7750irq/56-dwc_otg12:18:461
3113099563,22cyclictest0-21swapper/011:55:200
31130995631,11cyclictest7750irq/56-dwc_otg09:40:220
3113099561,38cyclictest0-21swapper/008:05:250
3113199553,4cyclictest7750irq/56-dwc_otg12:31:001
3113199551,6cyclictest15663-21ssh11:14:411
3113199551,45cyclictest5060-21ssh10:21:421
3113199551,3cyclictest0-21swapper/109:30:261
3113099555,24cyclictest0-21swapper/012:15:210
3113099555,24cyclictest0-21swapper/012:15:210
31130995537,4cyclictest0-21swapper/011:33:530
31130995531,13cyclictest11386-21ssh11:48:140
31130995528,16cyclictest0-21swapper/010:35:330
31130995528,16cyclictest0-21swapper/010:35:330
3113099552,40cyclictest0-21swapper/010:13:070
3113099552,40cyclictest0-21swapper/010:13:070
3113299546,29cyclictest0-21swapper/208:50:222
3113199546,34cyclictest7750irq/56-dwc_otg11:17:331
3113199546,34cyclictest7750irq/56-dwc_otg11:17:331
3113199542,44cyclictest18307-21ssh09:59:241
31130995425,15cyclictest10943-21sed09:10:350
31130995425,15cyclictest10943-21sed09:10:350
3113099542,41cyclictest0-21swapper/010:06:450
3113099542,38cyclictest0-21swapper/012:35:080
3113099541,43cyclictest908-21runrttasks09:34:220
3113099541,39cyclictest6319-21unixbench_singl12:20:350
3113099541,39cyclictest0-21swapper/011:35:230
307712543,45sleep00-21swapper/007:09:090
197662540,2sleep30-21swapper/309:20:463
3113199532,6cyclictest32478-21rm10:55:261
3113199532,40cyclictest0-21swapper/112:09:261
3113199531,40cyclictest7750irq/56-dwc_otg11:40:251
3113199531,3cyclictest7750irq/56-dwc_otg08:55:231
3113199531,39cyclictest0-21swapper/110:32:541
31130995339,3cyclictest0-21swapper/011:10:240
31130995335,4cyclictest0-21swapper/011:50:550
3113099531,40cyclictest0-21swapper/010:50:310
3113099531,39cyclictest28447-21latency_hist11:30:080
3113199524,45cyclictest4459-21fschecks_time07:25:261
3113199522,6cyclictest8019-21fschecks_count07:35:271
3113199521,7cyclictest23993-21apt-get08:20:211
3113099522,38cyclictest0-21swapper/008:30:090
3113099522,37cyclictest0-21swapper/007:50:360
3113099521,39cyclictest0-21swapper/009:25:260
3113099521,37cyclictest0-21swapper/012:25:260
3113099521,36cyclictest31597-21aten_rbpower_vo10:15:230
3113099521,35cyclictest3476-21unixbench_singl07:20:350
3113199514,4cyclictest7750irq/56-dwc_otg07:20:081
3113199514,4cyclictest7750irq/56-dwc_otg07:20:081
3113199512,6cyclictest0-21swapper/107:52:101
3113199512,3cyclictest7950irq/56-dwc_otg_08:52:191
3113199511,38cyclictest0-21swapper/108:00:241
3113199511,38cyclictest0-21swapper/108:00:241
31130995128,12cyclictest14950-21/usr/sbin/munin07:55:220
31130995127,12cyclictest0-21swapper/010:00:080
3113099511,40cyclictest9305-21idleruntime-cro07:40:080
3113199506,4cyclictest27872-21fschecks_count08:30:251
3113199502,39cyclictest7950irq/56-dwc_otg_08:28:371
31130995035,3cyclictest0-21swapper/012:36:440
31130995029,7cyclictest0-21swapper/008:16:380
31130995028,9cyclictest0-21swapper/008:40:330
3113099502,33cyclictest0-21swapper/010:01:550
31130995017,14cyclictest5044-21switchtime08:55:350
3113099501,36cyclictest0-21swapper/009:05:230
31132994916,6cyclictest19565-21unixbench_multi09:20:362
3113199493,38cyclictest7750irq/56-dwc_otg07:46:561
3113199491,7cyclictest29479-21aten_rbpower_po08:35:231
3113199491,5cyclictest14964-21apt-get07:55:191
3113199491,3cyclictest0-21swapper/108:40:231
31130994930,5cyclictest7850irq/56-dwc_otg_07:35:080
31130994928,11cyclictest0-21swapper/007:43:170
3113099491,37cyclictest0-21swapper/011:05:260
3113199481,6cyclictest5980-21grep07:30:221
31130994831,11cyclictest0-21swapper/011:00:270
31130994827,10cyclictest11463-21aten_rbpower_vo07:45:230
31130994824,15cyclictest7615-21ssh09:45:330
3113099481,37cyclictest149-21systemd-journal07:12:490
228232480,2sleep00-21swapper/010:43:530
3113199474,8cyclictest0-21swapper/107:20:351
52812460,2sleep20-21swapper/211:00:542
31130994625,11cyclictest0-21swapper/007:25:350
3113099462,30cyclictest0-21swapper/008:00:230
3113099462,30cyclictest0-21swapper/008:00:230
3113099452,30cyclictest0-21swapper/008:45:230
3113099452,30cyclictest0-21swapper/008:45:230
77192430,1sleep30-21swapper/309:05:083
31133994316,5cyclictest6509-21sh10:24:203
31132994314,6cyclictest9439-21sh10:27:072
3113399424,24cyclictest0-21swapper/308:55:223
31132994217,12cyclictest20503-21sh10:00:512
149512420,2sleep30-21swapper/311:13:233
35192410,4sleep33113399cyclictest12:17:573
35192410,4sleep33113399cyclictest12:17:573
3113299411,8cyclictest0-21swapper/208:25:192
3113399406,19cyclictest0-21swapper/308:20:213
31133994018,5cyclictest12954-21sh09:13:473
31133994018,5cyclictest12954-21sh09:13:473
3113299406,6cyclictest7950irq/56-dwc_otg_10:20:292
31132994018,15cyclictest17644-21sh11:55:302
3113399392,22cyclictest7950irq/56-dwc_otg_08:35:373
31133993917,14cyclictest16549-21sh11:55:073
3113299390,24cyclictest7950irq/56-dwc_otg_11:33:392
307302390,11sleep3381ktimersoftd/310:53:143
3113399382,4cyclictest7950irq/56-dwc_otg_08:00:213
3113399382,4cyclictest7950irq/56-dwc_otg_08:00:213
3113399381,5cyclictest21914-21ssh10:03:263
138592380,2sleep20-21swapper/212:30:282
3113399377,12cyclictest0-21swapper/309:36:143
3113399373,21cyclictest0-21swapper/308:15:213
31133993720,10cyclictest14351-21sshd07:50:403
3113399371,22cyclictest1067-21latency08:45:283
3113399371,22cyclictest1067-21latency08:45:283
301242377,17sleep230188-21switchtime07:05:352
3113399363,4cyclictest7750irq/56-dwc_otg12:05:083
31133993616,13cyclictest24667-21ssh10:05:543
31133993611,8cyclictest0-21swapper/309:30:283
3113299368,19cyclictest11-21rcu_preempt08:20:212
3113299367,6cyclictest0-21swapper/211:27:182
3113299366,4cyclictest0-21swapper/210:30:252
3113299365,6cyclictest0-21swapper/212:15:272
3113299365,6cyclictest0-21swapper/212:15:272
3113299363,4cyclictest7850irq/56-dwc_otg_08:00:212
3113299363,4cyclictest7850irq/56-dwc_otg_08:00:212
3113299362,5cyclictest0-21swapper/208:40:332
31132993615,8cyclictest24928-21timerwakeupswit10:45:332
301422360,2sleep30-21swapper/311:30:543
3113399354,6cyclictest0-21swapper/308:25:193
3113399353,21cyclictest17468-21diskmemload12:35:233
3113399352,7cyclictest7950irq/56-dwc_otg_07:35:293
3113399351,17cyclictest20550-21sh10:40:323
3113299357,5cyclictest0-21swapper/211:51:232
3113299355,6cyclictest20061-21cron11:20:082
3113299355,6cyclictest20061-21cron11:20:082
3113299352,5cyclictest31644-21ssh09:35:302
3113299352,15cyclictest0-21swapper/211:48:162
31132993512,7cyclictest7331-1kworker/u9:009:10:232
31132993512,7cyclictest7331-1kworker/u9:009:10:232
31132993511,4cyclictest7850irq/56-dwc_otg_10:50:302
94752340,2sleep20-21swapper/212:25:242
3113399348,10cyclictest7850irq/56-dwc_otg_08:09:043
3113399347,15cyclictest0-21swapper/308:30:213
31133993418,7cyclictest11941-21sh12:28:153
31133993414,6cyclictest5212-21sh09:43:023
3113299349,4cyclictest7950irq/56-dwc_otg_12:05:572
3113299347,4cyclictest7950irq/56-dwc_otg_12:10:292
3113299346,3cyclictest0-21swapper/212:35:452
3113299344,18cyclictest7950irq/56-dwc_otg_11:20:262
3113299341,6cyclictest1637650ssh10:35:342
3113299341,6cyclictest1637650ssh10:35:342
31132993411,4cyclictest20733-21timerandwakeup10:40:342
285092341,26sleep20-21swapper/208:30:352
158182340,4sleep339-21ksoftirqd/309:16:203
3113399336,15cyclictest0-21swapper/308:50:223
31133993324,6cyclictest6695-21ssh09:45:193
31133993321,5cyclictest8728-21ssh12:24:563
31133993317,8cyclictest1133-21sh10:17:053
31133993313,7cyclictest7850irq/56-dwc_otg_11:20:593
31133993310,3cyclictest0-21swapper/311:36:283
3113299336,4cyclictest7850irq/56-dwc_otg_09:55:232
3113299336,10cyclictest7950irq/56-dwc_otg_08:50:092
3113299336,10cyclictest7950irq/56-dwc_otg_08:50:092
3113299335,4cyclictest7850irq/56-dwc_otg_08:10:182
3113299335,14cyclictest7950irq/56-dwc_otg_07:25:082
3113299334,4cyclictest7850irq/56-dwc_otg_10:55:372
3113299332,5cyclictest0-21swapper/210:06:382
31132993319,5cyclictest14534-21sh09:54:582
31132993318,4cyclictest2787-21sh09:40:232
31132993314,8cyclictest7950irq/56-dwc_otg_11:09:442
31132993312,4cyclictest5431-21fw_packets12:20:242
3113299331,17cyclictest13393-21cut07:50:252
3113399328,5cyclictest29706-1kworker/u9:010:55:483
3113399328,11cyclictest24125-21fschecks_count10:45:233
3113399326,16cyclictest0-21swapper/308:10:203
3113399326,15cyclictest0-21swapper/310:30:203
31133993221,4cyclictest11096-21sh11:47:423
31133993217,4cyclictest28616-21sh12:09:393
31133993214,7cyclictest9478-21diskmemload10:25:503
31133993210,4cyclictest7750irq/56-dwc_otg11:40:283
31133993210,4cyclictest0-21swapper/311:00:213
3113299326,5cyclictest7850irq/56-dwc_otg_09:00:372
3113299325,4cyclictest0-21swapper/209:45:312
3113299323,5cyclictest12913-21cron07:50:082
3113299321,5cyclictest14619-21ssh09:15:292
31132993212,6cyclictest7850irq/56-dwc_otg_07:45:082
3113299321,20cyclictest27907-21ls10:10:282
3113299321,20cyclictest27907-21ls10:10:282
3113399319,3cyclictest0-21swapper/311:26:253
3113399314,4cyclictest7750irq/56-dwc_otg07:45:243
3113399311,22cyclictest7950irq/56-dwc_otg_11:15:363
3113399311,22cyclictest7950irq/56-dwc_otg_11:15:363
31133993110,4cyclictest0-21swapper/309:29:093
3113399310,13cyclictest17521-21ssh09:57:573
3113299312,15cyclictest7850irq/56-dwc_otg_07:15:302
3113299312,15cyclictest7850irq/56-dwc_otg_07:15:302
3113299312,11cyclictest7950irq/56-dwc_otg_10:15:282
3113299311,4cyclictest7750irq/56-dwc_otg09:05:212
31132993112,7cyclictest0-21swapper/209:31:412
3113399308,9cyclictest0-21swapper/311:06:263
3113399305,3cyclictest7950irq/56-dwc_otg_09:50:593
3113399303,4cyclictest7850irq/56-dwc_otg_10:11:533
3113399303,4cyclictest7850irq/56-dwc_otg_10:11:533
31133993016,5cyclictest1469750ssh12:30:433
3113299301,7cyclictest21505-21ssh12:00:282
3113299301,18cyclictest0-21swapper/211:15:082
3113399293,6cyclictest7950irq/56-dwc_otg_09:10:083
3113399291,6cyclictest0-21swapper/312:10:193
31133992912,5cyclictest6343-21if_err_wlan007:30:273
3113399291,23cyclictest0-21swapper/311:55:203
3113299297,14cyclictest0-21swapper/207:30:222
3113299296,14cyclictest0-21swapper/208:55:222
3113299293,5cyclictest16070-21grep07:55:362
31132992912,4cyclictest29395-21/usr/sbin/munin08:35:342
3113399285,5cyclictest7850irq/56-dwc_otg_07:20:073
3113399285,5cyclictest7850irq/56-dwc_otg_07:20:073
3113399284,5cyclictest7850irq/56-dwc_otg_07:55:363
31133992815,4cyclictest31207-21/usr/sbin/munin08:40:343
3113299284,10cyclictest7850irq/56-dwc_otg_07:10:252
31132992811,4cyclictest7850irq/56-dwc_otg_08:10:072
3113399273,7cyclictest11112-21idleruntime-cro07:45:083
31133992710,8cyclictest4151-21apt-get07:25:203
3113299273,5cyclictest0-21swapper/207:37:592
3113299271,4cyclictest7750irq/56-dwc_otg07:25:202
307422274,15sleep10-21swapper/107:08:461
31133992620,4cyclictest2419-21aten_rbpower_po07:20:213
3113299267,5cyclictest31-21ksoftirqd/208:15:352
306502262,15sleep30-21swapper/307:07:343
3113399251,18cyclictest0-21swapper/307:10:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional