You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 23:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Dec 19, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
325021070,3sleep30-21swapper/307:05:293
426199841,67cyclictest0-21swapper/307:55:173
121002760,5sleep3426199cyclictest10:30:193
4258996955,5cyclictest10792-21timerandwakeup08:55:320
425899677,56cyclictest0-21swapper/008:25:170
4258996646,9cyclictest101ktimersoftd/010:15:050
425899656,56cyclictest0-21swapper/007:25:180
4258996514,29cyclictest24742-21sh12:35:040
425899647,54cyclictest0-21swapper/007:45:170
425899645,43cyclictest21112-21fschecks_count09:25:210
425899644,46cyclictest29912-21modprobe11:20:040
4258996410,11cyclictest0-21swapper/010:50:160
22152640,1sleep30-21swapper/311:30:323
425899633,42cyclictest14108-21aten_rbpower_en12:05:190
425899632,49cyclictest0-21swapper/011:03:220
425899616,39cyclictest0-21swapper/010:55:180
4258996133,18cyclictest30713-21missed_timers11:20:280
425899605,51cyclictest321-21tail09:55:310
425899605,33cyclictest0-21swapper/008:15:250
4258996033,7cyclictest6028-21latency_hist07:15:050
29592603,12sleep20-21swapper/207:05:252
425899591,39cyclictest0-21swapper/011:35:050
425999573,5cyclictest0-21swapper/108:30:151
4258995719,4cyclictest6631-21latency_hist11:45:050
425899564,30cyclictest0-21swapper/009:40:340
4258995633,13cyclictest101ktimersoftd/008:45:050
425899563,28cyclictest0-21swapper/008:45:160
31882560,2sleep20-21swapper/210:05:222
425999554,10cyclictest7750irq/56-dwc_otg11:35:161
4258995528,9cyclictest29620-21modprobe09:50:040
425899551,31cyclictest935-21aten_rbpower_vo08:30:180
4258995427,14cyclictest0-21swapper/009:36:100
4258995427,14cyclictest0-21swapper/009:36:100
425899542,30cyclictest0-21swapper/009:10:040
4258995422,17cyclictest15847-21/usr/sbin/munin12:10:150
425999538,18cyclictest7850irq/56-dwc_otg_07:15:171
425999538,18cyclictest7850irq/56-dwc_otg_07:15:171
425899536,25cyclictest0-21swapper/010:40:170
425999523,8cyclictest7750irq/56-dwc_otg09:10:181
425999522,10cyclictest0-21swapper/111:40:181
4258995233,12cyclictest0-21swapper/010:06:350
4258995230,13cyclictest101ktimersoftd/008:10:310
425899521,40cyclictest7028-21missed_timers10:15:250
4259995117,17cyclictest0-21swapper/107:10:171
425899517,24cyclictest0-21swapper/007:30:230
425899514,37cyclictest13814-21wget10:35:170
4258995131,10cyclictest23767-21chrt08:02:200
4258995126,11cyclictest0-21swapper/008:05:180
4258995037,7cyclictest101ktimersoftd/009:03:570
4258995037,7cyclictest101ktimersoftd/009:03:570
4258995033,4cyclictest30424-21ls09:50:230
425899503,19cyclictest0-21swapper/011:25:180
425899501,6cyclictest9460-21apt-get07:25:040
425999497,23cyclictest0-21swapper/108:25:171
425899498,23cyclictest0-21swapper/007:55:150
4258994937,4cyclictest1232-21ls10:00:180
4258994931,9cyclictest12011-21latency_hist12:00:050
4258994929,8cyclictest0-21swapper/008:39:250
4258994928,9cyclictest101ktimersoftd/011:09:100
4258994928,9cyclictest101ktimersoftd/011:09:100
4258994928,8cyclictest0-21swapper/012:19:060
4258994926,4cyclictest0-21swapper/009:15:310
426199487,26cyclictest0-21swapper/307:40:193
425999487,27cyclictest0-21swapper/109:00:181
425999487,27cyclictest0-21swapper/109:00:181
425999487,26cyclictest0-21swapper/112:15:171
425999487,25cyclictest0-21swapper/108:45:171
425999484,5cyclictest0-21swapper/112:30:161
425999483,11cyclictest7850irq/56-dwc_otg_11:05:171
425999483,11cyclictest7850irq/56-dwc_otg_11:05:171
425899486,36cyclictest0-21swapper/010:30:180
4258994834,4cyclictest101ktimersoftd/007:41:210
4258994830,3cyclictest0-21swapper/007:15:400
4258994830,3cyclictest0-21swapper/007:15:400
425899481,17cyclictest0-21swapper/012:20:320
37962483,12sleep00-21swapper/007:07:420
426099477,25cyclictest0-21swapper/207:20:162
4259994722,9cyclictest7950irq/56-dwc_otg_09:25:241
425899473,25cyclictest8044-21taskset08:50:100
4258994730,11cyclictest0-21swapper/010:26:230
4258994730,11cyclictest0-21swapper/010:26:230
4258994725,3cyclictest0-21swapper/011:50:050
4258994721,14cyclictest25019-21wget12:35:180
425899471,4cyclictest3692-21ls11:35:250
425899471,31cyclictest0-21swapper/009:20:170
426199467,25cyclictest0-21swapper/307:30:173
425999467,24cyclictest0-21swapper/110:15:181
425999467,24cyclictest0-21swapper/107:25:101
425999467,23cyclictest0-21swapper/110:55:111
425899466,25cyclictest0-21swapper/010:20:110
425899466,25cyclictest0-21swapper/010:20:110
425899464,21cyclictest0-21swapper/012:25:310
425899463,21cyclictest0-21swapper/009:30:280
4258994616,15cyclictest19925-21grep07:50:320
425899461,14cyclictest0-21swapper/010:45:310
426199457,25cyclictest0-21swapper/310:20:183
426199457,25cyclictest0-21swapper/310:20:183
426199457,24cyclictest0-21swapper/307:20:163
426099453,10cyclictest7750irq/56-dwc_otg11:20:172
425999457,25cyclictest0-21swapper/109:35:181
425999457,25cyclictest0-21swapper/109:35:181
425999457,23cyclictest0-21swapper/110:30:181
425999457,23cyclictest0-21swapper/109:15:171
425899454,19cyclictest0-21swapper/011:10:340
4258994515,17cyclictest0-21swapper/011:51:200
426199446,23cyclictest0-21swapper/312:10:183
425999447,25cyclictest0-21swapper/111:15:171
425999447,23cyclictest0-21swapper/110:40:181
425999447,23cyclictest0-21swapper/108:20:191
425999446,23cyclictest0-21swapper/110:50:181
4259994422,10cyclictest7950irq/56-dwc_otg_09:20:211
4258994425,10cyclictest101ktimersoftd/008:21:060
425899442,25cyclictest0-21swapper/007:35:260
4260994317,6cyclictest1125-21runrttasks10:10:152
425999437,24cyclictest0-21swapper/112:20:101
4259994323,14cyclictest7950irq/56-dwc_otg_08:07:391
4261994221,6cyclictest1319-21fschecks_time10:00:203
426099428,25cyclictest0-21swapper/210:00:162
426099421,14cyclictest7750irq/56-dwc_otg11:25:242
425999426,24cyclictest0-21swapper/111:50:181
425899427,25cyclictest0-21swapper/012:00:180
425899421,23cyclictest0-21swapper/009:12:480
425999417,24cyclictest0-21swapper/111:00:191
4259994119,7cyclictest1373-21fschecks_time10:00:211
4259994114,11cyclictest7950irq/56-dwc_otg_08:10:181
4259994113,12cyclictest6119-21latency_hist08:45:051
426199401,30cyclictest20560-21ls12:20:323
426099401,9cyclictest19227-21latency_hist12:20:052
425999409,11cyclictest30504-21idleruntime11:20:251
4259994019,10cyclictest7950irq/56-dwc_otg_07:31:061
426199392,18cyclictest7950irq/56-dwc_otg_10:10:053
4260993914,5cyclictest7850irq/56-dwc_otg_09:10:332
425999398,27cyclictest0-21swapper/112:35:181
4259993926,11cyclictest7910-21dpkg07:20:141
4259993924,12cyclictest21456-21aten_rbpower_po12:25:171
425999392,14cyclictest16923-21sed12:10:321
425999390,23cyclictest0-21swapper/112:05:181
425999390,19cyclictest10679-21/usr/sbin/munin08:55:311
425999390,12cyclictest8905-21swap08:50:311
4260993816,5cyclictest25811-21latency_hist08:10:052
426099381,21cyclictest7950irq/56-dwc_otg_11:50:052
425999382,6cyclictest7750irq/56-dwc_otg07:35:171
425999380,22cyclictest14065-21/usr/sbin/munin10:35:211
37832380,5sleep12630-21/usr/sbin/munin08:35:341
187212380,3sleep339-21ksoftirqd/312:15:323
426199371,21cyclictest22559-21ls09:30:053
425999370,27cyclictest128-21systemd-journal10:13:141
425999370,14cyclictest28742-21sed08:15:281
214682370,1sleep20-21swapper/209:25:262
426199363,23cyclictest10749-21fschecks_time11:55:233
426199361,29cyclictest4890-21cut10:10:213
426199361,22cyclictest6086-21ls08:45:053
426099366,5cyclictest7850irq/56-dwc_otg_07:20:052
426099366,5cyclictest7850irq/56-dwc_otg_07:20:052
4260993618,5cyclictest5044-21aten_rbpower_vo11:40:202
425999367,6cyclictest0-21swapper/109:30:161
4259993624,10cyclictest7950irq/56-dwc_otg_10:26:301
4259993624,10cyclictest7950irq/56-dwc_otg_10:26:301
425999360,14cyclictest12106-21cat12:00:051
425999360,12cyclictest15088-21cat09:10:041
425999360,11cyclictest31665-21/usr/sbin/munin09:55:331
426199357,6cyclictest22890-21cut09:30:193
426199354,19cyclictest18590-21latency_hist07:50:043
426199351,23cyclictest0-21swapper/311:43:193
426199351,22cyclictest8989-21ls08:50:323
426099356,6cyclictest7850irq/56-dwc_otg_08:25:262
426099352,23cyclictest2966-21uniq08:35:232
4260993517,6cyclictest28063-21df08:15:182
4260993510,5cyclictest24271-21grep12:30:322
425999350,17cyclictest0-21swapper/107:41:251
425999350,16cyclictest31242-21chrt09:52:451
426199344,22cyclictest0-21swapper/311:20:203
426199341,17cyclictest39-21ksoftirqd/312:05:053
426099346,15cyclictest7950irq/56-dwc_otg_09:50:282
426099343,17cyclictest7950irq/56-dwc_otg_09:15:262
426099341,7cyclictest4239-21unixbench_multi11:35:322
426099341,26cyclictest19143-21ls10:50:052
426099341,21cyclictest27217-21timerandwakeup09:40:332
4259993426,6cyclictest18426-21sshd07:48:101
425999341,8cyclictest22499-21aten_rbpower_en08:00:181
4259993413,9cyclictest7850irq/56-dwc_otg_10:49:111
4259993410,6cyclictest0-21swapper/107:50:181
325502340,1sleep30-21swapper/309:55:293
269652341,2sleep20-21swapper/208:10:322
426199336,15cyclictest0-21swapper/311:45:183
426199336,15cyclictest0-21swapper/310:25:183
426199336,15cyclictest0-21swapper/310:25:183
426199335,15cyclictest7950irq/56-dwc_otg_11:05:313
426199335,15cyclictest7950irq/56-dwc_otg_11:05:313
426199331,29cyclictest23176-21cut11:00:233
426199331,25cyclictest31583-21taskset09:55:053
426199331,16cyclictest25958-21ls08:10:053
426099338,13cyclictest7950irq/56-dwc_otg_11:10:062
426099338,13cyclictest7950irq/56-dwc_otg_11:10:062
426099335,18cyclictest0-21swapper/210:50:162
426099334,17cyclictest7950irq/56-dwc_otg_09:20:202
426099332,3cyclictest7750irq/56-dwc_otg10:19:262
425999336,16cyclictest0-21swapper/110:20:121
425999336,16cyclictest0-21swapper/110:20:121
425999335,6cyclictest7750irq/56-dwc_otg11:45:181
425999330,11cyclictest0-21swapper/109:44:251
232892331,24sleep30-21swapper/308:00:313
426199326,18cyclictest0-21swapper/308:31:393
426199322,21cyclictest0-21swapper/310:48:053
426199321,8cyclictest0-21swapper/307:35:163
426199321,15cyclictest7950irq/56-dwc_otg_09:05:063
426199321,15cyclictest7950irq/56-dwc_otg_09:05:063
426199321,14cyclictest0-21swapper/311:14:203
426199321,13cyclictest0-21swapper/307:28:443
426099327,13cyclictest7950irq/56-dwc_otg_08:55:232
426099325,10cyclictest7950irq/56-dwc_otg_08:00:052
426099324,5cyclictest14528-21grep07:35:332
426099321,22cyclictest13818-21idleruntime09:05:232
425999324,19cyclictest0-21swapper/111:28:281
4259993225,5cyclictest12309-21wget12:00:181
4259993212,5cyclictest21715-21unixbench_multi07:55:311
425999320,13cyclictest128-21systemd-journal10:09:271
425999320,12cyclictest29729-21latency_hist09:50:051
158332321,2sleep30-21swapper/309:10:263
426199318,5cyclictest7950irq/56-dwc_otg_09:45:253
426199317,14cyclictest0-21swapper/312:25:173
426199316,5cyclictest7950irq/56-dwc_otg_07:15:273
426199316,5cyclictest7950irq/56-dwc_otg_07:15:273
426199316,15cyclictest0-21swapper/311:50:113
426199312,10cyclictest0-21swapper/311:20:043
426199311,9cyclictest32068-21wget11:25:213
426199311,4cyclictest7950irq/56-dwc_otg_12:35:153
426199311,18cyclictest0-21swapper/310:35:163
426099316,4cyclictest15901-21fschecks_time10:40:212
426099316,15cyclictest0-21swapper/207:10:182
426099314,16cyclictest7950irq/56-dwc_otg_09:55:252
426099313,7cyclictest7750irq/56-dwc_otg12:25:042
4260993122,6cyclictest13840-21aten_rbpower_en10:35:182
426099312,18cyclictest0-21swapper/211:51:112
426099311,20cyclictest0-21swapper/212:09:012
426199302,13cyclictest0-21swapper/308:25:233
4261993014,5cyclictest24983-21/usr/sbin/munin09:35:253
4261993014,5cyclictest24983-21/usr/sbin/munin09:35:253
426199301,19cyclictest20569-21latency_hist07:55:053
426199301,18cyclictest0-21swapper/312:10:053
426199301,13cyclictest7750irq/56-dwc_otg09:05:263
426099306,15cyclictest0-21swapper/211:55:192
426099305,6cyclictest7750irq/56-dwc_otg07:45:242
426099305,22cyclictest1085-21uniq08:30:212
426099304,4cyclictest7950irq/56-dwc_otg_12:35:322
426099302,5cyclictest7750irq/56-dwc_otg08:50:342
426099302,3cyclictest7950irq/56-dwc_otg_11:20:052
4260993014,9cyclictest13711-21latency_hist10:35:052
426099301,25cyclictest4651-21ls08:40:202
425999306,13cyclictest7750irq/56-dwc_otg11:30:331
426199299,5cyclictest7950irq/56-dwc_otg_11:35:323
426199297,5cyclictest20073-21switchtime10:50:303
426199295,4cyclictest0-21swapper/310:55:323
426199292,4cyclictest28144-21uniq08:15:203
426199292,19cyclictest0-21swapper/309:17:023
426199292,13cyclictest7950irq/56-dwc_otg_10:15:313
426199291,22cyclictest0-21swapper/308:35:283
426199291,17cyclictest0-21swapper/312:34:213
426199291,17cyclictest0-21swapper/309:40:123
426199291,16cyclictest381ktimersoftd/310:40:243
426099297,6cyclictest7950irq/56-dwc_otg_10:55:162
426099297,3cyclictest0-21swapper/208:45:302
426099292,19cyclictest0-21swapper/208:22:462
4260992920,7cyclictest8631-21fschecks_count10:20:212
4260992920,7cyclictest8631-21fschecks_count10:20:212
426099291,9cyclictest1812-21wc11:30:262
426099291,16cyclictest16118-21cut12:10:222
426099291,11cyclictest7850irq/56-dwc_otg_11:00:282
425999292,5cyclictest0-21swapper/111:10:201
426199281,6cyclictest4289-21apt-get07:10:163
426199281,6cyclictest11440-21latency_hist09:00:053
426199281,19cyclictest0-21swapper/309:24:223
426099287,6cyclictest0-21swapper/207:25:182
426099284,5cyclictest7950irq/56-dwc_otg_09:45:282
426099282,16cyclictest0-21swapper/207:52:422
426099282,12cyclictest7750irq/56-dwc_otg09:00:152
426099282,12cyclictest7750irq/56-dwc_otg09:00:152
4260992819,5cyclictest15309-21wget07:40:202
426099281,20cyclictest0-21swapper/212:29:282
426199272,15cyclictest6246-21/usr/sbin/munin08:45:283
426199271,12cyclictest7950irq/56-dwc_otg_08:10:333
426099275,6cyclictest7850irq/56-dwc_otg_09:30:212
426099273,14cyclictest0-21swapper/207:33:312
426099272,5cyclictest7850irq/56-dwc_otg_08:00:232
426099272,3cyclictest0-21swapper/209:35:272
426099272,3cyclictest0-21swapper/209:35:272
426099271,17cyclictest12489-21uniq12:00:222
38982277,13sleep10-21swapper/107:08:561
426199264,10cyclictest7950irq/56-dwc_otg_08:20:253
426099262,9cyclictest0-21swapper/210:26:142
426099262,9cyclictest0-21swapper/210:26:142
426099255,9cyclictest7750irq/56-dwc_otg11:11:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional