You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 09:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat Nov 22, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8699999854,15cyclictest0-21swapper/020:25:190
37042970,4sleep13708-21unixbench_singl23:35:351
8699999051,8cyclictest7950irq/56-dwc_otg_00:25:170
869999826,65cyclictest0-21swapper/020:50:180
869999823,53cyclictest0-21swapper/022:05:190
869999823,53cyclictest0-21swapper/022:05:190
108552770,1sleep30-21swapper/323:45:313
869999763,7cyclictest19594-21diskmemload22:50:220
8699997613,50cyclictest0-21swapper/019:10:060
24622740,3sleep231-21ksoftirqd/223:35:202
17812720,3sleep229-21rcuc/223:34:272
89942710,2sleep123-21ksoftirqd/123:43:361
869999717,41cyclictest0-21swapper/021:30:230
8699997117,16cyclictest3895-21ssh00:20:170
322552710,6sleep27850irq/56-dwc_otg_22:50:072
322552710,6sleep27850irq/56-dwc_otg_22:50:072
8699997012,38cyclictest101ktimersoftd/023:30:060
8702996917,22cyclictest7850irq/56-dwc_otg_20:25:192
8702996820,16cyclictest0-21swapper/223:55:182
869999676,58cyclictest0-21swapper/020:20:220
869999672,40cyclictest22634-21ssh22:36:300
869999651,61cyclictest101ktimersoftd/023:40:560
8701996420,11cyclictest7750irq/56-dwc_otg23:15:191
869999642,48cyclictest27071-21ntpq22:00:300
8699996417,26cyclictest101ktimersoftd/020:35:370
870199635,12cyclictest0-21swapper/121:40:181
869999639,38cyclictest0-21swapper/022:25:260
8699996323,17cyclictest29655-21if_wlan000:10:260
869999632,15cyclictest19504-21ssh21:50:330
869999628,35cyclictest22677-21ssh21:55:240
869999626,40cyclictest25647-21runrttasks21:16:410
869999626,40cyclictest25647-21runrttasks21:16:410
869999624,44cyclictest31971-21runrttasks20:14:000
8699996115,31cyclictest101ktimersoftd/020:00:340
869999611,4cyclictest10568-21cat21:40:070
869999611,36cyclictest0-21swapper/019:15:180
869999609,47cyclictest101ktimersoftd/000:00:350
869999607,38cyclictest0-21swapper/023:10:220
869999604,44cyclictest13587-21unixbench_singl19:20:360
869999600,37cyclictest0-21swapper/020:30:170
869999594,41cyclictest0-21swapper/021:14:170
130862590,4sleep113085-21grep19:20:291
869999588,4cyclictest0-21swapper/021:40:160
869999588,31cyclictest17363-21rm23:55:010
869999583,31cyclictest0-21swapper/019:25:200
869999578,31cyclictest0-21swapper/023:35:230
869999571,5cyclictest31645-21wget21:25:210
870199567,30cyclictest0-21swapper/119:30:211
870199567,30cyclictest0-21swapper/119:30:211
8699995648,4cyclictest0-21swapper/000:30:190
8699995614,39cyclictest101ktimersoftd/022:45:070
869999561,41cyclictest13732-21runrttasks23:07:160
869999560,49cyclictest24994-21grep19:55:060
869999554,35cyclictest7468-21ssh22:17:050
869999552,41cyclictest28871-21idleruntime-cro00:10:060
8699995512,5cyclictest3600-21ssh00:20:060
870199544,25cyclictest7766-21ssh23:00:201
8699995410,24cyclictest0-21swapper/021:00:170
869999540,41cyclictest0-21swapper/019:55:320
8701995315,22cyclictest21-21rcuc/100:00:311
8699995333,6cyclictest7850irq/56-dwc_otg_23:30:190
8699995331,5cyclictest10709-21/usr/sbin/munin20:45:290
8699995328,11cyclictest7950irq/56-dwc_otg_00:35:210
8699995325,15cyclictest7950irq/56-dwc_otg_20:15:340
8699995325,15cyclictest7950irq/56-dwc_otg_20:15:340
869999531,28cyclictest7404-21cron23:00:060
869999529,29cyclictest11648-21ssh22:22:390
869999528,32cyclictest0-21swapper/022:11:310
869999526,35cyclictest24463-21grep23:20:350
8699995228,13cyclictest0-21swapper/020:55:190
8699995227,4cyclictest7950irq/56-dwc_otg_22:30:230
8699995227,4cyclictest7950irq/56-dwc_otg_22:30:230
869999522,38cyclictest0-21swapper/023:55:130
870299517,29cyclictest0-21swapper/219:40:232
8699995126,7cyclictest28761-21aten_rbpower_en20:05:200
869999512,5cyclictest0-21swapper/019:40:060
80982513,41sleep20-21swapper/219:05:562
870199507,32cyclictest0-21swapper/120:10:181
8701995015,14cyclictest21-21rcuc/121:00:271
8701995011,6cyclictest0-21swapper/121:10:181
869999506,33cyclictest0-21swapper/021:24:530
8699995030,8cyclictest0-21swapper/021:46:500
869999501,23cyclictest17195-21grep19:30:360
869999501,23cyclictest17195-21grep19:30:360
869999500,34cyclictest22046-21ssh23:18:270
869999497,31cyclictest0-21swapper/023:00:340
8699994931,5cyclictest7950irq/56-dwc_otg_22:45:250
8699994931,5cyclictest7950irq/56-dwc_otg_22:45:250
870299484,8cyclictest0-21swapper/223:25:202
869999488,27cyclictest0-21swapper/019:49:480
8699994825,11cyclictest7950irq/56-dwc_otg_23:45:260
869999482,39cyclictest10370-21taskset20:43:520
8699994816,6cyclictest19600-21modprobe21:10:060
8699994815,21cyclictest0-21swapper/019:41:120
139462480,1sleep10-21swapper/122:25:301
870399471,13cyclictest0-21swapper/300:05:173
870199471,5cyclictest7950irq/56-dwc_otg_23:10:221
870299467,25cyclictest0-21swapper/220:10:182
870199462,10cyclictest0-21swapper/122:50:221
80962463,34sleep00-21swapper/019:05:540
870299457,25cyclictest0-21swapper/200:30:202
870299457,24cyclictest0-21swapper/219:30:212
870299457,24cyclictest0-21swapper/219:30:212
870299453,30cyclictest0-21swapper/221:15:212
870299453,30cyclictest0-21swapper/221:15:212
870199457,26cyclictest0-21swapper/122:40:111
870199457,25cyclictest0-21swapper/119:40:231
870299448,28cyclictest7850irq/56-dwc_otg_22:50:222
870299447,22cyclictest0-21swapper/219:15:112
8702994418,18cyclictest29-21rcuc/219:55:062
870299441,26cyclictest0-21swapper/223:50:172
8703994328,12cyclictest25200-21fschecks_time22:40:253
870299437,24cyclictest0-21swapper/220:50:122
8702994321,8cyclictest31779-21sh21:25:232
8701994310,25cyclictest7750irq/56-dwc_otg00:10:061
234432430,7sleep3381ktimersoftd/322:38:083
870399427,22cyclictest0-21swapper/321:40:113
870399422,14cyclictest0-21swapper/322:05:193
870399422,14cyclictest0-21swapper/322:05:193
8703994215,14cyclictest15026-21perl21:45:233
870299427,22cyclictest0-21swapper/222:40:112
870199428,24cyclictest0-21swapper/121:55:221
870199427,18cyclictest0-21swapper/119:25:201
870199421,4cyclictest7850irq/56-dwc_otg_20:20:211
870399413,8cyclictest7750irq/56-dwc_otg19:20:193
8703994113,5cyclictest3537-21timerwakeupswit23:35:333
870299414,7cyclictest7850irq/56-dwc_otg_23:00:212
870299412,3cyclictest0-21swapper/220:20:212
8702994115,13cyclictest195812chrt23:15:232
870199415,6cyclictest0-21swapper/100:20:191
36782410,2sleep20-21swapper/222:12:142
8703994026,8cyclictest381ktimersoftd/300:30:183
870399401,21cyclictest3691-21latency_hist22:55:063
870299407,4cyclictest7950irq/56-dwc_otg_19:40:062
8702994019,5cyclictest31-21ksoftirqd/220:00:232
870199404,26cyclictest7750irq/56-dwc_otg22:30:211
870199404,26cyclictest7750irq/56-dwc_otg22:30:211
870199403,5cyclictest19426-21apt19:40:051
191982401,4sleep20-21swapper/222:32:242
191982401,4sleep20-21swapper/222:32:242
870399399,4cyclictest0-21swapper/322:22:113
870399397,21cyclictest0-21swapper/321:30:213
870399394,3cyclictest7750irq/56-dwc_otg19:50:193
8702993915,4cyclictest30423-21grep00:10:352
870199397,21cyclictest2803-21latency_hist21:30:061
870199392,16cyclictest0-21swapper/120:40:161
136972390,1sleep20-21swapper/221:43:482
870399387,25cyclictest0-21swapper/319:15:203
8703993816,5cyclictest5564-21cpu22:15:203
870399381,25cyclictest28937-21ls22:45:243
870399381,25cyclictest28937-21ls22:45:243
870399381,22cyclictest32255-21ls20:15:073
870299385,4cyclictest0-21swapper/221:20:192
870299384,22cyclictest7850irq/56-dwc_otg_00:00:342
8702993819,5cyclictest24610-21latency_hist22:40:062
8702993818,4cyclictest15386-21sh00:35:362
870299381,3cyclictest7950irq/56-dwc_otg_21:55:192
870299381,19cyclictest7850irq/56-dwc_otg_23:07:092
870199388,6cyclictest7950irq/56-dwc_otg_23:25:201
870199384,21cyclictest7750irq/56-dwc_otg23:45:351
870199384,19cyclictest7750irq/56-dwc_otg22:25:061
8701993818,5cyclictest15211-21sensors00:35:341
870199381,7cyclictest20925-21aten_rbpower_cu22:35:211
870199381,6cyclictest0-21swapper/121:35:271
8701993815,5cyclictest26373-21wget22:00:211
8701993811,14cyclictest21-21rcuc/120:35:371
8701993811,13cyclictest23-21ksoftirqd/123:55:131
870399377,15cyclictest7736-21ssh21:35:343
870399371,23cyclictest32634-21uniq20:15:233
870399371,23cyclictest32634-21uniq20:15:233
870399371,23cyclictest13216-21ls20:50:263
870299375,21cyclictest7850irq/56-dwc_otg_19:10:332
8702993719,6cyclictest17588-21sh21:48:492
8702993713,4cyclictest26358-21sh23:24:192
870199376,21cyclictest7850irq/56-dwc_otg_00:15:071
870199375,17cyclictest7850irq/56-dwc_otg_22:55:231
870199374,5cyclictest14456-21cat23:50:251
870199374,5cyclictest0-21swapper/121:20:191
870199374,22cyclictest0-21swapper/119:55:171
870199374,19cyclictest7750irq/56-dwc_otg23:30:351
870199371,9cyclictest0-21swapper/122:05:321
870199371,9cyclictest0-21swapper/122:05:321
8701993715,6cyclictest23-21ksoftirqd/121:05:351
8701993711,10cyclictest0-21swapper/121:50:191
870399367,6cyclictest7950irq/56-dwc_otg_23:00:063
870399365,12cyclictest7850irq/56-dwc_otg_23:45:073
870399362,5cyclictest8181-21grep00:25:313
8703993617,5cyclictest24878-21sh21:57:543
8703993617,5cyclictest1783-21aten_rbpower_vo20:20:233
8703993613,4cyclictest23563-21grep23:20:233
870299367,24cyclictest0-21swapper/219:55:192
870299366,5cyclictest29-21rcuc/222:20:062
870299366,23cyclictest7850irq/56-dwc_otg_20:35:062
870299365,19cyclictest0-21swapper/200:10:072
870299364,21cyclictest0-21swapper/221:52:472
8702993624,7cyclictest10088-21sh23:45:202
8702993617,4cyclictest7950irq/56-dwc_otg_21:30:192
870299361,6cyclictest7850irq/56-dwc_otg_20:40:322
870299361,4cyclictest0-21swapper/222:00:222
870199365,4cyclictest948-21kernelversion00:15:281
870199365,20cyclictest7750irq/56-dwc_otg20:15:311
870199365,20cyclictest7750irq/56-dwc_otg20:15:311
870199365,18cyclictest7750irq/56-dwc_otg22:49:151
870199365,18cyclictest7750irq/56-dwc_otg22:49:151
870199363,22cyclictest7850irq/56-dwc_otg_20:05:171
870199362,25cyclictest7750irq/56-dwc_otg19:50:341
870199361,6cyclictest6017-21latency22:15:261
131432361,2sleep20-21swapper/219:20:312
870399356,14cyclictest7950irq/56-dwc_otg_20:50:063
870399356,14cyclictest7950irq/56-dwc_otg_20:25:333
870399355,20cyclictest3906-21ssh00:20:193
870399352,4cyclictest369-21users21:25:373
8703993517,10cyclictest19943-21sh21:10:183
870399351,22cyclictest19594-21diskmemload22:05:073
8703993511,4cyclictest7950irq/56-dwc_otg_00:05:073
870299357,8cyclictest7123-21grep21:35:252
870299356,17cyclictest7950irq/56-dwc_otg_22:55:242
870299353,20cyclictest7850irq/56-dwc_otg_22:28:332
8702993526,6cyclictest7950irq/56-dwc_otg_22:05:172
8702993526,6cyclictest7950irq/56-dwc_otg_22:05:172
8702993518,11cyclictest8463-21ssh23:42:312
8702993514,12cyclictest31-21ksoftirqd/200:20:292
870299351,17cyclictest0-21swapper/222:20:292
870199356,16cyclictest0-21swapper/123:20:191
870199355,8cyclictest0-21swapper/100:25:181
870199355,19cyclictest7850irq/56-dwc_otg_20:45:271
870199351,5cyclictest7750irq/56-dwc_otg19:10:331
8701993514,4cyclictest7850irq/56-dwc_otg_21:30:361
870199351,26cyclictest15239-21latency21:45:251
870199351,23cyclictest12682-21ssh23:05:331
8701993510,11cyclictest23-21ksoftirqd/120:30:271
870399346,6cyclictest7850irq/56-dwc_otg_21:25:053
870399341,5cyclictest7950irq/56-dwc_otg_21:05:173
8703993413,11cyclictest19594-21diskmemload22:26:293
870399341,27cyclictest25413-21cut19:55:233
8703993410,5cyclictest7950irq/56-dwc_otg_20:55:343
870299349,6cyclictest8556-1kworker/2:2H19:45:072
8702993416,4cyclictest1120-21runrttasks21:10:192
870299341,24cyclictest7850irq/56-dwc_otg_20:15:312
870299341,24cyclictest7850irq/56-dwc_otg_20:15:312
870199349,9cyclictest0-21swapper/122:10:201
870199347,17cyclictest0-21swapper/119:15:111
870199344,19cyclictest7750irq/56-dwc_otg19:45:231
870199343,5cyclictest7750irq/56-dwc_otg00:30:351
870199342,5cyclictest24106-21ssh21:15:261
870199342,5cyclictest24106-21ssh21:15:261
870199342,23cyclictest0-21swapper/120:50:111
870199342,17cyclictest7850irq/56-dwc_otg_20:01:201
870399339,4cyclictest7850irq/56-dwc_otg_20:05:343
870399334,20cyclictest7950irq/56-dwc_otg_23:15:203
870399334,18cyclictest7950irq/56-dwc_otg_22:35:073
870399334,18cyclictest7950irq/56-dwc_otg_22:35:073
8703993314,10cyclictest19594-21diskmemload23:11:073
8703993310,12cyclictest0-21swapper/323:25:303
870299335,19cyclictest15493-21ssh23:10:242
870399326,4cyclictest7850irq/56-dwc_otg_00:15:373
870399321,20cyclictest24929-21ssh21:15:363
870399321,20cyclictest24929-21ssh21:15:363
870399321,16cyclictest10474-21latency_hist19:15:063
8703993210,6cyclictest1593-21grep22:10:193
870299322,4cyclictest7850irq/56-dwc_otg_21:05:062
8702993213,9cyclictest12091-21seq20:47:552
870199328,4cyclictest0-21swapper/120:55:341
870199324,5cyclictest23-21ksoftirqd/120:25:161
870399319,6cyclictest7950irq/56-dwc_otg_23:30:323
870399318,5cyclictest7950irq/56-dwc_otg_23:05:173
870399318,11cyclictest7950irq/56-dwc_otg_00:10:223
870399317,5cyclictest0-21swapper/320:40:303
870399317,14cyclictest0-21swapper/323:50:193
870399316,14cyclictest15067-21grep19:25:333
870399312,20cyclictest0-21swapper/300:36:433
8703993114,5cyclictest11159-21sh23:04:543
870299317,15cyclictest0-21swapper/200:19:042
870299316,5cyclictest7750irq/56-dwc_otg20:35:362
8702993111,10cyclictest9160-21ssh00:26:572
840923111,11sleep10-21swapper/119:09:511
81262313,20sleep30-21swapper/319:06:173
870399304,4cyclictest7850irq/56-dwc_otg_21:00:283
870399302,14cyclictest7950irq/56-dwc_otg_19:35:063
870399302,14cyclictest7950irq/56-dwc_otg_19:35:063
870399301,26cyclictest5532-21cut20:30:233
870399301,14cyclictest7750irq/56-dwc_otg21:50:213
870299305,12cyclictest19594-21diskmemload21:10:062
8702993011,10cyclictest1120-21runrttasks20:06:072
870399294,5cyclictest7950irq/56-dwc_otg_20:35:223
870399292,8cyclictest7750irq/56-dwc_otg23:55:113
203372280,2sleep30-21swapper/319:40:323
870399271,17cyclictest0-21swapper/319:39:383
870299276,6cyclictest15208-21timerandwakeup19:25:342
8702992719,6cyclictest31331-21snmpd20:56:252
870399261,5cyclictest26968-21aten_rbpower_po20:00:223
227522261,4sleep30-21swapper/319:46:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional