You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 17:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Fri Aug 14, 2026 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18430991172,94cyclictest19410-21ntpq10:15:303
18427991164,82cyclictest2948-21fschecks_count10:35:230
18430991143,89cyclictest11576-21date10:45:293
18427991145,69cyclictest0-21swapper/011:35:260
18430991124,83cyclictest29215-21diskmemload10:26:403
18430991113,86cyclictest3927-21/usr/sbin/munin11:55:203
18430991113,86cyclictest3927-21/usr/sbin/munin11:55:203
18427991105,81cyclictest0-21swapper/011:40:230
184279910834,40cyclictest6178-21ssh10:00:170
18429991073,23cyclictest7850irq/56-dwc_otg_10:45:232
18430991066,78cyclictest32049-21ls10:30:323
18430991066,78cyclictest32049-21ls10:30:323
18427991063,65cyclictest0-21swapper/010:30:330
18427991063,65cyclictest0-21swapper/010:30:330
18430991053,71cyclictest8715-21ssh12:00:273
184279910368,21cyclictest101ktimersoftd/012:35:310
184279910321,14cyclictest0-21swapper/010:55:230
184309910227,69cyclictest2623-21/usr/sbin/munin10:35:293
18430991022,73cyclictest32628-21latency_hist12:30:063
184279910149,32cyclictest0-21swapper/009:35:220
18427999946,37cyclictest101ktimersoftd/012:30:070
31372980,3sleep121-21rcuc/112:33:131
1842799983,75cyclictest0-21swapper/011:10:190
183912980,5sleep20-21swapper/207:10:062
18429999612,8cyclictest7850irq/56-dwc_otg_10:26:392
18430999514,65cyclictest7850irq/56-dwc_otg_09:35:223
18429999511,10cyclictest7850irq/56-dwc_otg_11:55:202
18429999511,10cyclictest7850irq/56-dwc_otg_11:55:202
1842999945,10cyclictest0-21swapper/210:15:272
1842799946,57cyclictest17931-21kworker/0:011:55:210
1842799946,57cyclictest17931-21kworker/0:011:55:210
1843099933,49cyclictest0-21swapper/311:35:263
1842999929,17cyclictest7950irq/56-dwc_otg_09:35:222
1842999923,20cyclictest7750irq/56-dwc_otg12:00:262
1842799928,64cyclictest9-21ksoftirqd/010:45:210
1843099913,61cyclictest0-21swapper/312:30:083
1842799903,62cyclictest29325-21grep11:45:350
1843099894,67cyclictest6826-21rm10:00:263
18427998910,60cyclictest19272-21missed_timers10:15:270
1843099883,66cyclictest0-21swapper/312:10:263
18427998830,18cyclictest0-21swapper/010:25:260
1843099873,76cyclictest24303-21ssh11:40:243
18427998756,15cyclictest101ktimersoftd/012:00:290
1842799873,64cyclictest0-21swapper/011:05:270
1842799862,61cyclictest0-21swapper/009:55:290
1843099853,75cyclictest2696-21memory09:55:273
1843099852,75cyclictest5355-21/usr/sbin/munin12:35:303
18427998511,35cyclictest0-21swapper/008:05:180
1843099843,74cyclictest30350-21/usr/sbin/munin09:50:343
1843099843,74cyclictest29313-21grep11:45:353
1843099843,72cyclictest31589-21apt-get11:10:163
131322840,2sleep20-21swapper/208:25:182
1843099833,73cyclictest19600-21cut10:55:243
61562820,5sleep21842999cyclictest09:20:242
1843099822,72cyclictest28234-21ntp_kernel_pll_11:05:273
18429998110,22cyclictest7950irq/56-dwc_otg_11:40:202
1842799812,56cyclictest7950irq/56-dwc_otg_09:50:360
1843099803,69cyclictest10583-21ntp_states09:25:323
1842799807,8cyclictest7750irq/56-dwc_otg12:10:260
18429997920,12cyclictest301ktimersoftd/210:35:232
1842799773,10cyclictest10583-21ntp_states09:25:320
1842799766,54cyclictest0-21swapper/011:00:190
1842999753,21cyclictest29-21rcuc/212:30:062
1842899758,14cyclictest32056-21ssh10:30:331
1842899758,14cyclictest32056-21ssh10:30:331
18428997511,30cyclictest23914-21/usr/sbin/munin11:40:211
1842799752,61cyclictest32391-21wget11:50:190
35242740,1sleep10-21swapper/111:54:561
259022730,11sleep00-21swapper/010:24:040
1842799736,24cyclictest0-21swapper/012:05:190
182702736,53sleep10-21swapper/107:10:061
18429997128,23cyclictest7750irq/56-dwc_otg11:05:272
1842799713,10cyclictest0-21swapper/012:30:080
1843099702,7cyclictest15804-21/usr/sbin/munin10:50:263
18429997015,10cyclictest29222-21ls11:45:342
1842899706,44cyclictest0-21swapper/109:50:361
18428997031,10cyclictest3051-21idleruntime10:35:251
18428997012,34cyclictest17040-21ssh12:10:281
1842799701,65cyclictest0-21swapper/009:10:170
1842799692,8cyclictest7850irq/56-dwc_otg_07:10:070
1843099686,5cyclictest7850irq/56-dwc_otg_11:25:213
18427996815,24cyclictest0-21swapper/009:40:160
1842899678,42cyclictest19520-21ssh10:55:231
18428996721,23cyclictest0-21swapper/111:05:271
1842799672,6cyclictest16301-21grep11:30:250
114972670,14sleep11842899cyclictest10:45:271
259922660,2sleep20-21swapper/210:24:142
223992660,2sleep20-21swapper/212:16:432
1842999664,47cyclictest0-21swapper/211:35:272
18428996616,17cyclictest7850irq/56-dwc_otg_09:35:221
292252653,6sleep10-21swapper/111:45:351
1843099654,47cyclictest20762-21ssh12:15:233
18428996514,31cyclictest21-21rcuc/112:00:291
178122653,12sleep00-21swapper/007:05:440
189072640,2sleep20-21swapper/212:12:532
1842899648,31cyclictest7950irq/56-dwc_otg_10:25:271
18428996428,29cyclictest15084-21ssh10:50:171
162202640,3sleep2301ktimersoftd/209:32:282
1843099633,10cyclictest18431-21taskset07:10:073
1842999639,31cyclictest7950irq/56-dwc_otg_10:55:232
1842999635,13cyclictest0-21swapper/210:30:332
1842999635,13cyclictest0-21swapper/210:30:332
1842899639,7cyclictest0-21swapper/112:30:071
18427996318,42cyclictest101ktimersoftd/010:40:220
7462620,5sleep11842899cyclictest11:11:081
1843099627,38cyclictest0-21swapper/310:42:153
18429996211,29cyclictest32245-21missed_timers11:10:282
1842899628,31cyclictest7950irq/56-dwc_otg_11:55:211
1842899628,31cyclictest7950irq/56-dwc_otg_11:55:211
1842899623,43cyclictest0-21swapper/109:55:271
18428996211,33cyclictest27037-21kworker/1:110:15:301
18427996213,37cyclictest0-21swapper/009:21:180
1842999613,37cyclictest7750irq/56-dwc_otg10:00:262
1842899613,41cyclictest0-21swapper/111:35:261
18428996115,25cyclictest7850irq/56-dwc_otg_12:35:291
1842799611,5cyclictest12405-21grep08:20:340
1842999604,40cyclictest0-21swapper/212:30:082
18429996015,19cyclictest0-21swapper/212:35:292
1842899606,16cyclictest7408-21ssh11:20:151
1842799609,19cyclictest7950irq/56-dwc_otg_10:50:110
18427996033,8cyclictest0-21swapper/011:20:150
1843099599,36cyclictest29215-21diskmemload09:48:253
1843099599,36cyclictest29215-21diskmemload09:48:253
1843099591,5cyclictest7850irq/56-dwc_otg_08:26:213
1843099591,47cyclictest0-21swapper/310:20:323
1842999596,11cyclictest3281-21ssh11:15:172
18427995915,6cyclictest0-21swapper/009:15:170
18427995811,9cyclictest7750irq/56-dwc_otg08:45:180
18430995719,35cyclictest29120-21latency_hist07:40:053
1843099571,5cyclictest2379-21rm09:15:303
1843099571,5cyclictest18444-21ssh10:15:063
1843099571,5cyclictest18444-21ssh10:15:063
18430995715,38cyclictest26435-21sh12:21:333
1843099561,5cyclictest30960-21latency_hist07:45:053
71232550,1sleep10-21swapper/111:20:051
18430995515,36cyclictest31505-21/usr/sbin/munin07:45:233
1843099551,45cyclictest17732-21ssh09:35:053
1843099551,39cyclictest0-21swapper/308:04:233
1842799552,30cyclictest2300-21grep07:55:190
1843099547,37cyclictest7950irq/56-dwc_otg_08:45:293
1843099544,36cyclictest7950irq/56-dwc_otg_11:01:013
1843099541,9cyclictest7950irq/56-dwc_otg_09:10:253
1843099541,9cyclictest32237-21/usr/sbin/munin11:50:263
1843099541,5cyclictest25499-21idleruntime-cro09:00:053
1843099541,5cyclictest25499-21idleruntime-cro09:00:053
18429995412,20cyclictest7750irq/56-dwc_otg09:25:302
1842899549,12cyclictest23-21ksoftirqd/109:25:301
1842799546,38cyclictest0-21swapper/009:01:290
1842799543,35cyclictest1103-21ntp_kernel_err07:50:250
18427995425,7cyclictest14204-21unixbench_multi08:25:330
1842799542,29cyclictest9387-21latency_hist08:15:060
1843099531,4cyclictest7461-21/usr/sbin/munin11:20:173
1843099531,42cyclictest0-21swapper/308:21:133
18430995311,39cyclictest4495-21unixbench_multi11:15:323
1842999534,17cyclictest0-21swapper/207:10:072
1842799537,7cyclictest14320-21if_err_wlan009:30:240
1842799532,43cyclictest0-21swapper/007:30:230
18427995315,7cyclictest0-21swapper/012:15:190
1843099523,8cyclictest26807-21unixbench_multi07:30:363
1843099522,5cyclictest0-21swapper/309:24:203
1843099522,4cyclictest0-21swapper/307:30:053
1843099522,3cyclictest7750irq/56-dwc_otg08:10:163
1842999525,21cyclictest31713-21ssh09:50:372
18429995212,23cyclictest0-21swapper/209:55:282
1842899522,8cyclictest7950irq/56-dwc_otg_08:05:191
18428995210,19cyclictest6779-21ssh10:00:241
18427995227,15cyclictest7750irq/56-dwc_otg10:06:120
1842799522,30cyclictest23824-21latency_hist07:25:060
1842799521,7cyclictest3631-21ssh11:15:220
1842799521,5cyclictest15698-21users10:10:350
1842799521,5cyclictest15698-21users10:10:350
1843099517,41cyclictest25430-21cron07:30:043
1843099513,37cyclictest7950irq/56-dwc_otg_12:05:273
1843099512,46cyclictest20344-21aten_rbpower_po07:15:193
1843099511,3cyclictest7950irq/56-dwc_otg_10:07:563
18428995116,9cyclictest7950irq/56-dwc_otg_12:15:171
1842799515,30cyclictest15593-21tr08:30:280
1842799512,26cyclictest11234-21latency_hist08:20:050
1843099508,25cyclictest0-21swapper/309:40:153
1843099501,9cyclictest15000-21grep08:30:193
1843099501,4cyclictest0-21swapper/308:15:243
1843099501,3cyclictest0-21swapper/311:31:563
1842799502,33cyclictest0-21swapper/009:05:200
18427995021,6cyclictest4132-21tr08:00:190
1843099497,39cyclictest1514-21grep07:50:313
1843099492,7cyclictest22136-21aten_rbpower_en07:20:193
1842799492,6cyclictest25003-21unixbench_multi07:25:360
1842799492,6cyclictest25003-21unixbench_multi07:25:360
18427994919,16cyclictest31853-21ps07:45:290
1842799491,20cyclictest0-21swapper/012:20:250
1842799482,6cyclictest16779-21aten_rbpower_en08:35:200
1842799481,29cyclictest0-21swapper/008:40:170
1842799481,24cyclictest0-21swapper/008:55:190
1842799481,24cyclictest0-21swapper/008:55:190
1843099472,6cyclictest6214-21fschecks_time08:05:233
1843099472,10cyclictest0-21swapper/308:35:363
1843099471,6cyclictest27674-21grep09:05:173
1843099471,5cyclictest18779-21fschecks_count08:40:223
1842899477,25cyclictest0-21swapper/112:20:191
1842899477,25cyclictest0-21swapper/108:45:181
18427994720,14cyclictest7950irq/56-dwc_otg_07:40:260
1843099463,4cyclictest7950irq/56-dwc_otg_08:53:593
18430994610,33cyclictest26017-21fschecks_time09:00:223
18430994610,32cyclictest3120-21switchtime07:55:313
1842899467,25cyclictest0-21swapper/110:05:181
1842899465,10cyclictest7850irq/56-dwc_otg_10:20:161
18428994615,3cyclictest0-21swapper/108:50:241
18427994631,4cyclictest0-21swapper/011:25:210
1842799459,15cyclictest0-21swapper/008:50:240
1842999447,8cyclictest7750irq/56-dwc_otg11:30:162
1842999447,25cyclictest0-21swapper/209:00:182
1842999447,23cyclictest0-21swapper/208:30:182
1842999443,12cyclictest7750irq/56-dwc_otg09:45:182
1842999443,12cyclictest7750irq/56-dwc_otg09:45:182
18428994422,5cyclictest27397-21latency_hist09:05:051
18427994414,23cyclictest20821-21missed_timers07:15:260
1842999437,24cyclictest0-21swapper/208:50:182
1842799432,24cyclictest0-21swapper/009:45:380
1842799432,24cyclictest0-21swapper/009:45:380
1842899427,25cyclictest0-21swapper/111:25:171
1842899424,11cyclictest0-21swapper/107:10:071
18428994221,5cyclictest52950wget07:50:181
18428994211,20cyclictest60-21kswapd009:40:151
1842999417,24cyclictest0-21swapper/208:15:192
1842899418,23cyclictest0-21swapper/108:40:191
1842899414,25cyclictest0-21swapper/111:30:181
18427994116,9cyclictest0-21swapper/007:36:120
1842899408,7cyclictest0-21swapper/109:20:151
18428994016,6cyclictest13802-21sh08:25:271
136682400,1sleep20-21swapper/210:09:242
125092401,34sleep20-21swapper/211:25:312
30732390,2sleep20-21swapper/209:16:142
1842899399,5cyclictest0-21swapper/109:46:371
1842899399,5cyclictest0-21swapper/109:46:371
1842899386,22cyclictest0-21swapper/108:35:201
1842899386,21cyclictest14408-21ssh10:10:191
1842899386,21cyclictest14408-21ssh10:10:191
102332380,1sleep20-21swapper/211:23:162
29992370,2sleep10-21swapper/107:55:301
1842999378,11cyclictest12229-21ssh12:05:192
1842999372,25cyclictest7950irq/56-dwc_otg_11:50:152
18429993630,4cyclictest17216-21ls08:35:262
1842899361,22cyclictest4356-21cut08:00:221
1842999359,6cyclictest0-21swapper/208:55:232
1842999359,6cyclictest0-21swapper/208:55:232
1842999355,8cyclictest7750irq/56-dwc_otg10:50:312
1842999354,14cyclictest7950irq/56-dwc_otg_07:20:332
1842999352,3cyclictest0-21swapper/210:15:052
1842999352,3cyclictest0-21swapper/210:15:052
1842899357,19cyclictest23379-21ssh11:00:191
1842899356,19cyclictest21-21rcuc/109:05:111
18428993518,5cyclictest21170-21/usr/sbin/munin07:15:321
18428993515,11cyclictest1180-21runrttasks08:33:091
1842899351,25cyclictest7916-21cut08:10:221
179712353,23sleep30-21swapper/307:07:503
1842999347,18cyclictest0-21swapper/209:40:182
1842999341,5cyclictest21419-21grep08:45:322
18429993410,15cyclictest18426-21cyclictest09:06:062
1842899347,9cyclictest7950irq/56-dwc_otg_09:13:021
1842899346,12cyclictest7850irq/56-dwc_otg_07:20:221
1842899343,5cyclictest7079-21fschecks_time10:40:221
18428993411,8cyclictest0-21swapper/108:20:311
1842999337,6cyclictest7850irq/56-dwc_otg_10:40:192
1842999335,5cyclictest7850irq/56-dwc_otg_07:50:172
1842999332,19cyclictest0-21swapper/208:05:182
1842999331,5cyclictest0-21swapper/207:45:182
1842999331,20cyclictest29930-21cut09:10:232
1842899337,5cyclictest7950irq/56-dwc_otg_09:15:171
1842899335,3cyclictest0-21swapper/112:05:321
18428993311,10cyclictest7850irq/56-dwc_otg_07:40:041
18428993310,9cyclictest0-21swapper/108:55:371
18428993310,9cyclictest0-21swapper/108:55:371
1842999325,7cyclictest7850irq/56-dwc_otg_08:10:332
1842999321,21cyclictest23579-21ls11:00:212
18428993212,5cyclictest0-21swapper/107:45:281
18428993212,3cyclictest0-21swapper/109:31:471
1842999318,11cyclictest7850irq/56-dwc_otg_07:25:282
1842999318,11cyclictest7850irq/56-dwc_otg_07:25:282
1842999314,6cyclictest7850irq/56-dwc_otg_08:05:052
1842999313,17cyclictest29082-21latency_hist07:40:042
1842999312,5cyclictest7850irq/56-dwc_otg_07:55:312
1842899318,14cyclictest29968-21ntp_kernel_pll_07:40:261
1842899316,16cyclictest21-21rcuc/107:30:181
1842999302,4cyclictest25727-21timerwakeupswit12:20:332
1842899304,14cyclictest10177-21grep08:15:281
1842899302,5cyclictest24225-21fschecks_time07:25:261
1842899302,5cyclictest24225-21fschecks_time07:25:261
1842999292,4cyclictest11284-21/usr/sbin/munin08:20:272
1842999291,5cyclictest7850irq/56-dwc_otg_07:40:172
1842999282,5cyclictest7950irq/56-dwc_otg_07:30:332
1842999276,5cyclictest7850irq/56-dwc_otg_08:40:192
1842999272,15cyclictest0-21swapper/207:15:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional