You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 06:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat Dec 20, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2754521890,14sleep127548-21latency_hist19:10:051
2754521890,14sleep127548-21latency_hist19:10:051
27684999520,39cyclictest0-21swapper/019:30:180
2768499877,45cyclictest0-21swapper/021:30:190
2768499846,41cyclictest0-21swapper/022:45:180
27684998055,17cyclictest0-21swapper/022:50:110
27684997842,19cyclictest101ktimersoftd/019:10:070
54982770,2sleep10-21swapper/121:05:331
2768499775,47cyclictest28382-21runrttasks23:40:170
2768499757,25cyclictest0-21swapper/021:25:110
2768499757,25cyclictest0-21swapper/021:25:110
2768499756,23cyclictest0-21swapper/022:05:170
2768699711,46cyclictest3360-21df_inode00:00:232
2768799702,9cyclictest27688-21taskset19:10:073
2768699691,46cyclictest20796-21fschecks_count20:20:232
2768699681,45cyclictest6266-21apt22:40:052
2768499686,59cyclictest13-21rcuc/000:30:190
2768499673,29cyclictest0-21swapper/022:10:170
2768699661,48cyclictest29676-21apt20:45:182
2768699661,43cyclictest25431-21sed22:00:302
2768699651,49cyclictest29187-21chrt20:42:122
2768699651,43cyclictest32256-21ls22:20:222
2768799642,5cyclictest7850irq/56-dwc_otg_22:20:303
2768699641,46cyclictest1336-21apt-get23:55:162
2768699641,44cyclictest1056-21chrt23:54:512
2768699641,42cyclictest27926-21/usr/sbin/munin19:10:192
2768799631,13cyclictest7950irq/56-dwc_otg_23:30:323
2768699634,45cyclictest7850irq/56-dwc_otg_21:35:202
2768699631,40cyclictest29498-21/usr/sbin/munin19:15:342
2768599634,14cyclictest0-21swapper/121:50:161
2768599634,14cyclictest0-21swapper/121:50:161
2768499632,38cyclictest23986-21kworker/0:022:00:170
2768499631,14cyclictest0-21swapper/000:35:190
2768499631,14cyclictest0-21swapper/000:35:190
2768499628,3cyclictest0-21swapper/019:15:170
2768499626,39cyclictest0-21swapper/023:15:210
2768499626,21cyclictest0-21swapper/020:25:110
2768799615,4cyclictest7950irq/56-dwc_otg_20:45:273
2768499613,36cyclictest0-21swapper/020:21:140
2768799605,4cyclictest7950irq/56-dwc_otg_21:10:323
2768799605,19cyclictest7950irq/56-dwc_otg_23:20:183
2768799602,14cyclictest0-21swapper/300:25:323
2768799601,5cyclictest18481-21ls20:15:043
2768699601,42cyclictest6709-21df22:40:212
2768499603,32cyclictest14407-21unixbench_multi20:00:330
2768799595,4cyclictest17508-21cut23:10:203
2768799592,10cyclictest19135-21iwlist20:15:253
27687995916,4cyclictest7900-21/usr/sbin/munin19:45:273
27687995916,4cyclictest7900-21/usr/sbin/munin19:45:273
2768699591,45cyclictest25140-21/usr/sbin/munin23:30:242
2768699591,42cyclictest25496-21sshd20:31:152
2768699591,39cyclictest406-21sshd20:53:142
2768499591,41cyclictest7895-21grep00:10:330
27684995912,26cyclictest0-21swapper/022:20:240
2768699581,46cyclictest3600-21ntp_states22:30:292
2768699581,42cyclictest16186-21memory23:05:252
27684995823,19cyclictest0-21swapper/021:00:170
27684995818,8cyclictest0-21swapper/022:35:190
27684995812,11cyclictest0-21swapper/023:55:160
2768799573,8cyclictest7950irq/56-dwc_otg_21:55:203
2768699571,40cyclictest9486-21users22:45:332
2768499578,10cyclictest0-21swapper/021:55:150
2768499573,28cyclictest0-21swapper/019:25:160
2768499571,34cyclictest27926-21aten_rbpower_en20:40:170
2768499571,30cyclictest14107-21cut23:00:230
2768799561,7cyclictest21346-21/usr/sbin/munin20:20:313
27687995615,5cyclictest11428-21latency_hist21:25:053
2768699563,40cyclictest7950irq/56-dwc_otg_00:09:012
2768699561,51cyclictest1320-21ls19:25:262
2768699561,35cyclictest7950irq/56-dwc_otg_19:40:052
27684995629,5cyclictest12342-21fschecks_time22:55:240
2768499562,27cyclictest0-21swapper/000:15:160
2768499561,34cyclictest1125-21runrttasks21:15:230
2768499561,33cyclictest23187-21fschecks_count23:25:230
2768799553,6cyclictest3841-21ntp_states00:00:303
2768799551,6cyclictest0-21swapper/320:45:053
2768799551,5cyclictest810-21/usr/sbin/munin20:55:273
2768699551,41cyclictest6468-21df_inode21:10:212
2768499553,32cyclictest0-21swapper/019:50:150
2768499553,32cyclictest0-21swapper/019:50:150
27684995514,7cyclictest6054-21unixbench_multi00:05:330
27684995513,6cyclictest0-21swapper/023:30:160
2768799546,4cyclictest7950irq/56-dwc_otg_21:45:333
2768799541,44cyclictest0-21swapper/323:38:293
2768799541,3cyclictest7750irq/56-dwc_otg22:40:303
2768699543,39cyclictest7950irq/56-dwc_otg_21:00:192
2768699541,49cyclictest30186-21apt-get23:45:172
2768699541,49cyclictest30186-21apt-get23:45:172
2768699541,49cyclictest12177-21apt-get00:25:162
2768699541,43cyclictest27660-21sed23:35:332
2768699541,39cyclictest7950irq/56-dwc_otg_20:59:162
2768699541,39cyclictest7850irq/56-dwc_otg_20:15:042
27684995435,6cyclictest7950irq/56-dwc_otg_21:05:200
2768499542,29cyclictest16632-21grep21:35:340
2768499541,33cyclictest4701-21apt00:05:040
2768499541,32cyclictest6569-21aten_rbpower_en22:40:200
2768799538,6cyclictest0-21swapper/322:00:173
2768799532,3cyclictest0-21swapper/320:06:113
2768799531,4cyclictest7850irq/56-dwc_otg_23:55:243
2768699531,48cyclictest5486-21ls21:05:332
2768699531,47cyclictest15024-21ls00:30:352
2768699531,43cyclictest10073-21taskset00:19:352
27686995314,36cyclictest23954-21sed23:25:332
2768699531,42cyclictest10403-21ntp_states19:50:282
2768699531,42cyclictest10403-21ntp_states19:50:282
2768499532,29cyclictest26378-21fschecks_time20:35:230
2768499532,14cyclictest21502-21ls23:20:230
273392533,12sleep00-21swapper/019:09:080
273392533,12sleep00-21swapper/019:09:080
27687995213,4cyclictest32681-21/usr/sbin/munin23:50:273
27687995211,3cyclictest0-21swapper/320:30:233
2768699522,33cyclictest0-21swapper/200:11:392
2768699521,48cyclictest14031-21/usr/sbin/munin23:00:212
2768699521,39cyclictest8160-21fschecks_count19:45:232
2768699521,39cyclictest8160-21fschecks_count19:45:232
2768699521,36cyclictest0-21swapper/220:01:192
2768499521,29cyclictest6489-21fschecks_count21:10:220
27687995120,29cyclictest0-21swapper/300:10:163
2768799511,5cyclictest128-21systemd-journal21:51:313
2768799511,5cyclictest128-21systemd-journal21:51:313
2768799511,3cyclictest7950irq/56-dwc_otg_21:30:113
2768799511,11cyclictest7950irq/56-dwc_otg_22:18:023
2768699511,47cyclictest9865-21apt-get22:50:112
2768699511,47cyclictest27302-21sshd20:36:162
2768699511,45cyclictest19894-21/usr/sbin/munin23:15:292
2768699511,45cyclictest17409-21/usr/sbin/munin23:10:352
2768699511,39cyclictest0-21swapper/221:30:062
2768699511,39cyclictest0-21swapper/221:30:062
2768699511,34cyclictest7950irq/56-dwc_otg_20:15:242
2768699511,34cyclictest0-21swapper/222:15:302
2768699511,33cyclictest0-21swapper/221:40:532
27685995123,11cyclictest7750irq/56-dwc_otg20:50:261
27684995127,5cyclictest31507-21aten_rbpower_cu20:50:170
27684995124,10cyclictest0-21swapper/023:45:180
27684995124,10cyclictest0-21swapper/023:45:180
27684995122,5cyclictest12435-21fschecks_count00:25:220
27684995117,22cyclictest0-21swapper/021:48:580
2768499511,6cyclictest26650-21wget23:35:190
2768799502,5cyclictest7950irq/56-dwc_otg_19:34:243
2768799501,5cyclictest2993-21grep22:30:203
27687995012,3cyclictest7950irq/56-dwc_otg_00:22:493
27687995012,10cyclictest0-21swapper/321:05:053
2768799501,12cyclictest0-21swapper/322:54:043
2768699505,42cyclictest27328-21sort22:05:312
2768699501,46cyclictest15681-21/usr/sbin/munin20:05:272
2768699501,45cyclictest20814-21apt-get23:20:112
2768699501,45cyclictest16694-21awk00:35:312
2768699501,45cyclictest16694-21awk00:35:312
2768699501,36cyclictest7950irq/56-dwc_otg_23:41:322
2768699501,32cyclictest11117-21sshd21:21:152
2768599501,34cyclictest19323-21ls20:15:281
2768499505,5cyclictest7850irq/56-dwc_otg_00:20:190
2768499502,32cyclictest0-21swapper/023:50:140
2768499501,5cyclictest17041-21aten_rbpower_vo20:10:190
2768499501,32cyclictest6979-21taskset19:40:300
2768799494,6cyclictest0-21swapper/322:25:343
2768799491,4cyclictest7850irq/56-dwc_otg_20:26:203
2768799491,15cyclictest13878-21missed_timers20:00:263
2768699491,45cyclictest19089-21apt-get21:45:192
2768699491,45cyclictest12671-21/usr/sbin/munin22:55:282
2768699491,45cyclictest12421-21/usr/sbin/munin19:55:322
2768699491,44cyclictest11348-21ls00:20:332
2768799483,9cyclictest7950irq/56-dwc_otg_22:06:233
2768799482,5cyclictest0-21swapper/322:57:123
2768799481,5cyclictest466-21in:imuxsock19:16:003
2768799481,3cyclictest7950irq/56-dwc_otg_22:46:423
2768699482,34cyclictest7950irq/56-dwc_otg_19:20:582
27686994817,21cyclictest7950irq/56-dwc_otg_22:29:142
2768699481,44cyclictest22495-21sed21:55:052
2768699481,44cyclictest22495-21sed21:55:052
2768699481,44cyclictest15354-21date21:35:052
2768699481,35cyclictest22696-21apt-get21:55:152
27685994810,22cyclictest0-21swapper/119:10:071
2768499486,21cyclictest15355-21grep20:05:230
2768499482,26cyclictest0-21swapper/023:05:240
2768499482,21cyclictest20300-21munin-run20:20:050
27684994817,5cyclictest0-21swapper/019:55:180
2768499481,30cyclictest3999-21tail22:30:340
2768799477,25cyclictest0-21swapper/321:40:173
2768799473,5cyclictest7950irq/56-dwc_otg_19:40:083
2768799471,5cyclictest15701-21entropy21:35:223
2768799471,5cyclictest1-21systemd20:37:113
2768799471,18cyclictest17413-21ls00:40:063
2768799471,18cyclictest17413-21ls00:40:063
2768699471,43cyclictest2476-21sshd19:30:112
2768699471,43cyclictest22097-21apt-get20:25:102
2768699471,36cyclictest0-21swapper/222:12:132
27684994728,5cyclictest29684-21aten_rbpower_cu20:45:180
27684994725,6cyclictest7850irq/56-dwc_otg_23:10:250
2768499471,8cyclictest8026-21aten_rbpower_vo19:45:200
2768499471,8cyclictest8026-21aten_rbpower_vo19:45:200
2768499471,34cyclictest21180-21fschecks_time21:50:230
2768499471,34cyclictest21180-21fschecks_time21:50:230
2768799465,9cyclictest29296-21timerandwakeup23:40:323
2768799461,9cyclictest4499-21wget21:05:193
27687994615,4cyclictest1125-21runrttasks19:38:033
2768699461,42cyclictest9632-21latency_hist21:20:052
2768599461,18cyclictest7850irq/56-dwc_otg_23:00:241
2768799456,9cyclictest1125-21runrttasks23:08:203
2768799451,6cyclictest128-21systemd-journal00:17:163
2768799451,4cyclictest4954-21apt-get00:05:173
27687994513,5cyclictest1125-21runrttasks22:37:203
2768599457,24cyclictest0-21swapper/120:25:101
2768499452,33cyclictest0-21swapper/022:15:220
2768499451,28cyclictest0-21swapper/021:20:270
2768499451,24cyclictest24997-21proc_pri20:30:300
2768799441,4cyclictest11530-21aten_rbpower_cu19:55:193
2768699441,40cyclictest6427-21fschecks_time19:40:222
2768599447,23cyclictest0-21swapper/123:15:211
27685994410,20cyclictest25674-21/usr/sbin/munin22:00:321
2768499442,24cyclictest128-21systemd-journal21:00:050
27684994417,18cyclictest0-21swapper/021:44:510
27684994416,16cyclictest0-21swapper/019:21:170
27687994314,4cyclictest1125-21runrttasks00:32:113
2768599438,25cyclictest0-21swapper/122:40:191
27685994310,5cyclictest23274-21uniq23:25:241
2768499433,7cyclictest0-21swapper/022:25:170
2768599429,17cyclictest16145-21grep23:05:241
2768599428,14cyclictest8143-21forks19:45:231
2768599428,14cyclictest8143-21forks19:45:231
2768599427,25cyclictest0-21swapper/122:10:191
2768599427,23cyclictest0-21swapper/120:55:201
2768599427,23cyclictest0-21swapper/119:40:181
2768599427,21cyclictest0-21swapper/123:20:111
27685994211,12cyclictest9550irq/80-mmc021:25:051
27685994210,16cyclictest7750irq/56-dwc_otg20:45:211
2768799417,20cyclictest0-21swapper/321:15:193
2768599418,23cyclictest0-21swapper/123:40:201
2768599417,24cyclictest0-21swapper/119:55:191
2768599416,24cyclictest0-21swapper/120:20:201
2768599416,23cyclictest0-21swapper/121:30:101
27685994119,9cyclictest7950irq/56-dwc_otg_21:46:261
2768799407,21cyclictest0-21swapper/321:25:183
2768799407,21cyclictest0-21swapper/321:25:183
2768599405,13cyclictest6807-21ls21:10:251
27685994027,4cyclictest7750irq/56-dwc_otg21:35:331
2768599402,18cyclictest7750irq/56-dwc_otg19:30:181
27684994018,12cyclictest0-21swapper/019:35:280
2768599397,20cyclictest7750irq/56-dwc_otg23:49:471
2768599397,20cyclictest7750irq/56-dwc_otg23:49:471
27685993916,5cyclictest7750irq/56-dwc_otg22:20:211
27685993912,6cyclictest7850irq/56-dwc_otg_23:55:151
2768599386,23cyclictest0-21swapper/123:35:181
2768599386,17cyclictest7950irq/56-dwc_otg_19:15:341
2768599381,3cyclictest0-21swapper/122:35:171
2768599381,26cyclictest0-21swapper/119:35:201
2768599381,24cyclictest13635-21grep20:00:231
2768799371,9cyclictest0-21swapper/323:04:303
2768799371,10cyclictest0-21swapper/320:53:443
2768599376,14cyclictest22692-21sshd21:55:151
2768599371,24cyclictest12118-21ls21:25:231
2768599371,24cyclictest12118-21ls21:25:231
2768599371,21cyclictest10074-21apt-key22:50:061
2768799361,12cyclictest0-21swapper/323:15:383
2768599369,13cyclictest7950irq/56-dwc_otg_23:15:051
27685993617,8cyclictest7750irq/56-dwc_otg00:08:591
27685993615,6cyclictest7750irq/56-dwc_otg23:30:221
2768599361,3cyclictest7750irq/56-dwc_otg21:00:171
2768599361,25cyclictest15570-21ls00:35:051
27685993521,4cyclictest7750irq/56-dwc_otg00:25:211
27685993516,8cyclictest7850irq/56-dwc_otg_19:50:251
27685993516,8cyclictest7850irq/56-dwc_otg_19:50:251
2768799347,19cyclictest0-21swapper/322:10:193
2768799347,14cyclictest0-21swapper/319:50:183
2768799347,14cyclictest0-21swapper/319:50:183
2768599348,11cyclictest7850irq/56-dwc_otg_00:00:251
2768599341,9cyclictest10154-21latency_hist00:20:051
27685993413,10cyclictest0-21swapper/120:05:201
2768799331,12cyclictest0-21swapper/319:24:013
2768599337,14cyclictest0-21swapper/120:40:161
2768599336,16cyclictest0-21swapper/122:30:191
27685993318,4cyclictest7750irq/56-dwc_otg22:06:331
2768799321,12cyclictest0-21swapper/319:29:213
2768599326,14cyclictest0-21swapper/121:15:191
2768599323,7cyclictest18384-21grep21:40:321
2768599322,15cyclictest7950irq/56-dwc_otg_20:34:031
2768599321,3cyclictest7750irq/56-dwc_otg23:50:171
2768599321,23cyclictest16762-21timerwakeupswit00:35:321
2768599321,23cyclictest16762-21timerwakeupswit00:35:321
27685993211,7cyclictest7850irq/56-dwc_otg_22:17:351
273892324,19sleep30-21swapper/319:09:483
273892324,19sleep30-21swapper/319:09:483
2768599316,5cyclictest12358-21fschecks_time22:55:241
2768599311,7cyclictest11525-21grep00:20:351
27685993113,4cyclictest31440-21cut19:20:171
2768799306,5cyclictest7850irq/56-dwc_otg_23:50:063
2768799306,5cyclictest7850irq/56-dwc_otg_23:50:063
2768599307,13cyclictest0-21swapper/119:25:181
27685993013,4cyclictest7750irq/56-dwc_otg00:14:271
27685993011,6cyclictest221ktimersoftd/122:45:311
2768599296,8cyclictest7750irq/56-dwc_otg20:37:031
2768599296,13cyclictest0-21swapper/120:10:181
2768799282,19cyclictest0-21swapper/323:28:183
2768599274,5cyclictest7950irq/56-dwc_otg_22:25:241
271032263,14sleep20-21swapper/219:06:112
271032263,14sleep20-21swapper/219:06:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional