You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 00:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Aug 13, 2026 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2536922760,1sleep30-21swapper/307:05:293
264449914812,92cyclictest8430-21ssh11:50:250
264449912820,69cyclictest10314-21wc09:15:290
26445991147,84cyclictest7750irq/56-dwc_otg10:50:291
26446991094,62cyclictest24597-21ssh11:30:252
26444991095,79cyclictest24001-21ntp_states10:50:270
26444991095,77cyclictest0-21swapper/010:35:300
26445991083,77cyclictest7950irq/56-dwc_otg_11:10:241
26444991065,85cyclictest0-21swapper/010:15:180
26444991065,85cyclictest0-21swapper/010:15:180
26445991054,9cyclictest7850irq/56-dwc_otg_11:50:221
26446991043,65cyclictest11827-21fschecks_count11:15:222
264469910311,75cyclictest7950irq/56-dwc_otg_11:10:222
264449910353,24cyclictest24500-21/usr/sbin/munin12:10:220
26444991033,76cyclictest9-21ksoftirqd/011:15:200
26446991023,92cyclictest28342-21ssh11:35:202
26446991023,61cyclictest4899-21diskmemload12:10:212
26445991027,70cyclictest7950irq/56-dwc_otg_10:15:301
26445991027,70cyclictest7950irq/56-dwc_otg_10:15:301
26444991005,44cyclictest0-21swapper/011:30:250
291922990,2sleep00-21swapper/010:56:240
2644499993,45cyclictest0-21swapper/010:00:310
2644599983,10cyclictest24765-21df_inode12:10:221
2644699973,66cyclictest6997-21diskstats10:30:222
2644599973,11cyclictest0-21swapper/110:30:221
2644699963,71cyclictest10066-21ssh11:51:562
26444999658,10cyclictest7850irq/56-dwc_otg_11:05:320
26444999649,15cyclictest0-21swapper/007:35:170
26445999512,62cyclictest7850irq/56-dwc_otg_09:15:281
2644599933,69cyclictest7950irq/56-dwc_otg_10:35:291
26444999354,25cyclictest7850irq/56-dwc_otg_10:30:210
26445999213,72cyclictest5120-21unixbench_multi11:45:311
26444999269,8cyclictest7950irq/56-dwc_otg_09:35:290
26447999131,39cyclictest15181-21ssh10:00:303
2644699913,61cyclictest17163-21ps12:00:312
2644599914,16cyclictest26932-21ntp_states09:35:301
26445999113,72cyclictest27815-21idleruntime10:55:231
2644699903,59cyclictest10393-21sensors09:15:312
2644599904,17cyclictest24697-21taskset11:30:271
2644599904,13cyclictest0-21swapper/110:00:331
26445999019,53cyclictest0-21swapper/111:00:281
26445999012,12cyclictest4301-21switchtime11:05:311
2644499893,53cyclictest0-21swapper/011:35:200
128332890,2sleep30-21swapper/308:00:263
26446998712,55cyclictest7950irq/56-dwc_otg_10:55:122
2644699863,60cyclictest274722sh10:15:282
2644699863,60cyclictest274722sh10:15:282
2644699862,75cyclictest15215-21/usr/sbin/munin10:00:312
2644599862,9cyclictest7950irq/56-dwc_otg_11:15:191
26445998612,10cyclictest22764-21ntp_kernel_pll_09:30:271
2644499869,44cyclictest17161-21uname12:00:310
26446998515,41cyclictest0-21swapper/210:50:302
2644599854,13cyclictest28350-21diskstats11:35:211
2644599842,9cyclictest17173-21/usr/sbin/munin12:00:311
26444998440,14cyclictest7950irq/56-dwc_otg_12:15:260
52202830,4sleep00-21swapper/011:45:320
2644699833,72cyclictest4223-21/usr/sbin/munin11:05:292
2644599834,8cyclictest0-21swapper/112:15:281
2644499832,45cyclictest7817-21taskset11:10:220
2644699823,72cyclictest5214-21/usr/sbin/munin11:45:312
2644499822,9cyclictest32283-21load11:00:270
311892810,6sleep12644599cyclictest09:40:321
26447998114,40cyclictest381ktimersoftd/311:15:203
2644499814,31cyclictest0-21swapper/009:20:150
2644699804,70cyclictest11831-21switchtime10:35:302
2644699803,68cyclictest32286-21ssh11:00:272
2644499802,8cyclictest7950irq/56-dwc_otg_09:30:270
2644699792,71cyclictest27471-21ssh09:35:372
2644699792,70cyclictest22770-21ntpq09:30:272
168292790,4sleep32644799cyclictest12:00:253
6092780,2sleep3381ktimersoftd/311:00:463
26444997731,32cyclictest0-21swapper/007:20:100
2644799753,12cyclictest11777-21/usr/sbin/munin10:35:293
26446997513,55cyclictest29440-21cat12:15:282
2644499746,43cyclictest0-21swapper/007:10:160
260052740,2sleep30-21swapper/308:35:323
26444997227,6cyclictest60-21kswapd009:50:140
2644699712,46cyclictest0-21swapper/207:18:202
2644599717,13cyclictest17776-21ssh09:25:171
2644799706,28cyclictest4281-21cat11:05:313
26444996922,12cyclictest0-21swapper/007:25:170
26445996818,37cyclictest3375-21ssh10:25:271
26444996830,6cyclictest7950irq/56-dwc_otg_11:55:150
2644499682,55cyclictest19237-21latency_hist08:20:040
26447996729,17cyclictest7850irq/56-dwc_otg_09:15:283
2644699671,57cyclictest15817-21sshd11:59:532
26445996610,23cyclictest0-21swapper/107:30:181
56272650,1sleep30-21swapper/309:49:523
56272650,1sleep30-21swapper/309:49:523
2644799658,23cyclictest0-21swapper/307:35:173
2644699651,5cyclictest14777-21ls09:20:312
2644699651,54cyclictest12416-21/usr/sbin/munin08:00:212
26444996514,11cyclictest0-21swapper/008:55:150
245582650,4sleep324559-21sh12:10:193
26447996426,23cyclictest8520-21if_eth011:50:263
2644699641,55cyclictest0-21swapper/208:35:142
2644799635,43cyclictest0-21swapper/309:35:383
2644699634,46cyclictest7950irq/56-dwc_otg_10:06:312
2644699631,48cyclictest0-21swapper/209:50:252
26446996314,36cyclictest2577-21/usr/sbin/munin10:25:222
26447996233,8cyclictest381ktimersoftd/310:30:213
2644699625,42cyclictest7950irq/56-dwc_otg_11:44:022
2644699625,42cyclictest7950irq/56-dwc_otg_11:44:022
2644699611,6cyclictest17732-21/usr/sbin/munin09:25:182
26445996122,5cyclictest23-21ksoftirqd/110:45:151
26445996122,5cyclictest23-21ksoftirqd/110:45:151
26444996134,19cyclictest0-21swapper/009:45:190
26444996134,19cyclictest0-21swapper/009:45:190
2644499611,32cyclictest12553-21modprobe12:35:040
2644499611,32cyclictest12553-21modprobe12:35:040
26447996010,24cyclictest0-21swapper/310:15:303
26447996010,24cyclictest0-21swapper/310:15:303
2644699602,51cyclictest7850irq/56-dwc_otg_12:39:022
26446996010,34cyclictest0-21swapper/212:25:112
26444996017,22cyclictest20218-21/usr/sbin/munin10:45:320
26444996017,22cyclictest20218-21/usr/sbin/munin10:45:320
2644499601,31cyclictest12279-21grep08:00:190
2644499601,16cyclictest8842-21tune2fs07:50:210
2644699595,44cyclictest4875-21cron09:10:042
2644699594,42cyclictest7950irq/56-dwc_otg_07:26:082
2644699591,51cyclictest2640-21ssh12:22:072
2644699591,50cyclictest18353-21ssh11:23:022
2644699591,4cyclictest0-21swapper/207:40:222
2644699591,47cyclictest21505-21cat12:05:322
2644699591,47cyclictest0-21swapper/209:45:222
2644699591,47cyclictest0-21swapper/209:45:222
2644699591,42cyclictest0-21swapper/209:58:472
2644599594,5cyclictest5591-21tune2fs09:10:211
2644599593,5cyclictest7850irq/56-dwc_otg_09:50:281
2644599591,5cyclictest7750irq/56-dwc_otg07:15:311
2644599591,4cyclictest14206-21/usr/sbin/munin08:05:181
2644599591,4cyclictest14206-21/usr/sbin/munin08:05:181
2644799585,22cyclictest7950irq/56-dwc_otg_11:35:193
2644699585,38cyclictest14110-21aten_rbpower_vo08:05:172
2644699585,38cyclictest14110-21aten_rbpower_vo08:05:172
2644699583,43cyclictest7950irq/56-dwc_otg_09:45:042
26446995810,40cyclictest7894-21sh09:13:012
2644599588,22cyclictest60-21kswapd007:20:101
2644599586,9cyclictest7750irq/56-dwc_otg10:05:281
2644599582,5cyclictest7750irq/56-dwc_otg09:10:041
26445995817,5cyclictest22893-21sh10:10:211
2644499582,40cyclictest0-21swapper/009:55:320
26444995820,19cyclictest0-21swapper/007:45:150
26444995811,26cyclictest0-21swapper/012:05:250
2644799573,33cyclictest37-21rcuc/311:10:223
26447995727,10cyclictest246142sleep311:30:253
26447995726,9cyclictest7850irq/56-dwc_otg_11:45:313
2644699577,43cyclictest222742chrt10:48:572
2644699577,43cyclictest222742chrt10:48:572
2644699571,47cyclictest0-21swapper/208:00:052
2644699571,37cyclictest520-21rs:main2
26446995711,42cyclictest1815-21if_wlan007:30:242
2644599571,5cyclictest7950irq/56-dwc_otg_09:45:261
2644599571,5cyclictest7950irq/56-dwc_otg_09:45:261
2644599571,4cyclictest7950irq/56-dwc_otg_08:35:171
2644499578,14cyclictest0-21swapper/010:25:170
2644499571,5cyclictest29369-21unixbench_multi07:15:310
2644699563,41cyclictest0-21swapper/210:12:012
2644599563,4cyclictest7750irq/56-dwc_otg11:20:261
2644599562,45cyclictest0-21swapper/108:59:151
26444995632,4cyclictest7750irq/56-dwc_otg09:40:310
2644699556,36cyclictest7950irq/56-dwc_otg_10:20:072
2644699551,46cyclictest0-21swapper/210:43:442
2644599554,6cyclictest7950irq/56-dwc_otg_12:30:311
2644599554,6cyclictest7950irq/56-dwc_otg_12:30:311
2644599553,16cyclictest7750irq/56-dwc_otg08:45:141
2644599553,16cyclictest7750irq/56-dwc_otg08:45:141
2644599552,5cyclictest0-21swapper/111:56:461
2644599551,3cyclictest0-21swapper/110:20:241
26445995512,6cyclictest1180-21runrttasks07:55:051
2644599551,15cyclictest0-21swapper/107:47:021
2644499552,35cyclictest25925-21unixbench_multi08:35:310
2644499551,28cyclictest0-21swapper/008:25:150
2644799543,12cyclictest0-21swapper/310:55:093
2644699545,38cyclictest26710-21aten_rbpower_po08:40:172
2644699542,43cyclictest0-21swapper/212:35:012
2644699542,43cyclictest0-21swapper/212:35:012
2644699542,3cyclictest0-21swapper/207:54:482
2644699542,38cyclictest0-21swapper/207:21:102
26445995421,31cyclictest558-21/usr/sbin/munin11:40:281
26445995421,31cyclictest558-21/usr/sbin/munin11:40:281
2644599541,3cyclictest7950irq/56-dwc_otg_08:00:261
26444995424,14cyclictest7750irq/56-dwc_otg09:10:210
2644499542,26cyclictest2525-21unixbench_singl07:30:330
2644499541,40cyclictest0-21swapper/011:25:240
2644699533,4cyclictest31355-21timerwakeupswit08:50:292
2644699531,4cyclictest30263-21sed08:50:052
2644699531,42cyclictest0-21swapper/208:10:262
2644599538,6cyclictest9550irq/80-mmc012:40:041
2644599535,9cyclictest7950irq/56-dwc_otg_08:00:041
2644599532,4cyclictest7850irq/56-dwc_otg_08:40:201
2644599531,3cyclictest0-21swapper/108:25:191
2644499536,35cyclictest9-21ksoftirqd/010:06:160
2644499531,40cyclictest23631-21rm10:10:320
2644499531,33cyclictest23138-21aten_rbpower_vo08:30:190
2644599527,3cyclictest0-21swapper/111:27:581
2644599522,4cyclictest7750irq/56-dwc_otg09:20:201
2644599521,4cyclictest20610-21df_inode12:05:201
2644599521,15cyclictest0-21swapper/112:26:441
26444995224,4cyclictest0-21swapper/011:20:150
2644499522,33cyclictest30603-21fschecks_count08:50:190
2644499521,30cyclictest15026-21ssh10:40:180
2644799513,11cyclictest0-21swapper/312:15:283
2644699514,35cyclictest7850irq/56-dwc_otg_08:21:222
2644699512,37cyclictest0-21swapper/208:32:102
2644699511,5cyclictest4832-21sed07:40:042
2644699511,37cyclictest0-21swapper/207:10:362
2644699511,36cyclictest0-21swapper/208:25:192
2644699511,2cyclictest0-21swapper/208:15:152
2644599512,4cyclictest0-21swapper/109:55:301
2644599511,7cyclictest1443-21cat12:20:301
2644599511,3cyclictest7750irq/56-dwc_otg08:30:291
2644499517,26cyclictest0-21swapper/007:40:260
2644699502,44cyclictest0-21swapper/208:45:122
2644699501,40cyclictest1181-21seq08:59:122
2644699501,3cyclictest0-21swapper/209:00:212
2644599501,6cyclictest0-21swapper/110:43:151
2644599501,15cyclictest1180-21runrttasks09:02:181
26444995020,14cyclictest0-21swapper/009:10:050
2644499501,31cyclictest0-21swapper/009:25:250
2644699492,35cyclictest0-21swapper/207:49:492
2644499491,36cyclictest0-21swapper/008:45:270
2644499491,36cyclictest0-21swapper/008:45:270
2644499491,26cyclictest12857-21/usr/sbin/munin12:35:300
2644799488,11cyclictest39-21ksoftirqd/309:30:263
2644499484,32cyclictest101ktimersoftd/011:41:280
2644499484,32cyclictest101ktimersoftd/011:41:280
2644499481,6cyclictest4269-21munin-run12:25:040
2644799479,21cyclictest7850irq/56-dwc_otg_09:20:153
2644799467,25cyclictest0-21swapper/309:00:183
2644799467,18cyclictest381ktimersoftd/310:50:283
2644599468,25cyclictest0-21swapper/108:50:101
26444994621,4cyclictest16400-21grep08:10:250
26444994619,16cyclictest0-21swapper/010:24:220
2644499461,8cyclictest4927-21grep12:25:210
259582462,36sleep00-21swapper/007:07:180
2644499441,14cyclictest0-21swapper/008:20:330
173712440,2sleep3140-21systemd-journal08:15:033
2644599432,7cyclictest0-21swapper/107:29:031
26444994317,7cyclictest140-21systemd-journal09:00:050
2644499431,27cyclictest0-21swapper/008:40:230
26447994225,10cyclictest2000-21ssh12:20:583
2644499423,20cyclictest9-21ksoftirqd/008:05:210
2644499423,20cyclictest9-21ksoftirqd/008:05:210
2644799417,24cyclictest0-21swapper/308:45:153
2644799417,24cyclictest0-21swapper/308:45:153
2644599412,3cyclictest7750irq/56-dwc_otg07:10:161
2644499411,24cyclictest520-21rs:main0
253352418,16sleep20-21swapper/207:05:292
2644799408,3cyclictest0-21swapper/309:14:173
2644799407,22cyclictest12901-21ssh12:35:183
2644599404,18cyclictest7950irq/56-dwc_otg_08:20:181
26447993910,16cyclictest7950irq/56-dwc_otg_08:20:043
2644799387,20cyclictest7950irq/56-dwc_otg_12:30:163
2644799387,20cyclictest7950irq/56-dwc_otg_12:30:163
2644799383,26cyclictest7950irq/56-dwc_otg_10:45:153
2644799383,26cyclictest7950irq/56-dwc_otg_10:45:153
2644799381,27cyclictest32472-21cut08:55:213
2644599371,15cyclictest0-21swapper/108:13:421
2644799365,19cyclictest15701-21ssh11:20:173
2644799364,24cyclictest7950irq/56-dwc_otg_11:40:163
2644799364,24cyclictest7950irq/56-dwc_otg_11:40:163
26447993623,6cyclictest25667-21ssh10:13:423
2644799361,6cyclictest10153-21sh07:55:043
26447993615,5cyclictest2624-21aten_rbpower_cu10:25:183
2644799361,2cyclictest0-21swapper/309:29:323
2644799352,26cyclictest0-21swapper/311:27:483
2644799347,11cyclictest7950irq/56-dwc_otg_10:40:173
2644799341,4cyclictest7950irq/56-dwc_otg_08:25:163
2644799337,6cyclictest7950irq/56-dwc_otg_09:55:223
2644799337,14cyclictest0-21swapper/308:50:103
26447993312,4cyclictest1180-21runrttasks12:28:563
2644799327,5cyclictest7950irq/56-dwc_otg_10:05:333
2644799322,20cyclictest0-21swapper/310:20:283
2644799321,5cyclictest29328-21unixbench-2d07:15:313
26447993212,5cyclictest1541-21/usr/sbin/munin07:30:203
2644799321,18cyclictest30470-21grep07:20:213
2644599323,5cyclictest7750irq/56-dwc_otg07:35:311
2644799317,6cyclictest21466-21grep12:05:313
2644799317,14cyclictest0-21swapper/309:05:173
2644799317,14cyclictest0-21swapper/308:40:173
2644799317,14cyclictest0-21swapper/307:45:173
2644799312,13cyclictest7950irq/56-dwc_otg_08:05:213
2644799312,13cyclictest7950irq/56-dwc_otg_08:05:213
26447993114,7cyclictest23088-21aten_rbpower_en08:30:183
2644599313,5cyclictest0-21swapper/108:15:321
2644599311,19cyclictest6546-21latency_hist07:45:051
2644799302,6cyclictest7850irq/56-dwc_otg_11:55:153
26447993023,5cyclictest7786-21ssh09:51:143
2644799298,4cyclictest0-21swapper/309:41:583
2644799292,14cyclictest0-21swapper/307:58:003
2644799291,5cyclictest26791-21grep07:10:203
258342293,16sleep10-21swapper/107:05:411
2644799288,6cyclictest5653-21tr07:40:273
26447992815,5cyclictest0-21swapper/307:25:163
2644799276,13cyclictest19584-21ls08:20:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional