You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 21:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon May 11, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20506998921,34cyclictest0-21swapper/017:01:090
254282850,5sleep12050799cyclictest19:41:081
121162850,5sleep02050699cyclictest21:21:240
24252830,6sleep02050699cyclictest18:34:240
20506998314,50cyclictest9-21ksoftirqd/016:46:080
303232820,2sleep10-21swapper/120:26:091
182152750,2sleep10-21swapper/120:50:451
265912730,2sleep20-21swapper/218:16:152
253302730,2sleep30-21swapper/321:37:093
131252720,1sleep30-21swapper/320:05:303
20506996740,17cyclictest0-21swapper/021:31:190
20506996534,6cyclictest7850irq/56-dwc_otg_18:46:270
2050699642,40cyclictest21899-21unixbench_multi18:01:250
190492630,2sleep30-21swapper/316:26:173
20506996115,34cyclictest4327-21munin-run17:15:570
2050699604,52cyclictest101ktimersoftd/016:51:250
2050699602,36cyclictest0-21swapper/020:30:580
2050699602,30cyclictest0-21swapper/019:21:090
20506995933,6cyclictest7850irq/56-dwc_otg_20:50:570
20506995831,5cyclictest7750irq/56-dwc_otg19:00:570
2050699581,33cyclictest25279-21tail16:41:260
2050799574,9cyclictest0-21swapper/120:16:111
20506995726,7cyclictest6902-21sh21:15:580
20506995726,6cyclictest24863-21unixbench_multi21:36:270
167082560,12sleep00-21swapper/019:30:490
2050799555,7cyclictest0-21swapper/121:46:101
20506995536,3cyclictest0-21swapper/019:45:580
2050699551,29cyclictest19232-21grep20:11:270
2050799544,7cyclictest7850irq/56-dwc_otg_18:11:071
2050799544,5cyclictest7750irq/56-dwc_otg19:51:111
2050699546,23cyclictest0-21swapper/019:36:100
2050699546,23cyclictest0-21swapper/019:36:100
20506995415,7cyclictest908-21runrttasks19:46:270
2050699531,30cyclictest7838-21aten_rbpower_en21:56:120
2050699531,28cyclictest0-21swapper/020:09:100
2050999525,6cyclictest7950irq/56-dwc_otg_20:56:113
2050899525,8cyclictest0-21swapper/216:46:082
20507995217,13cyclictest7950irq/56-dwc_otg_19:15:571
2050699529,6cyclictest0-21swapper/019:16:110
20506995219,21cyclictest0-21swapper/019:31:250
20506995127,4cyclictest6454-21aten_rbpower_po17:21:120
2050699512,29cyclictest0-21swapper/021:07:200
2050699512,24cyclictest0-21swapper/021:00:580
20506995116,22cyclictest7850irq/56-dwc_otg_16:40:570
2050699511,28cyclictest15867-21uname17:46:170
2050699511,21cyclictest0-21swapper/020:41:110
2050999501,14cyclictest0-21swapper/319:36:083
2050999501,14cyclictest0-21swapper/319:36:083
2050899501,8cyclictest0-21swapper/216:36:082
2050799507,29cyclictest0-21swapper/117:06:111
2050799507,29cyclictest0-21swapper/116:46:101
2050799506,27cyclictest7950irq/56-dwc_otg_16:55:581
20507995010,8cyclictest7850irq/56-dwc_otg_18:26:101
2050699506,25cyclictest14738-21grep17:41:250
2050699504,22cyclictest0-21swapper/016:56:130
20506995036,6cyclictest1876-21grep19:11:270
20506995028,5cyclictest7750irq/56-dwc_otg20:21:140
20506995028,5cyclictest7750irq/56-dwc_otg20:21:140
2050699502,20cyclictest4216-21ssh18:36:170
2050699502,18cyclictest0-21swapper/018:51:420
20506995021,7cyclictest27135-21cut21:01:110
2050699501,31cyclictest0-21swapper/020:36:030
2050699501,31cyclictest0-21swapper/020:36:030
2050999497,6cyclictest7950irq/56-dwc_otg_17:01:093
2050999492,9cyclictest28861-21ssh19:06:073
2050999492,9cyclictest28861-21ssh19:06:073
20506994933,3cyclictest0-21swapper/020:53:350
2050699492,25cyclictest0-21swapper/017:06:190
20506994920,5cyclictest11114-21unixbench_singl17:31:260
2050699491,33cyclictest7414-21uniq21:16:150
2050699491,33cyclictest7414-21uniq21:16:150
2050699491,28cyclictest9810-21ssh20:01:120
2050699491,27cyclictest24632-21aten_rbpower_vo18:11:100
20509994811,7cyclictest7750irq/56-dwc_otg20:16:113
20506994827,4cyclictest5490-21aten_rbpower_cu19:56:110
2050699482,34cyclictest30567-1kworker/u9:121:51:170
2050699482,34cyclictest30567-1kworker/u9:121:51:170
2050799478,10cyclictest7969-21ssh18:41:101
2050799471,10cyclictest0-21swapper/119:36:081
2050799471,10cyclictest0-21swapper/119:36:081
2050699473,33cyclictest29226-21unixbench_multi18:21:250
2050699472,8cyclictest0-21swapper/019:51:090
2050699472,26cyclictest15384-21runrttasks21:26:110
20506994717,16cyclictest23301-21taskset20:16:270
20506994714,13cyclictest7850irq/56-dwc_otg_17:51:120
2050699471,26cyclictest9175-21grep18:41:250
2050999466,24cyclictest0-21swapper/320:11:033
2050999466,24cyclictest0-21swapper/316:46:103
2050999463,4cyclictest7950irq/56-dwc_otg_19:11:113
2050999461,4cyclictest7950irq/56-dwc_otg_17:06:093
2050899461,25cyclictest23278-21latency_hist21:35:582
2050799462,7cyclictest7750irq/56-dwc_otg20:56:111
20507994610,9cyclictest7750irq/56-dwc_otg21:11:091
2050699467,12cyclictest7850irq/56-dwc_otg_18:06:090
20506994621,15cyclictest0-21swapper/021:45:510
2050699461,25cyclictest5650-21grep17:16:260
186732463,34sleep00-21swapper/016:26:020
20509994525,4cyclictest7750irq/56-dwc_otg21:00:593
2050799458,8cyclictest0-21swapper/118:56:101
2050799458,23cyclictest7950irq/56-dwc_otg_21:01:211
20507994529,6cyclictest7750irq/56-dwc_otg16:35:571
2050799452,4cyclictest7750irq/56-dwc_otg19:06:071
2050799452,4cyclictest7750irq/56-dwc_otg19:06:071
2050699457,26cyclictest0-21swapper/018:26:110
20506994535,4cyclictest7950irq/56-dwc_otg_21:46:260
2050699452,5cyclictest3197-21grep20:31:260
20506994522,6cyclictest7850irq/56-dwc_otg_18:16:100
2050899447,25cyclictest0-21swapper/219:16:122
20507994411,8cyclictest7950irq/56-dwc_otg_21:21:121
2050699442,29cyclictest0-21swapper/019:09:570
2050699442,29cyclictest0-21swapper/019:09:570
2050699441,39cyclictest25397-21ls19:01:180
237752430,1sleep20-21swapper/220:17:092
2050999432,9cyclictest7950irq/56-dwc_otg_18:11:073
20509994323,6cyclictest7750irq/56-dwc_otg19:32:423
2050899432,9cyclictest7850irq/56-dwc_otg_17:56:102
2050799431,14cyclictest23-21ksoftirqd/119:56:111
20507994311,16cyclictest12680-21basename19:25:571
20507994310,25cyclictest0-21swapper/116:36:101
2050999423,7cyclictest7750irq/56-dwc_otg21:46:103
2050799429,18cyclictest7850irq/56-dwc_otg_18:37:111
2050699422,25cyclictest0-21swapper/017:39:490
2050699422,25cyclictest0-21swapper/017:39:490
212492410,2sleep30-21swapper/320:53:333
2050999416,25cyclictest0-21swapper/316:36:103
2050999411,21cyclictest0-21swapper/321:56:273
2050899417,17cyclictest7850irq/56-dwc_otg_17:11:172
20508994118,5cyclictest14843-21fschecks_time20:46:152
2050799419,24cyclictest0-21swapper/119:34:381
2050799418,4cyclictest11378-21chrt17:34:121
2050799417,30cyclictest0-21swapper/117:01:111
2050799416,25cyclictest16490-21ssh18:51:101
2050799415,24cyclictest18705-21ssh20:51:041
2050799414,25cyclictest7950irq/56-dwc_otg_20:07:051
2050799413,9cyclictest7750irq/56-dwc_otg19:16:141
2050799411,5cyclictest7850irq/56-dwc_otg_16:45:571
20507994112,14cyclictest4592-21aten_rbpower_cu17:16:111
20507994111,16cyclictest221ktimersoftd/117:00:571
2050699412,21cyclictest0-21swapper/017:26:150
2050999407,25cyclictest0-21swapper/321:21:113
20509994028,9cyclictest7950irq/56-dwc_otg_18:36:083
20509994019,10cyclictest29560-21cat16:55:583
20509994016,10cyclictest7950irq/56-dwc_otg_21:29:453
2050799405,16cyclictest7950irq/56-dwc_otg_18:46:121
2050799403,21cyclictest16693-21ssh21:26:311
2050799402,12cyclictest10151-21latency_hist20:40:581
2050799402,12cyclictest10151-21latency_hist20:40:581
20507994016,8cyclictest7850irq/56-dwc_otg_18:31:221
20507994013,25cyclictest10400-21ssh20:41:111
20506994012,20cyclictest0-21swapper/016:35:410
2050999397,24cyclictest0-21swapper/317:11:023
2050999397,24cyclictest0-21swapper/316:56:033
2050999392,12cyclictest0-21swapper/319:51:113
20509993917,8cyclictest7950irq/56-dwc_otg_18:56:143
20509993914,12cyclictest7950irq/56-dwc_otg_17:34:363
2050999391,15cyclictest7950irq/56-dwc_otg_17:26:093
2050799397,24cyclictest0-21swapper/117:21:111
2050799397,12cyclictest0-21swapper/119:04:371
2050799391,19cyclictest221ktimersoftd/117:55:581
20507993910,6cyclictest221ktimersoftd/120:21:231
20507993910,6cyclictest221ktimersoftd/120:21:231
192152390,4sleep1221ktimersoftd/116:26:191
187022396,23sleep218757-21/usr/sbin/munin16:26:122
2050999388,8cyclictest5230-21if_wlan019:16:173
2050999387,25cyclictest0-21swapper/316:31:113
2050999387,21cyclictest0-21swapper/317:36:093
2050999387,21cyclictest0-21swapper/317:36:093
2050999387,16cyclictest37-21rcuc/320:22:563
2050999387,16cyclictest37-21rcuc/320:22:563
2050999385,16cyclictest15987-21ssh18:50:423
20509993824,4cyclictest7750irq/56-dwc_otg19:41:093
2050999382,3cyclictest7950irq/56-dwc_otg_20:46:083
20509993822,6cyclictest7750irq/56-dwc_otg18:31:193
2050999382,21cyclictest13828-21uniq17:41:143
20509993816,4cyclictest6421-21switchtime19:56:233
20509993813,10cyclictest7750irq/56-dwc_otg18:51:033
2050799389,18cyclictest7950irq/56-dwc_otg_21:41:091
2050799387,23cyclictest0-21swapper/117:41:101
2050799386,17cyclictest5614-21ssh20:35:391
20507993815,15cyclictest7750irq/56-dwc_otg21:52:241
20507993815,15cyclictest7750irq/56-dwc_otg21:52:241
2050699382,20cyclictest0-21swapper/017:56:210
28532370,2sleep30-21swapper/321:11:043
2050999375,20cyclictest6164-21ssh20:36:043
2050999375,20cyclictest6164-21ssh20:36:043
2050999373,6cyclictest0-21swapper/318:26:093
20509993719,5cyclictest908-21runrttasks18:21:093
20509993714,8cyclictest7950irq/56-dwc_otg_19:46:533
20509993713,15cyclictest381ktimersoftd/320:29:553
2050999371,20cyclictest7950irq/56-dwc_otg_21:41:083
2050899371,28cyclictest9275-21ls20:00:572
2050899371,22cyclictest31613-21ls18:30:582
2050799379,12cyclictest7750irq/56-dwc_otg18:01:081
2050799376,22cyclictest0-21swapper/117:46:111
20507993714,11cyclictest7950irq/56-dwc_otg_21:06:401
2050999361,19cyclictest7950irq/56-dwc_otg_20:44:323
2050999360,10cyclictest5630-21ssh20:35:413
20508993615,6cyclictest16674-21sh19:30:442
2050799368,12cyclictest7950irq/56-dwc_otg_17:56:081
2050799367,24cyclictest0-21swapper/119:26:111
2050799364,5cyclictest3597-21switchtime17:11:231
2050799364,5cyclictest0-21swapper/121:18:461
2050799364,5cyclictest0-21swapper/121:18:461
2050799362,21cyclictest0-21swapper/118:21:081
20507993615,9cyclictest9877-21cat17:30:581
2050799361,20cyclictest7850irq/56-dwc_otg_21:32:361
20507993611,11cyclictest7750irq/56-dwc_otg17:39:341
20507993611,11cyclictest7750irq/56-dwc_otg17:39:341
2050999353,14cyclictest6821-21ssh21:54:543
2050999353,14cyclictest6821-21ssh21:54:543
2050999352,19cyclictest9403-21ssh21:18:203
2050999352,19cyclictest9403-21ssh21:18:203
2050999352,13cyclictest7950irq/56-dwc_otg_17:20:573
20509993514,7cyclictest24547-21/usr/sbin/munin16:41:173
20509993513,9cyclictest7950irq/56-dwc_otg_18:01:103
20509993513,11cyclictest9930-21timerandwakeup19:21:253
2050999351,18cyclictest0-21swapper/318:42:193
20509993511,12cyclictest7950irq/56-dwc_otg_20:09:533
20509993510,9cyclictest7950irq/56-dwc_otg_19:01:513
20508993514,5cyclictest11907-21sh19:24:362
20508993514,4cyclictest10345-21sh20:41:072
2050899351,23cyclictest21723-21timerandwakeup18:01:242
2050899351,22cyclictest0-21swapper/220:28:502
20508993510,3cyclictest0-21swapper/219:11:182
2050799357,5cyclictest7750irq/56-dwc_otg19:46:151
20507993520,4cyclictest7950irq/56-dwc_otg_20:11:031
2050999349,14cyclictest7950irq/56-dwc_otg_21:35:293
2050999348,11cyclictest7950irq/56-dwc_otg_17:21:203
2050999345,14cyclictest7950irq/56-dwc_otg_21:06:233
20509993414,13cyclictest167782chrt19:30:563
2050999341,3cyclictest0-21swapper/317:56:103
2050899346,7cyclictest7950irq/56-dwc_otg_20:05:582
2050899344,6cyclictest7850irq/56-dwc_otg_21:06:262
2050899342,6cyclictest16531-21unixbench_singl17:46:252
2050899341,24cyclictest16290-21ls18:50:582
2050899341,23cyclictest0-21swapper/219:31:092
2050799347,23cyclictest0-21swapper/118:06:111
20507993411,8cyclictest221ktimersoftd/120:03:531
2050999336,15cyclictest0-21swapper/317:46:113
2050899339,5cyclictest0-21swapper/219:36:072
2050899339,5cyclictest0-21swapper/219:36:072
2050899336,5cyclictest7750irq/56-dwc_otg20:21:232
2050899336,5cyclictest7750irq/56-dwc_otg20:21:232
2050899332,4cyclictest1120250ssh21:21:112
2050899332,3cyclictest0-21swapper/221:01:192
2050899331,6cyclictest7100-21cut21:16:112
2050899331,6cyclictest7100-21cut21:16:112
20508993314,5cyclictest20726-21cut18:56:112
2050899331,23cyclictest9736-21latency_hist17:30:582
2050899331,21cyclictest26021-21latency19:41:182
2050899331,18cyclictest0-21swapper/220:37:202
2050899331,18cyclictest0-21swapper/220:37:202
2050799332,6cyclictest9005-21grep21:56:271
2050899326,6cyclictest0-21swapper/221:30:352
2050899326,12cyclictest7950irq/56-dwc_otg_18:06:252
2050899324,8cyclictest0-21swapper/220:10:582
2050899324,23cyclictest0-21swapper/220:11:032
2050899322,21cyclictest0-21swapper/221:59:302
20508993215,4cyclictest6851-21latency17:21:172
2050899321,4cyclictest5378-21ssh21:13:312
20508993212,4cyclictest31495-21diskmemload19:08:412
20508993212,4cyclictest31495-21diskmemload19:08:412
2050899321,19cyclictest617-21ls17:05:572
2050899321,17cyclictest7729-21ssh18:40:582
20508993210,6cyclictest1684-21switchtime17:06:232
20507993210,10cyclictest0-21swapper/121:36:081
2050999312,5cyclictest22694-21apt-get18:06:033
2050999311,6cyclictest18310-21grep17:51:263
2050899316,4cyclictest7750irq/56-dwc_otg21:46:122
20508993120,9cyclictest7950irq/56-dwc_otg_21:38:022
2050899311,23cyclictest25711-21switchtime19:01:222
2050899311,21cyclictest8619-21missed_timers18:41:182
2050899311,21cyclictest26077-21latency_hist18:15:572
2050899311,21cyclictest1667-21uniq19:51:162
20508993110,4cyclictest19334-21ssh18:54:142
20508993110,4cyclictest0-21swapper/220:51:162
2050999304,15cyclictest7750irq/56-dwc_otg18:20:393
2050899309,5cyclictest0-21swapper/217:31:182
2050899304,6cyclictest7850irq/56-dwc_otg_16:31:252
2050899303,12cyclictest7950irq/56-dwc_otg_20:31:152
2050899301,8cyclictest30416-21ps16:56:212
20508993013,5cyclictest5195-21sh21:51:562
20508993013,5cyclictest5195-21sh21:51:562
2050899301,23cyclictest4818-21grep17:16:152
2050899301,23cyclictest30552-21switchtime19:46:242
2050899301,19cyclictest0-21swapper/218:21:012
2050899296,4cyclictest0-21swapper/218:33:012
2050899293,13cyclictest0-21swapper/221:41:282
2050899292,13cyclictest7950irq/56-dwc_otg_17:36:132
2050899292,13cyclictest7950irq/56-dwc_otg_17:36:132
2050899291,7cyclictest22992-21cut20:56:122
2050899291,5cyclictest18821-21latency_hist17:55:572
2050899281,24cyclictest28685-21timerandwakeup16:51:242
2050899281,18cyclictest0-21swapper/217:41:222
2050799281,5cyclictest27410-21grep18:16:261
2050899261,4cyclictest0-21swapper/216:41:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional