You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 22:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Mar 15, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
401022750,1sleep00-21swapper/007:05:240
5279991241,56cyclictest21587-21ssh09:55:160
1251321050,2sleep3381ktimersoftd/312:20:173
5279999810,4cyclictest31091-21ssh10:05:310
527999971,63cyclictest0-21swapper/012:25:190
5279998828,6cyclictest15503-21grep07:35:320
51522860,6sleep1528099cyclictest08:39:141
3672860,5sleep1528099cyclictest08:25:221
263982830,2sleep00-21swapper/011:18:390
527999822,25cyclictest0-21swapper/009:40:040
527999821,43cyclictest0-21swapper/009:20:140
527999811,28cyclictest26996-21uniq10:40:210
5279998112,18cyclictest0-21swapper/010:20:150
527999802,27cyclictest29858-21ntpq09:25:260
527999801,29cyclictest1211-21grep08:25:320
527999791,15cyclictest20401-21rm10:31:450
527999791,15cyclictest20401-21rm10:31:450
527999781,26cyclictest11687-21grep11:00:280
240662770,3sleep25276-21cyclictest12:34:132
223332760,2sleep30-21swapper/309:16:073
223332760,2sleep30-21swapper/309:16:073
113252760,2sleep17850irq/56-dwc_otg_09:42:061
527999751,22cyclictest12591-21/usr/sbin/munin11:40:300
527999751,22cyclictest12591-21/usr/sbin/munin11:40:300
527999744,20cyclictest0-21swapper/010:55:040
5279997324,4cyclictest12856-21fschecks_time07:30:220
527999731,21cyclictest0-21swapper/008:55:250
5279997222,4cyclictest0-21swapper/007:20:040
527999722,24cyclictest0-21swapper/007:10:150
5279997217,28cyclictest0-21swapper/008:35:170
5279997212,6cyclictest0-21swapper/008:05:160
5279997210,21cyclictest0-21swapper/010:11:280
133372720,1sleep20-21swapper/212:20:282
5279997116,46cyclictest0-21swapper/008:22:500
5279997010,12cyclictest7750irq/56-dwc_otg09:05:150
527999697,14cyclictest0-21swapper/009:52:580
527999697,14cyclictest0-21swapper/009:52:580
527999693,52cyclictest0-21swapper/012:00:160
5279996918,10cyclictest7950irq/56-dwc_otg_11:32:220
5279996914,4cyclictest0-21swapper/012:05:490
527999691,16cyclictest0-21swapper/009:13:490
527999689,11cyclictest0-21swapper/010:35:170
5279996816,13cyclictest0-21swapper/008:50:210
5279996815,9cyclictest0-21swapper/007:25:310
5279996815,9cyclictest0-21swapper/007:25:310
5279996813,11cyclictest917-21snmpd08:33:420
527999673,50cyclictest0-21swapper/012:35:150
527999672,21cyclictest0-21swapper/010:57:340
527999672,16cyclictest0-21swapper/011:20:200
5279996718,3cyclictest0-21swapper/012:30:340
5279996715,4cyclictest8643-21fschecks_count12:15:210
527999661,15cyclictest9826-21ls09:40:250
527999661,14cyclictest0-21swapper/011:35:300
233602660,2sleep30-21swapper/308:00:043
527999653,17cyclictest0-21swapper/012:11:490
527999652,18cyclictest0-21swapper/008:50:000
527999645,18cyclictest0-21swapper/007:20:220
527999643,15cyclictest0-21swapper/011:54:590
527999643,13cyclictest19210-21/usr/sbin/munin11:10:220
527999642,22cyclictest0-21swapper/010:17:180
527999642,20cyclictest0-21swapper/009:34:240
527999642,17cyclictest0-21swapper/010:26:210
527999641,25cyclictest0-21swapper/011:05:210
527999641,19cyclictest0-21swapper/011:26:040
527999636,18cyclictest0-21swapper/009:19:550
527999636,18cyclictest0-21swapper/009:19:550
5279996316,27cyclictest12923-21ssh12:20:220
527999621,22cyclictest0-21swapper/011:57:060
527999621,17cyclictest0-21swapper/011:45:430
527999621,17cyclictest0-21swapper/010:04:550
527999621,15cyclictest13291-21dpkg09:45:140
5279996211,12cyclictest0-21swapper/007:40:150
5279996113,5cyclictest16063-21diskmemload10:48:540
527999611,17cyclictest0-21swapper/008:01:330
273262610,4sleep227325-21fschecks_count08:10:222
205092610,4sleep1221ktimersoftd/109:15:051
87502600,2sleep3381ktimersoftd/310:18:273
5282996033,4cyclictest7950irq/56-dwc_otg_12:25:173
527999590,28cyclictest0-21swapper/008:10:160
527999553,7cyclictest0-21swapper/008:15:170
5279995527,13cyclictest7750irq/56-dwc_otg07:50:040
527999541,32cyclictest12750-21aten_rbpower_en09:00:170
5282995230,11cyclictest7750irq/56-dwc_otg11:57:423
5279995021,13cyclictest7950irq/56-dwc_otg_07:55:180
5282994914,7cyclictest7950irq/56-dwc_otg_10:25:183
5280994910,24cyclictest0-21swapper/107:40:161
181852490,6sleep2301ktimersoftd/209:11:192
5282994816,16cyclictest7750irq/56-dwc_otg08:25:223
49932483,12sleep10-21swapper/107:09:511
5281994719,11cyclictest7950irq/56-dwc_otg_09:25:032
528099477,3cyclictest0-21swapper/110:00:151
528099477,25cyclictest0-21swapper/109:50:161
528099477,25cyclictest0-21swapper/109:50:161
528099467,26cyclictest0-21swapper/107:25:171
528099467,26cyclictest0-21swapper/107:25:171
528099466,25cyclictest0-21swapper/109:00:171
5279994620,13cyclictest0-21swapper/007:51:070
45752460,2sleep20-21swapper/208:35:292
5282994517,10cyclictest7750irq/56-dwc_otg10:20:063
528199451,6cyclictest11668-21ssh11:40:152
528199451,6cyclictest11668-21ssh11:40:152
5279994512,9cyclictest0-21swapper/008:40:160
5279994512,9cyclictest0-21swapper/008:40:160
528299449,9cyclictest9080-21apt-get08:50:153
528299448,12cyclictest4142-21/usr/sbin/munin12:10:303
5282994411,16cyclictest7750irq/56-dwc_otg09:15:053
528099443,6cyclictest7750irq/56-dwc_otg08:50:161
5280994422,6cyclictest908-21runrttasks11:30:301
5282994314,16cyclictest7950irq/56-dwc_otg_11:00:253
150822430,4sleep115066-21apt-get11:05:151
5282994213,10cyclictest7950irq/56-dwc_otg_12:00:283
528299421,20cyclictest7950irq/56-dwc_otg_10:40:203
528099427,25cyclictest0-21swapper/112:30:181
528099427,25cyclictest0-21swapper/109:05:171
528099427,24cyclictest0-21swapper/107:55:181
528099426,23cyclictest19254-21ssh11:10:181
251362420,5sleep1528099cyclictest11:55:311
223742420,5sleep2528199cyclictest10:35:052
223742420,5sleep2528199cyclictest10:35:052
5282994112,14cyclictest7950irq/56-dwc_otg_07:15:143
528199417,24cyclictest0-21swapper/209:50:162
528199417,24cyclictest0-21swapper/209:50:162
528199412,8cyclictest7850irq/56-dwc_otg_07:15:172
528199412,25cyclictest4863-21ls09:35:052
5281994119,5cyclictest9219-21sh11:36:152
5281994119,4cyclictest15159-21aten_rbpower_po11:05:162
528099417,24cyclictest0-21swapper/112:10:101
55522400,2sleep121-21rcuc/110:53:361
528299407,24cyclictest0-21swapper/308:45:163
528299407,14cyclictest12308-21/usr/sbin/munin11:40:263
528299407,14cyclictest12308-21/usr/sbin/munin11:40:263
528299404,16cyclictest11422-21chrt07:25:263
528299404,16cyclictest11422-21chrt07:25:263
5282994022,4cyclictest7950irq/56-dwc_otg_11:29:283
5282994014,4cyclictest7850irq/56-dwc_otg_08:20:213
5282994011,4cyclictest7950irq/56-dwc_otg_09:40:173
528099402,9cyclictest24372-21timerandwakeup08:00:311
5280994015,5cyclictest25326-21fschecks_time12:35:211
289532400,32sleep128952-21wget08:15:201
528299394,3cyclictest7950irq/56-dwc_otg_07:20:153
5282993922,4cyclictest7750irq/56-dwc_otg09:53:583
5282993922,4cyclictest7750irq/56-dwc_otg09:53:583
5282993916,8cyclictest7950irq/56-dwc_otg_10:51:443
528299391,24cyclictest0-21swapper/307:50:173
528199397,23cyclictest0-21swapper/209:15:172
528199397,23cyclictest0-21swapper/209:15:172
528199393,6cyclictest0-21swapper/212:05:052
5281993914,9cyclictest301ktimersoftd/212:11:222
528099392,33cyclictest11206-21grep08:55:221
528099392,22cyclictest27585-21latency08:10:241
5280993921,9cyclictest908-21runrttasks07:50:161
5280993919,5cyclictest23-21ksoftirqd/109:40:041
5280993918,6cyclictest23-21ksoftirqd/112:25:221
528099391,18cyclictest0-21swapper/111:23:071
528299381,25cyclictest5674-21ls08:40:183
528299381,25cyclictest5674-21ls08:40:183
5282993811,9cyclictest7850irq/56-dwc_otg_10:02:583
5282993811,8cyclictest7850irq/56-dwc_otg_09:20:263
528199386,15cyclictest7850irq/56-dwc_otg_10:15:182
528199383,14cyclictest31-21ksoftirqd/211:20:302
528199382,7cyclictest31-21ksoftirqd/207:40:312
528199381,5cyclictest22442-21taskset11:52:432
528199381,23cyclictest29752-21ls10:05:052
528099384,8cyclictest6937-21df10:55:201
5280993814,7cyclictest0-21swapper/110:35:201
528099381,26cyclictest21594-21ssh09:55:171
528299377,5cyclictest30450-21idleruntime-cro08:20:043
528299375,20cyclictest39-21ksoftirqd/308:06:413
5282993721,5cyclictest7750irq/56-dwc_otg10:05:293
5282993720,8cyclictest7850irq/56-dwc_otg_12:19:423
5282993718,4cyclictest21231-21sh10:33:153
5282993718,4cyclictest21231-21sh10:33:153
5282993712,13cyclictest7950irq/56-dwc_otg_09:45:383
5282993710,7cyclictest8713-21unixbench-2d11:35:323
528199377,8cyclictest31-21ksoftirqd/210:05:222
528099374,7cyclictest7508-21ssh11:35:151
528099372,6cyclictest29505-21grep12:00:311
105272370,4sleep10-21swapper/110:20:231
528299369,5cyclictest20299-21fschecks_time11:50:213
528299366,21cyclictest0-21swapper/312:35:093
5282993621,4cyclictest7950irq/56-dwc_otg_11:30:063
5282993611,9cyclictest19131-21ssh11:49:133
5282993610,15cyclictest7950irq/56-dwc_otg_10:37:273
528199361,6cyclictest29-21rcuc/208:55:192
5281993613,4cyclictest11133-21timerandwakeup10:20:312
528099365,21cyclictest21679-21ls09:15:311
528099365,21cyclictest21679-21ls09:15:311
528099364,14cyclictest0-21swapper/110:31:271
528099364,14cyclictest0-21swapper/110:31:271
5280993616,4cyclictest520-21cron11:30:041
5280993614,4cyclictest0-21swapper/110:13:251
5280993610,16cyclictest0-21swapper/112:23:221
56902350,5sleep1528099cyclictest08:40:191
56902350,5sleep1528099cyclictest08:40:191
528299356,15cyclictest28605-21ls11:20:293
528299355,18cyclictest7950irq/56-dwc_otg_09:25:463
528299355,15cyclictest7950irq/56-dwc_otg_10:14:403
528299354,19cyclictest23822-21ssh09:57:133
5282993526,6cyclictest381ktimersoftd/310:55:183
528299350,20cyclictest1736-21ssh10:49:113
528199359,6cyclictest31-21ksoftirqd/210:40:302
528199358,18cyclictest7950irq/56-dwc_otg_11:00:162
528199357,5cyclictest7850irq/56-dwc_otg_10:25:102
528199355,4cyclictest7850irq/56-dwc_otg_08:35:052
528199352,19cyclictest31-21ksoftirqd/207:12:342
528199351,6cyclictest25390-21apt-get10:00:032
528199351,3cyclictest0-21swapper/209:30:042
528099355,6cyclictest7950irq/56-dwc_otg_12:10:051
5280993515,4cyclictest13722-21if_err_eth009:45:221
528099351,26cyclictest0-21swapper/111:42:471
528099351,26cyclictest0-21swapper/111:42:471
528299345,11cyclictest7950irq/56-dwc_otg_11:12:183
528299344,15cyclictest7950irq/56-dwc_otg_12:30:473
528299343,20cyclictest7750irq/56-dwc_otg09:36:303
528299343,11cyclictest7950irq/56-dwc_otg_12:08:363
5282993410,7cyclictest7850irq/56-dwc_otg_11:15:213
528199347,6cyclictest7850irq/56-dwc_otg_10:50:272
528199346,14cyclictest7850irq/56-dwc_otg_12:30:052
528199342,18cyclictest7850irq/56-dwc_otg_07:25:292
528199342,18cyclictest7850irq/56-dwc_otg_07:25:292
528199341,8cyclictest6624-21grep08:40:312
528199341,8cyclictest6624-21grep08:40:312
528099349,13cyclictest666050ssh10:15:291
528099346,5cyclictest7750irq/56-dwc_otg10:40:321
528099341,8cyclictest25596-21ssh09:20:241
528099341,28cyclictest8027-21ls07:15:301
528099341,23cyclictest31812-21switchtime10:45:291
528299337,7cyclictest0-21swapper/308:14:353
528299337,14cyclictest11235-21basename08:55:223
528299336,5cyclictest7750irq/56-dwc_otg07:35:293
5282993310,12cyclictest7950irq/56-dwc_otg_11:06:093
528199339,6cyclictest7850irq/56-dwc_otg_10:35:322
528199339,14cyclictest0-21swapper/211:25:212
528199338,5cyclictest31-21ksoftirqd/209:40:152
528199337,6cyclictest7850irq/56-dwc_otg_08:15:342
528199336,14cyclictest0-21swapper/207:55:102
528199334,5cyclictest9231-21grep07:20:202
528199333,6cyclictest29-21rcuc/208:25:182
528199331,24cyclictest0-21swapper/209:37:212
528199331,23cyclictest23643-21cut11:15:212
5280993311,4cyclictest7850irq/56-dwc_otg_10:05:451
528299324,15cyclictest5290-21taskset07:10:113
5282993213,8cyclictest7950irq/56-dwc_otg_09:03:473
528299321,24cyclictest4264-21ls08:35:243
528199327,10cyclictest7850irq/56-dwc_otg_10:55:222
528199326,15cyclictest0-21swapper/209:00:172
528199325,17cyclictest24921-21ssh12:35:102
528199323,14cyclictest7850irq/56-dwc_otg_10:45:172
528199321,6cyclictest9165-21aten_rbpower_po08:50:182
528199321,6cyclictest31-21ksoftirqd/207:40:042
5281993210,4cyclictest7750irq/56-dwc_otg10:10:272
528099329,7cyclictest23-21ksoftirqd/112:15:191
528099328,4cyclictest0-21swapper/111:02:441
528099323,5cyclictest0-21swapper/110:30:041
5280993217,7cyclictest2519550ssh11:16:291
528299314,10cyclictest7950irq/56-dwc_otg_09:09:183
528299313,14cyclictest16629-21latency07:40:233
528299311,6cyclictest874-21grep09:30:173
528299311,17cyclictest12523-21/usr/sbin/munin07:30:323
528199317,4cyclictest7750irq/56-dwc_otg12:15:222
528199314,12cyclictest7850irq/56-dwc_otg_11:55:312
5281993117,6cyclictest32743-21ssh12:05:212
528199311,6cyclictest26319-21grep08:05:322
5281993112,4cyclictest31-21ksoftirqd/209:45:292
5281993110,11cyclictest20145-21if_eth007:50:222
528099319,4cyclictest0-21swapper/111:50:311
528099317,13cyclictest0-21swapper/111:48:071
528099316,14cyclictest0-21swapper/108:30:161
528099315,7cyclictest30987-21grep08:20:211
5280993114,8cyclictest16063-21diskmemload09:28:091
5280993114,4cyclictest16063-21diskmemload09:30:061
528299305,12cyclictest7950irq/56-dwc_otg_08:30:523
528199306,5cyclictest7750irq/56-dwc_otg11:45:212
5281993010,4cyclictest0-21swapper/211:11:192
528099301,22cyclictest7444-21cpuspeed08:45:171
49002304,18sleep3917-21snmpd07:08:483
528299291,22cyclictest0-21swapper/308:01:563
528199298,5cyclictest7850irq/56-dwc_otg_08:00:162
528199296,5cyclictest7850irq/56-dwc_otg_09:05:282
528199296,15cyclictest31-21ksoftirqd/207:30:282
528199296,14cyclictest0-21swapper/208:20:172
528199294,6cyclictest7950irq/56-dwc_otg_11:33:522
528099291,5cyclictest19100-21grep07:45:321
528299281,4cyclictest7950irq/56-dwc_otg_07:46:133
528199284,14cyclictest7950irq/56-dwc_otg_07:45:312
528099285,11cyclictest7950irq/56-dwc_otg_08:05:281
528099281,4cyclictest7850irq/56-dwc_otg_07:35:171
528099281,17cyclictest0-21swapper/107:20:251
528199271,14cyclictest7950irq/56-dwc_otg_08:45:162
528099272,10cyclictest7950irq/56-dwc_otg_07:30:321
5280992718,6cyclictest7950irq/56-dwc_otg_07:10:151
43702270,5sleep24372-21switchtime07:05:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional