You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 05:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue May 05, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2545026500,27sleep024750-21/usr/sbin/munin04:26:180
3161321090,3sleep031616-21unixbench_singl06:11:230
398021050,2sleep02657799cyclictest09:06:080
130332940,2sleep00-21swapper/005:21:190
277292920,2sleep10-21swapper/109:35:001
40922890,2sleep00-21swapper/007:07:230
156592870,1sleep30-21swapper/309:20:293
63892810,3sleep06386-21smartpower06:31:190
26577997611,24cyclictest0-21swapper/005:01:060
253192730,4sleep12657899cyclictest05:56:101
103932700,2sleep22657999cyclictest08:34:262
2657799664,59cyclictest101ktimersoftd/009:01:160
2657799647,42cyclictest27726-21unixbench_multi04:31:220
2657799646,42cyclictest0-21swapper/004:46:220
26577996447,9cyclictest101ktimersoftd/008:15:540
2657799643,42cyclictest20481-21wget09:26:120
26577996412,29cyclictest15392-21sh08:40:540
26577996344,7cyclictest101ktimersoftd/007:36:140
26577996344,10cyclictest101ktimersoftd/009:35:540
26577996232,15cyclictest101ktimersoftd/005:31:100
26577996227,9cyclictest0-21swapper/006:51:150
26577996227,9cyclictest0-21swapper/006:51:150
2657799621,45cyclictest14000-21wget06:41:070
2657799621,43cyclictest8636-21grep08:31:220
26577996130,14cyclictest6743-21grep07:11:110
2657799612,43cyclictest3633-21fschecks_count08:26:130
2657799612,43cyclictest3633-21fschecks_count08:26:130
2657799611,43cyclictest11590-21grep05:16:210
125692610,3sleep10-21swapper/105:21:121
26577996027,25cyclictest0-21swapper/006:56:090
26577995953,4cyclictest101ktimersoftd/007:27:040
26577995938,10cyclictest24789-21munin-run05:55:540
2657799591,42cyclictest3484-21fschecks_time04:56:110
147372590,5sleep32658099cyclictest08:00:453
147372590,5sleep32658099cyclictest08:00:453
26577995843,4cyclictest101ktimersoftd/006:37:380
26577995836,10cyclictest101ktimersoftd/006:01:190
26577995836,10cyclictest101ktimersoftd/006:01:190
26577995834,7cyclictest0-21swapper/006:26:110
2657799582,40cyclictest0-21swapper/004:36:070
2657799582,40cyclictest0-21swapper/004:36:070
2657799581,41cyclictest0-21swapper/009:21:050
2657799576,48cyclictest101ktimersoftd/008:58:260
2657799575,30cyclictest101ktimersoftd/005:56:100
26577995733,5cyclictest19446-21grep06:46:220
2657799571,46cyclictest22986-21ssh07:31:080
2657799571,46cyclictest22986-21ssh07:31:080
2657799571,43cyclictest0-21swapper/006:21:090
2657799571,40cyclictest31446-21sort07:41:120
2657799566,27cyclictest27689-21apt08:55:540
2657799565,38cyclictest30780-21ls07:01:090
26577995638,8cyclictest0-21swapper/009:51:210
26577995636,9cyclictest2262-21ssh08:24:370
26577995635,11cyclictest101ktimersoftd/009:43:060
26577995538,6cyclictest101ktimersoftd/008:46:190
2657799553,38cyclictest11334-21fschecks_count07:56:090
2657799553,38cyclictest11334-21fschecks_count07:56:090
26577995529,9cyclictest17093-21ssh08:41:360
2657799552,41cyclictest12480-21uniq09:16:110
26577995431,9cyclictest9861-21chrt05:12:050
26577995425,15cyclictest7950irq/56-dwc_otg_08:06:220
2657799542,49cyclictest101ktimersoftd/009:50:540
2657799541,43cyclictest0-21swapper/008:01:120
2657799541,42cyclictest32490-21grep06:16:110
2657799541,40cyclictest0-21swapper/008:15:580
2657799541,37cyclictest0-21swapper/009:11:050
26577995341,3cyclictest0-21swapper/007:50:290
26577995336,4cyclictest18808-21/usr/sbin/munin05:36:220
26577995328,13cyclictest0-21swapper/007:25:210
2657799532,32cyclictest0-21swapper/005:10:540
2657799531,42cyclictest0-21swapper/007:16:200
2657799531,40cyclictest0-21swapper/010:00:540
129032530,1sleep30-21swapper/306:39:403
26577995233,8cyclictest7228-21ssh07:51:100
2657799516,21cyclictest21164-21cat05:45:540
2657799512,35cyclictest0-21swapper/006:06:150
2658099502,12cyclictest0-21swapper/307:36:043
2658099492,10cyclictest0-21swapper/306:06:073
26578994914,22cyclictest0-21swapper/106:26:101
26577994927,9cyclictest0-21swapper/009:37:140
289362480,1sleep20-21swapper/208:16:342
2657799471,38cyclictest0-21swapper/005:46:080
2657799471,26cyclictest14112-21aten_rbpower_po05:26:080
2657799471,26cyclictest14112-21aten_rbpower_po05:26:080
2657799461,32cyclictest0-21swapper/004:51:170
214702460,2sleep20-21swapper/206:49:572
2658099452,7cyclictest0-21swapper/309:11:063
2657799454,29cyclictest0-21swapper/004:41:200
2658099443,18cyclictest0-21swapper/306:11:073
2658099442,8cyclictest19267-21ssh08:06:053
2657999447,24cyclictest23913-21ssh08:51:082
26579994429,9cyclictest13144-21rm08:36:552
2657899447,25cyclictest0-21swapper/105:01:061
2657899446,25cyclictest7750irq/56-dwc_otg09:00:551
26578994419,6cyclictest0-21swapper/107:56:211
26578994419,6cyclictest0-21swapper/107:56:211
26580994324,7cyclictest28362-21ssh08:56:113
26580994318,12cyclictest7950irq/56-dwc_otg_07:16:333
26580994317,14cyclictest7950irq/56-dwc_otg_08:46:073
2657999437,26cyclictest0-21swapper/206:21:072
2657899431,6cyclictest7750irq/56-dwc_otg06:46:061
26578994316,8cyclictest0-21swapper/107:11:001
26578994315,4cyclictest0-21swapper/108:46:131
2657899431,34cyclictest0-21swapper/106:40:541
2658099428,16cyclictest7950irq/56-dwc_otg_08:39:103
2657899423,26cyclictest7750irq/56-dwc_otg08:35:531
26578994213,3cyclictest0-21swapper/107:46:081
26580994114,6cyclictest7750irq/56-dwc_otg06:51:123
26580994114,6cyclictest7750irq/56-dwc_otg06:51:123
26579994115,5cyclictest8006-21sh09:11:052
2657899417,23cyclictest0-21swapper/105:53:121
2657899417,21cyclictest0-21swapper/108:51:081
2657899414,27cyclictest7750irq/56-dwc_otg09:56:221
2657899412,6cyclictest0-21swapper/107:31:051
2657899412,6cyclictest0-21swapper/107:31:051
240442410,2sleep3381ktimersoftd/309:30:473
95962400,1sleep30-21swapper/308:32:583
2658099405,6cyclictest0-21swapper/308:51:063
26580994018,5cyclictest20503-21sh06:48:103
26580994018,4cyclictest7750irq/56-dwc_otg05:01:133
2657999401,5cyclictest7750irq/56-dwc_otg04:56:072
2657899406,6cyclictest27369-21grep07:36:101
2657899402,23cyclictest7950irq/56-dwc_otg_06:16:121
26578994015,5cyclictest32042-21sh09:40:201
26578994013,5cyclictest27186-21ssh08:15:541
2658099395,19cyclictest7950irq/56-dwc_otg_08:27:303
2658099395,19cyclictest7950irq/56-dwc_otg_08:27:303
26580993919,13cyclictest1010350rm07:54:483
26580993917,10cyclictest381ktimersoftd/309:01:153
2657999397,20cyclictest0-21swapper/206:26:102
26579993922,4cyclictest30009-21idleruntime-cro04:40:542
26579993922,4cyclictest30009-21idleruntime-cro04:40:542
26579993913,5cyclictest12322-21aten_rbpower_vo05:21:092
2657899392,27cyclictest0-21swapper/107:41:061
26578993913,4cyclictest19550-21latency_hist08:45:541
26578993911,13cyclictest7950irq/56-dwc_otg_04:51:051
2658099384,14cyclictest7850irq/56-dwc_otg_04:56:083
2658099383,16cyclictest27114-21fschecks_count06:01:103
2658099383,16cyclictest27114-21fschecks_count06:01:103
26580993815,5cyclictest5004-21diskmemload09:31:053
26580993813,4cyclictest381ktimersoftd/305:51:123
26580993810,4cyclictest7750irq/56-dwc_otg06:26:103
2657999387,19cyclictest7750irq/56-dwc_otg09:25:542
26579993825,3cyclictest301ktimersoftd/209:32:052
26579993815,5cyclictest6611-21cron07:50:542
2657899384,4cyclictest12185-21aten_rbpower_en09:16:071
26578993815,4cyclictest7750irq/56-dwc_otg09:11:091
190582380,2sleep10-21swapper/108:05:541
26580993716,5cyclictest5004-21diskmemload07:46:113
26580993711,13cyclictest381ktimersoftd/308:01:543
2657999372,23cyclictest7410-21ls07:11:202
26579993718,4cyclictest31-21ksoftirqd/206:51:052
26579993718,4cyclictest31-21ksoftirqd/206:51:052
26579993710,18cyclictest0-21swapper/209:06:082
2657899372,9cyclictest22432-21grep06:51:081
2657899372,9cyclictest22432-21grep06:51:081
26578993715,4cyclictest30339-21latency_hist06:10:541
2658099367,26cyclictest0-21swapper/306:16:073
2658099367,16cyclictest7950irq/56-dwc_otg_07:02:283
26580993617,6cyclictest7750irq/56-dwc_otg09:06:523
26580993617,11cyclictest31780-21kworker/u8:109:38:063
26580993610,7cyclictest37-21rcuc/309:51:213
2657999367,25cyclictest0-21swapper/205:53:122
26579993618,5cyclictest8175-21idleruntime-cro09:50:542
2657999361,22cyclictest0-21swapper/209:01:072
2657999361,20cyclictest31-21ksoftirqd/205:01:102
2657999361,19cyclictest8543-21ssh06:34:292
2657999361,18cyclictest27254-21latency_hist08:15:542
2657999361,11cyclictest0-21swapper/207:35:542
2657999361,11cyclictest0-21swapper/207:35:542
26578993623,9cyclictest0-21swapper/109:51:231
26578993616,4cyclictest15125-21sh06:41:211
26578993615,12cyclictest0-21swapper/109:01:051
26578993612,4cyclictest3972-21cpu09:06:081
227922360,5sleep22657999cyclictest08:10:262
170032360,2sleep2301ktimersoftd/207:23:152
15392360,3sleep121-21rcuc/109:41:201
2658099359,9cyclictest7950irq/56-dwc_otg_06:59:303
2658099355,4cyclictest7950irq/56-dwc_otg_07:11:053
2658099355,16cyclictest37-21rcuc/306:31:583
26580993522,10cyclictest26599-21idleruntime-cro06:00:543
26580993515,4cyclictest7950irq/56-dwc_otg_08:19:193
2658099351,23cyclictest9600-21timerwakeupswit05:11:213
2658099351,16cyclictest7950irq/56-dwc_otg_05:21:093
26580993510,9cyclictest7850irq/56-dwc_otg_09:48:403
2657999358,12cyclictest8468-21/usr/sbin/munin09:51:042
2657999355,18cyclictest0-21swapper/207:41:192
26579993523,5cyclictest7950irq/56-dwc_otg_06:56:062
2657999352,28cyclictest7950irq/56-dwc_otg_06:06:052
2657999352,10cyclictest14341-21munin-run07:20:542
26579993517,9cyclictest31-21ksoftirqd/204:31:212
26579993514,13cyclictest20198-21/usr/sbin/munin08:46:142
2657999351,24cyclictest0-21swapper/207:55:472
2657999351,16cyclictest0-21swapper/209:45:042
2657999351,14cyclictest0-21swapper/205:34:192
26579993511,11cyclictest0-21swapper/207:26:092
2657899355,5cyclictest0-21swapper/109:46:261
2657899355,4cyclictest11267-21/usr/sbin/munin05:16:191
2657899354,20cyclictest0-21swapper/106:59:131
2657899351,24cyclictest14629-21ls05:26:151
2657899351,24cyclictest14629-21ls05:26:151
2658099347,25cyclictest0-21swapper/307:31:073
2658099347,25cyclictest0-21swapper/307:31:073
2658099347,22cyclictest0-21swapper/305:06:073
2658099344,4cyclictest7950irq/56-dwc_otg_07:06:193
2658099342,20cyclictest37-21rcuc/307:25:573
2658099341,6cyclictest0-21swapper/304:51:063
26580993412,9cyclictest7850irq/56-dwc_otg_09:59:403
2658099341,15cyclictest7950irq/56-dwc_otg_07:42:003
2657999342,15cyclictest0-21swapper/209:56:212
26579993417,11cyclictest29000-21sh07:37:362
2657999341,24cyclictest2242-21ssh07:06:062
2657999341,14cyclictest0-21swapper/208:05:052
26579993411,14cyclictest301ktimersoftd/208:25:552
2657899345,11cyclictest6494-1kworker/u9:207:26:051
2657899344,20cyclictest7850irq/56-dwc_otg_09:26:261
2657899344,20cyclictest7850irq/56-dwc_otg_07:01:111
26578993427,5cyclictest32059-21wget04:46:081
26578993423,3cyclictest7950irq/56-dwc_otg_06:30:591
2657899342,24cyclictest0-21swapper/105:31:101
2657899342,23cyclictest7750irq/56-dwc_otg05:10:541
2657899341,6cyclictest310-21timerwakeupswit08:21:221
26578993413,6cyclictest23-21ksoftirqd/108:36:091
26578993412,6cyclictest6041-21http07:10:541
2658099334,14cyclictest7950irq/56-dwc_otg_05:44:543
26580993315,3cyclictest7750irq/56-dwc_otg09:41:053
2658099331,4cyclictest7750irq/56-dwc_otg07:22:333
26580993312,13cyclictest7850irq/56-dwc_otg_08:44:433
2657999335,11cyclictest7750irq/56-dwc_otg06:41:142
2657999331,26cyclictest0-21swapper/209:36:182
2657999331,26cyclictest0-21swapper/205:57:082
2657999331,23cyclictest0-21swapper/205:46:072
2657999331,19cyclictest0-21swapper/206:20:162
26579993310,12cyclictest0-21swapper/208:43:492
2657899338,5cyclictest23-21ksoftirqd/104:31:121
26578993315,10cyclictest5004-21diskmemload07:20:541
2657899331,4cyclictest7750irq/56-dwc_otg04:36:051
2657899331,4cyclictest7750irq/56-dwc_otg04:36:051
26578993312,13cyclictest19309-21wget08:06:071
2658099327,22cyclictest0-21swapper/304:36:073
2658099327,22cyclictest0-21swapper/304:36:073
2658099324,13cyclictest7750irq/56-dwc_otg05:20:183
2658099322,13cyclictest381ktimersoftd/304:31:223
26580993217,5cyclictest908-21runrttasks08:11:413
2658099321,5cyclictest37-21rcuc/304:42:313
26580993214,5cyclictest14013-21/usr/sbin/munin05:26:133
26580993214,5cyclictest14013-21/usr/sbin/munin05:26:133
26580993210,10cyclictest0-21swapper/308:23:103
2657999326,19cyclictest31-21ksoftirqd/207:56:092
2657999326,19cyclictest31-21ksoftirqd/207:56:092
2657999321,7cyclictest7850irq/56-dwc_otg_05:26:112
2657999321,7cyclictest7850irq/56-dwc_otg_05:26:112
26579993217,5cyclictest14330-21sh09:18:022
26579993212,4cyclictest20959-21sshd05:45:152
2657999321,23cyclictest0-21swapper/208:28:462
2657999321,23cyclictest0-21swapper/208:28:462
2657999321,23cyclictest0-21swapper/205:36:192
2657999321,21cyclictest20397-21ssh09:26:122
2657899324,4cyclictest14827-21cpu07:21:091
26578993226,4cyclictest27815-21sh08:16:111
26578993215,9cyclictest221ktimersoftd/108:30:071
26578993215,9cyclictest221ktimersoftd/108:30:071
2657899321,23cyclictest0-21swapper/109:21:081
254732327,15sleep10-21swapper/104:26:201
2658099315,9cyclictest7950irq/56-dwc_otg_05:31:183
26580993111,8cyclictest7950irq/56-dwc_otg_06:21:053
2657999313,22cyclictest0-21swapper/206:03:472
2657999313,22cyclictest0-21swapper/206:03:472
2657999311,8cyclictest30636-21wget07:01:082
2657999311,24cyclictest0-21swapper/208:56:542
2657999311,14cyclictest0-21swapper/205:19:532
2657899316,13cyclictest0-21swapper/107:51:091
2657899313,10cyclictest21-21rcuc/105:44:481
2658099304,14cyclictest7950irq/56-dwc_otg_06:42:543
2658099301,19cyclictest16565-21grep09:21:123
2657999305,17cyclictest371-21/usr/sbin/munin04:46:192
2657999302,12cyclictest7950irq/56-dwc_otg_06:36:222
2657999301,23cyclictest0-21swapper/204:51:342
2657999301,20cyclictest0-21swapper/206:13:202
2657899309,5cyclictest7750irq/56-dwc_otg06:01:151
2657899309,5cyclictest7750irq/56-dwc_otg06:01:151
2657899305,4cyclictest0-21swapper/105:47:301
2657899301,4cyclictest21-21rcuc/104:56:051
256022307,11sleep30-21swapper/304:26:223
2658099293,13cyclictest0-21swapper/305:47:583
2657999291,10cyclictest7750irq/56-dwc_otg05:13:552
2657899296,17cyclictest7850irq/56-dwc_otg_05:36:221
2657899292,19cyclictest7750irq/56-dwc_otg04:41:491
2658099288,4cyclictest0-21swapper/304:48:103
2657999282,19cyclictest0-21swapper/204:41:142
2657999281,17cyclictest908-21runrttasks05:06:492
2657899281,6cyclictest7750irq/56-dwc_otg06:21:081
2658099271,23cyclictest0-21swapper/305:36:073
2657899275,10cyclictest7850irq/56-dwc_otg_05:15:541
2657899272,5cyclictest7750irq/56-dwc_otg06:11:121
247472263,14sleep20-21swapper/204:26:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional