You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 08:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jun 19, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12903991024,20cyclictest12869-21ssh21:35:100
81622990,6sleep01290399cyclictest00:05:220
1290399877,22cyclictest0-21swapper/023:45:190
308822860,3sleep230875-21ssh22:35:262
12904998615,44cyclictest5740-21grep21:25:351
12903998320,28cyclictest0-21swapper/019:30:110
158702820,4sleep314734-21/usr/sbin/munin19:15:323
204642790,5sleep21290599cyclictest22:22:582
192422780,5sleep11290499cyclictest22:20:441
132552750,2sleep30-21swapper/323:31:563
42872740,3sleep24288-21latency00:00:292
42872740,3sleep24288-21latency00:00:292
239012740,2sleep0101ktimersoftd/021:48:010
306332720,2sleep30-21swapper/321:55:333
184662720,4sleep3381ktimersoftd/321:40:393
1290399717,42cyclictest0-21swapper/022:45:170
185492690,2sleep00-21swapper/021:40:490
1290499691,5cyclictest31104-21apt-get20:00:161
310082680,6sleep01290399cyclictest23:15:100
310082680,6sleep01290399cyclictest23:15:100
1290499689,16cyclictest7850irq/56-dwc_otg21:35:151
1290499686,17cyclictest0-21swapper/122:55:191
1290399681,9cyclictest16311-21ls00:15:200
1290499673,6cyclictest21866-21ssh00:21:021
1290499666,43cyclictest8050irq/56-dwc_otg_19:40:311
1290499661,49cyclictest25392-21ssh23:07:541
12904996611,40cyclictest16820-21ssh23:35:521
185322650,2sleep30-21swapper/320:55:183
1290499656,40cyclictest8050irq/56-dwc_otg_20:25:241
1290499655,42cyclictest7850irq/56-dwc_otg20:45:191
1290499652,5cyclictest29538-21rm23:51:131
1290499652,48cyclictest6175-21ssh22:45:151
1290499652,38cyclictest30333-21ssh21:55:301
1290499651,6cyclictest19440-21ssh21:42:261
1290499649,36cyclictest0-21swapper/123:56:241
1290499643,40cyclictest8050irq/56-dwc_otg_20:15:051
1290499641,8cyclictest8845-21ssh23:26:381
12904996414,46cyclictest14838-21aten_rbpower_vo19:15:181
1290399645,28cyclictest2116-21rm22:40:180
1290399642,34cyclictest0-21swapper/020:35:180
1290399642,34cyclictest0-21swapper/020:35:180
1290499635,37cyclictest7950irq/56-dwc_otg_21:45:221
1290499632,5cyclictest0-21swapper/121:14:111
12904996323,38cyclictest18121-21sh19:25:051
1290499631,49cyclictest10013-21ssh00:07:071
196712620,1sleep10-21swapper/100:19:371
1290499623,5cyclictest22644-21latency_hist23:05:051
1290499623,5cyclictest13890-21ssh22:15:201
1290499621,6cyclictest4072-21cron21:25:041
1290499621,6cyclictest11296-21wget23:30:191
12903996242,5cyclictest7950irq/56-dwc_otg_21:20:250
1290499615,43cyclictest8050irq/56-dwc_otg_00:12:241
1290499612,39cyclictest0-21swapper/122:25:241
1290499611,48cyclictest22929-21ssh23:44:201
1290499611,45cyclictest0-21swapper/119:13:171
1290499611,41cyclictest0-21swapper/119:35:151
12903996146,3cyclictest0-21swapper/022:30:160
12903996146,3cyclictest0-21swapper/022:30:160
1290499606,46cyclictest7950irq/56-dwc_otg_22:08:361
1290499595,33cyclictest0-21swapper/121:17:411
1290499595,33cyclictest0-21swapper/121:17:411
1290499592,33cyclictest0-21swapper/123:45:351
1290499591,6cyclictest26087-21/usr/sbin/munin22:30:151
1290499591,6cyclictest26087-21/usr/sbin/munin22:30:151
1290499591,5cyclictest21148-21sensors21:00:301
1290499591,5cyclictest20267-21wc21:00:051
12903995917,26cyclictest29780-21tail00:30:320
12903995917,26cyclictest29780-21tail00:30:320
1290399591,4cyclictest23637-21kworker/0:221:10:340
1290499584,41cyclictest0-21swapper/122:13:011
1290499582,45cyclictest7950irq/56-dwc_otg_22:42:271
1290499582,3cyclictest7850irq/56-dwc_otg00:04:181
1290499582,3cyclictest7850irq/56-dwc_otg00:04:181
12903995830,10cyclictest23698-21diskmemload21:10:040
12903995819,21cyclictest23357-21sh23:45:040
1290499572,7cyclictest8050irq/56-dwc_otg_00:34:011
1290499572,7cyclictest8050irq/56-dwc_otg_00:34:011
1290499572,6cyclictest22123-21/usr/sbin/munin21:05:341
1290499572,41cyclictest0-21swapper/123:22:131
1290499572,3cyclictest7850irq/56-dwc_otg19:30:051
1290499572,3cyclictest0-21swapper/120:05:271
1290499571,42cyclictest1921-21ssh23:19:071
1290499571,42cyclictest1921-21ssh23:19:071
12903995728,10cyclictest7850irq/56-dwc_otg22:00:330
12903995720,18cyclictest0-21swapper/019:10:260
1290499568,30cyclictest8050irq/56-dwc_otg_22:50:261
1290499564,36cyclictest8050irq/56-dwc_otg_20:20:051
1290499561,6cyclictest2367-21ssh00:36:581
1290499561,42cyclictest11861-21ssh21:33:541
1290499558,28cyclictest0-21swapper/123:14:061
12904995537,12cyclictest30271-21sh22:35:191
1290399553,30cyclictest0-21swapper/022:25:040
1290399551,14cyclictest13704-21wget22:15:180
27302540,2sleep20-21swapper/223:20:112
1290499541,3cyclictest7950irq/56-dwc_otg_21:51:261
1290399541,29cyclictest16473-21sh20:50:040
1290499536,37cyclictest7850irq/56-dwc_otg00:26:241
12903995329,5cyclictest7850irq/56-dwc_otg23:40:050
1290699526,7cyclictest7850irq/56-dwc_otg21:50:143
1290499521,8cyclictest13424-21grep20:40:201
1290499521,7cyclictest29385-21aten_rbpower_po19:55:171
1290499520,43cyclictest7950irq/56-dwc_otg_19:54:231
12903995230,7cyclictest8050irq/56-dwc_otg_22:35:190
1290399522,35cyclictest15147-21ssh23:35:060
1290399522,33cyclictest12070-21ssh00:10:180
124382523,14sleep10-21swapper/119:07:281
1290699515,21cyclictest9531-21ssh22:10:183
1290699513,10cyclictest7950irq/56-dwc_otg_19:35:173
1290599517,25cyclictest0-21swapper/223:45:192
1290499512,5cyclictest20258-21/usr/sbin/munin19:30:301
12904995113,35cyclictest9499-21apt-get20:30:151
1290399519,22cyclictest0-21swapper/020:40:220
1290399518,26cyclictest9-21ksoftirqd/023:10:220
12903995135,3cyclictest7950irq/56-dwc_otg_00:00:050
12903995123,12cyclictest16641-21aten_rbpower_po19:20:170
1290399511,9cyclictest14862-21aten_rbpower_vo19:15:180
12903995119,13cyclictest7950irq/56-dwc_otg_23:25:320
1290399511,23cyclictest368850ssh00:00:160
1290399511,23cyclictest368850ssh00:00:160
1290499502,7cyclictest8050irq/56-dwc_otg_22:02:591
1290499502,46cyclictest7462-21idleruntime-cro20:25:041
1290499501,5cyclictest18266-21cron20:55:041
1290399505,18cyclictest7950irq/56-dwc_otg_23:20:180
12903995032,4cyclictest8050irq/56-dwc_otg_21:25:340
12903995024,12cyclictest7950irq/56-dwc_otg_21:55:180
12903995023,7cyclictest0-21swapper/019:40:150
1290399502,29cyclictest4345-21fschecks_count20:15:200
12903995016,15cyclictest15202-21netstat22:55:270
1290399501,22cyclictest0-21swapper/020:20:150
1290699494,22cyclictest8050irq/56-dwc_otg_23:30:053
1290699492,11cyclictest0-21swapper/320:30:173
1290499491,7cyclictest25923-21fschecks_count19:45:201
1290499491,7cyclictest25923-21fschecks_count19:45:201
12903994932,3cyclictest7950irq/56-dwc_otg_19:35:330
1290699484,25cyclictest39-21ksoftirqd/319:55:143
1290699481,10cyclictest0-21swapper/320:00:163
1290499481,6cyclictest11364-21aten_rbpower_en20:35:191
1290499481,6cyclictest11364-21aten_rbpower_en20:35:191
12903994828,4cyclictest7950irq/56-dwc_otg_23:05:250
1290399482,4cyclictest0-21swapper/000:40:050
1290699472,7cyclictest0-21swapper/321:25:173
1290599477,29cyclictest11986-21runrttasks00:10:182
1290599477,25cyclictest0-21swapper/219:55:172
12905994735,7cyclictest13678-21sh22:15:172
12903994723,10cyclictest7950irq/56-dwc_otg_00:25:310
1290399472,28cyclictest13-21rcuc/000:20:260
1290399472,24cyclictest0-21swapper/021:00:150
12903994719,18cyclictest28589-21unixbench_multi19:50:310
1290699464,9cyclictest7950irq/56-dwc_otg_20:05:183
1290599467,25cyclictest0-21swapper/219:45:162
1290599467,25cyclictest0-21swapper/219:45:162
1290599467,25cyclictest0-21swapper/219:10:172
12905994629,6cyclictest23561-21ssh21:47:242
1290399462,36cyclictest24986-21kworker/0:221:53:310
12903994621,5cyclictest7950irq/56-dwc_otg_23:00:180
1290399461,28cyclictest11571-21taskset22:11:540
1290599458,23cyclictest0-21swapper/200:20:192
1290599457,24cyclictest0-21swapper/220:55:182
1290599457,23cyclictest0-21swapper/220:50:192
1290599456,24cyclictest0-21swapper/221:35:102
1290599456,24cyclictest0-21swapper/219:30:172
12903994522,14cyclictest18476-21wget19:25:180
12903994518,13cyclictest0-21swapper/020:10:140
1290599447,22cyclictest0-21swapper/219:25:172
12903994422,9cyclictest0-21swapper/020:52:430
12903994414,15cyclictest30156-21sh23:52:220
12903994413,21cyclictest28296-21acpi21:15:150
12903994413,21cyclictest28296-21acpi21:15:150
1290399441,26cyclictest0-21swapper/022:26:100
125912442,36sleep00-21swapper/019:09:300
1290699431,5cyclictest0-21swapper/320:15:163
1290599437,22cyclictest0-21swapper/219:50:162
1290599437,21cyclictest0-21swapper/219:20:162
1290599436,24cyclictest0-21swapper/222:50:182
1290399432,30cyclictest10855-21awk22:50:240
1290399431,27cyclictest0-21swapper/021:30:250
1290699427,24cyclictest0-21swapper/323:45:193
12906994216,11cyclictest8050irq/56-dwc_otg_21:15:203
12906994216,11cyclictest8050irq/56-dwc_otg_21:15:203
1290599423,4cyclictest7850irq/56-dwc_otg00:05:172
1290399424,25cyclictest9008-21kworker/0:020:33:360
12903994216,15cyclictest0-21swapper/020:27:200
12906994129,8cyclictest381ktimersoftd/321:10:163
12905994115,5cyclictest151822grep20:45:222
12903994125,10cyclictest101ktimersoftd/022:05:250
12903994120,10cyclictest0-21swapper/020:01:500
12903994116,13cyclictest0-21swapper/021:00:040
1290699408,16cyclictest0-21swapper/322:25:153
1290699407,20cyclictest0-21swapper/323:54:203
1290599407,23cyclictest0-21swapper/223:50:182
12905994027,6cyclictest9336-21ssh22:48:532
12905994017,6cyclictest7950irq/56-dwc_otg_21:20:152
1290599400,20cyclictest0-21swapper/223:40:182
12903994018,11cyclictest727-21runrttasks20:09:050
12903994016,11cyclictest30408-21grep19:55:310
1290699391,13cyclictest0-21swapper/319:30:173
12906993911,11cyclictest8050irq/56-dwc_otg_00:00:573
12906993911,11cyclictest8050irq/56-dwc_otg_00:00:573
1290399396,22cyclictest0-21swapper/019:45:160
1290399396,22cyclictest0-21swapper/019:45:160
1290699386,18cyclictest20668-21tune2fs21:00:233
1290699385,5cyclictest7950irq/56-dwc_otg_23:10:313
12906993821,4cyclictest8050irq/56-dwc_otg_22:05:293
1290599387,21cyclictest0-21swapper/200:35:182
1290699377,23cyclictest0-21swapper/322:40:103
1290699371,11cyclictest15656-21tune2fs23:35:233
12905993719,4cyclictest30753-21grep21:55:342
108702371,3sleep20-21swapper/222:10:392
1290699367,24cyclictest0-21swapper/323:05:193
1290699362,8cyclictest20095-21apt-config19:30:063
1290699361,4cyclictest7950irq/56-dwc_otg_22:55:163
12906993614,5cyclictest16828-21latency_hist21:40:043
12906993611,11cyclictest0-21swapper/321:47:003
1290599363,19cyclictest7850irq/56-dwc_otg00:25:332
1290599363,17cyclictest13279-21ssh23:31:582
12905993616,5cyclictest347-21/usr/sbin/munin23:55:302
1290599361,24cyclictest32095-21ls20:00:302
1290599361,24cyclictest23156-21ls21:05:322
1290599361,23cyclictest0-21swapper/220:15:212
1290699358,7cyclictest37-21rcuc/319:40:173
1290699358,4cyclictest0-21swapper/323:40:233
12906993528,5cyclictest39-21ksoftirqd/322:50:333
1290699352,21cyclictest8050irq/56-dwc_otg_23:20:233
1290699352,12cyclictest7950irq/56-dwc_otg_20:49:583
1290699351,20cyclictest9259-21latency_hist20:30:053
1290599358,14cyclictest0-21swapper/221:15:182
1290599358,14cyclictest0-21swapper/221:15:182
1290599352,22cyclictest7850irq/56-dwc_otg20:35:152
1290599352,22cyclictest7850irq/56-dwc_otg20:35:152
12905993516,5cyclictest1798-21cat22:00:242
1290599351,5cyclictest22464-21modprobe23:05:042
1290599351,23cyclictest15612-21grep23:35:222
1290699349,4cyclictest8050irq/56-dwc_otg_22:30:193
1290699349,4cyclictest8050irq/56-dwc_otg_22:30:193
1290699348,16cyclictest0-21swapper/300:15:193
1290699347,15cyclictest0-21swapper/320:20:163
1290699346,14cyclictest7850irq/56-dwc_otg00:25:243
1290699342,20cyclictest0-21swapper/322:04:383
1290699342,13cyclictest0-21swapper/320:45:053
1290699341,25cyclictest629-21ls23:55:333
1290699341,21cyclictest25846-21cpuspeed19:45:183
1290699341,21cyclictest25846-21cpuspeed19:45:183
1290599347,19cyclictest7950irq/56-dwc_otg_21:25:312
1290599344,16cyclictest0-21swapper/223:29:592
1290599343,25cyclictest8050irq/56-dwc_otg_23:15:162
1290599343,25cyclictest8050irq/56-dwc_otg_23:15:162
1290599342,21cyclictest24333-21ssh21:10:222
1290699335,6cyclictest7850irq/56-dwc_otg23:15:323
1290699335,6cyclictest7850irq/56-dwc_otg23:15:323
1290699333,5cyclictest0-21swapper/300:05:323
1290699331,6cyclictest7950irq/56-dwc_otg_19:10:213
1290699331,21cyclictest0-21swapper/321:30:173
1290699331,19cyclictest18373-21latency_hist20:55:053
1290699331,13cyclictest7850irq/56-dwc_otg00:31:143
1290699331,13cyclictest7850irq/56-dwc_otg00:31:143
12906993311,15cyclictest0-21swapper/322:39:223
1290599337,14cyclictest0-21swapper/220:20:162
1290599336,14cyclictest7950irq/56-dwc_otg_22:40:242
1290599331,7cyclictest29791-21grep00:30:322
1290599331,7cyclictest29791-21grep00:30:322
1290599331,7cyclictest2636450ssh22:30:212
1290599331,7cyclictest2636450ssh22:30:212
1290599331,6cyclictest29488-21ssh23:12:492
1290599331,21cyclictest14057-21timerandwakeup20:40:292
1290599331,19cyclictest11067-21latency_hist20:35:052
12905993311,8cyclictest31-21ksoftirqd/222:05:202
1290699329,7cyclictest7950irq/56-dwc_otg_22:20:373
1290699328,13cyclictest23698-21diskmemload21:22:333
1290699327,14cyclictest0-21swapper/321:05:323
1290699326,14cyclictest8050irq/56-dwc_otg_20:10:213
1290699323,4cyclictest7850irq/56-dwc_otg19:50:173
12906993210,5cyclictest8050irq/56-dwc_otg_20:35:243
12906993210,5cyclictest8050irq/56-dwc_otg_20:35:243
1290599327,15cyclictest0-21swapper/221:30:182
1290599326,23cyclictest0-21swapper/219:15:172
1290599326,11cyclictest7950irq/56-dwc_otg_21:45:052
1290599325,18cyclictest25369-21ssh21:50:162
119122326,13sleep20-21swapper/219:05:312
1290699319,10cyclictest8050irq/56-dwc_otg_00:10:273
1290699315,6cyclictest8050irq/56-dwc_otg_00:20:343
1290699312,18cyclictest0-21swapper/300:35:243
1290699311,20cyclictest19025-21ls23:00:213
12906993110,12cyclictest0-21swapper/322:50:003
1290599318,5cyclictest0-21swapper/223:05:212
1290599312,19cyclictest0-21swapper/222:29:072
1290599311,5cyclictest8050irq/56-dwc_otg_00:15:232
1290699303,24cyclictest13663-21ssh22:15:163
1290599303,5cyclictest7950irq/56-dwc_otg_21:00:162
1290599301,10cyclictest14725-21df_inode22:55:212
1290699291,4cyclictest0-21swapper/319:20:093
1290599295,15cyclictest7950irq/56-dwc_otg_20:07:202
1290599291,14cyclictest0-21swapper/220:14:452
1290599282,11cyclictest7850irq/56-dwc_otg20:25:212
123672288,12sleep30-21swapper/319:06:413
1290599263,5cyclictest7850irq/56-dwc_otg19:43:272
1290599256,9cyclictest7950irq/56-dwc_otg_19:35:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional