You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 21:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri May 15, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
279189911123,28cyclictest0-21swapper/016:31:020
27919991025,79cyclictest0-21swapper/116:31:011
2791899986,48cyclictest0-21swapper/016:41:130
35412840,3sleep23539-21grep20:21:252
27919998211,53cyclictest0-21swapper/119:06:201
263482810,14sleep17950irq/56-dwc_otg_21:28:231
250552800,2sleep22792099cyclictest17:51:172
2792199795,19cyclictest7950irq/56-dwc_otg_20:21:123
159782790,2sleep30-21swapper/319:19:353
86542770,5sleep02791899cyclictest18:32:290
46032730,5sleep22792099cyclictest21:01:312
27920997321,11cyclictest7750irq/56-dwc_otg19:41:112
27918997335,8cyclictest9-21ksoftirqd/019:36:090
2792199724,19cyclictest0-21swapper/316:31:023
114492720,2sleep10-21swapper/119:51:371
129892690,2sleep10-21swapper/118:37:381
27919996812,4cyclictest7750irq/56-dwc_otg20:51:001
2791899682,31cyclictest0-21swapper/021:31:110
2791999677,36cyclictest0-21swapper/116:36:141
2791899673,53cyclictest1213-21chrt18:16:110
2791899673,53cyclictest1213-21chrt18:16:110
2791899671,38cyclictest17528-21fschecks_count19:21:170
27918996538,7cyclictest7850irq/56-dwc_otg_20:31:090
105002650,2sleep30-21swapper/321:48:143
81382640,2sleep10-21swapper/121:46:031
2791999644,6cyclictest7950irq/56-dwc_otg_18:26:261
27918996423,17cyclictest0-21swapper/020:16:100
2791999637,6cyclictest0-21swapper/121:11:101
2791999637,4cyclictest0-21swapper/120:01:521
2791999637,4cyclictest0-21swapper/120:01:521
2791899632,46cyclictest7620-21grep17:01:260
2791899631,38cyclictest24532-21grep20:46:290
2791899631,35cyclictest21101-21aten_rbpower_po17:41:130
2791899631,32cyclictest17711-21fschecks_time17:31:150
14362630,2sleep20-21swapper/221:36:502
104172630,1sleep20-21swapper/220:30:512
2791899623,34cyclictest11-21rcu_preempt18:25:590
2791899621,43cyclictest15965-21kworker/0:120:36:140
2791899621,36cyclictest22853-21tune2fs20:06:170
2791899621,32cyclictest0-21swapper/021:36:130
2791999618,3cyclictest0-21swapper/119:24:231
2791999616,4cyclictest7750irq/56-dwc_otg20:14:071
2791999613,9cyclictest5705-21ssh21:42:011
2791899616,37cyclictest0-21swapper/019:51:100
27918996113,7cyclictest2747-21latency_hist16:50:590
27918996113,7cyclictest2747-21latency_hist16:50:590
2791899608,30cyclictest0-21swapper/018:01:160
2791899603,38cyclictest0-21swapper/019:41:270
2791999592,47cyclictest7950irq/56-dwc_otg_17:41:281
2791999592,12cyclictest30866-21ssh20:16:111
27919995919,37cyclictest13530-21sh17:20:591
2791899592,44cyclictest4740-21aten_rbpower_po16:56:130
2791899592,39cyclictest3539-21fschecks_count21:01:160
2791899591,30cyclictest0-21swapper/021:26:120
2791999587,20cyclictest7750irq/56-dwc_otg19:01:121
2791999581,3cyclictest7750irq/56-dwc_otg16:41:281
27918995814,22cyclictest32512-21ntp_kernel_err18:11:210
164362580,2sleep20-21swapper/220:36:432
2791999572,8cyclictest7950irq/56-dwc_otg_20:31:241
2791999571,7cyclictest30013-21ssh19:36:091
2791999571,5cyclictest7750irq/56-dwc_otg20:51:191
2791999571,5cyclictest0-21swapper/118:56:021
2791999571,5cyclictest0-21swapper/118:56:021
2791999571,3cyclictest0-21swapper/121:36:111
2791899573,33cyclictest0-21swapper/021:06:110
2791899572,35cyclictest4190-21ssh21:41:150
2791899571,32cyclictest28089-21runrttasks18:56:050
2791899571,32cyclictest28089-21runrttasks18:56:050
2791999564,4cyclictest7950irq/56-dwc_otg_20:36:431
2791999564,4cyclictest21710-21sh21:22:301
2791999564,3cyclictest0-21swapper/120:22:581
2791999562,5cyclictest30660-21rm21:33:301
2791999561,8cyclictest7750irq/56-dwc_otg18:34:341
2791999561,3cyclictest0-21swapper/119:56:111
2791999561,13cyclictest0-21swapper/118:21:161
2791899562,9cyclictest308-21sed20:56:270
2791899561,40cyclictest11317-21fschecks_count18:36:170
27918995613,4cyclictest8463-21seq21:46:140
2791999557,4cyclictest0-21swapper/121:54:221
2791999557,10cyclictest7950irq/56-dwc_otg_20:31:001
2791899554,32cyclictest0-21swapper/018:51:270
27918995521,15cyclictest9727-21idleruntime-cro19:50:590
2791999542,12cyclictest0-21swapper/119:43:051
2791999541,5cyclictest7750irq/56-dwc_otg16:46:181
2791999541,5cyclictest7750irq/56-dwc_otg16:46:181
2791999541,4cyclictest7750irq/56-dwc_otg21:57:001
2791999541,4cyclictest7750irq/56-dwc_otg16:51:271
2791999541,4cyclictest0-21swapper/117:04:261
27918995425,12cyclictest7950irq/56-dwc_otg_17:51:160
27918995424,14cyclictest13090-21ntpq21:51:200
2791899542,37cyclictest0-21swapper/019:26:240
27918995421,19cyclictest12848-21/usr/sbin/munin19:16:130
2792099536,30cyclictest28248-21ssh21:31:122
2791999536,38cyclictest0-21swapper/120:43:231
2791999532,4cyclictest7750irq/56-dwc_otg19:16:221
2791999532,44cyclictest0-21swapper/120:59:341
2791999532,12cyclictest0-21swapper/119:50:281
2791999531,8cyclictest7850irq/56-dwc_otg_18:51:161
2791999531,3cyclictest0-21swapper/121:01:271
2791999531,10cyclictest7850irq/56-dwc_otg_17:35:591
2791899531,6cyclictest4248-21ssh19:06:110
2791899531,26cyclictest26523-21awk19:31:220
2791899531,26cyclictest26523-21awk19:31:220
65542520,2sleep30-21swapper/319:46:203
2791999521,3cyclictest7750irq/56-dwc_otg18:45:461
27919995211,3cyclictest19998-21if_eth018:46:171
27919995211,3cyclictest19998-21if_eth018:46:171
2791999521,12cyclictest0-21swapper/118:07:541
2791899521,39cyclictest14933-21grep19:56:220
2792099513,30cyclictest0-21swapper/216:31:012
2791999516,4cyclictest7850irq/56-dwc_otg_19:12:511
2791999512,10cyclictest7950irq/56-dwc_otg_19:27:501
2791999511,14cyclictest0-21swapper/121:08:211
2791899511,35cyclictest8730-21sh19:11:130
2791899511,35cyclictest17323-21ps21:56:220
2791899511,35cyclictest10639-21cat17:11:200
2791899511,30cyclictest0-21swapper/020:01:210
2791899511,30cyclictest0-21swapper/020:01:210
2791899511,29cyclictest0-21swapper/016:55:590
2791899511,26cyclictest0-21swapper/019:01:100
27918995111,19cyclictest14232-21/usr/sbin/munin17:21:180
276002513,42sleep20-21swapper/216:30:232
27920995024,17cyclictest31-21ksoftirqd/217:26:102
27920995024,17cyclictest31-21ksoftirqd/217:26:102
2791999506,8cyclictest7950irq/56-dwc_otg_17:22:481
2791999503,6cyclictest2783-21idleruntime-cro18:21:001
2791999503,6cyclictest2783-21idleruntime-cro18:21:001
2791999501,5cyclictest23551-21/usr/sbin/munin17:46:241
2791999501,3cyclictest0-21swapper/121:16:281
2791999501,3cyclictest0-21swapper/121:16:281
27918995027,5cyclictest7750irq/56-dwc_otg18:46:110
27918995027,5cyclictest7750irq/56-dwc_otg18:46:110
2791899501,34cyclictest0-21swapper/017:17:080
27920994922,4cyclictest22499-21aten_rbpower_cu20:06:132
2791999491,4cyclictest7750irq/56-dwc_otg19:33:161
2791999491,4cyclictest7750irq/56-dwc_otg19:33:161
2791999491,4cyclictest24563-21ssh20:07:541
2791999491,4cyclictest0-21swapper/117:54:381
2791999491,3cyclictest7950irq/56-dwc_otg_17:36:121
2791899492,25cyclictest0-21swapper/020:41:220
2791899491,9cyclictest15370-21ssh21:15:590
2791899491,34cyclictest0-21swapper/020:21:220
27918994823,2cyclictest0-21swapper/018:41:130
2791899481,33cyclictest0-21swapper/021:16:240
2791899481,33cyclictest0-21swapper/021:16:240
2791899481,16cyclictest0-21swapper/020:12:230
2792199477,25cyclictest0-21swapper/317:26:123
2792199477,25cyclictest0-21swapper/317:26:123
2792199477,25cyclictest0-21swapper/317:21:123
2792099473,7cyclictest0-21swapper/218:31:092
2791999472,4cyclictest0-21swapper/117:10:011
2791899479,20cyclictest25577-21systemd-journal17:26:010
2791899479,20cyclictest25577-21systemd-journal17:26:010
2791899472,26cyclictest0-21swapper/021:21:500
2792099469,7cyclictest7950irq/56-dwc_otg_21:11:122
2792099467,26cyclictest0-21swapper/217:56:122
2792099467,25cyclictest0-21swapper/217:16:042
2792099467,25cyclictest0-21swapper/217:06:132
2791999466,26cyclictest0-21swapper/118:11:121
27920994537,5cyclictest6392-21diskmemload19:11:092
2791899451,24cyclictest23142-21fschecks_time17:46:170
273592453,35sleep00-21swapper/016:27:220
2792199446,24cyclictest0-21swapper/318:11:123
2792199443,5cyclictest28241-21ssh21:31:113
2792099447,10cyclictest0-21swapper/218:56:122
2792099447,10cyclictest0-21swapper/218:56:122
2791999449,4cyclictest221ktimersoftd/116:56:171
2791999441,12cyclictest0-21swapper/117:59:361
2791899441,26cyclictest0-21swapper/018:30:190
2791899441,24cyclictest7609-21/usr/sbin/munin20:26:240
2792199437,24cyclictest0-21swapper/318:16:113
2792199437,24cyclictest0-21swapper/318:16:113
2792199434,8cyclictest7950irq/56-dwc_otg_20:31:093
2792199433,9cyclictest7950irq/56-dwc_otg_17:46:173
2792099437,24cyclictest0-21swapper/218:26:142
27919994311,5cyclictest10768-21ntp_states17:11:221
27919994310,4cyclictest29292-21unixbench-2d18:01:271
2791899437,20cyclictest0-21swapper/017:06:250
2791899433,37cyclictest101ktimersoftd/018:07:260
2791899432,8cyclictest20340-21sed17:36:280
27918994315,15cyclictest0-21swapper/016:36:270
2791899431,13cyclictest27941-21irqstats20:51:190
27921994220,17cyclictest0-21swapper/318:51:103
2792099427,23cyclictest0-21swapper/218:01:122
2792099425,24cyclictest0-21swapper/217:11:122
2792099422,6cyclictest7850irq/56-dwc_otg_17:36:132
27920994226,13cyclictest7950irq/56-dwc_otg_18:41:122
276442427,22sleep127672-21munin-run16:30:591
269252420,4sleep2301ktimersoftd/220:50:442
294412410,3sleep327589-1kworker/u9:018:56:283
294412410,3sleep327589-1kworker/u9:018:56:283
2792199411,21cyclictest0-21swapper/319:35:223
2792199411,21cyclictest0-21swapper/319:35:223
2792099417,22cyclictest0-21swapper/220:11:042
27920994121,6cyclictest31-21ksoftirqd/219:36:142
2792099411,7cyclictest7950irq/56-dwc_otg_17:01:102
2791899412,24cyclictest0-21swapper/017:59:590
2792199407,22cyclictest0-21swapper/319:01:473
2792199406,24cyclictest0-21swapper/320:51:133
27920994015,5cyclictest27685-21fschecks_count20:51:162
275692400,31sleep30-21swapper/316:30:003
2792199393,6cyclictest0-21swapper/321:36:133
27921993920,9cyclictest19873-21ssh20:41:213
2792199391,28cyclictest32099-21ssh20:56:193
2792099397,23cyclictest0-21swapper/221:41:142
2792099396,7cyclictest29-21rcuc/220:16:272
144392390,8sleep27950irq/56-dwc_otg_21:52:522
34932380,2sleep30-21swapper/319:42:133
2792199385,19cyclictest23970-21ssh21:26:103
27921993823,6cyclictest5014-21ssh19:06:223
27921993813,14cyclictest0-21swapper/320:26:183
2792199381,21cyclictest7950irq/56-dwc_otg_16:36:273
2792099387,22cyclictest0-21swapper/218:06:132
2792099385,6cyclictest7750irq/56-dwc_otg21:50:592
2792099384,14cyclictest0-21swapper/218:36:132
2792099381,4cyclictest0-21swapper/216:36:142
27920993810,4cyclictest0-21swapper/221:21:232
2791999383,3cyclictest7750irq/56-dwc_otg17:27:511
2791999383,3cyclictest7750irq/56-dwc_otg17:27:511
27921993722,5cyclictest312-21sed16:41:283
2792199371,26cyclictest1273-21cut16:46:163
2792199371,26cyclictest1273-21cut16:46:163
2792099377,3cyclictest0-21swapper/219:29:282
2792099377,17cyclictest7950irq/56-dwc_otg_18:51:162
2792099373,16cyclictest26088-21ssh19:31:172
2792099373,16cyclictest26088-21ssh19:31:172
2792099371,29cyclictest1148-21ls18:16:002
2792099371,24cyclictest2918-21latency_hist21:01:002
2792099371,24cyclictest22209-21ls20:06:002
2792099371,24cyclictest22209-21ls20:06:002
2792099371,24cyclictest0-21swapper/217:46:172
27920993710,12cyclictest0-21swapper/220:31:102
2792199366,24cyclictest0-21swapper/320:46:143
2792199365,18cyclictest7850irq/56-dwc_otg_19:25:503
2792099367,6cyclictest7850irq/56-dwc_otg_19:01:232
2792099366,6cyclictest7950irq/56-dwc_otg_19:21:002
2792099366,21cyclictest10041-21ssh19:51:102
27920993611,14cyclictest0-21swapper/219:48:122
2792199352,7cyclictest7850irq/56-dwc_otg_20:16:123
27921993524,8cyclictest22562-21cron17:45:593
2792199352,23cyclictest0-21swapper/318:21:133
27920993515,5cyclictest31-21ksoftirqd/219:56:202
2792099351,25cyclictest3187-21ls18:21:142
27920993511,5cyclictest31-21ksoftirqd/218:51:002
27920993511,5cyclictest31-21ksoftirqd/218:51:002
27920993511,4cyclictest0-21swapper/220:41:172
27921993422,5cyclictest27395-21rm20:11:193
27921993421,10cyclictest7950irq/56-dwc_otg_21:41:303
2792099348,10cyclictest31-21ksoftirqd/216:46:122
2792099348,10cyclictest31-21ksoftirqd/216:46:122
2792099347,5cyclictest7850irq/56-dwc_otg_16:41:282
27920993413,5cyclictest29-21rcuc/219:10:202
27920993412,5cyclictest19563-21sh21:59:402
2792199335,17cyclictest7950irq/56-dwc_otg_19:56:223
2792199335,17cyclictest7850irq/56-dwc_otg_18:30:183
27921993321,5cyclictest20225-21rm20:02:313
27921993321,5cyclictest20225-21rm20:02:313
2792199331,4cyclictest8476-21/usr/sbin/munin19:11:203
27921993313,5cyclictest6392-21diskmemload18:38:263
2792199331,18cyclictest7750irq/56-dwc_otg21:11:193
2792099335,11cyclictest7850irq/56-dwc_otg_21:16:212
2792099335,11cyclictest7850irq/56-dwc_otg_21:16:212
2792099331,4cyclictest7750irq/56-dwc_otg19:21:192
95592320,1sleep30-21swapper/321:08:003
2792199329,13cyclictest25577-21systemd-journal19:54:483
2792199328,5cyclictest0-21swapper/318:46:113
2792199328,5cyclictest0-21swapper/318:46:113
2792199327,15cyclictest0-21swapper/321:21:133
27921993217,4cyclictest25577-21systemd-journal21:20:063
27921993217,4cyclictest25577-21systemd-journal21:20:063
2792199321,23cyclictest15602-21cut18:41:183
27921993210,4cyclictest29852-21cat18:06:003
2792099324,6cyclictest29-21rcuc/216:51:222
2792099322,13cyclictest2489350ssh21:26:252
2792199317,4cyclictest0-21swapper/321:03:433
2792199312,6cyclictest15045-21aten_rbpower_en20:36:143
2792199311,6cyclictest1647250ssh21:56:093
2792199311,22cyclictest11860-21apt-config17:16:003
27921993112,14cyclictest0-21swapper/321:51:093
2792099318,5cyclictest31-21ksoftirqd/221:10:592
2792099313,7cyclictest31-21ksoftirqd/216:56:272
36412300,2sleep30-21swapper/316:51:253
2792199308,11cyclictest0-21swapper/318:31:113
2792199306,15cyclictest0-21swapper/320:06:133
2792199306,14cyclictest0-21swapper/317:20:063
2792199303,19cyclictest0-21swapper/317:33:503
2792199302,7cyclictest30308-21sort19:36:163
27921993011,4cyclictest0-21swapper/319:30:503
27920993012,9cyclictest31-21ksoftirqd/217:41:152
2792099301,24cyclictest14091-21grep17:21:162
2792199294,4cyclictest0-21swapper/317:36:173
2792199293,6cyclictest4711-21aten_rbpower_en16:56:133
2792199292,6cyclictest0-21swapper/317:54:233
2792199292,17cyclictest0-21swapper/317:01:123
2792199292,13cyclictest30728-21tr18:06:223
2792099297,5cyclictest7950irq/56-dwc_otg_18:16:252
2792099297,5cyclictest7950irq/56-dwc_otg_18:16:252
2792199282,20cyclictest7850irq/56-dwc_otg_17:56:203
27920992816,5cyclictest18941-21cron17:35:592
2792199272,13cyclictest7950irq/56-dwc_otg_17:06:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional