You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 17:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Mar 16, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29532991282,92cyclictest29524-21cyclictest07:10:070
574021220,2sleep10-21swapper/111:00:311
295329910413,70cyclictest14457-21ssh09:55:040
2953299952,79cyclictest0-21swapper/009:35:310
2953299922,69cyclictest7081-21sort09:05:220
2953299881,66cyclictest21096-21cut11:20:050
29532998816,37cyclictest0-21swapper/008:40:180
2953299877,65cyclictest0-21swapper/010:27:080
2953299872,69cyclictest0-21swapper/007:20:050
29532998720,57cyclictest26013-21resolvconf08:27:410
2953299861,5cyclictest4082-21fschecks_time10:20:220
2953299852,5cyclictest2590-21wc07:25:050
2953299852,59cyclictest4517-21aten_rbpower_cu07:30:170
29532998421,32cyclictest0-21swapper/011:55:160
2953299841,5cyclictest25452-21apt09:30:050
2953299841,5cyclictest25452-21apt09:30:050
2953299831,5cyclictest6380-21wget07:35:190
2953299827,56cyclictest8328-21diskmemload11:00:050
2953299821,5cyclictest9936-21munin-plugin-st12:25:040
2953299812,4cyclictest8803-21sshd07:40:290
2953299811,6cyclictest26480-21fschecks_time12:05:210
29532998113,57cyclictest30738-21sh10:51:380
2953299801,5cyclictest20386-21users10:00:330
2953299801,4cyclictest23831-21ssh11:22:530
2953299801,4cyclictest21096-21ssh12:37:480
220472800,4sleep022050-21unixbench_multi08:15:330
212132800,5sleep22954399cyclictest10:01:582
29532997810,52cyclictest13682-21ntp_kernel_err09:15:250
2953299771,7cyclictest28426-21ssh09:32:510
2953299771,6cyclictest26569-21aten_rbpower_vo08:30:180
2953299771,34cyclictest0-21swapper/008:45:180
178122770,2sleep20-21swapper/211:55:112
2953299762,5cyclictest917-21snmpd11:48:290
2953299757,25cyclictest0-21swapper/008:05:090
29532997538,8cyclictest7850irq/56-dwc_otg_12:25:160
2953299751,5cyclictest18650-21ssh10:37:150
2953299751,5cyclictest17423-21rm09:57:490
2953299742,4cyclictest0-21swapper/012:19:280
2953299741,5cyclictest9308-21irqstats11:05:240
2953299741,4cyclictest0-21swapper/008:35:310
2953299736,51cyclictest3285-21grep07:25:260
2953299732,61cyclictest0-21swapper/011:00:290
40862720,1sleep20-21swapper/211:38:122
2953299721,6cyclictest0-21swapper/009:43:460
2953299719,44cyclictest27765-21ssh10:48:480
2953299718,46cyclictest5124-21modprobe11:40:040
2953299712,5cyclictest0-21swapper/011:40:200
2953299712,48cyclictest0-21swapper/012:30:210
2953299711,5cyclictest0-21swapper/009:10:500
2953299711,4cyclictest0-21swapper/011:34:160
2953299711,4cyclictest0-21swapper/011:34:160
143462710,2sleep10-21swapper/109:54:531
29532997028,13cyclictest0-21swapper/011:50:170
2953299701,5cyclictest26707-21ssh10:09:070
2953299699,46cyclictest0-21swapper/011:26:100
2953299697,46cyclictest8328-21diskmemload11:11:090
2953299696,15cyclictest0-21swapper/010:15:170
2953299692,51cyclictest0-21swapper/010:14:490
2953299691,5cyclictest0-21swapper/010:40:290
2953299691,57cyclictest15726-21ssh10:34:290
2953299691,4cyclictest0-21swapper/012:11:010
2953299691,4cyclictest0-21swapper/009:49:220
29532996820,5cyclictest0-21swapper/012:00:170
29538996727,14cyclictest7750irq/56-dwc_otg07:10:071
2953299671,5cyclictest0-21swapper/009:21:520
2953299665,45cyclictest24314-21munin-plugin-st08:25:050
2953299661,5cyclictest19333-21df_inode08:10:220
29532996612,42cyclictest13134-21taskset07:53:300
29543996415,29cyclictest31-21ksoftirqd/207:10:082
2953299642,4cyclictest0-21swapper/008:57:090
2953299641,43cyclictest0-21swapper/008:00:320
2953299637,44cyclictest0-21swapper/009:00:190
2953299631,4cyclictest91750snmpd07:47:400
29532996211,30cyclictest14956-21taskset07:58:430
29532996211,30cyclictest14956-21taskset07:58:430
2953299611,45cyclictest2850-21chrt08:54:290
2953299611,45cyclictest2850-21chrt08:54:290
2953899581,26cyclictest7850irq/56-dwc_otg_09:05:161
156002570,2sleep10-21swapper/111:51:321
2955599543,8cyclictest0-21swapper/307:10:083
2953899546,7cyclictest7850irq/56-dwc_otg_09:30:171
2955599524,4cyclictest0-21swapper/310:35:163
2953899502,7cyclictest0-21swapper/110:00:161
2953899501,4cyclictest0-21swapper/107:20:151
59402490,19sleep27950irq/56-dwc_otg_09:00:312
2953899496,6cyclictest0-21swapper/109:55:151
2953899495,4cyclictest0-21swapper/111:45:171
2953899493,20cyclictest7950irq/56-dwc_otg_10:45:301
289202493,12sleep00-21swapper/007:06:410
2953899484,12cyclictest21237-21ssh11:20:161
2955599473,6cyclictest0-21swapper/309:55:173
2953899479,30cyclictest0-21swapper/108:40:181
2953899477,26cyclictest0-21swapper/107:50:171
2953899473,9cyclictest6128-21ssh12:20:161
17272470,2sleep30-21swapper/310:17:133
2955599461,37cyclictest0-21swapper/308:50:053
29543994628,9cyclictest7950irq/56-dwc_otg_09:50:212
2953899465,7cyclictest7850irq/56-dwc_otg_09:25:151
2953899465,7cyclictest7850irq/56-dwc_otg_09:25:151
2955599457,25cyclictest0-21swapper/310:25:183
2954399457,22cyclictest0-21swapper/207:50:172
2953899457,25cyclictest0-21swapper/108:55:171
260012450,3sleep337-21rcuc/311:25:253
2955599447,24cyclictest0-21swapper/308:50:183
2955599447,24cyclictest0-21swapper/308:50:183
2955599431,25cyclictest1998-21ssh12:15:183
29543994328,9cyclictest7850irq/56-dwc_otg_12:35:162
24542430,1sleep10-21swapper/110:18:281
2955599427,24cyclictest0-21swapper/308:35:173
29543994236,4cyclictest301ktimersoftd/207:55:162
29543994236,4cyclictest301ktimersoftd/207:55:162
2953899421,5cyclictest0-21swapper/111:25:161
227552420,3sleep10-21swapper/110:42:111
2955599411,4cyclictest7850irq/56-dwc_otg_11:05:163
2954399417,21cyclictest0-21swapper/207:45:192
2954399411,6cyclictest0-21swapper/212:12:292
29543994113,4cyclictest7750irq/56-dwc_otg12:05:152
2953899417,25cyclictest0-21swapper/107:45:191
2953899413,7cyclictest7850irq/56-dwc_otg_10:55:151
29538994122,4cyclictest27429-21unixbench_singl12:05:321
29538994120,5cyclictest4288-21sh09:42:251
29538994119,4cyclictest877650wget09:10:181
29538994118,5cyclictest24421-21latency_hist08:25:051
29555994019,5cyclictest7850irq/56-dwc_otg_07:20:183
2954399405,9cyclictest7950irq/56-dwc_otg_11:45:172
2954399401,31cyclictest0-21swapper/208:00:232
2953899407,20cyclictest7850irq/56-dwc_otg_09:20:051
2953899407,18cyclictest7950irq/56-dwc_otg_12:30:051
2953899406,30cyclictest0-21swapper/108:10:091
29538994020,5cyclictest12321-21aten_rbpower_en10:30:181
29555993912,15cyclictest0-21swapper/309:30:153
29543993914,15cyclictest0-21swapper/210:33:142
2953899392,9cyclictest7850irq/56-dwc_otg_11:10:171
29538993919,5cyclictest30261-21sh11:30:261
29538993919,5cyclictest30261-21sh11:30:261
29538993918,4cyclictest21626-21sh08:15:281
29538993917,4cyclictest8328-21diskmemload12:35:251
29538993911,12cyclictest0-21swapper/111:09:301
29538993910,12cyclictest0-21swapper/109:40:051
2954399381,8cyclictest1905-21sh09:40:172
2953899387,23cyclictest0-21swapper/112:30:161
2953899387,23cyclictest0-21swapper/107:55:181
2953899387,23cyclictest0-21swapper/107:55:181
2953899383,23cyclictest7850irq/56-dwc_otg_07:30:141
292602386,19sleep129393-21gpgv07:10:061
2954399379,16cyclictest28211-21ssh10:10:252
2954399376,21cyclictest8328-21diskmemload11:18:002
29543993720,11cyclictest0-21swapper/209:17:162
29543993716,6cyclictest1573-21idleruntime-cro09:40:042
2953899378,6cyclictest23-21ksoftirqd/107:30:041
2953899377,14cyclictest7950irq/56-dwc_otg_07:35:331
2953899375,24cyclictest27619-21ssh10:10:181
29538993717,5cyclictest27321-21latency_hist10:10:051
69152362,29sleep26282-21/usr/sbin/munin07:35:272
2955599368,15cyclictest0-21swapper/308:10:103
2955599361,23cyclictest7660-21latency_hist10:25:053
2954399366,5cyclictest7750irq/56-dwc_otg11:25:332
2954399363,25cyclictest0-21swapper/211:21:592
29543993615,5cyclictest8696-21sh09:47:272
29543993614,5cyclictest4714-21sh09:00:042
2954399361,24cyclictest30741-21missed_timers08:40:262
2953899366,22cyclictest17873-21ssh11:55:161
2953899363,7cyclictest7750irq/56-dwc_otg11:35:171
29538993611,4cyclictest0-21swapper/112:01:461
2955599356,8cyclictest11-21rcu_preempt09:45:273
2955599355,14cyclictest7950irq/56-dwc_otg_11:45:223
2955599352,14cyclictest7750irq/56-dwc_otg09:10:053
29555993514,11cyclictest1037-21cat11:35:053
29555993514,11cyclictest1037-21cat11:35:053
29555993512,4cyclictest28670-21/usr/sbin/munin10:10:323
2954399358,14cyclictest2776-21grep07:25:192
29543993512,4cyclictest7750irq/56-dwc_otg11:08:372
2954399351,23cyclictest0-21swapper/211:53:122
2953899355,19cyclictest6191-21ssh09:45:101
2953899355,16cyclictest23-21ksoftirqd/110:25:321
29538993510,4cyclictest7750irq/56-dwc_otg12:10:261
243322351,4sleep324342-21ssh11:23:493
2955599347,15cyclictest0-21swapper/310:45:173
2955599345,20cyclictest0-21swapper/311:50:173
2955599344,4cyclictest7850irq/56-dwc_otg_10:05:163
2955599343,21cyclictest0-21swapper/308:03:033
29555993421,11cyclictest7950irq/56-dwc_otg_11:02:173
29555993418,5cyclictest1035550wget12:25:173
29555993415,4cyclictest17262-21ssh09:20:183
2955599340,21cyclictest0-21swapper/312:35:183
2954399342,21cyclictest0-21swapper/207:30:262
29543993415,5cyclictest908-21runrttasks12:25:292
29543993413,12cyclictest31-21ksoftirqd/208:25:052
2954399341,19cyclictest0-21swapper/210:25:162
2954399341,18cyclictest0-21swapper/212:19:532
2953899346,13cyclictest0-21swapper/111:15:151
2953899344,16cyclictest7950irq/56-dwc_otg_08:05:061
2953899341,17cyclictest27413-21timerandwakeup08:30:301
290872345,20sleep2917-21snmpd07:08:462
2955599338,12cyclictest0-21swapper/310:30:263
2955599335,19cyclictest8692-21ssh09:10:173
2955599333,14cyclictest7750irq/56-dwc_otg08:30:213
2955599331,7cyclictest1668-21latency_hist12:15:043
29555993317,6cyclictest2312550ssh09:26:043
29555993317,6cyclictest2312550ssh09:26:043
2955599331,3cyclictest0-21swapper/309:35:183
2954399338,7cyclictest31-21ksoftirqd/210:20:322
2954399332,6cyclictest0-21swapper/209:55:222
2954399332,14cyclictest0-21swapper/210:46:482
2954399331,22cyclictest0-21swapper/209:12:422
2954399331,17cyclictest0-21swapper/208:50:112
2954399331,17cyclictest0-21swapper/208:50:112
2953899339,4cyclictest0-21swapper/109:24:221
2953899332,8cyclictest23-21ksoftirqd/110:20:341
29538993319,4cyclictest16943-21apt-config08:05:041
29538993310,4cyclictest0-21swapper/110:54:411
2955599325,16cyclictest7850irq/56-dwc_otg_11:55:303
2955599325,15cyclictest7950irq/56-dwc_otg_08:05:283
2955599324,16cyclictest7850irq/56-dwc_otg_09:55:043
2955599322,4cyclictest19369-21fschecks_count10:00:213
29555993214,5cyclictest14599-21tail07:55:323
29555993214,5cyclictest14599-21tail07:55:323
29555993213,4cyclictest26683-21latency12:05:243
2955599321,28cyclictest14719-21cut12:30:203
2955599321,20cyclictest2182-21grep09:40:223
2955599321,14cyclictest7750irq/56-dwc_otg12:23:433
2954399329,7cyclictest9450irq/48-DMA09:12:252
2954399329,15cyclictest149-21systemd-journal08:30:272
2954399328,11cyclictest32548-21wc08:45:262
2954399324,19cyclictest28805-21ssh10:50:152
2954399323,5cyclictest0-21swapper/209:20:282
29543993221,4cyclictest30450-21sshd11:30:292
29543993221,4cyclictest30450-21sshd11:30:292
29543993220,5cyclictest20914-21/usr/sbin/munin08:15:322
29543993218,7cyclictest7750irq/56-dwc_otg11:04:202
29543993218,5cyclictest301ktimersoftd/211:44:532
29543993212,8cyclictest24425-21latency_hist10:45:052
2954399321,11cyclictest908-21runrttasks12:23:152
29543993211,15cyclictest0-21swapper/212:30:342
2953899329,10cyclictest31579-21cat07:15:191
2953899328,6cyclictest7850irq/56-dwc_otg_12:15:251
2953899327,15cyclictest0-21swapper/108:25:111
2953899326,16cyclictest0-21swapper/110:35:161
2953899325,14cyclictest7950irq/56-dwc_otg_09:05:041
2953899324,6cyclictest7850irq/56-dwc_otg_08:35:181
2953899321,11cyclictest0-21swapper/111:43:551
145112320,1sleep30-21swapper/309:15:583
2955599317,14cyclictest0-21swapper/310:50:193
29555993113,5cyclictest256250ssh10:56:523
2955599311,19cyclictest0-21swapper/307:30:163
2955599311,13cyclictest7750irq/56-dwc_otg07:15:323
2954399317,4cyclictest0-21swapper/212:00:152
2954399317,21cyclictest0-21swapper/208:05:092
29543993112,5cyclictest301ktimersoftd/210:35:162
2954399311,20cyclictest23877-21ssh10:05:252
2954399311,14cyclictest0-21swapper/207:19:212
2954399311,13cyclictest7950irq/56-dwc_otg_09:08:212
2954399311,13cyclictest0-21swapper/208:26:092
29538993110,6cyclictest149-21systemd-journal08:51:291
29538993110,6cyclictest149-21systemd-journal08:51:291
2955599307,14cyclictest0-21swapper/311:40:193
2955599305,6cyclictest5921-21kworker/3:207:50:213
29555993023,4cyclictest5136-21idleruntime-cro11:40:043
2955599301,4cyclictest20947-21ssh11:20:043
2955599301,21cyclictest0-21swapper/307:49:313
2955599301,18cyclictest5910-21timerandwakeup09:00:313
2955599301,14cyclictest0-21swapper/311:10:193
2954399307,9cyclictest0-21swapper/210:20:052
2954399306,16cyclictest7950irq/56-dwc_otg_10:55:232
2954399303,13cyclictest7950irq/56-dwc_otg_08:10:292
29543993016,4cyclictest14636-21sh11:11:152
2954399301,11cyclictest0-21swapper/207:42:532
227752300,2sleep30-21swapper/310:42:133
2955599296,7cyclictest7950irq/56-dwc_otg_08:55:153
2955599296,6cyclictest0-21swapper/308:15:263
2955599295,5cyclictest0-21swapper/308:40:313
2955599292,18cyclictest0-21swapper/307:37:073
2955599292,15cyclictest0-21swapper/308:25:333
29555992919,7cyclictest22766-21ssh12:00:273
2954399299,5cyclictest7850irq/56-dwc_otg_09:25:312
2954399299,5cyclictest7850irq/56-dwc_otg_09:25:312
2954399297,20cyclictest0-21swapper/207:20:172
2955599286,11cyclictest7950irq/56-dwc_otg_07:41:463
2955599281,14cyclictest7950irq/56-dwc_otg_07:25:233
2953899281,11cyclictest7950irq/56-dwc_otg_07:40:231
29555992720,5cyclictest24308-21apt-get08:25:053
2954399271,15cyclictest7950irq/56-dwc_otg_08:35:182
2953899275,5cyclictest7950irq/56-dwc_otg_08:45:271
289182264,15sleep30-21swapper/307:06:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional