You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 17:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Mon May 25, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
54102970,2sleep3381ktimersoftd/308:50:043
1088999238,9cyclictest0-21swapper/009:00:140
156162850,9sleep2301ktimersoftd/211:47:452
1088998415,14cyclictest0-21swapper/009:15:170
1088998415,14cyclictest0-21swapper/008:30:150
108899821,5cyclictest0-21swapper/012:15:150
1088998210,6cyclictest0-21swapper/010:25:140
108899819,57cyclictest18440-21unixbench_multi11:50:320
108899819,57cyclictest18440-21unixbench_multi11:50:320
108899814,30cyclictest29262-21kworker/u8:212:35:160
108899814,30cyclictest29262-21kworker/u8:212:35:160
108899804,19cyclictest14560-21ssh09:50:180
1088998035,33cyclictest0-21swapper/007:15:090
108899802,34cyclictest0-21swapper/012:10:160
108899801,35cyclictest0-21swapper/011:00:150
86762790,2sleep30-21swapper/307:30:143
322422790,5sleep10-21swapper/107:05:221
112252760,2sleep20-21swapper/211:42:182
113302730,4sleep1108999cyclictest11:42:291
1088997310,46cyclictest21586-21wget09:20:190
108899721,51cyclictest7490-21wget08:55:170
148162700,2sleep20-21swapper/211:07:272
108899709,44cyclictest20234-21cut10:35:210
76332690,2sleep20-21swapper/210:59:382
108899691,34cyclictest0-21swapper/009:25:150
1088996853,6cyclictest0-21swapper/008:15:110
1088996853,6cyclictest0-21swapper/008:15:110
108899676,36cyclictest0-21swapper/011:35:160
108899676,15cyclictest0-21swapper/010:15:160
109099666,5cyclictest28072-21ssh10:05:272
1088996635,9cyclictest0-21swapper/011:25:240
1088996432,14cyclictest10156-21grep11:40:320
1088996415,10cyclictest0-21swapper/007:25:150
108899635,28cyclictest7577-21cut10:20:180
108899634,40cyclictest18582-21ssh12:30:240
108899631,36cyclictest27144-21wget08:20:150
61932620,2sleep16190-21ssh12:15:261
108899628,50cyclictest101ktimersoftd/010:10:280
108899623,34cyclictest6129-21wget09:40:170
108899621,34cyclictest953-21sshd11:30:200
1088996131,7cyclictest6034-21sed07:20:310
108899612,43cyclictest11769-21seq09:45:490
201222600,3sleep00-21swapper/011:14:380
108899603,13cyclictest0-21swapper/007:35:140
1088996030,4cyclictest0-21swapper/009:05:150
1088996030,4cyclictest0-21swapper/009:05:150
108899596,47cyclictest0-21swapper/010:45:160
1088995935,5cyclictest7950irq/56-dwc_otg_12:00:030
108899585,32cyclictest22283-21ssh09:59:210
1088995838,4cyclictest8050irq/56-dwc_otg_09:10:160
1088995831,5cyclictest8050irq/56-dwc_otg_10:05:240
1088995822,16cyclictest7850irq/56-dwc_otg08:10:310
1088995822,16cyclictest7850irq/56-dwc_otg08:10:310
1088995736,8cyclictest0-21swapper/011:48:060
1088995733,5cyclictest7950irq/56-dwc_otg_11:15:170
1088995733,14cyclictest32585-21cpuspeed10:50:180
1088995730,12cyclictest18188-21sh10:32:370
1088995715,24cyclictest0-21swapper/012:25:160
108899571,31cyclictest16358-21fschecks_count07:50:220
138362560,2sleep10-21swapper/110:27:121
109099561,20cyclictest396-21ssh11:30:032
1088995626,6cyclictest21289-21latency_hist08:05:040
108899562,5cyclictest4064-21ssh10:55:140
108899559,42cyclictest101ktimersoftd/012:04:380
1088995527,6cyclictest7850irq/56-dwc_otg11:05:180
1088995527,5cyclictest24989-21aten_rbpower_vo11:20:180
1088995527,5cyclictest24989-21aten_rbpower_vo11:20:180
108899552,32cyclictest10879-21grep12:20:330
108899552,32cyclictest10879-21grep12:20:330
109099545,18cyclictest8050irq/56-dwc_otg_08:15:112
109099545,18cyclictest8050irq/56-dwc_otg_08:15:112
108899546,37cyclictest0-21swapper/010:00:160
1088995432,9cyclictest5228-21apt08:50:030
108899542,6cyclictest7850irq/56-dwc_otg10:40:140
108899531,28cyclictest30555-21dpkg08:30:040
108899531,24cyclictest3029850ssh09:30:190
140582520,2sleep30-21swapper/309:49:503
1088995227,7cyclictest0-21swapper/007:10:140
1088995210,39cyclictest101ktimersoftd/009:39:530
1088995210,39cyclictest101ktimersoftd/009:39:530
109199518,7cyclictest8050irq/56-dwc_otg_10:05:153
108899512,27cyclictest0-21swapper/007:55:150
108899511,27cyclictest1165-21snmpd08:50:090
1090995023,4cyclictest8050irq/56-dwc_otg_11:20:202
1090995023,4cyclictest8050irq/56-dwc_otg_11:20:202
109099502,26cyclictest31-21ksoftirqd/210:25:152
1089995031,3cyclictest0-21swapper/107:20:151
108899501,25cyclictest8922-21fschecks_count07:30:200
1088995010,36cyclictest101ktimersoftd/008:05:170
108899495,34cyclictest9-21ksoftirqd/008:35:590
1088994930,5cyclictest13648-21chrt07:41:300
1088994927,8cyclictest0-21swapper/008:41:190
1088994831,5cyclictest15777-21chrt07:49:230
5112473,11sleep00-21swapper/007:06:110
118992470,4sleep311901-21switchtime09:05:303
118992470,4sleep311901-21switchtime09:05:303
108899470,11cyclictest30283-21ssh12:05:240
109099467,24cyclictest0-21swapper/207:40:162
108999463,5cyclictest26086-21grep11:20:321
108999463,5cyclictest26086-21grep11:20:321
1090994516,9cyclictest21948-21if_wlan009:20:242
109199444,21cyclictest8050irq/56-dwc_otg_10:55:043
109099447,23cyclictest0-21swapper/207:20:102
109099446,11cyclictest29233-21ls10:45:292
1089994422,5cyclictest2405950wget10:40:171
1089994422,12cyclictest4512-21/usr/sbin/munin10:55:211
1089994418,5cyclictest13729-21unixbench_singl11:05:331
148852430,3sleep3381ktimersoftd/310:29:083
1091994320,4cyclictest18594-21latency_hist09:55:043
109099437,22cyclictest0-21swapper/211:30:162
109099437,22cyclictest0-21swapper/208:45:162
109099437,22cyclictest0-21swapper/208:30:052
108999436,14cyclictest12947-21missed_timers07:40:241
1089994320,5cyclictest506-21latency_hist11:30:041
4682420,4sleep1443-21latency10:50:231
109099426,22cyclictest0-21swapper/207:15:092
1090994222,5cyclictest1133-21runrttasks10:43:512
179322410,2sleep3381ktimersoftd/310:32:103
202772400,1sleep30-21swapper/309:18:383
169592400,2sleep123-21ksoftirqd/112:29:031
109099407,24cyclictest0-21swapper/212:00:172
109099407,24cyclictest0-21swapper/208:50:092
109099401,3cyclictest0-21swapper/210:20:152
108999402,17cyclictest4731-21dpkg07:20:041
1091993917,5cyclictest9001-21sh09:43:263
109099394,5cyclictest7850irq/56-dwc_otg10:30:162
1090993917,9cyclictest7950irq/56-dwc_otg_12:36:232
1090993917,9cyclictest7950irq/56-dwc_otg_12:36:232
108999398,21cyclictest7950irq/56-dwc_otg_09:00:141
108999393,9cyclictest29620-21munin-run09:30:031
1089993916,11cyclictest18262-21timerandwakeup09:15:321
1089993911,5cyclictest0-21swapper/111:00:321
25642380,10sleep1221ktimersoftd/112:10:311
109199384,4cyclictest8050irq/56-dwc_otg_12:20:223
109199384,4cyclictest8050irq/56-dwc_otg_12:20:223
1091993817,6cyclictest9578-21unixbench_singl11:00:333
109099382,10cyclictest0-21swapper/209:10:162
109099381,22cyclictest22889-21ls10:00:042
108999387,7cyclictest9550irq/80-mmc010:10:041
108999382,8cyclictest7850irq/56-dwc_otg08:35:171
108999382,26cyclictest21463-21ssh09:20:171
1089993817,5cyclictest18379-21sh11:11:251
108999381,24cyclictest9939-21latency_hist09:45:041
109199371,5cyclictest22643-21ssh09:20:333
1091993715,7cyclictest8525-21unixbench_singl08:55:323
1091993714,12cyclictest8050irq/56-dwc_otg_11:25:273
109199371,23cyclictest10769-21cpuspeed07:35:183
109199371,20cyclictest27140-21ls10:05:043
1091993711,14cyclictest0-21swapper/311:30:153
1090993731,4cyclictest301ktimersoftd/210:15:132
1090993715,4cyclictest18607-21sh09:15:452
108999373,3cyclictest7950irq/56-dwc_otg_11:50:161
108999373,3cyclictest7950irq/56-dwc_otg_11:50:161
108999373,10cyclictest8050irq/56-dwc_otg_08:20:051
108999373,10cyclictest8050irq/56-dwc_otg_08:20:051
108999372,7cyclictest141-21systemd-journal11:20:041
108999372,5cyclictest0-21swapper/110:49:011
109199368,6cyclictest8050irq/56-dwc_otg_11:20:053
109199361,7cyclictest14943-21tail12:25:323
109199361,3cyclictest0-21swapper/311:45:283
109099369,6cyclictest7850irq/56-dwc_otg12:10:302
109099369,13cyclictest11173-21missed_timers07:35:242
109099365,19cyclictest8050irq/56-dwc_otg_09:35:042
1090993617,4cyclictest9566-21/usr/sbin/munin12:20:312
1090993617,4cyclictest9566-21/usr/sbin/munin12:20:312
1090993617,4cyclictest1827-21sh10:13:082
109099360,10cyclictest8050irq/56-dwc_otg_09:50:062
108999363,3cyclictest8050irq/56-dwc_otg_11:30:281
108999362,6cyclictest0-21swapper/108:40:151
108999362,5cyclictest9694-21grep12:20:181
108999362,5cyclictest9694-21grep12:20:181
108999361,6cyclictest29511-21ntpq08:25:271
1089993615,9cyclictest12551-21diskmemload09:37:201
1089993615,9cyclictest12551-21diskmemload09:37:201
1089993614,13cyclictest22942-21sh10:38:441
287522350,11sleep2301ktimersoftd/208:25:042
286982350,1sleep30-21swapper/309:28:293
27442350,3sleep32727-21ssh09:35:293
27442350,3sleep32727-21ssh09:35:293
109199353,4cyclictest8050irq/56-dwc_otg_07:10:293
1091993517,10cyclictest7950irq/56-dwc_otg_12:10:163
109199351,4cyclictest7850irq/56-dwc_otg08:20:043
109199351,4cyclictest7850irq/56-dwc_otg08:20:043
1091993514,4cyclictest13092-21cpuspeed09:10:193
1091993512,5cyclictest21442-21cut11:55:173
109099357,19cyclictest0-21swapper/212:30:092
109099356,20cyclictest4822-21ssh11:35:162
109099356,18cyclictest2352-21latency08:40:222
108999358,7cyclictest29608-21ssh12:05:151
108999355,11cyclictest8050irq/56-dwc_otg_10:00:321
108999354,5cyclictest8050irq/56-dwc_otg_08:05:161
108999354,5cyclictest7542-21/usr/sbin/munin10:20:181
108999353,7cyclictest2138550ssh11:55:171
108999351,8cyclictest0-21swapper/110:15:101
1089993518,5cyclictest1133-21runrttasks07:25:161
1089993515,5cyclictest11086-21sh09:45:281
1089993515,4cyclictest12879-21/usr/sbin/munin09:10:301
1089993513,11cyclictest1133-21runrttasks07:53:421
1089993512,5cyclictest9550-21switchtime07:30:291
90152340,1sleep30-21swapper/312:19:393
109199343,19cyclictest0-21swapper/308:12:163
109199343,19cyclictest0-21swapper/308:12:163
109199342,4cyclictest7950irq/56-dwc_otg_08:25:303
109199341,7cyclictest0-21swapper/307:45:153
109199341,4cyclictest25471-21apt-get12:00:153
109199341,21cyclictest31824-21cut10:10:203
1091993412,15cyclictest211652chrt08:04:033
109099348,6cyclictest0-21swapper/212:26:482
109099347,15cyclictest0-21swapper/211:00:092
108999349,3cyclictest0-21swapper/112:35:151
108999349,3cyclictest0-21swapper/112:35:151
108999346,15cyclictest8050irq/56-dwc_otg_12:30:301
108999345,6cyclictest7850irq/56-dwc_otg08:55:321
108999343,17cyclictest16361-21ssh10:30:241
1089993428,4cyclictest15106-21ls09:50:241
1089993417,5cyclictest8318-21sh11:39:321
108999341,6cyclictest6327-21ps08:50:291
1089993412,3cyclictest0-21swapper/110:10:151
1089993412,3cyclictest0-21swapper/107:45:161
271942330,3sleep10-21swapper/108:20:161
228632330,2sleep20-21swapper/211:55:552
109199337,5cyclictest7950irq/56-dwc_otg_11:40:033
1091993327,4cyclictest19751-21ls09:55:293
109199331,6cyclictest27797-21cron10:45:043
109199331,6cyclictest13741-21grep11:05:333
109199331,5cyclictest32034-21munin-run10:50:033
109199331,3cyclictest0-21swapper/310:40:033
1091993313,4cyclictest511-21latency08:35:243
109099338,5cyclictest7950irq/56-dwc_otg_11:50:192
109099338,5cyclictest7950irq/56-dwc_otg_11:50:192
109099338,14cyclictest20551-21latency_hist11:15:042
109099337,14cyclictest0-21swapper/210:50:172
109099332,8cyclictest12076-21grep09:05:322
109099332,8cyclictest12076-21grep09:05:322
108999337,8cyclictest0-21swapper/108:00:151
108999336,6cyclictest0-21swapper/107:35:141
108999336,10cyclictest7950irq/56-dwc_otg_08:10:171
108999336,10cyclictest7950irq/56-dwc_otg_08:10:171
108999333,6cyclictest0-21swapper/109:55:241
108999333,5cyclictest25830-21grep12:00:211
1089993315,5cyclictest14136-21timerandwakeup11:45:291
1089993315,4cyclictest1133-21runrttasks08:32:571
108999331,4cyclictest0-21swapper/109:05:251
108999331,4cyclictest0-21swapper/109:05:251
1089993311,12cyclictest0-21swapper/109:31:211
1089993310,14cyclictest221ktimersoftd/107:10:261
109199329,16cyclictest0-21swapper/311:11:483
109199321,6cyclictest22646-21taskset12:35:243
109199321,6cyclictest22646-21taskset12:35:243
109199321,5cyclictest4996-21wget07:20:173
109099326,14cyclictest0-21swapper/207:30:172
109099323,4cyclictest7850irq/56-dwc_otg07:55:192
1090993228,2cyclictest0-21swapper/207:50:042
1090993214,9cyclictest9191-21ssh09:43:462
1090993213,10cyclictest7610-21ssh12:17:052
109099321,16cyclictest2119-21cut09:35:202
109099321,16cyclictest2119-21cut09:35:202
109199315,5cyclictest7850irq/56-dwc_otg10:15:103
109199313,23cyclictest8050irq/56-dwc_otg_08:05:143
1091993116,5cyclictest19443-21kworker/u8:011:50:173
1091993116,5cyclictest19443-21kworker/u8:011:50:173
109199311,5cyclictest28124-21grep08:20:293
109099317,15cyclictest0-21swapper/208:55:162
109099313,13cyclictest2889-21apt-key07:15:042
109099312,5cyclictest0-21swapper/209:46:082
109099312,15cyclictest8050irq/56-dwc_otg_08:25:232
109099311,27cyclictest8535-21ls07:30:042
1090993112,11cyclictest1133-21runrttasks10:05:022
109199309,11cyclictest0-21swapper/312:05:463
109199303,16cyclictest8050irq/56-dwc_otg_10:24:013
109199301,5cyclictest2609-21wc08:40:273
1091993015,9cyclictest28030-21sh11:23:573
1091993015,9cyclictest28030-21sh11:23:573
109199301,4cyclictest0-21swapper/308:50:293
1091993010,13cyclictest381ktimersoftd/312:30:093
109099307,10cyclictest24525-21munin-run11:20:042
109099302,6cyclictest26972-21grep09:25:322
109099302,4cyclictest7850irq/56-dwc_otg12:05:302
109099301,19cyclictest20423-21ls10:35:232
109199299,9cyclictest8050irq/56-dwc_otg_09:33:493
109199295,5cyclictest4623-21idleruntime-cro07:20:033
109199291,7cyclictest10324-21head09:00:313
109199291,22cyclictest16420-21cut07:50:233
109199291,13cyclictest8050irq/56-dwc_otg_07:55:263
1091992910,12cyclictest81782chrt07:27:043
109099297,4cyclictest0-21swapper/209:00:102
109099295,12cyclictest8050irq/56-dwc_otg_08:00:252
109099292,4cyclictest8050irq/56-dwc_otg_08:40:042
108999291,3cyclictest0-21swapper/108:45:211
109199289,4cyclictest0-21swapper/311:40:323
109199281,6cyclictest4975-21/usr/sbin/munin10:55:283
109099282,16cyclictest0-21swapper/207:53:192
109099281,5cyclictest7850irq/56-dwc_otg08:10:312
109099281,5cyclictest7850irq/56-dwc_otg08:10:312
108999287,4cyclictest7950irq/56-dwc_otg_07:55:221
322982274,14sleep232389-21nfs4_client07:05:242
109199271,6cyclictest8050irq/56-dwc_otg_08:30:163
109099273,4cyclictest21752-21fschecks_count08:05:202
5022253,14sleep30-21swapper/307:06:063
109199251,6cyclictest0-21swapper/307:42:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional