You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 08:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Wed Dec 17, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
426522380,3sleep123-21ksoftirqd/119:05:291
218432970,2sleep00-21swapper/019:55:170
44622880,5sleep0517399cyclictest23:35:180
517399807,64cyclictest0-21swapper/000:10:160
146492800,2sleep30-21swapper/322:35:033
517599792,70cyclictest5172-21cyclictest19:10:052
517399736,15cyclictest0-21swapper/020:35:150
43792730,2sleep10-21swapper/120:35:231
28472720,2sleep20-21swapper/220:30:292
517599711,6cyclictest0-21swapper/223:20:142
517399702,33cyclictest12752-21grep19:30:190
517599655,42cyclictest7950irq/56-dwc_otg_00:00:292
517599655,42cyclictest7950irq/56-dwc_otg_00:00:292
517599651,8cyclictest24569-21grep20:00:292
517599651,50cyclictest8043-21awk23:45:182
517599651,48cyclictest6294-21open_inodes20:40:302
517399656,44cyclictest0-21swapper/019:20:150
46792654,13sleep30-21swapper/319:07:183
517499641,11cyclictest0-21swapper/120:45:131
5173996433,16cyclictest101ktimersoftd/023:00:190
5173996427,18cyclictest0-21swapper/020:40:170
517599632,46cyclictest0-21swapper/220:10:222
517599617,36cyclictest7950irq/56-dwc_otg_21:05:032
39622610,5sleep1517499cyclictest22:05:191
517399604,43cyclictest0-21swapper/021:25:210
517599595,31cyclictest0-21swapper/221:45:142
517399594,31cyclictest0-21swapper/023:55:200
517399594,31cyclictest0-21swapper/023:55:200
517399584,31cyclictest8987-21grep23:45:310
517399581,24cyclictest0-21swapper/021:00:170
5175995716,37cyclictest10715-21/usr/sbin/munin19:25:312
517699561,22cyclictest0-21swapper/320:00:083
517399561,19cyclictest11318-21ls20:55:210
517599555,39cyclictest7950irq/56-dwc_otg_23:00:192
517599552,10cyclictest7750irq/56-dwc_otg23:35:152
517599551,44cyclictest7950irq/56-dwc_otg_00:19:132
5173995520,6cyclictest20984-21grep19:50:320
5173995520,6cyclictest20984-21grep19:50:320
517599544,45cyclictest0-21swapper/221:25:312
517599543,5cyclictest7950irq/56-dwc_otg_00:08:342
517599542,5cyclictest0-21swapper/222:50:392
517599542,4cyclictest0-21swapper/200:36:042
517599541,42cyclictest24364-21grep21:30:282
517599541,3cyclictest7950irq/56-dwc_otg_19:35:252
5175995411,40cyclictest5456-21latency_hist22:10:032
517399543,6cyclictest7950irq/56-dwc_otg_22:05:090
5173995432,5cyclictest10323-21head20:50:320
517599533,39cyclictest7850irq/56-dwc_otg_19:34:562
517599532,4cyclictest0-21swapper/200:25:312
517399539,22cyclictest0-21swapper/020:25:140
517399536,32cyclictest1237-21kworker/0:019:10:060
5173995325,8cyclictest15415-21cut00:05:210
517399531,49cyclictest24476-21ls00:30:220
517599523,45cyclictest7268-21basename22:15:032
517599523,40cyclictest0-21swapper/223:34:462
517599522,9cyclictest7750irq/56-dwc_otg20:35:132
517599521,8cyclictest0-21swapper/222:00:142
517599521,6cyclictest0-21swapper/220:55:462
517499521,9cyclictest7950irq/56-dwc_otg_00:15:141
5173995223,4cyclictest29135-21aten_rbpower_vo21:45:170
517599512,9cyclictest10233-21chrt19:22:072
517599511,7cyclictest9018-21latency_hist20:50:032
517599511,5cyclictest0-21swapper/221:55:162
517599511,5cyclictest0-21swapper/221:55:162
517599511,36cyclictest14557-21/usr/sbin/munin21:05:272
5173995132,5cyclictest7950irq/56-dwc_otg_19:50:030
5173995129,5cyclictest7850irq/56-dwc_otg_22:15:030
517599502,7cyclictest29366-21latency_hist23:15:042
517599501,5cyclictest0-21swapper/219:40:172
517599501,40cyclictest0-21swapper/223:16:302
517399504,25cyclictest32382-21idleruntime-cro21:55:030
517399503,24cyclictest0-21swapper/019:20:040
517399501,5cyclictest18953-21latency22:45:230
517399501,44cyclictest0-21swapper/000:36:180
5173995011,20cyclictest22835-21/usr/sbin/munin22:55:260
517599491,6cyclictest11209-21/usr/sbin/munin22:25:292
5175994915,30cyclictest19237-21rt-features22:45:282
517599491,3cyclictest7950irq/56-dwc_otg_21:40:032
517599491,3cyclictest0-21swapper/221:51:212
517599491,39cyclictest0-21swapper/200:23:202
5175994911,35cyclictest20294-21irqstats19:50:232
5175994911,35cyclictest20294-21irqstats19:50:232
5175994911,34cyclictest24340-21seq00:30:202
517399493,31cyclictest0-21swapper/022:50:140
5173994914,32cyclictest10885-21munin-run22:25:030
5173994914,32cyclictest10885-21munin-run22:25:030
517599484,4cyclictest0-21swapper/223:50:282
517599482,8cyclictest21574-21munin-run21:25:032
517599482,6cyclictest9227-21latency_hist22:20:042
517599482,6cyclictest18604-21latency_hist00:15:042
517599482,5cyclictest0-21swapper/219:55:262
517599481,6cyclictest25654-21fschecks_time20:05:202
517599481,6cyclictest0-21swapper/220:15:422
517599481,5cyclictest25973-21grep23:05:172
517599481,3cyclictest0-21swapper/219:45:212
517399486,24cyclictest0-21swapper/023:20:150
5173994823,5cyclictest0-21swapper/021:10:260
5173994823,5cyclictest0-21swapper/021:10:260
5173994819,22cyclictest0-21swapper/000:02:170
5173994819,22cyclictest0-21swapper/000:02:170
517699477,26cyclictest0-21swapper/300:35:093
517599477,25cyclictest0-21swapper/221:15:162
517599472,4cyclictest0-21swapper/223:25:202
517599472,3cyclictest0-21swapper/220:50:152
517599472,38cyclictest7950irq/56-dwc_otg_22:20:512
517599472,38cyclictest7950irq/56-dwc_otg_22:20:512
517599471,7cyclictest28817-21latency_hist21:45:032
517599471,7cyclictest15684-21ls22:35:292
5175994716,29cyclictest16622-21/usr/sbin/munin22:40:222
517599471,5cyclictest16446-21aten_rbpower_en21:10:172
517599471,5cyclictest16446-21aten_rbpower_en21:10:172
517599471,4cyclictest7950irq/56-dwc_otg_23:56:432
517599471,4cyclictest7950irq/56-dwc_otg_23:56:432
517499472,10cyclictest0-21swapper/120:25:141
5174994712,12cyclictest7750irq/56-dwc_otg22:15:131
517399477,13cyclictest0-21swapper/021:55:150
517399477,13cyclictest0-21swapper/021:55:150
5173994724,3cyclictest0-21swapper/023:15:180
517399472,31cyclictest0-21swapper/021:42:500
5173994721,14cyclictest0-21swapper/021:15:240
517399471,29cyclictest0-21swapper/022:40:190
517599463,40cyclictest0-21swapper/223:40:172
517599461,6cyclictest30809-21grep20:20:172
517599461,6cyclictest30809-21grep20:20:172
517599461,6cyclictest23692-21cron23:00:032
517599461,5cyclictest13141-21grep22:30:182
517599461,5cyclictest128-21systemd-journal20:30:032
517599461,5cyclictest0-21swapper/219:15:202
517399461,28cyclictest13922-21kworker/0:119:40:240
517499458,23cyclictest0-21swapper/119:20:161
517399452,25cyclictest0-21swapper/022:35:020
5173994520,7cyclictest14830-21fschecks_count21:05:190
5173994517,6cyclictest25355-21latency_hist21:35:040
517399451,41cyclictest0-21swapper/020:20:170
517399451,41cyclictest0-21swapper/020:20:170
517399451,17cyclictest29085-21aten_rbpower_vo20:15:190
5173994510,18cyclictest2216-21ls23:30:030
517499442,9cyclictest0-21swapper/122:00:161
517399446,9cyclictest13-21rcuc/000:15:250
517399446,24cyclictest0-21swapper/020:30:170
5173994422,8cyclictest0-21swapper/023:15:040
5173994420,5cyclictest7797-21grep22:15:200
5173994419,2cyclictest0-21swapper/000:20:150
5173994416,16cyclictest0-21swapper/023:09:450
517399441,18cyclictest15442-21unixbench_multi19:35:330
517399437,27cyclictest0-21swapper/020:50:030
517399432,23cyclictest0-21swapper/000:25:320
5173994318,14cyclictest0-21swapper/020:05:220
5173994317,4cyclictest0-21swapper/023:35:030
5173994314,13cyclictest6358-21fschecks_time23:40:200
517399431,29cyclictest2900-21timerandwakeup22:00:290
495324311,16sleep20-21swapper/219:10:042
45602432,35sleep00-21swapper/019:05:450
5176994228,9cyclictest0-21swapper/322:35:313
517499421,4cyclictest7950irq/56-dwc_otg_22:50:141
517499421,27cyclictest0-21swapper/100:10:161
517399422,23cyclictest0-21swapper/022:25:300
5173994217,15cyclictest24711-21unixbench_singl20:00:300
5173994217,11cyclictest0-21swapper/021:20:250
517399421,26cyclictest12283-21munin-run19:30:030
517399411,26cyclictest0-21swapper/023:54:150
517499404,24cyclictest0-21swapper/100:35:091
517499401,16cyclictest7950irq/56-dwc_otg_21:25:321
5173994017,10cyclictest0-21swapper/022:38:480
517399401,27cyclictest28295-21grep20:10:330
517399401,22cyclictest0-21swapper/021:35:230
5176993928,6cyclictest89252sleep319:20:143
517499398,4cyclictest17277-21unixbench_singl19:40:311
517499385,19cyclictest7950irq/56-dwc_otg_23:55:311
517499385,19cyclictest7950irq/56-dwc_otg_23:55:311
517699377,15cyclictest0-21swapper/300:10:163
5176993715,13cyclictest17389-21processes22:40:273
517699371,21cyclictest3509-21latency_hist20:35:043
517499378,8cyclictest0-21swapper/121:15:141
517499376,16cyclictest7950irq/56-dwc_otg_23:00:031
517499371,25cyclictest31886-21ls23:20:241
5174993712,5cyclictest27428-21munin-plugin-st23:10:031
5176993616,6cyclictest25927-21latency_hist00:35:043
517699361,21cyclictest17951-21latency_hist21:15:043
517699361,21cyclictest17951-21latency_hist21:15:043
517499362,5cyclictest23208-21apt-get20:00:091
5174993616,11cyclictest0-21swapper/100:00:331
5174993616,11cyclictest0-21swapper/100:00:331
5174993614,5cyclictest25889-21ls00:35:041
5174993610,5cyclictest0-21swapper/121:50:181
50512368,12sleep30-21swapper/319:10:053
517699352,18cyclictest7950irq/56-dwc_otg_21:45:183
5176993520,12cyclictest432-21systemd-logind22:15:133
5176993517,9cyclictest27504-21latency_hist23:10:043
517699351,22cyclictest8278-21wc20:45:283
517699351,10cyclictest7950irq/56-dwc_otg_00:15:133
517499359,5cyclictest7850irq/56-dwc_otg_22:40:301
517499353,5cyclictest7750irq/56-dwc_otg21:00:151
517499352,16cyclictest7950irq/56-dwc_otg_22:31:161
517699347,23cyclictest10633-21ls23:50:293
517699345,14cyclictest0-21swapper/320:56:273
517699343,11cyclictest12196-21kworker/3:019:35:033
517699341,3cyclictest7950irq/56-dwc_otg_21:25:303
517699341,12cyclictest20586-21cut00:20:173
517499343,10cyclictest0-21swapper/119:10:051
517499343,10cyclictest0-21swapper/100:25:151
517499342,3cyclictest0-21swapper/120:05:151
517499342,14cyclictest7950irq/56-dwc_otg_22:00:031
517499342,14cyclictest7950irq/56-dwc_otg_22:00:031
5174993411,13cyclictest12019-21runrttasks19:26:541
517699337,14cyclictest0-21swapper/323:40:173
517699334,10cyclictest7750irq/56-dwc_otg22:50:043
517699331,24cyclictest24315-21uniq23:00:223
517699331,19cyclictest0-21swapper/321:20:173
517499339,5cyclictest7850irq/56-dwc_otg_00:20:201
517499338,10cyclictest7950irq/56-dwc_otg_22:35:141
5174993312,6cyclictest0-21swapper/120:20:231
5174993312,6cyclictest0-21swapper/120:20:231
5176993221,8cyclictest381ktimersoftd/323:35:163
517699321,4cyclictest19722-21apt-config19:50:043
517499328,6cyclictest12633-21aten_rbpower_vo19:30:161
517499327,16cyclictest0-21swapper/123:40:171
517499322,16cyclictest0-21swapper/120:10:061
5174993214,5cyclictest21-21rcuc/123:00:231
5174993213,9cyclictest221ktimersoftd/122:10:201
517499321,20cyclictest20242-21cut21:20:201
517499321,12cyclictest7750irq/56-dwc_otg20:15:291
105922320,3sleep123-21ksoftirqd/123:50:281
517699316,14cyclictest0-21swapper/319:40:173
517699316,11cyclictest1125-21runrttasks20:53:293
517699315,13cyclictest0-21swapper/322:59:223
517699312,15cyclictest0-21swapper/321:40:183
5176993119,5cyclictest39-21ksoftirqd/322:20:303
5176993119,5cyclictest39-21ksoftirqd/322:20:303
517699311,14cyclictest0-21swapper/321:15:153
517499316,12cyclictest0-21swapper/121:35:311
517499312,7cyclictest14365-21taskset19:35:191
517499311,9cyclictest7950irq/56-dwc_otg_21:05:271
517499311,22cyclictest11234-21uniq20:55:211
517499311,21cyclictest18169-21uniq19:45:211
517499311,20cyclictest0-21swapper/119:50:551
517499311,20cyclictest0-21swapper/119:50:551
517699308,12cyclictest20320-21kernelversion19:50:243
517699308,12cyclictest20320-21kernelversion19:50:243
517699306,10cyclictest7850irq/56-dwc_otg_23:26:373
5176993022,5cyclictest16285-21latency_hist21:10:043
517699301,17cyclictest0-21swapper/319:55:133
517499309,11cyclictest221ktimersoftd/123:45:221
517499306,4cyclictest28783-21grep23:10:311
517499305,5cyclictest21-21rcuc/120:40:331
517499303,14cyclictest7950irq/56-dwc_otg_19:55:191
517499302,6cyclictest7950irq/56-dwc_otg_19:15:171
517499302,5cyclictest16169-21sed00:05:331
5174993024,4cyclictest1125-21runrttasks23:30:041
5174993015,4cyclictest21-21rcuc/121:30:321
517499301,14cyclictest7950irq/56-dwc_otg_23:35:161
517699292,6cyclictest128-21systemd-journal23:15:033
517699292,5cyclictest7850irq/56-dwc_otg_20:15:273
517699292,18cyclictest0-21swapper/321:36:043
5176992919,5cyclictest3940-21wget22:05:193
517699291,19cyclictest0-21swapper/300:02:213
517699291,19cyclictest0-21swapper/300:02:213
517699291,18cyclictest0-21swapper/320:35:293
517699291,16cyclictest3433-21timerwakeupswit23:30:303
517699291,15cyclictest7950irq/56-dwc_otg_00:05:283
5176992910,5cyclictest698-21cat21:55:243
5176992910,5cyclictest698-21cat21:55:243
517499292,5cyclictest9349-21grep22:20:181
517499292,5cyclictest9349-21grep22:20:181
517499291,15cyclictest0-21swapper/122:45:291
517699287,9cyclictest7950irq/56-dwc_otg_20:05:183
517699286,13cyclictest1124-21snmpd23:20:193
517699282,16cyclictest11-21rcu_preempt20:40:333
5176992812,7cyclictest381ktimersoftd/323:55:263
5176992812,7cyclictest381ktimersoftd/323:55:263
5176992812,4cyclictest31124-21ls20:20:213
5176992812,4cyclictest31124-21ls20:20:213
5176992811,6cyclictest381ktimersoftd/320:10:183
5176992811,4cyclictest24537-21timerandwakeup21:30:303
517699281,12cyclictest1152-21ls20:25:303
517699281,11cyclictest10758-21aten_rbpower_cu19:25:183
517499287,5cyclictest27297-21aten_rbpower_po21:40:181
517499284,5cyclictest7750irq/56-dwc_otg23:15:261
517499283,5cyclictest3592-21sed23:30:311
517499282,8cyclictest17476-21grep21:10:321
517499282,8cyclictest17476-21grep21:10:321
5174992822,4cyclictest23-21ksoftirqd/120:50:181
517499282,14cyclictest0-21swapper/120:30:141
5174992815,4cyclictest1125-21runrttasks21:46:121
517499281,20cyclictest11545-21uniq22:25:211
517699276,10cyclictest29962-21wc23:15:233
517699274,10cyclictest8392-21sed23:45:233
517699272,12cyclictest7750irq/56-dwc_otg22:50:203
517699271,6cyclictest0-21swapper/319:15:153
517699271,4cyclictest0-21swapper/321:50:253
517699271,19cyclictest0-21swapper/321:00:173
517699271,11cyclictest7950irq/56-dwc_otg_22:00:173
517699261,16cyclictest0-21swapper/322:14:533
517699261,13cyclictest7950irq/56-dwc_otg_00:29:503
517699255,9cyclictest14314-21/usr/sbin/munin19:35:163
517699251,13cyclictest0-21swapper/322:25:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional