You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 18:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat Jun 27, 2026 12:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2014421690,2sleep31247099cyclictest11:41:413
124569911337,68cyclictest4031-21/usr/sbin/munin09:25:350
124569910526,58cyclictest11424-21processes12:10:350
1245699988,10cyclictest0-21swapper/012:05:210
1245699966,8cyclictest1419-21munin-run11:20:090
1245699952,10cyclictest0-21swapper/012:25:210
1245699952,10cyclictest0-21swapper/012:25:210
76482940,4sleep231-21ksoftirqd/210:08:482
178432920,4sleep31247099cyclictest07:25:093
12456999212,64cyclictest0-21swapper/012:17:220
233022850,2sleep30-21swapper/311:45:353
12456998522,19cyclictest23651-21diskmemload09:10:220
12456998211,52cyclictest9323-21sed09:30:350
1245699817,59cyclictest0-21swapper/008:20:220
12456998014,44cyclictest30627-21sh09:56:340
1245699783,13cyclictest31412-21runrttasks12:35:210
1245699783,13cyclictest31412-21runrttasks12:35:210
12456997814,42cyclictest5231-21awk08:20:080
285252770,2sleep30-21swapper/311:12:583
12456997711,48cyclictest0-21swapper/010:10:340
12456997611,42cyclictest0-21swapper/009:45:090
12456997536,14cyclictest0-21swapper/009:50:210
1246699744,54cyclictest31-21ksoftirqd/209:25:352
12456997419,13cyclictest8375-21unixbench_multi08:25:370
12456997410,52cyclictest0-21swapper/007:22:070
1245699738,46cyclictest0-21swapper/008:00:300
1245699727,51cyclictest0-21swapper/011:08:450
12456997215,36cyclictest21485-21awk09:05:080
12456997215,36cyclictest21485-21awk09:05:080
12456997212,45cyclictest0-21swapper/010:59:580
12456997210,47cyclictest24356-21latency_hist09:50:090
83082710,2sleep121-21rcuc/109:30:221
1245699716,22cyclictest0-21swapper/007:45:220
1245699713,30cyclictest0-21swapper/008:05:210
12456997113,12cyclictest14582-21grep08:45:240
12456997010,13cyclictest22315-21proc_pri07:35:350
12456997010,13cyclictest22315-21proc_pri07:35:350
1245699696,52cyclictest0-21swapper/012:03:050
1245699696,52cyclictest0-21swapper/012:03:050
31622680,2sleep20-21swapper/210:03:082
12456996811,42cyclictest19096-21wc11:40:340
12456996810,42cyclictest23861-21ssh11:45:560
1245699679,38cyclictest0-21swapper/010:02:200
12456996711,42cyclictest29172-21rm12:31:420
12456996710,54cyclictest19544-21cat07:30:090
1246399667,14cyclictest0-21swapper/109:25:351
1245699669,36cyclictest1192550kworker/u8:112:20:190
1245699668,47cyclictest0-21swapper/010:37:010
1245699667,51cyclictest29191-21ssh09:15:380
12456996615,35cyclictest6206-21cat10:45:360
12456996612,39cyclictest0-21swapper/009:22:150
85692650,3sleep337-21rcuc/310:10:103
12456996513,32cyclictest17815-21aten_rbpower_vo11:00:230
12456996513,32cyclictest17815-21aten_rbpower_vo11:00:230
12456996511,50cyclictest21269-21cut10:25:250
1245699649,45cyclictest0-21swapper/011:10:370
1245699649,31cyclictest20282-21latency07:30:290
12456996419,29cyclictest727-21runrttasks08:10:300
12456996419,29cyclictest727-21runrttasks08:10:300
12456996411,9cyclictest0-21swapper/011:31:360
1245699639,41cyclictest4558-21sh10:05:170
12456996311,36cyclictest4561-21rm10:44:370
12456996213,37cyclictest2473-21ls11:20:320
12456996211,36cyclictest14076-21ssh09:36:310
12456996210,43cyclictest24586-21taskset07:40:590
1245699619,49cyclictest475-21ssh11:57:280
1245699618,38cyclictest12780-21sh10:54:200
1245699615,44cyclictest0-21swapper/008:59:340
12456996115,20cyclictest18123-21cat11:40:090
12456996110,38cyclictest29931-21ssh11:54:310
1246399602,8cyclictest7850irq/56-dwc_otg12:10:341
1245699609,39cyclictest26909-21ssh10:31:140
1245699608,40cyclictest12443-21latency_hist08:40:090
12456996012,11cyclictest4608-21kworker/u8:307:55:150
12456996011,39cyclictest0-21swapper/007:16:440
12456996011,37cyclictest15972-21ssh10:18:550
12456996011,37cyclictest15972-21ssh10:18:550
1245699599,41cyclictest0-21swapper/011:25:560
12456995813,35cyclictest22399-21grep09:05:320
1246699565,9cyclictest8050irq/56-dwc_otg_12:10:332
12456995612,29cyclictest13766-21taskset07:11:210
12456995517,17cyclictest0-21swapper/007:50:210
12456995513,29cyclictest17732-21ntp_kernel_err10:20:300
227072540,1sleep10-21swapper/110:26:061
216692540,2sleep20-21swapper/212:23:272
12470995414,21cyclictest8050irq/56-dwc_otg_12:10:333
12470995413,8cyclictest7950irq/56-dwc_otg_09:25:363
1246399535,3cyclictest7950irq/56-dwc_otg_11:20:241
1245699529,9cyclictest0-21swapper/008:53:530
12466994917,6cyclictest12002-21latency_hist09:35:092
12470994819,5cyclictest7191-21sh11:25:393
1245699481,25cyclictest13835-21unixbench_singl08:40:380
12456994810,27cyclictest727-21runrttasks08:31:240
1246399458,32cyclictest18236-21ssh11:40:201
118262453,35sleep00-21swapper/007:06:210
118262453,35sleep00-21swapper/007:06:210
309642440,4sleep130963-21sh11:15:341
129912440,3sleep312987-21ls09:35:313
219452430,5sleep121699-21/usr/sbin/munin09:05:261
1247099435,7cyclictest37-21rcuc/311:35:203
134842420,3sleep3381ktimersoftd/311:34:363
12470994225,9cyclictest7850irq/56-dwc_otg09:11:523
12470994223,11cyclictest30722-21ssh09:56:453
12466994223,4cyclictest3254550wget08:05:232
12463994115,6cyclictest24649-21ls09:10:371
1247099401,4cyclictest17423-21ssh11:00:093
1247099401,25cyclictest0-21swapper/307:10:153
12466994019,5cyclictest12920-21sh09:35:312
12466994016,6cyclictest19056-21unixbench_multi07:25:372
239512390,4sleep223954-21missed_timers07:40:302
1246699391,30cyclictest8050irq/56-dwc_otg_12:35:252
1246699391,30cyclictest8050irq/56-dwc_otg_12:35:252
12463993919,10cyclictest10810-21ssh11:30:351
12463993917,17cyclictest8050irq/56-dwc_otg_10:50:211
1246399391,4cyclictest17705-21ssh11:00:231
1246399391,4cyclictest17705-21ssh11:00:231
1246399391,24cyclictest0-21swapper/109:01:551
1246399391,24cyclictest0-21swapper/109:01:551
1247099383,23cyclictest0-21swapper/307:35:103
1247099381,23cyclictest10751-21ls08:35:093
1247099381,23cyclictest0-21swapper/308:10:233
1247099381,23cyclictest0-21swapper/308:10:233
1246699389,4cyclictest0-21swapper/211:12:022
12466993818,4cyclictest20342-21sh09:45:102
1246399381,5cyclictest9550irq/48-DMA15:26:471
1246399381,5cyclictest9550irq/48-DMA15:26:471
12463993815,11cyclictest1154-21sh09:20:521
1246399381,4cyclictest866-21ssh10:40:101
1246399381,4cyclictest7850irq/56-dwc_otg11:50:191
1246399381,22cyclictest0-21swapper/111:58:181
113882387,12sleep20-21swapper/207:05:352
113882387,12sleep20-21swapper/207:05:352
69252371,2sleep20-21swapper/211:25:382
140782371,31sleep10-21swapper/110:55:301
1247099379,5cyclictest7850irq/56-dwc_otg10:25:083
12470993717,9cyclictest7950irq/56-dwc_otg_09:35:093
1247099371,25cyclictest19121-21cut12:20:253
12470993712,12cyclictest30086-21netstat10:35:313
12470993711,15cyclictest9981-21latency10:50:303
12470993710,14cyclictest0-21swapper/312:20:093
1246699376,4cyclictest0-21swapper/212:04:112
1246699376,4cyclictest0-21swapper/212:04:112
1246699371,22cyclictest14819-21uniq08:45:272
1246399373,5cyclictest7850irq/56-dwc_otg12:00:371
1246399373,5cyclictest7850irq/56-dwc_otg12:00:371
1246399372,5cyclictest12449-21munin-run08:40:091
12463993721,5cyclictest13017-21fschecks_time08:40:271
12463993714,15cyclictest9229-21sh12:08:211
12463993713,16cyclictest26089-1kworker/u9:109:15:261
181652360,5sleep21246699cyclictest10:20:352
1247099361,26cyclictest0-21swapper/308:08:263
1247099361,23cyclictest0-21swapper/308:45:233
1246699367,3cyclictest0-21swapper/211:20:242
1246399362,8cyclictest5793-21ssh10:05:371
1246399361,4cyclictest18920-21rm12:20:221
1246399361,24cyclictest0-21swapper/112:18:181
1247099352,25cyclictest0-21swapper/311:19:093
12470993517,12cyclictest3848-21rm11:22:023
1247099351,25cyclictest0-21swapper/312:28:553
1247099351,25cyclictest0-21swapper/312:28:553
1247099351,25cyclictest0-21swapper/312:07:503
1247099351,15cyclictest0-21swapper/308:40:313
1246699356,9cyclictest0-21swapper/212:05:532
1246699352,18cyclictest7950irq/56-dwc_otg_11:00:262
1246699352,18cyclictest7950irq/56-dwc_otg_11:00:262
1246399359,14cyclictest650-21cat10:00:231
1246399359,13cyclictest0-21swapper/110:31:031
1246399358,16cyclictest0-21swapper/112:33:021
1246399354,14cyclictest12801-21ssh09:35:291
12463993518,8cyclictest221ktimersoftd/111:05:221
12463993517,6cyclictest28867-21/usr/sbin/munin07:55:361
1246399351,24cyclictest339-21cut08:05:261
1246399351,24cyclictest0-21swapper/110:24:551
12463993512,12cyclictest0-21swapper/109:57:441
12470993420,7cyclictest381ktimersoftd/309:24:463
12470993419,8cyclictest37-21rcuc/310:00:203
12470993415,5cyclictest2450-21sh10:40:433
1247099341,25cyclictest0-21swapper/308:19:373
1247099341,24cyclictest0-21swapper/307:53:583
12466993418,4cyclictest30678-21wget11:55:232
12466993416,9cyclictest23604-21latency12:25:302
12466993416,9cyclictest23604-21latency12:25:302
1246699341,4cyclictest7850irq/56-dwc_otg09:10:392
1246699341,23cyclictest23042-21missed_timers11:45:312
12466993410,9cyclictest0-21swapper/209:56:412
1246399345,5cyclictest0-21swapper/112:40:091
1246399345,5cyclictest0-21swapper/112:40:091
1246399344,4cyclictest7850irq/56-dwc_otg12:25:331
1246399344,4cyclictest7850irq/56-dwc_otg12:25:331
1246399344,16cyclictest0-21swapper/107:25:361
1246399342,4cyclictest3032-21users08:10:391
1246399342,4cyclictest3032-21users08:10:391
1246399342,13cyclictest26968-21/usr/sbin/munin07:50:361
12463993417,8cyclictest23356-21/usr/sbin/munin07:40:331
1246399341,5cyclictest16602-21ssh09:40:241
12463993413,10cyclictest26002-21/usr/sbin/munin07:45:341
12463993412,9cyclictest25479-21sh11:48:541
1246399341,15cyclictest0-21swapper/107:10:331
1246399341,13cyclictest0-21swapper/110:45:231
280002330,3sleep229-21rcuc/211:50:582
1247099337,13cyclictest0-21swapper/310:13:043
1247099331,6cyclictest19107-21grep07:25:373
12470993315,6cyclictest12427-21cron08:40:093
1247099331,25cyclictest0-21swapper/311:00:583
1247099331,25cyclictest0-21swapper/311:00:583
1247099331,23cyclictest0-21swapper/311:59:493
12470993312,10cyclictest23284-21diskmemload10:19:203
12470993312,10cyclictest23284-21diskmemload10:19:203
1247099331,14cyclictest0-21swapper/310:45:233
1246699331,6cyclictest66750ssh11:18:492
12466993315,6cyclictest7684-21if_eth008:25:282
1246699331,20cyclictest0-21swapper/210:27:042
12466993310,4cyclictest0-21swapper/210:11:292
1246399338,5cyclictest7950irq/56-dwc_otg_09:45:261
1246399336,5cyclictest18135-21wget08:55:221
1246399335,10cyclictest7850irq/56-dwc_otg11:35:271
1246399333,16cyclictest0-21swapper/109:50:221
1246399331,3cyclictest0-21swapper/111:13:301
1246399331,23cyclictest3816-21cpuspeed08:15:241
1246399331,23cyclictest32392-21ls08:05:101
1247099327,12cyclictest7850irq/56-dwc_otg12:30:153
12470993224,6cyclictest23268-21latency_hist07:40:103
12470993224,6cyclictest23268-21latency_hist07:40:103
1247099322,3cyclictest0-21swapper/310:30:233
1247099321,5cyclictest125-21systemd-journal11:54:403
1247099321,23cyclictest0-21swapper/307:15:243
1247099321,22cyclictest0-21swapper/311:07:063
1247099321,20cyclictest25746-21ls07:45:293
1247099321,16cyclictest8050irq/56-dwc_otg_07:55:203
12470993210,15cyclictest0-21swapper/310:28:083
1246699329,9cyclictest8050irq/56-dwc_otg_12:30:272
1246699328,4cyclictest0-21swapper/211:33:532
1246699327,9cyclictest7950irq/56-dwc_otg_10:30:242
1246699327,14cyclictest0-21swapper/210:51:012
1246699324,6cyclictest7850irq/56-dwc_otg09:49:302
1246699322,20cyclictest2939250ssh10:35:222
1246699321,9cyclictest19421-21ssh11:40:372
12466993219,6cyclictest8050irq/56-dwc_otg_10:43:272
1246699321,6cyclictest22276-21memory09:05:302
12466993214,5cyclictest5227-21latency_hist08:20:092
1246399324,14cyclictest0-21swapper/110:40:311
1246399323,19cyclictest0-21swapper/107:24:191
1246399322,5cyclictest5874-21/usr/sbin/munin11:25:401
1246399321,4cyclictest0-21swapper/108:20:231
1246399321,3cyclictest0-21swapper/107:15:231
12463993211,5cyclictest17863-21latency_hist08:55:091
1247099314,5cyclictest18162-21wget08:55:223
12470993116,5cyclictest24774-21wget09:50:243
1247099311,24cyclictest0-21swapper/309:01:003
1247099311,24cyclictest0-21swapper/309:01:003
1247099311,20cyclictest2449-21aten_rbpower_vo12:00:243
1247099311,20cyclictest2449-21aten_rbpower_vo12:00:243
1247099311,18cyclictest0-21swapper/309:50:023
1247099311,14cyclictest0-21swapper/308:55:093
1246699316,4cyclictest8050irq/56-dwc_otg_11:35:272
12466993117,5cyclictest2669-21ssh09:23:312
12466993113,8cyclictest8698-21ssh10:49:382
12466993112,5cyclictest8050irq/56-dwc_otg_09:15:202
1246399313,17cyclictest0-21swapper/108:50:081
12463993111,4cyclictest9395-21cat10:10:301
1247099302,5cyclictest0-21swapper/309:43:063
1247099301,8cyclictest30914-21ssh09:18:373
1247099301,17cyclictest0-21swapper/309:05:303
1247099301,16cyclictest0-21swapper/308:25:233
1246699306,12cyclictest0-21swapper/209:54:452
1246699306,10cyclictest7850irq/56-dwc_otg08:10:212
1246699306,10cyclictest7850irq/56-dwc_otg08:10:212
1246699302,6cyclictest0-21swapper/212:19:062
1246699301,4cyclictest13806-21df_inode10:55:262
12466993013,5cyclictest13112-21if_wlan008:40:292
1246699301,23cyclictest25435-21uniq07:45:252
1246399302,4cyclictest0-21swapper/108:25:271
1246399302,3cyclictest0-21swapper/108:30:301
111352306,11sleep10-21swapper/107:05:301
111352306,11sleep10-21swapper/107:05:301
1247099291,20cyclictest0-21swapper/308:20:243
1247099291,19cyclictest0-21swapper/307:40:333
12470992910,4cyclictest0-21swapper/312:35:153
12470992910,4cyclictest0-21swapper/312:35:153
1246699297,10cyclictest0-21swapper/211:09:152
1246699293,6cyclictest17227-21grep07:20:362
1246699293,10cyclictest7950irq/56-dwc_otg_07:35:242
1246699293,10cyclictest7950irq/56-dwc_otg_07:35:242
12466992921,5cyclictest125-21systemd-journal07:55:082
1246699291,24cyclictest19088-21timerwakeupswit08:55:362
1246699291,17cyclictest9369-21cut08:30:282
12466992910,5cyclictest5756-21fschecks_time08:20:272
1246399297,4cyclictest7950irq/56-dwc_otg_07:35:211
1246399297,4cyclictest7950irq/56-dwc_otg_07:35:211
12463992910,11cyclictest20457-21ntpq07:30:311
12470992818,7cyclictest30744-21aten_rbpower_po08:00:223
1246699282,16cyclictest14564-21fschecks_count07:15:262
1246699275,10cyclictest8050irq/56-dwc_otg_09:00:302
1246699275,10cyclictest8050irq/56-dwc_otg_09:00:302
1246699272,6cyclictest7850irq/56-dwc_otg08:35:282
1246699271,5cyclictest13556-21ls10:15:302
1246699271,5cyclictest13556-21ls10:15:302
12466992713,6cyclictest13325-21rt-features07:10:362
1246699271,10cyclictest0-21swapper/208:04:132
117892273,14sleep30-21swapper/307:05:523
117892273,14sleep30-21swapper/307:05:523
1246699266,4cyclictest8050irq/56-dwc_otg_08:00:092
1246699263,5cyclictest7850irq/56-dwc_otg07:30:312
1246699242,3cyclictest8050irq/56-dwc_otg_08:50:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional