You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 07:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 27, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1693322850,3sleep20-21swapper/204:26:052
18636991081,93cyclictest0-21swapper/009:56:120
227002990,8sleep21863899cyclictest07:00:492
1863699951,75cyclictest16798-21sed08:50:500
18636999342,44cyclictest10441-21timerwakeupswit09:21:200
253982900,3sleep30-21swapper/308:21:143
18636998922,53cyclictest12945-21/usr/sbin/munin08:06:120
183132880,3sleep01863699cyclictest07:33:320
183132880,3sleep01863699cyclictest07:33:320
1863699861,60cyclictest8473-21chrt07:21:160
1863699851,5cyclictest8883-21ntp_states08:01:150
1863699842,67cyclictest0-21swapper/007:06:130
1863699832,68cyclictest28941-21df_inode08:26:060
92612820,5sleep10-21swapper/108:01:191
18636998124,37cyclictest0-21swapper/004:31:160
18636998124,37cyclictest0-21swapper/004:31:160
1863699811,70cyclictest0-21swapper/007:05:500
1863699811,70cyclictest0-21swapper/007:05:500
100002810,3sleep11863799cyclictest08:41:211
1863699807,24cyclictest0-21swapper/006:36:040
1863699802,59cyclictest11175-21ssh06:46:150
1863699801,58cyclictest9019-21sed05:31:180
1863699801,13cyclictest0-21swapper/009:40:510
1863699791,67cyclictest16666-21chrt06:52:420
18636997742,17cyclictest0-21swapper/009:46:080
18636997719,41cyclictest13412-21/usr/sbin/munin09:26:160
1863699771,65cyclictest3396-21/usr/sbin/munin07:16:010
264602760,3sleep226441-21ssh09:01:192
1863699761,58cyclictest0-21swapper/006:31:550
1863699761,57cyclictest0-21swapper/008:36:060
1863699761,57cyclictest0-21swapper/008:36:060
181772760,5sleep31863999cyclictest08:12:163
1863699741,59cyclictest22345-21ssh08:17:220
1863699737,33cyclictest0-21swapper/007:26:040
18636997330,25cyclictest7850irq/56-dwc_otg_08:31:200
1863699732,56cyclictest0-21swapper/006:26:190
18636997320,42cyclictest0-21swapper/005:01:020
1863699731,6cyclictest24958-21ssh07:41:160
1863699731,60cyclictest17668-21sort08:51:100
1863699731,60cyclictest0-21swapper/005:56:070
1863699731,22cyclictest5495-21/usr/sbin/munin09:16:070
185622730,4sleep121-21rcuc/108:12:581
18636997222,35cyclictest0-21swapper/008:56:160
1863699721,58cyclictest3992-21dhclient-script05:20:250
1863699721,58cyclictest3992-21dhclient-script05:20:250
1863699721,58cyclictest20138-21/usr/sbin/munin06:01:190
1863699721,55cyclictest0-21swapper/009:54:230
210842710,2sleep10-21swapper/106:57:581
1863699712,6cyclictest3985-21ssh07:56:020
1863699712,6cyclictest3985-21ssh07:56:020
18636997124,29cyclictest10004-21fschecks_time05:36:070
1863699711,59cyclictest17907-21diskmemload08:11:460
1863799705,45cyclictest7850irq/56-dwc_otg_06:01:191
18636997018,40cyclictest0-21swapper/008:25:450
1863799692,53cyclictest17332-21fschecks_time05:56:061
1863799691,53cyclictest7750irq/56-dwc_otg08:06:101
18636996919,41cyclictest0-21swapper/008:45:150
1863699691,56cyclictest8408-21ssh06:43:060
1863699691,55cyclictest0-21swapper/009:43:230
1863799683,50cyclictest15955-21ssh06:51:261
1863799681,58cyclictest8460-21awk07:21:161
1863699687,43cyclictest0-21swapper/007:36:110
1863699681,20cyclictest4323-21ssh09:14:550
1863799674,50cyclictest22709-21rm08:56:471
1863799673,49cyclictest7750irq/56-dwc_otg05:01:131
1863799671,51cyclictest26280-21cpu06:21:051
1863799671,51cyclictest26280-21cpu06:21:051
1863699672,12cyclictest5349-21unixbench-2d05:21:170
1863699671,56cyclictest0-21swapper/006:56:290
1863799669,35cyclictest9550irq/80-mmc009:45:491
1863799664,51cyclictest3320-21ssh09:13:031
1863799664,47cyclictest32503-21apt-get08:30:541
1863799663,56cyclictest7970-21ssh08:40:001
1863799663,56cyclictest7970-21ssh08:40:001
1863799661,49cyclictest31851-21ssh09:08:271
1863699662,55cyclictest0-21swapper/006:07:120
1863699662,19cyclictest2072-21ssh07:52:570
18636996618,38cyclictest219372sleep004:39:420
1863699661,20cyclictest0-21swapper/009:07:270
1863799651,5cyclictest5284-21aten_rbpower_vo09:16:041
1863899643,5cyclictest16699-21awk05:55:502
1863899643,5cyclictest16699-21awk05:55:502
1863699642,48cyclictest0-21swapper/009:32:220
1863699641,52cyclictest0-21swapper/005:41:540
18636996414,2cyclictest0-21swapper/009:04:430
172472640,1sleep30-21swapper/308:51:053
1863799632,51cyclictest0-21swapper/107:07:321
1863799631,48cyclictest8947-21/usr/sbin/munin05:31:181
18636996319,5cyclictest29641-21sshd07:46:340
1863699631,50cyclictest0-21swapper/006:15:070
18638996213,5cyclictest17632-21missed_timers08:51:102
1863799626,43cyclictest7950irq/56-dwc_otg_04:31:161
1863799626,43cyclictest7950irq/56-dwc_otg_04:31:161
1863799624,46cyclictest10601-21grep09:21:211
1863799623,48cyclictest10235-21ssh06:45:551
1863799621,50cyclictest530-21munin-node07:41:011
1863799621,44cyclictest12611-21/usr/sbin/munin07:26:161
1863799620,51cyclictest0-21swapper/108:00:501
1863799620,51cyclictest0-21swapper/108:00:501
18636996216,36cyclictest0-21swapper/007:15:400
1863699621,53cyclictest0-21swapper/004:46:150
1863899611,6cyclictest0-21swapper/205:01:042
1863899611,15cyclictest7750irq/56-dwc_otg08:25:502
1863799614,45cyclictest26812-21ssh07:05:341
1863799614,45cyclictest26812-21ssh07:05:341
1863799614,44cyclictest21662sleep105:15:011
1863799614,42cyclictest7850irq/56-dwc_otg_09:32:441
1863799612,6cyclictest16100-21rm09:28:471
1863799611,6cyclictest420-21ssh09:48:511
1863799611,35cyclictest16696-21latency_hist08:50:501
1863699619,4cyclictest16677-21cron05:55:500
1863699619,4cyclictest16677-21cron05:55:500
1863699612,47cyclictest0-21swapper/006:16:090
1863699612,19cyclictest26235-21aten_rbpower_po06:21:040
1863699612,19cyclictest26235-21aten_rbpower_po06:21:040
1863899602,5cyclictest7850irq/56-dwc_otg_08:06:152
18638996015,6cyclictest9109-21fschecks_time08:41:092
1863799605,45cyclictest7850irq/56-dwc_otg_06:37:511
1863799604,42cyclictest7750irq/56-dwc_otg09:02:151
1863799602,48cyclictest19588-21ssh08:53:371
1863799601,49cyclictest3162-21ssh07:54:581
1863799601,48cyclictest27423-21sshd04:55:351
18636996012,9cyclictest0-21swapper/004:55:360
1863799598,31cyclictest0-21swapper/108:20:551
1863799596,31cyclictest0-21swapper/106:45:141
1863799594,42cyclictest7850irq/56-dwc_otg_08:28:521
1863799592,37cyclictest0-21swapper/105:25:291
1863799591,46cyclictest7950irq/56-dwc_otg_07:49:231
1863799591,45cyclictest0-21swapper/105:39:131
1863799591,37cyclictest9260-21chrt09:59:521
1863699592,47cyclictest0-21swapper/005:47:590
1863999587,15cyclictest0-21swapper/308:56:003
1863899581,8cyclictest10506-21ssh09:21:212
1863899581,5cyclictest19756-21sed04:31:172
1863899581,5cyclictest19756-21sed04:31:172
1863899581,5cyclictest17503-21/usr/sbin/munin05:56:092
1863799583,42cyclictest7850irq/56-dwc_otg_07:15:391
1863799581,45cyclictest166712sleep105:55:461
1863799581,45cyclictest166712sleep105:55:461
1863799581,44cyclictest0-21swapper/109:51:341
1863699581,4cyclictest22270-21/usr/sbin/munin04:41:070
1863699581,49cyclictest0-21swapper/005:11:540
1863899576,4cyclictest0-21swapper/206:45:052
1863899572,3cyclictest0-21swapper/206:21:052
1863899572,3cyclictest0-21swapper/206:21:052
18638995716,5cyclictest29730-21latency_hist06:30:502
1863899571,3cyclictest7850irq/56-dwc_otg_09:16:102
18638995711,4cyclictest7850irq/56-dwc_otg_08:04:442
1863799571,45cyclictest25682-21taskset06:17:551
1863799571,45cyclictest0-21swapper/109:35:561
1863799571,3cyclictest0-21swapper/108:18:081
1863799571,3cyclictest0-21swapper/107:16:171
18636995740,5cyclictest6352-21fschecks_count05:26:060
1863699571,21cyclictest29054-21chrt04:58:170
1863899567,5cyclictest7850irq/56-dwc_otg_07:21:192
1863899562,5cyclictest0-21swapper/209:07:192
1863899561,9cyclictest7850irq/56-dwc_otg_09:41:012
18638995618,4cyclictest20061-21grep06:01:172
1863899561,5cyclictest25299-21sed07:41:202
1863799562,43cyclictest19489-21ssh07:35:391
1863799562,43cyclictest19489-21ssh07:35:391
1863799562,42cyclictest7850irq/56-dwc_otg_06:31:541
1863799561,6cyclictest29012-21timerwakeupswit06:26:171
1863799561,5cyclictest20141-21rm07:36:041
1863799561,37cyclictest7950irq/56-dwc_otg_04:36:011
18637995610,36cyclictest7850irq/56-dwc_otg_05:20:281
18637995610,36cyclictest7850irq/56-dwc_otg_05:20:281
1863699562,40cyclictest0-21swapper/005:06:140
1863899554,6cyclictest7950irq/56-dwc_otg_06:48:442
1863899553,3cyclictest7950irq/56-dwc_otg_09:27:152
1863799556,32cyclictest0-21swapper/105:06:221
1863999545,15cyclictest11679-21ssh07:26:023
1863899543,6cyclictest9450irq/48-DMA09:12:252
1863899541,4cyclictest7950irq/56-dwc_otg_08:30:552
1863899541,11cyclictest0-21swapper/205:20:072
1863899541,11cyclictest0-21swapper/205:20:072
1863799543,36cyclictest21983-21runrttasks06:06:481
1863799541,42cyclictest0-21swapper/105:44:391
1863799541,41cyclictest0-21swapper/104:50:221
204962530,2sleep30-21swapper/309:34:053
1863899531,6cyclictest19444-21chrt08:14:362
1863799532,36cyclictest0-21swapper/104:45:001
1863799531,42cyclictest0-21swapper/105:47:511
182562533,44sleep10-21swapper/104:29:331
1863899527,3cyclictest0-21swapper/208:56:232
1863899526,4cyclictest0-21swapper/209:46:042
1863899524,3cyclictest7950irq/56-dwc_otg_07:50:502
18638995225,25cyclictest1340850wget05:46:052
18638995218,4cyclictest22906-21sh07:39:222
1863899521,5cyclictest2331-21rm07:53:262
1863799524,39cyclictest7850irq/56-dwc_otg_04:58:181
1863899513,4cyclictest7950irq/56-dwc_otg_07:01:142
1863899513,4cyclictest7950irq/56-dwc_otg_07:01:142
1863899512,9cyclictest0-21swapper/206:38:172
1863899511,5cyclictest7950irq/56-dwc_otg_07:18:322
1863899511,4cyclictest7850irq/56-dwc_otg_08:36:572
1863899511,4cyclictest7850irq/56-dwc_otg_08:36:572
1863799511,4cyclictest7458-21sshd05:28:351
1863899502,3cyclictest0-21swapper/206:51:422
18638995016,31cyclictest7950irq/56-dwc_otg_08:49:042
1863899501,4cyclictest7850irq/56-dwc_otg_09:15:492
1863899496,4cyclictest7750irq/56-dwc_otg09:30:532
1863899494,8cyclictest5598-21chrt05:22:512
1863899492,4cyclictest7850irq/56-dwc_otg_09:51:152
1863899491,9cyclictest0-21swapper/209:56:162
1863899491,5cyclictest28956-21ssh08:26:072
18638994915,4cyclictest21950-21seq06:06:242
1863899491,4cyclictest7950irq/56-dwc_otg_06:33:442
1863799495,35cyclictest7850irq/56-dwc_otg_06:12:211
1863899481,4cyclictest12376-21ls07:26:122
1863999472,23cyclictest0-21swapper/307:06:013
1863899475,4cyclictest7750irq/56-dwc_otg07:12:532
1863899471,5cyclictest4705-21ssh07:56:122
1863899471,5cyclictest4705-21ssh07:56:122
1863899471,3cyclictest7950irq/56-dwc_otg_09:35:512
1863899471,11cyclictest0-21swapper/207:10:052
1863899462,9cyclictest27219-21seq04:52:542
181522463,35sleep00-21swapper/004:28:100
1863899455,8cyclictest7850irq/56-dwc_otg_04:41:022
85202440,5sleep31863999cyclictest08:01:093
18639994430,12cyclictest0-21swapper/306:46:033
1863899442,5cyclictest0-21swapper/208:17:502
18639994226,10cyclictest381ktimersoftd/309:01:013
1863999417,22cyclictest0-21swapper/304:31:023
1863999417,22cyclictest0-21swapper/304:31:023
1863999412,4cyclictest7950irq/56-dwc_otg_08:31:023
1863999407,23cyclictest0-21swapper/305:55:553
1863999405,25cyclictest4643-21ssh08:36:033
1863999405,25cyclictest4643-21ssh08:36:033
173192400,1sleep30-21swapper/307:31:423
173192400,1sleep30-21swapper/307:31:423
1863999387,23cyclictest0-21swapper/305:06:023
1863999383,7cyclictest7850irq/56-dwc_otg_05:36:033
18639993819,5cyclictest29540-21diskmemload08:41:163
1863899381,5cyclictest18485-21ssh07:33:512
1863899381,5cyclictest18485-21ssh07:33:512
1863999375,5cyclictest7750irq/56-dwc_otg07:11:203
18639993723,4cyclictest7950irq/56-dwc_otg_06:56:053
18639993720,5cyclictest28608-21/usr/sbin/munin07:46:093
1863999371,15cyclictest7750irq/56-dwc_otg07:23:063
1863999369,8cyclictest7750irq/56-dwc_otg07:01:023
1863999369,8cyclictest7750irq/56-dwc_otg07:01:023
1863999361,4cyclictest7750irq/56-dwc_otg06:36:023
1863999361,24cyclictest21270-21latency06:06:093
1863899369,14cyclictest7850irq/56-dwc_otg_05:45:502
1863899362,21cyclictest32466-21grep05:06:172
1863999355,5cyclictest7850irq/56-dwc_otg_09:10:503
1863999354,3cyclictest0-21swapper/306:01:013
1863999351,23cyclictest14788-21ls06:51:063
18638993517,5cyclictest908-21runrttasks06:11:162
1863999348,5cyclictest7950irq/56-dwc_otg_09:46:183
1863999348,4cyclictest0-21swapper/308:20:513
1863999345,7cyclictest13508-21/usr/sbin/munin08:06:193
1863999342,15cyclictest7750irq/56-dwc_otg04:36:183
18639993414,5cyclictest3031-21sh09:12:323
1863999341,25cyclictest6036-21ls05:25:503
1863999336,6cyclictest7950irq/56-dwc_otg_06:26:163
18639993327,4cyclictest32631-21ls08:30:513
1863999331,5cyclictest7950irq/56-dwc_otg_05:41:023
1863999331,23cyclictest5731-21ls09:16:103
1863899335,12cyclictest7653-21cron05:30:492
1863899332,6cyclictest7950irq/56-dwc_otg_04:50:492
1863899332,16cyclictest7950irq/56-dwc_otg_05:15:502
98672320,2sleep39870-21latency_hist10:00:503
1863999326,14cyclictest7950irq/56-dwc_otg_07:42:473
1863999326,14cyclictest0-21swapper/304:46:043
18639993226,4cyclictest5648-21cron09:55:503
1863999322,20cyclictest0-21swapper/307:54:553
1863999322,18cyclictest0-21swapper/306:33:263
1863999322,17cyclictest0-21swapper/307:56:023
1863999322,17cyclictest0-21swapper/307:56:023
18639993218,11cyclictest0-21swapper/304:45:493
1863999321,23cyclictest20106-21ls07:36:043
1863999321,22cyclictest14047-21ls08:46:193
178282326,11sleep30-21swapper/304:26:193
1863999316,6cyclictest7950irq/56-dwc_otg_09:25:503
1863999315,6cyclictest7850irq/56-dwc_otg_04:56:183
1863999312,5cyclictest0-21swapper/309:26:003
18639993124,5cyclictest26423-21fschecks_time09:41:073
1863999308,4cyclictest0-21swapper/306:41:573
1863999302,4cyclictest7750irq/56-dwc_otg05:26:143
1863999301,21cyclictest0-21swapper/306:16:093
1863999301,12cyclictest7750irq/56-dwc_otg05:11:313
18639993010,4cyclictest0-21swapper/309:37:233
1863899302,10cyclictest28362-21tr04:56:112
1863999297,19cyclictest0-21swapper/306:11:053
1863999293,15cyclictest26834-21ntpq06:21:123
1863999293,15cyclictest26834-21ntpq06:21:123
1863999292,16cyclictest0-21swapper/305:51:033
1863999292,16cyclictest0-21swapper/305:51:033
18639992915,5cyclictest29560-21basename05:01:043
1863999291,18cyclictest0-21swapper/304:51:553
1863999282,16cyclictest0-21swapper/307:16:443
18638992815,4cyclictest25059-21ntp_states06:16:142
1863999275,11cyclictest7950irq/56-dwc_otg_05:46:173
1863999273,16cyclictest0-21swapper/305:31:083
1863899276,10cyclictest7850irq/56-dwc_otg_04:36:092
1863899272,4cyclictest7950irq/56-dwc_otg_05:36:042
1863999262,10cyclictest0-21swapper/305:16:083
1863999262,10cyclictest0-21swapper/305:16:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional