You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 13:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Thu Jan 08, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
45462980,1sleep10-21swapper/121:16:151
2259099896,62cyclictest0-21swapper/000:00:300
2259099896,49cyclictest0-21swapper/020:50:280
2259099871,25cyclictest0-21swapper/020:55:290
2259099851,67cyclictest5332-21grep00:15:430
2259099841,64cyclictest21158-21grep20:35:270
22590998121,40cyclictest0-21swapper/023:05:330
22590998033,36cyclictest0-21swapper/022:00:280
2259099802,55cyclictest0-21swapper/023:25:150
2259099801,60cyclictest27538-21aten_rbpower_po23:50:290
28052790,5sleep32259399cyclictest00:10:343
22590997922,37cyclictest3930-21aten_rbpower_vo22:45:290
2259099771,56cyclictest29216-21cat22:25:340
22590997614,45cyclictest7656-21/usr/sbin/munin21:25:360
2259099742,57cyclictest0-21swapper/021:20:320
22590997425,31cyclictest23719-21grep20:40:390
2259099732,56cyclictest0-21swapper/020:20:160
2259099721,20cyclictest27378-21tail19:20:410
326832710,5sleep32259399cyclictest22:35:313
2259099712,20cyclictest0-21swapper/019:54:100
249902700,1sleep10-21swapper/120:45:321
249902700,1sleep10-21swapper/120:45:321
2259099701,48cyclictest0-21swapper/021:01:320
2259099672,38cyclictest0-21swapper/023:45:270
2259099672,31cyclictest0-21swapper/019:10:260
2259099672,20cyclictest0-21swapper/021:07:120
2259099671,6cyclictest0-21swapper/000:10:270
2259099662,18cyclictest0-21swapper/023:25:350
22590996621,20cyclictest0-21swapper/021:45:280
2259099661,14cyclictest0-21swapper/000:35:380
22590996512,9cyclictest0-21swapper/021:20:130
290942640,2sleep30-21swapper/322:25:333
2259099648,14cyclictest0-21swapper/023:45:150
22590996441,7cyclictest7750irq/56-dwc_otg19:45:310
22590996325,10cyclictest24436-21sshd19:15:270
22590996325,10cyclictest24436-21sshd19:15:270
2259099632,18cyclictest0-21swapper/020:12:320
22590996321,38cyclictest12842sleep022:36:530
22590996316,8cyclictest0-21swapper/023:00:270
2259099622,20cyclictest0-21swapper/021:44:200
22590996214,9cyclictest8728-21runrttasks22:56:570
133252620,2sleep10-21swapper/123:10:351
133252620,2sleep10-21swapper/123:10:351
2259099619,4cyclictest7850irq/56-dwc_otg_20:05:380
2259099619,13cyclictest0-21swapper/020:25:270
2259099614,17cyclictest0-21swapper/020:48:270
2259099614,17cyclictest0-21swapper/020:48:270
2259099612,7cyclictest11367-21latency_hist00:35:150
22590996114,39cyclictest0-21swapper/023:14:010
22590996114,39cyclictest0-21swapper/023:14:010
2259099611,35cyclictest7736-21grep19:55:440
22590996112,39cyclictest0-21swapper/021:39:210
2259099611,20cyclictest0-21swapper/022:22:490
2259099611,20cyclictest0-21swapper/022:22:490
2259099605,5cyclictest17149-21grep21:50:420
2259099601,33cyclictest15775-21unixbench_multi23:15:420
22590996012,6cyclictest7950irq/56-dwc_otg_21:15:150
22590996012,6cyclictest7950irq/56-dwc_otg_21:15:150
2259099599,10cyclictest0-21swapper/020:01:550
2259099599,10cyclictest0-21swapper/020:01:550
2259099591,15cyclictest20098-21/usr/sbin/munin23:30:340
22590995911,10cyclictest0-21swapper/022:06:120
2259099582,34cyclictest0-21swapper/000:30:150
2259099582,34cyclictest0-21swapper/000:30:150
22590995816,8cyclictest0-21swapper/023:36:280
2259099581,53cyclictest0-21swapper/020:30:360
2259099572,28cyclictest0-21swapper/022:30:320
11612570,2sleep00-21swapper/019:40:270
2259099564,31cyclictest0-21swapper/021:35:150
2259099564,31cyclictest0-21swapper/021:35:150
2259099562,25cyclictest0-21swapper/022:10:250
2259099561,2cyclictest0-21swapper/021:55:410
2259099561,29cyclictest31555-21latency_hist19:35:150
22591995528,6cyclictest31839-21latency_hist21:05:151
2259099551,12cyclictest0-21swapper/022:51:330
2259099549,12cyclictest0-21swapper/020:22:320
2259099542,13cyclictest0-21swapper/019:26:250
2259099541,18cyclictest1125-21runrttasks00:07:190
2259099541,16cyclictest0-21swapper/019:38:510
2259099541,13cyclictest0-21swapper/022:17:180
22590995310,9cyclictest0-21swapper/000:21:200
2259099528,9cyclictest0-21swapper/023:56:290
22591994921,14cyclictest24621-21latency_hist20:45:161
22590994910,9cyclictest0-21swapper/022:42:360
2259199471,13cyclictest0-21swapper/122:55:271
2259299467,24cyclictest0-21swapper/222:55:272
2259299457,25cyclictest0-21swapper/221:24:192
2259299457,24cyclictest0-21swapper/223:00:292
22591994520,5cyclictest8613-21fschecks_count20:00:331
22591994520,5cyclictest8613-21fschecks_count20:00:331
22591994417,11cyclictest1751750wget20:25:291
221792443,33sleep00-21swapper/019:08:270
2259299437,26cyclictest0-21swapper/222:40:212
2259199437,23cyclictest0-21swapper/123:55:211
2259199437,22cyclictest0-21swapper/122:25:281
69302420,4sleep36931-21timerandwakeup00:20:413
2259299427,25cyclictest0-21swapper/222:50:282
22592994213,6cyclictest29107-21latency_hist23:55:152
22591994229,8cyclictest1125-21runrttasks20:50:261
22591994219,5cyclictest29867-21idleruntime-cro21:00:141
2259199412,3cyclictest0-21swapper/119:55:271
22591994118,5cyclictest22254-21fschecks_time23:35:331
2259399402,5cyclictest7950irq/56-dwc_otg_21:05:273
22592994015,16cyclictest5424-21netstat19:50:382
2259399399,14cyclictest7950irq/56-dwc_otg_23:50:153
2259399391,30cyclictest0-21swapper/320:20:013
2259299397,20cyclictest0-21swapper/223:20:252
2259299391,25cyclictest0-21swapper/223:55:212
2259299391,24cyclictest0-21swapper/220:20:272
2259299391,12cyclictest9933-21grep21:30:422
2259299391,12cyclictest9933-21grep21:30:422
2259199397,23cyclictest0-21swapper/121:24:191
2259199391,28cyclictest26067-21latency23:45:331
2259399387,19cyclictest0-21swapper/323:55:213
2259399384,7cyclictest0-21swapper/319:20:243
22592993814,4cyclictest31677-21latency_hist21:05:152
2259299381,25cyclictest31668-21apt-config19:35:162
2259299381,14cyclictest0-21swapper/200:15:262
2259199383,25cyclictest0-21swapper/119:25:361
2259199382,24cyclictest7204-21modprobe22:55:151
2259199381,4cyclictest7750irq/56-dwc_otg00:30:261
221802380,7sleep122181-21sshd19:08:271
163072380,2sleep30-21swapper/323:20:153
2259399377,16cyclictest2216-21unixbench-2d19:40:403
22593993729,6cyclictest39-21ksoftirqd/319:15:153
2259299376,25cyclictest0-21swapper/222:45:202
2259299376,25cyclictest0-21swapper/222:25:282
2259299376,25cyclictest0-21swapper/220:30:292
2259299371,6cyclictest26652-21grep23:45:412
2259299371,28cyclictest8052-21cut00:25:312
2259299371,28cyclictest8052-21cut00:25:312
2259299371,24cyclictest13977-21latency_hist21:45:152
2259299371,17cyclictest0-21swapper/219:40:372
2259199372,4cyclictest7950irq/56-dwc_otg_23:15:261
2259199371,6cyclictest4158-21tr00:15:261
22591993713,4cyclictest23672-21unixbench-2d19:10:411
22591993712,6cyclictest26830-21date22:20:161
2259199371,21cyclictest32342-21grep21:05:331
24462360,2sleep20-21swapper/221:10:412
24462360,2sleep20-21swapper/221:10:412
22593993612,11cyclictest28603-21cut20:55:333
2259299361,19cyclictest0-21swapper/222:00:352
22591993613,4cyclictest11842-21fschecks_time00:35:311
2259199361,25cyclictest0-21swapper/120:30:291
2259199361,18cyclictest0-21swapper/122:20:371
2259199361,18cyclictest0-21swapper/122:20:371
2259399353,25cyclictest7950irq/56-dwc_otg_21:24:183
2259299357,15cyclictest0-21swapper/222:05:302
2259299351,8cyclictest16058-21wget21:50:282
2259299351,25cyclictest0-21swapper/222:20:292
2259299351,25cyclictest0-21swapper/222:20:292
2259299351,23cyclictest579-21timerwakeupswit21:05:422
22592993512,12cyclictest301ktimersoftd/221:56:082
2259199353,11cyclictest7610-21latency_hist00:25:161
2259199352,5cyclictest29037-21munin-run23:55:141
2259199352,11cyclictest26242-21/usr/sbin/munin19:20:411
2259199351,6cyclictest9943-21unixbench_singl21:30:421
2259199351,6cyclictest9943-21unixbench_singl21:30:421
22591993516,4cyclictest19408-21latency_hist22:00:151
2259199351,4cyclictest1904-21proc_pri19:40:371
2259199351,28cyclictest2586-21grep00:10:321
2259399345,16cyclictest7950irq/56-dwc_otg_23:00:453
2259399342,17cyclictest37-21rcuc/321:17:403
2259299343,6cyclictest7750irq/56-dwc_otg00:05:422
22592993416,4cyclictest6038-21aten_rbpower_en00:20:282
2259299341,23cyclictest12247-21cut20:10:312
2259199342,4cyclictest0-21swapper/120:20:411
2259199342,16cyclictest7950irq/56-dwc_otg_19:35:321
2259199342,13cyclictest7950irq/56-dwc_otg_19:20:151
2259199342,13cyclictest7950irq/56-dwc_otg_19:20:151
2259199341,5cyclictest16113-21aten_rbpower_po21:50:281
22591993413,5cyclictest1125-21runrttasks20:38:321
2259199341,30cyclictest30235-21ls19:30:321
2259199341,13cyclictest0-21swapper/100:30:161
2259399338,11cyclictest7950irq/56-dwc_otg_23:35:403
2259399337,11cyclictest7950irq/56-dwc_otg_20:35:153
2259399335,5cyclictest7750irq/56-dwc_otg20:00:383
2259399335,5cyclictest7750irq/56-dwc_otg20:00:383
2259399334,15cyclictest7950irq/56-dwc_otg_22:55:343
22593993327,4cyclictest32329-21latency_hist22:35:153
2259399331,25cyclictest16288-21cut21:50:313
2259399331,19cyclictest3476-21latency_hist22:45:153
2259199334,4cyclictest14140-21fschecks_time20:15:311
2259199331,6cyclictest6327-21apt19:55:151
2259199331,5cyclictest18517-21df_inode23:25:311
22591993312,5cyclictest2249-21fschecks_time22:40:331
22591993312,12cyclictest221ktimersoftd/122:45:311
2259399327,20cyclictest0-21swapper/323:50:283
2259399322,12cyclictest7950irq/56-dwc_otg_20:55:013
22592993211,4cyclictest23996-21proc_pri22:10:372
2259299321,14cyclictest14971-21wc21:45:392
2259199328,11cyclictest10857-21fschecks_count21:35:311
22591993225,5cyclictest23-21ksoftirqd/122:05:281
2259199322,20cyclictest1056-21ls00:05:351
2259199321,7cyclictest10008-21grep23:00:411
2259199321,5cyclictest20397-21fschecks_time23:30:331
2259199321,23cyclictest0-21swapper/123:20:251
2259199321,21cyclictest0-21swapper/121:40:271
2259199321,15cyclictest0-21swapper/100:01:261
2259199321,13cyclictest0-21swapper/120:12:271
2259399318,10cyclictest7950irq/56-dwc_otg_19:15:433
2259399318,10cyclictest7950irq/56-dwc_otg_19:15:433
2259399315,13cyclictest7950irq/56-dwc_otg_00:31:433
2259399314,14cyclictest7950irq/56-dwc_otg_23:40:423
2259299313,8cyclictest7950irq/56-dwc_otg_19:10:422
2259299311,7cyclictest31871-21cut19:35:292
22592993110,11cyclictest0-21swapper/220:40:412
2259199316,13cyclictest7950irq/56-dwc_otg_00:25:311
2259199315,12cyclictest0-21swapper/122:36:371
2259199314,10cyclictest3270-21fschecks_time19:45:311
2259199312,15cyclictest0-21swapper/123:43:591
22591993117,5cyclictest11366-21fschecks_time23:05:331
22591993115,4cyclictest221ktimersoftd/122:35:141
22591993113,10cyclictest7861-21/usr/sbin/munin21:25:401
2259199311,20cyclictest0-21swapper/122:10:271
171862311,24sleep30-21swapper/323:20:363
2259399306,11cyclictest7750irq/56-dwc_otg22:15:303
2259399305,12cyclictest7950irq/56-dwc_otg_20:40:243
2259399304,6cyclictest39-21ksoftirqd/300:15:313
2259399304,13cyclictest7950irq/56-dwc_otg_22:10:243
2259399303,8cyclictest20098-21/usr/sbin/munin23:30:373
2259399301,3cyclictest0-21swapper/321:25:403
2259299308,14cyclictest25537-21unixbench_multi19:15:432
2259299308,14cyclictest25537-21unixbench_multi19:15:432
2259299308,10cyclictest0-21swapper/223:05:352
2259299306,15cyclictest1124-21snmpd23:25:312
2259299306,14cyclictest0-21swapper/200:30:282
22592993022,6cyclictest128-21systemd-journal23:40:152
2259299301,5cyclictest20398-21fschecks_time23:30:332
22592993010,5cyclictest0-21swapper/221:35:312
2259199301,10cyclictest0-21swapper/121:45:391
2259399295,12cyclictest7950irq/56-dwc_otg_19:30:283
2259399294,4cyclictest7750irq/56-dwc_otg20:10:253
2259399292,6cyclictest6655-21wget19:55:303
22593992922,4cyclictest18996-21sh20:30:153
2259399292,19cyclictest0-21swapper/300:29:293
2259399292,19cyclictest0-21swapper/300:29:293
2259399291,8cyclictest15386-21head21:45:433
22593992914,7cyclictest0-21swapper/323:05:333
2259299296,14cyclictest0-21swapper/222:35:272
2259299296,11cyclictest0-21swapper/220:05:142
2259299296,11cyclictest0-21swapper/220:05:142
2259299292,5cyclictest28357-21wget20:55:292
2259299291,5cyclictest0-21swapper/200:35:302
22592992912,4cyclictest22131-21unixbench_multi20:35:402
2259299291,14cyclictest27962-21ls19:25:152
22592992911,11cyclictest14103-21grep20:15:312
22592992910,5cyclictest4089-21unixbench_singl19:45:412
2259199297,6cyclictest10463-21fschecks_count20:05:311
2259199296,11cyclictest0-21swapper/122:00:401
2259199292,5cyclictest2222-21sensors21:10:391
2259199292,5cyclictest2222-21sensors21:10:391
220742294,17sleep31124-21snmpd19:07:043
2259399286,5cyclictest0-21swapper/321:35:273
2259399286,16cyclictest7950irq/56-dwc_otg_22:50:263
2259399283,12cyclictest7950irq/56-dwc_otg_19:35:333
2259399281,8cyclictest7950irq/56-dwc_otg_19:25:273
22593992812,11cyclictest39-21ksoftirqd/321:10:263
22593992812,11cyclictest39-21ksoftirqd/321:10:263
2259399281,11cyclictest7950irq/56-dwc_otg_20:45:343
2259399281,11cyclictest7950irq/56-dwc_otg_20:45:343
2259299288,10cyclictest30856-21cut22:30:322
2259299287,12cyclictest7750irq/56-dwc_otg23:10:392
2259299287,12cyclictest7750irq/56-dwc_otg23:10:392
2259299281,9cyclictest26253-21idleruntime-cro20:50:152
2259299281,9cyclictest26253-21idleruntime-cro20:50:152
22592992815,4cyclictest3286-21/usr/sbin/munin21:15:252
22592992815,4cyclictest15731-21/usr/sbin/munin23:15:422
2259299281,4cyclictest0-21swapper/223:45:152
16852280,2sleep30-21swapper/300:05:433
2259399275,11cyclictest7950irq/56-dwc_otg_21:40:423
2259399274,12cyclictest0-21swapper/322:45:343
2259399273,13cyclictest7950irq/56-dwc_otg_21:30:333
2259399273,13cyclictest7950irq/56-dwc_otg_21:30:333
2259399271,6cyclictest7950irq/56-dwc_otg_20:40:163
2259399271,3cyclictest7750irq/56-dwc_otg20:05:353
2259399271,15cyclictest18529-21diskstats23:25:313
2259299276,14cyclictest18343-21taskset20:25:412
22592992721,4cyclictest31226-21aten_rbpower_en00:00:302
2259399268,6cyclictest7950irq/56-dwc_otg_00:35:363
2259399265,10cyclictest7750irq/56-dwc_otg22:00:253
2259399263,8cyclictest0-21swapper/321:55:263
2259399262,20cyclictest0-21swapper/320:20:313
2259399261,7cyclictest26891-21/usr/sbin/munin22:20:353
2259399261,7cyclictest26891-21/usr/sbin/munin22:20:353
2259299266,11cyclictest7950irq/56-dwc_otg_21:25:332
2259299264,14cyclictest0-21swapper/220:05:332
2259299262,7cyclictest3442-21tail00:10:432
2259299261,4cyclictest0-21swapper/222:15:312
222082264,15sleep20-21swapper/219:08:492
2259399252,15cyclictest0-21swapper/321:05:013
2259399251,5cyclictest12931-21grep23:10:303
2259399251,5cyclictest12931-21grep23:10:303
2259299256,4cyclictest7850irq/56-dwc_otg_19:55:292
22592992512,7cyclictest7950irq/56-dwc_otg_20:54:472
2259299251,13cyclictest0-21swapper/219:27:322
2259399241,5cyclictest4884-21wget19:50:303
2259399241,12cyclictest7950irq/56-dwc_otg_22:08:443
2259399232,3cyclictest7950irq/56-dwc_otg_00:00:423
2259399232,18cyclictest0-21swapper/319:45:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional