You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 18:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Oct 30, 2025 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1839921940,5sleep10-21swapper/107:05:511
1952299861,39cyclictest0-21swapper/009:00:360
233392820,6sleep21952499cyclictest12:05:412
1952299731,39cyclictest0-21swapper/007:50:380
1952299731,33cyclictest0-21swapper/012:35:390
76542720,2sleep17623-21ssh12:25:531
20082710,2sleep10-21swapper/110:50:121
1952499716,48cyclictest8050irq/56-dwc_otg_07:30:532
1952499702,50cyclictest0-21swapper/211:25:402
1952499702,50cyclictest0-21swapper/211:25:402
1952299701,50cyclictest24373-21sed07:21:000
100042700,1sleep10-21swapper/109:26:381
100042700,1sleep10-21swapper/109:26:381
1952499694,47cyclictest8050irq/56-dwc_otg_07:20:422
19522996947,14cyclictest101ktimersoftd/007:40:520
28282670,1sleep30-21swapper/310:50:423
1952499674,49cyclictest1785-21tune2fs09:15:422
1952499664,49cyclictest8050irq/56-dwc_otg_10:40:552
1952299654,51cyclictest0-21swapper/011:05:370
319512640,2sleep10-21swapper/111:31:021
1952499643,39cyclictest7850irq/56-dwc_otg10:15:442
1952499641,7cyclictest30728-21/usr/sbin/munin12:15:492
19524996410,41cyclictest32492-21netstat10:00:472
1952299641,36cyclictest0-21swapper/012:30:370
1952299641,35cyclictest0-21swapper/008:55:380
1952299638,32cyclictest0-21swapper/011:55:560
1952299632,35cyclictest0-21swapper/009:40:380
1952299632,30cyclictest0-21swapper/010:35:410
19522996318,29cyclictest0-21swapper/010:05:410
1952499624,41cyclictest8050irq/56-dwc_otg_07:25:482
1952499622,5cyclictest5393-21/usr/sbin/munin09:20:432
1952299622,38cyclictest9-21ksoftirqd/011:20:240
1952499616,38cyclictest8050irq/56-dwc_otg_12:30:512
1952499615,41cyclictest8050irq/56-dwc_otg_10:23:502
1952499612,3cyclictest0-21swapper/210:06:332
1952499611,5cyclictest9812-21cut11:00:452
19522996122,4cyclictest0-21swapper/012:20:370
183542615,46sleep00-21swapper/007:05:510
1952499603,48cyclictest8050irq/56-dwc_otg_11:31:592
1952499602,46cyclictest0-21swapper/208:07:152
1952499601,47cyclictest26447-21ssh10:38:042
1952499592,40cyclictest0-21swapper/211:23:232
1952499591,5cyclictest16614-21ssh11:10:402
1952499591,45cyclictest13411-21switchtime08:20:532
1952499591,45cyclictest13411-21switchtime08:20:532
19522995941,5cyclictest12718-21ls08:20:420
19522995941,5cyclictest12718-21ls08:20:420
19522995940,11cyclictest101ktimersoftd/012:05:370
19522995931,10cyclictest101ktimersoftd/008:25:540
1952299591,33cyclictest21116-21munin-plugin-st07:15:250
1952299591,33cyclictest21116-21munin-plugin-st07:15:250
135492590,3sleep10-21swapper/112:33:381
1952499583,39cyclictest8050irq/56-dwc_otg_12:00:082
1952499582,42cyclictest7950irq/56-dwc_otg_07:15:402
1952499581,47cyclictest0-21swapper/211:40:492
19524995814,33cyclictest4281-21ssh10:52:092
19522995843,4cyclictest14420-21idleruntime-cro09:35:240
19522995836,7cyclictest7950irq/56-dwc_otg_11:50:560
19522995836,6cyclictest20006-21fschecks_count08:40:420
19522995836,5cyclictest26863-21/usr/sbin/munin07:30:480
19522995834,5cyclictest18579-21tail10:25:540
19522995827,13cyclictest7950irq/56-dwc_otg_11:35:440
19522995820,23cyclictest13-21rcuc/010:40:400
1952499571,42cyclictest8050irq/56-dwc_otg_09:35:512
1952499571,39cyclictest7850irq/56-dwc_otg09:44:162
1952299572,29cyclictest0-21swapper/009:35:480
19522995717,12cyclictest0-21swapper/007:25:370
159762570,2sleep0101ktimersoftd/010:23:010
1952499566,36cyclictest8050irq/56-dwc_otg_10:32:232
1952499562,49cyclictest21080-21tail11:15:532
1952499562,38cyclictest8050irq/56-dwc_otg_12:39:152
1952499561,4cyclictest0-21swapper/208:48:262
1952499561,3cyclictest0-21swapper/208:17:542
1952399561,40cyclictest7850irq/56-dwc_otg07:45:381
1952299561,30cyclictest28939-21fschecks_count07:35:430
1952299561,30cyclictest28939-21fschecks_count07:35:430
19522995612,5cyclictest0-21swapper/011:00:380
1952599555,9cyclictest8050irq/56-dwc_otg_08:45:373
1952499552,9cyclictest0-21swapper/209:45:362
1952499551,42cyclictest8050irq/56-dwc_otg_10:27:032
1952499551,42cyclictest0-21swapper/212:30:142
1952499551,3cyclictest0-21swapper/211:35:372
1952499551,38cyclictest8050irq/56-dwc_otg_09:50:382
19522995542,7cyclictest0-21swapper/012:25:370
19522995529,3cyclictest0-21swapper/008:15:370
1952299551,33cyclictest24822-21latency_hist09:50:250
1952299551,31cyclictest5620-21fschecks_time08:00:420
1952499544,4cyclictest7950irq/56-dwc_otg_12:20:472
1952499544,4cyclictest0-21swapper/209:10:532
1952499544,4cyclictest0-21swapper/209:10:532
1952499544,3cyclictest0-21swapper/211:06:212
1952499543,4cyclictest7950irq/56-dwc_otg_12:14:282
1952499541,4cyclictest6639-21apt-get10:10:362
1952299543,35cyclictest0-21swapper/008:33:240
19522995429,14cyclictest32264-21ssh11:31:450
19522995429,12cyclictest0-21swapper/009:25:400
19522995429,12cyclictest0-21swapper/009:25:400
19522995422,15cyclictest31971-21ntpq10:45:480
1952299541,39cyclictest0-21swapper/010:15:520
1952499535,29cyclictest0-21swapper/210:48:192
1952499533,3cyclictest8050irq/56-dwc_otg_09:30:252
1952499533,3cyclictest8050irq/56-dwc_otg_09:30:252
1952499532,6cyclictest28963-21fschecks_time07:35:432
1952499532,6cyclictest28963-21fschecks_time07:35:432
1952499531,3cyclictest0-21swapper/207:55:442
1952399532,18cyclictest9550irq/48-DMA10:48:091
1952299538,15cyclictest0-21swapper/007:45:380
1952299537,29cyclictest9-21ksoftirqd/010:00:430
1952299534,29cyclictest18195-21fschecks_count08:35:420
19522995331,3cyclictest0-21swapper/010:55:520
19522995331,3cyclictest0-21swapper/010:55:520
1952299532,26cyclictest0-21swapper/011:25:370
1952299532,26cyclictest0-21swapper/011:25:370
19522995316,21cyclictest13-21rcuc/011:45:500
71042520,2sleep30-21swapper/308:05:373
1952499527,41cyclictest11850-21apt-get09:30:362
1952499524,3cyclictest8050irq/56-dwc_otg_08:00:392
1952499524,35cyclictest8050irq/56-dwc_otg_09:05:512
1952499522,6cyclictest6023-21ssh10:55:382
1952499522,6cyclictest6023-21ssh10:55:382
1952499521,4cyclictest0-21swapper/207:10:442
1952499521,4cyclictest0-21swapper/207:10:442
1952499521,47cyclictest19778-21aten_rbpower_cu08:40:382
1952499521,3cyclictest0-21swapper/208:12:552
1952299527,24cyclictest2427-21taskset10:50:350
19522995234,4cyclictest8050irq/56-dwc_otg_12:15:370
1952499512,45cyclictest30531-21sshd07:40:352
1952499511,5cyclictest8050irq/56-dwc_otg_10:00:252
1952499511,4cyclictest15440-21ssh11:54:302
1952499511,2cyclictest0-21swapper/211:45:392
1952299511,34cyclictest0-21swapper/012:00:430
1952299511,34cyclictest0-21swapper/012:00:430
1952299511,31cyclictest24733-21unixbench_singl11:20:580
1952299511,13cyclictest1273-21latency_hist09:15:250
1952299511,13cyclictest1273-21latency_hist09:15:250
1952499502,45cyclictest15991-21latency_hist08:30:242
1952499502,45cyclictest0-21swapper/207:45:512
1952499501,46cyclictest26990-21grep09:00:392
1952499501,3cyclictest8050irq/56-dwc_otg_08:37:362
1952499492,4cyclictest17751-21latency_hist08:35:242
1952499491,4cyclictest0-21swapper/208:50:512
1952499491,44cyclictest8050irq/56-dwc_otg_12:00:532
1952499491,44cyclictest8050irq/56-dwc_otg_12:00:532
1952499491,3cyclictest0-21swapper/207:51:202
1952399497,25cyclictest0-21swapper/109:15:381
19522994926,4cyclictest7950irq/56-dwc_otg_11:40:490
1952299492,18cyclictest0-21swapper/009:55:380
1952299491,28cyclictest21603-21grep08:45:390
1952299491,27cyclictest0-21swapper/010:30:450
1952299491,21cyclictest0-21swapper/008:10:370
1952599484,11cyclictest7950irq/56-dwc_otg_08:20:383
1952599484,11cyclictest7950irq/56-dwc_otg_08:20:383
19522994835,5cyclictest8050irq/56-dwc_otg_09:55:240
19522994821,12cyclictest0-21swapper/010:12:390
19522994812,4cyclictest0-21swapper/009:20:370
1952499471,5cyclictest25819-21ntpq08:55:472
19523994722,9cyclictest2511450wget07:25:401
19523994710,25cyclictest0-21swapper/108:00:381
1952299471,30cyclictest0-21swapper/009:18:360
1952399467,24cyclictest0-21swapper/109:45:301
1952399462,30cyclictest0-21swapper/112:35:391
1952299463,5cyclictest9-21ksoftirqd/012:11:040
1952299462,7cyclictest22453-21unixbench_multi07:15:530
1952299462,25cyclictest0-21swapper/011:10:450
19523994512,8cyclictest8050irq/56-dwc_otg_09:10:391
19523994512,8cyclictest8050irq/56-dwc_otg_09:10:391
1952399451,28cyclictest22915-21awk12:05:251
1952399451,28cyclictest22915-21awk12:05:251
1952599448,26cyclictest0-21swapper/308:50:383
1952399444,7cyclictest11-21rcu_preempt07:35:381
1952399444,7cyclictest11-21rcu_preempt07:35:381
19523994425,5cyclictest27900-21wget10:40:401
1952299449,18cyclictest9-21ksoftirqd/008:50:380
1952299442,31cyclictest0-21swapper/008:06:010
1952599437,24cyclictest0-21swapper/312:10:383
1952599437,22cyclictest0-21swapper/310:30:393
1952599437,22cyclictest0-21swapper/307:50:393
1952599436,23cyclictest0-21swapper/311:30:413
1952399432,5cyclictest0-21swapper/108:35:361
19522994311,18cyclictest13-21rcuc/007:59:280
1952599427,24cyclictest0-21swapper/312:15:303
1952599427,23cyclictest0-21swapper/309:15:383
1952599427,22cyclictest0-21swapper/309:45:303
19525994220,5cyclictest13887-21grep10:20:403
1952399427,23cyclictest0-21swapper/109:20:371
19523994215,4cyclictest3236-21fschecks_count10:05:451
1952299421,22cyclictest0-21swapper/009:06:410
146642420,3sleep3381ktimersoftd/311:06:443
1952599417,24cyclictest0-21swapper/311:00:383
1952599414,5cyclictest8050irq/56-dwc_otg_09:25:363
1952599414,5cyclictest8050irq/56-dwc_otg_09:25:363
1952399415,14cyclictest0-21swapper/109:00:361
19523994115,14cyclictest0-21swapper/110:10:301
1952599407,6cyclictest0-21swapper/307:25:373
1952399407,21cyclictest0-21swapper/107:50:391
1952399405,29cyclictest8050irq/56-dwc_otg_10:50:351
1952399403,16cyclictest0-21swapper/108:45:371
19523994019,4cyclictest0-21swapper/110:15:461
19525993918,10cyclictest12440-21sh12:31:373
19525993917,12cyclictest21880-21sh12:03:053
19525993917,12cyclictest21880-21sh12:03:053
1952399397,6cyclictest23831-21grep11:20:471
19523993918,5cyclictest28609-21df09:55:411
1952399391,3cyclictest0-21swapper/111:10:371
1952399391,19cyclictest0-21swapper/109:40:381
1952399388,8cyclictest0-21swapper/108:55:361
1952399387,24cyclictest0-21swapper/110:30:391
19523993810,18cyclictest26331-1kworker/u9:010:00:541
19525993721,7cyclictest37-21rcuc/309:40:383
19525993716,5cyclictest13440-21sh10:20:243
1952399378,12cyclictest0-21swapper/111:45:441
1952399375,5cyclictest8050irq/56-dwc_otg_07:55:501
1952399372,3cyclictest0-21swapper/112:20:591
19523993714,5cyclictest0-21swapper/111:35:521
1952599362,5cyclictest0-21swapper/311:15:373
19525993617,10cyclictest5905-21sh10:09:433
1952399368,7cyclictest0-21swapper/109:50:361
1952399367,15cyclictest0-21swapper/111:55:371
1952399366,7cyclictest7950irq/56-dwc_otg_09:10:251
1952399366,12cyclictest23-21ksoftirqd/107:30:501
1952399361,4cyclictest20219-21perl11:15:421
1952399361,21cyclictest30587-21ssh12:15:251
1952399361,21cyclictest27294-21ssh11:25:411
1952399361,21cyclictest27294-21ssh11:25:411
1952599359,6cyclictest7850irq/56-dwc_otg09:10:243
1952599359,13cyclictest0-21swapper/310:39:023
1952599352,3cyclictest7950irq/56-dwc_otg_09:10:393
1952599352,3cyclictest7950irq/56-dwc_otg_09:10:393
19525993512,14cyclictest28053-21latency_hist09:55:253
1952399359,13cyclictest0-21swapper/111:40:381
1952399358,4cyclictest0-21swapper/110:25:381
1952399358,16cyclictest0-21swapper/110:55:351
1952399358,16cyclictest0-21swapper/110:55:351
1952399353,11cyclictest0-21swapper/109:30:371
19523993510,4cyclictest0-21swapper/111:55:251
19523993510,12cyclictest7950irq/56-dwc_otg_08:10:451
286422340,8sleep1221ktimersoftd/112:11:561
1952599349,4cyclictest0-21swapper/312:39:323
1952599341,22cyclictest3416-21ls12:20:523
1952599341,22cyclictest0-21swapper/312:27:313
1952399346,16cyclictest0-21swapper/110:20:391
1952399342,12cyclictest0-21swapper/110:35:391
1952399341,20cyclictest12822-21fschecks_time08:20:451
1952399341,20cyclictest12822-21fschecks_time08:20:451
1952399341,15cyclictest0-21swapper/107:15:361
1952599338,4cyclictest7850irq/56-dwc_otg07:55:503
1952599336,15cyclictest0-21swapper/308:00:383
19525993315,4cyclictest15711-21switchtime09:35:533
1952599331,20cyclictest0-21swapper/309:23:503
1952599331,18cyclictest32019-21ls10:00:403
1952399337,14cyclictest0-21swapper/108:50:381
1952399336,5cyclictest7850irq/56-dwc_otg07:10:391
1952399336,5cyclictest7850irq/56-dwc_otg07:10:391
1952399331,3cyclictest7850irq/56-dwc_otg08:25:381
1952599327,15cyclictest0-21swapper/312:05:393
1952599324,11cyclictest39-21ksoftirqd/308:15:373
1952599323,4cyclictest8050irq/56-dwc_otg_08:55:363
1952599322,8cyclictest0-21swapper/310:27:493
1952599321,23cyclictest12231-21latency_hist11:50:253
19525993210,3cyclictest0-21swapper/311:25:413
19525993210,3cyclictest0-21swapper/311:25:413
1952399326,7cyclictest0-21swapper/111:04:381
1952399324,13cyclictest8050irq/56-dwc_otg_12:10:251
1952399321,6cyclictest8661-21latency_hist08:10:241
1952599317,5cyclictest11966-21aten_rbpower_vo09:30:423
1952599315,19cyclictest0-21swapper/310:43:413
1952599314,11cyclictest8050irq/56-dwc_otg_08:35:243
1952599313,11cyclictest7950irq/56-dwc_otg_07:35:393
1952599313,11cyclictest7950irq/56-dwc_otg_07:35:393
1952599312,6cyclictest7850irq/56-dwc_otg11:40:413
1952599311,6cyclictest7850irq/56-dwc_otg08:35:403
19525993115,4cyclictest19195-21sh12:00:073
1952399313,6cyclictest0-21swapper/108:30:361
19523993114,4cyclictest1111-21runrttasks07:44:461
1952399311,13cyclictest0-21swapper/109:36:151
1952599309,4cyclictest0-21swapper/311:35:483
1952599306,4cyclictest8050irq/56-dwc_otg_10:14:433
1952599305,6cyclictest7950irq/56-dwc_otg_08:10:423
1952599303,14cyclictest8050irq/56-dwc_otg_07:15:533
1952599303,13cyclictest8050irq/56-dwc_otg_11:20:543
1952599301,5cyclictest9158-21idleruntime-cro11:00:253
1952599301,5cyclictest9158-21idleruntime-cro11:00:253
19525993011,4cyclictest0-21swapper/309:57:243
1952599301,12cyclictest8050irq/56-dwc_otg_07:21:003
19525993010,4cyclictest0-21swapper/310:48:093
1952399301,4cyclictest0-21swapper/111:05:491
1952599297,19cyclictest0-21swapper/307:30:383
1952599296,4cyclictest7850irq/56-dwc_otg11:50:513
19525992919,7cyclictest19787-21fschecks_count07:10:423
19525992919,7cyclictest19787-21fschecks_count07:10:423
19525992916,4cyclictest19734-21/usr/sbin/munin08:40:523
19523992911,11cyclictest11946-21taskset08:16:301
1952599289,4cyclictest7950irq/56-dwc_otg_11:10:273
1952599280,22cyclictest0-21swapper/307:45:493
177292284,14sleep30-21swapper/307:05:423
1952399272,11cyclictest0-21swapper/108:40:481
1952599263,4cyclictest30528-21apt-get07:40:353
176962263,14sleep20-21swapper/207:05:292
1952599234,4cyclictest7950irq/56-dwc_otg_08:25:393
1952599231,6cyclictest27071-21wget09:00:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional