You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 11:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Jul 05, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2651921800,5sleep31377799cyclictest22:28:393
1670621120,2sleep00-21swapper/022:55:330
1375999966,74cyclictest0-21swapper/021:55:260
70122860,7sleep11376599cyclictest00:01:371
23422770,3sleep10-21swapper/100:35:311
23422770,3sleep10-21swapper/100:35:311
121522743,11sleep00-21swapper/019:05:290
121522743,11sleep00-21swapper/019:05:290
1375999727,30cyclictest0-21swapper/020:20:260
102742690,1sleep20-21swapper/222:09:212
26522680,1sleep30-21swapper/322:38:573
303772630,2sleep20-21swapper/223:50:392
1375999612,41cyclictest0-21swapper/000:20:230
1375999612,41cyclictest0-21swapper/000:20:230
13759996043,3cyclictest0-21swapper/022:05:230
13759996035,10cyclictest15019-21grep19:10:390
13759995937,4cyclictest19151-21ssh23:37:250
13759995937,10cyclictest101ktimersoftd/023:25:340
13759995921,23cyclictest13849-21latency_hist21:35:130
13759995836,10cyclictest101ktimersoftd/021:17:530
1375999582,47cyclictest0-21swapper/020:45:230
313962570,2sleep30-21swapper/323:52:053
1375999578,28cyclictest3402-21ls20:10:280
1375999574,8cyclictest0-21swapper/000:05:250
13759995738,5cyclictest0-21swapper/021:05:250
13759995726,8cyclictest0-21swapper/000:35:140
13759995726,8cyclictest0-21swapper/000:35:140
13759995713,36cyclictest0-21swapper/021:25:330
13759995713,36cyclictest0-21swapper/021:25:330
13759995637,9cyclictest101ktimersoftd/022:20:270
13759995631,12cyclictest0-21swapper/021:40:240
1375999562,29cyclictest5650-21grep23:20:400
1375999561,28cyclictest0-21swapper/019:30:130
13759995536,11cyclictest0-21swapper/019:49:400
140882540,2sleep30-21swapper/320:40:253
13759995437,3cyclictest0-21swapper/023:15:330
13759995434,9cyclictest32109-21aten_rbpower_vo22:35:270
13759995433,7cyclictest101ktimersoftd/000:15:320
13759995430,11cyclictest101ktimersoftd/022:00:310
13759995430,10cyclictest0-21swapper/019:20:250
1375999536,24cyclictest9-21ksoftirqd/023:40:130
1375999531,37cyclictest14711-21/usr/sbin/munin20:40:330
1375999531,27cyclictest12065-21latency_hist20:35:120
1376599526,12cyclictest7950irq/56-dwc_otg_22:30:271
1376599526,12cyclictest7950irq/56-dwc_otg_22:30:271
13765995213,28cyclictest0-21swapper/122:10:251
13759995240,4cyclictest101ktimersoftd/022:50:130
13759995231,7cyclictest29791-21grep23:10:390
13759995230,6cyclictest23098-21cut19:35:260
13759995230,12cyclictest24125-21ssh21:45:590
1375999522,7cyclictest18025-21munin-run21:40:120
1375999522,39cyclictest25284-21sort21:10:310
1375999521,29cyclictest2348-21sed21:20:400
66462510,5sleep31377799cyclictest22:43:413
1376599512,11cyclictest32736-21ssh23:15:261
13765995114,18cyclictest14275-21ssh00:10:361
13759995133,5cyclictest25902-21grep19:40:400
13759995133,5cyclictest25902-21grep19:40:400
13759995132,4cyclictest988-21sh23:55:220
13759995132,11cyclictest101ktimersoftd/022:32:300
13759995132,11cyclictest101ktimersoftd/022:32:300
13759995129,12cyclictest0-21swapper/000:30:410
1376599509,6cyclictest8050irq/56-dwc_otg_20:50:251
13765995016,4cyclictest8050irq/56-dwc_otg_21:40:251
13759995037,5cyclictest0-21swapper/022:50:260
1375999502,27cyclictest0-21swapper/022:20:140
1375999502,27cyclictest0-21swapper/022:20:140
1375999501,45cyclictest101ktimersoftd/020:05:400
1375999496,40cyclictest0-21swapper/020:15:250
13759994937,5cyclictest19728-21cut20:55:310
1375999493,43cyclictest31556-21ssh23:52:210
13759994929,12cyclictest7850irq/56-dwc_otg22:15:130
13759994921,7cyclictest7950irq/56-dwc_otg_20:00:120
13759994918,21cyclictest0-21swapper/000:01:590
13759994917,23cyclictest0-21swapper/020:50:380
1376599485,30cyclictest0-21swapper/120:45:231
13765994821,8cyclictest8050irq/56-dwc_otg_21:57:241
1376599481,27cyclictest0-21swapper/122:55:141
13759994831,4cyclictest23832-21fschecks_count22:25:290
13759994828,6cyclictest7850irq/56-dwc_otg21:05:130
13759994826,8cyclictest0-21swapper/023:04:380
13759994821,5cyclictest7950irq/56-dwc_otg_22:41:330
1375999481,38cyclictest16081-21ssh00:13:010
1375999481,33cyclictest16087-21awk19:15:310
1375999481,28cyclictest25988-21aten_rbpower_en00:25:270
1375999481,23cyclictest25619-21ssh23:45:310
1376599479,4cyclictest0-21swapper/100:15:311
13765994723,8cyclictest8050irq/56-dwc_otg_23:55:271
13765994722,8cyclictest7850irq/56-dwc_otg23:40:231
13765994719,13cyclictest221ktimersoftd/121:15:281
1375999472,28cyclictest0-21swapper/020:00:240
13759994719,21cyclictest0-21swapper/023:07:580
270152460,3sleep121-21rcuc/123:46:401
1377199463,7cyclictest31-21ksoftirqd/221:15:252
13759994630,3cyclictest0-21swapper/021:50:310
13759994628,10cyclictest101ktimersoftd/019:55:130
1376599456,13cyclictest9008-21ssh23:25:311
13765994520,9cyclictest7850irq/56-dwc_otg00:06:491
13765994520,5cyclictest7850irq/56-dwc_otg00:30:121
13765994520,12cyclictest7950irq/56-dwc_otg_23:10:261
13765994519,6cyclictest7850irq/56-dwc_otg22:29:111
13759994532,4cyclictest12833-21cut23:30:270
13759994514,17cyclictest9111-21uname20:25:320
13759994514,17cyclictest9111-21uname20:25:320
217592440,2sleep2301ktimersoftd/222:22:342
1376599442,21cyclictest19932-21kernelversion22:20:311
13765994411,11cyclictest8050irq/56-dwc_otg_20:40:121
13765994410,6cyclictest8050irq/56-dwc_otg_21:45:261
1375999441,10cyclictest21540-21cut19:30:310
52062430,2sleep20-21swapper/222:02:412
13777994324,5cyclictest1010650ssh00:05:373
13771994317,12cyclictest727-21runrttasks20:15:392
1376599436,20cyclictest14510-21ssh23:31:301
13765994322,9cyclictest7850irq/56-dwc_otg23:22:541
13765994320,8cyclictest7850irq/56-dwc_otg21:32:011
13765994316,9cyclictest7850irq/56-dwc_otg19:10:391
13765994311,17cyclictest8050irq/56-dwc_otg_20:00:131
47082420,21sleep28050irq/56-dwc_otg_21:24:362
13771994213,18cyclictest30005-1kworker/u9:223:15:342
1376599428,15cyclictest31604-21ssh23:52:271
1376599425,21cyclictest24015-21sh23:05:061
1376599425,19cyclictest5469-21ssh22:03:091
13765994210,6cyclictest1560-21ls20:05:271
47552410,1sleep30-21swapper/321:24:413
292522411,35sleep30-21swapper/319:50:393
222462410,1sleep30-21swapper/322:23:273
1376599417,27cyclictest0-21swapper/119:40:261
1376599417,27cyclictest0-21swapper/119:40:261
1376599417,23cyclictest0-21swapper/121:05:251
13765994115,12cyclictest7850irq/56-dwc_otg22:44:041
13765994113,7cyclictest221ktimersoftd/121:15:141
13759994114,6cyclictest12564-21fschecks_time20:35:290
1377799402,6cyclictest37-21rcuc/319:20:123
1376599407,25cyclictest0-21swapper/120:00:261
1376599403,30cyclictest8050irq/56-dwc_otg_19:45:251
13765994018,4cyclictest7950irq/56-dwc_otg_20:21:231
13765994011,5cyclictest2386-21unixbench_singl21:20:411
144882390,2sleep20-21swapper/222:52:502
1377199393,5cyclictest0-21swapper/221:55:262
13771993919,4cyclictest1119-21sh23:55:262
13771993915,14cyclictest7950irq/56-dwc_otg_19:15:422
13771993913,16cyclictest3256-21ssh22:40:022
1377199391,26cyclictest0-21swapper/221:38:302
1377199391,19cyclictest11078-21grep21:30:412
13771993910,6cyclictest31-21ksoftirqd/200:10:252
1376599397,23cyclictest0-21swapper/121:25:251
1376599397,23cyclictest0-21swapper/121:25:251
1376599395,20cyclictest27034-21ssh23:08:021
1376599392,23cyclictest16494-21sh21:37:351
13765993920,4cyclictest8050irq/56-dwc_otg_22:56:151
13765993917,6cyclictest221ktimersoftd/122:35:261
13777993813,12cyclictest22082-21cut00:20:303
13777993813,12cyclictest22082-21cut00:20:303
1377799381,22cyclictest1259-21ls20:05:133
13771993825,7cyclictest7950irq/56-dwc_otg_22:30:132
1377199381,25cyclictest29023-21ls19:50:342
1376599388,13cyclictest11759-21sshd20:32:251
1376599387,27cyclictest0-21swapper/120:25:261
1376599387,27cyclictest0-21swapper/120:25:261
1376599385,18cyclictest14209-21cpu20:40:261
13765993817,8cyclictest7850irq/56-dwc_otg22:06:011
13765993815,10cyclictest221ktimersoftd/120:10:421
13777993718,11cyclictest5193-21apt-get00:00:283
1377799371,5cyclictest8050irq/56-dwc_otg_21:35:133
1377199374,19cyclictest7950irq/56-dwc_otg_00:17:392
1377199373,5cyclictest0-21swapper/219:40:262
1377199373,5cyclictest0-21swapper/219:40:262
13771993718,11cyclictest31-21ksoftirqd/220:45:402
1377199371,4cyclictest8050irq/56-dwc_otg_00:35:242
1377199371,4cyclictest8050irq/56-dwc_otg_00:35:242
13771993711,18cyclictest0-21swapper/222:32:062
13771993711,18cyclictest0-21swapper/222:32:062
13771993710,14cyclictest24644-21diskmemload21:50:342
1376599378,15cyclictest0-21swapper/119:25:181
1376599372,14cyclictest0-21swapper/119:33:451
1376599371,9cyclictest8029-21ssh22:45:271
13765993715,11cyclictest221ktimersoftd/100:21:121
13765993715,11cyclictest221ktimersoftd/100:21:121
13765993714,5cyclictest17236-21cron19:20:121
13765993714,10cyclictest221ktimersoftd/123:37:251
1377799367,14cyclictest7950irq/56-dwc_otg_21:45:133
1377799362,13cyclictest7950irq/56-dwc_otg_22:45:253
13777993619,6cyclictest25399-21df23:45:283
13777993619,5cyclictest18780-21sh22:19:413
13777993619,5cyclictest18780-21sh22:19:413
13777993616,4cyclictest25233-21if_eth019:40:323
13777993616,4cyclictest25233-21if_eth019:40:323
1377799361,5cyclictest9358-21apt-get21:30:133
1377799361,5cyclictest9358-21apt-get21:30:133
1377799361,22cyclictest25962-21ssh23:06:053
13771993612,7cyclictest7950irq/56-dwc_otg_23:35:302
1377199361,25cyclictest3069-21ls20:10:132
13771993610,13cyclictest0-21swapper/223:10:122
1376599366,23cyclictest30118-21ssh00:30:271
90982350,15sleep21377199cyclictest20:25:312
90982350,15sleep21377199cyclictest20:25:312
322242350,5sleep21377199cyclictest20:00:272
273552350,4sleep327348-21ssh21:50:303
185072350,2sleep30-21swapper/323:36:153
142182350,2sleep30-21swapper/322:13:573
138222350,3sleep213823-21modprobe20:40:122
1377799359,19cyclictest0-21swapper/321:11:073
13777993518,9cyclictest727-21runrttasks22:33:433
13777993518,9cyclictest727-21runrttasks22:33:433
1377799351,19cyclictest12592-21ls23:30:143
1377199359,13cyclictest0-21swapper/223:33:292
13771993516,9cyclictest727-21runrttasks19:36:422
13771993516,5cyclictest31-21ksoftirqd/200:05:272
1377199351,19cyclictest24223-21ssh23:05:142
1377199350,24cyclictest0-21swapper/200:30:392
1376599358,4cyclictest518-21rs:main1
13765993513,5cyclictest16017-21ntp_kernel_pll_22:15:361
13765993513,5cyclictest16017-21ntp_kernel_pll_22:15:361
1376599351,19cyclictest0-21swapper/121:52:341
216682340,4sleep324644-21diskmemload23:40:343
1377799341,23cyclictest29528-21switchtime23:10:363
13777993411,14cyclictest0-21swapper/323:04:293
1377199345,12cyclictest8050irq/56-dwc_otg_23:40:332
1377199344,20cyclictest7950irq/56-dwc_otg_22:55:412
1377199343,14cyclictest27373-21wc19:45:372
1377199342,9cyclictest32389-21munin-run23:15:122
13771993420,8cyclictest7950irq/56-dwc_otg_19:55:382
13771993417,5cyclictest727-21runrttasks20:55:352
1377199341,6cyclictest16429-21ls22:15:412
1377199341,6cyclictest16429-21ls22:15:412
13771993414,18cyclictest2453750ssh21:46:422
13771993412,14cyclictest0-21swapper/222:49:432
13771993412,11cyclictest25086-21rm00:24:182
13771993412,11cyclictest25086-21rm00:24:182
13771993410,13cyclictest0-21swapper/221:26:352
13771993410,13cyclictest0-21swapper/221:26:352
1376599347,4cyclictest0-21swapper/121:00:241
1376599347,15cyclictest0-21swapper/119:50:261
1377799333,5cyclictest22327-21grep19:30:413
13777993327,4cyclictest39-21ksoftirqd/322:50:403
13777993317,6cyclictest16213-21sh23:34:373
1377799331,4cyclictest8050irq/56-dwc_otg_00:35:313
1377799331,4cyclictest8050irq/56-dwc_otg_00:35:313
13777993313,11cyclictest381ktimersoftd/323:20:223
1377799331,24cyclictest24681-21ls21:10:133
1377199336,16cyclictest8050irq/56-dwc_otg_23:20:392
1377199335,14cyclictest8050irq/56-dwc_otg_20:20:412
13771993325,6cyclictest8688-21ssh00:04:412
1377199332,2cyclictest0-21swapper/200:25:252
1377199331,23cyclictest0-21swapper/221:13:112
13771993311,13cyclictest0-21swapper/221:41:382
1376599337,14cyclictest0-21swapper/119:35:261
13765993316,9cyclictest7850irq/56-dwc_otg20:15:591
1377799326,6cyclictest27119-21unixbench-2d00:25:413
1377799325,13cyclictest8050irq/56-dwc_otg_22:00:353
1377799323,22cyclictest8050irq/56-dwc_otg_19:20:243
13777993224,4cyclictest0-21swapper/321:45:243
13777993212,4cyclictest381ktimersoftd/300:10:263
1377799321,21cyclictest21030-21latency_hist21:00:133
13777993212,13cyclictest308932chrt00:30:383
13777993210,14cyclictest0-21swapper/323:20:103
1377199329,6cyclictest0-21swapper/220:10:332
1377199328,7cyclictest0-21swapper/219:10:262
1377199327,5cyclictest7950irq/56-dwc_otg_19:30:412
1377199327,18cyclictest7950irq/56-dwc_otg_21:05:412
13771993210,7cyclictest0-21swapper/223:28:392
1377799318,11cyclictest32723-21sh21:56:193
1377799317,6cyclictest7770-21/usr/sbin/munin20:20:393
1377799314,5cyclictest7950irq/56-dwc_otg_22:05:263
13777993115,6cyclictest29531-21ssh21:15:303
13777993111,14cyclictest0-21swapper/321:35:403
13777993110,9cyclictest17031-21processes22:55:383
1377199319,4cyclictest0-21swapper/222:40:292
1377199319,4cyclictest0-21swapper/220:50:312
1377199312,10cyclictest7950irq/56-dwc_otg_20:35:122
1377199311,7cyclictest14511-21/usr/sbin/munin20:40:312
13771993110,10cyclictest0-21swapper/222:10:182
1377799309,10cyclictest6581-21modprobe20:20:133
1377799308,12cyclictest0-21swapper/321:01:293
1377799302,10cyclictest0-21swapper/320:05:323
1377799301,6cyclictest17482-21wc20:50:133
13777993011,5cyclictest3663-21/usr/sbin/munin20:10:333
1377799301,12cyclictest7850irq/56-dwc_otg19:55:373
1377199308,11cyclictest7950irq/56-dwc_otg_23:45:282
1377199301,14cyclictest21558-21uniq21:00:302
1377799298,9cyclictest27694-21unixbench-2d19:45:403
1377799294,5cyclictest7850irq/56-dwc_otg19:10:283
1377799293,10cyclictest8050irq/56-dwc_otg_20:35:123
1377799292,4cyclictest0-21swapper/323:57:023
1377799291,4cyclictest37-21rcuc/300:15:413
13777992910,11cyclictest18266-21cat20:50:343
1376599291,6cyclictest17521-21wget19:20:261
1377799289,5cyclictest381ktimersoftd/319:25:153
1377199282,11cyclictest8050irq/56-dwc_otg_19:25:292
133462283,16sleep30-21swapper/319:08:403
133462283,16sleep30-21swapper/319:08:403
128992287,11sleep212999-21/usr/sbin/munin19:05:392
128992287,11sleep212999-21/usr/sbin/munin19:05:392
13777992720,5cyclictest39-21ksoftirqd/320:25:233
13777992720,5cyclictest39-21ksoftirqd/320:25:233
1377199275,4cyclictest0-21swapper/219:20:372
131102273,14sleep10-21swapper/119:05:431
131102273,14sleep10-21swapper/119:05:431
1377799252,6cyclictest521-21munin-node19:35:233
1377799251,4cyclictest12242-21apt-get20:35:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional