You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 19:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 23, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275999912017,85cyclictest0-21swapper/009:35:410
27600991183,78cyclictest27197-21perl09:35:411
27600991073,78cyclictest28108-21ssh11:35:501
27599991064,72cyclictest0-21swapper/011:40:510
275999910115,22cyclictest10997-21ssh09:15:440
2759999983,62cyclictest0-21swapper/012:04:500
27599999554,14cyclictest7850irq/56-dwc_otg09:52:470
2759999933,58cyclictest0-21swapper/011:35:510
2759999904,18cyclictest2885-21ssh12:25:350
2760099883,78cyclictest24012-21proc_pri12:10:491
2760099882,79cyclictest32144-21sort11:40:531
2759999884,77cyclictest0-21swapper/012:10:500
2760099833,58cyclictest31520-21apt-get08:50:361
2760099832,74cyclictest8965-21ssh09:52:471
2760099824,70cyclictest11193-21taskset09:15:481
2760099823,73cyclictest18374-21ssh12:04:481
2759999786,20cyclictest8050irq/56-dwc_otg_07:20:290
27602997621,24cyclictest11100-21ssh09:15:463
2759999766,35cyclictest0-21swapper/008:20:350
59002740,1sleep30-21swapper/310:29:083
2760099742,8cyclictest18987-21cut09:25:411
210462740,4sleep00-21swapper/012:07:160
210462740,4sleep00-21swapper/012:07:160
44652730,2sleep20-21swapper/210:26:252
2760099734,59cyclictest29905-21df08:45:361
2760199723,39cyclictest0-21swapper/212:04:502
2759999727,48cyclictest10995-21wget10:35:360
27599997237,15cyclictest6815-21ssh11:50:340
2759999721,5cyclictest10524-21modprobe11:15:220
2759999721,5cyclictest10524-21modprobe11:15:220
27602997041,23cyclictest27105-21fschecks_count09:35:403
2760099707,46cyclictest7850irq/56-dwc_otg12:30:411
166552690,2sleep10-21swapper/109:22:171
27602996830,15cyclictest381ktimersoftd/309:52:493
27601996826,16cyclictest24031-21wc12:10:492
2759999681,51cyclictest0-21swapper/011:30:450
2759999681,51cyclictest0-21swapper/011:30:450
27602996738,23cyclictest31075-21/usr/sbin/munin11:40:503
27601996711,9cyclictest27114-21ssh09:35:402
2760099671,53cyclictest11646-21aten_rbpower_vo07:55:381
2759999678,47cyclictest0-21swapper/008:35:370
2759999676,22cyclictest0-21swapper/008:55:380
2759999674,60cyclictest6902-21aten_rbpower_cu10:30:360
2759999674,60cyclictest6902-21aten_rbpower_cu10:30:360
27601996616,26cyclictest11002-21ssh09:15:452
2760099661,46cyclictest7850irq/56-dwc_otg10:55:371
27599996648,5cyclictest101ktimersoftd/012:19:150
14842660,2sleep20-21swapper/211:43:422
27602996515,9cyclictest8050irq/56-dwc_otg_12:10:483
2760199653,39cyclictest0-21swapper/211:35:512
2760099657,40cyclictest7850irq/56-dwc_otg11:08:051
27600996510,37cyclictest0-21swapper/112:05:361
27600996510,37cyclictest0-21swapper/112:05:361
27599996531,7cyclictest7950irq/56-dwc_otg_09:25:220
2760099642,49cyclictest0-21swapper/110:35:481
2759999647,25cyclictest0-21swapper/009:40:350
27599996441,10cyclictest0-21swapper/011:03:480
27602996335,22cyclictest750-21snmpd11:35:513
2760099635,44cyclictest7950irq/56-dwc_otg_09:48:211
2760099634,50cyclictest7950irq/56-dwc_otg_10:27:261
27600996310,37cyclictest23319-21sort10:10:391
27600996310,37cyclictest23319-21sort10:10:391
2759999632,38cyclictest14722-21latency_hist12:00:230
302972620,1sleep10-21swapper/112:19:051
2760199621,30cyclictest0-21swapper/210:05:292
2760099625,45cyclictest1652-1kworker/u9:011:04:341
2760099624,43cyclictest7850irq/56-dwc_otg10:47:161
2759999621,28cyclictest0-21swapper/010:25:340
27602996117,14cyclictest14672-21cat12:00:233
2760099611,8cyclictest0-21swapper/112:37:141
2760099611,8cyclictest0-21swapper/112:37:141
2759999613,42cyclictest15333-21aten_rbpower_po08:05:370
2760299604,27cyclictest0-21swapper/312:04:483
27602996016,25cyclictest11389-21idleruntime12:35:403
27602996016,25cyclictest11389-21idleruntime12:35:403
2760099601,3cyclictest0-21swapper/112:29:331
27600996011,45cyclictest11997-21ssh09:55:551
27599996036,11cyclictest16576-21chrt10:41:170
27599996035,11cyclictest101ktimersoftd/009:25:500
27602995915,7cyclictest18589-21munin-run11:25:223
27602995913,13cyclictest1469-21ssh11:03:243
27601995913,17cyclictest11212-21/usr/sbin/munin10:35:392
2760099591,4cyclictest7850irq/56-dwc_otg10:30:351
2760099591,4cyclictest7850irq/56-dwc_otg10:30:351
2760099591,4cyclictest7850irq/56-dwc_otg10:02:151
2760099591,43cyclictest7850irq/56-dwc_otg11:20:221
2759999596,38cyclictest18585-21latency_hist11:25:230
2759999594,44cyclictest4594-21chrt11:46:540
2759999592,44cyclictest24045-21ssh09:30:540
2759999591,48cyclictest0-21swapper/008:50:510
27599995911,24cyclictest0-21swapper/011:25:470
27599995910,35cyclictest11213-21df12:35:360
27599995910,35cyclictest11213-21df12:35:360
2760199584,31cyclictest0-21swapper/209:52:472
2760099588,38cyclictest8050irq/56-dwc_otg_08:10:521
2760099582,5cyclictest0-21swapper/110:07:481
2760099582,43cyclictest0-21swapper/111:46:521
2760099582,40cyclictest7850irq/56-dwc_otg11:28:361
27600995814,40cyclictest15153-21aten_rbpower_po10:40:371
2760099581,42cyclictest8050irq/56-dwc_otg_08:05:421
27599995832,4cyclictest101ktimersoftd/008:10:430
2759999582,35cyclictest14624-21wc10:00:230
2759999581,53cyclictest0-21swapper/010:05:230
27602995714,13cyclictest4439-21ssh11:06:303
2760099576,4cyclictest7950irq/56-dwc_otg_09:31:451
2760099572,5cyclictest9342-21ssh09:13:571
2760099572,42cyclictest0-21swapper/111:23:061
2760099572,40cyclictest0-21swapper/109:42:211
27600995719,35cyclictest9971-21fschecks_count07:50:401
2760099571,5cyclictest0-21swapper/109:00:351
2760099571,42cyclictest0-21swapper/111:59:291
27599995739,5cyclictest7648-21ntp_kernel_pll_12:30:450
2759999572,38cyclictest0-21swapper/007:45:490
27599995720,16cyclictest0-21swapper/011:06:270
2759999571,24cyclictest0-21swapper/012:20:460
221312570,2sleep20-21swapper/211:29:202
2760299568,14cyclictest9300-21ssh09:13:513
27602995622,15cyclictest698-21ssh09:42:383
2760099562,7cyclictest0-21swapper/110:21:261
2760099561,8cyclictest0-21swapper/108:00:491
2760099561,6cyclictest734-21ssh12:21:591
2759999564,46cyclictest0-21swapper/009:10:360
27602995517,11cyclictest10573-21ssh12:35:133
2760199551,22cyclictest0-21swapper/208:05:342
27601995510,31cyclictest20211-21ls12:05:542
27601995510,31cyclictest20211-21ls12:05:542
2760099552,49cyclictest5464-21grep07:35:521
2760099551,8cyclictest28688-21grep07:10:501
2760099551,5cyclictest0-21swapper/107:50:221
2760099551,4cyclictest0-21swapper/111:52:491
27599995529,5cyclictest27533-21fschecks_count10:55:390
2759999552,33cyclictest26807-21rm10:55:110
2759999552,33cyclictest26807-21rm10:55:110
2759999551,14cyclictest0-21swapper/010:22:090
27602995417,12cyclictest17638-21cat08:10:443
27602995416,18cyclictest19667-21memory12:05:463
27602995416,18cyclictest19667-21memory12:05:463
2760099545,45cyclictest32459-21unixbench_multi07:20:521
2760099542,3cyclictest0-21swapper/110:18:381
27600995415,36cyclictest27972-21/usr/sbin/munin08:40:361
27600995415,36cyclictest27972-21/usr/sbin/munin08:40:361
2760099541,40cyclictest0-21swapper/111:32:051
2760099541,40cyclictest0-21swapper/111:32:051
2760099541,39cyclictest26015-1kworker/u9:111:10:481
2760099541,39cyclictest26015-1kworker/u9:111:10:481
27599995424,13cyclictest0-21swapper/008:45:390
2759999541,31cyclictest14500-21unixbench_singl08:00:510
2759999541,29cyclictest0-21swapper/007:55:390
27602995319,12cyclictest6317-21ssh12:29:543
27602995317,10cyclictest28550-21ssh10:16:073
27602995315,23cyclictest11742-21/usr/sbin/munin10:35:473
2760199539,16cyclictest14018-21ntpq08:00:452
2760199539,12cyclictest8795-21sh11:12:012
2760199539,12cyclictest8795-21sh11:12:012
27601995311,15cyclictest1785-21ls08:55:512
2760099533,6cyclictest22590-21cut08:25:371
27600995312,37cyclictest26769-21ntp_kernel_pll_08:35:461
27600995311,39cyclictest55282chrt09:05:501
27599995334,7cyclictest0-21swapper/011:17:120
27599995327,17cyclictest101ktimersoftd/010:10:490
27599995327,17cyclictest101ktimersoftd/010:10:490
2760299527,27cyclictest0-21swapper/308:55:383
2760199526,22cyclictest24236-21awk10:10:512
2760199526,22cyclictest24236-21awk10:10:512
2760099522,7cyclictest26993-21latency_hist10:55:231
2760099522,7cyclictest26993-21latency_hist10:55:231
2759999522,17cyclictest0-21swapper/010:45:340
27599995211,31cyclictest101ktimersoftd/008:45:240
27599995211,31cyclictest101ktimersoftd/008:45:240
27602995120,15cyclictest19989-21ssh09:25:543
2760199519,25cyclictest8050irq/56-dwc_otg_11:15:392
2760199519,15cyclictest16703-21rm10:41:312
2760099512,8cyclictest25089-21grep08:30:471
2760099512,7cyclictest1816-21grep07:25:501
2760099511,6cyclictest2830-21fschecks_time07:30:401
27599995137,7cyclictest101ktimersoftd/009:45:400
27599995131,9cyclictest0-21swapper/010:15:500
27602995015,14cyclictest16536-21sh10:41:123
2760199508,5cyclictest0-21swapper/207:40:362
27601995011,12cyclictest24358-21ssh11:30:532
27601995011,12cyclictest24358-21ssh11:30:532
2760099502,6cyclictest1968-21sed08:55:531
2760099502,5cyclictest0-21swapper/108:15:381
27602994921,8cyclictest19435-21irqstats11:25:423
27602994919,11cyclictest32265-21/usr/sbin/munin08:50:473
27602994918,20cyclictest24551-21rm11:31:163
27602994918,20cyclictest24551-21rm11:31:163
27602994917,16cyclictest15257-21aten_rbpower_cu08:05:363
27602994917,12cyclictest22552-21sh10:49:553
27602994913,11cyclictest6051-21diskmemload11:47:253
27602994912,22cyclictest10219-21ssh11:14:493
27602994912,22cyclictest10219-21ssh11:14:493
2760199498,16cyclictest9131-21ssh12:32:352
2760099493,3cyclictest7950irq/56-dwc_otg_07:19:471
2760099492,8cyclictest7265-21grep07:40:511
2759999492,33cyclictest30707-21taskset07:16:510
27599994923,13cyclictest4308-21/usr/sbin/munin07:35:360
2759999491,26cyclictest4801-21fschecks_time09:05:400
27602994817,14cyclictest29587-21ssh10:57:533
27602994814,19cyclictest8050irq/56-dwc_otg_10:10:363
27602994814,19cyclictest8050irq/56-dwc_otg_10:10:363
27602994812,28cyclictest0-21swapper/307:12:073
27602994810,15cyclictest0-21swapper/308:00:463
2760199488,25cyclictest0-21swapper/210:15:372
2760199483,13cyclictest11289-21fschecks_count12:35:382
2760199483,13cyclictest11289-21fschecks_count12:35:382
2759999484,24cyclictest0-21swapper/009:00:420
27599994832,8cyclictest0-21swapper/010:07:440
27599994828,7cyclictest0-21swapper/008:25:400
27599994822,10cyclictest0-21swapper/007:27:200
27599994812,17cyclictest27909-21fschecks_time07:10:390
2760299478,25cyclictest0-21swapper/307:55:363
27602994718,13cyclictest18191-21rm10:04:513
2760199474,18cyclictest1985-21ssh10:24:272
27601994720,9cyclictest8050irq/56-dwc_otg_12:21:592
27599994725,7cyclictest25761-21chrt08:34:330
2760299468,9cyclictest0-21swapper/312:20:253
2760299467,9cyclictest7850irq/56-dwc_otg10:55:143
2760299467,9cyclictest7850irq/56-dwc_otg10:55:143
2760299467,13cyclictest21268-21ssh10:07:453
2760299465,26cyclictest0-21swapper/309:31:313
27602994619,14cyclictest12146-21/usr/sbin/munin11:15:533
27602994618,15cyclictest8050irq/56-dwc_otg_07:20:523
27602994617,11cyclictest28870-21ssh12:16:303
27602994613,13cyclictest568-21ssh10:21:483
27601994611,8cyclictest29788-21taskset10:58:162
27601994610,15cyclictest5697-21ssh11:08:542
2760099462,5cyclictest0-21swapper/108:20:381
2759999462,31cyclictest0-21swapper/007:40:400
2760299458,20cyclictest8050irq/56-dwc_otg_09:57:003
2760199458,29cyclictest0-21swapper/211:52:302
271112453,35sleep00-21swapper/007:07:330
2760299448,19cyclictest8050irq/56-dwc_otg_10:32:583
2760299448,19cyclictest8050irq/56-dwc_otg_10:32:583
27602994416,16cyclictest3622-21cat09:45:513
27602994414,7cyclictest24290-21apt-get08:30:343
27602994411,10cyclictest7850irq/56-dwc_otg08:45:353
27601994425,11cyclictest8050irq/56-dwc_otg_10:46:512
2759999441,22cyclictest0-21swapper/008:18:290
27602994311,16cyclictest17982-21ssh09:24:453
2760299428,22cyclictest7850irq/56-dwc_otg11:52:273
2760199428,13cyclictest1483-21rm11:03:262
2760199426,12cyclictest3696-21sed09:45:512
27601994219,8cyclictest7950irq/56-dwc_otg_09:42:432
27601994215,13cyclictest301ktimersoftd/209:31:512
2760299419,28cyclictest0-21swapper/307:45:373
2760299415,16cyclictest200892sleep308:16:203
27602994117,20cyclictest22711-21fschecks_count08:25:393
2760199413,12cyclictest30571-21ssh12:19:362
27601994117,6cyclictest8050irq/56-dwc_otg_12:27:192
27601994117,10cyclictest8050irq/56-dwc_otg_11:23:022
27601994111,20cyclictest0-21swapper/210:00:372
27599994120,12cyclictest101ktimersoftd/007:52:540
27599994116,14cyclictest0-21swapper/007:30:500
2760299407,15cyclictest8050irq/56-dwc_otg_09:01:253
27602994014,12cyclictest5820-21seq07:39:463
27601994019,10cyclictest8050irq/56-dwc_otg_09:26:002
2760199401,8cyclictest8050irq/56-dwc_otg_09:05:352
27601994014,7cyclictest8050irq/56-dwc_otg_10:55:112
27601994014,7cyclictest8050irq/56-dwc_otg_10:55:112
2760299396,14cyclictest0-21swapper/308:23:563
2760199397,14cyclictest32261-21proc_pri08:50:472
2760199393,3cyclictest8050irq/56-dwc_otg_07:10:352
27601993916,8cyclictest6871-21apt-get10:30:332
27601993916,8cyclictest6871-21apt-get10:30:332
2760199391,4cyclictest0-21swapper/207:30:362
2760199391,27cyclictest7370-21ls09:10:492
27602993814,10cyclictest5963-21taskset09:09:333
27602993810,10cyclictest29435-21taskset08:45:143
27602993810,10cyclictest29435-21taskset08:45:143
2760199382,22cyclictest8050irq/56-dwc_otg_08:45:372
27601993817,7cyclictest8050irq/56-dwc_otg_09:56:362
27601993810,13cyclictest14549-21/usr/sbin/munin09:20:332
2760199380,9cyclictest0-21swapper/207:50:052
2760299376,18cyclictest1811-21grep07:25:503
27602993723,11cyclictest29935-21latency07:15:443
27602993721,13cyclictest6679-21load07:40:433
2760199376,20cyclictest29-21rcuc/207:20:292
2760299362,14cyclictest7850irq/56-dwc_otg07:30:373
27602993610,11cyclictest27597-21seq08:39:523
27601993622,4cyclictest7950irq/56-dwc_otg_08:13:442
27601993616,4cyclictest8050irq/56-dwc_otg_11:47:252
2760199361,11cyclictest3933-21taskset09:01:412
2760199342,5cyclictest31010-21apt-get07:20:222
2760199342,17cyclictest0-21swapper/211:55:372
27601993414,4cyclictest7950irq/56-dwc_otg_07:55:342
27601993414,10cyclictest0-21swapper/208:32:182
2760199341,18cyclictest0-21swapper/208:35:362
204312340,4sleep2125-21systemd-journal08:20:222
27602993311,12cyclictest9708-21chrt07:50:243
2760199327,4cyclictest7950irq/56-dwc_otg_08:20:412
2760199322,17cyclictest0-21swapper/207:50:332
27601993217,4cyclictest8050irq/56-dwc_otg_07:28:032
2760199321,14cyclictest29252-21runrttasks08:42:502
2760199321,14cyclictest29252-21runrttasks08:42:502
257982325,8sleep10-21swapper/107:05:361
2760199315,8cyclictest0-21swapper/208:26:352
2760199291,16cyclictest0-21swapper/207:37:302
271282273,14sleep20-21swapper/207:07:472
267892265,9sleep30-21swapper/307:05:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional