You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 19:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Jul 28, 2026 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25594991276,99cyclictest7850irq/56-dwc_otg11:00:431
255939912143,20cyclictest0-21swapper/010:15:420
255939911710,45cyclictest25830-21ssh11:35:450
255949911010,67cyclictest5014-21grep12:30:251
255949911010,67cyclictest5014-21grep12:30:251
25593991084,14cyclictest0-21swapper/011:00:420
25593991083,75cyclictest0-21swapper/012:15:520
25594991073,79cyclictest25370-21ssh10:15:431
255939910757,9cyclictest8050irq/56-dwc_otg_10:45:350
255939910757,9cyclictest8050irq/56-dwc_otg_10:45:350
25593999726,59cyclictest0-21swapper/008:15:400
25593999347,29cyclictest0-21swapper/011:25:530
2559499923,10cyclictest0-21swapper/111:25:531
2559399924,51cyclictest0-21swapper/007:45:350
25593999110,66cyclictest0-21swapper/010:25:530
173732910,2sleep30-21swapper/310:45:393
173732910,2sleep30-21swapper/310:45:393
2559499893,60cyclictest0-21swapper/112:15:521
2559399893,58cyclictest0-21swapper/009:35:570
156662890,5sleep02559399cyclictest12:02:570
2559499884,77cyclictest25820-21tune2fs11:35:441
2559399873,60cyclictest0-21swapper/012:30:270
2559399873,60cyclictest0-21swapper/012:30:270
2559499866,74cyclictest26059-21timerwakeupswit09:35:551
2559499863,59cyclictest0-21swapper/112:30:291
2559499854,11cyclictest21412-21apt-get12:10:401
164042840,2sleep1221ktimersoftd/110:44:411
172252830,5sleep20-21swapper/210:45:382
172252830,5sleep20-21swapper/210:45:382
208242810,2sleep10-21swapper/112:09:591
2559499803,10cyclictest2136-21taskset10:25:561
115272790,3sleep20-21swapper/212:37:262
25595997836,22cyclictest7950irq/56-dwc_otg_12:30:252
25595997836,22cyclictest7950irq/56-dwc_otg_12:30:252
25596997751,20cyclictest8050irq/56-dwc_otg_11:30:383
25596997718,31cyclictest381ktimersoftd/311:00:423
2559699766,25cyclictest381ktimersoftd/310:15:423
2559399751,31cyclictest0-21swapper/010:35:370
2559399735,42cyclictest0-21swapper/012:10:400
25593997352,5cyclictest7950irq/56-dwc_otg_09:40:350
2559399732,8cyclictest20608-21idleruntime-cro10:10:250
189912720,2sleep10-21swapper/109:27:371
2559399712,46cyclictest5243-21grep07:40:530
251932714,12sleep20-21swapper/207:08:542
25596996821,26cyclictest26451-21awk12:15:523
25596996810,10cyclictest7950irq/56-dwc_otg_10:25:543
25595996818,35cyclictest125-21systemd-journal12:30:282
2559399682,38cyclictest4305-21sh09:50:250
2559399682,15cyclictest8050irq/56-dwc_otg_12:30:280
2559499674,11cyclictest0-21swapper/111:20:351
2559399673,44cyclictest0-21swapper/008:50:420
2559399673,44cyclictest0-21swapper/008:50:420
2559399672,43cyclictest0-21swapper/012:25:260
2559399671,52cyclictest17831-21if_wlan012:05:450
2559399671,46cyclictest16831-21ssh09:25:400
2559399671,44cyclictest0-21swapper/007:10:440
2559399671,41cyclictest5714-21unixbench_multi09:10:560
25594996616,29cyclictest7950irq/56-dwc_otg_11:40:361
2559499661,5cyclictest0-21swapper/110:45:351
2559499661,5cyclictest0-21swapper/110:45:351
2559399661,48cyclictest1461-21grep11:45:430
2559499653,7cyclictest8050irq/56-dwc_otg_09:10:401
25593996515,4cyclictest22035-21sh08:30:250
2559499645,5cyclictest7850irq/56-dwc_otg12:03:221
2559499642,5cyclictest7850irq/56-dwc_otg09:40:481
2559399641,31cyclictest8869-21ls11:15:260
2559599633,40cyclictest0-21swapper/210:25:562
2559499638,11cyclictest7850irq/56-dwc_otg09:58:581
2559499632,11cyclictest8050irq/56-dwc_otg_10:10:411
25594996314,12cyclictest7950irq/56-dwc_otg_10:30:361
25593996322,6cyclictest0-21swapper/007:30:360
2559399631,44cyclictest2433-21wget07:35:400
2559399631,43cyclictest13232-21aten_rbpower_vo11:20:380
2559399631,17cyclictest0-21swapper/009:50:550
2559399631,17cyclictest0-21swapper/009:50:550
25595996220,19cyclictest29570-21ssh11:00:422
2559499624,9cyclictest7850irq/56-dwc_otg12:35:461
2559399622,6cyclictest29195-21ssh10:20:370
2559399622,47cyclictest21209-21/usr/sbin/munin10:50:480
2559399622,42cyclictest0-21swapper/010:00:460
2559399621,40cyclictest750-21snmpd08:10:480
25595996110,25cyclictest25358-21ssh10:15:422
2559499615,9cyclictest8050irq/56-dwc_otg_10:05:521
2559399612,45cyclictest20854-21aten_rbpower_cu09:30:380
2559499605,12cyclictest0-21swapper/111:55:561
2559499603,4cyclictest7850irq/56-dwc_otg08:40:261
25594996019,38cyclictest22336-21wget08:30:391
2559499601,15cyclictest8050irq/56-dwc_otg_11:50:431
2559499601,10cyclictest8050irq/56-dwc_otg_10:55:471
25593996013,7cyclictest0-21swapper/009:20:380
25595995924,17cyclictest29-21rcuc/211:25:532
2559499597,4cyclictest0-21swapper/109:24:101
2559499593,6cyclictest8859-21ssh09:15:421
2559499593,6cyclictest6608-21ssh11:11:281
2559399594,40cyclictest30901-21cron07:25:250
2559399594,40cyclictest30901-21cron07:25:250
2559399592,29cyclictest5836-21ps10:30:500
2559399591,41cyclictest31335-21aten_rbpower_cu08:55:390
2559699583,34cyclictest0-21swapper/311:35:443
2559599583,34cyclictest0-21swapper/209:35:572
2559599583,11cyclictest26437-21ssh12:15:522
2559499582,10cyclictest26135-21fschecks_count08:40:431
25594995818,37cyclictest21101-21fschecks_count09:30:411
25594995813,5cyclictest1859-21sensors11:05:531
2559399585,11cyclictest0-21swapper/009:00:380
25593995836,14cyclictest0-21swapper/009:10:250
2559399582,41cyclictest0-21swapper/011:18:010
2559399582,41cyclictest0-21swapper/011:18:010
2559699573,39cyclictest0-21swapper/312:30:263
2559699573,39cyclictest0-21swapper/312:30:263
2559499571,16cyclictest0-21swapper/110:37:041
144662570,2sleep30-21swapper/310:01:343
2559499568,9cyclictest0-21swapper/109:50:191
2559499561,4cyclictest0-21swapper/110:52:421
25594995612,4cyclictest41762chrt11:49:121
2559399561,32cyclictest9420-21proc_pri09:15:500
2559699558,28cyclictest11-21rcu_preempt09:35:573
2559699555,9cyclictest7950irq/56-dwc_otg_11:25:553
25595995510,10cyclictest25799-21ssh11:35:442
2559499553,11cyclictest7850irq/56-dwc_otg08:25:371
2559499551,6cyclictest7850irq/56-dwc_otg11:16:431
2559499551,6cyclictest7850irq/56-dwc_otg11:16:431
2559399555,11cyclictest0-21swapper/010:55:370
2559399552,26cyclictest12119-21ntp_kernel_pll_08:00:470
25593995522,16cyclictest9512-21fschecks_count11:55:430
2559399551,30cyclictest13082-21/usr/sbin/munin10:40:510
25594995419,33cyclictest495-21latency_hist09:00:251
2559499541,6cyclictest8050irq/56-dwc_otg_08:45:541
2559499541,6cyclictest8050irq/56-dwc_otg_08:45:541
2559399548,6cyclictest0-21swapper/012:35:370
2559499536,4cyclictest8050irq/56-dwc_otg_08:00:421
2559499535,5cyclictest0-21swapper/110:24:041
2559499533,5cyclictest16651-21latency_hist10:05:251
2559499532,4cyclictest9925-21fschecks_time07:55:431
2559399538,20cyclictest6408-21unixbench_multi11:50:550
47202520,2sleep00-21swapper/011:10:250
2559599527,29cyclictest4924-21ssh10:30:322
2559499523,9cyclictest0-21swapper/109:05:391
2559499521,6cyclictest7850irq/56-dwc_otg07:55:261
25594995215,5cyclictest29569-21sh12:20:371
25594995213,5cyclictest4586-21taskset09:50:331
25594995213,5cyclictest4586-21taskset09:50:331
2559499521,11cyclictest7950irq/56-dwc_otg_08:19:121
2559399522,33cyclictest0-21swapper/007:50:480
2559399522,23cyclictest0-21swapper/011:34:130
25596995121,16cyclictest8050irq/56-dwc_otg_12:30:283
2559599513,13cyclictest21412-21apt-get12:10:382
2559499511,5cyclictest27413-21/usr/sbin/munin07:15:501
25594995111,37cyclictest0-21swapper/108:50:531
25594995111,37cyclictest0-21swapper/108:50:531
2559399519,22cyclictest0-21swapper/011:40:540
2559399512,33cyclictest0-21swapper/007:18:460
2559699509,9cyclictest8050irq/56-dwc_otg_12:10:383
2559499502,3cyclictest0-21swapper/108:10:541
2559499501,3cyclictest0-21swapper/111:32:581
2559399502,5cyclictest0-21swapper/008:31:300
2559399502,10cyclictest9946-21grep09:55:520
25595994922,23cyclictest0-21swapper/210:10:392
2559499493,3cyclictest8050irq/56-dwc_otg_08:06:181
2559499493,3cyclictest8050irq/56-dwc_otg_08:06:181
2559399491,34cyclictest0-21swapper/008:35:450
2559499481,9cyclictest7850irq/56-dwc_otg09:02:271
2559499481,13cyclictest0-21swapper/108:24:301
2559399485,22cyclictest0-21swapper/008:05:360
2559399485,22cyclictest0-21swapper/008:05:360
2559399482,25cyclictest0-21swapper/008:24:360
249962483,10sleep00-21swapper/007:06:210
2559599471,35cyclictest8050irq/56-dwc_otg_09:55:442
2559499473,3cyclictest7850irq/56-dwc_otg07:25:411
2559499471,4cyclictest7850irq/56-dwc_otg07:42:211
2559499471,3cyclictest0-21swapper/107:20:321
2559499471,3cyclictest0-21swapper/107:20:321
2559399472,31cyclictest0-21swapper/008:44:450
2559399471,14cyclictest0-21swapper/010:14:270
2559699463,16cyclictest0-21swapper/308:50:423
2559699463,16cyclictest0-21swapper/308:50:423
2559599466,28cyclictest9261-21ssh11:55:392
2559499467,25cyclictest0-21swapper/107:35:391
2559399462,7cyclictest27691-21/usr/sbin/munin08:45:480
2559399462,7cyclictest27691-21/usr/sbin/munin08:45:480
2559699457,25cyclictest0-21swapper/312:35:403
2559499454,4cyclictest0-21swapper/107:11:561
2559599448,6cyclictest31-21ksoftirqd/209:29:192
2559599448,22cyclictest0-21swapper/209:00:312
25595994414,6cyclictest1160150rm09:19:042
2559399441,35cyclictest0-21swapper/007:27:590
325262430,4sleep337-21rcuc/312:23:543
2559699435,12cyclictest8050irq/56-dwc_otg_10:30:533
2559399422,22cyclictest0-21swapper/007:56:320
2559699417,25cyclictest0-21swapper/307:20:303
2559699417,25cyclictest0-21swapper/307:20:303
2559699416,24cyclictest16776-21ssh09:25:393
2559599417,22cyclictest0-21swapper/207:45:372
2559499417,2cyclictest0-21swapper/107:45:371
243272418,22sleep324365-21/usr/sbin/munin07:05:483
283272400,1sleep20-21swapper/210:59:162
2559699407,24cyclictest0-21swapper/311:12:523
2559699406,27cyclictest948-21ssh11:05:403
2559699406,22cyclictest0-21swapper/307:45:373
2559699401,24cyclictest0-21swapper/311:55:363
2559599406,22cyclictest0-21swapper/211:20:372
2559599405,18cyclictest0-21swapper/208:35:382
25595994019,12cyclictest23717-21sh11:32:512
2559599401,6cyclictest0-21swapper/211:15:552
2559599401,6cyclictest0-21swapper/211:15:552
2559599398,7cyclictest31-21ksoftirqd/209:45:482
2559599397,23cyclictest0-21swapper/207:20:302
2559599397,23cyclictest0-21swapper/207:20:302
2559599397,18cyclictest0-21swapper/211:05:402
25595993918,11cyclictest0-21swapper/211:13:032
25595993912,5cyclictest29-21rcuc/212:24:362
25595993911,6cyclictest31-21ksoftirqd/210:10:252
25595993910,6cyclictest8050irq/56-dwc_otg_11:50:422
2559499393,20cyclictest0-21swapper/107:30:361
2559699388,19cyclictest8050irq/56-dwc_otg_11:50:393
2559699385,20cyclictest13115-21ssh11:20:373
2559699381,27cyclictest0-21swapper/312:02:333
2559699381,16cyclictest0-21swapper/309:44:253
2559599381,6cyclictest0-21swapper/210:23:332
2559599381,24cyclictest4900-21ls11:50:252
2559699377,4cyclictest0-21swapper/310:09:043
25596993714,12cyclictest1484-21fschecks_time11:45:433
2559699371,14cyclictest8050irq/56-dwc_otg_09:55:253
2559699371,14cyclictest8050irq/56-dwc_otg_09:55:253
2559699371,10cyclictest8050irq/56-dwc_otg_09:55:383
2559599378,10cyclictest0-21swapper/209:15:252
25595993715,5cyclictest31-21ksoftirqd/209:20:532
25595993712,7cyclictest9710-21/usr/sbin/munin10:35:482
2559599371,24cyclictest29-21rcuc/208:45:332
2559599371,24cyclictest29-21rcuc/208:45:332
2559699367,20cyclictest0-21swapper/310:10:393
2559699366,20cyclictest0-21swapper/309:15:383
2559699366,14cyclictest7950irq/56-dwc_otg_09:10:563
2559699365,11cyclictest0-21swapper/308:05:443
2559699365,11cyclictest0-21swapper/308:05:443
2559699362,20cyclictest0-21swapper/309:30:353
25596993621,8cyclictest381ktimersoftd/310:58:203
2559699361,28cyclictest0-21swapper/309:23:213
2559699361,22cyclictest0-21swapper/311:16:273
2559699361,22cyclictest0-21swapper/311:16:273
2559699361,17cyclictest37-21rcuc/312:05:563
25596993611,17cyclictest0-21swapper/309:48:573
2559599369,13cyclictest0-21swapper/210:51:172
2559599368,17cyclictest0-21swapper/209:50:392
2559599368,17cyclictest0-21swapper/209:50:392
2559599364,12cyclictest8050irq/56-dwc_otg_12:00:442
2559599362,27cyclictest0-21swapper/212:09:282
2559699356,17cyclictest0-21swapper/309:00:313
2559699352,14cyclictest0-21swapper/310:22:263
2559699351,27cyclictest0-21swapper/310:51:503
2559699351,24cyclictest0-21swapper/310:36:093
2559599357,15cyclictest8050irq/56-dwc_otg_08:55:532
25595993528,5cyclictest13955-21grep10:00:542
2559599352,16cyclictest7850irq/56-dwc_otg11:40:432
2559599341,9cyclictest7950irq/56-dwc_otg_09:40:402
25595993414,9cyclictest29-21rcuc/208:08:442
25595993414,9cyclictest29-21rcuc/208:08:442
25595993413,11cyclictest24000-21cut08:35:262
2559599341,23cyclictest28017-21missed_timers07:15:462
2559599341,22cyclictest31735-21missed_timers07:25:472
2559699337,13cyclictest8050irq/56-dwc_otg_11:45:253
2559699331,23cyclictest0-21swapper/310:41:313
2559699331,17cyclictest8050irq/56-dwc_otg_08:20:263
2559599337,10cyclictest31-21ksoftirqd/209:10:262
2559599332,17cyclictest13079-21ssh10:40:362
2559599331,15cyclictest8050irq/56-dwc_otg_08:20:372
2559699329,11cyclictest22371-21aten_rbpower_vo08:30:393
2559699326,17cyclictest0-21swapper/307:53:253
2559699321,8cyclictest0-21swapper/309:05:403
2559599327,15cyclictest29-21rcuc/208:26:592
2559599324,5cyclictest31-21ksoftirqd/208:40:382
2559599321,13cyclictest7950irq/56-dwc_otg_08:00:502
25595993210,4cyclictest0-21swapper/209:32:112
2559699317,14cyclictest2078-21latency_hist07:35:243
2559699313,18cyclictest39-21ksoftirqd/308:45:543
2559699313,18cyclictest39-21ksoftirqd/308:45:543
2559699312,21cyclictest0-21swapper/308:00:463
2559699311,21cyclictest0-21swapper/307:56:163
2559599316,11cyclictest8050irq/56-dwc_otg_07:55:432
25595993125,4cyclictest31-21ksoftirqd/207:10:382
2559599311,18cyclictest0-21swapper/207:34:242
2559699308,5cyclictest7950irq/56-dwc_otg_08:25:533
2559699307,14cyclictest0-21swapper/308:35:383
2559699304,22cyclictest31717-21ls07:25:463
2559699303,15cyclictest0-21swapper/307:10:373
2559699302,14cyclictest0-21swapper/308:23:123
2559699302,13cyclictest0-21swapper/307:37:553
2559699291,9cyclictest0-21swapper/308:58:223
2559699291,11cyclictest0-21swapper/307:17:153
2559599292,17cyclictest0-21swapper/208:15:482
2559699285,11cyclictest5272-21sed07:40:533
2559699281,10cyclictest0-21swapper/308:43:533
2559599281,4cyclictest7950irq/56-dwc_otg_07:35:552
249662285,14sleep10-21swapper/107:05:581
2559599272,6cyclictest0-21swapper/207:50:442
25595992721,4cyclictest29-21rcuc/208:10:442
2559699262,9cyclictest0-21swapper/308:10:283
25595992620,4cyclictest31-21ksoftirqd/208:50:412
25595992620,4cyclictest31-21ksoftirqd/208:50:412
2559599251,3cyclictest0-21swapper/207:40:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional