You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Sep 23, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24532950,2sleep30-21swapper/300:14:463
836999874,67cyclictest0-21swapper/019:10:280
8369998044,15cyclictest0-21swapper/023:15:530
8369998027,12cyclictest7950irq/56-dwc_otg_22:40:370
8369998027,12cyclictest7950irq/56-dwc_otg_22:40:370
239732800,2sleep30-21swapper/322:37:503
8369997751,11cyclictest7950irq/56-dwc_otg_20:25:260
836999771,23cyclictest0-21swapper/022:50:320
8369997647,16cyclictest0-21swapper/022:00:250
8369997639,16cyclictest18634-21unixbench_multi19:35:570
836999753,58cyclictest0-21swapper/000:15:400
8369997414,52cyclictest9550irq/80-mmc000:05:540
8369997352,9cyclictest101ktimersoftd/023:30:350
836999722,60cyclictest0-21swapper/023:00:400
266122720,1sleep00-21swapper/023:22:120
219952720,2sleep30-21swapper/319:45:513
8369997033,21cyclictest0-21swapper/023:50:260
836999701,59cyclictest0-21swapper/022:00:520
836999692,44cyclictest0-21swapper/023:25:410
836999691,58cyclictest30511-21/usr/sbin/munin22:05:490
277952690,2sleep00-21swapper/021:20:410
277952690,2sleep00-21swapper/021:20:410
836999682,51cyclictest4460-21unixbench_multi20:25:550
836999682,45cyclictest16542-21ssh23:10:260
836999681,49cyclictest1180-21runrttasks21:50:420
836999671,57cyclictest0-21swapper/021:41:590
8369996646,11cyclictest7850irq/56-dwc_otg_21:05:250
8369996642,11cyclictest5336-21sed23:35:560
8369996638,15cyclictest10337-21latency_hist20:45:260
836999662,45cyclictest0-21swapper/021:10:520
8369996613,38cyclictest0-21swapper/000:26:570
8369996448,5cyclictest101ktimersoftd/020:45:550
8369996445,5cyclictest7032-21aten_rbpower_po20:35:390
8369996442,11cyclictest7850irq/56-dwc_otg_19:40:430
8369996436,18cyclictest30817-21missed_timers20:10:470
8369996433,16cyclictest0-21swapper/022:35:540
8369996431,16cyclictest0-21swapper/021:25:410
836999642,51cyclictest23978-21wget00:00:420
836999642,49cyclictest5958-21grep22:55:540
8369996419,31cyclictest7186-21wget00:20:400
836999641,51cyclictest13-21rcuc/021:15:480
836999641,47cyclictest0-21swapper/019:30:310
212042640,2sleep10-21swapper/100:37:531
836999638,40cyclictest8271-21sed21:35:560
8369996345,5cyclictest7850irq/56-dwc_otg_22:25:250
8369996344,5cyclictest7750irq/56-dwc_otg23:45:250
8369996344,4cyclictest7750irq/56-dwc_otg22:10:480
836999632,51cyclictest0-21swapper/019:45:420
836999631,50cyclictest19538-21ssh23:55:260
836999631,44cyclictest0-21swapper/019:50:490
47132630,4sleep14684-21ssh22:55:251
8369996243,10cyclictest101ktimersoftd/000:40:260
836999622,43cyclictest0-21swapper/020:50:470
836999622,41cyclictest0-21swapper/023:15:270
8369996214,4cyclictest7950irq/56-dwc_otg_22:45:400
8369996147,6cyclictest7750irq/56-dwc_otg22:15:440
8369996147,6cyclictest7750irq/56-dwc_otg22:15:440
836999612,54cyclictest0-21swapper/022:25:530
836999612,54cyclictest0-21swapper/022:25:530
836999611,33cyclictest6891-21ls20:35:270
151622610,6sleep2837199cyclictest23:49:512
8369996041,6cyclictest23404-21apt-get00:00:260
836999602,46cyclictest16695-21ssh00:31:410
8369996017,35cyclictest0-21swapper/021:32:080
836999601,47cyclictest32134-21wget20:15:400
836999601,47cyclictest32134-21wget20:15:400
8369996010,36cyclictest0-21swapper/020:05:490
8369995945,4cyclictest7750irq/56-dwc_otg22:35:250
836999591,41cyclictest0-21swapper/000:15:250
836999582,43cyclictest0-21swapper/021:46:330
836999582,31cyclictest0-21swapper/021:05:540
836999582,31cyclictest0-21swapper/021:05:540
836999571,46cyclictest0-21swapper/019:57:070
8369995538,7cyclictest101ktimersoftd/020:55:470
837299538,29cyclictest0-21swapper/320:25:313
8369995330,9cyclictest150192sleep019:26:020
8369995324,19cyclictest0-21swapper/020:00:510
837299522,9cyclictest7950irq/56-dwc_otg_20:00:363
837299518,28cyclictest0-21swapper/319:30:393
836999511,41cyclictest0-21swapper/019:20:390
85232500,2sleep3381ktimersoftd/322:18:273
85232500,2sleep3381ktimersoftd/322:18:273
8370995014,7cyclictest23-21ksoftirqd/123:35:441
78572502,12sleep00-21swapper/019:07:240
837099495,8cyclictest21-21rcuc/122:36:581
837099492,30cyclictest0-21swapper/119:30:391
8370994918,5cyclictest5309-21/usr/sbin/munin22:55:451
8372994816,21cyclictest0-21swapper/323:50:443
837099487,4cyclictest7950irq/56-dwc_otg_00:25:471
837099483,21cyclictest7850irq/56-dwc_otg_20:55:321
8370994831,6cyclictest27338-21ssh00:05:191
8370994822,23cyclictest884650wget20:40:401
837099481,30cyclictest0-21swapper/120:35:381
836999483,35cyclictest13-21rcuc/019:17:140
324302480,2sleep10-21swapper/122:07:051
121272480,2sleep30-21swapper/300:25:543
8370994716,6cyclictest16203-21systemd-journal21:30:251
8370994712,21cyclictest13703-1kworker/u9:223:55:391
235952470,2sleep20-21swapper/223:19:002
837199467,11cyclictest7950irq/56-dwc_otg_21:00:422
8371994618,9cyclictest7750irq/56-dwc_otg00:26:262
837099462,16cyclictest0-21swapper/121:35:511
8370994616,9cyclictest0-21swapper/122:45:571
8370994614,15cyclictest23-21ksoftirqd/121:35:251
837299452,24cyclictest0-21swapper/319:35:413
837199456,16cyclictest19846-21ssh22:32:232
837199452,6cyclictest7950irq/56-dwc_otg_19:10:362
8370994522,6cyclictest23-21ksoftirqd/119:35:431
837099451,14cyclictest0-21swapper/122:35:261
280932450,2sleep30-21swapper/322:43:143
280932450,2sleep30-21swapper/322:43:143
837299444,21cyclictest7950irq/56-dwc_otg_19:19:213
8372994414,17cyclictest39-21ksoftirqd/321:00:503
8370994424,7cyclictest17133-21fschecks_count23:10:491
837099441,20cyclictest0-21swapper/120:26:541
8370994411,6cyclictest23-21ksoftirqd/100:16:551
837099441,11cyclictest29553-21unixbench_multi20:05:561
8370994410,6cyclictest23-21ksoftirqd/100:21:471
75632440,2sleep30-21swapper/322:58:213
30962440,6sleep2837199cyclictest21:30:352
214912441,3sleep20-21swapper/221:53:412
195642440,2sleep20-21swapper/223:13:522
837299439,16cyclictest2826-21ssh21:30:253
837299436,26cyclictest39-21ksoftirqd/321:20:183
8372994318,6cyclictest20168-21fschecks_time23:55:443
8372994318,5cyclictest9532-21sh00:23:133
8371994315,4cyclictest301ktimersoftd/223:37:572
837099436,24cyclictest14974-21/usr/sbin/munin00:30:471
8370994315,6cyclictest21-21rcuc/122:02:401
216532431,3sleep30-21swapper/321:11:513
837299429,6cyclictest39-21ksoftirqd/323:35:253
837299426,24cyclictest18815-21ssh21:50:393
8372994211,18cyclictest0-21swapper/323:05:593
8371994214,8cyclictest7850irq/56-dwc_otg_23:45:122
837099427,5cyclictest0-21swapper/121:15:251
8370994217,7cyclictest7927-21sh23:40:331
8370994213,5cyclictest3784-21sed23:35:261
188822421,34sleep10-21swapper/121:05:561
188822421,34sleep10-21swapper/121:05:561
93002410,2sleep20-21swapper/220:40:462
837299417,7cyclictest39-21ksoftirqd/321:35:253
8372994120,11cyclictest2461-21ssh22:10:503
8372994115,13cyclictest0-21swapper/322:00:403
837199419,12cyclictest0-21swapper/223:32:412
837199419,12cyclictest0-21swapper/222:00:292
837199416,13cyclictest7950irq/56-dwc_otg_21:50:112
8371994115,11cyclictest301ktimersoftd/221:10:422
8371994114,7cyclictest301ktimersoftd/200:16:462
8371994110,5cyclictest7850irq/56-dwc_otg_22:45:462
837099414,7cyclictest21-21rcuc/123:20:511
837099413,8cyclictest14123-21diskstats22:25:431
837099413,8cyclictest14123-21diskstats22:25:431
8370994120,11cyclictest30799-21ssh00:09:241
8370994114,11cyclictest29009-21unixbench_singl21:20:581
8370994114,11cyclictest29009-21unixbench_singl21:20:581
8370994113,5cyclictest23-21ksoftirqd/122:42:041
8370994113,5cyclictest23-21ksoftirqd/122:42:041
837099411,17cyclictest12816-21awk19:20:521
8370994111,7cyclictest0-21swapper/123:10:091
8370994111,5cyclictest21-21rcuc/100:14:291
837099411,15cyclictest0-21swapper/120:15:131
837099411,12cyclictest0-21swapper/122:14:241
837299408,14cyclictest39-21ksoftirqd/321:20:283
837299408,14cyclictest39-21ksoftirqd/321:20:283
837299407,29cyclictest0-21swapper/322:50:323
837299407,19cyclictest10950-21apt-get21:40:353
837299405,4cyclictest7950irq/56-dwc_otg_00:15:513
8372994023,9cyclictest381ktimersoftd/320:50:533
8372994019,11cyclictest18178-21df21:05:443
8372994019,11cyclictest18178-21df21:05:443
8372994016,16cyclictest381ktimersoftd/323:25:503
8372994014,5cyclictest37-21rcuc/320:55:403
837199408,13cyclictest29-21rcuc/220:55:442
837199403,18cyclictest31628-21ssh21:25:362
8371994020,9cyclictest7850irq/56-dwc_otg_22:23:102
8371994016,5cyclictest11046-21aten_rbpower_po21:40:382
8371994012,9cyclictest301ktimersoftd/221:25:142
8371994012,9cyclictest301ktimersoftd/221:25:142
837199401,24cyclictest17647-21wget19:35:422
837099403,17cyclictest0-21swapper/120:20:371
837099401,6cyclictest9064-21sh22:19:301
837099401,6cyclictest9064-21sh22:19:301
8370994014,7cyclictest23-21ksoftirqd/120:50:471
8370994013,8cyclictest0-21swapper/121:59:531
8370994013,6cyclictest21-21rcuc/123:28:251
8370994011,4cyclictest0-21swapper/123:00:481
837299399,13cyclictest15542-21apt-config23:50:263
837299397,21cyclictest0-21swapper/322:35:033
837299393,6cyclictest7039-21apt-get21:35:373
837299393,25cyclictest23887-21timerandwakeup19:50:553
837299392,27cyclictest7950irq/56-dwc_otg_19:40:483
837299392,25cyclictest0-21swapper/300:09:093
8372993916,13cyclictest10748-21aten_rbpower_vo20:45:433
8372993912,17cyclictest0-21swapper/322:05:443
837199396,20cyclictest7750irq/56-dwc_otg22:40:522
837199396,20cyclictest7750irq/56-dwc_otg22:40:522
837199394,15cyclictest31-21ksoftirqd/221:15:362
8371993919,5cyclictest7950irq/56-dwc_otg_00:33:452
8371993915,8cyclictest7950irq/56-dwc_otg_20:50:552
837199391,26cyclictest21062-21ssh00:37:362
8371993911,12cyclictest0-21swapper/221:55:272
837099392,6cyclictest24007-21cpuspeed21:15:421
8370993919,10cyclictest20328-21ssh21:51:211
8370993916,6cyclictest10695-21aten_rbpower_en20:45:421
837099391,12cyclictest12235-21ssh22:23:031
310032390,2sleep20-21swapper/223:27:582
837299389,14cyclictest13907-21wget19:25:403
837299389,12cyclictest37-21rcuc/323:45:253
837299387,6cyclictest39-21ksoftirqd/323:15:553
837299385,10cyclictest3263-21apt-config20:25:263
837299384,21cyclictest39-21ksoftirqd/322:27:083
837299384,21cyclictest39-21ksoftirqd/322:27:083
8372993822,9cyclictest381ktimersoftd/321:48:003
837299382,20cyclictest26360-21ssh22:00:263
8372993819,6cyclictest1180-21runrttasks00:00:383
8372993813,11cyclictest37-21rcuc/323:23:243
837299381,22cyclictest5231-21grep20:30:413
837199389,10cyclictest7850irq/56-dwc_otg_22:37:192
8371993823,4cyclictest7750irq/56-dwc_otg23:55:092
8371993812,5cyclictest20144-21fschecks_time23:55:442
837099386,5cyclictest22927-21/usr/sbin/munin19:50:521
837099386,13cyclictest23-21ksoftirqd/121:40:321
837099382,6cyclictest15222-21fschecks_time21:45:451
837099381,7cyclictest24930-21aten_rbpower_po19:55:411
8370993812,5cyclictest0-21swapper/123:45:471
837299379,5cyclictest37-21rcuc/322:49:243
837299378,5cyclictest39-21ksoftirqd/323:35:443
837299377,16cyclictest0-21swapper/320:35:383
837299376,19cyclictest39-21ksoftirqd/300:30:423
837299375,22cyclictest0-21swapper/322:21:303
837299372,9cyclictest0-21swapper/300:37:223
837199379,13cyclictest11753-21ssh23:03:532
837199378,5cyclictest7750irq/56-dwc_otg23:05:412
8371993723,4cyclictest7950irq/56-dwc_otg_23:23:382
837199372,15cyclictest757450/usr/sbin/munin21:35:462
837099377,5cyclictest23-21ksoftirqd/123:15:561
837099377,3cyclictest0-21swapper/123:54:091
837099372,10cyclictest0-21swapper/119:25:411
8370993712,8cyclictest7750irq/56-dwc_otg21:00:551
837299368,12cyclictest0-21swapper/323:10:513
837199368,11cyclictest7950irq/56-dwc_otg_00:00:412
837199367,7cyclictest7850irq/56-dwc_otg_19:55:442
837199361,24cyclictest0-21swapper/222:08:402
837199361,23cyclictest0-21swapper/219:25:552
837299354,19cyclictest0-21swapper/323:03:493
837199352,13cyclictest29-21rcuc/200:21:512
8371993518,4cyclictest7950irq/56-dwc_otg_00:07:022
837199351,5cyclictest4872-21ssh22:13:592
837199351,4cyclictest7750irq/56-dwc_otg19:30:482
8371993513,7cyclictest7850irq/56-dwc_otg_21:05:512
8371993513,7cyclictest7850irq/56-dwc_otg_21:05:512
837199351,12cyclictest4503-21ssh22:55:032
8371993510,6cyclictest5789-21netstat20:30:492
8371993510,12cyclictest0-21swapper/222:58:472
837099353,20cyclictest0-21swapper/119:10:281
837099352,5cyclictest7950irq/56-dwc_otg_19:45:251
837099351,12cyclictest5142-21awk20:30:371
837299349,7cyclictest26407-21idleruntime-cro20:00:253
837299346,18cyclictest0-21swapper/320:42:003
837299342,22cyclictest0-21swapper/320:16:433
837299342,22cyclictest0-21swapper/320:16:433
837299341,21cyclictest0-21swapper/320:10:403
837199349,4cyclictest0-21swapper/200:13:202
8371993414,4cyclictest7750irq/56-dwc_otg20:45:492
8371993413,7cyclictest7850irq/56-dwc_otg_19:40:382
837099344,5cyclictest7950irq/56-dwc_otg_20:15:431
837099344,5cyclictest7950irq/56-dwc_otg_20:15:431
837099341,12cyclictest27997-21chrt20:02:581
8370993410,6cyclictest10287-21grep19:15:401
25802340,4sleep22561-21unixbench_singl20:20:562
837299333,13cyclictest7950irq/56-dwc_otg_19:10:383
837299332,15cyclictest0-21swapper/319:25:263
837199334,6cyclictest13949-21aten_rbpower_en22:25:402
837199334,6cyclictest13949-21aten_rbpower_en22:25:402
8370993310,12cyclictest0-21swapper/119:49:541
837199326,6cyclictest31-21ksoftirqd/220:16:472
837199326,6cyclictest31-21ksoftirqd/220:16:472
837199322,10cyclictest10313-21wget19:15:402
8371993219,8cyclictest0-21swapper/219:20:492
837199321,6cyclictest23135-21sort19:50:422
78882322,23sleep10-21swapper/119:07:501
837299311,5cyclictest0-21swapper/320:05:413
837199316,4cyclictest7950irq/56-dwc_otg_20:10:512
837199315,12cyclictest7750irq/56-dwc_otg19:45:352
837199314,6cyclictest7750irq/56-dwc_otg22:15:382
837199314,6cyclictest7750irq/56-dwc_otg22:15:382
8371993121,6cyclictest7030-21aten_rbpower_po20:35:392
837199312,14cyclictest29937-21chrt20:09:442
837199312,12cyclictest7750irq/56-dwc_otg20:25:492
837199311,6cyclictest28417-21latency_hist20:05:272
80962318,13sleep28138-21ls19:10:262
82022286,13sleep38323-21sed19:10:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional