You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 06:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Tue Aug 04, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2428422550,4sleep0101ktimersoftd/019:05:500
1394121170,2sleep10-21swapper/120:05:561
1837621130,5sleep12549999cyclictest00:09:451
740421120,6sleep02549899cyclictest22:35:470
25498991059,80cyclictest3678-21kworker/0:021:10:440
25498999712,54cyclictest0-21swapper/023:10:430
25498999712,54cyclictest0-21swapper/023:10:430
2549999964,11cyclictest4329-21sh21:10:451
25498999662,13cyclictest0-21swapper/000:25:540
2549999952,14cyclictest8050irq/56-dwc_otg_00:25:521
2549999894,9cyclictest0-21swapper/122:10:421
2549899899,45cyclictest0-21swapper/023:40:570
2549899897,68cyclictest0-21swapper/019:20:430
25500998634,31cyclictest8050irq/56-dwc_otg_21:10:462
25499998619,13cyclictest8050irq/56-dwc_otg_23:25:291
2549899863,55cyclictest14252-21cat23:25:290
276012850,5sleep02549899cyclictest19:15:470
2549999834,22cyclictest28077-21grep23:40:571
25498998250,10cyclictest0-21swapper/000:16:000
25498998054,18cyclictest101ktimersoftd/021:45:460
2549899803,62cyclictest0-21swapper/019:10:310
2550099798,37cyclictest0-21swapper/221:45:362
25500997729,24cyclictest27999-21taskset23:40:562
310342760,2sleep10-21swapper/123:05:461
25500997647,24cyclictest14244-21date23:25:292
25500997346,22cyclictest2288-21apt-get23:10:432
25500997346,22cyclictest2288-21apt-get23:10:432
2549999731,28cyclictest21202-21timerwakeupswit20:25:561
25500997228,28cyclictest24423-21sed00:15:592
2549999725,42cyclictest23-21ksoftirqd/100:16:001
25498997260,4cyclictest9-21ksoftirqd/022:47:230
149672720,2sleep00-21swapper/020:10:420
2550299712,37cyclictest24359-21ssh00:15:593
25500997025,26cyclictest14380-21ssh22:45:112
130002690,5sleep32550299cyclictest22:42:313
2550299686,13cyclictest8050irq/56-dwc_otg_00:25:543
25500996840,22cyclictest19570-21ssh22:10:432
25498996832,19cyclictest0-21swapper/022:20:050
25498996832,19cyclictest0-21swapper/022:20:050
2549899682,46cyclictest13-21rcuc/022:10:400
2549899682,28cyclictest11320-21latency_hist22:00:290
175972680,2sleep10-21swapper/123:29:141
25502996715,36cyclictest7950irq/56-dwc_otg_23:10:413
25502996715,36cyclictest7950irq/56-dwc_otg_23:10:413
25500996725,23cyclictest30740-21ssh21:43:342
25498996738,6cyclictest8050irq/56-dwc_otg_23:30:410
25498996641,5cyclictest0-21swapper/022:40:410
25498996623,25cyclictest3292-21sh22:30:470
2550099656,20cyclictest8050irq/56-dwc_otg_23:35:432
25500996537,22cyclictest32345-21/usr/sbin/munin00:25:522
25500996525,19cyclictest7950irq/56-dwc_otg_22:29:282
25500996523,26cyclictest26445-21rm22:19:052
25500996523,26cyclictest26445-21rm22:19:052
2549899653,37cyclictest31480-21ssh22:25:420
2549899651,49cyclictest20138-21/usr/sbin/munin21:30:500
2549899651,48cyclictest31965-21ssh23:45:580
2549899651,29cyclictest0-21swapper/020:30:400
25500996423,25cyclictest20654-21cat21:30:482
25500996421,26cyclictest27236-21if_wlan023:00:502
2549899641,46cyclictest31759-21ls21:45:290
158212640,4sleep0101ktimersoftd/023:25:590
25500996319,26cyclictest21172-21ssh22:53:002
25500996224,15cyclictest7889-21ssh23:55:582
25500996221,22cyclictest13166-21df20:05:432
185502620,2sleep30-21swapper/323:30:333
25500996133,22cyclictest25501-21runrttasks19:10:302
25498996132,4cyclictest27526-21cut00:20:440
2550099603,3cyclictest0-21swapper/220:25:402
25500996026,14cyclictest25024-21ssh22:57:482
25500996020,27cyclictest17291-21ssh22:06:412
25500996020,21cyclictest17137-21sh22:47:522
2549999607,33cyclictest0-21swapper/119:55:421
25498996043,4cyclictest7850irq/56-dwc_otg22:55:400
25498996024,20cyclictest30777-21cut23:05:420
74542590,2sleep10-21swapper/122:35:481
25500995923,13cyclictest16159-21ssh21:25:372
25500995923,13cyclictest16159-21ssh21:25:372
25500995920,19cyclictest161372chrt20:13:102
25499995926,3cyclictest0-21swapper/100:00:401
25498995920,23cyclictest16180-21grep00:05:580
25500995825,19cyclictest24079-21/usr/sbin/munin21:35:562
25500995824,20cyclictest19440-21ls20:20:582
25500995823,20cyclictest16572-21apt20:15:402
25500995821,23cyclictest5023-21taskset22:32:222
25500995818,19cyclictest8050irq/56-dwc_otg_00:00:552
25500995818,12cyclictest2709-21basename23:50:422
25500995818,12cyclictest2709-21basename23:50:422
25498995832,5cyclictest9-21ksoftirqd/000:35:560
2549899582,36cyclictest27734-21latency_hist21:40:290
25500995723,14cyclictest4512-21/usr/sbin/munin21:50:552
25500995721,26cyclictest13223-21sh22:01:282
2549999574,16cyclictest2288-21apt-get23:10:401
2549999574,16cyclictest2288-21apt-get23:10:401
25498995731,7cyclictest7950irq/56-dwc_otg_21:15:590
25498995728,6cyclictest0-21swapper/021:25:430
25498995728,6cyclictest0-21swapper/021:25:430
2549899571,25cyclictest0-21swapper/020:25:410
25500995629,22cyclictest14435-21sh23:25:322
25500995619,20cyclictest20445-21ssh23:31:542
25500995618,14cyclictest9003-21/usr/sbin/munin21:56:002
25500995615,21cyclictest8050irq/56-dwc_otg_21:20:352
2549899562,33cyclictest0-21swapper/019:40:420
2550299555,20cyclictest39-21ksoftirqd/323:25:323
25502995515,20cyclictest7850irq/56-dwc_otg23:40:573
25502995510,18cyclictest0-21swapper/321:10:453
25500995524,14cyclictest10160-21ssh22:39:422
25500995524,12cyclictest9322-21ssh23:19:022
25500995523,9cyclictest799-21ssh23:08:142
25500995519,20cyclictest32708-21rm23:46:512
25499995525,5cyclictest27758-21latency_hist21:40:291
25499995518,25cyclictest0-21swapper/119:50:431
25498995533,14cyclictest101ktimersoftd/020:20:560
25498995530,11cyclictest0-21swapper/020:10:290
25498995526,7cyclictest8064-21grep23:55:590
25498995522,5cyclictest101ktimersoftd/023:55:290
25498995522,5cyclictest101ktimersoftd/023:55:290
2549899551,51cyclictest25490-21awk20:40:290
2549899551,29cyclictest11731-21aten_rbpower_en22:00:430
2550299543,11cyclictest19549-21apt-get22:10:423
25502995411,22cyclictest7850irq/56-dwc_otg23:25:303
25500995425,20cyclictest29318-21ssh22:21:582
2549999548,30cyclictest0-21swapper/122:30:431
2549899544,12cyclictest3190-21grep19:35:560
25498995428,11cyclictest4536-21wc21:50:550
2549899542,31cyclictest7593-21unixbench_singl23:15:590
25498995418,19cyclictest26772-21aten_rbpower_cu23:00:440
2550099538,8cyclictest8050irq/56-dwc_otg_00:05:412
25500995321,13cyclictest10301-21ssh21:17:362
25500995315,15cyclictest7950irq/56-dwc_otg_19:28:532
25500995310,13cyclictest0-21swapper/220:40:412
25500995310,13cyclictest0-21swapper/220:40:412
2549899533,38cyclictest13-21rcuc/021:07:360
25498995325,12cyclictest16614-21aten_rbpower_en20:15:430
2549899531,30cyclictest20065-21switchtime00:10:540
25498995312,26cyclictest0-21swapper/022:05:440
2550299522,29cyclictest7950irq/56-dwc_otg_20:10:403
2550099528,30cyclictest0-21swapper/219:50:412
25500995221,21cyclictest28747-21taskset19:18:322
25500995221,11cyclictest29490-21ssh00:22:262
25500995219,19cyclictest3225-21ssh00:30:352
25498995224,15cyclictest8050irq/56-dwc_otg_22:51:010
2549899521,35cyclictest9-21ksoftirqd/000:00:410
25500995130,17cyclictest5749-21/usr/sbin/munin19:46:012
25500995119,11cyclictest2110-21cat21:05:292
25500995117,20cyclictest22299-21ssh00:14:222
2549999518,31cyclictest0-21swapper/122:25:441
25498995129,5cyclictest7950irq/56-dwc_otg_20:50:470
2549899512,33cyclictest0-21swapper/021:20:340
25498995118,19cyclictest23988-21unixbench_singl23:35:590
2550099508,26cyclictest0-21swapper/220:00:432
25500995034,13cyclictest3486-21/usr/sbin/munin21:05:592
25500995020,10cyclictest25169-21taskset20:36:522
2549999508,30cyclictest0-21swapper/121:20:351
2549999504,13cyclictest516-21in:imuxsock20:35:391
25498995022,18cyclictest0-21swapper/019:50:390
2550299495,5cyclictest0-21swapper/319:20:423
2550299493,6cyclictest7950irq/56-dwc_otg_20:05:413
25502994920,5cyclictest2757-21latency_hist22:30:293
25500994914,11cyclictest8050irq/56-dwc_otg_00:40:292
2549999497,13cyclictest0-21swapper/119:10:301
2549999492,8cyclictest7850irq/56-dwc_otg20:00:421
2549999492,7cyclictest11661-21ssh22:00:411
2549899491,7cyclictest591-21tune2fs19:30:450
25498994815,18cyclictest727-21runrttasks22:20:320
25498994813,18cyclictest13-21rcuc/000:30:480
158372480,5sleep12549999cyclictest22:46:001
25500994716,12cyclictest727-21runrttasks19:23:362
25500994712,21cyclictest5436-21runrttasks19:44:502
25500994710,31cyclictest0-21swapper/219:55:422
2549899472,33cyclictest0-21swapper/020:55:540
2549899472,25cyclictest0-21swapper/019:55:500
2549899472,24cyclictest1345-21grep21:00:530
2550299462,9cyclictest8050irq/56-dwc_otg_22:15:403
2550299462,9cyclictest8050irq/56-dwc_otg_22:15:403
25498994616,16cyclictest0-21swapper/019:46:090
2549899461,29cyclictest727-21runrttasks20:05:130
2549899461,10cyclictest0-21swapper/019:25:400
2550299458,26cyclictest0-21swapper/321:30:423
25500994513,12cyclictest0-21swapper/220:59:572
2549999457,24cyclictest0-21swapper/119:30:411
2549999456,26cyclictest0-21swapper/120:50:431
2549999452,8cyclictest0-21swapper/122:15:421
2549999452,8cyclictest0-21swapper/122:15:421
2549999452,29cyclictest0-21swapper/123:15:431
2549999452,25cyclictest18985-21missed_timers20:20:521
2549899442,23cyclictest0-21swapper/020:47:090
25502994312,18cyclictest15734-21wget22:05:453
25500994316,10cyclictest8050irq/56-dwc_otg_20:54:332
25500994312,12cyclictest8050irq/56-dwc_otg_19:34:072
2549999438,25cyclictest0-21swapper/121:30:421
2549999437,24cyclictest0-21swapper/119:25:411
2549999428,22cyclictest18841-21ssh22:50:441
25499994222,5cyclictest22739-21/usr/sbin/munin22:55:521
2549999421,31cyclictest6113-21idleruntime19:46:021
25498994216,12cyclictest0-21swapper/020:43:340
25498994216,12cyclictest0-21swapper/020:43:340
2550299419,6cyclictest11322-21cron22:00:283
2550299413,7cyclictest7950irq/56-dwc_otg_22:50:433
25500994123,14cyclictest28672-21unixbench_singl20:45:582
25500994112,21cyclictest3385-21chrt19:36:422
2549999414,7cyclictest8454-21cut21:15:471
25499994121,5cyclictest2817950wget21:40:441
86612400,1sleep30-21swapper/300:35:563
2550299408,4cyclictest0-21swapper/322:38:233
2550299407,7cyclictest12279-21grep20:00:593
2550299405,4cyclictest8050irq/56-dwc_otg_23:45:433
2550299403,4cyclictest8050irq/56-dwc_otg_21:25:283
2550299403,23cyclictest0-21swapper/319:10:303
2550299402,7cyclictest0-21swapper/322:45:453
25502994014,4cyclictest334-21latency_hist21:00:293
2550299401,26cyclictest0-21swapper/321:50:453
2549999408,5cyclictest22910-21modprobe00:15:281
2549999406,6cyclictest7850irq/56-dwc_otg22:00:281
2549999402,7cyclictest2652250rm23:40:291
25499994013,12cyclictest0-21swapper/122:40:541
25499994010,18cyclictest0-21swapper/121:25:431
25499994010,18cyclictest0-21swapper/121:25:431
43202391,3sleep10-21swapper/121:50:521
2550299399,16cyclictest0-21swapper/321:50:163
2550299394,18cyclictest8050irq/56-dwc_otg_20:00:303
2550299393,21cyclictest24167-21aten_rbpower_en21:35:433
2550299391,7cyclictest28779-21users22:20:573
2550299391,5cyclictest0-21swapper/321:15:433
2549999398,19cyclictest0-21swapper/123:00:411
2549999391,21cyclictest10738-21latency_hist00:00:291
2550299387,20cyclictest27290-21latency_hist20:45:293
2550299387,20cyclictest27290-21latency_hist20:45:293
2550299385,15cyclictest7950irq/56-dwc_otg_20:20:443
2550299384,18cyclictest6706-21timerandwakeup19:46:103
2550299382,19cyclictest0-21swapper/323:09:393
2549999389,14cyclictest8050irq/56-dwc_otg_22:20:511
25499993818,5cyclictest2208650wget20:30:421
25499993816,10cyclictest7850irq/56-dwc_otg23:50:371
25499993816,10cyclictest7850irq/56-dwc_otg23:50:371
2550299379,14cyclictest0-21swapper/323:00:243
2550299377,5cyclictest8050irq/56-dwc_otg_00:30:583
2550299372,22cyclictest21173-21grep20:25:553
25502993715,5cyclictest300-21/usr/sbin/munin19:30:383
2550299371,15cyclictest750-21snmpd19:50:303
2549999376,6cyclictest7850irq/56-dwc_otg21:50:291
25499993716,6cyclictest8050irq/56-dwc_otg_20:55:461
2549999371,11cyclictest7082-21latency_hist00:35:291
249952376,22sleep324996-21sshd19:07:283
2550299368,25cyclictest0-21swapper/320:15:433
2550299364,6cyclictest7850irq/56-dwc_otg21:05:283
2550299362,24cyclictest0-21swapper/321:43:373
2550299361,6cyclictest1279350ssh22:00:583
25500993623,10cyclictest22343-21fw_packets20:30:462
2549999361,8cyclictest18149-21sh20:20:281
2550299358,11cyclictest7850irq/56-dwc_otg23:50:453
2550299358,11cyclictest7850irq/56-dwc_otg23:50:453
2550299356,12cyclictest7950irq/56-dwc_otg_22:30:473
2550299353,4cyclictest7950irq/56-dwc_otg_21:25:583
2550299353,4cyclictest7950irq/56-dwc_otg_21:25:583
2550299353,15cyclictest8050irq/56-dwc_otg_19:35:563
2550299352,7cyclictest26990-21ssh23:00:463
2550299351,3cyclictest0-21swapper/319:40:433
2550299351,14cyclictest7850irq/56-dwc_otg20:50:433
2549999358,6cyclictest7850irq/56-dwc_otg19:40:291
2549999358,5cyclictest7850irq/56-dwc_otg00:35:411
2549999357,12cyclictest15778-21timerwakeupswit20:10:561
25499993516,5cyclictest3863-21diskmemload22:09:181
2549999351,27cyclictest0-21swapper/123:49:151
25499993510,6cyclictest7850irq/56-dwc_otg23:30:591
254582351,27sleep30-21swapper/320:40:293
2550299347,8cyclictest8050irq/56-dwc_otg_23:16:263
2550299346,14cyclictest8050irq/56-dwc_otg_23:35:553
2550299341,20cyclictest19815-21ssh00:10:503
2549999349,5cyclictest7850irq/56-dwc_otg19:41:001
2549999343,20cyclictest0-21swapper/120:48:501
2549999341,22cyclictest27974-21sh00:20:501
2549999334,6cyclictest0-21swapper/121:05:561
25499993313,6cyclictest1106-21sed21:00:481
2549999331,21cyclictest26077-21ls20:40:461
2549999331,21cyclictest26077-21ls20:40:461
2550299329,6cyclictest8050irq/56-dwc_otg_23:55:443
2550299326,6cyclictest29825-21ssh00:23:033
2550299325,14cyclictest8050irq/56-dwc_otg_19:19:103
2550299324,14cyclictest8050irq/56-dwc_otg_00:05:433
2550299323,6cyclictest2411-21aten_rbpower_vo21:05:433
252802324,16sleep125396-21latency_hist19:10:291
25502993116,4cyclictest10980-21aten_rbpower_po00:00:433
2549999312,8cyclictest0-21swapper/119:24:431
2550299306,5cyclictest8050irq/56-dwc_otg_20:30:563
2550299301,25cyclictest0-21swapper/320:45:473
2549999292,20cyclictest0-21swapper/119:17:301
2550299281,14cyclictest0-21swapper/319:26:323
249672289,12sleep20-21swapper/219:07:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional