You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 20:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Aug 25, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
355521110,5sleep2301ktimersoftd/207:05:302
49159910859,40cyclictest0-21swapper/010:35:230
4915999960,8cyclictest7750irq/56-dwc_otg09:15:160
4923999839,25cyclictest0-21swapper/109:25:301
491599931,48cyclictest0-21swapper/009:50:230
111892920,2sleep00-21swapper/012:20:180
491599915,63cyclictest0-21swapper/012:30:250
491599915,63cyclictest0-21swapper/012:30:250
491599895,60cyclictest0-21swapper/009:25:300
491599817,34cyclictest0-21swapper/008:05:240
138042810,13sleep27950irq/56-dwc_otg_12:22:582
491599791,33cyclictest0-21swapper/009:55:220
491599777,9cyclictest0-21swapper/007:35:210
491599775,19cyclictest7950irq/56-dwc_otg_10:15:160
42002760,2sleep2492699cyclictest08:35:372
4923997545,24cyclictest19780-21apt-get09:15:161
4915997517,15cyclictest4317-21sh09:35:230
4923997328,25cyclictest16154-21basename07:40:301
65502720,2sleep20-21swapper/207:15:112
65502720,2sleep20-21swapper/207:15:112
4923997230,19cyclictest9296-21apt-get10:20:201
4923997230,19cyclictest9296-21apt-get10:20:201
491599725,16cyclictest9309-21sshd10:20:200
491599725,16cyclictest9309-21sshd10:20:200
108702720,2sleep30-21swapper/312:20:093
4923997134,12cyclictest25800-21apt-get10:40:241
491599716,8cyclictest0-21swapper/011:20:230
491599701,31cyclictest0-21swapper/009:45:220
221962700,2sleep10-21swapper/112:33:121
221962700,2sleep10-21swapper/112:33:121
4915996821,25cyclictest31175-21sh10:45:380
61892660,2sleep06191-21unixbench_singl08:40:380
4923996625,26cyclictest32229-21ssh11:26:111
4923996529,13cyclictest11756-21irqstats12:20:311
4923996429,23cyclictest14315-21grep07:35:271
4923996428,13cyclictest12253-21aten_rbpower_en07:30:241
491599648,6cyclictest0-21swapper/012:05:250
4915996423,21cyclictest0-21swapper/007:20:230
4923996330,17cyclictest2544-21ssh09:32:361
4923996328,22cyclictest31464-21ssh10:46:031
4915996310,6cyclictest101ktimersoftd/012:25:320
138652630,2sleep30-21swapper/309:45:383
4928996232,10cyclictest39-21ksoftirqd/309:15:143
492699629,37cyclictest0-21swapper/209:15:152
4923996227,24cyclictest9845-21ssh11:00:181
4923996227,13cyclictest14379-21taskset09:46:131
4923996225,22cyclictest4541-21ssh10:14:501
4923996223,13cyclictest22320-21/usr/sbin/munin09:55:401
491599627,15cyclictest0-21swapper/009:10:240
4923996129,11cyclictest26437-21ssh11:20:161
4923996128,19cyclictest26054-21ssh12:37:411
4923996123,21cyclictest28080-21switchtime12:00:371
4915996117,11cyclictest0-21swapper/009:40:240
491599611,40cyclictest23295-21fschecks_time08:00:260
491599611,40cyclictest23295-21fschecks_time08:00:260
4926996030,24cyclictest28745-21ssh09:25:282
4923996033,8cyclictest25146-21apt-get10:00:231
4923996027,17cyclictest25332-21sh11:57:381
4923996027,16cyclictest9865-21ssh09:40:411
4923996025,12cyclictest7950irq/56-dwc_otg_09:10:231
4923996022,11cyclictest7850irq/56-dwc_otg_07:25:381
4915996033,6cyclictest7850irq/56-dwc_otg_09:10:100
4915996021,20cyclictest22341-21/usr/sbin/munin11:15:400
205792600,1sleep20-21swapper/210:33:482
4923995935,16cyclictest11929-21sshd08:59:311
4923995932,17cyclictest9736-21ssh12:18:051
4923995931,13cyclictest8696-21aten_rbpower_vo07:20:251
4923995930,12cyclictest8929-21ssh11:37:501
4923995930,12cyclictest4363-21ssh11:32:011
4923995930,12cyclictest4363-21ssh11:32:011
4923995921,24cyclictest9093-21cat10:20:111
4923995921,19cyclictest0-21swapper/111:09:541
4923995921,18cyclictest5549-21sed08:40:291
492399591,9cyclictest2910-21ssh12:10:221
4923995918,11cyclictest0-21swapper/109:51:271
491599591,35cyclictest31517-21systemd-journal11:30:100
4915995912,26cyclictest15775-21timerwakeupswit11:45:370
492899587,36cyclictest8092-21kworker/u8:209:40:173
4923995828,11cyclictest6531-21latency_hist07:15:111
4923995828,11cyclictest6531-21latency_hist07:15:111
4923995827,18cyclictest30869-21sed08:20:311
4923995825,22cyclictest30749-21ssh10:05:571
4923995822,13cyclictest18769-21unixbench_singl07:45:401
4923995729,12cyclictest3690-21sh10:52:031
4923995729,12cyclictest3690-21sh10:52:031
4923995727,22cyclictest21799-21ssh11:14:571
4923995723,15cyclictest4371-21aten_rbpower_en09:35:241
4923995721,11cyclictest10838-21fschecks_count11:40:281
491599571,31cyclictest7227-21taskset08:45:290
4915995710,27cyclictest15196-21unixbench-2d11:05:380
492699568,30cyclictest15775-21ssh11:06:272
4923995627,15cyclictest8077-21chrt10:57:301
4923995626,12cyclictest22681-21fschecks_count11:15:281
4923995623,21cyclictest24791-21latency_hist08:05:121
4923995615,10cyclictest7850irq/56-dwc_otg_12:26:431
4915995625,6cyclictest32140-21aten_rbpower_po08:25:250
491599562,4cyclictest0-21swapper/011:50:240
4915995613,27cyclictest21066-21latency_hist07:55:110
491599561,30cyclictest6806-21ssh11:35:280
4923995527,17cyclictest19634-21ssh10:32:041
4923995527,16cyclictest26106-21chrt08:06:501
4923995524,20cyclictest16325-21ssh11:46:151
4923995521,15cyclictest25891-21ssh09:21:041
4923995520,16cyclictest3380-21/usr/sbin/munin08:35:251
4923995519,11cyclictest9550irq/80-mmc012:10:111
4915995535,3cyclictest7750irq/56-dwc_otg09:20:240
4915995533,7cyclictest26949-21latency_hist12:00:110
4915995527,7cyclictest8390-21unixbench_singl12:15:390
491599552,26cyclictest0-21swapper/008:15:100
4923995427,17cyclictest908-21taskset08:27:161
4923995423,21cyclictest15612-21diskmemload10:38:171
4923995421,12cyclictest22623-21ssh11:55:111
4923995419,23cyclictest16649-21ssh10:29:101
4915995431,7cyclictest25183-21/usr/sbin/munin10:00:250
4923995327,16cyclictest8146-21taskset07:17:501
4923995321,13cyclictest0-21swapper/108:04:091
4923995316,12cyclictest15838-21latency_hist09:10:111
491599532,29cyclictest14331-21ssh11:45:100
491599531,27cyclictest9941-21aten_rbpower_cu11:00:260
4915995232,9cyclictest7950irq/56-dwc_otg_10:15:110
491599522,29cyclictest29386-21ssh10:05:250
491599521,30cyclictest11289-21switchtime08:55:370
4923995117,20cyclictest7950irq/56-dwc_otg_09:04:551
491599519,20cyclictest12825-21/usr/sbin/munin07:30:310
4915995120,4cyclictest27082-21unixbench_singl10:40:400
491599511,47cyclictest2079-21tr08:30:350
4915995112,20cyclictest9450irq/48-DMA08:56:170
4928995010,23cyclictest7950irq/56-dwc_otg_09:25:303
4926995016,24cyclictest5619-21missed_timers08:40:312
4926995015,20cyclictest10884-21fschecks_time11:40:292
4923995021,12cyclictest0-21swapper/107:53:481
4923994936,10cyclictest22534-21sshd07:57:211
4923994933,13cyclictest26957-21grep08:10:281
4923994916,11cyclictest0-21swapper/108:30:111
491599492,25cyclictest0-21swapper/012:00:370
491599491,31cyclictest0-21swapper/011:30:290
491599491,31cyclictest0-21swapper/011:30:290
491599491,25cyclictest3407-21ssh12:10:290
4915994912,20cyclictest1180-21runrttasks12:37:520
492699489,13cyclictest16462-21ntpq07:40:342
492699487,14cyclictest7950irq/56-dwc_otg_11:00:262
4926994811,26cyclictest32294-21ssh10:08:482
4923994817,17cyclictest7950irq/56-dwc_otg_08:48:341
491599486,24cyclictest14566-21timerwakeupswit10:25:400
491599483,30cyclictest12272-21aten_rbpower_cu09:00:250
4915994824,12cyclictest0-21swapper/008:15:360
491599482,27cyclictest0-21swapper/007:15:330
4915994822,13cyclictest7950irq/56-dwc_otg_10:55:250
492899477,25cyclictest0-21swapper/308:25:243
492699476,21cyclictest7950irq/56-dwc_otg_09:35:322
4926994714,21cyclictest27856-21ntp_states12:00:342
4926994712,24cyclictest25751-21ssh10:40:182
4923994716,11cyclictest0-21swapper/108:51:091
4923994716,11cyclictest0-21swapper/108:51:091
4915994717,5cyclictest4786-21sh08:40:100
492699466,25cyclictest30645-21ssh11:25:242
4923994616,11cyclictest29721-21taskset08:17:001
491599462,35cyclictest17499-21latency_hist07:45:110
491599462,21cyclictest0-21swapper/008:00:120
491599461,17cyclictest0-21swapper/009:32:140
44792462,14sleep00-21swapper/007:09:050
492899456,23cyclictest0-21swapper/307:40:253
492699459,24cyclictest7463-21ssh10:17:242
492699457,25cyclictest0-21swapper/210:10:242
4915994530,4cyclictest21332-21sh10:35:100
491599451,41cyclictest0-21swapper/007:45:220
491599451,23cyclictest0-21swapper/010:54:480
491599451,23cyclictest0-21swapper/010:54:480
4915994511,18cyclictest10557-21cpuspeed07:25:250
4915994510,15cyclictest7850irq/56-dwc_otg_08:55:110
4915994510,15cyclictest7850irq/56-dwc_otg_08:55:110
492899436,22cyclictest0-21swapper/308:40:233
492699438,21cyclictest27688-21unixbench-2d11:20:402
492699438,12cyclictest31424-21ssh10:45:592
492699437,25cyclictest0-21swapper/208:05:242
492699437,23cyclictest0-21swapper/211:35:242
492699436,24cyclictest0-21swapper/210:55:242
492699436,13cyclictest12217-21basename07:30:232
492699432,26cyclictest7950irq/56-dwc_otg_10:51:492
492699432,26cyclictest7950irq/56-dwc_otg_10:51:492
491599433,20cyclictest0-21swapper/011:13:310
4915994325,4cyclictest6209-21sshd07:11:200
4915994325,4cyclictest6209-21sshd07:11:200
492699428,21cyclictest15941-21ssh09:10:212
492699427,24cyclictest0-21swapper/212:05:272
492699427,23cyclictest0-21swapper/210:25:252
492699427,13cyclictest23446-21ls12:35:112
492699427,13cyclictest23446-21ls12:35:112
4926994228,7cyclictest7950irq/56-dwc_otg_09:41:072
4926994223,11cyclictest7750irq/56-dwc_otg09:54:382
4926994210,13cyclictest21551-21sh11:14:312
145242420,2sleep30-21swapper/311:45:113
4928994118,6cyclictest22861-21if_wlan011:15:293
4928994117,4cyclictest22208-21latency10:35:303
492899411,4cyclictest7750irq/56-dwc_otg07:45:223
492699419,13cyclictest24867-21ssh11:17:492
492699417,24cyclictest7950irq/56-dwc_otg_12:15:232
492699417,16cyclictest7950irq/56-dwc_otg_08:17:032
492699417,13cyclictest14340-21cut07:35:282
492699411,25cyclictest0-21swapper/207:20:242
4926994112,14cyclictest10635-21/usr/sbin/munin10:20:382
4926994112,14cyclictest10635-21/usr/sbin/munin10:20:382
492899407,24cyclictest0-21swapper/307:35:243
4928994019,6cyclictest107750wget10:10:253
492899401,3cyclictest7950irq/56-dwc_otg_11:20:263
492699407,19cyclictest2411-21ssh09:32:222
4926994025,8cyclictest2692650ssh10:01:312
4926994021,13cyclictest7750irq/56-dwc_otg12:12:002
4926994014,7cyclictest24792-21cron10:00:102
4926994013,11cyclictest22379-21ssh11:54:482
4926994012,17cyclictest301ktimersoftd/209:21:322
4926994012,14cyclictest23521-21rm12:35:122
4928993922,5cyclictest25666-21sh10:00:313
492899391,4cyclictest0-21swapper/308:05:383
4928993911,16cyclictest0-21swapper/312:10:333
492699399,17cyclictest14432-21ssh09:46:182
492699396,20cyclictest18060-21ssh11:49:272
4926993916,11cyclictest11047-21/usr/sbin/munin07:25:322
4926993915,4cyclictest10235-21latency_hist08:55:112
4926993915,4cyclictest10235-21latency_hist08:55:112
492899381,25cyclictest8104-21latency_hist09:40:123
492699386,16cyclictest17040-21sshd12:26:202
492699383,13cyclictest3119-21cat08:35:112
492699381,25cyclictest0-21swapper/209:00:252
4926993812,12cyclictest18774-21uptime07:45:402
4928993722,8cyclictest7950irq/56-dwc_otg_11:15:113
492899371,8cyclictest31600-21fschecks_time12:05:313
4928993713,13cyclictest0-21swapper/309:50:303
492699379,4cyclictest13089-21kworker/u8:310:37:532
4926993720,10cyclictest7950irq/56-dwc_otg_11:56:042
4926993712,17cyclictest4189-21ssh11:31:422
4926993712,17cyclictest4189-21ssh11:31:422
492899367,26cyclictest0-21swapper/307:10:243
492899367,26cyclictest0-21swapper/307:10:243
492899363,18cyclictest12094-21latency_hist07:30:113
492899362,18cyclictest7950irq/56-dwc_otg_09:00:393
492899361,24cyclictest1197-21timerandwakeup09:30:393
4926993612,7cyclictest1157-21munin-run08:30:102
145512360,2sleep20-21swapper/209:05:322
492899357,24cyclictest0-21swapper/307:30:233
492899357,23cyclictest0-21swapper/311:30:233
492899357,23cyclictest0-21swapper/311:30:233
492899356,24cyclictest0-21swapper/310:05:253
492899353,17cyclictest7950irq/56-dwc_otg_10:50:273
492899353,17cyclictest7950irq/56-dwc_otg_10:50:273
492899351,4cyclictest0-21swapper/308:50:223
492899351,4cyclictest0-21swapper/308:50:223
492699351,24cyclictest8535-21ls07:20:112
492699351,24cyclictest0-21swapper/208:20:222
492899348,16cyclictest0-21swapper/311:47:133
492899347,20cyclictest0-21swapper/312:20:253
4928993422,4cyclictest22602-21latency_hist11:55:113
492899341,17cyclictest1469-21ls10:50:113
492699341,23cyclictest23178-21ls08:00:232
492699341,23cyclictest23178-21ls08:00:232
492699341,16cyclictest0-21swapper/208:11:012
492899339,12cyclictest7850irq/56-dwc_otg_10:30:243
492899338,5cyclictest0-21swapper/310:00:103
492899335,5cyclictest7092-21cut08:45:283
492899334,9cyclictest7950irq/56-dwc_otg_10:40:313
492899333,7cyclictest7950irq/56-dwc_otg_08:20:263
492899333,15cyclictest7950irq/56-dwc_otg_11:25:323
492899332,18cyclictest0-21swapper/311:00:173
4928993319,4cyclictest15929-21latency12:25:313
492899327,15cyclictest0-21swapper/310:15:173
492899322,21cyclictest0-21swapper/308:19:223
492699321,15cyclictest7780-21cat08:45:372
492899316,5cyclictest7750irq/56-dwc_otg11:35:363
492899316,12cyclictest7950irq/56-dwc_otg_09:20:253
492899315,5cyclictest19808-21uniq12:30:293
492899315,5cyclictest19808-21uniq12:30:293
492899313,5cyclictest7850irq/56-dwc_otg_12:00:253
492899311,24cyclictest20997-21ls07:55:113
4928993111,5cyclictest27229-21missed_timers08:10:313
492699316,10cyclictest20639-21seq07:51:112
492699315,4cyclictest7950irq/56-dwc_otg_08:56:562
42962313,20sleep10-21swapper/107:06:481
492899309,11cyclictest0-21swapper/311:55:203
492899306,6cyclictest15962-21ssh09:10:213
492899306,4cyclictest7750irq/56-dwc_otg10:55:373
492899301,20cyclictest14608-21missed_timers09:05:333
492899301,14cyclictest7850irq/56-dwc_otg_10:20:263
492899301,14cyclictest7850irq/56-dwc_otg_10:20:263
4928993011,4cyclictest0-21swapper/312:36:543
4926993018,3cyclictest31-21ksoftirqd/207:56:052
492899295,5cyclictest7850irq/56-dwc_otg_08:30:383
492899295,5cyclictest0-21swapper/307:20:303
492899291,3cyclictest7850irq/56-dwc_otg_11:05:393
492899291,14cyclictest7750irq/56-dwc_otg10:25:373
492899284,7cyclictest3740-21cat08:35:303
492899281,8cyclictest23930-21cat08:00:353
492899281,8cyclictest23930-21cat08:00:353
492899273,3cyclictest7950irq/56-dwc_otg_08:55:373
492899272,7cyclictest0-21swapper/307:55:313
32772276,9sleep30-21swapper/307:05:283
492899262,13cyclictest7850irq/56-dwc_otg_07:19:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional