You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 22:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 02, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2311921070,6sleep1221ktimersoftd/109:54:331
1161621030,9sleep3381ktimersoftd/310:18:503
1335699955,65cyclictest0-21swapper/011:45:260
1336199883,15cyclictest7750irq/56-dwc_otg10:50:241
187292850,1sleep10-21swapper/111:05:181
187292850,1sleep10-21swapper/111:05:181
269162830,2sleep20-21swapper/211:15:142
13356997835,10cyclictest0-21swapper/012:35:240
1335699769,46cyclictest0-21swapper/010:55:270
1335699762,40cyclictest0-21swapper/010:35:250
1335699762,40cyclictest0-21swapper/010:35:250
126662760,2sleep20-21swapper/207:05:372
227452750,5sleep11336199cyclictest12:27:351
13356997515,33cyclictest25172-21unixbench_singl07:55:360
54872730,1sleep10-21swapper/112:06:301
197502720,2sleep20-21swapper/209:11:582
13356997134,10cyclictest3727-21grep09:30:410
1335699707,55cyclictest0-21swapper/012:00:260
206242660,2sleep20-21swapper/209:50:392
1336399661,51cyclictest7750irq/56-dwc_otg10:35:392
1336399661,51cyclictest7750irq/56-dwc_otg10:35:392
13356996633,7cyclictest2414-21kworker/u8:109:40:240
1335699656,53cyclictest101ktimersoftd/011:35:140
13356996510,52cyclictest101ktimersoftd/009:25:140
13356996510,52cyclictest101ktimersoftd/009:25:140
1336399649,40cyclictest7950irq/56-dwc_otg_10:15:292
13361996412,16cyclictest15483-21ssh11:40:271
1335699636,22cyclictest0-21swapper/010:40:260
13363996220,11cyclictest0-21swapper/210:10:252
1335699626,44cyclictest16561-21fschecks_count12:20:300
1335699624,54cyclictest101ktimersoftd/008:10:130
13356996235,3cyclictest7750irq/56-dwc_otg11:55:270
13356996217,4cyclictest13-21rcuc/009:50:140
13356996217,4cyclictest13-21rcuc/009:50:140
1335699621,32cyclictest15844-21wget09:05:270
1336399619,37cyclictest0-21swapper/212:20:312
1336399613,49cyclictest7950irq/56-dwc_otg_11:55:362
13356996134,10cyclictest24522-21wc09:55:330
1335699613,27cyclictest0-21swapper/007:15:220
1335699612,35cyclictest149-21systemd-journal10:30:130
13356996123,21cyclictest16433-21idleruntime-cro07:25:130
13356996030,2cyclictest0-21swapper/012:15:230
1335699601,42cyclictest0-21swapper/008:15:250
1335699601,42cyclictest0-21swapper/008:15:250
1335699601,39cyclictest0-21swapper/009:35:280
1335699601,32cyclictest13309-21unixbench_singl08:55:400
1336399597,19cyclictest0-21swapper/211:35:272
1336399596,7cyclictest7950irq/56-dwc_otg_12:31:412
1336399595,8cyclictest7750irq/56-dwc_otg11:33:552
1336399594,6cyclictest7950irq/56-dwc_otg_09:25:422
1336399593,9cyclictest7950irq/56-dwc_otg_10:55:372
13356995944,4cyclictest15045-21unixbench-2d09:00:410
13356995939,4cyclictest17358-21diskmemload10:10:350
1335699592,40cyclictest11237-21ssh11:35:190
13356995915,27cyclictest9450irq/48-DMA09:12:250
1335699591,34cyclictest13417-21systemd07:10:280
1335699591,34cyclictest13417-21systemd07:10:280
1336399584,5cyclictest7950irq/56-dwc_otg_10:10:142
1336399582,6cyclictest0-21swapper/211:15:312
1336399581,5cyclictest13319-21sed10:20:322
1336399581,4cyclictest17358-21diskmemload09:15:352
1336199587,32cyclictest0-21swapper/109:45:261
1336199587,32cyclictest0-21swapper/109:45:261
1335699589,29cyclictest0-21swapper/008:10:410
1335699584,51cyclictest101ktimersoftd/008:55:130
13356995819,18cyclictest21961-21sh09:15:160
1336399578,39cyclictest7950irq/56-dwc_otg_11:41:052
1336399571,5cyclictest29443-21grep08:10:412
1336399571,10cyclictest0-21swapper/212:00:542
13361995714,13cyclictest21635-21rm11:47:001
1335699576,47cyclictest101ktimersoftd/008:20:410
13356995740,5cyclictest19538-21taskset09:11:360
13356995723,8cyclictest0-21swapper/012:30:240
1335699572,31cyclictest3855-21sshd10:10:140
1336399565,8cyclictest7950irq/56-dwc_otg_09:33:302
1336399562,10cyclictest7850irq/56-dwc_otg_09:47:402
1336399562,10cyclictest7850irq/56-dwc_otg_09:47:402
13356995638,9cyclictest31359-21ntpq09:25:360
13356995624,29cyclictest101ktimersoftd/008:25:250
1335699562,33cyclictest0-21swapper/010:15:250
1335699562,30cyclictest0-21swapper/008:30:410
1335699561,27cyclictest0-21swapper/009:50:310
315352550,2sleep30-21swapper/312:38:323
1336399559,43cyclictest4047-21ssh11:25:412
1336399552,6cyclictest7950irq/56-dwc_otg_12:06:332
1336399551,4cyclictest23783-21ssh10:32:442
13363995513,33cyclictest908-21runrttasks12:26:032
1336399551,13cyclictest0-21swapper/207:34:072
1336399551,10cyclictest21147-21latency_hist10:30:142
1336199556,14cyclictest7850irq/56-dwc_otg_08:50:241
13356995538,5cyclictest16931-21ssh11:40:580
13356995536,9cyclictest30008-21ssh10:01:420
13356995536,8cyclictest0-21swapper/011:55:080
13356995536,11cyclictest11917-21grep12:15:140
13356995532,13cyclictest917-21snmpd11:03:490
1335699551,4cyclictest31242-21ssh11:20:250
1336399548,5cyclictest24959-21unixbench_multi11:50:402
1336399544,7cyclictest7950irq/56-dwc_otg_09:41:402
1336399544,39cyclictest7950irq/56-dwc_otg_09:55:322
1336399543,4cyclictest7850irq/56-dwc_otg_11:50:152
1336399541,3cyclictest0-21swapper/210:50:362
1336399541,12cyclictest5634-21latency_hist10:50:142
13361995422,19cyclictest908-21runrttasks12:18:041
13356995437,8cyclictest0-21swapper/011:12:330
1335699542,40cyclictest8126-21kworker/0:012:10:140
13356995418,21cyclictest30972-21latency_hist11:20:130
1335699541,42cyclictest0-21swapper/007:35:290
13356995414,12cyclictest13348-21cyclictest12:25:240
212312530,2sleep30-21swapper/311:46:153
1336399534,4cyclictest0-21swapper/211:01:112
1336399533,8cyclictest7850irq/56-dwc_otg_11:10:082
1336399533,8cyclictest7850irq/56-dwc_otg_11:10:082
1336399533,5cyclictest29812-21rm10:40:232
1336399532,9cyclictest7850irq/56-dwc_otg_09:39:132
1336399531,4cyclictest0-21swapper/210:01:272
13356995337,5cyclictest20750-21sh11:06:510
13356995337,5cyclictest20750-21sh11:06:510
1336399523,44cyclictest24041-21sshd07:55:202
1336399521,5cyclictest26937-21/usr/sbin/munin08:05:282
13363995215,27cyclictest301ktimersoftd/207:14:022
13363995215,27cyclictest301ktimersoftd/207:14:022
1336199529,22cyclictest28981-21ntpq08:10:371
13361995214,18cyclictest10202-21ssh12:12:211
13356995217,13cyclictest0-21swapper/010:53:000
1335699521,27cyclictest0-21swapper/010:20:260
1335699521,25cyclictest7055-21grep08:40:300
1336399518,4cyclictest5839-21switchtime08:35:382
1336399515,4cyclictest7950irq/56-dwc_otg_08:47:132
1336399512,10cyclictest0-21swapper/208:55:212
1336399511,4cyclictest30780-21awk08:15:352
1336399511,4cyclictest30780-21awk08:15:352
1336399511,3cyclictest7750irq/56-dwc_otg12:15:222
1336399511,3cyclictest0-21swapper/209:21:262
1336399511,3cyclictest0-21swapper/209:21:262
1336199513,7cyclictest7750irq/56-dwc_otg09:15:261
13361995115,18cyclictest10871-21apt-get11:35:201
13361995114,20cyclictest28416-21ssh11:15:431
1335699515,35cyclictest9-21ksoftirqd/011:29:320
1336399501,41cyclictest0-21swapper/211:21:122
1336399501,3cyclictest0-21swapper/212:35:372
13363995013,34cyclictest3760-21cat08:30:342
1336199508,26cyclictest0-21swapper/110:25:261
13361995012,24cyclictest28631-21sh10:38:451
13361995012,24cyclictest28631-21sh10:38:451
13361995012,24cyclictest0-21swapper/107:25:231
13356995030,10cyclictest101ktimersoftd/008:05:000
13356995024,15cyclictest5015-21aten_rbpower_en08:35:270
1335699501,5cyclictest25227-21munin-run10:35:140
1335699501,29cyclictest8651-21wget08:45:270
1336499497,26cyclictest7974-21ssh12:10:253
1336399494,6cyclictest0-21swapper/207:45:242
1336399492,7cyclictest15028-21ls09:00:402
1336399491,8cyclictest20681-21latency_hist07:40:142
1336399491,8cyclictest15337-21sed07:15:372
1336399491,4cyclictest0-21swapper/207:55:042
1336399491,3cyclictest9550irq/80-mmc012:10:292
1336199497,26cyclictest0-21swapper/109:30:251
13361994914,19cyclictest16347-21ssh11:01:321
13361994913,12cyclictest31404-21ssh09:25:371
1336399482,7cyclictest7073-21fschecks_time08:40:302
1336399482,7cyclictest12001-21grep08:55:142
1336399481,9cyclictest16714-21grep07:25:232
1336199488,22cyclictest26788-21/usr/sbin/munin08:05:261
1336199487,25cyclictest0-21swapper/107:15:201
1336199487,24cyclictest7850irq/56-dwc_otg_11:29:411
1336199487,16cyclictest8498-21ssh09:36:321
13361994822,15cyclictest7750irq/56-dwc_otg11:24:171
1335699481,44cyclictest101ktimersoftd/007:31:230
1335699481,25cyclictest908-21runrttasks07:48:530
127772483,11sleep00-21swapper/007:06:170
1336399471,7cyclictest16643-21ls09:05:392
1336199478,24cyclictest0-21swapper/110:10:271
1336199476,24cyclictest5537-21ssh10:50:121
1336199474,13cyclictest5553-21kworker/u8:212:00:491
13361994722,7cyclictest7950irq/56-dwc_otg_10:15:431
13361994711,12cyclictest9559-21ssh11:32:501
13361994710,13cyclictest28311-21ls11:55:321
1336499468,23cyclictest0-21swapper/307:15:193
1336399465,4cyclictest7750irq/56-dwc_otg08:30:132
1336399461,7cyclictest0-21swapper/208:20:512
1336199469,11cyclictest27371-21ssh11:54:551
1336199468,23cyclictest0-21swapper/109:05:271
13361994612,18cyclictest32275-21ssh12:39:541
13361994611,12cyclictest25255-21ssh11:12:311
13356994629,8cyclictest0-21swapper/007:52:050
13356994628,3cyclictest0-21swapper/007:26:220
143692450,19sleep37950irq/56-dwc_otg_10:21:123
1336399457,25cyclictest0-21swapper/207:40:192
1336399452,8cyclictest16556-21latency_hist07:25:142
1336199459,20cyclictest31463-21ssh10:41:151
13361994514,16cyclictest12805-21ls08:55:341
13361994513,13cyclictest30414-21cut09:25:141
13361994513,13cyclictest30414-21cut09:25:141
13361994513,12cyclictest31627-21rm10:04:311
13361994510,20cyclictest12977-21ssh10:58:181
1336499447,23cyclictest0-21swapper/311:35:193
1336399442,8cyclictest0-21swapper/208:03:022
13356994414,21cyclictest0-21swapper/007:44:030
1336499436,29cyclictest10975-21ssh09:40:263
13364994323,18cyclictest7950irq/56-dwc_otg_11:20:263
1336199439,11cyclictest26522-21ssh12:31:561
1336199437,24cyclictest0-21swapper/107:55:221
1336199432,10cyclictest7750irq/56-dwc_otg10:30:231
20442420,2sleep30-21swapper/310:07:033
13361994215,4cyclictest7850irq/56-dwc_otg_09:41:351
13361994213,16cyclictest24710-21/usr/sbin/munin09:55:371
13361994210,22cyclictest0-21swapper/107:40:191
1336499417,21cyclictest7950irq/56-dwc_otg_09:50:203
1336499416,25cyclictest0-21swapper/309:25:273
1336199413,23cyclictest18732-21seq07:34:031
13361994114,17cyclictest18970-21rm12:23:151
13361994112,14cyclictest23123-21chrt07:49:461
13361994110,15cyclictest19867-21ssh09:12:101
123992410,1sleep30-21swapper/308:55:293
26782400,1sleep10-21swapper/110:08:101
1336499407,22cyclictest23871-21sh09:55:253
1336499401,8cyclictest0-21swapper/308:05:203
1336199409,12cyclictest13970-21/usr/sbin/munin09:00:431
1336199403,14cyclictest30680-21sed08:15:331
1336199403,14cyclictest30680-21sed08:15:331
1336499391,15cyclictest28120-21ssh10:00:253
1336199396,21cyclictest237482sleep107:54:451
13361993917,12cyclictest2781-21seq08:28:371
1336199391,4cyclictest7850irq/56-dwc_otg_07:40:131
13361993914,15cyclictest908-21runrttasks08:39:321
1336499381,6cyclictest18804-21ssh11:05:253
1336499381,6cyclictest18804-21ssh11:05:253
1336499381,21cyclictest20820-21ls07:40:143
1336199388,12cyclictest14537-21ssh10:21:291
1336499372,17cyclictest0-21swapper/310:35:253
1336499372,17cyclictest0-21swapper/310:35:253
13364993711,13cyclictest3381-21/usr/sbin/munin08:30:303
1336199378,15cyclictest908-21runrttasks08:41:501
1336199376,18cyclictest0-21swapper/108:02:481
1336499367,14cyclictest0-21swapper/311:00:203
1336499361,27cyclictest10536-21cut10:55:313
1336499361,23cyclictest23577-21missed_timers11:10:343
1336199367,17cyclictest0-21swapper/108:20:201
1336199363,15cyclictest0-21swapper/108:31:381
1336499352,10cyclictest4205-21ssh10:10:243
13364993516,4cyclictest15442-21/usr/sbin/munin11:40:253
1336499351,22cyclictest0-21swapper/312:19:333
1336199353,18cyclictest9901-21taskset08:47:281
1336499349,13cyclictest11088-21apt-config11:35:153
1336499348,15cyclictest0-21swapper/310:50:273
1336499347,23cyclictest0-21swapper/309:35:263
1336499346,6cyclictest7850irq/56-dwc_otg_09:00:273
1336499346,5cyclictest7750irq/56-dwc_otg12:25:133
1336499345,20cyclictest21980-21ssh09:15:193
1336499342,17cyclictest0-21swapper/309:45:353
1336499342,17cyclictest0-21swapper/309:45:353
1336499341,9cyclictest0-21swapper/311:16:493
1336499341,6cyclictest7750irq/56-dwc_otg10:40:233
13364993414,5cyclictest1883-21sh10:45:283
1336499338,4cyclictest0-21swapper/309:10:273
1336499337,12cyclictest7850irq/56-dwc_otg_10:30:253
1336499336,12cyclictest7950irq/56-dwc_otg_07:45:263
1336499333,20cyclictest0-21swapper/311:27:413
1336499332,6cyclictest23791-21kworker/u8:209:10:143
1336499331,25cyclictest26736-21ls09:20:303
1336499331,25cyclictest26736-21ls09:20:303
1336499331,22cyclictest21050-21missed_timers12:25:333
1336499331,20cyclictest24804-21uniq12:30:303
13364993311,10cyclictest0-21swapper/309:30:413
306352320,2sleep30-21swapper/308:15:323
306352320,2sleep30-21swapper/308:15:323
1336499327,15cyclictest0-21swapper/312:05:293
13364993224,5cyclictest381ktimersoftd/310:25:253
1336499319,5cyclictest28461-21cat11:55:343
1336499314,6cyclictest7750irq/56-dwc_otg07:25:363
1336499311,22cyclictest0-21swapper/311:50:183
13364993111,4cyclictest0-21swapper/308:10:303
1336199319,6cyclictest0-21swapper/107:13:371
1336199319,6cyclictest0-21swapper/107:13:371
125482318,11sleep10-21swapper/107:05:371
1336499306,14cyclictest0-21swapper/308:20:193
1336499306,12cyclictest7950irq/56-dwc_otg_08:25:353
1336499303,13cyclictest7850irq/56-dwc_otg_08:40:323
1336499301,4cyclictest32310-21fschecks_count12:00:303
1336499301,20cyclictest0-21swapper/308:02:313
13361993019,3cyclictest7950irq/56-dwc_otg_07:23:051
1336499292,13cyclictest0-21swapper/307:56:183
13364992914,5cyclictest908-21runrttasks07:11:173
13364992914,5cyclictest908-21runrttasks07:11:173
1336499291,13cyclictest7950irq/56-dwc_otg_08:50:293
1336499288,11cyclictest7950irq/56-dwc_otg_07:31:443
1336499282,7cyclictest9257-21ntpq08:45:353
1336499281,20cyclictest23803-21latency_hist07:55:133
1336499281,18cyclictest0-21swapper/307:43:523
1336499262,17cyclictest0-21swapper/308:39:483
128432254,14sleep30-21swapper/307:07:093
1336499242,14cyclictest0-21swapper/307:23:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional