You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 14:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Mar 15, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
905521080,2sleep20-21swapper/222:20:042
3099991065,57cyclictest0-21swapper/019:50:160
3099991065,57cyclictest0-21swapper/019:50:160
251332970,2sleep10-21swapper/100:35:261
309999957,81cyclictest0-21swapper/019:45:170
309999878,45cyclictest0-21swapper/022:30:160
3099998626,48cyclictest5017-21aten_rbpower_cu20:45:160
272422860,2sleep30-21swapper/322:02:583
3099998325,5cyclictest19291-21ssh21:15:270
3099998017,31cyclictest0-21swapper/022:35:180
3099997924,35cyclictest0-21swapper/022:50:310
309999791,3cyclictest7950irq/56-dwc_otg_22:45:230
3099997824,5cyclictest917-21snmpd23:40:540
3099997720,4cyclictest26832-21tune2fs20:15:210
24452760,2sleep20-21swapper/223:29:252
309999759,3cyclictest0-21swapper/021:10:060
309999751,19cyclictest0-21swapper/023:52:060
309999742,54cyclictest0-21swapper/019:35:270
309999741,24cyclictest14266-21/usr/sbin/munin22:25:270
309999739,14cyclictest0-21swapper/022:13:030
309999732,55cyclictest0-21swapper/000:00:290
309999732,55cyclictest0-21swapper/000:00:290
309999731,23cyclictest0-21swapper/000:31:080
3099997311,3cyclictest0-21swapper/023:46:230
309999728,13cyclictest0-21swapper/023:55:520
3099997227,30cyclictest7850irq/56-dwc_otg_21:40:300
309999722,23cyclictest29301-21ssh22:05:220
3099997214,10cyclictest7398-21aten_rbpower_vo23:35:180
309999712,5cyclictest0-21swapper/020:40:040
309999712,5cyclictest0-21swapper/020:40:040
309999712,55cyclictest0-21swapper/021:48:560
309999711,29cyclictest0-21swapper/019:10:150
309999711,25cyclictest0-21swapper/021:05:230
309999711,17cyclictest26280-21ssh00:36:380
263322710,2sleep00-21swapper/021:24:260
309999701,60cyclictest0-21swapper/022:41:010
309999701,58cyclictest0-21swapper/020:05:020
309999701,55cyclictest0-21swapper/023:25:240
309999701,24cyclictest0-21swapper/020:53:460
309999701,22cyclictest0-21swapper/020:59:110
309999691,24cyclictest0-21swapper/022:00:140
309999691,15cyclictest27279-21/usr/sbin/munin23:20:280
309999681,55cyclictest0-21swapper/021:31:080
309999681,55cyclictest0-21swapper/021:31:080
309999672,56cyclictest0-21swapper/022:56:060
309999672,53cyclictest0-21swapper/019:21:210
309999671,25cyclictest0-21swapper/022:19:150
309999661,26cyclictest0-21swapper/020:34:380
3099996520,5cyclictest7850irq/56-dwc_otg_20:10:030
3099996517,5cyclictest7850irq/56-dwc_otg_21:54:370
3099996517,5cyclictest7850irq/56-dwc_otg_21:54:370
3099996515,10cyclictest0-21swapper/023:15:440
3099996512,45cyclictest0-21swapper/000:28:480
3099996512,45cyclictest0-21swapper/000:28:480
309999651,16cyclictest0-21swapper/000:10:040
309999642,53cyclictest0-21swapper/020:28:490
3099996422,3cyclictest7850irq/56-dwc_otg_00:20:280
309999641,4cyclictest11826-21latency_hist00:20:040
309999641,22cyclictest0-21swapper/022:20:080
309999641,17cyclictest0-21swapper/021:29:070
309999631,23cyclictest0-21swapper/021:36:110
309999631,15cyclictest6419-21ssh23:34:070
309999631,15cyclictest6419-21ssh23:34:070
3099996311,14cyclictest0-21swapper/021:57:480
186492630,4sleep2301ktimersoftd/222:30:272
3099996242,5cyclictest11614-21grep21:00:320
3099996213,4cyclictest11409-21grep19:30:310
309999621,23cyclictest0-21swapper/000:11:430
3099996210,15cyclictest0-21swapper/023:04:310
3099996210,15cyclictest0-21swapper/023:04:310
3099996122,4cyclictest0-21swapper/019:17:030
309999611,51cyclictest0-21swapper/023:10:060
310299609,4cyclictest30702-21cut22:45:173
309999603,15cyclictest0-21swapper/023:07:120
309999592,24cyclictest0-21swapper/019:40:450
3099995911,28cyclictest149-21systemd-journal20:25:040
3099995910,10cyclictest0-21swapper/020:10:220
309999589,10cyclictest0-21swapper/019:57:050
309999584,4cyclictest7950irq/56-dwc_otg_19:25:190
309999581,34cyclictest3267-21wget20:40:180
310199572,10cyclictest7850irq/56-dwc_otg_22:45:142
3100995710,9cyclictest7950irq/56-dwc_otg_23:05:161
310299567,17cyclictest0-21swapper/322:50:183
143222540,2sleep30-21swapper/323:04:383
143222540,2sleep30-21swapper/323:04:383
310199526,5cyclictest7950irq/56-dwc_otg_19:20:162
310099521,6cyclictest7950irq/56-dwc_otg_21:15:171
310299513,4cyclictest7950irq/56-dwc_otg_21:52:083
310299513,4cyclictest7950irq/56-dwc_otg_21:52:083
3100995111,26cyclictest0-21swapper/120:05:161
22922510,2sleep10-21swapper/100:08:051
310099497,27cyclictest0-21swapper/120:10:161
310299477,26cyclictest0-21swapper/321:00:173
310199477,24cyclictest0-21swapper/222:55:152
310299466,24cyclictest19680-21ssh23:50:173
310099462,25cyclictest0-21swapper/100:10:161
310199457,25cyclictest0-21swapper/219:25:182
310099456,18cyclictest7950irq/56-dwc_otg_19:30:181
310299437,23cyclictest0-21swapper/300:15:183
310299437,22cyclictest0-21swapper/320:10:163
310199437,23cyclictest0-21swapper/220:55:092
310199436,5cyclictest0-21swapper/200:05:162
310099436,25cyclictest17771-21ssh22:30:091
310099431,16cyclictest7750irq/56-dwc_otg00:20:041
25562433,12sleep00-21swapper/019:06:390
25562433,12sleep00-21swapper/019:06:390
310299427,21cyclictest0-21swapper/320:05:163
310299423,5cyclictest7950irq/56-dwc_otg_22:10:143
310199427,25cyclictest0-21swapper/223:55:172
310199427,23cyclictest0-21swapper/220:50:172
310199426,17cyclictest7850irq/56-dwc_otg_23:40:032
310199423,3cyclictest7850irq/56-dwc_otg_19:55:142
310299417,24cyclictest0-21swapper/322:20:093
3102994115,5cyclictest3600-21if_eth021:35:223
310199412,13cyclictest0-21swapper/219:30:152
310099413,14cyclictest7750irq/56-dwc_otg20:20:131
141752410,2sleep10-21swapper/100:22:061
310299407,22cyclictest0-21swapper/320:30:173
3102994018,5cyclictest13884-21diskmemload21:10:253
310199406,24cyclictest0-21swapper/222:20:162
310199405,26cyclictest7850irq/56-dwc_otg_22:10:232
310199403,9cyclictest0-21swapper/223:50:162
310099407,18cyclictest0-21swapper/123:20:141
310099406,24cyclictest0-21swapper/120:35:171
310099406,24cyclictest0-21swapper/120:35:171
310099402,7cyclictest0-21swapper/120:25:161
310099401,26cyclictest13884-21diskmemload23:15:211
142892400,3sleep214287-21ssh22:25:282
3102993914,4cyclictest908-21runrttasks22:05:223
310299391,29cyclictest16100-21ls23:45:243
3101993912,5cyclictest0-21swapper/221:50:012
310099397,12cyclictest15907-21ssh21:50:141
310099397,12cyclictest15907-21ssh21:50:141
3100993917,6cyclictest2265-21apt-key22:50:051
3100993913,19cyclictest24516-21/usr/sbin/munin22:00:261
310299381,3cyclictest7850irq/56-dwc_otg_23:40:243
310199381,22cyclictest22919-21ls20:05:042
310099388,19cyclictest0-21swapper/123:50:491
310099382,4cyclictest20212-21apt-get21:55:101
3100993819,5cyclictest5023-21sh21:36:321
3100993813,6cyclictest16750-21unixbench_multi23:45:321
310099381,28cyclictest2822-21grep22:50:221
310299379,18cyclictest0-21swapper/321:15:173
310299379,13cyclictest0-21swapper/322:40:283
310299371,22cyclictest18588-21ls19:50:303
310299371,22cyclictest18588-21ls19:50:303
310199372,7cyclictest6050-21grep20:45:302
310199372,7cyclictest0-21swapper/223:20:142
3101993716,6cyclictest7750irq/56-dwc_otg21:55:152
310099376,3cyclictest0-21swapper/122:36:391
3100993718,10cyclictest908-21runrttasks21:25:391
3100993712,6cyclictest19371-21ssh23:10:231
310099370,26cyclictest0-21swapper/122:40:121
235552370,6sleep3310299cyclictest23:15:253
310299364,13cyclictest7950irq/56-dwc_otg_00:05:043
310299364,13cyclictest7950irq/56-dwc_otg_00:05:043
310299363,10cyclictest0-21swapper/323:30:163
310299363,10cyclictest0-21swapper/323:30:163
310299362,17cyclictest7950irq/56-dwc_otg_22:30:053
310299361,8cyclictest767350rm23:35:223
310199368,5cyclictest7750irq/56-dwc_otg00:05:042
310199368,5cyclictest7750irq/56-dwc_otg00:05:042
310199362,27cyclictest0-21swapper/223:46:092
3101993618,9cyclictest11712-21fschecks_count23:40:212
310199361,23cyclictest13041-21ls19:35:292
3101993610,7cyclictest29949-21cron20:25:042
310099368,6cyclictest7850irq/56-dwc_otg_23:40:321
310099367,22cyclictest0-21swapper/122:55:151
310099367,20cyclictest0-21swapper/122:05:161
310099362,25cyclictest0-21swapper/120:45:151
3100993612,9cyclictest17389-21ssh22:30:041
310299357,4cyclictest7950irq/56-dwc_otg_19:30:033
3102993514,4cyclictest16010-21fschecks_count19:45:213
310299351,18cyclictest25554-21ssh21:23:053
310199357,24cyclictest0-21swapper/219:40:172
310199356,6cyclictest7750irq/56-dwc_otg23:10:222
310199353,18cyclictest7950irq/56-dwc_otg_00:14:472
310099356,20cyclictest7415-21ssh21:40:161
310099353,23cyclictest28586-21missed_timers00:00:261
310099353,23cyclictest28586-21missed_timers00:00:261
310099351,27cyclictest14124-21ls19:40:181
310299348,5cyclictest39-21ksoftirqd/323:20:313
310299347,15cyclictest0-21swapper/320:55:163
310299347,15cyclictest0-21swapper/319:55:163
310299341,5cyclictest20917-21memory21:55:253
310199342,5cyclictest7950irq/56-dwc_otg_21:30:162
310199342,5cyclictest7950irq/56-dwc_otg_21:30:162
3101993417,9cyclictest10696-21sh21:00:192
3101993416,11cyclictest908-21runrttasks19:10:172
310099344,4cyclictest7750irq/56-dwc_otg21:45:321
310099342,6cyclictest0-21swapper/119:45:151
310299339,10cyclictest7850irq/56-dwc_otg_22:15:263
310299338,11cyclictest0-21swapper/300:05:183
310299336,17cyclictest0-21swapper/323:26:063
3102993323,7cyclictest381ktimersoftd/323:05:153
310299331,23cyclictest12762-21ls19:35:243
310299331,15cyclictest7750irq/56-dwc_otg21:30:213
310299331,15cyclictest7750irq/56-dwc_otg21:30:213
310199338,6cyclictest7950irq/56-dwc_otg_22:10:052
310199337,10cyclictest0-21swapper/200:37:512
310199336,5cyclictest7850irq/56-dwc_otg_20:15:242
310199333,14cyclictest0-21swapper/221:39:492
310199332,5cyclictest2514-21apt-get22:50:172
310199332,4cyclictest11833-21timerandwakeup23:00:302
310199332,4cyclictest11833-21timerandwakeup23:00:302
310099338,3cyclictest0-21swapper/123:30:321
310099338,3cyclictest0-21swapper/123:30:321
310099338,14cyclictest0-21swapper/122:15:301
310099337,15cyclictest0-21swapper/121:20:141
310099335,4cyclictest0-21swapper/123:55:151
310099335,21cyclictest7301-21ssh23:35:171
3100993326,5cyclictest9282-21sh22:20:161
310099331,6cyclictest7750irq/56-dwc_otg22:15:041
310099331,21cyclictest11442-21missed_timers23:00:251
310099331,21cyclictest11442-21missed_timers23:00:251
310099331,18cyclictest7750irq/56-dwc_otg21:33:151
310099331,18cyclictest7750irq/56-dwc_otg21:33:151
18402334,11sleep30-21swapper/319:05:253
18402334,11sleep30-21swapper/319:05:253
310299321,23cyclictest2244-21switchtime20:35:293
310299321,23cyclictest2244-21switchtime20:35:293
3102993211,5cyclictest917-21snmpd21:40:303
310199329,12cyclictest0-21swapper/223:30:282
310199329,12cyclictest0-21swapper/223:30:282
3101993220,4cyclictest31-21ksoftirqd/221:15:232
310199321,9cyclictest24133-21grep20:05:302
310199321,3cyclictest0-21swapper/222:00:172
310199321,10cyclictest26432-21latency_hist20:15:042
310099327,5cyclictest7109-21sort20:50:211
310099325,7cyclictest7750irq/56-dwc_otg00:30:321
310099322,5cyclictest0-21swapper/121:10:321
310099321,10cyclictest7950irq/56-dwc_otg_20:40:311
232052321,5sleep223215-21ssh00:33:352
310299318,5cyclictest37-21rcuc/300:10:223
310299318,14cyclictest0-21swapper/322:30:103
310299316,12cyclictest0-21swapper/321:46:393
3102993116,7cyclictest7410-21ssh22:55:243
310299311,4cyclictest16134-21/usr/sbin/munin00:25:273
310299311,4cyclictest16134-21/usr/sbin/munin00:25:273
3102993113,5cyclictest0-21swapper/300:21:143
310199318,4cyclictest0-21swapper/223:05:212
310199317,14cyclictest0-21swapper/223:15:212
310199317,14cyclictest0-21swapper/219:15:182
310199317,11cyclictest7750irq/56-dwc_otg00:20:042
310199315,15cyclictest7950irq/56-dwc_otg_19:50:042
310199314,6cyclictest7850irq/56-dwc_otg_20:40:222
310199312,6cyclictest7950irq/56-dwc_otg_21:50:142
310199312,6cyclictest7950irq/56-dwc_otg_21:50:142
310199311,7cyclictest24839-21ssh21:21:502
310199311,23cyclictest22735-21missed_timers22:35:262
3101993110,3cyclictest0-21swapper/200:20:102
310099318,21cyclictest0-21swapper/119:25:181
310099315,10cyclictest0-21swapper/119:20:161
310099314,5cyclictest27738-21unixbench_singl20:15:321
310099311,6cyclictest32084-21ssh23:25:241
3100993116,4cyclictest19718-21ssh00:29:411
3100993116,4cyclictest19718-21ssh00:29:411
3100993115,4cyclictest908-21runrttasks21:07:301
3100993114,14cyclictest0-21swapper/119:15:181
310099311,24cyclictest19599-21cut19:55:201
310099311,18cyclictest22737-21latency_hist20:05:041
310299303,5cyclictest7850irq/56-dwc_otg_20:45:193
310299302,7cyclictest3235-21cut19:10:183
3102993021,7cyclictest29049-21ssh21:26:453
310299302,12cyclictest0-21swapper/322:35:303
310199304,10cyclictest7850irq/56-dwc_otg_20:33:142
310199302,13cyclictest7950irq/56-dwc_otg_21:10:282
310199301,6cyclictest7950irq/56-dwc_otg_20:25:212
3101993013,4cyclictest0-21swapper/221:25:332
310199301,19cyclictest8503-21ssh21:40:302
310099306,15cyclictest0-21swapper/120:55:091
310099302,8cyclictest0-21swapper/119:35:191
310099302,17cyclictest17556-21/usr/sbin/munin19:50:311
310099302,17cyclictest17556-21/usr/sbin/munin19:50:311
310299298,5cyclictest0-21swapper/323:14:563
310299293,9cyclictest0-21swapper/323:57:163
310299292,13cyclictest7850irq/56-dwc_otg_19:15:223
310299291,14cyclictest7950irq/56-dwc_otg_20:25:323
310299291,12cyclictest7850irq/56-dwc_otg_19:20:163
310199295,10cyclictest7850irq/56-dwc_otg_00:25:222
310199295,10cyclictest7850irq/56-dwc_otg_00:25:222
310199291,22cyclictest0-21swapper/222:42:252
310199291,19cyclictest0-21swapper/220:35:172
310199291,19cyclictest0-21swapper/220:35:172
310199291,17cyclictest0-21swapper/221:05:272
310099299,4cyclictest11253-21processes21:00:281
310099292,4cyclictest7850irq/56-dwc_otg_19:10:281
123732290,4sleep312376-21aten_rbpower_po21:05:173
310299284,4cyclictest7950irq/56-dwc_otg_20:40:223
310299283,14cyclictest0-21swapper/320:15:483
310299281,13cyclictest7950irq/56-dwc_otg_00:35:143
310099281,14cyclictest0-21swapper/120:30:211
310299273,4cyclictest20581-21fschecks_count00:30:213
310299273,11cyclictest7950irq/56-dwc_otg_19:40:213
310299272,8cyclictest0-21swapper/320:20:153
310299272,7cyclictest7950irq/56-dwc_otg_19:30:173
310199271,5cyclictest7850irq/56-dwc_otg_19:51:282
310199271,5cyclictest7850irq/56-dwc_otg_19:51:282
27502273,16sleep10-21swapper/119:09:031
27502273,16sleep10-21swapper/119:09:031
83222261,4sleep30-21swapper/320:54:523
310299261,14cyclictest22162-21ls20:00:303
17922263,13sleep20-21swapper/219:05:242
17922263,13sleep20-21swapper/219:05:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional