You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 14:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Jun 14, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7279991005,13cyclictest0-21swapper/021:25:150
7279999512,63cyclictest29833-21wc20:10:260
7279999131,17cyclictest7850irq/56-dwc_otg23:35:170
7279998516,40cyclictest0-21swapper/022:50:140
271992840,2sleep20-21swapper/220:05:022
727999827,60cyclictest18623-21grep19:40:210
7279998232,7cyclictest7850irq/56-dwc_otg00:35:130
7279998212,52cyclictest8906-21grep22:15:310
7279998210,58cyclictest0-21swapper/000:05:010
727999816,54cyclictest17840-21grep21:05:300
727999779,55cyclictest20448-21ssh00:27:490
727999779,52cyclictest12984-21latency_hist20:55:030
727999779,52cyclictest12984-21latency_hist20:55:030
727999778,57cyclictest0-21swapper/020:55:190
7279997736,6cyclictest0-21swapper/023:55:140
727999757,53cyclictest22583-21latency_hist21:15:020
727999757,49cyclictest21795-21grep23:10:270
727999756,60cyclictest24453-21ssh21:55:280
727999749,54cyclictest10124-21ssh21:38:310
7279997434,4cyclictest8050irq/56-dwc_otg_22:05:150
7279997434,4cyclictest8050irq/56-dwc_otg_22:05:150
7279997410,51cyclictest6120-21awk23:30:310
727999725,53cyclictest0-21swapper/000:22:050
727999725,53cyclictest0-21swapper/000:22:050
7279997214,29cyclictest2281450ssh21:15:060
7279997214,29cyclictest2281450ssh21:15:060
727999718,47cyclictest29248-21rm23:20:190
727999709,33cyclictest0-21swapper/023:50:310
727999705,51cyclictest5594-21aten_rbpower_en00:10:200
727999701,53cyclictest0-21swapper/000:15:150
727999688,49cyclictest3958-21ssh00:07:500
727999687,49cyclictest0-21swapper/022:00:330
727999679,45cyclictest24854-21/usr/sbin/munin23:15:130
7279996724,28cyclictest0-21swapper/022:30:160
7279996711,44cyclictest16426-21basename19:35:170
7279996710,47cyclictest6883-21runrttasks20:35:350
7279996614,43cyclictest3156-21ssh23:27:390
7279996614,43cyclictest3156-21ssh23:27:390
7279996611,40cyclictest27692-21iwlist20:05:210
7279996610,45cyclictest17413-21basename23:45:160
727999657,41cyclictest17426-21ssh21:46:470
727999656,47cyclictest0-21swapper/020:30:230
7279996515,40cyclictest18193-21diskmemload22:27:110
7279996511,46cyclictest2358-21taskset22:47:230
192162650,2sleep30-21swapper/321:50:023
727999644,48cyclictest27107-21sort20:05:020
7279996412,37cyclictest13473-21/usr/sbin/munin23:40:180
7279996411,45cyclictest27282-21sh22:38:460
7279996410,37cyclictest11611-21uniq21:40:190
272642640,1sleep30-21swapper/322:38:453
727999635,45cyclictest16445-21latency_hist23:05:020
727999631,30cyclictest14482-21latency_hist19:30:020
727999628,41cyclictest12173-21ssh23:00:020
727999628,41cyclictest12173-21ssh23:00:020
7279996215,7cyclictest3537-21ssh22:10:100
728499613,10cyclictest17863-21ssh00:25:132
727999618,35cyclictest19858-21rm23:09:010
7279996116,23cyclictest19987-21tr19:45:160
727999608,44cyclictest0-21swapper/000:33:270
7279996016,40cyclictest13523-21rm22:21:100
728499596,6cyclictest0-21swapper/222:55:112
728499596,6cyclictest0-21swapper/222:55:112
7279995912,29cyclictest21925-21ssh21:52:290
170842590,3sleep2301ktimersoftd/223:45:022
727999588,27cyclictest0-21swapper/019:15:210
727999588,27cyclictest0-21swapper/019:15:210
7279995811,37cyclictest10706-21chrt20:46:540
7279995810,33cyclictest30217-21ssh21:23:320
88982570,22sleep28050irq/56-dwc_otg_23:35:042
727999573,5cyclictest0-21swapper/020:25:150
727999573,5cyclictest0-21swapper/020:25:150
7279995716,37cyclictest101ktimersoftd/020:21:190
727999568,31cyclictest5516-21ssh21:32:370
727999567,33cyclictest0-21swapper/022:41:570
69372562,12sleep20-21swapper/219:09:042
67242561,48sleep30-21swapper/319:06:183
7279995522,18cyclictest31964-21sed20:15:290
7284995423,18cyclictest0-21swapper/221:40:122
727999542,41cyclictest0-21swapper/020:44:030
728399536,6cyclictest7850irq/56-dwc_otg21:10:151
7283995314,25cyclictest0-21swapper/100:35:161
727999531,33cyclictest0-21swapper/019:55:080
728499524,4cyclictest0-21swapper/220:05:152
728399522,5cyclictest0-21swapper/120:30:121
727999529,31cyclictest0-21swapper/019:32:100
727999526,31cyclictest0-21swapper/019:51:230
728399516,19cyclictest0-21swapper/122:50:171
7283995127,11cyclictest7950irq/56-dwc_otg_21:50:141
7285995018,20cyclictest13164-21ssh23:40:093
728499503,8cyclictest7950irq/56-dwc_otg_19:15:122
728499503,8cyclictest7950irq/56-dwc_otg_19:15:122
7283995036,7cyclictest7850irq/56-dwc_otg23:05:141
7279995026,5cyclictest11257-21fschecks_time19:20:190
727999501,39cyclictest8756-21runrttasks19:13:390
728399499,6cyclictest7850irq/56-dwc_otg22:25:191
69092483,37sleep00-21swapper/019:08:420
728399473,33cyclictest8050irq/56-dwc_otg_22:55:111
728399473,33cyclictest8050irq/56-dwc_otg_22:55:111
728599467,25cyclictest0-21swapper/321:35:143
728599467,25cyclictest0-21swapper/319:15:143
728599467,25cyclictest0-21swapper/319:15:143
728499467,25cyclictest0-21swapper/219:55:082
728399461,5cyclictest11306-21ssh21:40:151
7279994628,8cyclictest8050irq/56-dwc_otg_21:00:270
728599457,5cyclictest8050irq/56-dwc_otg_22:20:163
728599456,7cyclictest8050irq/56-dwc_otg_23:20:143
728599452,11cyclictest0-21swapper/323:55:143
728599451,2cyclictest0-21swapper/323:35:173
728499457,26cyclictest0-21swapper/220:15:152
728499457,23cyclictest0-21swapper/221:05:162
728399457,25cyclictest0-21swapper/120:40:161
728399457,25cyclictest0-21swapper/119:20:151
728399456,26cyclictest0-21swapper/122:45:151
728399452,12cyclictest7850irq/56-dwc_otg23:30:141
728599447,24cyclictest0-21swapper/321:00:163
728499447,22cyclictest0-21swapper/220:20:162
728599437,22cyclictest0-21swapper/320:40:163
728599437,18cyclictest37-21rcuc/323:50:583
728599431,29cyclictest381ktimersoftd/319:40:273
7283994323,3cyclictest7850irq/56-dwc_otg21:25:151
7285994219,10cyclictest381ktimersoftd/321:29:273
728499427,23cyclictest0-21swapper/220:45:072
728399424,5cyclictest7850irq/56-dwc_otg00:10:281
7283994216,15cyclictest0-21swapper/123:35:091
728599418,20cyclictest37-21rcuc/323:10:483
7285994135,4cyclictest26122-21sh00:35:143
7285994114,13cyclictest8050irq/56-dwc_otg_22:30:163
728399414,4cyclictest0-21swapper/100:05:141
192962410,8sleep229-21rcuc/222:29:142
728599407,23cyclictest0-21swapper/319:55:073
7285994017,9cyclictest7950irq/56-dwc_otg_22:50:173
728599401,17cyclictest8050irq/56-dwc_otg_19:45:193
728499406,16cyclictest0-21swapper/223:50:132
161952400,2sleep30-21swapper/300:22:353
161952400,2sleep30-21swapper/300:22:353
728599398,17cyclictest39-21ksoftirqd/320:50:023
728599397,5cyclictest0-21swapper/321:50:143
728599397,24cyclictest0-21swapper/300:05:163
728599397,23cyclictest0-21swapper/300:25:153
728599391,7cyclictest112850ssh23:25:213
728599391,7cyclictest112850ssh23:25:213
728599391,23cyclictest0-21swapper/319:20:153
7285993910,16cyclictest37-21rcuc/320:30:173
728499395,25cyclictest16655-21ssh23:05:152
728399397,23cyclictest0-21swapper/123:25:151
728399397,23cyclictest0-21swapper/123:25:151
728399397,22cyclictest0-21swapper/119:10:151
728399396,24cyclictest0-21swapper/120:05:161
7283993917,6cyclictest30953-21aten_rbpower_po20:15:161
728399391,24cyclictest3085-21timerwakeupswit20:25:301
728399391,24cyclictest3085-21timerwakeupswit20:25:301
728599386,5cyclictest39-21ksoftirqd/300:10:473
728599384,11cyclictest8050irq/56-dwc_otg_21:55:293
7285993812,5cyclictest0-21swapper/322:00:163
728599381,20cyclictest39-21ksoftirqd/323:19:033
728599381,15cyclictest18296-21diskmemload21:10:313
728499385,20cyclictest4186-21ssh22:50:082
7284993813,4cyclictest7950irq/56-dwc_otg_19:30:192
728399387,21cyclictest0-21swapper/119:15:141
728399387,21cyclictest0-21swapper/119:15:141
7283993813,12cyclictest23-21ksoftirqd/120:55:031
7283993813,12cyclictest23-21ksoftirqd/120:55:031
7285993711,6cyclictest7850irq/56-dwc_otg20:05:243
728499378,12cyclictest8050irq/56-dwc_otg_00:15:022
728499377,17cyclictest10249-21missed_timers00:15:232
728499373,19cyclictest8050irq/56-dwc_otg_21:45:192
728499371,24cyclictest5160-21cut23:30:192
728399377,6cyclictest7850irq/56-dwc_otg20:45:151
728599365,17cyclictest8050irq/56-dwc_otg_23:05:043
728599365,17cyclictest37-21rcuc/321:15:243
728599365,17cyclictest37-21rcuc/321:15:243
728599361,3cyclictest8050irq/56-dwc_otg_20:20:153
7285993612,6cyclictest39-21ksoftirqd/320:55:283
728599361,17cyclictest0-21swapper/321:05:163
728499367,21cyclictest0-21swapper/222:40:162
728499366,12cyclictest8050irq/56-dwc_otg_21:15:202
728499366,12cyclictest8050irq/56-dwc_otg_21:15:202
7284993614,9cyclictest7950irq/56-dwc_otg_00:35:152
7283993617,5cyclictest23-21ksoftirqd/100:30:291
7283993617,5cyclictest17109-21/usr/sbin/munin21:05:211
728399361,4cyclictest0-21swapper/100:25:031
728399361,4cyclictest0-21swapper/100:25:031
7283993614,15cyclictest0-21swapper/122:15:241
728399361,24cyclictest25818-21switchtime23:15:271
7283993610,4cyclictest0-21swapper/121:20:231
728599356,18cyclictest37-21rcuc/300:16:313
728599355,16cyclictest37-21rcuc/319:10:413
728599355,13cyclictest8050irq/56-dwc_otg_22:50:023
728599354,18cyclictest39-21ksoftirqd/322:06:593
728599354,18cyclictest39-21ksoftirqd/322:06:593
728599352,19cyclictest0-21swapper/322:16:293
728599351,27cyclictest30090-21ls00:00:193
728599351,14cyclictest381ktimersoftd/323:30:303
728599351,11cyclictest7950irq/56-dwc_otg_23:45:153
728499351,22cyclictest2666-21missed_timers20:25:242
728499351,22cyclictest2666-21missed_timers20:25:242
728499351,22cyclictest19762-21ls21:50:182
728399358,6cyclictest8050irq/56-dwc_otg_19:55:031
728399357,6cyclictest8050irq/56-dwc_otg_21:55:191
728399355,19cyclictest0-21swapper/123:50:081
728399352,6cyclictest14008-21/usr/sbin/munin23:40:251
7283993526,6cyclictest221ktimersoftd/123:45:161
7283993511,4cyclictest0-21swapper/122:24:001
728599347,16cyclictest381ktimersoftd/320:10:463
728599346,26cyclictest12531-21ssh23:00:153
7285993421,5cyclictest28963-21ssh21:21:153
7285993414,8cyclictest7850irq/56-dwc_otg19:35:173
728599341,3cyclictest7850irq/56-dwc_otg22:25:133
7285993412,12cyclictest11267-21sh21:40:123
7285993411,4cyclictest0-21swapper/300:30:303
728499349,5cyclictest0-21swapper/221:55:222
728499347,7cyclictest8050irq/56-dwc_otg_22:00:322
728499347,7cyclictest8050irq/56-dwc_otg_21:25:022
728499347,6cyclictest8050irq/56-dwc_otg_22:20:302
728499345,5cyclictest7950irq/56-dwc_otg_23:20:302
7284993414,4cyclictest29-21rcuc/223:28:592
7284993414,4cyclictest29-21rcuc/223:28:592
728499341,23cyclictest0-21swapper/223:12:142
728399349,5cyclictest7850irq/56-dwc_otg22:35:231
728399345,20cyclictest3057-21ssh21:30:151
728399344,24cyclictest8050irq/56-dwc_otg_19:55:141
728399343,8cyclictest0-21swapper/122:05:151
728399343,8cyclictest0-21swapper/122:05:151
728399343,6cyclictest23-21ksoftirqd/123:20:321
728399342,8cyclictest31716-21ssh22:05:031
7283993411,12cyclictest0-21swapper/122:11:031
728599338,9cyclictest37-21rcuc/322:44:343
728599336,15cyclictest37-21rcuc/322:55:163
728599336,15cyclictest37-21rcuc/322:55:163
728599335,19cyclictest381ktimersoftd/320:30:023
728599335,19cyclictest381ktimersoftd/320:30:023
728599334,8cyclictest37-21rcuc/320:05:033
7285993312,15cyclictest37-21rcuc/322:13:243
728499338,11cyclictest0-21swapper/221:30:172
728499336,5cyclictest8050irq/56-dwc_otg_21:10:172
728499332,4cyclictest7850irq/56-dwc_otg22:30:102
728499332,4cyclictest32449-21apt-get22:45:132
7284993317,5cyclictest11943-21open_inodes20:50:272
7284993317,5cyclictest11943-21open_inodes20:50:272
728499331,21cyclictest25150-21uniq23:15:192
728499331,20cyclictest8076-21ls20:40:242
728399338,6cyclictest8050irq/56-dwc_otg_21:35:251
728399338,13cyclictest0-21swapper/121:15:161
728399338,13cyclictest0-21swapper/121:15:161
728399336,17cyclictest0-21swapper/100:25:151
728399334,4cyclictest8050irq/56-dwc_otg_00:15:131
728399333,18cyclictest5800-21wget20:35:161
728399332,14cyclictest7950irq/56-dwc_otg_23:55:271
728399331,9cyclictest0-21swapper/119:30:151
728399331,7cyclictest16652-21tune2fs19:35:201
7283993315,5cyclictest727-21runrttasks21:00:161
728399331,22cyclictest13285-21cpuspeed20:55:171
728599327,9cyclictest37-21rcuc/320:52:303
728599327,9cyclictest37-21rcuc/320:52:303
728599326,15cyclictest16121-21idleruntime-cro19:35:023
728599325,5cyclictest37-21rcuc/321:32:213
728499329,5cyclictest8050irq/56-dwc_otg_21:35:282
728499325,6cyclictest8050irq/56-dwc_otg_23:55:302
728499324,16cyclictest8050irq/56-dwc_otg_19:45:022
7284993214,5cyclictest18296-21diskmemload23:04:082
728499321,19cyclictest11807-21switchtime19:20:272
728499321,18cyclictest0-21swapper/221:00:082
728499321,17cyclictest0-21swapper/200:00:222
7284993211,5cyclictest0-21swapper/200:20:172
7284993211,5cyclictest0-21swapper/200:20:172
728399327,5cyclictest7850irq/56-dwc_otg22:30:301
728399326,5cyclictest8050irq/56-dwc_otg_21:45:311
728399324,5cyclictest7850irq/56-dwc_otg22:45:021
728399323,5cyclictest30091-21idleruntime00:00:191
728399322,8cyclictest21-21rcuc/120:10:191
7283993223,6cyclictest8050irq/56-dwc_otg_23:00:241
7283993210,10cyclictest0-21swapper/123:13:361
728599317,14cyclictest39-21ksoftirqd/320:35:343
728499319,4cyclictest0-21swapper/221:28:452
728499318,6cyclictest8050irq/56-dwc_otg_20:35:182
728499313,14cyclictest0-21swapper/200:30:182
728499312,6cyclictest7850irq/56-dwc_otg00:05:172
7284993115,4cyclictest20045-21aten_rbpower_vo19:45:162
728499311,20cyclictest8050irq/56-dwc_otg_22:05:292
728499311,20cyclictest8050irq/56-dwc_otg_22:05:292
728399313,8cyclictest0-21swapper/119:48:251
728399312,7cyclictest0-21swapper/119:25:161
728599305,13cyclictest37-21rcuc/319:54:023
728599301,3cyclictest8050irq/56-dwc_otg_20:15:143
728499306,12cyclictest8050irq/56-dwc_otg_22:10:032
728499303,6cyclictest0-21swapper/223:45:202
7284993012,4cyclictest18296-21diskmemload22:15:142
728399304,12cyclictest7950irq/56-dwc_otg_20:20:211
7283993010,7cyclictest0-21swapper/119:40:131
728499299,3cyclictest0-21swapper/222:38:022
728499297,9cyclictest8050irq/56-dwc_otg_20:31:212
728499295,11cyclictest7950irq/56-dwc_otg_19:51:012
728499292,7cyclictest7297-21/usr/sbin/munin19:10:202
728499291,18cyclictest0-21swapper/219:35:172
728399293,4cyclictest7850irq/56-dwc_otg20:00:111
728599282,5cyclictest0-21swapper/319:25:133
728499286,4cyclictest8050irq/56-dwc_otg_19:25:252
728499281,12cyclictest7950irq/56-dwc_otg_20:13:302
67692283,14sleep10-21swapper/119:06:531
728499273,12cyclictest8050irq/56-dwc_otg_20:55:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional