You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 06:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Jun 16, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2770122780,1sleep30-21swapper/319:05:303
2693022750,2sleep10-21swapper/119:05:201
2921321160,1sleep00-21swapper/022:54:510
5802940,4sleep0581-21/usr/sbin/munin22:20:170
199812910,2sleep20-21swapper/220:15:232
200012790,4sleep2301ktimersoftd/222:04:412
28360997410,52cyclictest18411-21latency_hist00:00:030
181302740,4sleep10-21swapper/123:59:461
97062720,2chrt0-21swapper/023:10:170
2836099722,39cyclictest0-21swapper/022:05:150
2836099722,39cyclictest0-21swapper/022:05:150
28363997115,34cyclictest8050irq/56-dwc_otg_00:15:153
2836099701,23cyclictest0-21swapper/023:00:160
271422690,5sleep0101ktimersoftd/019:05:230
2836299677,4cyclictest8050irq/56-dwc_otg_20:50:292
2836299677,4cyclictest8050irq/56-dwc_otg_20:50:292
28360996521,26cyclictest30060-21idleruntime-cro20:45:030
28360996421,6cyclictest0-21swapper/022:15:140
28360996319,23cyclictest27195-21/usr/sbin/munin00:10:200
2836399624,14cyclictest0-21swapper/323:45:133
2836399624,14cyclictest0-21swapper/323:45:133
28360996233,7cyclictest7850irq/56-dwc_otg23:30:030
28360996232,13cyclictest7850irq/56-dwc_otg20:50:030
28361996112,24cyclictest28110-21apt-get21:35:151
28360996131,14cyclictest7850irq/56-dwc_otg21:25:030
28360996031,14cyclictest8050irq/56-dwc_otg_22:50:030
28360996031,14cyclictest8050irq/56-dwc_otg_22:50:030
28362995922,2cyclictest0-21swapper/220:30:032
2836299591,9cyclictest0-21swapper/219:40:132
28360995914,22cyclictest8680-21aten_rbpower_po19:45:160
2836099591,38cyclictest0-21swapper/021:50:170
28360995816,27cyclictest28359-21cyclictest21:45:140
2836099581,40cyclictest5241-21fschecks_count19:35:200
2836299577,32cyclictest0-21swapper/222:05:172
2836299577,32cyclictest0-21swapper/222:05:172
2836099579,27cyclictest10433-21aten_rbpower_cu19:50:160
2836399562,24cyclictest0-21swapper/319:25:163
2836199562,11cyclictest26290-21ssh23:30:181
28360995640,9cyclictest138462sleep023:54:250
28360995628,4cyclictest7850irq/56-dwc_otg23:15:140
28360995625,17cyclictest0-21swapper/022:00:240
2836099559,33cyclictest0-21swapper/022:25:280
28361995421,8cyclictest32105-21apt-get20:50:141
28361995421,8cyclictest32105-21apt-get20:50:141
28360995423,17cyclictest0-21swapper/022:10:210
2836099542,30cyclictest0-21swapper/019:10:250
28360995419,17cyclictest0-21swapper/019:30:230
2836099541,30cyclictest7021-21fschecks_count19:40:180
145482540,2sleep20-21swapper/223:55:162
2836199532,7cyclictest7950irq/56-dwc_otg_00:35:161
2836199532,7cyclictest7950irq/56-dwc_otg_00:35:161
28360995325,4cyclictest7850irq/56-dwc_otg00:35:310
28360995325,4cyclictest7850irq/56-dwc_otg00:35:310
2836099532,31cyclictest28411-21latency_hist20:40:040
2836099532,31cyclictest0-21swapper/021:55:190
28360995320,16cyclictest7950irq/56-dwc_otg_20:55:030
28360995320,16cyclictest7950irq/56-dwc_otg_20:55:030
28360995320,12cyclictest0-21swapper/019:55:170
2836099531,34cyclictest0-21swapper/022:36:150
2836199525,33cyclictest19048-21ssh22:02:551
28360995225,7cyclictest3029-21idleruntime-cro21:00:020
28360995224,10cyclictest7850irq/56-dwc_otg23:05:310
2836099521,37cyclictest2628-21ssh00:20:040
2836099521,32cyclictest6747-21diskmemload22:40:170
2836099521,26cyclictest9-21ksoftirqd/000:30:280
28360995212,6cyclictest8050irq/56-dwc_otg_20:15:160
2836199519,19cyclictest24247-21ssh00:05:471
28361995114,21cyclictest5133-21ssh00:22:241
2836099511,7cyclictest26601-21apt00:10:030
28360995117,16cyclictest5496-21fschecks_time21:05:210
2836099511,47cyclictest0-21swapper/023:40:130
2836099511,27cyclictest6580-21uniq23:45:200
2836099511,27cyclictest6580-21uniq23:45:200
2836099511,26cyclictest32611-21wc19:20:240
2836199509,12cyclictest23710-21ls20:25:241
2836199504,23cyclictest21776-21ssh21:26:421
28361995012,24cyclictest30233-21sed20:45:041
2836099501,40cyclictest0-21swapper/021:34:340
2836299492,7cyclictest0-21swapper/219:30:142
2836199498,26cyclictest15673-21ssh21:20:171
2836199497,27cyclictest0-21swapper/123:50:151
2836099491,26cyclictest1695-21fschecks_time19:25:210
2836199486,26cyclictest10263-21sh21:52:021
2836199482,30cyclictest20812-21chrt23:23:181
2836199481,3cyclictest0-21swapper/120:20:151
28361994813,17cyclictest1858-21if_wlan023:00:221
28361994811,23cyclictest3692-21sed19:30:211
28360994823,13cyclictest30763-21grep19:15:250
28360994823,13cyclictest30763-21grep19:15:250
28360994818,23cyclictest0-21swapper/021:16:420
28360994818,21cyclictest0-21swapper/000:02:540
2836099481,5cyclictest0-21swapper/021:35:230
2836099481,25cyclictest18268-21grep23:20:200
2836099481,23cyclictest0-21swapper/021:25:310
2836099481,11cyclictest24747-21latency_hist20:30:030
321002470,16sleep18050irq/56-dwc_otg_22:19:011
2836299477,25cyclictest0-21swapper/220:55:162
2836199474,14cyclictest920-21timerwakeupswit21:40:301
28361994727,8cyclictest7950irq/56-dwc_otg_22:23:121
28361994710,22cyclictest6309-21ssh22:26:021
2836099474,3cyclictest0-21swapper/021:40:150
28360994722,7cyclictest7950irq/56-dwc_otg_22:55:150
2836099471,44cyclictest7713-21ssh21:10:240
2836099471,44cyclictest7713-21ssh21:10:240
2836099471,29cyclictest30221-21grep23:35:040
28360994712,3cyclictest0-21swapper/022:35:030
28360994712,3cyclictest0-21swapper/022:35:030
2836299467,25cyclictest0-21swapper/221:05:082
2836299461,19cyclictest8050irq/56-dwc_otg_00:35:032
2836199468,24cyclictest27563-21mv22:51:511
2836199467,25cyclictest0-21swapper/120:30:171
2836199467,24cyclictest0-21swapper/120:00:161
28361994611,20cyclictest2097-21apt23:40:121
2836099462,31cyclictest4835-21latency_hist21:05:030
28360994612,19cyclictest14001-21tr20:00:140
285422450,2sleep20-21swapper/223:32:172
2836199457,25cyclictest0-21swapper/119:45:161
2836199457,25cyclictest0-21swapper/119:35:161
2836199457,25cyclictest0-21swapper/119:25:171
2836199457,23cyclictest0-21swapper/121:05:091
2836199453,10cyclictest0-21swapper/121:15:171
28361994527,7cyclictest7850irq/56-dwc_otg22:40:181
2836199452,27cyclictest23691-21rm22:08:501
2836199452,27cyclictest23691-21rm22:08:501
28361994512,18cyclictest7585-21ssh23:45:321
28361994512,18cyclictest7585-21ssh23:45:321
28360994521,15cyclictest0-21swapper/020:05:160
28360994521,15cyclictest0-21swapper/020:05:160
28360994519,20cyclictest0-21swapper/020:23:290
2836199449,13cyclictest10650-21ssh22:31:251
2836199449,13cyclictest10650-21ssh22:31:251
28361994410,20cyclictest11710-21ssh23:11:511
28360994419,13cyclictest0-21swapper/000:25:420
207082440,5sleep320692-21sh21:25:283
2836399432,6cyclictest8050irq/56-dwc_otg_22:05:153
2836399432,6cyclictest8050irq/56-dwc_otg_22:05:153
28361994320,12cyclictest7950irq/56-dwc_otg_21:46:221
28361994314,13cyclictest9369-21ssh00:27:351
28361994311,21cyclictest6747-21diskmemload00:34:131
28360994316,6cyclictest125-21systemd-journal20:10:150
2836099431,12cyclictest0-21swapper/000:20:160
256432430,4sleep231-21ksoftirqd/200:08:202
28363994224,7cyclictest1228-21idleruntime-cro20:55:033
28363994224,7cyclictest1228-21idleruntime-cro20:55:033
28363994215,3cyclictest7950irq/56-dwc_otg_00:00:313
2836299427,25cyclictest0-21swapper/219:10:162
2836299427,21cyclictest0-21swapper/220:10:172
2836299425,24cyclictest25525-21ssh22:50:162
2836299424,9cyclictest8050irq/56-dwc_otg_19:45:132
2836299421,13cyclictest0-21swapper/219:55:302
2836199425,22cyclictest2042-21ssh00:19:201
28361994217,10cyclictest7850irq/56-dwc_otg00:02:581
28361994210,13cyclictest30178-21ssh22:55:231
28361994210,12cyclictest16246-21rm23:17:331
28361994210,11cyclictest22998-21runrttasks22:46:051
28361994210,11cyclictest22998-21runrttasks22:46:051
28360994219,14cyclictest0-21swapper/023:36:290
28360994218,14cyclictest0-21swapper/020:31:470
2836399419,4cyclictest0-21swapper/300:13:003
2836399416,22cyclictest8050irq/56-dwc_otg_22:30:153
2836399416,22cyclictest8050irq/56-dwc_otg_22:30:153
2836399416,22cyclictest0-21swapper/321:05:093
2836399413,8cyclictest0-21swapper/321:55:133
28363994120,6cyclictest696050wget00:25:153
2836299418,21cyclictest0-21swapper/219:15:162
2836299418,21cyclictest0-21swapper/219:15:162
2836299416,22cyclictest0-21swapper/220:05:182
2836299416,22cyclictest0-21swapper/220:05:182
28362994113,6cyclictest727-21runrttasks23:40:242
2836199419,16cyclictest7850irq/56-dwc_otg20:10:171
2836199417,19cyclictest6974-21ssh23:05:471
2836199416,24cyclictest0-21swapper/122:35:091
28361994114,12cyclictest8050irq/56-dwc_otg_23:26:101
28361994110,16cyclictest6747-21diskmemload00:14:051
28361994110,11cyclictest26104-21ssh21:32:031
2836399407,23cyclictest0-21swapper/322:25:163
2836399401,4cyclictest8050irq/56-dwc_otg_22:55:213
28362994015,6cyclictest32287-21grep00:15:312
2836199409,4cyclictest0-21swapper/122:14:171
2836199403,4cyclictest8050irq/56-dwc_otg_20:35:221
28361994010,4cyclictest12494-21fschecks_time19:55:211
171912400,2sleep30-21swapper/323:58:023
2836299397,5cyclictest0-21swapper/223:00:162
2836299397,24cyclictest0-21swapper/222:35:172
2836299396,23cyclictest0-21swapper/220:00:162
2836299393,3cyclictest0-21swapper/223:20:142
28362993919,5cyclictest13467-21grep21:55:302
28362993916,5cyclictest4057-21grep00:20:312
2836199399,12cyclictest10150-21ssh21:13:331
2836199399,12cyclictest10150-21ssh21:13:331
2836199398,13cyclictest7950irq/56-dwc_otg_19:40:131
28363993821,5cyclictest8050irq/56-dwc_otg_00:05:163
2836299387,22cyclictest0-21swapper/220:40:162
2836299387,21cyclictest0-21swapper/222:40:182
2836299383,30cyclictest8050irq/56-dwc_otg_00:10:142
28362993820,12cyclictest31-21ksoftirqd/200:35:152
28362993820,12cyclictest31-21ksoftirqd/200:35:152
28362993812,6cyclictest750-21snmpd21:35:322
2836199386,19cyclictest30282-21taskset20:45:081
2836199384,13cyclictest1083-21taskset19:24:331
28361993822,7cyclictest7850irq/56-dwc_otg22:00:021
28361993815,9cyclictest221ktimersoftd/123:39:531
28361993814,17cyclictest20770-21sshd20:17:241
2836399375,16cyclictest0-21swapper/321:42:483
2836399371,6cyclictest8050irq/56-dwc_otg_23:10:153
28362993715,5cyclictest727-21runrttasks21:32:062
2836299371,18cyclictest0-21swapper/200:25:182
233942370,2sleep229-21rcuc/223:25:332
232392370,1sleep20-21swapper/222:46:322
232392370,1sleep20-21swapper/222:46:322
2836399368,16cyclictest0-21swapper/323:15:163
2836399367,15cyclictest0-21swapper/320:25:163
2836399363,20cyclictest23968-21ssh21:30:163
2836399362,11cyclictest0-21swapper/323:20:143
2836399361,24cyclictest16324-21missed_timers21:20:243
2836299366,19cyclictest0-21swapper/221:29:192
2836299365,8cyclictest7850irq/56-dwc_otg23:15:152
2836299364,4cyclictest7850irq/56-dwc_otg19:35:142
28362993621,4cyclictest4690-21/usr/sbin/munin22:25:212
28362993611,5cyclictest7950irq/56-dwc_otg_21:10:272
28362993611,5cyclictest7950irq/56-dwc_otg_21:10:272
28362993611,14cyclictest0-21swapper/222:12:112
2836199369,4cyclictest7850irq/56-dwc_otg19:55:031
2836399356,12cyclictest8050irq/56-dwc_otg_22:00:283
2836399354,10cyclictest8050irq/56-dwc_otg_22:19:043
28363993520,4cyclictest7798-21sh21:10:253
28363993520,4cyclictest7798-21sh21:10:253
2836399351,4cyclictest8050irq/56-dwc_otg_20:20:213
28363993510,8cyclictest12099-21latency_hist21:55:043
2836299357,22cyclictest0-21swapper/220:23:052
28362993526,5cyclictest301ktimersoftd/219:20:162
2836299351,24cyclictest9907-21ls22:30:302
2836299351,24cyclictest9907-21ls22:30:302
2836299351,23cyclictest10747-21ls19:50:202
28362993511,13cyclictest0-21swapper/223:15:032
2836199355,6cyclictest4969-21cat21:05:031
28361993510,6cyclictest0-21swapper/120:55:321
28363993421,4cyclictest8050irq/56-dwc_otg_23:50:183
28363993416,5cyclictest17469-21cpuspeed22:40:193
2836399341,23cyclictest847-21cut22:20:203
2836299348,14cyclictest31437-21grep20:45:302
2836299345,9cyclictest61-21kswapd022:15:142
2836299345,20cyclictest6270-21ssh23:45:162
2836299345,20cyclictest6270-21ssh23:45:162
2836299343,7cyclictest7850irq/56-dwc_otg23:05:242
2836299342,18cyclictest0-21swapper/220:30:212
2836199347,15cyclictest0-21swapper/119:10:161
2836399339,14cyclictest0-21swapper/300:30:123
2836399333,3cyclictest7850irq/56-dwc_otg00:35:533
2836399333,3cyclictest7850irq/56-dwc_otg00:35:533
2836399332,5cyclictest6056-21kernelversion23:05:243
2836399331,4cyclictest0-21swapper/322:45:183
2836399331,4cyclictest0-21swapper/322:45:183
2836399331,21cyclictest28690-21cpuspeed20:40:183
2836299335,17cyclictest8050irq/56-dwc_otg_21:50:172
2836299334,5cyclictest7850irq/56-dwc_otg22:20:142
28362993317,9cyclictest996-21ssh23:38:392
2836299331,16cyclictest1062950ssh23:50:182
2836299331,14cyclictest7950irq/56-dwc_otg_21:40:272
292722320,2sleep337-21rcuc/322:54:583
2836399328,10cyclictest0-21swapper/321:15:163
2836399326,10cyclictest0-21swapper/321:46:313
2836399322,4cyclictest8050irq/56-dwc_otg_23:00:283
28363993217,5cyclictest19409-21seq20:15:133
28363993212,7cyclictest8050irq/56-dwc_otg_23:30:273
28363993210,8cyclictest0-21swapper/300:20:303
2836299327,15cyclictest0-21swapper/222:55:172
2836299326,6cyclictest7950irq/56-dwc_otg_00:00:312
2836299325,5cyclictest8050irq/56-dwc_otg_21:20:292
2836299321,16cyclictest8050irq/56-dwc_otg_21:15:302
2836199322,6cyclictest7850irq/56-dwc_otg19:15:191
2836199322,6cyclictest7850irq/56-dwc_otg19:15:191
2836199321,11cyclictest17292-21taskset20:09:101
2836199321,11cyclictest17292-21taskset20:09:101
2836399317,10cyclictest0-21swapper/323:35:273
2836399316,15cyclictest0-21swapper/322:35:093
2836399315,13cyclictest8050irq/56-dwc_otg_20:35:323
2836399312,6cyclictest32206-21cpuspeed19:20:173
2836399312,6cyclictest2617-21unixbench_singl20:55:323
2836399312,20cyclictest0-21swapper/319:18:033
2836399312,20cyclictest0-21swapper/319:18:033
2836399311,7cyclictest7850irq/56-dwc_otg19:45:233
28363993111,10cyclictest22672-21kworker/3:221:40:033
2836299319,12cyclictest0-21swapper/221:45:162
2836299314,6cyclictest8050irq/56-dwc_otg_19:30:032
2836299311,5cyclictest0-21swapper/220:35:322
2836299311,2cyclictest0-21swapper/221:00:152
2836399306,6cyclictest39-21ksoftirqd/323:40:233
2836399302,4cyclictest25901-21unixbench-2d22:10:323
33572290,2sleep30-21swapper/321:00:163
2836399294,5cyclictest12469-21fschecks_count19:55:203
2836399292,9cyclictest22378-21/usr/sbin/munin23:25:213
2836399292,13cyclictest8050irq/56-dwc_otg_20:30:163
2836399291,18cyclictest7049-21uniq19:40:183
2836399285,13cyclictest8050irq/56-dwc_otg_20:45:293
2836399283,5cyclictest7950irq/56-dwc_otg_20:05:193
2836399283,5cyclictest7950irq/56-dwc_otg_20:05:193
2836399282,4cyclictest8050irq/56-dwc_otg_20:10:233
2836399282,3cyclictest7850irq/56-dwc_otg20:00:143
2836399281,5cyclictest7850irq/56-dwc_otg19:30:473
2836399271,3cyclictest8050irq/56-dwc_otg_19:35:163
2836399266,5cyclictest8050irq/56-dwc_otg_19:50:223
2836399262,15cyclictest0-21swapper/319:10:173
277482253,15sleep20-21swapper/219:05:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional