You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 19:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 01, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4114991322,54cyclictest7950irq/56-dwc_otg_07:10:160
3008321020,4sleep330074-21ssh11:20:393
411499935,74cyclictest3914-21cron09:35:130
411499935,74cyclictest3914-21cron09:35:130
411499911,5cyclictest6551-21sed11:30:430
411499881,7cyclictest2873-21fschecks_count12:05:300
411499881,70cyclictest928-21cut08:30:280
217732880,2sleep20-21swapper/209:17:062
411499872,73cyclictest4116-21latency_hist07:10:150
411499841,7cyclictest2250-21apt12:05:120
411499841,5cyclictest21819-21unixbench-2d11:10:400
82892830,5sleep2412699cyclictest09:40:132
411499812,5cyclictest0-21swapper/010:36:400
411499802,5cyclictest0-21swapper/012:30:400
411499802,5cyclictest0-21swapper/009:54:400
411499802,5cyclictest0-21swapper/009:54:400
411499801,5cyclictest0-21swapper/010:00:380
105632800,1sleep20-21swapper/212:14:532
411499791,6cyclictest18554-21sed11:45:340
246252790,4sleep124624-21memory12:30:341
413299784,12cyclictest0-21swapper/307:10:153
411499781,6cyclictest15006-21fschecks_time10:25:320
4114997814,52cyclictest19172-21sh12:25:010
411499761,6cyclictest19187-21ssh10:30:290
411499761,47cyclictest31240-21latency_hist08:25:130
227192760,4sleep122718-21unixbench-2d09:55:421
225912760,5sleep1411899cyclictest11:11:471
4126997523,30cyclictest0-21swapper/211:30:272
411499751,6cyclictest0-21swapper/007:33:500
411499751,4cyclictest19613-21/usr/sbin/munin12:25:240
413299743,9cyclictest4127-21apt-config07:10:153
411499748,55cyclictest16356-21ssh09:49:150
411499742,5cyclictest0-21swapper/009:35:350
411499741,5cyclictest18142-21ssh11:06:260
4126997341,26cyclictest4124-21chrt07:10:152
411499732,5cyclictest0-21swapper/011:45:140
123482730,1sleep30-21swapper/311:38:223
411499729,53cyclictest11264-21ssh10:59:190
411499727,52cyclictest0-21swapper/009:40:290
4114997228,23cyclictest7850irq/56-dwc_otg_10:50:410
411499721,60cyclictest9929-21ssh12:13:460
80772710,5sleep1411899cyclictest10:55:281
411499711,9cyclictest3759-21sed08:35:410
411499711,8cyclictest0-21swapper/010:08:410
411499711,5cyclictest15450-21/usr/sbin/munin09:10:270
411499711,59cyclictest23775-21sh11:51:020
4114997111,49cyclictest474-21ssh11:25:000
4114997110,46cyclictest0-21swapper/009:18:170
277012700,1sleep20-21swapper/212:34:542
203932700,5sleep220394-21ntpq12:25:362
118552700,2sleep0101ktimersoftd/011:37:300
411499691,5cyclictest0-21swapper/010:11:330
411499691,48cyclictest13478-21/usr/sbin/munin09:05:230
309262690,1sleep10-21swapper/112:00:261
411499682,7cyclictest0-21swapper/007:36:190
411499682,6cyclictest0-21swapper/011:02:060
411499681,5cyclictest0-21swapper/009:57:530
411499681,52cyclictest29395-21ssh11:58:260
411499679,42cyclictest28208-21/usr/sbin/munin12:35:300
411499672,8cyclictest0-21swapper/008:19:580
4114996713,42cyclictest12931-21ssh10:22:490
4126996635,26cyclictest4175-21sh07:10:152
411499662,4cyclictest0-21swapper/009:21:180
411499661,54cyclictest0-21swapper/010:45:250
4114996611,43cyclictest908-21runrttasks10:45:070
411499659,43cyclictest183452sleep007:49:330
411499652,5cyclictest0-21swapper/007:51:440
411499652,4cyclictest0-21swapper/012:16:500
411499651,54cyclictest30227-21ssh09:26:570
4126996422,30cyclictest31359-21ssh10:45:042
411499647,45cyclictest929-21seq11:25:230
411499647,45cyclictest929-21seq11:25:230
411499642,6cyclictest0-21swapper/010:17:310
411499641,6cyclictest26607-21ssh11:16:220
411499641,40cyclictest24156-21/usr/sbin/munin08:05:360
411899635,3cyclictest9529-21ssh11:35:271
411499631,50cyclictest27513-21cron08:15:130
4114996310,42cyclictest7539-21runrttasks08:46:460
4126996226,11cyclictest7850irq/56-dwc_otg_10:50:412
4114996211,37cyclictest0-21swapper/008:25:330
4126996130,15cyclictest15108-21/usr/sbin/munin10:25:332
4126996129,12cyclictest31701-21ssh10:06:192
4126996127,18cyclictest2055-21ssh10:48:062
4126996127,11cyclictest19133-21ssh09:51:412
4126996127,11cyclictest19133-21ssh09:51:412
4126996125,20cyclictest26588-21ssh11:16:202
411499619,41cyclictest0-21swapper/007:26:160
411499611,43cyclictest0-21swapper/007:40:340
4132996020,38cyclictest26146-21fschecks_time10:00:313
4132996018,39cyclictest10839-21latency_hist12:15:133
4114996024,20cyclictest0-21swapper/008:55:270
4114996020,26cyclictest908-21runrttasks07:20:400
411499601,48cyclictest21750-21chrt07:56:540
413299596,9cyclictest7750irq/56-dwc_otg11:56:303
4132995918,4cyclictest24870-21/usr/sbin/munin11:15:413
4132995915,4cyclictest0-21swapper/309:40:283
4126995932,11cyclictest14526-21ssh12:19:212
413299583,3cyclictest7950irq/56-dwc_otg_09:52:273
413299583,3cyclictest7950irq/56-dwc_otg_09:52:273
413299581,5cyclictest31895-21ssh10:45:293
4126995830,9cyclictest3564-21ls08:35:392
4126995821,25cyclictest3768-21ssh11:28:152
4126995821,25cyclictest3768-21ssh11:28:152
411499581,47cyclictest0-21swapper/008:02:210
4126995731,18cyclictest2376-21ssh09:32:322
4126995731,18cyclictest2376-21ssh09:32:322
4126995727,21cyclictest29563-21ssh10:05:012
4126995726,16cyclictest30223-21ssh09:26:572
411499571,46cyclictest908-21runrttasks07:16:080
4132995619,35cyclictest6389-21cat11:30:413
4126995625,12cyclictest14884-21ssh09:46:422
4126995625,11cyclictest22147-21rm11:11:012
4126995623,22cyclictest11871-21ssh09:44:032
4118995628,24cyclictest4104-21cyclictest07:10:151
411499562,5cyclictest0-21swapper/008:52:010
261292560,2sleep10-21swapper/111:55:111
413299556,8cyclictest0-21swapper/310:23:513
413299556,4cyclictest7950irq/56-dwc_otg_11:25:263
413299556,4cyclictest7950irq/56-dwc_otg_11:25:263
413299556,3cyclictest0-21swapper/312:36:133
413299553,11cyclictest908-21runrttasks09:16:003
413299551,2cyclictest0-21swapper/311:05:283
4126995527,12cyclictest28598-21ssh11:57:012
4126995526,7cyclictest29345-21apt-get08:20:172
4126995526,19cyclictest849-21ssh12:02:492
4126995525,12cyclictest31238-21ssh11:22:002
4126995525,11cyclictest23856-21ssh09:57:282
4126995523,11cyclictest18548-21ssh11:45:342
4126995521,18cyclictest7850irq/56-dwc_otg_10:20:012
4126995518,27cyclictest15054-21diskmemload10:56:252
413299542,4cyclictest16847-21ssh09:10:543
4132995417,4cyclictest25438-21cron10:00:123
4132995414,5cyclictest23542-21diskstats10:35:303
413299541,13cyclictest7950irq/56-dwc_otg_10:56:323
4126995427,18cyclictest25223-21ssh10:36:462
4126995423,19cyclictest18975-21ssh11:07:542
282342540,3sleep1221ktimersoftd/110:40:351
413299534,3cyclictest7950irq/56-dwc_otg_08:20:273
413299533,9cyclictest7950irq/56-dwc_otg_10:54:003
413299531,6cyclictest7950irq/56-dwc_otg_09:05:273
413299531,5cyclictest922-21ssh09:30:383
413299531,5cyclictest922-21ssh09:30:383
413299531,4cyclictest0-21swapper/312:34:123
413299531,3cyclictest7950irq/56-dwc_otg_12:25:313
4126995325,17cyclictest12173-21ssh10:21:302
4126995321,18cyclictest3535-21sh12:05:392
411499535,40cyclictest917-21snmpd08:41:080
411499531,44cyclictest0-21swapper/009:02:200
411499531,44cyclictest0-21swapper/009:02:200
413299522,6cyclictest7950irq/56-dwc_otg_11:45:293
413299522,3cyclictest7850irq/56-dwc_otg_10:25:413
4132995216,4cyclictest15011-21sh09:46:553
413299521,5cyclictest7950irq/56-dwc_otg_08:25:393
413299521,4cyclictest7950irq/56-dwc_otg_12:05:433
413299521,4cyclictest7850irq/56-dwc_otg_11:54:003
413299521,4cyclictest27539-21apt-get10:40:263
4126995224,13cyclictest31494-21aten_rbpower_cu08:25:272
4126995224,13cyclictest15642-21awk09:10:282
4126995223,17cyclictest13156-21ssh11:39:482
4126995220,12cyclictest17673-21ssh11:45:132
4126995214,27cyclictest0-21swapper/207:18:592
411899526,17cyclictest0-21swapper/111:55:281
413299514,10cyclictest7950irq/56-dwc_otg_10:10:253
413299513,4cyclictest0-21swapper/310:10:093
4132995120,28cyclictest6923-21open_files10:15:363
413299511,7cyclictest7950irq/56-dwc_otg_07:30:253
413299511,4cyclictest7850irq/56-dwc_otg_09:29:503
413299511,3cyclictest0-21swapper/310:30:563
4126995127,11cyclictest25548-21ssh11:54:092
4126995126,13cyclictest20647-21ssh10:31:242
4126995126,12cyclictest14224-21ssh11:02:022
4126995123,10cyclictest7850irq/56-dwc_otg_09:05:232
4126995120,13cyclictest17511-21ssh12:22:072
413299506,3cyclictest0-21swapper/311:40:493
413299502,3cyclictest7950irq/56-dwc_otg_11:02:153
413299502,19cyclictest23913-21ssh09:20:243
413299501,5cyclictest0-21swapper/308:32:153
413299501,4cyclictest305-1kworker/u9:212:02:213
413299501,4cyclictest0-21swapper/312:20:323
413299501,3cyclictest0-21swapper/309:36:043
4126995027,12cyclictest29836-21ssh12:36:102
4126995021,13cyclictest13226-21/usr/sbin/munin07:35:332
4126995021,10cyclictest25538-21ssh09:21:122
4118995011,21cyclictest19603-21rm12:25:231
413299491,5cyclictest908-21runrttasks07:23:473
413299491,4cyclictest13098-21runrttasks12:16:523
4126994923,13cyclictest13216-21chrt09:05:022
4126994923,13cyclictest13216-21chrt09:05:022
4126994913,27cyclictest0-21swapper/208:30:272
4118994919,25cyclictest4159-21find07:10:151
413299481,4cyclictest0-21swapper/311:12:043
413299481,3cyclictest7950irq/56-dwc_otg_08:44:113
4126994825,12cyclictest3670-21ssh10:11:342
4126994825,11cyclictest20132-21sshd07:54:202
4126994818,11cyclictest7750-21seq08:49:232
411899489,20cyclictest26950-21sed10:00:411
411899485,5cyclictest23689-21latency_hist09:20:131
4118994814,16cyclictest221ktimersoftd/108:20:161
4126994722,13cyclictest27542-21idleruntime-cro08:15:132
4126994720,10cyclictest0-21swapper/208:07:282
4126994716,12cyclictest0-21swapper/207:41:512
4118994714,22cyclictest530-21munin-node08:25:241
411899469,18cyclictest3730-21/usr/sbin/munin08:35:411
411899468,24cyclictest10331-21sh10:20:191
411899468,13cyclictest5026-21ssh09:35:351
411899467,24cyclictest0-21swapper/108:00:261
4118994615,11cyclictest221ktimersoftd/110:08:411
4118994611,20cyclictest3117-21/usr/sbin/munin12:05:331
61532450,3sleep1221ktimersoftd/111:30:391
4132994516,4cyclictest22023-21sh08:00:133
4126994531,11cyclictest17397-21ls07:45:322
4126994523,13cyclictest5689-21runrttasks08:41:052
4126994519,12cyclictest9363-21chrt08:52:022
4126994518,16cyclictest23762-21runrttasks08:04:352
4126994518,15cyclictest217772sleep207:57:162
411899456,25cyclictest0-21swapper/111:15:261
4118994518,9cyclictest835-21apt-get08:30:251
411899451,30cyclictest0-21swapper/109:10:271
4118994510,12cyclictest32309-21latency_hist09:30:141
4126994424,10cyclictest29317-21taskset08:20:032
411899448,24cyclictest0-21swapper/108:05:201
411899447,23cyclictest0-21swapper/107:45:261
411899447,12cyclictest16358-21sh11:05:211
411899446,26cyclictest0-21swapper/110:35:181
4126994331,9cyclictest10542-21ls07:25:392
4126994329,11cyclictest9015-21/usr/sbin/munin07:20:422
4126994316,9cyclictest0-21swapper/207:30:402
411899439,16cyclictest880-21ssh11:25:181
411899439,16cyclictest880-21ssh11:25:181
411899436,24cyclictest18925-21ssh10:30:251
4118994312,9cyclictest0-21swapper/110:10:391
413299425,8cyclictest7950irq/56-dwc_otg_07:41:353
4126994219,11cyclictest9864-21/usr/sbin/munin08:55:292
411899427,23cyclictest0-21swapper/108:40:251
411899424,25cyclictest0-21swapper/111:45:351
411899423,24cyclictest17490-21ssh10:28:211
4118994229,4cyclictest7750irq/56-dwc_otg09:55:051
4118994229,4cyclictest7750irq/56-dwc_otg09:55:051
4118994222,10cyclictest7750irq/56-dwc_otg07:20:271
4118994210,17cyclictest9998-21ssh09:40:471
411899416,26cyclictest11130-21ssh12:15:271
411899416,19cyclictest0-21swapper/110:50:371
4118994111,15cyclictest17500-21ssh11:44:571
150922410,2sleep1221ktimersoftd/111:03:341
411899409,12cyclictest8265-21ssh10:17:081
411899406,22cyclictest0-21swapper/108:50:261
411899406,19cyclictest14908-21ssh09:46:441
411899403,23cyclictest0-21swapper/109:21:341
4118994018,9cyclictest8704-21ssh12:11:371
4118994017,9cyclictest149-21systemd-journal07:33:371
4118994011,15cyclictest0-21swapper/107:54:341
411899398,22cyclictest7850irq/56-dwc_otg_11:24:461
411899397,23cyclictest0-21swapper/107:35:281
4118993910,19cyclictest13219-21taskset09:05:041
4118993910,19cyclictest13219-21taskset09:05:041
411899389,8cyclictest19173-21ssh12:25:011
411899387,14cyclictest221ktimersoftd/110:45:231
4118993818,8cyclictest908-21runrttasks09:00:051
413299371,13cyclictest0-21swapper/308:14:583
411899376,22cyclictest0-21swapper/112:35:261
411899375,19cyclictest7532-21sshd08:46:421
4118993711,11cyclictest7850irq/56-dwc_otg_09:32:551
4118993711,11cyclictest7850irq/56-dwc_otg_09:32:551
411899368,11cyclictest27508-21chrt08:15:101
411899366,23cyclictest0-21swapper/108:15:271
411899364,19cyclictest13483-21awk09:05:231
411899357,22cyclictest0-21swapper/107:15:261
411899346,4cyclictest23-21ksoftirqd/107:25:311
411899344,9cyclictest7950irq/56-dwc_otg_07:55:241
413299331,21cyclictest24489-21cut08:05:323
411899331,4cyclictest0-21swapper/107:40:261
413299326,13cyclictest7950irq/56-dwc_otg_08:50:333
413299325,11cyclictest7950irq/56-dwc_otg_07:30:123
413299303,5cyclictest7850irq/56-dwc_otg_08:00:243
413299295,13cyclictest7950irq/56-dwc_otg_07:50:383
413299293,13cyclictest7950irq/56-dwc_otg_08:37:163
413299291,6cyclictest7950irq/56-dwc_otg_09:05:133
413299291,6cyclictest7950irq/56-dwc_otg_09:05:133
4132992915,5cyclictest9994-21cpu08:55:283
413299281,5cyclictest7950irq/56-dwc_otg_07:35:363
413299271,4cyclictest7750irq/56-dwc_otg08:15:313
413299271,13cyclictest7950irq/56-dwc_otg_08:48:083
413299266,3cyclictest7950irq/56-dwc_otg_07:15:253
413299261,3cyclictest7950irq/56-dwc_otg_07:45:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional