You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 12:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Mar 31, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
216921600,5sleep21355299cyclictest22:37:252
226052900,2sleep10-21swapper/121:05:121
57622880,2sleep20-21swapper/221:25:342
13549998049,12cyclictest660-21ssh23:15:240
11772790,2sleep00-21swapper/022:35:420
1355399757,31cyclictest0-21swapper/320:00:253
306792740,2sleep31355399cyclictest21:16:213
306792740,2sleep31355399cyclictest21:16:213
275532730,2sleep20-21swapper/200:25:562
13553997211,9cyclictest7850irq/56-dwc_otg_19:35:233
13553997211,9cyclictest7850irq/56-dwc_otg_19:35:233
173972710,2sleep01354999cyclictest20:50:260
153622710,2sleep10-21swapper/120:45:121
1354999705,55cyclictest0-21swapper/023:20:220
1354999705,55cyclictest0-21swapper/023:20:220
13549997016,36cyclictest9-21ksoftirqd/022:25:270
1354999693,61cyclictest0-21swapper/019:10:140
1354999692,57cyclictest30936-21fschecks_count21:55:310
13549996819,35cyclictest14904-21grep20:40:410
13549996810,48cyclictest9-21ksoftirqd/000:25:120
185192670,2sleep10-21swapper/122:56:451
204302660,1sleep10-21swapper/120:56:391
13549996614,32cyclictest0-21swapper/000:30:120
50682650,2sleep20-21swapper/200:39:042
46492650,1sleep30-21swapper/321:25:073
1354999651,42cyclictest0-21swapper/021:40:260
1354999651,42cyclictest0-21swapper/021:40:260
1354999642,49cyclictest25994-21cron21:50:110
1354999631,53cyclictest3118-21grep00:35:420
1354999624,36cyclictest0-21swapper/023:45:250
1354999622,47cyclictest0-21swapper/021:50:340
1354999622,46cyclictest0-21swapper/022:10:430
1354999621,5cyclictest20393-21ssh23:00:110
1354999621,48cyclictest21952-21unixbench_singl19:30:410
13549996139,15cyclictest101ktimersoftd/021:00:340
13549996043,4cyclictest0-21swapper/021:35:260
1354999602,47cyclictest0-21swapper/000:30:410
1354999602,46cyclictest0-21swapper/000:00:260
1354999602,46cyclictest0-21swapper/000:00:260
1354999602,3cyclictest0-21swapper/022:35:120
1354999601,9cyclictest13374-21fschecks_count23:30:300
1354999601,48cyclictest0-21swapper/023:40:380
1354999594,44cyclictest9-21ksoftirqd/023:10:310
1354999592,6cyclictest17800-21sort23:35:330
1354999591,45cyclictest908-21runrttasks23:06:560
1354999589,38cyclictest30947-21unixbench_singl23:50:400
1354999587,40cyclictest13684-21latency_hist20:40:130
1354999587,40cyclictest13684-21latency_hist20:40:130
1354999585,41cyclictest9-21ksoftirqd/019:20:340
1354999585,38cyclictest8643-21grep20:25:310
13549995835,14cyclictest2603-21grep23:55:440
1354999582,43cyclictest0-21swapper/021:30:230
13549995822,19cyclictest10310-21ssh22:09:340
132542583,12sleep20-21swapper/219:09:542
121192583,14sleep011728-21/usr/sbin/munin19:05:310
282522570,3sleep128255-21cpuspeed22:30:281
1354999575,39cyclictest4676-21ssh22:40:330
1354999572,39cyclictest0-21swapper/000:06:460
13552995612,14cyclictest31-21ksoftirqd/221:10:122
1354999565,37cyclictest9-21ksoftirqd/023:26:350
1354999565,26cyclictest0-21swapper/020:55:250
13549995641,5cyclictest7850irq/56-dwc_otg_22:20:250
1354999563,38cyclictest22885-21grep21:05:280
1354999562,5cyclictest8499-21aten_rbpower_vo22:45:260
1354999562,40cyclictest0-21swapper/000:15:130
1354999562,39cyclictest0-21swapper/022:00:410
1354999561,45cyclictest15583-21grep22:15:300
1355399557,31cyclictest7850irq/56-dwc_otg_19:10:153
1354999557,36cyclictest0-21swapper/021:10:260
13549995532,7cyclictest28277-21fschecks_count19:50:280
1354999551,42cyclictest32604-21switchtime20:00:370
1354999549,32cyclictest0-21swapper/021:19:460
1354999549,32cyclictest0-21swapper/021:19:460
1354999549,27cyclictest0-21swapper/022:50:260
1354999547,25cyclictest11-21rcu_preempt20:30:230
1354999541,44cyclictest0-21swapper/021:20:320
1354999541,41cyclictest0-21swapper/020:45:360
1355099533,10cyclictest7950irq/56-dwc_otg_00:00:241
1355099533,10cyclictest7950irq/56-dwc_otg_00:00:241
13549995333,7cyclictest7850irq/56-dwc_otg_19:41:220
13549995333,7cyclictest0-21swapper/021:25:230
1354999532,41cyclictest0-21swapper/020:20:370
13549995323,17cyclictest22865-21cut19:35:290
13549995323,17cyclictest22865-21cut19:35:290
1354999530,45cyclictest18641-21kworker/0:023:00:510
1355399524,6cyclictest0-21swapper/322:10:273
1354999521,4cyclictest1179-21aten_rbpower_po20:05:260
1354999521,39cyclictest26970-21processes19:45:380
176802510,2sleep20-21swapper/222:17:322
1355399512,35cyclictest0-21swapper/323:05:263
1355099491,10cyclictest7950irq/56-dwc_otg_20:10:231
13549994914,17cyclictest7750irq/56-dwc_otg00:15:260
1355099482,9cyclictest7950irq/56-dwc_otg_19:10:141
13550994825,14cyclictest18140-21fschecks_count21:40:301
13550994825,14cyclictest18140-21fschecks_count21:40:301
1355099481,9cyclictest7950irq/56-dwc_otg_23:20:211
1355099481,9cyclictest7950irq/56-dwc_otg_23:20:211
1354999482,8cyclictest20154-21grep19:25:410
1355399477,25cyclictest0-21swapper/321:05:273
1355099465,7cyclictest7850irq/56-dwc_otg_00:25:241
13550994624,5cyclictest23-21ksoftirqd/123:15:401
13550994623,5cyclictest22380-21idleruntime-cro19:35:121
13549994623,7cyclictest7950irq/56-dwc_otg_20:15:120
1355399457,27cyclictest6706-21rm22:05:193
1354999456,25cyclictest9-21ksoftirqd/019:55:370
261082440,4sleep326094-21apt-get00:25:243
1355399447,25cyclictest0-21swapper/321:00:253
1355399447,24cyclictest0-21swapper/320:30:253
1355399447,23cyclictest0-21swapper/322:25:263
13550994411,23cyclictest21-21rcuc/120:30:381
1354999442,4cyclictest13-21rcuc/019:15:350
1355399432,26cyclictest0-21swapper/320:25:273
1354999432,7cyclictest0-21swapper/020:15:410
1355099429,12cyclictest0-21swapper/122:35:261
13550994217,10cyclictest2506-21aten_rbpower_po22:00:261
1355399416,22cyclictest0-21swapper/300:30:243
1355399411,28cyclictest7950irq/56-dwc_otg_23:00:313
13552994120,12cyclictest9015-21sh23:25:272
1355299411,6cyclictest0-21swapper/223:52:502
13552994114,15cyclictest28951-21ssh22:30:382
1355099414,11cyclictest5390-21ssh22:40:411
13550994118,13cyclictest28048-21aten_rbpower_cu19:50:251
310712400,2sleep10-21swapper/121:17:041
310712400,2sleep10-21swapper/121:17:041
224982400,1sleep20-21swapper/223:01:292
1355399407,8cyclictest0-21swapper/319:45:263
1355299401,9cyclictest7950irq/56-dwc_otg_19:55:262
1355299401,6cyclictest7950irq/56-dwc_otg_19:50:222
13550994019,11cyclictest28079-21sh21:14:171
13550994019,11cyclictest21-21rcuc/122:53:211
13550994018,11cyclictest23182-21sh23:02:441
13550994013,5cyclictest23-21ksoftirqd/122:10:311
13550994010,9cyclictest7950irq/56-dwc_otg_19:15:251
38952390,2sleep30-21swapper/323:19:093
1355399398,16cyclictest0-21swapper/322:22:143
13553993919,5cyclictest908-21runrttasks21:50:243
1355399391,7cyclictest7950irq/56-dwc_otg_23:45:243
1355299397,4cyclictest0-21swapper/223:44:252
1355099398,6cyclictest0-21swapper/121:20:291
1355099394,5cyclictest18242-21fw_packets00:15:301
1355099394,3cyclictest0-21swapper/121:32:161
1355099393,6cyclictest7950irq/56-dwc_otg_23:35:261
1355099393,6cyclictest0-21swapper/123:05:241
13550993919,10cyclictest18050-21ntp_states20:50:361
13550993918,12cyclictest27168-21sh21:50:361
168392380,1sleep10-21swapper/100:13:591
1355299384,4cyclictest0-21swapper/220:25:312
1355299381,24cyclictest0-21swapper/219:45:262
1355099387,22cyclictest1724-21perl23:55:331
1355099383,5cyclictest31468-21latency_hist20:00:121
13550993821,5cyclictest908-21runrttasks22:20:441
13550993812,4cyclictest23-21ksoftirqd/100:25:121
1355099381,13cyclictest25547-21ssh23:45:251
81442370,4sleep21355299cyclictest00:03:092
81442370,4sleep21355299cyclictest00:03:092
1355399371,4cyclictest7850irq/56-dwc_otg_22:00:213
1355399371,23cyclictest19762-21ssh23:37:583
1355299374,15cyclictest0-21swapper/223:10:262
1355299372,23cyclictest15902-21ls19:15:322
13552993717,8cyclictest7050-21/usr/sbin/munin22:05:302
1355299371,5cyclictest1176-21aten_rbpower_po20:05:262
13552993712,16cyclictest25518-21/usr/sbin/munin23:45:282
1355099377,4cyclictest7950irq/56-dwc_otg_19:45:411
1355099377,26cyclictest0-21swapper/120:15:261
1355099376,4cyclictest0-21swapper/121:49:031
1355099375,24cyclictest0-21swapper/119:38:121
1355099375,24cyclictest0-21swapper/119:38:121
1355099372,18cyclictest6961-21ssh22:05:281
13550993717,4cyclictest13327-21/usr/sbin/munin23:30:291
1355099371,27cyclictest3063-21timerwakeupswit00:35:421
13550993711,3cyclictest0-21swapper/122:25:161
13550993711,23cyclictest0-21swapper/120:35:261
13550993711,23cyclictest0-21swapper/120:35:261
13550993710,3cyclictest0-21swapper/122:17:271
13550993710,17cyclictest0-21swapper/123:14:591
1355399365,19cyclictest5070-21ssh21:25:253
1355399364,16cyclictest7950irq/56-dwc_otg_20:15:373
13553993626,8cyclictest13548-21cyclictest23:25:253
13553993617,5cyclictest15617-21sh00:11:443
13553993615,9cyclictest24338-21diskmemload22:57:123
1355399361,23cyclictest24924-21latency19:40:323
1355299366,18cyclictest0-21swapper/222:27:492
1355299365,4cyclictest7950irq/56-dwc_otg_20:20:392
1355299362,5cyclictest7321-21ssh23:22:532
1355299362,5cyclictest7321-21ssh23:22:532
1355299362,5cyclictest21913-21grep19:30:412
13552993623,4cyclictest14040-21fschecks_count00:10:302
13552993618,8cyclictest27409-21timerandwakeup21:50:392
1355299361,7cyclictest13589-21ls00:10:132
1355299361,4cyclictest0-21swapper/221:15:362
1355299361,4cyclictest0-21swapper/221:15:362
1355299361,3cyclictest0-21swapper/221:30:312
13552993612,17cyclictest0-21swapper/221:59:302
13552993611,5cyclictest14010-21smartpower23:30:382
13552993610,14cyclictest31-21ksoftirqd/223:35:272
1355099367,4cyclictest0-21swapper/123:54:201
1355099367,4cyclictest0-21swapper/123:28:091
13550993615,4cyclictest0-21swapper/121:25:231
13550993610,5cyclictest7850irq/56-dwc_otg_21:05:311
1355399359,16cyclictest7950irq/56-dwc_otg_20:10:123
1355399356,12cyclictest21061-21grep19:30:303
13553993516,4cyclictest7750irq/56-dwc_otg00:15:413
13553993513,5cyclictest39-21ksoftirqd/323:35:123
13553993513,13cyclictest0-21swapper/322:40:263
1355299358,13cyclictest31-21ksoftirqd/222:10:392
1355299356,19cyclictest0-21swapper/222:48:222
1355299351,7cyclictest17860-21tr19:20:342
1355299351,7cyclictest13050-21ssh22:50:332
13552993517,11cyclictest3882-21ssh23:19:082
1355299351,4cyclictest22352-21ssh21:45:292
1355299351,20cyclictest29-21rcuc/220:35:582
1355299351,20cyclictest29-21rcuc/220:35:582
13552993510,15cyclictest0-21swapper/222:21:532
1355099359,15cyclictest30745-21aten_rbpower_vo21:55:281
1355099357,25cyclictest0-21swapper/119:25:261
1355099356,5cyclictest0-21swapper/119:23:551
1355099355,4cyclictest0-21swapper/100:07:401
13550993512,5cyclictest8550-21cpu22:45:271
13553993414,5cyclictest4761-21/usr/sbin/munin23:20:323
13553993414,5cyclictest4761-21/usr/sbin/munin23:20:323
1355299343,22cyclictest0-21swapper/219:45:112
1355299342,7cyclictest0-21swapper/220:52:462
13552993417,10cyclictest31-21ksoftirqd/220:30:302
1355299341,6cyclictest13914-21grep19:10:292
13552993413,5cyclictest14150-21sed21:35:332
1355299341,26cyclictest0-21swapper/223:58:202
13552993412,5cyclictest3590-21taskset20:10:352
1355299341,20cyclictest6416-21latency_hist22:05:132
1355099345,24cyclictest0-21swapper/120:20:251
1355099344,4cyclictest7850irq/56-dwc_otg_21:35:341
1355099342,25cyclictest7950irq/56-dwc_otg_23:40:231
13550993414,5cyclictest1165-21/usr/sbin/munin20:05:261
13550993410,9cyclictest7850irq/56-dwc_otg_20:25:251
1355399336,4cyclictest7750irq/56-dwc_otg23:50:343
13553993316,5cyclictest28637-21modprobe21:15:123
1355399331,28cyclictest20744-21ls19:30:133
1355399331,22cyclictest18037-21ls21:40:273
1355399331,22cyclictest18037-21ls21:40:273
1355399331,19cyclictest32759-21ssh23:15:123
1355399331,19cyclictest32068-21ssh22:35:123
1355299338,4cyclictest0-21swapper/200:30:262
1355299337,23cyclictest0-21swapper/220:15:262
1355299337,17cyclictest0-21swapper/222:40:412
1355299336,10cyclictest0-21swapper/219:25:212
1355299333,3cyclictest0-21swapper/221:10:262
1355299332,7cyclictest0-21swapper/222:55:422
1355299331,6cyclictest21292-21fschecks_time21:00:302
1355299331,20cyclictest16123-21ls20:45:322
1355099335,7cyclictest7850irq/56-dwc_otg_20:00:401
1355399329,12cyclictest7950irq/56-dwc_otg_00:06:463
1355399321,21cyclictest8673-21uniq22:45:293
1355399321,17cyclictest21439-21aten_rbpower_vo23:40:273
1355399321,15cyclictest7750irq/56-dwc_otg20:10:313
1355399321,14cyclictest7750irq/56-dwc_otg22:18:003
1355399321,14cyclictest7750irq/56-dwc_otg00:23:483
13553993210,5cyclictest7750irq/56-dwc_otg20:55:263
1355299329,5cyclictest18621-21ntp_states21:40:362
1355299329,5cyclictest18621-21ntp_states21:40:362
1355299328,5cyclictest18209-21fschecks_time00:15:302
1355299328,11cyclictest0-21swapper/223:05:402
1355299325,18cyclictest29-21rcuc/200:22:302
1355299322,4cyclictest0-21swapper/220:55:312
1355299321,5cyclictest0-21swapper/220:00:362
1355099325,3cyclictest0-21swapper/100:32:161
1355399317,14cyclictest9440-21wget21:30:263
1355399317,14cyclictest0-21swapper/321:45:263
1355399317,11cyclictest7850irq/56-dwc_otg_22:50:283
1355399316,6cyclictest7750irq/56-dwc_otg20:45:303
1355399316,14cyclictest0-21swapper/321:55:273
1355399314,6cyclictest13944-21ssh21:35:303
13553993120,8cyclictest0-21swapper/319:15:343
1355299311,4cyclictest1295-21/usr/sbin/munin21:20:322
13552993112,6cyclictest13962-21/usr/sbin/munin20:40:292
1355099317,14cyclictest0-21swapper/119:40:251
1355099316,4cyclictest7850irq/56-dwc_otg_20:45:381
1355399307,14cyclictest0-21swapper/323:55:293
1355399303,6cyclictest1042-21ls22:35:413
13553993019,7cyclictest1991-21aten_rbpower_en00:35:283
1355399301,25cyclictest28185-21ls19:50:263
1355399301,17cyclictest0-21swapper/320:25:133
1355399301,13cyclictest7750irq/56-dwc_otg20:50:293
1355399294,6cyclictest0-21swapper/320:35:253
1355399294,6cyclictest0-21swapper/320:35:253
1355399291,18cyclictest6428-21switchtime00:00:383
1355399291,18cyclictest6428-21switchtime00:00:383
1355299291,3cyclictest0-21swapper/219:35:272
1355299291,3cyclictest0-21swapper/219:35:272
127222293,14sleep10-21swapper/119:05:401
1355399274,4cyclictest13734-21/usr/sbin/munin20:40:283
1355399272,12cyclictest20406-21kworker/3:119:55:303
117582277,12sleep30-21swapper/319:05:263
13553992612,4cyclictest18875-21sed19:25:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional