You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 18:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Sun Jan 11, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2949999902,80cyclictest27851-21unixbench_multi11:30:430
152522840,2sleep00-21swapper/009:30:170
29499998227,3cyclictest0-21swapper/010:05:450
29499998227,3cyclictest0-21swapper/010:05:450
29499998214,5cyclictest28157-21fschecks_count08:35:350
29499997714,4cyclictest25876-21latency_hist08:30:170
29499997320,13cyclictest15742-21latency_hist11:00:170
9462710,6sleep12950099cyclictest07:20:331
92582710,1sleep30-21swapper/310:40:403
298622710,2sleep20-21swapper/207:10:362
134722710,2sleep10-21swapper/107:55:311
35052700,2sleep30-21swapper/308:55:423
29499997017,11cyclictest0-21swapper/009:25:170
231052700,2sleep00-21swapper/011:20:180
29499996620,23cyclictest11922-21sh10:50:170
29499996620,23cyclictest11922-21sh10:50:170
29499996616,12cyclictest1125-21runrttasks08:51:140
29499996616,12cyclictest1125-21runrttasks08:51:140
12062640,2sleep10-21swapper/110:20:171
2949999615,9cyclictest0-21swapper/010:15:280
113332610,2sleep10-21swapper/110:45:451
113332610,2sleep10-21swapper/110:45:451
29501996022,13cyclictest18158-21irqstats11:05:372
29501996022,13cyclictest18158-21irqstats11:05:372
29501996021,18cyclictest0-21swapper/209:20:172
29501996018,23cyclictest15943-21aten_rbpower_en11:00:312
2949999601,32cyclictest0-21swapper/008:45:300
2949999596,25cyclictest0-21swapper/011:40:290
2949999596,25cyclictest0-21swapper/011:40:290
2949999592,29cyclictest0-21swapper/009:50:270
29501995824,17cyclictest24137-21latency_hist08:25:172
29501995819,16cyclictest3320-21sed11:55:172
2949999588,10cyclictest7750irq/56-dwc_otg12:00:280
29501995722,18cyclictest26916-21/usr/sbin/munin11:30:302
2949999572,33cyclictest8618-21aten_rbpower_en10:40:300
29501995625,11cyclictest26517-21apt-get10:00:282
29501995622,19cyclictest30515-21gzip08:40:422
29501995523,17cyclictest14334-21/usr/sbin/munin12:25:472
2949999555,25cyclictest0-21swapper/007:10:290
2949999553,6cyclictest0-21swapper/009:40:280
29499995512,22cyclictest9760-21aten_rbpower_cu07:45:300
29501995419,12cyclictest5748-21seq07:33:442
2949999544,34cyclictest21083-21awk11:15:160
2949999542,31cyclictest21707-21grep08:15:460
2949999541,50cyclictest3268-21grep07:25:420
29501995322,11cyclictest3462-21/usr/sbin/munin11:55:382
2949999539,26cyclictest30847-21grep08:40:450
29499995333,6cyclictest7950irq/56-dwc_otg_11:35:410
2949999532,32cyclictest1751-21tr11:50:310
29499995310,30cyclictest25035-21aten_rbpower_vo11:25:310
292052533,12sleep00-21swapper/007:09:580
2950299527,28cyclictest0-21swapper/312:20:273
29501995219,18cyclictest13867-21df09:25:302
29501995214,19cyclictest7850irq/56-dwc_otg_10:05:382
29501995214,19cyclictest7850irq/56-dwc_otg_10:05:382
29501995130,12cyclictest15472-21apt-get09:30:302
29501995114,18cyclictest7950irq/56-dwc_otg_07:35:362
2949999512,28cyclictest22431-21grep11:15:460
2949999512,27cyclictest23402-21sshd08:20:430
2950299507,24cyclictest0-21swapper/308:05:303
2950299507,24cyclictest0-21swapper/308:05:303
29502995013,19cyclictest13153-21apt-get12:25:283
29501995020,11cyclictest29998-21chrt11:38:472
2949999502,28cyclictest13738-21latency_hist10:55:160
2950299497,25cyclictest0-21swapper/311:35:303
2950299497,25cyclictest0-21swapper/307:45:303
29502994916,17cyclictest15506-21/usr/sbin/munin09:30:303
29501994915,20cyclictest26301-21/usr/sbin/munin08:30:342
2949999498,24cyclictest13-21rcuc/007:40:280
2949999492,23cyclictest0-21swapper/012:10:420
2949999491,29cyclictest5816-21switchtime10:30:420
2950299487,25cyclictest0-21swapper/307:55:293
29502994825,7cyclictest13664-21apt-get09:25:283
29502994817,11cyclictest18261-21memory11:05:393
29502994817,11cyclictest18261-21memory11:05:393
29502994816,12cyclictest30226-21/usr/sbin/munin08:40:373
29502994811,20cyclictest4874-21taskset11:58:263
2950199487,26cyclictest0-21swapper/212:20:272
29501994820,17cyclictest11810-21sshd10:49:022
29501994820,17cyclictest11810-21sshd10:49:022
29501994819,12cyclictest11194-21latency_hist12:20:182
29501994819,12cyclictest11194-21latency_hist12:20:182
29501994818,14cyclictest8616-21wget10:40:302
29501994813,9cyclictest7850irq/56-dwc_otg_12:09:242
29501994813,21cyclictest31642-21taskset11:41:372
29501994813,21cyclictest31642-21taskset11:41:372
29501994813,10cyclictest7850irq/56-dwc_otg_09:49:492
2950099481,5cyclictest7750irq/56-dwc_otg08:10:271
2949999483,28cyclictest0-21swapper/009:30:290
29499994828,3cyclictest0-21swapper/008:15:160
29499994825,4cyclictest0-21swapper/009:10:380
29499994824,5cyclictest7850irq/56-dwc_otg_07:55:170
29499994820,5cyclictest435-21cron12:10:170
29499994817,21cyclictest0-21swapper/012:32:090
2949999481,44cyclictest12185-21wc12:20:390
2949999481,18cyclictest31091-21timerandwakeup10:10:440
2950299477,26cyclictest0-21swapper/309:50:303
2950299477,25cyclictest0-21swapper/311:20:293
29502994720,15cyclictest28743-21if_eth010:05:353
29502994720,15cyclictest28743-21if_eth010:05:353
29502994714,18cyclictest29077-21/usr/sbin/munin08:35:463
2950199477,25cyclictest0-21swapper/209:20:312
29501994720,10cyclictest6834-21wget10:35:302
29501994718,11cyclictest7950irq/56-dwc_otg_08:35:292
29499994719,14cyclictest17439-21cron11:05:170
29499994718,17cyclictest0-21swapper/007:15:290
29499994718,17cyclictest0-21swapper/007:15:290
29499994717,21cyclictest0-21swapper/007:21:100
2950299468,10cyclictest7850irq/56-dwc_otg_10:00:373
2950199467,24cyclictest0-21swapper/207:40:302
29501994610,11cyclictest9550-21seq09:12:262
2950099468,28cyclictest0-21swapper/110:00:221
29500994612,22cyclictest0-21swapper/107:35:311
29499994632,3cyclictest7750irq/56-dwc_otg09:55:310
29499994619,4cyclictest4242-21unixbench_multi10:25:470
2950299457,23cyclictest0-21swapper/309:05:303
29502994522,8cyclictest15891-21apt-get11:00:293
2950199457,25cyclictest0-21swapper/208:55:302
2950199457,23cyclictest0-21swapper/211:45:222
29501994520,5cyclictest15201-21aten_rbpower_en08:00:312
29501994519,8cyclictest31511-21apt-get08:45:302
29501994515,14cyclictest6107-21/usr/sbin/munin12:00:432
29501994511,15cyclictest26177-21sh10:00:172
2950099457,23cyclictest0-21swapper/109:50:301
2950099452,7cyclictest7850irq/56-dwc_otg_11:35:281
29499994527,11cyclictest1125-21runrttasks11:05:360
29499994527,11cyclictest1125-21runrttasks11:05:360
2949999452,26cyclictest14821-21latency_hist12:30:170
29499994520,5cyclictest17515-21cut09:35:340
2949999451,24cyclictest5190-21ntp_states09:00:400
2950299448,24cyclictest0-21swapper/311:45:223
2950299447,23cyclictest0-21swapper/310:50:313
29502994419,5cyclictest4711-21idleruntime-cro10:30:173
2950199448,25cyclictest0-21swapper/207:20:292
29501994418,12cyclictest10193-21chrt12:12:082
29501994416,14cyclictest1124-21snmpd10:19:222
29501994416,12cyclictest20060-21sshd08:11:442
29501994414,17cyclictest0-21swapper/208:06:572
29501994414,17cyclictest0-21swapper/208:06:572
29501994412,20cyclictest24933-21chrt11:25:272
29501994412,11cyclictest7950irq/56-dwc_otg_08:30:172
2950099447,25cyclictest0-21swapper/110:50:311
2950099447,23cyclictest0-21swapper/111:20:291
2950099446,24cyclictest0-21swapper/109:40:291
2950099446,24cyclictest0-21swapper/109:05:301
29500994418,14cyclictest0-21swapper/107:15:311
29500994418,14cyclictest0-21swapper/107:15:311
2949999442,23cyclictest0-21swapper/007:55:330
2949999441,34cyclictest0-21swapper/008:01:440
2949999441,25cyclictest0-21swapper/007:30:460
29502994324,7cyclictest6455-21users12:00:473
29502994319,12cyclictest31498-21/usr/sbin/munin08:45:483
29502994312,13cyclictest11751-21latency_hist09:20:173
2950199437,24cyclictest0-21swapper/207:45:302
29501994316,11cyclictest22141-21seq08:20:052
29501994315,15cyclictest12812-21runrttasks07:51:352
29501994312,12cyclictest0-21swapper/211:11:162
2950099438,14cyclictest7950irq/56-dwc_otg_10:00:171
2950099437,24cyclictest0-21swapper/111:45:231
2950099437,23cyclictest0-21swapper/109:10:221
2950099437,22cyclictest0-21swapper/109:00:311
2950099436,23cyclictest0-21swapper/108:05:301
2950099436,23cyclictest0-21swapper/108:05:301
2950099436,22cyclictest0-21swapper/108:55:301
29499994324,3cyclictest0-21swapper/011:45:350
2949999432,28cyclictest0-21swapper/008:09:240
2949999432,28cyclictest0-21swapper/008:09:240
2949999432,27cyclictest0-21swapper/012:15:310
2949999432,27cyclictest0-21swapper/012:15:310
2949999432,25cyclictest4623-21grep11:55:460
29499994318,4cyclictest0-21swapper/010:35:370
29502994217,18cyclictest27032-21/usr/sbin/munin11:30:333
2950199429,20cyclictest7950irq/56-dwc_otg_11:19:352
29501994215,12cyclictest4478-21chrt10:27:392
29500994210,3cyclictest0-21swapper/110:35:421
2949999421,37cyclictest21982-21grep09:45:460
2949999421,21cyclictest3066-21latency_hist10:25:170
2950299419,21cyclictest16381-21taskset12:33:413
2950199419,19cyclictest0-21swapper/210:24:542
2950199419,10cyclictest0-21swapper/211:00:102
2950199417,24cyclictest0-21swapper/211:20:292
2950199417,24cyclictest0-21swapper/207:55:292
29501994121,12cyclictest5752-21seq09:01:352
29501994116,16cyclictest1124-21snmpd10:30:192
2950099417,23cyclictest0-21swapper/109:20:311
2950099416,24cyclictest0-21swapper/107:30:301
2950099411,9cyclictest7750irq/56-dwc_otg12:05:281
2949999416,25cyclictest0-21swapper/010:00:220
29499994118,11cyclictest0-21swapper/009:08:320
29499994116,5cyclictest0-21swapper/012:39:500
283842412,5sleep20-21swapper/207:05:432
2950299407,29cyclictest0-21swapper/307:20:293
29502994018,5cyclictest471-21munin-node12:05:273
29502994017,12cyclictest13002-21grep09:20:463
2950199409,21cyclictest7950irq/56-dwc_otg_09:44:042
2950199407,25cyclictest0-21swapper/209:50:302
29501994022,8cyclictest12172-21apt-get10:50:292
29501994019,11cyclictest2504-21/usr/sbin/munin07:25:362
29501994011,16cyclictest0-21swapper/212:39:102
29501994011,12cyclictest16163-21chrt12:30:522
2950099407,12cyclictest23-21ksoftirqd/111:05:411
2950099407,12cyclictest23-21ksoftirqd/111:05:411
2950099406,25cyclictest0-21swapper/108:25:311
2950099406,22cyclictest0-21swapper/111:15:301
2950099401,22cyclictest26195-21aten_rbpower_vo08:30:311
2949999406,16cyclictest7750irq/56-dwc_otg08:55:300
2949999401,24cyclictest0-21swapper/008:34:290
2950299396,23cyclictest10374-21taskset10:45:313
2950299396,23cyclictest10374-21taskset10:45:313
29502993920,11cyclictest26292-21df_inode08:30:333
29502993910,10cyclictest0-21swapper/312:39:093
2950199397,23cyclictest0-21swapper/209:35:302
29501993917,15cyclictest1124-21snmpd07:18:312
29501993917,15cyclictest1124-21snmpd07:18:312
2950099396,20cyclictest0-21swapper/107:10:311
2950299384,21cyclictest0-21swapper/311:14:013
29502993813,11cyclictest25877-21latency_hist08:30:173
29501993824,7cyclictest7850irq/56-dwc_otg_08:50:342
29501993824,7cyclictest7850irq/56-dwc_otg_08:50:342
29501993812,10cyclictest7584-21chrt09:06:522
2950099386,15cyclictest0-21swapper/107:45:301
2950099384,19cyclictest10544-21latency_hist12:15:181
29499993817,12cyclictest0-21swapper/007:35:410
2950299379,11cyclictest5775-21taskset07:34:033
2950299378,11cyclictest31643-21taskset11:41:383
2950299378,11cyclictest31643-21taskset11:41:383
2950299377,9cyclictest7950irq/56-dwc_otg_09:46:473
2950099372,4cyclictest28662-21fschecks_time10:05:341
2950099372,4cyclictest28662-21fschecks_time10:05:341
2950099371,18cyclictest221ktimersoftd/109:40:171
2949999372,25cyclictest0-21swapper/009:15:390
2950299369,7cyclictest7950irq/56-dwc_otg_11:00:163
2950299369,13cyclictest0-21swapper/307:11:223
2950299369,11cyclictest468-21rs:main3
2950299365,14cyclictest6866-21/usr/sbin/munin10:35:313
29502993610,17cyclictest128-21systemd-journal10:24:303
29501993623,11cyclictest31234-21awk10:10:452
2950099367,22cyclictest0-21swapper/112:20:271
2950099366,7cyclictest3255-21tr10:25:331
2950099365,17cyclictest221ktimersoftd/109:20:161
29500993614,6cyclictest128-21systemd-journal12:29:481
29500993612,13cyclictest9550irq/80-mmc008:50:171
2950299358,11cyclictest20561-21chrt09:44:253
2950299357,9cyclictest0-21swapper/307:41:413
2950299357,11cyclictest16464-21taskset08:02:083
2950299356,9cyclictest7950irq/56-dwc_otg_09:12:283
2950299356,7cyclictest17563-21fschecks_time09:35:343
29502993526,7cyclictest868-21apt-get08:50:283
29502993526,7cyclictest868-21apt-get08:50:283
29502993520,11cyclictest5188-21grep09:00:403
2950299351,7cyclictest7850irq/56-dwc_otg_10:00:173
2950299351,24cyclictest19636-21ls08:10:423
2950099356,12cyclictest21-21rcuc/109:30:271
2950099354,22cyclictest221ktimersoftd/111:02:581
2950099353,6cyclictest9071-21grep07:40:451
29500993510,7cyclictest221ktimersoftd/108:40:341
2950299348,9cyclictest7850irq/56-dwc_otg_12:11:483
2950299346,19cyclictest0-21swapper/310:30:223
29502993421,10cyclictest6758-21ls07:35:393
29502993411,14cyclictest3737-21taskset07:26:053
2950099345,6cyclictest21-21rcuc/111:55:171
2950099342,14cyclictest221ktimersoftd/107:55:171
29500993414,5cyclictest1749-21if_eth010:20:351
29500993410,6cyclictest7750irq/56-dwc_otg10:10:341
2950299336,10cyclictest7850irq/56-dwc_otg_12:17:173
2950299336,10cyclictest7850irq/56-dwc_otg_12:17:173
29502993319,11cyclictest22984-21cut11:20:173
29502993318,12cyclictest11788-21fschecks_count07:50:343
29502993310,11cyclictest31352-21seq10:10:583
2950099336,15cyclictest0-21swapper/108:00:311
2950099335,6cyclictest898-21wget08:50:301
2950099335,6cyclictest898-21wget08:50:301
2950099335,6cyclictest21-21rcuc/109:25:271
29500993313,7cyclictest221ktimersoftd/107:26:441
2950099331,14cyclictest23-21ksoftirqd/111:55:391
2950299327,15cyclictest0-21swapper/311:50:303
2950299326,15cyclictest7950irq/56-dwc_otg_11:25:273
29502993218,11cyclictest22900-21/usr/sbin/munin08:20:363
2950099326,16cyclictest0-21swapper/110:40:301
2950099325,5cyclictest7750irq/56-dwc_otg08:15:351
2950099325,14cyclictest23-21ksoftirqd/111:30:291
2950099324,11cyclictest7850irq/56-dwc_otg_10:30:291
29500993215,8cyclictest17817-21unixbench-2d12:35:451
29500993212,7cyclictest7850irq/56-dwc_otg_08:35:411
2950099316,15cyclictest0-21swapper/111:10:301
2950099314,7cyclictest221ktimersoftd/112:20:181
2950099313,11cyclictest221ktimersoftd/112:00:461
2950099312,11cyclictest0-21swapper/112:35:161
2950099311,20cyclictest0-21swapper/111:40:311
2950099311,20cyclictest0-21swapper/111:40:311
2950299306,14cyclictest0-21swapper/310:16:223
2950099303,6cyclictest21-21rcuc/108:25:171
2950099301,17cyclictest0-21swapper/111:28:181
29502992915,11cyclictest32002-21ntpq07:15:413
29502992915,11cyclictest32002-21ntpq07:15:413
2950099298,6cyclictest14511-21latency10:55:361
2950099294,4cyclictest21195-21fschecks_time09:45:351
2950099287,9cyclictest221ktimersoftd/112:17:161
290612283,16sleep10-21swapper/107:08:131
291492262,15sleep30-21swapper/307:09:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional