You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-02 - 05:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Wed Sep 02, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7786991273,78cyclictest0-21swapper/021:25:420
2798121000,3sleep10-21swapper/121:20:291
778699974,9cyclictest0-21swapper/000:05:300
7789999517,58cyclictest0-21swapper/100:05:291
778699891,54cyclictest0-21swapper/021:10:250
778699865,10cyclictest27073-21runrttasks00:35:270
7786998323,5cyclictest30867-21sort00:00:320
7786998320,50cyclictest7950irq/56-dwc_otg_21:20:140
778699822,61cyclictest0-21swapper/020:35:290
778699821,69cyclictest0-21swapper/019:40:370
778699821,61cyclictest4759-21modprobe22:50:140
778699812,63cyclictest14497-21latency_hist00:20:150
778699805,33cyclictest0-21swapper/020:40:260
7786997944,8cyclictest7950irq/56-dwc_otg_21:50:260
778699791,24cyclictest17484-21/usr/sbin/munin21:05:310
220372790,4sleep0778699cyclictest00:29:230
778699781,24cyclictest0-21swapper/023:54:520
778699771,57cyclictest0-21swapper/022:30:390
7786997712,50cyclictest8228-21/usr/sbin/munin21:35:320
7786997517,43cyclictest0-21swapper/000:31:530
778699751,70cyclictest6652-21sort00:10:320
778699749,15cyclictest0-21swapper/023:20:400
778699731,38cyclictest0-21swapper/021:45:270
778699727,3cyclictest0-21swapper/022:22:170
778699727,3cyclictest0-21swapper/022:22:170
7786997218,33cyclictest26276-21sh23:55:250
778699721,57cyclictest26668-21sh22:36:150
778699721,24cyclictest12728-21ntp_states21:40:380
778699721,24cyclictest12728-21ntp_states21:40:380
7786997120,4cyclictest26171-21if_wlan023:15:340
778699706,14cyclictest17237-21fschecks_time19:35:310
7786997013,11cyclictest0-21swapper/023:37:550
778999694,39cyclictest0-21swapper/121:25:401
778699696,46cyclictest32659-21grep20:15:430
778699692,46cyclictest0-21swapper/022:55:140
7786996913,27cyclictest13024-21idleruntime-cro23:00:140
7789996711,8cyclictest0-21swapper/100:35:271
778699672,47cyclictest0-21swapper/020:10:150
7786996715,5cyclictest7750irq/56-dwc_otg22:25:280
7786996713,9cyclictest7353-21rm22:14:030
778699671,22cyclictest0-21swapper/000:20:160
778699671,22cyclictest0-21swapper/000:20:160
7786996618,2cyclictest0-21swapper/023:25:390
7786996616,9cyclictest0-21swapper/021:24:380
778699661,52cyclictest0-21swapper/023:46:240
778699661,41cyclictest0-21swapper/019:25:140
778699661,20cyclictest8950-21grep19:10:430
778699661,14cyclictest0-21swapper/022:03:010
778699653,45cyclictest16265-21grep19:30:430
778699652,4cyclictest0-21swapper/023:40:430
778699652,4cyclictest0-21swapper/023:40:430
7786996515,10cyclictest0-21swapper/023:01:250
778699651,20cyclictest0-21swapper/022:08:300
778699646,13cyclictest0-21swapper/019:16:180
778699646,13cyclictest0-21swapper/019:16:180
778699642,43cyclictest0-21swapper/022:15:410
778699641,45cyclictest22624-21fschecks_count19:50:310
7786996410,10cyclictest0-21swapper/021:57:330
111542640,2sleep20-21swapper/223:35:542
778699632,44cyclictest0-21swapper/020:50:140
778699631,43cyclictest0-21swapper/020:20:400
7786996313,8cyclictest7769-21diskmemload23:32:160
7786996313,10cyclictest0-21swapper/022:44:370
7786996313,10cyclictest0-21swapper/022:44:370
74752633,14sleep10-21swapper/119:09:401
779199622,41cyclictest658-21ssh21:25:423
778699624,17cyclictest0-21swapper/023:12:570
778699624,17cyclictest0-21swapper/023:12:570
778699621,25cyclictest151522sleep021:00:040
778699621,25cyclictest0-21swapper/023:06:580
778699621,17cyclictest6144-21ssh21:32:490
7791996121,14cyclictest0-21swapper/300:05:293
778699612,33cyclictest0-21swapper/020:00:210
7786996115,11cyclictest0-21swapper/020:32:450
100712610,1sleep20-21swapper/200:15:082
7786995911,3cyclictest0-21swapper/020:15:150
155422590,5sleep3779199cyclictest22:23:443
155422590,5sleep3779199cyclictest22:23:443
779099581,9cyclictest9229-21ssh22:55:272
778699586,13cyclictest21811-21taskset19:46:290
778699584,11cyclictest0-21swapper/020:50:300
778699584,11cyclictest0-21swapper/020:50:300
778699581,16cyclictest25694-21chrt19:59:500
778999565,21cyclictest0-21swapper/121:10:251
779099553,35cyclictest571-21ls21:25:402
7786995510,11cyclictest0-21swapper/021:00:250
7789995421,15cyclictest7950irq/56-dwc_otg_20:35:241
778699542,3cyclictest0-21swapper/020:27:130
10202540,2sleep20-21swapper/223:25:032
779199536,34cyclictest39-21ksoftirqd/319:15:193
779199536,34cyclictest39-21ksoftirqd/319:15:193
7789995312,13cyclictest0-21swapper/122:35:271
7786995212,10cyclictest0-21swapper/019:29:390
74452513,41sleep30-21swapper/319:09:183
7789994914,3cyclictest7750irq/56-dwc_otg22:00:281
779099483,26cyclictest31-21ksoftirqd/200:05:302
778999477,26cyclictest0-21swapper/120:00:201
7789994720,5cyclictest456-21fschecks_time22:05:321
778999462,6cyclictest7750irq/56-dwc_otg21:40:251
778999462,6cyclictest7750irq/56-dwc_otg21:40:251
778999461,2cyclictest0-21swapper/121:00:261
778999461,17cyclictest0-21swapper/119:20:281
71872463,35sleep00-21swapper/019:06:030
779099459,28cyclictest0-21swapper/219:15:192
779099459,28cyclictest0-21swapper/219:15:192
778999457,24cyclictest0-21swapper/120:30:291
7789994514,12cyclictest221ktimersoftd/123:20:481
779199447,24cyclictest0-21swapper/300:35:213
778999447,27cyclictest0-21swapper/121:30:281
778999446,25cyclictest18786-21diskmemload00:10:291
778999438,23cyclictest0-21swapper/123:00:261
778999432,6cyclictest0-21swapper/100:20:271
778999432,6cyclictest0-21swapper/100:20:271
7789994320,5cyclictest27624-21sh23:55:451
779099427,20cyclictest0-21swapper/220:50:272
779099427,20cyclictest0-21swapper/220:50:272
779099422,24cyclictest0-21swapper/220:25:152
7790994219,12cyclictest18786-21diskmemload23:55:252
778999427,22cyclictest0-21swapper/120:55:281
778999426,27cyclictest0-21swapper/121:05:271
778999426,16cyclictest0-21swapper/122:20:281
778999426,16cyclictest0-21swapper/122:20:281
778999425,12cyclictest0-21swapper/100:30:281
7789994221,5cyclictest4340-21aten_rbpower_po22:10:301
7789994216,5cyclictest25945-21latency_hist23:55:141
7789994216,14cyclictest916-21aten_rbpower_en20:20:291
778999416,5cyclictest30562-21apt-get00:00:261
7789994119,14cyclictest31517-21systemd-journal23:07:411
7789994113,11cyclictest221ktimersoftd/122:15:391
778999411,22cyclictest24971-21missed_timers21:55:371
779199407,22cyclictest0-21swapper/320:50:273
779199407,22cyclictest0-21swapper/320:50:273
779099407,20cyclictest0-21swapper/200:35:212
778999401,25cyclictest0-21swapper/119:55:311
7789994012,4cyclictest7750irq/56-dwc_otg21:35:281
7789994011,25cyclictest0-21swapper/119:30:281
256142400,2sleep30-21swapper/323:54:473
778999398,5cyclictest23-21ksoftirqd/122:30:331
778999398,23cyclictest0-21swapper/120:25:291
778999396,5cyclictest7750irq/56-dwc_otg20:50:421
778999396,5cyclictest7750irq/56-dwc_otg20:50:421
7789993914,5cyclictest7950irq/56-dwc_otg_23:20:141
779199387,23cyclictest0-21swapper/320:00:203
7791993813,5cyclictest24580-21/usr/sbin/munin21:55:323
779199381,23cyclictest22702-21ls00:30:153
7791993811,7cyclictest18011-21aten_rbpower_cu23:45:283
779099387,22cyclictest0-21swapper/221:15:312
779099384,4cyclictest7950irq/56-dwc_otg_19:25:282
779099383,25cyclictest0-21swapper/219:20:282
779099382,3cyclictest7850irq/56-dwc_otg_19:45:352
7790993813,8cyclictest26039-21fschecks_time23:15:322
779099381,24cyclictest28867-21http22:40:142
7790993811,16cyclictest22899-21wget00:30:292
778999382,23cyclictest0-21swapper/100:15:261
7789993819,7cyclictest32076-1kworker/u9:122:54:581
778999381,4cyclictest0-21swapper/123:50:151
7789993813,11cyclictest7750irq/56-dwc_otg22:25:261
778999381,26cyclictest8555-21ls20:40:331
778999381,20cyclictest10066-21aten_rbpower_en20:45:291
7789993811,15cyclictest29756-21wget20:10:301
7791993728,7cyclictest5127-21ssh22:50:283
7791993713,6cyclictest28571-21sh22:39:463
779199371,18cyclictest4051-21ls22:10:153
779099372,4cyclictest24687-21fschecks_time21:55:332
779099372,25cyclictest0-21swapper/223:25:412
779099372,20cyclictest0-21swapper/223:10:322
779099372,20cyclictest0-21swapper/223:10:322
779099372,18cyclictest18786-21diskmemload23:05:262
7790993712,6cyclictest7480-21unixbench_multi20:35:412
778999378,7cyclictest7750irq/56-dwc_otg20:05:411
778999378,27cyclictest0-21swapper/119:25:271
778999377,17cyclictest0-21swapper/119:10:291
7789993719,9cyclictest7750irq/56-dwc_otg19:40:281
7789993714,11cyclictest7750irq/56-dwc_otg23:15:151
7789993714,11cyclictest7750irq/56-dwc_otg23:15:151
779199369,3cyclictest0-21swapper/323:09:543
779199368,8cyclictest0-21swapper/322:40:203
779199368,8cyclictest0-21swapper/322:40:203
779199365,13cyclictest7950irq/56-dwc_otg_21:40:283
779199365,13cyclictest7950irq/56-dwc_otg_21:40:283
779099367,13cyclictest7850irq/56-dwc_otg_19:30:432
779099362,4cyclictest0-21swapper/222:05:262
7790993616,12cyclictest2057-21cat00:05:152
7790993614,7cyclictest5311-21open_files20:30:402
7790993613,4cyclictest20231-21sh23:47:282
778999369,13cyclictest7950irq/56-dwc_otg_23:35:251
778999369,13cyclictest7750irq/56-dwc_otg23:43:501
778999369,13cyclictest7750irq/56-dwc_otg23:43:501
778999364,13cyclictest7750irq/56-dwc_otg21:51:551
7789993621,4cyclictest7850irq/56-dwc_otg_23:25:411
7789993614,6cyclictest0-21swapper/123:30:281
778999361,26cyclictest19310-21ssh00:25:341
779199358,5cyclictest0-21swapper/321:16:013
779199358,3cyclictest0-21swapper/323:17:403
7791993514,4cyclictest30695-21aten_rbpower_vo00:00:293
779199351,23cyclictest2520-21ls20:25:153
779099359,24cyclictest2465150http22:35:142
779099357,24cyclictest0-21swapper/220:00:202
779099356,14cyclictest7950irq/56-dwc_otg_00:20:282
779099356,14cyclictest7950irq/56-dwc_otg_00:20:282
779099355,7cyclictest7750irq/56-dwc_otg21:10:242
779099351,15cyclictest0-21swapper/221:45:232
778999356,23cyclictest0-21swapper/119:45:271
778999353,8cyclictest0-21swapper/122:55:271
778999351,8cyclictest2213-21unixbench_multi22:45:451
7789993515,9cyclictest7850irq/56-dwc_otg_21:49:091
779199349,15cyclictest0-21swapper/323:36:573
7791993412,4cyclictest26495-21cpuspeed23:55:283
779199341,22cyclictest4747-21latency_hist22:50:143
779199341,22cyclictest11830-21grep19:20:323
779199341,21cyclictest14824-21switchtime23:40:403
779199341,21cyclictest14824-21switchtime23:40:403
7791993411,4cyclictest0-21swapper/321:30:363
7791993410,5cyclictest31659-21latency_hist21:25:153
7791993410,4cyclictest0-21swapper/322:17:463
779099349,3cyclictest7750irq/56-dwc_otg22:50:482
779099341,8cyclictest9489-21taskset23:35:152
7790993415,13cyclictest301ktimersoftd/223:50:332
7790993414,4cyclictest17647-21munin-run23:45:152
7790993414,4cyclictest17647-21munin-run23:45:152
778999349,15cyclictest0-21swapper/122:40:201
778999349,15cyclictest0-21swapper/122:40:201
7789993428,4cyclictest22456-21aten_rbpower_po19:50:281
778999341,22cyclictest17823-21switchtime19:35:401
7789993410,8cyclictest221ktimersoftd/121:18:591
114012341,4sleep311490-21sed00:15:423
779199338,4cyclictest0-21swapper/322:28:523
779199338,12cyclictest4644-21idleruntime22:10:343
7791993322,7cyclictest1849-21fschecks_time23:25:313
7791993316,11cyclictest10224-21ssh21:37:433
779199331,18cyclictest32568-21grep20:15:433
7791993311,4cyclictest10215-21df_inode20:45:313
779099339,14cyclictest0-21swapper/200:25:272
779099338,6cyclictest7950irq/56-dwc_otg_19:50:442
779099335,13cyclictest7950irq/56-dwc_otg_21:50:362
7790993318,4cyclictest1905-21sh22:45:432
779099331,20cyclictest20429-21ls19:45:152
779199328,13cyclictest0-21swapper/322:31:443
779199326,6cyclictest0-21swapper/323:31:573
7791993211,10cyclictest0-21swapper/300:10:283
779099325,12cyclictest7950irq/56-dwc_otg_21:35:142
779099325,12cyclictest7850irq/56-dwc_otg_21:20:282
779099322,15cyclictest0-21swapper/220:57:312
7790993217,4cyclictest29708-21if_err_eth022:40:322
7790993217,4cyclictest29708-21if_err_eth022:40:322
779099321,19cyclictest16525-21ls22:25:152
779099321,19cyclictest16525-21ls22:25:152
778999324,14cyclictest21-21rcuc/119:18:291
778999324,14cyclictest21-21rcuc/119:18:291
123842320,2sleep20-21swapper/200:16:492
779199317,4cyclictest0-21swapper/323:10:563
779199317,4cyclictest0-21swapper/323:10:563
779199315,15cyclictest20188-21timerandwakeup21:10:403
779199312,18cyclictest0-21swapper/319:44:463
779199311,23cyclictest22564-21cpuspeed19:50:293
779199311,18cyclictest4380-21ls20:30:143
7791993110,4cyclictest0-21swapper/321:50:303
779099315,20cyclictest7950irq/56-dwc_otg_23:00:242
779099312,8cyclictest12858-21sensors_temp21:40:402
779099312,8cyclictest12858-21sensors_temp21:40:402
7790993124,5cyclictest16845-21aten_rbpower_vo22:25:302
779099312,15cyclictest0-21swapper/222:10:542
779099311,4cyclictest7850irq/56-dwc_otg_20:10:282
779099311,16cyclictest7950irq/56-dwc_otg_20:40:422
779099311,16cyclictest28148-21grep20:05:312
779099311,15cyclictest10345-21grep20:45:332
778999312,3cyclictest0-21swapper/120:16:111
101422311,4sleep20-21swapper/222:16:362
779199306,11cyclictest0-21swapper/321:49:363
779199305,6cyclictest39-21ksoftirqd/323:00:423
779199302,18cyclictest0-21swapper/319:35:563
779199301,26cyclictest16976-21latency_hist21:05:143
7791993010,4cyclictest0-21swapper/319:25:403
779099306,4cyclictest7950irq/56-dwc_otg_22:03:462
7790993012,9cyclictest1180-21runrttasks20:26:062
779099301,17cyclictest15410-21/usr/sbin/munin21:00:352
779099301,15cyclictest16939-21/usr/sbin/munin19:35:392
779199299,10cyclictest0-21swapper/300:21:093
779199299,10cyclictest0-21swapper/300:21:093
779199297,4cyclictest7850irq/56-dwc_otg_20:05:243
779199296,12cyclictest0-21swapper/323:20:203
7791992923,4cyclictest9766-21munin-plugin-st20:45:143
779199291,6cyclictest29286-21ssh22:00:403
779099295,12cyclictest7850irq/56-dwc_otg_20:15:432
779099294,12cyclictest7850irq/56-dwc_otg_21:05:302
779199286,13cyclictest0-21swapper/322:56:443
779199283,12cyclictest7950irq/56-dwc_otg_19:50:143
7791992819,6cyclictest16912-21latency_hist19:35:153
779199281,2cyclictest0-21swapper/320:55:263
779199281,2cyclictest0-21swapper/300:33:433
779199281,15cyclictest0-21swapper/320:39:573
779199281,10cyclictest305062grep20:10:413
779099286,4cyclictest7950irq/56-dwc_otg_19:10:362
779099281,13cyclictest0-21swapper/221:37:102
779099281,12cyclictest24167-21/usr/sbin/munin19:55:402
779199278,10cyclictest39-21ksoftirqd/320:30:293
779199273,14cyclictest7950irq/56-dwc_otg_19:10:433
779199271,19cyclictest0-21swapper/321:05:443
59052273,12sleep20-21swapper/219:05:162
779199252,15cyclictest0-21swapper/319:58:583
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional