You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 09:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Mar 14, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
691221730,6sleep22907999cyclictest21:46:442
59252980,9sleep22907999cyclictest19:35:172
2907799946,25cyclictest817-21ssh21:40:160
323012870,1sleep30-21swapper/323:35:313
305612840,1sleep00-21swapper/022:15:280
193012830,1sleep20-21swapper/200:39:232
256452770,2sleep20-21swapper/220:30:032
75492720,4sleep22907999cyclictest00:25:102
175832720,2sleep20-21swapper/223:57:362
136352710,2sleep113634-21ssh21:15:311
205402690,1sleep30-21swapper/322:04:003
205402690,1sleep30-21swapper/322:04:003
2907799684,36cyclictest0-21swapper/021:05:140
29079996710,14cyclictest11746-21ssh00:30:142
29079996710,14cyclictest11746-21ssh00:30:142
34742640,5sleep339-21ksoftirqd/322:21:523
29079996412,13cyclictest7750irq/56-dwc_otg00:00:142
29077996422,22cyclictest9970-21sshd19:45:230
2907799632,31cyclictest14385-21sed19:55:330
29077996313,33cyclictest22018-21idleruntime-cro20:20:030
2907899626,11cyclictest7950irq/56-dwc_otg_00:00:141
2907999604,14cyclictest17944-21ssh22:40:142
2907799608,33cyclictest101ktimersoftd/020:10:170
29077996019,22cyclictest31881-21sh00:15:170
2907799594,25cyclictest2808250ssh23:30:300
2907799593,4cyclictest10296-21ssh23:10:130
2907799592,32cyclictest505-21rs:main0
306662570,2sleep30-21swapper/323:34:583
2907999575,9cyclictest0-21swapper/221:55:142
29077995739,4cyclictest8373-21fschecks_count21:10:200
2907799572,9cyclictest19689-21modprobe00:40:030
29077995729,13cyclictest7950irq/56-dwc_otg_22:20:170
2907799567,46cyclictest0-21swapper/020:30:160
29077995626,11cyclictest7950irq/56-dwc_otg_21:00:300
2907799561,39cyclictest0-21swapper/000:05:040
2907799561,29cyclictest6944-21aten_rbpower_en23:45:160
29077995514,29cyclictest15175-21wget23:55:160
29077995513,30cyclictest11-21rcu_preempt23:15:120
2907799551,28cyclictest17005-21idleruntime-cro22:00:030
2907799551,26cyclictest21221-21uniq21:25:210
2907799551,26cyclictest21221-21uniq21:25:210
29077995511,33cyclictest11347-21fschecks_time23:50:200
29077995511,33cyclictest11347-21fschecks_time23:50:200
29077995428,5cyclictest16190-21unixbench_singl20:00:300
29077995424,13cyclictest10298-21tail21:50:320
2907799542,27cyclictest25217-21unixbench_singl20:25:310
2907799541,5cyclictest2763-21tr23:00:240
2907799541,5cyclictest2763-21tr23:00:240
2907799541,27cyclictest5683-21wget22:25:180
2907799541,25cyclictest0-21swapper/021:20:270
2907799533,27cyclictest0-21swapper/000:30:300
2907799533,27cyclictest0-21swapper/000:30:300
29077995327,9cyclictest0-21swapper/022:45:180
29077995313,21cyclictest25685-21sh21:30:240
2907799531,26cyclictest3695-21fschecks_count00:20:230
29077995310,19cyclictest0-21swapper/000:25:150
2907799529,13cyclictest0-21swapper/020:55:290
29077995234,5cyclictest18015-21wget22:40:160
29077995232,5cyclictest7850irq/56-dwc_otg_21:20:030
29077995216,18cyclictest0-21swapper/022:00:200
29077995216,18cyclictest0-21swapper/022:00:200
2907799521,25cyclictest2342850ssh23:25:240
2907899513,11cyclictest14422-21ssh23:15:141
29077995127,4cyclictest9-21ksoftirqd/020:45:260
29077995126,4cyclictest7950irq/56-dwc_otg_21:35:250
2907799512,30cyclictest0-21swapper/020:40:250
29077995122,6cyclictest1704-21munin-plugin-st23:00:030
2907799512,25cyclictest10015-21cut22:30:190
2907799512,25cyclictest10015-21cut22:30:190
29077995117,17cyclictest9-21ksoftirqd/023:05:300
29077995114,21cyclictest908-21runrttasks22:36:300
29077995110,19cyclictest0-21swapper/022:10:300
29077995029,5cyclictest19232-21ls23:20:230
29077995029,5cyclictest19232-21ls23:20:230
2907799502,27cyclictest0-21swapper/020:35:140
2907799502,27cyclictest0-21swapper/020:35:140
2907799501,6cyclictest5926-21/usr/sbin/munin19:35:170
2907799501,24cyclictest5727-21ssh21:45:270
2908099488,25cyclictest0-21swapper/320:20:153
2907999485,6cyclictest0-21swapper/222:55:162
2907799482,31cyclictest0-21swapper/023:39:360
2907799481,34cyclictest4259-21cut19:30:190
2907799481,24cyclictest29567-21grep19:10:220
2908099477,25cyclictest0-21swapper/321:25:173
2908099477,25cyclictest0-21swapper/321:25:173
2907999476,27cyclictest0-21swapper/219:55:102
2907999472,8cyclictest0-21swapper/222:10:142
29077994725,9cyclictest0-21swapper/020:05:240
29077994723,13cyclictest24276-21processes00:05:270
29077994722,22cyclictest32757-21aten_rbpower_cu19:20:140
2907799471,26cyclictest1495-21grep20:50:270
2908099467,25cyclictest0-21swapper/319:25:173
2907999463,8cyclictest7950irq/56-dwc_otg_20:00:142
2907999463,6cyclictest7950irq/56-dwc_otg_19:30:142
2907999462,26cyclictest7850irq/56-dwc_otg_20:05:222
2907899467,26cyclictest0-21swapper/119:50:081
2907899466,24cyclictest0-21swapper/119:10:161
29078994622,6cyclictest23-21ksoftirqd/121:55:251
2907799462,26cyclictest27478-21rm00:10:120
2908099457,25cyclictest0-21swapper/321:05:143
2908099457,25cyclictest0-21swapper/320:25:163
2908099457,24cyclictest0-21swapper/319:15:163
2908099457,24cyclictest0-21swapper/319:15:163
2907999457,25cyclictest0-21swapper/221:50:162
2907899457,27cyclictest0-21swapper/120:30:161
29077994525,5cyclictest7850irq/56-dwc_otg_19:55:030
2907799451,29cyclictest0-21swapper/019:40:250
2907799451,29cyclictest0-21swapper/019:40:250
285632452,13sleep00-21swapper/019:06:530
323122440,5sleep12907899cyclictest22:17:511
2907999447,22cyclictest0-21swapper/219:10:162
2907999441,5cyclictest1404-21/usr/sbin/munin22:20:282
2907899446,25cyclictest0-21swapper/120:15:161
2907899443,25cyclictest0-21swapper/119:40:161
2907899443,25cyclictest0-21swapper/119:40:161
29078994421,6cyclictest23-21ksoftirqd/100:30:221
29078994421,6cyclictest23-21ksoftirqd/100:30:221
29078994410,12cyclictest0-21swapper/123:10:141
2907799441,19cyclictest6556-21munin-plugin-st23:45:030
2907899437,17cyclictest0-21swapper/119:55:101
2907899436,27cyclictest0-21swapper/122:10:151
2907899434,24cyclictest0-21swapper/120:00:151
2907899433,22cyclictest0-21swapper/119:35:161
29078994324,5cyclictest23-21ksoftirqd/122:40:191
29078994318,13cyclictest21-21rcuc/100:07:481
29077994320,3cyclictest0-21swapper/019:25:140
2907799431,23cyclictest0-21swapper/022:09:020
2907999427,23cyclictest0-21swapper/219:50:082
2907899422,16cyclictest0-21swapper/123:00:041
29078994215,7cyclictest21-21rcuc/123:25:031
29078994215,7cyclictest21-21rcuc/123:25:031
2788624210,13sleep10-21swapper/119:05:241
2908099417,23cyclictest0-21swapper/323:20:153
2908099417,23cyclictest0-21swapper/323:20:153
2908099412,25cyclictest0-21swapper/319:20:143
2907999415,24cyclictest10318-21ssh23:10:162
2907999414,19cyclictest301ktimersoftd/221:30:252
29079994122,4cyclictest7750irq/56-dwc_otg20:10:222
2907899417,25cyclictest0-21swapper/123:25:091
2907899416,7cyclictest7850irq/56-dwc_otg_22:20:281
2907799411,29cyclictest0-21swapper/020:23:530
2908099406,24cyclictest0-21swapper/323:00:153
2908099406,24cyclictest0-21swapper/323:00:153
29080994018,14cyclictest18671-21aten_rbpower_cu20:10:173
2907999403,6cyclictest31-21ksoftirqd/221:35:172
2907899407,21cyclictest0-21swapper/123:30:151
2907799401,35cyclictest0-21swapper/019:15:090
2907799401,35cyclictest0-21swapper/019:15:090
2908099392,15cyclictest20406-21ssh00:00:303
29079993919,5cyclictest12841-21kernelversion21:15:222
29079993911,15cyclictest301ktimersoftd/223:45:202
2907899399,4cyclictest21-21rcuc/123:05:201
2907899396,22cyclictest7547-21ssh00:25:091
2907899391,21cyclictest0-21swapper/121:00:041
175242391,4sleep117534-21sh00:36:091
2908099387,21cyclictest0-21swapper/323:05:173
2908099386,19cyclictest15147-21ssh23:55:163
2908099383,23cyclictest21537-21grep20:15:303
2908099382,8cyclictest2564450ssh21:30:233
29080993816,5cyclictest12531-21sh00:30:263
29080993816,5cyclictest12531-21sh00:30:263
2908099381,25cyclictest0-21swapper/300:25:143
29080993810,17cyclictest0-21swapper/323:45:173
2907999389,6cyclictest7850irq/56-dwc_otg_20:15:302
2907999387,16cyclictest23426-21ls23:25:242
2907999381,4cyclictest7950irq/56-dwc_otg_20:35:132
2907999381,4cyclictest7950irq/56-dwc_otg_20:35:132
2907999381,3cyclictest0-21swapper/200:10:162
2907899381,9cyclictest0-21swapper/120:10:141
29078993814,5cyclictest0-21swapper/123:05:031
29078993814,5cyclictest0-21swapper/123:05:031
2907899381,34cyclictest26833-21ssh22:50:241
29080993719,9cyclictest7950irq/56-dwc_otg_21:11:333
2907999379,15cyclictest0-21swapper/220:55:162
2907999379,12cyclictest7850irq/56-dwc_otg_22:04:022
2907999379,12cyclictest7850irq/56-dwc_otg_22:04:022
2907999377,24cyclictest0-21swapper/222:15:172
2907999376,4cyclictest61-21kswapd023:00:132
2907999376,4cyclictest61-21kswapd023:00:132
2907999375,15cyclictest7850irq/56-dwc_otg_22:32:012
2907999375,15cyclictest7850irq/56-dwc_otg_22:32:012
29079993718,6cyclictest7850irq/56-dwc_otg_21:20:272
2907999371,25cyclictest25271-21ls22:10:032
2907899378,19cyclictest0-21swapper/123:35:251
29078993716,7cyclictest23-21ksoftirqd/121:40:161
29078993716,7cyclictest21-21rcuc/121:05:201
2907899371,17cyclictest558-21latency_hist20:50:031
2908099368,15cyclictest0-21swapper/321:35:173
2908099366,20cyclictest0-21swapper/300:35:163
29080993620,5cyclictest10013-21fschecks_time22:30:193
29080993620,5cyclictest10013-21fschecks_time22:30:193
2908099361,25cyclictest12537-21ls21:15:173
29080993611,8cyclictest0-21swapper/321:48:543
2908099361,14cyclictest7750irq/56-dwc_otg23:45:043
2907999369,16cyclictest0-21swapper/223:40:172
2907999362,22cyclictest0-21swapper/219:20:142
2907999361,25cyclictest7889-21cut19:40:192
2907999361,25cyclictest7889-21cut19:40:192
2907899365,9cyclictest0-21swapper/100:10:141
2907899364,16cyclictest7950irq/56-dwc_otg_21:25:041
29078993625,3cyclictest221ktimersoftd/121:36:561
29078993616,5cyclictest12966-21sh22:34:061
29078993616,5cyclictest12966-21sh22:34:061
29078993616,4cyclictest1517-21seq19:21:021
2907899361,15cyclictest0-21swapper/119:30:291
2907899361,13cyclictest17121-21fschecks_time20:05:201
29078993610,3cyclictest0-21swapper/123:40:291
281912361,4sleep328197-21ssh22:12:553
2908099355,4cyclictest7950irq/56-dwc_otg_22:25:183
29080993521,4cyclictest7850irq/56-dwc_otg_21:00:163
29080993517,4cyclictest13153-21/usr/sbin/munin21:55:303
29080993512,5cyclictest15179-21grep23:15:243
29080993511,4cyclictest0-21swapper/321:20:333
2907999357,12cyclictest7850irq/56-dwc_otg_23:08:362
2907999354,4cyclictest4658-21unixbench_singl00:20:342
2907999353,7cyclictest5825-21fschecks_count22:25:202
2907999353,4cyclictest7750irq/56-dwc_otg23:30:232
2907999352,7cyclictest22370-21aten_rbpower_vo20:20:162
2907999351,18cyclictest301ktimersoftd/219:30:032
2907999351,15cyclictest26040-21ssh00:07:582
2907999351,15cyclictest0-21swapper/219:45:382
2907899356,22cyclictest21194-21ls22:05:041
2907899356,22cyclictest21194-21ls22:05:041
2907899354,15cyclictest23-21ksoftirqd/122:37:151
29078993516,5cyclictest23-21ksoftirqd/120:40:191
2907899351,5cyclictest0-21swapper/119:25:211
29078993512,10cyclictest4604-21cut21:00:191
2907899351,16cyclictest29338-21sh20:40:031
2907899351,16cyclictest29338-21sh20:40:031
29078993511,14cyclictest5827-21fschecks_count22:25:201
97382340,1sleep10-21swapper/121:11:081
2908099349,5cyclictest3010-21interrupts20:55:223
2908099345,13cyclictest7850irq/56-dwc_otg_22:05:303
2908099342,16cyclictest0-21swapper/319:53:213
29080993418,9cyclictest3785-21ssh21:43:333
29080993416,5cyclictest22737-21sh22:45:273
2908099341,23cyclictest0-21swapper/322:35:233
2907999342,5cyclictest0-21swapper/220:40:202
2907999342,4cyclictest0-21swapper/222:36:482
29079993412,9cyclictest7750irq/56-dwc_otg22:51:462
29079993411,5cyclictest3040-21apt00:20:032
29078993416,10cyclictest221ktimersoftd/123:45:161
29078993413,11cyclictest7750irq/56-dwc_otg21:45:331
29078993411,4cyclictest0-21swapper/123:50:241
29078993411,4cyclictest0-21swapper/123:50:241
271942341,30sleep3917-21snmpd20:31:023
29080993320,11cyclictest7950irq/56-dwc_otg_00:22:093
29080993317,5cyclictest5895-21aten_rbpower_po19:35:173
2907999339,4cyclictest7950irq/56-dwc_otg_21:05:122
2907999336,4cyclictest7850irq/56-dwc_otg_22:50:032
2907999333,17cyclictest7850irq/56-dwc_otg_23:36:512
2907999333,14cyclictest7850irq/56-dwc_otg_23:24:492
2907999333,14cyclictest7850irq/56-dwc_otg_23:24:492
2907899337,20cyclictest23-21ksoftirqd/100:16:221
2907899336,15cyclictest0-21swapper/122:05:161
2907899336,15cyclictest0-21swapper/121:50:161
29078993312,4cyclictest23816-21latency_hist20:25:041
2907899331,19cyclictest32490-21latency_hist19:20:041
2907899331,19cyclictest32490-21latency_hist19:20:041
2907899331,12cyclictest23-21ksoftirqd/121:30:231
2908099327,15cyclictest0-21swapper/323:25:173
2908099327,15cyclictest0-21swapper/319:30:093
2908099326,11cyclictest7950irq/56-dwc_otg_19:45:243
2908099323,14cyclictest7950irq/56-dwc_otg_19:55:223
2908099322,16cyclictest0-21swapper/322:20:033
2908099322,15cyclictest7950irq/56-dwc_otg_22:59:543
2908099322,12cyclictest7850irq/56-dwc_otg_00:06:043
2908099321,5cyclictest7850irq/56-dwc_otg_20:40:273
2907999329,8cyclictest31-21ksoftirqd/220:45:552
2907999327,7cyclictest7750irq/56-dwc_otg23:19:292
2907999327,15cyclictest0-21swapper/220:30:162
2907999324,11cyclictest29-21rcuc/223:50:582
2907999324,11cyclictest29-21rcuc/223:50:582
29079993212,12cyclictest21767-21sensors_temp21:25:282
29079993212,12cyclictest21767-21sensors_temp21:25:282
2907899329,11cyclictest7950irq/56-dwc_otg_20:30:031
2907899325,15cyclictest4335-21ssh00:20:321
2907899324,13cyclictest7950irq/56-dwc_otg_20:55:041
2908099312,8cyclictest11362-21grep23:50:203
2908099312,8cyclictest11362-21grep23:50:203
29080993124,5cyclictest9099-21grep21:50:163
29080993122,6cyclictest29360-21latency_hist20:40:033
29080993122,6cyclictest29360-21latency_hist20:40:033
29080993119,9cyclictest18402-21sh20:10:033
2908099311,24cyclictest3187-21ls00:20:043
2907999319,4cyclictest0-21swapper/221:43:012
2907899318,4cyclictest0-21swapper/123:55:171
2907899311,6cyclictest20957-21grep21:25:171
2907899311,6cyclictest20957-21grep21:25:171
29078993115,4cyclictest21-21rcuc/119:45:281
29078993111,10cyclictest7734-21diskmemload22:47:511
284692313,20sleep20-21swapper/219:05:492
2908099305,15cyclictest7850irq/56-dwc_otg_23:15:033
2908099302,5cyclictest0-21swapper/300:12:253
2908099302,21cyclictest7850irq/56-dwc_otg_19:11:133
29080993017,4cyclictest26174-21/usr/sbin/munin22:50:293
2908099301,19cyclictest0-21swapper/320:45:133
29080993011,4cyclictest0-21swapper/322:42:473
2908099301,11cyclictest7750irq/56-dwc_otg19:40:243
2908099301,11cyclictest7750irq/56-dwc_otg19:40:243
2907999307,5cyclictest8179-21tr21:10:172
2907999303,11cyclictest31-21ksoftirqd/220:50:142
2908099291,7cyclictest15782-21ps20:00:263
2908099291,21cyclictest1339-21missed_timers20:50:243
2907999292,7cyclictest31202-21fschecks_time19:15:202
2907999292,7cyclictest31202-21fschecks_time19:15:202
2907999273,11cyclictest7850irq/56-dwc_otg_21:00:272
286432273,15sleep30-21swapper/319:07:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional