You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 06:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat Aug 15, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3192829885,9sleep00-21swapper/019:09:460
3184829469,8sleep20-21swapper/219:08:452
3177129164,9sleep30-21swapper/319:07:453
3159628465,15sleep10-21swapper/119:05:291
247212580,0sleep00-21swapper/021:29:180
160812490,0sleep10-21swapper/121:24:051
239502440,4sleep125-21ksoftirqd/120:00:161
239502440,4sleep125-21ksoftirqd/120:00:151
3203599311,2cyclictest17623-21apache223:27:062
3203599260,26cyclictest0-21swapper/222:39:242
3203599240,24cyclictest0-21swapper/222:13:432
3203599240,23cyclictest0-21swapper/223:30:182
3203199240,1cyclictest28046-21cat23:45:251
3203799211,17cyclictest30934-21ssh22:50:123
3203599210,20cyclictest0-21swapper/200:30:292
3203199216,10cyclictest225-21systemd-journal22:30:001
3203599200,18cyclictest0-21swapper/221:29:382
3203599200,0cyclictest0-21swapper/221:20:102
32031992020,0cyclictest0-21swapper/121:00:141
3203199200,19cyclictest0-21swapper/121:50:051
32029992018,1cyclictest0-21swapper/022:48:290
3203799191,14cyclictest25165-21ssh23:25:123
32035991916,1cyclictest0-21swapper/221:17:182
3203199190,1cyclictest141rcu_preempt00:00:121
3202999197,6cyclictest7634-21perf23:15:010
3203599180,18cyclictest0-21swapper/221:56:302
3203199180,18cyclictest0-21swapper/123:20:241
3202999180,8cyclictest0-21swapper/022:35:150
32037991714,2cyclictest537-21snmpd21:16:273
32035991716,1cyclictest0-21swapper/200:18:022
3203599163,13cyclictest0-21swapper/219:29:132
32035991615,1cyclictest0-21swapper/222:06:332
3203599160,2cyclictest24615-21diskmemload23:36:272
3203199169,3cyclictest25-21ksoftirqd/123:20:001
32037991513,1cyclictest24615-21diskmemload22:35:153
3203799150,10cyclictest0-21swapper/321:30:103
32035991515,0cyclictest0-21swapper/223:06:032
3203599150,1cyclictest0-21swapper/222:28:142
3203599150,15cyclictest0-21swapper/219:59:022
3203599150,14cyclictest8731-21cut23:15:132
3203199155,4cyclictest455-21dbus-daemon00:09:591
3203199151,13cyclictest11257-21taskset19:32:181
3203199150,15cyclictest0-21swapper/122:09:351
3203199150,14cyclictest0-21swapper/123:35:391
32029991513,1cyclictest24615-21diskmemload22:03:200
3202999150,15cyclictest0-21swapper/000:23:570
320002150,0sleep00-21swapper/022:11:450
190652150,0sleep00-21swapper/023:01:470
3203799149,3cyclictest2353-21tr21:35:163
3203799146,4cyclictest225-21systemd-journal23:55:003
32037991413,1cyclictest0-21swapper/320:16:113
32037991412,1cyclictest0-21swapper/321:56:093
32037991410,3cyclictest14270-21wc22:20:213
32037991410,1cyclictest17753-21apache_accesses21:25:113
32035991413,1cyclictest0-21swapper/223:47:332
32035991413,1cyclictest0-21swapper/223:41:442
32035991413,1cyclictest0-21swapper/221:40:232
32035991413,1cyclictest0-21swapper/200:24:422
32035991412,2cyclictest32-21ksoftirqd/223:53:492
32035991412,1cyclictest19650-21cut00:00:172
32035991412,1cyclictest12924-21apache_volume20:45:122
32035991411,2cyclictest26417-21tr21:30:112
32035991411,2cyclictest18379-21grep20:55:162
3203599140,14cyclictest0-21swapper/200:39:452
32031991413,1cyclictest38150irq/131-snd_hda21:39:261
32031991410,3cyclictest545-21cut23:10:161
32031991410,2cyclictest8224-21tr19:25:221
3203199140,14cyclictest0-21swapper/122:12:071
3203199140,13cyclictest32356-21pmu-power22:50:211
3202999146,4cyclictest225-21systemd-journal21:16:280
3202999142,7cyclictest27830-21perf22:10:010
32029991410,3cyclictest28628-21tr00:05:230
32029991410,3cyclictest28178-21grep23:45:260
32029991410,2cyclictest26934-21sed21:30:140
32029991410,2cyclictest18614-21cut23:40:160
3202999140,1cyclictest0-21swapper/021:36:130
270242140,0sleep30-21swapper/323:06:273
3203799139,3cyclictest29539-21ssh22:30:033
3203799139,3cyclictest26652-21ls21:50:013
3203799139,3cyclictest19225-21grep19:50:143
3203799139,2cyclictest5096-21head22:15:163
3203799135,4cyclictest225-21systemd-journal23:00:013
3203799133,3cyclictest0-21swapper/300:00:173
3203799132,2cyclictest0-21swapper/321:50:203
32037991310,2cyclictest20581-21sh00:40:013
32037991310,2cyclictest17242-21grep23:00:263
32037991310,2cyclictest1569-21grep23:10:243
32037991310,2cyclictest11899-21tr00:15:143
32037991310,2cyclictest11687-21grep20:40:203
32035991313,0cyclictest0-21swapper/223:04:372
32035991312,1cyclictest3699-21/usr/sbin/munin00:10:132
32035991312,1cyclictest0-21swapper/221:46:032
32035991310,2cyclictest3961-21grep21:35:272
32035991310,2cyclictest29359-21tr00:25:182
32035991310,2cyclictest27598-21wc20:05:262
32035991310,2cyclictest2652-21grep20:20:262
32035991310,2cyclictest13849-21sh19:40:002
3203599130,12cyclictest0-21swapper/222:50:552
3203199139,3cyclictest6721-21cron19:25:001
3203199139,3cyclictest632-21cat23:30:001
3203199139,3cyclictest30068-21grep20:10:271
3203199139,3cyclictest27505-21wc20:05:261
3203199138,4cyclictest123322sleep122:58:071
3203199133,7cyclictest13685-21grep22:20:151
32031991311,1cyclictest25-21ksoftirqd/121:48:421
32031991311,1cyclictest25-21ksoftirqd/100:34:401
32031991310,2cyclictest18253-21cron21:45:011
32031991310,2cyclictest12252-21cat23:55:231
3203199130,12cyclictest8870-21cron21:19:591
3202999138,3cyclictest25358-21grep20:00:280
3202999138,3cyclictest25358-21grep20:00:270
3202999135,2cyclictest1142-21runrttasks21:14:420
3202999131,6cyclictest23433-21munin-run20:00:010
32029991311,1cyclictest0-21swapper/023:31:100
32029991311,1cyclictest0-21swapper/023:09:380
32029991310,2cyclictest8194-21grep20:35:160
32029991310,2cyclictest3196-21ssh00:10:110
32029991310,2cyclictest19930-21wc19:50:180
32029991310,2cyclictest12296-21grep21:40:260
32029991310,2cyclictest11146-21sort22:00:000
3202999130,7cyclictest0-21swapper/023:55:350
3203799129,2cyclictest3464-21tr00:10:123
3203799129,2cyclictest28810-21tr00:25:153
3203799129,2cyclictest2716-21df19:15:183
3203799129,2cyclictest23237-21grep21:05:163
3203799129,2cyclictest21797-21grep21:00:263
3203799129,2cyclictest17125-21cut19:45:133
3203799129,2cyclictest16088-21ssh23:20:003
3203799129,2cyclictest15999-21ssh22:40:253
3203799121,4cyclictest225-21systemd-journal23:44:413
3203799120,11cyclictest107052sleep300:33:463
3203799120,11cyclictest10367-21grep23:35:183
3203599129,2cyclictest8910-21head20:35:192
3203599129,2cyclictest19590-21grep19:50:162
3203599129,2cyclictest15615-21cat19:40:262
3203599129,2cyclictest14990-21sed22:40:162
3203599129,2cyclictest10827-21tr20:40:142
3203599124,2cyclictest225-21systemd-journal22:34:592
32035991211,1cyclictest0-21swapper/221:10:242
32035991210,1cyclictest13948-21rm22:00:472
3203599120,1cyclictest0-21swapper/223:14:142
3203199129,2cyclictest5630-21grep22:15:211
3203199129,2cyclictest4330-21cron19:19:591
3203199129,2cyclictest4061-21sh00:10:151
3203199129,2cyclictest3956-21sed21:55:181
3203199129,2cyclictest235182sleep123:05:041
3203199129,2cyclictest22208-21chrt00:20:371
3203199129,2cyclictest20598-21sed19:50:251
3203199129,2cyclictest18515-21cut20:55:161
3203199129,2cyclictest16101-21head23:00:181
3203199129,2cyclictest15574-21tr20:50:141
3203199129,2cyclictest13809-21uname00:35:181
3203199129,2cyclictest13303-21wc22:00:271
3203199129,2cyclictest11958-21wc00:15:141
3203199127,4cyclictest25-21ksoftirqd/123:52:111
32031991210,1cyclictest25-21ksoftirqd/122:31:481
3203199120,2cyclictest0-21swapper/100:28:141
3202999129,2cyclictest7705-21grep19:25:170
3202999129,2cyclictest3217-21ls20:25:000
3202999129,2cyclictest19526-21cut21:45:150
3202999129,2cyclictest19198-21tr00:00:140
3202999129,2cyclictest16735-21tr19:45:120
3202999129,2cyclictest12302-21sort22:20:010
3202999129,2cyclictest10375-21sed23:35:180
3202999128,3cyclictest14096-21tr22:20:190
3202999126,5cyclictest12285-21munin-run00:35:010
3202999121,9cyclictest16384-21apt-config23:20:020
3203799118,2cyclictest7884-21cat19:25:183
3203799118,2cyclictest4951-21expr20:25:243
3203799118,2cyclictest31488-21ssh22:11:223
3203799118,2cyclictest28198-21ls20:10:003
3203799118,2cyclictest27608-21sed23:45:213
3203799118,2cyclictest26062-21basename21:10:183
3203799118,2cyclictest25897-21sort20:05:013
3203799118,2cyclictest25897-21sort20:05:003
3203799118,2cyclictest24792-21ssh22:45:373
3203799118,2cyclictest21222-21ssh22:25:083
3203799118,2cyclictest20589-21ssh00:20:153
3203799118,2cyclictest20168-21cat22:05:133
3203799118,2cyclictest19635-21ssh23:21:053
3203799118,2cyclictest18015-21tr20:55:153
3203799118,2cyclictest15345-21basename20:50:133
3203799118,2cyclictest14917-21grep19:40:173
3203799118,2cyclictest12114-21ls21:40:243
3203799118,2cyclictest1188-21aten2_r7power_p23:30:143
3203799118,2cyclictest11715-21cut23:55:173
3203799118,2cyclictest11602-21sh22:00:153
3203799118,2cyclictest11265-21chrt19:32:253
3203799118,2cyclictest10646-21ssh21:20:203
3203799115,2cyclictest225-21systemd-journal19:22:363
3203799114,6cyclictest17623-21apache200:06:233
3203599119,1cyclictest24312-21grep20:00:182
3203599119,1cyclictest24312-21grep20:00:172
3203599119,1cyclictest14768-21sed22:20:242
3203599118,2cyclictest7663-21head22:55:152
3203599118,2cyclictest27126-21ssh00:05:092
3203599118,2cyclictest23920-21head22:45:202
3203599118,2cyclictest20889-21grep21:00:182
3203599118,2cyclictest11660-21cat23:55:172
3203599117,3cyclictest18691-21sed23:20:272
3203599114,1cyclictest225-21systemd-journal20:27:382
3203599113,2cyclictest0-21swapper/220:15:132
3203599110,7cyclictest28385-21needreboot21:50:202
3203199118,2cyclictest7452-21sed20:30:231
3203199118,2cyclictest6946-21grep22:35:211
3203199118,2cyclictest4800-21taskset20:25:231
3203199118,2cyclictest3830-21ssh23:31:011
3203199118,2cyclictest24872-21ssh21:10:021
3203199118,2cyclictest18616-21sh23:40:171
3203199118,2cyclictest1638-21seq19:11:071
3203199118,2cyclictest15425-21tr22:40:191
3203199118,2cyclictest13600-21sort20:45:161
3203199117,3cyclictest537-21snmpd21:34:301
3203199117,3cyclictest23717-21ssh21:28:311
3203199115,4cyclictest22920-21cat22:45:131
3203199110,10cyclictest0-21swapper/119:55:121
3202999118,2cyclictest7910-21ssh23:15:040
3202999118,2cyclictest6658-21ssh22:55:030
3202999118,2cyclictest6544-21cut20:30:140
3202999118,2cyclictest5125-21grep20:25:260
3202999118,2cyclictest31699-21cut22:50:160
3202999118,2cyclictest30596-21cut22:30:160
3202999118,2cyclictest29779-21fgrep20:10:220
3202999118,2cyclictest28588-21grep00:25:140
3202999118,2cyclictest2851-21cut19:15:180
3202999118,2cyclictest26301-21sh23:25:170
3202999118,2cyclictest2478-21tr23:50:140
3202999118,2cyclictest22182-21sed22:25:180
3202999118,2cyclictest19431-21head20:55:250
3202999118,2cyclictest18568-21sh00:38:250
3202999118,2cyclictest16095-21taskset19:43:130
3202999118,2cyclictest13362-21ssh21:21:570
3202999118,2cyclictest13181-21grep20:45:130
3202999118,2cyclictest1292-21wc19:10:300
3202999118,2cyclictest12290-21chrt19:35:150
3202999118,2cyclictest11668-21ssh00:15:120
3202999118,2cyclictest11257-21grep20:40:160
3202999117,3cyclictest22227-21cron21:05:000
3202999113,4cyclictest225-21systemd-journal21:51:360
3203799108,1cyclictest322242sleep319:10:043
3203799108,1cyclictest28903-21cat20:10:163
3203799108,1cyclictest21404-21grep19:55:123
3203799108,1cyclictest1166-21wget20:20:143
3203799107,2cyclictest7595-21chrt20:31:183
3203799107,2cyclictest13054-21cut20:45:143
3203599108,2cyclictest9843-21expr19:30:132
3203599108,2cyclictest30249-21taskset20:11:022
3203599108,2cyclictest23317-21ls21:05:172
3203599108,1cyclictest7845-21ssh22:16:402
3203599108,1cyclictest18202-21grep19:45:232
3203599108,1cyclictest1531-21grep19:10:302
3203199108,1cyclictest14139-21apt19:40:121
3203199108,1cyclictest12366-21taskset20:41:211
3203199107,2cyclictest8627-21expr20:35:171
3203199107,2cyclictest31333-21grep20:15:151
3203199107,2cyclictest18623-21taskset19:49:281
3203199107,2cyclictest13348-21grep19:35:241
3203199100,9cyclictest24506-21taskset21:08:481
3202999108,1cyclictest64212sleep019:21:080
3202999108,1cyclictest27448-21cat20:05:230
3202999108,1cyclictest23473-21sed21:05:180
3202999108,1cyclictest13982-21seq22:40:060
3202999107,2cyclictest16515-21sed20:50:220
320379997,1cyclictest8427-21cut20:35:173
320379994,2cyclictest452-21systemd-logind19:35:213
320359998,1cyclictest6160-21expr20:30:122
320359998,1cyclictest2193-21taskset19:15:102
320359997,2cyclictest5316-21grep19:20:172
320359997,2cyclictest15547-21expr20:50:142
320319997,1cyclictest1962-21cut20:20:181
320299996,2cyclictest31630-21dnf20:15:160
320299995,3cyclictest9371-21apt-get19:30:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional