You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 20:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Dec 22, 2025 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2612629979,15sleep20-21swapper/207:09:272
2593329066,8sleep00-21swapper/007:06:560
2600628366,13sleep10-21swapper/107:07:531
2599828168,9sleep30-21swapper/307:07:473
2625599322,2cyclictest8291-21idleruntime-cro07:40:002
2624399250,1cyclictest0-21swapper/007:10:150
26247992119,1cyclictest0-21swapper/112:25:141
2625599172,14cyclictest32601-21apt-get10:45:142
2624399173,11cyclictest13380-21apache_processe07:50:130
2624799163,12cyclictest0-21swapper/111:20:141
26247991614,1cyclictest0-21swapper/111:40:141
2624399161,9cyclictest0-21swapper/007:40:150
26243991610,2cyclictest25293-21wc11:35:230
2625799153,8cyclictest9525-21apt-get08:50:153
2625599151,11cyclictest0-21swapper/208:20:152
26247991513,1cyclictest0-21swapper/112:00:201
2624399159,2cyclictest13566-21grep08:55:270
26257991410,3cyclictest32263-21wc09:35:183
2625799140,13cyclictest11614-21grep11:05:283
26255991413,1cyclictest0-21swapper/208:35:292
26247991412,1cyclictest13682-21cstates07:50:151
26247991410,2cyclictest6430-21tr07:35:121
2624399141,3cyclictest0-21swapper/010:35:130
2625799139,3cyclictest747-21sed08:30:173
2625799139,2cyclictest12623-21tr11:10:163
2625799138,4cyclictest31519-21wc10:40:273
2625799138,3cyclictest19844-21grep08:00:263
2625799134,5cyclictest225-21systemd-journal08:20:003
26257991310,2cyclictest31588-21wc08:25:263
26257991310,2cyclictest31588-21wc08:25:253
26257991310,2cyclictest24286-21tail10:25:293
26257991310,2cyclictest22645-21awk08:10:013
26257991310,2cyclictest19208-21grep12:35:163
26257991310,2cyclictest12268-21wc07:45:233
2625599138,2cyclictest455-21dbus-daemon07:19:592
2625599132,8cyclictest7032-21idleruntime-cro10:59:592
26255991310,2cyclictest761-21sed08:30:172
26255991310,2cyclictest6009-21tr08:40:242
26255991310,2cyclictest22163-21tail08:05:282
26255991310,2cyclictest1171-21basename07:24:592
2624799139,3cyclictest30318-21grep08:25:141
2624799139,3cyclictest30318-21grep08:25:141
2624799137,5cyclictest16995-21grep09:05:141
26247991311,1cyclictest208802sleep108:05:151
26247991310,2cyclictest7912-21grep08:45:171
26247991310,2cyclictest4910-21grep07:30:201
26247991310,2cyclictest3782-21tr10:50:211
26247991310,2cyclictest24996-21grep11:35:191
2624799130,12cyclictest31207-21perf07:20:001
2624799130,10cyclictest0-21swapper/108:55:161
26243991310,2cyclictest31075-21cat08:25:180
26243991310,2cyclictest31075-21cat08:25:180
2625799129,2cyclictest9939-21tail09:55:273
2625799129,2cyclictest8561-21wc11:00:203
2625799129,2cyclictest4813-21wc09:45:233
2625799129,2cyclictest4463-21grep12:00:183
2625799129,2cyclictest31145-21wc11:50:013
2625799129,2cyclictest29299-21sed08:20:283
2625799129,2cyclictest29112-21tr09:30:153
2625799129,2cyclictest26125-21grep10:30:203
2625799129,2cyclictest22404-21uname09:15:193
2625799129,2cyclictest19745-21tr11:25:143
2625799129,2cyclictest13634-21grep12:20:243
2625799128,3cyclictest1894-21tail09:40:183
2625799128,2cyclictest21974-21head10:20:293
2625799128,2cyclictest21974-21head10:20:293
2625799123,2cyclictest225-21systemd-journal08:00:003
2625799121,5cyclictest225-21systemd-journal10:59:593
2625599129,2cyclictest32649-21wc09:35:242
2625599129,2cyclictest29758-21sed11:45:202
2625599129,2cyclictest28148-21tail07:10:272
2625599129,2cyclictest2651-21ls07:25:232
2625599129,2cyclictest25492-21wc11:35:242
2625599129,2cyclictest20868-21cut11:25:222
2625599129,2cyclictest18378-21cut11:20:212
2625599129,2cyclictest16475-21sleep12:30:012
2625599129,2cyclictest14119-21wc10:05:202
2625599121,2cyclictest0-21swapper/210:25:172
26255991210,1cyclictest31664-21tr11:50:152
26255991210,1cyclictest11701-21sh12:20:002
2624799129,2cyclictest5666-21turbostat.cron09:50:001
2624799129,2cyclictest3489-21uname08:35:221
2624799129,2cyclictest25309-21tr08:15:151
2624799129,2cyclictest25256-21tr10:30:141
2624799129,2cyclictest20312-21cut10:20:161
2624799129,2cyclictest20312-21cut10:20:161
2624799129,2cyclictest11685-21cron12:20:001
2624799129,2cyclictest10622-21grep11:05:191
2624799128,3cyclictest7405-21grep12:10:151
2624799128,3cyclictest2097-21grep11:55:171
2624799128,3cyclictest19510-21basename11:25:131
26247991210,1cyclictest31143-21cat11:50:011
26247991210,1cyclictest23089-21grep09:15:261
26247991210,1cyclictest15488-21head11:15:181
2624799120,11cyclictest9784-21wc09:55:271
2624399129,2cyclictest8199-21cat08:45:210
2624399129,2cyclictest3744-21ls10:50:210
2624399129,2cyclictest3005-21grep08:35:190
2624399129,2cyclictest28872-21sed08:20:220
2624399129,2cyclictest21108-21sed11:25:250
2624399129,2cyclictest19798-21grep09:10:180
2624399129,2cyclictest19239-21basename11:25:010
2624399129,2cyclictest17391-21sed07:55:270
2624399129,2cyclictest15092-21grep09:00:170
2624399129,2cyclictest11706-21cat12:20:010
2624399128,3cyclictest10866-21cat11:05:200
2624399126,2cyclictest225-21systemd-journal10:35:000
2624399125,3cyclictest225-21systemd-journal07:50:000
2625799119,1cyclictest31438-21taskset07:20:063
2625799118,2cyclictest7835-21sed07:35:243
2625799118,2cyclictest7766-21taskset09:51:403
2625799118,2cyclictest27130-21cut09:25:183
2625799118,2cyclictest2682-21grep11:55:233
2625799118,2cyclictest25773-21sed09:20:273
2625799118,2cyclictest23183-21tr08:10:153
2625799118,2cyclictest22084-21tr11:30:153
2625799118,2cyclictest19344-21tr09:10:153
2625799118,2cyclictest17211-21chrt12:30:173
2625799118,2cyclictest15430-21tail11:15:173
2625799118,2cyclictest15142-21taskset07:50:403
2625799118,2cyclictest14806-21ls09:00:153
2625799118,2cyclictest1405-21tr10:45:233
2625799118,2cyclictest13038-21grep08:55:223
2625799118,2cyclictest11430-21cut10:00:183
2625799117,2cyclictest6842-21sort12:10:003
2625799110,10cyclictest24490-21cut11:35:173
2625599119,2cyclictest10711-21wc07:45:012
2625599119,1cyclictest30788-21cut10:40:182
2625599119,1cyclictest27413-21cut11:40:182
2625599119,1cyclictest23588-21munin-plugin-st09:19:592
2625599119,1cyclictest23504-21tail11:30:252
2625599119,1cyclictest17496-21tail12:30:192
2625599119,1cyclictest16209-21grep10:10:162
2625599119,1cyclictest13059-21grep11:10:192
2625599118,3cyclictest32-21ksoftirqd/210:35:122
2625599118,2cyclictest9364-21sort09:55:212
2625599118,2cyclictest7815-21tr11:00:152
2625599118,2cyclictest7427-21expr12:10:152
2625599118,2cyclictest5721-21sed09:50:002
2625599118,2cyclictest4911-21grep07:30:202
2625599118,2cyclictest30752-21grep08:25:162
2625599118,2cyclictest30752-21grep08:25:162
2625599118,2cyclictest29606-21sed09:30:192
2625599118,2cyclictest2656-21grep10:50:142
2625599118,2cyclictest26020-21sed08:15:182
2625599118,2cyclictest2067-21grep11:55:172
2625599118,2cyclictest20248-21cron08:05:002
2625599118,2cyclictest19327-21tr12:35:162
2625599118,2cyclictest18900-21wc10:15:202
2625599118,2cyclictest12824-21taskset07:46:232
2625599118,2cyclictest1276-21basename09:40:152
2625599118,2cyclictest11843-21tail10:00:202
2625599113,4cyclictest225-21systemd-journal08:50:002
2625599110,4cyclictest225-21systemd-journal12:20:192
2624799119,1cyclictest31437-21aten2_r7power_e11:50:141
2624799118,2cyclictest654-21df10:45:161
2624799118,2cyclictest6408-21taskset12:06:431
2624799118,2cyclictest26739-21grep09:25:161
2624799118,2cyclictest24743-21cron10:30:011
2624799118,2cyclictest23487-21cat08:10:171
2624799118,2cyclictest20706-21unixbench-2d12:35:271
2624799118,2cyclictest18472-21expr08:00:151
2624799118,2cyclictest13104-21cron07:50:001
2624799118,2cyclictest12789-21chrt10:04:111
2624799117,3cyclictest6680-21head09:50:181
2624799117,3cyclictest31174-21awk10:40:231
2624799117,3cyclictest17511-21head12:30:191
2624799116,4cyclictest13211-21cut10:05:161
2624799112,1cyclictest0-21swapper/108:20:131
2624799111,2cyclictest225-21systemd-journal08:00:011
2624799110,10cyclictest32300-21cat09:35:181
2624399119,1cyclictest8851-21sed12:10:270
2624399119,1cyclictest6249-21tr07:35:120
2624399119,1cyclictest454-21grep09:35:240
2624399119,1cyclictest4508-21grep12:00:190
2624399119,1cyclictest2201-21head07:25:170
2624399119,1cyclictest1803-21grep10:45:270
2624399119,1cyclictest16604-21strings10:10:180
2624399118,2cyclictest8516-21cat11:00:190
2624399118,2cyclictest3834-21cut07:30:150
2624399118,2cyclictest29995-21cat11:45:210
2624399118,2cyclictest28029-21cat09:25:280
2624399118,2cyclictest24007-21cut08:10:220
2624399118,2cyclictest23749-21grep10:25:210
2624399118,2cyclictest20540-21hostname08:05:130
2624399118,2cyclictest18442-21sed12:30:290
2624399118,2cyclictest16433-21grep11:15:270
2624399118,2cyclictest14026-21grep11:10:280
2624399112,6cyclictest21861-21apache_volume11:30:120
2624399111,4cyclictest27029-21sshd08:15:320
2624399111,4cyclictest12785-21sshd10:04:090
2624399110,10cyclictest18103-21chrt08:00:010
2625799108,1cyclictest9129-21sed07:40:163
2625799108,1cyclictest67302sleep308:43:163
2625799108,1cyclictest5917-21munin-run07:34:593
2625799108,1cyclictest28668-21wc10:35:203
2625799108,1cyclictest285972sleep311:43:173
2625799108,1cyclictest28300-21chrt07:11:203
2625799108,1cyclictest2510-21chrt10:50:053
2625799108,1cyclictest17935-21grep09:05:213
2625799108,1cyclictest17362-21taskset10:11:583
2625799108,1cyclictest14446-21grep12:25:153
2625799107,2cyclictest8850-21head12:10:283
2625799107,2cyclictest7744-21cut08:45:163
2625799107,2cyclictest4035-21tail08:35:283
2625799107,2cyclictest30892-21chrt07:16:203
2625799107,2cyclictest17758-21cut11:20:163
2625799107,2cyclictest13361-21tr10:05:153
2625799107,2cyclictest11689-21cron12:20:003
2625599108,1cyclictest7139-21cut09:50:222
2625599108,1cyclictest6304-21chrt12:05:242
2625599108,1cyclictest27106-21cut09:25:182
2625599108,1cyclictest22970-21cut08:10:152
2625599107,2cyclictest9978-21wget11:05:162
2625599107,2cyclictest4436-21tr12:00:182
2625599107,2cyclictest24043-21taskset09:20:082
2625599107,2cyclictest19284-21grep09:10:152
2625599107,2cyclictest16150-21expr07:55:142
2625599107,2cyclictest15825-21wc09:00:262
2625599107,2cyclictest13651-21tr07:50:152
2625599107,2cyclictest12404-21expr08:55:172
2625599106,3cyclictest32-21ksoftirqd/210:20:232
2625599106,3cyclictest32-21ksoftirqd/210:20:232
2625599103,5cyclictest16724-21apt-get09:05:132
2625599100,9cyclictest25954-21grep10:30:192
2624799109,1cyclictest0-21swapper/108:40:261
2624799108,1cyclictest7416-21dpkg11:00:001
2624799108,1cyclictest28337-21taskset07:11:481
2624799108,1cyclictest27436-21apt-config10:35:001
2624799108,1cyclictest19886-21sh09:10:181
2624799108,1cyclictest18024-21wc10:15:141
2624799108,1cyclictest17357-21taskset10:11:541
2624799108,1cyclictest12220-21expr12:20:131
2624799107,2cyclictest32189-21tail07:20:181
2624799107,2cyclictest28905-21grep09:30:141
2624799107,2cyclictest2645-21cat09:40:261
2624799107,2cyclictest22293-21expr11:30:141
2624799107,2cyclictest1722-21grep07:25:151
2624799107,2cyclictest12533-21grep11:10:161
2624799105,4cyclictest8636-21cut07:40:151
2624399108,2cyclictest6757-21taskset10:56:380
2624399108,2cyclictest23358-21chrt09:17:180
2624399108,1cyclictest67372sleep008:43:200
2624399108,1cyclictest31755-21expr07:20:160
2624399108,1cyclictest30361-21expr10:40:170
2624399108,1cyclictest28974-21chmod07:15:010
2624399108,1cyclictest19106-21apache_volume12:35:140
2624399108,1cyclictest18499-21grep10:15:160
2624399108,1cyclictest1621-21cut11:55:150
2624399108,1cyclictest12402-21expr12:20:130
2624399107,2cyclictest6365-21expr09:50:150
2624399107,2cyclictest5667-21dump-pmu-power09:50:000
2624399107,2cyclictest29277-21expr09:30:150
2624399107,2cyclictest22372-21cron10:25:010
2624399107,2cyclictest22372-21cron10:25:000
2624399107,2cyclictest18940-21cat09:09:590
2624399105,2cyclictest273-21systemd-udevd09:57:030
2624399100,9cyclictest28416-21chrt11:40:580
2624399100,9cyclictest14337-21cat12:25:120
262579997,1cyclictest32317-21sed11:50:193
262579997,1cyclictest18084-21grep10:15:153
262579996,2cyclictest3311-21taskset07:27:123
262559997,1cyclictest146732sleep211:15:042
262559997,1cyclictest11583-21taskset08:53:532
262479997,1cyclictest915-21uname08:30:201
262479997,1cyclictest6716-21seq10:56:061
262479997,1cyclictest254662sleep109:20:231
262479997,1cyclictest14553-21grep09:00:151
262479996,2cyclictest9917-21cat08:50:171
262439998,1cyclictest25477-21smartctl09:20:250
262439997,1cyclictest9646-21basename08:50:160
262439997,1cyclictest6692-21awk12:10:000
262439997,1cyclictest31368-21basename11:50:120
262439997,1cyclictest1295-21aten2_r7power_v09:40:150
262439996,2cyclictest32466-21awk08:30:150
262439996,2cyclictest13729-21grep10:05:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional