You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 14:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 30, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4749210152,45sleep00-21swapper/019:05:480
477129649,42sleep20-21swapper/219:06:072
492428570,10sleep30-21swapper/319:08:053
488628063,13sleep10-21swapper/119:07:351
516099264,22cyclictest0-21swapper/000:26:550
516099259,16cyclictest0-21swapper/021:40:530
5160992521,3cyclictest30382-21diskmemload21:28:330
516299240,24cyclictest0-21swapper/122:36:191
5160992422,1cyclictest9012-21apt-get21:55:130
5160992419,5cyclictest0-21swapper/000:13:150
516099240,0cyclictest0-21swapper/022:59:130
5167992320,3cyclictest8813-21kworker/3:000:14:223
5160992313,8cyclictest0-21swapper/000:16:080
5167992220,2cyclictest8813-21kworker/3:000:39:233
5167992219,3cyclictest8813-21kworker/3:000:33:423
516299221,3cyclictest19218-21wc22:00:281
5160992215,6cyclictest30382-21diskmemload00:34:060
516799210,1cyclictest0-21swapper/322:16:263
516799210,17cyclictest0-21swapper/322:55:143
516599211,19cyclictest0-21swapper/220:15:132
516599210,20cyclictest0-21swapper/221:27:402
5162992120,0cyclictest0-21swapper/123:13:111
5162992119,1cyclictest0-21swapper/121:20:211
516299210,20cyclictest0-21swapper/123:19:191
516299210,20cyclictest0-21swapper/123:19:191
516299210,20cyclictest0-21swapper/121:05:141
516099215,11cyclictest0-21swapper/000:22:000
5160992117,2cyclictest30382-21diskmemload23:45:170
5160992116,4cyclictest0-21swapper/022:40:190
5160992113,5cyclictest30382-21diskmemload22:28:210
5160992112,9cyclictest0-21swapper/000:00:130
5160992112,7cyclictest30382-21diskmemload23:11:000
5167992019,1cyclictest0-21swapper/322:45:533
516099206,7cyclictest19329-21munin-run22:40:000
5160992017,2cyclictest6317-21awk20:20:000
5160992013,5cyclictest0-21swapper/021:36:450
5160992013,5cyclictest0-21swapper/021:36:440
5167991917,2cyclictest8813-21kworker/3:000:18:233
516599190,19cyclictest0-21swapper/222:46:592
516299192,1cyclictest141rcu_preempt22:55:141
5160991917,0cyclictest30382-21diskmemload22:12:130
5160991913,5cyclictest23172-1kworker/u9:223:05:200
5160991912,6cyclictest0-21swapper/022:09:220
5160991910,8cyclictest0-21swapper/023:41:510
5160991910,8cyclictest0-21swapper/019:45:190
5160991910,8cyclictest0-21swapper/019:45:180
516599180,0cyclictest0-21swapper/223:37:422
5162991818,0cyclictest0-21swapper/122:19:061
5162991814,1cyclictest0-21swapper/123:20:521
516299180,2cyclictest0-21swapper/121:48:251
516299180,1cyclictest0-21swapper/122:05:051
516099189,7cyclictest0-21swapper/023:04:020
516099189,7cyclictest0-21swapper/000:05:590
516099187,10cyclictest575-21sh21:50:120
5160991816,2cyclictest0-21swapper/021:19:310
5160991814,3cyclictest30382-21diskmemload23:39:250
5160991814,3cyclictest30382-21diskmemload23:24:120
5160991814,1cyclictest0-21swapper/023:31:460
5160991813,4cyclictest9981-21awk19:20:000
5160991811,4cyclictest30382-21diskmemload22:45:200
5167991715,2cyclictest8813-21kworker/3:000:26:133
5167991715,2cyclictest24860-21kworker/3:122:22:433
516599170,1cyclictest0-21swapper/222:37:222
516299173,2cyclictest141rcu_preempt22:52:221
516099178,7cyclictest14981-21pmu-power20:35:200
5160991712,4cyclictest19151-21fschecks_count20:45:170
242572170,0sleep20-21swapper/222:04:142
5167991614,2cyclictest8813-21kworker/3:000:22:523
516799160,1cyclictest0-21swapper/323:04:183
516599169,4cyclictest32-21ksoftirqd/220:00:012
516599162,9cyclictest10158-21apt-get22:15:142
516599162,2cyclictest141rcu_preempt22:55:532
5165991615,1cyclictest0-21swapper/223:40:292
5162991615,1cyclictest0-21swapper/123:56:361
516099168,3cyclictest0-21swapper/020:40:110
516099165,10cyclictest1-21systemd00:40:000
516099162,1cyclictest0-21swapper/022:32:010
5160991614,1cyclictest30382-21diskmemload23:59:420
5160991613,2cyclictest30382-21diskmemload22:16:050
516799154,8cyclictest16157-21apt-get20:40:123
5167991512,3cyclictest39-21ksoftirqd/321:19:013
516599154,8cyclictest3589-21sh23:46:512
516599150,15cyclictest0-21swapper/221:59:292
516599150,14cyclictest0-21swapper/223:50:462
516599150,13cyclictest0-21swapper/200:22:332
5162991514,0cyclictest0-21swapper/120:00:121
5162991512,2cyclictest0-21swapper/122:35:001
5162991511,3cyclictest30318-21cut23:05:161
516099158,6cyclictest0-21swapper/023:53:500
516099155,5cyclictest985-21timerwakeupswit20:05:250
516099154,10cyclictest0-21swapper/021:30:140
516099153,6cyclictest3933-21awk20:15:010
5160991514,1cyclictest0-21swapper/022:51:330
5160991512,2cyclictest0-21swapper/021:46:530
193002150,1sleep232-21ksoftirqd/223:56:022
516799146,7cyclictest9746-21apache222:09:073
516799140,1cyclictest0-21swapper/322:34:233
516599149,2cyclictest23005-21sh20:54:592
5165991410,3cyclictest19580-21ls00:35:152
516599140,13cyclictest14660-21sh21:20:012
516599140,12cyclictest0-21swapper/200:29:302
516299142,8cyclictest10635-21/usr/sbin/munin00:30:241
5162991413,1cyclictest0-21swapper/120:17:311
5162991411,2cyclictest20437-21chrt23:37:451
5162991410,1cyclictest0-21swapper/123:28:011
516299140,13cyclictest8836-31find00:10:121
516299140,13cyclictest0-21swapper/123:40:151
516099149,4cyclictest24131-21fschecks_time20:55:150
516099149,4cyclictest19470-21sleep19:40:000
516099147,7cyclictest0-21swapper/021:13:390
516099146,8cyclictest0-21swapper/022:04:110
5160991411,2cyclictest19679-21wc22:20:190
5160991410,3cyclictest10450-21wc20:25:410
516099140,11cyclictest0-21swapper/023:26:320
516799139,3cyclictest23211-21grep19:45:203
516799139,3cyclictest23211-21grep19:45:193
516799134,5cyclictest26163-21grep19:50:253
5167991310,2cyclictest27588-21grep21:00:263
5167991310,2cyclictest25321-21ssh21:25:263
5167991310,2cyclictest2018-21cat00:05:213
516799130,12cyclictest6585-21head23:10:163
516799130,12cyclictest29002-21grep21:05:183
516799130,11cyclictest29976-21head20:00:153
516599139,3cyclictest17389-21head20:40:192
516599139,3cyclictest0-21swapper/222:44:482
5165991311,2cyclictest32-21ksoftirqd/221:42:492
5165991310,2cyclictest9956-21grep21:35:272
5165991310,2cyclictest9956-21grep21:35:262
5165991310,2cyclictest5003-21sed22:50:172
5165991310,2cyclictest24594-21cut19:50:132
5165991310,2cyclictest2435-21sort22:10:172
5165991310,2cyclictest15141-21cat23:35:012
5165991310,2cyclictest13308-21grep19:25:162
516299139,2cyclictest2533-21wc22:30:011
516299131,1cyclictest258062rm00:39:051
5162991311,1cyclictest30696-21ssh00:22:161
5162991310,2cyclictest24225-21tail20:55:161
5162991310,2cyclictest1286-21expr00:05:171
516299130,12cyclictest18608-21wc19:35:201
516099139,3cyclictest16392-21wc19:30:270
516099137,6cyclictest0-21swapper/021:10:000
516099132,3cyclictest5477-21aten2_r7power_p19:10:140
5160991310,2cyclictest30954-21grep20:00:250
5160991310,2cyclictest14961-21cut23:15:180
5160991310,2cyclictest14961-21cut23:15:170
5160991310,2cyclictest11455-21grep20:30:130
516099130,11cyclictest0-21swapper/021:00:130
516799129,2cyclictest518-21grep23:25:273
516799129,2cyclictest4725-21ssh22:50:153
516799129,2cyclictest26597-21grep21:45:283
516799129,2cyclictest24675-21grep23:40:183
516799129,2cyclictest22089-21cron23:20:013
516799129,2cyclictest22089-21cron23:20:003
516799129,2cyclictest21364-21grep22:40:233
516799129,2cyclictest18123-21grep19:35:163
516799129,2cyclictest17816-21wc21:40:223
516799129,2cyclictest1732-21sed23:45:263
516799129,2cyclictest15831-21cron21:40:013
516799129,2cyclictest15831-21cron21:40:003
516799129,2cyclictest15493-21tr21:20:143
516799129,2cyclictest12329-21cron19:25:003
516799128,3cyclictest26970-21grep22:25:133
516799128,3cyclictest11446-21ssh22:35:113
516799125,4cyclictest9950-21munin-run19:20:013
516799120,12cyclictest0-21swapper/323:08:123
516799120,10cyclictest0-21swapper/323:50:133
516599129,2cyclictest9659-21tr00:10:122
516599129,2cyclictest31966-21chrt22:08:422
516599129,2cyclictest29377-21taskset23:05:022
516599129,2cyclictest28755-21grep22:25:262
516599129,2cyclictest26056-21ls00:00:192
516599129,2cyclictest2557-21head20:10:162
516599129,2cyclictest24518-21wc20:55:182
516599129,2cyclictest216552sleep200:16:462
516599129,2cyclictest14562-21tr23:15:162
516599129,2cyclictest14562-21tr23:15:152
516599129,2cyclictest12607-21grep20:30:212
516599128,3cyclictest7629-21cat23:10:252
516599128,3cyclictest2961-21sshd22:30:092
516599128,2cyclictest23371-21grep23:20:182
516299129,2cyclictest9605-21df21:55:151
516299129,2cyclictest8885-21sort23:50:151
516299129,2cyclictest3932-21seq22:10:371
516299129,2cyclictest24130-21cron21:45:001
516299129,2cyclictest23000-21cron20:54:591
516299129,2cyclictest13316-21head19:25:161
516299129,1cyclictest0-21swapper/122:20:521
516299128,2cyclictest2107-21ssh21:50:231
516299128,2cyclictest10461-21grep19:20:141
516299121,1cyclictest0-21swapper/119:40:141
516299121,10cyclictest30382-21diskmemload22:40:281
516299120,3cyclictest0-21swapper/121:19:181
516099128,3cyclictest12659-21basename19:25:130
516099128,3cyclictest0-21swapper/020:24:590
516099126,2cyclictest30382-21diskmemload21:20:040
516799118,2cyclictest9760-21sed20:25:163
516799118,2cyclictest9486-21expr21:55:163
516799118,2cyclictest9346-21ssh23:30:423
516799118,2cyclictest567-21rm21:50:113
516799118,2cyclictest27511-21ssh00:00:523
516799118,2cyclictest24946-21cat20:55:243
516799118,2cyclictest24430-21cat23:20:263
516799118,2cyclictest2211-21ssh21:30:553
516799118,2cyclictest21788-21ssh23:57:583
516799118,2cyclictest21379-21sed19:40:263
516799118,2cyclictest18573-21load22:00:193
516799118,2cyclictest1823-21grep22:10:153
516799118,2cyclictest17096-21basename19:35:013
516799118,2cyclictest1621-21ssh21:11:203
516799118,2cyclictest13334-21head19:25:163
516799117,3cyclictest17405-21grep23:35:263
516599118,2cyclictest9767-21sed20:25:162
516599118,2cyclictest8916-21ntpq19:15:212
516599118,2cyclictest737-21cut21:50:132
516599118,2cyclictest3274-21ssh00:05:432
516599118,2cyclictest31844-21ssh21:30:062
516599118,2cyclictest31241-21taskset20:03:282
516599118,2cyclictest25207-21expr21:45:172
516599118,2cyclictest23368-21grep19:45:222
516599118,2cyclictest23368-21grep19:45:212
516599118,2cyclictest20645-21apt20:50:002
516599118,2cyclictest17712-21rm21:20:512
516599118,2cyclictest10946-21expr00:30:152
516599118,2cyclictest1069-21sh21:10:542
516599118,1cyclictest55850irq/129-enp2s0-22:20:112
516599117,2cyclictest13994-21tr20:35:132
516599110,9cyclictest0-21swapper/223:00:122
516599110,10cyclictest5100-21ssh23:28:552
516299119,1cyclictest38150irq/131-snd_hda19:15:201
516299118,2cyclictest7490-21cron19:15:001
516299118,2cyclictest6921-21ssh23:30:101
516299118,2cyclictest5051-21ssh21:14:021
516299118,2cyclictest27033-21ssh21:26:391
516299118,2cyclictest25426-21expr00:00:141
516299118,2cyclictest2494-21ssh22:49:081
516299118,2cyclictest23804-21grep19:45:271
516299118,2cyclictest23804-21grep19:45:261
516299118,2cyclictest1938-21ssh23:45:331
516299118,2cyclictest1926-21ssh21:30:421
516299118,2cyclictest15373-21grep20:35:241
516299118,2cyclictest10976-21ssh21:36:131
516299118,2cyclictest10976-21ssh21:36:121
516299114,5cyclictest18898-21idleruntime00:15:191
516299111,9cyclictest246482sleep123:01:431
516099118,2cyclictest21449-21df20:50:150
516099114,4cyclictest225-21systemd-journal20:00:000
516099114,4cyclictest225-21systemd-journal19:55:000
516099114,3cyclictest225-21systemd-journal19:45:010
516099110,10cyclictest12177-21chrt19:23:080
516799108,1cyclictest4582-21expr20:15:143
516799108,1cyclictest3589-21taskset20:11:003
516799107,2cyclictest31807-21dpkg20:05:033
516799107,2cyclictest21663-21grep20:50:163
516799107,2cyclictest18919-21cat20:45:153
516799107,2cyclictest14467-21head20:35:143
516799106,3cyclictest8687-21dump-pmu-power20:25:003
516799100,9cyclictest27443-21df19:55:153
516799100,9cyclictest132852sleep320:32:273
516599108,1cyclictest7976-21python20:20:252
516599108,1cyclictest31954-21wget20:05:142
516599108,1cyclictest26180-21expr21:00:142
516599108,1cyclictest17443-21cut19:35:132
516599107,2cyclictest5415-21basename19:10:122
516599107,2cyclictest15067-21basename19:30:162
516599107,2cyclictest10669-21ls19:20:152
516599100,9cyclictest20116-21expr19:40:152
516299108,1cyclictest4340-21ssh00:25:281
516299108,1cyclictest32122-21tr20:05:141
516299108,1cyclictest27483-21tune2fs19:55:161
516299108,1cyclictest20654-21basename20:50:001
516299108,1cyclictest11712-21expr20:30:141
516299107,2cyclictest17756-21unixbench-2d20:40:241
516299107,2cyclictest15610-21grep19:30:181
516299107,2cyclictest10057-21cat20:25:201
516299100,9cyclictest6990-21expr20:20:141
516299100,9cyclictest26941-21wc21:00:191
51679996,2cyclictest7115-21head19:10:303
51659996,2cyclictest29228-21ls21:05:202
51629997,1cyclictest26431-21taskset19:52:561
51629996,2cyclictest2075-21grep20:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional