You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 06:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat May 09, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11668210063,33sleep00-21swapper/019:05:540
1195329669,9sleep30-21swapper/319:09:343
1194129165,6sleep10-21swapper/119:09:241
1174429178,9sleep20-21swapper/219:06:522
145712600,0sleep00-21swapper/022:13:540
270952590,0sleep30-21swapper/323:58:203
239952530,1sleep20-21swapper/223:55:552
1207799260,24cyclictest0-21swapper/223:22:132
1207799248,4cyclictest4666-21diskmemload21:30:132
1207799240,24cyclictest0-21swapper/222:07:332
1207799240,19cyclictest0-21swapper/200:20:042
1207399240,1cyclictest0-21swapper/122:55:141
12077992322,1cyclictest0-21swapper/221:11:312
1207799230,22cyclictest0-21swapper/222:29:112
1207799230,19cyclictest0-21swapper/222:17:092
12073992314,5cyclictest452-21systemd-logind00:09:011
31532220,1sleep30-21swapper/322:25:423
1207799220,22cyclictest0-21swapper/200:01:212
12068992221,1cyclictest0-21swapper/022:05:040
1207999211,19cyclictest0-21swapper/300:35:143
1207799210,14cyclictest0-21swapper/223:06:402
1207399210,3cyclictest31592-21python20:55:241
1207799202,18cyclictest0-21swapper/221:56:172
12077992019,1cyclictest0-21swapper/223:10:582
1207399202,2cyclictest949-21smartctl21:00:181
12073992019,1cyclictest4666-21diskmemload23:00:481
12073992018,1cyclictest19490-21aten2_r7power_c23:15:131
12073992012,8cyclictest38150irq/131-snd_hda22:07:141
1207399200,1cyclictest0-21swapper/123:31:321
1207399200,1cyclictest0-21swapper/123:21:431
12068992017,3cyclictest13-21ksoftirqd/022:04:220
1207999195,10cyclictest5723-21sed23:25:253
1207999190,18cyclictest0-21swapper/319:20:133
12077991918,1cyclictest0-21swapper/222:14:342
1207799190,15cyclictest0-21swapper/223:15:512
12073991918,1cyclictest0-21swapper/121:43:471
12073991917,1cyclictest0-21swapper/123:10:101
12073991914,5cyclictest38150irq/131-snd_hda21:19:371
1206899196,7cyclictest6742-21perf21:50:000
1206899196,6cyclictest30857-21perf21:44:590
1207999182,2cyclictest141rcu_preempt22:30:413
12079991816,1cyclictest25500-21kworker/3:122:05:523
1207799180,18cyclictest0-21swapper/222:55:422
1207799180,16cyclictest0-21swapper/223:45:372
1207399183,14cyclictest4666-21diskmemload00:38:401
1207399180,16cyclictest4666-21diskmemload21:36:141
1207999176,9cyclictest0-21swapper/322:20:153
12077991717,0cyclictest0-21swapper/222:43:312
1207799170,17cyclictest0-21swapper/223:29:592
1207799170,17cyclictest0-21swapper/200:39:492
1207799170,13cyclictest0-21swapper/200:31:282
1207399170,0cyclictest0-21swapper/121:24:121
1206899170,16cyclictest0-21swapper/022:27:430
70232160,0sleep20-21swapper/221:50:042
1207799160,15cyclictest0-21swapper/222:48:112
1207399162,0cyclictest141rcu_preempt21:13:401
12073991616,0cyclictest0-21swapper/122:53:121
12073991615,1cyclictest0-21swapper/123:28:221
1207399160,16cyclictest0-21swapper/122:45:131
1207399160,0cyclictest0-21swapper/100:12:261
12068991615,1cyclictest0-21swapper/020:40:120
12068991615,1cyclictest0-21swapper/020:40:110
1207999155,5cyclictest6292-21/usr/sbin/munin21:30:163
12079991514,1cyclictest0-21swapper/323:04:263
12079991514,1cyclictest0-21swapper/321:25:153
12077991513,1cyclictest0-21swapper/200:29:242
12077991512,3cyclictest0-21swapper/222:36:072
12077991512,3cyclictest0-21swapper/222:36:072
12077991510,4cyclictest6151-21tr00:05:142
1207799150,1cyclictest0-21swapper/221:36:522
1207799150,15cyclictest0-21swapper/221:43:082
1207399153,1cyclictest0-21swapper/100:01:211
12073991514,1cyclictest25-21ksoftirqd/122:30:201
12073991511,4cyclictest0-21swapper/120:05:131
12073991510,5cyclictest27538-21kworker/1:123:55:191
12068991514,1cyclictest0-21swapper/023:26:450
1206899151,1cyclictest0-21swapper/023:38:230
1206899150,1cyclictest0-21swapper/022:30:100
12079991410,3cyclictest31128-21cron19:50:003
1207999140,1cyclictest0-21swapper/322:40:113
1207799141,12cyclictest27397-21tail19:40:172
1207799140,3cyclictest0-21swapper/223:00:482
1207799140,14cyclictest0-21swapper/223:36:292
1207799140,0cyclictest0-21swapper/223:52:412
12073991414,0cyclictest0-21swapper/122:16:541
12073991412,1cyclictest25-21ksoftirqd/120:20:141
1207399141,12cyclictest10206-21rm21:31:551
1207399140,13cyclictest0-21swapper/121:45:411
1206899146,4cyclictest225-21systemd-journal20:40:000
1206899141,6cyclictest26383-21munin-run19:40:000
12068991412,1cyclictest4666-21diskmemload23:07:190
1206899140,1cyclictest0-21swapper/023:00:280
1207999139,3cyclictest6069-21cut23:45:203
1207999139,3cyclictest16812-21perf19:20:013
1207999139,2cyclictest22467-21ssh21:40:003
1207999138,2cyclictest31338-21apt-get21:45:123
1207999133,8cyclictest5376-21munin-run00:05:003
12079991310,2cyclictest26253-21tr23:00:003
12079991310,2cyclictest22464-21wc00:15:003
12079991310,2cyclictest13196-21apt21:15:013
12079991310,2cyclictest11590-21grep20:15:163
12079991310,2cyclictest10028-21ssh22:10:253
1207999130,12cyclictest23869-21sed21:40:163
1207999130,12cyclictest0-21swapper/322:18:133
1207799139,3cyclictest30688-21tr21:25:142
1207799139,3cyclictest24057-21wc20:40:182
1207799139,3cyclictest24057-21wc20:40:172
1207799139,3cyclictest11013-21tail22:30:272
1207799131,10cyclictest24781-21/usr/sbin/munin22:20:152
12077991310,2cyclictest31650-21tr21:45:142
12077991310,2cyclictest18180-21grep19:20:192
12077991310,2cyclictest10587-21grep22:50:142
1207399139,3cyclictest12453-21cat20:15:251
1207399137,5cyclictest24777-21grep22:00:171
12073991311,1cyclictest25-21ksoftirqd/123:37:081
12073991310,2cyclictest28833-21cat19:45:001
1207399130,1cyclictest0-21swapper/122:27:031
1207399130,1cyclictest0-21swapper/100:18:251
1207399130,12cyclictest0-21swapper/123:05:061
1206899139,3cyclictest6168-21head21:10:160
1206899139,3cyclictest15381-21cron20:24:590
1206899131,6cyclictest31652-21sendmail-msp00:40:010
1206899131,6cyclictest22292-21awk00:15:000
1206899131,12cyclictest0-21swapper/000:29:480
12068991310,2cyclictest9595-21tr20:10:190
12068991310,2cyclictest31010-21chrt23:40:380
12068991310,2cyclictest29444-21grep20:50:250
12068991310,2cyclictest13949-21grep19:10:250
1206899130,7cyclictest21585-21sendmail-msp21:20:010
1206899130,13cyclictest0-21swapper/000:04:210
1206899130,12cyclictest21032-21grep23:54:590
1207999129,2cyclictest6698-21grep00:05:173
1207999129,2cyclictest30768-21sort00:20:013
1207999129,2cyclictest28164-21tr23:20:123
1207999129,2cyclictest25197-21uname19:35:183
1207999129,2cyclictest18204-21head22:35:173
1207999129,2cyclictest18204-21head22:35:163
1207999129,2cyclictest15298-21ssh23:12:183
1207999129,2cyclictest14795-21ls23:50:263
1207999129,2cyclictest10778-21tail22:50:163
1207999129,2cyclictest10200-21grep20:10:263
1207999128,3cyclictest2860-21cat22:45:183
1207999128,3cyclictest21896-21wget19:30:133
1207999121,2cyclictest0-21swapper/323:15:133
1207799129,2cyclictest5474-21sed20:00:262
1207799129,2cyclictest30295-21grep20:55:142
1207799129,2cyclictest24240-21tr22:00:142
1207799129,2cyclictest19458-21ls20:30:202
1207799129,2cyclictest15912-21grep19:15:192
1207799129,2cyclictest15820-21cut00:10:212
1207799128,3cyclictest23985-21cron19:34:592
1207799128,2cyclictest4212-21grep21:05:242
1207799128,2cyclictest15879-21tr20:25:142
1207799127,1cyclictest32-21ksoftirqd/200:20:002
1207799121,1cyclictest0-21swapper/223:40:262
1207799120,4cyclictest0-21swapper/221:23:022
1207799120,2cyclictest0-21swapper/221:16:552
1207799120,1cyclictest0-21swapper/223:33:222
1207799120,10cyclictest25251-21wc19:35:182
1207399129,2cyclictest29595-21grep23:40:161
1207399129,2cyclictest26862-21cat20:45:241
1207399129,2cyclictest26415-21sendmail-msp19:40:001
1207399129,2cyclictest17850-21grep19:20:151
1207399129,2cyclictest16746-21ntpq00:30:221
1207399128,1cyclictest15987-21expr20:25:141
1207399126,4cyclictest12638-21apache200:24:251
1207399122,10cyclictest0-21swapper/123:51:231
1207399121,0cyclictest141rcu_preempt22:20:471
1206899129,2cyclictest31210-21ssh21:25:160
1206899129,2cyclictest29104-21tr19:45:130
1206899129,2cyclictest23977-21grep21:20:240
1206899129,2cyclictest19199-21basename19:24:590
1206899129,2cyclictest16936-21awk21:55:200
1206899129,2cyclictest1174-21idleruntime-cro22:45:010
1206899128,3cyclictest6423-21tr20:05:140
1206899124,4cyclictest225-21systemd-journal00:35:000
1206899120,11cyclictest7802-21tail00:05:260
1206899120,11cyclictest611-21grep19:50:260
1206899120,11cyclictest17374-21head20:25:260
1207999118,2cyclictest7182-21apache_accesses21:50:113
1207999118,2cyclictest7095-21grep00:25:133
1207999118,2cyclictest5241-21ssh23:05:553
1207999118,2cyclictest4090-21cat21:05:253
1207999118,2cyclictest32752-21grep19:50:223
1207999118,2cyclictest31571-21ssh22:04:493
1207999118,2cyclictest3131-21chrt19:55:493
1207999118,2cyclictest28600-21ssh23:40:063
1207999118,2cyclictest26163-21head20:45:163
1207999118,2cyclictest23493-21wc21:20:193
1207999118,2cyclictest23068-21apache_volume20:40:123
1207999118,2cyclictest23068-21apache_volume20:40:123
1207999118,2cyclictest18963-21tail20:30:163
1207999118,2cyclictest17405-21grep21:55:253
1207999118,2cyclictest15887-21cut20:25:143
1207999118,2cyclictest15531-21wc21:15:263
1207999118,2cyclictest14945-21ssh23:31:093
1207999118,2cyclictest13090-21cut19:10:173
1207999116,3cyclictest4241-21df_abs20:00:153
1207999112,1cyclictest0-21swapper/300:30:153
1207999110,10cyclictest28263-21ls20:50:133
1207999110,10cyclictest24250-21chrt23:36:513
1207999110,10cyclictest1786-21ssh00:21:173
1207799118,2cyclictest29763-21grep19:45:172
1207799118,2cyclictest28445-21sort20:50:152
1207799118,2cyclictest26705-21wc20:45:212
1207799118,2cyclictest20689-21grep19:25:212
1207799118,2cyclictest14220-21uname20:20:192
1207799117,3cyclictest6888-21grep20:05:172
1207799117,3cyclictest20871-21cut20:35:132
1207399119,1cyclictest29422-21cat19:45:141
1207399118,2cyclictest422-21wc19:50:261
1207399118,2cyclictest30567-21tr21:25:141
1207399118,2cyclictest20589-21basename20:35:101
1207399118,2cyclictest16840-21cat21:55:191
1207399118,2cyclictest16040-21which19:15:201
1207399118,2cyclictest10017-21ssh21:51:021
1207399117,3cyclictest8952-21sed22:10:151
1207399117,3cyclictest23778-21head20:40:161
1207399117,3cyclictest23778-21head20:40:151
1207399114,6cyclictest38150irq/131-snd_hda22:40:111
1207399113,3cyclictest1142-21runrttasks23:49:151
1207399110,11cyclictest0-21swapper/122:35:431
1207399110,11cyclictest0-21swapper/122:35:431
1207399110,11cyclictest0-21swapper/119:55:141
1206899118,2cyclictest4628-21ssh22:46:110
1206899118,2cyclictest31718-21ssh23:21:530
1206899118,2cyclictest28102-21cat19:40:250
1206899118,2cyclictest28010-21ssh22:21:210
1206899118,2cyclictest26638-21ssh00:16:530
1206899118,2cyclictest22570-21ssh22:18:390
1206899118,2cyclictest22325-21ssh23:55:170
1206899118,2cyclictest20565-21cut19:25:190
1206899118,2cyclictest20407-21ssh22:36:140
1206899118,2cyclictest20407-21ssh22:36:130
1206899118,2cyclictest20004-21cat22:55:250
1206899118,2cyclictest17884-21ssh21:36:250
1206899118,2cyclictest15755-21ssh23:31:480
1206899118,2cyclictest11318-21expr20:15:140
1206899118,2cyclictest10437-21expr22:50:140
1206899118,2cyclictest10338-21chrt21:51:180
1206899117,3cyclictest10253-21ssh21:31:560
1206899115,4cyclictest25413-21/usr/sbin/munin20:45:120
1206899112,4cyclictest225-21systemd-journal23:45:140
1207999108,1cyclictest8140-21taskset20:08:513
1207999108,1cyclictest21380-21chrt19:27:483
1207999108,1cyclictest20642-21hostname20:35:133
1207999107,2cyclictest1860-21grep21:00:273
1207999107,2cyclictest15355-21chrt20:24:423
1207999104,5cyclictest31446-21proc_pri20:55:213
1207799108,1cyclictest31452sleep219:55:592
1207799108,1cyclictest10482-21chrt20:13:272
1207799107,2cyclictest13390-21needreboot19:10:202
1207799107,2cyclictest10881-21apt-get20:15:122
1207799100,9cyclictest32564-21cat21:00:122
1207399108,1cyclictest7016-21ssh00:25:111
1207399108,1cyclictest3717-21taskset20:00:071
1207399108,1cyclictest29610-21taskset20:51:041
1207399108,1cyclictest23946-21chrt19:34:321
1207399108,1cyclictest19467-21basename19:25:131
1207399108,1cyclictest18070-21apt-config20:30:011
1207399108,1cyclictest12346-21sshd19:10:131
1207399107,2cyclictest8575-21tr20:10:131
1207399107,2cyclictest3310-21grep21:05:151
1206899108,1cyclictest4018-21expr20:00:140
1206899108,1cyclictest30992-21smartctl20:55:160
1206899108,1cyclictest2622-21aten2_r7power_p21:05:120
1206899108,1cyclictest22268-21expr19:30:150
1206899108,1cyclictest15073-21expr19:15:130
1206899108,1cyclictest11338-21expr23:10:130
1206899107,2cyclictest31373-21grep00:20:140
1206899107,2cyclictest21446-21grep23:15:260
1206899107,2cyclictest18919-21cut20:30:170
1206899106,3cyclictest1450-21chrt19:55:140
120799996,2cyclictest28723-21taskset19:44:363
120779997,1cyclictest929-21taskset19:53:122
120689997,1cyclictest32584-21taskset21:00:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional