You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 20:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Jul 18, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20430210289,9sleep00-21swapper/007:06:010
2063928660,8sleep30-21swapper/307:08:453
2072028370,9sleep20-21swapper/207:09:492
2045128164,13sleep10-21swapper/107:06:181
299972730,0sleep20-21swapper/210:36:522
20831992727,0cyclictest0-21swapper/312:08:473
2082599260,2cyclictest0-21swapper/109:35:261
2082599250,24cyclictest0-21swapper/109:15:401
20831992118,2cyclictest39-21ksoftirqd/311:45:163
20829992120,1cyclictest0-21swapper/210:11:582
2082999210,4cyclictest0-21swapper/212:18:432
20825992119,1cyclictest31479-21fschecks_time09:20:161
20825992020,0cyclictest0-21swapper/112:38:511
20825992017,3cyclictest0-21swapper/110:11:231
2082599200,3cyclictest0-21swapper/112:34:341
20822992020,0cyclictest0-21swapper/012:37:530
2082999191,16cyclictest0-21swapper/212:33:582
2082299190,18cyclictest0-21swapper/010:49:480
2083199180,0cyclictest0-21swapper/309:59:233
2082999180,1cyclictest0-21swapper/207:40:132
20825991816,1cyclictest13629-21diskmemload09:32:411
20831991717,0cyclictest0-21swapper/309:20:583
20829991715,2cyclictest0-21swapper/210:22:372
2082999170,1cyclictest0-21swapper/212:12:362
2082999170,0cyclictest0-21swapper/211:51:112
20825991712,4cyclictest15021-21ssh12:05:051
2082599170,2cyclictest0-21swapper/110:27:521
2082599170,0cyclictest0-21swapper/108:50:241
2082299177,5cyclictest19594-21perf08:15:000
20829991615,1cyclictest0-21swapper/210:57:252
2082999160,16cyclictest12437-21latency_hist08:00:002
2082599160,5cyclictest0-21swapper/110:59:211
2082599160,13cyclictest0-21swapper/110:30:131
2082299162,7cyclictest5281-21perf11:40:000
2082299160,2cyclictest0-21swapper/011:00:240
20831991515,0cyclictest0-21swapper/311:25:333
20831991514,1cyclictest0-21swapper/310:34:523
20831991514,1cyclictest0-21swapper/310:14:223
2083199150,15cyclictest0-21swapper/309:39:433
20829991512,2cyclictest32-21ksoftirqd/211:31:152
2082999150,4cyclictest0-21swapper/209:43:272
2082999150,2cyclictest0-21swapper/211:27:492
2082999150,2cyclictest0-21swapper/210:05:392
2082999150,15cyclictest0-21swapper/211:19:392
2082999150,11cyclictest0-21swapper/212:00:162
2082599157,4cyclictest25673-21ssh11:32:081
2082599155,9cyclictest13629-21diskmemload12:10:571
20825991512,1cyclictest0-21swapper/110:06:371
2082599150,2cyclictest0-21swapper/111:29:161
2082599150,1cyclictest0-21swapper/110:00:521
2082599150,14cyclictest0-21swapper/110:42:411
20822991514,1cyclictest13-21ksoftirqd/009:15:190
20822991514,1cyclictest0-21swapper/011:18:060
2083199149,4cyclictest740-21sh10:39:213
2083199149,3cyclictest9592-21grep07:50:273
2083199148,4cyclictest11232-21cut10:45:173
20831991412,1cyclictest21062-21apt-get07:10:153
20831991411,2cyclictest30873-21wc08:35:243
2083199140,0cyclictest0-21swapper/310:51:573
2082999142,5cyclictest141rcu_preempt12:07:512
20829991413,1cyclictest55750irq/128-enp2s0-11:40:142
20829991413,1cyclictest0-21swapper/211:05:112
20829991411,2cyclictest16172-21wc09:30:202
20829991411,2cyclictest0-21swapper/211:40:012
2082999140,2cyclictest0-21swapper/209:19:182
2082999140,13cyclictest0-21swapper/210:00:522
2082599142,1cyclictest0-21swapper/109:13:071
2082599140,14cyclictest0-21swapper/109:58:221
2082299149,3cyclictest11227-21cron09:05:000
2082299146,4cyclictest4792-21apt-get11:20:140
2082299144,9cyclictest1458-21munin-run08:45:000
2082299141,3cyclictest0-21swapper/010:28:400
20822991412,2cyclictest0-21swapper/012:14:560
20822991410,3cyclictest16979-21tr12:25:150
2082299140,1cyclictest0-21swapper/009:35:480
2082299140,1cyclictest0-21swapper/008:45:120
2083199139,3cyclictest31965-21ssh12:15:053
2083199139,3cyclictest25542-21cron07:20:003
2083199131,1cyclictest0-21swapper/309:11:233
20831991311,1cyclictest34932sleep309:42:113
20831991311,1cyclictest0-21swapper/310:08:193
20831991311,1cyclictest0-21swapper/309:32:263
20831991310,2cyclictest7439-21sed08:55:183
20831991310,2cyclictest30003-21basename11:35:133
20831991310,2cyclictest25535-21grep12:10:293
20831991310,2cyclictest23465-21head11:30:263
2083199130,2cyclictest141rcu_preempt10:21:503
2083199130,1cyclictest0-21swapper/312:31:443
2083199130,12cyclictest12448-21sed11:05:163
2082999139,2cyclictest31239-21sh11:55:142
20829991310,2cyclictest4986-21cat11:00:262
20829991310,2cyclictest2962-21sed10:40:182
20829991310,2cyclictest29463-21wc07:25:202
20829991310,2cyclictest18361-21cat12:25:262
20829991310,2cyclictest17207-21munin-plugin-st08:10:002
20829991310,2cyclictest12859-21cat11:25:012
20829991310,2cyclictest12032-21wc10:45:232
2082999130,1cyclictest0-21swapper/209:59:472
20825991313,0cyclictest0-21swapper/110:20:511
20825991312,1cyclictest55650irq/127-enp2s0-11:36:241
20825991310,3cyclictest13629-21diskmemload11:49:251
20825991310,2cyclictest27202-21grep10:15:281
20825991310,2cyclictest19215-21expr10:50:151
20825991310,2cyclictest18498-21cat08:10:221
20825991310,2cyclictest14830-21turbostat.cron08:05:001
2082599130,13cyclictest0-21swapper/111:01:401
2082599130,12cyclictest5838-21uname11:20:191
2082299139,3cyclictest8352-21sh12:20:150
2082299139,3cyclictest25004-21wc07:15:240
2082299139,3cyclictest13217-21grep09:05:260
2082299134,3cyclictest225-21systemd-journal11:14:320
2082299134,3cyclictest225-21systemd-journal11:14:320
20822991310,2cyclictest6298-21readlink12:00:000
20822991310,2cyclictest15051-21grep11:25:260
20822991310,2cyclictest0-21swapper/012:32:180
2082299130,11cyclictest413-21cut12:15:180
2083199129,2cyclictest8137-21wc09:25:213
2083199129,2cyclictest7691-21cut12:00:203
2083199129,2cyclictest6574-21taskset11:40:173
2083199129,2cyclictest20780-21tr08:15:193
2083199129,2cyclictest17420-21grep12:25:173
2083199129,2cyclictest1635-21wc08:45:003
2083199128,3cyclictest7676-21awk09:45:153
2083199128,3cyclictest3908-21ssh10:40:273
2083199128,3cyclictest2481-21ssh12:35:233
2083199128,3cyclictest23690-21wc08:20:253
2083199126,5cyclictest8458-21apache211:17:423
2083199120,9cyclictest0-21swapper/311:20:143
2083199120,5cyclictest0-21swapper/308:10:163
2083199120,11cyclictest14416-21grep08:00:273
2082999129,2cyclictest581-21sh10:20:002
2082999129,2cyclictest18979-21cat08:10:282
2082999129,2cyclictest13606-21cron09:10:002
2082999129,2cyclictest10783-21strings10:25:212
2082999128,3cyclictest18602-21taskset09:12:242
20829991211,1cyclictest0-21swapper/212:22:312
20829991210,1cyclictest19717-21head10:50:182
2082999120,1cyclictest0-21swapper/209:27:432
2082999120,10cyclictest32466-21head08:40:172
2082599129,2cyclictest7221-21df08:55:171
2082599129,2cyclictest5242-21cron07:45:001
2082599129,2cyclictest31723-21head11:55:171
2082599129,2cyclictest28434-21grep07:25:141
2082599129,2cyclictest26736-21perf08:30:001
2082599129,2cyclictest26526-21tr10:35:171
2082599129,2cyclictest26086-21tr07:20:141
2082599129,2cyclictest16939-21if_err_enp2s009:50:171
2082599129,2cyclictest16939-21if_err_enp2s009:50:171
2082599129,2cyclictest14470-21cron09:30:001
2082599129,2cyclictest12392-21sed10:45:271
2082599128,3cyclictest27183-21awk08:30:161
2082599128,2cyclictest0-21swapper/108:20:131
2082599127,4cyclictest27375-21ssh11:14:261
2082599127,4cyclictest27375-21ssh11:14:261
2082599127,4cyclictest10576-21expr07:55:141
20825991211,1cyclictest0-21swapper/112:00:571
20825991210,1cyclictest9849-21grep09:00:181
2082599120,9cyclictest31493-21ssh11:16:071
2082599120,12cyclictest0-21swapper/111:07:461
2082599120,0cyclictest0-21swapper/112:29:501
2082299129,2cyclictest8707-21sed07:50:190
2082299129,2cyclictest5783-21grep07:45:150
2082299129,2cyclictest5309-21sed08:50:230
2082299129,2cyclictest30393-21cat07:30:000
2082299129,2cyclictest26656-21perf10:55:000
2082299129,2cyclictest2617-21sendmail-msp11:00:000
2082299129,2cyclictest26067-21grep07:20:140
2082299129,2cyclictest25693-21cat10:15:160
2082299129,2cyclictest23790-21perf09:55:000
2082299129,2cyclictest23790-21perf09:55:000
2082299129,2cyclictest21108-21grep08:15:220
2082299129,2cyclictest2047-21grep10:00:250
2082299129,2cyclictest13903-21sh11:05:270
2082299128,3cyclictest15356-21aten2_r7power_v12:05:140
20822991211,1cyclictest0-21swapper/009:10:430
20822991210,1cyclictest58002sleep011:40:130
84872110,1sleep00-21swapper/010:24:370
6152110,1sleep20-21swapper/207:35:002
222072110,1sleep00-21swapper/007:10:230
2083199118,2cyclictest9820-21tr10:25:163
2083199118,2cyclictest5382-21ssh11:00:303
2083199118,2cyclictest32449-21ssh11:55:233
2083199118,2cyclictest29433-21ssh10:17:123
2083199118,2cyclictest27466-21tr10:55:153
2083199118,2cyclictest24472-21ssh11:50:273
2083199118,2cyclictest23141-21ssh11:11:083
2083199118,2cyclictest23141-21ssh11:11:083
2083199118,2cyclictest1970-21grep08:45:133
2083199118,2cyclictest18085-21ssh09:50:323
2083199118,2cyclictest18085-21ssh09:50:323
2083199118,2cyclictest11090-21ssh12:21:023
2083199117,3cyclictest8473-21apache207:55:183
2083199117,3cyclictest686-21ssh10:00:143
2083199117,3cyclictest5147-21wc08:50:213
2083199117,3cyclictest29823-21sed07:25:253
2083199117,2cyclictest23271-21grep09:15:163
2083199110,10cyclictest1241-21sed07:35:153
2082999118,2cyclictest8830-21cron09:00:002
2082999118,2cyclictest5112-21missed_timers08:50:212
2082999118,2cyclictest31027-21tr09:20:142
2082999118,2cyclictest30942-21cat08:35:252
2082999118,2cyclictest2812-21grep08:45:182
2082999118,2cyclictest27461-21expr08:30:162
2082999118,2cyclictest24223-21seq07:15:152
2082999118,2cyclictest24036-21ssh09:35:142
2082999118,2cyclictest23579-21sh10:33:212
2082999118,2cyclictest22591-21expr08:20:142
2082999118,2cyclictest21523-21sed08:15:262
2082999118,2cyclictest20668-21expr11:10:162
2082999118,2cyclictest20668-21expr11:10:162
2082999118,2cyclictest19665-21ssh09:51:472
2082999118,2cyclictest19665-21ssh09:51:472
2082999118,2cyclictest17681-21ssh11:46:382
2082999118,2cyclictest12943-21tr08:00:152
2082999111,4cyclictest18915-21kworker/u8:212:35:252
2082999110,11cyclictest0-21swapper/209:46:232
2082599118,2cyclictest6087-21rm09:44:141
2082599118,2cyclictest32508-21head08:40:171
2082599118,2cyclictest22280-21ssh11:50:031
2082599118,2cyclictest21839-21grep07:10:181
2082599118,2cyclictest14465-21ssh09:49:151
2082599118,2cyclictest13017-21cat09:05:261
2082599117,3cyclictest21874-21taskset08:19:061
2082599117,2cyclictest15742-21grep08:05:171
2082599113,6cyclictest25963-21kworker/1:012:16:211
2082599110,10cyclictest7752-21sed11:40:271
2082299119,1cyclictest15789-21grep09:30:180
2082299119,1cyclictest13443-21grep08:00:170
2082299119,1cyclictest0-21swapper/011:48:500
2082299118,2cyclictest989-21tr07:35:140
2082299118,2cyclictest8946-21awk09:45:220
2082299118,2cyclictest8831-21cron09:00:000
2082299118,2cyclictest30117-21sed08:35:150
2082299118,2cyclictest29039-21sh10:36:060
2082299118,2cyclictest25346-21tail08:25:170
2082299118,2cyclictest24375-21basename09:55:150
2082299118,2cyclictest23791-21sort08:20:250
2082299118,2cyclictest16901-21ssh10:10:140
2082299118,2cyclictest15190-21cut08:05:140
2082299118,2cyclictest11944-21sed07:55:260
2082299118,2cyclictest10651-21ssh10:05:290
2082299117,3cyclictest22026-21ssh11:30:150
2082299115,2cyclictest225-21systemd-journal12:05:000
2082299114,6cyclictest2075-21/usr/sbin/munin10:40:230
2082299114,2cyclictest0-21swapper/009:40:100
2082299110,2cyclictest0-21swapper/009:26:060
2083199108,1cyclictest686-21sed07:35:003
2083199108,1cyclictest11516-21cut09:05:143
2083199108,1cyclictest10885-21chrt09:00:363
2083199107,2cyclictest3413-21expr07:40:153
2083199107,2cyclictest27591-21fschecks_count08:30:183
2083199107,2cyclictest26572-21tail07:20:163
2083199107,2cyclictest15354-21tr08:05:153
2082999108,1cyclictest97742sleep207:51:272
2082999108,1cyclictest2651-21taskset07:37:292
2082999108,1cyclictest22810-21taskset07:10:372
2082999107,2cyclictest6916-21python07:45:262
2082999107,2cyclictest24849-21tr08:25:142
2082999107,2cyclictest10983-21chrt09:01:512
2082599108,1cyclictest7949-21aten2_r7power_e07:50:141
2082599108,1cyclictest31476-21uniq07:30:181
2082599108,1cyclictest24395-21expr07:15:171
2082599108,1cyclictest2052-21expr08:45:131
2082599107,2cyclictest6407-21cat07:45:191
2082599100,9cyclictest7332sleep107:35:111
2082599100,9cyclictest31704-21cat08:40:001
2082599100,9cyclictest0-21swapper/112:20:151
2082299108,1cyclictest3683-21sed07:40:160
2082299108,1cyclictest20154-21ssh10:30:400
2082299107,2cyclictest31293-21ssh09:20:150
2082299107,2cyclictest31044-21grep07:30:160
2082299107,2cyclictest25756-21ssh11:51:250
2082299106,3cyclictest28599-21grep08:30:280
208319997,1cyclictest24579-21taskset08:25:053
208319995,3cyclictest7090-21wc07:45:273
208299997,2cyclictest27085-21proc_pri07:20:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional