You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 10:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot5s.osadl.org (updated Fri Aug 28, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2050051995239,9cyclictest311ktimers/223:05:132
2050051995239,9cyclictest311ktimers/223:05:132
2050042995125,9cyclictest2161825-21/usr/sbin/munin00:30:170
2050051995043,5cyclictest2066232-21kworker/u8:1+events_unbound20:02:152
2050042995018,17cyclictest0-21swapper/023:15:250
2050042995015,19cyclictest0-21swapper/019:41:380
2050042994918,13cyclictest2120143-21cpu22:30:140
2050047994840,5cyclictest2061020-21munin-plugin-st19:40:011
2050042994835,10cyclictest121ktimers/020:40:120
2050042994739,4cyclictest2164702-21sendmail00:40:000
2050042994620,10cyclictest2142698-21chrt23:35:110
2050051994537,5cyclictest2118182-21idleruntime-cro22:25:022
2050042994524,7cyclictest2125954-21/usr/sbin/munin22:45:250
2050042994431,10cyclictest121ktimers/000:10:140
2050042994431,10cyclictest121ktimers/000:10:130
2050042994423,18cyclictest2130795-21fw_packets23:00:170
2050042994423,18cyclictest2130795-21fw_packets23:00:170
2050042994416,11cyclictest21218012sleep022:35:090
2050055994335,6cyclictest391ktimers/323:50:133
2050042994323,10cyclictest0-21swapper/020:30:130
2050055994227,11cyclictest391ktimers/300:30:113
2050051994230,7cyclictest311ktimers/223:45:132
2050042994232,7cyclictest2124299-21/usr/sbin/munin22:40:240
2050042994231,8cyclictest438-21cron21:10:000
2050042994228,9cyclictest2087872-21/usr/sbin/munin20:55:220
2050042994228,11cyclictest2099554-21/usr/sbin/munin21:30:180
2050042994228,11cyclictest2056893-21/usr/sbin/munin19:25:230
2050055994126,12cyclictest391ktimers/321:15:143
2050051994126,10cyclictest2103886-21sshd21:42:272
2050042994129,9cyclictest2142293-21cron23:33:000
2050042994123,10cyclictest0-21swapper/022:15:120
2050042994116,9cyclictest0-21swapper/023:48:410
2050055994032,6cyclictest391ktimers/320:50:123
2050055994032,5cyclictest2061018-21munin-plugin-st19:40:013
2050047994033,4cyclictest2059301-21cron19:35:011
2050042994023,12cyclictest2080347-21/usr/sbin/munin20:35:150
2050042994018,17cyclictest0-21swapper/020:25:170
2050055993929,6cyclictest0-21swapper/322:15:253
2050051993933,4cyclictest2112266-21kworker/u8:0+events_unbound22:07:322
2050051993932,4cyclictest2116279-21idleruntime-cro22:20:002
2050051993930,5cyclictest2121588-21munin-plugin-st22:35:002
2050042993927,8cyclictest2139533-21/usr/sbin/munin23:25:150
2050042993926,10cyclictest2106879-21/usr/sbin/munin21:50:250
2050042993926,10cyclictest2104333-21sshd21:45:110
2050042993926,10cyclictest2090107-21cron21:01:000
2050042993924,11cyclictest0-21swapper/023:40:090
2050042993923,11cyclictest2059554-21egrep19:35:140
2050042993922,10cyclictest0-21swapper/022:50:130
2050055993831,4cyclictest2144171-21munin-plugin-st23:40:003
2050055993827,7cyclictest0-21swapper/323:00:003
2050047993829,7cyclictest2121114-21unixbench_multi22:30:261
2050047993827,7cyclictest2147105-21grep23:45:251
2050042993827,8cyclictest2059156-21sshd19:33:120
2050042993827,7cyclictest2153515-21ntp_states00:05:210
2050042993827,7cyclictest2153515-21ntp_states00:05:210
2050042993826,9cyclictest2160473-21/usr/sbin/munin00:25:230
2050042993826,7cyclictest2065630-21/usr/sbin/munin19:50:230
2050042993825,8cyclictest2148085-21/usr/sbin/munin23:50:180
2050042993823,9cyclictest0-21swapper/019:45:130
2050042993822,12cyclictest53-21kcompactd021:55:140
2050042993822,12cyclictest2128971-21uname22:55:180
2050042993821,13cyclictest2118801-21sed22:25:230
2050055993726,7cyclictest2090042-21grep21:00:253
2050042993731,3cyclictest2111164-21apt-get22:05:150
2050042993728,5cyclictest2066817-21/usr/sbin/munin19:55:150
2050042993727,7cyclictest2137721-21/usr/sbin/munin23:20:160
2050042993727,7cyclictest2098272-21/usr/sbin/munin21:25:240
2050042993726,8cyclictest2103615-21/usr/sbin/munin21:40:260
2050042993726,8cyclictest2085934-21/usr/sbin/munin20:50:180
2050042993726,7cyclictest2094540-21meminfo21:15:210
2050042993726,7cyclictest2075694-21ntp_states20:20:210
2050042993724,9cyclictest2158194-21/usr/sbin/munin00:20:150
2050042993723,11cyclictest2093406-21awk21:10:250
2050042993721,8cyclictest0-21swapper/020:10:120
2050042993720,14cyclictest2053105-21open_inodes19:15:220
2050042993716,18cyclictest2074979-21latency_hist20:20:020
2050042993715,18cyclictest2149797-21if_err_eth023:55:170
2050055993627,6cyclictest2140366-21unixbench_singl23:25:253
2050055993627,5cyclictest2108794-21head21:55:273
2050051993627,5cyclictest0-21swapper/221:00:022
2050051993626,7cyclictest2108728-21grep21:55:262
2050051993623,5cyclictest2162847-21kworker/u8:0+events_unbound00:38:042
2050051993613,6cyclictest131rcu_preempt23:25:112
2050047993628,4cyclictest0-21swapper/100:38:141
2050047993626,7cyclictest2150557-21grep23:55:271
2050042993629,3cyclictest2151250-21apt-get00:00:120
2050042993628,4cyclictest2133732-21munin-run23:10:000
2050042993628,4cyclictest2133732-21munin-run23:10:000
2050042993627,6cyclictest2095813-21/usr/sbin/munin21:20:120
2050042993626,8cyclictest2112628-21apt-get22:10:050
2050042993626,7cyclictest2055207-21swap19:20:250
2050042993621,12cyclictest2157880-21latency_hist00:20:000
2050042993620,13cyclictest2116972-21if_err_eth022:20:180
2050042993619,14cyclictest2101947-21unixbench_singl21:35:260
2050042993619,14cyclictest2101947-21unixbench_singl21:35:260
2050042993618,4cyclictest0-21swapper/023:10:240
2050042993617,17cyclictest0-21swapper/020:05:240
2050042993615,16cyclictest0-21swapper/019:10:120
2050051993523,8cyclictest311ktimers/222:00:122
2050051993516,4cyclictest131rcu_preempt21:45:132
2050047993528,4cyclictest0-21swapper/121:35:161
2050047993528,4cyclictest0-21swapper/121:35:161
2050047993524,7cyclictest0-21swapper/121:20:001
2050042993526,6cyclictest2109579-21diskstats22:00:150
2050042993526,6cyclictest2085084-21sshd20:47:120
2050051993430,2cyclictest2136999-21kworker/u8:2+events_unbound23:57:502
2050051993429,2cyclictest0-21swapper/220:40:122
2050051993427,4cyclictest2071481-21idleruntime-cro20:10:012
2050051993425,7cyclictest2051554-21kworker/u8:3+events_unbound19:38:412
2050051993424,7cyclictest2098534-21grep21:25:262
2050047993427,4cyclictest2068052-21latency_hist20:00:011
2050047993425,5cyclictest0-21swapper/120:20:161
2050055993325,4cyclictest0-21swapper/300:00:293
2050055993323,6cyclictest0-21swapper/322:00:263
2050051993327,4cyclictest2142250-21kworker/u8:0+events_unbound23:32:442
2050047993324,6cyclictest0-21swapper/120:40:021
2050047993323,7cyclictest2088731-21auditctl21:00:011
2050047993321,8cyclictest2155681-21grep00:10:251
2050047993321,8cyclictest2155681-21grep00:10:251
2050055993221,7cyclictest2097277-21sh21:25:003
2050047993224,4cyclictest0-21swapper/119:25:251
2050047993222,7cyclictest2162943-21audit_send_reply00:35:001
2050047993222,7cyclictest2055488-21grep19:20:261
2050047993221,7cyclictest0-21swapper/120:15:251
2050055993123,4cyclictest0-21swapper/319:50:243
2050055993122,4cyclictest0-21swapper/321:25:253
2050055993121,7cyclictest438-21cron19:50:013
2050051993124,4cyclictest2106250-21fschecks_time21:50:172
2050051993123,5cyclictest0-21swapper/219:55:012
2050051993123,4cyclictest0-21swapper/219:20:152
2050051993123,4cyclictest0-21swapper/200:15:262
205005199311,6cyclictest0-21swapper/222:25:162
205004799319,4cyclictest21357732cut23:15:011
2050047993124,4cyclictest2136938-21unixbench_singl23:15:261
2050047993123,5cyclictest2077930-21kworker/u8:0+events_unbound20:27:171
2050047993121,7cyclictest2130245-21sort23:00:011
2050047993121,7cyclictest2065914-21unixbench_singl19:50:241
2050055993022,5cyclictest2074394-21unixbench_multi20:15:253
2050055993022,4cyclictest0-21swapper/320:00:253
2050055993020,7cyclictest2153181-21grep00:05:153
2050055993020,7cyclictest2153181-21grep00:05:153
2050051993020,6cyclictest0-21swapper/223:00:262
2050051993020,6cyclictest0-21swapper/223:00:262
205005199301,5cyclictest0-21swapper/222:45:142
205004799304,23cyclictest2052696-21cut19:15:161
2050047993023,4cyclictest0-21swapper/119:40:161
2050047993020,6cyclictest0-21swapper/122:37:311
2050042993019,8cyclictest0-21swapper/019:10:010
2050055992923,3cyclictest0-21swapper/322:10:013
2050055992921,4cyclictest0-21swapper/321:35:003
2050055992921,4cyclictest0-21swapper/320:35:163
205005199292,3cyclictest131rcu_preempt20:20:122
2050051992922,5cyclictest2115930-21kworker/u8:1+events_unbound22:52:412
2050047992922,4cyclictest2157837-21sendmail00:20:001
2050047992922,4cyclictest2112632-21latency_hist22:10:011
2050047992922,4cyclictest0-21swapper/121:10:161
2050047992920,6cyclictest0-21swapper/122:25:201
2050047992920,6cyclictest0-21swapper/100:00:291
2050055992822,3cyclictest0-21swapper/319:35:013
2050055992820,4cyclictest0-21swapper/320:20:163
2050055992818,7cyclictest2117238-21ntpq22:20:213
2050055992817,7cyclictest2102402-21basename21:40:003
2050055992817,7cyclictest2102402-21basename21:40:003
2050051992821,4cyclictest2155699-21grep00:10:252
2050051992821,4cyclictest2155699-21grep00:10:252
2050051992821,3cyclictest0-21swapper/219:25:252
2050051992820,6cyclictest2115930-21kworker/u8:1+events_unbound22:55:222
2050051992820,4cyclictest0-21swapper/219:15:012
2050051992818,7cyclictest2124577-21grep22:40:262
2050051992818,6cyclictest0-21swapper/223:35:162
2050047992818,7cyclictest2125459-21tune2fs22:45:171
2050055992720,3cyclictest0-21swapper/322:35:283
2050055992719,5cyclictest0-21swapper/320:30:263
2050055992719,4cyclictest0-21swapper/321:50:263
2050055992719,4cyclictest0-21swapper/320:10:253
2050055992719,3cyclictest0-21swapper/323:20:013
2050055992715,8cyclictest0-21swapper/319:20:003
205005199276,6cyclictest131rcu_preempt00:00:122
205005199272,5cyclictest32-21ksoftirqd/200:07:582
205005199272,5cyclictest32-21ksoftirqd/200:07:582
2050051992721,4cyclictest2088975-21kworker/u8:0+events_unbound21:04:082
2050051992720,5cyclictest311ktimers/200:20:132
2050051992720,5cyclictest2131916-21kworker/u8:0+events_unbound23:14:412
2050051992720,5cyclictest2060742-21kworker/u8:0+events_unbound20:10:222
2050051992720,5cyclictest2060742-21kworker/u8:0+events_unbound19:44:242
2050051992720,4cyclictest0-21swapper/200:25:252
2050051992719,5cyclictest2114076-21tail22:10:252
2050051992718,6cyclictest2073087-21kworker/u8:2+events_unbound21:05:212
2050051992716,9cyclictest2050036-21cyclictest23:50:132
2050047992721,3cyclictest0-21swapper/121:05:181
2050047992720,5cyclictest2115930-21kworker/u8:1+events_unbound22:52:351
2050047992719,4cyclictest0-21swapper/119:10:161
2050047992718,5cyclictest0-21swapper/123:55:001
2050047992716,7cyclictest0-21swapper/100:20:261
2050055992619,4cyclictest2088975-21kworker/u8:0+events_unbound21:07:243
2050055992619,4cyclictest2057615-21kthreadd19:30:013
2050055992618,5cyclictest0-21swapper/323:00:263
2050055992618,5cyclictest0-21swapper/323:00:253
2050055992617,5cyclictest0-21swapper/320:00:013
2050055992615,8cyclictest2127502-21ntp_states22:50:223
2050055992615,7cyclictest2083211-21grep20:40:263
2050051992619,5cyclictest2093467-21kworker/u8:3+events_unbound21:15:242
2050051992619,5cyclictest2049766-21kworker/u8:2+events_unbound19:18:402
2050051992619,4cyclictest0-21swapper/200:30:152
2050051992618,4cyclictest0-21swapper/221:35:262
2050051992618,4cyclictest0-21swapper/221:35:262
2050051992617,7cyclictest2083509-21kworker/u8:1+events_unbound20:47:222
2050051992615,7cyclictest204-21dbus-daemon21:12:242
2050051992614,4cyclictest2071315-21kworker/u8:1+events_unbound20:17:532
2050047992620,3cyclictest0-21swapper/120:30:191
2050047992619,5cyclictest2086617-21unixbench_singl20:50:251
2050047992617,6cyclictest1-21systemd21:32:251
2050055992519,3cyclictest0-21swapper/300:38:033
2050055992519,3cyclictest0-21swapper/300:20:003
2050055992518,4cyclictest2133731-21idleruntime-cro23:10:003
2050055992518,4cyclictest2133731-21idleruntime-cro23:10:003
2050055992518,3cyclictest0-21swapper/321:00:013
2050055992517,5cyclictest0-21swapper/300:00:003
2050055992515,6cyclictest204-21dbus-daemon19:12:083
205005199253,4cyclictest0-21swapper/223:40:222
2050051992519,4cyclictest2051554-21kworker/u8:3+events_unbound19:34:512
2050051992519,3cyclictest0-21swapper/221:30:262
2050051992517,6cyclictest2073087-21kworker/u8:2+events_unbound20:35:062
2050051992517,5cyclictest0-21swapper/223:20:152
205005199251,12cyclictest0-21swapper/220:26:042
2050047992519,4cyclictest2144176-21grep23:40:001
2050047992518,4cyclictest0-21swapper/120:40:261
2050047992516,6cyclictest2117259-21awk22:20:221
2050055992417,5cyclictest145-21jbd2/mmcblk1p2-822:10:063
2050055992417,4cyclictest2147591-21kthreadd23:50:003
2050055992413,7cyclictest0-21swapper/323:20:253
2050055992412,9cyclictest2158525-21memory00:20:213
205005199248,3cyclictest131rcu_preempt21:20:162
205005199247,7cyclictest32-21ksoftirqd/219:55:112
2050051992416,4cyclictest0-21swapper/223:20:002
205005199241,13cyclictest0-21swapper/219:48:512
2050047992418,3cyclictest0-21swapper/123:40:111
2050047992414,6cyclictest0-21swapper/123:20:241
2050047992414,6cyclictest0-21swapper/121:25:251
2050047992412,7cyclictest0-21swapper/123:05:261
2050047992412,7cyclictest0-21swapper/123:05:261
2050055992317,3cyclictest0-21swapper/322:30:273
2050055992316,4cyclictest0-21swapper/323:10:233
205005199232,12cyclictest948274-21kworker/2:1+mm_percpu_wq22:36:082
2050047992317,3cyclictest0-21swapper/121:55:211
2050047992316,4cyclictest0-21swapper/123:30:261
2050047992316,4cyclictest0-21swapper/121:20:161
2050047992315,5cyclictest2159987-21fschecks_time00:25:161
2050055992215,5cyclictest391ktimers/322:45:133
2050055992215,4cyclictest2104293-21latency_hist21:45:023
2050055992214,5cyclictest2062733-21apt19:45:013
2050055992214,5cyclictest0-21swapper/321:45:263
2050055992213,4cyclictest0-21swapper/322:25:303
205005599221,18cyclictest629-21snmpd20:25:113
2050051992216,4cyclictest2086627-21grep20:50:252
205004799222,16cyclictest629-21snmpd22:00:121
2050047992212,6cyclictest0-21swapper/100:10:001
2050047992212,6cyclictest0-21swapper/100:10:001
205004799221,11cyclictest0-21swapper/120:12:231
205005599218,8cyclictest2145884-21sh23:45:003
2050055992116,2cyclictest0-21swapper/323:30:183
2050055992115,3cyclictest0-21swapper/322:40:233
2050055992114,5cyclictest0-21swapper/320:45:243
2050055992112,6cyclictest391ktimers/300:10:133
2050055992112,6cyclictest391ktimers/300:10:133
205005199218,6cyclictest32-21ksoftirqd/220:30:182
205004799214,3cyclictest131rcu_preempt20:45:121
2050047992114,5cyclictest231ktimers/120:05:291
2050047992114,4cyclictest2106184-21fschecks_count21:50:161
2050047992113,4cyclictest0-21swapper/123:27:421
205004799211,10cyclictest0-21swapper/121:42:341
2050055992015,2cyclictest0-21swapper/300:30:003
2050055992013,5cyclictest391ktimers/321:10:133
2050055992013,5cyclictest391ktimers/320:05:133
2050055992010,7cyclictest2054548-21df19:20:133
2050047992014,4cyclictest2051984-21munin-node21:45:121
2050047992014,3cyclictest0-21swapper/121:00:211
2050047991912,3cyclictest0-21swapper/120:00:231
2050047991911,6cyclictest231ktimers/122:40:121
2050047991910,6cyclictest2130667-21df23:00:141
2050047991910,6cyclictest2130667-21df23:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional