You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 08:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sun May 24, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2279107995446,5cyclictest2307341-21ls20:30:013
2279103995043,4cyclictest2342088-21idleruntime-cro22:10:002
2279094995041,5cyclictest2301732-21unixbench_multi20:10:240
2279094995041,5cyclictest2301732-21unixbench_multi20:10:240
2279098994938,7cyclictest2326319-21grep21:20:241
2279094994923,22cyclictest2309458-21fschecks_time20:35:150
2279098994738,6cyclictest2318160-21fschecks_count21:00:141
2279094994724,19cyclictest0-21swapper/023:40:260
227909499471,3cyclictest2342318-21apt-get22:10:120
2279107994638,5cyclictest2335268-21ls21:50:013
2279098994434,6cyclictest2336500-21unixbench_multi21:50:241
227909499432,9cyclictest2319861-21/usr/sbin/munin21:05:160
227909499431,9cyclictest2327092-21/usr/sbin/munin21:25:150
227909499431,39cyclictest2290942-21/usr/sbin/munin19:40:260
2279103994229,9cyclictest311ktimers/219:10:112
2279098994235,4cyclictest2348915-21latency_hist22:30:011
2279098994235,4cyclictest2348915-21latency_hist22:30:001
2279098994231,7cyclictest2280850-21grep19:15:011
227909499421,9cyclictest2316662-21/usr/sbin/munin20:55:210
227909499421,9cyclictest2305925-21/usr/sbin/munin20:25:140
2279094994216,23cyclictest2345611-21latency_hist22:20:000
227909499421,10cyclictest2338272-21/usr/sbin/munin21:55:250
2279107994130,7cyclictest2308579-21grep20:30:253
2279103994135,4cyclictest2308910-21kworker/u8:2+events_unbound20:52:382
227909499412,6cyclictest2304702-21/usr/sbin/munin20:20:220
227909499412,10cyclictest2350983-21fschecks_count22:35:150
227909499411,9cyclictest2382065-21cron00:03:590
227909499411,8cyclictest2325383-21df21:20:130
227909499411,8cyclictest2288934-21ntpq19:35:210
227909499411,12cyclictest2303495-21head20:15:270
227909499411,10cyclictest2287238-21/usr/sbin/munin19:30:210
2279107994029,7cyclictest2323739-21cut21:15:173
2279107994029,6cyclictest2377101-21apt-config23:50:003
2279103994031,6cyclictest2356102-21fschecks_count22:50:152
2279098994032,5cyclictest2345492-21munin-plugin-st22:20:001
227909499409,26cyclictest2292787-21grep19:45:260
227909499407,5cyclictest2297980-21/usr/sbin/munin20:00:250
2279094994032,5cyclictest2280819-21latency_hist19:15:020
227909499402,5cyclictest2312690-21munin-node20:45:120
227909499401,9cyclictest2390126-21/usr/sbin/munin00:25:240
227909499401,9cyclictest2341162-21/usr/sbin/munin22:05:220
227909499401,8cyclictest2349142-21munin-node22:30:120
227909499401,8cyclictest2348247-21/usr/sbin/munin22:25:250
227909499401,8cyclictest2348247-21/usr/sbin/munin22:25:240
227909499401,8cyclictest2338919-21/usr/sbin/munin22:00:140
227909499401,8cyclictest2283280-21/usr/sbin/munin19:20:110
227909499401,14cyclictest2281563-21chrt19:15:120
2279094994011,26cyclictest2384092-21latency_hist00:10:000
2279103993932,4cyclictest2289854-21grep19:40:012
2279103993915,4cyclictest131rcu_preempt23:05:112
2279098993931,4cyclictest0-21swapper/121:15:011
2279098993931,4cyclictest0-21swapper/121:15:001
227909499399,27cyclictest2314391-21apt-get20:50:120
227909499399,26cyclictest2309084-21latency_hist20:35:010
227909499394,7cyclictest2358648-21grep22:55:260
227909499393,4cyclictest2366862-21memory23:20:180
227909499393,4cyclictest2334372-21ntp_states21:45:250
2279094993928,7cyclictest2286444-21sh19:30:010
227909499392,10cyclictest204-21dbus-daemon23:58:090
227909499392,10cyclictest204-21dbus-daemon23:58:080
227909499391,9cyclictest2293909-21grep19:50:170
227909499391,6cyclictest2363338-21/usr/sbin/munin23:10:200
227909499391,6cyclictest2356483-21ntp_kernel_err22:50:200
227909499391,6cyclictest2336059-21ntp_states21:50:200
227909499391,6cyclictest2299047-21/usr/sbin/munin20:05:110
227909499391,5cyclictest2385893-21latency_hist00:15:010
227909499391,5cyclictest2330823-21meminfo21:35:190
227909499391,4cyclictest2370122-21/usr/sbin/munin23:30:200
2279094993914,21cyclictest2319085-21grep21:00:240
227909499391,13cyclictest2296166-21tr19:55:220
227909499391,10cyclictest53-21kcompactd021:15:070
227909499391,10cyclictest2377712-21meminfo23:50:190
2279107993828,7cyclictest2391950-21runrttasks00:30:243
2279098993831,4cyclictest2355736-21munin-plugin-st22:50:011
227909499386,6cyclictest2388538-21ls00:20:260
227909499383,9cyclictest2311327-21irqstats20:40:180
227909499383,3cyclictest2359952-21ntp_states23:00:220
227909499382,5cyclictest2394304-21munin-run00:40:000
227909499381,9cyclictest2391653-21ntpq00:30:200
227909499381,7cyclictest2332341-21/usr/sbin/munin21:40:170
227909499381,4cyclictest2353166-21/usr/sbin/munin22:40:210
227909499381,11cyclictest2364818-21irqstats23:15:190
227909499381,10cyclictest2354251-21sshd22:45:050
2279103993728,6cyclictest2353574-21unixbench_multi22:40:252
227909499371,9cyclictest2328675-21apt-get21:30:100
227909499371,6cyclictest2345932-21fschecks_time22:20:160
227909499371,10cyclictest2375471-21df_inode23:45:160
2279103993629,3cyclictest0-21swapper/222:32:582
2279103993627,6cyclictest2310732-21sh20:40:012
2279103993625,8cyclictest2369582-21latency_hist23:30:002
227909499363,5cyclictest0-21swapper/023:35:200
227909499362,9cyclictest2321190-21apt-get21:10:030
227909499362,9cyclictest2321190-21apt-get21:10:020
227909499362,9cyclictest13550irq/42-30be0000.ethernet23:05:120
227909499361,9cyclictest2368231-21diskstats23:25:150
227909499361,5cyclictest2386696-21wc00:15:260
2279107993526,5cyclictest0-21swapper/319:50:173
227910399352,13cyclictest0-21swapper/200:00:132
227910399351,15cyclictest0-21swapper/222:00:132
2279098993526,6cyclictest2316253-21fschecks_count20:55:151
2279098993525,7cyclictest0-21swapper/123:20:001
2279098993520,11cyclictest231ktimers/120:40:121
2279107993424,6cyclictest0-21swapper/300:35:263
227909899344,4cyclictest131rcu_preempt19:30:121
2279098993427,4cyclictest0-21swapper/120:35:011
2279107993325,3cyclictest0-21swapper/323:35:013
2279103993325,5cyclictest2298343-21unixbench_multi20:00:262
2279098993326,4cyclictest2376994-21latency_hist23:50:011
2279098993326,4cyclictest2373193-21latency_hist23:40:001
2279107993224,5cyclictest2359120-21cron23:00:003
2279107993224,4cyclictest0-21swapper/321:20:243
2279103993225,5cyclictest2328125-21kworker/u8:2+events_unbound21:42:512
2279103993224,4cyclictest0-21swapper/219:20:252
2279103993221,7cyclictest0-21swapper/223:15:282
2279098993226,4cyclictest2377256-21kworker/u8:4+events_unbound23:53:081
2279098993224,4cyclictest0-21swapper/121:20:001
2279098993224,4cyclictest0-21swapper/100:34:591
2279098993222,6cyclictest0-21swapper/121:55:261
2279107993125,4cyclictest2279087-21cyclictest20:05:153
2279107993123,4cyclictest0-21swapper/320:15:173
227910399313,4cyclictest131rcu_preempt19:25:132
2279103993124,5cyclictest2377249-21kworker/u8:0+events_unbound00:24:422
2279103993124,4cyclictest2328444-21sh21:30:012
2279103993123,4cyclictest0-21swapper/221:55:012
2279103993121,7cyclictest2375542-21seq23:45:172
2279103993115,5cyclictest2333409-21kworker/u8:0+events_unbound21:55:212
2279098993125,3cyclictest0-21swapper/122:10:151
2279098993123,4cyclictest0-21swapper/123:40:261
2279098993123,4cyclictest0-21swapper/123:10:001
2279098993123,4cyclictest0-21swapper/121:35:161
2279107993024,4cyclictest2295617-21kworker/u8:0+events_unbound20:22:313
227910799302,24cyclictest2323185-21grep21:15:013
227910799302,24cyclictest2323185-21grep21:15:003
2279107993021,4cyclictest0-21swapper/320:42:353
2279103993023,5cyclictest2295630-21kworker/u8:3+events_unbound21:18:422
2279103993022,6cyclictest2278619-21kworker/u8:1+events_unbound19:50:132
2279103993022,4cyclictest0-21swapper/223:45:012
2279103993020,7cyclictest2359125-21sh23:00:002
227910399300,6cyclictest0-21swapper/220:25:122
2279098993023,4cyclictest2307339-21kthreadd20:30:001
2279098993023,4cyclictest0-21swapper/122:50:261
2279098993020,6cyclictest2387074-21grep00:15:271
2279098993020,6cyclictest0-21swapper/122:35:251
2279107992923,3cyclictest0-21swapper/322:10:253
2279107992922,5cyclictest2328125-21kworker/u8:2+events_unbound21:57:533
2279107992922,4cyclictest0-21swapper/321:10:013
2279107992921,4cyclictest0-21swapper/322:05:273
2279107992919,7cyclictest2282479-21ls19:15:253
2279107992919,6cyclictest2372767-21grep23:35:263
227910399299,12cyclictest32-21ksoftirqd/200:25:182
2279103992921,4cyclictest0-21swapper/221:30:242
2279103992921,4cyclictest0-21swapper/220:45:162
227910399290,5cyclictest0-21swapper/222:20:142
2279098992921,5cyclictest2328029-21unixbench_singl21:25:261
2279098992921,4cyclictest0-21swapper/123:30:181
2279098992919,7cyclictest2283024-21sh19:20:001
2279107992821,4cyclictest2357543-21latency_hist22:55:013
2279107992820,4cyclictest0-21swapper/300:05:253
2279107992819,6cyclictest0-21swapper/319:30:153
2279107992817,7cyclictest2351883-21grep22:35:263
2279103992822,4cyclictest2295617-21kworker/u8:0+events_unbound19:58:382
2279103992820,6cyclictest311ktimers/223:10:122
2279103992818,7cyclictest2351852-21unixbench_multi22:35:262
2279103992818,6cyclictest0-21swapper/220:15:262
2279103992813,6cyclictest131rcu_preempt23:00:122
2279103992811,8cyclictest131rcu_preempt22:10:162
2279098992822,4cyclictest2319107-21kworker/u8:0+events_unbound21:07:411
2279098992821,4cyclictest0-21swapper/100:10:271
2279098992820,4cyclictest0-21swapper/123:55:251
2279098992820,4cyclictest0-21swapper/123:55:251
2279098992820,4cyclictest0-21swapper/119:55:271
2279098992817,8cyclictest2364348-21latency_hist23:15:001
2279107992720,4cyclictest0-21swapper/300:00:253
2279107992719,5cyclictest2280821-21auditctl19:15:023
2279107992719,4cyclictest2380455-21sendmail00:00:013
2279107992719,4cyclictest2380455-21sendmail00:00:013
2279107992719,4cyclictest0-21swapper/320:45:263
2279107992710,4cyclictest131rcu_preempt19:30:013
227910399277,11cyclictest32-21ksoftirqd/200:35:192
2279103992721,4cyclictest2289528-21kworker/u8:0+events_unbound19:45:132
2279103992720,5cyclictest2295630-21kworker/u8:3+events_unbound21:22:312
2279103992720,5cyclictest2295630-21kworker/u8:3+events_unbound20:09:482
2279103992720,5cyclictest2295617-21kworker/u8:0+events_unbound20:30:112
2279103992719,6cyclictest2295630-21kworker/u8:3+events_unbound20:11:042
2279103992719,6cyclictest2295630-21kworker/u8:3+events_unbound20:11:042
2279103992719,5cyclictest2333409-21kworker/u8:0+events_unbound22:45:162
2279103992719,5cyclictest0-21swapper/223:35:262
2279098992719,4cyclictest0-21swapper/121:30:251
2279098992718,5cyclictest0-21swapper/119:20:251
2279098992717,5cyclictest0-21swapper/100:35:171
2279098992715,8cyclictest204-21dbus-daemon22:22:561
227909899271,4cyclictest131rcu_preempt20:45:141
2279098992714,9cyclictest2295178-21apt-get19:55:011
2279107992619,5cyclictest2363771-21unixbench_multi23:10:243
2279107992617,6cyclictest0-21swapper/323:05:253
2279103992618,6cyclictest2343479-21kworker/u8:1+events_unbound22:28:312
2279103992618,6cyclictest2343479-21kworker/u8:1+events_unbound22:28:312
2279103992618,4cyclictest0-21swapper/219:40:182
2279098992620,3cyclictest0-21swapper/100:10:021
2279098992619,3cyclictest0-21swapper/100:25:261
2279098992618,4cyclictest2289848-21latency_hist19:40:011
2279098992618,4cyclictest0-21swapper/122:55:171
2279098992617,7cyclictest145-21jbd2/mmcblk1p2-819:40:291
2279107992518,4cyclictest0-21swapper/320:10:243
2279107992518,4cyclictest0-21swapper/320:10:243
2279107992518,3cyclictest0-21swapper/320:40:013
2279107992517,5cyclictest0-21swapper/300:25:013
2279107992516,6cyclictest0-21swapper/319:55:223
2279107992516,5cyclictest0-21swapper/323:23:033
2279107992516,5cyclictest0-21swapper/321:00:023
2279107992515,6cyclictest2346793-21grep22:20:263
2279107992513,8cyclictest2315392-21grep20:50:263
227910399258,10cyclictest32-21ksoftirqd/219:30:202
2279103992519,3cyclictest0-21swapper/220:42:352
2279103992517,4cyclictest0-21swapper/200:05:252
2279103992515,7cyclictest2331224-21cat21:35:242
2279107992417,4cyclictest0-21swapper/323:00:263
2279107992416,5cyclictest0-21swapper/321:40:163
227910399248,9cyclictest32-21ksoftirqd/221:10:172
227910399248,9cyclictest32-21ksoftirqd/221:10:172
227910399248,9cyclictest32-21ksoftirqd/200:30:132
227910399244,4cyclictest131rcu_preempt23:55:132
227910399244,4cyclictest131rcu_preempt23:55:122
2279103992419,2cyclictest0-21swapper/221:45:162
2279103992417,5cyclictest2317813-21latency_hist21:00:022
227910399241,6cyclictest0-21swapper/223:20:092
2279103992416,6cyclictest2383700-21kworker/u8:2+events_unbound00:15:132
2279098992418,3cyclictest0-21swapper/122:05:171
2279098992417,5cyclictest2388962-21kworker/u8:1+events_unbound00:24:401
2279098992415,6cyclictest0-21swapper/121:45:281
2279098992412,8cyclictest0-21swapper/122:40:241
2279107992317,3cyclictest0-21swapper/319:35:233
2279107992317,3cyclictest0-21swapper/300:25:263
2279107992316,5cyclictest2345312-21kworker/u8:3+events_unbound22:30:573
2279107992316,4cyclictest0-21swapper/323:20:003
2279107992314,6cyclictest2368448-21irqstats23:25:163
227910399236,6cyclictest131rcu_preempt21:05:122
2279103992317,4cyclictest2383704-21kworker/u8:3+events_unbound00:10:292
2279103992317,4cyclictest2340616-21kworker/u8:2+events_unbound22:16:362
2279103992317,3cyclictest0-21swapper/223:50:182
227910399231,4cyclictest0-21swapper/223:30:172
2279098992318,3cyclictest0-21swapper/123:20:061
2279098992316,5cyclictest145-21jbd2/mmcblk1p2-820:10:161
2279098992316,5cyclictest145-21jbd2/mmcblk1p2-820:10:151
2279098992316,5cyclictest0-21swapper/119:50:011
2279098992315,5cyclictest0-21swapper/119:30:021
2279098992314,6cyclictest1-21systemd20:37:341
227910799229,9cyclictest204-21dbus-daemon22:42:593
2279107992216,3cyclictest0-21swapper/322:25:133
2279107992216,3cyclictest0-21swapper/322:25:133
2279107992216,3cyclictest0-21swapper/319:20:243
2279107992214,5cyclictest0-21swapper/322:15:213
2279107992213,5cyclictest0-21swapper/319:45:253
2279098992216,4cyclictest2345312-21kworker/u8:3+events_unbound22:30:401
2279098992216,4cyclictest2333575-21idleruntime-cro21:45:001
2279098992216,4cyclictest2333409-21kworker/u8:0+events_unbound23:03:021
2279098992216,3cyclictest0-21swapper/120:05:201
227909899221,4cyclictest0-21swapper/120:02:041
2279107992115,3cyclictest0-21swapper/323:40:203
2279107992115,3cyclictest0-21swapper/321:50:153
2279107992114,5cyclictest391ktimers/300:20:023
2279107992114,5cyclictest2330909-21ntpq21:35:203
2279107992113,5cyclictest2384487-21fschecks_time00:10:163
2279107992112,6cyclictest2377629-21cut23:50:183
2279107992111,6cyclictest0-21swapper/320:05:013
2279098992115,4cyclictest2333409-21kworker/u8:0+events_unbound22:03:211
2279098992112,6cyclictest0-21swapper/100:05:001
227910799202,13cyclictest2328044-21head21:25:263
2279107992014,3cyclictest0-21swapper/321:30:143
2279107992012,5cyclictest0-21swapper/322:45:203
227910799197,8cyclictest1-21systemd21:02:393
227909899193,13cyclictest2368110-21munin-node23:25:111
227909899191,10cyclictest0-21swapper/120:53:541
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional