You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 06:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed May 27, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1881773995311,26cyclictest0-21swapper/022:00:000
1881777994941,5cyclictest1982067-21sendmail00:00:011
1881777994614,6cyclictest131rcu_preempt23:45:111
1881773994637,6cyclictest1897221-21grep19:50:260
1881777994437,5cyclictest1947567-21kworker/u8:3+events_unbound00:22:331
1881773994433,6cyclictest1934094-21sh21:40:000
188177399442,9cyclictest1958047-21sshd22:48:190
188177399440,9cyclictest1914738-21/usr/sbin/munin20:40:250
1881782994332,7cyclictest1950903-21grep22:25:252
1881773994336,4cyclictest1921940-21cron21:05:000
1881773994323,16cyclictest0-21swapper/021:00:010
188177399431,9cyclictest1895244-21/usr/sbin/munin19:45:210
1881773994312,26cyclictest0-21swapper/021:20:010
188177399431,10cyclictest1942149-21/usr/sbin/munin22:00:260
188178299428,14cyclictest1907342-21sed20:20:212
1881777994233,5cyclictest0-21swapper/122:55:011
188177399429,3cyclictest0-21swapper/022:25:170
188177399422,8cyclictest1953605-21/usr/sbin/munin22:35:150
188177399422,6cyclictest1906581-21cron20:20:000
188177399421,9cyclictest1981210-21/usr/sbin/munin23:55:230
188177399421,9cyclictest1903413-21/usr/sbin/munin20:10:110
188177399421,9cyclictest1892900-21/usr/sbin/munin19:40:160
188177399421,8cyclictest1952472-21/usr/sbin/munin22:30:240
1881782994113,10cyclictest1977068-21cut23:45:002
1881782994113,10cyclictest1977068-21cut23:45:002
1881777994131,6cyclictest0-21swapper/100:32:341
188177399419,5cyclictest1979890-21unixbench_multi23:50:260
1881773994135,4cyclictest1908727-21fschecks_time20:25:170
1881773994134,4cyclictest1949272-21unixbench_singl22:20:270
1881773994134,4cyclictest1949272-21unixbench_singl22:20:260
1881773994133,5cyclictest1968419-21munin-plugin-st23:20:010
188177399412,6cyclictest1987837-21fschecks_time00:15:160
188177399411,9cyclictest1962519-21/usr/sbin/munin23:00:240
188177399411,9cyclictest1934974-21/usr/sbin/munin21:40:230
1881773994119,18cyclictest0-21swapper/019:15:010
188177399411,5cyclictest1922343-21cut21:05:160
188177399411,10cyclictest1960527-21/usr/sbin/munin22:55:190
1881782994030,6cyclictest1964530-21unixbench_multi23:05:262
1881782994010,12cyclictest1882802-21timerandwakeup19:10:242
1881777994033,5cyclictest1947567-21kworker/u8:3+events_unbound00:07:321
1881777994033,5cyclictest1928613-21kworker/u8:2+events_unbound21:20:401
188177399408,6cyclictest1992038-21grep00:25:230
188177399403,8cyclictest1915915-21/usr/sbin/munin20:45:220
188177399403,6cyclictest1912155-21grep20:35:180
188177399403,6cyclictest1912155-21grep20:35:170
188177399403,6cyclictest1907896-21head20:20:280
188177399402,8cyclictest1890814-21cron19:35:010
188177399402,6cyclictest1899362-21sendmail20:00:010
188177399402,11cyclictest1944985-21grep22:10:160
188177399401,9cyclictest1946921-21meminfo22:15:170
188177399401,8cyclictest1982637-21/usr/sbin/munin00:00:160
188177399401,8cyclictest1976946-21cron23:45:000
188177399401,8cyclictest1976946-21cron23:45:000
188177399401,8cyclictest1884368-21fschecks_count19:15:160
188177399401,11cyclictest1969663-21unixbench_multi23:20:250
188177399401,10cyclictest1971831-21cron23:30:000
1881773994010,27cyclictest0-21swapper/021:10:170
1881786993931,4cyclictest1892529-21munin-plugin-st19:40:013
1881777993931,5cyclictest1961602-21munin-plugin-st23:00:001
1881777993931,5cyclictest1918518-21cron20:55:001
1881777993930,7cyclictest1867129-21kworker/u8:3+events_unbound19:11:311
188177399395,5cyclictest1972202-21fschecks_count23:30:140
188177399392,8cyclictest1911284-21head20:30:270
188177399392,7cyclictest1944114-21grep22:05:240
188177399392,6cyclictest1937053-21grep21:45:240
188177399392,6cyclictest1932379-21cron21:35:000
188177399392,6cyclictest1932379-21cron21:34:590
188177399392,5cyclictest1977397-21fschecks_time23:45:150
188177399392,4cyclictest1955182-21grep22:40:140
188177399392,12cyclictest1928514-21grep21:20:250
188177399391,9cyclictest1995908-21cron00:40:010
188177399391,9cyclictest1985164-21/usr/sbin/munin00:05:280
188177399391,12cyclictest1986124-21fschecks_time00:10:150
188177399391,10cyclictest1929795-21ntp_states21:25:220
1881786993829,6cyclictest0-21swapper/321:05:013
1881786993825,9cyclictest391ktimers/319:20:123
1881782993826,8cyclictest1974106-21/usr/sbin/munin23:35:192
1881782993825,8cyclictest1918255-21sshd20:51:482
1881782993812,10cyclictest1959930-21latency_hist22:55:002
188177799380,15cyclictest0-21swapper/123:35:121
188177399382,9cyclictest1918519-21cron20:55:010
188177399382,10cyclictest1888695-21grep19:25:240
188177399381,9cyclictest1900868-21head20:00:290
188177399381,8cyclictest1965935-21wc23:10:240
188177399381,7cyclictest1964536-21grep23:05:250
188177399381,7cyclictest1886139-21fschecks_time19:20:160
1881773993812,23cyclictest0-21swapper/023:35:150
188177399381,10cyclictest1994215-21debian-sa100:35:010
1881786993715,12cyclictest0-21swapper/321:10:273
188178299379,10cyclictest1955588-21ntp_states22:40:212
1881782993724,7cyclictest1929566-21/usr/sbin/munin21:25:192
1881782993713,9cyclictest1946836-21kernelversion22:15:182
1881782993713,8cyclictest1886610-21processes19:20:242
188177399378,2cyclictest0-21swapper/021:50:030
188177399373,6cyclictest1901614-21df20:05:150
188177399372,5cyclictest1959883-21cron22:55:010
188177399371,9cyclictest1989910-21ntp_states00:20:220
1881786993628,4cyclictest1942854-21munin-plugin-st22:05:013
1881782993627,6cyclictest1900822-21unixbench_multi20:00:282
1881782993627,6cyclictest0-21swapper/200:35:172
1881782993622,11cyclictest1993725-21/usr/sbin/munin00:30:262
1881782993621,10cyclictest1934860-21/usr/sbin/munin21:40:202
1881777993629,4cyclictest0-21swapper/121:20:001
1881782993528,3cyclictest1936040-21apt-get21:45:122
1881782993524,7cyclictest1901846-21/usr/sbin/munin20:05:192
1881782993522,8cyclictest1893613-21/usr/sbin/munin19:40:242
1881782993521,11cyclictest1972905-21/usr/sbin/munin23:30:242
1881782993519,12cyclictest1922322-21fschecks_count21:05:172
1881782993519,11cyclictest1966700-21cron23:15:012
1881782993513,5cyclictest1960234-21df_inode22:55:142
188177799353,6cyclictest0-21swapper/119:30:121
188177399354,6cyclictest1881858-21date19:10:000
1881786993426,4cyclictest0-21swapper/323:40:153
1881786993426,4cyclictest0-21swapper/323:40:153
188178299348,9cyclictest1884797-21ntp_states19:15:222
1881782993427,4cyclictest1921955-21sh21:05:012
1881782993422,9cyclictest1985472-21sshd00:07:312
1881782993421,10cyclictest1889561-21/usr/sbin/munin19:30:152
1881782993419,11cyclictest1978850-21cut23:50:012
1881782993416,13cyclictest1927290-21sendmail21:20:002
1881782993415,10cyclictest0-21swapper/219:55:102
1881782993411,9cyclictest1910617-21latency20:30:182
1881777993428,4cyclictest1937265-21kworker/u8:1+events_unbound22:08:401
1881777993427,4cyclictest1945324-21kworker/u8:0+events_unbound22:23:421
1881777993427,4cyclictest1945324-21kworker/u8:0+events_unbound22:23:421
1881777993425,6cyclictest1930685-21sh21:29:591
1881777993416,5cyclictest1887657-21kworker/u8:0+events_unbound20:28:211
188178699339,5cyclictest40-21ksoftirqd/321:30:193
188178699339,5cyclictest40-21ksoftirqd/321:30:193
188178699338,5cyclictest40-21ksoftirqd/319:55:173
188178699336,7cyclictest0-21swapper/300:00:203
1881782993327,3cyclictest1941374-21apt-get22:00:132
1881782993323,7cyclictest1982976-21/usr/sbin/munin00:00:202
1881782993322,8cyclictest1937987-21/usr/sbin/munin21:50:182
1881782993321,9cyclictest1991279-21/usr/sbin/munin00:25:132
1881782993320,9cyclictest0-21swapper/220:15:132
1881782993320,10cyclictest1904738-21sshd20:13:192
1881782993319,11cyclictest1897387-21cron19:52:012
1881782993319,10cyclictest1894478-21/usr/sbin/munin19:45:112
1881782993317,13cyclictest1989953-21/usr/sbin/munin00:20:232
1881782993315,13cyclictest1941122-21auditctl22:00:012
1881782993314,15cyclictest1944644-21cat22:10:012
188177799335,5cyclictest131rcu_preempt21:40:111
188177799333,6cyclictest131rcu_preempt22:40:121
1881777993326,4cyclictest0-21swapper/121:10:271
1881777993325,5cyclictest1989020-21kworker/u8:0+events_unbound00:35:411
1881777993318,5cyclictest1937265-21kworker/u8:1+events_unbound22:27:411
188177799331,5cyclictest131rcu_preempt20:40:121
1881777993314,6cyclictest1887657-21kworker/u8:0+events_unbound21:00:281
1881786993225,4cyclictest1995922-21munin-plugin-st00:40:013
1881786993221,7cyclictest1900828-21grep20:00:293
1881786993210,6cyclictest381rcuc/322:10:223
1881782993226,3cyclictest1930922-21apt-get21:30:142
1881782993226,3cyclictest1930922-21apt-get21:30:132
1881782993224,5cyclictest1978885-21apt-get23:50:142
1881782993222,7cyclictest1952172-21/usr/sbin/munin22:30:212
1881782993222,7cyclictest1928042-21/usr/sbin/munin21:20:212
1881782993222,7cyclictest1920139-21cron20:59:012
1881782993222,7cyclictest1915704-21/usr/sbin/munin20:45:182
1881782993222,7cyclictest1912629-21/usr/sbin/munin20:35:252
1881782993222,7cyclictest1912629-21/usr/sbin/munin20:35:252
1881782993221,8cyclictest1970643-21/usr/sbin/munin23:25:172
1881782993221,8cyclictest1914184-21/usr/sbin/munin20:40:202
1881782993221,7cyclictest1953304-21/usr/sbin/munin22:35:122
1881782993220,9cyclictest1957476-21/usr/sbin/munin22:45:242
1881782993214,13cyclictest1934208-21latency_hist21:40:002
1881777993225,4cyclictest0-21swapper/123:05:001
1881777993223,5cyclictest0-21swapper/100:27:341
188178699318,5cyclictest40-21ksoftirqd/300:25:233
1881786993124,4cyclictest0-21swapper/320:50:173
1881786993124,3cyclictest0-21swapper/321:55:003
1881786993120,7cyclictest0-21swapper/323:55:253
1881782993125,3cyclictest1968665-21apt-get23:20:122
1881782993125,3cyclictest1962143-21if_eth023:00:162
1881782993121,7cyclictest1945216-21/usr/sbin/munin22:10:182
1881782993120,8cyclictest1967061-21diskstats23:15:142
1881782993119,9cyclictest1981738-21sshd23:56:172
1881782993118,10cyclictest1891728-21/usr/sbin/munin19:35:222
1881782993113,11cyclictest1909554-21grep20:25:252
188177799319,3cyclictest199-21systemd-network20:10:061
1881777993125,4cyclictest1894086-21kworker/u8:1+events_unbound20:16:441
188178699308,4cyclictest131rcu_preempt22:15:173
1881786993020,7cyclictest438-21cron22:40:013
1881786993020,6cyclictest0-21swapper/322:45:263
1881786993019,6cyclictest0-21swapper/321:15:263
1881786993010,11cyclictest40-21ksoftirqd/319:45:113
1881782993021,7cyclictest1948208-21/usr/sbin/munin22:20:152
1881782993021,7cyclictest1948208-21/usr/sbin/munin22:20:142
1881782993016,10cyclictest1986125-21fschecks_time00:10:162
1881782993013,11cyclictest1924159-21/usr/sbin/munin21:10:172
1881777993022,5cyclictest1888651-21unixbench_multi19:25:251
1881777993022,4cyclictest0-21swapper/120:20:281
1881777993021,7cyclictest1937898-21cut21:50:171
1881777993020,6cyclictest0-21swapper/122:45:261
1881777993010,5cyclictest131rcu_preempt19:35:111
188178699297,7cyclictest0-21swapper/319:25:233
188178699295,6cyclictest40-21ksoftirqd/323:00:133
1881786992915,4cyclictest131rcu_preempt20:05:153
188177799296,5cyclictest131rcu_preempt21:35:131
188177799293,4cyclictest131rcu_preempt23:50:131
1881777992922,5cyclictest1937265-21kworker/u8:1+events_unbound22:35:351
1881777992921,4cyclictest0-21swapper/123:10:011
1881777992920,7cyclictest1975624-21fschecks_count23:40:151
1881777992920,7cyclictest1975624-21fschecks_count23:40:151
1881777992914,10cyclictest231ktimers/119:55:111
188177799291,14cyclictest0-21swapper/100:15:521
188178699288,6cyclictest0-21swapper/323:35:143
188178699288,5cyclictest381rcuc/321:20:173
188178699287,5cyclictest40-21ksoftirqd/321:30:013
188178699282,10cyclictest1910354-21sh20:30:163
1881786992820,4cyclictest0-21swapper/320:25:173
1881786992818,7cyclictest391ktimers/321:36:143
1881786992818,6cyclictest0-21swapper/319:40:253
1881786992818,4cyclictest131rcu_preempt19:50:393
1881786992816,5cyclictest131rcu_preempt23:23:253
1881786992810,11cyclictest131rcu_preempt00:20:133
1881777992819,5cyclictest0-21swapper/122:00:011
1881777992816,6cyclictest1947567-21kworker/u8:3+events_unbound23:12:201
1881777992815,8cyclictest0-21swapper/120:40:011
1881777992815,8cyclictest0-21swapper/120:40:001
1881777992813,6cyclictest1904604-21kworker/u8:2+events_unbound20:31:491
188178699279,9cyclictest131rcu_preempt22:34:333
188178699274,5cyclictest40-21ksoftirqd/300:35:013
1881786992719,4cyclictest0-21swapper/300:05:183
1881786992718,5cyclictest0-21swapper/323:54:593
1881786992716,7cyclictest1-21systemd21:57:073
1881786992715,9cyclictest1950251-21cut22:25:193
1881786992714,4cyclictest1881766-21cyclictest21:07:153
1881786992714,4cyclictest131rcu_preempt22:40:193
1881786992712,4cyclictest131rcu_preempt23:46:033
1881786992710,10cyclictest1881766-21cyclictest20:44:553
1881777992720,5cyclictest1921632-21kworker/u8:1+events_unbound21:32:041
1881777992720,5cyclictest1921632-21kworker/u8:1+events_unbound21:32:041
188177799271,4cyclictest0-21swapper/122:00:141
188177799271,3cyclictest1985921-21apt-get00:10:121
188178699268,12cyclictest40-21ksoftirqd/319:16:443
188178699266,4cyclictest0-21swapper/322:21:323
188178699266,4cyclictest0-21swapper/322:21:313
188178699266,12cyclictest0-21swapper/323:34:383
188178699263,7cyclictest40-21ksoftirqd/323:25:223
1881786992621,3cyclictest0-21swapper/300:10:123
1881786992618,5cyclictest0-21swapper/321:00:013
1881786992618,4cyclictest0-21swapper/323:15:163
1881786992617,7cyclictest391ktimers/320:47:053
1881786992617,6cyclictest0-21swapper/320:40:013
1881786992617,6cyclictest0-21swapper/320:40:003
1881786992613,7cyclictest131rcu_preempt19:14:143
1881786992613,4cyclictest131rcu_preempt23:08:183
188178699261,19cyclictest0-21swapper/320:11:243
1881777992620,3cyclictest1923661-21diskmemload21:10:021
1881777992620,3cyclictest0-21swapper/119:50:271
188177799261,9cyclictest1895950-21munin-plugin-st19:50:011
1881777992619,5cyclictest1887657-21kworker/u8:0+events_unbound20:00:131
1881777992618,4cyclictest0-21swapper/123:15:161
1881777992618,4cyclictest0-21swapper/119:40:151
1881777992612,6cyclictest1916617-21kworker/u8:3+events_unbound20:47:351
188177799261,13cyclictest0-21swapper/120:05:041
188178699259,9cyclictest131rcu_preempt20:21:513
1881786992518,4cyclictest1966275-21unixbench_singl23:10:263
1881786992514,4cyclictest131rcu_preempt22:52:163
188178699251,16cyclictest0-21swapper/322:56:063
188178699251,14cyclictest0-21swapper/321:43:583
1881786992510,8cyclictest131rcu_preempt22:05:463
1881777992519,4cyclictest1858173-21kworker/u8:2+events_unbound19:16:331
188177799251,5cyclictest0-21swapper/120:55:121
188177799251,12cyclictest0-21swapper/121:46:271
188178699246,12cyclictest40-21ksoftirqd/300:18:423
188178699241,16cyclictest0-21swapper/321:48:503
188178699241,13cyclictest0-21swapper/320:18:343
1881786992410,7cyclictest131rcu_preempt19:31:263
188177799246,10cyclictest0-21swapper/122:15:201
188177799245,4cyclictest0-21swapper/122:30:161
188177799244,12cyclictest1983206-21grep00:00:251
1881777992417,5cyclictest1964953-21kworker/u8:1+events_unbound23:33:561
1881777992417,5cyclictest1858173-21kworker/u8:2+events_unbound19:23:171
188177799231,12cyclictest0-21swapper/122:14:281
1881777992214,6cyclictest1964953-21kworker/u8:1+events_unbound23:20:561
188177799221,11cyclictest0-21swapper/123:25:461
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional