You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 04:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Mar 25, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1907188995033,13cyclictest311ktimers/221:45:122
1907193994839,5cyclictest1992973-21unixbench_singl23:15:233
1907188994841,5cyclictest1997267-21kworker/u8:3+events_unbound23:42:092
1907193994722,22cyclictest391ktimers/319:50:113
1907184994735,8cyclictest1931817-21idleruntime-cro20:20:001
1907193994537,4cyclictest0-21swapper/322:25:003
1907193994534,8cyclictest2014100-21sendmail00:20:013
1907184994538,5cyclictest1892164-21kworker/u8:1+events_unbound19:46:241
190719399442,37cyclictest1943726-21cron20:55:003
1907188994437,5cyclictest1969298-21kworker/u8:0+events_unbound22:31:552
190719399432,36cyclictest1921355-21cron19:50:013
1907188994334,6cyclictest1911121-21apt19:20:002
1907188994334,6cyclictest1911121-21apt19:20:002
1907182994336,4cyclictest2014082-21munin-plugin-st00:20:000
1907182994336,4cyclictest1985251-21fschecks_count22:55:150
1907193994235,4cyclictest2006635-21unixbench_singl23:55:263
1907193994231,7cyclictest391ktimers/323:35:133
190719399421,5cyclictest1945688-21apt-get21:00:143
190719399421,3cyclictest0-21swapper/323:10:113
1907193994213,26cyclictest2010904-21/usr/sbin/munin00:10:263
1907184994236,4cyclictest1958685-21kworker/u8:3+events_unbound22:57:001
1907182994213,10cyclictest1965302-21sed21:55:250
190719399412,35cyclictest1962025-21grep21:45:263
190719399411,3cyclictest1914773-21apt-get19:30:123
190719399411,3cyclictest0-21swapper/319:35:133
190719399411,35cyclictest1952258-21cron21:20:003
190719399411,13cyclictest2005353-21basename23:55:003
190719399410,3cyclictest0-21swapper/320:35:123
190718899411,16cyclictest0-21swapper/220:40:122
190718299419,7cyclictest0-21swapper/021:20:000
190718299417,17cyclictest0-21swapper/023:55:190
1907182994118,18cyclictest1961572-21/usr/sbin/munin21:45:210
190719399403,32cyclictest1971227-21cron22:15:013
190719399403,32cyclictest1971227-21cron22:15:003
190719399402,33cyclictest1981474-21cron22:45:003
1907193994015,22cyclictest0-21swapper/319:45:023
190719399401,3cyclictest0-21swapper/320:55:113
190719399401,3cyclictest0-21swapper/320:00:133
190719399401,31cyclictest1982246-21ntpq22:45:203
190719399401,10cyclictest1996343-21unixbench_multi23:25:263
190718899401,14cyclictest0-21swapper/200:35:132
1907188994010,4cyclictest0-21swapper/221:25:122
1907182994033,4cyclictest2020921-21sendmail00:40:010
1907182994014,23cyclictest1925130-21latency_hist20:00:010
190719399397,29cyclictest1989488-21cat23:05:253
190719399393,32cyclictest1960368-21grep21:40:253
190719399393,30cyclictest2016194-21fschecks_count00:25:163
190719399392,8cyclictest2010286-21/usr/sbin/munin00:05:253
190719399392,32cyclictest2020994-21cat00:40:003
190719399392,32cyclictest1972933-21cron22:20:003
190719399392,31cyclictest2002403-21fschecks_time23:45:173
190719399392,11cyclictest1977223-21ps22:30:233
190719399391,9cyclictest2000629-21/usr/sbin/munin23:40:163
190719399391,5cyclictest1985112-21apt-get22:55:123
190719399391,4cyclictest1953501-21/usr/sbin/munin21:20:253
190719399391,3cyclictest2007292-21apt-get00:00:113
190719399391,3cyclictest1907399-21/usr/sbin/munin19:10:153
190719399391,3cyclictest1907399-21/usr/sbin/munin19:10:143
190719399391,3cyclictest0-21swapper/320:15:103
190719399391,3cyclictest0-21swapper/300:30:113
190719399391,34cyclictest1995113-21cron23:25:003
190719399391,33cyclictest2014437-21df00:20:143
1907188993929,6cyclictest0-21swapper/219:15:032
1907188993929,6cyclictest0-21swapper/219:15:022
1907184993928,8cyclictest1999777-21grep23:35:271
190719399382,6cyclictest1959079-21cron21:40:013
190719399382,4cyclictest1954217-21apt-get21:25:123
1907193993822,12cyclictest0-21swapper/319:55:243
190719399381,9cyclictest1947386-21/usr/sbin/munin21:05:193
190719399381,9cyclictest1947386-21/usr/sbin/munin21:05:183
190719399381,4cyclictest1997052-21apt-get23:30:133
190719399381,4cyclictest1997052-21apt-get23:30:133
190719399381,4cyclictest1983988-21ntp_states22:50:233
190719399381,4cyclictest1964595-21/usr/sbin/munin21:55:173
190719399381,4cyclictest1939945-21users20:40:273
190719399381,3cyclictest1975011-21diskstats22:25:163
190719399381,3cyclictest0-21swapper/320:30:143
190719399381,3cyclictest0-21swapper/320:20:133
190719399381,3cyclictest0-21swapper/319:25:143
190719399381,10cyclictest1963746-21grep21:50:243
190719399380,3cyclictest0-21swapper/322:05:133
190719399380,3cyclictest0-21swapper/322:05:133
1907188993821,5cyclictest1958685-21kworker/u8:3+events_unbound22:47:462
1907184993832,4cyclictest1958685-21kworker/u8:3+events_unbound21:51:501
1907182993811,25cyclictest0-21swapper/023:30:250
1907182993811,25cyclictest0-21swapper/023:30:250
190719399372,8cyclictest1934436-21sendmail_mailqu20:25:233
190719399371,4cyclictest1909549-21memory19:15:043
190719399371,4cyclictest1909549-21memory19:15:043
190719399371,3cyclictest1986830-21apt-get23:00:143
190719399371,3cyclictest1949666-21ntp_states21:10:223
190719399371,3cyclictest1949666-21ntp_states21:10:223
190719399371,3cyclictest1940566-21apt-get20:45:133
190719399371,3cyclictest0-21swapper/322:00:123
190719399371,3cyclictest0-21swapper/320:05:133
190719399371,3cyclictest0-21swapper/319:20:113
190719399370,3cyclictest0-21swapper/322:35:143
190719399370,3cyclictest0-21swapper/321:30:123
1907188993729,4cyclictest0-21swapper/221:40:022
1907188993729,4cyclictest0-21swapper/221:00:162
1907188993720,14cyclictest1907173-21cyclictest19:30:132
1907184993728,6cyclictest0-21swapper/122:00:001
1907184993727,7cyclictest0-21swapper/121:25:261
1907184993727,6cyclictest0-21swapper/122:15:241
1907184993726,7cyclictest0-21swapper/122:35:261
1907184993726,7cyclictest0-21swapper/120:35:011
1907182993726,8cyclictest2014679-21/usr/sbin/munin00:20:180
1907182993726,7cyclictest1975501-21/usr/sbin/munin22:25:230
1907182993723,10cyclictest121ktimers/000:00:090
1907182993721,10cyclictest1999204-21/usr/sbin/munin23:35:200
1907182993719,14cyclictest1921356-21cron19:50:020
1907182993717,7cyclictest2003869-21apt-get23:50:110
190719399361,8cyclictest1929671-21unixbench_singl20:10:283
1907188993629,4cyclictest0-21swapper/223:05:162
1907188993626,6cyclictest0-21swapper/223:05:002
1907184993627,6cyclictest0-21swapper/100:20:001
1907184993626,6cyclictest0-21swapper/123:10:241
1907182993629,4cyclictest1921592-21apt-get19:50:110
1907182993627,4cyclictest0-21swapper/023:02:000
1907182993626,6cyclictest1926278-21grep20:00:260
1907182993620,12cyclictest1918728-21tr19:40:200
1907184993526,6cyclictest0-21swapper/120:50:001
1907184993525,6cyclictest0-21swapper/119:50:251
1907184993525,6cyclictest0-21swapper/119:35:261
1907184993524,7cyclictest0-21swapper/123:45:251
1907182993528,4cyclictest1949093-21apt-get21:10:130
1907182993528,4cyclictest1949093-21apt-get21:10:130
1907182993521,11cyclictest1929598-21sed20:10:280
1907182993519,13cyclictest2009198-21apt-get00:05:070
1907182993519,13cyclictest1980765-21/usr/sbin/munin22:40:240
1907182993517,13cyclictest1969866-21cstates22:10:150
1907182993517,13cyclictest1969866-21cstates22:10:150
1907182993517,13cyclictest1956127-21sort21:30:160
1907182993516,15cyclictest1960816-21munin-plugin-st21:45:010
1907182993516,10cyclictest2011215-21if_eth100:10:190
1907182993515,15cyclictest1947941-21ntpq21:05:240
1907182993515,15cyclictest1947941-21ntpq21:05:230
1907182993514,17cyclictest629-21snmpd23:05:180
1907182993514,17cyclictest0-21swapper/022:00:110
1907188993428,4cyclictest2008948-21apt-key00:05:012
1907188993428,4cyclictest1940259-21kworker/u8:2+events_unbound20:51:382
1907188993425,5cyclictest0-21swapper/220:50:002
1907188993416,4cyclictest1926338-21kworker/u8:2+events_unbound20:10:592
1907184993426,5cyclictest1993415-21run-parts23:20:001
1907184993426,4cyclictest0-21swapper/100:35:211
1907184993424,7cyclictest0-21swapper/120:50:251
1907184993423,8cyclictest0-21swapper/119:35:011
1907184993423,7cyclictest0-21swapper/100:05:171
1907182993426,4cyclictest1937309-21fschecks_count20:35:160
1907182993424,6cyclictest0-21swapper/022:50:170
1907182993421,9cyclictest1964197-21cron21:55:010
1907182993419,10cyclictest1947145-21cron21:05:000
1907182993418,13cyclictest1936519-21/usr/sbin/munin20:30:270
1907182993418,11cyclictest1940853-21/usr/sbin/munin20:45:160
1907182993417,13cyclictest1993762-21df23:20:140
1907182993417,10cyclictest2002400-21fschecks_time23:45:160
1907182993414,16cyclictest1927508-21seq20:05:230
1907182993410,12cyclictest1969146-21awk22:05:270
1907182993410,12cyclictest1969146-21awk22:05:270
1907188993327,4cyclictest1993188-21kworker/u8:0+events_unbound23:27:042
1907188993327,4cyclictest1949778-21kworker/u8:1+events_unbound21:41:472
190718899330,6cyclictest0-21swapper/223:15:102
1907184993326,4cyclictest0-21swapper/123:55:161
1907184993325,6cyclictest1958685-21kworker/u8:3+events_unbound22:30:411
1907184993324,6cyclictest0-21swapper/119:45:011
1907184993322,7cyclictest0-21swapper/122:50:281
1907184993321,5cyclictest1949778-21kworker/u8:1+events_unbound21:10:251
1907184993321,5cyclictest1949778-21kworker/u8:1+events_unbound21:10:251
1907182993326,3cyclictest1930322-21apt-get20:15:110
1907182993323,5cyclictest0-21swapper/022:20:240
1907182993319,9cyclictest1939314-21memory20:40:190
1907182993319,11cyclictest1913359-21/usr/sbin/munin19:25:170
1907182993318,10cyclictest13550irq/42-30be0000.ethernet21:20:120
1907182993316,13cyclictest2018727-21unixbench_multi00:30:240
1907182993315,13cyclictest1993595-21latency_hist23:20:010
1907182993314,14cyclictest1917943-21latency_hist19:40:020
1907182993312,7cyclictest1913055-21latency_hist19:25:010
1907188993226,4cyclictest2006704-21kworker/u8:0+events_unbound23:57:142
1907188993226,3cyclictest2010678-21kthreadd00:10:002
1907188993225,5cyclictest1949778-21kworker/u8:1+events_unbound21:11:432
1907188993225,5cyclictest1949778-21kworker/u8:1+events_unbound21:11:432
1907184993224,4cyclictest0-21swapper/121:35:161
1907184993224,4cyclictest0-21swapper/100:20:251
1907184993223,6cyclictest0-21swapper/120:40:251
1907184993223,6cyclictest0-21swapper/120:40:001
1907184993222,6cyclictest0-21swapper/122:50:001
1907184993221,7cyclictest0-21swapper/100:25:261
1907184993215,6cyclictest1958685-21kworker/u8:3+events_unbound22:10:241
1907184993215,6cyclictest1958685-21kworker/u8:3+events_unbound22:10:241
1907182993221,8cyclictest2016032-21apt-get00:25:120
1907182993220,9cyclictest1934052-21/usr/sbin/munin20:25:170
1907182993219,8cyclictest1991245-21tail23:10:240
1907182993217,9cyclictest1979753-21cron22:40:000
1907182993217,12cyclictest1915832-21grep19:30:250
1907182993217,11cyclictest1945435-21auditctl21:00:000
1907182993217,11cyclictest1933506-21cron20:25:000
1907182993217,11cyclictest1909789-21sort19:15:160
1907182993217,11cyclictest1909789-21sort19:15:160
1907182993216,11cyclictest1995491-21grep23:25:170
1907182993215,14cyclictest1908080-21tr19:10:240
1907182993215,14cyclictest1908080-21tr19:10:240
1907182993215,13cyclictest1977050-21wc22:30:200
1907182993215,12cyclictest1942566-21if_eth020:50:170
190718899313,4cyclictest131rcu_preempt23:50:122
1907188993123,5cyclictest0-21swapper/220:40:002
1907188993120,8cyclictest311ktimers/222:50:132
1907184993124,5cyclictest0-21swapper/119:20:011
1907184993124,5cyclictest0-21swapper/119:20:001
1907184993121,6cyclictest0-21swapper/121:20:011
1907182993124,4cyclictest1971467-21apt-get22:15:110
1907182993120,7cyclictest2000469-21munin-node23:40:110
1907182993119,8cyclictest1954337-21/usr/sbin/munin21:25:160
1907182993116,10cyclictest1982248-21tr22:45:200
1907182993113,14cyclictest1959092-21sendmail21:40:010
1907188993024,4cyclictest1883441-21kworker/u8:0+events_unbound19:28:422
1907188993023,5cyclictest1960566-21kworker/u8:0+events_unbound21:50:402
1907184993023,4cyclictest0-21swapper/120:25:161
1907184993021,6cyclictest0-21swapper/121:45:011
1907184993021,6cyclictest0-21swapper/119:10:171
1907184993021,6cyclictest0-21swapper/119:10:171
1907184993020,6cyclictest0-21swapper/121:00:261
190718899298,4cyclictest131rcu_preempt20:00:132
1907188992923,4cyclictest1958685-21kworker/u8:3+events_unbound22:56:422
1907188992919,6cyclictest1916660-21cut19:35:172
1907184992922,4cyclictest0-21swapper/121:30:251
1907184992920,6cyclictest0-21swapper/100:30:161
1907184992919,7cyclictest0-21swapper/123:40:161
190718899285,10cyclictest1971300-21cat22:15:002
190718899285,10cyclictest1971300-21cat22:15:002
1907188992820,5cyclictest0-21swapper/223:10:252
1907188992820,4cyclictest0-21swapper/220:15:242
1907184992822,3cyclictest0-21swapper/119:25:261
1907188992720,5cyclictest2009197-21kworker/u8:4+events_unbound00:14:552
1907188992720,4cyclictest0-21swapper/220:25:142
1907188992720,3cyclictest0-21swapper/223:20:252
1907188992720,3cyclictest0-21swapper/220:35:002
1907188992719,5cyclictest1975475-21kworker/u8:2+events_unbound22:25:262
1907188992719,5cyclictest0-21swapper/219:55:012
1907188992713,5cyclictest1949778-21kworker/u8:1+events_unbound22:02:232
1907184992718,6cyclictest0-21swapper/119:55:241
190718499271,3cyclictest131rcu_preempt23:00:251
190718899262,12cyclictest0-21swapper/221:21:462
1907188992621,2cyclictest0-21swapper/200:20:142
1907188992620,3cyclictest0-21swapper/200:15:242
1907188992619,5cyclictest311ktimers/223:35:132
1907188992619,5cyclictest1926338-21kworker/u8:2+events_unbound20:06:272
1907188992619,4cyclictest1997267-21kworker/u8:3+events_unbound23:47:162
1907188992617,5cyclictest0-21swapper/220:55:232
1907188992616,7cyclictest0-21swapper/222:00:002
190718899261,3cyclictest0-21swapper/221:15:112
1907184992618,6cyclictest2006704-21kworker/u8:0+events_unbound00:02:141
1907184992618,4cyclictest0-21swapper/123:50:241
1907184992617,6cyclictest1997848-21cat23:30:241
1907184992617,6cyclictest1997848-21cat23:30:241
1907184992617,6cyclictest0-21swapper/123:25:171
1907184992617,6cyclictest0-21swapper/121:00:011
1907184992617,5cyclictest1967418-21grep22:00:251
1907188992520,3cyclictest53-21kcompactd022:15:162
1907188992519,3cyclictest0-21swapper/223:35:012
1907188992519,3cyclictest0-21swapper/223:35:002
1907188992518,5cyclictest2009197-21kworker/u8:4+events_unbound00:32:042
1907184992519,4cyclictest1926338-21kworker/u8:2+events_unbound20:04:511
1907184992517,5cyclictest0-21swapper/121:20:251
1907184992517,5cyclictest0-21swapper/100:10:161
1907184992516,6cyclictest0-21swapper/120:05:171
1907184992515,7cyclictest1969058-21unixbench_multi22:05:281
1907184992515,7cyclictest1969058-21unixbench_multi22:05:271
1907188992419,3cyclictest53-21kcompactd022:35:162
1907188992417,5cyclictest1890395-21kworker/u8:2+events_unbound19:42:192
1907188992416,6cyclictest1958685-21kworker/u8:3+events_unbound22:42:042
1907188992416,6cyclictest1923371-21kworker/u8:3+events_unbound19:55:152
1907188992413,8cyclictest1957378-21sh21:35:012
190718899240,3cyclictest131rcu_preempt22:20:132
1907184992418,3cyclictest0-21swapper/123:20:161
1907184992416,4cyclictest0-21swapper/123:10:001
1907184992415,6cyclictest0-21swapper/120:10:191
1907184992415,6cyclictest0-21swapper/119:20:181
1907188992317,4cyclictest1933276-21kworker/u8:1+events_unbound20:22:122
1907188992317,4cyclictest1890395-21kworker/u8:2+events_unbound19:49:012
1907188992315,6cyclictest1958685-21kworker/u8:3+events_unbound22:05:222
1907188992315,6cyclictest1958685-21kworker/u8:3+events_unbound22:05:212
1907188992315,5cyclictest0-21swapper/219:20:162
1907188992314,6cyclictest2017091-21grep00:25:262
1907184992317,3cyclictest0-21swapper/120:20:211
1907184992316,5cyclictest1940259-21kworker/u8:2+events_unbound21:06:421
1907184992316,5cyclictest1940259-21kworker/u8:2+events_unbound21:06:421
1907184992315,5cyclictest1976371-21idleruntime-cro22:30:011
1907188992216,4cyclictest1921266-21kworker/u8:0+events_unbound21:06:042
1907188992216,4cyclictest1921266-21kworker/u8:0+events_unbound21:06:042
1907184992216,4cyclictest1980716-21awk22:40:241
1907184992113,5cyclictest0-21swapper/122:20:241
1907184992113,4cyclictest0-21swapper/121:45:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional