You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 08:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Thu Jun 18, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1755882996353,6cyclictest1757610-21idleruntime-cro19:15:002
1755877995422,12cyclictest1815492-21sed22:00:171
1755877995119,14cyclictest1764292-21sed19:30:231
1755885995041,5cyclictest1790717-21cron20:50:003
1755885995041,5cyclictest1790717-21cron20:49:593
1755877994819,9cyclictest1849471-21awk23:40:111
1755877994719,9cyclictest1855040-21ntpq23:55:221
1755877994618,10cyclictest1788039-21missed_timers20:40:201
1755885994533,9cyclictest438-21cron22:30:003
1755885994533,9cyclictest438-21cron22:30:003
1755877994538,3cyclictest1808243-21apt-get21:40:131
1755877994536,5cyclictest1845742-21idleruntime-cro23:30:001
1755877994516,10cyclictest1760630-21ntp_states19:20:221
1755877994427,12cyclictest0-21swapper/120:35:121
1755877994422,9cyclictest1789830-21ntp_states20:45:211
1755877994422,9cyclictest1789830-21ntp_states20:45:211
1755877994418,9cyclictest1847369-21taskset23:34:171
1755873994437,4cyclictest1772185-21seq19:55:170
1755885994337,4cyclictest1852399-21kworker/u8:0+events_unbound00:24:073
1755885994332,7cyclictest1851269-21cut23:45:153
1755877994329,10cyclictest231ktimers/100:25:111
1755877994318,9cyclictest1858007-21cstates00:05:161
1755882994234,4cyclictest1806285-21cron21:35:012
1755877994232,7cyclictest1772896-21/usr/sbin/munin19:55:261
1755877994231,7cyclictest1779712-21/usr/sbin/munin20:15:231
1755877994228,11cyclictest1820762-21/usr/sbin/munin22:15:211
1755877994226,12cyclictest1757726-21latency_hist19:15:021
1755885994135,4cyclictest1773076-21kworker/u8:1+events_unbound20:24:423
1755885994134,5cyclictest1776708-21kworker/u8:2+events_unbound20:11:093
1755885994133,6cyclictest1823933-21kworker/u8:0+events_unbound23:31:463
1755885994131,6cyclictest1854622-21fschecks_count23:55:153
1755877994131,7cyclictest1758665-21/usr/sbin/munin19:15:171
1755877994130,7cyclictest1801422-21/usr/sbin/munin21:20:091
1755877994129,9cyclictest1868718-21/usr/sbin/munin00:35:201
1755877994129,9cyclictest1837443-21munin-node23:05:121
1755877994129,9cyclictest1820068-21cat22:15:001
1755877994117,9cyclictest1852888-21df23:50:151
1755882994026,11cyclictest311ktimers/222:30:122
1755882994026,11cyclictest311ktimers/222:30:122
1755877994034,3cyclictest1794953-21ntp_states21:00:211
1755877994033,3cyclictest1777312-21apt-get20:10:121
1755877994032,5cyclictest1830622-21apt-get22:45:111
1755877994030,7cyclictest1856736-21/usr/sbin/munin00:00:261
1755877994030,7cyclictest1842564-21/usr/sbin/munin23:20:111
1755877994030,7cyclictest1827508-21/usr/sbin/munin22:35:171
1755877994030,7cyclictest1807067-21/usr/sbin/munin21:35:201
1755877994030,6cyclictest1784135-21apt-get20:30:081
1755877994029,7cyclictest1839141-21/usr/sbin/munin23:10:111
1755877994028,9cyclictest1776935-21sshd20:08:161
1755877994027,10cyclictest1863121-21/usr/sbin/munin00:20:131
1755877994026,9cyclictest1832867-21/usr/sbin/munin22:50:201
1755877994025,9cyclictest0-21swapper/120:55:121
1755877994025,12cyclictest1797236-21/usr/sbin/munin21:05:241
1755877994022,13cyclictest1810929-21tail21:45:221
1755877994014,10cyclictest0-21swapper/121:15:251
1755882993931,5cyclictest1794139-21munin-plugin-st21:00:002
1755877993930,4cyclictest1766882-21apt-get19:40:151
1755877993929,7cyclictest1769034-21/usr/sbin/munin19:45:191
1755877993929,7cyclictest1769034-21/usr/sbin/munin19:45:181
1755877993928,7cyclictest53-21kcompactd019:29:041
1755877993927,9cyclictest53-21kcompactd020:52:381
1755877993915,9cyclictest645-21runrttasks23:00:491
1755885993829,5cyclictest1814802-21cron22:00:003
1755885993826,7cyclictest0-21swapper/321:35:003
1755885993811,6cyclictest131rcu_preempt19:40:143
1755877993832,3cyclictest1840842-21apt-get23:15:111
1755877993832,3cyclictest1840842-21apt-get23:15:101
1755877993832,3cyclictest1816764-21apt-get22:05:121
1755877993832,3cyclictest1765161-21apt-get19:35:131
1755877993831,3cyclictest1828923-21apt-get22:40:131
1755877993831,3cyclictest1803124-21apt-get21:25:121
1755877993829,6cyclictest1824145-21/usr/sbin/munin22:25:201
1755877993829,6cyclictest1824145-21/usr/sbin/munin22:25:191
1755877993828,7cyclictest1774780-21users20:00:251
1755877993827,8cyclictest1866605-21/usr/sbin/munin00:30:141
1755877993827,7cyclictest1862690-21cron00:19:001
1755877993827,7cyclictest1848117-21memory23:35:191
1755877993826,8cyclictest1805226-21/usr/sbin/munin21:30:191
1755877993826,10cyclictest53-21kcompactd021:10:391
1755877993825,9cyclictest1851204-21/usr/sbin/munin23:45:141
1755877993825,9cyclictest1835484-21cron23:00:001
1755877993822,11cyclictest1782548-21/usr/sbin/munin20:25:141
1755877993820,13cyclictest1813651-21if_eth121:55:171
1755877993815,8cyclictest629-21snmpd00:11:451
1755877993814,11cyclictest645-21runrttasks22:24:431
1755877993814,11cyclictest645-21runrttasks22:24:431
1755877993728,6cyclictest53-21kcompactd022:31:281
1755877993728,6cyclictest53-21kcompactd022:31:281
1755877993723,10cyclictest1770865-21/usr/sbin/munin19:50:231
1755877993723,10cyclictest1770865-21/usr/sbin/munin19:50:231
1755877993715,8cyclictest645-21runrttasks21:50:151
1755873993728,7cyclictest1764465-21unixbench_multi19:30:230
1755873993721,11cyclictest121ktimers/020:05:110
1755885993630,4cyclictest1805905-21kworker/u8:2+events_unbound21:51:253
1755885993626,6cyclictest0-21swapper/319:50:013
1755885993626,6cyclictest0-21swapper/319:50:013
1755882993627,4cyclictest0-21swapper/222:41:372
1755873993629,4cyclictest0-21swapper/022:20:010
1755885993527,4cyclictest0-21swapper/319:15:023
1755885993526,5cyclictest0-21swapper/300:40:003
1755885993520,6cyclictest1848924-21kworker/u8:3+events_unbound00:16:073
1755885993520,5cyclictest1757373-21kworker/u8:3+events_unbound19:21:133
1755885993516,6cyclictest131rcu_preempt21:35:113
1755882993529,3cyclictest0-21swapper/200:15:002
1755882993526,4cyclictest0-21swapper/221:40:142
1755877993519,12cyclictest1780734-21/usr/sbin/munin20:20:161
175588599348,16cyclictest40-21ksoftirqd/320:50:263
1755885993427,5cyclictest1823933-21kworker/u8:0+events_unbound22:46:413
1755885993426,4cyclictest0-21swapper/323:09:593
1755885993414,11cyclictest40-21ksoftirqd/323:55:013
1755882993424,6cyclictest0-21swapper/200:02:462
1755873993426,4cyclictest0-21swapper/021:00:240
1755873993423,7cyclictest1840195-21grep23:10:260
175588599338,6cyclictest131rcu_preempt20:15:093
1755885993327,4cyclictest1823933-21kworker/u8:0+events_unbound22:33:423
1755885993327,4cyclictest1823933-21kworker/u8:0+events_unbound22:33:413
1755885993327,4cyclictest1787553-21kworker/u8:0+events_unbound21:11:183
1755885993325,4cyclictest0-21swapper/319:15:253
1755885993323,6cyclictest1829970-21grep22:40:263
1755885993323,6cyclictest0-21swapper/319:30:233
1755885993317,6cyclictest1801055-21kworker/u8:1+events_unbound21:47:473
1755885993311,6cyclictest131rcu_preempt23:20:133
1755882993326,3cyclictest0-21swapper/222:00:152
1755882993326,3cyclictest0-21swapper/219:30:242
1755882993322,7cyclictest0-21swapper/221:10:002
1755873993324,4cyclictest0-21swapper/023:06:420
1755873993323,6cyclictest0-21swapper/019:35:250
1755885993225,3cyclictest0-21swapper/322:05:253
1755885993224,4cyclictest0-21swapper/322:55:253
1755885993222,6cyclictest0-21swapper/320:00:003
1755885993211,12cyclictest40-21ksoftirqd/322:20:263
1755885993211,12cyclictest40-21ksoftirqd/322:20:253
1755885993210,11cyclictest40-21ksoftirqd/320:35:253
1755882993225,4cyclictest1792436-21idleruntime-cro20:55:002
1755882993223,6cyclictest1867433-21unixbench_multi00:30:222
1755882993223,4cyclictest0-21swapper/220:06:072
1755882993222,6cyclictest0-21swapper/200:09:592
1755873993225,4cyclictest0-21swapper/020:55:250
1755873993224,4cyclictest0-21swapper/023:01:400
1755885993125,4cyclictest1852399-21kworker/u8:0+events_unbound00:14:423
1755885993124,5cyclictest1828378-21kworker/u8:3+events_unbound22:54:423
1755882993124,5cyclictest1823933-21kworker/u8:0+events_unbound23:36:482
1755882993122,4cyclictest0-21swapper/223:40:242
1755882993122,4cyclictest0-21swapper/220:40:262
1755882993120,6cyclictest0-21swapper/200:40:002
1755873993121,7cyclictest1818229-21sh22:10:010
1755873993120,6cyclictest0-21swapper/020:00:250
1755885993023,4cyclictest1763234-21latency_hist19:30:013
1755885993022,4cyclictest0-21swapper/321:25:253
1755885993017,6cyclictest1818122-21kworker/u8:3+events_unbound22:17:193
1755885993015,6cyclictest1781875-21kworker/u8:3+events_unbound20:30:443
1755885993013,6cyclictest1792111-21kworker/u8:4+events_unbound21:23:253
1755882993024,4cyclictest1808003-21sendmail21:40:012
1755882993022,4cyclictest0-21swapper/221:05:002
1755882993019,6cyclictest0-21swapper/219:35:252
175588299301,14cyclictest1778773-21cron20:15:012
1755873993023,5cyclictest1855576-21kworker/u8:1+events_unbound00:18:340
175588599296,4cyclictest1800060-21kernelversion21:15:193
1755885992921,4cyclictest0-21swapper/320:05:243
1755885992915,6cyclictest1792111-21kworker/u8:4+events_unbound21:01:153
1755882992922,5cyclictest1781875-21kworker/u8:3+events_unbound20:29:382
1755882992922,3cyclictest0-21swapper/222:10:172
1755882992920,5cyclictest0-21swapper/222:40:012
1755882992919,6cyclictest0-21swapper/221:50:242
1755882992919,6cyclictest0-21swapper/219:50:172
1755882992919,6cyclictest0-21swapper/219:50:172
1755882992919,6cyclictest0-21swapper/219:25:252
1755873992923,4cyclictest1796284-21kworker/u8:3+events_unbound21:06:160
1755873992922,4cyclictest0-21swapper/023:40:010
1755873992922,4cyclictest0-21swapper/000:05:270
1755873992922,3cyclictest0-21swapper/000:25:250
1755873992918,9cyclictest121ktimers/021:30:130
175588599288,11cyclictest40-21ksoftirqd/323:35:153
175588599288,11cyclictest1828378-21kworker/u8:3+events_power_efficient23:28:173
1755885992812,8cyclictest131rcu_preempt22:00:173
1755882992822,4cyclictest1801055-21kworker/u8:1+events_unbound21:26:122
1755882992821,3cyclictest0-21swapper/223:20:002
1755882992821,3cyclictest0-21swapper/223:20:002
1755882992820,5cyclictest1781875-21kworker/u8:3+events_unbound20:45:162
1755882992820,5cyclictest1781875-21kworker/u8:3+events_unbound20:45:152
1755882992819,7cyclictest1828378-21kworker/u8:3+events_unbound22:51:412
1755882992818,6cyclictest0-21swapper/220:35:262
1755882992817,6cyclictest0-21swapper/200:30:002
1755882992816,8cyclictest1-21systemd20:21:092
175588299281,4cyclictest131rcu_preempt19:45:122
175588299281,4cyclictest131rcu_preempt19:45:122
1755873992821,3cyclictest0-21swapper/019:40:270
1755873992820,6cyclictest121ktimers/022:35:180
1755873992819,7cyclictest1834177-21fschecks_count22:55:150
175588599277,12cyclictest131rcu_preempt22:10:173
175588599275,11cyclictest40-21ksoftirqd/322:40:013
1755885992720,5cyclictest1848924-21kworker/u8:3+events_unbound00:05:463
1755885992720,3cyclictest0-21swapper/323:45:013
1755885992719,6cyclictest53-21kcompactd021:05:103
1755885992719,4cyclictest0-21swapper/300:00:313
1755885992717,6cyclictest0-21swapper/323:15:243
1755885992717,6cyclictest0-21swapper/323:15:243
1755885992714,5cyclictest1823933-21kworker/u8:0+events_unbound23:04:213
1755885992713,4cyclictest131rcu_preempt21:43:333
1755882992721,3cyclictest0-21swapper/222:00:012
1755882992720,5cyclictest1787553-21kworker/u8:0+events_unbound21:15:142
1755882992719,4cyclictest0-21swapper/220:35:012
1755882992717,6cyclictest0-21swapper/220:01:072
1755882992716,6cyclictest0-21swapper/223:55:262
1755873992720,3cyclictest0-21swapper/019:30:010
1755873992718,5cyclictest0-21swapper/020:46:540
1755873992718,5cyclictest0-21swapper/020:46:540
175588599268,11cyclictest131rcu_preempt19:35:173
1755885992613,5cyclictest131rcu_preempt20:25:163
1755885992610,6cyclictest40-21ksoftirqd/321:00:003
1755882992620,3cyclictest0-21swapper/220:00:002
1755882992619,5cyclictest0-21swapper/222:20:012
1755882992619,3cyclictest0-21swapper/223:00:242
175588299261,5cyclictest131rcu_preempt19:40:132
1755873992622,2cyclictest13550irq/42-30be0000.ethernet19:18:050
1755873992620,3cyclictest0-21swapper/022:50:010
1755873992619,4cyclictest1787308-21latency_hist20:40:000
1755873992619,4cyclictest0-21swapper/023:20:000
1755873992619,4cyclictest0-21swapper/023:20:000
1755873992617,6cyclictest1787642-21ls20:40:130
175588599258,4cyclictest0-21swapper/300:30:103
175588599256,3cyclictest1839343-21fschecks_time23:10:163
175588599252,12cyclictest40-21ksoftirqd/320:00:253
175588599252,12cyclictest0-21swapper/319:53:253
175588599252,12cyclictest0-21swapper/319:53:243
1755885992518,4cyclictest0-21swapper/300:25:153
1755882992520,3cyclictest1824658-21unixbench_singl22:25:242
1755882992520,3cyclictest1824658-21unixbench_singl22:25:232
1755882992516,5cyclictest0-21swapper/220:15:252
1755882992515,6cyclictest0-21swapper/200:24:072
1755873992519,3cyclictest0-21swapper/021:50:140
1755873992518,4cyclictest13850irq/36-mmc121:20:000
1755873992518,4cyclictest0-21swapper/020:30:230
175587399251,6cyclictest645-21runrttasks21:23:450
1755873992516,5cyclictest0-21swapper/023:24:260
1755873992515,7cyclictest121ktimers/021:55:230
1755873992515,6cyclictest0-21swapper/022:15:000
175587399251,20cyclictest0-21swapper/021:12:180
175588599241,15cyclictest0-21swapper/320:40:253
175588299242,5cyclictest131rcu_preempt23:25:112
1755882992418,3cyclictest0-21swapper/223:50:242
1755882992417,5cyclictest1828378-21kworker/u8:3+events_unbound22:48:072
1755882992416,5cyclictest0-21swapper/221:20:192
1755882992415,5cyclictest0-21swapper/223:20:252
1755882992414,5cyclictest0-21swapper/223:45:242
1755882992413,7cyclictest32-21ksoftirqd/222:55:152
1755882992413,7cyclictest0-21swapper/219:15:252
175587399241,9cyclictest1806686-21uniq21:35:150
1755873992417,3cyclictest0-21swapper/023:41:210
1755873992417,3cyclictest0-21swapper/000:36:350
1755873992417,3cyclictest0-21swapper/000:24:590
1755873992415,6cyclictest1757049-21grep19:10:260
175587399241,19cyclictest0-21swapper/023:26:280
175587399241,19cyclictest0-21swapper/019:45:540
175587399241,19cyclictest0-21swapper/019:45:540
175587399241,19cyclictest0-21swapper/019:24:320
175587399241,19cyclictest0-21swapper/000:32:260
175587399241,19cyclictest0-21swapper/000:13:580
1755882992316,5cyclictest1855576-21kworker/u8:1+events_unbound00:19:052
1755882992316,3cyclictest0-21swapper/221:45:232
175587399238,11cyclictest121ktimers/022:20:120
175587399238,11cyclictest121ktimers/022:20:110
175587399231,9cyclictest1825499-21fschecks_time22:30:150
175587399231,9cyclictest1825499-21fschecks_time22:30:140
1755873992316,4cyclictest0-21swapper/020:20:010
1755873992316,3cyclictest0-21swapper/022:54:300
1755873992316,3cyclictest0-21swapper/020:52:540
1755873992315,5cyclictest1766416-21kworker/u8:0+events_unbound19:50:410
1755873992315,5cyclictest1766416-21kworker/u8:0+events_unbound19:50:400
1755873992313,6cyclictest0-21swapper/023:45:240
1755873992312,8cyclictest1811432-21wc21:50:010
175587399231,19cyclictest1815783-21runrttasks22:00:220
175587399231,18cyclictest0-21swapper/022:43:430
175587399231,18cyclictest0-21swapper/021:40:100
175588299226,7cyclictest32-21ksoftirqd/223:05:172
1755882992217,3cyclictest0-21swapper/223:30:242
1755882992216,3cyclictest0-21swapper/223:10:152
1755882992212,2cyclictest0-21swapper/219:20:272
1755873992217,2cyclictest0-21swapper/020:20:120
1755873992215,3cyclictest0-21swapper/020:11:200
175587399221,18cyclictest1783240-21switchtime20:25:240
1755882992113,4cyclictest0-21swapper/222:20:252
1755882992113,4cyclictest0-21swapper/222:20:242
175587399211,4cyclictest1804752-21latency_hist21:30:010
175587399211,3cyclictest1855620-21taskset23:56:070
175587399211,3cyclictest0-21swapper/023:52:470
175587399211,3cyclictest0-21swapper/022:27:100
175587399211,3cyclictest0-21swapper/022:27:100
1755873992111,7cyclictest1846526-21ntpq23:30:200
175587399202,13cyclictest13550irq/42-30be0000.ethernet00:00:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional