You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 12:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Sat May 09, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37462217165,39sleep20-21swapper/219:09:462
35612202165,25sleep10-21swapper/119:07:191
34712201164,24sleep30-21swapper/319:06:123
34952200165,24sleep00-21swapper/019:06:290
1015321680,5sleep2386099cyclictest21:50:012
38609910198,2cyclictest0-21swapper/221:39:452
3860997168,2cyclictest0-21swapper/222:53:562
3860996561,3cyclictest0-21swapper/220:28:142
3860995956,2cyclictest0-21swapper/200:33:402
78652580,2sleep30-21swapper/322:55:243
153962490,4sleep1385699cyclictest20:45:231
109392490,2sleep2311ktimersoftd/223:05:012
48262480,1sleep20-21swapper/219:10:162
248542470,2sleep10-21swapper/123:35:141
206312450,1sleep30-21swapper/321:00:023
3860994239,2cyclictest0-21swapper/221:03:332
384899369,12cyclictest13319-21awk19:30:000
384899347,12cyclictest21780-21idleruntime-cro23:30:020
384899337,12cyclictest18338-21perf20:55:010
384899337,12cyclictest18338-21perf20:55:010
164052330,2sleep00-21swapper/019:35:140
164052330,2sleep00-21swapper/019:35:140
385699310,22cyclictest0-21swapper/120:44:131
3848993012,15cyclictest20200-21munin-run00:40:020
384899282,13cyclictest1338-21munin-run21:30:020
384899279,15cyclictest5753-21munin-run21:40:010
384899271,6cyclictest0-21swapper/021:41:050
385699260,16cyclictest0-21swapper/122:58:521
3856992515,3cyclictest9966-21kernelversion19:20:181
384899251,5cyclictest0-21swapper/021:59:500
386699249,3cyclictest0-21swapper/322:15:483
385699247,14cyclictest0-21swapper/100:30:011
3856992418,3cyclictest7586-21gltestperf19:15:141
3856992413,4cyclictest26373-21kworker/u8:223:45:111
385699241,17cyclictest0-21swapper/123:21:071
385699240,17cyclictest0-21swapper/121:02:141
384899248,2cyclictest26715-21kworker/0:023:57:210
384899241,4cyclictest0-21swapper/000:30:460
384899241,3cyclictest23059-21kworker/0:322:27:540
384899241,11cyclictest11004-21/usr/sbin/munin23:05:180
384899241,11cyclictest11004-21/usr/sbin/munin23:05:180
11182240,3sleep10-21swapper/121:27:251
3866992310,7cyclictest87350irq/126-enp1s0-21:58:363
386099237,3cyclictest22444-21ntp_states22:15:212
386099237,3cyclictest0-21swapper/219:18:022
386099237,3cyclictest0-21swapper/200:38:462
386099236,3cyclictest0-21swapper/223:47:162
386099231,19cyclictest2492-21grep20:15:252
385699233,9cyclictest0-21swapper/121:53:291
385699231,6cyclictest0-21swapper/122:00:281
385699231,19cyclictest0-21swapper/122:29:501
385699231,16cyclictest0-21swapper/123:25:251
385699231,16cyclictest0-21swapper/120:23:481
385699231,16cyclictest0-21swapper/119:47:471
385699231,12cyclictest13642-21cpuspeed_turbos19:30:131
384899239,3cyclictest18548-21/usr/sbin/munin20:55:180
384899236,2cyclictest10088-21kworker/0:222:06:140
384899232,10cyclictest6943-21/usr/sbin/munin19:15:170
384899231,6cyclictest0-21swapper/023:31:070
384899231,11cyclictest26379-21/usr/sbin/munin23:40:250
384899231,11cyclictest11949-21/usr/sbin/munin20:40:240
107492230,4sleep07-21ksoftirqd/020:35:190
386699222,4cyclictest6894-21memory21:40:173
386699221,4cyclictest18146-21perl23:20:143
3866992210,3cyclictest5371-21df00:05:133
386099225,9cyclictest25556-21awk22:25:022
386099222,12cyclictest20347-21proc_pri22:10:222
386099221,7cyclictest24924-21munin-run21:10:012
385699229,4cyclictest9539-21awk20:35:001
385699226,9cyclictest0-21swapper/122:30:331
385699221,4cyclictest0-21swapper/121:10:111
385699221,18cyclictest12900-21smart_sda00:20:241
385699221,18cyclictest12900-21smart_sda00:20:241
385699221,16cyclictest0-21swapper/123:14:131
385699221,16cyclictest0-21swapper/122:53:481
385699221,16cyclictest0-21swapper/120:10:481
385699221,16cyclictest0-21swapper/119:13:211
385699221,11cyclictest7335-21munin-run20:30:001
385699220,16cyclictest0-21swapper/122:05:521
384899229,3cyclictest28501-21kworker/0:220:09:140
384899228,3cyclictest808-21snmpd23:51:080
384899222,10cyclictest23059-21kworker/0:322:23:080
384899222,10cyclictest1915-21kworker/0:122:54:530
384899221,18cyclictest808-21snmpd21:02:220
384899221,10cyclictest9441-21/usr/sbin/munin00:15:280
384899221,10cyclictest808-21snmpd23:01:100
384899221,10cyclictest0-21swapper/021:17:160
384899220,20cyclictest0-21swapper/022:45:140
384899220,20cyclictest0-21swapper/020:30:240
384899220,10cyclictest13609-21munin-run00:25:010
384899220,10cyclictest13609-21munin-run00:25:010
12892221,2sleep10-21swapper/120:15:131
386699211,7cyclictest0-21swapper/323:49:513
386699211,12cyclictest10369-21/usr/sbin/munin21:50:153
386699211,11cyclictest0-21swapper/300:30:183
386099215,3cyclictest0-21swapper/222:25:272
386099214,9cyclictest3646-21kernelversion00:00:182
386099213,5cyclictest22937-21gltestperf19:50:162
386099212,6cyclictest7927-21awk20:30:142
386099211,12cyclictest0-21swapper/223:52:242
386099211,11cyclictest378-21pmu-power21:25:212
386099211,11cyclictest28316-21diskstats22:30:152
386099211,11cyclictest14608-21ntp_states19:30:222
386099211,11cyclictest10567-21irqstats20:35:162
3856992117,3cyclictest0-21swapper/122:40:181
3856992113,3cyclictest0-21swapper/123:56:141
3856992113,3cyclictest0-21swapper/123:17:041
385699211,2cyclictest18581-21kworker/u8:121:15:181
385699211,15cyclictest0-21swapper/123:00:571
385699211,15cyclictest0-21swapper/121:58:531
385699211,15cyclictest0-21swapper/121:47:431
385699211,15cyclictest0-21swapper/119:56:341
385699211,15cyclictest0-21swapper/119:52:331
385699211,15cyclictest0-21swapper/119:36:481
385699211,15cyclictest0-21swapper/119:36:481
385699211,15cyclictest0-21swapper/119:27:421
385699211,13cyclictest0-21swapper/100:35:191
385699211,11cyclictest4462-21chrt21:35:151
385699211,11cyclictest22249-21/usr/sbin/munin22:15:181
385699210,9cyclictest23-21ksoftirqd/120:50:181
385699210,9cyclictest23-21ksoftirqd/120:50:171
384899217,3cyclictest13537-21/usr/sbin/munin19:30:200
384899216,3cyclictest10088-21kworker/0:221:53:260
384899211,9cyclictest0-21swapper/000:01:160
384899211,8cyclictest0-21swapper/021:21:290
384899211,10cyclictest12918-21kworker/0:023:10:150
384899210,5cyclictest0-21swapper/022:55:240
386699203,9cyclictest25443-21sensors23:35:223
386699201,9cyclictest5883-21grep22:50:243
386699201,6cyclictest18245-1kworker/u9:222:47:123
3866992015,3cyclictest41-21ksoftirqd/321:15:243
386699201,10cyclictest0-21swapper/323:50:273
386099206,7cyclictest0-21swapper/223:07:182
386099206,7cyclictest0-21swapper/223:07:182
386099205,7cyclictest0-21swapper/220:24:152
386099205,5cyclictest24921-21diskmemload23:20:132
386099201,7cyclictest26438-21ls21:10:222
386099201,6cyclictest24182-21/usr/sbin/munin23:35:162
386099201,6cyclictest0-21swapper/200:09:352
3860992015,3cyclictest12722-21grep21:55:112
3860992014,3cyclictest27915-21unixbench_multi23:40:242
386099201,10cyclictest24764-21expr19:55:132
385699209,3cyclictest3171-21diskstats00:00:141
385699204,10cyclictest0-21swapper/122:39:131
3856992016,3cyclictest0-21swapper/119:40:381
3856992012,3cyclictest0-21swapper/123:30:031
385699201,17cyclictest0-21swapper/120:55:051
385699201,14cyclictest0-21swapper/121:41:071
385699201,14cyclictest0-21swapper/120:07:061
385699201,11cyclictest24214-21smart_sda21:05:231
385699201,11cyclictest19489-21expr22:10:121
385699201,10cyclictest13150-21wc23:10:011
385699201,10cyclictest13150-21wc23:10:011
385699200,15cyclictest0-21swapper/121:30:501
385699200,15cyclictest0-21swapper/121:21:311
385699200,14cyclictest0-21swapper/123:40:141
384899207,3cyclictest19906-21awk19:45:000
384899206,3cyclictest1575-21/usr/sbin/munin21:30:120
384899205,3cyclictest87550irq/128-enp1s0-23:35:190
384899205,3cyclictest808-21snmpd21:45:540
384899201,9cyclictest13488-21kworker/0:100:25:190
384899201,7cyclictest0-21swapper/022:34:470
384899201,6cyclictest3159-21/usr/sbin/munin20:20:170
384899201,12cyclictest0-21swapper/000:05:120
384899200,1cyclictest0-21swapper/022:35:180
384899200,1cyclictest0-21swapper/019:45:220
384899200,10cyclictest0-21swapper/022:10:150
235672201,15sleep123653-21cpuspeed_turbos22:20:111
386699193,9cyclictest41-21ksoftirqd/322:25:143
386699192,14cyclictest2080-21grep23:55:243
386699191,9cyclictest26373-21kworker/u8:200:10:353
386699191,6cyclictest0-21swapper/322:30:223
386699191,4cyclictest0-21swapper/321:30:133
386699191,3cyclictest0-21swapper/322:05:143
386699191,17cyclictest0-21swapper/321:11:083
386699191,10cyclictest32699-21df_inode22:40:123
386699191,10cyclictest0-21swapper/322:12:533
386699191,10cyclictest0-21swapper/322:01:133
386699191,10cyclictest0-21swapper/300:23:503
386699191,10cyclictest0-21swapper/300:23:503
386699191,10cyclictest0-21swapper/300:15:093
386099197,3cyclictest0-21swapper/220:53:002
386099197,3cyclictest0-21swapper/220:53:002
386099195,3cyclictest0-21swapper/220:59:222
386099194,9cyclictest26373-21kworker/u8:200:18:402
386099191,9cyclictest11338-21/usr/sbin/munin19:25:122
386099191,9cyclictest0-21swapper/223:16:202
386099191,6cyclictest3301-21/usr/sbin/munin22:45:212
386099191,6cyclictest28612-21sh21:15:232
386099191,5cyclictest7923-21smartctl00:10:152
386099191,5cyclictest11984-21diskstats00:20:132
386099191,5cyclictest11984-21diskstats00:20:132
386099191,5cyclictest0-21swapper/223:29:572
386099191,5cyclictest0-21swapper/222:55:182
3860991912,4cyclictest14358-21fschecks_time00:25:142
386099191,15cyclictest195852sleep219:41:072
386099191,11cyclictest0-21swapper/222:00:572
386099191,10cyclictest884-21cat23:55:112
386099191,10cyclictest16079-21diskstats19:35:132
386099191,10cyclictest16079-21diskstats19:35:132
385699198,2cyclictest26373-21kworker/u8:223:53:071
385699194,3cyclictest5613-21fschecks_time00:05:161
385699190,17cyclictest0-21swapper/100:10:071
385699190,14cyclictest0-21swapper/122:48:071
384899197,4cyclictest8646-21kworker/0:119:20:120
384899195,3cyclictest87550irq/128-enp1s0-21:10:240
384899195,3cyclictest87550irq/128-enp1s0-00:10:250
384899194,3cyclictest25148-21sshd19:55:150
384899194,3cyclictest19058-21unixbench-2d23:20:240
384899193,8cyclictest569-21kworker/0:120:17:260
384899193,3cyclictest87550irq/128-enp1s0-23:46:350
384899191,9cyclictest20907-21kworker/0:122:16:300
384899191,7cyclictest819-21lldpd21:06:540
384899191,4cyclictest0-21swapper/023:16:190
384899190,1cyclictest0-21swapper/020:45:220
255862190,4sleep339-21rcuc/319:55:213
386699185,6cyclictest0-21swapper/323:07:273
386699185,6cyclictest0-21swapper/323:07:273
386699182,2cyclictest87350irq/126-enp1s0-21:21:263
386699181,8cyclictest13026-21ntpq20:40:213
386699181,8cyclictest0-21swapper/320:00:213
386699181,2cyclictest0-21swapper/319:45:163
3866991812,4cyclictest24146-21perl22:20:143
3866991812,3cyclictest41-21ksoftirqd/321:45:253
386699181,13cyclictest0-21swapper/323:31:133
386699181,11cyclictest0-21swapper/323:40:173
386699181,10cyclictest0-21swapper/323:15:233
386699181,10cyclictest0-21swapper/321:35:273
386699181,10cyclictest0-21swapper/300:02:123
386099188,3cyclictest0-21swapper/219:24:072
386099185,3cyclictest87550irq/128-enp1s0-21:30:172
386099181,9cyclictest0-21swapper/221:50:542
386099181,6cyclictest21991-21/usr/sbin/munin23:30:262
386099181,5cyclictest0-21swapper/222:06:092
3860991813,3cyclictest32104-21needreboot20:10:182
3860991813,3cyclictest13617-21sshd20:41:132
3860991812,3cyclictest29542-21/usr/sbin/munin21:20:232
3860991811,4cyclictest28307-21uptime20:00:252
386099180,5cyclictest0-21swapper/221:43:352
385699181,16cyclictest0-21swapper/120:35:301
385699181,16cyclictest0-21swapper/100:31:171
385699181,16cyclictest0-21swapper/100:17:021
384899184,3cyclictest87550irq/128-enp1s0-20:10:230
384899184,3cyclictest87350irq/126-enp1s0-22:40:140
384899181,6cyclictest4052-21/usr/sbin/munin19:10:190
384899181,14cyclictest14771-21/usr/sbin/munin22:00:120
384899180,9cyclictest0-21swapper/020:00:150
242872180,1sleep30-21swapper/321:05:253
239902181,3sleep00-21swapper/019:51:120
386699177,2cyclictest19792-21/usr/sbin/munin23:25:193
386699172,3cyclictest10387-21users23:00:283
386699172,12cyclictest839-21lldpd20:10:203
386699171,9cyclictest808-21snmpd19:15:263
386699171,9cyclictest0-21swapper/320:16:513
386699171,3cyclictest87350irq/126-enp1s0-22:35:143
3866991713,3cyclictest0-21swapper/321:02:283
386699171,2cyclictest13725-21fschecks_time23:10:153
3866991712,3cyclictest3268-21tr20:20:133
3866991712,3cyclictest1398-21wc21:30:013
3866991712,2cyclictest18906-21hddtemp_smartct00:35:163
3866991711,3cyclictest10077-21cat19:20:183
386699171,10cyclictest0-21swapper/320:51:283
386699171,10cyclictest0-21swapper/320:51:283
386699171,10cyclictest0-21swapper/300:25:233
3866991710,2cyclictest87350irq/126-enp1s0-19:50:263
386099172,4cyclictest0-21swapper/222:38:592
386099172,3cyclictest32355-21/usr/sbin/munin22:40:232
386099171,5cyclictest20139-21acpi19:45:122
386099171,5cyclictest15890-21sshd20:47:142
3860991711,3cyclictest827-21runrttasks23:10:112
386099171,10cyclictest0-21swapper/220:05:202
385699171,14cyclictest0-21swapper/120:00:551
384899171,14cyclictest808-21snmpd20:25:400
386699161,5cyclictest0-21swapper/320:30:263
3866991611,3cyclictest4052-21/usr/sbin/munin19:10:183
3866991611,3cyclictest28930-21/usr/sbin/munin20:05:163
3866991611,3cyclictest15730-21/usr/sbin/munin19:35:153
3866991611,3cyclictest15730-21/usr/sbin/munin19:35:153
386699161,13cyclictest0-21swapper/320:25:133
3866991610,3cyclictest41-21ksoftirqd/320:50:023
113552160,2sleep30-21swapper/320:35:273
386699151,6cyclictest13537-21/usr/sbin/munin19:30:113
386699151,5cyclictest0-21swapper/319:40:143
3866991510,3cyclictest12213-21sed19:25:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional