You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 16:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Mar 27, 2026 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212177699566,40cyclictest2272607-21ssh11:25:322
2121774995426,18cyclictest12350irq/32-eth010:40:250
2121775995340,9cyclictest2157821-21latency_hist09:10:021
2121775994938,8cyclictest2166124-21cat09:20:001
2121775994938,5cyclictest0-21swapper/109:50:311
2121774994938,8cyclictest2300812-21sh12:00:010
212177599485,35cyclictest2318804-21ssh12:20:311
2121775994837,7cyclictest2242567-21grep10:50:201
212177599474,35cyclictest2289279-21sh11:45:281
212177599472,38cyclictest2306234-21ssh12:05:321
2121774994736,8cyclictest1230504-21cups-browsed12:30:290
2121774994736,8cyclictest1230504-21cups-browsed12:30:290
2121774994733,10cyclictest2274441-40mandb11:28:030
2121776994635,7cyclictest2252005-21unixbench_multi11:00:372
212177599466,32cyclictest2310382-21ssh12:10:291
212177599463,30cyclictest2259852-21ssh11:10:271
2121775994629,11cyclictest439-21systemd-logind07:25:021
2121774994634,7cyclictest12450irq/33-eth009:15:270
2121775994537,5cyclictest1-21systemd11:55:021
212177599453,34cyclictest2171563-21ssh09:25:321
212177599452,33cyclictest2200927-21ssh10:00:271
2121776994434,7cyclictest2201339-21/usr/sbin/munin10:00:382
2121775994431,9cyclictest420-21avahi-daemon09:43:001
2121774994432,5cyclictest2218401-21tail10:20:380
2121774994432,5cyclictest2218401-21tail10:20:380
2121776994334,6cyclictest2187139-21munin-plugin-st09:45:012
2121776994314,6cyclictest171rcu_preempt11:10:262
2121775994335,5cyclictest2212443-21cron10:15:011
2121775994331,8cyclictest2159025-21sh09:10:321
2121774994336,5cyclictest2279960-21latency_hist11:35:030
2121774994330,11cyclictest0-21swapper/011:00:010
2121775994114,8cyclictest2222038-21ntpq10:25:311
2121774994133,5cyclictest420-21avahi-daemon10:05:330
2121774994133,5cyclictest2263803-21latency11:15:210
2121774994133,5cyclictest2263803-21latency11:15:210
2121774994131,7cyclictest421-21dbus-daemon12:13:250
212177699403,31cyclictest2314616-21ssh12:15:312
2121776994032,4cyclictest0-21swapper/209:50:292
2121775994033,3cyclictest0-21swapper/109:55:201
212177599403,27cyclictest2327133-21sh12:30:291
212177599403,27cyclictest2327133-21sh12:30:291
2121775994032,5cyclictest2214203-21unixbench_singl10:15:391
2121775994032,5cyclictest2214203-21unixbench_singl10:15:391
2121775994032,4cyclictest0-21swapper/111:05:251
2121775994030,7cyclictest420-21avahi-daemon10:30:521
212177599401,34cyclictest2268276-21ssh11:20:271
2121776993933,3cyclictest0-21swapper/211:00:012
2121776993932,4cyclictest2231009-21unixbench_singl10:35:382
2121776993931,5cyclictest2298270-21unixbench_multi11:55:392
2121775993931,4cyclictest0-21swapper/110:45:011
2121775993930,5cyclictest261-21systemd-journal10:55:381
2121775993926,5cyclictest0-21swapper/112:00:261
212177599392,29cyclictest2331220-21sshd12:35:251
212177599392,27cyclictest2230552-21ssh10:35:331
2121774993932,3cyclictest0-21swapper/011:40:000
2121774993913,21cyclictest12350irq/32-eth012:15:290
2121776993829,5cyclictest1230504-21cups-browsed09:25:022
2121775993830,4cyclictest0-21swapper/111:35:301
2121775993828,5cyclictest0-21swapper/109:35:221
2121775993814,8cyclictest2238787-21ntpq10:45:261
2121774993830,5cyclictest1051-21mta-sts-daemon10:15:090
2121774993830,5cyclictest1051-21mta-sts-daemon10:15:090
212177699373,26cyclictest2331328-21ssh12:35:292
2121776993729,5cyclictest421-21dbus-daemon12:35:012
2121776993729,5cyclictest421-21dbus-daemon12:35:012
2121776993729,5cyclictest2288169-21cron11:45:002
2121776993729,5cyclictest2241917-21ssh10:50:012
2121776993729,4cyclictest0-21swapper/209:15:282
2121775993728,6cyclictest0-21swapper/108:20:011
2121775993727,7cyclictest2284440-21aten_r2power_fr11:40:161
2121775993727,6cyclictest2128127-21aten_r2power_po07:30:161
2121775993726,8cyclictest2205187-21ntp_offset10:05:281
2121775993722,10cyclictest2175582-21sh09:30:281
2121776993630,3cyclictest0-21swapper/207:20:012
2121776993628,5cyclictest2216675-21sendmail10:20:012
2121776993628,5cyclictest2216675-21sendmail10:20:012
2121776993628,4cyclictest0-21swapper/210:40:252
2121776993624,5cyclictest0-21swapper/211:50:332
2121775993629,3cyclictest0-21swapper/111:25:291
2121775993621,9cyclictest2145234-21ntpq08:25:291
2121775993610,8cyclictest2136170-21ntpq07:55:281
2121774993631,3cyclictest0-21swapper/012:40:000
2121774993630,4cyclictest2151773-21systemd-run08:50:010
2121774993630,4cyclictest2151773-21systemd-run08:50:010
2121776993528,3cyclictest0-21swapper/207:35:272
2121776993527,4cyclictest0-21swapper/212:10:312
2121775993530,3cyclictest0-21swapper/112:00:011
2121775993526,5cyclictest2250741-21aten_r2power_en11:00:151
2121775993523,7cyclictest2167890-21unixbench_singl09:20:381
2121775993517,10cyclictest2133186-21ntpq07:45:271
2121774993527,5cyclictest1230504-21cups-browsed10:10:190
2121774993526,4cyclictest0-21swapper/011:05:260
2121774993524,6cyclictest2221943-21ntpq10:25:270
212177699343,24cyclictest2158968-21ssh09:10:302
2121776993427,4cyclictest1055-21mta-sts-daemon08:10:262
2121776993426,5cyclictest1230504-21cups-browsed08:20:022
2121776993426,4cyclictest0-21swapper/209:45:212
2121776993425,6cyclictest2258739-21ssh11:10:012
212177699341,27cyclictest2318877-21ssh12:20:342
212177599347,11cyclictest2153929-21sed08:55:221
2121775993426,5cyclictest2134283-21fschecks_time07:50:201
2121775993425,5cyclictest0-21swapper/111:15:321
2121775993425,5cyclictest0-21swapper/111:15:321
2121774993425,7cyclictest2144377-21latency_hist08:25:020
212177699333,24cyclictest2322950-21ssh12:25:292
2121776993326,3cyclictest0-21swapper/209:10:012
2121776993325,5cyclictest2283936-21cron11:40:002
2121776993325,5cyclictest1230504-21cups-browsed12:02:352
2121776993323,6cyclictest2147511-21aten_r2power_cu08:35:152
212177699331,27cyclictest2226146-21ssh10:30:272
2121776993310,18cyclictest2217939-21ntpq10:20:322
2121776993310,18cyclictest2217939-21ntpq10:20:322
212177599332,22cyclictest2323036-21ssh12:25:321
2121775993320,9cyclictest2187689-21aten_r2power_vo09:45:171
2121775993320,8cyclictest2155653-21ntpq09:00:271
2121775993314,13cyclictest2121730-21/usr/sbin/munin07:10:021
212177499333,26cyclictest2259832-21ssh11:10:260
212177499333,25cyclictest2184147-21sh09:40:310
2121774993327,4cyclictest2326121-21latency_hist12:30:030
2121774993326,5cyclictest261-21systemd-journal12:05:010
2121774993324,7cyclictest2170302-21latency_hist09:25:020
2121774993322,8cyclictest12350irq/32-eth010:50:250
2121776993226,3cyclictest0-21swapper/209:00:012
2121776993225,4cyclictest2221489-21cpuspeed10:25:182
2121776993225,3cyclictest0-21swapper/209:30:022
2121776993224,4cyclictest0-21swapper/211:15:272
2121776993224,4cyclictest0-21swapper/211:15:272
2121775993222,6cyclictest2140092-21aten_r2power_fr08:10:171
212177599322,23cyclictest2276660-21ssh11:30:261
212177599322,22cyclictest2314512-21ssh12:15:271
2121775993219,8cyclictest2143683-21ntpq08:20:271
2121775993218,8cyclictest2130095-21ntpq07:35:261
212177699315,6cyclictest2157748-21diskmemload12:05:302
2121776993126,2cyclictest0-21swapper/211:50:012
2121776993124,3cyclictest0-21swapper/210:05:252
2121776993123,5cyclictest2196873-21processes09:55:332
2121776993122,6cyclictest0-21swapper/211:20:172
2121775993124,4cyclictest2124764-21grep07:20:011
2121775993122,4cyclictest0-21swapper/108:40:011
2121775993121,6cyclictest2217896-21sh10:20:301
2121775993121,6cyclictest2217896-21sh10:20:301
2121775993118,8cyclictest2149688-21ntpq08:40:271
2121775993118,8cyclictest2131729-21ntpq07:40:331
2121775993118,8cyclictest2122612-21ntpq07:10:241
2121775993114,10cyclictest1051-21mta-sts-daemon08:05:291
2121774993123,6cyclictest2286125-21sshd11:41:170
2121774993118,9cyclictest151ktimers/012:05:290
2121774993117,10cyclictest421-21dbus-daemon09:10:300
2121774993111,4cyclictest171rcu_preempt12:20:250
2121776993024,3cyclictest0-21swapper/209:35:012
2121776993024,3cyclictest0-21swapper/208:35:012
2121776993023,3cyclictest0-21swapper/210:10:252
2121776993022,5cyclictest2156252-21latency_hist09:05:012
2121775993019,7cyclictest2127166-21ntpq07:25:271
2121775993015,12cyclictest2151718-21sshd08:45:381
2121775993015,12cyclictest2151718-21sshd08:45:381
2121774993024,4cyclictest2133771-21systemd-run07:50:010
2121774993024,4cyclictest2126263-21cron07:25:010
2121774993023,4cyclictest2224281-21kworker/u8:1-xprtiod10:35:280
2121774993023,4cyclictest2201113-21proc_pri10:00:340
2121774993022,5cyclictest420-21avahi-daemon09:52:320
2121774993022,5cyclictest2195912-21/usr/sbin/munin09:55:280
2121776992920,7cyclictest0-21swapper/207:15:022
212177599298,16cyclictest0-21swapper/108:00:231
2121775992920,7cyclictest2153017-21timerwakeupswit08:50:371
212177499293,20cyclictest12350irq/32-eth009:25:280
2121774992923,4cyclictest439-21systemd-logind07:10:010
2121774992923,4cyclictest2296761-21latency_hist11:55:020
2121774992923,4cyclictest2153355-21latency_hist08:55:020
2121774992923,4cyclictest2130755-21munin-plugin-st07:40:010
2121774992923,4cyclictest2124599-21unixbench-2d07:15:360
2121774992922,5cyclictest2129260-21debian-sa107:35:010
2121774992917,9cyclictest518-21rs:main0
2121774992911,13cyclictest151ktimers/009:45:270
2121776992822,3cyclictest0-21swapper/210:50:202
2121776992820,5cyclictest0-21swapper/209:35:252
2121775992818,7cyclictest2146624-21ntpq08:30:261
2121774992822,4cyclictest427-21polkitd08:11:140
2121774992822,4cyclictest2254555-21systemd11:05:020
2121774992822,4cyclictest2226754-21tail10:30:380
2121774992822,3cyclictest0-21swapper/010:45:230
212177499282,22cyclictest2268207-21ssh11:20:250
2121774992821,5cyclictest2157594-21sed09:05:360
212177499281,5cyclictest171rcu_preempt07:25:280
212177699273,4cyclictest171rcu_preempt08:05:252
2121776992720,3cyclictest0-21swapper/208:45:312
2121776992720,3cyclictest0-21swapper/208:45:312
212177499272,4cyclictest171rcu_preempt07:55:240
2121774992720,5cyclictest420-21avahi-daemon09:30:360
2121776992621,2cyclictest0-21swapper/208:45:002
2121776992621,2cyclictest0-21swapper/207:30:192
2121776992620,3cyclictest0-21swapper/207:50:182
2121776992618,7cyclictest0-21swapper/211:30:242
2121776992617,7cyclictest2145698-21unixbench_singl08:25:362
2121774992621,3cyclictest0-21swapper/011:50:000
2121774992620,4cyclictest2138786-21fschecks_time08:05:190
2121774992613,5cyclictest12450irq/33-eth009:40:010
2121776992519,3cyclictest0-21swapper/208:05:012
2121776992517,5cyclictest2117692-21kworker/u8:8+xprtiod08:00:012
2121776992517,4cyclictest0-21swapper/207:20:242
2121774992518,5cyclictest421-21dbus-daemon08:43:330
2121774992518,5cyclictest2147512-21aten_r2power_cu08:35:150
2121774992516,7cyclictest2147055-21timerwakeupswit08:30:380
2121776992416,3cyclictest0-21swapper/207:40:352
2121774992419,3cyclictest12450irq/33-eth008:00:290
2121774992418,3cyclictest0-21swapper/008:55:170
2121774992414,8cyclictest0-21swapper/007:50:370
2121774992411,4cyclictest2131054-21aten_r2power_en07:40:160
212177699232,16cyclictest2143739-21ntpq08:20:312
2121776992316,4cyclictest2121854-21latency_hist07:10:022
2121776992311,9cyclictest2127758-21sh07:30:012
2121774992318,3cyclictest2142747-21cron08:20:010
2121776992216,4cyclictest2152149-21aten2.4-expect08:50:162
2121776992216,3cyclictest0-21swapper/207:45:232
2121774992216,3cyclictest151ktimers/009:00:290
2121774992211,3cyclictest12350irq/32-eth007:10:370
212177799193,8cyclictest0-21swapper/311:28:033
212177799192,10cyclictest0-21swapper/310:00:273
212177799191,8cyclictest0-21swapper/312:30:293
212177799191,8cyclictest0-21swapper/312:30:293
212177799182,9cyclictest0-21swapper/310:55:273
212177799182,8cyclictest0-21swapper/309:20:273
212177799181,10cyclictest0-21swapper/310:45:273
212177799172,9cyclictest0-21swapper/311:55:263
212177799172,8cyclictest0-21swapper/309:55:253
212177799171,9cyclictest0-21swapper/310:35:303
212177799171,7cyclictest0-21swapper/308:25:233
212177799164,8cyclictest0-21swapper/311:20:263
212177799164,7cyclictest0-21swapper/311:05:253
212177799163,8cyclictest0-21swapper/309:35:273
212177799163,7cyclictest0-21swapper/311:30:233
212177799163,7cyclictest0-21swapper/311:10:273
212177799162,9cyclictest0-21swapper/311:40:303
212177799162,8cyclictest0-21swapper/311:00:263
212177799162,8cyclictest0-21swapper/310:50:263
212177799162,8cyclictest0-21swapper/310:05:243
212177799162,8cyclictest0-21swapper/309:40:293
212177799162,7cyclictest0-21swapper/308:35:263
212177799162,6cyclictest0-21swapper/307:40:303
212177799161,9cyclictest0-21swapper/310:30:233
212177799161,9cyclictest0-21swapper/309:25:203
212177799152,8cyclictest0-21swapper/311:15:203
212177799152,8cyclictest0-21swapper/311:15:203
212177799152,7cyclictest0-21swapper/312:35:273
212177799152,7cyclictest0-21swapper/309:05:283
212177799151,9cyclictest0-21swapper/312:15:163
212177799151,8cyclictest0-21swapper/312:25:313
212177799151,8cyclictest0-21swapper/310:25:273
212177799151,8cyclictest0-21swapper/309:30:283
212177799151,7cyclictest0-21swapper/308:50:243
212177799151,7cyclictest0-21swapper/308:15:273
212177799142,8cyclictest0-21swapper/312:05:303
212177799142,8cyclictest0-21swapper/311:45:183
212177799142,8cyclictest0-21swapper/309:50:313
212177799142,8cyclictest0-21swapper/309:12:373
212177799142,7cyclictest0-21swapper/311:35:203
212177799142,7cyclictest0-21swapper/310:40:233
212177799142,7cyclictest0-21swapper/310:20:203
212177799142,7cyclictest0-21swapper/310:20:203
212177799142,7cyclictest0-21swapper/308:50:013
212177799142,7cyclictest0-21swapper/308:50:013
212177799142,7cyclictest0-21swapper/308:40:253
212177799141,8cyclictest0-21swapper/312:20:163
212177799141,8cyclictest0-21swapper/312:10:243
212177799141,8cyclictest0-21swapper/312:00:163
212177799141,8cyclictest0-21swapper/309:45:323
212177799141,7cyclictest0-21swapper/308:30:293
212177799141,7cyclictest0-21swapper/307:50:293
212177799141,7cyclictest0-21swapper/307:10:283
212177799141,6cyclictest0-21swapper/308:05:253
212177799141,6cyclictest0-21swapper/307:55:273
212177799134,5cyclictest0-21swapper/309:15:193
212177799132,7cyclictest0-21swapper/311:50:183
212177799132,7cyclictest0-21swapper/310:10:263
212177799132,6cyclictest0-21swapper/310:15:193
212177799132,6cyclictest0-21swapper/310:15:193
212177799131,6cyclictest0-21swapper/308:10:273
212177799131,5cyclictest0-21swapper/307:20:243
212177799112,6cyclictest0-21swapper/308:55:283
212177799112,6cyclictest0-21swapper/307:30:013
212177799111,5cyclictest0-21swapper/307:45:233
212177799101,6cyclictest0-21swapper/308:00:273
212177799101,5cyclictest0-21swapper/307:30:243
21217779994,3cyclictest0-21swapper/307:15:303
21217779993,4cyclictest0-21swapper/309:00:193
21217779993,4cyclictest0-21swapper/307:35:283
21217779992,5cyclictest0-21swapper/308:20:253
21217779981,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional