You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 20:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Jul 06, 2026 12:43:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2169616996242,17cyclictest0-21swapper/109:35:361
2169617995545,6cyclictest2284634-21munin-run11:20:002
2169616995342,7cyclictest2236109-21cron10:00:011
2169616995228,20cyclictest0-21swapper/110:15:361
216961699512,20cyclictest2206716-21ssh09:10:401
216961699482,17cyclictest2258456-21ssh10:35:371
2169617994737,6cyclictest2254388-21ssh10:30:012
2169616994739,5cyclictest2212123-21aten_r2power_cu09:20:261
216961699461,12cyclictest2196043-21ntpq08:35:411
2169617994534,5cyclictest2245752-21aten_r2power_vo10:15:252
216961699453,7cyclictest2173539-21ntpq07:20:361
2169616994512,2cyclictest0-21swapper/107:50:351
2169615994533,9cyclictest2220888-21ssh09:35:010
2169617994434,5cyclictest0-21swapper/212:25:012
2169617994433,8cyclictest0-21swapper/210:05:432
2169617994433,8cyclictest0-21swapper/210:05:432
216961699443,7cyclictest2185568-21ntpq08:00:381
216961699441,12cyclictest2294753-21ntpq11:35:341
216961699441,10cyclictest2194500-21ntpq08:30:371
2169615994436,5cyclictest1370086-21cups-browsed12:14:010
2169617994333,7cyclictest2287667-21/usr/sbin/munin11:25:012
216961699434,33cyclictest2264635-21ssh10:45:411
2169616994336,3cyclictest0-21swapper/109:45:391
2169616994334,5cyclictest2288554-21latency11:25:331
2169617994232,6cyclictest421-21dbus-daemon12:10:432
2169616994235,3cyclictest0-21swapper/110:10:391
216961699422,34cyclictest2315980-21ssh12:10:371
2169616994221,18cyclictest2263497-21ssh10:45:021
216961699421,12cyclictest2198983-21ntpq08:45:381
216961699421,10cyclictest2189985-21ntpq08:15:391
216961699421,10cyclictest2189985-21ntpq08:15:391
2169615994228,10cyclictest1-21systemd11:55:020
2169615994217,19cyclictest2270702-21ssh10:55:440
2169617994133,5cyclictest2196582-21cron08:40:012
2169617994132,6cyclictest1370086-21cups-browsed12:25:462
2169617994129,7cyclictest420-21avahi-daemon11:52:542
216961699414,7cyclictest2193077-21ntpq08:25:411
216961699413,8cyclictest2291789-21ntpq11:30:401
2169616994131,5cyclictest0-21swapper/111:50:421
216961699412,14cyclictest2204965-21ntpq09:05:361
216961699411,8cyclictest2225076-21ssh09:40:381
2169616994118,20cyclictest2210243-21unixbench-2d09:15:471
2169616994117,21cyclictest0-21swapper/110:25:301
216961699411,13cyclictest2240263-21ntpq10:05:371
216961699411,13cyclictest2240263-21ntpq10:05:371
216961699411,12cyclictest2249458-21ntp_states10:20:401
216961699411,12cyclictest2177840-21memory07:35:331
216961699411,12cyclictest2177840-21memory07:35:331
2169616994111,2cyclictest0-21swapper/108:40:431
216961699411,11cyclictest2179492-21ntpq07:40:351
2169615994134,5cyclictest2280269-21head11:10:500
2169615994132,5cyclictest2224163-21latency_hist09:40:030
216961699403,9cyclictest2285545-21memory11:20:331
216961699402,12cyclictest2175046-21ntpq07:25:381
2169616994015,22cyclictest0-21swapper/111:15:391
2169616994012,25cyclictest2275083-21sh11:03:281
216961699401,11cyclictest2219043-21ntpq09:30:421
2169615994033,5cyclictest2211707-21ssh09:20:010
2169617993932,4cyclictest2260464-21kworker/u8:6-xprtiod10:40:012
2169617993930,4cyclictest0-21swapper/211:55:442
2169617993929,6cyclictest2224163-21latency_hist09:40:032
2169617993928,8cyclictest2331735-21sed12:35:522
216961699394,7cyclictest2300895-21ntpq11:45:371
216961699394,10cyclictest428-21mta-sts-daemon10:56:051
2169616993922,14cyclictest0-21swapper/111:40:441
2169616993918,18cyclictest2231720-21unixbench_singl09:50:481
2169616993918,18cyclictest0-21swapper/110:50:281
2169616993912,24cyclictest0-21swapper/107:15:381
216961699391,21cyclictest2276646-21ssh11:05:401
216961699391,17cyclictest2306898-21ssh11:55:381
216961699391,12cyclictest2279746-21ntpq11:10:421
216961699391,11cyclictest2329138-21ssh12:31:591
216961699391,11cyclictest2200474-21ntpq08:50:371
2169615993929,7cyclictest2177254-21latency_hist07:35:020
2169615993929,7cyclictest2177254-21latency_hist07:35:020
2169617993832,3cyclictest0-21swapper/209:20:272
2169617993831,4cyclictest1001-21mta-sts-daemon12:07:172
2169617993828,6cyclictest420-21avahi-daemon09:20:012
216961699383,7cyclictest2325159-21ssh12:25:401
216961699382,8cyclictest2318610-21diskstats12:15:311
216961699382,8cyclictest2318610-21diskstats12:15:311
2169616993815,19cyclictest0-21swapper/108:55:331
2169616993812,2cyclictest0-21swapper/109:28:591
2169616993810,24cyclictest0-21swapper/110:00:391
2169616993810,24cyclictest0-21swapper/110:00:391
2169615993832,3cyclictest0-21swapper/010:15:360
2169615993830,5cyclictest2283201-21tail11:15:450
2169615993830,5cyclictest2264803-21mailstats10:45:450
2169615993829,7cyclictest2290702-21true11:30:010
216961799373,26cyclictest2193018-21sshd08:25:382
2169617993730,3cyclictest0-21swapper/212:15:412
2169617993730,3cyclictest0-21swapper/212:15:412
2169617993729,5cyclictest2239182-21grep10:05:002
2169617993729,5cyclictest2239182-21grep10:05:002
2169617993729,4cyclictest0-21swapper/209:30:322
2169617993728,6cyclictest420-21avahi-daemon08:10:472
2169617993728,6cyclictest420-21avahi-daemon08:10:472
2169617993727,6cyclictest420-21avahi-daemon11:13:132
216961699373,11cyclictest2186957-21ntpq08:05:341
216961699372,9cyclictest2205600-21diskmemload12:05:381
216961699372,8cyclictest2255418-21ssh10:30:371
216961699372,7cyclictest2180974-21ntpq07:45:361
2169616993713,2cyclictest0-21swapper/112:35:341
216961699371,12cyclictest2309949-21ntpq12:00:401
216961699371,11cyclictest2203589-21ntpq09:00:521
2169615993731,3cyclictest0-21swapper/012:30:440
216961799363,21cyclictest2261555-21ssh10:40:382
2169617993628,5cyclictest2301366-21unixbench-2d11:45:472
2169617993628,5cyclictest2264140-21fschecks_time10:45:302
2169617993628,5cyclictest2227047-21systemd09:45:022
2169617993628,5cyclictest2184594-21cut08:00:012
216961799362,26cyclictest2291859-21ssh11:30:432
216961699362,15cyclictest2188109-21sshd08:10:331
216961699362,15cyclictest2188109-21sshd08:10:331
2169616993611,2cyclictest0-21swapper/108:21:241
2169615993631,2cyclictest0-21swapper/008:10:010
2169615993630,4cyclictest151ktimers/007:55:400
2169615993629,4cyclictest2236111-21cron10:00:010
2169615993628,5cyclictest2333218-21latency_hist12:40:020
216961799353,23cyclictest2294822-21ssh11:35:362
2169617993528,3cyclictest0-21swapper/210:30:342
2169617993526,5cyclictest1000-21mta-sts-daemon10:10:252
2169617993526,5cyclictest0-21swapper/212:00:452
2169617993523,9cyclictest2269850-21/usr/sbin/munin10:55:292
2169617993522,9cyclictest2234168-21ntp_kernel_pll_09:55:382
2169616993527,5cyclictest2324065-21systemd-run12:25:011
2169616993512,3cyclictest0-21swapper/107:30:251
2169616993512,3cyclictest0-21swapper/107:30:251
2169615993526,6cyclictest1-21systemd11:30:320
216961599352,27cyclictest2212804-21ssh09:20:370
216961599352,27cyclictest2206772-21ssh09:10:410
2169615993521,11cyclictest2269493-21idleruntime-cro10:55:010
2169615993517,10cyclictest151ktimers/008:15:370
2169615993517,10cyclictest151ktimers/008:15:370
2169617993428,4cyclictest2169614-21cyclictest08:50:352
2169617993427,3cyclictest0-21swapper/211:05:512
2169617993426,5cyclictest2233080-21rm09:55:012
2169617993424,7cyclictest420-21avahi-daemon11:45:002
2169617993423,6cyclictest0-21swapper/207:15:412
2169615993427,4cyclictest12650irq/32-eth009:00:460
2169615993427,4cyclictest1003-21mta-sts-daemon12:09:220
2169615993426,5cyclictest1370086-21cups-browsed11:24:470
2169615993426,5cyclictest0-21swapper/010:05:010
2169615993426,5cyclictest0-21swapper/010:05:010
2169615993425,6cyclictest2299802-21latency_hist11:45:020
2169615993424,7cyclictest12650irq/32-eth010:20:360
216961799336,19cyclictest1003-21mta-sts-daemon10:20:382
2169617993325,5cyclictest2175517-21unixbench_multi07:25:472
216961799332,25cyclictest2267697-21ssh10:50:472
2169617993312,13cyclictest351ktimers/208:30:352
2169616993312,2cyclictest0-21swapper/107:15:011
2169616993311,3cyclictest0-21swapper/107:10:011
2169615993329,2cyclictest0-21swapper/012:20:000
2169615993329,2cyclictest0-21swapper/012:20:000
2169615993326,4cyclictest2201017-21head08:50:480
2169615993325,5cyclictest1370083-21cupsd09:46:120
2169615993324,7cyclictest2278128-21sshd11:08:330
2169615993320,10cyclictest2233091-21munin-plugin-st09:55:010
2169615993320,10cyclictest0-21swapper/011:45:300
2169615993314,14cyclictest2296842-21latency_hist11:40:020
2169617993225,4cyclictest0-21swapper/208:20:012
2169617993225,4cyclictest0-21swapper/208:20:012
216961799322,23cyclictest2288704-21ssh11:25:372
2169617993221,8cyclictest2206622-21sh09:10:362
216961699323,10cyclictest2184007-21ntpq07:55:391
2169615993225,5cyclictest2178551-21head07:35:480
2169615993225,5cyclictest2178551-21head07:35:480
2169615993219,11cyclictest2227034-21kthreadd09:45:010
2169617993125,4cyclictest2177753-21kernelversion07:35:322
2169617993125,4cyclictest2177753-21kernelversion07:35:322
216961799312,24cyclictest2273611-21sh11:00:372
2169617993114,11cyclictest351ktimers/209:25:402
216961599313,22cyclictest2273693-21ssh11:00:400
2169615993126,3cyclictest0-21swapper/010:25:330
2169615993125,3cyclictest0-21swapper/012:20:340
2169615993124,4cyclictest2242204-21cron10:10:000
2169615993124,4cyclictest2242204-21cron10:10:000
2169615993123,5cyclictest0-21swapper/012:00:030
2169615993122,7cyclictest2182121-21fschecks_count07:50:300
2169615993115,13cyclictest151ktimers/010:35:400
2169617993024,3cyclictest0-21swapper/207:25:012
2169617993023,4cyclictest0-21swapper/209:50:012
2169617993022,5cyclictest2199618-21systemd-run08:50:012
2169617993020,6cyclictest420-21avahi-daemon12:32:562
216961599303,22cyclictest2242936-21ssh10:10:300
216961599303,22cyclictest1370086-21cups-browsed08:20:400
2169615993023,5cyclictest2174228-21latency_hist07:25:020
216961799296,17cyclictest428-21mta-sts-daemon08:00:352
2169617992921,5cyclictest2202430-21unixbench-2d08:55:482
2169617992921,5cyclictest2192001-21sed08:20:502
216961599293,19cyclictest421-21dbus-daemon08:00:390
2169615992923,4cyclictest2170129-21fschecks_time07:10:310
2169615992922,4cyclictest1370086-21cups-browsed10:44:350
216961599292,20cyclictest2310002-21ssh12:00:420
2169615992921,6cyclictest2325705-21sed12:25:500
2169615992920,6cyclictest2217874-21unin-run09:30:020
2169617992822,2cyclictest0-21swapper/208:40:422
216961599284,21cyclictest2190641-21kworker/u8:7+rpciod08:45:400
2169615992822,4cyclictest2175630-21systemd-run07:30:010
2169615992820,5cyclictest439-21systemd-logind09:00:010
216961599281,21cyclictest665-21snmpd10:30:400
2169617992713,11cyclictest351ktimers/209:05:422
2169615992720,4cyclictest421-21dbus-daemon08:45:010
2169615992712,12cyclictest0-21swapper/009:10:020
2169617992621,2cyclictest0-21swapper/207:45:002
2169617992618,5cyclictest1370086-21cups-browsed07:12:562
2169617992615,8cyclictest694-21cron08:10:012
2169615992610,13cyclictest0-21swapper/007:15:490
2169617992517,5cyclictest2183031-21unixbench_singl07:50:482
2169617992515,7cyclictest2176446-21/usr/sbin/munin07:30:332
2169617992515,7cyclictest2176446-21/usr/sbin/munin07:30:332
216961599252,19cyclictest2196600-21awk08:40:010
2169615992419,3cyclictest1003-21mta-sts-daemon07:40:420
2169615992417,5cyclictest2193550-21unixbench_singl08:25:480
2169615992415,7cyclictest151ktimers/007:45:350
216961799231,19cyclictest2203100-21sshd09:00:322
216961599238,12cyclictest2188528-21ntp_offset08:10:420
216961599238,12cyclictest2188528-21ntp_offset08:10:420
2169617992215,4cyclictest2181495-21unixbench_multi07:45:482
216961599222,15cyclictest2194585-21ntpq08:30:410
216961599215,5cyclictest439-21systemd-logind07:10:010
2169617991814,2cyclictest0-21swapper/207:10:022
216961899172,8cyclictest0-21swapper/310:20:373
216961899172,8cyclictest0-21swapper/309:55:433
216961899171,8cyclictest0-21swapper/308:25:363
216961899163,7cyclictest0-21swapper/311:40:383
216961899163,6cyclictest0-21swapper/311:30:403
216961899163,6cyclictest0-21swapper/310:00:373
216961899163,6cyclictest0-21swapper/310:00:373
216961899161,8cyclictest0-21swapper/312:25:363
216961899161,8cyclictest0-21swapper/309:35:373
216961899161,8cyclictest0-21swapper/308:30:383
216961899161,7cyclictest0-21swapper/312:30:373
216961899154,7cyclictest0-21swapper/310:35:363
216961899153,5cyclictest0-21swapper/307:25:383
216961899152,8cyclictest0-21swapper/310:30:373
216961899152,8cyclictest0-21swapper/309:40:343
216961899152,6cyclictest0-21swapper/309:45:343
216961899152,6cyclictest0-21swapper/309:10:333
216961899151,9cyclictest0-21swapper/312:35:403
216961899151,8cyclictest0-21swapper/312:00:403
216961899151,7cyclictest0-21swapper/312:15:373
216961899151,7cyclictest0-21swapper/312:15:373
216961899151,7cyclictest0-21swapper/312:10:413
216961899151,7cyclictest0-21swapper/309:30:393
216961899142,7cyclictest0-21swapper/312:20:423
216961899142,7cyclictest0-21swapper/310:15:393
216961899141,8cyclictest0-21swapper/311:45:263
216961899141,8cyclictest0-21swapper/311:10:373
216961899141,8cyclictest0-21swapper/310:55:443
216961899141,8cyclictest0-21swapper/310:25:433
216961899141,8cyclictest0-21swapper/309:20:413
216961899141,7cyclictest0-21swapper/311:15:323
216961899141,7cyclictest0-21swapper/311:05:383
216961899141,7cyclictest0-21swapper/311:00:393
216961899141,6cyclictest0-21swapper/308:50:353
216961899141,6cyclictest0-21swapper/308:15:383
216961899141,6cyclictest0-21swapper/308:15:383
216961899141,6cyclictest0-21swapper/307:55:383
216961899133,7cyclictest0-21swapper/311:25:473
216961899132,6cyclictest0-21swapper/311:50:253
216961899132,5cyclictest0-21swapper/309:00:493
216961899131,8cyclictest0-21swapper/309:15:423
216961899131,7cyclictest0-21swapper/312:05:383
216961899131,7cyclictest0-21swapper/310:50:403
216961899131,7cyclictest0-21swapper/310:10:393
216961899131,7cyclictest0-21swapper/309:25:293
216961899131,7cyclictest0-21swapper/307:45:013
216961899131,6cyclictest0-21swapper/308:40:403
216961899131,6cyclictest0-21swapper/308:00:363
216961899131,6cyclictest0-21swapper/307:50:353
216961899131,6cyclictest0-21swapper/307:30:373
216961899131,6cyclictest0-21swapper/307:30:373
216961899131,6cyclictest0-21swapper/307:15:373
216961899131,5cyclictest0-21swapper/307:10:373
216961899124,6cyclictest0-21swapper/307:45:363
216961899122,6cyclictest0-21swapper/311:35:383
216961899122,5cyclictest0-21swapper/308:20:353
216961899122,4cyclictest0-21swapper/311:20:363
216961899121,7cyclictest0-21swapper/310:40:253
216961899121,7cyclictest0-21swapper/310:05:293
216961899121,7cyclictest0-21swapper/310:05:293
216961899121,7cyclictest0-21swapper/309:50:283
216961899121,7cyclictest0-21swapper/308:10:413
216961899121,7cyclictest0-21swapper/308:10:413
216961899121,6cyclictest0-21swapper/311:55:293
216961899121,6cyclictest0-21swapper/310:45:343
216961899121,6cyclictest0-21swapper/309:08:323
216961899121,6cyclictest0-21swapper/308:45:383
216961899111,7cyclictest0-21swapper/308:35:383
216961899111,6cyclictest0-21swapper/308:55:333
216961899111,6cyclictest0-21swapper/308:05:363
216961899101,6cyclictest0-21swapper/307:35:303
216961899101,6cyclictest0-21swapper/307:35:303
216961899101,5cyclictest0-21swapper/307:20:333
21696189981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional