You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 19:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Jul 26, 2026 12:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2107269995948,4cyclictest0-21swapper/209:35:472
2107269995847,7cyclictest2229268-21sshd11:30:522
2107268995545,4cyclictest0-21swapper/111:25:541
2107267995443,6cyclictest14-21ksoftirqd/008:55:540
210726999503,35cyclictest2162759-21ssh09:40:522
2107267995033,7cyclictest2166106-21timerandwakeup09:45:580
2107269994738,6cyclictest421-21dbus-daemon11:25:012
2107268994725,5cyclictest171rcu_preempt07:40:471
2107269994538,3cyclictest0-21swapper/211:20:012
2107269994436,5cyclictest2211628-21grep11:00:592
2107269994436,5cyclictest2211628-21grep11:00:592
2107269994435,5cyclictest2148013-21unixbench_singl09:15:592
2107267994413,27cyclictest2177882-21ssh10:05:510
2107269994335,5cyclictest2197803-21ls10:40:012
2107268994337,3cyclictest0-21swapper/109:30:491
2107268994333,6cyclictest2210607-21fschecks_count11:00:401
2107268994333,6cyclictest2210607-21fschecks_count11:00:401
2107268994332,5cyclictest0-21swapper/112:00:481
2107268994332,5cyclictest0-21swapper/112:00:481
2107267994336,4cyclictest2231041-21cron11:35:010
2107268994234,5cyclictest2110288-21systemd-run07:20:011
2107267994229,9cyclictest2168177-21aten_r2power_vo09:50:370
2107269994131,6cyclictest420-21avahi-daemon12:02:162
2107269994131,6cyclictest420-21avahi-daemon12:02:162
2107269994131,6cyclictest1287404-21cupsd10:30:382
210726899414,30cyclictest2153595-21rm09:25:511
2107268994133,4cyclictest0-21swapper/110:35:581
2107268994132,5cyclictest2204436-21aten_r2power_vo10:50:381
2107267994132,6cyclictest2205083-21ntp_states10:50:530
2107267994129,9cyclictest151ktimers/008:21:010
2107267994128,9cyclictest421-21dbus-daemon11:20:550
210726799412,32cyclictest2192963-21ssh10:30:510
2107269994032,5cyclictest2167619-21systemd-run09:50:022
2107269994032,5cyclictest2111759-21cron07:25:012
2107269994031,6cyclictest1-21systemd10:50:022
2107268994034,3cyclictest0-21swapper/110:20:011
2107268994034,3cyclictest0-21swapper/110:20:011
2107267994033,4cyclictest1000-21mta-sts-daemon11:05:390
2107267994033,4cyclictest1000-21mta-sts-daemon11:05:390
2107267994030,7cyclictest2173654-21sendmail10:00:010
2107269993932,3cyclictest0-21swapper/211:50:532
2107269993929,7cyclictest2228034-21/usr/sbin/munin11:30:012
2107269993929,6cyclictest2264491-21latency_hist12:30:022
2107268993930,5cyclictest2126937-21latency_hist08:15:031
2107268993930,4cyclictest0-21swapper/110:20:501
2107268993928,8cyclictest2221974-21munin-plugin-st11:20:021
210726899392,28cyclictest2256355-21ssh12:15:501
2107267993931,5cyclictest2203887-21ssh10:50:020
2107269993829,6cyclictest1-21systemd10:30:012
2107268993831,3cyclictest0-21swapper/108:20:011
2107268993829,6cyclictest439-21systemd-logind12:25:011
2107268993827,7cyclictest2228438-21acpi11:30:361
210726799387,20cyclictest428-21mta-sts-daemon09:05:480
210726799383,25cyclictest265-21systemd-journal12:30:530
2107267993830,5cyclictest2243136-21ls11:55:010
2107267993829,7cyclictest2174366-21fschecks_count10:00:400
2107267993827,8cyclictest2234481-21/usr/sbin/munin11:40:410
2107267993824,11cyclictest0-21swapper/011:20:010
210726799382,25cyclictest2217158-21ssh11:10:530
2107267993817,16cyclictest12650irq/32-eth012:15:510
2107267993815,3cyclictest0-21swapper/012:20:550
2107269993729,5cyclictest2176676-21cut10:05:012
2107269993726,8cyclictest2135068-21needreboot08:40:432
2107269993721,11cyclictest2180923-21sh10:10:522
2107268993728,5cyclictest2240838-21uniq11:50:421
2107268993726,8cyclictest2120058-21needreboot07:50:441
2107268993726,8cyclictest2120058-21needreboot07:50:441
2107267993732,4cyclictest2107266-21cyclictest12:30:020
2107267993726,7cyclictest1000-21mta-sts-daemon11:27:450
2107267993725,9cyclictest2134287-21kworker/u8:0+events_unbound08:45:490
2107267993722,9cyclictest12650irq/32-eth009:25:530
2107267993710,19cyclictest2186809-21diskmemload10:20:450
2107269993629,4cyclictest2210135-21kworker/u8:12-nfsiod11:10:002
2107269993629,4cyclictest2210135-21kworker/u8:12-nfsiod11:10:002
2107269993629,4cyclictest2149365-21kworker/u8:6-xprtiod11:50:012
2107269993627,4cyclictest0-21swapper/210:20:482
210726899364,24cyclictest2232294-21ssh11:35:571
2107268993628,5cyclictest2262074-21if_eth012:25:421
2107267993629,4cyclictest1003-21mta-sts-daemon12:11:000
2107267993626,4cyclictest2150074-21fschecks_time09:20:390
210726799362,25cyclictest665-21snmpd12:35:520
2107267993619,12cyclictest2141240-21kworker/u8:9+rpciod09:15:490
2107267993616,14cyclictest421-21dbus-daemon12:05:460
2107267993616,14cyclictest421-21dbus-daemon12:05:460
2107269993528,4cyclictest0-21swapper/207:10:012
2107269993528,3cyclictest0-21swapper/209:50:462
2107269993524,4cyclictest2261212-21diskmemload12:24:522
210726999352,24cyclictest2208029-21ssh10:55:522
2107268993529,3cyclictest0-21swapper/111:40:471
2107268993528,4cyclictest2267815-21aten_r2power_fr12:35:391
2107268993527,5cyclictest1000-21mta-sts-daemon10:41:011
2107267993529,4cyclictest2146488-21latency_hist09:15:020
2107269993427,4cyclictest0-21swapper/207:20:012
2107269993426,5cyclictest2206929-21latency_hist10:55:022
2107269993426,4cyclictest0-21swapper/211:55:442
2107269993426,4cyclictest0-21swapper/209:11:012
2107269993422,9cyclictest2231656-21cpu11:35:392
210726999341,9cyclictest171rcu_preempt11:40:502
2107269993418,13cyclictest623-21sshd09:25:532
2107268993427,4cyclictest2252298-21latency_hist12:10:011
2107268993427,4cyclictest2252298-21latency_hist12:10:011
2107268993427,4cyclictest2194816-21systemd10:35:021
2107268993427,4cyclictest1001-21mta-sts-daemon10:12:121
2107268993426,5cyclictest2255439-21latency_hist12:15:021
2107268993426,5cyclictest2244462-21sendmail_mailst11:55:521
2107268993426,5cyclictest2222506-21aten_r2power_fr11:20:371
2107268993426,5cyclictest2133204-21cpuspeed08:35:381
2107268993425,6cyclictest2216766-21if_wlan011:10:431
210726899342,10cyclictest2137248-21ls08:50:011
210726799347,7cyclictest171rcu_preempt08:30:480
2107267993426,4cyclictest0-21swapper/009:41:520
2107267993422,8cyclictest2183719-21needreboot10:15:430
2107267993422,8cyclictest2183719-21needreboot10:15:430
2107268993329,2cyclictest0-21swapper/110:09:471
2107268993327,3cyclictest0-21swapper/108:00:011
2107268993327,3cyclictest0-21swapper/107:25:011
2107268993326,4cyclictest2216015-21latency_hist11:10:021
2107268993326,4cyclictest2216015-21latency_hist11:10:021
2107268993326,4cyclictest0-21swapper/111:45:541
2107268993322,6cyclictest1000-21mta-sts-daemon09:37:111
210726799337,19cyclictest2189876-21sshd10:25:520
210726799334,19cyclictest2143250-21diskmemload09:30:500
2107267993326,4cyclictest2107327-21latency_hist07:10:020
2107267993324,7cyclictest2108092-21ntp_kernel_err07:10:440
2107267993324,6cyclictest421-21dbus-daemon07:20:010
2107267993316,10cyclictest12650irq/32-eth012:00:500
2107267993316,10cyclictest12650irq/32-eth012:00:500
210726799331,25cyclictest2244219-21ssh11:55:440
210726799331,15cyclictest2181317-21/usr/sbin/munin10:10:570
210726999325,22cyclictest2217158-21ssh11:10:532
2107269993226,3cyclictest0-21swapper/207:40:542
2107269993225,4cyclictest2140238-21cut09:00:012
2107269993225,4cyclictest2134287-21kworker/u8:0-xprtiod09:35:012
2107269993225,3cyclictest0-21swapper/212:20:012
2107269993225,3cyclictest0-21swapper/210:09:322
2107269993224,5cyclictest421-21dbus-daemon09:55:452
2107269993224,3cyclictest0-21swapper/212:35:522
2107268993226,3cyclictest0-21swapper/108:05:001
2107268993226,3cyclictest0-21swapper/107:40:011
2107268993226,2cyclictest0-21swapper/109:53:221
2107268993224,5cyclictest2162834-21proc_pri09:40:551
210726899321,25cyclictest2147586-21ssh09:15:541
210726899321,23cyclictest2190074-21ssh10:26:001
2107267993223,6cyclictest2195190-21/usr/sbin/munin10:35:400
2107267993217,10cyclictest12650irq/32-eth009:35:460
210726999313,20cyclictest2150638-21sshd09:20:522
2107269993124,4cyclictest2183427-21uniq10:15:392
2107269993124,4cyclictest2183427-21uniq10:15:392
2107269993124,4cyclictest2114777-21kworker/u8:5-xprtiod08:14:082
2107269993123,5cyclictest2123759-21cron08:05:002
210726999312,22cyclictest2199036-21sshd10:40:532
2107268993126,2cyclictest0-21swapper/109:10:011
2107268993123,5cyclictest439-21systemd-logind07:30:011
2107268993122,6cyclictest421-21dbus-daemon09:24:341
210726899312,22cyclictest2265489-21sshd12:30:531
2107268993120,7cyclictest1287407-21cups-browsed09:48:581
2107267993114,11cyclictest421-21dbus-daemon11:39:200
210726999302,23cyclictest2250398-21sh12:05:502
210726999302,23cyclictest2250398-21sh12:05:502
210726999302,22cyclictest2120301-21ps07:50:532
210726999302,22cyclictest2120301-21ps07:50:532
2107268993024,3cyclictest0-21swapper/109:55:381
2107268993023,3cyclictest0-21swapper/109:14:381
2107268993022,5cyclictest2174512-21kernelversion10:00:431
2107268993022,5cyclictest2130286-21fschecks_time08:25:411
210726799306,14cyclictest12650irq/32-eth011:45:530
210726799304,18cyclictest428-21mta-sts-daemon11:00:520
210726799304,18cyclictest428-21mta-sts-daemon11:00:520
210726799302,21cyclictest1287404-21cupsd08:05:470
2107267993012,11cyclictest151ktimers/007:40:470
210726999293,21cyclictest2253294-21ssh12:10:452
2107269992922,3cyclictest0-21swapper/208:20:012
2107269992920,6cyclictest1287407-21cups-browsed08:45:382
2107269992914,11cyclictest265-21systemd-journal08:24:092
2107268992923,3cyclictest0-21swapper/111:00:011
2107268992923,3cyclictest0-21swapper/109:00:011
2107268992920,6cyclictest2125200-21unixbench_singl08:05:591
2107268992919,7cyclictest2203682-21sh10:49:301
2107267992923,4cyclictest2125477-21/usr/sbin/munin08:10:490
2107267992922,5cyclictest2200849-21/usr/sbin/munin10:45:010
2107267992922,4cyclictest487-21rs:main0
2107267992922,4cyclictest2128292-21systemd-run08:20:020
2107269992822,4cyclictest2107266-21cyclictest09:00:472
2107269992812,11cyclictest421-21dbus-daemon08:05:482
210726899283,19cyclictest443-21gpm08:30:501
2107268992822,4cyclictest1003-21mta-sts-daemon08:50:351
2107267992820,5cyclictest2112130-21aten_r2power_po07:25:370
2107267992819,7cyclictest2114773-21sh07:35:010
2107267992817,4cyclictest2209872-21ssh11:00:010
210726999272,20cyclictest2265393-21ssh12:30:492
2107269992720,4cyclictest0-21swapper/207:55:522
2107269992719,5cyclictest2134146-21unixbench_multi08:35:582
210726899276,11cyclictest1386149-21munin-node07:10:451
2107268992720,4cyclictest1001-21mta-sts-daemon07:30:401
2107268992717,7cyclictest420-21avahi-daemon08:20:391
2107267992718,7cyclictest1-21systemd07:21:520
2107267992718,5cyclictest1002-21mta-sts-daemon08:35:500
2107267992715,9cyclictest2121206-21cpuspeed07:55:390
2107269992621,2cyclictest0-21swapper/208:50:572
2107269992620,3cyclictest0-21swapper/207:30:012
2107269992614,9cyclictest2142560-21needreboot09:05:422
2107268992619,4cyclictest2119403-21latency_hist07:50:021
2107268992618,5cyclictest421-21dbus-daemon09:04:101
210726799264,15cyclictest428-21mta-sts-daemon09:00:510
210726799263,16cyclictest750-21ntpd07:50:470
210726799263,16cyclictest750-21ntpd07:50:470
2107267992618,6cyclictest2130008-21aten_r2power_cu08:25:370
2107267992617,6cyclictest2123265-21proc_pri08:00:530
2107269992519,3cyclictest0-21swapper/207:40:012
2107269992519,3cyclictest0-21swapper/207:10:482
2107269992512,10cyclictest2130728-21sshd08:25:522
2107268992515,8cyclictest420-21avahi-daemon08:40:531
210726799252,17cyclictest1287407-21cups-browsed07:45:460
2107269992418,2cyclictest0-21swapper/207:45:562
2107269992313,6cyclictest421-21dbus-daemon08:35:012
2107267992312,2cyclictest171rcu_preempt08:40:410
2107267992310,10cyclictest2137734-21df_inode08:50:380
2107269992215,5cyclictest2110222-21kworker/u8:3+xprtiod07:30:552
2107268992114,5cyclictest2107266-21cyclictest07:10:011
210727099171,5cyclictest0-21swapper/308:50:453
210727099162,6cyclictest0-21swapper/309:05:443
210727099161,8cyclictest0-21swapper/310:35:483
210727099153,5cyclictest0-21swapper/312:00:493
210727099153,5cyclictest0-21swapper/312:00:493
210727099152,8cyclictest0-21swapper/311:40:413
210727099151,8cyclictest0-21swapper/311:45:533
210727099151,8cyclictest0-21swapper/309:40:553
210727099151,7cyclictest0-21swapper/311:15:493
210727099143,7cyclictest0-21swapper/310:15:463
210727099143,7cyclictest0-21swapper/310:15:463
210727099143,4cyclictest0-21swapper/308:15:483
210727099142,7cyclictest0-21swapper/311:05:503
210727099142,7cyclictest0-21swapper/311:05:503
210727099142,7cyclictest0-21swapper/309:30:483
210727099142,7cyclictest0-21swapper/308:30:543
210727099142,6cyclictest0-21swapper/312:15:533
210727099142,6cyclictest0-21swapper/309:10:463
210727099142,5cyclictest0-21swapper/308:10:453
210727099141,8cyclictest0-21swapper/312:10:493
210727099141,8cyclictest0-21swapper/311:30:513
210727099141,8cyclictest0-21swapper/309:15:503
210727099141,7cyclictest0-21swapper/312:20:493
210727099141,7cyclictest0-21swapper/310:20:533
210727099141,7cyclictest0-21swapper/310:05:473
210727099141,7cyclictest0-21swapper/308:25:483
210727099141,7cyclictest0-21swapper/307:50:473
210727099141,7cyclictest0-21swapper/307:50:473
210727099141,6cyclictest0-21swapper/311:50:473
210727099141,6cyclictest0-21swapper/309:45:463
210727099141,6cyclictest0-21swapper/309:00:473
210727099141,6cyclictest0-21swapper/308:05:473
210727099134,7cyclictest0-21swapper/310:45:453
210727099134,6cyclictest0-21swapper/312:35:503
210727099132,8cyclictest0-21swapper/312:05:463
210727099132,8cyclictest0-21swapper/312:05:463
210727099132,7cyclictest0-21swapper/311:35:533
210727099132,7cyclictest0-21swapper/310:50:533
210727099132,7cyclictest0-21swapper/310:40:533
210727099132,5cyclictest0-21swapper/307:40:463
210727099131,8cyclictest0-21swapper/309:55:483
210727099131,7cyclictest0-21swapper/311:10:433
210727099131,7cyclictest0-21swapper/309:35:463
210727099131,7cyclictest0-21swapper/309:20:473
210727099131,6cyclictest0-21swapper/310:30:463
210727099131,6cyclictest0-21swapper/308:45:483
210727099131,6cyclictest0-21swapper/308:20:523
210727099131,6cyclictest0-21swapper/307:55:513
210727099131,6cyclictest0-21swapper/307:20:483
210727099131,5cyclictest0-21swapper/308:55:523
210727099131,5cyclictest0-21swapper/307:10:453
210727099123,6cyclictest0-21swapper/312:30:533
210727099122,6cyclictest0-21swapper/310:25:523
210727099122,5cyclictest0-21swapper/312:25:463
210727099122,5cyclictest0-21swapper/310:55:433
210727099122,4cyclictest0-21swapper/307:35:453
210727099121,8cyclictest0-21swapper/310:15:023
210727099121,7cyclictest0-21swapper/311:55:483
210727099121,6cyclictest0-21swapper/309:50:503
210727099121,6cyclictest0-21swapper/308:40:463
210727099121,6cyclictest0-21swapper/308:35:453
210727099121,6cyclictest0-21swapper/307:25:483
210727099121,5cyclictest0-21swapper/307:45:443
210727099120,7cyclictest0-21swapper/311:20:503
210727099112,6cyclictest0-21swapper/310:00:453
210727099112,6cyclictest0-21swapper/308:00:403
210727099112,5cyclictest0-21swapper/311:25:363
210727099111,7cyclictest0-21swapper/311:00:463
210727099111,7cyclictest0-21swapper/311:00:463
210727099111,5cyclictest0-21swapper/307:15:453
210727099102,5cyclictest0-21swapper/309:25:453
210727099102,5cyclictest0-21swapper/307:35:023
21072709982,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional