You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 17:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Jul 17, 2026 12:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1956005996224,12cyclictest12650irq/32-eth010:30:410
1956007995746,7cyclictest2038363-21kernelversion10:25:412
1956006995638,15cyclictest2082924-21grep11:40:011
1956006995633,20cyclictest2092690-21fschecks_count11:55:361
1956007995445,6cyclictest1002-21mta-sts-daemon12:25:412
1956007995442,7cyclictest2089642-21fschecks_count11:50:342
1956007995442,7cyclictest2089642-21fschecks_count11:50:342
1956007995242,5cyclictest0-21swapper/208:40:012
1956006995024,5cyclictest0-21swapper/109:15:021
1956006995023,3cyclictest0-21swapper/112:00:481
1956005995013,32cyclictest2081056-21ssh11:35:450
1956007994938,6cyclictest0-21swapper/211:10:492
195600699497,5cyclictest171rcu_preempt10:30:431
1956007994840,4cyclictest0-21swapper/212:30:402
1956007994734,10cyclictest1-21systemd09:25:512
1956006994728,15cyclictest0-21swapper/112:35:431
1956006994726,17cyclictest567-21gdbus09:15:141
1956006994725,2cyclictest0-21swapper/108:35:011
1956007994635,7cyclictest2001164-21cron09:25:012
195600699468,15cyclictest441-21thd10:05:461
1956006994626,16cyclictest0-21swapper/109:35:531
1956006994622,4cyclictest0-21swapper/111:45:011
1956005994635,6cyclictest151ktimers/010:50:430
1956005994633,6cyclictest151ktimers/008:05:510
1956005994633,6cyclictest151ktimers/008:05:510
1956007994432,7cyclictest420-21avahi-daemon12:21:082
1956005994437,4cyclictest1995085-21latency_hist09:15:020
1956005994433,5cyclictest151ktimers/008:45:460
1956007994333,7cyclictest2024054-21unixbench_multi10:00:552
1956006994335,5cyclictest1963514-21systemd-run07:35:021
1956005994329,11cyclictest2007248-21systemd-run09:35:020
1956007994232,6cyclictest2050702-21ntp_kernel_pll_10:45:422
195600799422,32cyclictest2080940-21ssh11:35:402
1956005994234,5cyclictest439-21systemd-logind10:00:010
1956006994134,4cyclictest0-21swapper/110:10:511
1956005994131,7cyclictest1998118-21idleruntime-cro09:20:010
195600799409,20cyclictest421-21dbus-daemon07:20:452
1956007994033,3cyclictest2092104-21kworker/u8:9-xprtiod11:57:502
1956007994032,5cyclictest1000-21mta-sts-daemon11:16:002
1956007994031,4cyclictest0-21swapper/210:40:012
1956007994030,6cyclictest1002-21mta-sts-daemon11:00:122
195600799400,33cyclictest2096155-21ssh12:00:432
1956006994031,6cyclictest0-21swapper/112:05:441
1956005994031,4cyclictest0-21swapper/010:25:520
1956007993931,4cyclictest2031479-21kworker/u8:4-xprtiod10:55:572
1956007993930,6cyclictest1987699-21latency_hist08:55:032
195600699395,28cyclictest2005465-21grep09:30:501
1956006993930,6cyclictest2108809-21unixbench_singl12:20:541
1956005993932,4cyclictest2098438-21/usr/sbin/munin12:05:410
1956005993928,8cyclictest2085934-21cron11:45:010
1956007993828,5cyclictest0-21swapper/210:15:472
1956007993827,8cyclictest1991306-21needreboot09:05:402
195600799381,29cyclictest2105226-21ssh12:15:472
1956006993830,5cyclictest2087632-21unixbench_singl11:45:541
1956006993830,5cyclictest2087632-21unixbench_singl11:45:541
1956006993830,5cyclictest2060357-21unixbench_multi11:00:551
1956006993829,3cyclictest0-21swapper/110:40:421
1956006993828,7cyclictest1969472-21cron07:55:011
1956006993828,4cyclictest0-21swapper/110:20:461
1956005993832,3cyclictest0-21swapper/010:55:370
1956005993831,4cyclictest2010266-21grep09:40:010
1956005993829,5cyclictest151ktimers/008:15:470
1956005993829,5cyclictest151ktimers/008:15:470
195600799373,25cyclictest2035686-21ssh10:20:482
1956007993729,5cyclictest1976002-21fschecks_count08:15:372
1956007993729,5cyclictest1976002-21fschecks_count08:15:372
1956007993728,6cyclictest1973030-21fschecks_time08:05:392
1956007993728,6cyclictest1973030-21fschecks_time08:05:392
1956007993728,5cyclictest1003-21mta-sts-daemon11:48:192
1956007993728,5cyclictest1003-21mta-sts-daemon11:48:192
1956007993722,10cyclictest1149763-21cups-browsed10:14:192
1956006993730,4cyclictest2113133-21ssh12:30:011
1956006993729,5cyclictest2016896-21aten_r2power_vo09:50:321
1956006993729,5cyclictest1992150-21latency_hist09:10:021
1956006993727,7cyclictest2114010-21latency12:30:371
1956006993727,6cyclictest420-21avahi-daemon11:05:541
1956006993726,8cyclictest2058669-21sendmail11:00:011
195600599374,27cyclictest665-21snmpd07:10:420
1956005993730,4cyclictest2102732-21unixbench_multi12:10:550
1956005993730,4cyclictest2050674-21ntpq10:45:420
1956005993730,3cyclictest0-21swapper/012:20:430
1956005993728,7cyclictest1981459-21kworker/u8:0-nfsiod08:35:010
1956005993726,8cyclictest1964296-21needreboot07:35:400
1956005993722,11cyclictest420-21avahi-daemon12:25:430
1956005993719,13cyclictest2043789-21kworker/u8:10+rpciod10:40:420
195600599371,33cyclictest1991994-21diskmemload11:25:430
1956007993628,5cyclictest2071517-21sort11:20:362
1956007993628,5cyclictest2014701-21mailstats09:45:532
1956006993628,5cyclictest1975489-21grep08:15:001
1956005993630,4cyclictest2013778-21aten_r2power_en09:45:340
1956005993628,6cyclictest2034479-21grep10:20:010
195600799353,23cyclictest2084126-21rm11:40:492
1956007993527,4cyclictest1002-21mta-sts-daemon09:12:172
1956007993527,4cyclictest0-21swapper/211:35:032
1956007993526,6cyclictest1986010-21systemd-run08:50:012
1956007993526,6cyclictest1003-21mta-sts-daemon11:06:082
1956007993526,5cyclictest1001-21mta-sts-daemon12:09:432
195600799352,23cyclictest2020461-21ssh09:55:442
195600699354,23cyclictest2068909-21ssh11:15:451
1956006993528,4cyclictest2105471-21mailstats12:15:541
1956006993528,4cyclictest1970974-21cron08:00:011
1956006993527,5cyclictest421-21dbus-daemon09:44:581
1956006993527,5cyclictest420-21avahi-daemon09:25:521
1956006993527,5cyclictest1998817-21fschecks_count09:20:351
1956006993527,5cyclictest1964983-21idleruntime-cro07:40:011
195600699351,24cyclictest2032684-21ssh10:15:511
1956005993524,6cyclictest12750irq/33-eth012:30:440
195600799349,18cyclictest2053832-21ssh10:50:482
195600799344,22cyclictest2102300-21ssh12:10:492
1956007993428,3cyclictest0-21swapper/212:40:012
1956007993425,7cyclictest1998107-21munin-run09:20:012
1956007993425,6cyclictest1970003-21fschecks_count07:55:362
195600799342,25cyclictest2005339-21ssh09:30:452
1956006993425,6cyclictest0-21swapper/110:45:431
1956006993421,10cyclictest2090504-21timerwakeupswit11:50:521
1956006993421,10cyclictest2090504-21timerwakeupswit11:50:521
1956005993429,2cyclictest0-21swapper/012:40:010
1956005993428,3cyclictest2046581-21sendmail10:40:010
1956005993428,3cyclictest0-21swapper/011:05:330
1956005993427,4cyclictest1963602-21latency_hist07:35:020
195600799334,24cyclictest2074891-21ssh11:25:412
195600799333,25cyclictest2041834-21sh10:30:492
1956007993325,5cyclictest2048313-21unixbench_singl10:40:572
1956007993321,9cyclictest1977901-21ntp_kernel_pll_08:20:432
1956007993321,8cyclictest421-21dbus-daemon09:35:482
1956006993326,3cyclictest0-21swapper/108:40:451
1956006993323,8cyclictest1986798-21needreboot08:50:421
195600599333,28cyclictest0-21swapper/011:55:470
1956005993328,2cyclictest0-21swapper/009:55:020
1956005993324,7cyclictest1972503-21/usr/sbin/munin08:05:020
1956005993323,6cyclictest1991989-21cron09:10:010
195600799325,17cyclictest428-21mta-sts-daemon07:15:472
1956007993227,2cyclictest0-21swapper/209:00:012
1956007993224,4cyclictest0-21swapper/209:40:382
1956007993224,4cyclictest0-21swapper/208:40:512
1956007993223,6cyclictest1989515-21fschecks_count09:00:342
1956007993223,6cyclictest1975509-21systemd-run08:15:012
1956007993223,5cyclictest1963514-21systemd-run07:35:022
1956007993223,5cyclictest0-21swapper/209:50:352
1956006993225,4cyclictest1961030-21fschecks_time07:25:351
1956006993225,4cyclictest1000-21mta-sts-daemon10:37:491
1956006993222,7cyclictest2078375-21timerandwakeup11:30:551
1956006993222,6cyclictest2020590-21sh09:55:501
1956006993221,8cyclictest1985338-21ntp_kernel_err08:45:401
1956005993224,5cyclictest2080073-21latency_hist11:35:030
195600599322,23cyclictest2011439-21ssh09:40:460
1956005993221,4cyclictest1987415-21unixbench_singl08:50:560
195600799317,17cyclictest351ktimers/208:25:442
1956007993125,3cyclictest0-21swapper/208:05:012
1956007993123,5cyclictest1-21systemd07:15:022
1956007993122,6cyclictest1965514-21fschecks_count07:40:362
1956007993121,5cyclictest0-21swapper/210:07:202
1956006993124,4cyclictest1959200-21latency_hist07:20:031
1956006993119,9cyclictest2074437-21cpu11:25:331
1956005993127,2cyclictest0-21swapper/012:20:010
1956005993125,3cyclictest0-21swapper/007:25:460
1956005993124,4cyclictest1999730-21unixbench-2d09:20:540
1956005993124,4cyclictest1983516-21fschecks_count08:40:350
1956005993123,6cyclictest1970983-21/usr/sbin/munin08:00:020
1956005993123,6cyclictest1-21systemd07:50:020
1956007993018,9cyclictest1961401-21ntp_kernel_pll_07:25:432
1956006993025,3cyclictest1944973-21kworker/u8:7-rpciod08:05:011
1956006993024,3cyclictest0-21swapper/111:20:561
195600699302,20cyclictest2102300-21ssh12:10:491
1956006993021,6cyclictest1149763-21cups-browsed08:26:041
1956005993024,4cyclictest1990691-21latency_hist09:05:020
1956005993023,4cyclictest2001707-21aten_r2power_vo09:25:320
195600599301,23cyclictest2096108-21ssh12:00:410
1956005992922,4cyclictest2029406-21missed_timers10:10:440
1956005992922,4cyclictest2024039-21unixbench_multi10:00:550
1956005992914,10cyclictest12650irq/32-eth010:05:480
1956006992822,4cyclictest0-21swapper/108:40:021
1956006992822,3cyclictest0-21swapper/110:00:531
1956006992821,3cyclictest0-21swapper/109:00:481
1956006992820,5cyclictest2014402-21ntp_kernel_pll_09:45:441
1956006992820,5cyclictest1988994-21idleruntime-cro09:00:011
1956006992820,5cyclictest1960428-21unixbench_singl07:20:541
1956005992822,4cyclictest2087549-21unixbench-2d11:45:530
1956005992822,4cyclictest2087549-21unixbench-2d11:45:530
1956005992822,4cyclictest0-21swapper/011:00:320
1956005992822,3cyclictest151ktimers/008:55:420
1956005992821,5cyclictest1149763-21cups-browsed08:35:250
1956005992817,7cyclictest12650irq/32-eth011:10:500
1956006992719,5cyclictest2054346-21unixbench_singl10:50:551
1956006992719,5cyclictest1-21systemd07:10:021
1956006992718,7cyclictest2038708-21sh10:25:501
1956005992723,2cyclictest0-21swapper/010:20:440
1956005992720,5cyclictest421-21dbus-daemon08:25:530
1956005992719,2cyclictest0-21swapper/011:20:550
1956005992718,6cyclictest12750irq/33-eth007:40:450
1956005992716,8cyclictest0-21swapper/011:50:530
1956005992716,8cyclictest0-21swapper/011:50:530
1956006992621,2cyclictest0-21swapper/108:10:001
1956006992621,2cyclictest0-21swapper/108:10:001
1956006992619,5cyclictest2065597-21latency11:10:391
1956006992619,4cyclictest1149763-21cups-browsed07:48:501
195600799255,13cyclictest0-21swapper/207:35:492
1956007992518,4cyclictest1003-21mta-sts-daemon08:31:012
1956006992518,4cyclictest1956909-21ntp_kernel_pll_07:10:441
195600599253,16cyclictest665-21snmpd07:15:440
1956005992513,8cyclictest2068595-21cut11:15:370
195600799245,12cyclictest421-21dbus-daemon07:55:012
1956006992418,3cyclictest0-21swapper/108:15:371
1956006992418,3cyclictest0-21swapper/108:15:371
1956005992419,3cyclictest1977270-21aten_r2power_cu08:20:310
1956005992419,2cyclictest0-21swapper/007:50:360
1956007992316,3cyclictest0-21swapper/207:50:012
1956006992317,3cyclictest0-21swapper/108:20:451
1956005992316,4cyclictest1149763-21cups-browsed08:10:170
195600699221,16cyclictest1965888-21ntpq07:40:441
1956005992215,4cyclictest0-21swapper/007:20:450
195600899206,8cyclictest0-21swapper/310:00:433
195600899202,10cyclictest0-21swapper/309:15:433
195600899194,8cyclictest0-21swapper/309:40:453
195600899192,10cyclictest0-21swapper/311:35:403
1956007991913,3cyclictest0-21swapper/207:10:032
195600899183,7cyclictest0-21swapper/312:05:443
195600899182,9cyclictest0-21swapper/311:15:423
195600899182,9cyclictest0-21swapper/309:45:463
195600899182,9cyclictest0-21swapper/309:20:483
195600899182,10cyclictest0-21swapper/310:45:423
195600899182,10cyclictest0-21swapper/309:30:453
195600599181,14cyclictest1956011-21true07:10:020
195600899174,10cyclictest0-21swapper/311:55:473
195600899173,8cyclictest0-21swapper/310:15:473
195600899172,9cyclictest0-21swapper/310:25:473
195600899172,8cyclictest0-21swapper/312:00:423
195600899172,8cyclictest0-21swapper/310:20:433
195600899172,7cyclictest0-21swapper/309:35:413
195600899172,11cyclictest0-21swapper/309:25:423
195600899171,9cyclictest0-21swapper/312:35:413
195600899164,7cyclictest0-21swapper/311:20:423
195600899163,8cyclictest0-21swapper/311:30:433
195600899162,9cyclictest0-21swapper/312:25:473
195600899162,8cyclictest0-21swapper/312:15:473
195600899162,8cyclictest0-21swapper/310:40:413
195600899162,8cyclictest0-21swapper/309:55:363
195600899162,8cyclictest0-21swapper/309:10:423
195600899162,7cyclictest0-21swapper/309:05:433
195600899162,7cyclictest0-21swapper/308:15:493
195600899162,7cyclictest0-21swapper/308:15:493
195600899161,9cyclictest0-21swapper/310:50:463
195600899161,9cyclictest0-21swapper/310:05:493
195600899161,8cyclictest0-21swapper/308:20:453
195600899161,8cyclictest0-21swapper/308:05:473
195600899161,8cyclictest0-21swapper/308:05:473
195600899153,8cyclictest0-21swapper/310:35:423
195600899153,7cyclictest0-21swapper/312:20:473
195600899153,6cyclictest0-21swapper/308:25:423
195600899152,8cyclictest0-21swapper/311:45:423
195600899152,8cyclictest0-21swapper/311:45:423
195600899152,8cyclictest0-21swapper/311:05:423
195600899152,8cyclictest0-21swapper/310:30:413
195600899152,6cyclictest0-21swapper/307:10:453
195600899151,8cyclictest0-21swapper/307:25:403
195600899151,7cyclictest0-21swapper/307:15:443
195600899151,6cyclictest0-21swapper/308:00:433
195600899151,6cyclictest0-21swapper/307:55:413
195600899145,6cyclictest0-21swapper/311:10:503
195600899143,7cyclictest0-21swapper/312:10:433
195600899143,7cyclictest0-21swapper/311:50:403
195600899143,7cyclictest0-21swapper/311:50:403
195600899142,8cyclictest0-21swapper/311:40:423
195600899142,8cyclictest0-21swapper/310:55:433
195600899142,8cyclictest0-21swapper/309:50:353
195600899142,7cyclictest0-21swapper/312:30:413
195600899142,7cyclictest0-21swapper/311:25:343
195600899141,7cyclictest0-21swapper/307:30:433
195600899141,6cyclictest0-21swapper/308:30:423
195600899133,8cyclictest0-21swapper/311:00:323
195600899132,7cyclictest0-21swapper/310:10:413
195600899132,6cyclictest0-21swapper/308:45:403
195600899132,6cyclictest0-21swapper/307:20:453
195600899132,4cyclictest0-21swapper/308:35:403
195600899131,7cyclictest0-21swapper/308:55:473
195600899131,6cyclictest0-21swapper/308:40:463
195600899131,6cyclictest0-21swapper/307:45:423
195600899131,6cyclictest0-21swapper/307:35:463
195600899123,5cyclictest0-21swapper/307:50:433
195600899122,7cyclictest0-21swapper/308:15:013
195600899122,7cyclictest0-21swapper/307:40:413
195600899112,5cyclictest0-21swapper/309:00:403
195600899111,6cyclictest0-21swapper/308:50:373
19560089981,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional