You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 16:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Tue Aug 04, 2026 12:43:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1995241996253,6cyclictest2048268-21grep09:36:042
1995240996149,8cyclictest1215473-21cups-browsed09:25:551
1995240996149,8cyclictest1215473-21cups-browsed09:25:551
1995240995529,3cyclictest0-21swapper/109:00:011
199523999551,20cyclictest2041663-21ssh09:25:560
199523999551,20cyclictest2041663-21ssh09:25:560
1995240995327,6cyclictest2016251-21systemd-run08:20:011
199523999511,12cyclictest2059986-21ssh09:55:570
1995241995042,4cyclictest0-21swapper/212:10:532
1995240995028,6cyclictest2038469-21ntp_kernel_err09:20:501
1995240995028,6cyclictest2038469-21ntp_kernel_err09:20:501
1995240995021,6cyclictest0-21swapper/112:05:541
1995240994735,8cyclictest1215473-21cups-browsed11:06:011
1995239994723,21cyclictest2093944-21unixbench_multi10:51:030
1995240994637,6cyclictest1-21systemd12:30:011
1995240994634,8cyclictest1215473-21cups-browsed11:45:501
199523999453,36cyclictest2130024-21ssh11:50:550
1995239994529,9cyclictest12650irq/32-eth012:25:560
199523999450,15cyclictest0-21swapper/012:35:540
1995241994437,3cyclictest0-21swapper/212:10:012
1995241994434,6cyclictest2111426-21kernelversion11:20:502
1995241994434,6cyclictest2010250-21systemd-run08:00:022
1995241994434,4cyclictest0-21swapper/210:15:522
1995240994436,5cyclictest2025236-21systemd-run08:50:011
199523999443,9cyclictest2011114-21ntpq08:00:530
199523999443,7cyclictest2014173-21ntpq08:10:550
199523999441,12cyclictest1999168-21ntpq07:20:550
1995241994324,12cyclictest351ktimers/207:35:522
1995240994336,4cyclictest1003-21mta-sts-daemon10:03:471
199523999431,17cyclictest1215473-21cups-browsed11:40:540
1995241994236,3cyclictest0-21swapper/207:20:012
1995241994234,5cyclictest2014706-21cron08:15:012
1995241994234,5cyclictest0-21swapper/211:00:012
1995241994231,7cyclictest1215473-21cups-browsed09:22:242
1995241994231,7cyclictest1215473-21cups-browsed09:22:242
1995240994234,4cyclictest0-21swapper/110:10:561
199523999422,8cyclictest2024694-21ntpq08:45:550
1995239994217,21cyclictest12650irq/32-eth012:00:510
199523999421,13cyclictest2069009-21ntpq10:10:500
199524199414,6cyclictest171rcu_preempt11:10:492
1995241994133,5cyclictest2153169-21grep12:30:002
1995241994131,7cyclictest1215473-21cups-browsed11:42:552
1995241994129,8cyclictest2067919-21systemd-run10:10:012
1995240994133,5cyclictest421-21dbus-daemon07:50:021
1995240994132,6cyclictest2100077-21unixbench_multi11:01:061
1995240994132,5cyclictest2154224-21needreboot12:30:491
1995240994132,5cyclictest2154224-21needreboot12:30:491
1995239994119,19cyclictest0-21swapper/010:50:000
199523999411,13cyclictest2020137-21ntpq08:30:530
199523999411,13cyclictest2003674-21ntpq07:35:550
199523999411,11cyclictest2029147-21ntpq09:00:540
1995241994032,5cyclictest2017728-21cron08:25:012
1995241994031,5cyclictest2086144-21ssh10:40:002
199524099405,26cyclictest2031214-21diskmemload12:35:541
1995240994032,5cyclictest2052676-21cron09:45:011
1995240994032,5cyclictest2037355-21idleruntime-cro09:20:011
1995240994032,5cyclictest2037355-21idleruntime-cro09:20:011
1995240994032,3cyclictest0-21swapper/110:15:591
1995240994031,6cyclictest265-21systemd-journal08:15:011
1995240994031,5cyclictest420-21avahi-daemon08:26:001
199523999404,9cyclictest2081241-21ntpq10:30:530
1995239994025,12cyclictest2127487-21ssh11:46:020
1995239994019,18cyclictest0-21swapper/010:56:020
1995239994018,20cyclictest2139499-21timerandwakeup12:06:070
1995239994016,21cyclictest0-21swapper/011:20:550
199523999401,13cyclictest2050878-21ntpq09:40:570
199523999401,11cyclictest2109474-21ssh11:16:450
1995239994010,4cyclictest0-21swapper/007:40:420
1995241993931,5cyclictest2077808-21fschecks_time10:25:452
1995241993927,6cyclictest0-21swapper/212:30:522
1995241993927,6cyclictest0-21swapper/212:30:522
1995240993932,3cyclictest0-21swapper/112:15:021
1995239993923,13cyclictest1998276-21latency_hist07:20:020
1995239993922,14cyclictest0-21swapper/009:45:540
1995239993916,20cyclictest0-21swapper/010:06:000
1995239993915,21cyclictest0-21swapper/012:15:010
1995239993912,24cyclictest0-21swapper/009:50:450
199523999391,17cyclictest2044670-21ntpq09:30:530
199523999391,17cyclictest2044670-21ntpq09:30:530
199523999391,14cyclictest2105608-21ntpq11:10:510
199523999391,13cyclictest2144888-21irqstats12:15:450
199523999391,13cyclictest2072126-21ntpq10:15:540
199523999391,12cyclictest2031930-21df09:10:440
199523999391,12cyclictest2009211-21diskstats07:55:440
1995241993832,3cyclictest0-21swapper/209:41:022
1995241993830,4cyclictest0-21swapper/211:00:512
1995241993829,5cyclictest0-21swapper/208:20:012
1995240993830,5cyclictest2137951-21grep12:05:011
199523999382,17cyclictest2035571-21ssh09:16:030
199523999382,17cyclictest2035571-21ssh09:16:030
199523999381,8cyclictest2015573-21ntpq08:15:510
1995239993818,17cyclictest0-21swapper/011:55:540
1995239993814,21cyclictest2114016-21aten_r2power_cu11:25:420
1995239993813,22cyclictest1-21systemd08:40:010
199523999381,14cyclictest2154484-21ps12:30:590
199523999381,14cyclictest2154484-21ps12:30:590
199523999381,13cyclictest2038223-21irqstats09:20:470
199523999381,13cyclictest2038223-21irqstats09:20:470
1995241993729,4cyclictest2011746-21kthreadd08:05:012
1995241993728,6cyclictest2127479-21grep11:46:022
1995240993729,5cyclictest2131841-21munin-plugin-st11:55:001
1995240993729,5cyclictest2017753-21systemd-run08:25:011
1995240993727,7cyclictest2150124-21cut12:25:021
1995239993731,4cyclictest2104444-21munin-run11:10:010
199523999372,13cyclictest2023662-21unixbench_singl08:41:030
199523999372,13cyclictest2023662-21unixbench_singl08:41:030
1995239993713,3cyclictest2086655-21telnet10:40:420
1995239993712,2cyclictest0-21swapper/008:59:450
199523999371,12cyclictest2121180-21ssh11:36:070
199523999371,12cyclictest2099481-21ld11:00:510
199523999371,12cyclictest2085056-21ssh10:36:330
1995239993710,9cyclictest2075117-21ssh10:20:510
1995241993627,6cyclictest420-21avahi-daemon11:55:442
1995241993626,7cyclictest2061822-21sendmail10:00:012
1995241993624,9cyclictest2117499-21kernelversion11:30:462
1995240993630,2cyclictest0-21swapper/111:55:471
1995240993629,4cyclictest2120572-21latency11:35:511
1995240993627,7cyclictest2008717-21munin-plugin-st07:55:011
1995240993623,7cyclictest0-21swapper/110:35:571
199523999362,4cyclictest2063084-21ntp_states10:01:000
1995239993613,3cyclictest0-21swapper/009:35:410
1995239993613,20cyclictest0-21swapper/008:25:420
1995239993610,3cyclictest2017725-21cron08:25:010
199524199357,19cyclictest1001-21mta-sts-daemon10:00:592
1995241993529,4cyclictest1995238-21cyclictest12:05:012
1995241993528,4cyclictest2001113-21tail07:26:032
1995241993528,4cyclictest2001113-21tail07:26:032
1995241993527,5cyclictest1-21systemd10:46:012
199524199352,6cyclictest171rcu_preempt07:20:562
1995241993525,6cyclictest2121404-21unixbench_singl11:36:092
1995241993525,6cyclictest1002-21mta-sts-daemon07:10:532
1995240993528,4cyclictest2034294-21munin-run09:15:011
1995240993528,4cyclictest2029795-21latency_hist09:05:021
199524099352,26cyclictest2114700-21ssh11:25:531
199523999354,7cyclictest2012571-21ntpq08:05:490
199523999352,13cyclictest2007715-21diskstats07:50:430
199523999351,14cyclictest2030772-21ps09:05:590
199523999351,11cyclictest2078614-21/usr/sbin/munin10:26:060
199523999351,10cyclictest2147948-21/usr/sbin/munin12:20:470
199524199343,25cyclictest2087299-21ssh10:40:512
199524199343,25cyclictest2032430-21ssh09:10:562
1995241993426,5cyclictest1215473-21cups-browsed08:40:012
1995241993426,4cyclictest0-21swapper/209:45:572
1995241993425,6cyclictest2130501-21unixbench_multi11:51:042
1995241993424,7cyclictest2157703-21switchtime12:36:022
199524199342,25cyclictest2075272-21ssh10:20:572
1995240993429,2cyclictest0-21swapper/107:10:011
1995240993428,3cyclictest0-21swapper/111:30:401
1995240993428,3cyclictest0-21swapper/109:30:471
1995240993428,3cyclictest0-21swapper/109:30:471
1995240993427,4cyclictest2039936-21kworker/u8:6-nfsiod09:40:011
1995240993426,5cyclictest2098347-21grep11:00:011
1995240993426,5cyclictest2019747-21fschecks_time08:30:441
199523999345,10cyclictest1001-21mta-sts-daemon07:25:540
199523999345,10cyclictest1001-21mta-sts-daemon07:25:540
199523999343,6cyclictest2119677-21munin-run11:35:020
199523999341,12cyclictest2026174-21ntpq08:50:560
199523999341,11cyclictest2001860-21irqstats07:30:460
199524199336,13cyclictest2102646-21ssh11:05:572
1995241993327,3cyclictest0-21swapper/209:20:012
1995241993327,3cyclictest0-21swapper/209:20:012
1995241993327,3cyclictest0-21swapper/207:45:002
1995241993326,3cyclictest0-21swapper/209:30:482
1995241993326,3cyclictest0-21swapper/209:30:482
199524099333,23cyclictest2108609-21ssh11:15:511
1995240993326,4cyclictest2111868-21proc_pri11:21:031
1995240993325,5cyclictest2077798-21cut10:25:451
1995240993323,7cyclictest2054425-21grep09:46:041
199523999332,16cyclictest1996178-21ntp_states07:10:550
1995241993225,4cyclictest2083141-21kworker/u8:5-xprtiod10:51:032
1995241993224,5cyclictest2070946-21idleruntime-cro10:15:012
1995241993224,4cyclictest0-21swapper/208:25:542
1995241993221,4cyclictest0-21swapper/209:50:512
1995241993214,14cyclictest351ktimers/208:05:542
1995240993225,3cyclictest0-21swapper/110:05:581
1995240993225,3cyclictest0-21swapper/109:55:431
1995240993224,5cyclictest1215473-21cups-browsed07:32:281
1995241993125,3cyclictest0-21swapper/211:20:012
1995241993123,5cyclictest2150227-21ssh12:25:032
1995240993124,4cyclictest1002-21mta-sts-daemon07:55:511
1995240993123,5cyclictest1996794-21latency_hist07:15:021
1995240993122,7cyclictest2086978-21sh10:40:471
1995240993122,7cyclictest2030933-21timerandwakeup09:06:031
1995241993024,3cyclictest0-21swapper/212:20:012
1995241993022,5cyclictest2081217-21ntp_kernel_pll_10:30:522
1995241993021,6cyclictest2006246-21fschecks_count07:45:462
1995240993023,4cyclictest1002-21mta-sts-daemon08:00:421
1995240993022,5cyclictest2000251-21fschecks_count07:25:451
1995240993022,5cyclictest2000251-21fschecks_count07:25:451
1995240993022,5cyclictest0-21swapper/110:45:401
199524199294,5cyclictest171rcu_preempt08:50:552
1995241992922,4cyclictest2030362-21kernelversion09:05:482
1995241992921,5cyclictest2023714-21cron08:45:002
1995241992921,5cyclictest2023714-21cron08:45:002
199524099292,23cyclictest2105623-21ssh11:10:521
1995240992922,3cyclictest0-21swapper/109:50:461
1995240992921,5cyclictest2077081-21latency_hist10:25:021
1995241992821,4cyclictest2008026-21needreboot07:50:482
1995240992823,2cyclictest0-21swapper/108:40:011
1995240992821,4cyclictest2145681-21unixbench-2d12:16:031
1995240992821,4cyclictest2083123-21grep10:35:011
1995240992820,5cyclictest2092759-21aten2.4-expect10:50:401
1995239992818,8cyclictest2006536-21needreboot07:45:500
1995241992719,5cyclictest2002654-21unixbench_singl07:31:032
1995240992721,3cyclictest1001-21mta-sts-daemon07:35:421
1995240992719,5cyclictest1997532-21uname07:15:501
1995240992719,4cyclictest0-21swapper/111:40:511
1995240992718,6cyclictest2026232-21proc_pri08:50:581
1995241992617,5cyclictest2024089-21aten_r2power_po08:45:402
199524199261,12cyclictest2115014-21sensors_volt11:26:042
1995240992619,4cyclictest2004693-21cpuspeed07:40:441
199524199252,17cyclictest2041547-21ssh09:25:522
199524199252,17cyclictest2041547-21ssh09:25:522
1995241992510,12cyclictest351ktimers/209:00:502
199524199243,15cyclictest428-21mta-sts-daemon08:55:522
1995241992414,7cyclictest2020073-21ntpq08:30:502
1995240992316,4cyclictest1998642-21aten_r2power_vo07:20:431
199524099231,17cyclictest2012749-21ps08:05:571
1995241992215,4cyclictest1995313-21latency_hist07:10:022
1995240992216,3cyclictest0-21swapper/108:40:451
1995240992216,3cyclictest0-21swapper/108:40:451
199524299203,7cyclictest0-21swapper/311:30:493
199524299202,10cyclictest0-21swapper/311:40:533
199524299183,9cyclictest0-21swapper/311:15:513
199524299183,9cyclictest0-21swapper/310:45:543
199524299182,9cyclictest0-21swapper/309:35:563
199524299182,8cyclictest0-21swapper/310:25:593
199524299181,9cyclictest0-21swapper/309:50:513
199523999180,8cyclictest665-21snmpd07:10:010
199524299175,6cyclictest0-21swapper/310:15:523
199524299173,8cyclictest0-21swapper/310:20:523
199524299173,7cyclictest0-21swapper/309:16:003
199524299173,7cyclictest0-21swapper/309:16:003
199524299172,9cyclictest0-21swapper/310:05:493
199524299172,8cyclictest0-21swapper/312:20:553
199524299172,8cyclictest0-21swapper/311:21:023
199524299172,8cyclictest0-21swapper/311:00:543
199524299172,10cyclictest0-21swapper/312:00:543
199524299171,8cyclictest0-21swapper/309:00:533
199524299171,10cyclictest0-21swapper/312:30:563
199524299171,10cyclictest0-21swapper/312:30:563
199524299165,7cyclictest0-21swapper/312:10:503
199524299164,7cyclictest0-21swapper/309:40:513
199524299163,7cyclictest0-21swapper/310:50:503
199524299162,9cyclictest0-21swapper/309:25:533
199524299162,9cyclictest0-21swapper/309:25:533
199524299162,8cyclictest0-21swapper/312:05:573
199524299162,6cyclictest0-21swapper/307:40:553
199524299161,8cyclictest0-21swapper/310:10:533
199524299161,7cyclictest0-21swapper/308:30:543
199524299152,8cyclictest0-21swapper/312:35:583
199524299152,8cyclictest0-21swapper/311:45:563
199524299152,8cyclictest0-21swapper/311:05:513
199524299152,7cyclictest0-21swapper/310:55:413
199524299152,6cyclictest0-21swapper/307:30:503
199524299151,9cyclictest0-21swapper/309:30:513
199524299151,9cyclictest0-21swapper/309:30:513
199524299151,8cyclictest0-21swapper/310:40:433
199524299151,6cyclictest0-21swapper/308:25:573
199524299150,6cyclictest0-21swapper/308:45:483
199524299143,7cyclictest0-21swapper/310:35:463
199524299143,6cyclictest0-21swapper/309:10:433
199524299143,4cyclictest0-21swapper/308:15:523
199524299141,8cyclictest0-21swapper/312:26:023
199524299141,8cyclictest0-21swapper/311:10:513
199524299141,7cyclictest0-21swapper/309:20:513
199524299141,7cyclictest0-21swapper/309:20:513
199524299141,6cyclictest0-21swapper/307:25:523
199524299141,6cyclictest0-21swapper/307:25:523
199524299134,6cyclictest0-21swapper/310:00:493
199524299133,5cyclictest0-21swapper/311:35:483
199524299132,7cyclictest0-21swapper/310:30:453
199524299132,7cyclictest0-21swapper/309:45:403
199524299132,7cyclictest0-21swapper/307:50:493
199524299132,5cyclictest0-21swapper/307:45:523
199524299131,8cyclictest0-21swapper/308:00:013
199524299131,7cyclictest0-21swapper/312:15:493
199524299131,6cyclictest0-21swapper/308:55:553
199524299131,6cyclictest0-21swapper/308:05:533
199524299131,5cyclictest0-21swapper/308:10:503
199524299122,7cyclictest0-21swapper/311:55:423
199524299122,7cyclictest0-21swapper/309:55:433
199524299122,7cyclictest0-21swapper/309:05:523
199524299122,6cyclictest0-21swapper/311:25:423
199524299122,6cyclictest0-21swapper/307:15:013
199524299122,5cyclictest0-21swapper/307:20:503
199524299121,6cyclictest0-21swapper/308:20:503
199524299121,6cyclictest0-21swapper/307:35:533
199524299113,5cyclictest0-21swapper/307:20:023
199524299110,8cyclictest0-21swapper/311:50:453
199524299104,4cyclictest0-21swapper/308:50:513
199524299101,5cyclictest0-21swapper/308:35:403
199524299101,5cyclictest0-21swapper/308:00:413
19952429991,5cyclictest0-21swapper/307:10:023
19952429990,6cyclictest0-21swapper/308:40:533
19952429990,6cyclictest0-21swapper/308:40:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional