You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 16:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 17, 2026 12:43:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
700704996554,5cyclictest0-21swapper/112:15:591
70070399602,48cyclictest874903-21ssh12:00:560
70070399602,48cyclictest874903-21ssh12:00:560
700704995837,17cyclictest893708-21ssh12:25:021
700703995530,17cyclictest151ktimers/012:35:520
700704995330,20cyclictest750554-21timerwakeupswit09:26:041
700703995245,4cyclictest744746-21basename09:20:010
700704995040,6cyclictest757928-21missed_timers09:35:511
700704995040,6cyclictest757928-21missed_timers09:35:511
700705994941,5cyclictest776956-21munin-run10:00:012
700704994939,6cyclictest4037003-21cups-browsed10:10:151
700705994842,3cyclictest0-21swapper/211:20:012
700703994840,5cyclictest736731-21munin-plugin-st09:10:000
700705994737,6cyclictest897678-21ssh12:30:012
700705994734,9cyclictest420-21avahi-daemon12:18:342
70070499473,16cyclictest818699-21ssh10:50:581
700703994735,5cyclictest151ktimers/007:15:570
700703994716,27cyclictest762112-21ssh09:40:540
700705994635,8cyclictest877580-21systemd-run12:05:012
700705994635,8cyclictest877580-21systemd-run12:05:012
700704994637,4cyclictest0-21swapper/112:36:011
700703994639,4cyclictest721716-21kworker/u8:9-xprtiod09:10:500
700705994537,5cyclictest829367-21ssh11:05:012
700705994537,5cyclictest829367-21ssh11:05:012
700704994527,15cyclictest863444-21sh11:46:081
700703994534,8cyclictest790314-21cat10:15:560
700703994534,8cyclictest790314-21cat10:15:560
700705994435,5cyclictest0-21swapper/212:15:012
700705994432,7cyclictest814422-21needreboot10:45:512
700704994436,5cyclictest899222-21mailstats12:31:061
700704994427,13cyclictest727710-21cron08:40:011
70070499441,12cyclictest716668-21ntpq08:01:011
700703994428,4cyclictest833399-21kworker/u8:12+rpciod11:50:010
700705994335,5cyclictest1056-21mta-sts-daemon11:35:542
70070499432,6cyclictest838874-21ntpq11:16:001
700704994317,23cyclictest765809-21ls09:45:471
700703994334,6cyclictest420-21avahi-daemon11:20:560
700703994330,6cyclictest857519-21ssh11:40:010
700703994327,13cyclictest721734-21latency_hist08:20:020
700705994234,5cyclictest845428-21sh11:25:012
700705994234,5cyclictest0-21swapper/209:00:012
700705994232,6cyclictest439-21systemd-logind11:30:012
70070499428,31cyclictest799254-21ssh10:26:231
700704994236,3cyclictest0-21swapper/110:40:011
70070499422,5cyclictest701572-21ntpq07:10:551
700704994224,15cyclictest850733-21netstat11:30:531
700704994216,22cyclictest0-21swapper/111:40:591
70070499421,15cyclictest712090-21ntpq07:45:551
70070499421,11cyclictest774291-21ntpq09:55:561
700703994231,8cyclictest776956-21munin-run10:00:010
700705994133,5cyclictest727764-21systemd-run08:40:012
70070499419,29cyclictest825559-21ssh11:00:051
70070499419,29cyclictest825559-21ssh11:00:051
70070499412,7cyclictest734624-21ntpq09:00:561
70070499412,6cyclictest731601-21ntpq08:50:521
70070499412,5cyclictest722571-21ntpq08:20:531
70070499411,9cyclictest719181-21diskstats08:10:471
700704994118,20cyclictest874219-21aten_r2power_fr12:00:441
700704994118,20cyclictest874219-21aten_r2power_fr12:00:441
70070499411,5cyclictest822714-21ntpq10:56:001
70070499411,5cyclictest709080-21ntpq07:35:551
70070499411,12cyclictest707185-21diskstats07:30:451
70070499411,10cyclictest780991-21ssh10:04:591
70070499411,10cyclictest710638-21ntpq07:40:581
70070499411,10cyclictest710638-21ntpq07:40:581
700703994134,4cyclictest420-21avahi-daemon12:20:010
700703994132,7cyclictest754758-21sh09:31:070
700703994132,6cyclictest421-21dbus-daemon09:55:010
700703994130,5cyclictest829337-21cut11:05:000
700703994130,5cyclictest829337-21cut11:05:000
70070499408,29cyclictest747801-21ssh09:23:111
70070499407,30cyclictest815974-21ssh10:47:351
70070499402,5cyclictest715062-21ntpq07:55:541
70070499402,4cyclictest728717-21ntpq08:41:011
70070499401,9cyclictest736182-21ntpq09:05:591
70070499401,9cyclictest704594-21ntpq07:20:531
70070499401,13cyclictest846963-21ssh11:26:021
70070499401,11cyclictest881177-21ssh12:09:111
70070499401,11cyclictest833261-21ssh11:09:491
70070499401,11cyclictest762879-21ssh09:41:131
70070499401,10cyclictest896923-21ssh12:28:371
70070499401,10cyclictest884763-21ssh12:13:211
70070499401,10cyclictest869370-21ssh11:54:471
70070499401,10cyclictest833815-21ssh11:10:271
70070499401,10cyclictest796503-21ssh10:23:521
70070499401,10cyclictest796503-21ssh10:23:521
70070499401,10cyclictest744231-21ssh09:19:031
70070499400,10cyclictest755190-21ssh09:31:571
70070399403,32cyclictest814489-21sh10:45:530
700703994028,9cyclictest893758-21latency_hist12:25:020
700703994025,11cyclictest0-21swapper/011:56:010
70070399402,28cyclictest838781-21ssh11:15:560
700703994022,14cyclictest881595-21cron12:10:000
700705993931,5cyclictest793089-21sendmail10:20:012
700705993931,5cyclictest793089-21sendmail10:20:012
700705993927,4cyclictest858292-21aten_r2power_vo11:40:452
70070499399,27cyclictest718321-21/usr/sbin/munin08:06:051
700704993932,4cyclictest703689-21grep07:20:011
70070499391,9cyclictest856973-21ssh11:38:591
70070499391,9cyclictest769217-21ssh09:50:141
70070499391,8cyclictest724157-21ntpq08:25:581
70070499391,5cyclictest725587-21ntpq08:30:541
70070499391,4cyclictest730100-21ntpq08:45:541
70070499391,12cyclictest738080-21ntpq09:10:541
70070499391,12cyclictest706673-21sshd07:29:271
70070499391,11cyclictest811809-21ssh10:42:221
70070499391,11cyclictest733204-21ntpq08:55:591
70070499391,10cyclictest791266-21ssh10:16:311
70070499391,10cyclictest791266-21ssh10:16:311
70070499391,10cyclictest784884-21ssh10:09:411
700703993932,4cyclictest766782-21grep09:46:050
700703993924,12cyclictest732210-21cron08:55:000
700705993829,5cyclictest778376-21ntp_kernel_pll_10:00:572
700705993829,5cyclictest764864-21munin-plugin-st09:45:002
70070499382,5cyclictest721150-21ntpq08:15:571
70070499381,9cyclictest843567-21ssh11:21:241
70070499381,10cyclictest869768-21ssh11:55:081
70070499371,8cyclictest801496-21ssh10:30:181
70070499371,8cyclictest713411-21memory07:50:481
700703993724,7cyclictest151ktimers/009:20:540
700703993722,8cyclictest12350irq/32-eth011:41:030
700703993712,19cyclictest782345-21ssh10:05:540
70070399364,25cyclictest802558-21ssh10:30:570
700703993629,4cyclictest735305-21latency_hist09:05:020
700703993629,4cyclictest151ktimers/008:10:590
700703993627,6cyclictest4037003-21cups-browsed09:27:540
700703993624,9cyclictest1-21systemd09:40:020
700703993624,9cyclictest1-21systemd09:40:020
700703993617,14cyclictest151ktimers/008:55:540
700705993529,3cyclictest0-21swapper/211:55:522
700705993528,4cyclictest709681-21munin-run07:40:002
700705993527,5cyclictest711192-21munin-plugin-st07:45:002
700705993527,5cyclictest711192-21munin-plugin-st07:45:002
700705993527,5cyclictest1-21systemd10:35:012
700705993527,5cyclictest0-21swapper/209:00:562
700705993522,10cyclictest822986-21switchtime10:56:062
700705993517,9cyclictest821229-21ssh10:54:572
700703993529,4cyclictest833403-21systemd11:10:010
700703993528,4cyclictest718757-21latency_hist08:10:020
700703993528,3cyclictest0-21swapper/011:11:000
700703993526,6cyclictest866105-21/usr/sbin/munin11:50:510
700703993525,7cyclictest795026-21unixbench_multi10:21:070
700703993525,7cyclictest795026-21unixbench_multi10:21:070
700703993525,7cyclictest4037003-21cups-browsed12:10:400
70070599349,9cyclictest0-21swapper/208:00:012
70070599343,25cyclictest754174-21ssh09:30:572
700705993427,4cyclictest809152-21basename10:40:012
700705993426,5cyclictest881633-21systemd-run12:10:012
700705993426,4cyclictest0-21swapper/212:36:002
700705993422,8cyclictest866697-21missed_timers11:50:542
700703993427,5cyclictest798353-21/usr/sbin/munin10:25:520
700703993427,4cyclictest849508-21latency_hist11:30:020
70070399340,7cyclictest898948-21ssh12:30:570
700705993326,4cyclictest853186-21sshd11:34:272
700705993325,5cyclictest0-21swapper/209:25:592
700705993324,5cyclictest782359-21ntpq10:05:552
700705993313,4cyclictest36-21ksoftirqd/209:45:592
70070399332,23cyclictest850958-21ssh11:31:020
70070599323,22cyclictest862918-21ssh11:45:562
700705993226,3cyclictest0-21swapper/212:31:062
700705993224,5cyclictest831265-21unixbench_multi11:06:062
700705993224,5cyclictest738476-21threads09:11:052
700705993223,6cyclictest736755-21unin-run09:10:012
700705993210,4cyclictest36-21ksoftirqd/207:15:552
70070399324,16cyclictest420-21avahi-daemon08:35:580
700703993224,5cyclictest708197-21systemd-run07:35:010
700703993223,6cyclictest730630-21unixbench_multi08:46:060
700703993219,9cyclictest895054-21ntpq12:26:020
700703993214,11cyclictest806458-21ssh10:35:530
70070599319,5cyclictest36-21ksoftirqd/207:10:472
700705993125,3cyclictest0-21swapper/208:46:062
700705993124,4cyclictest0-21swapper/210:26:002
700705993115,4cyclictest717175-21kworker/u8:8-rpciod08:05:032
700705993111,4cyclictest36-21ksoftirqd/209:51:012
700705993110,4cyclictest36-21ksoftirqd/210:45:012
700705993110,4cyclictest36-21ksoftirqd/210:20:482
700705993110,4cyclictest36-21ksoftirqd/210:20:482
700705993110,4cyclictest36-21ksoftirqd/209:40:022
700705993110,4cyclictest36-21ksoftirqd/209:40:022
700704993115,13cyclictest0-21swapper/107:10:011
700703993123,5cyclictest420-21avahi-daemon10:55:350
700703993123,3cyclictest0-21swapper/010:01:030
700703993121,8cyclictest722541-21ntp_kernel_err08:20:520
700703993120,9cyclictest1054-21mta-sts-daemon10:41:120
700703993118,11cyclictest710108-21cpu07:40:470
700703993118,11cyclictest710108-21cpu07:40:470
70070599309,5cyclictest36-21ksoftirqd/208:30:482
700705993024,3cyclictest0-21swapper/209:20:012
700705993022,5cyclictest707217-21fschecks_time07:30:462
70070599302,23cyclictest834758-21diskmemload11:10:582
700705993010,4cyclictest36-21ksoftirqd/207:45:462
700703993024,3cyclictest0-21swapper/010:55:020
70070599299,11cyclictest36-21ksoftirqd/208:16:042
700705992923,3cyclictest0-21swapper/212:21:022
700705992922,4cyclictest1056-21mta-sts-daemon08:00:562
700705992911,4cyclictest36-21ksoftirqd/207:25:462
700705992910,4cyclictest36-21ksoftirqd/209:20:582
700705992910,4cyclictest36-21ksoftirqd/208:25:462
700703992921,6cyclictest786224-21ssh10:10:520
700703992919,8cyclictest1051-21mta-sts-daemon07:35:590
700703992917,4cyclictest726077-21unixbench_multi08:31:040
700703992916,11cyclictest4037003-21cups-browsed07:10:590
70070599289,5cyclictest36-21ksoftirqd/207:20:552
70070599289,4cyclictest36-21ksoftirqd/208:55:012
700705992823,2cyclictest0-21swapper/210:12:272
700705992810,5cyclictest36-21ksoftirqd/208:40:522
700705992810,4cyclictest36-21ksoftirqd/208:20:482
700705992810,4cyclictest36-21ksoftirqd/208:10:582
70070599279,4cyclictest36-21ksoftirqd/207:50:562
700703992720,5cyclictest714116-21unixbench_singl07:51:040
700703992613,10cyclictest729127-21unixbench_multi08:41:100
700703992613,10cyclictest0-21swapper/007:26:040
700703992519,4cyclictest717183-21cron08:05:010
700703992519,4cyclictest712699-21systemd-run07:50:010
700703992518,5cyclictest4037003-21cups-browsed07:24:340
700703992516,3cyclictest0-21swapper/008:25:470
700703992412,9cyclictest714699-21fschecks_time07:55:460
700705992212,7cyclictest700652-21latency_hist07:10:022
70070699193,7cyclictest0-21swapper/310:45:523
70070699182,9cyclictest0-21swapper/310:35:563
70070399181,14cyclictest1-21systemd07:10:010
70070699165,5cyclictest0-21swapper/310:50:553
70070699163,6cyclictest0-21swapper/309:50:543
70070699162,7cyclictest0-21swapper/309:10:483
70070699161,9cyclictest0-21swapper/310:01:003
70070699161,8cyclictest0-21swapper/312:10:543
70070699161,7cyclictest0-21swapper/308:25:543
70070699161,7cyclictest0-21swapper/307:25:543
70070699161,6cyclictest0-21swapper/308:00:553
70070699154,5cyclictest0-21swapper/311:35:573
70070699152,8cyclictest0-21swapper/312:00:543
70070699152,8cyclictest0-21swapper/312:00:543
70070699152,7cyclictest0-21swapper/307:50:533
70070699152,6cyclictest0-21swapper/307:55:573
70070699151,7cyclictest0-21swapper/310:55:583
70070699151,7cyclictest0-21swapper/307:45:543
70070699143,7cyclictest0-21swapper/312:30:573
70070699143,5cyclictest0-21swapper/309:05:553
70070699142,8cyclictest0-21swapper/309:21:073
70070699142,7cyclictest0-21swapper/311:00:473
70070699142,7cyclictest0-21swapper/311:00:473
70070699142,7cyclictest0-21swapper/310:25:553
70070699142,7cyclictest0-21swapper/310:20:593
70070699142,7cyclictest0-21swapper/310:20:593
70070699142,6cyclictest0-21swapper/307:40:543
70070699142,6cyclictest0-21swapper/307:40:543
70070699141,9cyclictest0-21swapper/311:16:083
70070699141,8cyclictest0-21swapper/311:56:073
70070699141,8cyclictest0-21swapper/311:51:023
70070699141,8cyclictest0-21swapper/309:45:523
70070699141,7cyclictest0-21swapper/312:15:443
70070699141,7cyclictest0-21swapper/311:45:543
70070699141,7cyclictest0-21swapper/311:25:553
70070699141,7cyclictest0-21swapper/309:35:473
70070699141,7cyclictest0-21swapper/309:35:473
70070699141,7cyclictest0-21swapper/308:50:563
70070699141,7cyclictest0-21swapper/308:20:563
70070699141,6cyclictest0-21swapper/308:30:513
70070699141,6cyclictest0-21swapper/307:30:513
70070699135,5cyclictest0-21swapper/309:40:553
70070699135,5cyclictest0-21swapper/309:25:523
70070699134,6cyclictest0-21swapper/312:26:023
70070699133,6cyclictest0-21swapper/310:05:523
70070699133,5cyclictest0-21swapper/310:30:523
70070699132,7cyclictest0-21swapper/311:10:593
70070699132,7cyclictest0-21swapper/310:15:443
70070699132,7cyclictest0-21swapper/310:15:443
70070699132,6cyclictest0-21swapper/309:55:443
70070699132,6cyclictest0-21swapper/309:15:563
70070699132,5cyclictest0-21swapper/308:35:583
70070699131,8cyclictest0-21swapper/312:20:443
70070699131,8cyclictest0-21swapper/309:31:273
70070699131,7cyclictest0-21swapper/312:35:543
70070699131,7cyclictest0-21swapper/311:40:563
70070699131,7cyclictest0-21swapper/311:30:543
70070699131,7cyclictest0-21swapper/310:40:533
70070699131,6cyclictest0-21swapper/308:55:523
70070699131,6cyclictest0-21swapper/308:10:593
70070699131,6cyclictest0-21swapper/307:11:003
70070699131,5cyclictest0-21swapper/308:15:513
70070699122,7cyclictest0-21swapper/311:21:273
70070699122,6cyclictest0-21swapper/309:00:563
70070699121,7cyclictest0-21swapper/310:10:593
70070699121,6cyclictest0-21swapper/308:05:533
70070699112,5cyclictest0-21swapper/307:15:483
70070699111,7cyclictest0-21swapper/312:05:433
70070699111,6cyclictest0-21swapper/311:05:443
70070699111,6cyclictest0-21swapper/308:45:553
70070699111,6cyclictest0-21swapper/307:20:543
70070699104,4cyclictest0-21swapper/308:40:553
70070699102,4cyclictest0-21swapper/307:35:463
7007069992,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional