You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 23:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Aug 27, 2026 12:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3207733995342,6cyclictest0-21swapper/212:31:102
3207733995242,5cyclictest3198046-21kworker/u8:6-rpciod08:13:462
3207733995242,5cyclictest3198046-21kworker/u8:6-rpciod08:13:462
3207733995241,7cyclictest3357732-21/usr/sbin/munin12:16:092
3207733995240,8cyclictest2389223-21cups-browsed08:26:082
3207732995225,23cyclictest3332110-21ssh11:35:011
3207731995138,5cyclictest151ktimers/008:26:060
320773299495,36cyclictest3318070-21ssh11:11:011
3207731994935,6cyclictest3350517-21latency_hist12:05:020
3207733994739,4cyclictest0-21swapper/211:45:012
3207732994731,12cyclictest3346079-21grep11:56:211
3207733994635,7cyclictest420-21avahi-daemon10:23:442
3207732994623,20cyclictest3274383-21latency_hist10:00:021
320773299461,18cyclictest3351575-21rm12:06:071
3207733994533,9cyclictest3303173-21mailstats10:46:122
3207733994433,6cyclictest0-21swapper/211:06:032
320773299442,15cyclictest3302869-21ntpq10:46:021
320773299441,14cyclictest3227194-40mandb08:12:571
320773299441,14cyclictest3227194-40mandb08:12:571
320773299441,12cyclictest3214688-21ntpq07:31:051
3207731994432,9cyclictest3341257-21/usr/sbin/munin11:50:010
3207731994429,11cyclictest3227194-40mandb08:13:180
3207731994429,11cyclictest3227194-40mandb08:13:180
320773199441,18cyclictest3222135-21ntpq07:56:040
320773199441,13cyclictest3214569-21ntpq07:30:590
3207733994333,6cyclictest3370094-21unixbench-2d12:36:132
3207733994333,6cyclictest2389223-21cups-browsed09:01:142
320773299439,11cyclictest3342233-21sed11:51:011
320773299433,5cyclictest3223640-21ntpq08:01:031
3207731994325,16cyclictest421-21dbus-daemon09:30:020
3207731994323,17cyclictest0-21swapper/012:15:550
320773199431,16cyclictest3234201-21ntpq08:36:080
3207731994311,2cyclictest0-21swapper/008:20:540
3207731994311,2cyclictest0-21swapper/008:20:540
320773199431,11cyclictest3237138-21ntpq08:46:040
3207733994233,6cyclictest3308573-21cut10:55:552
3207732994234,5cyclictest3230263-21systemd-run08:25:011
3207732994234,5cyclictest3230263-21systemd-run08:25:011
320773299422,6cyclictest3237088-21ntpq08:46:011
320773299422,6cyclictest3217692-21ntpq07:41:041
3207732994220,17cyclictest0-21swapper/110:00:571
3207732994218,20cyclictest0-21swapper/109:35:531
320773299421,12cyclictest3368175-21ssh12:34:211
320773199421,13cyclictest3241220-21df09:00:530
3207731994211,2cyclictest0-21swapper/007:26:060
3207733994131,6cyclictest3279135-21grep10:06:152
320773299412,14cyclictest3222071-21ntpq07:56:011
320773299411,9cyclictest3234128-21ntpq08:36:041
320773299411,9cyclictest3216201-21ntpq07:36:071
320773299411,15cyclictest3363589-21ntpq12:26:021
320773299411,12cyclictest3271128-21ssh09:54:421
320773299411,11cyclictest3211649-21ntpq07:21:031
320773299411,10cyclictest3354627-21ntp_states12:11:051
320773199414,9cyclictest3299939-21ssh10:41:040
3207731994114,23cyclictest0-21swapper/012:21:040
320773199411,14cyclictest3366158-21diskstats12:30:560
320773199411,12cyclictest3273361-21ssh09:57:070
3207733994032,3cyclictest0-21swapper/212:25:012
3207733994031,5cyclictest0-21swapper/210:41:012
3207733994030,6cyclictest3262124-21awk09:40:002
3207733994030,5cyclictest1002-21mta-sts-daemon11:01:402
320773299404,8cyclictest3299805-21ntpq10:40:581
320773299402,14cyclictest3266393-21ntpq09:46:041
320773299401,9cyclictest3240690-21sshd08:57:491
320773299401,9cyclictest3219103-21ntpq07:46:021
3207732994015,22cyclictest0-21swapper/110:36:071
320773299401,12cyclictest3287747-21ntpq10:21:051
3207732994011,26cyclictest3208111-21aten_r2power_vo07:10:521
320773199405,29cyclictest3257357-21grep09:31:090
320773199403,8cyclictest3220577-21ntpq07:51:010
3207731994028,8cyclictest3292598-21grep10:30:010
320773199402,7cyclictest3354476-21ntpq12:10:590
3207731994018,18cyclictest421-21dbus-daemon09:45:110
320773199401,13cyclictest3333330-21ntpq11:36:030
320773199401,11cyclictest3228181-21ntpq08:16:070
320773199401,11cyclictest3228181-21ntpq08:16:070
3207733993933,4cyclictest0-21swapper/212:00:012
3207733993928,5cyclictest0-21swapper/209:16:052
3207733993924,9cyclictest0-21swapper/210:16:032
3207733993924,9cyclictest0-21swapper/210:16:032
320773299395,27cyclictest3284720-21ssh10:16:051
320773299395,27cyclictest3284720-21ssh10:16:051
320773299394,7cyclictest3244897-21ntpq09:11:021
320773299392,7cyclictest3360679-21ntpq12:21:081
320773299392,7cyclictest3315120-21ntpq11:06:041
320773299392,7cyclictest3220613-21ntpq07:51:021
3207732993924,12cyclictest0-21swapper/112:20:021
3207732993920,16cyclictest0-21swapper/110:30:561
320773299391,9cyclictest3308916-21ntpq10:56:001
320773299391,12cyclictest3249320-21ssh09:18:251
320773299391,10cyclictest3228115-21ntpq08:16:041
320773299391,10cyclictest3228115-21ntpq08:16:041
3207731993933,4cyclictest3353348-21systemd-run12:10:010
3207731993930,6cyclictest3283446-21munin-plugin-st10:15:010
320773199392,8cyclictest3217700-21ntpq07:41:050
320773199392,6cyclictest421-21dbus-daemon11:58:450
3207731993920,16cyclictest1003-21mta-sts-daemon09:10:560
320773199391,7cyclictest3363553-21ntpq12:26:010
320773199391,18cyclictest3306166-21mailstats10:51:110
320773199391,15cyclictest3263343-21ntpq09:41:030
320773199391,14cyclictest3336393-21ntpq11:41:030
320773199391,11cyclictest3225199-21ntpq08:06:050
320773199391,11cyclictest3225199-21ntpq08:06:050
320773399383,31cyclictest3363531-21ssh12:26:002
3207733993829,5cyclictest0-21swapper/210:36:022
320773299388,6cyclictest3237947-21sshd08:50:481
320773299383,9cyclictest420-21avahi-daemon10:25:451
320773299383,8cyclictest3327295-21ntpq11:26:041
3207732993829,5cyclictest0-21swapper/111:16:051
320773299382,7cyclictest3241618-21ntpq09:01:021
3207732993824,11cyclictest3236235-21munin-plugin-st08:45:011
320773299381,9cyclictest3232675-21ntpq08:31:041
3207732993819,17cyclictest265-21systemd-journal09:31:161
320773299381,15cyclictest3324196-21ntpq11:21:021
320773299381,14cyclictest3254255-21ntpq09:26:091
320773299381,11cyclictest3264473-21ssh09:42:511
320773299381,10cyclictest3283281-21ssh10:14:321
320773299381,10cyclictest3279381-21modprobe10:06:491
320773299380,11cyclictest3370389-21ssh12:36:471
3207731993832,4cyclictest3210736-21sendmail07:20:010
3207731993831,4cyclictest3329693-21aten_r2power_fr11:30:520
320773199382,13cyclictest3269516-21ntpq09:51:080
3207731993817,18cyclictest0-21swapper/012:36:020
320773199381,14cyclictest3240195-21ntpq08:56:070
320773199381,12cyclictest3313405-21ssh11:03:330
320773199381,12cyclictest3296917-21ntpq10:36:090
320773199381,12cyclictest3216120-21ntpq07:36:030
320773399373,24cyclictest3275546-21ssh10:01:042
3207733993728,6cyclictest3223265-21fschecks_time08:00:562
3207733993728,5cyclictest2389223-21cups-browsed10:11:032
3207733993726,8cyclictest3219613-21unixbench_multi07:46:132
320773299372,5cyclictest3213096-21ntpq07:26:021
3207732993718,15cyclictest0-21swapper/112:01:091
3207732993712,22cyclictest2389223-21cups-browsed11:46:171
320773299371,11cyclictest3225180-21ntpq08:06:041
320773299371,11cyclictest3225180-21ntpq08:06:041
320773299371,10cyclictest3334810-21ssh11:38:541
320773299371,10cyclictest3210668-21users07:16:141
320773199377,4cyclictest3262290-21latency_hist09:40:020
320773199373,7cyclictest3278615-21ntpq10:06:060
320773199373,6cyclictest3208568-21ntpq07:10:590
320773199372,15cyclictest3321206-21ntpq11:16:050
320773199372,12cyclictest3324328-21ntpq11:21:070
3207731993717,18cyclictest0-21swapper/010:55:520
3207731993715,19cyclictest0-21swapper/010:15:520
3207731993715,19cyclictest0-21swapper/010:15:520
320773199371,12cyclictest3328401-21ssh11:27:490
320773199371,12cyclictest3219186-21ntpq07:46:060
320773199371,11cyclictest3243084-21ntpq09:06:010
320773199370,12cyclictest3303897-21ssh10:47:230
320773399363,23cyclictest3339496-21ssh11:46:092
3207733993628,3cyclictest0-21swapper/209:11:062
320773399362,27cyclictest3254090-21ssh09:26:022
320773299361,9cyclictest3312707-21ssh11:01:201
3207732993614,19cyclictest0-21swapper/109:06:031
320773299361,10cyclictest3243724-21diskmemload10:51:421
320773299361,10cyclictest3231160-21ntpq08:26:041
320773299360,10cyclictest3337521-21ssh11:42:481
320773299360,10cyclictest3251267-21ntpq09:21:371
320773199369,5cyclictest3243724-21diskmemload10:03:160
320773199364,10cyclictest428-21mta-sts-daemon09:20:420
320773199363,8cyclictest3232686-21ntpq08:31:050
320773199363,12cyclictest3293090-21tail10:30:520
320773199363,10cyclictest3318070-21ssh11:11:000
320773199361,15cyclictest3247546-21df09:15:540
3207731993611,2cyclictest0-21swapper/011:05:500
320773199361,10cyclictest3235623-21ntpq08:41:060
320773199361,10cyclictest3211703-21ntpq07:21:060
3207733993527,5cyclictest3321373-21ssh11:16:092
3207731993519,13cyclictest1000-21mta-sts-daemon10:22:050
320773199351,10cyclictest3344011-21ssh11:54:130
3207733993428,3cyclictest0-21swapper/207:40:012
3207733993426,5cyclictest3207730-21cyclictest09:45:542
3207733993426,3cyclictest0-21swapper/212:06:072
3207733993425,5cyclictest3271284-21latency_hist09:55:022
3207733993423,8cyclictest3240060-21ntp_kernel_err08:56:012
320773199348,7cyclictest3238236-21grep08:50:550
320773199341,11cyclictest3224160-21unixbench_singl08:01:130
3207733993326,4cyclictest1001-21mta-sts-daemon08:51:022
3207733993326,4cyclictest0-21swapper/209:10:012
3207733993326,3cyclictest0-21swapper/208:00:012
3207733993324,6cyclictest1001-21mta-sts-daemon09:21:192
320773399332,25cyclictest2389223-21cups-browsed10:26:072
320773399324,20cyclictest3293885-21ssh10:31:142
3207733993225,4cyclictest3354315-21latency12:10:562
3207733993225,4cyclictest3234722-21/usr/sbin/munin08:40:022
3207733993222,5cyclictest1003-21mta-sts-daemon12:02:092
3207733993221,6cyclictest1000-21mta-sts-daemon10:53:392
3207733993125,3cyclictest0-21swapper/209:35:022
3207733993124,4cyclictest0-21swapper/211:13:482
3207733993123,5cyclictest3221168-21unixbench_singl07:51:132
320773399312,20cyclictest3324179-21ssh11:21:022
3207733993117,6cyclictest0-21swapper/211:26:032
320773399304,16cyclictest428-21mta-sts-daemon08:21:032
320773399304,16cyclictest428-21mta-sts-daemon08:21:032
320773399295,13cyclictest3263382-21ssh09:41:052
3207733992921,4cyclictest0-21swapper/208:31:052
3207733992920,6cyclictest2389223-21cups-browsed09:57:282
3207733992920,5cyclictest3209207-21cron07:15:012
3207733992913,11cyclictest2389223-21cups-browsed11:36:222
3207732992910,16cyclictest0-21swapper/107:10:021
320773199291,13cyclictest3207773-21latency_hist07:10:020
3207733992820,5cyclictest3341887-21aten_r2power_po11:50:542
3207733992820,5cyclictest1003-21mta-sts-daemon11:30:172
3207733992721,3cyclictest0-21swapper/208:45:012
3207733992719,5cyclictest3218221-21cron07:45:012
3207733992621,4cyclictest3207730-21cyclictest08:50:012
3207733992620,3cyclictest1003-21mta-sts-daemon08:07:372
3207733992620,3cyclictest1003-21mta-sts-daemon08:07:372
3207733992618,4cyclictest0-21swapper/207:16:052
3207733992616,8cyclictest3214623-21ntp_kernel_pll_07:31:022
3207733992519,3cyclictest0-21swapper/207:25:562
3207733992417,3cyclictest0-21swapper/207:10:012
320773399232,15cyclictest3211621-21ntpq07:21:022
3207733992317,3cyclictest0-21swapper/208:20:002
3207733992317,3cyclictest0-21swapper/208:20:002
320773499171,9cyclictest0-21swapper/310:36:153
320773499162,7cyclictest0-21swapper/310:01:063
320773499162,6cyclictest0-21swapper/307:26:043
320773499161,8cyclictest0-21swapper/312:01:123
320773499161,6cyclictest0-21swapper/308:21:073
320773499161,6cyclictest0-21swapper/308:21:073
320773499154,5cyclictest0-21swapper/311:36:063
320773499153,8cyclictest0-21swapper/311:51:003
320773499153,8cyclictest0-21swapper/310:26:073
320773499153,7cyclictest0-21swapper/308:13:353
320773499153,7cyclictest0-21swapper/308:13:353
320773499153,6cyclictest0-21swapper/309:46:153
320773499152,7cyclictest0-21swapper/312:06:083
320773499152,7cyclictest0-21swapper/309:36:023
320773499152,6cyclictest0-21swapper/307:41:053
320773499151,8cyclictest0-21swapper/312:21:103
320773499151,7cyclictest0-21swapper/312:26:013
320773499151,6cyclictest0-21swapper/307:56:053
320773499151,6cyclictest0-21swapper/307:16:003
320773499151,5cyclictest0-21swapper/307:46:023
320773499150,7cyclictest0-21swapper/309:26:013
320773499143,5cyclictest0-21swapper/312:35:563
320773499142,8cyclictest0-21swapper/311:26:003
320773499142,8cyclictest0-21swapper/309:31:073
320773499142,6cyclictest0-21swapper/309:56:063
320773499142,6cyclictest0-21swapper/308:46:063
320773499141,8cyclictest0-21swapper/312:11:413
320773499141,7cyclictest0-21swapper/311:16:073
320773499141,7cyclictest0-21swapper/310:06:033
320773499141,7cyclictest0-21swapper/308:51:003
320773499141,6cyclictest0-21swapper/311:31:023
320773499141,6cyclictest0-21swapper/310:55:593
320773499141,6cyclictest0-21swapper/308:06:063
320773499141,6cyclictest0-21swapper/308:06:063
320773499141,6cyclictest0-21swapper/307:31:043
320773499134,4cyclictest0-21swapper/311:21:023
320773499133,5cyclictest0-21swapper/311:56:063
320773499133,5cyclictest0-21swapper/310:46:023
320773499133,5cyclictest0-21swapper/310:20:563
320773499133,4cyclictest0-21swapper/308:56:013
320773499132,7cyclictest0-21swapper/310:15:523
320773499132,7cyclictest0-21swapper/310:15:523
320773499132,7cyclictest0-21swapper/309:10:573
320773499132,6cyclictest0-21swapper/312:31:053
320773499132,6cyclictest0-21swapper/310:30:523
320773499132,6cyclictest0-21swapper/308:20:013
320773499132,6cyclictest0-21swapper/308:20:013
320773499131,8cyclictest0-21swapper/310:10:543
320773499131,7cyclictest0-21swapper/310:51:043
320773499131,7cyclictest0-21swapper/310:40:543
320773499131,7cyclictest0-21swapper/309:40:563
320773499131,6cyclictest0-21swapper/311:40:563
320773499131,6cyclictest0-21swapper/309:01:013
320773499131,5cyclictest0-21swapper/311:05:033
320773499131,5cyclictest0-21swapper/309:10:023
320773499131,5cyclictest0-21swapper/307:11:023
320773499123,5cyclictest0-21swapper/312:16:013
320773499122,6cyclictest0-21swapper/311:46:063
320773499122,6cyclictest0-21swapper/311:06:003
320773499122,6cyclictest0-21swapper/309:51:033
320773499121,7cyclictest0-21swapper/311:10:553
320773499121,7cyclictest0-21swapper/308:25:553
320773499111,6cyclictest0-21swapper/308:01:083
320773499101,7cyclictest0-21swapper/309:16:103
320773499101,6cyclictest0-21swapper/308:40:023
320773499101,5cyclictest0-21swapper/309:20:493
320773499101,5cyclictest0-21swapper/307:50:593
32077349992,5cyclictest0-21swapper/308:41:003
32077349991,5cyclictest0-21swapper/307:35:523
32077349991,5cyclictest0-21swapper/307:20:553
32077349991,4cyclictest0-21swapper/307:10:033
32077349990,6cyclictest0-21swapper/308:30:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional