You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 22:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed May 27, 2026 12:42:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3727673996244,15cyclictest3853476-21ssh11:01:001
3727673995226,3cyclictest0-21swapper/110:35:021
372767399513,27cyclictest3825381-21ssh10:26:011
3727672995026,21cyclictest3729171-21systemd-run07:15:010
3727674994940,6cyclictest2903711-21cups-browsed10:31:142
372767399491,18cyclictest3885720-21ssh11:41:061
372767399491,18cyclictest3885720-21ssh11:41:061
3727674994736,7cyclictest3789514-21mailstats09:41:082
3727673994721,4cyclictest0-21swapper/109:31:081
3727672994736,5cyclictest151ktimers/007:56:050
372767499462,37cyclictest3904377-21ssh12:05:342
372767399462,19cyclictest3877488-21sh11:30:571
3727672994633,7cyclictest151ktimers/008:11:000
3727672994630,13cyclictest3885233-21if_eth011:40:520
3727672994630,13cyclictest3885233-21if_eth011:40:520
3727672994629,14cyclictest2903711-21cups-browsed11:31:050
372767399452,24cyclictest3917114-21ssh12:20:591
3727673994521,4cyclictest0-21swapper/111:06:031
3727672994536,6cyclictest1055-21mta-sts-daemon10:21:020
3727674994436,5cyclictest3807817-21cron10:05:012
3727673994435,5cyclictest3931565-21cron12:40:011
372767399443,18cyclictest3765125-21ssh09:11:031
3727673994420,5cyclictest420-21avahi-daemon12:04:181
3727673994419,3cyclictest0-21swapper/110:10:591
3727673994410,14cyclictest421-21dbus-daemon11:56:031
372767299445,31cyclictest3764974-21ssh09:10:580
3727672994436,5cyclictest3730691-21systemd-run07:20:010
3727672994432,9cyclictest3819878-21/usr/sbin/munin10:20:010
3727672994424,13cyclictest0-21swapper/009:20:580
3727672994420,21cyclictest3735143-21cron07:35:010
372767299441,12cyclictest3761568-21ntpq09:01:020
3727674994335,5cyclictest439-21systemd-logind08:45:012
3727674994335,5cyclictest420-21avahi-daemon11:25:012
3727672994332,8cyclictest3736645-21munin-run07:40:010
3727674994235,5cyclictest0-21swapper/210:10:472
3727674994234,5cyclictest3864081-21cron11:15:012
3727674994234,5cyclictest3727677-21systemd-run07:10:012
3727673994234,3cyclictest0-21swapper/112:20:011
3727673994233,6cyclictest3924430-21aten_r2power_po12:30:501
372767399422,20cyclictest3797322-21ssh09:51:101
3727673994217,9cyclictest548-21gdbus11:36:071
372767299426,33cyclictest0-21swapper/009:30:050
3727672994233,6cyclictest3756163-21systemd-run08:45:010
372767299422,7cyclictest3754056-21ntpq08:36:010
372767299422,4cyclictest3733011-21ntpq07:25:580
372767299422,4cyclictest3731617-21ntpq07:21:010
372767299422,34cyclictest3801287-21ssh09:56:050
3727672994216,23cyclictest0-21swapper/012:05:580
372767299421,16cyclictest3909178-21ntpq12:10:570
3727673994135,3cyclictest0-21swapper/107:35:011
3727673994133,5cyclictest3835966-21idleruntime-cro10:40:011
3727673994132,7cyclictest3872174-21sh11:25:011
3727673994124,2cyclictest0-21swapper/108:50:011
372767399412,32cyclictest3769006-21ssh09:15:561
372767399411,31cyclictest3817213-21ssh10:15:581
372767399411,20cyclictest3865496-21ssh11:16:001
372767399411,20cyclictest3865496-21ssh11:16:001
372767299419,28cyclictest3857459-21rm11:05:580
372767299412,6cyclictest3737518-21ntpq07:40:570
372767299412,13cyclictest3869599-21ntpq11:21:030
3727672994117,21cyclictest3758171-21fschecks_time08:50:550
3727674994030,7cyclictest548-21gdbus12:26:072
3727673994032,4cyclictest3835988-21kworker/u8:9+xprtiod11:30:011
3727673994031,6cyclictest3763664-21systemd-run09:10:011
3727673994030,6cyclictest3844045-21latency_hist10:50:021
372767299402,6cyclictest3913313-21ntpq12:16:030
372767299402,6cyclictest3865601-21ntpq11:16:040
372767299402,6cyclictest3865601-21ntpq11:16:040
372767299402,6cyclictest3739002-21ntpq07:45:590
372767299402,6cyclictest3739002-21ntpq07:45:590
372767299402,5cyclictest3929220-21awk12:36:070
372767299402,10cyclictest3783834-21kworker/u8:7+xprtiod09:40:570
372767299401,9cyclictest3891593-21ssh11:49:200
3727672994019,19cyclictest3807853-21kworker/u8:0+nfsiod10:41:090
3727672994018,20cyclictest3763638-21diskmemload11:01:100
372767299401,5cyclictest3925146-21ntpq12:31:060
372767299401,35cyclictest3843131-21ssh10:48:190
372767299401,11cyclictest3777236-21ntpq09:26:040
372767299400,13cyclictest3863900-21ssh11:14:410
372767499396,23cyclictest3917098-21ssh12:20:582
372767499393,29cyclictest3889298-21ssh11:45:572
3727674993931,5cyclictest3852026-21grep11:00:002
3727674993931,5cyclictest261-21systemd-journal12:00:012
3727674993931,5cyclictest261-21systemd-journal10:40:012
3727674993930,6cyclictest2903711-21cups-browsed12:32:562
3727673993932,3cyclictest0-21swapper/109:26:051
3727673993928,8cyclictest3785018-21missed_timers09:35:551
3727673993928,8cyclictest3785018-21missed_timers09:35:551
372767299392,6cyclictest3881673-21ntpq11:36:030
372767299392,5cyclictest3747993-21ntpq08:15:560
372767299392,5cyclictest3743635-21ntpq08:01:030
372767299392,30cyclictest3740507-21ntpq07:50:580
372767299392,12cyclictest3805236-21ntpq10:01:010
372767299392,11cyclictest3825285-21ssh10:25:580
372767299391,9cyclictest3814036-21ssh10:11:280
372767299391,9cyclictest3768147-21ssh09:15:290
3727672993918,19cyclictest3727671-21cyclictest11:51:000
372767299391,14cyclictest3785152-21ssh09:35:580
372767299391,14cyclictest3785152-21ssh09:35:580
372767299391,12cyclictest3917429-21ssh12:21:090
372767299391,12cyclictest3897576-21sensors_temp11:56:080
372767299390,13cyclictest3792448-21ssh09:45:440
3727674993833,2cyclictest0-21swapper/208:00:002
3727674993831,3cyclictest0-21swapper/211:05:012
3727674993830,4cyclictest3787870-21kworker/u8:1409:40:022
3727674993830,4cyclictest3787870-21kworker/u8:1409:40:022
3727674993818,14cyclictest351ktimers/211:30:592
3727673993822,8cyclictest3763638-21diskmemload09:21:041
3727672993831,4cyclictest1051-21mta-sts-daemon12:29:000
3727672993830,5cyclictest3849642-21mailstats10:56:070
372767299382,6cyclictest3845305-21ntpq10:50:560
372767299382,6cyclictest3845305-21ntpq10:50:560
372767299382,6cyclictest3809271-21ntpq10:06:010
372767299382,6cyclictest3751006-21ntpq08:25:580
372767299382,4cyclictest3745017-21ntpq08:05:580
3727672993821,14cyclictest2903711-21cups-browsed12:00:590
372767299381,9cyclictest3833264-21ntpq10:35:580
372767299381,5cyclictest3760048-21ntpq08:56:020
372767299381,4cyclictest3829334-21ssh10:31:030
372767299381,12cyclictest3757065-21ntpq08:46:000
372767299381,11cyclictest3762989-21ntpq09:05:560
372767299381,10cyclictest3796219-21ssh09:50:240
3727674993731,3cyclictest0-21swapper/210:00:012
3727674993730,3cyclictest0-21swapper/210:30:012
3727674993726,7cyclictest420-21avahi-daemon09:28:062
3727674993725,9cyclictest3821529-21mailstats10:21:072
3727673993730,4cyclictest0-21swapper/109:56:021
3727673993730,3cyclictest0-21swapper/109:05:011
3727673993727,7cyclictest3905551-21mailstats12:06:081
3727673993727,6cyclictest3760661-21systemd-run09:00:011
372767299371,8cyclictest3749392-21/usr/sbin/munin08:20:540
372767299371,4cyclictest3873661-21ntpq11:26:050
372767499363,26cyclictest3837299-21ssh10:40:572
3727673993628,5cyclictest3893854-21unixbench-2d11:51:091
3727673993628,5cyclictest3746923-21timerwakeupswit08:11:091
3727673993626,7cyclictest3921449-21switchtime12:26:101
3727673993626,7cyclictest3811860-21sh10:10:011
372767299361,5cyclictest3752488-21ntpq08:30:570
3727674993529,3cyclictest0-21swapper/209:50:002
3727674993528,4cyclictest3882199-21unixbench_singl11:36:112
3727674993528,4cyclictest3760516-21unixbench_multi08:56:112
3727674993527,5cyclictest3911844-21systemd-run12:15:012
3727673993529,3cyclictest0-21swapper/110:55:571
3727673993529,3cyclictest0-21swapper/107:40:011
3727673993527,5cyclictest2903711-21cups-browsed09:49:231
3727673993526,6cyclictest3791845-21cron09:45:001
3727673993526,4cyclictest0-21swapper/110:20:581
3727673993525,7cyclictest3889768-21timerandwakeup11:46:091
372767499347,18cyclictest1056-21mta-sts-daemon08:00:592
3727674993426,5cyclictest3738150-21cron07:45:012
3727674993422,8cyclictest3765504-21timerandwakeup09:11:112
372767499342,25cyclictest3873625-21ssh11:26:042
3727673993427,4cyclictest3744155-21cron08:05:001
3727673993427,3cyclictest0-21swapper/110:50:491
3727673993427,3cyclictest0-21swapper/110:50:491
3727673993426,5cyclictest3909464-21sendmail_mailst12:11:061
3727673993424,8cyclictest3749586-21ntp_offset08:21:011
372767299341,4cyclictest3727628-21munin-run07:10:010
3727674993326,4cyclictest3913489-21ssh12:16:092
3727674993325,5cyclictest3893926-21grep11:51:102
3727674993325,5cyclictest3783840-21systemd-run09:35:022
3727674993325,4cyclictest3810372-21kworker/u8:1-rpciod11:44:302
3727674993325,4cyclictest3810372-21kworker/u8:1-rpciod11:44:302
3727674993324,5cyclictest3750177-21systemd-run08:25:022
3727673993325,5cyclictest3751764-21kworker/u8:1+xprtiod08:40:011
3727673993325,5cyclictest3730647-21cron07:20:011
3727673993325,4cyclictest0-21swapper/107:45:011
3727673993324,4cyclictest0-21swapper/111:11:071
3727673993323,6cyclictest0-21swapper/110:00:571
3727674993226,3cyclictest0-21swapper/207:40:012
3727674993225,3cyclictest0-21swapper/209:21:112
3727674993224,5cyclictest3931617-21systemd-run12:40:012
372767499321,27cyclictest3845439-21ssh10:51:002
372767499321,27cyclictest3845439-21ssh10:51:002
3727673993225,3cyclictest0-21swapper/110:45:021
372767499315,16cyclictest3865411-21ssh11:15:572
372767499315,16cyclictest3865411-21ssh11:15:572
3727674993125,3cyclictest0-21swapper/208:15:012
3727674993125,3cyclictest0-21swapper/207:31:132
3727674993124,5cyclictest548-21gdbus07:26:262
3727674993122,7cyclictest3860096-21sh11:10:022
3727674993122,6cyclictest0-21swapper/210:06:092
3727674993116,12cyclictest351ktimers/207:21:002
3727673993122,6cyclictest421-21dbus-daemon07:50:011
3727673993122,6cyclictest421-21dbus-daemon07:50:011
3727673993121,7cyclictest2903711-21cups-browsed08:30:011
372767499304,20cyclictest3797137-21ssh09:51:022
3727674993025,2cyclictest0-21swapper/209:10:002
3727674993020,7cyclictest3752514-21ntp_kernel_pll_08:30:592
372767499301,23cyclictest3769058-21ssh09:15:582
3727673993024,3cyclictest0-21swapper/108:00:001
3727673993023,4cyclictest1056-21mta-sts-daemon07:20:511
3727673993020,8cyclictest3744383-21/usr/sbin/munin08:05:521
372767499295,15cyclictest420-21avahi-daemon07:46:042
372767499295,15cyclictest420-21avahi-daemon07:46:042
3727674992923,3cyclictest0-21swapper/210:16:052
372767499292,21cyclictest3901257-21ssh12:00:592
3727674992921,5cyclictest2903711-21cups-browsed07:54:402
3727674992921,4cyclictest3729097-21unixbench_singl07:11:092
3727674992919,7cyclictest3841807-21timerandwakeup10:46:092
3727673992921,5cyclictest421-21dbus-daemon07:15:011
3727674992821,4cyclictest3748647-21munin-run08:20:002
3727673992822,3cyclictest0-21swapper/108:35:011
3727673992819,5cyclictest0-21swapper/108:41:091
3727673992817,8cyclictest3748637-21sh08:20:001
3727674992720,3cyclictest0-21swapper/209:05:012
3727674992617,6cyclictest548-21gdbus08:36:062
3727673992618,5cyclictest3733729-21latency_hist07:30:021
3727674992519,3cyclictest0-21swapper/208:30:012
3727674992518,4cyclictest302151450kworker/u8:25+xprtiod07:20:012
3727674992514,3cyclictest171rcu_preempt08:06:002
3727673992519,3cyclictest0-21swapper/107:55:001
3727674992419,2cyclictest0-21swapper/208:46:012
3727674992315,5cyclictest548-21gdbus08:54:462
3727673992213,6cyclictest3758971-21grep08:51:121
372767599183,8cyclictest0-21swapper/312:10:593
372767599182,9cyclictest0-21swapper/312:16:013
372767599182,8cyclictest0-21swapper/311:11:003
372767599173,7cyclictest0-21swapper/309:55:593
372767599172,9cyclictest0-21swapper/311:01:043
372767599171,9cyclictest0-21swapper/310:01:043
372767599171,7cyclictest0-21swapper/308:01:013
372767599163,9cyclictest0-21swapper/309:46:023
372767599163,7cyclictest0-21swapper/312:21:053
372767599163,7cyclictest0-21swapper/310:26:003
372767599162,8cyclictest0-21swapper/312:01:003
372767599162,8cyclictest0-21swapper/311:56:003
372767599162,8cyclictest0-21swapper/310:05:593
372767599161,7cyclictest0-21swapper/308:56:033
372767399161,12cyclictest3727684-21munin-run07:10:021
372767599153,7cyclictest0-21swapper/310:55:593
372767599153,7cyclictest0-21swapper/309:15:563
372767599153,6cyclictest0-21swapper/309:10:593
372767599152,8cyclictest0-21swapper/310:46:043
372767599152,7cyclictest0-21swapper/311:30:563
372767599152,7cyclictest0-21swapper/310:50:583
372767599152,7cyclictest0-21swapper/310:50:583
372767599151,8cyclictest0-21swapper/311:05:553
372767599151,7cyclictest0-21swapper/311:35:533
372767599143,7cyclictest0-21swapper/311:46:023
372767599143,7cyclictest0-21swapper/311:26:013
372767599143,6cyclictest0-21swapper/310:15:593
372767599142,7cyclictest0-21swapper/312:26:023
372767599142,7cyclictest0-21swapper/312:05:583
372767599142,7cyclictest0-21swapper/310:31:023
372767599142,6cyclictest0-21swapper/309:06:093
372767599141,8cyclictest0-21swapper/309:25:593
372767599141,7cyclictest0-21swapper/310:35:583
372767599141,6cyclictest0-21swapper/307:46:033
372767599141,6cyclictest0-21swapper/307:46:033
372767599134,6cyclictest0-21swapper/310:41:003
372767599133,5cyclictest0-21swapper/309:01:003
372767599132,7cyclictest0-21swapper/309:51:023
372767599132,6cyclictest0-21swapper/312:40:033
372767599132,6cyclictest0-21swapper/311:15:473
372767599132,6cyclictest0-21swapper/311:15:473
372767599131,7cyclictest0-21swapper/311:50:583
372767599131,7cyclictest0-21swapper/311:40:573
372767599131,7cyclictest0-21swapper/311:40:573
372767599131,7cyclictest0-21swapper/311:21:103
372767599131,7cyclictest0-21swapper/310:10:553
372767599131,7cyclictest0-21swapper/309:40:593
372767599131,7cyclictest0-21swapper/309:35:573
372767599131,7cyclictest0-21swapper/309:35:573
372767599122,7cyclictest0-21swapper/310:21:023
372767599122,5cyclictest0-21swapper/308:40:513
372767599121,7cyclictest0-21swapper/312:30:513
372767599121,7cyclictest0-21swapper/309:30:583
372767599121,6cyclictest0-21swapper/308:35:593
372767599121,6cyclictest0-21swapper/308:10:593
372767599113,6cyclictest0-21swapper/308:21:023
372767599112,6cyclictest0-21swapper/308:46:103
372767599112,6cyclictest0-21swapper/308:31:093
372767599111,7cyclictest0-21swapper/309:20:533
372767599111,7cyclictest0-21swapper/308:51:053
372767599111,6cyclictest0-21swapper/307:40:013
372767599111,6cyclictest0-21swapper/307:20:583
372767599111,5cyclictest0-21swapper/308:25:583
372767599111,5cyclictest0-21swapper/307:10:493
372767599104,3cyclictest0-21swapper/308:06:043
372767599101,6cyclictest0-21swapper/307:55:583
372767599101,6cyclictest0-21swapper/307:31:063
372767599101,6cyclictest0-21swapper/307:26:003
372767599101,5cyclictest0-21swapper/308:15:473
372767599101,5cyclictest0-21swapper/307:55:023
372767599101,5cyclictest0-21swapper/307:40:583
372767599101,5cyclictest0-21swapper/307:15:563
37276759971,3cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional