You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-03 - 22:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Sep 03, 2026 12:42:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
117218997257,10cyclictest188291-21mailstats10:06:160
117218997257,10cyclictest188291-21mailstats10:06:160
117219996352,5cyclictest202049-21systemd10:30:021
117220995242,6cyclictest1-21systemd10:40:022
117220995242,6cyclictest1-21systemd10:40:022
11721899511,45cyclictest14-21ksoftirqd/012:26:100
117219995041,5cyclictest1003-21mta-sts-daemon12:36:131
117219995038,9cyclictest153202-21diskmemload10:41:161
117219995038,9cyclictest153202-21diskmemload10:41:161
117220994939,5cyclictest123231-21systemd-run07:30:012
117220994822,9cyclictest171rcu_preempt09:51:102
117218994827,13cyclictest12650irq/32-eth009:51:080
117219994736,7cyclictest3527540-21cups-browsed12:06:081
11721999472,37cyclictest267048-21ssh12:16:151
11721999472,37cyclictest267048-21ssh12:16:151
117218994738,6cyclictest208728-21aten_r2power_po10:40:560
117218994738,6cyclictest208728-21aten_r2power_po10:40:560
117218994538,4cyclictest156555-21sshd09:15:160
117218994513,10cyclictest0-21swapper/011:31:140
117220994435,6cyclictest220201-21sendmail11:00:012
117220994435,4cyclictest1001-21mta-sts-daemon10:15:012
117220994435,4cyclictest1001-21mta-sts-daemon10:15:012
117219994435,6cyclictest183783-21systemd-run10:00:011
117218994423,14cyclictest12650irq/32-eth011:06:040
117220994334,5cyclictest151726-21kworker/u8:12+xprtiod09:05:012
117219994334,5cyclictest126196-21ls07:40:011
11721999432,34cyclictest175960-21ssh09:46:111
117219994234,5cyclictest1000-21mta-sts-daemon11:18:221
117219994233,6cyclictest215072-21kernelversion10:51:011
11721899421,33cyclictest258043-21ssh12:01:130
117220994131,6cyclictest187948-21needreboot10:06:042
117220994131,6cyclictest187948-21needreboot10:06:042
117219994130,6cyclictest420-21avahi-daemon09:34:231
117219994127,11cyclictest240108-21timerandwakeup11:31:251
117218994132,6cyclictest421-21dbus-daemon11:30:020
117220994032,5cyclictest242418-21ssh11:36:022
117220994031,4cyclictest0-21swapper/210:21:122
11722099401,10cyclictest0-21swapper/209:26:072
117219994032,5cyclictest421-21dbus-daemon11:45:011
11721899404,29cyclictest265-21systemd-journal10:11:160
11721899404,29cyclictest265-21systemd-journal10:11:160
11721899403,30cyclictest215334-21ssh10:51:050
117218994032,5cyclictest153202-21diskmemload11:40:010
117220993931,5cyclictest148710-21systemd-run08:55:012
117220993931,5cyclictest117230-21systemd-run07:10:012
11722099392,31cyclictest194185-21ssh10:16:092
117219993921,6cyclictest171rcu_preempt08:16:121
117218993928,8cyclictest279540-21unixbench_singl12:36:210
11722099384,27cyclictest239732-21ssh11:31:142
11722099382,31cyclictest665-21snmpd08:46:042
117219993831,4cyclictest115554-21kworker/u8:3-nfsiod07:35:011
117219993830,4cyclictest263312-21aten_r2power_en12:10:591
117219993829,6cyclictest150177-21cron09:00:001
117219993826,8cyclictest242982-21sensors_temp11:36:161
11721999382,26cyclictest567-21gdbus10:01:081
11721999382,26cyclictest567-21gdbus10:01:081
117218993829,6cyclictest420-21avahi-daemon11:04:230
117220993732,2cyclictest0-21swapper/209:00:012
117220993730,4cyclictest226308-21idleruntime-cro11:10:012
117220993729,5cyclictest171630-21latency_hist09:40:022
117220993728,7cyclictest195968-21kworker/u8:8-xprtiod11:49:162
117220993728,5cyclictest157859-21timerandwakeup09:16:172
11722099372,25cyclictest264002-21ssh12:11:152
11722099370,31cyclictest227524-21ssh11:11:062
11722099370,30cyclictest175877-21ssh09:46:072
117219993729,5cyclictest1001-21mta-sts-daemon08:41:091
117219993728,6cyclictest148711-21true08:55:011
117219993727,6cyclictest420-21avahi-daemon10:37:531
117219993727,6cyclictest420-21avahi-daemon10:37:531
117219993724,9cyclictest193964-21missed_timers10:16:031
117218993730,5cyclictest140953-21timerandwakeup08:26:160
117218993721,11cyclictest151ktimers/009:21:100
117218993719,13cyclictest12650irq/32-eth012:06:100
117220993628,5cyclictest205051-21cron10:35:012
117220993627,6cyclictest183735-21ssh10:00:012
117220993627,5cyclictest1002-21mta-sts-daemon10:00:392
117220993627,5cyclictest1002-21mta-sts-daemon10:00:392
117220993626,7cyclictest694-21cron12:05:012
117219993630,3cyclictest0-21swapper/111:25:001
117219993630,3cyclictest0-21swapper/109:20:011
117219993628,5cyclictest162383-21ssh09:25:011
117219993628,4cyclictest0-21swapper/111:00:021
117219993626,5cyclictest0-21swapper/111:56:101
117218993627,5cyclictest151ktimers/007:56:070
117218993627,5cyclictest151ktimers/007:56:070
117218993626,7cyclictest0-21swapper/011:50:010
117218993625,8cyclictest139031-21needreboot08:21:040
11721899362,24cyclictest212382-21ssh10:46:090
11721899362,24cyclictest212382-21ssh10:46:090
117220993528,4cyclictest0-21swapper/208:40:012
117220993527,5cyclictest232413-21systemd-run11:20:012
117220993527,5cyclictest1-21systemd12:20:012
117220993527,5cyclictest1-21systemd12:20:012
117220993526,7cyclictest279001-21sh12:36:122
117220993524,8cyclictest134525-21needreboot08:06:032
117219993529,4cyclictest164168-21unixbench_multi09:26:181
117219993529,3cyclictest0-21swapper/109:36:161
117219993524,7cyclictest3527540-21cups-browsed12:21:131
117219993524,7cyclictest3527540-21cups-browsed12:21:131
117219993515,3cyclictest171rcu_preempt07:56:041
117219993515,3cyclictest171rcu_preempt07:56:041
11721899353,28cyclictest206402-21sshd10:36:160
11721899353,28cyclictest206402-21sshd10:36:160
117218993518,8cyclictest12650irq/32-eth009:46:040
117218993516,13cyclictest12650irq/32-eth007:16:080
117220993427,3cyclictest0-21swapper/208:05:012
117220993427,3cyclictest0-21swapper/208:05:012
117220993425,6cyclictest1-21systemd08:00:012
117220993425,6cyclictest1-21systemd08:00:012
117220993423,8cyclictest153214-21idleruntime-cro09:10:012
11722099342,24cyclictest245692-21ssh11:41:042
11722099342,23cyclictest209404-21ssh10:41:112
11722099342,23cyclictest209404-21ssh10:41:112
117219993427,3cyclictest0-21swapper/111:06:161
117219993427,3cyclictest0-21swapper/109:10:011
117219993424,8cyclictest275930-21ntp_kernel_pll_12:31:111
117219993414,11cyclictest251815-21ssh11:51:091
117218993429,3cyclictest183763-21sendmail10:00:010
117218993429,3cyclictest1000-21mta-sts-daemon10:35:000
117218993426,5cyclictest421-21dbus-daemon07:40:010
117218993425,6cyclictest0-21swapper/011:40:540
117218993424,7cyclictest232357-21ssh11:20:000
117218993420,7cyclictest0-21swapper/010:25:580
117218993420,11cyclictest123737-21fschecks_time07:30:590
117220993327,3cyclictest0-21swapper/212:30:002
117220993326,4cyclictest271729-21munin-plugin-st12:25:002
117220993326,4cyclictest271729-21munin-plugin-st12:25:002
117220993325,4cyclictest205197-21kworker/u8:0-xprtiod12:05:572
117220993325,4cyclictest168546-21kworker/u8:7-xprtiod10:50:012
117220993325,4cyclictest168546-21kworker/u8:7-xprtiod10:50:012
117220993324,5cyclictest223262-21cron11:05:012
11722099331,28cyclictest200287-21ssh10:26:092
11721999335,18cyclictest237297-21sh11:26:201
11721999334,21cyclictest172877-21ssh09:41:101
117219993326,4cyclictest151705-21cron09:05:001
117219993326,4cyclictest1001-21mta-sts-daemon10:05:301
117219993326,4cyclictest1001-21mta-sts-daemon10:05:301
117219993324,6cyclictest3527540-21cups-browsed08:06:131
11721999331,15cyclictest28-21ksoftirqd/108:01:141
11721999331,15cyclictest28-21ksoftirqd/108:01:141
117218993327,3cyclictest0-21swapper/009:00:010
117218993325,6cyclictest184408-21aten_r2power_po10:00:550
117218993325,6cyclictest184408-21aten_r2power_po10:00:550
117218993325,5cyclictest217990-21fschecks_time10:55:580
117218993325,4cyclictest0-21swapper/012:16:120
117218993325,4cyclictest0-21swapper/012:16:120
117218993323,7cyclictest233521-21needreboot11:21:020
117218993321,9cyclictest127574-21unixbench-2d07:41:170
117218993316,7cyclictest12650irq/32-eth010:16:090
117220993225,4cyclictest0-21swapper/209:25:012
117220993225,4cyclictest0-21swapper/208:25:012
117220993222,7cyclictest233459-21missed_timers11:21:012
117219993224,5cyclictest3527540-21cups-browsed09:53:591
117219993222,7cyclictest127424-21threads07:41:161
11721999322,22cyclictest212314-21ssh10:46:071
11721999322,22cyclictest212314-21ssh10:46:071
117219993219,2cyclictest171rcu_preempt08:21:081
117218993223,6cyclictest420-21avahi-daemon12:21:570
117218993223,6cyclictest420-21avahi-daemon12:21:570
117220993124,3cyclictest0-21swapper/211:25:592
117220993123,5cyclictest127715-21munin-plugin-st07:45:012
117220993122,6cyclictest174649-21systemd-run09:45:012
117220993116,11cyclictest1000-21mta-sts-daemon08:16:112
11721999312,22cyclictest221467-21ssh11:01:091
117219993120,8cyclictest226848-21sh11:10:541
117219993119,7cyclictest257988-21sh12:01:101
11721899317,17cyclictest151ktimers/011:56:090
117218993126,3cyclictest1000-21mta-sts-daemon09:06:120
117218993125,4cyclictest265709-21ssh12:15:020
117218993125,3cyclictest0-21swapper/011:51:000
117218993124,4cyclictest151ktimers/008:01:130
117218993124,4cyclictest151ktimers/008:01:130
117218993123,6cyclictest173030-21sendmail_mailqu09:41:150
117218993122,6cyclictest3626617-21munin-node09:10:540
11721899312,24cyclictest197162-21ssh10:21:060
117218993116,7cyclictest276101-21ssh12:31:170
11721899311,24cyclictest227497-21sh11:11:060
117220993022,6cyclictest123748-21gltestperf07:30:592
117220993021,6cyclictest1002-21mta-sts-daemon09:11:042
117220993011,12cyclictest3527537-21cupsd08:11:102
117219993025,2cyclictest0-21swapper/109:10:591
117219993021,6cyclictest249374-21unixbench_singl11:46:161
117219993021,6cyclictest142697-21latency_hist08:35:031
117219993021,4cyclictest0-21swapper/112:25:081
117219993015,2cyclictest171rcu_preempt07:16:081
117219993015,10cyclictest421-21dbus-daemon10:10:221
117219993015,10cyclictest421-21dbus-daemon10:10:221
117218993013,5cyclictest169952-21sendmail_mailst09:36:130
117220992924,2cyclictest0-21swapper/207:20:012
117220992922,3cyclictest0-21swapper/212:31:222
117219992921,5cyclictest123160-21unixbench_singl07:26:211
117219992921,5cyclictest121613-21unixbench_multi07:21:161
117219992919,6cyclictest117279-21latency_hist07:10:021
117219992918,8cyclictest129094-21unixbench_multi07:46:181
117219992918,8cyclictest129094-21unixbench_multi07:46:181
117219992917,6cyclictest171rcu_preempt08:11:071
117219992915,3cyclictest171rcu_preempt10:21:051
11721899293,20cyclictest163600-21ssh09:26:060
117218992922,3cyclictest0-21swapper/009:31:090
117220992821,4cyclictest0-21swapper/212:00:012
117220992821,4cyclictest0-21swapper/210:50:552
117220992820,5cyclictest129199-21cron07:50:002
117220992820,5cyclictest129199-21cron07:50:002
117220992819,5cyclictest251828-21ntp_kernel_pll_11:51:102
11722099281,21cyclictest166821-21sshd09:31:162
11721999286,14cyclictest3527540-21cups-browsed07:11:111
117219992816,3cyclictest171rcu_preempt10:31:091
117218992814,11cyclictest151ktimers/007:11:080
117220992719,5cyclictest3527540-21cups-browsed07:54:382
11721999276,3cyclictest171rcu_preempt08:36:071
11721999272,17cyclictest265-21systemd-journal08:26:101
117218992721,4cyclictest1000-21mta-sts-daemon08:45:580
117218992720,5cyclictest0-21swapper/008:20:010
117218992720,4cyclictest150741-21fschecks_time09:00:590
117218992720,4cyclictest135727-21fschecks_time08:10:570
117218992718,6cyclictest420-21avahi-daemon08:40:540
117220992618,5cyclictest118228-21proc_pri07:11:132
11721999266,4cyclictest261rcuc/108:50:011
117218992621,3cyclictest0-21swapper/008:55:010
117218992620,2cyclictest0-21swapper/008:10:010
11722099253,17cyclictest420-21avahi-daemon07:36:082
117220992520,3cyclictest1002-21mta-sts-daemon08:45:012
117219992519,3cyclictest0-21swapper/107:55:001
117218992514,8cyclictest123214-21munin-plugin-st07:30:010
117220992416,5cyclictest121602-21unixbench_multi07:21:162
11721899242,17cyclictest128658-21ntpq07:46:100
11721899242,17cyclictest128658-21ntpq07:46:100
117218992416,6cyclictest130689-21cron07:55:000
117218992415,6cyclictest0-21swapper/007:21:160
117218992411,10cyclictest0-21swapper/008:36:190
117218992317,4cyclictest265-21systemd-journal08:34:160
11722099222,15cyclictest142132-21ntpq08:31:072
117220992114,4cyclictest140575-21ntp_kernel_err08:26:052
11722199173,6cyclictest0-21swapper/309:36:083
11722199172,6cyclictest0-21swapper/310:46:073
11722199172,6cyclictest0-21swapper/310:46:073
11722199171,8cyclictest0-21swapper/312:21:103
11722199171,8cyclictest0-21swapper/312:21:103
117218991712,3cyclictest107512-21kworker/u8:9-xprtiod07:10:030
11722199163,9cyclictest0-21swapper/309:46:053
11722199163,7cyclictest0-21swapper/310:41:163
11722199163,7cyclictest0-21swapper/310:41:163
11722199162,9cyclictest0-21swapper/312:31:163
11722199162,6cyclictest0-21swapper/308:01:063
11722199162,6cyclictest0-21swapper/308:01:063
11722199161,7cyclictest0-21swapper/308:56:093
11722199152,8cyclictest0-21swapper/312:11:173
11722199151,8cyclictest0-21swapper/311:31:143
11722199151,7cyclictest0-21swapper/309:06:063
11722199151,7cyclictest0-21swapper/307:51:133
11722199151,6cyclictest0-21swapper/307:26:093
11722199143,6cyclictest0-21swapper/310:11:113
11722199143,6cyclictest0-21swapper/310:11:113
11722199142,8cyclictest0-21swapper/312:36:053
11722199142,7cyclictest0-21swapper/312:20:003
11722199142,7cyclictest0-21swapper/312:20:003
11722199142,7cyclictest0-21swapper/312:01:093
11722199142,7cyclictest0-21swapper/311:11:073
11722199142,6cyclictest0-21swapper/311:01:103
11722199141,9cyclictest0-21swapper/309:40:573
11722199141,7cyclictest0-21swapper/309:26:083
11722199141,6cyclictest0-21swapper/310:31:093
11722199141,6cyclictest0-21swapper/308:36:073
11722199141,6cyclictest0-21swapper/307:36:063
11722199141,6cyclictest0-21swapper/307:31:053
11722199141,5cyclictest0-21swapper/308:46:033
11722199134,6cyclictest0-21swapper/311:15:533
11722199133,7cyclictest0-21swapper/310:16:073
11722199132,7cyclictest0-21swapper/310:06:113
11722199132,7cyclictest0-21swapper/310:06:113
11722199132,6cyclictest0-21swapper/310:36:093
11722199132,6cyclictest0-21swapper/310:36:093
11722199132,5cyclictest0-21swapper/309:55:033
11722199132,5cyclictest0-21swapper/307:41:063
11722199131,7cyclictest0-21swapper/311:41:053
11722199131,7cyclictest0-21swapper/310:50:543
11722199131,7cyclictest0-21swapper/310:26:073
11722199131,7cyclictest0-21swapper/309:21:053
11722199131,6cyclictest0-21swapper/311:45:543
11722199131,6cyclictest0-21swapper/309:01:053
11722199131,6cyclictest0-21swapper/307:11:053
11722199130,6cyclictest0-21swapper/308:26:043
11722199130,6cyclictest0-21swapper/308:21:093
11722199123,4cyclictest0-21swapper/309:31:043
11722199122,7cyclictest0-21swapper/311:36:173
11722199122,6cyclictest0-21swapper/311:56:013
11722199122,6cyclictest0-21swapper/309:15:593
11722199122,5cyclictest0-21swapper/308:11:043
11722199121,7cyclictest0-21swapper/311:25:593
11722199121,7cyclictest0-21swapper/310:00:593
11722199121,7cyclictest0-21swapper/310:00:593
11722199121,7cyclictest0-21swapper/309:55:593
11722199121,6cyclictest0-21swapper/310:55:593
11722199121,6cyclictest0-21swapper/310:20:583
11722199121,6cyclictest0-21swapper/308:06:113
11722199121,6cyclictest0-21swapper/307:56:053
11722199121,6cyclictest0-21swapper/307:56:053
11722199121,5cyclictest0-21swapper/309:10:163
11722199114,5cyclictest0-21swapper/311:21:073
11722199114,4cyclictest0-21swapper/312:06:123
11722199113,6cyclictest0-21swapper/307:16:033
11722199113,5cyclictest0-21swapper/311:51:103
11722199112,6cyclictest0-21swapper/311:05:563
11722199112,5cyclictest0-21swapper/308:16:043
11722199111,7cyclictest0-21swapper/312:25:593
11722199111,6cyclictest0-21swapper/307:20:553
11722199102,5cyclictest0-21swapper/307:45:563
11722199102,5cyclictest0-21swapper/307:45:563
11722199101,6cyclictest0-21swapper/308:45:013
11722199101,5cyclictest0-21swapper/308:31:043
1172219993,4cyclictest0-21swapper/308:51:063
1172219981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional