You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 23:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sat Dec 06, 2025 12:44:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1768821070,2sleep017690-21latency_hist08:23:490
1768821070,2sleep017690-21latency_hist08:23:490
2158621060,3sleep20-21swapper/211:24:042
2158621060,3sleep20-21swapper/211:24:042
213832780,3sleep20-21swapper/211:23:562
23953997549,16cyclictest55150irq/47-eth008:29:010
23953997549,16cyclictest55150irq/47-eth008:29:010
236082690,3sleep20-21swapper/211:29:192
236082690,3sleep20-21swapper/211:29:192
158772640,3sleep20-21swapper/211:09:042
158772640,3sleep20-21swapper/211:09:042
91922600,3sleep20-21swapper/207:59:102
91922600,3sleep20-21swapper/207:59:102
23962590,2sleep10-21swapper/111:59:041
23962590,2sleep10-21swapper/111:59:041
2395499562,43cyclictest17956-21taskset08:24:011
2395499562,43cyclictest17956-21taskset08:24:011
23953995619,14cyclictest171ktimers/011:14:010
23953995619,14cyclictest171ktimers/011:14:010
23954995449,3cyclictest0-21swapper/111:54:001
2395499512,8cyclictest181rcu_preempt11:24:011
2395499512,8cyclictest181rcu_preempt11:24:011
128872510,2sleep20-21swapper/212:24:102
128872510,2sleep20-21swapper/212:24:102
2395499492,7cyclictest181rcu_preempt08:39:001
23953994924,18cyclictest171ktimers/012:19:000
23953994924,18cyclictest171ktimers/012:19:000
23342490,1sleep00-21swapper/011:59:040
23342490,1sleep00-21swapper/011:59:040
23955994815,18cyclictest0-21swapper/211:44:022
23955994815,18cyclictest0-21swapper/211:44:022
2395499482,9cyclictest181rcu_preempt07:59:041
2395499482,9cyclictest181rcu_preempt07:59:041
34692470,3sleep20-21swapper/207:44:072
34692470,3sleep20-21swapper/207:44:072
23954994737,5cyclictest0-21swapper/109:58:581
23954994737,5cyclictest0-21swapper/109:58:581
23954994535,6cyclictest0-21swapper/111:19:021
2395399457,25cyclictest55150irq/47-eth007:29:020
2395399457,25cyclictest55150irq/47-eth007:29:020
23953994525,15cyclictest171ktimers/007:54:020
23953994425,13cyclictest171ktimers/009:34:060
23953994425,13cyclictest171ktimers/009:34:060
14332430,2sleep10-21swapper/107:39:091
70422420,3sleep30-21swapper/312:09:153
70422420,3sleep30-21swapper/312:09:153
313632420,3chrt0-21swapper/211:49:082
313632420,3chrt0-21swapper/211:49:082
320802410,3sleep123952-21cyclictest08:59:111
23953994131,5cyclictest0-21swapper/011:04:020
23953994131,5cyclictest0-21swapper/011:04:020
12402400,2sleep20-21swapper/210:29:092
12402400,2sleep20-21swapper/210:29:092
103822400,2sleep20-21swapper/209:29:042
23954993929,8cyclictest30904-21sendmail10:23:491
23954993929,8cyclictest30904-21sendmail10:23:491
183042390,3chrt0-21swapper/112:38:561
23954993731,4cyclictest23952-21cyclictest12:09:151
23954993731,4cyclictest23952-21cyclictest12:09:151
2395499371,5cyclictest181rcu_preempt10:44:041
23953993725,8cyclictest0-21swapper/010:49:000
23953993725,8cyclictest0-21swapper/010:49:000
227182370,2sleep10-21swapper/109:59:141
227182370,2sleep10-21swapper/109:59:141
290142350,2sleep029022-21apt08:53:480
290142350,2sleep029022-21apt08:53:480
23954993527,6cyclictest5371-21threads10:39:151
23954993527,6cyclictest5371-21threads10:39:151
23954993525,8cyclictest15688-21sendmail09:43:501
23954993525,8cyclictest15688-21sendmail09:43:501
23953993529,5cyclictest12404-21kernelversion10:59:100
23953993520,11cyclictest0-21swapper/010:04:010
23953993520,11cyclictest0-21swapper/010:04:010
155122350,6sleep1281ktimers/112:30:161
241722340,2sleep124152-21latency10:04:071
241722340,2sleep124152-21latency10:04:071
2395499343,24cyclictest17629-21chrt11:13:581
2395499343,24cyclictest17629-21chrt11:13:581
23953993326,5cyclictest1194-21fschecks_time07:39:060
23953993325,6cyclictest4264-21fschecks_count12:04:060
23953993325,6cyclictest4264-21fschecks_count12:04:060
23953993325,6cyclictest31763-21fschecks_time07:34:050
23953993325,6cyclictest31763-21fschecks_time07:34:050
23953993324,7cyclictest22701-21threads09:59:130
23953993324,7cyclictest22701-21threads09:59:130
23953993323,8cyclictest0-21swapper/008:44:010
23953993323,8cyclictest0-21swapper/008:44:010
23955993211,16cyclictest64962sleep209:19:032
23955993211,16cyclictest64962sleep209:19:032
2395499323,7cyclictest181rcu_preempt12:14:011
23954993226,5cyclictest23952-21cyclictest08:29:051
23954993226,5cyclictest23952-21cyclictest08:29:051
23954993223,7cyclictest3718-21threads09:09:151
23954993223,7cyclictest3718-21threads09:09:151
2395499321,21cyclictest290132sleep111:44:021
2395499321,21cyclictest290132sleep111:44:021
23953993221,10cyclictest480-21sshd12:14:310
167892320,3sleep20-21swapper/208:19:122
167892320,3sleep20-21swapper/208:19:122
23954993124,5cyclictest8884-21kernelversion09:24:061
23954993124,5cyclictest8884-21kernelversion09:24:061
23953993121,8cyclictest29941-21netstat10:19:090
23953993121,8cyclictest29941-21netstat10:19:090
2395699301,16cyclictest255062sleep308:44:023
2395699301,16cyclictest255062sleep308:44:023
23954993025,3cyclictest29326-21kworker/u9:5+nfsiod08:58:561
23954993025,3cyclictest29326-21kworker/u9:5+nfsiod08:58:561
23953993021,7cyclictest2605-21latency_hist09:08:540
23953993021,7cyclictest2605-21latency_hist09:08:540
118242300,1sleep1271rcuc/112:23:531
118242300,1sleep1271rcuc/112:23:531
2395499292,7cyclictest181rcu_preempt09:49:021
23954992923,5cyclictest3436-21switchtime10:34:141
23954992923,5cyclictest3436-21switchtime10:34:141
23954992923,4cyclictest23952-21cyclictest08:18:531
23954992923,4cyclictest23952-21cyclictest08:18:531
23954992922,5cyclictest12609-21needreboot12:24:061
23954992922,5cyclictest12609-21needreboot12:24:061
23953992925,3cyclictest23952-21cyclictest10:14:040
23953992925,3cyclictest14291-21gltestperf12:29:050
23953992921,5cyclictest21667-21cut11:24:040
23953992921,5cyclictest21667-21cut11:24:040
23953992917,10cyclictest0-21swapper/012:13:490
23953992917,10cyclictest0-21swapper/012:13:490
152462290,3sleep20-21swapper/209:39:162
152462290,3sleep20-21swapper/209:39:162
10262290,3sleep129-21ksoftirqd/109:04:061
10262290,3sleep129-21ksoftirqd/109:04:061
40872280,2sleep14090-21unixbench_singl07:44:141
40872280,2sleep14090-21unixbench_singl07:44:141
2395699285,9cyclictest0-21swapper/310:49:023
2395699285,9cyclictest0-21swapper/310:49:023
23954992823,4cyclictest407-21in:imuxsock08:53:501
23954992823,4cyclictest407-21in:imuxsock08:53:501
23954992822,5cyclictest14091-21gltestperf11:04:071
23954992822,5cyclictest14091-21gltestperf11:04:071
23954992822,3cyclictest0-21swapper/111:14:151
23954992822,3cyclictest0-21swapper/111:14:151
23954992821,5cyclictest10762-21kernelversion09:29:061
23953992824,3cyclictest30112-21ntp_offset08:54:140
23953992824,3cyclictest30112-21ntp_offset08:54:140
23953992824,3cyclictest23952-21cyclictest07:49:110
23953992822,3cyclictest20222-21head11:19:090
23953992820,6cyclictest16402-21ntp_kernel_pll_11:09:100
23953992820,6cyclictest16402-21ntp_kernel_pll_11:09:100
23953992819,7cyclictest31489-21gltestperf10:24:050
2395599271,20cyclictest6279-21chrt10:44:042
2395499275,3cyclictest25493-21apt-get10:09:031
2395499275,3cyclictest25493-21apt-get10:09:031
23954992722,3cyclictest0-21swapper/108:23:491
23954992722,3cyclictest0-21swapper/108:23:491
23954992721,3cyclictest0-21swapper/112:04:161
23954992721,3cyclictest0-21swapper/112:04:161
23954992721,3cyclictest0-21swapper/108:08:481
23954992721,3cyclictest0-21swapper/108:08:481
23954992721,3cyclictest0-21swapper/108:08:481
23953992722,3cyclictest0-21swapper/012:24:020
23953992722,3cyclictest0-21swapper/012:24:020
2395399272,20cyclictest394-21snmpd09:19:030
2395399272,20cyclictest394-21snmpd09:19:030
23953992721,5cyclictest5380-21timerandwakeup10:39:150
23953992721,5cyclictest5380-21timerandwakeup10:39:150
23953992721,4cyclictest17594-21latency_hist09:48:490
23953992721,4cyclictest17594-21latency_hist09:48:490
23953992720,5cyclictest2908-21cpuspeed07:44:020
23953992720,5cyclictest2908-21cpuspeed07:44:020
23953992720,5cyclictest27067-21latency_hist11:38:580
23953992720,5cyclictest27067-21latency_hist11:38:580
2395399271,9cyclictest236072fschecks_time11:29:190
2395399271,9cyclictest236072fschecks_time11:29:190
267322260,4chrt0-21swapper/210:09:152
267322260,4chrt0-21swapper/210:09:152
23954992621,3cyclictest23952-21cyclictest11:39:141
23954992621,3cyclictest23952-21cyclictest11:39:141
23954992618,5cyclictest0-21swapper/108:09:041
23954992618,5cyclictest0-21swapper/108:09:041
23953992619,5cyclictest12996-21ntp_kernel_pll_08:09:100
23953992619,5cyclictest12996-21ntp_kernel_pll_08:09:100
23953992618,6cyclictest5312-21ntp_kernel_err09:14:100
23953992618,6cyclictest5312-21ntp_kernel_err09:14:100
23953992617,7cyclictest1204-21wc11:54:110
271622250,2sleep30-21swapper/310:13:503
271622250,2sleep30-21swapper/310:13:503
23954992521,2cyclictest0-21swapper/109:48:491
23954992521,2cyclictest0-21swapper/109:48:491
23954992521,2cyclictest0-21swapper/107:53:491
23954992520,4cyclictest31337-21fschecks_count10:24:041
23954992520,3cyclictest0-21swapper/110:59:091
23954992519,3cyclictest0-21swapper/110:14:101
23954992518,5cyclictest25484-21/usr/sbin/munin08:44:101
23954992518,5cyclictest25484-21/usr/sbin/munin08:44:101
2395499251,19cyclictest21721-21taskset08:34:041
23953992519,4cyclictest26736-21sed10:09:150
23953992519,4cyclictest26736-21sed10:09:150
23953992518,5cyclictest526-21sshd09:03:310
23953992518,5cyclictest16671-21kernelversion12:34:070
23953992518,4cyclictest0-21swapper/009:13:510
23953992518,4cyclictest0-21swapper/009:13:510
23953992516,7cyclictest0-21swapper/009:58:550
23953992516,7cyclictest0-21swapper/009:58:550
312942243,8sleep20-21swapper/208:58:572
312942243,8sleep20-21swapper/208:58:572
2395599241,18cyclictest21721-21taskset08:34:042
23954992420,3cyclictest23952-21cyclictest09:16:021
23954992420,3cyclictest23952-21cyclictest09:16:021
23954992419,3cyclictest10691-21needreboot10:54:091
23954992419,3cyclictest10691-21needreboot10:54:091
23954992419,3cyclictest10691-21needreboot10:54:091
23954992417,5cyclictest942-21gltestperf10:29:061
23954992417,5cyclictest942-21gltestperf10:29:061
23954992417,5cyclictest31032-21fschecks_count11:49:061
23954992417,5cyclictest31032-21fschecks_count11:49:061
23953992419,4cyclictest11924-21sh09:33:480
23953992419,3cyclictest0-21swapper/011:49:130
23953992419,3cyclictest0-21swapper/011:49:130
23953992419,3cyclictest0-21swapper/010:34:070
23953992419,3cyclictest0-21swapper/010:34:070
23953992418,4cyclictest0-21swapper/008:24:120
23953992418,4cyclictest0-21swapper/008:24:120
23953992415,7cyclictest663-21runrttasks11:44:120
23953992415,7cyclictest663-21runrttasks11:44:120
146302240,4sleep20-21swapper/212:29:082
108202240,3chrt0-21swapper/210:54:132
108202240,3chrt0-21swapper/210:54:132
108202240,3chrt0-21swapper/210:54:132
2395599233,11cyclictest0-21swapper/212:18:582
23954992318,3cyclictest0-21swapper/107:54:071
23954992317,3cyclictest0-21swapper/111:34:021
23954992317,3cyclictest0-21swapper/111:34:021
23953992319,2cyclictest0-21swapper/010:44:060
2395399231,5cyclictest0-21swapper/008:00:020
2395399231,5cyclictest0-21swapper/008:00:020
23953992315,5cyclictest19476-21cat09:53:490
261592220,3sleep20-21swapper/208:44:092
261592220,3sleep20-21swapper/208:44:092
23954992218,2cyclictest0-21swapper/107:34:051
23954992218,2cyclictest0-21swapper/107:34:051
23954992218,2cyclictest0-21swapper/107:29:041
23954992218,2cyclictest0-21swapper/107:29:041
23954992216,3cyclictest0-21swapper/109:19:151
23954992216,3cyclictest0-21swapper/109:19:151
23954992215,5cyclictest20033-21latency_hist12:43:491
23954992215,5cyclictest20033-21latency_hist12:43:491
2395399228,12cyclictest0-21swapper/010:57:020
2395399228,12cyclictest0-21swapper/010:57:020
2395399228,12cyclictest0-21swapper/010:57:020
23953992217,4cyclictest23952-21cyclictest11:39:040
23953992217,4cyclictest23952-21cyclictest11:39:040
23953992216,4cyclictest10874-21/usr/sbin/munin08:04:060
23953992216,4cyclictest10874-21/usr/sbin/munin08:04:060
23953992216,4cyclictest10874-21/usr/sbin/munin08:04:060
23953992215,5cyclictest14402-21fschecks_time08:14:030
23953992215,5cyclictest14402-21fschecks_time08:14:030
23953992215,4cyclictest0-21swapper/009:24:050
23953992215,4cyclictest0-21swapper/009:24:050
2395699213,9cyclictest0-21swapper/309:58:543
2395699213,9cyclictest0-21swapper/309:58:543
2395699213,8cyclictest0-21swapper/309:19:023
2395699213,8cyclictest0-21swapper/309:19:023
2395499216,8cyclictest0-21swapper/111:30:481
2395499216,8cyclictest0-21swapper/111:30:481
23954992117,2cyclictest0-21swapper/109:33:581
23954992117,2cyclictest0-21swapper/109:33:581
23953992116,3cyclictest0-21swapper/012:39:260
23953992116,3cyclictest0-21swapper/012:39:260
23953992116,3cyclictest0-21swapper/008:39:050
23953992114,5cyclictest15688-21sendmail09:43:490
23953992114,5cyclictest15688-21sendmail09:43:490
44332200,2sleep20-21swapper/209:13:512
44332200,2sleep20-21swapper/209:13:512
2395699203,9cyclictest0-21swapper/310:34:043
2395699203,9cyclictest0-21swapper/310:34:043
2395699203,9cyclictest0-21swapper/309:14:033
2395699203,9cyclictest0-21swapper/309:14:033
2395699203,9cyclictest0-21swapper/308:14:013
2395699203,9cyclictest0-21swapper/308:14:013
2395699203,9cyclictest0-21swapper/308:04:033
2395699203,9cyclictest0-21swapper/308:04:033
2395699203,9cyclictest0-21swapper/308:04:033
2395699202,9cyclictest0-21swapper/312:14:013
23954992015,3cyclictest0-21swapper/110:53:491
23954992015,3cyclictest0-21swapper/110:53:491
23953992015,3cyclictest0-21swapper/010:29:110
23953992015,3cyclictest0-21swapper/010:29:110
2395399201,15cyclictest21716-21apt-get08:34:020
2395699193,9cyclictest0-21swapper/312:25:093
2395699193,9cyclictest0-21swapper/312:25:093
2395699193,9cyclictest0-21swapper/312:04:043
2395699193,9cyclictest0-21swapper/312:04:043
2395699193,9cyclictest0-21swapper/311:29:163
2395699193,9cyclictest0-21swapper/311:29:163
2395699193,9cyclictest0-21swapper/310:59:063
2395699193,9cyclictest0-21swapper/310:24:033
2395699193,9cyclictest0-21swapper/310:19:393
2395699193,9cyclictest0-21swapper/310:19:393
2395699193,9cyclictest0-21swapper/309:59:023
2395699193,9cyclictest0-21swapper/309:59:023
2395699193,9cyclictest0-21swapper/308:09:023
2395699193,9cyclictest0-21swapper/308:09:023
2395699193,9cyclictest0-21swapper/307:59:043
2395699193,9cyclictest0-21swapper/307:59:043
2395699193,8cyclictest0-21swapper/310:44:043
2395699193,8cyclictest0-21swapper/307:29:023
2395699193,8cyclictest0-21swapper/307:29:023
2395699192,9cyclictest0-21swapper/307:54:033
2395699192,4cyclictest0-21swapper/308:29:013
2395699192,4cyclictest0-21swapper/308:29:013
2395699191,14cyclictest17842-21chrt11:14:043
2395699191,14cyclictest17842-21chrt11:14:043
2395599193,9cyclictest0-21swapper/211:59:032
2395599193,9cyclictest0-21swapper/211:59:032
2395599193,9cyclictest0-21swapper/208:53:552
2395599193,9cyclictest0-21swapper/208:53:552
2395599193,7cyclictest0-21swapper/208:04:012
2395599193,7cyclictest0-21swapper/208:04:012
2395599193,7cyclictest0-21swapper/208:04:012
98572180,2sleep30-21swapper/312:19:003
98572180,2sleep30-21swapper/312:19:003
294052180,3sleep20-21swapper/210:19:042
294052180,3sleep20-21swapper/210:19:042
2395699187,6cyclictest138222sleep311:04:043
2395699187,6cyclictest138222sleep311:04:043
2395699183,9cyclictest0-21swapper/311:34:033
2395699183,9cyclictest0-21swapper/311:34:033
2395699183,9cyclictest0-21swapper/310:54:043
2395699183,9cyclictest0-21swapper/310:54:043
2395699183,9cyclictest0-21swapper/310:54:043
2395699183,9cyclictest0-21swapper/309:44:033
2395699183,9cyclictest0-21swapper/309:44:033
2395699183,9cyclictest0-21swapper/308:59:043
2395699183,8cyclictest0-21swapper/311:54:593
2395699183,8cyclictest0-21swapper/309:29:023
2395699182,9cyclictest0-21swapper/308:24:023
2395699182,9cyclictest0-21swapper/308:24:023
2395699182,8cyclictest0-21swapper/308:39:023
2395599183,9cyclictest0-21swapper/209:58:582
2395599183,9cyclictest0-21swapper/209:58:582
23955991813,3cyclictest215462sleep208:33:522
23955991813,3cyclictest215462sleep208:33:522
2395699177,4cyclictest0-21swapper/309:04:023
2395699177,4cyclictest0-21swapper/309:04:023
2395699173,9cyclictest0-21swapper/311:13:553
2395699173,9cyclictest0-21swapper/311:13:553
2395699173,9cyclictest0-21swapper/309:49:023
2395699173,8cyclictest0-21swapper/309:24:013
2395699173,8cyclictest0-21swapper/309:24:013
2395699173,5cyclictest0-21swapper/309:34:053
2395699173,5cyclictest0-21swapper/309:34:053
2395699172,9cyclictest0-21swapper/310:39:033
2395699172,9cyclictest0-21swapper/310:39:033
2395699171,12cyclictest23897-21taskset10:04:053
2395699171,12cyclictest23897-21taskset10:04:053
2395699171,11cyclictest4831-21chrt07:49:023
2395699171,11cyclictest29036-21chrt11:44:033
2395699171,11cyclictest29036-21chrt11:44:033
2395699171,11cyclictest14906-21chrt12:29:133
2395599173,9cyclictest0-21swapper/208:39:022
2395699164,8cyclictest0-21swapper/311:49:033
2395699164,8cyclictest0-21swapper/311:49:033
2395699163,9cyclictest0-21swapper/311:59:033
2395699163,9cyclictest0-21swapper/311:59:033
2395699163,9cyclictest0-21swapper/308:54:063
2395699163,9cyclictest0-21swapper/308:54:063
2395699162,5cyclictest0-21swapper/307:34:013
2395699162,5cyclictest0-21swapper/307:34:013
2395699161,11cyclictest21483-21chrt11:24:013
2395699161,11cyclictest21483-21chrt11:24:013
2395699161,10cyclictest29173-21chrt10:18:573
2395599163,9cyclictest0-21swapper/211:39:042
2395599163,9cyclictest0-21swapper/211:39:042
2395599163,9cyclictest0-21swapper/211:14:022
2395599163,9cyclictest0-21swapper/211:14:022
2395599163,9cyclictest0-21swapper/210:14:032
2395599163,9cyclictest0-21swapper/210:05:322
2395599163,9cyclictest0-21swapper/210:05:322
2395599163,9cyclictest0-21swapper/209:34:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional