You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 06:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun Dec 21, 2025 00:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4369872760,8sleep0201rcuc/019:39:030
4578412730,2sleep3481ktimers/319:59:043
409316995544,9cyclictest435096-21ntpq19:34:091
409311995242,7cyclictest430841-21latency_hist19:30:010
409316994944,4cyclictest414178-21kworker/u8:2+flush-8:021:04:121
409316994536,7cyclictest278-21systemd-journal23:05:011
409311994541,3cyclictest414178-21kworker/u8:2+events_unbound21:40:000
409311994538,5cyclictest589197-21latency_hist22:10:010
409316994333,8cyclictest278-21systemd-journal19:20:001
409316994135,4cyclictest687476-21kworker/u8:3+flush-8:023:50:101
409316994135,4cyclictest446194-21apt-get19:48:541
409316994133,5cyclictest722557-21latency_hist00:25:001
409311994132,6cyclictest613714-21grep22:34:140
409311994132,6cyclictest490097-21latency_hist20:30:010
409311994125,11cyclictest278-21systemd-journal00:09:090
409316993935,3cyclictest717893-21kworker/u8:0+events_unbound00:34:121
409316993934,4cyclictest394562-21kworker/u8:0+flush-8:019:59:011
409311993934,3cyclictest414178-21kworker/u8:2+flush-8:022:24:000
409311993934,3cyclictest414178-21kworker/u8:2+events_unbound21:44:100
409311993933,4cyclictest608812-21kworker/u8:3+events_unbound22:44:100
409311993932,5cyclictest608812-21kworker/u8:3+flush-8:022:39:000
409311993931,6cyclictest737298-21latency_hist00:40:010
409311993930,6cyclictest278-21systemd-journal23:30:000
409311993833,3cyclictest414178-21kworker/u8:2+flush-8:021:49:080
409311993832,4cyclictest653634-21kworker/u8:2+events_unbound23:54:110
409311993825,9cyclictest450531-21latency_hist19:50:010
409311993731,4cyclictest579620-21kworker/u8:1+flush-8:022:13:590
409311993731,4cyclictest524889-21kworker/u8:1+flush-8:021:14:030
409311993729,5cyclictest598900-21grep22:19:140
409311993724,9cyclictest278-21systemd-journal21:19:100
409311993630,4cyclictest530299-21kworker/u8:0+events_unbound22:28:580
409311993630,4cyclictest480297-21kworker/u8:3+flush-8:020:54:120
409311993629,5cyclictest712604-21kworker/u8:2+flush-8:000:33:590
409321993527,6cyclictest278-21systemd-journal23:20:002
409316993528,6cyclictest414178-21kworker/u8:2+flush-8:021:24:121
409311993531,3cyclictest666301-21kworker/u8:1+flush-8:000:19:060
409311993531,3cyclictest3749471-21kworker/u8:3+events_unbound19:19:040
409311993529,4cyclictest666301-21kworker/u8:1+flush-8:000:05:000
409311993529,4cyclictest579620-21kworker/u8:1+flush-8:022:48:590
409311993529,4cyclictest394562-21kworker/u8:0+flush-8:019:59:050
409311993527,6cyclictest638633-21kworker/u8:0+flush-8:023:18:590
409316993426,6cyclictest394562-21kworker/u8:0+flush-8:019:29:161
409311993426,6cyclictest425286-21ntpq19:24:070
409311993423,2cyclictest697943-21idleruntime-cro00:00:000
409311993421,9cyclictest687840-21latency_hist23:50:010
409311993329,3cyclictest414178-21kworker/u8:2+flush-8:022:59:000
409311993328,3cyclictest530299-21kworker/u8:0+flush-8:021:29:070
409311993328,3cyclictest435687-21kworker/u8:1+events_unbound20:19:010
409311993327,4cyclictest414178-21kworker/u8:2+flush-8:020:34:150
409311993327,4cyclictest394562-21kworker/u8:0+flush-8:019:14:090
409311993326,5cyclictest414178-21kworker/u8:2+flush-8:020:59:120
409321993214,9cyclictest0-21swapper/219:48:562
409316993224,5cyclictest409141-21latency_hist19:10:011
409316993222,8cyclictest278-21systemd-journal20:50:001
409311993227,3cyclictest470430-21systemd-run20:10:000
409311993225,5cyclictest648241-21cron23:10:000
40932199318,7cyclictest688147-21cut23:50:012
409321993120,3cyclictest161rcu_preempt23:55:002
40932199310,29cyclictest593431-21netstat22:14:072
409316993124,5cyclictest480297-21kworker/u8:3+flush-8:020:53:581
40932199306,7cyclictest0-21swapper/221:24:032
409321993024,4cyclictest678017-21awk23:40:002
409316993026,3cyclictest589219-21systemd-run22:10:001
409316993025,4cyclictest697944-21kworker/u8:0+events_unbound00:00:001
409316993025,3cyclictest737293-21ls00:40:001
409316993025,3cyclictest515582-21apt-get20:58:571
409316993020,7cyclictest623651-21latency_hist22:45:011
409311993026,3cyclictest414178-21kworker/u8:2+flush-8:021:24:140
409311993025,4cyclictest717893-21kworker/u8:0+events_unbound00:25:370
409311993025,4cyclictest414178-21kworker/u8:2+flush-8:019:54:050
409311993025,4cyclictest394562-21kworker/u8:0+events_unbound20:13:590
409311993025,4cyclictest394562-21kworker/u8:0+events_unbound20:13:590
409311993022,6cyclictest530299-21kworker/u8:0+flush-8:021:54:080
40932199296,14cyclictest701960-21kthreadcore00:04:052
409321992926,2cyclictest470992-21aten2_r5power_c20:13:592
409321992926,2cyclictest470992-21aten2_r5power_c20:13:592
409321992924,4cyclictest579620-21kworker/u8:1+flush-8:022:29:162
409316992925,3cyclictest501697-21kthreadcore20:44:021
409316992923,4cyclictest559607-21sh21:40:001
409316992921,6cyclictest278-21systemd-journal23:35:001
409311992925,3cyclictest653634-21kworker/u8:2+events_unbound23:34:110
409311992923,4cyclictest579620-21kworker/u8:1+flush-8:022:04:130
409311992921,6cyclictest481072-21diskstats20:24:010
40931199291,3cyclictest509762-21runrttasks20:49:130
40931199291,27cyclictest440731-21kworker/0:1+events20:44:070
409316992825,2cyclictest705575-21kthreadcore00:09:051
409316992824,3cyclictest414178-21kworker/u8:2+flush-8:019:39:011
409316992824,2cyclictest724033-21kthreadcore00:29:041
409316992823,3cyclictest579620-21kworker/u8:1+events_unbound22:54:131
409316992823,3cyclictest579620-21kworker/u8:1+events_unbound22:54:131
409316992823,3cyclictest569507-21systemd-run21:50:011
409316992822,4cyclictest470960-21apt-get20:13:571
409316992822,4cyclictest470960-21apt-get20:13:571
409316992822,4cyclictest421564-21apt-get19:23:541
409316992819,7cyclictest278-21systemd-journal22:25:001
409311992824,3cyclictest608812-21kworker/u8:3+flush-8:022:54:000
409311992824,3cyclictest608812-21kworker/u8:3+flush-8:022:54:000
409311992824,2cyclictest0-21swapper/021:10:000
409311992822,4cyclictest653634-21kworker/u8:2+flush-8:023:39:100
409311992822,4cyclictest435687-21kworker/u8:1+flush-8:020:04:090
5783552270,4sleep20-21swapper/221:59:042
409316992723,2cyclictest708936-21irqcore00:14:021
409316992721,4cyclictest638633-21kworker/u8:0+flush-8:023:05:401
409311992722,3cyclictest500245-21munin-run20:40:020
409311992721,4cyclictest435687-21kworker/u8:1+flush-8:019:44:140
409311992721,4cyclictest414178-21kworker/u8:2+flush-8:021:59:030
40932199263,8cyclictest612616-21tr22:34:052
409321992623,2cyclictest481047-21cpuspeed_turbos20:24:002
409321992610,7cyclictest0-21swapper/221:49:082
409316992621,3cyclictest0-21swapper/123:25:011
409316992619,6cyclictest666301-21kworker/u8:1+flush-8:000:19:081
40931699261,24cyclictest539306-21sshd21:19:101
40931699261,23cyclictest687886-21munin-run23:50:001
409311992622,2cyclictest524889-21kworker/u8:1+flush-8:021:34:030
409311992622,2cyclictest524889-21kworker/u8:1+flush-8:021:34:030
409311992620,5cyclictest666301-21kworker/u8:1+flush-8:000:24:130
40932199254,6cyclictest553233-21cut21:34:052
40932199254,6cyclictest553233-21cut21:34:052
409321992522,2cyclictest614603-21tr22:38:582
40932199250,12cyclictest454799-21cut19:54:052
409316992521,3cyclictest678016-21turbostat.cron23:40:001
409316992521,3cyclictest618769-21latency_hist22:40:001
409316992521,2cyclictest0-21swapper/121:54:091
409316992521,2cyclictest0-21swapper/121:29:111
409311992521,3cyclictest394562-21kworker/u8:0+events_unbound19:34:170
409311992519,4cyclictest638633-21kworker/u8:0+flush-8:023:24:010
40932199242,14cyclictest0-21swapper/223:09:122
409321992421,2cyclictest727340-21unixbench_multi00:29:162
409321992421,2cyclictest514856-21unixbench_multi20:54:132
40932199240,12cyclictest584839-21apt-get22:08:582
409316992420,3cyclictest435687-21kworker/u8:1+flush-8:019:44:091
409316992420,2cyclictest0-21swapper/120:29:041
409316992419,4cyclictest435687-21kworker/u8:1+flush-8:020:04:011
409316992418,5cyclictest524889-21kworker/u8:1+flush-8:021:34:101
409316992418,5cyclictest524889-21kworker/u8:1+flush-8:021:34:101
409311992419,3cyclictest653634-21kworker/u8:2+flush-8:023:44:030
409311992419,3cyclictest414178-21kworker/u8:2+events_unbound21:03:580
40932199238,2cyclictest161rcu_preempt20:15:252
40932199233,4cyclictest0-21swapper/222:55:002
40932199233,4cyclictest0-21swapper/222:55:002
409321992320,2cyclictest505951-21cpuspeed_turbos20:48:572
409321992319,2cyclictest559602-21cron21:40:002
409321992314,2cyclictest161rcu_preempt19:24:042
40932199230,13cyclictest491898-21cut20:34:032
409316992320,2cyclictest530846-21cpuspeed_turbos21:13:561
409316992319,3cyclictest590336-21irqcore22:14:021
409316992319,2cyclictest0-21swapper/123:29:071
409316992319,2cyclictest0-21swapper/123:19:021
409316992319,2cyclictest0-21swapper/122:48:591
409316992318,3cyclictest412384-21kthreadcore19:14:051
4614202220,6sleep20-21swapper/220:04:012
40932199225,2cyclictest161rcu_preempt23:43:542
40932199222,4cyclictest0-21swapper/219:19:062
409321992219,2cyclictest737325-21sendmail00:40:002
409321992219,2cyclictest371rcuc/221:13:232
409321992218,2cyclictest0-21swapper/200:20:002
409321992213,2cyclictest161rcu_preempt20:59:062
40932199220,8cyclictest574482-21sed21:55:012
40931699228,5cyclictest0-21swapper/122:55:591
409316992219,2cyclictest609952-21irqcore22:34:011
409316992218,3cyclictest564544-21sh21:45:001
409316992218,2cyclictest0-21swapper/120:19:041
409316992218,2cyclictest0-21swapper/120:09:021
409316992217,4cyclictest414178-21kworker/u8:2+events_unbound19:54:091
409316992217,3cyclictest526015-21irqcore21:09:011
409311992218,2cyclictest0-21swapper/000:13:560
409311992217,4cyclictest579620-21kworker/u8:1+flush-8:023:14:090
409311992216,4cyclictest608812-21kworker/u8:3+events_unbound23:04:000
40932199214,2cyclictest161rcu_preempt22:44:122
40932199213,4cyclictest0-21swapper/223:04:062
40932199212,9cyclictest0-21swapper/220:43:432
40932199212,5cyclictest0-21swapper/223:29:092
40932199212,4cyclictest0-21swapper/220:38:552
40932199212,4cyclictest0-21swapper/219:43:592
40932199212,4cyclictest0-21swapper/200:09:052
409321992118,2cyclictest371rcuc/223:23:352
409321992117,3cyclictest371rcuc/219:59:162
409321992117,3cyclictest371rcuc/219:27:382
40932199211,5cyclictest721940-21ntpq00:24:072
409321992113,6cyclictest278-21systemd-journal00:00:002
409321992110,9cyclictest709283-21tr00:14:022
40931699218,4cyclictest0-21swapper/122:01:581
409316992117,3cyclictest500053-21unixbench_multi20:39:151
409316992117,2cyclictest0-21swapper/121:58:571
409316992117,2cyclictest0-21swapper/120:34:051
409316992117,2cyclictest0-21swapper/120:23:591
409316992117,2cyclictest0-21swapper/100:04:021
409316992115,4cyclictest0-21swapper/123:13:591
409316992110,3cyclictest0-21swapper/122:25:211
40467322111,6sleep20-21swapper/219:07:202
40932199209,2cyclictest161rcu_preempt20:30:012
40932199207,2cyclictest161rcu_preempt20:09:102
40932199202,6cyclictest0-21swapper/221:40:322
40932199202,6cyclictest0-21swapper/221:04:022
40932199202,5cyclictest0-21swapper/221:29:002
40932199202,4cyclictest0-21swapper/222:19:012
40932199202,4cyclictest0-21swapper/219:39:062
409321992017,2cyclictest371rcuc/219:33:582
409321992010,7cyclictest535667-21cut21:18:582
40932199200,5cyclictest731739-21ntpq00:34:072
40932199200,4cyclictest624201-21apt-get22:48:562
40931699208,3cyclictest0-21swapper/123:41:011
409316992016,2cyclictest0-21swapper/122:18:561
4045792202,12sleep00-21swapper/019:06:100
40932199192,7cyclictest0-21swapper/222:57:592
40932199192,4cyclictest0-21swapper/222:24:042
409321991916,2cyclictest371rcuc/223:11:482
409321991916,2cyclictest371rcuc/221:05:212
40932199190,5cyclictest583620-21sed22:04:062
409321991815,2cyclictest371rcuc/223:31:302
409321991815,2cyclictest371rcuc/219:12:272
4046272183,8sleep30-21swapper/319:06:453
7056372170,3sleep3575-21kworker/3:2+events00:09:053
6475792170,2sleep3471rcuc/323:09:083
5078222178,5sleep30-21swapper/320:49:053
6886212162,9sleep30-21swapper/323:53:543
4862632166,6sleep30-21swapper/320:29:063
4166142162,9sleep30-21swapper/319:18:573
6794712155,6sleep30-21swapper/323:44:033
6878252142,8sleep30-21swapper/323:50:003
4456292142,7sleep30-21swapper/319:45:013
6236352132,6sleep30-21swapper/322:45:003
6138542132,6sleep30-21swapper/322:35:013
5371602132,7sleep30-21swapper/321:19:073
5343752132,7sleep30-21swapper/321:14:083
5179222134,5sleep30-21swapper/320:59:083
7233552122,6sleep30-21swapper/300:29:013
6952322122,6sleep30-21swapper/323:59:033
6690782122,6sleep30-21swapper/323:34:003
6643142122,6sleep30-21swapper/323:29:023
6632532122,6sleep30-21swapper/323:25:013
6384392122,5sleep30-21swapper/323:00:013
6377312122,6sleep30-21swapper/322:59:093
6053942122,6sleep30-21swapper/322:29:033
5935542123,5sleep30-21swapper/322:14:113
5850822122,6sleep30-21swapper/322:09:013
5739542122,6sleep30-21swapper/321:54:133
5693102122,6sleep30-21swapper/321:49:153
5596012122,5sleep30-21swapper/321:40:003
5508132122,6sleep30-21swapper/321:34:023
5508132122,6sleep30-21swapper/321:34:023
5486202122,6sleep30-21swapper/321:29:093
5253872122,6sleep30-21swapper/321:08:473
5205362122,6sleep30-21swapper/321:03:573
5109332122,6sleep30-21swapper/320:53:583
5050492122,6sleep30-21swapper/320:44:153
4908942122,6sleep30-21swapper/320:34:003
4664482122,6sleep30-21swapper/320:09:003
4606442122,5sleep30-21swapper/320:00:013
4367782122,6sleep30-21swapper/319:39:033
4313932122,6sleep30-21swapper/319:33:573
4220812124,5sleep30-21swapper/319:24:023
7327032113,5sleep30-21swapper/300:36:193
7278362112,5sleep30-21swapper/300:31:373
7178762112,5sleep30-21swapper/300:20:103
7131922112,5sleep30-21swapper/300:18:383
7080422112,5sleep30-21swapper/300:10:193
6984512112,5sleep30-21swapper/300:03:373
6735672112,5sleep30-21swapper/323:37:573
6537882112,5sleep30-21swapper/323:17:103
6485912112,6sleep30-21swapper/323:11:333
6289782112,6sleep30-21swapper/322:52:323
6289782112,6sleep30-21swapper/322:52:313
6240962113,5sleep30-21swapper/322:47:533
6094272112,6sleep30-21swapper/322:33:573
5993692112,5sleep30-21swapper/322:21:213
5950612112,5sleep30-21swapper/322:19:003
5836602112,5sleep30-21swapper/322:04:083
5746672113,5sleep30-21swapper/321:55:213
5600012112,5sleep30-21swapper/321:41:533
5401742112,5sleep30-21swapper/321:20:103
4961472112,6sleep30-21swapper/320:39:013
4851682112,5sleep30-21swapper/320:25:013
4758422112,6sleep30-21swapper/320:18:353
4706612112,6sleep30-21swapper/320:10:253
4706612112,6sleep30-21swapper/320:10:253
4263122113,5sleep30-21swapper/319:27:073
4099652112,5sleep30-21swapper/319:13:593
4510152102,5sleep30-21swapper/319:50:223
4458692102,5sleep30-21swapper/319:45:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional