You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 05:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Dec 25, 2025 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
74112216164,39sleep20-21swapper/219:06:272
75032215163,39sleep00-21swapper/019:07:380
75162202167,23sleep10-21swapper/119:07:481
73432201164,25sleep30-21swapper/319:05:363
7786998784,2cyclictest0-21swapper/223:35:102
7786997369,3cyclictest0-21swapper/220:17:192
69732500,1sleep20-21swapper/200:00:162
64482450,1sleep30-21swapper/321:30:203
7779993217,9cyclictest0-21swapper/121:44:441
777499328,11cyclictest4665-21idleruntime-cro20:15:000
7786992925,3cyclictest0-21swapper/220:03:532
7779992916,7cyclictest0-21swapper/122:03:121
777499291,13cyclictest9054-21munin-run20:25:000
267382290,2sleep1777999cyclictest21:05:011
7779992815,7cyclictest0-21swapper/123:48:341
778699271,5cyclictest0-21swapper/223:56:262
777999272,6cyclictest9280-21df_inode00:05:141
7779992715,8cyclictest0-21swapper/121:55:471
221852270,3sleep30-21swapper/323:20:163
121702270,2sleep10-21swapper/121:45:111
121702270,2sleep10-21swapper/121:45:101
778699261,6cyclictest0-21swapper/223:48:332
778699261,6cyclictest0-21swapper/222:03:132
778699261,12cyclictest0-21swapper/221:55:162
7779992614,7cyclictest0-21swapper/100:00:381
7779992613,9cyclictest0-21swapper/122:11:091
7779992613,7cyclictest0-21swapper/123:23:481
7779992613,7cyclictest0-21swapper/122:52:131
7774992621,2cyclictest19671-21grep22:00:200
777499261,12cyclictest19429-21munin-run19:35:020
778699251,5cyclictest0-21swapper/222:33:362
778699251,4cyclictest0-21swapper/221:10:182
777999252,15cyclictest87150irq/128-enp1s0-21:10:231
7779992515,3cyclictest0-21swapper/123:17:241
7779992515,3cyclictest0-21swapper/122:33:361
7779992515,3cyclictest0-21swapper/121:23:211
7779992513,8cyclictest0-21swapper/123:52:011
7779992513,7cyclictest0-21swapper/121:33:131
7779992513,7cyclictest0-21swapper/100:22:251
7779992513,6cyclictest0-21swapper/100:27:081
7774992519,3cyclictest28467-21cut22:20:190
202612250,2sleep20-21swapper/200:30:152
779199243,3cyclictest86950irq/126-enp1s0-22:45:163
778699241,7cyclictest0-21swapper/221:33:162
778699241,6cyclictest0-21swapper/223:17:242
778699241,6cyclictest0-21swapper/200:26:402
778699241,5cyclictest0-21swapper/223:32:082
778699241,5cyclictest0-21swapper/200:22:252
778699241,4cyclictest0-21swapper/221:52:302
7779992415,3cyclictest0-21swapper/122:16:491
7779992415,2cyclictest0-21swapper/122:48:531
7779992414,3cyclictest0-21swapper/123:32:081
777999240,17cyclictest0-21swapper/121:37:071
7774992420,2cyclictest3315-21grep23:50:190
7774992419,3cyclictest25878-21grep22:15:140
777499241,11cyclictest28476-21sshd21:05:390
109512240,6sleep3401ktimersoftd/322:55:163
779199231,9cyclictest0-21swapper/322:25:193
779199231,10cyclictest0-21swapper/323:46:163
779199231,10cyclictest0-21swapper/300:22:513
779199230,10cyclictest0-21swapper/322:12:553
778699232,8cyclictest24823-21wc21:00:122
778699232,7cyclictest3686-21munin-run22:40:002
778699232,6cyclictest0-21swapper/219:26:462
778699232,6cyclictest0-21swapper/219:26:462
778699231,6cyclictest0-21swapper/222:42:542
778699231,6cyclictest0-21swapper/221:40:092
778699231,6cyclictest0-21swapper/200:11:152
778699231,6cyclictest0-21swapper/200:11:152
778699231,5cyclictest0-21swapper/222:12:272
778699231,4cyclictest0-21swapper/222:52:132
778699231,4cyclictest0-21swapper/200:37:392
778699230,7cyclictest25512-21kworker/u8:000:20:002
7779992316,3cyclictest0-21swapper/123:40:151
7779992316,3cyclictest0-21swapper/121:52:291
777999231,4cyclictest18367-21memory19:30:171
7779992312,6cyclictest0-21swapper/122:55:121
777999231,19cyclictest8886-21memory19:10:181
777999230,10cyclictest29196-21pluginstate23:35:231
777499239,3cyclictest6104-21/usr/sbin/munin22:45:210
777499233,3cyclictest21287-21idleruntime-cro23:20:020
777499232,10cyclictest1687-21/usr/sbin/munin22:35:210
7774992319,2cyclictest21249-21needreboot20:50:220
7774992319,2cyclictest13692-21grep23:00:200
7774992319,2cyclictest13692-21grep23:00:190
777499231,6cyclictest0-21swapper/021:12:120
777499231,6cyclictest0-21swapper/000:39:400
777499231,11cyclictest14284-21/usr/sbin/munin21:50:140
307802230,2sleep00-21swapper/023:40:150
779199222,11cyclictest27924-21ntp_states21:05:213
779199221,8cyclictest0-21swapper/323:36:033
779199221,6cyclictest0-21swapper/321:57:093
779199221,6cyclictest0-21swapper/321:25:243
779199220,8cyclictest0-21swapper/321:52:213
778699228,3cyclictest87250irq/129-enp1s0-22:20:252
778699224,4cyclictest15753-21grep23:05:182
778699221,8cyclictest9439-21chrt21:37:172
778699221,6cyclictest0-21swapper/221:27:242
778699221,5cyclictest0-21swapper/223:22:362
778699221,4cyclictest0-21swapper/223:52:002
778699221,4cyclictest0-21swapper/221:23:342
778699220,5cyclictest18126-21kworker/u8:122:28:132
7779992213,3cyclictest0-21swapper/122:44:371
777999221,18cyclictest8103-21nfs4_client20:20:191
777999220,15cyclictest0-21swapper/123:00:241
777999220,15cyclictest0-21swapper/123:00:241
777499229,3cyclictest87150irq/128-enp1s0-21:47:450
777499229,3cyclictest87150irq/128-enp1s0-21:47:450
777499227,3cyclictest87150irq/128-enp1s0-00:00:290
777499224,8cyclictest0-21swapper/023:35:260
777499221,7cyclictest0-21swapper/021:38:470
777499221,6cyclictest0-21swapper/022:55:340
7774992214,5cyclictest26740-21latency_hist21:05:020
777499221,3cyclictest87150irq/128-enp1s0-23:10:190
779199219,9cyclictest39-21rcuc/321:50:013
779199219,9cyclictest39-21rcuc/321:50:013
779199212,5cyclictest5736-21processes23:55:233
779199212,3cyclictest31602-21diskstats21:15:143
779199212,11cyclictest21845-21/usr/sbin/munin19:40:203
779199211,4cyclictest9435-21kworker/u8:022:00:143
779199211,4cyclictest577-21systemd-journal23:05:023
779199211,4cyclictest577-21systemd-journal23:05:023
779199211,4cyclictest3230-21/usr/sbin/munin23:50:193
7791992114,4cyclictest32701-21uname22:30:213
779199211,12cyclictest0-21swapper/300:18:103
779199211,10cyclictest0-21swapper/300:28:063
779199210,9cyclictest0-21swapper/322:23:053
779199210,10cyclictest0-21swapper/300:36:133
778699212,3cyclictest13764-21ntp_states23:00:212
778699212,3cyclictest13764-21ntp_states23:00:212
778699211,8cyclictest17455-21diskstats23:10:142
778699211,7cyclictest8957-21cpu00:05:122
778699211,7cyclictest31350-21ntp_states23:40:212
778699211,7cyclictest0-21swapper/222:46:032
778699211,7cyclictest0-21swapper/221:46:062
778699211,7cyclictest0-21swapper/221:46:062
778699211,6cyclictest27198-21df21:05:132
778699211,5cyclictest0-21swapper/222:55:122
778699211,5cyclictest0-21swapper/222:19:332
778699211,5cyclictest0-21swapper/221:16:182
778699211,4cyclictest0-21swapper/223:29:362
778699210,7cyclictest0-21swapper/219:20:232
777999219,4cyclictest11299-21cat20:30:011
777999218,11cyclictest31110-21kworker/u8:220:59:081
777999217,6cyclictest0-21swapper/123:06:531
777999215,9cyclictest21109-21ps00:30:231
7779992117,2cyclictest21004-21cat19:35:251
777999211,3cyclictest5435-21memory23:55:201
777999211,3cyclictest28593-21pmu-power22:20:221
777999211,18cyclictest0-21swapper/120:15:131
777999211,11cyclictest24370-21hddtemp_smartct23:25:181
777999210,14cyclictest0-21swapper/121:25:141
777499218,3cyclictest27495-21vmstat23:30:290
777499218,3cyclictest202622cat00:30:140
777499217,8cyclictest0-21swapper/023:45:250
777499215,9cyclictest0-21swapper/021:40:240
777499212,6cyclictest26237-21/usr/sbin/munin19:50:130
777499212,3cyclictest87150irq/128-enp1s0-00:05:190
7774992118,2cyclictest0-21swapper/023:20:260
7774992117,3cyclictest0-21swapper/021:15:260
7774992117,2cyclictest16763-21tail19:25:250
7774992117,2cyclictest16763-21tail19:25:250
777499211,6cyclictest0-21swapper/022:43:400
777499211,6cyclictest0-21swapper/022:30:380
777499211,6cyclictest0-21swapper/022:29:370
777499211,6cyclictest0-21swapper/022:10:560
777499211,5cyclictest0-21swapper/021:59:450
777499211,5cyclictest0-21swapper/021:26:530
777499211,12cyclictest0-21swapper/022:05:110
777499211,10cyclictest30633-21/usr/sbin/munin20:00:140
777499211,10cyclictest13368-21egrep00:15:110
779199209,3cyclictest801-21snmpd20:19:253
779199208,4cyclictest0-21swapper/322:36:293
779199208,4cyclictest0-21swapper/322:17:503
779199207,4cyclictest0-21swapper/323:32:153
779199204,6cyclictest751-21rngd19:55:103
779199201,9cyclictest86950irq/126-enp1s0-22:05:183
779199201,5cyclictest0-21swapper/300:12:543
779199201,5cyclictest0-21swapper/300:12:543
779199201,4cyclictest0-21swapper/321:43:273
779199201,11cyclictest30288-21/usr/sbin/munin23:40:243
779199201,11cyclictest1075-21/usr/sbin/munin21:20:253
779199201,11cyclictest0-21swapper/323:11:383
779199201,11cyclictest0-21swapper/323:07:493
779199201,11cyclictest0-21swapper/321:36:203
779199201,11cyclictest0-21swapper/300:32:243
779199201,11cyclictest0-21swapper/300:05:003
779199201,10cyclictest0-21swapper/320:28:443
779199200,8cyclictest0-21swapper/322:54:343
778699204,4cyclictest2864-21cstates20:10:142
778699204,3cyclictest86950irq/126-enp1s0-20:05:152
778699203,7cyclictest28202-21crond19:55:012
778699203,4cyclictest0-21swapper/219:45:172
778699201,7cyclictest21919-21sort19:40:132
778699201,6cyclictest10929-21egrep19:15:112
778699201,6cyclictest0-21swapper/220:35:142
778699201,6cyclictest0-21swapper/220:35:142
778699201,5cyclictest9625-21diskstats20:25:152
778699201,5cyclictest0-21swapper/222:06:312
778699201,10cyclictest0-21swapper/220:46:402
777999209,3cyclictest14644-21cat00:15:251
777999200,15cyclictest0-21swapper/122:37:101
777499205,2cyclictest0-21swapper/023:05:190
777499203,3cyclictest86950irq/126-enp1s0-23:30:020
7774992017,2cyclictest0-21swapper/020:55:230
777499201,6cyclictest21845-21/usr/sbin/munin19:40:140
777499201,6cyclictest0-21swapper/021:20:030
777499201,6cyclictest0-21swapper/000:11:420
777499201,6cyclictest0-21swapper/000:11:420
777499201,5cyclictest9244-21swap19:10:230
7774992015,3cyclictest20640-21head19:35:190
777499201,10cyclictest4875-21/usr/sbin/munin20:15:200
777499200,6cyclictest0-21swapper/000:21:440
779199198,3cyclictest801-21snmpd19:14:093
779199197,4cyclictest0-21swapper/322:43:313
779199195,3cyclictest276532timerwakeupswit19:50:243
779199193,6cyclictest32519-21seq20:03:563
779199191,10cyclictest801-21snmpd20:39:293
779199191,10cyclictest801-21snmpd20:39:293
779199191,10cyclictest29060-21/usr/sbin/munin21:10:153
779199191,10cyclictest23699-21/usr/sbin/munin23:25:203
779199191,10cyclictest0-21swapper/300:07:143
778699194,5cyclictest751-21rngd20:50:082
778699194,3cyclictest12050irq/123-ahci[0019:40:012
778699192,4cyclictest17944-21tune2fs19:30:132
778699191,8cyclictest0-21swapper/220:55:152
778699191,6cyclictest16146-21cat20:40:112
778699191,6cyclictest12285-21irqstats20:30:182
778699191,4cyclictest8796-21kernelversion19:10:182
778699191,3cyclictest86950irq/126-enp1s0-19:55:142
777999196,8cyclictest0-21swapper/100:36:231
777999195,4cyclictest2682-21/usr/sbin/munin20:10:171
777999192,14cyclictest434-40dnf21:15:581
7779991916,2cyclictest0-21swapper/120:35:451
7779991916,2cyclictest0-21swapper/120:35:451
777999191,5cyclictest11825-21hddtemp_smartct00:10:181
777999191,5cyclictest11825-21hddtemp_smartct00:10:181
7779991915,3cyclictest0-21swapper/119:21:251
777999191,3cyclictest0-21swapper/123:14:401
777999191,2cyclictest0-21swapper/119:58:411
777999191,17cyclictest0-21swapper/121:08:541
777999191,16cyclictest0-21swapper/122:25:381
777999191,16cyclictest0-21swapper/120:05:241
777999191,16cyclictest0-21swapper/119:28:031
777999191,16cyclictest0-21swapper/119:28:031
777499194,4cyclictest24067-21cpu19:45:120
777499193,4cyclictest13693-21cpu20:35:120
777499193,4cyclictest13693-21cpu20:35:120
777499191,6cyclictest8306-21/usr/sbin/munin22:50:180
777499191,6cyclictest0-21swapper/021:32:220
777499191,6cyclictest0-21swapper/020:05:180
777499191,5cyclictest0-21swapper/023:55:560
777499191,5cyclictest0-21swapper/000:25:100
779199187,3cyclictest801-21snmpd19:34:133
779199182,3cyclictest41-21ksoftirqd/320:50:013
779199181,8cyclictest0-21swapper/319:38:553
779199181,8cyclictest0-21swapper/319:29:173
779199181,8cyclictest0-21swapper/319:29:173
779199181,6cyclictest0-21swapper/320:50:213
779199181,4cyclictest0-21swapper/323:15:433
779199181,13cyclictest0-21swapper/320:21:173
779199181,11cyclictest0-21swapper/319:17:393
779199181,10cyclictest801-21snmpd20:10:193
779199181,10cyclictest13061-21/usr/sbin/munin19:20:243
7786991811,4cyclictest7169-21cpuspeed_turbos20:20:132
777999186,7cyclictest0-21swapper/122:06:301
7779991815,2cyclictest0-21swapper/119:15:161
7779991813,3cyclictest22899-21ntp_offset19:40:221
777999181,16cyclictest0-21swapper/120:49:021
777999181,16cyclictest0-21swapper/120:00:461
777999181,16cyclictest0-21swapper/119:46:101
777999181,14cyclictest11607-21cut20:30:131
777999180,17cyclictest0-21swapper/120:51:561
777999180,16cyclictest0-21swapper/119:51:021
777499187,2cyclictest0-21swapper/020:45:150
777499183,3cyclictest14988-21sshd19:24:380
777499181,5cyclictest10557-21switchtime20:25:250
777499181,5cyclictest0-21swapper/020:40:210
7774991814,3cyclictest0-21swapper/019:15:170
779199171,7cyclictest24652-21/usr/sbin/munin21:00:203
779199171,7cyclictest24431-21munin-run21:00:013
779199171,6cyclictest0-21swapper/320:40:213
779199171,3cyclictest0-21swapper/320:06:393
779199171,13cyclictest0-21swapper/320:32:403
779199170,10cyclictest0-21swapper/319:45:383
777999170,16cyclictest0-21swapper/120:41:491
777499171,6cyclictest0-21swapper/019:55:260
777499171,2cyclictest87150irq/128-enp1s0-20:30:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional