You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 08:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Aug 12, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
128082244169,23sleep20-21swapper/219:09:022
128862205167,25sleep10-21swapper/119:10:011
128582203166,25sleep00-21swapper/019:09:400
126812203167,24sleep30-21swapper/319:07:223
1992021850,2sleep01297499cyclictest21:50:120
3035821340,4sleep21298699cyclictest19:45:212
12986998269,7cyclictest0-21swapper/222:36:172
12986997268,3cyclictest0-21swapper/200:23:282
250632530,1sleep081ktimersoftd/022:00:230
250632530,1sleep081ktimersoftd/022:00:230
128512510,2sleep20-21swapper/221:35:012
1299299402,24cyclictest27074-21/usr/sbin/munin19:40:153
1299299394,30cyclictest31189-40dnf19:49:173
1299299394,30cyclictest13360-40dnf22:50:003
1299299392,28cyclictest21850-21sshd23:05:573
326052380,1sleep00-21swapper/023:30:250
1299299381,29cyclictest31192-21/usr/sbin/munin23:30:203
1299299381,22cyclictest755-21rngd22:25:143
1297499389,14cyclictest22182-21munin-run23:10:010
212112370,2sleep10-21swapper/119:25:171
1299299378,21cyclictest808-21snmpd22:20:463
1299299376,22cyclictest17382-21/usr/sbin/munin21:45:153
1299299354,18cyclictest87650irq/129-enp1s0-23:00:133
1299299351,31cyclictest17578-21ls00:10:233
1299299345,16cyclictest87350irq/126-enp1s0-22:15:173
12986993420,3cyclictest9004-21ntp_states20:10:222
1298699341,25cyclictest0-21swapper/223:48:152
1298699341,15cyclictest0-21swapper/221:36:032
1299299333,15cyclictest5689-21latency22:30:203
1299299331,5cyclictest808-21snmpd22:14:423
1299299331,23cyclictest20617-21munin-run00:20:013
1299299331,21cyclictest12365-21/usr/sbin/munin20:20:253
12986993320,8cyclictest0-21swapper/219:44:472
12986993317,3cyclictest0-21swapper/221:01:512
1298699331,22cyclictest0-21swapper/223:12:472
1299299325,19cyclictest24604-21/usr/sbin/munin23:15:273
1299299324,3cyclictest23487-21sshd21:56:473
1299299321,4cyclictest0-21swapper/321:40:453
1299299321,3cyclictest87650irq/129-enp1s0-21:25:023
1299299321,3cyclictest12050irq/123-ahci[0023:29:553
12986993219,8cyclictest0-21swapper/223:23:192
1298699321,19cyclictest0-21swapper/223:57:552
1298699321,15cyclictest0-21swapper/220:36:512
1298699321,14cyclictest0-21swapper/222:27:472
1299299314,3cyclictest1998-21/usr/sbin/munin21:10:253
1299299312,3cyclictest26798-21/usr/sbin/munin23:20:183
1299299311,7cyclictest0-21swapper/320:40:153
1299299311,4cyclictest25907-21diskstats20:50:143
1299299311,4cyclictest25266-40dnf20:49:233
1299299311,4cyclictest25266-40dnf20:49:233
1299299311,4cyclictest14406-21/usr/sbin/munin22:50:183
12992993114,14cyclictest827-21runrttasks19:30:143
1299299311,3cyclictest87350irq/126-enp1s0-23:10:203
1298699311,24cyclictest0-21swapper/200:18:232
1298699311,14cyclictest0-21swapper/221:56:472
1298699310,17cyclictest0-21swapper/221:28:472
1297499312,14cyclictest19984-21munin-run23:05:010
1299299304,3cyclictest1007-21/usr/sbin/munin23:35:213
1299299304,23cyclictest25023-21/usr/sbin/munin19:35:133
1299299304,23cyclictest25023-21/usr/sbin/munin19:35:133
1299299303,24cyclictest10954-21/usr/sbin/munin21:30:143
1299299302,3cyclictest12144-21munin-run20:20:023
1299299302,25cyclictest20456-21/usr/sbin/munin19:25:153
1299299301,4cyclictest16132-21sshd00:09:043
1299299301,4cyclictest13209-21/usr/sbin/munin00:00:243
1299299301,3cyclictest827-21runrttasks23:55:213
1299299301,3cyclictest13712-21/usr/sbin/munin21:35:173
12986993014,9cyclictest0-21swapper/222:14:232
1298699301,12cyclictest0-21swapper/222:59:022
1298699301,11cyclictest0-21swapper/219:26:152
1298699300,21cyclictest0-21swapper/221:07:392
1298699300,11cyclictest0-21swapper/200:01:312
1299299298,18cyclictest6792-21ntpq20:05:283
1299299296,20cyclictest9349-21cpuspeed_turbos22:40:113
1299299293,3cyclictest87650irq/129-enp1s0-20:10:213
1299299292,3cyclictest27433-21/usr/sbin/munin00:35:203
1299299291,6cyclictest9417-21kernelversion21:25:213
1299299291,4cyclictest32536-21/usr/sbin/munin19:50:183
1299299291,4cyclictest24110-21uname00:25:233
1299299291,4cyclictest24110-21uname00:25:233
1299299291,3cyclictest3871-21df20:00:173
1299299291,3cyclictest3231-21/usr/sbin/munin23:40:273
1299299291,3cyclictest25907-21/usr/sbin/munin00:30:163
1299299291,20cyclictest0-21swapper/320:34:233
12992992910,16cyclictest8077-21cut23:50:153
12986992917,3cyclictest0-21swapper/219:56:342
12986992917,3cyclictest0-21swapper/219:34:552
12986992915,7cyclictest0-21swapper/200:39:312
1298699291,20cyclictest0-21swapper/222:34:222
1298699291,14cyclictest0-21swapper/223:39:272
1298699291,14cyclictest0-21swapper/200:27:552
1298699291,14cyclictest0-21swapper/200:27:552
1298699291,13cyclictest0-21swapper/223:28:072
1298699291,12cyclictest0-21swapper/222:04:592
1298699291,12cyclictest0-21swapper/222:04:592
1298699291,12cyclictest0-21swapper/220:20:272
1298699291,10cyclictest23706-21diskstats20:45:152
1298699291,10cyclictest23706-21diskstats20:45:152
1299299289,16cyclictest7706-21sort22:35:183
1299299281,3cyclictest32162-21/usr/sbin/munin21:05:263
1299299281,3cyclictest27166-21sed22:05:203
1299299281,3cyclictest21945-21/usr/sbin/munin00:20:233
1299299281,2cyclictest29952-21/usr/sbin/munin21:00:183
1299299281,24cyclictest19582-21/usr/sbin/munin21:50:133
12992992811,14cyclictest6162-21dump-pmu-power21:20:013
12992992811,14cyclictest18682-21grep19:20:153
12992992810,15cyclictest827-21runrttasks20:57:463
12992992810,15cyclictest1309-21/usr/sbin/munin19:55:133
1298699289,3cyclictest0-21swapper/223:18:552
12986992816,3cyclictest0-21swapper/222:53:342
12986992815,8cyclictest0-21swapper/219:37:552
12986992815,8cyclictest0-21swapper/219:37:552
12986992813,9cyclictest0-21swapper/221:10:352
1298699281,25cyclictest0-21swapper/219:13:382
1298699281,15cyclictest0-21swapper/220:50:292
1298699281,13cyclictest0-21swapper/222:06:432
1297499280,13cyclictest26849-21munin-run19:40:010
1297499280,13cyclictest26849-21munin-run19:40:000
1299299276,18cyclictest19387-21fschecks_count20:35:173
1299299271,4cyclictest15866-21/usr/sbin/munin22:55:123
1299299271,3cyclictest5610-21/usr/sbin/munin23:45:153
1299299271,3cyclictest15242-21hddtemp_smartct20:25:173
1299299271,3cyclictest13170-21/usr/sbin/munin19:10:153
1299299271,2cyclictest17129-21ntp_states19:15:213
12986992715,3cyclictest0-21swapper/220:08:382
1298699271,19cyclictest0-21swapper/219:51:392
1298699271,12cyclictest0-21swapper/200:05:252
1298699271,11cyclictest0-21swapper/200:30:352
12986992711,10cyclictest0-21swapper/219:19:412
1297499271,13cyclictest2396-21munin-run22:25:010
1299299261,4cyclictest24145-21/usr/sbin/munin22:00:143
1299299261,4cyclictest24145-21/usr/sbin/munin22:00:143
1298699269,11cyclictest0-21swapper/223:09:152
1298699268,8cyclictest0-21swapper/220:56:432
12986992615,3cyclictest0-21swapper/221:53:232
1298699261,18cyclictest0-21swapper/223:40:552
1298699261,13cyclictest0-21swapper/223:30:472
1298699261,11cyclictest0-21swapper/223:53:152
1298699261,11cyclictest0-21swapper/222:45:462
1298699260,18cyclictest0-21swapper/222:17:472
1297499268,15cyclictest31937-21munin-run21:05:010
1297499260,12cyclictest31279-21perf19:50:010
12986992515,3cyclictest0-21swapper/221:22:192
12986992513,3cyclictest18355-21users20:30:272
1298699251,18cyclictest0-21swapper/222:24:552
1298699251,11cyclictest0-21swapper/221:18:552
12986992510,12cyclictest15394-21kernelversion20:25:192
12981992514,9cyclictest0-21swapper/121:54:201
1298699249,3cyclictest0-21swapper/223:00:032
12986992415,3cyclictest0-21swapper/220:00:062
1298699241,13cyclictest0-21swapper/222:42:182
1298699240,10cyclictest0-21swapper/220:19:152
1298199241,5cyclictest0-21swapper/120:49:281
1298199241,5cyclictest0-21swapper/120:49:281
1297499241,9cyclictest21906-21kworker/0:123:14:070
1298699237,10cyclictest0-21swapper/200:14:272
12986992312,7cyclictest0-21swapper/221:49:352
1298699231,12cyclictest0-21swapper/220:42:232
1298699230,16cyclictest0-21swapper/221:40:032
1298199231,8cyclictest0-21swapper/123:43:441
12981992310,4cyclictest15380-21irqstats20:25:191
1298699221,10cyclictest0-21swapper/219:21:452
12981992217,3cyclictest0-21swapper/123:06:111
12981992213,2cyclictest9388-21cstates22:40:121
1298199221,18cyclictest15304-21ntp_states00:05:241
1298199221,16cyclictest0-21swapper/123:56:181
1298199221,15cyclictest0-21swapper/123:20:151
1298199221,15cyclictest0-21swapper/100:25:311
1298199221,15cyclictest0-21swapper/100:25:311
1298199221,12cyclictest32166-21users22:15:261
1298199221,11cyclictest11518-21grep22:45:121
1298199220,16cyclictest0-21swapper/100:15:131
1297499224,15cyclictest23144-21awk20:45:010
1297499221,8cyclictest29450-21kworker/0:122:11:510
180192210,2sleep118020-21grep21:45:151
1298199219,3cyclictest7887-21grep22:35:191
1298199219,3cyclictest15528-21df_inode21:40:131
1298199213,15cyclictest18734-21ls20:35:011
1298199212,10cyclictest17152-21idleruntime00:10:171
1298199211,8cyclictest0-21swapper/121:24:101
1298199211,3cyclictest0-21swapper/123:50:161
12981992113,7cyclictest8447-21kworker/u8:120:13:171
12981992113,2cyclictest0-21swapper/123:31:161
1298199211,17cyclictest13170-21/usr/sbin/munin19:10:121
1298199211,15cyclictest0-21swapper/123:15:111
1298199211,14cyclictest0-21swapper/123:25:221
1298199211,11cyclictest1947-21memory23:35:191
1298199211,11cyclictest14827-21/usr/sbin/munin22:50:241
1298199211,10cyclictest23134-21munin-run20:45:011
12981992110,3cyclictest4592-21munin-run22:30:021
1298199210,15cyclictest0-21swapper/100:35:161
1297499219,3cyclictest10886-21ntp_states23:55:230
1297499212,7cyclictest8671-21kworker/0:123:52:390
1297499211,8cyclictest32226-21kworker/0:022:17:270
1298199208,11cyclictest21628-21kworker/u8:220:04:501
1298199204,6cyclictest23651-21strings19:30:201
1298199201,7cyclictest30799-21grep19:45:241
12981992013,2cyclictest0-21swapper/122:31:021
12981992012,3cyclictest0-21swapper/123:45:191
1298199201,18cyclictest0-21swapper/100:04:161
1298199201,15cyclictest0-21swapper/100:21:221
1298199201,11cyclictest31023-21ntp_states21:00:231
1298199201,11cyclictest13285-21memory20:20:191
1298199201,10cyclictest26093-21cat22:05:001
1298199201,10cyclictest26093-21cat22:05:001
12981992010,2cyclictest5456-21smart_sda21:15:241
1297499207,4cyclictest0-21swapper/021:32:120
1297499204,4cyclictest0-21swapper/023:35:510
1297499202,8cyclictest467-21kworker/0:200:20:190
1297499201,9cyclictest0-21swapper/021:29:090
1297499201,8cyclictest1371750systemd-journal00:01:210
1297499201,11cyclictest18964-21/usr/sbin/munin20:35:260
1297499201,11cyclictest13608-21/usr/sbin/munin22:50:230
1297499201,10cyclictest16256-21ntp_states21:40:210
1297499201,10cyclictest0-21swapper/021:45:020
1297499201,10cyclictest0-21swapper/000:25:080
1297499201,10cyclictest0-21swapper/000:25:080
12974992010,3cyclictest23759-21awk22:00:010
1297499200,9cyclictest16170-21grep22:55:150
1298199198,4cyclictest29076-21awk19:45:001
1298199198,3cyclictest7001-21smart_sda20:05:321
1298199193,5cyclictest29309-21grep20:55:261
1298199192,9cyclictest0-21swapper/121:55:251
1298199191,9cyclictest3301-21munin-run20:00:011
1298199191,8cyclictest0-21swapper/121:27:221
12981991915,3cyclictest0-21swapper/123:01:191
12981991915,3cyclictest0-21swapper/121:08:241
12981991915,3cyclictest0-21swapper/120:35:501
1298199191,4cyclictest19371-21grep19:20:221
12981991914,4cyclictest0-21swapper/121:30:251
1298199191,2cyclictest0-21swapper/120:17:211
1298199191,16cyclictest0-21swapper/119:50:461
1298199191,13cyclictest0-21swapper/121:13:031
1298199191,10cyclictest661-21df22:20:121
1298199191,10cyclictest26855-21awk19:40:011
1298199191,10cyclictest26855-21awk19:40:001
1298199191,10cyclictest17137-21/usr/sbin/munin22:55:251
1298199190,14cyclictest0-21swapper/123:10:191
1298199190,14cyclictest0-21swapper/100:32:571
1297499197,3cyclictest0-21swapper/021:35:280
1297499196,6cyclictest0-21swapper/022:25:180
1297499195,6cyclictest0-21swapper/021:17:350
1297499195,5cyclictest0-21swapper/022:31:410
1297499193,4cyclictest167172sed19:15:160
1297499191,6cyclictest0-21swapper/000:30:200
1297499191,11cyclictest5427-21/usr/sbin/munin23:45:170
1297499191,11cyclictest0-21swapper/023:40:280
1297499191,11cyclictest0-21swapper/021:22:440
1297499191,11cyclictest0-21swapper/019:30:210
1297499191,10cyclictest7007-21/usr/sbin/munin22:35:260
1297499191,10cyclictest1769-21diskmemload23:29:470
1297499191,10cyclictest14153-21sshd20:22:450
1297499191,10cyclictest0-21swapper/000:13:340
1297499190,6cyclictest0-21swapper/022:40:350
1297499190,5cyclictest0-21swapper/000:08:280
54532181,2sleep00-21swapper/020:04:300
1298199188,3cyclictest0-21swapper/122:05:121
12981991815,2cyclictest0-21swapper/121:36:451
12981991815,2cyclictest0-21swapper/120:51:441
1298199181,16cyclictest0-21swapper/122:10:431
1298199181,16cyclictest0-21swapper/119:17:021
1297499188,6cyclictest0-21swapper/020:49:560
1297499188,6cyclictest0-21swapper/020:49:560
1297499186,4cyclictest0-21swapper/022:45:090
1297499184,8cyclictest0-21swapper/022:06:170
1297499184,4cyclictest1782-21awk21:10:010
1297499183,4cyclictest808-21snmpd20:54:280
1297499181,9cyclictest819-21lldpd20:58:290
1297499181,9cyclictest19894-21sendmail_mailtr00:15:250
1297499181,15cyclictest12155-21perf20:20:020
1297499181,11cyclictest0-21swapper/000:35:110
1297499181,10cyclictest0-21swapper/023:15:040
1297499180,6cyclictest0-21swapper/023:20:520
1297499176,3cyclictest12050irq/123-ahci[0019:45:010
1297499171,7cyclictest13170-21/usr/sbin/munin19:10:220
1297499171,6cyclictest2913-21memory21:10:190
1297499171,4cyclictest14570-21/usr/sbin/munin20:25:190
12974991712,3cyclictest9251-21sed20:10:270
12974991712,3cyclictest5742-21/usr/sbin/munin20:05:310
12974991712,3cyclictest18259-21/usr/sbin/munin19:20:190
1297499165,3cyclictest0-21swapper/020:30:210
12974991611,3cyclictest603-21head19:50:240
12974991611,3cyclictest11-21rcuc/019:26:160
1297499161,10cyclictest0-21swapper/019:55:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional