You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 11:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Aug 11, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72722236162,25sleep00-21swapper/019:06:270
55842236163,24sleep10-21swapper/119:05:031
73392233163,56sleep30-21swapper/319:07:193
73852226155,57sleep20-21swapper/219:07:552
2660421790,4sleep3765499cyclictest21:05:013
2914121600,5sleep2764799cyclictest19:55:172
928421420,3sleep0763799cyclictest00:05:170
118522950,4sleep2764799cyclictest00:10:172
7647999385,7cyclictest0-21swapper/220:32:252
172912660,1sleep00-21swapper/020:40:260
235922600,3sleep323596-21fschecks_time22:10:143
288482520,9sleep121-21rcuc/123:35:161
115952520,2sleep20-21swapper/219:15:202
214822490,5sleep2764799cyclictest22:05:172
214822490,5sleep2764799cyclictest22:05:162
199972480,2sleep20-21swapper/223:15:192
199972480,2sleep20-21swapper/223:15:192
47192400,2sleep00-21swapper/020:15:020
7647993936,2cyclictest0-21swapper/221:14:112
763799378,13cyclictest13402-21munin-run20:35:000
763799369,13cyclictest17509-21munin-run00:25:010
765499354,22cyclictest28794-21perf21:10:013
765499354,17cyclictest14576-21df21:50:133
765499335,20cyclictest23644-21unixbench_multi00:35:263
7647993330,2cyclictest0-21swapper/222:58:382
764799331,12cyclictest0-21swapper/223:43:412
765499326,16cyclictest26005-21if_err_enp1s022:15:173
765499314,15cyclictest16544-21tr21:55:113
765499312,19cyclictest29136-21cpuspeed_turbos21:10:123
765499311,11cyclictest0-21swapper/321:40:233
764199312,16cyclictest0-21swapper/119:38:591
158612310,4sleep2764799cyclictest23:05:192
765499309,11cyclictest2104-21/usr/sbin/munin23:50:013
764799301,23cyclictest0-21swapper/220:24:012
764799301,20cyclictest0-21swapper/220:17:492
764799301,12cyclictest0-21swapper/200:06:572
763799304,13cyclictest9010-21munin-run20:25:010
765499299,10cyclictest12692-21latency_hist19:20:013
765499297,9cyclictest1601-21fschecks_time21:20:173
765499295,14cyclictest2628-21/usr/sbin/munin20:10:163
765499294,17cyclictest5211-21latency_hist21:30:023
765499294,15cyclictest0-21swapper/323:40:233
764799291,16cyclictest0-21swapper/223:46:572
764199291,22cyclictest0-21swapper/122:24:211
763799294,12cyclictest5890-21awk22:45:010
765499289,5cyclictest26920-21ntp_kernel_err23:30:213
765499287,18cyclictest21625-21tr23:20:133
765499283,10cyclictest20278-21cpuspeed_turbos20:50:123
764799281,13cyclictest0-21swapper/222:36:172
764799281,13cyclictest0-21swapper/222:29:092
765499277,6cyclictest13718-21/usr/sbin/munin23:00:193
765499273,15cyclictest22410-21/usr/sbin/munin20:55:223
765499273,15cyclictest22410-21/usr/sbin/munin20:55:223
764799270,11cyclictest0-21swapper/220:25:232
765499266,8cyclictest15858-21missed_timers23:05:193
765499265,3cyclictest87350irq/126-enp1s0-00:30:253
7647992622,3cyclictest0-21swapper/219:28:352
764799261,11cyclictest0-21swapper/220:37:592
764799261,11cyclictest0-21swapper/219:45:042
764799261,10cyclictest0-21swapper/220:40:032
764199264,16cyclictest4924-40dnf22:40:221
764199261,16cyclictest3589-21diskstats21:25:121
7641992611,12cyclictest0-21swapper/121:30:261
763799269,14cyclictest30999-21basename21:15:020
765499257,16cyclictest3407-21sshd22:36:473
765499255,10cyclictest27472-21cpu22:20:133
765499253,5cyclictest2323-21/usr/sbin/munin23:50:143
765499253,13cyclictest10686-21cstates22:55:133
765499251,17cyclictest0-21swapper/323:55:243
765499251,12cyclictest0-21swapper/300:00:193
764799250,9cyclictest0-21swapper/200:33:492
764799250,11cyclictest0-21swapper/222:32:332
764799250,10cyclictest0-21swapper/222:44:172
764799250,10cyclictest0-21swapper/200:02:452
764199259,4cyclictest0-21swapper/122:06:541
764199259,4cyclictest0-21swapper/122:06:541
764199251,15cyclictest0-21swapper/100:05:241
763799256,16cyclictest1459-21munin-run22:35:010
763799250,12cyclictest4311-21idleruntime-cro23:55:010
237162250,4sleep123720-21crond19:45:011
765499246,5cyclictest808-21snmpd21:49:373
765499246,10cyclictest6147-21cpu22:45:143
765499243,14cyclictest30658-21netstat22:25:213
765499243,10cyclictest477-21sh22:30:213
765499242,13cyclictest17663-21diskstats19:30:153
764799241,9cyclictest0-21swapper/219:14:172
764799241,9cyclictest0-21swapper/200:25:032
764799241,9cyclictest0-21swapper/200:25:032
764799241,8cyclictest0-21swapper/220:47:152
764799240,10cyclictest0-21swapper/200:00:022
764199242,4cyclictest21907-21munin-run00:35:011
764199241,13cyclictest23385-21diskstats22:10:131
764199241,10cyclictest0-21swapper/123:12:491
765499237,13cyclictest16409-21grep20:40:163
765499236,5cyclictest87550irq/128-enp1s0-00:15:213
765499235,6cyclictest17733-21sed23:10:163
765499233,12cyclictest10371-21date22:55:013
7654992315,5cyclictest24423-21cut19:45:143
765499231,10cyclictest0-21swapper/323:25:293
764799238,3cyclictest0-21swapper/220:07:352
764799231,8cyclictest0-21swapper/222:52:172
764799231,6cyclictest0-21swapper/223:02:522
764799230,9cyclictest0-21swapper/200:37:492
764799230,9cyclictest0-21swapper/200:18:172
764799230,8cyclictest0-21swapper/222:04:222
764199231,6cyclictest0-21swapper/123:29:351
7641992314,2cyclictest0-21swapper/120:00:131
764199231,15cyclictest0-21swapper/100:10:021
764199230,15cyclictest0-21swapper/122:50:441
763799231,10cyclictest0-21swapper/020:55:140
763799231,10cyclictest0-21swapper/020:55:140
164692231,18sleep016762-21expr21:55:110
765499226,10cyclictest808-21snmpd00:21:073
765499224,9cyclictest5864-21uniq21:30:173
765499224,15cyclictest28784-21diskmemload22:05:173
765499224,15cyclictest28784-21diskmemload22:05:173
765499223,9cyclictest9570-21gltestperf00:05:203
765499223,15cyclictest87350irq/126-enp1s0-22:01:593
765499222,11cyclictest0-21swapper/300:10:083
7654992212,7cyclictest31562-21diskstats21:15:173
765499221,16cyclictest87350irq/126-enp1s0-23:36:043
764799228,4cyclictest27166-21diskstats21:05:152
764799228,3cyclictest0-21swapper/221:46:592
764799228,3cyclictest0-21swapper/219:34:112
764799220,8cyclictest0-21swapper/222:14:052
764799220,11cyclictest0-21swapper/221:18:512
764199229,3cyclictest0-21swapper/122:00:111
764199227,9cyclictest0-21swapper/121:40:081
764199223,12cyclictest1432-21kworker/u8:223:42:141
764199222,9cyclictest0-21swapper/123:05:131
764199222,17cyclictest21-21rcuc/120:45:131
764199221,6cyclictest0-21swapper/123:51:351
764199221,15cyclictest16705-21kworker/1:221:15:361
764199221,15cyclictest0-21swapper/119:29:361
764199221,11cyclictest790-21in:imjournal19:15:021
764199221,11cyclictest0-21swapper/123:23:171
764199221,11cyclictest0-21swapper/123:03:541
764199221,10cyclictest19231-21grep00:25:241
764199221,10cyclictest19231-21grep00:25:241
764199220,13cyclictest0-21swapper/122:48:571
763799221,9cyclictest26880-21grep23:30:200
763799221,9cyclictest154512grep00:20:010
763799221,10cyclictest19233-21grep22:00:160
765499215,5cyclictest7956-21cpuspeed_turbos19:10:133
765499214,14cyclictest19369-21taskset00:25:513
765499214,14cyclictest19369-21taskset00:25:513
765499214,14cyclictest10104-21kworker/u8:123:15:273
765499214,14cyclictest10104-21kworker/u8:123:15:273
765499213,7cyclictest4957-21ntpq22:40:213
7654992114,4cyclictest26994-21latency19:50:183
7654992113,5cyclictest19279-21smart_sda20:45:253
7654992112,6cyclictest29409-21ntp_states19:55:213
7654992112,5cyclictest22804-21ntp_states19:40:223
765499211,12cyclictest12050irq/123-ahci[0019:35:173
764799217,3cyclictest0-21swapper/220:14:392
764199211,7cyclictest802-21sssd_be23:30:231
764199211,5cyclictest0-21swapper/122:59:051
7641992113,3cyclictest0-21swapper/120:34:531
7641992113,2cyclictest0-21swapper/121:00:211
764199211,15cyclictest0-21swapper/121:06:051
764199211,14cyclictest0-21swapper/121:20:031
764199211,14cyclictest0-21swapper/120:41:361
764199211,11cyclictest20560-21diskstats20:50:141
7641992110,3cyclictest11451-21/usr/sbin/munin19:15:181
764199210,16cyclictest0-21swapper/119:51:341
763799211,7cyclictest0-21swapper/000:31:280
763799211,6cyclictest0-21swapper/023:36:170
763799211,6cyclictest0-21swapper/022:17:440
763799211,6cyclictest0-21swapper/022:14:220
326202210,10sleep339-21rcuc/320:05:013
765499208,2cyclictest9356-21kworker/u8:221:38:263
765499202,15cyclictest11421-21/usr/sbin/munin20:30:133
7654992012,6cyclictest5186-21/usr/sbin/munin20:15:173
7654992012,5cyclictest13271-21diskstats19:20:153
7654992012,5cyclictest10295-21ntp_states20:25:203
7654992011,6cyclictest13971-21/usr/sbin/munin20:35:143
765499201,11cyclictest808-21snmpd19:29:453
764799205,3cyclictest0-21swapper/221:00:192
764799201,5cyclictest0-21swapper/222:16:282
764799201,4cyclictest0-21swapper/223:39:082
764799201,10cyclictest5667-21expr21:30:152
764199209,3cyclictest26822-21users22:15:261
764199208,3cyclictest0-21swapper/121:58:591
764199202,9cyclictest0-21swapper/121:13:171
7641992016,3cyclictest0-21swapper/123:19:541
7641992016,3cyclictest0-21swapper/123:19:541
7641992015,3cyclictest0-21swapper/121:49:151
7641992013,2cyclictest0-21swapper/120:36:181
764199201,17cyclictest0-21swapper/122:30:091
764199201,17cyclictest0-21swapper/119:58:191
764199201,14cyclictest0-21swapper/120:58:401
764199201,14cyclictest0-21swapper/120:58:401
764199201,14cyclictest0-21swapper/120:28:431
764199201,14cyclictest0-21swapper/120:14:431
764199201,14cyclictest0-21swapper/119:32:511
764199201,14cyclictest0-21swapper/119:21:581
764199201,11cyclictest24300-21diskstats19:45:131
764199201,11cyclictest1174-21ntp_states23:45:231
764199201,11cyclictest0-21swapper/100:38:181
764199201,10cyclictest0-21swapper/121:37:131
763799209,3cyclictest30284-21/usr/sbin/munin23:40:170
763799201,7cyclictest0-21swapper/022:05:070
763799201,7cyclictest0-21swapper/022:05:070
763799201,6cyclictest0-21swapper/022:47:270
763799201,6cyclictest0-21swapper/022:36:050
763799201,6cyclictest0-21swapper/021:38:570
763799201,5cyclictest0-21swapper/021:53:480
763799201,16cyclictest31820-21latency_hist22:30:020
763799201,12cyclictest0-21swapper/019:25:140
763799201,12cyclictest0-21swapper/000:35:280
47042201,15sleep04830-21users21:25:260
7654991912,4cyclictest1645-21/usr/sbin/munin20:05:233
7654991910,6cyclictest8294-21smart_sda20:20:253
764799190,10cyclictest0-21swapper/222:20:492
764199192,9cyclictest7791-21ntpq00:00:231
7641991913,2cyclictest0-21swapper/120:20:501
764199191,2cyclictest0-21swapper/100:22:551
7641991912,2cyclictest0-21swapper/120:15:131
764199191,16cyclictest0-21swapper/100:17:421
764199191,10cyclictest15503-21smart_sda21:50:241
764199190,17cyclictest0-21swapper/122:25:161
764199190,14cyclictest0-21swapper/120:07:561
763799199,2cyclictest349-21cpu20:05:120
763799191,6cyclictest0-21swapper/023:49:040
763799191,6cyclictest0-21swapper/023:29:430
763799191,5cyclictest0-21swapper/022:59:130
763799191,4cyclictest0-21swapper/023:57:480
7637991913,3cyclictest6486-21latency_hist00:00:020
763799191,10cyclictest17732-21/usr/sbin/munin00:25:160
763799191,10cyclictest17732-21/usr/sbin/munin00:25:160
763799191,10cyclictest17102-21/usr/sbin/munin23:10:170
763799191,10cyclictest0-21swapper/023:24:190
55062191,2sleep00-21swapper/020:15:190
172572191,2sleep20-21swapper/221:55:182
764799185,3cyclictest0-21swapper/223:30:512
764799182,8cyclictest0-21swapper/223:26:562
764799182,7cyclictest1027-21/usr/sbin/munin21:20:202
764799182,3cyclictest0-21swapper/219:40:162
764799181,8cyclictest0-21swapper/220:51:492
764799181,4cyclictest0-21swapper/221:50:442
764799181,11cyclictest0-21swapper/223:50:362
764799181,11cyclictest0-21swapper/223:10:402
764199186,3cyclictest909-21sssd_nss22:35:211
764199181,16cyclictest0-21swapper/123:55:401
763799188,2cyclictest0-21swapper/019:36:460
763799187,3cyclictest87350irq/126-enp1s0-20:00:160
763799187,3cyclictest32057-21kernelversion21:15:220
763799185,3cyclictest0-21swapper/019:42:560
763799181,9cyclictest0-21swapper/020:35:240
763799181,7cyclictest6361-21memory21:30:220
763799181,5cyclictest0-21swapper/023:16:480
763799181,5cyclictest0-21swapper/023:16:470
763799181,5cyclictest0-21swapper/000:12:300
763799181,4cyclictest0-21swapper/021:22:400
7637991812,4cyclictest28781-21crond21:10:010
763799181,11cyclictest0-21swapper/021:49:590
764799174,4cyclictest21484-21latency_hist23:20:012
764799173,5cyclictest0-21swapper/221:35:112
764799171,7cyclictest0-21swapper/219:53:542
764799171,15cyclictest0-21swapper/220:00:422
764799171,11cyclictest0-21swapper/219:39:102
764799170,12cyclictest0-21swapper/200:20:532
763799175,4cyclictest13994-21ntp_states19:20:240
763799171,7cyclictest26793-21cat21:05:010
763799171,7cyclictest0-21swapper/019:55:120
7637991712,3cyclictest21724-21grep20:50:260
763799171,14cyclictest0-21swapper/019:15:080
763799171,13cyclictest27577-21tr22:20:130
763799171,13cyclictest18016-21/usr/sbin/munin20:45:130
763799171,10cyclictest0-21swapper/023:00:050
763799170,5cyclictest10340-21fschecks_time21:40:150
763799170,15cyclictest0-21swapper/019:34:130
764799161,2cyclictest0-21swapper/221:25:362
764799161,14cyclictest0-21swapper/219:22:512
7647991611,3cyclictest6729-21gltestperf22:45:182
763799161,7cyclictest0-21swapper/022:50:100
763799161,5cyclictest0-21swapper/019:46:460
763799161,5cyclictest0-21swapper/019:10:110
7637991612,3cyclictest0-21swapper/020:29:400
7637991611,3cyclictest15970-21sh23:05:220
7647991512,2cyclictest0-21swapper/221:43:122
764799150,14cyclictest0-21swapper/220:56:542
764799150,14cyclictest0-21swapper/220:56:542
763799159,5cyclictest0-21swapper/019:53:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional