You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 01:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Fri Apr 03, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151952240163,24sleep30-21swapper/307:06:433
153732239162,25sleep00-21swapper/007:09:030
153842203166,24sleep10-21swapper/107:09:121
151722201165,24sleep20-21swapper/207:06:262
15545991091,106cyclictest0-21swapper/210:41:492
15545997673,2cyclictest0-21swapper/210:47:092
287432660,11sleep011-21rcuc/012:30:180
114942580,3sleep339-21rcuc/308:10:203
114942580,3sleep339-21rcuc/308:10:203
305622510,4sleep011-21rcuc/008:55:130
171752450,3sleep117179-21cpuspeed_turbos08:25:111
15762400,1sleep10-21swapper/111:30:131
1554999334,15cyclictest0-21swapper/311:35:143
1554999311,23cyclictest0-21swapper/310:27:343
1553299315,12cyclictest24039-21perf09:55:010
1554999291,20cyclictest0-21swapper/310:36:263
1554999291,19cyclictest0-21swapper/311:40:053
1554999291,19cyclictest0-21swapper/311:40:053
15532992911,15cyclictest15859-21idleruntime-cro10:50:020
1554999281,19cyclictest0-21swapper/311:15:083
1554999273,18cyclictest856-21kworker/u8:012:20:013
15549992713,8cyclictest0-21swapper/311:59:443
15549992713,5cyclictest0-21swapper/310:07:113
1554999271,18cyclictest0-21swapper/311:52:353
1554999271,18cyclictest0-21swapper/309:51:123
1554599271,11cyclictest0-21swapper/212:32:562
1553299275,10cyclictest17440-21perf09:40:020
1553299271,23cyclictest18270-21/usr/sbin/munin10:55:200
1554999261,19cyclictest0-21swapper/309:34:213
1554999261,17cyclictest0-21swapper/310:02:463
1554999261,17cyclictest0-21swapper/309:13:273
1554999261,16cyclictest39-21rcuc/310:10:173
1553299268,3cyclictest11394-21kworker/0:211:22:260
164162250,5sleep011-21rcuc/008:20:240
1554999257,9cyclictest15449-21/usr/sbin/munin09:35:263
15549992521,2cyclictest87350irq/126-enp1s0-09:46:533
1554999251,18cyclictest0-21swapper/312:37:063
1554999251,18cyclictest0-21swapper/311:32:053
1554999251,18cyclictest0-21swapper/311:05:213
1554999251,18cyclictest0-21swapper/310:47:393
1554999251,18cyclictest0-21swapper/310:32:413
1554999251,18cyclictest0-21swapper/310:32:413
1554999251,17cyclictest0-21swapper/310:52:223
1554999251,16cyclictest0-21swapper/311:26:303
1554999251,10cyclictest0-21swapper/311:03:043
1554999250,17cyclictest0-21swapper/310:56:403
1553899251,12cyclictest0-21swapper/112:38:131
1553299253,10cyclictest28654-21/usr/sbin/munin10:05:240
1553299253,10cyclictest22807-21awk07:25:010
1553299251,21cyclictest17649-21/usr/sbin/munin09:40:210
1554999243,14cyclictest0-21swapper/312:00:433
15549992417,2cyclictest0-21swapper/312:08:523
1554999241,17cyclictest0-21swapper/310:40:303
1554999241,16cyclictest0-21swapper/311:24:243
1554999241,16cyclictest0-21swapper/309:26:543
1554999241,15cyclictest0-21swapper/309:55:043
1554599240,8cyclictest0-21swapper/212:03:342
1553899248,14cyclictest14256-21kworker/u8:008:44:141
1553299245,6cyclictest87350irq/126-enp1s0-09:20:210
1553299244,3cyclictest16157-21kworker/0:112:10:210
1553299241,20cyclictest9475-21/usr/sbin/munin10:35:130
1553299241,20cyclictest808-21snmpd10:17:540
1553299241,20cyclictest3841-21/usr/sbin/munin07:55:180
1553299241,11cyclictest15734-21/usr/sbin/munin07:10:160
1553299241,11cyclictest10093-21/usr/sbin/munin11:50:180
1554999232,15cyclictest0-21swapper/311:45:243
1554999231,9cyclictest0-21swapper/309:17:473
1554999231,17cyclictest0-21swapper/312:10:133
1554999231,17cyclictest0-21swapper/310:23:393
1554999231,16cyclictest0-21swapper/312:21:503
15549992311,6cyclictest0-21swapper/309:43:123
1554999231,16cyclictest0-21swapper/309:23:253
15538992317,3cyclictest28028-21df_inode12:30:131
15538992317,3cyclictest15157-21hddtemp_smartct12:00:151
1553899231,19cyclictest8536-21chrt09:18:581
1553899231,12cyclictest0-21swapper/111:51:411
1553899231,11cyclictest3619-21munin-run07:55:011
1553899231,11cyclictest0-21swapper/107:46:501
1553899230,16cyclictest0-21swapper/107:28:101
1553299239,5cyclictest81ktimersoftd/010:25:180
1553299239,3cyclictest24247-21/usr/sbin/munin09:55:140
1553299237,14cyclictest26833-21crond11:15:020
1553299233,6cyclictest12416-21munin-run08:15:010
1553299233,6cyclictest12416-21munin-run08:15:000
1553299232,9cyclictest11-21rcuc/011:15:140
1553299231,11cyclictest22675-21/usr/sbin/munin11:05:180
1553299231,11cyclictest13252-21/usr/sbin/munin09:30:180
1553299231,10cyclictest5703-21/usr/sbin/munin11:40:210
1553299231,10cyclictest5703-21/usr/sbin/munin11:40:210
1554999228,3cyclictest87350irq/126-enp1s0-08:35:023
1554999221,15cyclictest0-21swapper/312:34:083
1553899227,4cyclictest0-21swapper/110:13:051
1553899221,4cyclictest196082sort08:30:141
1553899221,3cyclictest8935-21cpuspeed_turbos09:20:121
1553899221,3cyclictest24032-21perf09:55:021
1553899221,3cyclictest0-21swapper/111:58:271
1553899221,18cyclictest7984-21sshd08:04:431
1553899221,15cyclictest0-21swapper/111:00:171
1553899221,15cyclictest0-21swapper/110:23:181
1553899221,15cyclictest0-21swapper/107:21:131
1553899221,14cyclictest0-21swapper/112:23:091
1553899221,12cyclictest16338-21kworker/u8:112:06:231
1553899221,11cyclictest0-21swapper/109:46:301
1553299228,3cyclictest808-21snmpd08:27:350
1553299222,6cyclictest18050-21munin-run10:55:020
1553299221,7cyclictest0-21swapper/012:15:100
15532992210,3cyclictest87550irq/128-enp1s0-07:35:160
1553299220,9cyclictest12289-21/usr/sbin/munin11:55:250
1554999211,7cyclictest20819-21/usr/sbin/munin07:20:213
15549992115,3cyclictest22035-21grep08:35:153
1554999211,14cyclictest0-21swapper/312:27:563
1554999211,14cyclictest0-21swapper/308:45:233
1554999211,13cyclictest0-21swapper/311:10:153
1554999211,11cyclictest28023-21/usr/sbin/munin08:50:153
1554599216,5cyclictest5781-21cat11:40:152
1554599216,5cyclictest5781-21cat11:40:142
1554599211,11cyclictest579-21systemd-journal12:40:012
1553899218,3cyclictest0-21swapper/111:13:191
15538992117,3cyclictest0-21swapper/112:26:481
15538992114,4cyclictest581-21grep09:00:151
1553899211,18cyclictest0-21swapper/109:55:241
1553899211,14cyclictest0-21swapper/110:36:411
1553899211,14cyclictest0-21swapper/110:03:011
1553899211,14cyclictest0-21swapper/107:12:191
1553899211,12cyclictest0-21swapper/112:19:441
1553899211,11cyclictest4570-21ntp_states11:35:211
1553899211,11cyclictest13896-21smart_sda08:15:231
1553899211,11cyclictest0-21swapper/111:50:001
1553899211,11cyclictest0-21swapper/110:50:531
1553899211,11cyclictest0-21swapper/110:43:271
15538992110,2cyclictest6341-21smart_sda10:25:221
15538992110,2cyclictest12124-21ntp_states09:25:211
1553899210,5cyclictest0-21swapper/111:20:251
1553899210,15cyclictest0-21swapper/107:42:231
1553899210,11cyclictest0-21swapper/109:44:131
1553299218,3cyclictest87550irq/128-enp1s0-10:20:240
1553299212,16cyclictest18675-21munin-run12:10:010
1553299211,11cyclictest11-21rcuc/011:25:150
1553299211,10cyclictest7827-21sshd08:02:430
1553299211,10cyclictest14624-21perf08:20:000
1553299211,10cyclictest0-21swapper/010:42:590
1554999206,9cyclictest16464-40dnf08:20:453
1554999203,8cyclictest26936-21chrt07:31:563
1554999202,9cyclictest2314-21hddtemp_smartct07:50:143
1554999201,13cyclictest0-21swapper/310:16:473
1554999201,12cyclictest0-21swapper/308:55:203
1554999201,11cyclictest0-21swapper/308:26:193
1554599201,6cyclictest10444-21users09:20:242
1554599201,6cyclictest0-21swapper/210:36:552
1554599201,5cyclictest0-21swapper/212:27:102
1554599201,5cyclictest0-21swapper/210:21:422
1554599201,12cyclictest0-21swapper/210:30:212
1554599201,12cyclictest0-21swapper/210:30:212
1553899209,2cyclictest0-21swapper/110:45:251
1553899201,3cyclictest0-21swapper/109:37:301
15538992013,2cyclictest0-21swapper/110:34:311
15538992013,2cyclictest0-21swapper/110:34:311
15538992012,3cyclictest0-21swapper/107:18:521
1553899201,18cyclictest0-21swapper/111:26:481
1553899201,17cyclictest0-21swapper/108:56:141
1553899201,17cyclictest0-21swapper/108:20:211
1553899201,11cyclictest1759-21open_files10:15:211
1553899201,10cyclictest27884-21cat08:50:011
1553899201,10cyclictest0-21swapper/110:57:361
1553899200,18cyclictest87650irq/129-enp1s0-09:30:131
1553899200,14cyclictest0-21swapper/108:11:381
1553899200,14cyclictest0-21swapper/108:11:381
1553899200,11cyclictest0-21swapper/110:05:331
1553299206,3cyclictest9133-21sshd10:33:440
1553299206,3cyclictest9133-21sshd10:33:430
1553299205,3cyclictest11-21rcuc/012:26:210
1554999199,2cyclictest29607-21/usr/sbin/munin07:40:223
1554999191,6cyclictest41-21ksoftirqd/309:04:163
1554999191,6cyclictest18624-21/usr/sbin/munin07:15:153
1554999191,5cyclictest3841-21/usr/sbin/munin07:55:143
1554999191,12cyclictest0-21swapper/308:05:423
1554999191,12cyclictest0-21swapper/307:10:033
1554999191,10cyclictest25522-21sshd08:44:123
1554599195,3cyclictest87350irq/126-enp1s0-10:25:252
1554599194,9cyclictest0-21swapper/210:16:262
1554599193,8cyclictest0-21swapper/209:39:192
1554599193,4cyclictest25597-21sh08:45:022
1554599192,11cyclictest0-21swapper/211:58:572
1554599191,6cyclictest0-21swapper/211:18:082
1554599191,5cyclictest28987-21grep07:35:242
1554599191,5cyclictest22774-21cpuspeed_turbos11:05:112
1554599191,5cyclictest0-21swapper/210:02:182
15545991913,3cyclictest609-21smart_sda07:45:222
1554599191,15cyclictest10774-21df08:10:142
1554599191,15cyclictest10774-21df08:10:142
1553899195,4cyclictest0-21swapper/109:12:131
1553899192,4cyclictest6964-21smart_sda11:40:271
1553899192,4cyclictest6964-21smart_sda11:40:271
1553899191,8cyclictest0-21swapper/107:32:011
1553899191,11cyclictest0-21swapper/111:08:461
1553899191,10cyclictest29476-21cat07:40:011
1553899190,5cyclictest0-21swapper/111:16:491
1553899190,17cyclictest0-21swapper/112:10:141
1553899190,14cyclictest0-21swapper/107:57:391
1553299196,3cyclictest87550irq/128-enp1s0-12:20:200
1553299195,3cyclictest87550irq/128-enp1s0-09:05:140
1553299195,3cyclictest12050irq/123-ahci[0008:55:010
1553299194,6cyclictest755-21rngd10:10:320
1553299193,3cyclictest3500-21/usr/sbin/munin11:35:130
1553299192,2cyclictest9881-21awk11:50:000
1553299191,6cyclictest0-21swapper/009:45:240
1553299191,5cyclictest3813-21cat07:55:020
15532991912,4cyclictest31122-21grep07:40:260
1553299191,15cyclictest29883-21/usr/sbin/munin12:35:110
1553299191,15cyclictest20468-21/usr/sbin/munin11:00:170
1553299191,15cyclictest19238-21/usr/sbin/munin08:30:110
1553299191,15cyclictest1537-21crond09:30:010
9292181,2sleep20-21swapper/211:28:372
51032181,13sleep20-21swapper/207:55:232
308342181,3sleep20-21swapper/210:10:042
29452181,2sleep00-21swapper/011:30:570
1554999181,9cyclictest23034-21cpu07:25:113
15549991812,2cyclictest87350irq/126-enp1s0-08:15:223
1554599189,2cyclictest30-21rcuc/208:20:152
1554599184,9cyclictest87550irq/128-enp1s0-09:45:512
1554599184,3cyclictest12050irq/123-ahci[0009:20:002
1554599182,9cyclictest0-21swapper/211:52:402
1554599181,7cyclictest19361-21expr08:30:122
1554599181,7cyclictest0-21swapper/209:43:592
1554599181,6cyclictest0-21swapper/210:59:192
15545991815,2cyclictest311ktimersoftd/211:35:202
1554599181,4cyclictest0-21swapper/211:14:442
15545991813,3cyclictest25682-21grep07:30:122
15545991812,3cyclictest23522-21grep09:50:262
1554599181,15cyclictest0-21swapper/208:18:572
1553899187,3cyclictest30049-21cat08:55:001
1553899181,7cyclictest0-21swapper/108:06:241
15538991812,3cyclictest3851-21users09:05:251
1553899181,16cyclictest0-21swapper/108:37:441
1553299187,2cyclictest87350irq/126-enp1s0-09:15:140
1553299185,4cyclictest87550irq/128-enp1s0-12:05:010
1553299183,3cyclictest23018-21/usr/sbin/munin07:25:140
1553299182,3cyclictest0-21swapper/007:34:280
1553299181,5cyclictest4464-21/usr/sbin/munin09:10:230
1553299181,5cyclictest26441-21/usr/sbin/munin10:00:150
1553299181,11cyclictest0-21swapper/008:40:250
311312171,3sleep20-21swapper/211:23:482
1554999173,8cyclictest0-21swapper/308:00:183
1554999173,3cyclictest5750irq/9-acpi09:05:123
1554999171,3cyclictest87350irq/126-enp1s0-07:35:123
1554599175,3cyclictest87650irq/129-enp1s0-07:10:192
1554599174,2cyclictest12050irq/123-ahci[0007:30:012
1554599172,5cyclictest1652-21diskstats11:30:132
1554599172,3cyclictest25820-21unixbench_singl09:55:262
1554599171,8cyclictest87350irq/126-enp1s0-09:00:162
1554599171,8cyclictest0-21swapper/210:07:462
1554599171,7cyclictest17039-21/usr/sbin/munin08:25:142
1554599171,7cyclictest0-21swapper/212:20:242
1554599171,6cyclictest2736-21fschecks_time09:05:142
1554599171,6cyclictest19318-21fschecks_count12:10:152
1554599171,5cyclictest16702-21/usr/sbin/munin12:05:252
1554599171,5cyclictest0-21swapper/209:25:232
1554599171,4cyclictest0-21swapper/209:14:582
15545991712,3cyclictest9152-21sensors_temp11:45:232
15545991711,3cyclictest827-21runrttasks07:15:032
1554599171,13cyclictest6697-21sed08:00:142
1554599171,10cyclictest0-21swapper/212:18:302
1554599171,10cyclictest0-21swapper/211:03:432
1553299172,3cyclictest8245-21/usr/sbin/munin08:05:130
15532991713,3cyclictest0-21swapper/008:36:160
15532991712,3cyclictest26886-21ntp_states08:45:200
15532991711,3cyclictest31924-21cpuspeed_turbos07:45:120
1554999160,14cyclictest0-21swapper/307:45:243
1554599169,4cyclictest2873-21/usr/sbin/munin07:50:212
1554599162,5cyclictest31555-21/usr/sbin/munin08:55:232
1554599161,6cyclictest8306-21egrep08:05:112
15545991612,3cyclictest0-21swapper/210:52:012
15545991611,3cyclictest30715-21pmu-power07:40:212
15545991611,3cyclictest30-21rcuc/208:52:372
15545991611,3cyclictest13460-21ls09:30:132
1554599161,13cyclictest0-21swapper/208:39:262
15532991612,3cyclictest0-21swapper/009:00:150
1553299161,14cyclictest0-21swapper/007:16:160
1554599151,5cyclictest20842-21cpu07:20:112
1554599151,2cyclictest87350irq/126-enp1s0-08:45:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional