You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 12:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Sun Mar 22, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
38772822210,1sleep00-21swapper/019:05:190
388000499192,12cyclictest4118679-21turbostat22:05:003
388000499192,12cyclictest4118679-21turbostat22:05:003
3880004991916,1cyclictest0-21swapper/321:00:013
388000499181,13cyclictest39770-21taskset23:20:003
388000499181,12cyclictest3990014-21turbostat20:35:003
161862181,6sleep04034474-21diskmemload23:03:380
3880004991715,1cyclictest0-21swapper/321:24:253
3880004991715,1cyclictest0-21swapper/300:12:193
3880004991715,0cyclictest0-21swapper/321:49:523
3880004991714,1cyclictest0-21swapper/323:35:193
388000499171,12cyclictest4049804-21sleep21:20:013
388000499165,7cyclictest1370122sleep300:20:213
388000499164,8cyclictest40859142sleep321:40:223
388000499161,6cyclictest0-21swapper/320:05:143
388000499161,6cyclictest0-21swapper/320:05:143
3880004991615,0cyclictest0-21swapper/322:17:353
3880004991614,1cyclictest0-21swapper/323:23:333
3880004991614,0cyclictest0-21swapper/319:25:513
3880004991613,1cyclictest0-21swapper/322:25:213
3880004991613,1cyclictest0-21swapper/322:06:063
3880004991613,1cyclictest0-21swapper/322:06:063
3880004991613,1cyclictest0-21swapper/320:00:243
3880004991613,1cyclictest0-21swapper/319:31:443
3880004991613,1cyclictest0-21swapper/300:25:083
388000499161,12cyclictest4015422-21turbostat20:55:003
388000499161,12cyclictest3945577-21turbostat20:00:003
388000499161,11cyclictest5500-21taskset22:55:173
388000499157,5cyclictest47428-21sleep23:29:553
388000499155,7cyclictest40399712sleep321:10:223
388000499155,7cyclictest38801732sleep319:10:123
3880004991514,0cyclictest0-21swapper/300:05:013
3880004991513,1cyclictest0-21swapper/323:55:203
3880004991513,1cyclictest0-21swapper/323:55:203
3880004991512,1cyclictest0-21swapper/323:40:243
3880004991512,1cyclictest0-21swapper/323:00:233
3880004991512,1cyclictest0-21swapper/322:50:213
3880004991512,1cyclictest0-21swapper/321:56:213
388000499151,11cyclictest4128813-21taskset22:10:173
388000499151,11cyclictest4102032-21taskset21:51:123
388000499151,11cyclictest3977593-21taskset20:25:023
388000499151,11cyclictest23598-21taskset23:07:373
3880004991511,1cyclictest0-21swapper/321:28:233
388000499151,10cyclictest4182554-21taskset22:45:153
388000499151,10cyclictest4164988-21taskset22:35:083
388000499151,10cyclictest4073236-21taskset21:35:123
388000499150,6cyclictest0-21swapper/322:20:133
388000499147,3cyclictest0-21swapper/320:40:093
388000499141,9cyclictest39902912sleep320:35:133
3880004991413,0cyclictest0-21swapper/319:40:203
3880004991412,1cyclictest0-21swapper/321:30:183
3880004991412,1cyclictest0-21swapper/319:20:213
3880004991412,1cyclictest0-21swapper/300:00:193
3880004991411,1cyclictest0-21swapper/322:41:253
3880004991411,1cyclictest0-21swapper/319:51:453
388000499141,10cyclictest84618-21taskset23:46:473
388000499141,10cyclictest4163741-21taskset22:32:113
388000499141,10cyclictest4008752-21taskset20:46:273
388000499141,10cyclictest3964868-21taskset20:15:033
388000499141,10cyclictest3964412-21taskset20:12:263
388000499141,10cyclictest3932651-21taskset19:47:123
388000499141,10cyclictest3920107-21taskset19:39:503
388000499141,10cyclictest152533-21taskset00:30:233
388000499141,10cyclictest128211-21taskset00:15:193
388000499140,11cyclictest3894756-21taskset19:19:573
388000499140,11cyclictest31874-21taskset23:14:343
388000499140,10cyclictest4034175-21taskset21:06:203
247612141,9sleep00-21swapper/023:10:110
388000499131,9cyclictest3971234-21taskset20:20:043
388000499131,8cyclictest860002sleep323:50:133
3880004991311,0cyclictest0-21swapper/323:31:403
3880004991310,1cyclictest0-21swapper/321:00:223
388000499130,10cyclictest154867-21taskset00:35:043
40851742121,8sleep04085766-21/usr/sbin/munin21:40:190
3880004991210,1cyclictest0-21swapper/319:10:013
1074602121,8sleep2107464-21ssh00:01:272
40410772110,3sleep14041079-21ssh21:12:041
39776382112,5sleep13977639-21apt-get20:25:131
39709552111,3sleep21-21systemd20:20:002
38738312111,4sleep10-21swapper/119:05:131
41864112101,5sleep20-21swapper/222:46:272
41568172100,7sleep24156821-21ssh22:29:262
41343322101,5sleep24134488-21diskstats22:15:142
40723812101,2sleep10-21swapper/121:34:021
39712722102,4sleep13971275-21apt-get20:20:111
1622462101,6sleep1162345-21munin-run00:40:011
160022101,6sleep116006-21ssh23:02:581
1482142100,3sleep00-21swapper/000:30:170
1148182101,2sleep20-21swapper/200:05:282
61434291,5sleep04034474-21diskmemload23:31:130
469290,3sleep1471-21ssh22:52:441
45524290,6sleep045648-21ls23:20:260
4165020291,5sleep04165054-21apt-get22:35:140
4162503291,4sleep04162553-21ntpq22:30:210
4133975290,2sleep00-21swapper/022:15:000
4131818290,5sleep24131862-21ntpq22:10:212
4124237291,4sleep14124283-21ntpq22:05:231
4124237291,4sleep14124283-21ntpq22:05:231
4087301290,7sleep1933-21sshd21:42:531
4023037291,5sleep00-21swapper/021:00:160
4015691291,3sleep20-21swapper/220:55:132
4009296291,4sleep14009328-21apt-get20:50:101
4003352290,6sleep24003754-21sed20:45:152
3940957291,6sleep13942562-21cut19:55:171
3927154290,2sleep20-21swapper/219:45:162
3926968290,6sleep13927151-21cut19:45:151
3895046291,4sleep03895079-21apt-get19:20:110
22986290,1sleep10-21swapper/123:05:471
21196290,6sleep022081-21/usr/sbin/munin23:05:180
140944292,4sleep2141944-21cut00:25:182
130595291,5sleep04034474-21diskmemload00:16:570
114552291,4sleep0114800-21users00:05:260
95510281,4sleep297315-21kthreadcore23:55:172
95510281,4sleep297315-21kthreadcore23:55:172
85907280,5sleep185913-21ssh23:50:031
84444281,4sleep20-21swapper/223:46:192
75885280,4sleep175929-21ntpq23:40:201
70373281,4sleep170612-21latency_hist23:40:001
54863280,2sleep154869-21ssh23:29:301
4186770280,1sleep00-21swapper/022:47:330
4172730281,3sleep04172729-21apt-get22:40:110
4171226280,2sleep14171232-21ssh22:36:351
4162605281,4sleep24162651-21ntpq22:30:232
4157348281,4sleep14157505-21diskstats22:30:141
4152265281,4sleep04154107-21ssh22:25:190
4143483280,5sleep04144855-21kthreadcore22:20:180
4140169280,2sleep00-21swapper/022:15:410
4126634281,3sleep14126637-21apt-get22:10:121
4093617281,5sleep04093700-21/usr/sbin/munin21:45:240
4080333280,5sleep11-21systemd21:40:001
4057437281,4sleep14057493-21unin-run21:25:001
4047652281,4sleep14047698-21ntpq21:15:211
4042372281,3sleep24042409-21apt-get21:15:102
40361280,5sleep240038-21/usr/sbin/munin23:20:162
4034386280,3sleep00-21swapper/021:09:030
4019970280,6sleep14020681-21cat20:55:201
3995446280,5sleep13995488-21ntpq20:35:231
3987844281,4sleep23988884-21memory20:30:192
39585280,6sleep139591-21ssh23:19:351
3950994281,4sleep23951043-21ntpq20:00:232
3946252284,2sleep00-21swapper/020:00:170
3939533281,3sleep03939529-21apt-get19:55:120
3925568281,4sleep13925605-21ntpq19:40:211
3920604280,5sleep23920737-21grep19:40:142
3914268280,5sleep13914455-21gltestperf19:35:161
3910042280,4sleep13911982-21cut19:30:191
3903509280,5sleep23905419-21tr19:25:192
3901438281,4sleep03901443-21apt-get19:25:130
3893853280,4sleep03893898-21ntpq19:15:220
23662280,5sleep24034474-21diskmemload23:07:492
145651281,4sleep0145645-40updatedb.plocat00:26:120
144638280,5sleep1144764-21ntpq00:25:211
134503281,5sleep10-21swapper/100:20:191
131985281,4sleep0132140-21df_inode00:20:140
130657280,5sleep2130663-21ssh00:17:072
116665281,3sleep2116662-21apt-get00:10:142
8960270,4sleep08999-21munin-run23:00:010
85958271,3sleep285995-21apt-get23:50:122
46876270,4sleep14034474-21diskmemload23:23:321
4125063270,1sleep24125069-21ssh22:06:202
4125063270,1sleep24125069-21ssh22:06:202
4124189271,3sleep04124233-21ntpq22:05:220
4124189271,3sleep04124233-21ntpq22:05:220
4118005270,4sleep24118011-21ssh22:03:072
4118005270,4sleep24118011-21ssh22:03:072
4110537270,1sleep24110543-21ssh21:58:462
4102970270,2sleep00-21swapper/021:53:590
4095962271,3sleep24095997-21apt-get21:50:122
4095492270,4sleep14034474-21diskmemload21:49:351
4095117270,3sleep24034474-21diskmemload21:48:232
4082150270,5sleep20-21swapper/221:40:172
4079899271,3sleep04034474-21diskmemload21:38:390
4071547270,2sleep20-21swapper/221:31:272
4063036270,4sleep04063084-21ntpq21:25:230
4047602271,4sleep04047650-21ntpq21:15:200
4034775270,1sleep00-21swapper/021:10:110
4018059270,5sleep04019969-21grep20:55:190
4014671270,5sleep24014781-21xargs20:50:242
4003151270,4sleep14003351-21hddtemp_smartct20:45:141
3995653270,4sleep03995847-21grep20:35:260
3985924270,4sleep13987843-21cut20:30:181
3981690270,4sleep03982566-21sort20:25:200
3960138271,3sleep20-21swapper/220:10:182
3954790270,5sleep13956779-21cut20:05:201
3954790270,5sleep13956779-21cut20:05:201
3933137271,3sleep13933169-21apt-get19:50:111
3925697270,4sleep03925820-21python319:40:240
3914458270,4sleep23914598-21cut19:35:172
3886666271,3sleep13886928-21kthreadcore19:15:031
3874492271,3sleep23875388-21kthreadcore19:05:182
32382271,2sleep00-21swapper/023:15:120
1716272,3sleep11835-21cpuspeed_turbos22:55:121
154958270,4sleep0155119-21diskstats00:35:140
153385270,3sleep10-21swapper/100:31:221
132262270,1sleep20-21swapper/200:20:162
122811270,4sleep1122816-21sh00:11:291
11042271,4sleep212855-21cut23:00:192
102800270,4sleep0104245-21kthreadcore00:00:170
98994261,2sleep199071-21systemctl23:55:251
98994261,2sleep199071-21systemctl23:55:251
93638260,2sleep093661-21apt-get23:55:110
93638260,2sleep093661-21apt-get23:55:110
91155262,1sleep0221rcuc/023:50:220
78841260,3sleep079829-21kernelversion23:45:170
64177260,3sleep065195-21kthreadcore23:35:180
53214260,1sleep20-21swapper/223:25:242
47686261,2sleep00-21swapper/023:25:110
4194070260,4sleep20-21swapper/222:51:282
4193987260,1sleep00-21swapper/022:51:140
4180017260,3sleep14180222-21ssh22:45:021
4173704260,3sleep14174237-21cut22:40:161
4149655260,3sleep14149696-21ssh22:25:121
4148125260,3sleep14148131-21ssh22:21:231
4148109260,4sleep24148111-21ssh22:21:202
4140983260,1sleep14140987-21ssh22:18:061
4117169260,4sleep04034474-21diskmemload22:00:370
4117169260,4sleep04034474-21diskmemload22:00:370
4111309261,2sleep10-21swapper/122:00:131
4111309261,2sleep10-21swapper/122:00:131
4110312260,1sleep00-21swapper/021:58:080
4108941260,3sleep14109023-21processes21:55:241
4102162260,3sleep14102168-21ssh21:51:351
4074515260,3sleep24075951-21grep21:35:162
4064972260,1sleep14064978-21ssh21:29:371
4064559261,3sleep20-21swapper/221:28:262
4034179260,1sleep20-21swapper/221:06:242
4015140260,1sleep00-21swapper/020:51:460
3989236261,2sleep00-21swapper/020:30:250
3983301260,1sleep20-21swapper/220:25:302
3972453261,2sleep20-21swapper/220:20:162
3958141260,1sleep20-21swapper/220:08:372
3958141260,1sleep20-21swapper/220:08:372
3907767261,3sleep03907915-21/usr/sbin/munin19:30:140
3668261,2sleep20-21swapper/222:55:172
35089260,3sleep236944-21grep23:15:192
26205260,3sleep127587-21kthreadcore23:10:161
154910261,2sleep20-21swapper/200:35:122
153823260,3sleep20-21swapper/200:32:442
119750261,2sleep00-21swapper/000:10:200
78386250,2sleep178548-21ssh23:45:141
72478250,2sleep2421rcuc/223:40:172
70697250,2sleep00-21swapper/023:40:110
63256251,2sleep263430-21ssh23:35:162
56959251,2sleep20-21swapper/223:30:192
55414250,2sleep10-21swapper/123:30:141
4172682251,2sleep20-21swapper/222:40:092
4171350250,2sleep24171355-21sshd-session22:36:562
4051156251,2sleep00-21swapper/021:20:180
4040863250,2sleep24040869-21ssh21:11:292
4029429250,3sleep14030520-21kthreadcore21:05:191
4001820250,2sleep14001853-21ntpq20:40:201
3998649250,3sleep04000577-21kthreadcore20:40:150
3996614250,2sleep23996646-21apt-get20:40:092
3990255250,2sleep20-21swapper/220:35:122
3975523250,3sleep03976265-21/usr/sbin/munin20:20:180
3959205250,3sleep13960134-21kthreadcore20:10:171
3958516250,2sleep00-21swapper/020:10:120
3952215250,2sleep00-21swapper/020:05:150
3952215250,2sleep00-21swapper/020:05:150
3939052250,1sleep20-21swapper/219:52:412
3928264251,2sleep00-21swapper/019:45:180
3915564251,2sleep00-21swapper/019:35:190
3913790250,1sleep20-21swapper/219:34:532
3902441250,2sleep10-21swapper/119:25:181
3896625250,2sleep20-21swapper/219:20:172
3895082250,3sleep10-21swapper/119:20:121
3894521250,1sleep20-21swapper/219:17:012
125310250,3sleep1126383-21cut00:15:181
109259250,3sleep1109438-21if_eno100:05:151
104248250,2sleep10-21swapper/100:00:181
4066990240,2sleep04068782-21cut21:30:160
4050314240,1sleep20-21swapper/221:20:162
4027789240,3sleep20-21swapper/221:00:472
4021735240,2sleep14021835-21appstreamcli21:00:011
4004666240,2sleep04006565-21kthreadcore20:45:170
3970911240,3sleep10-21swapper/120:19:271
3944660240,1sleep20-21swapper/219:55:222
3938898240,3sleep00-21swapper/019:50:430
3885385240,2sleep20-21swapper/219:10:232
3880210240,1sleep00-21swapper/019:10:130
30876240,2sleep20-21swapper/223:11:342
3964915230,1sleep00-21swapper/020:15:130
3946879230,1sleep10-21swapper/120:00:181
3879916230,1sleep10-21swapper/119:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional