You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 22:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8.osadl.org (updated Tue Dec 02, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3878173993934,3cyclictest3937070-21sleep07:55:003
3878173993229,1cyclictest0-21swapper/309:40:013
3878173992522,1cyclictest0-21swapper/312:10:013
387817399236,8cyclictest154532sleep310:59:193
387817399220,2cyclictest778062sleep311:35:223
387817399220,16cyclictest41274552sleep310:06:243
387817399217,8cyclictest38841682sleep307:12:453
3878173992117,1cyclictest0-21swapper/309:30:013
387817399206,8cyclictest234152sleep311:03:553
387817399204,7cyclictest1445082sleep312:18:093
387817399200,10cyclictest4006370-21taskset08:47:573
387817399193,5cyclictest39055372sleep307:29:083
3878173991916,1cyclictest0-21swapper/311:40:193
3878173991916,1cyclictest0-21swapper/311:15:193
3878173991916,1cyclictest0-21swapper/310:10:233
387817399191,14cyclictest4088626-21chrt09:44:523
387817399191,14cyclictest3918275-21turbostat07:40:013
387817399190,14cyclictest4104837-21sleep09:55:013
41065632180,9sleep14108120-21kthreadcore09:55:161
387817399186,6cyclictest39871112sleep308:30:313
387817399184,7cyclictest1756632sleep312:35:363
387817399182,13cyclictest4177442-21turbostat10:40:003
3878173991815,1cyclictest0-21swapper/310:15:163
3878173991815,1cyclictest0-21swapper/309:49:343
387817399181,3cyclictest0-21swapper/309:00:133
387817399181,13cyclictest4031905-21turbostat09:10:013
387817399181,13cyclictest30075-21chrt11:05:353
387817399181,13cyclictest120598-21chrt12:03:533
387817399180,13cyclictest3924760-21taskset07:45:023
387817399180,10cyclictest4184353-21taskset10:42:143
40890682171,5sleep04089231-21diskstats09:45:130
387817399174,7cyclictest40002262sleep308:44:223
387817399173,8cyclictest40464072sleep309:15:523
387817399171,13cyclictest3943627-21sleep08:00:013
387817399171,12cyclictest4169365-21turbostat10:35:003
387817399171,12cyclictest4056260-21taskset09:24:593
387817399171,12cyclictest3886734-21taskset07:15:123
387817399171,12cyclictest159522-21taskset12:25:293
387817399171,12cyclictest136193-21chrt12:12:323
387817399170,7cyclictest39921242sleep308:35:193
387817399170,13cyclictest153418-21turbostat12:25:013
387817399165,1cyclictest0-21swapper/311:30:233
3878173991614,1cyclictest0-21swapper/307:25:003
387817399161,3cyclictest0-21swapper/311:55:133
387817399161,3cyclictest0-21swapper/310:50:123
387817399161,3cyclictest0-21swapper/310:00:123
387817399161,3cyclictest0-21swapper/309:55:113
387817399161,3cyclictest0-21swapper/308:00:113
3878173991613,1cyclictest0-21swapper/312:33:153
3878173991613,1cyclictest0-21swapper/308:19:223
387817399161,2cyclictest0-21swapper/311:20:123
387817399161,12cyclictest96492-21taskset11:49:203
387817399161,12cyclictest57500-21taskset11:25:153
387817399161,12cyclictest57500-21taskset11:25:143
387817399161,12cyclictest4151800-21taskset10:21:433
387817399161,12cyclictest4038302-21taskset09:11:133
387817399161,12cyclictest4018941-21chrt08:58:523
387817399161,12cyclictest4012485-21taskset08:51:223
387817399161,12cyclictest3974831-21taskset08:24:013
387817399161,12cyclictest104090-21chrt11:53:113
387817399161,11cyclictest3980851-21taskset08:25:323
387817399161,11cyclictest3955829-21chrt08:05:573
387817399160,4cyclictest0-21swapper/307:45:003
40262722155,7sleep24026993-21cut09:05:172
387817399151,3cyclictest383722chrt11:11:033
3878173991512,1cyclictest0-21swapper/309:32:243
387817399151,11cyclictest3911636-21chrt07:31:583
387817399151,10cyclictest4192112-21taskset10:46:323
387817399150,3cyclictest0-21swapper/310:25:143
387817399150,12cyclictest3962061-21chrt08:10:353
495872145,6sleep150730-21tr11:20:171
41527842140,6sleep14152788-21ssh10:23:581
38721202142,6sleep23872152-21apt-get07:05:152
721242130,10sleep172131-21sh11:34:391
40465802130,10sleep14046588-21ssh09:16:141
147598130,9rtkit-daemon166979-21ssh12:30:202
147598130,1rtkit-daemon0-21swapper/107:48:161
41434222120,8sleep14143430-21ssh10:16:011
41879822110,8sleep14189999-21grep10:45:191
41023212111,7sleep04102451-21grep09:50:230
40134252111,8sleep24013777-21cut08:55:192
39881462110,8sleep03988893-21cut08:35:170
39062822110,8sleep23906947-21cut07:30:182
368562110,3sleep037436-21grep11:10:220
147598110,8rtkit-daemon4039713-21seq09:14:302
147598110,7rtkit-daemon744-21runrttasks11:27:151
147598110,7rtkit-daemon744-21runrttasks11:27:141
147598110,7rtkit-daemon720-21snmpd10:51:091
147598110,7rtkit-daemon295-21systemd-journal08:09:421
147598110,5rtkit-daemon214-21plymouthd10:22:470
147598110,4rtkit-daemon214-21plymouthd09:34:310
1272142110,1sleep10-21swapper/112:05:281
1066562110,3sleep0108288-21kthreadcore11:55:180
1033062110,8sleep1103309-21sh11:51:271
706572100,7sleep070665-21ssh11:31:090
454282100,7sleep240370-21/usr/sbin/munin11:15:202
40193222102,4sleep04019344-21apt-get09:00:130
39994462100,7sleep23999510-21ps08:40:242
39689002101,5sleep13968933-21apt-get08:20:111
1591442100,7sleep0159371-21grep12:25:240
147598100,7rtkit-daemon456-21avahi-daemon07:13:511
147598100,6rtkit-daemon97107-21/usr/sbin/munin11:50:122
147598100,6rtkit-daemon4185136-21ssh10:44:082
147598100,5rtkit-daemon4175126-21python310:35:250
147598100,5rtkit-daemon40404-21apt-get11:15:120
147598100,4rtkit-daemon34-21ksoftirqd/109:24:401
147598100,3rtkit-daemon720-21snmpd11:44:292
147598100,3rtkit-daemon720-21snmpd07:30:141
147598100,3rtkit-daemon214-21plymouthd10:57:570
147598100,1rtkit-daemon0-21swapper/012:19:470
1429552100,7sleep1143068-21find12:15:231
88523290,7sleep188990-21ssh11:45:011
58564291,6sleep060509-21kthreadcore11:25:170
58564291,6sleep060509-21kthreadcore11:25:160
50731290,6sleep252760-21kthreadcore11:20:182
4124159290,6sleep24126219-21kthreadcore10:05:192
4028094290,6sleep04030149-21grep09:05:190
3994672291,5sleep03995199-21cut08:40:170
3962660291,4sleep23962693-21apt-get08:15:102
3950344290,6sleep23950376-21fschecks_count08:05:172
3924809290,6sleep23924964-21/usr/sbin/munin07:45:142
163156290,6sleep0164295-21cat12:30:160
152593290,6sleep2152599-21ssh12:23:102
14759890,6rtkit-daemon515-21gmain07:08:491
14759890,6rtkit-daemon3884511-21munin-node08:28:391
14759890,6rtkit-daemon1540-21sendmail-mta09:11:251
14759890,4rtkit-daemon88487-21apt-get11:45:022
14759890,4rtkit-daemon72826-21df11:35:141
14759890,4rtkit-daemon4003582-21cut08:45:201
14759890,4rtkit-daemon3943150-21grep07:55:291
14759890,4rtkit-daemon214-21plymouthd12:01:561
14759890,4rtkit-daemon1-21systemd12:14:260
14759890,2rtkit-daemon214-21plymouthd11:22:280
14759890,2rtkit-daemon214-21plymouthd11:01:090
14759890,2rtkit-daemon214-21plymouthd09:37:350
14759890,1rtkit-daemon0-21swapper/012:20:250
95566280,1sleep00-21swapper/011:47:120
4168221280,3sleep10-21swapper/110:32:231
4159442280,1sleep10-21swapper/110:25:431
4112510280,1sleep20-21swapper/209:59:102
4097049282,3sleep24097086-21apt-get09:50:132
4080973281,3sleep04080976-21apt-get09:40:130
4040467280,3sleep00-21swapper/009:15:040
4025339281,4sleep24025396-21sh09:05:002
4025227280,1sleep10-21swapper/109:03:341
3970178280,1sleep23971286-21kthreadcore08:20:162
3878018280,2sleep30-21swapper/307:09:553
33678280,2sleep234782-21kthreadcore11:10:202
23432280,5sleep123439-21sh11:03:581
161926280,1sleep10-21swapper/112:30:141
160324281,4sleep1160332-21ssh12:27:291
159833280,1sleep20-21swapper/212:26:182
14759880,5rtkit-daemon4103238-21kworker/u16:4+rpciod10:40:020
14759880,4rtkit-daemon729-21lldpd12:36:332
14759880,4rtkit-daemon729-21lldpd07:35:021
14759880,4rtkit-daemon4191643-21ssh10:45:332
14759880,4rtkit-daemon295-21systemd-journal11:55:132
14759880,4rtkit-daemon295-21systemd-journal10:10:120
14759880,4rtkit-daemon214-21plymouthd07:27:181
14759880,3rtkit-daemon4119087-21ssh10:00:410
14759880,3rtkit-daemon4063458-21ssh09:28:000
14759880,3rtkit-daemon214-21plymouthd10:15:150
14759880,1rtkit-daemon0-21swapper/108:41:171
144049280,5sleep2144057-21ssh12:17:042
123015280,2sleep0124918-21kthreadcore12:05:200
104420280,1sleep00-21swapper/011:54:020
4186854270,1sleep00-21swapper/010:45:180
4185041270,1sleep10-21swapper/110:43:491
4176788270,4sleep24071358-21kworker/u16:3+rpciod10:38:382
4144457271,4sleep24144463-21ssh10:18:212
4129740270,5sleep24130067-21irqcore10:10:172
4126537270,4sleep14121291-21/usr/sbin/munin10:05:221
4103046270,2sleep14103054-21ssh09:50:561
4096730270,5sleep24096787-21latency_hist09:50:002
4089232271,3sleep14071358-21kworker/u16:3+xprtiod09:45:141
4087112270,5sleep1214-21plymouthd09:41:141
4073647270,5sleep24074664-21cut09:35:172
4056589271,3sleep24056628-21apt-get09:25:122
4038805275,1sleep00-21swapper/009:12:240
3943871271,3sleep10-21swapper/108:00:121
3931212270,1sleep00-21swapper/007:50:130
3912173271,3sleep23912322-21df_inode07:35:162
3905462270,1sleep00-21swapper/007:28:090
3893008271,3sleep23893040-21apt-get07:20:122
3880487271,4sleep03882530-21kthreadcore07:10:160
22194270,1sleep20-21swapper/211:01:062
170332270,4sleep1171266-21tail12:35:191
14759870,4rtkit-daemon730-21ntpd08:56:071
14759870,4rtkit-daemon3884511-21munin-node11:59:561
14759870,4rtkit-daemon214-21plymouthd08:50:021
14759870,4rtkit-daemon1-21systemd10:05:090
14759870,3rtkit-daemon4167245-21unixbench_multi10:30:232
14759870,3rtkit-daemon4161130-21ssh10:29:372
14759870,3rtkit-daemon3931040-21apt-get07:50:101
14759870,3rtkit-daemon3884511-21munin-node11:09:220
14759870,3rtkit-daemon3884511-21munin-node08:35:171
14759870,3rtkit-daemon313-21systemd-udevd07:41:431
14759870,3rtkit-daemon295-21systemd-journal11:39:132
14759870,3rtkit-daemon214-21plymouthd10:52:460
14759870,3rtkit-daemon214-21plymouthd09:55:100
14759870,3rtkit-daemon214-21plymouthd09:25:181
14759870,3rtkit-daemon214-21plymouthd08:32:531
14759870,3rtkit-daemon1-21systemd10:34:310
14759870,3rtkit-daemon1-21systemd09:15:172
14759870,3rtkit-daemon1-21systemd08:15:161
14759870,3rtkit-daemon1-21systemd08:11:181
14759870,3rtkit-daemon1-21systemd07:21:591
14759870,2rtkit-daemon3891832-21/usr/sbin/munin07:15:201
14759870,2rtkit-daemon214-21plymouthd10:25:430
14759870,2rtkit-daemon214-21plymouthd09:06:291
136611270,1sleep10-21swapper/112:13:351
127294270,1sleep20-21swapper/212:05:382
87200260,1sleep10-21swapper/111:42:041
85916260,3sleep085995-21ntp_states11:40:240
56508260,3sleep256551-21ssh11:25:112
56508260,3sleep256551-21ssh11:25:102
47022260,1sleep1214-21plymouthd11:17:321
4175677260,1sleep10-21swapper/110:36:011
4151713260,1sleep20-21swapper/210:21:272
4136041260,4sleep10-21swapper/110:12:281
4113127260,1sleep20-21swapper/210:00:032
4080306260,4sleep10-21swapper/109:39:531
4072041260,1sleep20-21swapper/209:34:262
4070553260,1sleep10-21swapper/109:30:481
4055505260,1sleep20-21swapper/209:23:152
4018198260,3sleep04018270-21/usr/sbin/munin08:55:250
4006979260,3sleep04007179-21hddtemp_smartct08:50:150
3993398260,1sleep20-21swapper/208:35:422
3984173260,3sleep23986180-21grep08:30:182
3980883260,1sleep00-21swapper/008:25:590
3968429260,1sleep00-21swapper/008:17:290
3962326260,1sleep00-21swapper/008:14:010
3956435260,3sleep23956578-21df08:10:152
3956130260,1sleep00-21swapper/008:09:510
3943351260,1sleep20-21swapper/207:56:562
3943296260,1sleep00-21swapper/007:56:130
3924254260,1sleep20-21swapper/207:41:402
3923819260,3sleep03923930-21/usr/sbin/munin07:40:230
3918214260,1sleep00-21swapper/007:39:300
3911633260,1sleep00-21swapper/007:31:570
3892454260,1sleep00-21swapper/007:16:310
3878333260,3sleep23878366-21apt-get07:10:102
3872117260,1sleep00-21swapper/007:05:130
32343261,2sleep132380-21apt-get11:10:151
30778261,3sleep2214-21plymouthd11:07:122
15301260,4sleep215309-21ssh10:58:542
152326260,3sleep1152334-21ssh12:22:291
120305260,4sleep20-21swapper/212:03:102
120039260,1sleep00-21swapper/012:02:360
74068250,2sleep00-21swapper/011:35:180
64946250,2sleep20-21swapper/211:30:152
4112919250,3sleep10-21swapper/110:00:021
4086966250,1sleep20-21swapper/209:40:552
4054439250,3sleep00-21swapper/009:20:420
4006189250,3sleep20-21swapper/208:45:372
3976330250,3sleep23977466-21grep08:25:162
3931977250,3sleep23933141-21kthreadcore07:50:142
3904772251,2sleep20-21swapper/207:25:222
3886699251,2sleep20-21swapper/207:15:122
31637250,3sleep10-21swapper/111:09:141
305251,2sleep20-21swapper/210:50:152
175671251,2sleep00-21swapper/012:35:380
14826250,3sleep10-21swapper/110:57:491
135548250,3sleep2135556-21ssh12:11:042
4006815240,2sleep2431ktimers/208:50:132
4000538240,1sleep00-21swapper/008:45:150
3981453240,2sleep00-21swapper/008:30:110
3945131240,2sleep2214-21plymouthd08:00:172
3930140240,2sleep03930388-21awk07:45:230
3968889230,1sleep00-21swapper/008:20:010
3949214230,1sleep00-21swapper/008:00:230
3892993230,1sleep00-21swapper/007:20:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional