You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 09:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Tue Nov 18, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2458324839,0sleep20-21swapper/218:17:532
2447423827,0sleep30-21swapper/318:16:383
2442923824,0sleep00-21swapper/018:16:070
2449523324,0sleep10-21swapper/118:16:521
329970,0watchdog/3299250irq/32-eth0-rx-23:40:443
329970,0watchdog/30-21swapper/322:30:283
29925070,0irq/32-eth0-rx-29229-21cpuspeed_turbos18:45:223
248449970,0cyclictest0-21swapper/223:05:262
339960,0migration/32600-21munin-node21:50:283
339960,0migration/324320-21sh22:56:153
339960,0migration/317568-21sh21:57:293
329960,0watchdog/38541-21kthreadcore21:15:293
329960,0watchdog/32896-21ssh23:20:523
329960,0watchdog/323687-21timerandwakeup20:45:373
329960,0watchdog/30-21swapper/323:47:563
329960,0watchdog/30-21swapper/323:33:443
329960,0watchdog/30-21swapper/323:25:203
329960,0watchdog/30-21swapper/323:17:043
329960,0watchdog/30-21swapper/323:10:283
329960,0watchdog/30-21swapper/323:00:083
329960,0watchdog/30-21swapper/322:53:043
329960,0watchdog/30-21swapper/322:25:523
329960,0watchdog/30-21swapper/322:21:283
329960,0watchdog/30-21swapper/322:16:363
329960,0watchdog/30-21swapper/322:10:283
329960,0watchdog/30-21swapper/322:05:373
329960,0watchdog/30-21swapper/322:00:093
329960,0watchdog/30-21swapper/321:45:373
329960,0watchdog/30-21swapper/321:36:413
329960,0watchdog/30-21swapper/321:31:213
329960,0watchdog/30-21swapper/321:25:293
329960,0watchdog/30-21swapper/321:14:533
329960,0watchdog/30-21swapper/321:01:573
329960,0watchdog/30-21swapper/320:55:453
329960,0watchdog/30-21swapper/320:40:093
329960,0watchdog/30-21swapper/320:36:173
329960,0watchdog/30-21swapper/320:31:333
329960,0watchdog/30-21swapper/320:29:213
329960,0watchdog/30-21swapper/319:50:093
329960,0watchdog/30-21swapper/319:15:493
329960,0watchdog/30-21swapper/319:00:293
329960,0watchdog/30-21swapper/318:24:253
29935060,0irq/33-eth0-tx-0-21swapper/221:16:392
29925060,0irq/32-eth0-rx-31984-21python19:05:353
29925060,0irq/32-eth0-rx-22531-21ntp_states21:40:303
29925060,0irq/32-eth0-rx-0-21swapper/323:39:313
29925060,0irq/32-eth0-rx-0-21swapper/323:05:343
29925060,0irq/32-eth0-rx-0-21swapper/322:40:373
29925060,0irq/32-eth0-rx-0-21swapper/322:35:433
29925060,0irq/32-eth0-rx-0-21swapper/321:21:143
29925060,0irq/32-eth0-rx-0-21swapper/320:20:303
29925060,0irq/32-eth0-rx-0-21swapper/320:05:253
248449960,0cyclictest0-21swapper/221:11:372
248449960,0cyclictest0-21swapper/219:35:362
248429960,0cyclictest0-21swapper/020:58:180
149960,0migration/024999-21sh22:58:070
149960,0migration/011073-21gltestperf21:55:270
8315050,0irq/26-0000:00:17511-21kthreadcore19:20:282
339950,0migration/3525-21aten_r1power_cu19:10:213
339950,0migration/321727-21kthreadcore21:05:283
339950,0migration/31942-21kthreadcore22:45:263
339950,0migration/31310-21kthreadcore20:15:283
329950,0watchdog/37144-21latency19:55:293
329950,0watchdog/30-21swapper/320:50:213
329950,0watchdog/30-21swapper/320:11:053
329950,0watchdog/30-21swapper/319:45:173
329950,0watchdog/30-21swapper/319:40:413
329950,0watchdog/30-21swapper/319:37:493
329950,0watchdog/30-21swapper/319:31:453
329950,0watchdog/30-21swapper/319:25:573
329950,0watchdog/30-21swapper/319:20:093
329950,0watchdog/30-21swapper/318:56:173
329950,0watchdog/30-21swapper/318:41:573
329950,0watchdog/30-21swapper/318:36:293
329950,0watchdog/30-21swapper/318:30:173
329950,0watchdog/30-21swapper/318:25:573
29935050,0irq/33-eth0-tx-531-21cut23:40:262
29935050,0irq/33-eth0-tx-4601-21munin-node23:25:292
29935050,0irq/33-eth0-tx-32259-21munin-node22:45:262
29935050,0irq/33-eth0-tx-26945-21perl20:50:232
29935050,0irq/33-eth0-tx-26746-21grep23:35:282
29935050,0irq/33-eth0-tx-23066-21munin-node20:10:360
29935050,0irq/33-eth0-tx-20885-21perl21:05:252
29935050,0irq/33-eth0-tx-13205-21munin-node23:30:332
29935050,0irq/33-eth0-tx-12208-21munin-node22:30:292
29935050,0irq/33-eth0-tx-0-21swapper/223:03:012
29935050,0irq/33-eth0-tx-0-21swapper/222:20:012
29935050,0irq/33-eth0-tx-0-21swapper/221:28:112
29935050,0irq/33-eth0-tx-0-21swapper/020:20:300
29935050,0irq/33-eth0-tx-0-21swapper/019:55:230
29935050,0irq/33-eth0-tx-0-21swapper/018:25:060
29925050,0irq/32-eth0-rx-9320-21fschecks_time20:00:233
29925050,0irq/32-eth0-rx-6166-21kthreadcore18:50:293
259950,0migration/231408-21latency_hist20:35:082
259950,0migration/219335-21kthreadcore21:40:272
259950,0migration/214127-21cpuspeed_turbos21:20:212
248449955,0cyclictest0-21swapper/223:47:232
248449955,0cyclictest0-21swapper/221:35:252
248449955,0cyclictest0-21swapper/220:35:492
248449950,0cyclictest0-21swapper/223:10:202
248449950,0cyclictest0-21swapper/222:04:442
248439955,0cyclictest0-21swapper/119:40:301
248429955,0cyclictest0-21swapper/022:05:280
248429955,0cyclictest0-21swapper/020:41:260
248429954,0cyclictest18214-21munin-run19:00:080
248429950,0cyclictest0-21swapper/023:48:160
248429950,0cyclictest0-21swapper/021:02:150
249950,0watchdog/224332-21sed19:00:292
249950,0watchdog/20-21swapper/219:15:252
249950,0watchdog/20-21swapper/218:30:292
19150,0ktimersoftd/10-21swapper/123:47:571
179950,0migration/16848-21munin-node22:30:231
179950,0migration/13875-21ksmtuned21:12:261
179950,0migration/132574-21munin-node19:50:321
179950,0migration/128664-21latency20:30:281
179950,0migration/123429-21pmu-power20:45:331
179950,0migration/122723-21aten_r1power_vo21:25:251
179950,0migration/122516-21sh22:38:011
179950,0migration/122184-21kthreadcore19:45:311
179950,0migration/111103-21switchtime23:25:341
169950,0watchdog/120332-21kthreadcore19:20:291
149950,0migration/06825-21taskset18:28:290
149950,0migration/05510-21sh22:29:050
149950,0migration/04786-21kthreadcore22:45:270
149950,0migration/03839-21kthreadcore19:10:270
149950,0migration/032172-21timerwakeupswit19:05:380
149950,0migration/031326-21kthreadcore23:20:290
149950,0migration/030509-21taskset21:28:480
149950,0migration/030279-21latency_hist20:15:080
149950,0migration/028386-21taskset18:40:420
149950,0migration/027110-21kthreadcore20:30:280
149950,0migration/02620-21irqcore21:50:290
149950,0migration/023300-21kthreadcore21:05:290
149950,0migration/023004-21munin-node23:35:260
149950,0migration/022489-21irqcore22:20:250
149950,0migration/01728-21kthreadcore18:45:280
149950,0migration/016679-21fschecks_time20:05:270
149950,0migration/016270-21kthreadcore18:35:270
149950,0migration/014678-21latency_hist22:35:080
149950,0migration/014036-21fschecks_time23:30:220
149950,0migration/012290-21kthreadcore18:30:300
149950,0migration/012276-21if_eth123:10:250
149950,0migration/011497-21sh22:13:110
149950,0migration/011088-21kthreadcore21:15:290
8315040,0irq/26-0000:00:0-21swapper/219:25:082
8315040,0irq/26-0000:00:0-21swapper/219:05:382
29935040,0irq/33-eth0-tx-8268-21ssh20:40:112
29935040,0irq/33-eth0-tx-2980-21kthreadcore21:30:272
29935040,0irq/33-eth0-tx-17686-21ssh21:00:332
29935040,0irq/33-eth0-tx-14964-21tail22:35:222
29935040,0irq/33-eth0-tx-0-21swapper/223:20:502
29935040,0irq/33-eth0-tx-0-21swapper/223:16:092
29935040,0irq/33-eth0-tx-0-21swapper/222:50:132
29935040,0irq/33-eth0-tx-0-21swapper/222:40:392
29935040,0irq/33-eth0-tx-0-21swapper/222:25:242
29935040,0irq/33-eth0-tx-0-21swapper/222:20:292
29935040,0irq/33-eth0-tx-0-21swapper/222:11:522
29935040,0irq/33-eth0-tx-0-21swapper/221:55:102
29935040,0irq/33-eth0-tx-0-21swapper/220:25:142
29935040,0irq/33-eth0-tx-0-21swapper/220:21:372
29935040,0irq/33-eth0-tx-0-21swapper/019:20:250
29935040,0irq/33-eth0-tx-0-21swapper/019:00:130
259940,0migration/23184-21kthreadcore21:50:292
259940,0migration/231628-21kthreadcore18:45:272
259940,0migration/230212-21fschecks_count19:30:222
259940,0migration/229196-21kthreadcore22:05:282
259940,0migration/227409-21kthreadcore18:20:282
259940,0migration/216343-21kthreadcore19:40:272
259940,0migration/214325-21kthreadcore20:00:282
248449944,0cyclictest0-21swapper/218:35:222
248449943,0cyclictest4971-21sadc20:20:082
248449943,0cyclictest3640-21awk20:55:242
248449943,0cyclictest3599-21lldpd20:05:432
248449943,0cyclictest3599-21lldpd19:48:432
248449943,0cyclictest3599-21lldpd18:42:132
248449943,0cyclictest26165-21awk21:45:262
248449943,0cyclictest24203-21awk20:10:262
248449943,0cyclictest1606-21awk19:10:252
248449943,0cyclictest1043-21awk18:25:252
248449941,0cyclictest0-21swapper/222:55:202
248439943,0cyclictest9525-21awk20:40:251
248439943,0cyclictest3599-21lldpd23:13:241
248439943,0cyclictest3599-21lldpd22:46:511
248439943,0cyclictest3599-21lldpd21:38:501
248439943,0cyclictest30094-21awk18:45:251
248439943,0cyclictest2545-21awk21:50:281
248439943,0cyclictest23871-21snmpd22:50:401
248439943,0cyclictest23871-21snmpd22:00:401
248439943,0cyclictest23871-21snmpd20:55:101
248439943,0cyclictest23871-21snmpd18:40:401
248439943,0cyclictest17443-21awk21:40:251
248429943,0cyclictest8711-21awk21:35:260
248429943,0cyclictest3262-21irqbalance23:15:220
248429943,0cyclictest3262-21irqbalance19:36:130
248429943,0cyclictest3218-17auditd21:10:080
248429943,0cyclictest27522-21awk19:50:260
248429943,0cyclictest27201-21awk20:50:250
248429943,0cyclictest23871-21snmpd23:00:250
248429943,0cyclictest23871-21snmpd21:30:400
248429943,0cyclictest19982-21awk22:00:250
248429943,0cyclictest14990-21awk21:20:250
248429943,0cyclictest13586-21awk22:15:270
248429941,0cyclictest0-21swapper/019:30:210
249940,0watchdog/27136-21latency19:55:292
249940,0watchdog/26218-21kthreadcore18:50:292
249940,0watchdog/232624-21python19:50:332
249940,0watchdog/20-21swapper/220:45:292
19140,0ktimersoftd/10-21swapper/120:51:361
179940,0migration/13871-21irqrtprio23:05:261
179940,0migration/132197-21kthreadcore22:25:271
179940,0migration/131252-21runrttasks21:30:141
179940,0migration/131202-21irqrtprio19:30:261
179940,0migration/130692-21kthreadcore23:20:291
179940,0migration/130438-21aten_r1power_vo20:15:221
179940,0migration/129534-21unixbench-2d20:10:361
179940,0migration/129325-21kthreadcore22:05:281
179940,0migration/128225-21kthreadcore19:05:291
179940,0migration/125146-21aten_r1power_fr18:20:221
179940,0migration/124311-21kthreadcore23:35:281
179940,0migration/119779-21taskset22:17:101
179940,0migration/117181-21latency_hist22:55:081
179940,0migration/114392-21kthreadcore21:00:301
179940,0migration/112962-21taskset21:18:251
179940,0migration/11278-21kthreadcore20:35:311
179940,0migration/110970-21timerandwakeup19:35:361
169940,0watchdog/19786-21tr18:30:291
169940,0watchdog/15137-21sh23:41:481
169940,0watchdog/13740-21ssh22:10:161
169940,0watchdog/129219-21kthreadcore21:45:291
169940,0watchdog/113899-21aten_r1power_fr20:25:211
169940,0watchdog/113385-21cpu23:30:201
169940,0watchdog/11035-21iwlist18:25:251
169940,0watchdog/10-21swapper/123:15:161
169940,0watchdog/10-21swapper/122:20:201
169940,0watchdog/10-21swapper/121:20:291
169940,0watchdog/10-21swapper/121:05:251
169940,0watchdog/10-21swapper/118:50:251
169940,0watchdog/10-21swapper/118:35:291
149940,0migration/0841-21irqstats20:35:300
149940,0migration/03287-21gltestperf23:05:230
149940,0migration/028731-21kthreadcore21:45:280
149940,0migration/02456-21kthreadcore23:40:270
149940,0migration/023860-21irqrtprio19:25:270
149940,0migration/023208-21kthreadcore19:45:310
149940,0migration/021880-21runrttasks22:36:160
149940,0migration/019463-21kthreadcore21:40:280
149940,0migration/010066-21irqcore22:50:290
29935030,0irq/33-eth0-tx-8606-21aten2.4-expect20:00:210
248449932,0cyclictest3599-21lldpd18:55:422
248439932,0cyclictest3599-21lldpd21:55:201
248439932,0cyclictest23871-21snmpd20:00:241
248439932,0cyclictest23871-21snmpd19:55:231
248439932,0cyclictest23871-21snmpd18:55:401
248439932,0cyclictest23477-21awk19:25:261
248439932,0cyclictest22070-21sensors20:05:351
248429933,0cyclictest3340-21kworker/0:223:26:050
248429933,0cyclictest30765-21kworker/0:122:40:270
248429933,0cyclictest28574-21kworker/0:118:50:290
248429932,0cyclictest7921-21munin-node19:15:230
248429932,0cyclictest3262-21irqbalance19:40:130
248429932,0cyclictest23476-21processes20:45:330
248429930,0cyclictest14630-21perl20:25:230
169930,0watchdog/10-21swapper/123:01:161
169930,0watchdog/10-21swapper/122:41:521
169930,0watchdog/10-21swapper/120:20:251
169930,0watchdog/10-21swapper/119:15:251
169930,0watchdog/10-21swapper/119:10:371
169930,0watchdog/10-21swapper/119:00:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional