You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-22 - 00:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Tue Jan 21, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1684628683,0sleep10-21swapper/106:35:351
17168994343,0cyclictest17018-21md1_resync08:30:482
1675723329,0sleep00-21swapper/006:34:340
1695022824,0sleep30-21swapper/306:36:453
1633822824,0sleep20-21swapper/206:33:442
329980,0watchdog/316406-21kthreadcore11:28:383
8315070,0irq/26-0000:00:20405-21sed06:58:403
8315070,0irq/26-0000:00:125142sleep308:02:163
8315070,0irq/26-0000:00:0-21swapper/311:38:073
8315070,0irq/26-0000:00:0-21swapper/311:17:333
8315070,0irq/26-0000:00:0-21swapper/310:48:373
8315070,0irq/26-0000:00:0-21swapper/308:52:293
8315070,0irq/26-0000:00:0-21swapper/307:18:333
8315070,0irq/26-0000:00:0-21swapper/306:46:223
329970,0watchdog/327750-21cpu08:13:303
329970,0watchdog/315867-21sh09:35:343
329970,0watchdog/30-21swapper/311:54:343
329970,0watchdog/30-21swapper/310:18:383
329970,0watchdog/30-21swapper/309:31:263
329970,0watchdog/30-21swapper/308:54:263
29925070,0irq/32-eth0-rx-0-21swapper/311:21:453
171649970,0cyclictest31927-21irqrtprio09:23:351
8315060,0irq/26-0000:00:6386-21runrttasks07:44:143
8315060,0irq/26-0000:00:4815-21chrt07:53:513
8315060,0irq/26-0000:00:4331-21ssh10:38:313
8315060,0irq/26-0000:00:29439-21sensors_volt07:48:393
8315060,0irq/26-0000:00:28124-21if_err_eth1.208:58:293
8315060,0irq/26-0000:00:26099-21perl09:43:323
8315060,0irq/26-0000:00:23413-21kthreadcore07:23:373
8315060,0irq/26-0000:00:16415-21kthreadcore08:03:363
8315060,0irq/26-0000:00:15802-21kthreadcore08:23:363
8315060,0irq/26-0000:00:0-21swapper/311:48:353
8315060,0irq/26-0000:00:0-21swapper/311:10:073
8315060,0irq/26-0000:00:0-21swapper/311:03:453
8315060,0irq/26-0000:00:0-21swapper/310:54:453
8315060,0irq/26-0000:00:0-21swapper/310:33:213
8315060,0irq/26-0000:00:0-21swapper/310:28:483
8315060,0irq/26-0000:00:0-21swapper/310:23:303
8315060,0irq/26-0000:00:0-21swapper/310:03:453
8315060,0irq/26-0000:00:0-21swapper/309:13:593
8315060,0irq/26-0000:00:0-21swapper/308:08:353
8315060,0irq/26-0000:00:0-21swapper/307:08:363
8315060,0irq/26-0000:00:0-21swapper/307:08:363
339960,0migration/38448-21aten_r1power_cu07:38:303
329960,0watchdog/30-21swapper/311:46:543
329960,0watchdog/30-21swapper/311:25:303
329960,0watchdog/30-21swapper/310:59:383
329960,0watchdog/30-21swapper/310:08:423
329960,0watchdog/30-21swapper/309:59:063
329960,0watchdog/30-21swapper/309:55:223
329960,0watchdog/30-21swapper/308:45:383
329960,0watchdog/30-21swapper/308:38:503
329960,0watchdog/30-21swapper/308:33:463
329960,0watchdog/30-21swapper/308:31:143
329960,0watchdog/30-21swapper/307:32:503
329960,0watchdog/30-21swapper/307:13:223
329960,0watchdog/30-21swapper/306:53:343
329960,0watchdog/30-21swapper/306:40:023
29925060,0irq/32-eth0-rx-30906-21ssh11:38:493
29925060,0irq/32-eth0-rx-0-21swapper/311:58:253
29925060,0irq/32-eth0-rx-0-21swapper/310:43:283
29925060,0irq/32-eth0-rx-0-21swapper/310:16:563
29925060,0irq/32-eth0-rx-0-21swapper/309:49:563
29925060,0irq/32-eth0-rx-0-21swapper/309:41:293
29925060,0irq/32-eth0-rx-0-21swapper/309:23:433
29925060,0irq/32-eth0-rx-0-21swapper/309:20:183
29925060,0irq/32-eth0-rx-0-21swapper/309:09:233
29925060,0irq/32-eth0-rx-0-21swapper/309:03:423
19160,0ktimersoftd/10-21swapper/109:04:501
19160,0ktimersoftd/10-21swapper/108:03:391
171639965,0cyclictest5095-21ssh09:24:070
171639965,0cyclictest21818-21ssh09:18:070
179960,0migration/121855-21munin-node11:58:301
149960,0migration/024281-21sh08:50:430
8315050,0irq/26-0000:00:0-21swapper/306:48:443
329950,0watchdog/30-21swapper/308:18:183
329950,0watchdog/30-21swapper/307:33:343
329950,0watchdog/30-21swapper/307:03:383
29935050,0irq/33-eth0-tx-6315-21iceccd07:04:000
29935050,0irq/33-eth0-tx-32033-21munin-node06:48:450
29935050,0irq/33-eth0-tx-27586-21munin-node08:13:380
29935050,0irq/33-eth0-tx-22739-21cut11:58:312
29935050,0irq/33-eth0-tx-15780-21munin-node07:43:390
29935050,0irq/33-eth0-tx-0-21swapper/211:38:332
29935050,0irq/33-eth0-tx-0-21swapper/208:44:062
29925050,0irq/32-eth0-rx-0-21swapper/312:03:333
27150,0ktimersoftd/20-21swapper/211:46:192
259950,0migration/24541-21sh10:12:152
249950,0watchdog/20-21swapper/210:43:302
19150,0ktimersoftd/10-21swapper/111:46:191
171689953,0cyclictest3599-21lldpd06:42:132
171649955,0cyclictest0-21swapper/110:11:121
171649955,0cyclictest0-21swapper/109:56:421
171649955,0cyclictest0-21swapper/109:13:581
171649955,0cyclictest0-21swapper/109:10:591
171649955,0cyclictest0-21swapper/107:32:451
171649955,0cyclictest0-21swapper/107:01:421
171649950,0cyclictest0-21swapper/111:10:291
171649950,0cyclictest0-21swapper/110:23:581
171649950,0cyclictest0-21swapper/109:01:221
171639955,0cyclictest0-21swapper/011:32:360
171639955,0cyclictest0-21swapper/011:16:030
171639955,0cyclictest0-21swapper/010:10:180
171639955,0cyclictest0-21swapper/007:13:250
171639950,0cyclictest0-21swapper/009:33:440
171639950,0cyclictest0-21swapper/009:21:070
171639950,0cyclictest0-21swapper/007:59:370
179950,0migration/130412-21irqcore12:03:391
179950,0migration/129841-21kthreadcore08:33:461
179950,0migration/125464-21kthreadcore07:48:341
179950,0migration/122777-21taskset07:47:061
179950,0migration/111070-21kthreadcore11:03:371
179950,0migration/110200-21munin-node07:13:381
169950,0watchdog/131845-21df_abs10:58:261
169950,0watchdog/130114-21kthreadcore06:43:341
149950,0migration/09553-21kthreadcore06:53:340
149950,0migration/08687-21munin-node07:38:320
149950,0migration/08282-21kthreadcore09:28:420
149950,0migration/07556-21latency09:48:370
149950,0migration/06731-21sh08:39:490
149950,0migration/05394-21kthreadcore10:38:310
149950,0migration/05327-21kthreadcore08:18:400
149950,0migration/030105-21kthreadcore10:28:360
149950,0migration/029268-21munin-node07:48:370
149950,0migration/029156-21kthreadcore06:43:340
149950,0migration/029091-21kthreadcore08:58:310
149950,0migration/024043-21aten2.4-expect11:38:240
149950,0migration/023330-21sendmail_mailst09:38:370
149950,0migration/022640-21kthreadcore10:03:380
149950,0migration/022393-21timerandwakeup10:23:430
149950,0migration/019125-21kthreadcore06:38:360
149950,0migration/016598-21kthreadcore07:18:390
149950,0migration/016401-21switchtime08:23:440
149950,0migration/01560-21munin-node11:43:340
149950,0migration/014173-21kthreadcore10:43:370
149950,0migration/011793-21sh08:43:350
149950,0migration/010925-21timerwakeupswit11:23:440
29935040,0irq/33-eth0-tx-31805-21diskmemload08:39:182
29935040,0irq/33-eth0-tx-29403-21ssh10:53:342
29935040,0irq/33-eth0-tx-29170-21ssh12:03:332
29935040,0irq/33-eth0-tx-27058-21munin-node08:58:292
29935040,0irq/33-eth0-tx-2354-21ssh11:19:112
29935040,0irq/33-eth0-tx-0-21swapper/211:54:572
29935040,0irq/33-eth0-tx-0-21swapper/211:34:472
29935040,0irq/33-eth0-tx-0-21swapper/211:24:012
29935040,0irq/33-eth0-tx-0-21swapper/211:13:382
29935040,0irq/33-eth0-tx-0-21swapper/211:08:312
29935040,0irq/33-eth0-tx-0-21swapper/211:03:262
29935040,0irq/33-eth0-tx-0-21swapper/210:33:582
29935040,0irq/33-eth0-tx-0-21swapper/210:31:102
29935040,0irq/33-eth0-tx-0-21swapper/210:18:372
29935040,0irq/33-eth0-tx-0-21swapper/210:17:022
29935040,0irq/33-eth0-tx-0-21swapper/209:58:392
29935040,0irq/33-eth0-tx-0-21swapper/209:55:272
29935040,0irq/33-eth0-tx-0-21swapper/209:44:392
29935040,0irq/33-eth0-tx-0-21swapper/209:39:422
29935040,0irq/33-eth0-tx-0-21swapper/209:28:462
29935040,0irq/33-eth0-tx-0-21swapper/209:23:282
29935040,0irq/33-eth0-tx-0-21swapper/209:18:342
29935040,0irq/33-eth0-tx-0-21swapper/209:15:482
29935040,0irq/33-eth0-tx-0-21swapper/008:33:320
29935040,0irq/33-eth0-tx-0-21swapper/007:28:390
29935040,0irq/33-eth0-tx-0-21swapper/007:23:310
259940,0migration/27147-21kthreadcore09:03:372
259940,0migration/232749-21runrttasks07:28:542
259940,0migration/22768-21aten2.4-expect10:38:252
259940,0migration/224559-21kthreadcore10:03:382
259940,0migration/215214-21sh09:33:332
249940,0watchdog/29477-21aten2.4-expect11:48:302
249940,0watchdog/226178-21ssh08:55:422
249940,0watchdog/217517-21grep08:48:302
249940,0watchdog/212991-21idleruntime09:08:302
249940,0watchdog/20-21swapper/210:58:262
249940,0watchdog/20-21swapper/206:43:342
171689943,0cyclictest3599-21lldpd08:27:002
171689943,0cyclictest3599-21lldpd08:04:302
171689943,0cyclictest3599-21lldpd07:10:242
171689943,0cyclictest3599-21lldpd07:10:242
171689943,0cyclictest3599-21lldpd07:03:232
171689943,0cyclictest3262-21irqbalance08:36:122
171689943,0cyclictest3262-21irqbalance07:58:222
171689943,0cyclictest3262-21irqbalance07:54:222
171689943,0cyclictest3262-21irqbalance07:34:122
171689943,0cyclictest3262-21irqbalance07:25:222
171689943,0cyclictest3262-21irqbalance06:53:122
171689943,0cyclictest28622-21awk08:13:332
171689943,0cyclictest24256-21awk07:48:322
171689943,0cyclictest23871-21snmpd09:48:472
171689943,0cyclictest23871-21snmpd08:20:172
171689943,0cyclictest23871-21snmpd07:44:322
171689943,0cyclictest23871-21snmpd06:53:472
171689943,0cyclictest21548-21awk10:48:342
171689943,0cyclictest16977-21awk10:23:342
171689943,0cyclictest15496-21awk06:58:372
171689943,0cyclictest14231-21awk11:28:362
171689943,0cyclictest12437-21awk07:18:362
171689943,0cyclictest11150-21sadc07:18:182
171649944,0cyclictest0-21swapper/111:54:191
171649944,0cyclictest0-21swapper/110:33:071
171649944,0cyclictest0-21swapper/108:47:481
171649944,0cyclictest0-21swapper/108:33:081
171649944,0cyclictest0-21swapper/107:09:251
171649944,0cyclictest0-21swapper/107:09:251
171649943,0cyclictest700-21awk06:48:351
171649943,0cyclictest3599-21lldpd11:33:181
171649943,0cyclictest3599-21lldpd11:15:181
171649943,0cyclictest3599-21lldpd09:36:301
171649943,0cyclictest3599-21lldpd09:29:271
171649943,0cyclictest3599-21lldpd08:19:001
171649943,0cyclictest31526-21awk07:53:331
171649943,0cyclictest25135-21crond08:53:181
171649943,0cyclictest23871-21snmpd10:55:181
171649943,0cyclictest21441-21awk10:03:351
171649943,0cyclictest21421-21awk08:08:331
171649943,0cyclictest11023-21awk08:23:321
171649943,0cyclictest0-21swapper/108:53:571
171639944,0cyclictest0-21swapper/010:56:300
171639944,0cyclictest0-21swapper/010:33:570
171639943,0cyclictest4538-21awk09:03:340
171639943,0cyclictest3262-21irqbalance11:59:520
171639943,0cyclictest3262-21irqbalance09:56:320
171639943,0cyclictest3262-21irqbalance09:45:020
171639943,0cyclictest3262-21irqbalance08:28:320
171639943,0cyclictest3262-21irqbalance08:09:220
171639943,0cyclictest3262-21irqbalance08:05:420
171639943,0cyclictest23871-21snmpd12:03:180
171639943,0cyclictest23871-21snmpd07:56:320
171639943,0cyclictest2070-21awk07:33:330
171639943,0cyclictest10566-21chrt11:48:340
179940,0migration/15658-21kthreadcore09:48:361
179940,0migration/15331-21taskset10:13:551
179940,0migration/130938-21kthreadcore08:13:361
179940,0migration/122225-21kthreadcore07:23:371
179940,0migration/122087-21latency_hist09:18:181
179940,0migration/118925-21kthreadcore09:38:321
179940,0migration/117833-21fschecks_count06:38:321
179940,0migration/114337-21kthreadcore09:58:311
179940,0migration/112622-21kthreadcore10:18:321
179940,0migration/111512-21aten2.4-expect07:18:321
169940,0watchdog/17262-21ssh08:41:261
169940,0watchdog/132082-21ssh09:43:461
169940,0watchdog/116405-21kthreadcore11:28:381
169940,0watchdog/113272-21processes06:53:381
169940,0watchdog/111641-21aten_r1power_cu10:43:301
149940,0migration/03706-21kthreadcore10:58:310
149940,0migration/018672-21sh09:58:520
149940,0migration/015031-21runrttasks11:04:240
149940,0migration/01288-21kthreadcore11:18:320
171689932,0cyclictest3599-21lldpd07:38:272
171689932,0cyclictest20219-21date08:08:182
171649932,0cyclictest9510-21awk07:38:351
171649932,0cyclictest26899-21sensors10:48:411
171649931,0cyclictest25077-21awk11:38:281
171649930,0cyclictest5898-21df_inode07:58:471
171639933,0cyclictest0-21swapper/007:08:380
171639933,0cyclictest0-21swapper/007:08:380
171639932,0cyclictest31805-21diskmemload10:17:460
171639932,0cyclictest23871-21snmpd08:54:470
171639932,0cyclictest21629-21awk10:48:350
171639932,0cyclictest21205-21cut11:08:380
171639930,0cyclictest19629-21runrttasks11:57:130
171639930,0cyclictest14368-21munin-node06:58:360
179930,0migration/19770-21df_abs11:48:311
169930,0watchdog/117018-21md1_resync07:35:221
169930,0watchdog/10-21swapper/111:23:181
169930,0watchdog/10-21swapper/111:18:341
169930,0watchdog/10-21swapper/110:39:261
169930,0watchdog/10-21swapper/110:34:421
149930,0migration/021890-21diskmemload11:33:220
171639920,0cyclictest0-21swapper/010:18:190
171639920,0cyclictest0-21swapper/009:08:280
169920,0watchdog/10-21swapper/107:03:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional