You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-23 - 23:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack1slot1.osadl.org (updated Tue Apr 23, 2024 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
297842209168,27sleep20-21swapper/207:05:572
300282198159,25sleep00-21swapper/007:09:090
298572198160,24sleep30-21swapper/307:06:553
298302198160,25sleep10-21swapper/107:06:331
143942680,1sleep09-21ksoftirqd/011:50:210
22182160,0sleep30-21swapper/309:25:003
30184991311,1cyclictest28-21ksoftirqd/211:35:002
3018499120,11cyclictest0-21swapper/210:55:182
3018499119,1cyclictest28-21ksoftirqd/207:55:002
3018499118,1cyclictest28-21ksoftirqd/207:50:002
3018999109,1cyclictest35-21ksoftirqd/310:15:003
3018999109,1cyclictest35-21ksoftirqd/309:15:003
3018999108,1cyclictest2332-21ssh12:18:423
3018499109,1cyclictest28-21ksoftirqd/207:24:592
3018499100,10cyclictest0-21swapper/209:40:592
3016899109,1cyclictest9-21ksoftirqd/010:25:000
3016899109,0cyclictest9-21ksoftirqd/009:56:100
3016899102,4cyclictest27811-21perf12:10:000
3016899102,4cyclictest27811-21perf12:10:000
3016899102,4cyclictest10493-21cat12:30:000
3016899100,6cyclictest7144-21/usr/sbin/munin10:15:150
301899990,9cyclictest32539-21munin-node11:52:183
301899990,8cyclictest0-21swapper/310:52:203
301899990,6cyclictest8123-21apt-get11:00:133
301849998,1cyclictest28-21ksoftirqd/210:50:182
301849998,1cyclictest28-21ksoftirqd/208:40:002
301849997,1cyclictest12892-21idleruntime-cro11:50:002
301689998,1cyclictest9-21ksoftirqd/009:30:140
301689998,1cyclictest9-21ksoftirqd/009:20:160
301689997,1cyclictest9-21ksoftirqd/010:00:150
301689997,1cyclictest9-21ksoftirqd/009:20:000
301689990,6cyclictest10733-21apt-get12:30:110
32411280,0sleep10-21swapper/110:48:541
301899988,0cyclictest35-21ksoftirqd/311:25:013
301899987,1cyclictest35-21ksoftirqd/310:28:403
301899987,1cyclictest35-21ksoftirqd/309:15:133
301899987,1cyclictest35-21ksoftirqd/307:50:003
301899986,1cyclictest8849-21snmpd12:39:283
301899986,1cyclictest8849-21snmpd10:49:463
301899986,1cyclictest8849-21snmpd08:49:363
301899984,2cyclictest530-21systemd-logind11:25:233
301899983,3cyclictest259-21systemd-journal09:00:003
301899980,7cyclictest0-21swapper/310:01:583
301899980,6cyclictest15704-1kworker/u9:209:42:153
301899980,6cyclictest0-21swapper/310:43:013
301899980,3cyclictest6997-21/usr/sbin/munin12:25:243
301899980,1cyclictest0-21swapper/312:32:563
301849988,0cyclictest28-21ksoftirqd/212:25:132
301849987,1cyclictest28-21ksoftirqd/210:30:132
301849986,1cyclictest8849-21snmpd10:05:082
301849986,1cyclictest61250irq/122-eno110:40:162
301849986,1cyclictest28-21ksoftirqd/212:20:012
301849984,4cyclictest11909-21kworker/2:011:10:002
301849981,6cyclictest11486-21munin-node08:45:102
301849981,0cyclictest23056-21diskmemload11:21:182
301849980,4cyclictest24308-21apt-get12:05:092
301849980,4cyclictest24308-21apt-get12:05:092
301849980,0cyclictest0-21swapper/212:39:462
301689987,1cyclictest9-21ksoftirqd/011:35:110
301689987,1cyclictest9-21ksoftirqd/011:10:200
301689987,1cyclictest9-21ksoftirqd/010:50:000
301689987,1cyclictest9-21ksoftirqd/010:35:130
301689987,1cyclictest9-21ksoftirqd/008:15:010
301689987,1cyclictest1422-21ntpd12:18:190
301689986,1cyclictest8849-21snmpd08:45:450
301689982,5cyclictest31848-21cron08:20:010
301689982,5cyclictest23056-21diskmemload10:10:100
301689981,6cyclictest32607-21latency_hist07:15:000
301689981,6cyclictest20273-21ssh12:00:000
301689981,3cyclictest23056-21diskmemload12:11:560
301689981,3cyclictest0-21swapper/009:35:180
301689981,1cyclictest259-21systemd-journal09:10:000
301689981,1cyclictest13767-21ssh11:05:450
301689980,7cyclictest30613-21ssh11:30:030
301689980,1cyclictest8042-21ssh11:41:530
8709270,2sleep00-21swapper/008:38:010
301899976,1cyclictest8849-21snmpd09:33:223
301899976,1cyclictest35-21ksoftirqd/310:05:153
301899976,1cyclictest1422-21ntpd10:55:293
301899975,1cyclictest8849-21snmpd10:39:093
301899975,1cyclictest8849-21snmpd10:19:063
301899975,1cyclictest8849-21snmpd08:51:213
301899975,1cyclictest8849-21snmpd08:36:293
301899975,1cyclictest8849-21snmpd07:15:363
301899975,0cyclictest23056-21diskmemload10:31:033
301899971,5cyclictest8849-21snmpd07:51:253
301899971,4cyclictest4402-21apt-get08:30:103
301899970,6cyclictest0-21swapper/311:42:523
301899970,5cyclictest32112-21apt-get08:20:093
301899970,4cyclictest0-21swapper/311:45:023
301899970,4cyclictest0-21swapper/309:27:383
301899970,2cyclictest0-21swapper/312:05:333
301899970,2cyclictest0-21swapper/312:05:333
301849976,1cyclictest8849-21snmpd10:45:252
301849976,1cyclictest8849-21snmpd09:21:142
301849976,1cyclictest61250irq/122-eno112:33:502
301849976,1cyclictest61250irq/122-eno111:55:172
301849976,1cyclictest61250irq/122-eno109:39:482
301849976,1cyclictest61250irq/122-eno109:30:212
301849976,1cyclictest61250irq/122-eno109:10:132
301849976,1cyclictest61250irq/122-eno107:10:252
301849976,1cyclictest28-21ksoftirqd/208:55:002
301849976,1cyclictest28-21ksoftirqd/207:20:002
301849976,0cyclictest61250irq/122-eno112:00:132
301849975,2cyclictest28-21ksoftirqd/208:12:352
301849975,1cyclictest61250irq/122-eno112:10:272
301849975,1cyclictest61250irq/122-eno109:45:102
301849972,5cyclictest32539-21munin-node08:04:232
301849972,4cyclictest8849-21snmpd09:57:352
301849971,3cyclictest28-21ksoftirqd/209:52:162
301849971,3cyclictest0-21swapper/207:35:132
301849970,7cyclictest0-21swapper/209:25:162
301849970,5cyclictest534-21nscd10:26:382
301849970,5cyclictest2436-21kworker/u8:110:15:322
301849970,4cyclictest0-21swapper/211:17:352
301849970,2cyclictest0-21swapper/211:50:222
301849970,1cyclictest0-21swapper/207:57:372
301799977,0cyclictest0-21swapper/109:20:551
301799972,2cyclictest0-21swapper/111:09:421
301799971,6cyclictest0-21swapper/108:41:341
301799971,3cyclictest0-21swapper/107:50:101
301799971,1cyclictest0-21swapper/108:05:111
301799970,4cyclictest0-21swapper/110:25:301
301799970,1cyclictest0-21swapper/108:17:411
301689977,0cyclictest9-21ksoftirqd/012:35:130
301689977,0cyclictest9-21ksoftirqd/011:15:130
301689976,1cyclictest9-21ksoftirqd/009:10:240
301689975,1cyclictest9-21ksoftirqd/009:50:110
301689975,1cyclictest9-21ksoftirqd/008:05:000
301689975,1cyclictest8849-21snmpd08:12:200
301689975,1cyclictest8849-21snmpd07:23:240
301689975,1cyclictest29110-21munin-run10:45:000
301689974,1cyclictest23056-21diskmemload09:25:200
301689972,1cyclictest0-21swapper/008:30:170
301689971,5cyclictest9974-21ssh11:45:140
301689971,1cyclictest11240-21expr07:35:170
301689970,6cyclictest4224-21perf08:30:000
301689970,6cyclictest23056-21diskmemload11:20:200
301689970,6cyclictest23056-21diskmemload10:05:000
301689970,6cyclictest15552-21ssh09:41:110
301689970,6cyclictest14848-21taskset08:50:160
301689970,6cyclictest0-21swapper/012:20:160
301689970,6cyclictest0-21swapper/010:55:200
301689970,6cyclictest0-21swapper/010:25:090
301689970,6cyclictest0-21swapper/007:15:160
301689970,4cyclictest18195-21apt-get10:30:100
301689970,4cyclictest0-21swapper/007:25:110
301689970,3cyclictest0-21swapper/010:50:180
301689970,2cyclictest0-21swapper/012:00:130
301689970,1cyclictest19690-21ssh09:47:210
301689970,1cyclictest0-21swapper/011:04:230
22229270,0sleep10-21swapper/109:50:211
12290270,0sleep10-21swapper/109:37:301
301899965,1cyclictest8849-21snmpd09:57:053
301899965,1cyclictest8849-21snmpd09:09:033
301899965,1cyclictest8849-21snmpd08:41:363
301899965,1cyclictest21395-21sshd11:16:103
301899965,1cyclictest21323-21ssh12:00:143
301899965,1cyclictest20854-21ssh09:50:033
301899965,1cyclictest18857-21ssh11:14:293
301899965,1cyclictest12005-21ssh09:36:413
301899964,1cyclictest8849-21snmpd11:55:423
301899964,1cyclictest8849-21snmpd07:36:333
301899964,1cyclictest14884-21ssh11:08:453
301899964,1cyclictest13711-21cat07:40:163
301899962,1cyclictest1973-21apt-get11:35:103
301899960,4cyclictest0-21swapper/312:12:573
301899960,3cyclictest0-21swapper/309:45:093
301899960,2cyclictest30823-21aten_r1power_vo11:30:123
301849966,0cyclictest61250irq/122-eno111:25:082
301849966,0cyclictest61250irq/122-eno111:10:062
301849966,0cyclictest61250irq/122-eno110:10:232
301849966,0cyclictest61250irq/122-eno109:06:502
301849966,0cyclictest61250irq/122-eno108:58:552
301849966,0cyclictest61250irq/122-eno108:05:192
301849965,1cyclictest8849-21snmpd08:32:442
301849965,1cyclictest61250irq/122-eno112:21:232
301849965,1cyclictest61250irq/122-eno110:00:112
301849965,1cyclictest61250irq/122-eno108:40:142
301849965,1cyclictest28-21ksoftirqd/211:00:172
301849965,1cyclictest28-21ksoftirqd/210:35:112
301849964,1cyclictest61250irq/122-eno109:15:012
301849964,1cyclictest61250irq/122-eno109:00:232
301849964,1cyclictest28-21ksoftirqd/207:40:142
301849962,3cyclictest5715-21apt-get11:40:112
301849960,5cyclictest0-21swapper/207:32:212
301849960,3cyclictest0-21swapper/210:20:102
301849960,2cyclictest0-21swapper/211:35:112
301849960,2cyclictest0-21swapper/208:15:122
301849960,1cyclictest0-21swapper/208:25:112
301799966,0cyclictest0-21swapper/110:51:311
301799965,1cyclictest0-21swapper/107:18:541
301799962,1cyclictest0-21swapper/109:25:231
301799960,6cyclictest0-21swapper/112:12:571
301799960,6cyclictest0-21swapper/111:12:051
301799960,6cyclictest0-21swapper/110:14:061
301799960,6cyclictest0-21swapper/108:28:381
301799960,6cyclictest0-21swapper/108:13:111
301799960,6cyclictest0-21swapper/107:39:341
301799960,5cyclictest0-21swapper/111:15:591
301799960,5cyclictest0-21swapper/109:15:391
301799960,5cyclictest0-21swapper/109:05:511
301799960,5cyclictest0-21swapper/107:20:141
301799960,4cyclictest0-21swapper/112:37:241
301799960,4cyclictest0-21swapper/112:01:411
301799960,4cyclictest0-21swapper/111:40:251
301799960,4cyclictest0-21swapper/110:30:311
301799960,4cyclictest0-21swapper/109:31:121
301799960,3cyclictest0-21swapper/111:58:121
301799960,3cyclictest0-21swapper/111:38:261
301799960,3cyclictest0-21swapper/110:55:011
301799960,3cyclictest0-21swapper/110:41:311
301799960,3cyclictest0-21swapper/110:37:251
301799960,3cyclictest0-21swapper/107:30:121
301799960,1cyclictest0-21swapper/110:20:231
301799960,1cyclictest0-21swapper/109:56:411
301799960,1cyclictest0-21swapper/108:50:151
301799960,0cyclictest0-21swapper/109:14:001
301689965,1cyclictest18577-21apt-get09:00:150
301689964,1cyclictest9-21ksoftirqd/008:58:460
301689964,1cyclictest8849-21snmpd07:53:410
301689963,2cyclictest8882-21sed07:30:160
301689962,2cyclictest23056-21diskmemload11:25:130
301689960,4cyclictest0-21swapper/007:45:170
301689960,3cyclictest0-21swapper/008:05:110
301689960,0cyclictest0-21swapper/007:55:120
11772260,1sleep10-21swapper/111:46:571
301899954,1cyclictest8849-21snmpd09:02:183
301899953,1cyclictest8849-21snmpd08:14:473
301899953,1cyclictest548-21in:imuxsock07:59:563
301899953,1cyclictest32436-21taskset07:14:183
301899953,1cyclictest11372-21expr10:20:153
301899951,3cyclictest3286-21/usr/sbin/munin12:20:213
301899950,3cyclictest0-21swapper/308:00:103
301899950,2cyclictest259-21systemd-journal08:17:003
301849954,1cyclictest28-21ksoftirqd/207:25:132
301849952,1cyclictest530-21systemd-logind08:20:182
301799951,4cyclictest0-21swapper/108:21:221
301799951,3cyclictest736-21chrt11:32:231
301799951,1cyclictest0-21swapper/109:45:101
301799950,5cyclictest0-21swapper/109:03:081
301799950,5cyclictest0-21swapper/107:43:381
301799950,5cyclictest0-21swapper/107:11:091
301799950,4cyclictest0-21swapper/112:27:311
301799950,3cyclictest0-21swapper/111:50:181
301799950,3cyclictest0-21swapper/111:03:471
301799950,3cyclictest0-21swapper/110:05:451
301799950,2cyclictest0-21swapper/112:30:221
301799950,2cyclictest0-21swapper/112:21:041
301799950,2cyclictest0-21swapper/112:17:051
301799950,2cyclictest0-21swapper/112:09:031
301799950,2cyclictest0-21swapper/112:09:021
301799950,2cyclictest0-21swapper/111:20:091
301799950,2cyclictest0-21swapper/110:18:501
301799950,2cyclictest0-21swapper/110:00:141
301799950,2cyclictest0-21swapper/109:44:221
301799950,1cyclictest0-21swapper/107:25:111
301799950,0cyclictest0-21swapper/111:25:191
301689954,1cyclictest9-21ksoftirqd/008:40:150
301689951,4cyclictest12896-21telnet07:40:130
301899944,0cyclictest9541-21sensors07:30:483
301899943,1cyclictest3897-21diskstats07:20:113
301899942,1cyclictest25610-21gltestperf08:05:163
301899940,2cyclictest2051-21apt-get08:25:123
301899940,1cyclictest5743-21apt-get07:25:103
301799944,0cyclictest0-21swapper/107:56:151
301799940,3cyclictest0-21swapper/108:30:101
301799940,1cyclictest0-21swapper/108:35:121
301799940,0cyclictest0-21swapper/108:00:101
301799931,1cyclictest0-21swapper/108:55:091
301799930,1cyclictest0-21swapper/108:45:001
15395230,1sleep10-21swapper/107:45:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional