You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-03 - 07:30

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.16.15-rt7 (Profile)

Latency plot of system in rack #b, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackbslot1.osadl.org (updated Sun Jul 03, 2022 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3253421000,2sleep21068099cyclictest21:30:272
1036326037,7sleep20-21swapper/219:08:102
1035625633,7sleep00-21swapper/019:08:040
1053125139,7sleep30-21swapper/319:09:473
1036225139,7sleep10-21swapper/119:08:091
10674991614,1cyclictest17929-21snmpd19:49:581
39622150,0sleep00-21swapper/023:25:180
10667991513,1cyclictest17929-21snmpd23:38:450
10687991413,1cyclictest17929-21snmpd23:26:113
10680991311,1cyclictest17929-21snmpd00:29:042
10674991311,1cyclictest31494-31munin-html19:40:311
10667991311,1cyclictest17929-21snmpd20:52:190
10667991311,1cyclictest17929-21snmpd00:34:590
10667991310,2cyclictest26332-31munin-graph21:20:350
231202120,1sleep00-21swapper/020:20:400
10687991211,0cyclictest17929-21snmpd21:32:553
10687991210,1cyclictest8562-21munin-update00:30:273
10687991210,1cyclictest8084-31munin-html19:55:293
10687991210,1cyclictest17929-21snmpd22:34:373
10687991210,1cyclictest16408-31munin-html23:45:343
1068799120,11cyclictest21808-31munin-graph23:00:403
1068099120,11cyclictest0-21swapper/222:50:162
10674991210,1cyclictest17929-21snmpd20:11:051
1067499120,11cyclictest0-21swapper/122:50:441
1066799124,4cyclictest12765-21munin-update21:55:000
10667991210,1cyclictest3864-31munin-html22:30:290
1066799120,1cyclictest5677-31munin-html20:45:320
1068799119,1cyclictest32069-31munin-html20:35:313
1068799119,1cyclictest0-21swapper/320:50:223
10687991111,0cyclictest17929-21snmpd23:13:563
1068799110,10cyclictest0-21swapper/323:59:573
1068099119,1cyclictest17929-21snmpd23:55:102
1068099119,1cyclictest0-21swapper/222:43:002
1068099119,1cyclictest0-21swapper/222:20:142
10680991111,0cyclictest17929-21snmpd19:40:212
10680991111,0cyclictest17929-21snmpd19:35:542
1068099110,1cyclictest0-21swapper/222:05:032
1068099110,10cyclictest0-21swapper/223:25:382
1068099110,10cyclictest0-21swapper/221:15:192
1067499119,1cyclictest0-21swapper/121:36:371
1067499110,1cyclictest0-21swapper/120:05:431
1067499110,1cyclictest0-21swapper/100:15:201
1066799119,1cyclictest17929-21snmpd22:43:100
1066799119,1cyclictest0-21swapper/022:50:410
1066799110,1cyclictest0-21swapper/019:55:390
1066799110,1cyclictest0-21swapper/000:15:370
1068799109,0cyclictest14043-31munin-graph00:35:373
1068799109,0cyclictest0-21swapper/321:46:543
1068799108,1cyclictest0-21swapper/322:59:143
1068799108,1cyclictest0-21swapper/322:50:503
1068799108,1cyclictest0-21swapper/321:53:403
1068799108,1cyclictest0-21swapper/320:29:523
1068799108,1cyclictest0-21swapper/320:09:043
1068799108,1cyclictest0-21swapper/300:14:193
1068799100,9cyclictest0-21swapper/323:18:463
1068799100,9cyclictest0-21swapper/319:52:253
1068799100,9cyclictest0-21swapper/319:17:043
1068799100,1cyclictest0-21swapper/320:43:163
1068799100,0cyclictest0-21swapper/321:55:273
1068799100,0cyclictest0-21swapper/320:45:043
1068099108,1cyclictest0-21swapper/200:00:292
1068099100,9cyclictest0-21swapper/221:13:132
1068099100,0cyclictest0-21swapper/223:00:272
1067499109,1cyclictest17929-21snmpd19:11:251
1067499108,1cyclictest0-21swapper/123:58:131
1067499108,1cyclictest0-21swapper/123:49:491
1067499108,1cyclictest0-21swapper/121:15:381
1067499100,9cyclictest0-21swapper/120:27:021
1067499100,1cyclictest0-21swapper/119:26:291
1067499100,1cyclictest0-21swapper/100:37:361
1066799108,1cyclictest0-21swapper/023:30:330
10667991010,0cyclictest17929-21snmpd21:37:580
106879997,1cyclictest0-21swapper/322:19:123
106879997,1cyclictest0-21swapper/319:33:223
106879990,1cyclictest0-21swapper/322:39:313
106809998,0cyclictest0-21swapper/223:11:152
106809997,1cyclictest17929-21snmpd00:11:222
106809997,1cyclictest0-21swapper/219:26:212
106749997,1cyclictest0-21swapper/122:11:111
106749997,1cyclictest0-21swapper/120:50:171
106749997,1cyclictest0-21swapper/120:41:051
106749990,8cyclictest0-21swapper/123:13:111
106679998,0cyclictest0-21swapper/023:55:050
106679997,1cyclictest1441-31munin-html23:20:320
106679997,1cyclictest0-21swapper/021:45:070
106679997,1cyclictest0-21swapper/019:22:530
106679990,8cyclictest0-21swapper/021:05:340
106879986,1cyclictest0-21swapper/323:35:423
106879986,1cyclictest0-21swapper/323:31:033
106879986,1cyclictest0-21swapper/321:29:273
106879986,1cyclictest0-21swapper/321:14:463
106879986,1cyclictest0-21swapper/320:31:403
106879986,1cyclictest0-21swapper/320:16:293
106879986,1cyclictest0-21swapper/320:01:563
106879986,1cyclictest0-21swapper/300:20:173
106879980,7cyclictest0-21swapper/322:13:413
106809986,1cyclictest17136-31munin-graph20:10:352
106809986,1cyclictest0-21swapper/222:17:202
106809986,1cyclictest0-21swapper/219:15:122
106809986,1cyclictest0-21swapper/200:35:422
106809980,7cyclictest0-21swapper/223:22:232
106809980,7cyclictest0-21swapper/221:59:252
106809980,7cyclictest0-21swapper/200:07:082
106809980,1cyclictest0-21swapper/223:32:052
106749986,1cyclictest8031-31munin-html00:25:311
106749986,1cyclictest0-21swapper/123:17:521
106749986,1cyclictest0-21swapper/121:44:521
106749986,1cyclictest0-21swapper/120:36:521
106749986,1cyclictest0-21swapper/119:51:311
106749980,1cyclictest0-21swapper/121:01:541
106749980,1cyclictest0-21swapper/119:39:541
106679987,0cyclictest0-21swapper/022:15:190
106679986,1cyclictest0-21swapper/023:50:350
106679986,1cyclictest0-21swapper/022:49:140
106679986,1cyclictest0-21swapper/022:01:450
106679986,1cyclictest0-21swapper/020:30:510
106679980,4cyclictest27768-21kworker/0:122:40:000
6803270,0sleep10-21swapper/123:30:161
23840270,0sleep20-21swapper/222:10:192
22997270,1sleep30-21swapper/319:29:493
22991270,1sleep00-21swapper/019:29:460
106879976,0cyclictest0-21swapper/300:28:223
106879975,1cyclictest17929-21snmpd23:40:183
106879975,1cyclictest0-21swapper/323:51:323
106879975,1cyclictest0-21swapper/323:20:133
106879975,1cyclictest0-21swapper/321:07:003
106809975,1cyclictest3736-21isc-worker000020:40:522
106809975,1cyclictest3736-21isc-worker000019:24:242
106809975,1cyclictest2039-31munin-html00:15:312
106809975,1cyclictest17929-21snmpd19:12:032
106809975,1cyclictest0-21swapper/222:47:212
106809975,1cyclictest0-21swapper/221:41:382
106809975,1cyclictest0-21swapper/221:21:382
106809975,1cyclictest0-21swapper/220:29:572
106809975,1cyclictest0-21swapper/220:02:502
106809975,1cyclictest0-21swapper/219:55:432
106809970,6cyclictest0-21swapper/223:51:472
106809970,6cyclictest0-21swapper/223:47:342
106809970,6cyclictest0-21swapper/222:38:202
106809970,6cyclictest0-21swapper/222:01:112
106809970,6cyclictest0-21swapper/221:26:082
106749975,1cyclictest0-21swapper/123:41:261
106749975,1cyclictest0-21swapper/123:37:141
106749975,1cyclictest0-21swapper/123:25:461
106749975,1cyclictest0-21swapper/123:03:491
106749975,1cyclictest0-21swapper/122:49:161
106749975,1cyclictest0-21swapper/122:35:531
106749975,1cyclictest0-21swapper/122:21:021
106749975,1cyclictest0-21swapper/121:07:341
106749975,1cyclictest0-21swapper/120:58:011
106749970,6cyclictest0-21swapper/123:08:301
106749970,1cyclictest0-21swapper/123:24:181
106679976,1cyclictest17929-21snmpd22:58:430
106679976,0cyclictest17929-21snmpd20:56:330
106679975,1cyclictest0-21swapper/022:25:480
106679975,1cyclictest0-21swapper/020:41:400
106679975,1cyclictest0-21swapper/020:39:350
106679970,6cyclictest0-21swapper/023:17:200
106679970,6cyclictest0-21swapper/022:24:220
106679970,6cyclictest0-21swapper/021:56:070
106679970,6cyclictest0-21swapper/020:07:260
106679970,1cyclictest0-21swapper/020:03:250
106679970,1cyclictest0-21swapper/000:11:040
14250260,1sleep20-21swapper/220:06:292
106879965,1cyclictest17929-21snmpd00:06:223
106879965,0cyclictest24144-21awk23:05:153
106879964,1cyclictest752-21dbus-daemon21:40:013
106879964,1cyclictest752-21dbus-daemon19:35:593
106879964,1cyclictest0-21swapper/322:45:203
106879964,1cyclictest0-21swapper/322:42:273
106879964,1cyclictest0-21swapper/320:57:493
106879963,2cyclictest752-21dbus-daemon00:15:363
106879963,2cyclictest17929-21snmpd21:15:283
106809965,1cyclictest17929-21snmpd22:33:452
106809965,0cyclictest17929-21snmpd21:39:382
106809964,1cyclictest752-21dbus-daemon23:41:002
106809964,1cyclictest3738-21isc-worker000219:31:022
106809964,1cyclictest21458-21tr20:20:082
106809964,1cyclictest0-21swapper/223:19:102
106809964,1cyclictest0-21swapper/221:05:102
106809964,1cyclictest0-21swapper/220:35:552
106809963,2cyclictest0-21swapper/223:35:062
106809960,5cyclictest0-21swapper/220:19:382
106749966,0cyclictest17929-21snmpd20:19:031
106749965,1cyclictest17929-21snmpd21:25:491
106749965,1cyclictest17929-21snmpd00:31:261
106749965,0cyclictest17929-21snmpd21:14:041
106749964,1cyclictest24-21ksoftirqd/122:34:591
106749964,1cyclictest24-21ksoftirqd/122:10:001
106749964,1cyclictest24-21ksoftirqd/119:24:591
106749964,1cyclictest24-21ksoftirqd/119:15:011
106749964,1cyclictest0-21swapper/122:42:001
106749964,1cyclictest0-21swapper/100:23:251
106749964,1cyclictest0-21swapper/100:00:191
106749963,2cyclictest17929-21snmpd20:45:361
106749963,2cyclictest17929-21snmpd20:23:471
106749963,2cyclictest16809-21awk22:00:091
106749963,2cyclictest1302-21ping22:29:231
106679965,1cyclictest17929-21snmpd23:13:060
106679964,1cyclictest25251-21ls20:25:140
106679964,1cyclictest17929-21snmpd00:23:240
106679964,1cyclictest14034-21wc19:15:010
106679964,1cyclictest0-21swapper/023:48:280
106679964,1cyclictest0-21swapper/023:41:520
106679964,1cyclictest0-21swapper/022:10:580
106679964,1cyclictest0-21swapper/022:05:380
106679963,2cyclictest27710-21awk00:05:140
106679963,2cyclictest19411-21awk20:15:140
106679963,2cyclictest17929-21snmpd19:37:430
106679963,2cyclictest11659-21awk19:10:110
106679960,5cyclictest0-21swapper/023:05:510
106679960,5cyclictest0-21swapper/021:41:350
106679960,2cyclictest626-21isc-worker000021:34:000
106679960,1cyclictest0-21swapper/021:11:510
106679960,1cyclictest0-21swapper/020:11:490
106679960,1cyclictest0-21swapper/019:42:340
106679960,0cyclictest0-21swapper/019:32:240
106879953,1cyclictest87650irq/28-eno1-rx-22:00:063
106879953,1cyclictest20918-21ping_192.168.1720:20:013
106879953,1cyclictest19577-21ntpq22:05:053
106879953,1cyclictest17929-21snmpd21:40:113
106879953,1cyclictest17929-21snmpd20:10:113
106879953,1cyclictest17929-21snmpd19:45:373
106879953,1cyclictest17929-21snmpd19:40:013
106879953,1cyclictest13947-21cat21:00:143
106879953,1cyclictest1382-21munin-node22:25:013
106879953,1cyclictest11342-21cat19:10:073
106879953,1cyclictest0-21swapper/300:00:113
106879952,2cyclictest29535-21awk22:20:133
106879952,2cyclictest24193-21/usr/share/muni21:20:143
106879950,4cyclictest17843-21perl19:20:063
106809953,1cyclictest9422-21/usr/share/muni20:55:022
106809953,1cyclictest9104-21cat00:30:052
106809953,1cyclictest87950irq/34-eno2-rx-20:50:012
106809953,1cyclictest87950irq/34-eno2-rx-00:20:132
106809953,1cyclictest3753-21cat20:45:052
106809953,1cyclictest31315-21cat22:25:062
106809953,1cyclictest2876-21/usr/share/muni19:50:092
106809953,1cyclictest18817-31munin-graph22:55:412
106809953,1cyclictest1856-31munin-html19:45:332
106809953,1cyclictest17929-21snmpd21:50:292
106809953,1cyclictest17929-21snmpd21:45:092
106809953,1cyclictest13140-21cat21:00:082
106809953,1cyclictest11493-21openvpn23:05:012
106809953,1cyclictest0-21swapper/220:30:332
106749954,0cyclictest17929-21snmpd00:05:431
106749953,1cyclictest7370-21cat19:55:131
106749953,1cyclictest7325-21cat21:45:061
106749953,1cyclictest29449-21ntpq00:10:051
106749953,1cyclictest27314-21awk20:30:061
106749953,1cyclictest26519-21cat22:15:141
106749953,1cyclictest17929-21snmpd20:00:051
106749953,1cyclictest17650-21cat23:50:071
106749953,1cyclictest17067-21cat22:55:061
106749953,1cyclictest15243-31munin-html21:55:331
106749953,1cyclictest0-21swapper/121:20:081
106749952,2cyclictest17929-21snmpd21:50:201
106749952,2cyclictest0-21swapper/121:30:021
106749952,2cyclictest0-21swapper/119:30:041
106679953,1cyclictest81ktimersoftd/000:00:330
106679953,1cyclictest32554-21cat19:45:060
106679953,1cyclictest21789-31munin-html23:00:330
106679953,1cyclictest12098-21cat00:35:040
106679952,2cyclictest87750irq/30-eno1-tx-21:25:000
106679952,2cyclictest3736-21isc-worker000021:00:070
106679952,2cyclictest0-21swapper/021:15:130
106679951,3cyclictest2876-21/usr/share/muni19:50:040
106679950,4cyclictest6716-21/usr/sbin/munin00:25:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional