You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-28 - 01:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Wed Feb 28, 2024 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
68752220157,24sleep10-21swapper/119:06:581
69272209171,25sleep30-21swapper/319:07:403
68642201163,25sleep20-21swapper/219:06:512
50032200160,26sleep00-21swapper/019:05:120
238582650,0sleep30-21swapper/322:25:013
719499130,8cyclictest4909-21ssh23:25:250
720799120,11cyclictest14100-21/usr/sbin/munin00:20:243
719499123,5cyclictest3391-21perf21:15:000
720399119,1cyclictest4189-21/usr/sbin/munin20:10:292
7203991110,1cyclictest28-21ksoftirqd/223:00:002
7203991110,1cyclictest28-21ksoftirqd/200:40:002
720399110,10cyclictest1422-21ntpd21:20:592
719499113,4cyclictest21351-21latency_hist00:30:000
720799102,7cyclictest2702-1kworker/u9:223:24:383
720399109,1cyclictest28-21ksoftirqd/223:30:302
720399109,1cyclictest28-21ksoftirqd/222:20:312
720399109,1cyclictest28-21ksoftirqd/222:00:362
720399108,1cyclictest28-21ksoftirqd/222:35:002
720399108,1cyclictest28-21ksoftirqd/220:30:302
720399108,1cyclictest28-21ksoftirqd/219:20:002
720399108,1cyclictest28-21ksoftirqd/200:30:242
720399100,3cyclictest0-21swapper/200:15:212
720399100,1cyclictest0-21swapper/222:16:342
719499100,5cyclictest29882-21perf21:50:000
72079996,2cyclictest770-21snmpd21:16:503
72079993,3cyclictest35-21ksoftirqd/322:36:143
72079990,1cyclictest0-21swapper/300:00:253
72039999,0cyclictest32266-21diskmemload22:25:012
72039998,1cyclictest28-21ksoftirqd/223:00:012
72039998,1cyclictest28-21ksoftirqd/200:25:002
72039997,1cyclictest9574-21kworker/2:223:35:262
72039997,1cyclictest28-21ksoftirqd/220:45:242
72039997,1cyclictest28-21ksoftirqd/220:40:002
72039990,1cyclictest0-21swapper/219:55:242
71949994,2cyclictest259-21systemd-journal19:15:000
25472290,0sleep332266-21diskmemload23:50:373
72079986,1cyclictest770-21snmpd23:58:513
72079986,1cyclictest770-21snmpd22:04:243
72079986,1cyclictest770-21snmpd21:59:453
72079986,1cyclictest770-21snmpd21:11:443
72079986,1cyclictest770-21snmpd21:01:443
72079986,1cyclictest770-21snmpd20:51:343
72079986,1cyclictest770-21snmpd19:58:143
72079986,1cyclictest770-21snmpd00:06:253
72079986,1cyclictest770-21snmpd00:06:253
72079986,1cyclictest334-21cat20:00:303
72079983,4cyclictest7184-21cyclictest21:51:513
72079982,3cyclictest6957-21diskstats00:10:233
72079980,4cyclictest32266-21diskmemload00:30:063
72079980,2cyclictest0-21swapper/320:20:223
72039987,1cyclictest28-21ksoftirqd/219:50:002
72039986,1cyclictest61250irq/122-eno122:40:302
72039986,1cyclictest61250irq/122-eno121:30:272
72039986,1cyclictest61250irq/122-eno119:40:242
72039986,1cyclictest28-21ksoftirqd/221:10:282
72039986,1cyclictest28-21ksoftirqd/219:25:002
72039983,4cyclictest1422-21ntpd22:07:592
72039980,7cyclictest271rcuc/220:05:202
72039980,6cyclictest30901-21python23:15:332
72039980,4cyclictest32266-21diskmemload00:00:232
72039980,3cyclictest0-21swapper/220:00:242
71989988,0cyclictest0-21swapper/121:55:321
71989981,4cyclictest0-21swapper/119:15:001
71989980,7cyclictest196-21jbd2/sda2-821:35:061
71989980,4cyclictest0-21swapper/100:06:531
71989980,4cyclictest0-21swapper/100:06:521
71949987,1cyclictest9-21ksoftirqd/022:27:150
71949987,1cyclictest9-21ksoftirqd/021:55:340
71949987,1cyclictest7184-21cyclictest23:55:060
71949986,1cyclictest770-21snmpd19:50:050
71949986,1cyclictest770-21snmpd19:47:590
71949986,1cyclictest770-21snmpd19:16:400
71949984,1cyclictest259-21systemd-journal20:44:590
71949983,4cyclictest0-21swapper/020:30:370
71949982,5cyclictest9054-21apt-get22:05:240
71949982,5cyclictest1640-21cut20:05:000
71949982,5cyclictest11555-1kworker/u9:222:22:180
71949981,4cyclictest0-21swapper/022:14:450
71949980,7cyclictest20218-21ssh23:00:590
71949980,5cyclictest12841-21ssh23:35:270
71949980,2cyclictest5576-21ssh00:08:380
71949980,2cyclictest5576-21ssh00:08:380
72079976,0cyclictest32266-21diskmemload22:11:093
72079975,1cyclictest770-21snmpd23:38:093
72079975,1cyclictest770-21snmpd23:25:533
72079975,1cyclictest770-21snmpd22:07:543
72079975,1cyclictest770-21snmpd21:38:303
72079975,1cyclictest770-21snmpd21:31:173
72079975,1cyclictest770-21snmpd21:26:593
72079975,1cyclictest770-21snmpd21:09:233
72079975,1cyclictest770-21snmpd20:48:473
72079975,1cyclictest770-21snmpd19:20:343
72079975,1cyclictest16423-21ssh22:56:253
72079975,1cyclictest11280-21ssh23:33:413
72079974,1cyclictest259-21systemd-journal22:20:003
72079972,3cyclictest32266-21diskmemload22:32:333
72079970,3cyclictest0-21swapper/300:15:483
72079970,2cyclictest0-21swapper/323:03:323
72039977,0cyclictest32266-21diskmemload23:45:252
72039977,0cyclictest0-21swapper/222:35:512
72039976,1cyclictest770-21snmpd19:33:382
72039976,1cyclictest61250irq/122-eno123:44:012
72039976,1cyclictest61250irq/122-eno123:28:262
72039976,1cyclictest61250irq/122-eno123:21:402
72039976,1cyclictest61250irq/122-eno122:50:592
72039976,1cyclictest61250irq/122-eno122:10:002
72039976,1cyclictest61250irq/122-eno121:58:092
72039976,1cyclictest61250irq/122-eno121:26:442
72039976,1cyclictest61250irq/122-eno121:05:222
72039976,1cyclictest61250irq/122-eno100:06:522
72039976,1cyclictest61250irq/122-eno100:06:512
72039976,0cyclictest61250irq/122-eno121:15:332
72039975,1cyclictest770-21snmpd21:45:032
72039975,1cyclictest61250irq/122-eno120:55:222
72039975,1cyclictest28-21ksoftirqd/221:50:312
72039975,1cyclictest27547-21ssh23:10:382
72039974,3cyclictest61250irq/122-eno120:15:232
72039973,2cyclictest259-21systemd-journal21:05:002
72039972,3cyclictest6777-21/usr/sbin/munin22:45:182
72039971,5cyclictest21308-21runrttasks00:29:542
72039970,4cyclictest0-21swapper/223:50:212
72039970,2cyclictest0-21swapper/223:55:212
72039970,1cyclictest0-21swapper/220:51:132
71989970,7cyclictest0-21swapper/100:24:141
71989970,6cyclictest0-21swapper/123:50:341
71989970,5cyclictest0-21swapper/121:40:431
71989970,4cyclictest0-21swapper/122:52:031
71949976,1cyclictest770-21snmpd20:50:190
71949976,1cyclictest30933-1kworker/u9:022:46:580
71949975,1cyclictest770-21snmpd20:16:390
71949975,1cyclictest770-21snmpd19:56:590
71949975,1cyclictest770-21snmpd19:38:230
71949973,1cyclictest32266-21diskmemload00:11:490
71949972,1cyclictest32266-21diskmemload21:25:250
71949971,3cyclictest32266-21diskmemload00:39:090
71949971,3cyclictest3060-21apt-get22:40:230
71949971,2cyclictest259-21systemd-journal20:25:000
71949970,7cyclictest32266-21diskmemload22:01:360
71949970,7cyclictest0-21swapper/023:05:060
71949970,6cyclictest32533-1kworker/u9:221:15:290
71949970,6cyclictest32266-21diskmemload23:13:090
71949970,6cyclictest20906-1kworker/u9:000:30:170
71949970,6cyclictest16032-21ssh00:20:530
71949970,6cyclictest15084-21ssh21:30:060
71949970,6cyclictest1400-21ssh22:36:250
71949970,6cyclictest0-21swapper/023:24:450
71949970,1cyclictest32266-21diskmemload22:51:030
71949970,1cyclictest32266-21diskmemload21:40:310
71949970,1cyclictest32266-21diskmemload00:03:410
71949970,1cyclictest17742-21/usr/sbin/munin19:30:310
71949970,1cyclictest17352-21hddtemp_smartct22:15:280
71949970,1cyclictest16358-21ssh22:56:140
71949970,1cyclictest0-21swapper/023:17:340
71949970,1cyclictest0-21swapper/021:35:190
71949970,1cyclictest0-21swapper/021:20:320
71949970,1cyclictest0-21swapper/020:49:130
71949970,0cyclictest32266-21diskmemload23:34:050
19064270,0sleep10-21swapper/120:40:251
14784270,0sleep10-21swapper/123:37:251
72079965,1cyclictest9694-21ssh22:48:333
72079965,1cyclictest7378-21ssh21:20:093
72079965,1cyclictest32233-21ssh23:18:203
72079965,1cyclictest2968-21ssh22:40:073
72079965,1cyclictest28702-21ssh23:13:553
72079965,1cyclictest27919-21ssh00:37:133
72079965,1cyclictest12684-21ssh22:51:243
72079964,2cyclictest24819-21/usr/sbin/munin19:45:213
72079964,1cyclictest770-21snmpd20:43:013
72079964,1cyclictest770-21snmpd20:17:243
72079964,1cyclictest770-21snmpd19:54:203
72079964,1cyclictest770-21snmpd19:32:093
72079964,1cyclictest2620-21tail20:05:223
72079964,0cyclictest32266-21diskmemload23:43:343
72079962,1cyclictest19892-21apt-get23:45:223
72079961,4cyclictest23526-21sed21:40:273
72079961,2cyclictest27623-21/usr/sbin/munin21:45:253
72079960,3cyclictest0-21swapper/300:25:303
72079960,2cyclictest0-21swapper/322:25:223
72039966,0cyclictest61250irq/122-eno123:05:142
72039966,0cyclictest61250irq/122-eno121:41:562
72039966,0cyclictest61250irq/122-eno119:35:472
72039966,0cyclictest61250irq/122-eno100:11:112
72039965,1cyclictest28-21ksoftirqd/220:25:232
72039965,1cyclictest0-21swapper/219:24:432
72039964,1cyclictest61250irq/122-eno119:10:242
72039964,1cyclictest28-21ksoftirqd/220:20:242
72039960,3cyclictest0-21swapper/221:35:162
71989961,3cyclictest0-21swapper/122:02:131
71989961,3cyclictest0-21swapper/100:26:081
71989960,6cyclictest0-21swapper/123:55:391
71989960,6cyclictest0-21swapper/123:44:421
71989960,6cyclictest0-21swapper/123:26:351
71989960,6cyclictest0-21swapper/123:19:351
71989960,6cyclictest0-21swapper/123:10:261
71989960,6cyclictest0-21swapper/122:45:011
71989960,6cyclictest0-21swapper/122:42:251
71989960,6cyclictest0-21swapper/122:08:101
71989960,6cyclictest0-21swapper/121:34:061
71989960,6cyclictest0-21swapper/121:20:231
71989960,6cyclictest0-21swapper/120:27:031
71989960,6cyclictest0-21swapper/119:52:261
71989960,6cyclictest0-21swapper/119:15:281
71989960,6cyclictest0-21swapper/100:32:101
71989960,5cyclictest0-21swapper/121:13:411
71989960,5cyclictest0-21swapper/120:37:401
71989960,5cyclictest0-21swapper/100:35:071
71989960,5cyclictest0-21swapper/100:17:561
71989960,3cyclictest0-21swapper/119:45:221
71989960,3cyclictest0-21swapper/100:00:021
71989960,2cyclictest0-21swapper/121:25:281
71989960,1cyclictest0-21swapper/122:35:051
71949965,1cyclictest9-21ksoftirqd/019:24:270
71949965,1cyclictest770-21snmpd19:41:440
71949965,1cyclictest31441-21ssh21:50:350
71949965,1cyclictest2966-21ntpq20:05:260
71949965,1cyclictest28881-21ssh22:30:280
71949961,5cyclictest0-21swapper/020:26:030
71949960,6cyclictest0-21swapper/023:50:180
71949960,6cyclictest0-21swapper/023:45:440
71949960,4cyclictest0-21swapper/023:40:090
71949960,2cyclictest10176-21ssh00:15:090
71949960,1cyclictest0-21swapper/020:35:070
71949960,0cyclictest0-21swapper/021:00:200
71949960,0cyclictest0-21swapper/020:11:560
72079954,1cyclictest770-21snmpd20:56:143
72079954,1cyclictest0-21swapper/319:18:263
72079953,1cyclictest770-21snmpd19:10:163
72079953,1cyclictest23461-21python23:05:353
72079952,1cyclictest5132-21latency20:10:283
72079951,1cyclictest22470-21apt-get19:40:223
72079951,1cyclictest14150-21df_inode20:30:303
72079950,5cyclictest0-21swapper/320:28:323
72079950,5cyclictest0-21swapper/319:36:363
72079950,2cyclictest259-21systemd-journal20:40:003
72039954,1cyclictest770-21snmpd20:44:462
72039954,1cyclictest61250irq/122-eno119:50:232
71989955,0cyclictest0-21swapper/123:31:171
71989951,3cyclictest0-21swapper/119:20:211
71989951,0cyclictest0-21swapper/119:25:251
71989950,5cyclictest0-21swapper/123:45:401
71989950,5cyclictest0-21swapper/123:20:141
71989950,5cyclictest0-21swapper/123:07:381
71989950,5cyclictest0-21swapper/123:00:081
71989950,5cyclictest0-21swapper/122:55:521
71989950,5cyclictest0-21swapper/122:30:041
71989950,5cyclictest0-21swapper/122:25:021
71989950,5cyclictest0-21swapper/122:20:421
71989950,5cyclictest0-21swapper/122:15:101
71989950,5cyclictest0-21swapper/122:10:541
71989950,5cyclictest0-21swapper/121:50:201
71989950,5cyclictest0-21swapper/121:45:021
71989950,5cyclictest0-21swapper/121:15:321
71989950,5cyclictest0-21swapper/120:48:341
71989950,5cyclictest0-21swapper/120:20:421
71989950,5cyclictest0-21swapper/120:10:311
71989950,5cyclictest0-21swapper/100:10:321
71989950,3cyclictest0-21swapper/119:30:251
71989950,2cyclictest0-21swapper/119:40:011
71989950,1cyclictest0-21swapper/121:05:231
71989950,1cyclictest0-21swapper/119:40:371
71949952,2cyclictest15900-21diskstats19:25:250
71949950,1cyclictest30156-21/usr/sbin/munin21:05:380
71949950,0cyclictest0-21swapper/020:57:040
72079942,1cyclictest15631-21grep19:25:243
71989943,1cyclictest0-21swapper/120:09:151
71989943,1cyclictest0-21swapper/120:01:151
71989942,1cyclictest31505-21chrt19:58:281
71989940,4cyclictest0-21swapper/120:55:341
71989940,4cyclictest0-21swapper/120:50:311
71989940,4cyclictest0-21swapper/120:32:461
71989932,1cyclictest0-21swapper/121:00:001
71989930,2cyclictest0-21swapper/120:15:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional