You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 12:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rack1slot1.osadl.org (updated Fri May 02, 2025 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
261972206162,23sleep30-21swapper/319:05:513
263452202161,27sleep20-21swapper/219:07:472
264162200163,24sleep10-21swapper/119:08:421
263212199160,26sleep00-21swapper/019:07:280
1260421230,1sleep121-21ksoftirqd/123:45:171
253882630,1sleep319615-21diskmemload21:55:203
26606991513,1cyclictest9-21ksoftirqd/022:10:010
2660699143,5cyclictest9024-21grep19:40:000
26606991311,1cyclictest9-21ksoftirqd/000:40:000
2662399120,6cyclictest254-21systemd-journal00:08:252
26606991211,1cyclictest9-21ksoftirqd/021:40:160
2660699121,5cyclictest370-21perf20:30:010
26623991110,1cyclictest28-21ksoftirqd/221:12:182
2660699119,1cyclictest9-21ksoftirqd/023:20:000
2660699119,1cyclictest9-21ksoftirqd/020:10:000
26606991110,1cyclictest9-21ksoftirqd/023:35:150
26606991110,1cyclictest9-21ksoftirqd/023:25:000
26606991110,1cyclictest9-21ksoftirqd/022:10:260
26606991110,1cyclictest9-21ksoftirqd/021:20:170
2660699110,10cyclictest0-21swapper/020:10:150
2660699109,1cyclictest9-21ksoftirqd/023:25:150
2660699109,1cyclictest9-21ksoftirqd/022:53:330
2660699109,1cyclictest9-21ksoftirqd/021:50:150
2660699109,1cyclictest9-21ksoftirqd/021:15:150
2660699109,1cyclictest9-21ksoftirqd/000:20:140
2660699109,1cyclictest14812-21ssh23:47:380
2660699109,0cyclictest9-21ksoftirqd/000:10:010
2660699108,1cyclictest9-21ksoftirqd/022:44:590
2660699107,2cyclictest9-21ksoftirqd/020:35:120
26606991010,0cyclictest9-21ksoftirqd/023:30:150
26606991010,0cyclictest9-21ksoftirqd/022:45:200
13842100,0sleep10-21swapper/122:05:571
266309991,6cyclictest557-21nscd00:30:073
266309991,6cyclictest0-21swapper/300:19:073
266239999,0cyclictest0-21swapper/221:57:422
266239999,0cyclictest0-21swapper/221:49:112
266239993,4cyclictest60950irq/123-eno121:39:022
266069999,0cyclictest9-21ksoftirqd/023:50:170
266069999,0cyclictest9-21ksoftirqd/023:40:350
266069998,1cyclictest9-21ksoftirqd/021:45:210
266069998,1cyclictest9-21ksoftirqd/021:35:170
266069998,1cyclictest9-21ksoftirqd/000:05:160
266069997,1cyclictest9-21ksoftirqd/022:00:190
266069992,6cyclictest9-21ksoftirqd/019:10:170
266069990,8cyclictest19615-21diskmemload00:00:220
266309987,1cyclictest35-21ksoftirqd/320:54:593
266309986,1cyclictest19615-21diskmemload23:45:493
266309986,1cyclictest14827-21cat19:50:183
266309981,4cyclictest0-21swapper/322:30:203
266309980,7cyclictest15152-21apt-get21:00:163
266239987,1cyclictest28-21ksoftirqd/223:00:012
266239986,1cyclictest60950irq/123-eno122:34:192
266239986,1cyclictest60950irq/123-eno121:31:062
266239986,1cyclictest28330-21ssh21:20:012
266239986,1cyclictest1457-21cat23:30:192
266239985,2cyclictest28-21ksoftirqd/220:20:002
266239985,1cyclictest60950irq/123-eno123:49:552
266239982,5cyclictest1-21systemd21:40:152
266239981,4cyclictest12542-21perf20:55:002
266239981,3cyclictest3315-21sed22:50:212
266169981,2cyclictest0-21swapper/123:30:171
266169980,3cyclictest0-21swapper/100:20:141
266069988,0cyclictest9-21ksoftirqd/022:25:170
266069988,0cyclictest9-21ksoftirqd/021:25:500
266069987,1cyclictest9-21ksoftirqd/023:55:180
266069987,1cyclictest9-21ksoftirqd/023:05:210
266069987,1cyclictest9-21ksoftirqd/020:15:140
266069987,1cyclictest9-21ksoftirqd/019:45:180
266069987,1cyclictest9-21ksoftirqd/019:15:150
266069986,1cyclictest9-21ksoftirqd/021:30:210
266069983,0cyclictest23790-21ssh22:35:180
266069981,3cyclictest6656-21sort19:35:010
266069980,3cyclictest16657-21/usr/sbin/munin00:30:220
266309976,1cyclictest35-21ksoftirqd/321:50:123
266309976,1cyclictest35-21ksoftirqd/319:20:003
266309976,1cyclictest15791-21sh21:44:283
266309975,2cyclictest196162sleep321:49:213
266309975,1cyclictest3749-21ssh21:29:403
266309975,1cyclictest35-21ksoftirqd/319:45:003
266309974,1cyclictest254-21systemd-journal21:16:353
266309973,2cyclictest19615-21diskmemload22:05:183
266309972,2cyclictest19615-21diskmemload23:35:103
266309972,2cyclictest19615-21diskmemload23:26:213
266309971,3cyclictest31610-21gpgconf00:10:013
266309971,3cyclictest27095-21ntpq20:15:213
266309971,2cyclictest19615-21diskmemload23:14:203
266309970,3cyclictest29646-21sh00:05:493
266309970,3cyclictest0-21swapper/322:12:233
266309970,2cyclictest0-21swapper/323:00:203
266309970,1cyclictest0-21swapper/322:28:443
266239976,1cyclictest60950irq/123-eno123:55:172
266239976,1cyclictest60950irq/123-eno123:20:182
266239976,1cyclictest60950irq/123-eno122:37:082
266239976,1cyclictest60950irq/123-eno122:00:122
266239976,1cyclictest60950irq/123-eno120:25:182
266239976,1cyclictest60950irq/123-eno100:39:392
266239976,1cyclictest60950irq/123-eno100:20:312
266239976,1cyclictest60950irq/123-eno100:17:452
266239976,1cyclictest517-21ntpq22:05:212
266239976,1cyclictest0-21swapper/219:45:232
266239976,0cyclictest60950irq/123-eno122:17:322
266239975,1cyclictest60950irq/123-eno122:21:052
266239975,1cyclictest60950irq/123-eno100:10:122
266239972,2cyclictest254-21systemd-journal21:29:552
266239971,3cyclictest28451-21sh22:40:502
266239970,6cyclictest0-21swapper/220:00:182
266239970,1cyclictest0-21swapper/220:45:232
266169970,6cyclictest0-21swapper/122:40:161
266169970,1cyclictest0-21swapper/122:35:151
266069977,0cyclictest9-21ksoftirqd/022:55:130
266069977,0cyclictest9-21ksoftirqd/022:15:170
266069976,1cyclictest9-21ksoftirqd/021:10:190
266069976,1cyclictest9-21ksoftirqd/019:40:200
266069976,1cyclictest9-21ksoftirqd/019:25:000
266069976,1cyclictest9-21ksoftirqd/000:15:220
266069973,2cyclictest254-21systemd-journal20:24:550
266069972,2cyclictest19615-21diskmemload00:26:220
266069971,6cyclictest13011-21ssh22:20:550
266069971,5cyclictest13650irq/124-ahci[0021:10:000
266069970,6cyclictest18421-21apt-get20:00:000
266069970,6cyclictest0-21swapper/023:00:050
266069970,6cyclictest0-21swapper/020:00:220
266069970,1cyclictest21702-21ssh22:33:270
266069970,1cyclictest0-21swapper/021:57:440
266069970,1cyclictest0-21swapper/020:45:160
266309965,1cyclictest434-21ssh23:30:033
266309965,1cyclictest29763-21ssh22:00:243
266309965,1cyclictest2976-21ntpq22:50:203
266309965,1cyclictest26453-21ssh22:40:093
266309965,1cyclictest25785-21ssh00:00:543
266309965,1cyclictest24515-21ssh22:35:403
266309965,1cyclictest22100-21ssh23:56:273
266309965,1cyclictest20854-21sshd20:03:343
266309965,1cyclictest13353-21ssh23:05:013
266309965,1cyclictest10033-21ssh22:18:193
266309964,1cyclictest22356-21ntp_states23:15:203
266309964,1cyclictest19615-21diskmemload00:20:283
266309964,1cyclictest11591-21apt-get19:45:173
266309964,0cyclictest19615-21diskmemload23:44:473
266309963,1cyclictest254-21systemd-journal19:31:023
266309962,2cyclictest19615-21diskmemload21:10:453
266309962,1cyclictest19588-21latency_hist23:55:003
266309962,0cyclictest19615-21diskmemload21:20:313
266309961,4cyclictest0-21swapper/322:47:013
266309961,3cyclictest19929-21cut21:10:003
266309961,3cyclictest12053-21grep22:20:183
266309960,6cyclictest9148-21ssh22:59:503
266309960,4cyclictest546-21irqbalance20:21:263
266309960,4cyclictest0-21swapper/321:35:233
266309960,4cyclictest0-21swapper/300:27:453
266309960,3cyclictest0-21swapper/321:30:173
266309960,2cyclictest1093-21diskstats20:30:153
266309960,1cyclictest30862-21/usr/sbin/munin20:25:153
266309960,1cyclictest0-21swapper/323:21:103
266239966,0cyclictest60950irq/123-eno123:15:032
266239966,0cyclictest60950irq/123-eno123:10:202
266239966,0cyclictest60950irq/123-eno123:05:162
266239966,0cyclictest60950irq/123-eno121:50:232
266239966,0cyclictest60950irq/123-eno121:03:462
266239966,0cyclictest60950irq/123-eno120:30:402
266239966,0cyclictest60950irq/123-eno120:20:162
266239966,0cyclictest60950irq/123-eno119:40:102
266239966,0cyclictest60950irq/123-eno119:13:092
266239965,1cyclictest60950irq/123-eno123:50:172
266239965,1cyclictest60950irq/123-eno123:40:142
266239965,1cyclictest60950irq/123-eno123:25:022
266239965,1cyclictest60950irq/123-eno100:30:132
266239965,1cyclictest60950irq/123-eno100:25:122
266239965,1cyclictest60950irq/123-eno100:00:062
266239965,1cyclictest5099-21ssh22:10:282
266239964,1cyclictest7988-21ntpq19:35:172
266239964,1cyclictest60950irq/123-eno122:45:142
266239964,1cyclictest60950irq/123-eno121:10:002
266239964,1cyclictest60950irq/123-eno120:35:162
266239964,1cyclictest60950irq/123-eno120:05:232
266239964,1cyclictest60950irq/123-eno119:50:172
266239964,1cyclictest60950irq/123-eno119:30:152
266239964,1cyclictest60950irq/123-eno119:20:142
266239964,1cyclictest60950irq/123-eno119:15:132
266239964,1cyclictest5396-21ssh23:35:172
266239964,1cyclictest3044-21cat19:25:172
266239964,1cyclictest19615-21diskmemload22:25:142
266239960,3cyclictest0-21swapper/221:16:002
266239960,2cyclictest9670-21basename23:00:152
266239960,2cyclictest0-21swapper/220:55:162
266169961,4cyclictest0-21swapper/100:12:241
266169960,6cyclictest0-21swapper/122:55:151
266169960,6cyclictest0-21swapper/100:25:271
266169960,5cyclictest0-21swapper/120:10:191
266169960,5cyclictest0-21swapper/119:30:171
266169960,4cyclictest16371-21chrt22:25:221
266169960,4cyclictest0-21swapper/123:15:131
266169960,4cyclictest0-21swapper/100:18:041
266169960,3cyclictest0-21swapper/122:45:421
266169960,3cyclictest0-21swapper/121:30:001
266169960,3cyclictest0-21swapper/120:25:141
266169960,1cyclictest0-21swapper/120:15:151
266169960,0cyclictest0-21swapper/123:40:201
266169960,0cyclictest0-21swapper/123:05:261
266069965,1cyclictest9-21ksoftirqd/020:40:180
266069962,1cyclictest2703-21idleruntime-cro20:34:590
266069960,6cyclictest21549-21kworker/0:223:10:180
266069960,6cyclictest0-21swapper/021:00:520
266069960,5cyclictest2155-21/usr/sbin/munin19:25:140
266069960,2cyclictest0-21swapper/019:50:140
266069960,0cyclictest0-21swapper/020:59:500
266309954,1cyclictest29225-21tune2fs19:15:003
266309954,1cyclictest10190-21ls20:50:003
266309954,0cyclictest19615-21diskmemload00:38:053
266309953,1cyclictest3650-21df_inode20:35:133
266239955,0cyclictest60950irq/123-eno120:10:152
266239955,0cyclictest60950irq/123-eno119:55:152
266239954,1cyclictest60950irq/123-eno120:40:282
266169955,0cyclictest0-21swapper/121:35:041
266169955,0cyclictest0-21swapper/120:50:301
266169955,0cyclictest0-21swapper/100:05:591
266169952,0cyclictest0-21swapper/100:30:181
266169951,2cyclictest0-21swapper/123:20:021
266169951,2cyclictest0-21swapper/122:00:131
266169951,2cyclictest0-21swapper/121:41:441
266169951,1cyclictest0-21swapper/122:50:171
266169950,5cyclictest0-21swapper/121:45:011
266169950,5cyclictest0-21swapper/121:10:171
266169950,5cyclictest0-21swapper/100:01:451
266169950,4cyclictest0-21swapper/122:33:021
266169950,3cyclictest0-21swapper/123:50:191
266169950,3cyclictest0-21swapper/121:52:271
266169950,2cyclictest0-21swapper/123:11:001
266169950,1cyclictest0-21swapper/123:59:541
266169950,1cyclictest0-21swapper/123:38:131
266169950,1cyclictest0-21swapper/122:10:161
266169950,1cyclictest0-21swapper/121:21:011
266169950,1cyclictest0-21swapper/119:15:131
266169950,0cyclictest0-21swapper/122:16:031
266169950,0cyclictest0-21swapper/120:37:141
266169950,0cyclictest0-21swapper/100:35:211
266069950,1cyclictest0-21swapper/020:50:150
26296250,0sleep10-21swapper/121:16:211
266309943,1cyclictest5976-21grep20:40:193
266309942,1cyclictest6952-21tail19:35:123
266309942,1cyclictest16881-21tune2fs19:55:183
266309941,2cyclictest32223-21apt-get19:20:133
266309941,2cyclictest21219-21apt-get20:05:143
266309941,1cyclictest2769-21grep19:25:153
266309941,1cyclictest12843-21aten_r1power_fr20:55:173
266309940,1cyclictest254-21systemd-journal20:10:003
266169944,0cyclictest0-21swapper/123:26:171
266169944,0cyclictest0-21swapper/119:47:291
266169944,0cyclictest0-21swapper/119:13:501
266169943,1cyclictest16960-21chrt21:01:151
266169942,1cyclictest0-21swapper/119:50:141
266169941,1cyclictest0-21swapper/120:45:171
266169941,1cyclictest0-21swapper/120:40:241
266169940,2cyclictest0-21swapper/123:00:241
266169940,2cyclictest0-21swapper/121:31:001
266169940,2cyclictest0-21swapper/120:05:151
266169940,2cyclictest0-21swapper/120:00:221
266169940,2cyclictest0-21swapper/119:55:151
266169940,2cyclictest0-21swapper/119:35:181
266169940,1cyclictest0-21swapper/122:20:061
266169940,1cyclictest0-21swapper/121:55:161
266169940,1cyclictest0-21swapper/121:05:121
266169940,1cyclictest0-21swapper/119:40:231
266169940,0cyclictest0-21swapper/120:30:121
266169931,1cyclictest0-21swapper/120:55:161
266169931,1cyclictest0-21swapper/119:25:141
266169930,3cyclictest0-21swapper/119:20:131
266169920,1cyclictest0-21swapper/120:20:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional