You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 07:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Sun Jul 13, 2025 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
221022214160,42sleep20-21swapper/219:07:432
220432209154,22sleep10-21swapper/119:06:581
221252206149,45sleep00-21swapper/019:08:010
221082200162,24sleep30-21swapper/319:07:483
205522710,1sleep220533-21perf23:25:002
65382650,0sleep20-21swapper/222:20:432
252082210,0sleep00-21swapper/000:14:160
45842130,1sleep12236799cyclictest23:01:521
22365991310,3cyclictest9-21ksoftirqd/000:25:150
2236799120,12cyclictest0-21swapper/122:05:151
22365991211,1cyclictest9-21ksoftirqd/022:40:000
22365991211,1cyclictest9-21ksoftirqd/021:20:000
22384991110,1cyclictest35-21ksoftirqd/320:55:003
22384991110,1cyclictest35-21ksoftirqd/320:40:013
22384991110,1cyclictest35-21ksoftirqd/300:30:003
2238499110,5cyclictest0-21swapper/323:30:153
22365991111,0cyclictest9-21ksoftirqd/000:30:230
203022110,0sleep315423-21diskmemload00:06:033
96732100,0sleep20-21swapper/223:52:402
93452100,0sleep10-21swapper/121:41:461
2238499109,1cyclictest35-21ksoftirqd/319:20:003
2238499108,1cyclictest35-21ksoftirqd/322:25:003
2238499100,4cyclictest25836-21apt-get22:05:153
2237599100,9cyclictest24721-21munin-node23:08:092
2236799100,9cyclictest0-21swapper/119:10:171
2236599109,1cyclictest9-21ksoftirqd/021:20:160
2236599109,1cyclictest9-21ksoftirqd/020:35:000
2236599101,4cyclictest9717-21cat19:50:000
2236599101,4cyclictest9717-21cat19:50:000
2236599100,6cyclictest24721-21munin-node19:30:490
28661290,0sleep10-21swapper/100:18:271
223849993,3cyclictest25513-21nscd23:49:173
223759997,1cyclictest28-21ksoftirqd/220:15:012
223759997,1cyclictest28-21ksoftirqd/219:20:002
223759992,5cyclictest15423-21diskmemload21:15:162
223679990,7cyclictest31922sleep100:26:441
223659998,1cyclictest9-21ksoftirqd/023:40:000
223659998,1cyclictest9-21ksoftirqd/022:45:010
223659998,1cyclictest9-21ksoftirqd/022:15:000
223659998,1cyclictest9-21ksoftirqd/020:55:020
223659997,1cyclictest9-21ksoftirqd/023:45:000
223659997,1cyclictest9-21ksoftirqd/000:15:160
223659997,1cyclictest5665-21ssh22:20:200
223659992,6cyclictest28134-21ssh21:25:150
223659990,4cyclictest20793-21ssh23:25:090
223659990,4cyclictest16876-21date23:20:000
11082290,1sleep20-21swapper/222:28:012
223849988,0cyclictest35-21ksoftirqd/300:40:003
223849985,1cyclictest0-21swapper/319:35:003
223849982,5cyclictest25927-21kworker/u8:120:25:083
223849980,3cyclictest0-21swapper/321:00:153
223849980,1cyclictest0-21swapper/322:10:123
223759986,1cyclictest60950irq/123-eno123:48:382
223759986,1cyclictest60950irq/123-eno123:38:382
223759986,1cyclictest22809-21cat20:15:182
223759985,1cyclictest28-21ksoftirqd/219:24:552
223759984,2cyclictest254-21systemd-journal20:40:002
223759983,2cyclictest254-21systemd-journal21:38:552
223759981,5cyclictest24721-21munin-node19:41:472
223759980,5cyclictest6415-21unixbench_multi00:30:232
223679982,4cyclictest0-21swapper/123:32:481
223659988,0cyclictest9-21ksoftirqd/021:40:170
223659987,1cyclictest9-21ksoftirqd/023:50:140
223659987,1cyclictest9-21ksoftirqd/022:00:000
223659987,1cyclictest9-21ksoftirqd/021:50:240
223659987,1cyclictest9-21ksoftirqd/000:05:010
223659987,0cyclictest27924-21ps20:25:200
223659986,1cyclictest9-21ksoftirqd/023:15:000
223659986,1cyclictest9-21ksoftirqd/023:05:150
223659986,1cyclictest9-21ksoftirqd/000:00:010
223659986,1cyclictest30763-21apt-get22:55:140
223659986,1cyclictest1957-21perl20:40:160
223659982,5cyclictest24080-21perf20:20:000
223659980,7cyclictest17529-21cstates20:05:150
223659980,4cyclictest28042-21latency_hist23:35:000
223659980,1cyclictest3658-21ssh22:18:110
223849976,1cyclictest35-21ksoftirqd/319:50:013
223849976,1cyclictest35-21ksoftirqd/319:50:003
223849976,1cyclictest25184-21cat20:20:173
223849975,2cyclictest6080-21sendmail_mailqu00:30:223
223849975,1cyclictest912-21ssh22:57:403
223849975,1cyclictest35-21ksoftirqd/323:25:003
223849974,2cyclictest10694-21latency23:10:163
223849972,1cyclictest0-21swapper/322:35:513
223849970,7cyclictest0-21swapper/323:35:523
223849970,5cyclictest0-21swapper/321:51:113
223849970,5cyclictest0-21swapper/321:30:193
223849970,3cyclictest0-21swapper/321:40:563
223849970,3cyclictest0-21swapper/300:19:313
223849970,2cyclictest0-21swapper/320:40:203
223759976,1cyclictest60950irq/123-eno123:10:122
223759976,1cyclictest60950irq/123-eno122:04:062
223759976,1cyclictest60950irq/123-eno121:29:592
223759976,1cyclictest60950irq/123-eno121:20:252
223759976,1cyclictest60950irq/123-eno119:40:002
223759976,1cyclictest60950irq/123-eno100:23:492
223759976,1cyclictest60950irq/123-eno100:07:242
223759975,1cyclictest60950irq/123-eno123:15:162
223759975,1cyclictest60950irq/123-eno122:05:162
223759975,1cyclictest60950irq/123-eno121:55:152
223759975,1cyclictest60950irq/123-eno121:50:262
223759975,1cyclictest60950irq/123-eno121:10:182
223759975,1cyclictest5156-21kworker/2:120:01:212
223759975,1cyclictest4215-21ssh23:00:492
223759973,2cyclictest254-21systemd-journal00:11:282
223759973,1cyclictest254-21systemd-journal22:52:122
223759973,1cyclictest254-21systemd-journal21:40:012
223759973,1cyclictest13753-21ssh23:58:412
223759971,4cyclictest20703-21/usr/sbin/munin22:40:172
223679970,6cyclictest0-21swapper/121:10:551
223679970,4cyclictest0-21swapper/122:30:271
223679970,4cyclictest0-21swapper/100:10:141
223679970,3cyclictest0-21swapper/123:38:461
223679970,3cyclictest0-21swapper/121:36:071
223679970,3cyclictest0-21swapper/119:50:151
223679970,2cyclictest0-21swapper/122:37:031
223679970,2cyclictest0-21swapper/121:16:131
223679970,1cyclictest0-21swapper/123:22:201
223659976,1cyclictest9-21ksoftirqd/020:00:160
223659975,1cyclictest7656-1kworker/u9:000:35:200
223659973,0cyclictest15423-21diskmemload22:42:490
223659972,4cyclictest819-21lldpd00:20:180
223659971,6cyclictest10400-21ssh22:26:030
223659971,5cyclictest2216-21ssh23:00:030
223659970,7cyclictest19837-21ssh23:23:070
223659970,6cyclictest25930-21cstates19:15:160
223659970,6cyclictest19108-21ssh21:14:240
223659970,6cyclictest0-21swapper/021:30:010
223659970,1cyclictest28080-21ssh22:06:400
223659970,1cyclictest254-21systemd-journal21:49:420
223659970,1cyclictest254-21systemd-journal21:37:400
223659970,1cyclictest15423-21diskmemload22:00:100
223659970,1cyclictest15423-21diskmemload00:05:390
223849965,1cyclictest35-21ksoftirqd/323:15:143
223849965,1cyclictest35-21ksoftirqd/321:20:003
223849965,1cyclictest24451-21ssh00:12:093
223849965,1cyclictest22867-21ssh23:26:073
223849965,1cyclictest22647-21ssh21:18:223
223849965,1cyclictest10906-21ssh22:27:303
223849964,1cyclictest1861-21ssh23:41:373
223849964,1cyclictest18319-21ssh21:55:103
223849964,0cyclictest15423-21diskmemload22:46:223
223849963,2cyclictest22573-21apt-get19:10:153
223849963,2cyclictest15426-21basename21:10:003
223849963,1cyclictest254-21systemd-journal23:10:003
223849963,1cyclictest10848-21aten_r1power_en23:55:143
223849962,2cyclictest4160-21df20:45:163
223849962,1cyclictest2503-21/usr/sbin/munin22:15:243
223849961,3cyclictest0-21swapper/322:00:223
223849961,3cyclictest0-21swapper/321:27:113
223849961,2cyclictest0-21swapper/300:20:263
223849960,4cyclictest0-21swapper/321:14:053
223849960,4cyclictest0-21swapper/300:00:093
223849960,3cyclictest0-21swapper/322:30:183
223849960,2cyclictest0-21swapper/323:01:263
223849960,0cyclictest0-21swapper/322:50:503
223759966,0cyclictest60950irq/123-eno123:30:182
223759966,0cyclictest60950irq/123-eno123:25:132
223759966,0cyclictest60950irq/123-eno122:57:082
223759966,0cyclictest60950irq/123-eno122:45:102
223759966,0cyclictest60950irq/123-eno122:15:022
223759966,0cyclictest60950irq/123-eno121:45:292
223759966,0cyclictest60950irq/123-eno121:30:122
223759966,0cyclictest60950irq/123-eno121:07:322
223759966,0cyclictest60950irq/123-eno120:55:192
223759966,0cyclictest60950irq/123-eno120:46:582
223759966,0cyclictest60950irq/123-eno120:30:142
223759966,0cyclictest60950irq/123-eno119:30:142
223759966,0cyclictest60950irq/123-eno119:27:322
223759966,0cyclictest60950irq/123-eno100:15:112
223759966,0cyclictest60950irq/123-eno100:00:162
223759965,1cyclictest60950irq/123-eno100:35:132
223759965,1cyclictest60950irq/123-eno100:25:092
223759964,1cyclictest60950irq/123-eno122:30:162
223759964,1cyclictest60950irq/123-eno120:05:232
223759964,1cyclictest60950irq/123-eno119:45:152
223759964,1cyclictest60950irq/123-eno119:45:152
223759964,1cyclictest498-21cat23:40:172
223759961,2cyclictest15423-21diskmemload22:12:572
223759960,5cyclictest27793-21ntp_kernel_pll_20:25:192
223759960,5cyclictest10757-21fschecks_time19:50:162
223759960,3cyclictest24295-21apt-get20:20:132
223759960,3cyclictest0-21swapper/222:35:082
223759960,3cyclictest0-21swapper/219:55:142
223679966,0cyclictest0-21swapper/122:52:201
223679966,0cyclictest0-21swapper/122:23:051
223679961,2cyclictest0-21swapper/121:33:501
223679961,1cyclictest0-21swapper/123:14:111
223679960,6cyclictest0-21swapper/120:40:151
223679960,6cyclictest0-21swapper/100:38:391
223679960,5cyclictest0-21swapper/119:35:141
223679960,4cyclictest0-21swapper/122:57:291
223679960,4cyclictest0-21swapper/122:11:321
223679960,4cyclictest0-21swapper/121:23:431
223679960,4cyclictest0-21swapper/119:45:161
223679960,4cyclictest0-21swapper/119:45:161
223679960,3cyclictest0-21swapper/123:55:461
223679960,3cyclictest0-21swapper/123:07:141
223679960,3cyclictest0-21swapper/121:52:491
223679960,3cyclictest0-21swapper/121:25:431
223679960,2cyclictest0-21swapper/123:41:411
223679960,2cyclictest0-21swapper/122:40:511
223679960,2cyclictest0-21swapper/122:26:581
223679960,2cyclictest0-21swapper/122:15:211
223679960,2cyclictest0-21swapper/122:03:341
223679960,2cyclictest0-21swapper/100:20:201
223679960,1cyclictest0-21swapper/123:45:201
223679960,1cyclictest0-21swapper/123:15:081
223679960,1cyclictest0-21swapper/121:55:121
223679960,1cyclictest0-21swapper/100:31:391
223659966,0cyclictest9-21ksoftirqd/020:45:000
223659964,1cyclictest9-21ksoftirqd/021:05:000
223659964,1cyclictest9-21ksoftirqd/019:35:000
223659963,1cyclictest9-21ksoftirqd/020:21:390
223659960,6cyclictest29036-21ssh22:51:030
223659960,6cyclictest1700-1kworker/u9:123:45:100
223659960,6cyclictest0-21swapper/020:35:130
223659960,6cyclictest0-21swapper/019:14:320
223659960,5cyclictest6616-21smartctl19:40:230
223659960,5cyclictest0-21swapper/022:30:020
223659960,5cyclictest0-21swapper/019:50:180
223659960,5cyclictest0-21swapper/019:25:150
223849954,1cyclictest13362-21cat19:55:193
223849953,2cyclictest18364-21ntp_states20:05:213
223849953,1cyclictest30412-21aten_r1power_fr19:25:133
223849953,1cyclictest29101-21aten_r1power_po20:30:133
223849953,1cyclictest20052-21df20:10:163
223849952,2cyclictest0-21swapper/319:35:173
223849952,1cyclictest0-21swapper/319:54:553
223849951,2cyclictest0-21swapper/321:35:113
223849950,4cyclictest15191-21expr20:00:163
223849950,4cyclictest0-21swapper/322:40:113
223849950,3cyclictest10818-21apt-get21:45:123
223849950,3cyclictest0-21swapper/323:50:013
223849950,3cyclictest0-21swapper/319:40:183
223849950,1cyclictest254-21systemd-journal19:22:533
223849950,1cyclictest0-21swapper/320:55:173
223759955,0cyclictest60950irq/123-eno121:00:152
223759954,1cyclictest60950irq/123-eno119:10:232
223679951,1cyclictest0-21swapper/120:35:181
223679950,4cyclictest0-21swapper/123:28:021
223679950,3cyclictest0-21swapper/122:46:301
223679950,3cyclictest0-21swapper/120:35:001
223679950,2cyclictest0-21swapper/123:50:211
223679950,1cyclictest0-21swapper/121:46:511
223679950,1cyclictest0-21swapper/120:05:181
223679950,1cyclictest0-21swapper/119:40:181
223679950,1cyclictest0-21swapper/119:19:541
223659950,5cyclictest0-21swapper/019:58:050
223659950,4cyclictest27991-21/usr/sbin/munin19:20:200
223659950,4cyclictest0-21swapper/021:05:150
223659950,4cyclictest0-21swapper/020:10:210
223659950,2cyclictest254-21systemd-journal20:50:100
223849942,1cyclictest22597-21fschecks_time20:15:183
223759940,2cyclictest0-21swapper/220:50:122
223759940,1cyclictest254-21systemd-journal20:40:092
223679944,0cyclictest0-21swapper/120:16:091
223679944,0cyclictest0-21swapper/120:04:021
223679944,0cyclictest0-21swapper/119:59:161
223679943,1cyclictest28716-21chrt20:29:321
223679941,1cyclictest0-21swapper/120:45:231
223679940,3cyclictest0-21swapper/100:05:161
223679940,2cyclictest0-21swapper/121:10:001
223679940,2cyclictest0-21swapper/120:55:141
223679940,2cyclictest0-21swapper/120:50:151
223679940,1cyclictest0-21swapper/121:00:231
223679940,1cyclictest0-21swapper/100:00:121
223679940,0cyclictest0-21swapper/120:20:211
223679932,1cyclictest326852sleep119:30:051
223679931,1cyclictest0-21swapper/120:10:141
223679930,2cyclictest0-21swapper/119:20:141
223679930,1cyclictest0-21swapper/119:25:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional