You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-19 - 06:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sat Jul 19, 2025 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
130692201186,10sleep10-21swapper/119:10:001
108432185169,12sleep30-21swapper/319:05:083
128522184168,11sleep00-21swapper/019:07:100
108372175157,13sleep20-21swapper/219:05:032
121202490,1sleep012125-21latency_hist22:50:010
245842450,0sleep20-21swapper/222:06:152
86552430,0sleep30-21swapper/323:36:443
327192430,0sleep032718-21ntp_kernel_pll_19:45:170
267372420,0sleep10-21swapper/121:18:501
3432100,0sleep10-21swapper/100:21:411
132429993,3cyclictest15031-21munin-run22:35:010
132429992,3cyclictest1453-21awk20:55:000
132429981,6cyclictest0-21swapper/020:05:140
20915260,0sleep10-21swapper/100:15:251
132519960,5cyclictest0-21swapper/322:45:133
4203250,0sleep10-21swapper/122:12:391
31800250,0sleep331802-21unixbench_multi23:15:223
31800250,0sleep331802-21unixbench_multi23:15:223
132519950,5cyclictest0-21swapper/323:14:173
132519950,5cyclictest0-21swapper/322:42:423
132519950,5cyclictest0-21swapper/322:18:163
132519950,5cyclictest0-21swapper/300:33:003
132519950,0cyclictest0-21swapper/321:27:053
132469955,0cyclictest0-21swapper/222:20:032
132469950,5cyclictest0-21swapper/223:41:582
132469950,5cyclictest0-21swapper/222:53:482
132469950,5cyclictest0-21swapper/222:17:002
132469950,5cyclictest0-21swapper/222:11:372
132469950,5cyclictest0-21swapper/221:52:242
132469950,5cyclictest0-21swapper/200:07:142
132469950,4cyclictest12579-21hddtemp_smartct23:55:162
132469950,1cyclictest0-21swapper/223:20:232
132469950,0cyclictest0-21swapper/223:53:192
132469950,0cyclictest0-21swapper/200:17:112
132439955,0cyclictest0-21swapper/123:21:211
132439950,5cyclictest0-21swapper/122:22:081
132439950,5cyclictest0-21swapper/120:30:491
132429953,1cyclictest15540-21ntpq20:15:160
132429953,0cyclictest3-21ksoftirqd/022:10:010
132429952,1cyclictest5127-21sh22:30:010
132429950,5cyclictest0-21swapper/023:51:070
132429950,5cyclictest0-21swapper/023:45:110
132429950,5cyclictest0-21swapper/022:53:480
132429950,5cyclictest0-21swapper/022:20:460
132429950,5cyclictest0-21swapper/022:12:190
132429950,5cyclictest0-21swapper/021:43:130
132429950,5cyclictest0-21swapper/021:23:170
132429950,5cyclictest0-21swapper/000:19:290
132429950,0cyclictest0-21swapper/023:04:340
30289240,1sleep10-21swapper/121:20:191
20306240,1sleep00-21swapper/023:26:040
19172240,0sleep10-21swapper/123:25:211
132519944,0cyclictest9606-21kworker/3:223:56:333
132519944,0cyclictest0-21swapper/323:02:473
132519944,0cyclictest0-21swapper/322:21:193
132519944,0cyclictest0-21swapper/321:48:173
132519944,0cyclictest0-21swapper/320:00:453
132519944,0cyclictest0-21swapper/320:00:453
132519944,0cyclictest0-21swapper/300:13:123
132519943,1cyclictest57250irq/125-eth022:05:023
132519943,1cyclictest27697-21cron22:29:563
132519943,1cyclictest26505-21/usr/sbin/munin19:35:193
132519943,1cyclictest21732-21/usr/sbin/munin20:30:223
132519943,1cyclictest17340-21grep19:15:153
132519943,1cyclictest0-21swapper/319:56:323
132519941,2cyclictest26577-21ls23:30:013
132519941,2cyclictest0-21swapper/321:05:133
132519941,1cyclictest0-21swapper/321:15:123
132519940,4cyclictest31469-21apt-get00:05:163
132519940,4cyclictest29185-21/usr/sbin/munin21:20:223
132519940,4cyclictest1063-21apt21:55:123
132519940,4cyclictest0-21swapper/323:33:203
132519940,4cyclictest0-21swapper/321:30:323
132519940,4cyclictest0-21swapper/320:45:123
132519940,4cyclictest0-21swapper/300:01:163
132519940,3cyclictest27361-21perl20:40:153
132519940,3cyclictest19594-21sed23:10:013
132519940,3cyclictest1468-21cut19:50:013
132519940,3cyclictest12609-21/usr/sbin/munin22:50:133
132519940,3cyclictest0-21swapper/322:00:133
132519940,1cyclictest4164-21/usr/sbin/munin21:00:203
132519940,1cyclictest28649-21/usr/sbin/munin00:20:203
132519940,1cyclictest15781-21ntp_states23:40:193
132519940,0cyclictest0-21swapper/323:20:233
132519940,0cyclictest0-21swapper/321:42:193
132469944,0cyclictest0-21swapper/200:26:032
132469943,1cyclictest14161-21grep19:10:162
132469941,1cyclictest0-21swapper/221:55:122
132469941,1cyclictest0-21swapper/220:10:112
132469940,4cyclictest31637-21apt-get20:50:152
132469940,4cyclictest24647-21sh23:12:032
132469940,4cyclictest2207-21diskstats20:55:152
132469940,4cyclictest2039-21diskstats19:50:142
132469940,4cyclictest20356-21ssh22:05:002
132469940,4cyclictest1780-21tr23:00:192
132469940,4cyclictest16566-21diskstats00:30:152
132469940,4cyclictest13853-21ssh21:45:152
132469940,4cyclictest1278-21munin-run19:50:012
132469940,4cyclictest0-21swapper/223:36:242
132469940,4cyclictest0-21swapper/223:30:122
132469940,4cyclictest0-21swapper/223:25:062
132469940,4cyclictest0-21swapper/223:15:132
132469940,4cyclictest0-21swapper/223:15:132
132469940,4cyclictest0-21swapper/223:06:122
132469940,4cyclictest0-21swapper/222:45:132
132469940,4cyclictest0-21swapper/222:26:102
132469940,4cyclictest0-21swapper/221:40:172
132469940,4cyclictest0-21swapper/221:30:102
132469940,4cyclictest0-21swapper/221:16:212
132469940,4cyclictest0-21swapper/220:45:222
132469940,4cyclictest0-21swapper/200:21:212
132469940,4cyclictest0-21swapper/200:14:372
132469940,4cyclictest0-21swapper/200:01:272
132469940,3cyclictest7602-21/usr/sbin/munin21:25:152
132469940,3cyclictest7133-21/usr/sbin/munin22:30:212
132469940,3cyclictest30997-21snmpd23:45:242
132469940,3cyclictest30997-21snmpd00:38:462
132469940,3cyclictest29203-21/usr/sbin/munin21:20:142
132469940,3cyclictest28644-21ssh21:35:272
132469940,3cyclictest25294-21memory20:35:202
132469940,3cyclictest24560-21sh19:30:142
132469940,3cyclictest23973-21cat19:30:012
132469940,3cyclictest22224-21gpgv22:55:012
132469940,3cyclictest18854-21rm22:36:102
132469940,3cyclictest17118-21df19:15:142
132469940,3cyclictest16720-21cat20:20:012
132469940,0cyclictest0-21swapper/220:25:232
132439944,0cyclictest0-21swapper/123:52:381
132439944,0cyclictest0-21swapper/123:18:181
132439944,0cyclictest0-21swapper/123:18:181
132439944,0cyclictest0-21swapper/122:40:281
132439944,0cyclictest0-21swapper/121:14:021
132439941,2cyclictest0-21swapper/121:50:131
132439941,2cyclictest0-21swapper/100:30:131
132439941,1cyclictest0-21swapper/121:00:121
132439940,4cyclictest0-21swapper/123:45:041
132439940,4cyclictest0-21swapper/122:34:211
132439940,4cyclictest0-21swapper/121:44:271
132439940,0cyclictest0-21swapper/121:25:581
132429944,0cyclictest0-21swapper/023:32:040
132429944,0cyclictest0-21swapper/000:31:520
132429943,1cyclictest19029-21latency_hist20:25:010
132429941,2cyclictest0-21swapper/000:20:130
132429941,1cyclictest0-21swapper/020:40:130
132429940,4cyclictest7502-21hddtemp_smartct21:25:140
132429940,4cyclictest31013-21ssh00:37:210
132429940,4cyclictest21302-21ssh23:43:290
132429940,4cyclictest1063-21apt21:55:110
132429940,4cyclictest0-21swapper/023:56:180
132429940,4cyclictest0-21swapper/023:38:340
132429940,4cyclictest0-21swapper/023:12:040
132429940,4cyclictest0-21swapper/023:05:130
132429940,4cyclictest0-21swapper/022:59:490
132429940,4cyclictest0-21swapper/022:35:140
132429940,4cyclictest0-21swapper/022:00:130
132429940,4cyclictest0-21swapper/021:45:150
132429940,4cyclictest0-21swapper/021:32:050
132429940,4cyclictest0-21swapper/019:55:110
132429940,4cyclictest0-21swapper/019:16:060
132429940,4cyclictest0-21swapper/000:05:160
132429940,3cyclictest6009-21users21:00:210
132429940,3cyclictest29695-21fschecks_time19:40:160
132429940,3cyclictest29547-21chrt20:45:130
132429940,3cyclictest24659-21if_eth021:50:160
132429940,3cyclictest23082-21python19:25:200
132429940,3cyclictest21901-21ssh21:15:460
132429940,3cyclictest10151-21tune2fs21:10:160
132429940,3cyclictest0-21swapper/022:40:140
132429940,3cyclictest0-21swapper/020:25:140
132429940,3cyclictest0-21swapper/000:00:120
132429940,1cyclictest23976-21munin-run20:35:000
30406230,0sleep10-21swapper/122:58:591
132519933,0cyclictest1727-21cpuspeed19:50:133
132519932,1cyclictest25711-21users21:50:213
132519932,1cyclictest2082-21expr23:50:163
132519932,1cyclictest19328-21cut20:25:143
132519931,1cyclictest31740-21hostname20:50:163
132519931,1cyclictest11818-21apt-get20:10:113
132519931,1cyclictest0-21swapper/320:55:133
132519931,1cyclictest0-21swapper/320:15:113
132519930,3cyclictest6495-21tr00:25:143
132519930,3cyclictest26004-21cpu00:35:143
132519930,3cyclictest17241-21cstates20:20:123
132519930,3cyclictest10068-21perl20:05:163
132519930,3cyclictest0-21swapper/323:45:113
132519930,3cyclictest0-21swapper/322:55:083
132519930,3cyclictest0-21swapper/322:35:143
132519930,3cyclictest0-21swapper/320:35:153
132519930,3cyclictest0-21swapper/319:30:123
132519930,3cyclictest0-21swapper/319:25:053
132519930,3cyclictest0-21swapper/319:20:223
132519930,3cyclictest0-21swapper/319:10:213
132519930,3cyclictest0-21swapper/300:15:003
132519930,2cyclictest0-21swapper/322:30:153
132519930,1cyclictest9420-21ssh21:10:133
132519930,1cyclictest30824-21ssh22:10:063
132519930,1cyclictest28995-21/usr/sbin/munin19:40:213
132519930,1cyclictest26213-21ssh21:35:043
132469932,1cyclictest30997-21snmpd20:05:142
132469931,1cyclictest0-21swapper/221:05:132
132469930,3cyclictest6873-21apt20:00:132
132469930,3cyclictest6873-21apt20:00:132
132469930,3cyclictest25132-21ssh22:40:022
132469930,3cyclictest0-21swapper/221:10:022
132469930,3cyclictest0-21swapper/221:00:072
132469930,3cyclictest0-21swapper/220:40:092
132469930,3cyclictest0-21swapper/220:30:122
132469930,3cyclictest0-21swapper/220:20:112
132469930,3cyclictest0-21swapper/219:55:072
132469930,3cyclictest0-21swapper/219:40:132
132469930,3cyclictest0-21swapper/219:35:142
132469930,3cyclictest0-21swapper/219:20:162
132439933,0cyclictest0-21swapper/122:16:451
132439933,0cyclictest0-21swapper/120:42:181
132439933,0cyclictest0-21swapper/100:09:041
132439931,1cyclictest0-21swapper/123:30:121
132439931,1cyclictest0-21swapper/123:05:131
132439931,1cyclictest0-21swapper/122:45:131
132439931,1cyclictest0-21swapper/122:35:141
132439931,1cyclictest0-21swapper/122:25:101
132439931,1cyclictest0-21swapper/122:05:021
132439931,1cyclictest0-21swapper/121:55:121
132439931,1cyclictest0-21swapper/121:35:151
132439931,1cyclictest0-21swapper/121:30:121
132439931,1cyclictest0-21swapper/121:10:011
132439931,1cyclictest0-21swapper/120:55:131
132439931,1cyclictest0-21swapper/120:25:141
132439931,1cyclictest0-21swapper/120:10:111
132439931,1cyclictest0-21swapper/119:45:111
132439931,1cyclictest0-21swapper/119:30:121
132439931,1cyclictest0-21swapper/119:25:131
132439931,1cyclictest0-21swapper/119:20:161
132439931,1cyclictest0-21swapper/119:10:131
132439931,1cyclictest0-21swapper/100:35:151
132439930,3cyclictest0-21swapper/120:53:041
132439930,3cyclictest0-21swapper/119:57:211
132439930,3cyclictest0-21swapper/119:17:551
132439930,2cyclictest7254-21taskset23:35:501
132439930,2cyclictest0-21swapper/123:40:131
132439930,2cyclictest0-21swapper/122:00:131
132439930,2cyclictest0-21swapper/119:50:121
132439930,2cyclictest0-21swapper/100:00:121
132439930,0cyclictest0-21swapper/119:39:361
132429931,1cyclictest0-21swapper/022:15:120
132429931,1cyclictest0-21swapper/019:20:160
132429930,3cyclictest8573-21ssh00:10:030
132429930,3cyclictest1508-21apt-get19:50:120
132429930,3cyclictest0-21swapper/023:20:110
132429930,3cyclictest0-21swapper/023:15:130
132429930,3cyclictest0-21swapper/023:15:130
132429930,3cyclictest0-21swapper/021:35:120
132429930,3cyclictest0-21swapper/020:55:050
132429930,3cyclictest0-21swapper/020:35:050
132429930,3cyclictest0-21swapper/020:10:090
132429930,3cyclictest0-21swapper/020:00:090
132429930,3cyclictest0-21swapper/020:00:090
132429930,3cyclictest0-21swapper/019:35:140
132429930,3cyclictest0-21swapper/019:30:120
132429930,3cyclictest0-21swapper/019:10:130
132429930,3cyclictest0-21swapper/000:25:100
132429930,2cyclictest7156-21df21:05:140
132439922,0cyclictest0-21swapper/121:45:111
132439922,0cyclictest0-21swapper/120:07:581
132439921,1cyclictest312002chrt20:47:381
132439921,1cyclictest0-21swapper/123:55:131
132439920,1cyclictest26131-21chrt20:36:141
132439920,1cyclictest0-21swapper/123:10:091
132439920,1cyclictest0-21swapper/123:00:121
132439920,1cyclictest0-21swapper/122:50:111
132439920,1cyclictest0-21swapper/120:20:111
132439920,1cyclictest0-21swapper/120:15:111
132439920,1cyclictest0-21swapper/120:00:021
132439920,1cyclictest0-21swapper/120:00:021
132439920,1cyclictest0-21swapper/119:40:131
132439920,1cyclictest0-21swapper/100:25:131
132439920,1cyclictest0-21swapper/100:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional