You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 04:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Sun Jul 06, 2025 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121562225161,22sleep00-21swapper/019:06:570
121482212149,23sleep20-21swapper/219:06:512
121142199162,24sleep30-21swapper/319:06:233
120702197160,24sleep10-21swapper/119:05:491
206412630,0sleep30-21swapper/322:55:133
141692150,0sleep30-21swapper/322:02:153
25402140,0sleep00-21swapper/023:13:520
1248199133,5cyclictest28973-21perf21:39:590
1248199133,5cyclictest19415-21cat00:19:590
12481991312,1cyclictest9-21ksoftirqd/023:55:000
12481991311,1cyclictest9-21ksoftirqd/020:15:000
12481991211,1cyclictest9-21ksoftirqd/022:40:000
1251199119,1cyclictest35-21ksoftirqd/323:47:463
12511991110,1cyclictest35-21ksoftirqd/321:55:003
1251199110,6cyclictest0-21swapper/322:50:153
12503991110,1cyclictest0-21swapper/221:33:102
1250399110,1cyclictest4380-21irqstats23:15:182
1248199113,4cyclictest5147-21diskmemload23:20:000
1248199112,4cyclictest27945-21perf23:05:000
1248199112,4cyclictest17934-21date19:20:000
12481991110,1cyclictest9-21ksoftirqd/023:35:140
12481991110,1cyclictest9-21ksoftirqd/021:20:000
1251199109,1cyclictest35-21ksoftirqd/300:15:013
1251199100,8cyclictest0-21swapper/322:15:133
1251199100,7cyclictest21273-21apt-get20:35:123
1250399109,1cyclictest557-21nscd00:05:002
1248199109,1cyclictest9-21ksoftirqd/023:30:230
1248199109,1cyclictest9-21ksoftirqd/020:35:000
1248199109,1cyclictest0-21swapper/023:24:590
1248199104,5cyclictest29467-21perf23:50:000
12481991010,0cyclictest9-21ksoftirqd/023:25:210
29950290,0sleep20-21swapper/200:33:012
125119999,0cyclictest35-21ksoftirqd/320:10:173
125119999,0cyclictest35-21ksoftirqd/300:25:013
125119998,1cyclictest35-21ksoftirqd/322:10:183
125119998,0cyclictest35-21ksoftirqd/322:35:203
125039998,1cyclictest742-21runrttasks00:38:082
125039998,0cyclictest0-21swapper/221:56:012
125039997,1cyclictest5686-21runrttasks21:50:202
125039990,3cyclictest0-21swapper/223:40:262
124909990,4cyclictest0-21swapper/119:25:121
124819998,1cyclictest9-21ksoftirqd/022:20:150
124819998,1cyclictest9-21ksoftirqd/022:05:000
124819998,1cyclictest9-21ksoftirqd/021:10:160
124819997,1cyclictest9-21ksoftirqd/021:25:140
124819997,1cyclictest9-21ksoftirqd/019:35:000
124819997,1cyclictest9-21ksoftirqd/000:35:150
124819997,1cyclictest9-21ksoftirqd/000:30:160
124819994,1cyclictest15751-21cpuspeed_turbos22:05:130
124819994,1cyclictest0-21swapper/022:10:200
124819990,6cyclictest5147-21diskmemload23:05:150
124819990,5cyclictest22832-21sed19:30:000
124819990,4cyclictest29871-21latency_hist19:45:000
125119988,0cyclictest35-21ksoftirqd/321:10:233
125119986,1cyclictest5147-21diskmemload23:15:573
125119986,1cyclictest35-21ksoftirqd/319:50:003
125119986,1cyclictest35-21ksoftirqd/319:10:183
125119986,1cyclictest17740-21snmpd20:03:383
125119982,3cyclictest5147-21diskmemload23:28:053
125119980,6cyclictest29957-21sshd00:33:023
125119980,6cyclictest27419-21/usr/sbin/munin22:20:153
125039986,1cyclictest60950irq/123-eno123:08:442
125039986,1cyclictest60950irq/123-eno122:45:212
125039986,1cyclictest60950irq/123-eno122:15:562
125039986,1cyclictest60950irq/123-eno119:45:212
125039986,1cyclictest60950irq/123-eno100:20:182
125039986,1cyclictest17740-21snmpd23:47:502
125039986,1cyclictest17740-21snmpd23:38:112
125039983,4cyclictest17740-21snmpd22:40:092
125039983,1cyclictest8396-21ntp_states23:20:182
125039981,4cyclictest716-21open_inodes23:10:192
125039980,5cyclictest28-21ksoftirqd/223:56:182
125039980,3cyclictest0-21swapper/200:15:132
124909981,3cyclictest0-21swapper/123:05:121
124909980,7cyclictest2066-21taskset23:12:341
124819987,1cyclictest9-21ksoftirqd/021:55:000
124819986,1cyclictest9-21ksoftirqd/019:55:000
124819984,2cyclictest254-21systemd-journal00:25:000
124819980,7cyclictest24562-21rm00:25:170
124819980,7cyclictest15290-21ssh00:14:090
124819980,1cyclictest2919-21ssh22:30:200
124819980,1cyclictest11313-21ssh22:41:230
125119977,0cyclictest5147-21diskmemload23:30:173
125119977,0cyclictest35-21ksoftirqd/323:05:173
125119977,0cyclictest35-21ksoftirqd/320:50:133
125119976,1cyclictest35-21ksoftirqd/323:00:013
125119976,1cyclictest35-21ksoftirqd/321:10:013
125119976,1cyclictest35-21ksoftirqd/300:25:133
125119976,1cyclictest35-21ksoftirqd/300:15:143
125119976,1cyclictest17740-21snmpd20:07:353
125119975,1cyclictest35-21ksoftirqd/319:20:143
125119975,1cyclictest3278-21cat19:55:163
125119975,1cyclictest17740-21snmpd21:23:593
125119975,1cyclictest17740-21snmpd20:27:053
125119971,3cyclictest568-21nscd22:41:343
125119971,2cyclictest22457-21aten_r1power_po23:40:173
125119971,0cyclictest5147-21diskmemload21:29:303
125119970,4cyclictest0-21swapper/322:45:533
125119970,3cyclictest0-21swapper/321:35:133
125039976,1cyclictest60950irq/123-eno123:25:232
125039976,1cyclictest60950irq/123-eno122:35:212
125039976,1cyclictest60950irq/123-eno122:25:132
125039976,1cyclictest60950irq/123-eno121:10:262
125039976,1cyclictest60950irq/123-eno120:31:212
125039976,1cyclictest10524-21ntpq21:15:192
125039975,1cyclictest60950irq/123-eno121:25:132
125039975,1cyclictest60950irq/123-eno120:55:122
125039975,1cyclictest60950irq/123-eno100:10:152
125039975,1cyclictest17740-21snmpd21:04:422
125039975,1cyclictest17740-21snmpd00:28:222
125039974,2cyclictest60950irq/123-eno122:05:142
125039973,1cyclictest5147-21diskmemload23:50:142
125039972,3cyclictest0-21swapper/219:10:102
125039971,3cyclictest5147-21diskmemload22:30:152
125039970,1cyclictest0-21swapper/222:00:022
125039970,1cyclictest0-21swapper/220:12:022
125039970,1cyclictest0-21swapper/219:26:372
124909971,6cyclictest0-21swapper/123:31:291
124909971,6cyclictest0-21swapper/121:49:171
124909971,3cyclictest0-21swapper/100:26:391
124909970,7cyclictest0-21swapper/122:20:121
124909970,6cyclictest0-21swapper/121:39:231
124909970,5cyclictest0-21swapper/121:55:141
124909970,1cyclictest0-21swapper/123:15:581
124819977,0cyclictest9421-21sensors21:55:210
124819976,1cyclictest9-21ksoftirqd/022:50:140
124819976,1cyclictest9-21ksoftirqd/020:55:010
124819976,1cyclictest21696-21ntpq22:55:180
124819976,1cyclictest17740-21snmpd20:17:090
124819976,1cyclictest17740-21snmpd20:04:400
124819975,2cyclictest9-21ksoftirqd/021:20:480
124819975,1cyclictest9-21ksoftirqd/020:59:590
124819975,1cyclictest17740-21snmpd20:25:140
124819975,1cyclictest17740-21snmpd19:48:200
124819975,1cyclictest17740-21snmpd19:36:500
124819972,1cyclictest14833-21ssh22:45:500
124819971,6cyclictest6980-1kworker/u9:200:04:150
124819970,6cyclictest5147-21diskmemload21:40:070
124819970,6cyclictest31581-21ssh22:25:180
124819970,6cyclictest31225-1kworker/u9:121:45:020
124819970,6cyclictest0-21swapper/020:23:020
124819970,6cyclictest0-21swapper/000:05:170
124819970,3cyclictest0-21swapper/020:40:140
124819970,1cyclictest24612-21ssh23:41:510
124819970,1cyclictest21315-21ssh21:30:090
124819970,1cyclictest14077-21mailstats19:10:180
124819970,1cyclictest0-21swapper/023:55:180
15866260,0sleep10-21swapper/122:48:481
125119966,0cyclictest35-21ksoftirqd/321:30:203
125119965,1cyclictest9929-21ssh21:55:493
125119965,1cyclictest9742-21ssh23:22:283
125119965,1cyclictest7678-21ssh00:03:383
125119965,1cyclictest714-21ssh00:36:293
125119965,1cyclictest35-21ksoftirqd/319:35:133
125119965,1cyclictest35-21ksoftirqd/319:15:003
125119965,1cyclictest31530-21ssh21:41:533
125119965,1cyclictest30916-21ssh23:50:193
125119965,1cyclictest21154-21ssh23:37:383
125119965,1cyclictest17740-21snmpd23:59:193
125119964,1cyclictest35-21ksoftirqd/319:30:243
125119964,1cyclictest17740-21snmpd20:20:203
125119964,1cyclictest17740-21snmpd20:18:143
125119963,1cyclictest26667-21/usr/sbin/munin20:45:153
125119962,3cyclictest0-21swapper/319:50:123
125119961,3cyclictest0-21swapper/322:05:183
125119960,3cyclictest30402-21apt-get22:25:123
125119960,3cyclictest1743-21apt-get22:30:143
125119960,2cyclictest0-21swapper/321:48:343
125119960,1cyclictest8969-21diskstats00:05:133
125039966,0cyclictest60950irq/123-eno123:30:092
125039966,0cyclictest60950irq/123-eno122:56:032
125039966,0cyclictest60950irq/123-eno122:50:412
125039966,0cyclictest60950irq/123-eno121:45:132
125039966,0cyclictest60950irq/123-eno121:06:152
125039966,0cyclictest60950irq/123-eno120:53:222
125039966,0cyclictest60950irq/123-eno119:50:122
125039966,0cyclictest60950irq/123-eno119:19:412
125039966,0cyclictest60950irq/123-eno100:05:152
125039965,1cyclictest60950irq/123-eno123:00:242
125039965,1cyclictest60950irq/123-eno122:10:182
125039965,1cyclictest60950irq/123-eno121:40:112
125039965,1cyclictest60950irq/123-eno119:40:182
125039964,1cyclictest60950irq/123-eno120:35:112
125039964,1cyclictest60950irq/123-eno119:20:172
125039964,1cyclictest26661-21ssh22:20:082
125039962,3cyclictest25460-21aten2.4-expect21:35:132
125039961,4cyclictest0-21swapper/219:35:122
125039961,3cyclictest0-21swapper/221:20:112
125039960,5cyclictest0-21swapper/220:45:162
125039960,5cyclictest0-21swapper/220:40:232
125039960,5cyclictest0-21swapper/219:31:492
124909965,1cyclictest0-21swapper/123:04:191
124909965,1cyclictest0-21swapper/120:09:381
124909965,1cyclictest0-21swapper/119:36:221
124909965,1cyclictest0-21swapper/119:12:271
124909964,2cyclictest0-21swapper/122:27:391
124909961,3cyclictest0-21swapper/123:35:141
124909961,0cyclictest0-21swapper/119:48:041
124909960,6cyclictest0-21swapper/123:51:451
124909960,6cyclictest0-21swapper/122:01:211
124909960,6cyclictest0-21swapper/120:45:401
124909960,6cyclictest0-21swapper/119:16:071
124909960,6cyclictest0-21swapper/100:15:591
124909960,5cyclictest0-21swapper/122:50:561
124909960,5cyclictest0-21swapper/120:55:161
124909960,5cyclictest0-21swapper/120:01:181
124909960,4cyclictest0-21swapper/122:37:451
124909960,3cyclictest0-21swapper/100:10:201
124909960,3cyclictest0-21swapper/100:00:001
124909960,2cyclictest0-21swapper/121:13:571
124909960,1cyclictest0-21swapper/121:21:071
124909960,1cyclictest0-21swapper/120:16:141
124909960,1cyclictest0-21swapper/119:31:191
124819965,1cyclictest9-21ksoftirqd/021:05:000
124819965,1cyclictest9-21ksoftirqd/020:40:000
124819965,1cyclictest9-21ksoftirqd/020:00:000
124819965,1cyclictest17740-21snmpd20:09:050
124819964,1cyclictest30444-1kworker/u9:221:10:000
124819960,0cyclictest0-21swapper/022:15:000
124819960,0cyclictest0-21swapper/019:20:010
125119954,1cyclictest35-21ksoftirqd/320:30:203
125119954,0cyclictest5147-21diskmemload23:10:083
125119953,1cyclictest17740-21snmpd19:40:043
125119953,1cyclictest10605-21tr21:15:193
125119951,3cyclictest0-21swapper/319:25:123
125119951,2cyclictest0-21swapper/321:00:103
125039955,0cyclictest60950irq/123-eno119:55:132
125039954,1cyclictest60950irq/123-eno120:25:252
125039954,1cyclictest28-21ksoftirqd/220:17:472
125039953,1cyclictest17740-21snmpd20:00:012
125039950,4cyclictest0-21swapper/220:20:162
125039950,0cyclictest0-21swapper/220:05:012
124909955,0cyclictest0-21swapper/123:22:081
124909955,0cyclictest0-21swapper/121:31:561
124909953,1cyclictest279142sleep123:45:411
124909952,2cyclictest0-21swapper/122:09:111
124909951,2cyclictest0-21swapper/122:15:101
124909950,5cyclictest0-21swapper/122:11:061
124909950,5cyclictest0-21swapper/121:15:431
124909950,5cyclictest0-21swapper/120:54:241
124909950,5cyclictest0-21swapper/120:10:061
124909950,5cyclictest0-21swapper/119:40:411
124909950,5cyclictest0-21swapper/100:35:321
124909950,4cyclictest0-21swapper/100:08:031
124909950,3cyclictest0-21swapper/123:29:171
124909950,3cyclictest0-21swapper/120:25:261
124909950,3cyclictest0-21swapper/100:21:021
124909950,2cyclictest0-21swapper/122:43:241
124909950,2cyclictest0-21swapper/121:50:161
124909950,2cyclictest0-21swapper/121:26:451
124909950,2cyclictest0-21swapper/100:02:191
124909950,1cyclictest0-21swapper/123:40:181
124909950,1cyclictest0-21swapper/121:40:151
124819950,1cyclictest0-21swapper/020:45:120
125119943,1cyclictest30754-21aten_r1power_en20:55:143
125119941,2cyclictest23651-21apt-get20:40:143
124909944,0cyclictest0-21swapper/122:55:421
124909941,2cyclictest0-21swapper/120:20:121
124909941,1cyclictest0-21swapper/121:00:201
124909940,4cyclictest0-21swapper/121:09:531
124909940,3cyclictest0-21swapper/119:20:211
124909940,2cyclictest0-21swapper/120:35:211
124909940,1cyclictest0-21swapper/100:30:151
124909940,0cyclictest0-21swapper/122:30:091
124909940,0cyclictest0-21swapper/119:50:131
124909932,0cyclictest0-21swapper/119:55:121
124909931,1cyclictest0-21swapper/120:40:201
124909931,1cyclictest0-21swapper/120:30:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional