You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 08:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack1slot1.osadl.org (updated Sun Nov 16, 2025 00:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
249912215161,43sleep10-21swapper/119:08:511
248422215160,43sleep00-21swapper/019:06:570
247972202163,26sleep20-21swapper/219:06:232
247732196159,24sleep30-21swapper/319:06:033
219732690,0sleep10-21swapper/121:14:381
52792680,0sleep10-21swapper/121:35:121
11452660,0sleep20-21swapper/223:45:182
107542600,0sleep00-21swapper/022:27:220
25181991919,0cyclictest9-21ksoftirqd/023:58:030
25181991717,0cyclictest9-21ksoftirqd/022:06:290
25181991616,0cyclictest9-21ksoftirqd/000:35:030
249042160,0sleep00-21swapper/000:19:490
261072150,0sleep00-21swapper/023:35:150
25181991515,0cyclictest9-21ksoftirqd/023:11:540
25181991515,0cyclictest9-21ksoftirqd/021:39:110
25181991414,0cyclictest9-21ksoftirqd/022:46:320
25181991414,0cyclictest9-21ksoftirqd/022:16:080
25181991414,0cyclictest9-21ksoftirqd/021:57:150
25181991414,0cyclictest9-21ksoftirqd/021:31:030
25181991414,0cyclictest9-21ksoftirqd/000:33:280
25181991414,0cyclictest9-21ksoftirqd/000:13:270
25181991413,1cyclictest9-21ksoftirqd/022:03:360
25195991312,1cyclictest28-21ksoftirqd/221:40:002
2518199138,4cyclictest9-21ksoftirqd/023:06:340
2518199138,0cyclictest9-21ksoftirqd/021:50:580
25181991313,0cyclictest9-21ksoftirqd/023:47:440
25181991313,0cyclictest9-21ksoftirqd/022:57:370
25181991313,0cyclictest9-21ksoftirqd/021:15:350
25181991313,0cyclictest9-21ksoftirqd/000:28:000
25181991312,1cyclictest9-21ksoftirqd/022:32:490
25181991312,1cyclictest9-21ksoftirqd/000:20:160
25181991311,2cyclictest9-21ksoftirqd/021:13:360
2521099121,3cyclictest18520-21diskmemload23:25:153
2519599120,0cyclictest0-21swapper/223:10:122
2518199129,3cyclictest9-21ksoftirqd/023:15:030
25181991212,0cyclictest9-21ksoftirqd/023:24:100
25181991212,0cyclictest9-21ksoftirqd/023:00:260
25181991212,0cyclictest9-21ksoftirqd/022:50:190
25181991212,0cyclictest9-21ksoftirqd/022:50:190
25181991212,0cyclictest9-21ksoftirqd/022:12:060
25181991211,1cyclictest9-21ksoftirqd/023:28:280
25181991211,1cyclictest9-21ksoftirqd/022:38:060
25181991211,1cyclictest9-21ksoftirqd/020:26:300
25181991211,1cyclictest9-21ksoftirqd/000:06:550
2521099119,1cyclictest35-21ksoftirqd/320:00:003
2521099116,0cyclictest18520-21diskmemload22:15:263
25195991110,1cyclictest28-21ksoftirqd/223:01:042
2519599110,8cyclictest7449-21apt-get23:55:122
2519599110,0cyclictest0-21swapper/222:45:142
2518199119,2cyclictest9-21ksoftirqd/022:40:420
2518199115,5cyclictest9-21ksoftirqd/000:02:560
25181991111,0cyclictest9-21ksoftirqd/023:50:020
25181991111,0cyclictest9-21ksoftirqd/021:40:270
25181991111,0cyclictest9-21ksoftirqd/021:20:010
25181991110,1cyclictest9-21ksoftirqd/023:33:080
25181991110,1cyclictest9-21ksoftirqd/022:20:160
25181991110,1cyclictest9-21ksoftirqd/021:47:560
25181991110,1cyclictest9-21ksoftirqd/021:25:070
2521099100,6cyclictest0-21swapper/300:30:163
2521099100,5cyclictest0-21swapper/321:14:343
2521099100,2cyclictest0-21swapper/323:52:313
2519599109,1cyclictest28-21ksoftirqd/220:40:002
2519599100,9cyclictest27535-21munin-node22:30:242
2518199109,1cyclictest9-21ksoftirqd/019:30:130
2518199108,1cyclictest9-21ksoftirqd/019:16:220
2518199103,7cyclictest9-21ksoftirqd/019:54:370
25181991010,0cyclictest9-21ksoftirqd/023:40:130
25181991010,0cyclictest9-21ksoftirqd/023:40:130
252109991,5cyclictest8864-21/usr/sbin/munin21:40:193
252109991,4cyclictest18520-21diskmemload22:45:133
252109991,1cyclictest0-21swapper/319:16:223
252109990,5cyclictest0-21swapper/300:20:133
252109990,4cyclictest25683-21/usr/sbin/munin23:35:223
251959998,1cyclictest28-21ksoftirqd/221:11:262
251959996,1cyclictest28-21ksoftirqd/221:19:092
251819999,0cyclictest9-21ksoftirqd/019:25:210
251819998,1cyclictest9-21ksoftirqd/020:35:180
251819998,1cyclictest9-21ksoftirqd/020:10:150
251819998,1cyclictest9-21ksoftirqd/019:15:000
251819998,1cyclictest0-21swapper/020:40:250
251819998,1cyclictest0-21swapper/019:45:200
251819996,2cyclictest19110-21snmpd19:20:190
251819991,8cyclictest9-21ksoftirqd/021:05:120
251819991,8cyclictest9-21ksoftirqd/020:55:260
251819991,8cyclictest9-21ksoftirqd/020:48:490
251819991,8cyclictest9-21ksoftirqd/020:15:030
251819991,8cyclictest9-21ksoftirqd/019:55:010
252109987,1cyclictest0-21swapper/323:41:313
252109987,1cyclictest0-21swapper/323:41:313
252109986,1cyclictest19110-21snmpd22:34:393
252109986,1cyclictest19110-21snmpd20:44:313
252109986,1cyclictest19110-21snmpd20:03:243
252109980,6cyclictest19494-21apt-get21:55:133
252109980,5cyclictest0-21swapper/323:16:223
252109980,1cyclictest0-21swapper/321:47:253
251959987,1cyclictest28-21ksoftirqd/221:56:042
251959987,1cyclictest28-21ksoftirqd/220:29:032
251959986,1cyclictest60950irq/123-eno122:55:172
251959986,1cyclictest28-21ksoftirqd/220:55:162
251959986,1cyclictest19110-21snmpd22:11:512
251959983,4cyclictest20274-21apt-get20:05:122
251959980,4cyclictest0-21swapper/200:15:142
251899980,5cyclictest0-21swapper/123:55:121
251899980,4cyclictest0-21swapper/123:25:121
251819981,0cyclictest9-21ksoftirqd/020:20:040
251819981,0cyclictest9-21ksoftirqd/019:35:030
251819980,8cyclictest9-21ksoftirqd/021:00:050
251819980,8cyclictest9-21ksoftirqd/020:50:050
251819980,8cyclictest9-21ksoftirqd/020:30:030
251819980,8cyclictest9-21ksoftirqd/020:05:250
251819980,8cyclictest9-21ksoftirqd/020:00:450
251819980,0cyclictest9-21ksoftirqd/019:41:330
13974287,0sleep228-21ksoftirqd/221:45:252
252109976,1cyclictest27920-21fschecks_count20:20:173
252109976,1cyclictest19110-21snmpd19:51:443
252109975,1cyclictest6253-21ssh21:35:173
252109975,1cyclictest3717-21ssh21:31:433
252109975,1cyclictest32683-21ssh22:12:493
252109975,1cyclictest30655-21cat22:55:193
252109975,1cyclictest19110-21snmpd20:09:563
252109975,1cyclictest10660-21ssh22:27:043
252109974,1cyclictest18520-21diskmemload21:18:413
252109971,5cyclictest546-21irqbalance23:20:053
252109970,6cyclictest31395-21ssh00:27:383
252109970,4cyclictest0-21swapper/323:57:333
252109970,4cyclictest0-21swapper/300:15:073
252109970,4cyclictest0-21swapper/300:10:423
252109970,3cyclictest3661-21/usr/sbin/munin00:35:193
252109970,2cyclictest0-21swapper/322:40:223
252109970,2cyclictest0-21swapper/322:07:433
251959976,1cyclictest60950irq/123-eno123:50:312
251959976,1cyclictest60950irq/123-eno123:40:192
251959976,1cyclictest60950irq/123-eno123:40:182
251959976,1cyclictest60950irq/123-eno123:33:022
251959976,1cyclictest60950irq/123-eno122:23:382
251959976,1cyclictest19110-21snmpd22:02:382
251959976,0cyclictest60950irq/123-eno123:37:052
251959976,0cyclictest60950irq/123-eno119:45:232
251959975,1cyclictest60950irq/123-eno122:20:002
251959975,1cyclictest60950irq/123-eno121:31:472
251959975,1cyclictest60950irq/123-eno119:30:212
251959975,1cyclictest60950irq/123-eno100:00:132
251959975,1cyclictest4359-21ssh23:05:032
251959975,1cyclictest30085-21ssh21:25:052
251959975,1cyclictest28-21ksoftirqd/220:50:012
251959975,1cyclictest1449-21ntpd00:21:232
251959974,2cyclictest16812-21hwlatdetect21:50:162
251959973,3cyclictest25171-21cyclictest22:41:222
251959972,3cyclictest0-21swapper/222:50:192
251959972,3cyclictest0-21swapper/222:50:192
251959971,5cyclictest32475-21apt-get00:30:152
251959970,7cyclictest0-21swapper/221:42:262
251959970,6cyclictest0-21swapper/219:20:132
251899972,2cyclictest0-21swapper/100:21:101
251899970,7cyclictest0-21swapper/122:40:001
251899970,5cyclictest0-21swapper/119:55:191
17591275,1sleep228-21ksoftirqd/200:09:182
252109966,0cyclictest0-21swapper/321:51:353
252109965,1cyclictest29150-21ssh21:23:063
252109965,1cyclictest28025-21ssh22:50:273
252109965,1cyclictest28025-21ssh22:50:273
252109965,1cyclictest22069-21ssh23:30:013
252109965,1cyclictest19110-21snmpd21:03:393
252109965,1cyclictest16384-21sshd21:05:043
252109964,1cyclictest508-21apt-get19:25:013
252109964,1cyclictest4863-21aten2.4-expect22:20:143
252109964,1cyclictest19110-21snmpd19:11:573
252109964,1cyclictest15467-21irqstats00:05:193
252109962,4cyclictest18520-21diskmemload21:25:363
252109962,3cyclictest24089-21/usr/sbin/munin22:00:213
252109961,1cyclictest0-21swapper/323:00:163
252109961,1cyclictest0-21swapper/322:36:013
252109960,3cyclictest9684-21/usr/sbin/munin19:40:243
252109960,3cyclictest18520-21diskmemload23:45:203
252109960,3cyclictest0-21swapper/323:07:003
252109960,2cyclictest0-21swapper/300:00:003
252109960,1cyclictest0-21swapper/323:10:503
251959966,0cyclictest60950irq/123-eno123:15:152
251959966,0cyclictest60950irq/123-eno122:36:472
251959966,0cyclictest60950irq/123-eno122:25:012
251959966,0cyclictest60950irq/123-eno122:05:112
251959966,0cyclictest60950irq/123-eno120:20:312
251959966,0cyclictest60950irq/123-eno120:15:162
251959966,0cyclictest60950irq/123-eno120:04:262
251959966,0cyclictest60950irq/123-eno119:43:512
251959965,1cyclictest28-21ksoftirqd/221:00:122
251959965,1cyclictest19110-21snmpd19:27:302
251959964,1cyclictest60950irq/123-eno100:25:152
251959964,1cyclictest28-21ksoftirqd/219:50:042
251959964,1cyclictest1449-21ntpd19:16:222
251959964,1cyclictest10732-21/usr/sbin/munin20:50:212
251959963,2cyclictest60950irq/123-eno120:30:252
251959963,2cyclictest18134-21apt-get00:10:162
251959963,2cyclictest15103-21head23:20:132
251959962,2cyclictest27553-21ntpq21:20:182
251959960,6cyclictest0-21swapper/219:55:222
251959960,6cyclictest0-21swapper/219:13:212
251959960,6cyclictest0-21swapper/200:35:142
251959960,5cyclictest18931-21cat23:25:182
251959960,1cyclictest0-21swapper/219:38:262
251899964,1cyclictest0-21swapper/120:15:141
251899962,2cyclictest0-21swapper/122:22:031
251899961,2cyclictest0-21swapper/122:50:151
251899961,2cyclictest0-21swapper/122:50:141
251899960,6cyclictest0-21swapper/123:48:001
251899960,6cyclictest0-21swapper/122:10:161
251899960,6cyclictest0-21swapper/121:15:171
251899960,6cyclictest0-21swapper/119:39:011
251899960,5cyclictest0-21swapper/121:25:161
251899960,5cyclictest0-21swapper/120:28:021
251899960,5cyclictest0-21swapper/119:33:191
251899960,5cyclictest0-21swapper/100:35:171
251899960,4cyclictest0-21swapper/123:05:011
251899960,3cyclictest0-21swapper/123:21:521
251899960,2cyclictest0-21swapper/123:00:131
251899960,1cyclictest0-21swapper/119:45:071
251899960,0cyclictest0-21swapper/123:38:141
27751250,1sleep1111rcu_preempt20:20:151
252109954,1cyclictest19110-21snmpd20:49:413
252109954,1cyclictest19110-21snmpd19:39:283
252109953,1cyclictest32292-21apt-get20:30:183
252109953,1cyclictest2226-21apt-get20:35:143
252109952,2cyclictest0-21swapper/319:45:143
252109952,1cyclictest254-21systemd-journal19:35:003
252109951,3cyclictest0-21swapper/320:25:133
252109951,2cyclictest11674-21apt-get20:55:163
252109950,2cyclictest0-21swapper/320:15:213
252109950,1cyclictest9811-21df20:50:143
251959954,1cyclictest60950irq/123-eno120:40:192
251959954,1cyclictest60950irq/123-eno120:10:202
251959953,1cyclictest16654-21tr21:05:142
251899955,0cyclictest0-21swapper/122:45:001
251899955,0cyclictest0-21swapper/122:40:451
251899955,0cyclictest0-21swapper/119:18:101
251899953,2cyclictest0-21swapper/120:05:201
251899952,2cyclictest0-21swapper/122:19:251
251899952,0cyclictest0-21swapper/100:09:261
251899951,2cyclictest0-21swapper/123:10:461
251899951,1cyclictest0-21swapper/121:48:351
251899951,1cyclictest0-21swapper/100:25:181
251899950,5cyclictest0-21swapper/123:15:341
251899950,5cyclictest0-21swapper/121:01:541
251899950,5cyclictest0-21swapper/120:50:461
251899950,5cyclictest0-21swapper/120:45:181
251899950,5cyclictest0-21swapper/119:53:131
251899950,5cyclictest0-21swapper/119:42:021
251899950,5cyclictest0-21swapper/119:20:391
251899950,4cyclictest0-21swapper/123:40:411
251899950,4cyclictest0-21swapper/123:40:401
251899950,4cyclictest0-21swapper/121:50:111
251899950,4cyclictest0-21swapper/100:00:161
251899950,3cyclictest0-21swapper/123:50:191
251899950,3cyclictest0-21swapper/123:30:221
251899950,3cyclictest0-21swapper/122:55:051
251899950,3cyclictest0-21swapper/122:25:581
251899950,3cyclictest0-21swapper/121:55:471
251899950,3cyclictest0-21swapper/121:30:181
251899950,3cyclictest0-21swapper/121:20:081
251899950,3cyclictest0-21swapper/119:10:151
251899950,3cyclictest0-21swapper/100:16:341
251899950,1cyclictest0-21swapper/122:03:551
20688250,1sleep10-21swapper/100:12:501
252109944,0cyclictest24034-21sensors20:10:233
252109941,2cyclictest980-21apt-get19:25:163
251899944,0cyclictest0-21swapper/120:30:051
251899941,3cyclictest0-21swapper/120:00:191
251899940,4cyclictest0-21swapper/120:58:121
251899940,3cyclictest0-21swapper/121:40:131
251899940,3cyclictest0-21swapper/121:08:441
251899940,3cyclictest0-21swapper/119:30:001
251899940,2cyclictest0-21swapper/122:05:121
251899940,2cyclictest0-21swapper/100:30:211
251899940,1cyclictest0-21swapper/120:40:151
251899940,0cyclictest0-21swapper/120:35:221
11854240,0sleep10-21swapper/122:30:081
251899930,1cyclictest0-21swapper/120:10:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional