You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 21:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sat Nov 29, 2025 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
125232207192,10sleep10-21swapper/107:09:161
103602199182,12sleep30-21swapper/307:05:053
123012190172,13sleep00-21swapper/007:06:220
12507212187,12sleep20-21swapper/207:09:052
317712680,1sleep11277899cyclictest10:09:361
219322450,0sleep30-21swapper/310:52:363
155392120,0sleep10-21swapper/109:12:581
204012110,0sleep10-21swapper/109:47:181
122152110,0sleep10-21swapper/109:59:541
6721290,0sleep10-21swapper/111:00:521
17540290,0sleep00-21swapper/012:28:450
127779982,3cyclictest15354-21perf11:55:000
127839971,3cyclictest26278-21chrt10:38:542
127839971,1cyclictest0-21swapper/212:02:052
127839971,1cyclictest0-21swapper/210:21:502
127839971,1cyclictest0-21swapper/209:45:022
127839970,4cyclictest26385-21ssh11:27:192
127839970,3cyclictest3844-21cat09:00:002
127839970,2cyclictest301ktimersoftd/211:59:252
127839970,1cyclictest301ktimersoftd/209:56:332
127779971,3cyclictest6346-21munin-run08:00:000
15057260,0sleep10-21swapper/108:15:161
127849960,1cyclictest25945-21sh11:10:243
127839963,2cyclictest15224-21munin-node07:15:012
127839962,1cyclictest0-21swapper/209:40:122
127839961,3cyclictest10520-21ssh11:51:572
127839961,2cyclictest7732-21df_inode09:25:142
127839961,2cyclictest3389-21cut12:05:142
127839961,2cyclictest22726-21ssh12:30:462
127839961,2cyclictest1528-21cut12:20:192
127839961,1cyclictest31-21ksoftirqd/211:00:542
127839961,1cyclictest0-21swapper/212:13:262
127839961,1cyclictest0-21swapper/211:48:322
127839961,1cyclictest0-21swapper/211:40:182
127839961,1cyclictest0-21swapper/211:35:472
127839961,1cyclictest0-21swapper/211:21:102
127839961,1cyclictest0-21swapper/210:30:222
127839961,1cyclictest0-21swapper/209:53:512
127839960,4cyclictest910-21ssh11:30:252
127839960,4cyclictest30887-21ssh12:19:092
127839960,4cyclictest2478-21ssh10:42:532
127839960,4cyclictest24741-21python10:05:222
127839960,4cyclictest20562-21ssh10:51:442
127839960,4cyclictest1338-21ssh09:38:062
127839960,3cyclictest9438-21sh11:19:112
127839960,3cyclictest3406-21ssh10:27:052
127839960,3cyclictest31873-21ssh09:20:322
127839960,3cyclictest20872-21ssh11:08:302
127839960,3cyclictest13309-21ssh09:11:342
127839960,3cyclictest12373-21ssh12:25:242
127839960,2cyclictest3985-21ssh10:11:042
127839960,2cyclictest30217-21/usr/sbin/munin07:40:202
127839960,2cyclictest29071-21rm11:12:252
127839960,2cyclictest22123-21rm09:32:062
127839960,2cyclictest13682-21rm10:48:412
127839960,2cyclictest12837-21rm10:00:082
127839960,2cyclictest12837-21rm10:00:082
127839960,1cyclictest31-21ksoftirqd/209:15:142
127779960,5cyclictest0-21swapper/008:20:110
127779960,3cyclictest8787-21munin-run09:10:010
127849953,1cyclictest31368-21/usr/sbin/munin07:45:143
127849951,3cyclictest26303-21latency_hist08:40:003
127849950,5cyclictest0-21swapper/311:51:093
127849950,5cyclictest0-21swapper/310:25:183
127849950,0cyclictest0-21swapper/309:24:293
127839953,2cyclictest21046-21munin-run07:25:012
127839953,2cyclictest15505-21df_inode07:15:012
127839951,2cyclictest31-21ksoftirqd/212:35:032
127839951,2cyclictest26694-21grep08:40:142
127839951,1cyclictest16862-21cstates08:20:132
127839951,1cyclictest0-21swapper/208:25:232
127839950,3cyclictest12423-21ssh10:15:192
127839950,2cyclictest26234-21ssh10:55:112
127839950,2cyclictest25257-21python07:30:192
127839950,2cyclictest25169-21ntpq08:35:182
127839950,1cyclictest31-21ksoftirqd/208:00:142
127839950,1cyclictest0-21swapper/207:35:222
127789950,5cyclictest0-21swapper/111:48:031
127789950,5cyclictest0-21swapper/111:17:031
127789950,5cyclictest0-21swapper/110:42:431
127789950,5cyclictest0-21swapper/107:50:171
127779952,2cyclictest0-21swapper/007:35:170
127779950,5cyclictest0-21swapper/011:20:550
127779950,5cyclictest0-21swapper/010:50:310
127779950,5cyclictest0-21swapper/009:36:010
127779950,4cyclictest2996-21apt-get09:55:140
127779950,4cyclictest0-21swapper/011:30:120
127779950,4cyclictest0-21swapper/009:20:120
127779950,0cyclictest0-21swapper/010:10:390
127779950,0cyclictest0-21swapper/009:43:440
127849944,0cyclictest0-21swapper/312:00:543
127849943,1cyclictest31368-21tr11:30:133
127849943,1cyclictest29793-21/usr/sbin/munin12:35:213
127849943,1cyclictest29084-21df11:45:153
127849943,1cyclictest11732-21cut08:10:143
127849942,1cyclictest4942-21/usr/sbin/munin09:40:213
127849942,1cyclictest16308-21/usr/sbin/munin07:15:203
127849942,1cyclictest10196-21hddtemp_smartct09:10:153
127849941,2cyclictest0-21swapper/309:05:033
127849941,2cyclictest0-21swapper/307:20:143
127849941,1cyclictest0-21swapper/312:10:143
127849941,1cyclictest0-21swapper/307:55:123
127849940,4cyclictest0-21swapper/312:24:023
127849940,4cyclictest0-21swapper/312:15:143
127849940,4cyclictest0-21swapper/310:10:103
127849940,4cyclictest0-21swapper/310:05:273
127849940,4cyclictest0-21swapper/309:51:253
127849940,4cyclictest0-21swapper/309:00:143
127849940,3cyclictest8517-21ssh11:35:083
127849940,3cyclictest7403-21/usr/sbin/munin10:45:183
127849940,3cyclictest6891-21sh11:00:583
127849940,3cyclictest4462-21/usr/sbin/munin12:05:183
127849940,3cyclictest30706-21users08:45:213
127849940,3cyclictest29603-21perl10:40:163
127849940,3cyclictest27659-21/usr/sbin/munin07:35:233
127849940,3cyclictest26471-21/usr/sbin/munin10:55:133
127849940,3cyclictest26386-21/usr/sbin/munin08:40:203
127849940,3cyclictest24339-21sh11:59:283
127849940,3cyclictest24264-21df_inode07:30:133
127849940,3cyclictest23335-21/usr/sbin/munin11:25:233
127849940,3cyclictest21095-21sh10:35:383
127849940,3cyclictest19687-21/usr/sbin/munin11:40:183
127849940,3cyclictest17233-21ssh09:30:113
127849940,3cyclictest17152-21perl08:20:143
127849940,3cyclictest14975-21sh10:33:043
127849940,3cyclictest14440-21diskstats11:05:143
127849940,3cyclictest13991-21/usr/sbin/munin08:15:173
127849940,3cyclictest13707-21hddtemp_smartct07:10:153
127849940,3cyclictest13663-21ssh10:15:443
127849940,3cyclictest1254-21cat08:55:003
127849940,3cyclictest12539-21sh12:25:303
127849940,3cyclictest11101-21sh11:20:053
127849940,2cyclictest0-21swapper/311:17:213
127849940,1cyclictest8799-21cron08:05:003
127849940,1cyclictest28964-21munin-run09:20:013
127849940,1cyclictest27357-21/usr/sbin/munin09:35:213
127849940,1cyclictest21796-21perl10:20:173
127849940,1cyclictest21269-21/usr/sbin/munin07:25:173
127849940,1cyclictest20028-21/usr/sbin/munin12:30:193
127849940,0cyclictest0-21swapper/309:55:213
127839943,1cyclictest31410-21wget08:50:132
127839942,2cyclictest21947-21fschecks_time07:25:152
127839942,2cyclictest1479-21date07:50:012
127839942,1cyclictest11518-21apt-get08:10:132
127839941,3cyclictest6524-21chrt09:05:022
127839941,2cyclictest30935-21apt-get07:45:002
127839941,1cyclictest31-21ksoftirqd/208:05:132
127839941,1cyclictest0-21swapper/208:45:152
127839940,1cyclictest4622-21diskstats07:55:142
127839940,1cyclictest0-21swapper/209:00:132
127839940,1cyclictest0-21swapper/208:30:112
127839940,1cyclictest0-21swapper/208:15:122
127789941,2cyclictest0-21swapper/112:15:141
127789941,2cyclictest0-21swapper/111:20:121
127789941,2cyclictest0-21swapper/110:45:141
127789941,2cyclictest0-21swapper/110:10:021
127789941,2cyclictest0-21swapper/108:05:131
127789940,4cyclictest0-21swapper/110:51:091
127789940,4cyclictest0-21swapper/110:15:041
127789940,4cyclictest0-21swapper/109:41:101
127789940,0cyclictest0-21swapper/112:10:361
127789940,0cyclictest0-21swapper/111:38:181
127789940,0cyclictest0-21swapper/109:02:181
127779944,0cyclictest8777-21diskmemload11:35:200
127779944,0cyclictest0-21swapper/012:31:210
127779944,0cyclictest0-21swapper/010:05:160
127779943,1cyclictest22216-21grep08:30:150
127779941,3cyclictest0-21swapper/008:10:140
127779940,4cyclictest9968-21ntp_states10:30:180
127779940,4cyclictest8777-21diskmemload10:20:460
127779940,4cyclictest1190-21munin-run07:50:000
127779940,4cyclictest0-21swapper/012:10:130
127779940,4cyclictest0-21swapper/011:55:030
127779940,4cyclictest0-21swapper/011:40:150
127779940,4cyclictest0-21swapper/011:25:150
127779940,4cyclictest0-21swapper/010:56:550
127779940,4cyclictest0-21swapper/010:45:500
127779940,4cyclictest0-21swapper/009:50:130
127779940,4cyclictest0-21swapper/009:29:430
127779940,4cyclictest0-21swapper/009:16:490
127779940,4cyclictest0-21swapper/008:35:140
127779940,4cyclictest0-21swapper/007:25:120
127779940,3cyclictest7099-21df08:00:140
127779940,3cyclictest5441-21apt11:50:000
127779940,3cyclictest3870-21apt-get09:00:130
127779940,3cyclictest3282-21ssh12:05:140
127779940,3cyclictest32575-21sh12:20:040
127779940,3cyclictest30361-21df12:35:150
127779940,3cyclictest28520-21ssh10:40:010
127779940,3cyclictest2387-21ssh10:26:270
127779940,3cyclictest23380-21cat11:10:000
127779940,3cyclictest17332-21rm10:01:480
127779940,3cyclictest17332-21rm10:01:480
127779940,3cyclictest13390-21df07:10:130
127779940,3cyclictest10269-21apt-get10:15:030
127779940,3cyclictest0-21swapper/012:00:130
127779940,3cyclictest0-21swapper/011:10:020
127779940,3cyclictest0-21swapper/008:50:120
127779940,3cyclictest0-21swapper/008:40:120
127779940,3cyclictest0-21swapper/007:20:140
127779940,1cyclictest30001-21sed08:45:170
127779940,0cyclictest18860-21idleruntime-cro09:15:010
12436240,0sleep10-21swapper/112:09:571
127849932,0cyclictest0-21swapper/309:45:123
127849930,3cyclictest0-21swapper/308:55:013
127849930,3cyclictest0-21swapper/308:30:113
127849930,3cyclictest0-21swapper/308:25:013
127849930,3cyclictest0-21swapper/308:00:133
127849930,3cyclictest0-21swapper/307:50:013
127849930,3cyclictest0-21swapper/307:40:133
127849930,1cyclictest6988-21ssh09:25:053
127849930,1cyclictest12721-21ssh10:00:033
127849930,1cyclictest12721-21ssh10:00:033
127789933,0cyclictest0-21swapper/112:38:061
127789933,0cyclictest0-21swapper/111:43:321
127789933,0cyclictest0-21swapper/110:20:271
127789931,2cyclictest0-21swapper/111:25:151
127789931,1cyclictest0-21swapper/112:25:141
127789931,1cyclictest0-21swapper/112:00:141
127789931,1cyclictest0-21swapper/110:55:131
127789931,1cyclictest0-21swapper/110:25:141
127789931,1cyclictest0-21swapper/109:25:011
127789931,1cyclictest0-21swapper/109:20:121
127789931,1cyclictest0-21swapper/108:30:111
127789931,1cyclictest0-21swapper/108:20:121
127789931,1cyclictest0-21swapper/107:25:121
127789930,2cyclictest21656-21chrt09:31:471
127789930,2cyclictest0-21swapper/111:50:121
127789930,2cyclictest0-21swapper/107:40:121
127789930,2cyclictest0-21swapper/107:20:141
127779933,0cyclictest0-21swapper/008:05:040
127779931,1cyclictest0-21swapper/008:25:130
127779930,3cyclictest8777-21diskmemload11:15:040
127779930,3cyclictest8777-21diskmemload09:30:040
127779930,3cyclictest28702-21apt-get07:40:120
127779930,3cyclictest15113-21apt-get07:15:010
127779930,3cyclictest14519-21df_inode08:15:140
127779930,3cyclictest0-21swapper/012:15:020
127779930,3cyclictest0-21swapper/011:00:020
127779930,3cyclictest0-21swapper/010:35:130
127779930,3cyclictest0-21swapper/009:45:020
127779930,3cyclictest0-21swapper/008:55:120
127779930,3cyclictest0-21swapper/007:50:000
127779930,3cyclictest0-21swapper/007:30:110
127789922,0cyclictest0-21swapper/112:32:081
127789921,1cyclictest193232chrt09:15:101
127789921,1cyclictest0-21swapper/109:50:131
127789921,1cyclictest0-21swapper/108:50:131
127789921,1cyclictest0-21swapper/108:25:131
127789921,1cyclictest0-21swapper/107:55:121
127789921,1cyclictest0-21swapper/107:10:121
127789920,1cyclictest6566-21chrt08:00:031
127789920,1cyclictest263772chrt08:40:061
127789920,1cyclictest23571-21chrt11:10:041
127789920,1cyclictest0-21swapper/112:20:131
127789920,1cyclictest0-21swapper/111:55:131
127789920,1cyclictest0-21swapper/111:30:121
127789920,1cyclictest0-21swapper/111:05:121
127789920,1cyclictest0-21swapper/110:35:141
127789920,1cyclictest0-21swapper/110:30:121
127789920,1cyclictest0-21swapper/109:35:141
127789920,1cyclictest0-21swapper/109:05:031
127789920,1cyclictest0-21swapper/108:45:121
127789920,1cyclictest0-21swapper/108:35:121
127789920,1cyclictest0-21swapper/108:10:141
127789920,1cyclictest0-21swapper/107:45:011
127789920,1cyclictest0-21swapper/107:30:111
127789920,1cyclictest0-21swapper/107:15:021
127789920,0cyclictest0-21swapper/110:00:131
127789920,0cyclictest0-21swapper/110:00:131
127789920,0cyclictest0-21swapper/108:55:101
127789910,1cyclictest0-21swapper/107:35:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional