You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 07:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Fri Nov 28, 2025 00:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295742209177,21sleep30-21swapper/319:09:133
293792203149,21sleep10-21swapper/119:06:411
293182201163,24sleep20-21swapper/219:05:552
294992200138,50sleep00-21swapper/019:08:140
29708991411,1cyclictest9-21ksoftirqd/019:50:000
197922140,0sleep30-21swapper/321:00:213
2970999130,12cyclictest0-21swapper/119:50:171
29717991210,1cyclictest28-21ksoftirqd/222:39:592
29717991210,1cyclictest28-21ksoftirqd/222:25:002
29718991110,1cyclictest35-21ksoftirqd/320:25:013
29718991110,1cyclictest35-21ksoftirqd/320:09:593
29718991110,1cyclictest35-21ksoftirqd/300:09:593
2971799119,1cyclictest28-21ksoftirqd/222:00:012
2971799119,1cyclictest28-21ksoftirqd/221:15:002
29717991110,1cyclictest28-21ksoftirqd/222:55:162
2971799110,0cyclictest0-21swapper/200:24:402
2971899109,1cyclictest35-21ksoftirqd/320:19:593
29718991010,0cyclictest0-21swapper/323:38:113
29718991010,0cyclictest0-21swapper/323:38:103
2971899100,8cyclictest3140-21apt-get19:20:153
2971799109,1cyclictest28-21ksoftirqd/221:40:172
2971799109,1cyclictest28-21ksoftirqd/220:20:002
2971799109,1cyclictest28-21ksoftirqd/200:15:002
2971799108,1cyclictest28-21ksoftirqd/223:50:152
2971799108,1cyclictest28-21ksoftirqd/219:40:002
2971799107,1cyclictest28-21ksoftirqd/220:35:012
2970899107,1cyclictest278-21systemd-udevd00:11:090
2970899102,4cyclictest15646-21perf23:15:000
2970899100,9cyclictest32543-21ssh00:23:550
297189999,0cyclictest35-21ksoftirqd/321:59:583
297179999,0cyclictest28-21ksoftirqd/222:45:162
297179999,0cyclictest28-21ksoftirqd/221:30:142
297179998,1cyclictest28-21ksoftirqd/223:20:222
297179998,1cyclictest28-21ksoftirqd/222:10:012
297179998,1cyclictest28-21ksoftirqd/221:45:142
297179997,1cyclictest28-21ksoftirqd/223:45:002
297179997,1cyclictest28-21ksoftirqd/223:30:132
297179997,1cyclictest28-21ksoftirqd/219:20:002
297179990,5cyclictest60950irq/123-eno123:26:302
297089994,1cyclictest0-21swapper/022:05:320
297089990,8cyclictest0-21swapper/023:21:520
297089990,1cyclictest254-21systemd-journal19:26:420
12274290,0sleep00-21swapper/021:37:170
297189987,1cyclictest35-21ksoftirqd/323:00:013
297189984,1cyclictest23238-21diskmemload23:56:313
297189982,3cyclictest23238-21diskmemload22:40:123
297189980,6cyclictest12990-21/usr/sbin/munin22:25:173
297179988,0cyclictest28-21ksoftirqd/223:10:152
297179987,1cyclictest28-21ksoftirqd/221:25:172
297179986,1cyclictest8828-21cat00:35:162
297179986,1cyclictest60950irq/123-eno121:52:182
297179986,1cyclictest60950irq/123-eno121:35:172
297179986,1cyclictest60950irq/123-eno100:30:262
297179986,1cyclictest60950irq/123-eno100:00:212
297179984,2cyclictest570-21systemd-logind23:05:202
297179980,7cyclictest28-21ksoftirqd/223:02:332
297099980,4cyclictest0-21swapper/123:05:191
297089987,1cyclictest1449-21ntpd21:40:050
297089986,1cyclictest9-21ksoftirqd/020:55:000
297089986,1cyclictest8148-21chrt21:31:220
297089981,6cyclictest8433-21ssh00:35:130
297089980,7cyclictest8420-21ssh22:18:190
297089980,7cyclictest649-21ssh21:21:060
297089980,7cyclictest30657-21ssh22:50:120
297089980,7cyclictest22702-21ssh22:38:160
297089980,7cyclictest0-21swapper/022:40:160
8209270,0sleep10-21swapper/122:17:381
297189977,0cyclictest35-21ksoftirqd/323:06:233
297189976,1cyclictest35-21ksoftirqd/320:05:003
297189976,1cyclictest1449-21ntpd22:31:223
297189975,1cyclictest35-21ksoftirqd/319:45:013
297189975,1cyclictest25468-21ssh21:11:113
297189975,1cyclictest1449-21ntpd19:26:223
297189974,1cyclictest0-21swapper/323:20:173
297189973,3cyclictest564-21nscd21:33:073
297189972,4cyclictest23232-21rm00:10:173
297189972,3cyclictest35-21ksoftirqd/300:34:333
297189971,4cyclictest14933-21apt-get19:45:103
297189971,1cyclictest0-21swapper/323:12:163
297189970,6cyclictest226032sleep321:51:223
297189970,3cyclictest0-21swapper/300:19:103
297179977,0cyclictest28-21ksoftirqd/223:15:202
297179977,0cyclictest28-21ksoftirqd/221:20:182
297179976,1cyclictest60950irq/123-eno123:59:552
297179976,1cyclictest60950irq/123-eno122:33:402
297179976,1cyclictest60950irq/123-eno120:10:122
297179976,1cyclictest60950irq/123-eno119:20:162
297179976,1cyclictest28-21ksoftirqd/223:40:162
297179976,1cyclictest28-21ksoftirqd/222:20:192
297179976,1cyclictest28-21ksoftirqd/219:10:122
297179976,1cyclictest0-21swapper/220:25:172
297179976,0cyclictest0-21swapper/222:43:272
297179975,1cyclictest60950irq/123-eno121:00:162
297179975,1cyclictest60950irq/123-eno119:40:122
297179975,1cyclictest28465-21find21:15:202
297179975,1cyclictest10273-21ntpq20:40:192
297179974,2cyclictest8971-21munin-run22:20:002
297179973,3cyclictest29926-21diskmemload23:35:042
297179973,3cyclictest29926-21diskmemload23:35:032
297179970,6cyclictest0-21swapper/219:55:152
297179970,5cyclictest4069-21grep00:30:002
297179970,5cyclictest0-21swapper/200:10:202
297099970,4cyclictest0-21swapper/122:05:401
297089976,1cyclictest29173-21ssh21:16:070
297089976,1cyclictest25129-1kworker/u9:023:36:330
297089976,1cyclictest25129-1kworker/u9:023:36:320
297089975,1cyclictest7324-21ssh00:34:060
297089975,1cyclictest1449-21ntpd23:46:220
297089974,1cyclictest254-21systemd-journal19:40:000
297089972,2cyclictest254-21systemd-journal21:10:290
297089971,6cyclictest26809-21ssh21:58:340
297089971,6cyclictest24934-21ssh23:26:020
297089970,6cyclictest4173-21ssh21:25:400
297089970,6cyclictest28910-21ssh22:45:240
297089970,6cyclictest26994-21irqstats00:15:180
297089970,6cyclictest19942-21ssh21:49:060
297089970,6cyclictest19686-21sed21:00:210
297089970,6cyclictest17819-21ssh00:02:550
297089970,6cyclictest11912-21ssh22:23:080
297089970,6cyclictest11040-21ssh23:53:570
297089970,1cyclictest543-21cron20:40:010
297089970,1cyclictest3056-21ssh00:26:540
297089970,1cyclictest22977-21ssh21:52:310
297089970,1cyclictest19101-21cstates00:05:120
297089970,1cyclictest1782-21ssh22:55:120
297089970,1cyclictest13412-21ssh22:25:150
297089970,1cyclictest1281-21expr19:15:150
297089970,1cyclictest0-21swapper/023:40:240
297089970,1cyclictest0-21swapper/023:05:060
297089970,1cyclictest0-21swapper/020:14:500
297189965,1cyclictest9694-21ssh00:35:223
297189965,1cyclictest9334-21ntpq23:50:213
297189965,1cyclictest7627-21ssh22:15:463
297189965,1cyclictest5204-21ssh21:28:403
297189965,1cyclictest4316-21ssh22:11:243
297189965,1cyclictest2972-21ssh23:40:373
297189965,1cyclictest28266-21ssh23:30:273
297189965,1cyclictest27040-21ssh21:15:023
297189965,1cyclictest2641-21ssh00:25:373
297189965,1cyclictest2537-21diskstats20:25:163
297189965,1cyclictest24809-21ssh23:25:393
297189965,1cyclictest17923-21cat21:45:173
297189965,1cyclictest17711-21ssh23:15:303
297189965,1cyclictest10433-21ntpq22:20:183
297189964,1cyclictest35-21ksoftirqd/320:56:023
297189964,1cyclictest1449-21ntpd20:46:233
297189964,1cyclictest1449-21ntpd20:41:223
297189964,1cyclictest1449-21ntpd19:56:223
297189963,1cyclictest254-21systemd-journal20:36:443
297189963,1cyclictest254-21systemd-journal00:21:343
297189961,4cyclictest557-21nscd22:45:203
297189961,3cyclictest5669-21memory23:45:183
297189961,3cyclictest29399-21chrt22:00:293
297189961,3cyclictest23238-21diskmemload22:05:183
297189961,3cyclictest23238-21diskmemload21:20:013
297189960,6cyclictest7167-21ssh23:00:333
297189960,6cyclictest0-21swapper/300:00:063
297189960,1cyclictest23238-21diskmemload22:50:163
297179966,0cyclictest60950irq/123-eno122:52:462
297179966,0cyclictest60950irq/123-eno121:05:112
297179966,0cyclictest60950irq/123-eno120:55:122
297179966,0cyclictest60950irq/123-eno120:50:142
297179966,0cyclictest60950irq/123-eno120:07:122
297179966,0cyclictest60950irq/123-eno119:50:132
297179966,0cyclictest60950irq/123-eno100:06:102
297179965,1cyclictest28-21ksoftirqd/220:40:002
297179965,0cyclictest0-21swapper/222:00:162
297179964,1cyclictest28-21ksoftirqd/220:00:122
297179963,3cyclictest11668-21apt-get20:45:122
297179963,1cyclictest29294-21apt-get20:15:122
297179960,1cyclictest2320-21aten_r1power_po22:10:122
297099962,2cyclictest0-21swapper/100:02:091
297099961,4cyclictest0-21swapper/123:04:291
297099961,2cyclictest0-21swapper/119:40:121
297099961,2cyclictest0-21swapper/100:15:171
297099960,6cyclictest0-21swapper/122:52:281
297099960,6cyclictest0-21swapper/100:35:581
297099960,4cyclictest0-21swapper/122:46:271
297099960,4cyclictest0-21swapper/100:25:171
297099960,4cyclictest0-21swapper/100:20:141
297099960,3cyclictest0-21swapper/123:20:121
297099960,3cyclictest0-21swapper/122:34:361
297099960,3cyclictest0-21swapper/120:55:111
297099960,2cyclictest0-21swapper/123:10:121
297099960,0cyclictest0-21swapper/122:23:371
297089965,1cyclictest9-21ksoftirqd/020:45:120
297089965,1cyclictest17420-21ssh22:30:160
297089965,1cyclictest15874-21ssh23:15:110
297089965,1cyclictest11665-21/usr/sbin/munin23:55:140
297089964,1cyclictest13146-21fschecks_count19:40:140
297089964,1cyclictest11276-21http20:45:000
297089963,1cyclictest31883-21/usr/sbin/munin20:20:160
297089963,1cyclictest254-21systemd-journal20:03:580
297089960,5cyclictest27575-21ssh22:00:110
297089960,5cyclictest16379-21/usr/sbin/munin20:55:180
297089960,5cyclictest1599-21apt-get20:25:000
297089960,2cyclictest0-21swapper/019:55:110
297089960,2cyclictest0-21swapper/019:10:130
297089960,1cyclictest0-21swapper/019:20:140
297089960,0cyclictest0-21swapper/023:30:100
297089960,0cyclictest0-21swapper/023:01:040
297089960,0cyclictest0-21swapper/022:10:350
297189954,1cyclictest1449-21ntpd20:31:223
297189954,1cyclictest13477-21ssh21:40:113
297189953,1cyclictest30558-21cut19:10:163
297189953,1cyclictest17560-21tr19:50:163
297189953,1cyclictest1449-21ntpd20:51:223
297189953,1cyclictest1449-21ntpd20:11:233
297189953,1cyclictest1284-21grep19:15:153
297189953,1cyclictest10513-21cstates19:35:143
297189951,1cyclictest23238-21diskmemload22:35:053
297189950,5cyclictest0-21swapper/321:08:023
297189950,4cyclictest8642-21sed19:30:153
297189950,1cyclictest10391-21diskstats21:35:163
297179955,0cyclictest60950irq/123-eno119:30:122
297179953,1cyclictest28-21ksoftirqd/219:45:122
297179952,2cyclictest6829-21sendmail_mailqu19:25:192
297099955,0cyclictest0-21swapper/123:59:371
297099955,0cyclictest0-21swapper/123:30:441
297099955,0cyclictest0-21swapper/121:48:091
297099955,0cyclictest0-21swapper/121:33:361
297099955,0cyclictest0-21swapper/121:15:101
297099953,1cyclictest3746-21chrt22:10:211
297099953,1cyclictest0-21swapper/119:15:201
297099953,0cyclictest0-21swapper/120:50:241
297099952,2cyclictest13716-21chrt20:49:141
297099950,5cyclictest0-21swapper/123:50:191
297099950,5cyclictest0-21swapper/122:25:161
297099950,4cyclictest0-21swapper/123:38:191
297099950,4cyclictest0-21swapper/123:38:191
297099950,4cyclictest0-21swapper/122:41:391
297099950,4cyclictest0-21swapper/121:56:171
297099950,4cyclictest0-21swapper/121:40:461
297099950,3cyclictest0-21swapper/123:28:011
297099950,3cyclictest0-21swapper/121:38:481
297099950,2cyclictest0-21swapper/123:15:121
297099950,2cyclictest0-21swapper/121:21:521
297099950,2cyclictest0-21swapper/100:30:521
297099950,1cyclictest0-21swapper/121:50:151
297099950,1cyclictest0-21swapper/121:12:221
297099950,1cyclictest0-21swapper/119:45:111
297099950,1cyclictest0-21swapper/119:30:111
297099950,0cyclictest0-21swapper/121:26:381
297089953,1cyclictest24874-21tr20:05:140
297089950,5cyclictest0-21swapper/020:18:490
297089950,4cyclictest21116-21/usr/sbin/munin21:05:170
297089950,4cyclictest17291-21/usr/sbin/munin19:50:160
297089950,1cyclictest7842-21/usr/sbin/munin19:30:140
297089950,1cyclictest4379-21/usr/sbin/munin20:30:110
297099944,0cyclictest0-21swapper/120:14:121
297099942,1cyclictest0-21swapper/100:10:211
297099941,3cyclictest0-21swapper/122:00:211
297099940,3cyclictest0-21swapper/122:35:141
297099940,3cyclictest0-21swapper/119:10:161
297099940,3cyclictest0-21swapper/100:05:011
297099940,2cyclictest0-21swapper/123:40:021
297099940,2cyclictest0-21swapper/120:40:111
297099940,2cyclictest0-21swapper/120:00:201
297099940,2cyclictest0-21swapper/119:55:111
297099940,2cyclictest0-21swapper/119:20:181
297099940,1cyclictest0-21swapper/122:55:161
297099940,1cyclictest0-21swapper/121:00:171
297099940,1cyclictest0-21swapper/120:20:131
297099940,0cyclictest0-21swapper/123:45:311
297099940,0cyclictest0-21swapper/120:25:221
297099940,0cyclictest0-21swapper/119:25:201
297099932,1cyclictest0-21swapper/120:30:201
297099931,1cyclictest0-21swapper/121:05:131
297099931,1cyclictest0-21swapper/120:35:111
297099931,1cyclictest0-21swapper/120:05:131
297099930,2cyclictest0-21swapper/120:15:111
297099930,2cyclictest0-21swapper/119:35:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional