You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 02:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Fri Nov 07, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
213552194178,11sleep10-21swapper/119:07:271
212282187170,12sleep00-21swapper/019:05:510
212362180147,28sleep30-21swapper/319:05:583
215002174157,12sleep20-21swapper/219:09:222
273102460,0sleep00-21swapper/023:40:580
122612460,0sleep10-21swapper/121:00:011
141792430,0sleep20-21swapper/223:01:572
127962120,0sleep10-21swapper/123:01:041
2173499100,8cyclictest0-21swapper/322:20:133
15292100,0sleep20-21swapper/219:30:232
2350290,0sleep20-21swapper/223:12:182
217239990,4cyclictest14367-21munin-run22:30:000
217299970,5cyclictest22365-21chrt22:17:082
217299970,5cyclictest14262-21ssh22:13:182
217299970,4cyclictest0-21swapper/219:28:382
217299970,3cyclictest26024-21if_eth119:15:142
217299970,3cyclictest15863-21cat21:05:152
217299970,3cyclictest1422-21latency21:50:162
217299970,2cyclictest2885-21diskstats22:40:122
217239973,4cyclictest24791-21munin-run21:30:010
217299961,4cyclictest0-21swapper/223:25:262
217299960,4cyclictest711-21runrttasks22:25:382
217299960,4cyclictest711-21runrttasks20:07:202
217299960,4cyclictest30976-21ssh21:16:102
217299960,4cyclictest19728-21ssh00:26:112
217299960,4cyclictest18661-21ssh21:42:472
217299960,4cyclictest0-21swapper/223:45:512
217299960,3cyclictest8528-21cut19:45:152
217299960,3cyclictest30219-21grep20:30:142
217299960,3cyclictest25225-21grep20:20:142
217299960,3cyclictest23214-21expr21:45:162
217299960,3cyclictest20798-21ssh23:05:162
217299960,3cyclictest20402-21ls20:10:132
217299960,3cyclictest15434-21df_inode22:30:142
217299960,3cyclictest0-21swapper/222:50:192
217299960,3cyclictest0-21swapper/222:46:382
217299960,3cyclictest0-21swapper/219:59:172
217299960,3cyclictest0-21swapper/219:50:592
217299960,3cyclictest0-21swapper/219:35:302
217299960,3cyclictest0-21swapper/200:02:062
217299960,2cyclictest9477-21sshd20:51:342
217299960,2cyclictest71862sleep219:43:512
217299960,2cyclictest47842sleep220:44:402
217299960,2cyclictest3190-21ssh22:07:342
217299960,2cyclictest29540-21ssh00:31:082
217299960,2cyclictest27934-21ssh21:30:432
217299960,2cyclictest27442-21/usr/sbin/munin22:20:182
217299960,2cyclictest24483-21taskset20:15:312
217299960,2cyclictest237362sleep219:10:492
217299960,2cyclictest23048-21ssh22:00:532
217299960,2cyclictest17200-21diskmemload22:37:192
217299960,2cyclictest17200-21diskmemload21:29:342
217299960,2cyclictest15269-21ssh00:24:392
217299960,2cyclictest14999-21chrt00:07:502
217299960,2cyclictest143742sleep221:00:462
217299960,2cyclictest143742sleep221:00:452
217299960,2cyclictest118752sleep220:55:362
217299960,2cyclictest11720-21ssh00:39:092
217299960,2cyclictest0-21swapper/223:30:242
217299960,2cyclictest0-21swapper/221:36:062
217299960,2cyclictest0-21swapper/220:25:312
217299960,1cyclictest7934-21perl23:15:162
217299960,1cyclictest17662-21ssh21:10:112
217349955,0cyclictest3635-21kworker/3:223:32:003
217349955,0cyclictest3635-21kworker/3:223:09:363
217349955,0cyclictest3635-21kworker/3:200:16:433
217349955,0cyclictest0-21swapper/319:15:163
217349950,4cyclictest7728-21apt-get19:45:123
217349950,1cyclictest0-21swapper/322:47:083
217299950,4cyclictest18399-21cron00:10:012
217299950,4cyclictest18399-21cron00:10:012
217299950,3cyclictest11588-21ssh23:50:122
217299950,2cyclictest711-21runrttasks20:45:242
217299950,2cyclictest5007-21ssh21:20:042
217299950,2cyclictest32513-21taskset20:35:132
217299950,2cyclictest32035-21ssh22:55:102
217299950,2cyclictest15743-21fschecks_count20:00:142
217299950,2cyclictest0-21swapper/223:55:042
217299950,2cyclictest0-21swapper/223:40:032
217299950,2cyclictest0-21swapper/223:35:102
217299950,2cyclictest0-21swapper/223:20:062
217299950,2cyclictest0-21swapper/221:55:302
217299950,2cyclictest0-21swapper/219:20:022
217299950,2cyclictest0-21swapper/200:15:022
217259951,1cyclictest0-21swapper/121:55:111
217259950,5cyclictest0-21swapper/123:26:291
217259950,5cyclictest0-21swapper/121:44:031
217259950,5cyclictest0-21swapper/121:00:231
217259950,5cyclictest0-21swapper/121:00:221
217259950,5cyclictest0-21swapper/100:30:121
217259950,5cyclictest0-21swapper/100:24:591
217259950,0cyclictest0-21swapper/123:53:331
217239954,1cyclictest3232-21ssh22:07:350
217239950,5cyclictest0-21swapper/023:05:120
217239950,5cyclictest0-21swapper/022:12:060
217239950,5cyclictest0-21swapper/021:58:340
217239950,5cyclictest0-21swapper/021:48:060
217239950,5cyclictest0-21swapper/021:42:520
217239950,5cyclictest0-21swapper/021:32:230
217239950,4cyclictest20137-21apt-get20:10:120
217239950,0cyclictest0-21swapper/021:37:360
30954240,0sleep10-21swapper/123:43:191
28003240,0sleep30-21swapper/323:41:253
217349944,0cyclictest3635-21kworker/3:223:20:463
217349944,0cyclictest3635-21kworker/3:220:21:363
217349944,0cyclictest3635-21kworker/3:219:25:403
217349944,0cyclictest0-21swapper/322:10:103
217349944,0cyclictest0-21swapper/322:01:533
217349944,0cyclictest0-21swapper/300:21:513
217349943,1cyclictest55950irq/130-eth022:05:113
217349943,1cyclictest354-21cut22:55:143
217349942,2cyclictest10335-21ssh22:27:283
217349942,1cyclictest3085-21ssh00:34:573
217349942,1cyclictest21839-21/usr/sbin/munin19:10:213
217349941,3cyclictest29978-21sh21:15:313
217349941,2cyclictest0-21swapper/323:55:123
217349941,2cyclictest0-21swapper/323:25:123
217349940,4cyclictest12240-21ssh23:00:423
217349940,4cyclictest0-21swapper/323:53:223
217349940,4cyclictest0-21swapper/323:47:223
217349940,4cyclictest0-21swapper/322:15:133
217349940,4cyclictest0-21swapper/321:55:293
217349940,4cyclictest0-21swapper/321:50:133
217349940,4cyclictest0-21swapper/320:15:153
217349940,4cyclictest0-21swapper/320:00:123
217349940,4cyclictest0-21swapper/300:38:393
217349940,3cyclictest714-21snmpd23:15:083
217349940,3cyclictest5783-21df_inode21:20:153
217349940,3cyclictest32525-21irqstats00:00:203
217349940,3cyclictest29349-21grep19:20:233
217349940,3cyclictest27579-21apt-get20:25:123
217349940,3cyclictest27492-21sh21:30:263
217349940,3cyclictest25436-21df_inode22:35:173
217349940,3cyclictest2347-21munin-run20:40:013
217349940,3cyclictest22403-21ssh21:45:013
217349940,3cyclictest20343-21ntp_states00:10:193
217349940,3cyclictest20343-21ntp_states00:10:183
217349940,3cyclictest20141-21sh21:27:053
217349940,3cyclictest17531-21ssh21:10:053
217349940,3cyclictest16899-21/usr/sbin/munin00:25:133
217349940,3cyclictest15772-21/usr/sbin/munin23:35:193
217349940,3cyclictest15606-21grep22:30:143
217349940,3cyclictest15468-21df21:05:133
217349940,3cyclictest13730-21ssh21:40:153
217349940,3cyclictest0-21swapper/323:10:013
217349940,1cyclictest8332-21http00:05:013
217349940,1cyclictest22887-21cut22:50:153
217349940,1cyclictest10193-21/usr/sbin/munin19:50:163
217259944,0cyclictest0-21swapper/123:31:001
217259944,0cyclictest0-21swapper/123:21:591
217259944,0cyclictest0-21swapper/123:17:271
217259944,0cyclictest0-21swapper/122:32:571
217259944,0cyclictest0-21swapper/122:15:381
217259944,0cyclictest0-21swapper/100:35:271
217259944,0cyclictest0-21swapper/100:14:301
217259944,0cyclictest0-21swapper/100:14:291
217259944,0cyclictest0-21swapper/100:09:151
217259941,2cyclictest0-21swapper/122:25:121
217259941,2cyclictest0-21swapper/121:15:151
217259941,2cyclictest0-21swapper/120:05:161
217259940,4cyclictest0-21swapper/123:12:561
217259940,4cyclictest0-21swapper/122:36:441
217259940,2cyclictest0-21swapper/120:20:131
217259940,0cyclictest0-21swapper/121:53:481
217239944,0cyclictest0-21swapper/023:46:490
217239944,0cyclictest0-21swapper/021:16:400
217239944,0cyclictest0-21swapper/000:34:420
217239944,0cyclictest0-21swapper/000:29:270
217239943,1cyclictest4814-21cron20:45:010
217239943,1cyclictest24005-21wc20:15:210
217239942,1cyclictest9961-21latency_hist19:50:010
217239942,1cyclictest17671-21ssh23:20:140
217239941,3cyclictest7601-21if_eth400:20:190
217239941,3cyclictest714-21snmpd21:00:210
217239941,3cyclictest714-21snmpd21:00:210
217239941,2cyclictest0-21swapper/021:05:120
217239941,2cyclictest0-21swapper/019:35:130
217239940,4cyclictest21003-21users00:10:220
217239940,4cyclictest21003-21users00:10:210
217239940,4cyclictest0-21swapper/023:11:140
217239940,4cyclictest0-21swapper/023:00:110
217239940,4cyclictest0-21swapper/022:45:140
217239940,4cyclictest0-21swapper/022:40:120
217239940,4cyclictest0-21swapper/022:37:520
217239940,4cyclictest0-21swapper/022:00:130
217239940,4cyclictest0-21swapper/021:50:460
217239940,4cyclictest0-21swapper/021:10:120
217239940,4cyclictest0-21swapper/000:00:160
217239940,3cyclictest7298-21munin-run19:45:010
217239940,3cyclictest5963-21pluginstate23:30:190
217239940,3cyclictest24307-21munin-node19:15:010
217239940,3cyclictest18962-21/usr/sbin/munin20:05:220
217239940,3cyclictest17200-21diskmemload23:25:000
217239940,3cyclictest17200-21diskmemload00:35:130
217239940,3cyclictest14395-21/usr/sbin/munin23:35:170
217239940,3cyclictest11609-21/usr/sbin/munin23:50:190
217239940,3cyclictest0-21swapper/019:55:000
217239940,1cyclictest14928-21basename20:00:000
217239940,0cyclictest0-21swapper/022:16:370
217239940,0cyclictest0-21swapper/021:21:540
217349932,1cyclictest55950irq/130-eth022:40:123
217349931,2cyclictest55950irq/130-eth021:35:003
217349931,2cyclictest0-21swapper/320:55:133
217349931,1cyclictest7531-21apt-get20:50:143
217349931,1cyclictest0-21swapper/320:45:123
217349931,1cyclictest0-21swapper/320:10:123
217349931,1cyclictest0-21swapper/319:55:133
217349930,3cyclictest32720-21grep19:30:153
217349930,3cyclictest30786-21tune2fs20:30:173
217349930,3cyclictest12487-21apt-get21:00:123
217349930,3cyclictest12487-21apt-get21:00:123
217349930,3cyclictest0-21swapper/320:40:153
217349930,3cyclictest0-21swapper/320:05:163
217349930,3cyclictest0-21swapper/319:40:013
217349930,3cyclictest0-21swapper/319:35:053
217259933,0cyclictest0-21swapper/123:58:471
217259933,0cyclictest0-21swapper/123:35:301
217259933,0cyclictest0-21swapper/122:48:471
217259933,0cyclictest0-21swapper/100:04:021
217259931,1cyclictest0-21swapper/123:45:111
217259931,1cyclictest0-21swapper/123:05:121
217259931,1cyclictest0-21swapper/122:40:121
217259931,1cyclictest0-21swapper/122:10:131
217259931,1cyclictest0-21swapper/122:05:121
217259931,1cyclictest0-21swapper/121:45:161
217259931,1cyclictest0-21swapper/121:30:121
217259931,1cyclictest0-21swapper/121:25:021
217259931,1cyclictest0-21swapper/121:05:121
217259931,1cyclictest0-21swapper/120:30:151
217259931,1cyclictest0-21swapper/120:25:131
217259931,1cyclictest0-21swapper/119:35:011
217259931,1cyclictest0-21swapper/100:15:161
217259930,3cyclictest0-21swapper/120:48:101
217259930,3cyclictest0-21swapper/120:01:361
217259930,3cyclictest0-21swapper/119:25:051
217259930,2cyclictest2268-21chrt20:39:221
217259930,2cyclictest0-21swapper/121:20:131
217259930,2cyclictest0-21swapper/120:50:151
217259930,2cyclictest0-21swapper/119:10:121
217259930,2cyclictest0-21swapper/100:25:111
217239933,0cyclictest25110-21apt-get20:20:120
217239932,1cyclictest714-21snmpd19:10:090
217239932,1cyclictest31867-21ssh22:55:020
217239931,2cyclictest17200-21diskmemload00:15:030
217239931,1cyclictest0-21swapper/019:20:130
217239930,3cyclictest7729-21df23:15:150
217239930,3cyclictest21201-21ssh23:55:030
217239930,3cyclictest0-21swapper/022:50:110
217239930,3cyclictest0-21swapper/022:30:110
217239930,3cyclictest0-21swapper/022:20:130
217239930,3cyclictest0-21swapper/020:55:130
217239930,3cyclictest0-21swapper/020:50:140
217239930,3cyclictest0-21swapper/020:45:110
217239930,3cyclictest0-21swapper/020:35:140
217239930,3cyclictest0-21swapper/020:30:140
217239930,3cyclictest0-21swapper/020:25:130
217239930,3cyclictest0-21swapper/020:00:010
217239930,3cyclictest0-21swapper/019:30:150
217239930,3cyclictest0-21swapper/019:25:140
217239930,3cyclictest0-21swapper/000:05:010
217259922,0cyclictest0-21swapper/122:20:071
217259921,1cyclictest147862chrt19:58:441
217259921,1cyclictest0-21swapper/119:20:121
217259920,2cyclictest0-21swapper/119:50:131
217259920,1cyclictest25067-21chrt19:15:031
217259920,1cyclictest17632-21chrt21:10:101
217259920,1cyclictest0-21swapper/122:55:131
217259920,1cyclictest0-21swapper/122:50:121
217259920,1cyclictest0-21swapper/122:00:141
217259920,1cyclictest0-21swapper/121:35:111
217259920,1cyclictest0-21swapper/120:40:141
217259920,1cyclictest0-21swapper/120:15:151
217259920,1cyclictest0-21swapper/120:10:111
217259920,1cyclictest0-21swapper/119:45:111
217259920,1cyclictest0-21swapper/119:30:151
217259910,1cyclictest0-21swapper/119:40:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional