You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 08:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Fri May 09, 2025 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
255402207192,10sleep10-21swapper/119:07:351
256382186169,12sleep20-21swapper/219:08:542
254132186171,11sleep30-21swapper/319:05:583
255392180162,13sleep00-21swapper/019:07:340
47162500,0sleep30-21swapper/321:50:083
294792460,0sleep00-21swapper/023:24:280
131942460,0sleep00-21swapper/000:37:380
315042450,0sleep20-21swapper/221:30:212
185942420,0sleep10-21swapper/123:18:561
247092120,0sleep10-21swapper/121:43:481
53902110,1sleep22592799cyclictest23:11:502
277322110,0sleep30-21swapper/321:29:053
259199982,3cyclictest23053-21awk23:05:010
259199980,4cyclictest19283-21basename21:25:020
259199960,3cyclictest10394-21munin-run23:15:010
259329955,0cyclictest28483-21kworker/3:221:31:343
259329955,0cyclictest0-21swapper/300:18:493
259329950,5cyclictest29805-21apt-get00:30:023
259329950,5cyclictest0-21swapper/321:44:103
259329950,5cyclictest0-21swapper/321:19:093
259329950,4cyclictest12064-21ssh22:10:133
259329950,0cyclictest15331-21ssh23:16:503
259329950,0cyclictest0-21swapper/319:36:163
259279950,4cyclictest21975-21apt-get22:15:142
259279950,4cyclictest19762-21sh23:19:422
259279950,2cyclictest6080-21sh22:40:002
259239951,2cyclictest0-21swapper/123:10:121
259239951,2cyclictest0-21swapper/121:30:141
259239950,5cyclictest0-21swapper/123:31:511
259239950,5cyclictest0-21swapper/123:27:471
259239950,5cyclictest0-21swapper/122:22:351
259199950,5cyclictest0-21swapper/022:50:250
259199950,5cyclictest0-21swapper/020:33:120
259199950,4cyclictest0-21swapper/021:45:130
23294250,0sleep30-21swapper/323:05:033
259329944,0cyclictest28483-21kworker/3:223:57:083
259329944,0cyclictest28483-21kworker/3:223:12:223
259329944,0cyclictest0-21swapper/323:32:523
259329944,0cyclictest0-21swapper/322:39:193
259329944,0cyclictest0-21swapper/321:46:303
259329944,0cyclictest0-21swapper/320:20:213
259329942,1cyclictest16478-21ssh23:50:143
259329941,3cyclictest30997-21snmpd20:05:183
259329941,2cyclictest31936-21/usr/sbin/munin19:20:153
259329940,4cyclictest23136-21grep00:10:143
259329940,4cyclictest21644-21diskmemload23:04:113
259329940,4cyclictest1618-21cron22:05:003
259329940,4cyclictest10468-21cat00:20:123
259329940,4cyclictest0-21swapper/323:45:123
259329940,4cyclictest0-21swapper/323:35:063
259329940,4cyclictest0-21swapper/323:20:043
259329940,4cyclictest0-21swapper/322:51:233
259329940,4cyclictest0-21swapper/322:20:103
259329940,4cyclictest0-21swapper/322:15:143
259329940,4cyclictest0-21swapper/321:58:583
259329940,4cyclictest0-21swapper/321:10:143
259329940,4cyclictest0-21swapper/321:10:133
259329940,4cyclictest0-21swapper/320:13:563
259329940,4cyclictest0-21swapper/319:45:123
259329940,4cyclictest0-21swapper/300:37:553
259329940,4cyclictest0-21swapper/300:06:403
259329940,3cyclictest9866-21sh22:41:063
259329940,3cyclictest9222-21cron21:20:023
259329940,3cyclictest32508-21/usr/sbin/munin23:25:223
259329940,3cyclictest3014-21apt00:00:153
259329940,3cyclictest28245-21latency_hist19:15:013
259329940,3cyclictest21097-21irqstats00:25:173
259329940,3cyclictest20678-21ntp_states21:05:183
259329940,3cyclictest19669-21cat20:00:133
259329940,3cyclictest19221-21/usr/sbin/munin22:30:213
259329940,3cyclictest17462-21df21:00:153
259329940,3cyclictest17425-21hddtemp_smartct22:45:163
259329940,3cyclictest14456-21/usr/sbin/munin20:55:153
259329940,3cyclictest13828-21users20:50:193
259329940,3cyclictest12230-21ssh21:37:123
259329940,3cyclictest0-21swapper/322:55:123
259329940,1cyclictest9796-21cat22:25:143
259329940,0cyclictest0-21swapper/323:40:083
259279944,0cyclictest0-21swapper/223:36:202
259279944,0cyclictest0-21swapper/222:28:332
259279943,1cyclictest21111-21/usr/sbin/munin23:20:212
259279942,2cyclictest301ktimersoftd/221:10:152
259279942,2cyclictest301ktimersoftd/221:10:152
259279941,3cyclictest57250irq/125-eth021:55:562
259279941,1cyclictest0-21swapper/222:45:132
259279940,4cyclictest7216-21ssh22:56:122
259279940,4cyclictest26988-21diskstats22:50:152
259279940,4cyclictest14892-21ssh00:22:022
259279940,4cyclictest12574-21ssh22:42:502
259279940,4cyclictest0-21swapper/223:55:122
259279940,4cyclictest0-21swapper/223:51:002
259279940,4cyclictest0-21swapper/223:25:152
259279940,4cyclictest0-21swapper/223:01:352
259279940,4cyclictest0-21swapper/222:05:132
259279940,4cyclictest0-21swapper/221:50:162
259279940,4cyclictest0-21swapper/221:45:542
259279940,4cyclictest0-21swapper/200:35:122
259279940,4cyclictest0-21swapper/200:10:142
259279940,4cyclictest0-21swapper/200:06:072
259279940,4cyclictest0-21swapper/200:00:172
259279940,3cyclictest6107-21/usr/sbin/munin19:30:202
259279940,3cyclictest2629-21ssh00:15:312
259279940,3cyclictest25294-21sh21:27:342
259279940,3cyclictest19357-21ls21:05:022
259279940,3cyclictest1216-21runrttasks00:31:192
259279940,3cyclictest10586-21sh23:47:112
259279940,3cyclictest0-21swapper/219:40:112
259279940,3cyclictest0-21swapper/200:25:122
259279940,2cyclictest0-21swapper/222:12:052
259279940,1cyclictest9802-21/usr/sbin/munin21:20:162
259279940,1cyclictest8920-21df23:30:152
259279940,1cyclictest31833-21/usr/sbin/munin22:20:152
259279940,1cyclictest21961-21/usr/sbin/munin20:05:202
259279940,1cyclictest19221-21/usr/sbin/munin22:30:172
259279940,1cyclictest17195-21/usr/sbin/munin21:40:202
259279940,0cyclictest0-21swapper/222:01:282
259239944,0cyclictest0-21swapper/123:03:031
259239944,0cyclictest0-21swapper/100:20:271
259239944,0cyclictest0-21swapper/100:03:511
259239941,2cyclictest0-21swapper/123:55:121
259239941,2cyclictest0-21swapper/123:40:121
259239941,2cyclictest0-21swapper/122:15:131
259239941,2cyclictest0-21swapper/122:10:121
259239941,2cyclictest0-21swapper/122:00:151
259239941,2cyclictest0-21swapper/121:35:131
259239941,2cyclictest0-21swapper/121:00:121
259239941,2cyclictest0-21swapper/120:50:101
259239941,2cyclictest0-21swapper/119:50:111
259239941,2cyclictest0-21swapper/119:45:131
259239941,2cyclictest0-21swapper/119:35:121
259239941,1cyclictest0-21swapper/121:55:111
259239940,4cyclictest0-21swapper/123:53:321
259239940,4cyclictest0-21swapper/123:23:421
259239940,4cyclictest0-21swapper/123:06:141
259239940,4cyclictest0-21swapper/122:58:151
259239940,4cyclictest0-21swapper/122:47:451
259239940,4cyclictest0-21swapper/122:43:341
259239940,4cyclictest0-21swapper/122:29:521
259239940,2cyclictest0-21swapper/122:50:131
259199944,0cyclictest9015-21ssh00:19:360
259199944,0cyclictest28267-21perf19:15:000
259199944,0cyclictest0-21swapper/023:19:040
259199944,0cyclictest0-21swapper/021:56:170
259199944,0cyclictest0-21swapper/020:10:210
259199944,0cyclictest0-21swapper/019:45:020
259199944,0cyclictest0-21swapper/000:21:570
259199942,1cyclictest30997-21snmpd22:19:400
259199942,1cyclictest30997-21snmpd21:37:130
259199942,1cyclictest29756-21ssh23:08:000
259199941,3cyclictest32749-21ssh21:30:580
259199941,3cyclictest30997-21snmpd21:26:120
259199941,2cyclictest0-21swapper/022:00:140
259199941,2cyclictest0-21swapper/021:40:130
259199941,2cyclictest0-21swapper/019:20:080
259199940,4cyclictest61-21kswapd023:50:110
259199940,4cyclictest11683-21ssh22:25:310
259199940,4cyclictest0-21swapper/022:40:130
259199940,4cyclictest0-21swapper/020:05:120
259199940,4cyclictest0-21swapper/000:30:130
259199940,4cyclictest0-21swapper/000:05:110
259199940,3cyclictest6259-21diskstats23:45:140
259199940,3cyclictest585-21ssh22:36:320
259199940,3cyclictest4921-21ssh21:50:150
259199940,3cyclictest4529-21ssh21:17:060
259199940,3cyclictest3779-21perl00:00:170
259199940,3cyclictest32212-21latency_hist00:15:000
259199940,3cyclictest31844-21ssh22:20:100
259199940,3cyclictest3075-21memory19:25:170
259199940,3cyclictest28979-21munin-node19:15:000
259199940,3cyclictest28919-21fschecks_count23:40:140
259199940,3cyclictest26238-21/usr/sbin/munin23:55:160
259199940,3cyclictest21933-21unixbench-2d00:25:220
259199940,3cyclictest21893-21ssh21:10:030
259199940,3cyclictest21893-21ssh21:10:020
259199940,3cyclictest17659-21apt-get23:35:020
259199940,3cyclictest16532-21/usr/sbin/munin22:45:160
259199940,3cyclictest14221-21latency_hist20:55:010
259199940,3cyclictest11706-21sed22:10:010
259199940,3cyclictest0-21swapper/020:20:100
259199940,0cyclictest11551-21apt-get20:50:010
23780240,0sleep00-21swapper/022:31:570
259329931,1cyclictest0-21swapper/319:25:123
259329930,3cyclictest4887-21diskstats20:35:113
259329930,3cyclictest29081-21hddtemp_smartct19:15:003
259329930,3cyclictest1976-21sh22:05:063
259329930,3cyclictest0-21swapper/320:45:053
259329930,3cyclictest0-21swapper/320:40:053
259329930,3cyclictest0-21swapper/320:30:133
259329930,3cyclictest0-21swapper/320:25:053
259329930,3cyclictest0-21swapper/320:15:003
259329930,3cyclictest0-21swapper/319:55:123
259329930,3cyclictest0-21swapper/319:50:023
259329930,3cyclictest0-21swapper/319:40:113
259329930,3cyclictest0-21swapper/319:30:093
259279931,1cyclictest0-21swapper/220:40:112
259279931,1cyclictest0-21swapper/220:20:092
259279931,1cyclictest0-21swapper/219:15:132
259279930,3cyclictest6990-21apt-config19:35:002
259279930,3cyclictest26898-21apt-get20:15:112
259279930,3cyclictest2509-21df_inode19:25:142
259279930,3cyclictest1900-21apt-get20:30:132
259279930,3cyclictest17010-21apt-get19:55:112
259279930,3cyclictest14976-21df20:55:152
259279930,3cyclictest14757-21expr19:50:112
259279930,3cyclictest0-21swapper/223:40:012
259279930,3cyclictest0-21swapper/223:05:132
259279930,3cyclictest0-21swapper/221:15:112
259279930,3cyclictest0-21swapper/221:05:132
259279930,3cyclictest0-21swapper/220:50:002
259279930,3cyclictest0-21swapper/220:45:132
259279930,3cyclictest0-21swapper/220:35:102
259279930,3cyclictest0-21swapper/220:25:122
259279930,3cyclictest0-21swapper/220:10:132
259279930,3cyclictest0-21swapper/220:00:102
259279930,3cyclictest0-21swapper/219:45:112
259279930,3cyclictest0-21swapper/219:20:012
259279930,3cyclictest0-21swapper/219:10:082
259279930,2cyclictest7151-21sh21:35:092
259239933,0cyclictest211ktimersoftd/120:37:301
259239933,0cyclictest211ktimersoftd/120:25:101
259239933,0cyclictest211ktimersoftd/120:22:471
259239933,0cyclictest0-21swapper/122:05:081
259239931,1cyclictest0-21swapper/123:45:131
259239931,1cyclictest0-21swapper/123:35:121
259239931,1cyclictest0-21swapper/122:35:131
259239931,1cyclictest0-21swapper/122:30:121
259239931,1cyclictest0-21swapper/121:50:011
259239931,1cyclictest0-21swapper/121:45:121
259239931,1cyclictest0-21swapper/121:25:181
259239931,1cyclictest0-21swapper/121:20:141
259239931,1cyclictest0-21swapper/120:45:141
259239931,1cyclictest0-21swapper/120:30:121
259239931,1cyclictest0-21swapper/120:15:121
259239931,1cyclictest0-21swapper/120:10:131
259239931,1cyclictest0-21swapper/120:00:101
259239931,1cyclictest0-21swapper/119:55:121
259239931,1cyclictest0-21swapper/119:40:101
259239931,1cyclictest0-21swapper/119:10:111
259239931,1cyclictest0-21swapper/100:35:111
259239931,1cyclictest0-21swapper/100:30:111
259239931,1cyclictest0-21swapper/100:25:121
259239931,1cyclictest0-21swapper/100:15:151
259239931,1cyclictest0-21swapper/100:10:141
259239931,1cyclictest0-21swapper/100:05:011
259239930,3cyclictest0-21swapper/120:44:221
259239930,3cyclictest0-21swapper/120:07:061
259239930,2cyclictest0-21swapper/121:05:131
259199932,1cyclictest8067-21ssh23:30:040
259199932,1cyclictest30997-21snmpd20:55:120
259199932,1cyclictest30997-21snmpd20:15:080
259199930,3cyclictest9820-21expr19:40:110
259199930,3cyclictest3632-21ssh22:55:040
259199930,3cyclictest32093-21tr20:25:140
259199930,3cyclictest30665-21ssh23:25:050
259199930,3cyclictest0-21swapper/022:10:080
259199930,3cyclictest0-21swapper/021:05:120
259199930,3cyclictest0-21swapper/021:00:120
259199930,3cyclictest0-21swapper/020:40:100
259199930,3cyclictest0-21swapper/020:35:090
259199930,3cyclictest0-21swapper/020:00:030
259199930,3cyclictest0-21swapper/019:55:110
259199930,3cyclictest0-21swapper/019:50:100
259199930,3cyclictest0-21swapper/019:30:130
259199930,2cyclictest7118-21/usr/sbin/munin19:35:180
259239922,0cyclictest211ktimersoftd/119:25:051
259239921,1cyclictest16347-21chrt20:55:471
259239920,1cyclictest31950-21chrt21:15:061
259239920,1cyclictest0-21swapper/121:10:141
259239920,1cyclictest0-21swapper/121:10:141
259239920,1cyclictest0-21swapper/119:30:131
259239920,1cyclictest0-21swapper/119:20:021
259239920,1cyclictest0-21swapper/119:15:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional