You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-17 - 05:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Fri Jan 17, 2025 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162832188170,12sleep10-21swapper/119:09:561
161782184168,12sleep20-21swapper/219:08:372
159402181164,12sleep00-21swapper/019:05:310
159382181165,11sleep30-21swapper/319:05:293
5682460,0sleep20-21swapper/221:20:202
131022450,0sleep30-21swapper/321:44:113
8752430,0sleep10-21swapper/123:19:251
273762430,0sleep20-21swapper/223:50:132
235052430,0sleep30-21swapper/323:30:563
203212430,0sleep10-21swapper/122:21:041
107772430,0sleep10-21swapper/123:58:321
78732420,0sleep00-21swapper/000:30:510
25222120,0sleep10-21swapper/120:55:001
128582100,0sleep30-21swapper/323:42:053
24032290,0sleep30-21swapper/321:16:083
164489992,3cyclictest57250irq/125-eth021:40:010
164489990,3cyclictest32196-21memory21:20:160
13869290,0sleep30-21swapper/322:35:113
13085290,0sleep20-21swapper/221:10:402
164489981,1cyclictest0-21swapper/019:20:090
164489981,1cyclictest0-21swapper/019:20:090
164489970,3cyclictest31752-21awk20:45:000
164489970,3cyclictest27525-21df20:35:130
164489962,1cyclictest0-21swapper/020:54:550
164489960,5cyclictest0-21swapper/023:10:090
2319250,0sleep10-21swapper/122:46:061
164629955,0cyclictest19005-21kworker/3:222:51:243
164629955,0cyclictest19005-21kworker/3:222:17:473
164629955,0cyclictest19005-21kworker/3:219:52:153
164629955,0cyclictest19005-21kworker/3:200:32:083
164629950,5cyclictest0-21swapper/323:54:103
164629950,5cyclictest0-21swapper/322:56:163
164629950,5cyclictest0-21swapper/322:20:113
164629950,5cyclictest0-21swapper/300:12:233
164629950,5cyclictest0-21swapper/300:01:183
164629950,4cyclictest0-21swapper/322:10:073
164629950,4cyclictest0-21swapper/321:35:103
164629950,4cyclictest0-21swapper/321:20:023
164629950,4cyclictest0-21swapper/320:55:003
164629950,4cyclictest0-21swapper/319:20:103
164629950,4cyclictest0-21swapper/319:20:103
164589950,5cyclictest0-21swapper/222:16:412
164589950,5cyclictest0-21swapper/200:33:542
164589950,4cyclictest17041-21ssh21:30:082
164589950,0cyclictest0-21swapper/223:29:542
164589950,0cyclictest0-21swapper/221:49:462
164539951,2cyclictest0-21swapper/123:50:071
164539951,2cyclictest0-21swapper/123:20:101
164539950,5cyclictest0-21swapper/120:15:171
164489950,4cyclictest20187-21hddtemp_smartct00:20:130
164489950,4cyclictest13032-21ssh22:35:010
19135240,0sleep00-21swapper/021:47:490
164629944,0cyclictest19005-21kworker/3:222:40:113
164629944,0cyclictest19005-21kworker/3:221:55:273
164629944,0cyclictest0-21swapper/323:21:173
164629944,0cyclictest0-21swapper/322:25:353
164629942,1cyclictest30997-21snmpd19:19:273
164629940,4cyclictest20321-21grep20:20:143
164629940,4cyclictest16867-21apt-get21:30:033
164629940,4cyclictest11410-21perl21:10:183
164629940,4cyclictest10041-21diskmemload22:30:023
164629940,4cyclictest0-21swapper/323:45:143
164629940,4cyclictest0-21swapper/323:10:083
164629940,4cyclictest0-21swapper/323:00:113
164629940,4cyclictest0-21swapper/322:48:273
164629940,4cyclictest0-21swapper/321:51:053
164629940,4cyclictest0-21swapper/321:46:343
164629940,4cyclictest0-21swapper/320:30:143
164629940,4cyclictest0-21swapper/319:10:103
164629940,4cyclictest0-21swapper/300:25:403
164629940,4cyclictest0-21swapper/300:16:013
164629940,3cyclictest9703-21ssh21:25:493
164629940,3cyclictest4498-21hddtemp_smartct23:55:123
164629940,3cyclictest30997-21snmpd19:35:033
164629940,3cyclictest30451-21ssh23:35:103
164629940,3cyclictest29496-21/usr/sbin/munin20:40:113
164629940,3cyclictest27248-21/usr/sbin/munin19:30:253
164629940,3cyclictest23397-21ssh22:06:403
164629940,3cyclictest20784-21/usr/sbin/munin00:20:183
164629940,3cyclictest18144-21ssh23:28:513
164629940,3cyclictest14730-21ssh22:01:253
164629940,3cyclictest11850-21ssh23:08:293
164629940,0cyclictest0-21swapper/300:06:503
164589944,0cyclictest0-21swapper/223:56:392
164589944,0cyclictest0-21swapper/223:14:532
164589944,0cyclictest0-21swapper/223:07:022
164589944,0cyclictest0-21swapper/222:22:042
164589943,1cyclictest25335-21grep20:30:172
164589941,3cyclictest30997-21snmpd21:50:192
164589941,3cyclictest10041-21diskmemload23:22:522
164589941,2cyclictest0-21swapper/221:05:112
164589941,1cyclictest0-21swapper/222:40:132
164589941,1cyclictest0-21swapper/220:55:112
164589940,4cyclictest61-21kswapd022:25:022
164589940,4cyclictest3542-21sh21:39:302
164589940,4cyclictest1148-21irqstats20:50:132
164589940,4cyclictest10041-21diskmemload22:35:242
164589940,4cyclictest10041-21diskmemload22:08:112
164589940,4cyclictest0-21swapper/223:48:052
164589940,4cyclictest0-21swapper/223:36:102
164589940,4cyclictest0-21swapper/223:01:172
164589940,4cyclictest0-21swapper/222:48:102
164589940,4cyclictest0-21swapper/222:30:092
164589940,4cyclictest0-21swapper/222:00:112
164589940,4cyclictest0-21swapper/221:55:062
164589940,4cyclictest0-21swapper/221:17:512
164589940,4cyclictest0-21swapper/221:00:102
164589940,4cyclictest0-21swapper/219:25:132
164589940,4cyclictest0-21swapper/200:15:112
164589940,3cyclictest32172-21latency_hist19:40:012
164589940,3cyclictest31873-21date20:45:002
164589940,3cyclictest31385-21vmstat22:10:162
164589940,3cyclictest2183-21cat19:45:012
164589940,3cyclictest21701-21ssh23:30:162
164589940,3cyclictest20434-21hddtemp_smartct20:20:162
164589940,3cyclictest19579-21df19:15:012
164589940,3cyclictest19559-21df_inode22:55:122
164589940,3cyclictest19035-21ssh00:20:012
164589940,3cyclictest0-21swapper/220:35:102
164589940,0cyclictest15873-21ssh23:43:592
164539944,0cyclictest0-21swapper/122:40:061
164539944,0cyclictest0-21swapper/121:54:421
164539944,0cyclictest0-21swapper/120:42:391
164539944,0cyclictest0-21swapper/120:07:091
164539944,0cyclictest0-21swapper/100:26:451
164539941,2cyclictest0-21swapper/122:30:021
164539941,2cyclictest0-21swapper/121:20:121
164539941,2cyclictest0-21swapper/121:10:151
164539941,2cyclictest0-21swapper/120:30:141
164539941,2cyclictest0-21swapper/119:55:121
164539941,2cyclictest0-21swapper/100:35:011
164539941,2cyclictest0-21swapper/100:15:101
164539941,2cyclictest0-21swapper/100:00:131
164539941,1cyclictest0-21swapper/119:50:171
164539940,4cyclictest0-21swapper/123:28:231
164539940,4cyclictest0-21swapper/122:13:541
164539940,4cyclictest0-21swapper/121:34:191
164539940,2cyclictest0-21swapper/119:25:121
164539940,0cyclictest0-21swapper/121:05:221
164539940,0cyclictest0-21swapper/119:15:121
164489944,0cyclictest0-21swapper/023:59:240
164489944,0cyclictest0-21swapper/021:31:530
164489943,1cyclictest16021-21wc20:10:210
164489941,2cyclictest0-21swapper/021:10:150
164489940,4cyclictest0-21swapper/023:40:120
164489940,4cyclictest0-21swapper/023:30:110
164489940,4cyclictest0-21swapper/023:20:100
164489940,4cyclictest0-21swapper/022:50:060
164489940,4cyclictest0-21swapper/022:45:570
164489940,4cyclictest0-21swapper/021:50:280
164489940,4cyclictest0-21swapper/021:42:410
164489940,4cyclictest0-21swapper/020:30:140
164489940,4cyclictest0-21swapper/019:30:150
164489940,4cyclictest0-21swapper/000:15:100
164489940,3cyclictest8321-21df21:05:130
164489940,3cyclictest565-21sed22:45:010
164489940,3cyclictest5469-21perl19:50:190
164489940,3cyclictest29870-21cpu19:35:120
164489940,3cyclictest2949-21sh23:37:080
164489940,3cyclictest28652-21ssh23:50:380
164489940,3cyclictest23373-21ssh00:05:040
164489940,3cyclictest22593-21df_inode20:25:120
164489940,3cyclictest20876-21sh22:05:060
164489940,3cyclictest19602-21apt-get20:20:120
164489940,3cyclictest17562-21expr20:15:110
164489940,3cyclictest15387-21python00:00:200
164489940,3cyclictest14676-21gpgv00:35:010
164489940,3cyclictest12027-21sh22:34:320
164489940,3cyclictest1200-21cat21:55:140
164489940,3cyclictest11815-21cron23:25:000
164489940,3cyclictest10404-21perl20:00:130
164489940,3cyclictest0-21swapper/019:25:120
164489940,3cyclictest0-21swapper/019:15:020
164489940,1cyclictest32356-21latency_hist20:50:000
164489940,1cyclictest29108-21dpkg22:10:020
164489940,1cyclictest27020-21cat22:25:010
164489940,1cyclictest18825-21munin-run19:15:020
164489940,1cyclictest17048-21/usr/sbin/munin22:20:180
164629933,0cyclictest40-21ksoftirqd/319:25:143
164629932,1cyclictest30997-21snmpd21:00:063
164629932,1cyclictest30997-21snmpd20:45:123
164629930,3cyclictest1785-21sed20:50:173
164629930,3cyclictest0-21swapper/321:05:123
164629930,3cyclictest0-21swapper/320:35:113
164629930,3cyclictest0-21swapper/320:25:113
164629930,3cyclictest0-21swapper/320:15:113
164629930,3cyclictest0-21swapper/320:10:153
164629930,3cyclictest0-21swapper/320:05:103
164629930,3cyclictest0-21swapper/320:00:103
164629930,3cyclictest0-21swapper/319:55:123
164629930,3cyclictest0-21swapper/319:45:063
164629930,3cyclictest0-21swapper/319:40:113
164629930,3cyclictest0-21swapper/300:35:013
164629930,0cyclictest24752-21apt-get23:15:113
164589932,1cyclictest4941-21tr19:50:192
164589930,3cyclictest9041-21grep22:50:082
164589930,3cyclictest622-21ntp_states00:10:132
164589930,3cyclictest4618-21ssh21:40:022
164589930,3cyclictest27990-21perl19:30:182
164589930,3cyclictest23504-21sshd00:05:072
164589930,3cyclictest14632-21apt-get20:10:142
164589930,3cyclictest0-21swapper/223:15:102
164589930,3cyclictest0-21swapper/221:25:022
164589930,3cyclictest0-21swapper/220:45:322
164589930,3cyclictest0-21swapper/220:25:112
164589930,3cyclictest0-21swapper/220:15:102
164589930,3cyclictest0-21swapper/220:05:102
164589930,3cyclictest0-21swapper/220:00:112
164589930,3cyclictest0-21swapper/219:55:052
164589930,3cyclictest0-21swapper/219:45:122
164589930,3cyclictest0-21swapper/219:20:092
164589930,3cyclictest0-21swapper/219:20:092
164589930,3cyclictest0-21swapper/219:10:102
164589930,3cyclictest0-21swapper/200:35:012
164589930,3cyclictest0-21swapper/200:25:062
164589930,3cyclictest0-21swapper/200:00:022
164539933,0cyclictest0-21swapper/122:06:111
164539933,0cyclictest0-21swapper/121:28:221
164539933,0cyclictest0-21swapper/120:56:291
164539933,0cyclictest0-21swapper/119:13:361
164539933,0cyclictest0-21swapper/100:10:091
164539931,2cyclictest0-21swapper/123:30:121
164539931,1cyclictest0-21swapper/123:40:101
164539931,1cyclictest0-21swapper/123:00:111
164539931,1cyclictest0-21swapper/122:50:071
164539931,1cyclictest0-21swapper/122:35:091
164539931,1cyclictest0-21swapper/122:15:111
164539931,1cyclictest0-21swapper/121:35:091
164539931,1cyclictest0-21swapper/121:15:101
164539931,1cyclictest0-21swapper/121:00:101
164539931,1cyclictest0-21swapper/120:35:101
164539931,1cyclictest0-21swapper/120:25:121
164539931,1cyclictest0-21swapper/119:30:171
164539931,1cyclictest0-21swapper/119:20:091
164539931,1cyclictest0-21swapper/119:20:091
164539931,1cyclictest0-21swapper/100:30:071
164539931,1cyclictest0-21swapper/100:20:111
164539930,2cyclictest7251-21chrt21:58:021
164539930,2cyclictest4474-21chrt23:38:071
164539930,2cyclictest31968-21chrt20:45:111
164539930,2cyclictest0-21swapper/120:10:141
164539930,2cyclictest0-21swapper/119:40:101
164489933,0cyclictest0-21swapper/022:55:050
164489932,1cyclictest30997-21snmpd23:15:110
164489932,1cyclictest2553-21tr19:45:140
164489931,1cyclictest30997-21snmpd19:55:390
164489930,3cyclictest29108-21ssh00:25:030
164489930,3cyclictest28923-21sh23:00:060
164489930,3cyclictest20708-21cut21:15:100
164489930,3cyclictest18034-21sh23:45:120
164489930,3cyclictest0-21swapper/022:15:100
164489930,3cyclictest0-21swapper/021:00:100
164489930,3cyclictest0-21swapper/020:55:100
164489930,3cyclictest0-21swapper/020:06:490
164489930,3cyclictest0-21swapper/019:40:110
164489930,3cyclictest0-21swapper/000:10:080
164489930,2cyclictest6888-21sh23:05:210
164489930,2cyclictest30997-21snmpd22:00:070
164489930,1cyclictest0-21swapper/021:25:450
164539921,1cyclictest0-21swapper/123:10:071
164539921,1cyclictest0-21swapper/122:00:111
164539921,1cyclictest0-21swapper/120:00:111
164539921,1cyclictest0-21swapper/119:35:171
164539920,1cyclictest17887-21chrt23:45:071
164539920,1cyclictest0-21swapper/123:07:191
164539920,1cyclictest0-21swapper/122:55:091
164539920,1cyclictest0-21swapper/122:25:021
164539920,1cyclictest0-21swapper/121:48:111
164539920,1cyclictest0-21swapper/121:40:121
164539920,1cyclictest0-21swapper/120:20:121
164539920,1cyclictest0-21swapper/119:45:131
164539920,1cyclictest0-21swapper/100:05:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional