You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-21 - 04:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Sun Apr 21, 2024 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88772199184,11sleep00-21swapper/019:06:050
90262198182,11sleep10-21swapper/119:08:001
88432194177,12sleep30-21swapper/319:05:413
88692170152,13sleep20-21swapper/219:06:002
315882470,0sleep20-21swapper/222:44:392
17172440,0sleep00-21swapper/022:25:270
274802430,0sleep20-21swapper/222:21:442
65982420,0sleep00-21swapper/022:09:500
197752120,0sleep10-21swapper/123:35:381
93619980,7cyclictest0-21swapper/320:00:133
93479982,3cyclictest12447-21idleruntime-cro20:20:010
93619970,6cyclictest0-21swapper/319:15:133
93619970,5cyclictest0-21swapper/300:05:133
93479971,3cyclictest6388-21perf21:50:000
93579965,1cyclictest31-21ksoftirqd/223:00:012
93579965,0cyclictest31-21ksoftirqd/200:00:002
93519961,2cyclictest0-21swapper/122:30:131
93479962,4cyclictest24810-21grep23:20:010
93479960,3cyclictest14101-21munin-run21:35:010
93619950,5cyclictest14504-21/usr/sbin/munin21:35:213
93619950,5cyclictest0-21swapper/320:20:123
93619950,5cyclictest0-21swapper/300:10:243
93619950,4cyclictest17416-21perl22:55:133
93619950,4cyclictest0-21swapper/319:35:103
93619950,0cyclictest0-21swapper/323:54:493
93579950,5cyclictest0-21swapper/223:27:162
93579950,5cyclictest0-21swapper/223:14:012
93579950,5cyclictest0-21swapper/222:53:572
93579950,5cyclictest0-21swapper/221:36:412
93519951,3cyclictest0-21swapper/122:10:101
93519951,2cyclictest0-21swapper/122:05:121
93519951,2cyclictest0-21swapper/119:15:141
93519950,5cyclictest0-21swapper/122:44:181
93519950,5cyclictest0-21swapper/122:28:521
93519950,5cyclictest0-21swapper/121:46:321
93519950,5cyclictest0-21swapper/121:35:191
93519950,5cyclictest0-21swapper/100:05:211
93479955,0cyclictest0-21swapper/021:37:590
93479955,0cyclictest0-21swapper/021:25:190
93479951,2cyclictest0-21swapper/021:15:130
93479950,5cyclictest0-21swapper/023:38:100
93479950,5cyclictest0-21swapper/023:21:430
93479950,5cyclictest0-21swapper/020:30:180
93479950,5cyclictest0-21swapper/000:18:450
93479950,4cyclictest0-21swapper/023:30:090
93479950,4cyclictest0-21swapper/020:45:110
93479950,4cyclictest0-21swapper/020:10:110
93479950,1cyclictest15628-21munin-run22:35:010
93619944,0cyclictest0-21swapper/323:42:193
93619944,0cyclictest0-21swapper/323:36:033
93619944,0cyclictest0-21swapper/321:31:303
93619944,0cyclictest0-21swapper/300:39:383
93619943,1cyclictest16126-21diskstats22:15:153
93619941,3cyclictest3602-21snmpd21:08:213
93619940,4cyclictest12493-21python00:30:243
93619940,4cyclictest0-21swapper/323:55:093
93619940,4cyclictest0-21swapper/323:46:513
93619940,4cyclictest0-21swapper/323:26:173
93619940,4cyclictest0-21swapper/322:45:023
93619940,4cyclictest0-21swapper/322:30:133
93619940,4cyclictest0-21swapper/322:21:153
93619940,4cyclictest0-21swapper/322:10:093
93619940,4cyclictest0-21swapper/321:29:253
93619940,4cyclictest0-21swapper/321:12:213
93619940,4cyclictest0-21swapper/320:55:123
93619940,4cyclictest0-21swapper/320:35:113
93619940,4cyclictest0-21swapper/320:08:063
93619940,4cyclictest0-21swapper/319:25:103
93619940,4cyclictest0-21swapper/300:20:343
93619940,3cyclictest9627-21perl23:10:173
93619940,3cyclictest6507-21latency_hist21:50:003
93619940,3cyclictest26202-21if_err_eth119:40:163
93619940,3cyclictest25984-21perl23:20:153
93619940,3cyclictest21839-21sort21:20:013
93619940,3cyclictest20340-21/usr/sbin/munin19:30:243
93619940,3cyclictest0-21swapper/319:50:133
93619940,3cyclictest0-21swapper/319:50:133
93619940,1cyclictest8496-21/usr/sbin/munin22:50:203
93619940,1cyclictest537-21ssh22:05:203
93619940,1cyclictest32704-21/usr/sbin/munin19:55:223
93619940,1cyclictest32680-21/usr/sbin/munin21:00:133
93619940,1cyclictest32365-21grep22:25:143
93619940,1cyclictest15156-21/usr/sbin/munin20:25:223
93619940,0cyclictest0-21swapper/323:17:173
93619940,0cyclictest0-21swapper/321:54:383
93619940,0cyclictest0-21swapper/321:43:263
93579944,0cyclictest16416-21ssh21:55:182
93579944,0cyclictest0-21swapper/223:42:052
93579944,0cyclictest0-21swapper/221:23:182
93579943,1cyclictest31-21ksoftirqd/223:50:012
93579942,2cyclictest150192sleep223:33:242
93579941,2cyclictest0-21swapper/219:50:132
93579941,2cyclictest0-21swapper/219:50:122
93579941,1cyclictest0-21swapper/223:15:082
93579940,4cyclictest4973-21diskmemload23:45:102
93579940,4cyclictest27503-21perf20:50:002
93579940,4cyclictest17629-21apt-get20:30:112
93579940,4cyclictest0-21swapper/223:04:552
93579940,4cyclictest0-21swapper/222:30:132
93579940,4cyclictest0-21swapper/222:25:132
93579940,4cyclictest0-21swapper/222:14:232
93579940,4cyclictest0-21swapper/221:50:132
93579940,4cyclictest0-21swapper/221:45:142
93579940,4cyclictest0-21swapper/221:40:142
93579940,4cyclictest0-21swapper/221:28:462
93579940,4cyclictest0-21swapper/221:15:142
93579940,4cyclictest0-21swapper/219:55:112
93579940,4cyclictest0-21swapper/219:30:122
93579940,4cyclictest0-21swapper/200:22:582
93579940,4cyclictest0-21swapper/200:01:362
93579940,3cyclictest9186-21cron19:10:012
93579940,3cyclictest25099-21latency_hist20:45:002
93579940,3cyclictest2184-21apt-get00:05:122
93579940,3cyclictest0-21swapper/219:15:012
93519944,0cyclictest0-21swapper/123:46:331
93519942,1cyclictest0-21swapper/122:55:091
93519941,2cyclictest0-21swapper/123:10:131
93519941,2cyclictest0-21swapper/121:30:121
93519941,2cyclictest0-21swapper/121:20:131
93519941,2cyclictest0-21swapper/121:15:141
93519941,2cyclictest0-21swapper/120:50:101
93519941,2cyclictest0-21swapper/120:20:101
93519941,2cyclictest0-21swapper/120:10:111
93519941,2cyclictest0-21swapper/119:30:111
93519941,2cyclictest0-21swapper/100:00:131
93519941,1cyclictest0-21swapper/120:40:101
93519941,1cyclictest0-21swapper/120:00:131
93519940,4cyclictest0-21swapper/121:51:461
93519940,2cyclictest0-21swapper/120:05:111
93519940,2cyclictest0-21swapper/119:45:191
93519940,0cyclictest0-21swapper/100:16:141
93479944,0cyclictest22587-21apt19:35:010
93479944,0cyclictest0-21swapper/022:47:400
93479944,0cyclictest0-21swapper/021:11:560
93479944,0cyclictest0-21swapper/000:14:500
93479942,1cyclictest28130-21expr19:45:210
93479941,1cyclictest0-21swapper/021:20:120
93479940,4cyclictest4973-21diskmemload00:20:410
93479940,4cyclictest22006-21sh22:38:320
93479940,4cyclictest0-21swapper/022:55:100
93479940,4cyclictest0-21swapper/022:10:090
93479940,4cyclictest0-21swapper/021:57:100
93479940,4cyclictest0-21swapper/021:50:130
93479940,4cyclictest0-21swapper/000:37:580
93479940,3cyclictest781-21diskstats21:00:130
93479940,3cyclictest5864-21kworker/0:200:25:010
93479940,3cyclictest31664-21ssh23:04:120
93479940,3cyclictest30196-21apt20:55:130
93479940,3cyclictest28171-21ssh21:43:050
93479940,3cyclictest25382-21irqstats22:40:180
93479940,3cyclictest23595-21ssh22:20:030
93479940,3cyclictest1886-21cat00:05:000
93479940,3cyclictest1781-21tr19:55:210
93479940,3cyclictest17730-21apt-config19:25:010
93479940,3cyclictest1320-21ssh23:45:010
93479940,3cyclictest10027-21grep00:10:000
93479940,3cyclictest0-21swapper/022:00:120
93479940,3cyclictest0-21swapper/020:20:110
93479940,1cyclictest23332-21sh22:20:010
93479940,1cyclictest16319-21munin-run22:55:010
93479940,0cyclictest0-21swapper/023:59:150
93479940,0cyclictest0-21swapper/023:08:280
93479940,0cyclictest0-21swapper/020:27:140
93619933,0cyclictest0-21swapper/300:15:033
93619932,1cyclictest3435-21hddtemp_smartct00:25:153
93619931,1cyclictest16072-21aten2_r5power_e22:35:103
93619931,1cyclictest0-21swapper/320:40:103
93619931,1cyclictest0-21swapper/319:20:103
93619930,3cyclictest9341-21cyclictest23:05:083
93619930,3cyclictest9231-21apt-get23:30:093
93619930,3cyclictest61-21kswapd022:40:113
93619930,3cyclictest29183-21which20:50:203
93619930,3cyclictest17629-21apt-get20:30:103
93619930,3cyclictest0-21swapper/323:00:103
93619930,3cyclictest0-21swapper/322:00:123
93619930,3cyclictest0-21swapper/321:55:033
93619930,3cyclictest0-21swapper/320:45:013
93619930,3cyclictest0-21swapper/320:15:153
93619930,3cyclictest0-21swapper/320:10:113
93619930,3cyclictest0-21swapper/319:45:193
93619930,3cyclictest0-21swapper/319:10:103
93619930,3cyclictest0-21swapper/300:00:093
93619930,1cyclictest21924-21ssh21:20:043
93579933,0cyclictest31-21ksoftirqd/223:05:202
93579933,0cyclictest0-21swapper/219:20:032
93579932,1cyclictest23374-21apt-get22:00:122
93579932,1cyclictest19731-21fschecks_time00:35:142
93579930,3cyclictest7711-21apt-get20:10:102
93579930,3cyclictest3602-21snmpd21:30:062
93579930,3cyclictest3398-21fschecks_count20:00:162
93579930,3cyclictest25789-21df_inode23:20:142
93579930,3cyclictest20098-21apt-get20:35:102
93579930,3cyclictest0-21swapper/223:35:092
93579930,3cyclictest0-21swapper/222:45:012
93579930,3cyclictest0-21swapper/222:35:092
93579930,3cyclictest0-21swapper/222:15:122
93579930,3cyclictest0-21swapper/222:05:122
93579930,3cyclictest0-21swapper/221:10:102
93579930,3cyclictest0-21swapper/221:05:132
93579930,3cyclictest0-21swapper/221:00:092
93579930,3cyclictest0-21swapper/220:55:052
93579930,3cyclictest0-21swapper/220:50:102
93579930,3cyclictest0-21swapper/220:20:112
93579930,3cyclictest0-21swapper/220:15:122
93579930,3cyclictest0-21swapper/220:05:112
93579930,3cyclictest0-21swapper/219:45:192
93579930,3cyclictest0-21swapper/219:40:122
93579930,3cyclictest0-21swapper/219:35:102
93579930,3cyclictest0-21swapper/219:25:012
93579930,3cyclictest0-21swapper/200:30:092
93579930,3cyclictest0-21swapper/200:25:122
93579930,3cyclictest0-21swapper/200:15:032
93579930,3cyclictest0-21swapper/200:10:132
93579930,0cyclictest0-21swapper/220:25:052
93519933,0cyclictest0-21swapper/123:05:061
93519933,0cyclictest0-21swapper/121:41:171
93519933,0cyclictest0-21swapper/120:32:541
93519931,2cyclictest0-21swapper/120:15:131
93519931,1cyclictest162072chrt23:34:221
93519931,1cyclictest0-21swapper/123:40:081
93519931,1cyclictest0-21swapper/123:25:111
93519931,1cyclictest0-21swapper/123:20:111
93519931,1cyclictest0-21swapper/123:00:101
93519931,1cyclictest0-21swapper/122:35:101
93519931,1cyclictest0-21swapper/122:00:111
93519931,1cyclictest0-21swapper/121:55:101
93519931,1cyclictest0-21swapper/120:45:101
93519931,1cyclictest0-21swapper/120:35:111
93519931,1cyclictest0-21swapper/120:25:111
93519931,1cyclictest0-21swapper/119:40:121
93519931,1cyclictest0-21swapper/119:25:021
93519931,1cyclictest0-21swapper/119:20:111
93519931,1cyclictest0-21swapper/119:10:101
93519931,1cyclictest0-21swapper/100:30:131
93519931,1cyclictest0-21swapper/100:10:111
93519930,2cyclictest8690-21chrt21:11:211
93519930,2cyclictest262442sleep122:20:441
93519930,2cyclictest0-21swapper/123:50:011
93519930,2cyclictest0-21swapper/121:05:131
93519930,2cyclictest0-21swapper/120:55:131
93519930,2cyclictest0-21swapper/119:55:101
93519930,1cyclictest0-21swapper/122:15:131
93479932,1cyclictest521-21nscd23:40:080
93479931,1cyclictest12088-21aten2_r5power_e19:15:000
93479930,3cyclictest9564-21apt-config23:50:010
93479930,3cyclictest8726-21seq23:10:070
93479930,3cyclictest0-21swapper/023:25:060
93479930,3cyclictest0-21swapper/021:05:010
93479930,3cyclictest0-21swapper/020:50:090
93479930,3cyclictest0-21swapper/020:40:100
93479930,3cyclictest0-21swapper/020:35:110
93479930,3cyclictest0-21swapper/020:05:130
93479930,3cyclictest0-21swapper/020:00:120
93479930,3cyclictest0-21swapper/019:50:230
93479930,3cyclictest0-21swapper/019:50:230
93479930,3cyclictest0-21swapper/019:40:110
93479930,3cyclictest0-21swapper/019:35:100
93479930,3cyclictest0-21swapper/019:20:150
93479930,3cyclictest0-21swapper/019:10:090
93479930,3cyclictest0-21swapper/000:30:120
93519921,1cyclictest0-21swapper/123:55:121
93519921,1cyclictest0-21swapper/123:15:091
93519921,1cyclictest0-21swapper/121:25:141
93519921,1cyclictest0-21swapper/119:50:131
93519921,1cyclictest0-21swapper/119:50:121
93519920,1cyclictest0-21swapper/122:50:111
93519920,1cyclictest0-21swapper/122:45:021
93519920,1cyclictest0-21swapper/121:00:171
93519920,1cyclictest0-21swapper/119:35:091
93519920,1cyclictest0-21swapper/100:35:111
93519920,1cyclictest0-21swapper/100:25:091
93519920,1cyclictest0-21swapper/100:21:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional