You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-03 - 12:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Oct 03, 2024 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
313099280,0cyclictest0-21swapper/122:21:201
313099270,13cyclictest0-21swapper/121:06:181
312899250,24cyclictest593-21dbus-daemon20:55:000
3136992422,1cyclictest1-21systemd23:00:003
313099240,23cyclictest25693-21cron23:15:001
312899240,23cyclictest313-21systemd-journal19:20:000
3136992322,0cyclictest0-21swapper/322:41:193
3136992320,2cyclictest3644-21turbostat.cron22:30:003
313699230,22cyclictest28088-21sh23:20:003
313699230,22cyclictest26457-21perf22:10:003
3134992320,2cyclictest3645-21apt22:30:002
3134992320,2cyclictest30012-21idleruntime-cro21:10:012
3130992323,0cyclictest0-21swapper/121:31:191
3130992321,1cyclictest28125-21sendmail-msp23:20:001
3130992320,2cyclictest596-21in:imuxsock21:45:001
3128992321,1cyclictest2734-21grep20:15:000
3128992320,2cyclictest17468-21ls00:05:000
312899230,22cyclictest32243-21idleruntime-cro00:35:000
312899230,22cyclictest0-21swapper/000:40:000
312899230,0cyclictest0-21swapper/022:56:190
3136992222,0cyclictest0-21swapper/300:16:203
3136992221,0cyclictest4013-21apt-get22:31:193
3136992221,0cyclictest26810-21apt-get22:11:193
3136992220,1cyclictest593-21dbus-daemon20:35:003
313699220,22cyclictest2721-21apt-get20:15:023
313699220,21cyclictest30708-21apt-get20:06:203
313699220,21cyclictest28309-21apt-get20:01:173
313699220,21cyclictest25220-21awk21:00:003
313699220,21cyclictest19609-21apt-get21:56:193
313699220,1cyclictest22747-21latency_hist19:50:003
313499220,21cyclictest25564-21apt-get21:01:192
313499220,21cyclictest18880-21apt-get23:01:212
313499220,21cyclictest15428-21apt-get00:01:192
313499220,1cyclictest12781-21latency_hist20:35:002
313499220,1cyclictest10302-21grep19:25:002
3130992220,1cyclictest0-21swapper/119:56:201
3130992219,2cyclictest12563-21dump-pmu-power23:55:001
313099220,21cyclictest844-21apt-get23:31:181
313099220,21cyclictest6054-21perf22:35:001
313099220,21cyclictest211rcuc/123:26:201
313099220,21cyclictest0-21swapper/123:46:201
313099220,21cyclictest0-21swapper/122:36:201
313099220,21cyclictest0-21swapper/121:36:181
313099220,21cyclictest0-21swapper/119:51:191
313099220,1cyclictest13503-21apt-get19:31:191
313099220,1cyclictest0-21swapper/120:46:211
3128992222,0cyclictest13150-21perf19:30:010
3128992222,0cyclictest0-21swapper/022:26:190
3128992220,1cyclictest593-21dbus-daemon20:30:000
3128992220,1cyclictest0-21swapper/023:35:010
3128992219,3cyclictest22024-21apt-get22:01:200
312899220,21cyclictest32750-21apt-get21:16:190
312899220,21cyclictest0-21swapper/023:21:180
312899220,21cyclictest0-21swapper/023:16:200
312899220,21cyclictest0-21swapper/019:36:190
312899220,21cyclictest0-21swapper/000:21:190
3136992120,1cyclictest28478-21apt-get23:21:193
3136992120,1cyclictest25894-21apt-get19:56:203
3136992120,1cyclictest23101-21apt-get19:51:193
3136992119,1cyclictest10293-21awk19:25:003
313699210,21cyclictest0-21swapper/321:51:193
313699210,20cyclictest0-21swapper/323:36:183
313699210,20cyclictest0-21swapper/321:36:183
313699210,20cyclictest0-21swapper/319:16:193
313699210,1cyclictest13548-21apt-get20:36:183
313699210,0cyclictest0-21swapper/323:01:213
313699210,0cyclictest0-21swapper/321:16:193
313499210,21cyclictest0-21swapper/219:56:202
313499210,20cyclictest3356-21apt-get20:15:022
313499210,20cyclictest3263-21apt-get23:36:182
313499210,20cyclictest2760-21apt-get21:21:192
313499210,20cyclictest21272-21apt-get23:06:182
313499210,0cyclictest0-21swapper/222:46:202
3130992121,0cyclictest0-21swapper/119:25:021
3130992119,1cyclictest15164-21munin-plugin-st00:00:001
3130992119,1cyclictest10288-21sh19:25:001
313099210,21cyclictest0-21swapper/122:56:181
313099210,21cyclictest0-21swapper/120:16:201
313099210,20cyclictest30225-21apt-get00:31:221
313099210,20cyclictest0-21swapper/119:46:171
313099210,0cyclictest22-21ksoftirqd/120:56:171
313099210,0cyclictest0-21swapper/119:41:181
3128992121,0cyclictest10471-21apt-get23:51:200
3128992121,0cyclictest0-21swapper/019:11:200
3128992120,1cyclictest5680-21apt-get23:41:200
3128992120,1cyclictest0-21swapper/019:31:190
312899210,20cyclictest9951-21apt-get21:36:180
312899210,20cyclictest26439-21cron22:10:000
312899210,20cyclictest0-21swapper/020:56:170
312899210,20cyclictest0-21swapper/020:16:210
312899210,0cyclictest7560-21apt-get21:31:190
312899210,0cyclictest0-21swapper/023:06:180
312899210,0cyclictest0-21swapper/022:41:190
3136992020,0cyclictest8303-21apt-get20:26:193
3136992020,0cyclictest5911-21apt-get20:21:193
3136992020,0cyclictest0-21swapper/323:11:183
3136992020,0cyclictest0-21swapper/319:31:193
3136992020,0cyclictest0-21swapper/319:11:203
3136992020,0cyclictest0-21swapper/300:31:213
3136992019,0cyclictest0-21swapper/320:46:213
313699200,1cyclictest593-21dbus-daemon21:10:013
313699200,19cyclictest0-21swapper/321:31:193
313699200,19cyclictest0-21swapper/300:21:193
313699200,0cyclictest0-21swapper/323:51:213
313699200,0cyclictest0-21swapper/321:21:193
3134992020,0cyclictest3449-21apt-get19:11:202
3134992020,0cyclictest0-21swapper/223:16:192
3134992020,0cyclictest0-21swapper/200:26:192
3134992019,0cyclictest0-21swapper/219:51:192
3134992018,1cyclictest32263-21latency_hist00:35:002
3134992018,1cyclictest20397-21perf20:50:002
313499200,19cyclictest0-21swapper/220:06:202
313499200,0cyclictest17213-21apt-get21:51:192
313499200,0cyclictest12790-21apt-get20:35:012
313499200,0cyclictest0-21swapper/221:56:192
3130992020,0cyclictest0-21swapper/123:21:191
3130992020,0cyclictest0-21swapper/123:01:211
3130992020,0cyclictest0-21swapper/122:26:181
3130992020,0cyclictest0-21swapper/121:16:191
3130992020,0cyclictest0-21swapper/120:31:201
3130992020,0cyclictest0-21swapper/120:06:201
3130992019,1cyclictest0-21swapper/121:51:191
3130992019,1cyclictest0-21swapper/120:21:191
3130992019,1cyclictest0-21swapper/100:10:021
3130992019,1cyclictest0-21swapper/100:10:021
3130992018,1cyclictest11258-21wc22:45:011
3130992018,1cyclictest0-21swapper/100:01:191
313099200,0cyclictest0-21swapper/121:56:191
3128992020,0cyclictest0-21swapper/022:31:200
3128992020,0cyclictest0-21swapper/021:56:190
3128992020,0cyclictest0-21swapper/020:21:190
3128992020,0cyclictest0-21swapper/020:01:160
3128992019,1cyclictest8460-21turbostat22:40:010
3128992019,1cyclictest20349-21idleruntime-cro19:45:000
3128992019,0cyclictest0-21swapper/019:46:180
3128992018,1cyclictest7217-21idleruntime-cro21:30:000
3128992018,1cyclictest16110-21awk22:55:010
312899200,20cyclictest0-21swapper/021:06:180
3136991919,0cyclictest27826-21apt-get00:26:193
3136991919,0cyclictest19857-21apt-get00:10:023
3136991919,0cyclictest19857-21apt-get00:10:023
3136991919,0cyclictest11618-21apt-get22:46:203
3136991919,0cyclictest0-21swapper/322:51:193
3136991919,0cyclictest0-21swapper/300:06:213
3136991918,1cyclictest0-21swapper/323:30:013
3136991918,0cyclictest0-21swapper/321:26:193
3136991917,1cyclictest14471-21perf21:45:003
3136991917,1cyclictest10127-21awk23:50:003
3134991919,0cyclictest18313-21apt-get19:41:182
3134991919,0cyclictest0-21swapper/221:26:192
3134991919,0cyclictest0-21swapper/221:16:192
3134991919,0cyclictest0-21swapper/220:26:192
3134991919,0cyclictest0-21swapper/219:31:202
3134991919,0cyclictest0-21swapper/219:26:272
3134991918,1cyclictest29205-21apt-get22:16:192
3134991918,1cyclictest17825-21apt-get00:06:212
3134991917,1cyclictest578-21irqbalance00:36:202
313499190,0cyclictest0-21swapper/222:40:022
3130991917,1cyclictest18007-21apt20:45:001
313099190,18cyclictest0-21swapper/120:51:191
3128991919,0cyclictest0-21swapper/023:56:200
3128991919,0cyclictest0-21swapper/021:11:190
3128991919,0cyclictest0-21swapper/020:41:190
3128991919,0cyclictest0-21swapper/000:26:200
3128991919,0cyclictest0-21swapper/000:11:200
3128991919,0cyclictest0-21swapper/000:11:200
3128991918,1cyclictest0-21swapper/022:21:190
3128991918,0cyclictest0-21swapper/022:46:200
3128991917,1cyclictest313-21systemd-journal20:10:000
3128991917,1cyclictest20379-21idleruntime-cro20:50:000
313699183,14cyclictest15901-21apt-get19:36:193
3136991818,0cyclictest0-21swapper/323:06:183
3136991818,0cyclictest0-21swapper/321:01:193
3136991818,0cyclictest0-21swapper/320:41:193
3136991818,0cyclictest0-21swapper/300:01:193
3136991816,1cyclictest11060-21apt-get19:26:273
3134991818,0cyclictest24418-21apt-get22:06:192
3134991818,0cyclictest23146-21apt-get20:56:172
3134991818,0cyclictest0-21swapper/223:46:202
3134991818,0cyclictest0-21swapper/222:21:202
3134991818,0cyclictest0-21swapper/221:41:182
3134991818,0cyclictest0-21swapper/220:01:172
3134991817,1cyclictest5842-21apt-get19:16:192
3134991817,0cyclictest0-21swapper/200:25:012
3134991816,1cyclictest22757-21awk19:50:002
3134991816,1cyclictest14956-40find00:00:002
3130991818,0cyclictest0-21swapper/121:01:191
3130991818,0cyclictest0-21swapper/120:11:191
3130991818,0cyclictest0-21swapper/119:11:191
3130991818,0cyclictest0-21swapper/100:36:201
3130991818,0cyclictest0-21swapper/100:06:211
3130991817,1cyclictest0-21swapper/122:01:201
3130991817,0cyclictest0-21swapper/100:16:191
3136991717,0cyclictest0-21swapper/323:41:193
3136991717,0cyclictest0-21swapper/323:31:183
3136991717,0cyclictest0-21swapper/321:11:193
3134991717,0cyclictest20753-21apt-get20:51:192
3134991717,0cyclictest12024-21sendmail-msp21:40:002
3134991717,0cyclictest0-21swapper/222:36:202
3134991717,0cyclictest0-21swapper/219:36:192
3134991717,0cyclictest0-21swapper/200:10:022
3134991717,0cyclictest0-21swapper/200:10:022
3134991715,1cyclictest16463-21apt-get22:56:192
3130991717,0cyclictest0-21swapper/122:11:191
3130991717,0cyclictest0-21swapper/122:06:191
3130991717,0cyclictest0-21swapper/121:21:191
3130991717,0cyclictest0-21swapper/120:35:011
3130991717,0cyclictest0-21swapper/119:16:191
3130991716,1cyclictest0-21swapper/122:46:191
3130991715,1cyclictest313-21systemd-journal19:40:011
3128991717,0cyclictest30874-21apt-get23:26:200
3128991717,0cyclictest0-21swapper/023:36:180
3128991717,0cyclictest0-21swapper/021:46:180
3128991717,0cyclictest0-21swapper/021:46:180
3128991717,0cyclictest0-21swapper/020:31:200
3128991717,0cyclictest0-21swapper/019:21:170
3128991716,1cyclictest0-21swapper/021:51:190
312899170,1cyclictest28850-21turbostat.cron22:15:000
312899170,16cyclictest14164-21cut20:36:210
312899170,0cyclictest0-21swapper/023:46:200
3134991616,0cyclictest661-21apt-get20:11:192
3134991616,0cyclictest22024-21apt-get22:01:202
3134991616,0cyclictest0-21swapper/220:21:192
3134991614,1cyclictest30528-21munin-run23:25:002
3130991616,0cyclictest0-21swapper/123:06:181
3130991616,0cyclictest0-21swapper/120:01:171
313099160,9cyclictest4370-21grep23:36:241
3128991616,0cyclictest0-21swapper/022:16:200
3128991616,0cyclictest0-21swapper/019:51:190
312899160,1cyclictest0-21swapper/023:01:210
3136991515,0cyclictest0-21swapper/322:16:193
3136991515,0cyclictest0-21swapper/319:41:183
3136991513,1cyclictest14964-21dump-pmu-power00:00:003
313699150,13cyclictest32620-21apt-get00:36:203
3134991515,0cyclictest0-21swapper/223:41:202
3134991515,0cyclictest0-21swapper/221:15:012
3134991514,1cyclictest22982-21basename00:16:192
3130991513,1cyclictest0-21swapper/100:26:201
3136991414,0cyclictest6411-21apt-get22:36:203
3136991414,0cyclictest31624-21apt-get22:21:203
3136991414,0cyclictest0-21swapper/322:02:353
3136991414,0cyclictest0-21swapper/321:45:103
3136991414,0cyclictest0-21swapper/321:45:103
313699140,14cyclictest0-21swapper/320:50:293
313699140,0cyclictest0-21swapper/320:11:233
3134991414,0cyclictest0-21swapper/223:51:212
3134991414,0cyclictest0-21swapper/220:40:032
3134991412,1cyclictest32165-21ptp4l-jitter23:26:272
313499140,14cyclictest0-21swapper/223:30:252
313499140,14cyclictest0-21swapper/222:51:172
313499140,14cyclictest0-21swapper/221:45:092
313499140,14cyclictest0-21swapper/221:45:092
313499140,14cyclictest0-21swapper/221:30:522
313499140,0cyclictest0-21swapper/222:30:062
313099140,14cyclictest0-21swapper/121:25:251
313099140,14cyclictest0-21swapper/120:26:151
313099140,0cyclictest0-21swapper/123:40:241
313099140,0cyclictest0-21swapper/122:50:261
3128991414,0cyclictest0-21swapper/021:01:150
312899140,14cyclictest0-21swapper/000:15:130
312899140,14cyclictest0-21swapper/000:05:580
312899140,0cyclictest0-21swapper/021:42:160
312899140,0cyclictest0-21swapper/021:22:290
313499130,0cyclictest0-21swapper/222:11:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional