You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-09 - 12:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6.osadl.org (updated Wed Jul 09, 2025 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
733421000,6chrt7339-21unixbench_multi20:45:380
4486999463,18cyclictest26-21ksoftirqd/123:20:181
4486999369,15cyclictest26-21ksoftirqd/100:00:121
4486999364,16cyclictest26-21ksoftirqd/120:25:171
4486999363,20cyclictest26-21ksoftirqd/122:35:161
4486999268,16cyclictest26-21ksoftirqd/122:40:161
4486999264,18cyclictest26-21ksoftirqd/120:50:161
4486999060,20cyclictest26-21ksoftirqd/119:25:121
4486998963,17cyclictest26-21ksoftirqd/122:45:161
4484998935,36cyclictest14-21ksoftirqd/019:40:140
4486998863,16cyclictest26-21ksoftirqd/121:25:241
4486998859,19cyclictest26-21ksoftirqd/122:20:131
4486998858,19cyclictest26-21ksoftirqd/119:10:131
4486998758,19cyclictest26-21ksoftirqd/120:05:131
4484998746,21cyclictest14-21ksoftirqd/023:30:140
4486998654,21cyclictest26-21ksoftirqd/121:10:121
321902860,4chrt0-21swapper/020:25:270
4486998555,20cyclictest26-21ksoftirqd/100:30:131
4486998553,18cyclictest26-21ksoftirqd/122:55:131
4486998551,19cyclictest26-21ksoftirqd/119:45:161
448499837,58cyclictest0-21swapper/019:30:120
111952830,14chrt11190-21missed_timers19:25:260
4486998252,20cyclictest26-21ksoftirqd/121:00:141
4486998250,20cyclictest26-21ksoftirqd/123:50:141
4486998152,17cyclictest26-21ksoftirqd/121:45:261
4486998150,20cyclictest26-21ksoftirqd/120:00:131
4486998150,20cyclictest26-21ksoftirqd/120:00:131
4486998045,23cyclictest26-21ksoftirqd/119:55:121
4486997950,16cyclictest26-21ksoftirqd/119:30:111
4486997949,20cyclictest26-21ksoftirqd/121:30:131
4486997946,22cyclictest26-21ksoftirqd/121:40:111
4486997848,21cyclictest26-21ksoftirqd/121:55:131
4484997846,20cyclictest14-21ksoftirqd/020:40:160
4484997846,20cyclictest14-21ksoftirqd/020:40:160
4486997742,27cyclictest26-21ksoftirqd/119:35:161
4486997649,16cyclictest26-21ksoftirqd/120:15:131
4486997643,21cyclictest26-21ksoftirqd/121:05:161
4486997641,21cyclictest26-21ksoftirqd/122:15:161
4486997641,21cyclictest26-21ksoftirqd/122:15:151
4486997544,19cyclictest26-21ksoftirqd/100:15:161
4484997549,16cyclictest18077-21apt-get00:20:160
4486997444,19cyclictest26-21ksoftirqd/123:05:151
73022710,6sleep00-21swapper/019:15:260
4486997145,18cyclictest26-21ksoftirqd/100:25:151
4486997139,18cyclictest0-21swapper/120:20:161
4486997136,21cyclictest26-21ksoftirqd/122:50:151
4486997038,20cyclictest0-21swapper/100:05:141
4486997035,21cyclictest26-21ksoftirqd/120:10:151
4486996942,16cyclictest0-21swapper/120:45:141
425826919,40sleep00-21swapper/019:09:020
4486996811,43cyclictest24077-21seq21:35:151
4486996733,22cyclictest26-21ksoftirqd/120:55:191
4486996712,41cyclictest0-21swapper/120:30:181
448499676,41cyclictest0-21swapper/023:15:160
448499675,14cyclictest0-21swapper/020:20:130
448699669,44cyclictest0-21swapper/119:40:151
4486996631,23cyclictest26-21ksoftirqd/120:35:131
4486996613,42cyclictest0-21swapper/120:40:181
4486996613,42cyclictest0-21swapper/120:40:181
432126631,25sleep10-21swapper/119:09:441
4486996537,20cyclictest26-21ksoftirqd/122:25:131
4486996510,38cyclictest151rcu_preempt23:45:001
448499652,44cyclictest23056-21aten2.4-expect20:00:170
448499652,44cyclictest23056-21aten2.4-expect20:00:170
448699649,42cyclictest8910-21taskset19:20:231
448699648,42cyclictest594-21runrttasks00:10:141
448499647,39cyclictest0-21swapper/021:55:120
448499647,11cyclictest0-21swapper/022:50:130
448499645,17cyclictest6317-21apt-get22:15:160
448499645,17cyclictest6317-21apt-get22:15:160
4486996336,16cyclictest0-21swapper/123:00:131
4484996336,21cyclictest14-21ksoftirqd/023:05:180
4484996324,21cyclictest0-21swapper/023:55:130
4484996324,21cyclictest0-21swapper/023:55:130
4486996231,20cyclictest0-21swapper/123:10:141
448499625,39cyclictest14875-21diskmemload21:10:110
141082620,5chrt0-21swapper/021:05:350
448499617,39cyclictest0-21swapper/022:05:170
448499616,16cyclictest0-21swapper/019:35:150
4484996136,20cyclictest14-21ksoftirqd/000:04:580
4484996120,17cyclictest24768-21aten2.4-expect20:05:170
4484996120,15cyclictest22547-21aten2.4-expect23:00:190
4484996119,17cyclictest29523-21aten2.4-expect23:20:200
4484996118,18cyclictest19430-21aten2.4-expect19:50:160
448499603,50cyclictest20120-21apt-get21:25:150
4484996028,27cyclictest14-21ksoftirqd/022:30:280
448499602,21cyclictest23115-21latency_hist00:35:000
4484996022,15cyclictest28296-21aten2.4-expect20:15:180
307462600,8sleep014-21ksoftirqd/021:50:360
448699593,49cyclictest4603-21cut22:10:171
4486995929,18cyclictest0-21swapper/123:45:121
4486995831,19cyclictest3831-21cron23:40:001
4484995830,23cyclictest14-21ksoftirqd/021:15:330
4484995825,20cyclictest14-21ksoftirqd/019:45:300
448699574,14cyclictest24916-21munin-run00:40:001
4484995735,8cyclictest14-21ksoftirqd/021:20:270
4484995734,17cyclictest14-21ksoftirqd/020:10:170
4484995733,19cyclictest14-21ksoftirqd/021:35:200
4484995720,32cyclictest28039-21aten2.4-expect21:45:260
4486995618,10cyclictest76752sleep119:15:361
448499569,42cyclictest0-21swapper/019:20:360
4484995635,16cyclictest14-21ksoftirqd/020:55:210
4484995633,18cyclictest14-21ksoftirqd/023:25:170
4484995633,18cyclictest14-21ksoftirqd/022:30:010
4484995620,29cyclictest17280-21aten2.4-expect22:45:190
4484995617,16cyclictest2787-21aten2.4-expect20:35:170
4486995527,22cyclictest261-21systemd-journal23:31:421
4486995522,21cyclictest0-21swapper/123:25:141
448499556,37cyclictest0-21swapper/023:45:150
448499555,22cyclictest4567-21apt-get19:10:130
4484995538,10cyclictest14-21ksoftirqd/022:20:020
4484995532,8cyclictest14-21ksoftirqd/023:00:010
448499552,16cyclictest8572-21unixbench_multi23:50:360
4484995521,29cyclictest2345-21aten2.4-expect23:35:190
4484995516,17cyclictest15236-21aten2.4-expect22:40:180
448699546,11cyclictest0-21swapper/123:15:281
4486995428,20cyclictest261-21systemd-journal00:00:011
4486995428,20cyclictest261-21systemd-journal00:00:011
4486995421,27cyclictest0-21swapper/121:23:391
4486995413,27cyclictest0-21swapper/121:15:171
4486995413,26cyclictest0-21swapper/100:20:161
448499546,43cyclictest0-21swapper/000:15:270
4484995432,8cyclictest14-21ksoftirqd/020:35:010
448499542,32cyclictest151rcu_preempt00:35:400
4486995337,10cyclictest0-21swapper/122:05:261
4486995316,24cyclictest0-21swapper/121:50:171
448499535,42cyclictest0-21swapper/023:44:510
4484995337,9cyclictest14-21ksoftirqd/020:55:010
4484995333,8cyclictest14-21ksoftirqd/019:55:160
4484995331,8cyclictest14-21ksoftirqd/000:05:270
4484995325,22cyclictest14-21ksoftirqd/000:10:310
4484995321,28cyclictest14-21ksoftirqd/022:00:260
4484995317,17cyclictest27516-21sed21:45:010
448699526,10cyclictest0-21swapper/119:50:061
4486995223,14cyclictest26-21ksoftirqd/122:35:001
4484995236,10cyclictest14-21ksoftirqd/022:35:270
4484995227,19cyclictest12485-21/usr/sbin/munin21:00:370
4486995024,19cyclictest261-21systemd-journal22:01:261
448499505,40cyclictest0-21swapper/022:10:370
448499505,10cyclictest0-21swapper/023:10:290
4484995030,16cyclictest14-21ksoftirqd/000:25:200
4484995029,16cyclictest14-21ksoftirqd/021:35:010
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:233
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
none16:34:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional