You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 12:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Tue Nov 25, 2025 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5449712232172,18sleep40-21swapper/419:06:144
5450692230172,20sleep30-21swapper/319:07:333
5427882230166,21sleep70-21swapper/719:05:057
5449322212178,22sleep10-21swapper/119:05:391
5451992210176,21sleep20-21swapper/219:09:262
5451822209174,22sleep50-21swapper/519:09:135
5450902209176,21sleep60-21swapper/619:07:536
5451312208175,21sleep00-21swapper/019:08:270
54547799264,4cyclictest0-21swapper/723:00:257
54547799264,4cyclictest0-21swapper/700:06:197
54547799253,4cyclictest0-21swapper/723:33:227
54547799253,4cyclictest0-21swapper/721:54:317
54547799253,4cyclictest0-21swapper/721:32:337
54547799253,4cyclictest0-21swapper/721:32:337
54547799253,4cyclictest0-21swapper/720:59:367
54547299250,24cyclictest0-21swapper/521:10:145
54545499250,24cyclictest572219-21apt-get19:55:121
54547799245,18cyclictest0-21swapper/723:55:207
54547499240,23cyclictest0-21swapper/619:30:006
54547499240,1cyclictest563226-21munin-run19:40:006
54546799240,23cyclictest583233-21munin-run20:15:004
545463992422,1cyclictest646399-21turbostat22:05:003
54546399240,23cyclictest672795-21apt-get22:50:113
54546399240,23cyclictest623529-21munin-run21:25:003
545477992323,0cyclictest0-21swapper/719:55:117
54547799230,22cyclictest649666-21apt-get22:10:137
54547799230,22cyclictest557936-21apt-get19:30:147
54547799230,22cyclictest0-21swapper/722:00:117
545474992320,2cyclictest588962-21munin-run20:25:006
54547499230,2cyclictest683812-21munin-run23:10:016
54547499230,22cyclictest617797-21awk21:15:006
54547499230,22cyclictest0-21swapper/623:20:026
54547499230,22cyclictest0-21swapper/620:00:006
54547499230,1cyclictest26550irq/139-nvme0q723:30:016
54547299230,22cyclictest0-21swapper/522:20:115
54546799230,22cyclictest566514-21apt-get19:45:124
54546799230,22cyclictest0-21swapper/419:55:124
545463992322,1cyclictest566101-21perf19:45:003
545463992321,1cyclictest731295-21apt-get00:30:143
54546399230,22cyclictest0-21swapper/322:15:133
545459992322,1cyclictest609656-21apt-get21:00:112
54545999230,22cyclictest669944-21apt-get22:45:122
545454992322,1cyclictest692910-21apt-get23:25:011
54545499230,23cyclictest592088-21idleruntime-cro20:30:001
54545499230,23cyclictest0-21swapper/120:14:591
545452992323,0cyclictest0-21swapper/019:35:150
545452992321,1cyclictest606785-21apt-get20:55:130
54545299230,23cyclictest0-21swapper/019:49:590
54545299230,22cyclictest0-21swapper/022:20:110
54545299230,1cyclictest0-21swapper/022:35:000
545477992222,0cyclictest0-21swapper/721:40:147
545477992222,0cyclictest0-21swapper/700:00:137
545477992219,2cyclictest588954-21apt-get20:25:017
54547799220,22cyclictest0-21swapper/723:20:017
54547799220,21cyclictest0-21swapper/723:45:127
54547799220,21cyclictest0-21swapper/723:40:127
54547799220,21cyclictest0-21swapper/723:25:137
54547799220,21cyclictest0-21swapper/722:35:137
54547799220,21cyclictest0-21swapper/722:15:137
54547799220,21cyclictest0-21swapper/721:55:137
54547799220,21cyclictest0-21swapper/721:15:137
54547799220,21cyclictest0-21swapper/700:25:137
54547799220,1cyclictest595352-21apt-get20:35:117
54547799220,1cyclictest557517-21munin-run19:30:007
545474992220,1cyclictest0-21swapper/620:19:596
545474992220,1cyclictest0-21swapper/600:05:006
54547499220,21cyclictest0-21swapper/622:55:126
54547499220,21cyclictest0-21swapper/622:20:116
54547499220,1cyclictest0-21swapper/623:40:126
54547299220,22cyclictest0-21swapper/500:30:015
54547299220,21cyclictest0-21swapper/522:40:125
54547299220,1cyclictest0-21swapper/519:20:005
545467992221,1cyclictest692925-21awk23:25:014
545467992221,1cyclictest0-21swapper/422:00:124
545467992221,1cyclictest0-21swapper/420:30:004
545467992220,1cyclictest618217-21apt-get21:15:134
545467992220,1cyclictest575096-21apt-get20:00:134
54546799220,21cyclictest615364-21apt-get21:10:124
54546799220,21cyclictest0-21swapper/421:05:134
54546799220,21cyclictest0-21swapper/419:10:134
54546799220,21cyclictest0-21swapper/400:10:134
54546799220,21cyclictest0-21swapper/400:00:134
54546799220,1cyclictest678078-21turbostat23:00:014
54546399220,22cyclictest0-21swapper/323:20:013
54546399220,21cyclictest0-21swapper/300:29:593
545459992222,0cyclictest0-21swapper/223:30:142
545459992222,0cyclictest0-21swapper/221:20:122
545459992220,1cyclictest722188-21apt-get00:15:132
54545999220,22cyclictest563226-21munin-run19:40:002
54545999220,21cyclictest707880-21apt-get23:50:132
54545999220,21cyclictest0-21swapper/220:25:012
54545999220,21cyclictest0-21swapper/219:10:132
54545999220,1cyclictest678526-21apt-get23:00:142
545454992222,0cyclictest0-21swapper/122:00:111
545454992220,1cyclictest0-21swapper/123:55:131
545454992220,1cyclictest0-21swapper/120:15:141
54545499220,21cyclictest0-21swapper/123:05:121
54545499220,21cyclictest0-21swapper/120:30:121
54545499220,21cyclictest0-21swapper/119:30:141
54545499220,21cyclictest0-21swapper/119:25:131
54545499220,21cyclictest0-21swapper/100:05:141
54545499220,1cyclictest0-21swapper/123:30:011
54545499220,1cyclictest0-21swapper/121:05:131
545452992221,1cyclictest724628-21munin-run00:20:000
54545299220,22cyclictest0-21swapper/021:10:000
54545299220,21cyclictest0-21swapper/021:10:140
54545299220,1cyclictest549350-21apt-get19:15:120
545477992121,0cyclictest0-21swapper/722:40:127
545477992121,0cyclictest0-21swapper/720:50:137
545477992121,0cyclictest0-21swapper/720:15:007
545474992121,0cyclictest0-21swapper/623:45:126
545474992120,1cyclictest0-21swapper/600:10:136
545474992119,1cyclictest710731-21apt-get23:55:136
545474992119,1cyclictest621095-21apt-get21:20:126
54547499210,20cyclictest0-21swapper/620:25:016
54547499210,1cyclictest0-21swapper/600:25:006
54547499210,1cyclictest0-21swapper/600:15:136
545472992121,0cyclictest0-21swapper/522:25:125
545472992120,1cyclictest0-21swapper/519:50:005
545472992120,1cyclictest0-21swapper/500:20:005
54547299211,19cyclictest597772-21sh20:40:005
54547299210,21cyclictest563665-21apt-get19:40:145
54547299210,21cyclictest0-21swapper/522:15:135
54547299210,20cyclictest0-21swapper/500:10:135
54547299210,1cyclictest629656-21apt-get21:35:135
54547299210,1cyclictest629656-21apt-get21:35:135
54547299210,1cyclictest0-21swapper/521:25:135
54547299210,0cyclictest0-21swapper/520:50:135
545467992121,0cyclictest0-21swapper/420:30:124
545467992121,0cyclictest0-21swapper/400:20:144
545467992120,1cyclictest0-21swapper/420:55:134
545467992120,1cyclictest0-21swapper/419:20:004
545467992120,1cyclictest0-21swapper/400:30:014
545467992120,1cyclictest0-21swapper/400:20:004
545467992119,1cyclictest696449-21apt-get23:30:144
54546799210,20cyclictest0-21swapper/423:40:124
54546799210,20cyclictest0-21swapper/400:05:134
545463992121,0cyclictest0-21swapper/323:25:123
545463992121,0cyclictest0-21swapper/322:55:133
545463992121,0cyclictest0-21swapper/320:20:143
545463992121,0cyclictest0-21swapper/320:15:203
54546399210,20cyclictest545776-21apt-get19:10:133
54546399210,0cyclictest0-21swapper/323:00:143
54546399210,0cyclictest0-21swapper/322:35:133
54546399210,0cyclictest0-21swapper/319:15:123
545459992121,0cyclictest0-21swapper/222:10:132
545459992121,0cyclictest0-21swapper/221:15:002
545459992121,0cyclictest0-21swapper/200:30:142
545459992120,1cyclictest571805-21perf19:55:002
545459992120,0cyclictest0-21swapper/222:35:122
545459992119,1cyclictest683803-21apt-get23:10:022
54545999210,21cyclictest0-21swapper/220:45:192
54545999210,20cyclictest0-21swapper/220:10:122
545454992121,0cyclictest0-21swapper/120:55:121
545454992121,0cyclictest0-21swapper/100:30:011
545454992120,1cyclictest0-21swapper/123:30:141
545454992120,1cyclictest0-21swapper/122:15:131
545454992119,1cyclictest634962-21munin-run21:45:001
54545499210,20cyclictest0-21swapper/120:00:131
54545499210,0cyclictest0-21swapper/100:10:131
545452992121,0cyclictest0-21swapper/022:15:010
545452992121,0cyclictest0-21swapper/021:35:140
545452992121,0cyclictest0-21swapper/021:35:140
545452992121,0cyclictest0-21swapper/019:10:130
545452992120,1cyclictest0-21swapper/022:50:120
545452992119,1cyclictest687345-21apt-get23:15:140
545452992118,2cyclictest684357-21latency_hist23:10:010
54545299210,21cyclictest672371-21turbostat22:50:010
54545299210,21cyclictest0-21swapper/021:35:000
54545299210,21cyclictest0-21swapper/021:35:000
54545299210,20cyclictest0-21swapper/023:00:140
54545299210,0cyclictest0-21swapper/019:30:140
54545299210,0cyclictest0-21swapper/019:20:110
54545299210,0cyclictest0-21swapper/000:20:150
54547799203,0cyclictest0-21swapper/723:22:227
545477992020,0cyclictest0-21swapper/720:30:137
545477992020,0cyclictest0-21swapper/719:10:187
54547799200,0cyclictest0-21swapper/719:50:147
545474992020,0cyclictest0-21swapper/623:10:126
545474992020,0cyclictest0-21swapper/622:35:136
545474992020,0cyclictest0-21swapper/621:25:126
545474992020,0cyclictest0-21swapper/621:05:136
545474992020,0cyclictest0-21swapper/620:00:136
545474992019,1cyclictest667090-21apt-get22:40:126
54547499200,1cyclictest0-21swapper/622:30:106
54547499200,0cyclictest0-21swapper/620:55:136
545472992020,0cyclictest0-21swapper/522:35:135
545472992020,0cyclictest0-21swapper/520:15:155
545472992019,1cyclictest0-21swapper/521:20:005
545472992018,1cyclictest650-21dbus-daemon23:20:015
545467992020,0cyclictest0-21swapper/423:10:124
545467992020,0cyclictest0-21swapper/421:25:134
545467992020,0cyclictest0-21swapper/420:35:124
545467992020,0cyclictest0-21swapper/420:15:154
545467992019,0cyclictest658508-21apt-get22:25:124
545467992018,2cyclictest713228-21pluginstate00:00:004
54546799200,20cyclictest0-21swapper/423:25:024
545463992020,0cyclictest0-21swapper/323:35:143
545463992020,0cyclictest0-21swapper/320:25:123
545463992020,0cyclictest0-21swapper/320:05:123
545463992020,0cyclictest0-21swapper/300:20:013
545463992019,1cyclictest0-21swapper/320:45:143
545463992018,1cyclictest597778-21idleruntime-cro20:40:003
545459992020,0cyclictest0-21swapper/223:35:142
545459992020,0cyclictest0-21swapper/223:15:142
545459992020,0cyclictest0-21swapper/223:05:122
545459992020,0cyclictest0-21swapper/221:50:112
545459992020,0cyclictest0-21swapper/221:05:122
545454992020,0cyclictest0-21swapper/122:25:121
545454992020,0cyclictest0-21swapper/121:10:141
545454992020,0cyclictest0-21swapper/119:35:151
545454992020,0cyclictest0-21swapper/100:20:151
545454992019,1cyclictest0-21swapper/122:50:011
545454992019,0cyclictest0-21swapper/100:35:111
54545499200,0cyclictest0-21swapper/123:40:121
545452992020,0cyclictest0-21swapper/021:25:130
545452992020,0cyclictest0-21swapper/020:50:120
545452992020,0cyclictest0-21swapper/000:05:140
545452992018,1cyclictest0-21swapper/022:05:000
545477991919,0cyclictest0-21swapper/719:40:147
545477991918,1cyclictest0-21swapper/720:45:147
545474991919,0cyclictest0-21swapper/621:55:126
545474991919,0cyclictest0-21swapper/620:35:126
545474991919,0cyclictest0-21swapper/619:45:126
545474991919,0cyclictest0-21swapper/600:30:146
545474991918,1cyclictest0-21swapper/622:00:126
545474991918,1cyclictest0-21swapper/621:00:106
545474991918,1cyclictest0-21swapper/600:30:006
545472991919,0cyclictest0-21swapper/523:25:025
545472991919,0cyclictest0-21swapper/523:20:025
545472991919,0cyclictest0-21swapper/521:05:115
545472991919,0cyclictest0-21swapper/520:25:135
545472991918,1cyclictest681375-21apt-get23:05:125
545472991918,1cyclictest0-21swapper/521:50:115
545472991915,3cyclictest700620-21sshd23:35:215
54547299190,0cyclictest0-21swapper/522:00:115
545467991919,0cyclictest0-21swapper/422:15:124
545467991919,0cyclictest0-21swapper/419:35:154
545463991919,0cyclictest0-21swapper/322:40:113
545463991919,0cyclictest0-21swapper/321:55:123
545463991919,0cyclictest0-21swapper/321:40:143
545463991919,0cyclictest0-21swapper/320:30:133
545463991919,0cyclictest0-21swapper/320:10:133
545463991919,0cyclictest0-21swapper/319:45:123
545463991918,0cyclictest0-21swapper/323:20:023
545459991919,0cyclictest0-21swapper/220:35:122
545459991919,0cyclictest0-21swapper/200:20:142
545459991918,1cyclictest716483-21apt-get00:05:122
545459991918,1cyclictest678126-21idleruntime-cro23:00:012
545459991918,1cyclictest0-21swapper/223:55:132
545459991918,0cyclictest0-21swapper/221:35:132
545459991918,0cyclictest0-21swapper/221:35:122
545454991919,0cyclictest0-21swapper/119:40:141
545454991919,0cyclictest0-21swapper/119:10:131
545454991918,1cyclictest649253-21perf22:10:001
545454991918,1cyclictest598228-21apt-get20:40:131
545454991918,1cyclictest552230-21apt-get19:20:131
545452991919,0cyclictest0-21swapper/022:10:130
545452991919,0cyclictest0-21swapper/020:35:110
545452991919,0cyclictest0-21swapper/019:55:120
545452991919,0cyclictest0-21swapper/019:25:130
545452991918,1cyclictest696449-21apt-get23:30:130
545452991918,1cyclictest0-21swapper/021:25:000
545452991918,1cyclictest0-21swapper/020:10:130
545452991918,0cyclictest0-21swapper/021:50:100
545477991818,0cyclictest0-21swapper/723:50:127
545477991818,0cyclictest0-21swapper/723:10:177
545477991818,0cyclictest0-21swapper/722:55:137
545477991818,0cyclictest0-21swapper/722:05:127
545477991818,0cyclictest0-21swapper/721:45:137
545477991818,0cyclictest0-21swapper/720:05:127
545477991818,0cyclictest0-21swapper/719:20:127
545477991816,1cyclictest0-21swapper/723:40:007
545474991818,0cyclictest0-21swapper/623:30:146
545474991818,0cyclictest0-21swapper/622:15:136
545474991818,0cyclictest0-21swapper/621:35:136
545474991818,0cyclictest0-21swapper/621:35:136
545472991818,0cyclictest0-21swapper/523:00:135
545472991818,0cyclictest0-21swapper/522:30:105
545472991818,0cyclictest0-21swapper/522:10:125
545472991818,0cyclictest0-21swapper/520:45:145
545472991818,0cyclictest0-21swapper/500:00:135
545472991817,1cyclictest0-21swapper/520:40:135
545472991817,1cyclictest0-21swapper/520:05:125
545472991816,1cyclictest0-21swapper/522:45:135
54547299180,0cyclictest0-21swapper/519:10:135
545467991818,0cyclictest634967-21perf21:45:004
545467991818,0cyclictest0-21swapper/422:50:014
545467991818,0cyclictest0-21swapper/419:40:144
545467991818,0cyclictest0-21swapper/419:30:134
545467991818,0cyclictest0-21swapper/400:35:114
545467991817,1cyclictest0-21swapper/422:10:004
545467991817,1cyclictest0-21swapper/420:05:124
545467991816,2cyclictest586530-21apt-get20:20:144
545463991818,0cyclictest0-21swapper/323:30:143
545463991818,0cyclictest0-21swapper/323:10:023
545463991818,0cyclictest0-21swapper/322:25:173
545463991818,0cyclictest0-21swapper/321:35:133
545463991818,0cyclictest0-21swapper/321:35:133
545463991818,0cyclictest0-21swapper/321:30:133
545463991818,0cyclictest0-21swapper/321:30:133
545463991818,0cyclictest0-21swapper/321:25:133
545463991818,0cyclictest0-21swapper/300:10:133
545463991817,1cyclictest707460-21munin-run23:50:003
545459991818,0cyclictest0-21swapper/222:05:122
545459991817,1cyclictest574657-21munin-run20:00:002
545454991818,0cyclictest0-21swapper/120:05:121
54545499180,17cyclictest0-21swapper/122:50:121
545452991818,0cyclictest635380-21apt-get21:45:130
545452991818,0cyclictest0-21swapper/023:55:120
545452991818,0cyclictest0-21swapper/021:55:130
545452991818,0cyclictest0-21swapper/000:35:110
545452991818,0cyclictest0-21swapper/000:10:130
545452991817,1cyclictest0-21swapper/022:25:120
545477991717,0cyclictest0-21swapper/722:30:117
545477991717,0cyclictest0-21swapper/722:25:137
545477991717,0cyclictest0-21swapper/700:15:137
545474991717,0cyclictest0-21swapper/622:50:126
545474991717,0cyclictest0-21swapper/622:10:136
545474991717,0cyclictest0-21swapper/621:15:136
545474991717,0cyclictest0-21swapper/619:30:146
545474991717,0cyclictest0-21swapper/619:10:136
545474991717,0cyclictest0-21swapper/600:35:116
545474991716,0cyclictest638232-21apt-get21:50:106
54547499170,15cyclictest0-21swapper/621:40:186
54547499170,0cyclictest0-21swapper/623:35:146
545472991717,0cyclictest0-21swapper/523:10:125
545472991717,0cyclictest0-21swapper/522:05:125
545472991717,0cyclictest0-21swapper/521:20:125
545472991717,0cyclictest0-21swapper/520:30:125
545472991716,1cyclictest725329-21apt-get00:20:155
545472991716,1cyclictest0-21swapper/519:25:115
545472991715,1cyclictest632530-21apt-get21:40:145
545467991717,0cyclictest0-21swapper/422:50:124
545467991717,0cyclictest0-21swapper/420:40:134
545467991717,0cyclictest0-21swapper/419:20:134
545463991717,0cyclictest0-21swapper/323:50:133
545463991717,0cyclictest0-21swapper/320:40:133
545463991717,0cyclictest0-21swapper/320:00:133
545463991717,0cyclictest0-21swapper/319:50:143
545463991716,1cyclictest713629-21apt-get00:00:123
545459991717,0cyclictest0-21swapper/223:20:112
545459991717,0cyclictest0-21swapper/222:20:102
545459991717,0cyclictest0-21swapper/220:55:132
545459991717,0cyclictest0-21swapper/219:20:132
545459991717,0cyclictest0-21swapper/200:25:122
545459991716,1cyclictest643553-21sendmail21:59:592
545454991717,0cyclictest0-21swapper/123:45:111
545454991717,0cyclictest0-21swapper/123:35:141
545454991717,0cyclictest0-21swapper/120:20:141
545454991717,0cyclictest0-21swapper/119:45:121
545454991716,1cyclictest0-21swapper/123:10:021
545452991717,0cyclictest0-21swapper/023:50:120
545452991717,0cyclictest0-21swapper/022:55:130
545452991717,0cyclictest0-21swapper/021:40:140
545452991717,0cyclictest0-21swapper/020:25:010
545452991716,1cyclictest0-21swapper/022:35:130
545452991716,1cyclictest0-21swapper/000:25:130
545452991715,2cyclictest0-21swapper/020:35:000
545452991715,1cyclictest566097-21munin-run19:45:000
545477991616,0cyclictest0-21swapper/720:20:137
545477991616,0cyclictest0-21swapper/720:15:207
545477991616,0cyclictest0-21swapper/719:15:127
545477991616,0cyclictest0-21swapper/700:35:117
545477991616,0cyclictest0-21swapper/700:30:147
54547799160,15cyclictest0-21swapper/722:48:577
54547799160,15cyclictest0-21swapper/721:35:577
54547799160,15cyclictest0-21swapper/721:35:567
54547799160,15cyclictest0-21swapper/720:40:507
545474991616,0cyclictest0-21swapper/619:40:146
545474991615,1cyclictest707880-21apt-get23:50:126
545474991615,1cyclictest594954-21awk20:35:006
545474991615,1cyclictest0-21swapper/621:35:006
545474991615,1cyclictest0-21swapper/621:35:006
545474991615,1cyclictest0-21swapper/600:05:146
54547499160,15cyclictest0-21swapper/623:16:386
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional