You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 04:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Thu Nov 20, 2025 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3211402231167,20sleep50-21swapper/519:07:255
3212242219186,21sleep70-21swapper/719:08:407
3210462215181,22sleep10-21swapper/119:06:011
3211192213174,24sleep20-21swapper/219:07:062
3212892211179,21sleep00-21swapper/019:09:370
3211832210179,20sleep30-21swapper/319:08:043
3211142205169,22sleep60-21swapper/619:07:026
3211212200167,20sleep40-21swapper/419:07:084
32157199298,20cyclictest437251-21munin-run23:00:005
32157199290,2cyclictest481199-21basename00:14:595
321557992926,2cyclictest2278112-21in:imuxsock21:40:000
32157199270,26cyclictest457775-21apt-get23:35:015
32156899260,25cyclictest0-21swapper/419:40:004
32157499250,24cyclictest0-21swapper/621:55:126
321571992520,4cyclictest0-21swapper/521:20:005
32157199250,1cyclictest379487-21apt-get21:20:115
32157199250,1cyclictest0-21swapper/519:25:005
32156099250,24cyclictest355924-21grep20:40:001
32156099250,24cyclictest355924-21grep20:40:001
32157499240,23cyclictest396219-21sleep21:50:016
321571992420,3cyclictest0-21swapper/521:00:005
32157199240,1cyclictest0-21swapper/521:15:005
32157199240,0cyclictest0-21swapper/523:50:155
32156699240,23cyclictest0-21swapper/320:20:143
32156299240,24cyclictest0-21swapper/223:20:002
321560992423,1cyclictest466621-21munin-run23:50:001
32156099240,24cyclictest495493-21awk00:40:001
32155799241,22cyclictest2278114-21rs:main0
32155799241,22cyclictest2278114-21rs:main0
32155799240,24cyclictest0-21swapper/023:25:150
321577992323,0cyclictest0-21swapper/720:00:157
32157499230,22cyclictest358774-21cron20:45:006
32157499230,22cyclictest358774-21cron20:45:006
321571992318,4cyclictest399066-21perf21:55:015
32157199230,22cyclictest472623-21awk00:00:005
32157199230,22cyclictest414047-21apt-get22:20:125
32157199230,22cyclictest0-21swapper/522:05:125
32156899230,22cyclictest484059-21perf00:19:594
32156899230,22cyclictest0-21swapper/400:25:134
321566992321,1cyclictest0-21swapper/320:30:163
32156699230,22cyclictest0-21swapper/322:20:013
32156699230,22cyclictest0-21swapper/321:00:153
32156299231,1cyclictest401rcuc/219:35:002
32156299230,23cyclictest364514-21awk20:55:002
32156299230,22cyclictest0-21swapper/200:30:162
32156099230,22cyclictest69450irq/124-eno122:40:161
32156099230,22cyclictest0-21swapper/120:55:101
321557992322,1cyclictest0-21swapper/021:25:150
321557992321,1cyclictest0-21swapper/019:40:000
32155799230,22cyclictest470154-21apt-get23:55:120
32155799230,1cyclictest434128-21apt-get22:55:010
32155799230,1cyclictest0-21swapper/000:40:000
321577992220,1cyclictest0-21swapper/719:50:007
32157799220,21cyclictest449123-21apt-get23:20:117
32157799220,21cyclictest0-21swapper/721:15:117
32157799220,21cyclictest0-21swapper/720:10:157
32157799220,1cyclictest455352-21apt-get23:30:137
32157799220,1cyclictest0-21swapper/721:39:597
32157799220,1cyclictest0-21swapper/721:10:007
321574992220,1cyclictest344932-21apt-get20:20:146
32157499220,21cyclictest493056-21apt-get00:35:136
32157499220,21cyclictest475909-21apt-get00:05:136
32157499220,21cyclictest466663-21systemd-run23:50:006
32157499220,21cyclictest0-21swapper/623:05:006
32157499220,1cyclictest404787-21munin-run22:05:016
32157499220,1cyclictest322167-21latency_hist19:15:006
321571992221,1cyclictest0-21swapper/521:45:105
321571992220,1cyclictest353486-21apt-get20:35:125
321571992220,1cyclictest353486-21apt-get20:35:125
321571992220,1cyclictest0-21swapper/523:00:125
321571992219,3cyclictest440540-21apt-get23:05:115
32157199220,21cyclictest452241-21apt-get23:25:145
32157199220,21cyclictest0-21swapper/523:15:125
32157199220,21cyclictest0-21swapper/522:50:115
32157199220,21cyclictest0-21swapper/522:15:135
32157199220,1cyclictest390476-21sh21:40:005
32157199220,1cyclictest0-21swapper/500:05:005
321568992222,0cyclictest0-21swapper/422:15:134
32156899220,21cyclictest0-21swapper/420:20:004
32156899220,1cyclictest328560-21apt-get19:50:214
32156899220,1cyclictest0-21swapper/423:35:114
321566992221,1cyclictest0-21swapper/321:45:103
321566992221,1cyclictest0-21swapper/321:15:003
321566992220,1cyclictest443392-21apt-get23:10:113
32156699220,21cyclictest454664-21apt-get23:30:013
32156699220,21cyclictest0-21swapper/322:00:143
32156699220,21cyclictest0-21swapper/319:55:073
32156699220,21cyclictest0-21swapper/300:10:133
32156699220,1cyclictest0-21swapper/323:50:203
32156699220,15cyclictest0-21swapper/322:54:103
32156699220,15cyclictest0-21swapper/300:24:413
32156299220,22cyclictest0-21swapper/223:50:002
32156299220,22cyclictest0-21swapper/222:15:012
32156299220,22cyclictest0-21swapper/220:00:002
321560992222,0cyclictest0-21swapper/121:30:111
321560992221,1cyclictest650-21dbus-daemon19:40:001
32156099220,22cyclictest353085-21awk20:35:001
32156099220,21cyclictest0-21swapper/123:20:111
32156099220,21cyclictest0-21swapper/121:05:131
32156099220,21cyclictest0-21swapper/100:25:131
32156099220,1cyclictest393772-21apt-get21:45:101
32156099220,1cyclictest0-21swapper/122:35:001
32156099220,1cyclictest0-21swapper/122:20:021
321557992222,0cyclictest0-21swapper/021:30:110
321557992221,1cyclictest0-21swapper/000:25:120
32155799220,22cyclictest0-21swapper/020:00:000
32155799220,21cyclictest466613-21apt-get23:50:010
32155799220,21cyclictest0-21swapper/023:10:120
32155799220,21cyclictest0-21swapper/021:50:130
32155799220,0cyclictest0-21swapper/020:30:160
321577992121,0cyclictest0-21swapper/723:50:157
321577992121,0cyclictest0-21swapper/721:40:117
321577992121,0cyclictest0-21swapper/719:15:007
321577992120,1cyclictest353067-21munin-run20:35:007
32157799210,20cyclictest0-21swapper/722:20:127
321574992121,0cyclictest0-21swapper/622:45:126
321574992121,0cyclictest0-21swapper/621:00:136
321574992121,0cyclictest0-21swapper/620:55:106
321574992121,0cyclictest0-21swapper/620:35:136
321574992121,0cyclictest0-21swapper/620:35:136
321574992121,0cyclictest0-21swapper/620:00:156
321574992120,1cyclictest0-21swapper/623:30:006
321574992120,1cyclictest0-21swapper/600:05:006
321574992119,1cyclictest484490-21apt-get00:20:116
321574992119,1cyclictest437252-21awk23:00:006
321574992119,1cyclictest388047-21apt-get21:35:126
32157499210,0cyclictest0-21swapper/621:40:116
321571992121,0cyclictest0-21swapper/523:40:085
321571992120,1cyclictest0-21swapper/522:00:145
321571992119,2cyclictest0-21swapper/522:45:015
321571992119,2cyclictest0-21swapper/520:30:165
32157199210,20cyclictest0-21swapper/522:35:125
321568992121,0cyclictest0-21swapper/421:40:114
321568992120,1cyclictest427904-21apt-get22:45:014
321568992120,1cyclictest0-21swapper/423:45:174
32156899210,21cyclictest0-21swapper/420:35:004
32156899210,20cyclictest0-21swapper/420:35:134
32156899210,20cyclictest0-21swapper/420:35:134
32156899210,0cyclictest0-21swapper/423:00:114
321566992121,0cyclictest0-21swapper/321:00:003
321566992120,1cyclictest0-21swapper/322:10:013
321566992119,1cyclictest325876-21turbostat19:35:003
321566992119,1cyclictest0-21swapper/321:25:003
32156699210,20cyclictest0-21swapper/323:55:123
32156699210,20cyclictest0-21swapper/320:35:123
32156699210,20cyclictest0-21swapper/320:35:123
321562992121,0cyclictest0-21swapper/222:50:112
321562992121,0cyclictest0-21swapper/222:30:122
321562992120,1cyclictest401rcuc/221:35:112
321562992120,1cyclictest0-21swapper/222:20:012
32156299210,21cyclictest0-21swapper/223:40:092
32156299210,21cyclictest0-21swapper/222:25:112
32156299210,21cyclictest0-21swapper/200:20:122
32156299210,20cyclictest0-21swapper/221:05:132
32156299210,0cyclictest0-21swapper/223:35:012
321560992121,0cyclictest0-21swapper/121:25:151
321560992121,0cyclictest0-21swapper/119:55:071
321560992118,2cyclictest390489-21awk21:40:001
32156099210,21cyclictest428595-21apt-get22:45:121
32156099210,21cyclictest311rcuc/121:15:001
32156099210,21cyclictest0-21swapper/122:00:141
32156099210,21cyclictest0-21swapper/121:50:131
32156099210,0cyclictest0-21swapper/123:50:011
32156099210,0cyclictest0-21swapper/123:35:001
32156099210,0cyclictest0-21swapper/123:15:121
32156099210,0cyclictest0-21swapper/122:55:141
321557992121,0cyclictest0-21swapper/022:15:130
321557992119,1cyclictest0-21swapper/023:20:110
32155799210,1cyclictest481613-21apt-get00:15:120
32155799210,1cyclictest0-21swapper/019:50:210
32155799210,0cyclictest0-21swapper/023:45:170
32155799210,0cyclictest0-21swapper/023:15:120
32155799210,0cyclictest0-21swapper/021:14:590
32155799210,0cyclictest0-21swapper/021:05:130
32155799210,0cyclictest0-21swapper/020:55:100
321577992020,0cyclictest0-21swapper/722:50:117
321577992020,0cyclictest0-21swapper/720:35:127
321577992020,0cyclictest0-21swapper/720:35:127
321577992019,1cyclictest0-21swapper/720:05:127
32157799200,19cyclictest0-21swapper/719:50:217
321574992020,0cyclictest0-21swapper/623:20:116
321574992020,0cyclictest0-21swapper/600:30:166
321574992020,0cyclictest0-21swapper/600:10:136
32157499200,20cyclictest0-21swapper/622:50:116
32157499200,19cyclictest0-21swapper/620:30:006
32157499200,0cyclictest0-21swapper/620:05:126
321571992020,0cyclictest0-21swapper/522:30:135
321571992020,0cyclictest0-21swapper/520:25:135
321571992020,0cyclictest0-21swapper/520:20:155
321571992020,0cyclictest0-21swapper/500:15:125
321571992019,0cyclictest0-21swapper/522:10:095
321571992018,2cyclictest0-21swapper/520:45:125
321571992015,4cyclictest0-21swapper/521:00:125
32157199200,1cyclictest0-21swapper/522:45:005
32157199200,19cyclictest339203-21apt-get20:10:155
321568992020,0cyclictest0-21swapper/423:15:124
321568992020,0cyclictest0-21swapper/422:55:144
321568992020,0cyclictest0-21swapper/420:25:134
321568992019,1cyclictest0-21swapper/421:20:004
321568992019,1cyclictest0-21swapper/421:15:004
321568992018,1cyclictest325893-21awk19:35:004
321568992018,1cyclictest0-21swapper/420:05:004
32156899200,19cyclictest405205-21apt-get22:05:124
32156899200,0cyclictest0-21swapper/422:20:124
32156899200,0cyclictest0-21swapper/421:55:124
32156899200,0cyclictest0-21swapper/420:40:134
32156899200,0cyclictest0-21swapper/420:40:134
321566992020,0cyclictest0-21swapper/323:15:133
321566992020,0cyclictest0-21swapper/322:20:123
321566992020,0cyclictest0-21swapper/320:50:153
321566992020,0cyclictest0-21swapper/300:05:133
321566992019,1cyclictest0-21swapper/321:45:013
321566992018,2cyclictest325135-21dump-pmu-power19:30:003
32156699200,0cyclictest0-21swapper/323:05:123
321562992020,0cyclictest0-21swapper/223:25:152
321562992020,0cyclictest0-21swapper/222:45:012
321562992020,0cyclictest0-21swapper/221:40:112
321562992020,0cyclictest0-21swapper/221:20:112
321562992020,0cyclictest0-21swapper/221:10:142
321562992020,0cyclictest0-21swapper/200:00:122
321560992020,0cyclictest0-21swapper/122:25:101
321560992020,0cyclictest0-21swapper/100:00:111
321560992019,1cyclictest0-21swapper/121:55:121
321560992019,1cyclictest0-21swapper/120:20:151
321560992019,0cyclictest0-21swapper/121:15:021
321560992018,2cyclictest322417-21latency_hist19:15:001
321560992018,1cyclictest478762-21apt-get00:10:131
321560992018,1cyclictest0-21swapper/123:30:131
32156099200,20cyclictest0-21swapper/123:10:111
32156099200,20cyclictest0-21swapper/122:50:111
32156099200,19cyclictest390923-21apt-get21:40:111
32156099200,0cyclictest0-21swapper/120:00:151
321557992020,0cyclictest0-21swapper/022:20:120
321557992020,0cyclictest0-21swapper/021:55:120
321557992020,0cyclictest0-21swapper/020:20:150
321557992020,0cyclictest0-21swapper/000:00:120
321557992019,1cyclictest0-21swapper/023:40:000
321557992019,1cyclictest0-21swapper/023:00:120
321557992018,1cyclictest0-21swapper/022:35:000
32155799200,0cyclictest0-21swapper/022:50:100
321577991919,0cyclictest0-21swapper/723:25:157
321577991919,0cyclictest0-21swapper/722:45:127
321577991919,0cyclictest0-21swapper/721:20:117
321577991919,0cyclictest0-21swapper/720:50:157
321577991918,1cyclictest419755-21apt-get22:30:137
321577991918,1cyclictest347783-21apt-get20:25:137
32157799190,6cyclictest0-21swapper/722:55:207
321574991919,0cyclictest0-21swapper/623:10:116
321574991919,0cyclictest0-21swapper/622:40:166
321574991919,0cyclictest0-21swapper/622:35:126
321574991919,0cyclictest0-21swapper/622:20:026
321574991919,0cyclictest0-21swapper/600:25:136
321574991917,1cyclictest461331-21apt-get23:40:076
321571991919,0cyclictest0-21swapper/519:55:075
321571991918,0cyclictest0-21swapper/520:50:155
321571991917,1cyclictest327381-21munin-run19:45:005
32157199190,18cyclictest0-21swapper/500:05:135
321568991919,0cyclictest0-21swapper/421:30:114
321568991918,1cyclictest367792-21apt-get21:00:134
321568991917,1cyclictest470154-21apt-get23:55:124
321568991917,1cyclictest361662-21basename20:50:004
32156899190,18cyclictest0-21swapper/421:30:004
321566991919,0cyclictest0-21swapper/322:55:143
321566991919,0cyclictest0-21swapper/321:55:123
321566991919,0cyclictest0-21swapper/321:15:113
321566991919,0cyclictest0-21swapper/320:10:153
321566991918,1cyclictest422607-21apt-get22:35:123
321566991918,1cyclictest0-21swapper/319:54:593
32156699190,0cyclictest0-21swapper/320:15:133
321562991919,0cyclictest0-21swapper/223:20:112
321562991919,0cyclictest0-21swapper/221:19:242
321562991918,1cyclictest0-21swapper/222:00:012
321562991918,1cyclictest0-21swapper/221:00:372
321562991917,2cyclictest323659-21awk19:20:002
321562991917,1cyclictest367330-21cron21:00:002
321560991919,0cyclictest0-21swapper/123:00:121
321560991918,1cyclictest0-21swapper/123:05:111
321560991916,2cyclictest463759-21dump-pmu-power23:45:001
321557991919,0cyclictest0-21swapper/022:40:160
321557991919,0cyclictest0-21swapper/022:00:130
321557991919,0cyclictest0-21swapper/021:15:020
321557991918,0cyclictest0-21swapper/023:30:130
32155799190,19cyclictest0-21swapper/019:45:000
321577991818,0cyclictest0-21swapper/721:25:157
321577991818,0cyclictest0-21swapper/700:35:137
321577991818,0cyclictest0-21swapper/700:30:167
321577991818,0cyclictest0-21swapper/700:10:137
321577991817,1cyclictest0-21swapper/723:55:127
321574991818,0cyclictest0-21swapper/621:20:116
321574991818,0cyclictest0-21swapper/620:15:146
321574991817,1cyclictest330612-21apt-get19:55:076
321571991818,0cyclictest0-21swapper/520:05:125
32157199180,1cyclictest445826-21munin-run23:15:005
32157199180,13cyclictest0-21swapper/500:25:125
321568991818,0cyclictest0-21swapper/423:25:014
321568991818,0cyclictest0-21swapper/422:30:134
321568991818,0cyclictest0-21swapper/421:20:104
321568991818,0cyclictest0-21swapper/420:50:154
321568991817,1cyclictest495512-21plymouth00:40:004
321568991817,1cyclictest0-21swapper/400:15:004
321566991818,0cyclictest0-21swapper/323:35:013
321566991817,1cyclictest0-21swapper/319:15:003
321562991818,0cyclictest0-21swapper/221:25:152
321562991818,0cyclictest0-21swapper/220:35:132
321562991818,0cyclictest0-21swapper/220:35:132
321562991818,0cyclictest0-21swapper/219:23:272
321562991818,0cyclictest0-21swapper/200:10:122
321562991817,1cyclictest411ktimers/222:35:182
32156299180,1cyclictest0-21swapper/200:05:132
321560991817,1cyclictest0-21swapper/122:10:101
32156099180,17cyclictest0-21swapper/120:05:121
321557991818,0cyclictest0-21swapper/023:05:110
321557991818,0cyclictest0-21swapper/021:45:100
321557991818,0cyclictest0-21swapper/019:15:000
321557991818,0cyclictest0-21swapper/000:10:130
321557991817,1cyclictest0-21swapper/021:05:000
321557991817,1cyclictest0-21swapper/020:10:000
321557991817,0cyclictest0-21swapper/020:50:150
321557991816,2cyclictest463757-21idleruntime-cro23:45:000
321577991717,0cyclictest0-21swapper/722:15:137
321577991717,0cyclictest0-21swapper/700:25:137
321577991717,0cyclictest0-21swapper/700:15:127
32157799170,0cyclictest0-21swapper/723:10:127
321574991717,0cyclictest0-21swapper/622:05:116
321574991717,0cyclictest0-21swapper/621:15:026
321574991717,0cyclictest0-21swapper/620:45:126
321574991717,0cyclictest0-21swapper/600:00:006
321574991715,1cyclictest0-21swapper/619:55:016
32157199173,13cyclictest0-21swapper/520:00:145
321571991717,0cyclictest0-21swapper/521:30:115
321571991717,0cyclictest0-21swapper/521:05:135
32157199170,1cyclictest0-21swapper/520:40:195
32157199170,1cyclictest0-21swapper/520:40:195
32157199170,16cyclictest0-21swapper/519:47:525
32157199170,16cyclictest0-21swapper/519:36:465
32157199170,16cyclictest0-21swapper/500:20:115
32157199170,0cyclictest0-21swapper/520:15:145
321568991717,0cyclictest0-21swapper/422:50:114
321568991717,0cyclictest0-21swapper/419:55:014
321568991716,1cyclictest473054-21apt-get00:00:124
321566991717,0cyclictest0-21swapper/323:00:123
321562991717,0cyclictest0-21swapper/223:30:112
321562991717,0cyclictest0-21swapper/222:05:122
321562991717,0cyclictest0-21swapper/221:50:132
321562991717,0cyclictest0-21swapper/200:25:132
321562991716,1cyclictest0-21swapper/222:40:162
321562991715,2cyclictest0-21swapper/223:50:152
321562991715,2cyclictest0-21swapper/222:20:012
32156299170,1cyclictest0-21swapper/219:50:002
321560991717,0cyclictest0-21swapper/123:55:121
321560991717,0cyclictest0-21swapper/120:50:151
321560991716,1cyclictest452241-21apt-get23:25:151
321560991716,1cyclictest0-21swapper/121:05:001
321557991717,0cyclictest0-21swapper/022:10:100
321557991717,0cyclictest0-21swapper/020:00:150
321557991716,1cyclictest344490-21munin-run20:20:000
321557991716,1cyclictest0-21swapper/022:05:120
321557991716,1cyclictest0-21swapper/000:05:130
321557991716,0cyclictest0-21swapper/021:40:110
321577991616,0cyclictest0-21swapper/720:20:147
321577991616,0cyclictest0-21swapper/719:55:077
321577991616,0cyclictest0-21swapper/700:20:117
321574991616,0cyclictest0-21swapper/623:35:016
321574991616,0cyclictest0-21swapper/623:30:016
321574991616,0cyclictest0-21swapper/621:10:146
321574991616,0cyclictest0-21swapper/620:30:156
32157499160,1cyclictest385194-21apt-get21:30:116
32157199160,1cyclictest0-21swapper/500:35:195
32157199160,16cyclictest0-21swapper/521:27:255
32157199160,15cyclictest0-21swapper/521:40:485
32157199160,15cyclictest0-21swapper/519:50:435
32157199160,15cyclictest0-21swapper/519:32:325
32157199160,14cyclictest0-21swapper/522:25:405
32157199160,14cyclictest0-21swapper/519:15:555
32157199160,0cyclictest0-21swapper/519:28:265
32157199160,0cyclictest0-21swapper/500:30:095
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional