You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 12:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Thu Apr 30, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2566802244173,22sleep00-21swapper/019:09:130
2564772240168,60sleep10-21swapper/119:06:171
2567142228166,19sleep70-21swapper/719:09:447
2565722226162,18sleep40-21swapper/419:07:414
2542162217183,23sleep60-21swapper/619:05:096
2564502215181,22sleep30-21swapper/319:05:553
2565372213184,19sleep50-21swapper/519:07:095
2565052212182,20sleep20-21swapper/219:06:422
256948992723,3cyclictest281203-21idleruntime-cro19:50:016
256948992723,3cyclictest281203-21idleruntime-cro19:50:016
25693899270,25cyclictest45-21ksoftirqd/219:50:012
25693899270,25cyclictest45-21ksoftirqd/219:50:012
25694899260,25cyclictest366-21systemd-journal19:40:016
25694499260,25cyclictest400358-21awk00:00:014
25693899260,24cyclictest407823-21idleruntime-cro00:20:002
256950992521,3cyclictest377246-21turbostat22:55:017
25694899250,24cyclictest407017-21apt-get00:15:056
25694799250,24cyclictest346072-21grep21:40:025
256944992520,4cyclictest385454-21dump-pmu-power23:20:014
256938992521,3cyclictest45-21ksoftirqd/220:15:122
256934992520,4cyclictest263508-21perf19:20:010
25693499250,1cyclictest352395-21apt-get21:50:140
256950992422,1cyclictest0-21swapper/719:45:017
25695099240,24cyclictest346066-21turbostat21:40:027
25695099240,1cyclictest0-21swapper/719:50:027
25695099240,1cyclictest0-21swapper/719:50:017
256948992420,3cyclictest384680-21idleruntime-cro23:15:016
25694899240,23cyclictest354902-21apt-get21:55:046
25694799240,2cyclictest769-21dbus-daemon00:40:015
256944992423,1cyclictest0-21swapper/421:30:134
25694499240,23cyclictest0-21swapper/423:40:134
25694499240,23cyclictest0-21swapper/422:20:014
25694499240,23cyclictest0-21swapper/421:35:134
256942992422,1cyclictest385895-21apt-get23:20:133
25694299240,24cyclictest0-21swapper/321:40:173
25694299240,23cyclictest359744-21tar22:10:013
256938992421,2cyclictest269400-21ls19:30:012
25693899240,23cyclictest287111-21idleruntime-cro20:00:012
25693899240,23cyclictest287111-21idleruntime-cro20:00:002
25693899240,23cyclictest0-21swapper/220:30:122
25693899240,23cyclictest0-21swapper/220:20:142
25693699240,24cyclictest0-21swapper/100:20:131
256950992322,1cyclictest281256-21appstreamcli19:50:047
256950992322,1cyclictest281256-21appstreamcli19:50:047
25695099230,23cyclictest0-21swapper/721:20:187
25695099230,23cyclictest0-21swapper/719:55:127
25695099230,23cyclictest0-21swapper/719:55:127
25695099230,22cyclictest346513-21apt-get21:40:127
25695099230,22cyclictest293442-21apt-get20:10:157
25695099230,22cyclictest0-21swapper/721:30:127
25695099230,21cyclictest0-21swapper/722:10:037
25695099230,1cyclictest378025-21munin-run23:00:017
25695099230,1cyclictest0-21swapper/721:20:017
256948992323,0cyclictest0-21swapper/620:40:196
25694899230,23cyclictest0-21swapper/623:40:196
25694899230,22cyclictest0-21swapper/600:00:206
25694799230,22cyclictest322506-21awk21:00:025
25694799230,22cyclictest0-21swapper/520:40:155
25694799230,22cyclictest0-21swapper/520:10:195
25694799230,22cyclictest0-21swapper/500:00:205
256944992321,1cyclictest0-21swapper/423:30:134
256944992320,2cyclictest0-21swapper/400:40:014
25694499230,22cyclictest371036-21apt-get22:40:134
25694499230,22cyclictest284164-21ls19:55:014
25694499230,22cyclictest284164-21ls19:55:004
25694499230,22cyclictest0-21swapper/422:50:124
25694499230,22cyclictest0-21swapper/421:45:144
25694499230,22cyclictest0-21swapper/420:40:144
25694499230,22cyclictest0-21swapper/420:20:194
256942992322,1cyclictest0-21swapper/321:10:133
256942992322,1cyclictest0-21swapper/320:10:153
256942992322,0cyclictest0-21swapper/323:40:143
25694299230,23cyclictest0-21swapper/320:45:223
25694299230,22cyclictest0-21swapper/323:00:133
25694299230,22cyclictest0-21swapper/320:25:143
25693899236,16cyclictest387281-21ntpq23:20:202
256938992323,0cyclictest0-21swapper/223:00:132
256938992323,0cyclictest0-21swapper/222:40:122
25693899230,22cyclictest392876-21awk23:40:012
25693899230,22cyclictest0-21swapper/222:55:012
25693899230,22cyclictest0-21swapper/221:25:142
25693899230,22cyclictest0-21swapper/220:55:142
25693899230,22cyclictest0-21swapper/200:30:122
256936992323,0cyclictest0-21swapper/121:35:141
256936992323,0cyclictest0-21swapper/100:30:041
256936992322,1cyclictest0-21swapper/120:40:011
25693699230,23cyclictest407090-21appstreamcli00:15:021
25693699230,23cyclictest0-21swapper/121:50:191
25693699230,22cyclictest334713-21apt-get21:20:121
25693699230,22cyclictest0-21swapper/122:10:031
25693699230,22cyclictest0-21swapper/120:30:111
256934992322,1cyclictest359593-21cat22:10:010
256934992321,1cyclictest284596-21apt-get19:55:130
256934992321,1cyclictest284596-21apt-get19:55:130
256934992321,1cyclictest0-21swapper/019:45:180
256934992321,1cyclictest0-21swapper/019:45:170
256934992320,2cyclictest411648-21appstreamcli00:30:020
25693499230,23cyclictest404084-21munin-run00:10:010
25693499230,22cyclictest396647-21munin-run23:50:010
25693499230,22cyclictest396647-21munin-run23:50:010
25693499230,1cyclictest308186-21apt-get20:35:140
256950992221,1cyclictest0-21swapper/722:20:017
256950992221,1cyclictest0-21swapper/720:35:157
256950992220,1cyclictest0-21swapper/720:45:137
25695099220,21cyclictest0-21swapper/723:50:177
25695099220,21cyclictest0-21swapper/722:40:127
25695099220,21cyclictest0-21swapper/719:35:137
25695099220,21cyclictest0-21swapper/700:00:137
25695099220,1cyclictest328809-21apt-get21:10:147
25695099220,1cyclictest0-21swapper/721:05:157
25695099220,0cyclictest0-21swapper/723:10:117
25695099220,0cyclictest0-21swapper/700:30:117
256948992222,0cyclictest0-21swapper/600:20:046
256948992220,1cyclictest296382-21apt-get20:15:126
256948992218,3cyclictest769-21dbus-daemon20:00:016
256948992218,3cyclictest769-21dbus-daemon20:00:016
25694899220,22cyclictest334285-21systemctl21:20:026
25694899220,21cyclictest0-21swapper/621:40:126
25694899220,21cyclictest0-21swapper/621:35:136
25694899220,21cyclictest0-21swapper/621:00:136
256947992222,0cyclictest0-21swapper/500:20:135
256947992221,1cyclictest0-21swapper/521:05:225
256947992220,1cyclictest0-21swapper/521:55:015
256947992219,2cyclictest407853-21sleep00:20:015
25694799220,21cyclictest341891-21ntpq21:30:175
25694799220,21cyclictest0-21swapper/523:40:145
25694799220,21cyclictest0-21swapper/521:55:035
25694799220,21cyclictest0-21swapper/521:45:145
25694799220,21cyclictest0-21swapper/521:10:145
25694799220,21cyclictest0-21swapper/519:50:125
25694799220,21cyclictest0-21swapper/519:50:125
25694799220,21cyclictest0-21swapper/519:45:195
25694799220,21cyclictest0-21swapper/519:45:195
25694799220,21cyclictest0-21swapper/519:15:145
25694799220,1cyclictest314089-21apt-get20:45:145
256944992220,1cyclictest0-21swapper/420:05:144
25694499220,22cyclictest0-21swapper/420:05:014
25694499220,22cyclictest0-21swapper/420:05:014
25694499220,21cyclictest0-21swapper/420:35:154
25694499220,21cyclictest0-21swapper/420:15:124
25694499220,21cyclictest0-21swapper/419:55:134
25694499220,21cyclictest0-21swapper/419:55:134
25694499220,21cyclictest0-21swapper/419:20:124
25694499220,21cyclictest0-21swapper/400:30:044
25694499220,1cyclictest400800-21apt-get00:00:124
256942992222,0cyclictest0-21swapper/320:35:153
256942992222,0cyclictest0-21swapper/319:30:133
256942992221,1cyclictest852-21rs:main3
256942992221,1cyclictest343551-21apt-get21:35:133
256942992221,1cyclictest0-21swapper/319:50:023
256942992221,1cyclictest0-21swapper/319:50:023
25694299220,22cyclictest408307-21apt-get00:20:133
25694299220,21cyclictest325871-21apt-get21:05:143
25694299220,21cyclictest0-21swapper/323:50:123
25694299220,21cyclictest0-21swapper/320:55:213
25694299220,21cyclictest0-21swapper/320:50:133
25694299220,21cyclictest0-21swapper/319:10:123
25694299220,1cyclictest389598-21apt-get23:30:133
25694299220,1cyclictest287560-21apt-get20:00:133
25694299220,1cyclictest287560-21apt-get20:00:133
25694299220,0cyclictest0-21swapper/319:20:123
256938992221,1cyclictest0-21swapper/219:50:182
256938992221,1cyclictest0-21swapper/219:50:182
256938992221,1cyclictest0-21swapper/219:40:142
256938992221,1cyclictest0-21swapper/200:10:132
256938992220,1cyclictest396643-21turbostat23:50:012
256938992220,1cyclictest396643-21turbostat23:50:012
256938992220,1cyclictest331342-21idleruntime-cro21:15:022
256938992220,1cyclictest0-21swapper/222:15:012
25693899220,21cyclictest0-21swapper/221:15:122
25693899220,21cyclictest0-21swapper/221:05:142
25693899220,21cyclictest0-21swapper/220:05:142
25693899220,21cyclictest0-21swapper/219:30:142
25693899220,21cyclictest0-21swapper/219:10:132
25693899220,1cyclictest325435-21apt21:05:012
256936992221,1cyclictest378465-21apt-get23:00:121
256936992221,1cyclictest0-21swapper/120:40:201
25693699220,22cyclictest404099-21idleruntime-cro00:10:011
25693699220,22cyclictest332951-21sed21:15:161
25693699220,22cyclictest0-21swapper/121:10:201
25693699220,21cyclictest0-21swapper/122:30:141
25693699220,21cyclictest0-21swapper/121:30:121
25693699220,21cyclictest0-21swapper/120:45:131
25693699220,1cyclictest359425-21munin-run22:10:011
256934992221,1cyclictest374735-21apt-get22:50:130
256934992221,1cyclictest302299-21apt-get20:25:130
25693499220,22cyclictest337665-21apt-get21:25:140
25693499220,21cyclictest0-21swapper/021:55:020
25693499220,21cyclictest0-21swapper/020:40:130
25693499220,21cyclictest0-21swapper/000:30:040
256950992120,1cyclictest0-21swapper/723:40:137
256950992120,1cyclictest0-21swapper/723:30:147
256950992120,1cyclictest0-21swapper/720:40:147
256950992120,1cyclictest0-21swapper/720:00:137
256950992120,1cyclictest0-21swapper/720:00:137
25695099210,21cyclictest0-21swapper/719:20:017
25695099210,0cyclictest0-21swapper/721:45:137
25695099210,0cyclictest0-21swapper/720:33:287
25695099210,0cyclictest0-21swapper/719:30:147
256948992121,0cyclictest0-21swapper/623:00:136
256948992121,0cyclictest0-21swapper/622:00:146
256948992120,1cyclictest275734-21apt-get19:40:136
256948992120,1cyclictest266887-21apt-get19:25:146
256948992120,1cyclictest260972-21apt-get19:15:136
256948992120,1cyclictest0-21swapper/622:40:006
25694899210,21cyclictest0-21swapper/621:50:206
25694899210,21cyclictest0-21swapper/620:45:196
25694899210,21cyclictest0-21swapper/600:30:056
25694899210,0cyclictest0-21swapper/622:40:126
25694899210,0cyclictest0-21swapper/622:30:146
25694899210,0cyclictest0-21swapper/621:20:176
256947992121,0cyclictest0-21swapper/521:15:125
256947992121,0cyclictest0-21swapper/520:05:145
256947992121,0cyclictest0-21swapper/519:20:115
256947992120,1cyclictest404508-21apt-get00:10:125
256947992120,1cyclictest0-21swapper/519:55:185
256947992120,1cyclictest0-21swapper/519:55:185
256947992120,1cyclictest0-21swapper/519:30:145
25694799210,20cyclictest384683-21ls23:15:015
25694799210,20cyclictest0-21swapper/520:15:115
25694799210,0cyclictest0-21swapper/523:20:135
25694799210,0cyclictest0-21swapper/522:40:125
256944992120,1cyclictest0-21swapper/423:10:124
256944992120,0cyclictest0-21swapper/420:45:144
256944992119,1cyclictest0-21swapper/419:45:204
256944992119,1cyclictest0-21swapper/419:45:204
25694499210,1cyclictest263540-21basename19:20:014
25694499210,0cyclictest0-21swapper/421:15:124
25694499210,0cyclictest0-21swapper/400:20:044
256942992120,1cyclictest0-21swapper/322:20:143
256942992120,1cyclictest0-21swapper/321:20:023
256942992116,4cyclictest363180-21idleruntime-cro22:20:003
25694299210,21cyclictest0-21swapper/321:50:203
25694299210,21cyclictest0-21swapper/319:35:203
25694299210,21cyclictest0-21swapper/319:15:143
25694299210,0cyclictest0-21swapper/300:10:173
25693899214,16cyclictest0-21swapper/222:20:202
256938992121,0cyclictest0-21swapper/220:50:142
256938992121,0cyclictest0-21swapper/200:20:172
256938992120,1cyclictest45-21ksoftirqd/223:10:012
256936992121,0cyclictest0-21swapper/120:55:131
256936992121,0cyclictest0-21swapper/100:00:131
256936992120,1cyclictest377271-21basename22:55:011
25693699210,21cyclictest0-21swapper/120:50:191
25693699210,21cyclictest0-21swapper/120:10:151
25693699210,21cyclictest0-21swapper/119:55:131
25693699210,21cyclictest0-21swapper/119:55:131
25693699210,1cyclictest0-21swapper/120:20:141
25693699210,1cyclictest0-21swapper/119:50:021
25693699210,1cyclictest0-21swapper/119:50:021
25693699210,0cyclictest0-21swapper/123:10:121
25693699210,0cyclictest0-21swapper/119:50:121
25693699210,0cyclictest0-21swapper/119:50:121
25693699210,0cyclictest0-21swapper/119:20:121
256934992121,0cyclictest0-21swapper/020:55:200
256934992121,0cyclictest0-21swapper/020:50:130
256934992120,1cyclictest387178-21ntpq23:20:180
256934992119,1cyclictest410824-21grep00:25:010
256934992119,1cyclictest385455-21munin-run23:20:010
25693499210,21cyclictest0-21swapper/021:30:130
25693499210,21cyclictest0-21swapper/020:30:190
25693499210,1cyclictest265269-21ntpq19:20:180
25693499210,1cyclictest0-21swapper/022:10:030
25693499210,0cyclictest0-21swapper/021:05:150
256950992020,0cyclictest0-21swapper/720:25:137
256950992020,0cyclictest0-21swapper/720:20:197
256950992020,0cyclictest0-21swapper/719:20:127
256950992019,1cyclictest367314-21apt-get22:30:137
256950992019,1cyclictest0-21swapper/720:15:127
256950992019,1cyclictest0-21swapper/700:20:047
256950992018,1cyclictest407028-21idleruntime-cro00:15:017
25695099200,20cyclictest0-21swapper/720:55:137
256948992020,0cyclictest0-21swapper/622:10:136
256948992020,0cyclictest0-21swapper/621:25:146
256948992019,1cyclictest307759-21munin-run20:35:016
25694899200,0cyclictest0-21swapper/619:10:176
256947992020,0cyclictest0-21swapper/523:30:135
256947992020,0cyclictest0-21swapper/522:30:145
256947992020,0cyclictest0-21swapper/522:20:155
256947992020,0cyclictest0-21swapper/522:10:195
256947992020,0cyclictest0-21swapper/520:20:145
256947992020,0cyclictest0-21swapper/519:10:135
256947992019,1cyclictest0-21swapper/523:00:135
256947992019,1cyclictest0-21swapper/520:50:135
256947992018,1cyclictest392857-21cron23:40:015
256944992020,0cyclictest0-21swapper/421:20:124
256944992020,0cyclictest0-21swapper/420:10:154
256944992019,1cyclictest0-21swapper/423:20:134
256944992019,1cyclictest0-21swapper/420:25:134
256944992019,1cyclictest0-21swapper/400:15:044
256944992018,1cyclictest259787-21munin-run19:15:014
256944992018,1cyclictest0-21swapper/423:00:014
256944992018,1cyclictest0-21swapper/419:40:014
256944992017,2cyclictest399593-21perf23:55:014
256942992020,0cyclictest0-21swapper/322:50:013
256942992020,0cyclictest0-21swapper/322:00:233
256942992020,0cyclictest0-21swapper/321:55:033
256942992020,0cyclictest0-21swapper/321:50:013
256942992020,0cyclictest0-21swapper/320:20:133
256942992019,1cyclictest0-21swapper/323:10:123
256942992019,1cyclictest0-21swapper/300:30:123
25694299200,0cyclictest0-21swapper/321:25:143
25694299200,0cyclictest0-21swapper/300:15:053
256938992020,0cyclictest392105-21sh23:35:012
256938992020,0cyclictest0-21swapper/222:00:212
256938992020,0cyclictest0-21swapper/221:55:012
256938992020,0cyclictest0-21swapper/220:25:132
256938992020,0cyclictest0-21swapper/220:10:152
256938992020,0cyclictest0-21swapper/220:00:132
256938992020,0cyclictest0-21swapper/220:00:132
256938992019,1cyclictest393321-21apt-get23:40:132
256938992019,1cyclictest0-21swapper/219:15:142
256938992018,1cyclictest400374-21sleep00:00:012
256938992017,2cyclictest366-21systemd-journal00:10:012
256938992017,2cyclictest351953-21sh21:50:002
256936992020,0cyclictest0-21swapper/123:20:131
256936992020,0cyclictest0-21swapper/121:00:201
256936992020,0cyclictest0-21swapper/120:00:131
256936992020,0cyclictest0-21swapper/120:00:121
256936992020,0cyclictest0-21swapper/119:15:191
25693699200,0cyclictest0-21swapper/121:40:121
256934992020,0cyclictest0-21swapper/022:30:140
256934992020,0cyclictest0-21swapper/020:45:140
256934992020,0cyclictest0-21swapper/020:00:130
256934992020,0cyclictest0-21swapper/020:00:120
256934992020,0cyclictest0-21swapper/019:50:120
256934992020,0cyclictest0-21swapper/019:50:120
256934992019,1cyclictest407041-21idleruntime-cro00:15:010
256934992019,1cyclictest383497-21ntpq23:10:180
256934992019,1cyclictest294775-21ntpq20:10:210
256934992019,1cyclictest0-21swapper/000:00:120
256934992018,1cyclictest0-21swapper/023:30:010
256934992017,2cyclictest366-21systemd-journal22:50:010
25693499200,20cyclictest0-21swapper/023:30:140
25693499200,20cyclictest0-21swapper/023:00:130
25693499200,0cyclictest0-21swapper/019:10:130
256950991919,0cyclictest0-21swapper/719:25:197
256950991917,1cyclictest385453-21dump-pmu-power23:20:017
256948991919,0cyclictest0-21swapper/620:55:136
256948991919,0cyclictest0-21swapper/620:25:136
256948991919,0cyclictest0-21swapper/620:00:136
256948991919,0cyclictest0-21swapper/620:00:136
256948991919,0cyclictest0-21swapper/619:50:126
256948991919,0cyclictest0-21swapper/619:50:126
256948991914,4cyclictest387268-21collect223:20:196
25694899190,0cyclictest0-21swapper/623:35:016
256947991919,0cyclictest0-21swapper/521:25:135
256947991919,0cyclictest0-21swapper/521:00:215
256947991919,0cyclictest0-21swapper/519:40:145
256947991919,0cyclictest0-21swapper/500:30:185
256947991914,4cyclictest1-21systemd23:20:015
25694799190,18cyclictest288840-21ld20:00:185
25694799190,18cyclictest288840-21ld20:00:185
256944991919,0cyclictest0-21swapper/422:00:154
256944991919,0cyclictest0-21swapper/421:40:124
256944991919,0cyclictest0-21swapper/421:05:154
256944991919,0cyclictest0-21swapper/419:25:124
256944991918,1cyclictest0-21swapper/420:55:134
256944991916,3cyclictest325454-21awk21:05:024
256942991919,0cyclictest0-21swapper/321:30:123
256942991919,0cyclictest0-21swapper/319:55:123
256942991919,0cyclictest0-21swapper/319:55:113
256942991918,1cyclictest311148-21apt-get20:40:143
256942991918,1cyclictest0-21swapper/319:50:053
256942991918,1cyclictest0-21swapper/319:50:053
256942991918,0cyclictest0-21swapper/300:00:133
25694299191,17cyclictest324249-21ldconfig21:00:203
25694299190,1cyclictest400356-21cat00:00:013
25694299190,19cyclictest0-21swapper/322:40:123
25694299190,18cyclictest297707-21ntpq20:15:183
256938991919,0cyclictest0-21swapper/221:20:122
256938991919,0cyclictest0-21swapper/220:45:132
256938991917,1cyclictest0-21swapper/223:50:182
256936991919,0cyclictest0-21swapper/123:40:191
256936991919,0cyclictest0-21swapper/122:00:151
256936991919,0cyclictest0-21swapper/121:25:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional