You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 09:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Thu Jan 01, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4494192242178,21sleep60-21swapper/619:09:006
4470462230166,21sleep00-21swapper/019:05:090
4494802221188,21sleep10-21swapper/119:09:521
4492132221188,21sleep40-21swapper/419:06:024
4493552212182,19sleep70-21swapper/719:08:057
4493412212180,20sleep20-21swapper/219:07:522
4493442210180,19sleep50-21swapper/519:07:555
4492772207176,20sleep30-21swapper/319:06:583
44972299300,23cyclictest0-21swapper/320:20:003
44972299290,23cyclictest0-21swapper/319:25:003
449734992822,6cyclictest479157-21idleruntime-cro20:00:007
44972299280,22cyclictest0-21swapper/321:25:013
449727992523,1cyclictest584041-21apt-get23:00:125
449722992521,1cyclictest476722-21apt-get19:55:133
44972299250,23cyclictest0-21swapper/322:55:003
44971699250,3cyclictest467736-21perf19:40:001
44971299250,24cyclictest363-21systemd-journal22:20:000
44973499247,4cyclictest0-21swapper/722:55:567
449734992420,3cyclictest0-21swapper/722:10:137
44973499240,23cyclictest0-21swapper/720:55:127
44973499240,23cyclictest0-21swapper/719:40:227
449732992422,1cyclictest470588-21idleruntime-cro19:45:006
44973299240,23cyclictest528538-21awk21:25:006
44972299240,23cyclictest606737-21awk23:39:593
44972299240,23cyclictest0-21swapper/323:55:003
44972099240,23cyclictest0-21swapper/222:20:012
44972099240,22cyclictest0-21swapper/200:25:012
44971299240,3cyclictest569077-21idleruntime-cro22:35:000
449734992319,4cyclictest496840-21perf20:30:007
44973499230,22cyclictest0-21swapper/721:55:137
44973299230,22cyclictest513972-21munin-run21:00:006
44973299230,22cyclictest482708-21apt-get20:05:126
449727992313,3cyclictest0-21swapper/523:14:595
449725992321,1cyclictest0-21swapper/419:55:144
449725992321,1cyclictest0-21swapper/400:10:084
44972599230,22cyclictest0-21swapper/421:05:014
449722992322,1cyclictest0-21swapper/320:10:143
449722992321,1cyclictest0-21swapper/322:05:003
44972299230,22cyclictest0-21swapper/320:40:013
44972299230,22cyclictest0-21swapper/319:50:103
44972299230,22cyclictest0-21swapper/319:20:003
449720992321,1cyclictest549485-21apt-get22:00:122
449720992320,2cyclictest572613-21apt-get22:40:122
44972099230,22cyclictest464656-21/usr/sbin/munin19:35:002
44972099230,1cyclictest456037-21munin-run19:20:012
449716992322,1cyclictest311rcuc/121:05:141
449716992322,1cyclictest311rcuc/121:05:131
44971699230,22cyclictest0-21swapper/121:55:131
44971699230,0cyclictest0-21swapper/100:20:121
449712992321,1cyclictest618415-21munin-run23:59:590
449712992321,1cyclictest555189-21apt-get22:10:130
449712992321,1cyclictest363-21systemd-journal21:50:000
449712992321,1cyclictest0-21swapper/022:35:010
44971299230,2cyclictest630646-21awk00:25:000
44971299230,22cyclictest622474-21apt-get00:10:080
44971299230,22cyclictest609612-21basename23:44:590
44971299230,22cyclictest0-21swapper/021:45:000
44971299230,22cyclictest0-21swapper/019:20:130
44971299230,22cyclictest0-21swapper/019:10:110
44971299230,1cyclictest0-21swapper/000:30:130
449734992221,1cyclictest0-21swapper/723:45:007
449734992219,2cyclictest483903-21ntpq20:05:167
44973499220,2cyclictest554750-21cron22:10:007
44973499220,21cyclictest0-21swapper/723:15:007
44973499220,1cyclictest586898-21apt-get23:05:177
44973499220,1cyclictest0-21swapper/700:20:007
44973299220,22cyclictest572175-21munin-run22:40:006
44973299220,21cyclictest631057-21apt-get00:25:126
44973299220,21cyclictest535184-21apt-get21:35:136
449727992221,1cyclictest540910-21apt-get21:45:125
449727992220,1cyclictest0-21swapper/519:20:005
44972799220,22cyclictest537660-21awk21:40:015
44972799220,22cyclictest0-21swapper/500:20:005
44972799220,21cyclictest0-21swapper/520:25:015
44972799220,1cyclictest573875-21ntpq22:40:185
44972799220,1cyclictest525665-21munin-run21:20:015
44972799220,1cyclictest0-21swapper/522:25:115
449725992220,2cyclictest511132-21basename20:55:004
449725992220,1cyclictest0-21swapper/420:05:004
44972599220,21cyclictest601433-21apt-get23:30:134
44972599220,21cyclictest534496-21apt-get21:35:014
44972599220,21cyclictest0-21swapper/423:10:274
44972599220,21cyclictest0-21swapper/422:15:124
44972599220,21cyclictest0-21swapper/421:20:144
44972599220,21cyclictest0-21swapper/419:50:104
44972599220,1cyclictest0-21swapper/421:05:194
44972599220,1cyclictest0-21swapper/421:05:194
449722992221,1cyclictest618879-21apt-get00:00:133
449722992221,1cyclictest0-21swapper/323:45:013
449722992221,1cyclictest0-21swapper/319:35:163
449722992221,1cyclictest0-21swapper/300:10:083
449722992221,0cyclictest0-21swapper/321:50:093
44972299220,21cyclictest508265-21munin-run20:50:003
44972299220,21cyclictest0-21swapper/323:15:003
44972299220,21cyclictest0-21swapper/320:55:123
44972299220,21cyclictest0-21swapper/320:50:123
44972299220,21cyclictest0-21swapper/300:35:123
44972299220,1cyclictest563356-21perf22:25:003
44972299220,0cyclictest0-21swapper/323:30:133
449720992221,1cyclictest0-21swapper/223:45:012
449720992221,1cyclictest0-21swapper/222:15:002
449720992221,1cyclictest0-21swapper/221:15:142
449720992220,1cyclictest488412-21apt-get20:15:122
449720992220,1cyclictest0-21swapper/222:35:022
44972099220,21cyclictest0-21swapper/219:50:102
44972099220,21cyclictest0-21swapper/219:35:012
44972099220,21cyclictest0-21swapper/200:10:082
44972099220,1cyclictest0-21swapper/220:10:152
449716992221,1cyclictest558041-21apt-get22:15:121
449716992220,2cyclictest639193-21munin-run00:40:001
449716992220,1cyclictest0-21swapper/123:35:001
449716992220,1cyclictest0-21swapper/120:10:161
44971699220,22cyclictest636347-21perf00:35:001
44971699220,21cyclictest363-21systemd-journal00:00:001
44971699220,21cyclictest0-21swapper/122:00:121
44971699220,21cyclictest0-21swapper/119:55:121
44971699220,21cyclictest0-21swapper/119:10:121
44971699220,1cyclictest497254-21apt-get20:30:121
44971699220,1cyclictest0-21swapper/122:35:001
449712992220,1cyclictest0-21swapper/023:05:180
449712992220,1cyclictest0-21swapper/021:10:000
449712992220,1cyclictest0-21swapper/021:10:000
44971299220,21cyclictest0-21swapper/022:40:130
44971299220,21cyclictest0-21swapper/022:00:120
44971299220,21cyclictest0-21swapper/021:15:140
44971299220,21cyclictest0-21swapper/020:20:130
44971299220,21cyclictest0-21swapper/019:35:010
44971299220,1cyclictest552336-21apt-get22:05:110
449734992121,0cyclictest0-21swapper/721:15:147
449734992119,2cyclictest0-21swapper/722:40:137
449734992117,3cyclictest0-21swapper/719:10:127
44973499210,4cyclictest0-21swapper/700:34:487
44973499210,21cyclictest0-21swapper/720:00:147
44973499210,20cyclictest0-21swapper/719:20:147
44973499210,0cyclictest0-21swapper/700:25:127
449732992120,1cyclictest523228-21apt-get21:15:146
449732992120,1cyclictest467766-21sendmail-msp19:40:006
449732992119,1cyclictest554761-21idleruntime-cro22:10:006
449732992119,1cyclictest464799-21apt-key19:35:006
44973299210,21cyclictest0-21swapper/600:20:006
44973299210,0cyclictest0-21swapper/622:30:136
44973299210,0cyclictest0-21swapper/621:25:096
449727992120,1cyclictest471018-21apt-get19:45:125
449727992120,1cyclictest0-21swapper/520:30:125
44972799210,20cyclictest0-21swapper/523:35:115
44972799210,0cyclictest0-21swapper/521:10:135
44972799210,0cyclictest0-21swapper/521:10:125
449725992121,0cyclictest0-21swapper/422:00:004
449725992120,1cyclictest578315-21apt-get22:50:114
449725992120,1cyclictest0-21swapper/423:55:104
44972599210,21cyclictest0-21swapper/420:05:114
44972599210,20cyclictest452664-21date19:15:004
44972599210,1cyclictest639261-21apt-config00:40:004
44972599210,1cyclictest0-21swapper/400:20:114
44972599210,0cyclictest0-21swapper/423:00:124
44972599210,0cyclictest0-21swapper/400:15:134
449722992121,0cyclictest0-21swapper/321:05:143
449722992121,0cyclictest0-21swapper/321:05:133
449722992120,1cyclictest0-21swapper/322:35:013
449722992120,1cyclictest0-21swapper/300:30:123
449722992119,1cyclictest0-21swapper/320:40:123
449722992119,1cyclictest0-21swapper/319:35:003
44972299210,0cyclictest0-21swapper/319:40:233
44972299210,0cyclictest0-21swapper/300:15:123
449720992120,1cyclictest0-21swapper/222:55:122
449720992120,0cyclictest0-21swapper/223:05:182
449720992119,1cyclictest519690-21munin-run21:10:012
449720992119,1cyclictest519690-21munin-run21:10:002
449720992119,1cyclictest0-21swapper/219:45:122
44972099210,21cyclictest0-21swapper/221:45:002
44972099210,1cyclictest0-21swapper/221:55:122
44972099210,0cyclictest0-21swapper/200:30:122
449716992121,0cyclictest0-21swapper/123:45:011
449716992121,0cyclictest0-21swapper/120:25:011
449716992120,1cyclictest0-21swapper/121:30:141
449716992120,1cyclictest0-21swapper/119:30:121
44971699210,21cyclictest0-21swapper/100:00:131
44971699210,20cyclictest471018-21apt-get19:45:121
449712992120,1cyclictest601433-21apt-get23:30:130
449712992120,1cyclictest0-21swapper/022:55:120
44971299210,21cyclictest0-21swapper/021:35:010
44971299210,21cyclictest0-21swapper/020:15:130
44971299210,21cyclictest0-21swapper/020:15:010
44971299210,20cyclictest0-21swapper/023:45:010
44971299210,20cyclictest0-21swapper/023:25:120
44971299210,0cyclictest0-21swapper/023:35:120
44973499207,0cyclictest0-21swapper/720:11:127
44973499207,0cyclictest0-21swapper/719:38:147
44973499207,0cyclictest0-21swapper/719:27:167
449734992020,0cyclictest0-21swapper/723:25:127
449734992020,0cyclictest0-21swapper/720:30:127
449734992019,1cyclictest569051-21apt-get22:35:017
449734992019,1cyclictest0-21swapper/700:20:117
449734992017,3cyclictest0-21swapper/721:00:137
449732992020,0cyclictest0-21swapper/623:55:106
449732992020,0cyclictest0-21swapper/623:00:126
449732992020,0cyclictest0-21swapper/621:05:006
449732992019,1cyclictest546613-21apt-get21:55:126
44973299200,20cyclictest577905-21awk22:50:016
44973299200,0cyclictest0-21swapper/622:15:126
44973299200,0cyclictest0-21swapper/619:20:136
449727992020,0cyclictest0-21swapper/521:25:095
449727992020,0cyclictest0-21swapper/520:45:005
449727992020,0cyclictest0-21swapper/519:40:225
449727992019,1cyclictest0-21swapper/521:30:135
449727992018,1cyclictest598579-21apt-get23:25:125
44972799200,0cyclictest0-21swapper/520:15:125
449725992020,0cyclictest0-21swapper/423:15:144
449725992020,0cyclictest0-21swapper/421:10:014
449725992020,0cyclictest0-21swapper/421:10:014
449725992019,1cyclictest0-21swapper/422:10:144
449725992018,1cyclictest566205-21munin-run22:30:004
44972599200,0cyclictest0-21swapper/421:25:014
44972599200,0cyclictest0-21swapper/419:30:134
449722992020,0cyclictest0-21swapper/322:40:013
449722992020,0cyclictest0-21swapper/300:20:133
449722992019,1cyclictest0-21swapper/320:25:133
449720992020,0cyclictest0-21swapper/223:20:122
449720992020,0cyclictest0-21swapper/222:45:132
449720992019,1cyclictest0-21swapper/220:05:122
449720992018,1cyclictest0-21swapper/223:15:132
44972099200,20cyclictest520372-21apt-get21:10:122
44972099200,20cyclictest520372-21apt-get21:10:122
44972099200,19cyclictest0-21swapper/220:20:132
44972099200,0cyclictest0-21swapper/223:40:122
449716992020,0cyclictest0-21swapper/120:05:021
44971699200,19cyclictest0-21swapper/120:15:121
44971699200,0cyclictest0-21swapper/121:50:101
449712992020,0cyclictest0-21swapper/020:05:110
449712992020,0cyclictest0-21swapper/019:45:120
449712992019,1cyclictest0-21swapper/000:20:000
449712992018,1cyclictest578315-21apt-get22:50:120
44971299200,20cyclictest0-21swapper/022:45:130
449734991919,0cyclictest0-21swapper/722:25:107
449734991919,0cyclictest0-21swapper/722:20:137
449734991915,3cyclictest490851-21munin-run20:20:017
449732991919,0cyclictest0-21swapper/623:40:136
449732991919,0cyclictest0-21swapper/621:45:126
449732991919,0cyclictest0-21swapper/621:35:016
449732991919,0cyclictest0-21swapper/621:10:026
449732991919,0cyclictest0-21swapper/621:10:016
449732991919,0cyclictest0-21swapper/620:50:136
449732991918,1cyclictest604284-21apt-get23:35:116
449732991918,1cyclictest581166-21apt-get22:55:116
449732991918,1cyclictest453596-21apt-get19:15:116
449732991918,1cyclictest0-21swapper/620:15:136
44973299190,1cyclictest538059-21apt-get21:40:126
44973299190,19cyclictest493711-21dump-pmu-power20:25:016
44973299190,0cyclictest0-21swapper/622:55:016
449727991919,0cyclictest0-21swapper/522:55:115
449727991919,0cyclictest0-21swapper/522:05:125
449727991919,0cyclictest0-21swapper/520:50:125
449727991919,0cyclictest0-21swapper/520:35:145
449727991919,0cyclictest0-21swapper/520:05:115
449727991919,0cyclictest0-21swapper/500:20:115
449727991919,0cyclictest0-21swapper/500:10:105
449727991918,0cyclictest0-21swapper/522:10:135
449727991917,1cyclictest0-21swapper/500:40:005
449727991916,2cyclictest609714-21apt-key23:45:015
449725991919,0cyclictest0-21swapper/423:20:134
449725991919,0cyclictest0-21swapper/423:05:174
449725991919,0cyclictest0-21swapper/422:20:124
449725991919,0cyclictest0-21swapper/421:50:104
449725991919,0cyclictest0-21swapper/421:40:134
449725991919,0cyclictest0-21swapper/420:15:124
449725991919,0cyclictest0-21swapper/419:20:134
449725991918,1cyclictest500110-21apt-get20:35:144
449725991918,1cyclictest0-21swapper/420:25:014
449722991919,0cyclictest0-21swapper/322:25:103
449722991919,0cyclictest0-21swapper/321:15:143
449722991918,1cyclictest0-21swapper/319:45:123
449722991918,0cyclictest0-21swapper/321:20:153
449722991917,1cyclictest461755-21turbostat19:30:003
44972299190,1cyclictest0-21swapper/320:05:013
449720991919,0cyclictest0-21swapper/222:05:112
449720991919,0cyclictest0-21swapper/200:00:002
449720991918,1cyclictest631057-21apt-get00:25:132
449720991918,1cyclictest0-21swapper/200:20:002
449720991916,2cyclictest511107-21latency_hist20:55:002
44972099190,18cyclictest0-21swapper/220:00:152
449716991919,0cyclictest0-21swapper/123:25:121
449716991919,0cyclictest0-21swapper/122:55:111
449716991919,0cyclictest0-21swapper/122:50:111
449716991919,0cyclictest0-21swapper/121:20:141
449716991919,0cyclictest0-21swapper/120:45:131
449716991919,0cyclictest0-21swapper/120:35:141
449716991918,1cyclictest0-21swapper/100:09:591
449712991919,0cyclictest0-21swapper/023:15:130
449712991919,0cyclictest0-21swapper/021:25:090
449712991919,0cyclictest0-21swapper/000:25:120
449712991918,0cyclictest0-21swapper/021:30:140
449712991917,1cyclictest0-21swapper/000:35:120
449734991818,0cyclictest0-21swapper/723:20:117
449734991818,0cyclictest0-21swapper/719:50:107
449734991818,0cyclictest0-21swapper/700:10:097
449734991817,1cyclictest551919-21munin-run22:05:007
44973499180,14cyclictest0-21swapper/723:10:387
449732991818,0cyclictest0-21swapper/621:05:136
449732991818,0cyclictest0-21swapper/621:05:136
449732991818,0cyclictest0-21swapper/620:35:146
449732991818,0cyclictest0-21swapper/619:55:136
449732991817,1cyclictest505832-21apt-get20:45:126
449732991817,1cyclictest0-21swapper/620:30:006
44973299180,0cyclictest0-21swapper/622:10:136
449727991818,0cyclictest0-21swapper/523:05:175
449727991818,0cyclictest0-21swapper/521:40:125
449727991818,0cyclictest0-21swapper/519:10:125
449727991818,0cyclictest0-21swapper/500:30:125
449727991818,0cyclictest0-21swapper/500:25:125
449727991817,1cyclictest610274-21apt-get23:45:125
449727991817,1cyclictest0-21swapper/522:15:125
449727991816,1cyclictest569059-21munin-run22:35:005
449727991816,1cyclictest473869-21apt-get19:50:115
449727991815,2cyclictest519849-21latency_hist21:10:005
449727991815,2cyclictest519849-21latency_hist21:10:005
449725991818,0cyclictest0-21swapper/423:45:124
449725991818,0cyclictest0-21swapper/422:35:014
449725991818,0cyclictest0-21swapper/422:30:144
449725991818,0cyclictest0-21swapper/422:00:124
449725991818,0cyclictest0-21swapper/400:25:124
449722991818,0cyclictest0-21swapper/322:55:113
449722991818,0cyclictest0-21swapper/322:15:113
449722991818,0cyclictest0-21swapper/322:05:113
449722991818,0cyclictest0-21swapper/300:25:123
449722991816,1cyclictest0-21swapper/323:55:103
449720991818,0cyclictest0-21swapper/223:30:142
449720991818,0cyclictest0-21swapper/221:25:102
449720991816,1cyclictest604284-21apt-get23:35:112
449720991816,1cyclictest0-21swapper/200:00:132
449720991815,2cyclictest505832-21apt-get20:45:122
44972099180,17cyclictest0-21swapper/221:35:012
449716991818,0cyclictest0-21swapper/122:05:121
449716991818,0cyclictest0-21swapper/121:25:021
449716991817,1cyclictest473869-21apt-get19:50:101
449716991817,1cyclictest0-21swapper/100:30:011
449716991815,2cyclictest577889-21apt22:50:001
449712991818,0cyclictest0-21swapper/023:50:130
449712991817,1cyclictest511104-21idleruntime-cro20:55:010
449712991817,1cyclictest0-21swapper/022:00:010
449734991717,0cyclictest0-21swapper/721:30:147
449734991716,1cyclictest606721-21munin-run23:40:007
449734991716,1cyclictest0-21swapper/721:05:127
449734991716,1cyclictest0-21swapper/721:05:127
449734991716,1cyclictest0-21swapper/700:40:007
449734991715,2cyclictest0-21swapper/721:40:127
449734991715,2cyclictest0-21swapper/721:35:127
449734991712,1cyclictest473451-21munin-run19:50:017
44973499170,4cyclictest0-21swapper/720:22:107
44973499170,17cyclictest0-21swapper/723:56:577
44973499170,16cyclictest503207-21tr20:40:137
44973499170,14cyclictest0-21swapper/719:32:337
449732991717,0cyclictest0-21swapper/623:45:016
449732991717,0cyclictest0-21swapper/620:30:126
449727991717,0cyclictest0-21swapper/523:15:155
449727991716,1cyclictest0-21swapper/519:35:015
449725991717,0cyclictest0-21swapper/421:45:134
449725991717,0cyclictest0-21swapper/421:30:124
449725991717,0cyclictest0-21swapper/420:40:124
449725991717,0cyclictest0-21swapper/400:00:124
44972599170,16cyclictest0-21swapper/423:50:144
44972299170,16cyclictest537620-21munin-run21:40:003
449720991717,0cyclictest0-21swapper/223:50:132
449720991717,0cyclictest0-21swapper/221:30:132
449720991716,1cyclictest0-21swapper/222:20:122
449720991715,1cyclictest0-21swapper/221:45:122
44972099170,16cyclictest0-21swapper/220:25:132
449716991717,0cyclictest0-21swapper/123:45:001
449716991717,0cyclictest0-21swapper/121:10:121
449716991717,0cyclictest0-21swapper/121:10:111
449716991717,0cyclictest0-21swapper/119:40:221
449716991716,1cyclictest456474-21apt-get19:20:141
44971699170,3cyclictest572581-21/usr/sbin/munin22:40:241
449712991717,0cyclictest0-21swapper/022:20:130
449712991717,0cyclictest0-21swapper/021:50:100
449712991717,0cyclictest0-21swapper/020:00:140
449712991717,0cyclictest0-21swapper/000:00:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional