You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 01:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Thu Aug 06, 2026 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
574012257177,26sleep30-21swapper/307:07:493
572522238173,20sleep10-21swapper/107:05:461
574142230168,19sleep70-21swapper/707:08:017
574492228174,16sleep60-21swapper/607:08:336
575122222185,24sleep40-21swapper/407:09:274
575392219180,26sleep20-21swapper/207:09:512
573652219181,24sleep50-21swapper/507:07:195
572512211181,20sleep00-21swapper/007:05:450
57787992821,6cyclictest208901-21awk11:25:014
5778499280,24cyclictest0-21swapper/308:35:203
5778499280,24cyclictest0-21swapper/308:35:193
5779499260,25cyclictest229577-21basename12:00:007
5779499260,21cyclictest159177-21apt-get10:00:137
5779499260,0cyclictest0-21swapper/710:50:117
57784992622,4cyclictest1-21systemd11:35:003
5778499260,25cyclictest0-21swapper/312:15:103
57794992521,3cyclictest0-21swapper/711:00:127
57794992516,3cyclictest138105-21grep09:25:007
5779499250,25cyclictest150300-21/usr/sbin/munin09:45:187
5779499250,22cyclictest83846-21ntpq07:50:207
5779499250,1cyclictest164634-21munin-run10:10:007
5779499250,1cyclictest125055-21ntpq09:00:177
5779099250,24cyclictest96786-21dump-pmu-power08:15:005
57787992521,3cyclictest135425-21tar09:20:014
5778499250,24cyclictest0-21swapper/310:10:133
5778199250,24cyclictest0-21swapper/209:55:132
5778199250,1cyclictest0-21swapper/209:10:012
5777299250,24cyclictest218212-21apt-get11:40:110
57794992418,5cyclictest0-21swapper/709:40:127
5779499240,23cyclictest250241-21sleep12:35:007
5779499240,23cyclictest250241-21sleep12:35:007
5779499240,23cyclictest0-21swapper/711:05:137
57790992421,2cyclictest220733-21awk11:45:005
57787992422,2cyclictest185322-21perf-exec10:45:004
5778799240,23cyclictest182342-21sendmail10:40:004
5778799240,23cyclictest0-21swapper/410:10:174
57784992421,3cyclictest0-21swapper/310:05:133
57784992421,3cyclictest0-21swapper/308:40:193
57784992421,3cyclictest0-21swapper/308:40:193
57784992419,5cyclictest182349-21munin-run10:40:003
5778499240,24cyclictest61790-21apt-get07:15:123
5778499240,23cyclictest0-21swapper/309:30:123
5778499240,23cyclictest0-21swapper/307:30:113
5778199240,23cyclictest789-21dbus-daemon11:35:012
5778199240,23cyclictest0-21swapper/212:30:122
5778199240,23cyclictest0-21swapper/212:30:122
5777799240,1cyclictest0-21swapper/107:25:131
5777299240,23cyclictest0-21swapper/007:15:190
57794992322,1cyclictest0-21swapper/711:40:127
57794992321,2cyclictest0-21swapper/712:00:037
57794992321,2cyclictest0-21swapper/710:45:127
57794992320,3cyclictest0-21swapper/709:30:127
5779499230,22cyclictest0-21swapper/710:25:127
5779499230,1cyclictest0-21swapper/710:55:127
5779499230,1cyclictest0-21swapper/708:30:177
57792992323,0cyclictest0-21swapper/611:20:126
57792992323,0cyclictest0-21swapper/608:00:196
57792992322,1cyclictest91333-21apt-get08:05:116
5779299230,22cyclictest229567-21systemctl12:00:006
57790992321,1cyclictest0-21swapper/512:25:185
5779099230,22cyclictest200045-21perf11:10:005
5779099230,22cyclictest0-21swapper/510:50:115
5779099230,22cyclictest0-21swapper/508:50:125
5779099230,1cyclictest0-21swapper/507:45:115
57787992322,1cyclictest185732-21apt-get10:45:124
57787992322,1cyclictest0-21swapper/408:45:174
57787992322,1cyclictest0-21swapper/408:45:174
5778799230,23cyclictest92602-21ntpq08:05:174
5778799230,23cyclictest85034-21appstreamcli07:55:024
5778799230,23cyclictest0-21swapper/410:30:184
5778799230,22cyclictest0-21swapper/411:35:104
57784992321,2cyclictest0-21swapper/310:55:193
57784992320,3cyclictest64754-21apt-get07:20:103
57784992319,3cyclictest189946-21ntpq10:50:183
5778499230,4cyclictest117414-21cron08:50:003
5778499230,4cyclictest117414-21cron08:50:003
5778499230,3cyclictest146960-21basename09:40:003
5778499230,22cyclictest0-21swapper/311:35:033
5778499230,22cyclictest0-21swapper/311:20:123
5778499230,22cyclictest0-21swapper/310:40:123
5778499230,22cyclictest0-21swapper/308:20:183
5778499230,22cyclictest0-21swapper/307:45:123
5778199230,22cyclictest200041-21munin-run11:10:002
5778199230,22cyclictest179828-21apt-get10:35:122
5778199230,22cyclictest120802-21apt-get08:55:112
5778199230,22cyclictest0-21swapper/212:05:132
5778199230,22cyclictest0-21swapper/211:45:172
5778199230,22cyclictest0-21swapper/209:40:132
5778199230,22cyclictest0-21swapper/208:45:172
5778199230,22cyclictest0-21swapper/208:45:172
5778199230,22cyclictest0-21swapper/207:20:112
5778199230,1cyclictest229575-21basename12:00:002
5778199230,1cyclictest0-21swapper/208:20:122
57777992322,1cyclictest0-21swapper/107:45:131
5777799230,23cyclictest135107-21turbostat09:20:001
5777799230,22cyclictest203405-21apt-get11:15:131
5777799230,22cyclictest0-21swapper/109:20:121
5777799230,0cyclictest0-21swapper/109:55:181
5777799230,0cyclictest0-21swapper/107:10:121
57772992322,1cyclictest70220-21munin-run07:30:000
5777299230,23cyclictest90922-21idleruntime-cro08:05:000
5777299230,23cyclictest230034-21apt-get12:00:120
5777299230,22cyclictest3422186-21systemd-journal09:30:000
5777299230,22cyclictest244339-21grep12:25:000
5777299230,22cyclictest0-21swapper/012:10:120
5777299230,22cyclictest0-21swapper/010:35:120
5777299230,22cyclictest0-21swapper/010:05:130
5777299230,22cyclictest0-21swapper/008:45:120
5777299230,22cyclictest0-21swapper/008:45:120
57794992221,1cyclictest63160-21ntpq07:15:197
57794992221,1cyclictest0-21swapper/710:40:007
57794992221,1cyclictest0-21swapper/709:20:027
57794992220,2cyclictest0-21swapper/711:45:107
57794992219,3cyclictest0-21swapper/708:45:117
57794992219,3cyclictest0-21swapper/708:45:117
5779499220,22cyclictest0-21swapper/711:35:107
5779499220,22cyclictest0-21swapper/710:15:017
5779499220,22cyclictest0-21swapper/707:40:167
5779499220,21cyclictest64754-21apt-get07:20:117
5779499220,21cyclictest60604-21turbostat07:15:007
5779499220,21cyclictest0-21swapper/712:20:117
5779499220,21cyclictest0-21swapper/709:55:147
5779499220,21cyclictest0-21swapper/709:50:107
5779499220,21cyclictest0-21swapper/708:15:137
5779499220,1cyclictest0-21swapper/709:25:127
5779499220,19cyclictest0-21swapper/707:25:137
57792992222,0cyclictest58088-21apt-get07:10:106
57792992221,1cyclictest244763-21apt-get12:25:126
57792992221,1cyclictest171000-21apt-get10:20:106
57792992220,1cyclictest182791-21apt-get10:40:126
5779299220,22cyclictest0-21swapper/612:35:176
5779299220,21cyclictest100179-21apt-get08:20:126
5779299220,21cyclictest0-21swapper/611:05:116
5779299220,21cyclictest0-21swapper/609:50:116
5779299220,21cyclictest0-21swapper/608:45:116
5779299220,21cyclictest0-21swapper/608:45:116
5779299220,21cyclictest0-21swapper/607:55:136
5779299220,1cyclictest209311-21apt-get11:25:106
5779299220,1cyclictest0-21swapper/609:45:006
5779099229,0cyclictest0-21swapper/507:29:535
57790992221,1cyclictest70640-21apt-get07:30:115
57790992221,1cyclictest0-21swapper/511:15:145
57790992221,1cyclictest0-21swapper/509:50:005
57790992221,1cyclictest0-21swapper/508:20:125
57790992220,1cyclictest701rcuc/511:00:005
57790992220,1cyclictest0-21swapper/507:20:175
5779099220,22cyclictest0-21swapper/512:35:115
5779099220,21cyclictest229546-21apt-get12:00:035
5779099220,21cyclictest0-21swapper/511:35:165
5779099220,21cyclictest0-21swapper/509:40:125
5779099220,21cyclictest0-21swapper/507:35:095
5779099220,1cyclictest0-21swapper/510:05:205
5779099220,0cyclictest0-21swapper/508:25:125
57787992221,1cyclictest221153-21apt-get11:45:114
57787992221,1cyclictest147382-21apt-get09:40:124
5778799220,22cyclictest73579-21apt-get07:35:094
5778799220,22cyclictest252025-21sh12:35:194
5778799220,22cyclictest214804-21http11:35:004
5778799220,22cyclictest117861-21apt-get08:50:124
5778799220,22cyclictest0-21swapper/412:00:184
5778799220,22cyclictest0-21swapper/411:10:124
5778799220,22cyclictest0-21swapper/410:55:194
5778799220,21cyclictest243129-21ntpq12:20:164
5778799220,21cyclictest0-21swapper/412:05:134
5778799220,21cyclictest0-21swapper/410:25:134
5778799220,21cyclictest0-21swapper/407:50:124
5778799220,1cyclictest0-21swapper/408:15:124
57784992221,1cyclictest151718-21ntpq09:45:193
57784992221,1cyclictest138532-21apt-get09:25:113
57784992221,1cyclictest0-21swapper/308:10:123
57784992221,1cyclictest0-21swapper/307:15:003
57784992220,2cyclictest0-21swapper/311:45:163
57784992220,2cyclictest0-21swapper/311:40:113
57784992220,1cyclictest135589-21apt-get09:20:123
57784992219,3cyclictest0-21swapper/310:45:183
5778499220,22cyclictest84958-21apt07:55:003
5778499220,22cyclictest106073-21apt-get08:30:113
5778499220,22cyclictest0-21swapper/311:50:103
5778499220,22cyclictest0-21swapper/311:25:113
5778499220,22cyclictest0-21swapper/311:10:123
5778499220,22cyclictest0-21swapper/311:05:193
5778499220,21cyclictest0-21swapper/307:55:023
57781992222,0cyclictest0-21swapper/211:25:112
57781992221,1cyclictest114497-21munin-run08:45:002
57781992221,1cyclictest114497-21munin-run08:45:002
57781992221,1cyclictest0-21swapper/208:15:122
5778199220,22cyclictest175285-21ntpq10:25:192
5778199220,22cyclictest173522-21perf10:25:012
5778199220,22cyclictest0-21swapper/211:00:012
5778199220,21cyclictest0-21swapper/210:05:132
5778199220,21cyclictest0-21swapper/209:25:122
5778199220,21cyclictest0-21swapper/209:10:122
5778199220,21cyclictest0-21swapper/208:00:122
5778199220,21cyclictest0-21swapper/207:45:172
5778199220,21cyclictest0-21swapper/207:35:092
57777992222,0cyclictest0-21swapper/111:35:031
57777992222,0cyclictest0-21swapper/110:40:181
57777992221,1cyclictest190057-21ntpq10:50:191
57777992221,1cyclictest0-21swapper/110:25:011
57777992220,1cyclictest0-21swapper/110:05:131
5777799220,22cyclictest194582-21apt-get11:00:111
5777799220,22cyclictest0-21swapper/112:20:121
5777799220,22cyclictest0-21swapper/109:10:121
5777799220,22cyclictest0-21swapper/107:35:091
5777799220,21cyclictest82453-21apt-get07:50:121
5777799220,21cyclictest144421-21apt-get09:35:121
5777799220,21cyclictest0-21swapper/112:15:101
5777799220,21cyclictest0-21swapper/108:35:131
5777799220,21cyclictest0-21swapper/108:35:131
5777799220,21cyclictest0-21swapper/107:55:021
5777799220,1cyclictest206370-21apt-get11:20:121
5777799220,1cyclictest0-21swapper/109:10:001
57772992222,0cyclictest0-21swapper/009:00:180
57772992221,1cyclictest150332-21apt-get09:45:120
57772992221,1cyclictest0-21swapper/008:25:110
57772992221,1cyclictest0-21swapper/007:55:020
57772992220,1cyclictest0-21swapper/010:55:130
5777299220,2cyclictest97213-21apt-get08:15:110
5777299220,22cyclictest0-21swapper/010:25:120
5777299220,22cyclictest0-21swapper/009:55:130
5777299220,21cyclictest0-21swapper/009:35:120
5777299220,21cyclictest0-21swapper/009:10:120
5777299220,21cyclictest0-21swapper/007:50:130
5777299220,1cyclictest0-21swapper/012:20:000
5777299220,1cyclictest0-21swapper/011:50:000
5777299220,1cyclictest0-21swapper/010:45:120
5777299220,1cyclictest0-21swapper/009:40:120
5777299220,1cyclictest0-21swapper/008:10:000
57794992121,0cyclictest0-21swapper/711:25:107
57794992121,0cyclictest0-21swapper/710:30:137
57794992121,0cyclictest0-21swapper/707:55:037
57794992120,1cyclictest0-21swapper/708:25:117
57794992119,0cyclictest0-21swapper/712:15:097
57794992118,0cyclictest0-21swapper/712:25:197
57794992118,0cyclictest0-21swapper/708:40:137
57794992118,0cyclictest0-21swapper/708:40:127
5779499210,21cyclictest0-21swapper/712:10:117
57792992121,0cyclictest212254-21apt-get11:30:126
57792992121,0cyclictest0-21swapper/611:50:126
57792992121,0cyclictest0-21swapper/609:25:116
57792992121,0cyclictest0-21swapper/608:15:126
57792992121,0cyclictest0-21swapper/607:20:126
57792992120,1cyclictest141479-21apt-get09:30:126
57792992120,1cyclictest111577-21awk08:40:006
57792992120,1cyclictest111577-21awk08:40:006
57792992120,1cyclictest0-21swapper/608:55:116
57792992119,1cyclictest129208-21cron09:10:006
5779299210,21cyclictest0-21swapper/610:15:036
5779299210,21cyclictest0-21swapper/609:55:136
5779299210,1cyclictest241822-21apt-get12:20:116
5779299210,0cyclictest232975-21apt-get12:05:126
5779299210,0cyclictest0-21swapper/610:30:126
57790992121,0cyclictest0-21swapper/511:25:115
57790992121,0cyclictest0-21swapper/510:30:125
57790992120,1cyclictest0-21swapper/512:10:125
57790992120,1cyclictest0-21swapper/511:00:125
57790992120,1cyclictest0-21swapper/509:00:125
57790992120,1cyclictest0-21swapper/508:40:125
57790992120,1cyclictest0-21swapper/508:40:125
57790992120,1cyclictest0-21swapper/508:00:125
57790992117,3cyclictest185309-21grep10:45:015
5779099210,21cyclictest123342-21perf-exec09:00:005
5779099210,20cyclictest0-21swapper/512:20:115
5779099210,0cyclictest0-21swapper/509:50:115
57787992121,0cyclictest0-21swapper/412:10:124
57787992120,1cyclictest156216-21apt-get09:55:134
57787992120,1cyclictest0-21swapper/408:10:184
5778799210,21cyclictest143994-21turbostat09:35:004
5778799210,21cyclictest0-21swapper/411:00:124
5778799210,20cyclictest0-21swapper/409:35:124
5778799210,20cyclictest0-21swapper/409:20:034
5778799210,0cyclictest0-21swapper/407:30:124
57784992121,0cyclictest0-21swapper/312:35:113
57784992121,0cyclictest0-21swapper/312:30:113
57784992121,0cyclictest0-21swapper/312:30:113
57784992121,0cyclictest0-21swapper/312:20:113
57784992121,0cyclictest0-21swapper/308:00:123
57784992121,0cyclictest0-21swapper/307:35:103
57784992120,1cyclictest105622-21cron08:30:003
57784992119,2cyclictest0-21swapper/310:20:113
57784992116,5cyclictest0-21swapper/311:20:003
5778499210,21cyclictest0-21swapper/312:05:133
5778499210,21cyclictest0-21swapper/309:15:093
5778499210,0cyclictest0-21swapper/312:00:123
5778499210,0cyclictest0-21swapper/307:40:103
57781992121,0cyclictest0-21swapper/212:20:112
57781992121,0cyclictest0-21swapper/208:50:122
57781992121,0cyclictest0-21swapper/208:25:112
57781992121,0cyclictest0-21swapper/208:10:132
57781992121,0cyclictest0-21swapper/207:25:132
57781992120,1cyclictest0-21swapper/212:00:022
57781992120,0cyclictest0-21swapper/211:35:102
57781992119,1cyclictest79074-21turbostat07:45:002
57781992119,1cyclictest246045-21ntpq12:25:172
5778199210,21cyclictest0-21swapper/210:15:122
5778199210,0cyclictest0-21swapper/208:05:112
5778199210,0cyclictest0-21swapper/207:10:112
57777992121,0cyclictest0-21swapper/112:30:111
57777992121,0cyclictest0-21swapper/112:30:111
57777992121,0cyclictest0-21swapper/108:10:121
57777992118,3cyclictest214818-21store11:35:001
5777799210,21cyclictest235915-21apt-get12:10:121
5777799210,21cyclictest0-21swapper/112:05:131
5777799210,21cyclictest0-21swapper/111:40:111
5777799210,21cyclictest0-21swapper/110:15:031
5777799210,21cyclictest0-21swapper/107:15:121
5777799210,1cyclictest123763-21apt-get09:00:121
5777799210,0cyclictest0-21swapper/111:45:111
5777799210,0cyclictest0-21swapper/110:45:121
5777799210,0cyclictest0-21swapper/107:20:111
57772992121,0cyclictest0-21swapper/011:50:110
57772992121,0cyclictest0-21swapper/008:40:130
57772992121,0cyclictest0-21swapper/008:40:120
57772992120,1cyclictest214844-21appstreamcli11:35:020
57772992120,1cyclictest200465-21apt-get11:10:120
57772992120,1cyclictest0-21swapper/011:20:000
57772992120,1cyclictest0-21swapper/010:15:170
57772992116,4cyclictest76093-21latency_hist07:40:000
5777299210,21cyclictest0-21swapper/010:30:120
5777299210,21cyclictest0-21swapper/010:10:120
5777299210,21cyclictest0-21swapper/007:35:000
5777299210,0cyclictest0-21swapper/012:25:110
5777299210,0cyclictest0-21swapper/007:45:120
57794992020,0cyclictest0-21swapper/712:35:117
57794992020,0cyclictest0-21swapper/712:05:137
57794992020,0cyclictest0-21swapper/708:55:117
57794992020,0cyclictest0-21swapper/708:10:127
57794992019,1cyclictest0-21swapper/707:40:007
57794992018,0cyclictest0-21swapper/707:45:127
5779499200,19cyclictest0-21swapper/711:50:117
5779499200,0cyclictest0-21swapper/710:40:117
57792992020,0cyclictest188671-21apt-get10:50:116
57792992020,0cyclictest0-21swapper/612:15:096
57792992020,0cyclictest0-21swapper/612:00:116
57792992020,0cyclictest0-21swapper/611:45:126
57792992020,0cyclictest0-21swapper/610:25:126
57792992020,0cyclictest0-21swapper/610:10:126
57792992020,0cyclictest0-21swapper/609:00:126
57792992020,0cyclictest0-21swapper/608:10:126
57792992020,0cyclictest0-21swapper/607:45:196
57792992020,0cyclictest0-21swapper/607:35:146
57792992020,0cyclictest0-21swapper/607:30:116
57792992020,0cyclictest0-21swapper/607:15:116
57792992019,1cyclictest247707-21apt-get12:30:126
57792992019,1cyclictest247707-21apt-get12:30:116
57792992019,1cyclictest0-21swapper/611:15:006
57792992019,1cyclictest0-21swapper/610:55:186
5779299200,0cyclictest0-21swapper/609:20:016
57790992020,0cyclictest0-21swapper/511:55:145
57790992020,0cyclictest0-21swapper/511:10:185
57790992020,0cyclictest0-21swapper/510:15:035
57790992020,0cyclictest0-21swapper/509:55:135
57790992020,0cyclictest0-21swapper/509:20:035
57790992020,0cyclictest0-21swapper/508:35:135
57790992020,0cyclictest0-21swapper/508:35:135
57790992020,0cyclictest0-21swapper/508:15:125
57790992020,0cyclictest0-21swapper/507:10:115
57790992019,1cyclictest0-21swapper/509:05:125
5779099200,20cyclictest0-21swapper/507:15:185
5779099200,0cyclictest0-21swapper/508:05:115
57787992020,0cyclictest0-21swapper/409:50:004
57787992020,0cyclictest0-21swapper/409:05:184
57787992020,0cyclictest0-21swapper/408:40:124
57787992020,0cyclictest0-21swapper/408:40:124
57787992019,1cyclictest197524-21apt-get11:05:114
5778799200,0cyclictest0-21swapper/409:10:194
5778799200,0cyclictest0-21swapper/408:30:124
57784992020,0cyclictest0-21swapper/312:10:183
57784992020,0cyclictest0-21swapper/311:00:123
57784992020,0cyclictest0-21swapper/310:30:123
57784992020,0cyclictest0-21swapper/310:15:113
5778499200,4cyclictest175446-21awk10:25:213
5778499200,3cyclictest129213-21apt09:10:003
57781992020,0cyclictest0-21swapper/212:35:112
57781992020,0cyclictest0-21swapper/211:40:112
57781992020,0cyclictest0-21swapper/211:20:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional