You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 05:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Tue Jul 01, 2025 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6799732228164,21sleep00-21swapper/019:09:190
6799352213179,22sleep70-21swapper/719:08:467
6799102212179,21sleep20-21swapper/219:08:242
6798952211159,17sleep50-21swapper/519:08:125
6797542211178,21sleep60-21swapper/619:06:126
6799272208176,20sleep10-21swapper/119:08:401
6798842206178,18sleep30-21swapper/319:08:023
6797052203176,17sleep40-21swapper/419:05:294
68027899298,4cyclictest0-21swapper/719:44:557
68027899268,0cyclictest0-21swapper/722:51:387
68027899258,0cyclictest0-21swapper/722:29:407
68027899258,0cyclictest0-21swapper/721:23:467
68027899258,0cyclictest0-21swapper/720:50:497
68027899250,24cyclictest693240-21apt-get19:30:157
68027299250,1cyclictest770662-21apt-get21:45:185
680270992524,1cyclictest819167-21fwupd23:08:004
68026199250,23cyclictest0-21swapper/120:00:111
68027899247,0cyclictest0-21swapper/723:13:367
68027899247,0cyclictest0-21swapper/722:07:427
68027899247,0cyclictest0-21swapper/722:07:417
68027899247,0cyclictest0-21swapper/719:11:577
68026799240,24cyclictest1896684-21systemd-journal23:35:013
680261992422,1cyclictest0-21swapper/122:40:011
68027899231,4cyclictest0-21swapper/720:39:507
680276992321,1cyclictest0-21swapper/621:34:596
68027699230,22cyclictest0-21swapper/623:20:016
68027699230,22cyclictest0-21swapper/621:55:006
68027699230,22cyclictest0-21swapper/621:15:136
68027699230,21cyclictest802618-21apt-get22:40:146
680272992321,1cyclictest816897-21apt-get23:05:165
680272992321,1cyclictest773508-21apt-get21:50:145
68027299230,23cyclictest747392-21awk21:05:005
68027099230,22cyclictest851966-21apt-get00:10:094
68027099230,22cyclictest0-21swapper/419:45:124
68027099230,22cyclictest0-21swapper/400:05:004
68026799230,22cyclictest0-21swapper/322:25:153
68026799230,0cyclictest0-21swapper/320:05:123
680265992322,1cyclictest0-21swapper/223:50:012
68026599230,22cyclictest842621-21apt-get23:50:142
68026599230,22cyclictest0-21swapper/219:55:132
68026199230,22cyclictest836917-21apt-get23:40:131
68026199230,22cyclictest0-21swapper/121:50:141
68026199230,22cyclictest0-21swapper/121:20:001
68025999230,22cyclictest868958-21perf00:40:000
68025999230,22cyclictest816476-21munin-run23:05:000
68025999230,22cyclictest0-21swapper/020:50:120
680278992221,1cyclictest0-21swapper/700:10:007
680278992220,1cyclictest805471-21apt-get22:45:157
68027899221,4cyclictest0-21swapper/723:24:357
68027899220,21cyclictest0-21swapper/721:45:187
68027899220,21cyclictest0-21swapper/721:25:127
68027899220,1cyclictest0-21swapper/720:20:177
68027899220,1cyclictest0-21swapper/700:05:007
680276992222,0cyclictest0-21swapper/623:25:136
680276992221,1cyclictest857702-21apt-get00:20:156
680276992221,1cyclictest0-21swapper/623:50:146
680276992220,2cyclictest1896684-21systemd-journal23:25:016
680276992220,1cyclictest724933-21apt-get20:25:146
68027699220,21cyclictest0-21swapper/622:25:166
68027699220,21cyclictest0-21swapper/622:10:176
68027699220,21cyclictest0-21swapper/622:00:186
68027699220,21cyclictest0-21swapper/622:00:186
68027699220,21cyclictest0-21swapper/620:20:176
68027699220,21cyclictest0-21swapper/619:50:136
68027699220,21cyclictest0-21swapper/619:10:136
68027699220,21cyclictest0-21swapper/600:10:086
68027699220,1cyclictest767805-21apt-get21:40:136
68027699220,1cyclictest0-21swapper/622:50:016
68027699220,1cyclictest0-21swapper/600:35:116
680272992222,0cyclictest0-21swapper/522:40:135
680272992221,1cyclictest0-21swapper/522:30:155
680272992221,1cyclictest0-21swapper/520:05:125
680272992220,1cyclictest848371-21apt-get00:00:115
68027299220,22cyclictest845057-21perf23:55:015
68027299220,1cyclictest0-21swapper/520:30:155
680270992221,1cyclictest755971-21sendmail-msp21:20:004
680270992220,1cyclictest0-21swapper/420:25:004
68027099220,21cyclictest0-21swapper/423:40:134
68027099220,21cyclictest0-21swapper/420:05:124
68027099220,21cyclictest0-21swapper/419:40:184
68027099220,21cyclictest0-21swapper/419:15:114
68027099220,21cyclictest0-21swapper/400:20:154
680267992221,1cyclictest845046-21turbostat23:55:013
68026799220,22cyclictest0-21swapper/322:00:133
68026799220,22cyclictest0-21swapper/322:00:123
68026799220,21cyclictest0-21swapper/323:10:143
68026799220,21cyclictest0-21swapper/322:35:163
68026799220,21cyclictest0-21swapper/320:25:143
68026799220,1cyclictest710389-21apt-get20:00:123
68026599220,21cyclictest0-21swapper/222:40:032
68026599220,1cyclictest684388-21apt-get19:15:112
680261992220,1cyclictest756374-21apt-get21:20:141
680261992220,1cyclictest0-21swapper/123:10:011
68026199220,22cyclictest692821-21perf19:30:001
68026199220,21cyclictest0-21swapper/123:20:151
68026199220,21cyclictest0-21swapper/120:25:131
68026199220,21cyclictest0-21swapper/120:10:001
68026199220,21cyclictest0-21swapper/119:15:111
68026199220,21cyclictest0-21swapper/100:25:131
680259992222,0cyclictest0-21swapper/019:50:130
680259992221,1cyclictest0-21swapper/020:05:000
680259992219,2cyclictest770225-21idleruntime-cro21:45:000
68025999220,2cyclictest842203-21awk23:50:010
68025999220,22cyclictest807913-21idleruntime-cro22:50:010
68025999220,21cyclictest0-21swapper/021:35:160
680278992119,1cyclictest753081-21cut21:15:007
68027899210,21cyclictest0-21swapper/722:30:157
68027899210,20cyclictest0-21swapper/720:15:177
68027899210,20cyclictest0-21swapper/720:10:157
68027899210,0cyclictest0-21swapper/722:35:157
68027899210,0cyclictest0-21swapper/719:55:137
680276992121,0cyclictest0-21swapper/623:30:136
680276992120,1cyclictest0-21swapper/623:40:016
68027699210,0cyclictest0-21swapper/622:30:156
680272992121,0cyclictest0-21swapper/522:35:165
680272992120,1cyclictest698969-21apt-get19:40:175
680272992120,1cyclictest0-21swapper/521:20:155
680272992120,1cyclictest0-21swapper/521:20:005
680272992120,1cyclictest0-21swapper/500:20:155
680272992119,1cyclictest830791-21basename23:30:015
68027299210,21cyclictest762078-21apt-get21:30:145
68027299210,21cyclictest0-21swapper/522:25:165
68027299210,0cyclictest0-21swapper/521:25:135
68027299210,0cyclictest0-21swapper/520:40:135
680270992121,0cyclictest0-21swapper/419:35:194
680270992120,1cyclictest0-21swapper/420:30:004
680270992120,1cyclictest0-21swapper/400:30:004
680270992120,0cyclictest0-21swapper/421:10:134
680270992120,0cyclictest0-21swapper/400:35:124
680270992119,1cyclictest0-21swapper/420:44:594
68027099210,20cyclictest0-21swapper/423:00:134
68027099210,20cyclictest0-21swapper/422:40:144
68027099210,0cyclictest0-21swapper/423:15:134
680267992120,1cyclictest698525-21munin-run19:40:003
680267992119,1cyclictest816897-21apt-get23:05:153
68026799210,21cyclictest0-21swapper/323:35:133
68026799210,21cyclictest0-21swapper/320:30:153
68026799210,1cyclictest810756-21perf22:55:003
68026799210,0cyclictest0-21swapper/323:40:123
68026799210,0cyclictest0-21swapper/321:30:143
68026799210,0cyclictest0-21swapper/320:50:133
68026799210,0cyclictest0-21swapper/319:45:123
680265992121,0cyclictest0-21swapper/223:55:132
680265992121,0cyclictest0-21swapper/223:15:122
680265992121,0cyclictest0-21swapper/221:00:142
680265992120,1cyclictest860552-21apt-get00:25:132
680265992120,1cyclictest835272-21ntpq23:35:182
680265992120,1cyclictest0-21swapper/222:20:222
680265992120,1cyclictest0-21swapper/219:40:002
680265992119,1cyclictest0-21swapper/220:25:002
68026599210,21cyclictest801955-21sendmail-msp22:40:012
68026599210,21cyclictest781665-21munin-run22:05:002
68026599210,21cyclictest781665-21munin-run22:05:002
68026599210,0cyclictest0-21swapper/222:55:142
68026599210,0cyclictest0-21swapper/220:30:152
68026599210,0cyclictest0-21swapper/200:30:142
680261992120,1cyclictest0-21swapper/123:25:141
68026199210,20cyclictest0-21swapper/121:00:141
68026199210,0cyclictest0-21swapper/122:05:001
68026199210,0cyclictest0-21swapper/122:05:001
68026199210,0cyclictest0-21swapper/121:10:141
68026199210,0cyclictest0-21swapper/119:10:021
680259992121,0cyclictest0-21swapper/023:40:130
680259992121,0cyclictest0-21swapper/022:15:160
680259992121,0cyclictest0-21swapper/021:05:150
680259992121,0cyclictest0-21swapper/020:45:130
680259992121,0cyclictest0-21swapper/020:25:000
680259992120,1cyclictest0-21swapper/022:05:150
680259992120,1cyclictest0-21swapper/022:05:150
680259992119,1cyclictest650-21dbus-daemon20:39:590
68025999210,21cyclictest0-21swapper/022:30:020
68025999210,1cyclictest727358-21turbostat.cron20:30:000
68025999210,0cyclictest0-21swapper/000:20:140
680278992020,0cyclictest0-21swapper/722:00:137
680278992020,0cyclictest0-21swapper/722:00:137
68027899200,0cyclictest0-21swapper/700:30:147
68027899200,0cyclictest0-21swapper/700:25:137
680276992020,0cyclictest0-21swapper/619:40:176
680276992018,1cyclictest0-21swapper/621:20:166
68027699200,0cyclictest0-21swapper/620:55:136
68027699200,0cyclictest0-21swapper/600:15:126
680272992020,0cyclictest0-21swapper/523:45:145
680272992020,0cyclictest0-21swapper/521:40:145
680272992020,0cyclictest0-21swapper/520:25:145
680272992019,1cyclictest736365-21apt-get20:45:135
680272992019,0cyclictest0-21swapper/500:35:125
68027299200,20cyclictest0-21swapper/522:50:145
68027299200,0cyclictest0-21swapper/523:35:135
680270992020,0cyclictest0-21swapper/423:35:014
680270992020,0cyclictest0-21swapper/422:15:004
680270992020,0cyclictest0-21swapper/422:05:154
680270992020,0cyclictest0-21swapper/422:05:154
680270992020,0cyclictest0-21swapper/421:35:164
680270992020,0cyclictest0-21swapper/421:20:154
680270992019,1cyclictest0-21swapper/423:20:164
680270992018,1cyclictest839770-21apt-get23:45:144
680270992018,1cyclictest0-21swapper/421:00:004
68027099200,0cyclictest0-21swapper/421:00:144
680267992020,0cyclictest0-21swapper/322:40:143
680267992020,0cyclictest0-21swapper/322:30:023
680267992020,0cyclictest0-21swapper/321:55:163
680267992020,0cyclictest0-21swapper/321:50:143
680267992019,1cyclictest807894-21turbostat22:50:013
680267992019,1cyclictest0-21swapper/321:40:133
680267992017,2cyclictest747800-21http21:05:163
68026799200,0cyclictest0-21swapper/319:40:183
680265992020,0cyclictest0-21swapper/223:25:142
680265992020,0cyclictest0-21swapper/223:00:142
680265992020,0cyclictest0-21swapper/220:10:002
680265992020,0cyclictest0-21swapper/220:00:112
680265992019,1cyclictest827919-21munin-run23:25:012
680265992019,1cyclictest0-21swapper/200:19:592
680261992020,0cyclictest0-21swapper/122:40:141
680261992020,0cyclictest0-21swapper/122:20:171
680261992020,0cyclictest0-21swapper/122:15:161
680261992020,0cyclictest0-21swapper/121:35:161
680261992020,0cyclictest0-21swapper/120:20:171
680261992020,0cyclictest0-21swapper/119:55:131
680261992020,0cyclictest0-21swapper/100:30:141
680261992018,1cyclictest845471-21apt-get23:55:131
68026199200,19cyclictest0-21swapper/120:50:001
68026199200,0cyclictest0-21swapper/119:20:021
680259992020,0cyclictest0-21swapper/023:30:130
680259992020,0cyclictest0-21swapper/023:25:130
680259992020,0cyclictest0-21swapper/023:10:140
680259992020,0cyclictest0-21swapper/021:25:120
680259992020,0cyclictest0-21swapper/020:05:110
680259992020,0cyclictest0-21swapper/019:40:180
680259992020,0cyclictest0-21swapper/019:10:010
680259992020,0cyclictest0-21swapper/000:00:120
680259992019,1cyclictest847985-21cups-browsed00:00:010
68025999200,20cyclictest0-21swapper/022:05:000
68025999200,20cyclictest0-21swapper/022:05:000
680278991919,0cyclictest0-21swapper/723:45:147
680278991919,0cyclictest0-21swapper/723:05:167
680278991919,0cyclictest0-21swapper/722:40:147
680278991919,0cyclictest0-21swapper/721:50:137
680278991919,0cyclictest0-21swapper/721:15:147
680276991919,0cyclictest0-21swapper/620:55:006
680276991919,0cyclictest0-21swapper/620:30:156
680276991919,0cyclictest0-21swapper/619:25:136
680272991919,0cyclictest0-21swapper/500:10:095
680272991918,1cyclictest0-21swapper/521:05:155
680272991918,1cyclictest0-21swapper/520:00:005
680272991918,1cyclictest0-21swapper/500:25:135
680270991919,0cyclictest0-21swapper/423:10:134
680270991919,0cyclictest0-21swapper/422:55:154
680270991919,0cyclictest0-21swapper/422:30:154
680270991919,0cyclictest0-21swapper/420:00:124
680270991919,0cyclictest0-21swapper/419:20:174
680270991919,0cyclictest0-21swapper/419:10:024
680270991917,1cyclictest0-21swapper/421:10:004
680267991919,0cyclictest0-21swapper/323:55:133
680267991919,0cyclictest0-21swapper/323:45:143
680267991919,0cyclictest0-21swapper/320:55:133
680267991918,0cyclictest744943-21apt-get21:00:143
680267991916,2cyclictest0-21swapper/319:59:593
680265991919,0cyclictest776362-21apt-get21:55:162
680265991919,0cyclictest0-21swapper/221:45:172
680265991919,0cyclictest0-21swapper/220:55:132
680265991919,0cyclictest0-21swapper/220:45:132
680265991919,0cyclictest0-21swapper/220:15:172
680265991919,0cyclictest0-21swapper/219:45:122
680265991919,0cyclictest0-21swapper/219:20:022
680265991919,0cyclictest0-21swapper/200:35:022
680265991919,0cyclictest0-21swapper/200:10:002
680261991919,0cyclictest0-21swapper/122:10:171
680261991919,0cyclictest0-21swapper/120:10:141
680261991917,1cyclictest795953-21perf22:30:001
680261991917,1cyclictest747797-21apt-get21:05:151
680261991917,1cyclictest0-21swapper/121:50:001
680261991916,2cyclictest860122-21idleruntime-cro00:24:591
680259991919,0cyclictest0-21swapper/022:40:150
680259991919,0cyclictest0-21swapper/021:45:180
680259991919,0cyclictest0-21swapper/021:15:130
680259991919,0cyclictest0-21swapper/020:55:130
680259991919,0cyclictest0-21swapper/020:40:130
680259991919,0cyclictest0-21swapper/019:15:110
680259991918,1cyclictest799484-21apt-get22:35:150
680259991918,1cyclictest721377-21turbostat20:20:000
680259991918,1cyclictest716095-21apt-get20:10:140
680259991918,1cyclictest0-21swapper/021:54:590
680278991818,0cyclictest0-21swapper/721:40:137
680278991818,0cyclictest0-21swapper/721:35:167
680278991818,0cyclictest0-21swapper/721:30:147
680278991818,0cyclictest0-21swapper/720:45:137
680278991818,0cyclictest0-21swapper/719:35:187
680278991818,0cyclictest0-21swapper/719:20:187
680278991817,1cyclictest0-21swapper/722:20:167
680278991816,1cyclictest0-21swapper/719:20:017
68027899181,0cyclictest0-21swapper/723:57:327
68027899181,0cyclictest0-21swapper/722:18:417
680276991818,0cyclictest0-21swapper/623:40:136
680276991818,0cyclictest0-21swapper/623:05:156
680276991818,0cyclictest0-21swapper/621:10:136
680276991818,0cyclictest0-21swapper/620:35:126
680276991818,0cyclictest0-21swapper/619:20:186
680276991817,0cyclictest0-21swapper/600:30:146
680272991818,0cyclictest0-21swapper/521:00:005
680272991818,0cyclictest0-21swapper/519:24:595
680272991818,0cyclictest0-21swapper/500:30:135
680272991818,0cyclictest0-21swapper/500:15:125
680272991817,1cyclictest692833-21idleruntime-cro19:30:005
680272991817,1cyclictest686819-21apt-get19:20:005
680272991815,2cyclictest718508-21sh20:15:005
680270991818,0cyclictest0-21swapper/422:25:164
680270991818,0cyclictest0-21swapper/400:09:594
680270991817,1cyclictest0-21swapper/423:35:124
680270991815,2cyclictest721420-21idleruntime-cro20:19:594
680267991818,0cyclictest0-21swapper/321:20:153
680267991818,0cyclictest0-21swapper/321:19:593
680267991818,0cyclictest0-21swapper/300:20:153
680267991817,1cyclictest0-21swapper/323:30:013
680267991816,1cyclictest0-21swapper/300:30:143
680267991814,3cyclictest767403-21basename21:40:003
680265991818,0cyclictest0-21swapper/222:05:142
680265991818,0cyclictest0-21swapper/222:05:142
680265991818,0cyclictest0-21swapper/221:15:122
680261991818,0cyclictest0-21swapper/123:30:131
680261991818,0cyclictest0-21swapper/122:50:141
680261991818,0cyclictest0-21swapper/121:55:161
680261991818,0cyclictest0-21swapper/121:40:131
680261991818,0cyclictest0-21swapper/119:30:141
680261991817,1cyclictest0-21swapper/119:45:121
680259991818,0cyclictest0-21swapper/021:55:160
680259991818,0cyclictest0-21swapper/021:10:130
680259991818,0cyclictest0-21swapper/019:55:130
680259991817,1cyclictest851966-21apt-get00:10:080
68025999180,17cyclictest0-21swapper/023:35:130
68025999180,17cyclictest0-21swapper/022:10:170
680278991716,1cyclictest861ktimers/720:03:507
680278991715,1cyclictest744943-21apt-get21:00:137
680276991717,0cyclictest0-21swapper/623:45:146
680276991717,0cyclictest0-21swapper/622:35:166
680276991717,0cyclictest0-21swapper/620:45:136
680276991717,0cyclictest0-21swapper/619:45:126
680276991717,0cyclictest0-21swapper/619:35:186
680276991717,0cyclictest0-21swapper/600:25:126
680276991715,1cyclictest0-21swapper/621:35:166
68027699170,16cyclictest0-21swapper/622:20:176
68027699170,0cyclictest0-21swapper/623:55:126
680272991717,0cyclictest0-21swapper/522:10:005
680272991717,0cyclictest0-21swapper/522:10:005
680272991717,0cyclictest0-21swapper/522:00:185
680272991717,0cyclictest0-21swapper/522:00:175
680272991717,0cyclictest0-21swapper/519:10:135
680272991716,1cyclictest0-21swapper/523:00:015
680272991716,1cyclictest0-21swapper/521:39:595
68027299170,17cyclictest0-21swapper/521:57:285
680270991717,0cyclictest0-21swapper/422:15:154
680270991717,0cyclictest0-21swapper/420:45:134
680270991717,0cyclictest0-21swapper/419:50:124
680270991716,1cyclictest0-21swapper/422:35:164
680267991717,0cyclictest0-21swapper/320:45:123
680267991717,0cyclictest0-21swapper/320:35:133
680267991717,0cyclictest0-21swapper/319:25:003
680267991717,0cyclictest0-21swapper/300:35:113
680267991717,0cyclictest0-21swapper/300:10:083
680267991717,0cyclictest0-21swapper/300:00:123
680267991716,1cyclictest0-21swapper/319:20:013
680265991717,0cyclictest0-21swapper/223:40:112
680265991717,0cyclictest0-21swapper/220:10:142
680265991717,0cyclictest0-21swapper/219:30:142
680265991716,1cyclictest857702-21apt-get00:20:152
680265991716,1cyclictest819350-21awk23:10:012
680261991717,0cyclictest0-21swapper/122:45:151
680261991717,0cyclictest0-21swapper/100:35:021
680261991717,0cyclictest0-21swapper/100:15:121
680261991716,1cyclictest0-21swapper/120:30:141
680259991717,0cyclictest0-21swapper/022:25:160
680259991716,1cyclictest2278112-21in:imuxsock19:50:000
680278991616,0cyclictest0-21swapper/720:05:117
680278991616,0cyclictest0-21swapper/700:15:127
680278991616,0cyclictest0-21swapper/700:10:087
680278991614,1cyclictest814044-21apt-get23:00:137
68027899160,1cyclictest0-21swapper/700:35:127
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional