You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 22:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4s.osadl.org (updated Mon Nov 24, 2025 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1007512232195,24sleep10-21swapper/107:09:091
1007102222184,25sleep60-21swapper/607:08:336
1007792219180,26sleep00-21swapper/007:09:310
1006242216183,21sleep30-21swapper/307:07:183
1007552214174,25sleep50-21swapper/507:09:125
1007112212175,24sleep70-21swapper/707:08:347
1007182204177,17sleep40-21swapper/407:08:394
1006682201169,21sleep20-21swapper/207:07:572
101058992822,6cyclictest274254-21perf12:10:004
101056992524,1cyclictest866-21mta-sts-daemon08:25:123
10105399250,24cyclictest0-21swapper/209:10:142
10105399250,21cyclictest0-21swapper/212:35:132
10106799242,4cyclictest0-21swapper/709:49:387
101064992423,1cyclictest0-21swapper/611:50:136
10106299240,1cyclictest282826-21dump-pmu-power12:25:005
101058992423,1cyclictest0-21swapper/408:45:014
10105699240,23cyclictest0-21swapper/310:40:123
101053992420,4cyclictest0-21swapper/210:20:192
10105399240,23cyclictest0-21swapper/212:00:002
10106799232,4cyclictest0-21swapper/712:01:257
101067992321,1cyclictest291375-21journal-offline12:40:017
101067992321,1cyclictest0-21swapper/711:35:017
101067992319,3cyclictest142134-21apt-get08:20:137
10106799230,2cyclictest0-21swapper/711:05:017
101064992321,1cyclictest0-21swapper/609:35:136
101064992321,1cyclictest0-21swapper/607:35:136
101064992321,1cyclictest0-21swapper/607:35:136
101064992319,3cyclictest279967-21basename12:20:016
101064992319,3cyclictest279967-21basename12:20:016
10106499230,23cyclictest0-21swapper/609:50:136
10106499230,22cyclictest0-21swapper/611:15:116
10106299230,22cyclictest119261-21apt-get07:40:135
10106299230,22cyclictest0-21swapper/510:20:125
10106299230,1cyclictest156042-21systemd-run08:45:005
10105899230,22cyclictest217224-21apt-get10:30:124
10105899230,22cyclictest0-21swapper/408:15:134
10105899230,21cyclictest234385-21apt-get11:00:124
10105699230,22cyclictest0-21swapper/312:00:113
10105699230,22cyclictest0-21swapper/307:45:123
101053992319,3cyclictest0-21swapper/208:05:132
10105399230,23cyclictest0-21swapper/208:15:002
10105399230,22cyclictest0-21swapper/211:00:122
10105399230,22cyclictest0-21swapper/208:30:122
10105399230,0cyclictest0-21swapper/212:00:012
101051992322,1cyclictest0-21swapper/108:30:011
101051992321,1cyclictest0-21swapper/110:05:141
101051992321,1cyclictest0-21swapper/109:25:131
101046992321,1cyclictest271815-21apt-get12:05:110
10106799224,17cyclictest0-21swapper/709:59:597
10106799220,21cyclictest194097-21apt-get09:50:127
10106799220,21cyclictest0-21swapper/712:10:157
10106799220,21cyclictest0-21swapper/712:10:147
10106799220,21cyclictest0-21swapper/707:40:017
10106799220,21cyclictest0-21swapper/707:40:017
10106799220,21cyclictest0-21swapper/707:35:007
10106799220,21cyclictest0-21swapper/707:35:007
10106799220,1cyclictest124547-21munin-run07:50:007
10106499220,22cyclictest0-21swapper/610:04:596
10106499220,1cyclictest0-21swapper/610:24:596
101062992221,1cyclictest0-21swapper/509:34:595
10106299220,22cyclictest0-21swapper/509:10:005
10106299220,21cyclictest0-21swapper/512:15:125
10106299220,21cyclictest0-21swapper/512:15:125
10106299220,21cyclictest0-21swapper/511:35:125
10106299220,21cyclictest0-21swapper/510:55:115
10106299220,21cyclictest0-21swapper/510:50:115
10106299220,21cyclictest0-21swapper/509:40:135
10106299220,1cyclictest107364-21munin-run07:20:005
10106299220,0cyclictest0-21swapper/509:55:135
101058992221,1cyclictest842-21runrttasks12:35:124
10105899220,21cyclictest268264-21awk12:00:004
10105899220,21cyclictest0-21swapper/412:15:114
10105899220,21cyclictest0-21swapper/412:15:114
10105899220,21cyclictest0-21swapper/409:40:134
10105899220,21cyclictest0-21swapper/407:35:124
10105899220,21cyclictest0-21swapper/407:35:124
10105899220,1cyclictest225385-21munin-run10:45:004
10105899220,1cyclictest196946-21apt-get09:55:134
10105899220,1cyclictest0-21swapper/411:50:004
101056992222,0cyclictest0-21swapper/310:30:133
101056992221,1cyclictest0-21swapper/308:55:113
10105699220,21cyclictest0-21swapper/309:50:123
10105699220,21cyclictest0-21swapper/309:45:143
10105699220,0cyclictest0-21swapper/309:15:123
101053992222,0cyclictest0-21swapper/209:55:122
101053992221,0cyclictest0-21swapper/208:00:112
101053992220,1cyclictest185516-21apt-get09:35:142
101053992219,3cyclictest0-21swapper/209:45:132
10105399220,21cyclictest0-21swapper/212:25:132
10105399220,21cyclictest0-21swapper/210:25:132
10105399220,21cyclictest0-21swapper/209:20:122
10105399220,21cyclictest0-21swapper/208:25:122
101051992221,1cyclictest133124-21munin-run08:05:001
101051992221,1cyclictest0-21swapper/110:14:591
101051992221,0cyclictest274672-21apt-get12:10:151
101051992221,0cyclictest274672-21apt-get12:10:151
101051992221,0cyclictest0-21swapper/109:15:131
101051992220,1cyclictest0-21swapper/107:20:101
10105199220,22cyclictest0-21swapper/112:00:101
10105199220,21cyclictest0-21swapper/111:35:111
10105199220,21cyclictest0-21swapper/108:50:111
10105199220,1cyclictest155998-21munin-run08:45:001
10105199220,1cyclictest104924-21apt-get07:15:131
10105199220,1cyclictest0-21swapper/108:20:001
10105199220,0cyclictest0-21swapper/112:25:131
101046992221,1cyclictest0-21swapper/009:55:120
101046992221,1cyclictest0-21swapper/009:10:030
101046992220,1cyclictest0-21swapper/009:40:140
101046992220,1cyclictest0-21swapper/008:50:110
10104699220,22cyclictest212821-21ntpq10:20:180
10104699220,22cyclictest0-21swapper/011:55:010
10104699220,22cyclictest0-21swapper/010:50:110
10104699220,21cyclictest0-21swapper/011:20:130
10104699220,21cyclictest0-21swapper/011:05:130
10104699220,21cyclictest0-21swapper/009:30:110
10104699220,21cyclictest0-21swapper/009:20:120
10104699220,21cyclictest0-21swapper/009:05:130
10104699220,21cyclictest0-21swapper/007:55:130
10104699220,1cyclictest280399-21apt-get12:20:110
10106799218,0cyclictest0-21swapper/707:26:507
101067992121,0cyclictest0-21swapper/708:10:407
101067992121,0cyclictest0-21swapper/707:20:107
101067992120,0cyclictest0-21swapper/711:40:007
101067992119,2cyclictest0-21swapper/711:45:147
101067992116,4cyclictest0-21swapper/710:19:597
10106799210,20cyclictest0-21swapper/712:25:127
10106799210,0cyclictest0-21swapper/711:25:137
10106799210,0cyclictest0-21swapper/710:55:127
10106799210,0cyclictest0-21swapper/710:05:147
101064992121,0cyclictest0-21swapper/607:40:136
101064992120,1cyclictest0-21swapper/609:55:126
101064992120,1cyclictest0-21swapper/607:45:116
101064992120,0cyclictest0-21swapper/610:55:116
101064992119,1cyclictest0-21swapper/609:15:136
10106499210,21cyclictest0-21swapper/612:00:096
10106499210,20cyclictest0-21swapper/610:35:136
10106499210,20cyclictest0-21swapper/607:35:006
10106499210,20cyclictest0-21swapper/607:35:006
10106499210,0cyclictest0-21swapper/609:00:116
101062992121,0cyclictest0-21swapper/507:25:145
101062992120,1cyclictest222515-21munin-run10:40:005
10106299210,21cyclictest200085-21apt-get10:00:135
10106299210,21cyclictest0-21swapper/510:10:205
10106299210,20cyclictest0-21swapper/512:10:145
10106299210,20cyclictest0-21swapper/512:10:145
10106299210,0cyclictest0-21swapper/511:30:125
10106299210,0cyclictest0-21swapper/508:20:135
101058992120,1cyclictest0-21swapper/412:00:024
101058992120,1cyclictest0-21swapper/407:45:124
101058992119,1cyclictest0-21swapper/407:30:014
10105899211,0cyclictest0-21swapper/410:20:004
10105899210,21cyclictest0-21swapper/409:05:134
10105899210,21cyclictest0-21swapper/408:55:114
10105899210,0cyclictest0-21swapper/408:10:414
101056992121,0cyclictest0-21swapper/312:25:133
101056992121,0cyclictest0-21swapper/311:20:113
101056992120,1cyclictest268249-21apt-get12:00:003
101056992119,1cyclictest182659-21apt-get09:30:123
10105699210,21cyclictest0-21swapper/308:45:143
10105699210,0cyclictest0-21swapper/310:35:123
10105699210,0cyclictest0-21swapper/307:20:103
101053992121,0cyclictest0-21swapper/207:15:122
101053992120,1cyclictest0-21swapper/208:45:022
101053992120,1cyclictest0-21swapper/207:35:132
101053992120,1cyclictest0-21swapper/207:35:122
101053992120,0cyclictest0-21swapper/212:05:122
101053992119,1cyclictest0-21swapper/212:30:122
101053992118,0cyclictest0-21swapper/211:30:112
101053992118,0cyclictest0-21swapper/208:55:112
10105399210,0cyclictest0-21swapper/211:50:012
101051992120,1cyclictest0-21swapper/109:55:121
101051992120,1cyclictest0-21swapper/109:30:101
101051992120,1cyclictest0-21swapper/107:35:131
101051992120,1cyclictest0-21swapper/107:35:121
101051992120,0cyclictest0-21swapper/109:50:121
101051992119,1cyclictest115951-21perf07:35:011
101051992119,1cyclictest115951-21perf07:35:001
101051992119,1cyclictest0-21swapper/111:50:131
101046992121,0cyclictest0-21swapper/012:25:120
101046992121,0cyclictest0-21swapper/012:00:100
101046992121,0cyclictest0-21swapper/009:25:130
101046992119,2cyclictest288544-21idleruntime-cro12:35:000
101046992119,1cyclictest165257-21apt-get09:00:110
101046992119,1cyclictest0-21swapper/010:15:000
10104699210,21cyclictest0-21swapper/011:15:000
10104699210,20cyclictest0-21swapper/011:45:130
10104699210,0cyclictest0-21swapper/008:35:120
10104699210,0cyclictest0-21swapper/008:30:130
101067992020,0cyclictest0-21swapper/712:30:127
101067992020,0cyclictest0-21swapper/710:20:117
101067992018,1cyclictest171180-21apt-get09:10:037
101067992017,2cyclictest0-21swapper/712:20:127
101067992016,4cyclictest0-21swapper/709:30:117
10106799200,0cyclictest0-21swapper/710:35:007
101064992020,0cyclictest0-21swapper/610:45:006
101064992019,1cyclictest0-21swapper/611:35:116
101064992018,1cyclictest236819-21munin-run11:05:016
10106499200,20cyclictest0-21swapper/607:10:126
10106499200,0cyclictest0-21swapper/612:10:006
101062992020,0cyclictest0-21swapper/511:45:135
101062992020,0cyclictest0-21swapper/511:25:135
101062992020,0cyclictest0-21swapper/511:05:145
101062992020,0cyclictest0-21swapper/509:20:135
101062992020,0cyclictest0-21swapper/508:30:135
101062992019,1cyclictest193675-21munin-run09:49:595
101062992019,0cyclictest116365-21apt-get07:35:125
101062992019,0cyclictest116365-21apt-get07:35:125
101062992018,1cyclictest153125-21munin-run08:40:015
10106299200,20cyclictest0-21swapper/507:20:115
10106299200,0cyclictest0-21swapper/512:00:025
101058992020,0cyclictest0-21swapper/410:50:114
101058992020,0cyclictest0-21swapper/410:00:134
101058992020,0cyclictest0-21swapper/408:35:134
101058992020,0cyclictest0-21swapper/407:15:124
101058992019,1cyclictest0-21swapper/408:50:104
101058992019,0cyclictest0-21swapper/409:45:144
10105899200,19cyclictest0-21swapper/411:05:144
10105899200,0cyclictest0-21swapper/411:15:124
10105899200,0cyclictest0-21swapper/408:25:134
101056992020,0cyclictest0-21swapper/312:20:123
101056992020,0cyclictest0-21swapper/311:35:113
101056992020,0cyclictest0-21swapper/310:50:123
101056992020,0cyclictest0-21swapper/310:15:103
101056992020,0cyclictest0-21swapper/310:05:133
101056992020,0cyclictest0-21swapper/308:30:123
101056992020,0cyclictest0-21swapper/307:55:123
101056992020,0cyclictest0-21swapper/307:25:143
101056992019,1cyclictest199383-21munin-run10:00:003
101056992019,1cyclictest0-21swapper/311:55:003
101056992019,1cyclictest0-21swapper/311:10:153
101056992019,0cyclictest0-21swapper/308:15:133
101056992018,2cyclictest182244-21grep09:30:003
101053992020,0cyclictest0-21swapper/211:50:122
101053992020,0cyclictest0-21swapper/210:45:132
101053992020,0cyclictest0-21swapper/209:00:102
10105399200,19cyclictest0-21swapper/209:35:002
101051992020,0cyclictest0-21swapper/112:05:111
101051992020,0cyclictest0-21swapper/110:25:121
101051992020,0cyclictest0-21swapper/109:00:111
101051992019,1cyclictest242531-21awk11:15:001
101051992019,1cyclictest0-21swapper/111:45:141
101051992019,1cyclictest0-21swapper/110:50:121
101051992019,1cyclictest0-21swapper/110:40:131
101051992019,1cyclictest0-21swapper/108:45:021
101051992019,1cyclictest0-21swapper/107:50:131
101051992018,1cyclictest291394-21munin-run12:40:011
101051992018,1cyclictest0-21swapper/111:05:131
101051992018,1cyclictest0-21swapper/110:55:121
101051992018,1cyclictest0-21swapper/110:35:121
101046992020,0cyclictest0-21swapper/011:30:110
101046992020,0cyclictest0-21swapper/010:40:130
101046992020,0cyclictest0-21swapper/008:45:130
101046992020,0cyclictest0-21swapper/007:15:130
101046992019,1cyclictest0-21swapper/008:30:010
101046992018,1cyclictest0-21swapper/009:15:130
10104699200,20cyclictest0-21swapper/010:19:590
10106799192,4cyclictest0-21swapper/708:32:457
101067991919,0cyclictest0-21swapper/710:00:137
101067991919,0cyclictest0-21swapper/709:05:007
101067991919,0cyclictest0-21swapper/708:00:127
101067991918,0cyclictest0-21swapper/711:15:117
101067991917,2cyclictest0-21swapper/708:35:127
101067991916,3cyclictest0-21swapper/708:05:137
101064991919,0cyclictest0-21swapper/612:10:156
101064991919,0cyclictest0-21swapper/612:10:146
101064991919,0cyclictest0-21swapper/609:05:136
101064991918,1cyclictest156680-21apt-get08:45:136
101064991918,0cyclictest0-21swapper/611:40:126
101064991918,0cyclictest0-21swapper/610:15:096
101064991917,1cyclictest164807-21latency_hist09:00:016
10106499190,1cyclictest245820-21apt-get11:20:116
101062991919,0cyclictest0-21swapper/512:35:135
101062991919,0cyclictest0-21swapper/511:40:175
101062991919,0cyclictest0-21swapper/508:55:125
101062991919,0cyclictest0-21swapper/507:10:125
101062991917,1cyclictest0-21swapper/508:15:015
101058991919,0cyclictest0-21swapper/411:25:124
101058991919,0cyclictest0-21swapper/409:50:134
101058991919,0cyclictest0-21swapper/408:40:154
101058991919,0cyclictest0-21swapper/408:10:014
101058991918,1cyclictest214369-21apt-get10:25:124
101058991918,1cyclictest0-21swapper/410:45:144
101058991918,1cyclictest0-21swapper/407:50:124
101056991919,0cyclictest0-21swapper/312:30:123
101056991919,0cyclictest0-21swapper/310:20:193
101056991919,0cyclictest0-21swapper/309:00:123
101056991919,0cyclictest0-21swapper/308:35:123
101056991919,0cyclictest0-21swapper/307:35:193
101056991919,0cyclictest0-21swapper/307:35:193
101056991918,1cyclictest176955-21apt-get09:20:123
101056991918,1cyclictest0-21swapper/311:05:133
101053991919,0cyclictest0-21swapper/209:25:132
101053991919,0cyclictest0-21swapper/209:15:122
101053991919,0cyclictest0-21swapper/208:15:132
101053991918,1cyclictest0-21swapper/211:25:122
101053991917,2cyclictest258155-21fwupd11:40:182
101053991917,1cyclictest225392-21awk10:44:592
101051991919,0cyclictest0-21swapper/110:20:121
101051991918,1cyclictest0-21swapper/107:45:111
101051991918,0cyclictest0-21swapper/108:05:131
101051991917,2cyclictest245387-21run-parts11:20:001
101051991917,1cyclictest0-21swapper/107:10:131
101051991913,5cyclictest251494-21/usr/sbin/munin11:30:191
101046991919,0cyclictest225803-21apt-get10:45:130
101046991919,0cyclictest0-21swapper/011:35:110
101046991919,0cyclictest0-21swapper/009:50:120
101046991919,0cyclictest0-21swapper/009:40:000
101046991918,1cyclictest0-21swapper/008:10:000
101046991918,1cyclictest0-21swapper/007:20:100
101046991918,0cyclictest0-21swapper/012:10:150
101046991918,0cyclictest0-21swapper/012:10:150
101067991817,1cyclictest0-21swapper/709:20:127
101067991817,0cyclictest0-21swapper/707:50:127
10106799180,15cyclictest0-21swapper/711:53:447
101064991818,0cyclictest0-21swapper/612:35:146
101064991818,0cyclictest0-21swapper/611:55:146
101064991818,0cyclictest0-21swapper/608:40:146
101064991818,0cyclictest0-21swapper/608:00:136
101064991818,0cyclictest0-21swapper/607:50:126
101064991817,1cyclictest0-21swapper/607:25:136
101062991817,1cyclictest242513-21latency_hist11:15:015
101058991818,0cyclictest0-21swapper/412:25:124
101058991818,0cyclictest0-21swapper/410:35:124
101058991818,0cyclictest0-21swapper/408:20:134
101058991818,0cyclictest0-21swapper/407:20:114
101058991818,0cyclictest0-21swapper/407:10:124
101058991816,1cyclictest0-21swapper/411:35:124
101058991815,2cyclictest176509-21dump-pmu-power09:20:004
101056991818,0cyclictest0-21swapper/309:39:593
101056991818,0cyclictest0-21swapper/309:05:143
101056991818,0cyclictest0-21swapper/308:45:013
101056991818,0cyclictest0-21swapper/307:40:123
101056991817,0cyclictest0-21swapper/312:05:123
101056991817,0cyclictest0-21swapper/309:40:133
101053991818,0cyclictest0-21swapper/210:15:092
101053991818,0cyclictest0-21swapper/210:00:142
101053991816,1cyclictest0-21swapper/210:39:592
101051991818,0cyclictest0-21swapper/110:45:121
101051991817,1cyclictest190824-21munin-run09:45:001
101051991817,1cyclictest0-21swapper/111:00:131
101051991817,1cyclictest0-21swapper/109:00:001
101051991816,2cyclictest130254-21munin-run08:00:011
101046991818,0cyclictest0-21swapper/010:05:140
101046991818,0cyclictest0-21swapper/008:05:010
101046991818,0cyclictest0-21swapper/007:40:120
101046991817,1cyclictest0-21swapper/010:25:110
101067991716,1cyclictest101352-21apt-get07:10:137
101067991716,1cyclictest0-21swapper/710:10:157
101067991714,2cyclictest0-21swapper/709:37:247
10106799170,9cyclictest1041-21rtkit-daemon10:39:487
10106799170,16cyclictest154860-21ntpq08:40:217
10106799170,16cyclictest0-21swapper/711:56:227
10106799170,16cyclictest0-21swapper/709:15:557
10106799170,16cyclictest0-21swapper/709:06:237
101064991717,0cyclictest0-21swapper/610:30:146
101064991717,0cyclictest0-21swapper/609:45:156
101064991717,0cyclictest0-21swapper/609:30:116
101064991717,0cyclictest0-21swapper/608:20:196
101064991716,1cyclictest161968-21awk08:55:006
101064991716,1cyclictest0-21swapper/608:10:116
10106499170,16cyclictest0-21swapper/612:20:126
101062991717,0cyclictest0-21swapper/509:35:135
101062991716,1cyclictest147422-21dump-pmu-power08:30:005
101062991716,1cyclictest127389-21idleruntime-cro07:55:005
101058991717,0cyclictest0-21swapper/411:20:124
101058991717,0cyclictest0-21swapper/411:10:144
10105899170,1cyclictest0-21swapper/410:20:124
101056991717,0cyclictest0-21swapper/311:45:133
101056991717,0cyclictest0-21swapper/309:10:023
101056991717,0cyclictest0-21swapper/307:15:123
101056991716,0cyclictest0-21swapper/311:30:113
101056991715,1cyclictest208236-21sleep10:14:593
101053991717,0cyclictest0-21swapper/208:50:112
101051991716,1cyclictest0-21swapper/111:40:121
101051991716,1cyclictest0-21swapper/108:10:401
101046991717,0cyclictest0-21swapper/007:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional