You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 07:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Dec 25, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1687399250,24cyclictest0-21swapper/323:00:163
1687399240,23cyclictest19713-21apt-get23:40:153
1687399240,23cyclictest0-21swapper/300:00:143
16860992421,2cyclictest22895-21munin-run20:30:010
1687399231,21cyclictest2505-21sh22:00:003
1687399231,1cyclictest0-21swapper/323:15:163
16869992321,1cyclictest606-21dbus-daemon20:15:002
16869992320,3cyclictest8992-21apt-get00:25:132
1686999230,2cyclictest25281-21apt20:35:012
16864992321,1cyclictest0-21swapper/100:25:001
1686499230,22cyclictest24206-21perf22:45:001
1686499230,22cyclictest0-21swapper/121:20:021
1686499230,22cyclictest0-21swapper/120:35:161
16860992321,1cyclictest0-21swapper/019:55:150
16860992320,2cyclictest8731-21munin-run00:25:000
1686099230,22cyclictest14593-21apt22:25:010
16873992222,0cyclictest0-21swapper/320:40:143
16873992220,1cyclictest4927-21latency_hist22:05:003
16873992219,2cyclictest10852-21cron20:05:013
1687399220,21cyclictest0-21swapper/320:30:153
1687399220,21cyclictest0-21swapper/319:25:153
1687399220,1cyclictest19400-21munin-run22:35:013
16869992221,1cyclictest13264-21dump-pmu-power20:10:002
16869992220,2cyclictest14601-21perf22:25:012
16869992219,2cyclictest8470-21idleruntime-cro20:00:012
16869992219,2cyclictest3899-21basename00:15:002
16869992219,2cyclictest30120-21idleruntime-cro21:50:002
1686999220,21cyclictest336-21apt-get21:55:132
1686999220,21cyclictest30048-21apt-get22:55:152
1686999220,21cyclictest26902-21apt-get23:55:152
1686999220,21cyclictest24226-21apt23:50:012
1686999220,21cyclictest15942-21apt-get20:15:152
1686999220,1cyclictest17007-21awk23:35:002
16864992221,1cyclictest0-21swapper/119:25:141
16864992221,0cyclictest0-21swapper/100:25:131
16864992220,1cyclictest14859-21apt-get22:25:151
1686499220,21cyclictest23199-21apt-get21:35:151
1686499220,21cyclictest13830-21apt-get00:35:131
1686499220,21cyclictest0-21swapper/123:45:181
1686499220,21cyclictest0-21swapper/119:35:131
1686499220,21cyclictest0-21swapper/100:15:161
1686499220,1cyclictest25616-21apt-get21:40:141
1686499220,0cyclictest0-21swapper/120:05:141
16860992220,1cyclictest31545-21munin-run00:05:010
16860992220,1cyclictest0-21swapper/000:25:130
1686099220,21cyclictest24193-21apt-get22:45:000
1686099220,21cyclictest18409-21apt-get21:25:150
1686099220,21cyclictest0-21swapper/022:25:150
1686099220,21cyclictest0-21swapper/021:00:150
1686099220,21cyclictest0-21swapper/020:40:140
1686099220,21cyclictest0-21swapper/020:35:160
1686099220,1cyclictest24019-21sh19:25:000
1686099220,1cyclictest13270-21perf20:10:000
1686099220,1cyclictest0-21swapper/022:45:020
16873992121,0cyclictest11153-21apt-get20:05:143
16873992120,1cyclictest22113-21apt-get23:45:183
16873992120,0cyclictest30367-21apt-get20:45:143
16873992119,1cyclictest0-21swapper/322:50:023
16873992118,3cyclictest25557-21apt-get20:35:163
1687399210,21cyclictest0-21swapper/321:10:133
1687399210,21cyclictest0-21swapper/319:35:133
1687399210,20cyclictest29172-21ls21:45:223
1687399210,20cyclictest0-21swapper/322:25:153
1687399210,20cyclictest0-21swapper/321:00:003
1687399210,1cyclictest11812-21apt-get23:25:023
1687399210,0cyclictest0-21swapper/323:55:163
1687399210,0cyclictest0-21swapper/320:50:143
16869992120,1cyclictest2448-21apt-get23:05:142
16869992120,1cyclictest24193-21apt-get22:45:022
16869992120,1cyclictest0-21swapper/219:45:162
1686999210,20cyclictest24505-21apt-get23:50:152
1686999210,20cyclictest24505-21apt-get23:50:152
1686999210,20cyclictest16004-21apt-get21:20:152
1686999210,20cyclictest0-21swapper/220:35:162
1686999210,1cyclictest0-21swapper/223:00:162
1686999210,0cyclictest3145-21apt-get20:55:152
1686999210,0cyclictest19487-21apt-get19:15:122
1686999210,0cyclictest0-21swapper/222:35:152
16864992121,0cyclictest7235-21apt-get23:15:161
16864992121,0cyclictest0-21swapper/122:05:141
16864992121,0cyclictest0-21swapper/120:20:151
16864992120,1cyclictest0-21swapper/122:00:131
16864992120,0cyclictest0-21swapper/122:50:151
16864992120,0cyclictest0-21swapper/122:15:171
16864992119,1cyclictest0-21swapper/122:30:151
1686499210,21cyclictest0-21swapper/119:55:151
1686499210,1cyclictest0-21swapper/120:50:011
1686499210,0cyclictest0-21swapper/123:25:021
1686499210,0cyclictest0-21swapper/122:20:151
1686499210,0cyclictest0-21swapper/121:15:151
1686499210,0cyclictest0-21swapper/121:00:151
1686499210,0cyclictest0-21swapper/120:40:141
1686499210,0cyclictest0-21swapper/120:25:121
16860992121,0cyclictest0-21swapper/019:15:120
16860992120,1cyclictest0-21swapper/022:05:140
16860992119,1cyclictest0-21swapper/022:15:170
1686099210,21cyclictest0-21swapper/021:20:020
1686099210,20cyclictest32465-21apt-get23:00:160
1686099210,0cyclictest0-21swapper/021:55:130
1686099210,0cyclictest0-21swapper/021:15:150
16873992020,0cyclictest20759-21apt-get20:25:123
16873992020,0cyclictest11440-21apt-get00:30:163
16873992020,0cyclictest0-21swapper/322:35:153
16873992020,0cyclictest0-21swapper/321:15:153
16873992020,0cyclictest0-21swapper/321:05:133
16873992020,0cyclictest0-21swapper/319:50:163
16873992020,0cyclictest0-21swapper/319:30:143
16873992020,0cyclictest0-21swapper/319:20:143
16873992020,0cyclictest0-21swapper/319:15:123
16873992018,1cyclictest8698-21cron00:25:003
16873992018,1cyclictest19235-21grep19:15:013
16873992018,1cyclictest12209-21awk22:20:013
16873992015,4cyclictest0-21swapper/322:20:153
1687399200,20cyclictest0-21swapper/320:10:153
1687399200,19cyclictest23020-21ls22:40:203
1687399200,19cyclictest14892-21apt-get23:30:143
1687399200,19cyclictest0-21swapper/322:55:153
1687399200,19cyclictest0-21swapper/322:10:013
1687399200,0cyclictest0-21swapper/320:15:153
16869992020,0cyclictest0-21swapper/223:15:162
16869992020,0cyclictest0-21swapper/221:00:022
16869992020,0cyclictest0-21swapper/219:40:162
16869992020,0cyclictest0-21swapper/200:30:152
16869992019,0cyclictest0-21swapper/221:40:012
16869992018,1cyclictest21828-21awk23:45:012
1686999200,0cyclictest0-21swapper/219:35:132
1686999200,0cyclictest0-21swapper/219:25:152
16864992020,0cyclictest31543-21perf00:05:011
16864992020,0cyclictest0-21swapper/123:50:151
16864992020,0cyclictest0-21swapper/123:50:151
16864992020,0cyclictest0-21swapper/123:40:151
16864992020,0cyclictest0-21swapper/122:45:151
16864992020,0cyclictest0-21swapper/121:55:131
16864992020,0cyclictest0-21swapper/121:45:151
16864992020,0cyclictest0-21swapper/120:00:141
16864992020,0cyclictest0-21swapper/100:10:151
16864992019,0cyclictest0-21swapper/122:10:141
16860992020,0cyclictest27650-21apt-get22:50:140
16860992020,0cyclictest0-21swapper/023:55:160
16860992020,0cyclictest0-21swapper/023:50:160
16860992020,0cyclictest0-21swapper/023:50:160
16860992020,0cyclictest0-21swapper/023:30:140
16860992020,0cyclictest0-21swapper/023:15:160
16860992020,0cyclictest0-21swapper/021:40:140
16860992020,0cyclictest0-21swapper/020:30:140
16860992020,0cyclictest0-21swapper/020:20:150
16860992020,0cyclictest0-21swapper/019:50:160
16860992020,0cyclictest0-21swapper/019:40:150
16860992020,0cyclictest0-21swapper/019:25:140
16860992019,1cyclictest0-21swapper/000:15:160
16860992019,0cyclictest0-21swapper/000:10:150
1686099200,20cyclictest0-21swapper/000:35:130
1686099200,0cyclictest32763-21apt-get20:50:150
1686099200,0cyclictest0-21swapper/023:25:020
16873991919,0cyclictest1788-21apt-get00:10:153
16873991919,0cyclictest0-21swapper/323:20:153
16873991919,0cyclictest0-21swapper/321:25:153
16873991917,1cyclictest6946-21awk23:15:013
1687399190,19cyclictest0-21swapper/323:35:193
1687399190,18cyclictest22186-21sensors_temp21:30:213
1687399190,18cyclictest10027-21awk00:25:183
1687399190,16cyclictest15969-21sendmail-msp00:40:013
16869991919,0cyclictest5202-21apt-get22:05:142
16869991919,0cyclictest10049-21apt-get22:15:172
16869991919,0cyclictest0-21swapper/221:50:142
16869991918,1cyclictest0-21swapper/223:35:152
16869991918,0cyclictest9709-21apt-get23:20:152
16869991917,1cyclictest281ksoftirqd/222:15:012
16869991916,2cyclictest2475-21awk20:55:012
1686999190,18cyclictest0-21swapper/222:00:132
1686999190,0cyclictest0-21swapper/219:20:132
16864991919,0cyclictest0-21swapper/123:00:151
16864991918,0cyclictest26700-21apt-get19:30:141
16860991919,0cyclictest0-21swapper/023:40:150
16860991919,0cyclictest0-21swapper/021:45:150
16860991919,0cyclictest0-21swapper/020:00:140
16860991918,1cyclictest0-21swapper/022:10:140
16860991917,1cyclictest0-21swapper/020:55:150
16873991818,0cyclictest0-21swapper/321:00:153
16873991817,1cyclictest4184-21apt-get00:15:163
16873991816,1cyclictest1190-21cron19:45:013
1687399180,18cyclictest0-21swapper/321:50:223
1687399180,17cyclictest24485-21/usr/sbin/munin21:35:213
1687399180,17cyclictest0-21swapper/320:00:013
1687399180,13cyclictest744-21sensors_fan00:05:213
16869991818,0cyclictest0-21swapper/223:25:152
16869991818,0cyclictest0-21swapper/222:40:162
16869991818,0cyclictest0-21swapper/221:40:142
16869991818,0cyclictest0-21swapper/221:25:152
16869991818,0cyclictest0-21swapper/219:50:162
16869991818,0cyclictest0-21swapper/200:35:132
16869991816,1cyclictest6339-21awk00:20:002
16869991815,2cyclictest609-21rs:main2
16864991818,0cyclictest2486-21apt-get20:55:031
16864991818,0cyclictest0-21swapper/123:30:151
16864991818,0cyclictest0-21swapper/119:10:121
16864991817,1cyclictest19659-21apt-get22:35:161
16864991816,1cyclictest2488-21latency_hist20:55:011
1686499180,17cyclictest0-21swapper/121:30:131
16860991818,0cyclictest0-21swapper/022:30:150
16860991817,1cyclictest0-21swapper/020:15:150
1686099180,17cyclictest0-21swapper/021:30:130
16873991716,1cyclictest25616-21apt-get21:40:133
1687399170,16cyclictest0-21swapper/322:45:193
16869991717,0cyclictest31800-21apt-get00:05:142
16869991717,0cyclictest0-21swapper/220:20:142
16864991717,0cyclictest0-21swapper/120:30:151
16864991717,0cyclictest0-21swapper/120:15:151
16864991715,1cyclictest0-21swapper/119:25:001
1686499170,5cyclictest0-21swapper/121:05:131
16860991717,0cyclictest0-21swapper/019:30:140
16860991717,0cyclictest0-21swapper/019:10:120
16873991616,0cyclictest1489-21apt-get19:45:173
1687399160,0cyclictest16716-21mii-tool21:20:183
16869991616,0cyclictest0-21swapper/219:30:142
16869991615,0cyclictest0-21swapper/220:45:152
16869991614,1cyclictest19373-21sh22:35:012
1686999161,15cyclictest12474-21unixbench_singl21:10:232
16864991616,0cyclictest0-21swapper/123:35:151
16864991615,0cyclictest0-21swapper/100:05:141
16864991614,1cyclictest32338-21chrt21:52:571
16860991616,0cyclictest0-21swapper/022:35:160
16860991616,0cyclictest0-21swapper/021:35:140
16860991616,0cyclictest0-21swapper/020:45:140
16860991614,1cyclictest0-21swapper/000:05:140
1686099160,15cyclictest0-21swapper/023:20:150
16873991515,0cyclictest0-21swapper/323:06:453
1687399150,0cyclictest0-21swapper/322:10:273
16869991515,0cyclictest0-21swapper/221:15:152
1686999150,1cyclictest17520-21diskstats19:10:142
16864991515,0cyclictest0-21swapper/123:05:141
16864991514,1cyclictest0-21swapper/123:55:081
16864991513,1cyclictest0-21swapper/100:30:151
16860991515,0cyclictest17296-21apt-get23:35:150
16860991515,0cyclictest0-21swapper/023:45:180
16860991515,0cyclictest0-21swapper/023:05:140
16860991515,0cyclictest0-21swapper/021:10:130
16860991515,0cyclictest0-21swapper/020:10:150
1686099150,8cyclictest0-21swapper/019:35:130
1687399140,14cyclictest0-21swapper/323:51:043
1687399140,14cyclictest0-21swapper/323:51:043
16869991414,0cyclictest0-21swapper/222:52:112
16869991414,0cyclictest0-21swapper/221:05:042
16869991414,0cyclictest0-21swapper/220:29:402
16869991414,0cyclictest0-21swapper/220:00:032
1686999140,14cyclictest0-21swapper/221:30:062
1686999140,14cyclictest0-21swapper/200:00:152
1686999140,0cyclictest0-21swapper/223:10:082
1686999140,0cyclictest0-21swapper/222:25:052
16864991412,1cyclictest31517-21apt-get19:40:151
1686499140,14cyclictest0-21swapper/121:25:251
1686499140,14cyclictest0-21swapper/119:46:211
1686499140,0cyclictest0-21swapper/123:21:131
16860991414,0cyclictest0-21swapper/022:56:320
16860991414,0cyclictest0-21swapper/022:00:490
16860991414,0cyclictest0-21swapper/019:46:210
16860991413,0cyclictest0-21swapper/000:30:150
1686099140,14cyclictest0-21swapper/023:10:120
1686099140,0cyclictest0-21swapper/021:50:060
1686099140,0cyclictest0-21swapper/021:05:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional