You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-16 - 00:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Jul 15, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
95499270,7cyclictest12069-21latency_hist10:50:003
95499260,25cyclictest0-21swapper/310:55:593
95499260,25cyclictest0-21swapper/307:50:573
95499260,0cyclictest0-21swapper/309:21:003
95499250,25cyclictest0-21swapper/308:10:573
95499250,24cyclictest7407-21latency_hist09:35:003
95499250,0cyclictest0-21swapper/310:05:593
954992420,4cyclictest0-21swapper/308:26:023
948992418,5cyclictest6076-21awk11:45:001
954992319,4cyclictest0-21swapper/310:00:583
95499230,22cyclictest5772-21latency_hist07:20:003
95499230,22cyclictest29221-21apt-get11:25:593
95499230,22cyclictest0-21swapper/310:20:003
95499230,0cyclictest0-21swapper/312:00:023
95099230,22cyclictest10927-21dump-pmu-power11:54:592
95099230,22cyclictest0-21swapper/210:00:582
95099230,1cyclictest2395-21basename09:25:012
948992321,1cyclictest7866-21apt-get08:30:571
94899230,22cyclictest1243-21apt-get07:10:571
94899230,0cyclictest17226-21apt-get11:01:001
943992320,2cyclictest26075-21apt-get12:25:570
954992221,1cyclictest0-21swapper/309:15:023
954992221,0cyclictest0-21swapper/312:11:043
95499220,21cyclictest28159-21sed08:05:003
95499220,21cyclictest1669-21unixbench_singl10:26:093
95499220,21cyclictest0-21swapper/309:56:073
95499220,21cyclictest0-21swapper/309:56:063
95499220,21cyclictest0-21swapper/308:50:003
950992221,0cyclictest5118-21apt-get09:30:582
950992220,1cyclictest6426-21apt-get11:46:002
95099220,21cyclictest8141-21cat07:25:002
95099220,21cyclictest1269-21latency_hist11:35:002
95099220,21cyclictest0-21swapper/210:05:592
95099220,1cyclictest27522-21apt-get09:10:582
95099220,1cyclictest27207-21basename09:10:002
95099220,1cyclictest23356-21perf12:20:002
95099220,0cyclictest0-21swapper/210:56:002
948992221,1cyclictest0-21swapper/111:10:591
948992220,1cyclictest27554-21apt-get10:15:591
948992220,1cyclictest20917-21latency_hist12:15:001
948992220,1cyclictest10934-21perf11:54:591
94899220,21cyclictest9986-21apt-get09:40:571
94899220,21cyclictest3649-21apt-get07:15:581
94899220,21cyclictest31616-21apt-get11:30:591
94899220,21cyclictest23357-21munin-run12:20:001
94899220,21cyclictest20619-21apt-get07:50:571
94899220,21cyclictest19915-21apt-get08:55:561
94899220,21cyclictest15830-21apt-get07:40:581
94899220,1cyclictest12405-21idleruntime-cro08:40:001
943992221,1cyclictest0-21swapper/009:00:580
943992221,0cyclictest19585-21apt-get10:00:590
943992220,1cyclictest12093-21munin-run10:50:000
94399220,21cyclictest593-21dbus-daemon10:25:000
94399220,21cyclictest4763-21apt-get10:36:010
94399220,21cyclictest4032-21apt-get11:41:010
94399220,21cyclictest21654-21apt-get10:05:030
94399220,21cyclictest12412-21apt-get10:50:570
94399220,21cyclictest12038-21turbostat09:45:000
94399220,21cyclictest0-21swapper/009:56:010
94399220,21cyclictest0-21swapper/009:56:010
94399220,21cyclictest0-21swapper/009:20:000
94399220,21cyclictest0-21swapper/008:50:580
94399220,0cyclictest0-21swapper/011:11:010
954992119,2cyclictest0-21swapper/309:25:583
954992119,1cyclictest12036-21awk09:45:003
954992118,3cyclictest0-21swapper/310:40:583
95499210,21cyclictest0-21swapper/311:11:003
95499210,20cyclictest25666-21fschecks_time09:06:003
95499210,20cyclictest18621-21cat07:46:003
95499210,20cyclictest0-21swapper/312:05:553
95499210,20cyclictest0-21swapper/311:56:003
95499210,20cyclictest0-21swapper/311:05:593
95499210,20cyclictest0-21swapper/311:01:033
95499210,20cyclictest0-21swapper/310:50:583
95499210,20cyclictest0-21swapper/308:35:573
95499210,0cyclictest0-21swapper/309:45:593
95499210,0cyclictest0-21swapper/308:50:573
950992120,1cyclictest0-21swapper/212:31:012
950992120,1cyclictest0-21swapper/212:15:002
950992120,1cyclictest0-21swapper/212:00:012
950992120,0cyclictest22014-21apt-get11:11:002
950992120,0cyclictest0-21swapper/208:50:582
950992119,1cyclictest4763-21cron09:30:002
950992119,1cyclictest12407-21sendmail-msp08:40:002
950992119,1cyclictest10574-21turbostat.cron07:30:002
95099210,20cyclictest21980-21idleruntime-cro09:00:002
95099210,20cyclictest0-21swapper/211:25:592
95099210,20cyclictest0-21swapper/208:10:002
95099210,20cyclictest0-21swapper/207:55:572
95099210,0cyclictest22732-21apt-get09:00:582
95099210,0cyclictest0-21swapper/212:35:592
95099210,0cyclictest0-21swapper/210:40:582
95099210,0cyclictest0-21swapper/207:40:572
948992121,0cyclictest5475-21apt-get08:26:021
948992121,0cyclictest0-21swapper/107:25:571
948992120,1cyclictest10914-21apt-get07:30:581
948992120,0cyclictest16470-21apt-get12:05:551
948992119,1cyclictest14771-21apt-get09:50:571
94899210,21cyclictest0-21swapper/112:25:571
94899210,1cyclictest3077-21apt-get08:20:581
94899210,0cyclictest23064-21apt-get07:55:571
94899210,0cyclictest0-21swapper/111:55:591
94899210,0cyclictest0-21swapper/109:25:581
943992120,1cyclictest0-21swapper/008:00:010
943992120,0cyclictest0-21swapper/011:20:580
943992120,0cyclictest0-21swapper/008:35:570
943992120,0cyclictest0-21swapper/007:40:570
943992119,1cyclictest6071-21apt-get07:21:000
943992118,2cyclictest16836-21basename09:55:000
94399210,21cyclictest0-21swapper/012:00:580
94399210,20cyclictest0-21swapper/012:31:020
94399210,20cyclictest0-21swapper/011:45:590
94399210,20cyclictest0-21swapper/011:35:580
94399210,20cyclictest0-21swapper/009:05:580
94399210,20cyclictest0-21swapper/007:30:580
94399210,1cyclictest0-21swapper/010:15:000
94399210,0cyclictest0-21swapper/012:20:590
94399210,0cyclictest0-21swapper/007:10:570
954992020,0cyclictest0-21swapper/307:21:003
95499200,20cyclictest29538-21/usr/sbin/munin08:06:063
95499200,1cyclictest0-21swapper/312:31:043
95499200,19cyclictest27440-21sed11:21:003
95499200,19cyclictest0-21swapper/309:11:063
95499200,19cyclictest0-21swapper/307:56:043
95499200,19cyclictest0-21swapper/307:36:003
95499200,18cyclictest0-21swapper/311:31:033
95499200,0cyclictest0-21swapper/310:36:023
95499200,0cyclictest0-21swapper/309:00:593
95499200,0cyclictest0-21swapper/308:41:033
95499200,0cyclictest0-21swapper/307:40:583
950992020,0cyclictest15130-21apt-get08:46:002
950992020,0cyclictest0-21swapper/212:20:582
950992019,1cyclictest29577-21apt-get09:15:032
950992019,0cyclictest0-21swapper/208:26:022
950992018,1cyclictest25158-21apt-get10:10:562
95099200,0cyclictest0-21swapper/210:25:592
948992020,0cyclictest21993-21apt-get09:00:011
948992020,0cyclictest0-21swapper/111:45:591
948992020,0cyclictest0-21swapper/110:21:001
948992020,0cyclictest0-21swapper/110:05:021
948992018,1cyclictest4427-21perf10:35:001
94899200,20cyclictest0-21swapper/110:00:581
94899200,20cyclictest0-21swapper/109:10:591
94899200,19cyclictest0-21swapper/108:10:571
94899200,0cyclictest25139-21apt-get08:00:021
94899200,0cyclictest0-21swapper/110:50:571
943992020,0cyclictest0-21swapper/010:40:580
943992020,0cyclictest0-21swapper/010:15:580
943992020,0cyclictest0-21swapper/008:46:010
943992018,1cyclictest22406-21apt-config07:51:320
943992018,1cyclictest2037-21munin-run10:30:000
94399200,20cyclictest0-21swapper/007:36:000
94399200,20cyclictest0-21swapper/007:15:590
94399200,19cyclictest0-21swapper/009:10:580
94399200,0cyclictest0-21swapper/012:11:040
94399200,0cyclictest0-21swapper/011:06:010
954991919,0cyclictest0-21swapper/311:41:013
954991919,0cyclictest0-21swapper/310:30:573
95499190,19cyclictest0-21swapper/311:50:003
95499190,18cyclictest0-21swapper/307:11:043
95499190,0cyclictest0-21swapper/312:36:053
950991919,0cyclictest0-21swapper/210:50:572
950991919,0cyclictest0-21swapper/208:40:592
950991919,0cyclictest0-21swapper/208:00:562
950991917,1cyclictest32034-21awk10:25:002
950991917,1cyclictest26498-21sleep11:20:002
948991919,0cyclictest14810-21apt-get10:56:001
948991919,0cyclictest0-21swapper/111:15:591
948991919,0cyclictest0-21swapper/107:21:001
948991918,0cyclictest13312-21apt-get07:36:001
943991919,0cyclictest0-21swapper/011:50:580
943991919,0cyclictest0-21swapper/009:45:580
943991919,0cyclictest0-21swapper/009:30:570
943991919,0cyclictest0-21swapper/008:40:580
943991919,0cyclictest0-21swapper/008:15:590
943991919,0cyclictest0-21swapper/007:45:590
943991918,1cyclictest30870-21apt-get12:36:000
943991918,1cyclictest2366-21apt-get10:30:580
943991917,1cyclictest1263-21dump-pmu-power11:35:000
954991818,0cyclictest0-21swapper/312:25:573
954991818,0cyclictest0-21swapper/309:50:583
954991817,1cyclictest0-21swapper/308:15:593
95499180,0cyclictest0-21swapper/312:16:003
950991818,0cyclictest0-21swapper/212:05:552
950991818,0cyclictest0-21swapper/211:41:022
950991818,0cyclictest0-21swapper/210:36:022
950991818,0cyclictest0-21swapper/209:56:012
950991818,0cyclictest0-21swapper/209:56:012
950991818,0cyclictest0-21swapper/209:40:562
950991818,0cyclictest0-21swapper/207:15:582
948991818,0cyclictest621-21apt-get08:15:591
948991818,0cyclictest0-21swapper/112:31:021
948991818,0cyclictest0-21swapper/110:10:581
948991818,0cyclictest0-21swapper/109:20:591
948991817,1cyclictest23682-21apt-get12:20:581
948991815,2cyclictest16118-21cron12:05:001
943991818,0cyclictest0-21swapper/012:05:550
94399180,17cyclictest9639-21perf09:40:000
94399180,17cyclictest0-21swapper/007:55:580
94399180,0cyclictest0-21swapper/008:10:570
954991717,0cyclictest7515-21apt-get09:35:573
954991717,0cyclictest0-21swapper/312:20:583
954991717,0cyclictest0-21swapper/311:35:583
954991717,0cyclictest0-21swapper/310:20:583
950991717,0cyclictest11269-21apt-get11:55:592
950991717,0cyclictest0-21swapper/211:20:582
950991717,0cyclictest0-21swapper/208:30:572
950991717,0cyclictest0-21swapper/208:20:592
950991717,0cyclictest0-21swapper/207:30:592
950991717,0cyclictest0-21swapper/207:10:572
948991717,0cyclictest17522-21apt-get08:50:581
948991716,0cyclictest1602-21apt-get11:35:581
948991715,1cyclictest9661-21sendmail-msp09:40:001
948991715,1cyclictest12380-21apt-get09:45:581
94899170,0cyclictest0-21swapper/110:45:581
943991717,0cyclictest8462-21apt-get07:25:580
943991717,0cyclictest0-21swapper/011:55:590
943991716,1cyclictest24408-21apt-get11:16:000
954991616,0cyclictest0-21swapper/308:20:583
954991616,0cyclictest0-21swapper/307:25:583
95499160,16cyclictest0-21swapper/310:11:153
95499160,14cyclictest0-21swapper/308:30:423
950991616,0cyclictest0-21swapper/209:50:562
950991614,1cyclictest28150-21munin-run12:30:002
948991616,0cyclictest28272-21apt-get08:05:591
94899160,15cyclictest19881-21expr11:06:001
954991515,0cyclictest0-21swapper/307:30:583
954991514,1cyclictest0-21swapper/308:55:563
95499150,15cyclictest0-21swapper/311:51:063
950991515,0cyclictest0-21swapper/211:06:002
950991515,0cyclictest0-21swapper/210:30:582
950991515,0cyclictest0-21swapper/209:35:592
950991513,1cyclictest2750-21munin-run08:20:012
948991515,0cyclictest30313-21apt-get09:15:581
948991515,0cyclictest0-21swapper/112:36:011
948991515,0cyclictest0-21swapper/109:56:011
948991515,0cyclictest0-21swapper/109:56:011
948991515,0cyclictest0-21swapper/108:46:001
943991515,0cyclictest0-21swapper/008:05:590
94399150,1cyclictest0-21swapper/009:21:010
95499140,14cyclictest0-21swapper/311:16:243
950991414,0cyclictest0-21swapper/207:51:282
950991413,1cyclictest0-21swapper/210:45:582
95099140,14cyclictest0-21swapper/211:35:232
95099140,14cyclictest0-21swapper/211:00:042
95099140,14cyclictest0-21swapper/209:45:412
95099140,14cyclictest0-21swapper/208:10:342
95099140,14cyclictest0-21swapper/207:45:352
95099140,14cyclictest0-21swapper/207:37:532
95099140,0cyclictest0-21swapper/210:16:302
948991414,0cyclictest0-21swapper/111:21:021
948991414,0cyclictest0-21swapper/110:35:131
94899140,14cyclictest0-21swapper/110:40:041
94899140,14cyclictest0-21swapper/109:30:081
94899140,0cyclictest0-21swapper/111:25:471
94899140,0cyclictest0-21swapper/109:05:161
94899140,0cyclictest0-21swapper/108:40:211
94899140,0cyclictest0-21swapper/107:45:061
943991414,0cyclictest0-21swapper/012:15:060
943991414,0cyclictest0-21swapper/008:21:200
94399140,14cyclictest0-21swapper/011:00:330
94399140,14cyclictest0-21swapper/009:25:030
94399140,14cyclictest0-21swapper/008:30:050
94399140,0cyclictest0-21swapper/011:25:070
94399140,0cyclictest0-21swapper/010:55:050
94399140,0cyclictest0-21swapper/008:56:430
94399140,0cyclictest0-21swapper/008:25:370
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional