You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 14:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Tue Dec 02, 2025 00:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
808108997770,2cyclictest0-21swapper/023:15:020
808108997125,41cyclictest235-21systemd-journal23:10:000
808108997123,13cyclictest823134-21sshd23:55:180
808123997021,44cyclictest272-21dbus-daemon19:15:013
808123997015,5cyclictest823208-40mandb00:00:083
80812399692,15cyclictest823976-21munin-run00:15:003
80812399661,12cyclictest814568-21sshd21:10:143
808108996621,38cyclictest1-21systemd22:20:010
80812399659,6cyclictest821341-21sshd23:20:143
80810899642,17cyclictest809873-21cron19:45:000
808123996355,6cyclictest37-21ksoftirqd/322:35:023
80812399632,16cyclictest818026-21munin-run22:20:013
80812399632,16cyclictest811446-21sshd20:11:123
808108996327,31cyclictest272-21dbus-daemon22:35:010
808108996322,2cyclictest0-21swapper/021:13:150
80812399622,15cyclictest819563-21munin-run22:50:013
80812399622,14cyclictest816907-21sshd21:57:143
80812399622,13cyclictest813320-21munin-run20:50:013
80812399621,14cyclictest817693-21sshd22:12:143
808108996224,35cyclictest12-21ksoftirqd/023:48:450
80812399612,14cyclictest812486-21sshd20:30:153
80812399612,12cyclictest814366-21munin-run21:10:013
80812399611,13cyclictest809331-21sshd19:32:403
808108996145,12cyclictest1-21systemd20:10:280
808108996119,11cyclictest12-21ksoftirqd/021:30:010
80810899602,19cyclictest823956-21modprobe00:14:590
808108996021,7cyclictest824686-21latency_hist00:25:020
808108996021,7cyclictest824686-21latency_hist00:25:020
808108996019,35cyclictest16950irq/224-800000021:55:000
80812399592,15cyclictest823711-21munin-run00:10:013
80812399592,15cyclictest820332-21munin-run23:05:013
80812399592,13cyclictest822903-21modprobe23:54:193
80812399592,13cyclictest812271-21munin-run20:30:003
80812399592,12cyclictest808851-21munin-run19:25:003
80812399591,14cyclictest363-21snmpd00:18:373
80812399591,13cyclictest819312-21munin-run22:45:013
808112995947,7cyclictest0-21swapper/119:10:011
80812399582,13cyclictest816445-21munin-run21:50:013
80812399581,12cyclictest810360-21modprobe19:51:123
80812399581,12cyclictest809070-21sshd19:26:123
80810899582,20cyclictest306-21in:imuxsock20:10:010
80812399572,16cyclictest272-21dbus-daemon21:45:003
80812399572,15cyclictest821895-21munin-run23:35:013
808123995716,37cyclictest822637-21sshd23:47:183
808123995714,11cyclictest815756-21sed21:35:023
80810899574,15cyclictest813042-21cron20:45:000
808108995724,29cyclictest12-21ksoftirqd/023:35:020
80810899572,13cyclictest815652-21modprobe21:35:000
80812399562,17cyclictest822413-21munin-run23:45:003
80812399561,13cyclictest363-21snmpd23:16:203
80810899562,22cyclictest0-21swapper/000:35:010
808108995620,33cyclictest19050irq/34-mmc122:55:010
80812399553,15cyclictest813572-21munin-run20:55:013
80812399552,13cyclictest363-21snmpd00:22:473
80812399552,13cyclictest363-21snmpd00:22:473
808123995520,31cyclictest812185-21sed20:25:023
808123995520,31cyclictest37-21ksoftirqd/322:25:003
808123995519,31cyclictest272-21dbus-daemon19:20:013
80812399551,14cyclictest819780-21sshd22:53:283
80810899552,24cyclictest16950irq/224-800000021:43:110
808108995518,11cyclictest272-21dbus-daemon19:40:000
808108995516,12cyclictest272-21dbus-daemon20:20:010
80812399542,16cyclictest811746-21munin-run20:20:013
808123995421,29cyclictest37-21ksoftirqd/320:10:033
80810899541,4cyclictest0-21swapper/020:01:520
808108995412,11cyclictest1-21systemd21:25:000
80812399532,10cyclictest816670-21cron21:54:593
808123995314,7cyclictest812741-21sshd20:36:143
808123995311,4cyclictest810768-21latency_hist20:00:023
80812399531,12cyclictest820037-21sshd22:59:143
80812399531,11cyclictest235-21systemd-journal22:10:003
808119995329,19cyclictest235-21systemd-journal20:20:002
80810899533,15cyclictest813547-21modprobe20:55:000
80810899533,14cyclictest808159-21wc19:10:020
808108995325,25cyclictest0-21swapper/000:39:200
808108995322,10cyclictest819279-21sshd22:43:240
80810899532,19cyclictest812958-21latency_hist20:40:020
808108995313,10cyclictest12-21ksoftirqd/021:00:000
80812399524,11cyclictest235-21systemd-journal19:35:033
80812399523,8cyclictest822122-21cron23:40:003
808123995230,18cyclictest825278-21latency_hist00:40:003
808123995228,21cyclictest37-21ksoftirqd/323:30:033
80812399522,12cyclictest823198-21basename00:00:003
808119995231,17cyclictest272-21dbus-daemon21:25:002
808112995236,11cyclictest235-21systemd-journal23:30:011
808112995233,14cyclictest272-21dbus-daemon19:20:011
808108995235,12cyclictest1-21systemd19:25:000
808108995218,8cyclictest810507-21sed19:55:020
80812399512,15cyclictest819030-21munin-run22:40:003
80810899512,17cyclictest808604-21modprobe19:20:010
80810899512,17cyclictest235-21systemd-journal21:40:000
808108995118,8cyclictest12-21ksoftirqd/000:07:200
808108995115,6cyclictest822243-21latency_hist23:40:020
808108995112,9cyclictest820940-21ls23:15:020
80810899511,22cyclictest0-21swapper/020:47:000
808123995024,23cyclictest37-21ksoftirqd/321:36:153
80812399501,12cyclictest235-21systemd-journal22:25:023
808119995032,9cyclictest306-21in:imuxsock22:50:012
808112995041,7cyclictest23-21ksoftirqd/100:00:021
808112995035,10cyclictest23-21ksoftirqd/120:05:021
808108995020,5cyclictest810646-21latency_hist20:00:010
808108995020,4cyclictest0-21swapper/022:02:430
808108995020,4cyclictest0-21swapper/022:02:420
808108995011,8cyclictest306-21in:imuxsock23:00:000
80810899501,16cyclictest363-21snmpd22:00:010
80812399499,9cyclictest37-21ksoftirqd/321:00:013
808123994918,26cyclictest37-21ksoftirqd/322:05:013
808123994918,26cyclictest37-21ksoftirqd/322:05:013
808123994912,4cyclictest821040-21latency_hist23:15:023
80812399491,12cyclictest814875-21grep21:20:003
80812399491,12cyclictest814875-21grep21:20:003
80811999498,7cyclictest30-21ksoftirqd/200:15:012
808108994945,2cyclictest0-21swapper/022:10:010
80810899491,44cyclictest0-21swapper/020:25:020
80812399481,13cyclictest824767-21systemd-run00:30:013
80811999484,5cyclictest131rcu_preempt22:15:002
808119994832,12cyclictest817988-21cat22:20:002
808112994836,7cyclictest817992-21modprobe22:20:001
80810899483,9cyclictest12-21ksoftirqd/020:35:010
80810899482,15cyclictest822515-21sort23:45:020
80810899481,22cyclictest0-21swapper/020:25:020
80810899481,21cyclictest0-21swapper/022:20:120
80812399477,11cyclictest272-21dbus-daemon19:50:013
80812399473,14cyclictest815399-21cut21:30:003
80812399472,14cyclictest813056-21munin-plugin-st20:45:003
808123994713,7cyclictest37-21ksoftirqd/319:41:473
808119994732,10cyclictest823190-21tr00:00:012
808112994729,13cyclictest235-21systemd-journal21:55:001
808108994713,8cyclictest12-21ksoftirqd/000:17:460
80810899471,2cyclictest0-21swapper/000:30:020
80812399462,8cyclictest808122-21sort19:10:013
808123994627,13cyclictest272-21dbus-daemon20:05:013
80812399462,13cyclictest815132-21cron21:25:003
808119994631,7cyclictest306-21in:imuxsock20:05:002
808119994625,16cyclictest272-21dbus-daemon19:15:012
808119994620,16cyclictest235-21systemd-journal22:05:002
808119994620,16cyclictest235-21systemd-journal22:05:002
808112994631,11cyclictest235-21systemd-journal20:32:141
808112994629,11cyclictest235-21systemd-journal23:15:001
808112994628,13cyclictest1-21systemd21:20:011
808112994628,13cyclictest1-21systemd21:20:011
80810899462,23cyclictest0-21swapper/021:18:280
80810899462,23cyclictest0-21swapper/021:18:280
80810899462,17cyclictest814353-21grep21:10:010
808108994620,7cyclictest12-21ksoftirqd/023:25:020
80810899461,22cyclictest0-21swapper/019:45:370
808119994522,19cyclictest30-21ksoftirqd/219:20:012
808119994520,16cyclictest272-21dbus-daemon22:25:002
808112994529,13cyclictest811712-21sshd20:17:151
808112994529,11cyclictest272-21dbus-daemon00:40:011
808108994520,4cyclictest0-21swapper/022:48:080
80810899451,26cyclictest0-21swapper/019:13:170
80810899451,22cyclictest0-21swapper/023:50:040
808108994511,9cyclictest1-21systemd23:30:020
80811999444,31cyclictest812272-21munin-plugin-st20:29:592
80811999441,11cyclictest812879-21latency_hist20:40:022
808112994426,14cyclictest235-21systemd-journal00:25:011
808112994426,14cyclictest235-21systemd-journal00:25:001
808112994417,22cyclictest23-21ksoftirqd/100:35:011
808112994411,6cyclictest131rcu_preempt20:30:001
80810899442,12cyclictest818517-21munin-run22:30:000
80810899441,39cyclictest0-21swapper/022:36:090
80812399432,13cyclictest820750-21cat23:10:023
808119994331,7cyclictest30-21ksoftirqd/223:35:012
808119994329,10cyclictest1-21systemd21:45:002
808119994328,12cyclictest820070-21tr23:00:002
808119994324,13cyclictest272-21dbus-daemon19:45:012
808112994330,10cyclictest23-21ksoftirqd/120:40:021
808112994326,13cyclictest235-21systemd-journal00:05:021
808108994337,4cyclictest16950irq/224-800000000:00:030
808119994228,9cyclictest30-21ksoftirqd/220:25:012
808112994230,6cyclictest819800-21munin-plugin-st22:55:001
808112994228,10cyclictest306-21in:imuxsock00:00:201
808112994223,13cyclictest272-21dbus-daemon20:50:011
80810899421,37cyclictest0-21swapper/019:26:450
808119994131,6cyclictest818515-21munin-plugin-st22:30:012
808119994128,9cyclictest306-21in:imuxsock19:48:112
808119994124,11cyclictest306-21in:imuxsock23:10:002
808119994119,17cyclictest272-21dbus-daemon00:10:012
808112994125,11cyclictest1-21systemd21:50:011
808112994114,4cyclictest408-21lldpd23:53:221
80810899411,23cyclictest0-21swapper/023:04:410
80812399402,11cyclictest825159-21cat00:35:023
808119994029,6cyclictest306-21in:imuxsock21:40:002
808119994014,5cyclictest131rcu_preempt21:20:012
808119994014,5cyclictest131rcu_preempt21:20:012
80811299408,7cyclictest131rcu_preempt20:55:031
808112994031,5cyclictest306-21in:imuxsock22:30:001
808112994025,10cyclictest23-21ksoftirqd/123:25:011
808112994019,14cyclictest235-21systemd-journal22:50:001
808108994031,5cyclictest816430-21munin-plugin-st21:50:000
808119993929,7cyclictest822902-21sshd23:54:182
808119993928,6cyclictest824211-21cron00:20:002
808119993921,12cyclictest235-21systemd-journal22:55:002
808112993929,7cyclictest23-21ksoftirqd/119:30:001
808112993927,10cyclictest23-21ksoftirqd/122:15:011
808112993925,11cyclictest23-21ksoftirqd/123:05:021
808112993924,12cyclictest23-21ksoftirqd/120:10:301
808112993924,12cyclictest23-21ksoftirqd/119:40:021
808119993828,7cyclictest30-21ksoftirqd/200:40:002
808119993826,7cyclictest816939-21sendmail22:00:002
808112993828,7cyclictest810359-21sshd19:51:121
808112993826,9cyclictest235-21systemd-journal20:21:141
808112993825,10cyclictest23-21ksoftirqd/119:35:011
808112993823,10cyclictest272-21dbus-daemon23:10:001
80810899381,22cyclictest0-21swapper/021:04:290
80811999373,25cyclictest821631-21grep23:30:002
808119993723,9cyclictest306-21in:imuxsock00:03:482
808119993723,10cyclictest30-21ksoftirqd/200:35:012
808119993721,7cyclictest306-21in:imuxsock21:10:012
808119993717,15cyclictest272-21dbus-daemon23:20:012
80811999371,5cyclictest131rcu_preempt23:05:022
80811999371,5cyclictest131rcu_preempt19:25:012
808112993729,4cyclictest821103-21sendmail23:20:001
808112993728,5cyclictest814487-21latency_hist21:10:021
808112993727,6cyclictest819009-21cron22:40:001
808112993726,8cyclictest817215-21/usr/sbin/munin22:05:011
808112993726,8cyclictest817215-21/usr/sbin/munin22:05:011
808112993721,11cyclictest820077-21basename23:00:011
808112993719,12cyclictest272-21dbus-daemon22:00:001
808112993716,14cyclictest23-21ksoftirqd/100:29:591
80811999364,23cyclictest813799-21cron21:00:002
80811999363,20cyclictest822128-21modprobe23:40:002
808119993625,7cyclictest30-21ksoftirqd/222:40:012
808119993624,9cyclictest821356-21sshd23:23:342
808119993624,7cyclictest809621-21systemctl19:40:002
808119993623,9cyclictest30-21ksoftirqd/222:35:022
808119993622,11cyclictest811553-21sed20:15:012
808119993621,10cyclictest30-21ksoftirqd/220:10:022
808119993618,12cyclictest272-21dbus-daemon00:30:002
808112993628,6cyclictest23-21ksoftirqd/122:25:021
808112993626,7cyclictest822377-21sshd23:40:181
808112993625,8cyclictest23-21ksoftirqd/119:23:121
80811299362,3cyclictest131rcu_preempt19:13:111
808112993623,9cyclictest810109-21sshd19:46:121
808112993623,9cyclictest0-21swapper/122:35:011
808112993622,8cyclictest306-21in:imuxsock21:00:001
808108993626,7cyclictest12-21ksoftirqd/022:15:010
80811999357,19cyclictest819286-21cron22:45:002
808119993528,4cyclictest809099-21latency_hist19:30:022
808119993524,6cyclictest30-21ksoftirqd/220:35:022
808119993519,11cyclictest306-21in:imuxsock21:25:032
808119993518,9cyclictest306-21in:imuxsock20:45:002
808119993517,12cyclictest30-21ksoftirqd/221:35:002
808119993517,12cyclictest235-21systemd-journal21:55:002
80811299357,7cyclictest23-21ksoftirqd/121:45:021
808112993528,5cyclictest23-21ksoftirqd/120:00:021
808112993524,9cyclictest23-21ksoftirqd/121:30:021
80810899351,30cyclictest0-21swapper/019:30:220
808123993421,10cyclictest814177-21cat21:05:023
808119993425,6cyclictest822517-21latency_hist23:45:022
808119993422,9cyclictest30-21ksoftirqd/221:05:012
80811999342,24cyclictest814607-21cut21:15:002
80811299343,21cyclictest235-21systemd-journal20:45:001
808112993425,7cyclictest23-21ksoftirqd/123:40:001
808112993424,7cyclictest23-21ksoftirqd/100:20:021
808112993422,9cyclictest23-21ksoftirqd/123:35:001
808112993422,10cyclictest23-21ksoftirqd/121:15:001
80811999337,5cyclictest131rcu_preempt21:50:012
808119993325,5cyclictest810660-21sendmail20:00:002
808119993324,6cyclictest809493-21latency_hist19:35:022
808119993319,10cyclictest813753-21sed20:55:032
80811299332,3cyclictest131rcu_preempt19:40:001
808112993322,9cyclictest23-21ksoftirqd/120:10:021
808112993320,10cyclictest23-21ksoftirqd/122:10:011
80811999325,21cyclictest822666-21grep23:50:002
80811999324,23cyclictest810570-21grep19:55:032
808119993223,7cyclictest30-21ksoftirqd/220:50:012
808112993222,7cyclictest23-21ksoftirqd/121:39:141
808112993221,9cyclictest23-21ksoftirqd/122:45:001
808119993124,5cyclictest30-21ksoftirqd/222:10:012
808112993124,4cyclictest815161-21latency_hist21:25:011
808112993118,9cyclictest1-21systemd21:05:011
808112993116,11cyclictest272-21dbus-daemon23:50:001
808119993015,9cyclictest306-21in:imuxsock23:15:012
808112993021,6cyclictest23-21ksoftirqd/121:25:141
808112993018,8cyclictest235-21systemd-journal00:14:591
80811999291,6cyclictest131rcu_preempt19:10:012
80811999291,4cyclictest131rcu_preempt00:25:012
80811999291,4cyclictest131rcu_preempt00:25:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional