You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 14:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Tue Dec 09, 2025 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16717489911034,14cyclictest131rcu_preempt21:45:033
16717489911034,14cyclictest131rcu_preempt21:45:023
1671738998113,3cyclictest131rcu_preempt22:15:021
1671745998025,10cyclictest272-21dbus-daemon21:30:012
1671745998025,10cyclictest272-21dbus-daemon21:30:012
167174599777,66cyclictest1680808-21unin-run22:05:002
1671733997325,44cyclictest1679923-21sed21:45:030
1671733997325,44cyclictest1679923-21sed21:45:020
1671745997027,8cyclictest1676864-21sshd20:49:282
1671745997027,8cyclictest1676864-21sshd20:49:282
1671745996831,12cyclictest1-21systemd23:40:012
1671733996621,40cyclictest1-21systemd00:00:000
167173399652,17cyclictest1683350-21sshd22:52:420
167173399642,16cyclictest1681049-21munin-run22:10:000
1671745996322,11cyclictest30-21ksoftirqd/220:22:282
1671733996323,33cyclictest1674277-21systemctl20:00:000
1671733996321,38cyclictest12-21ksoftirqd/023:27:480
1671733996311,14cyclictest1685972-21cron23:45:000
1671745996225,5cyclictest1686772-21sendmail-msp00:00:002
1671745996123,11cyclictest30-21ksoftirqd/222:10:002
167173399612,15cyclictest48-21kcompactd000:05:000
167173399612,15cyclictest1675608-21munin-run20:25:010
1671745996029,4cyclictest1-21systemd00:05:002
167173399602,15cyclictest1676098-21munin-run20:35:000
1671733996020,33cyclictest1-21systemd22:15:010
1671733996020,33cyclictest1-21systemd22:15:010
167173399601,5cyclictest1687530-21latency_hist00:14:590
1671745995826,8cyclictest1682351-21systemd-run22:35:002
1671733995822,31cyclictest1685707-21sh23:40:010
1671745995723,7cyclictest1678705-21sshd21:24:282
167173399578,9cyclictest272-21dbus-daemon00:10:010
167173399578,9cyclictest272-21dbus-daemon00:10:010
1671733995753,2cyclictest0-21swapper/023:00:020
167173399572,26cyclictest1686440-21latency_hist23:50:020
1671733995718,34cyclictest1-21systemd20:45:010
1671733995711,7cyclictest0-21swapper/021:45:280
167173399571,15cyclictest1673360-21cat19:40:010
1671748995612,13cyclictest37-21ksoftirqd/300:40:013
1671745995626,10cyclictest30-21ksoftirqd/222:15:002
1671745995626,10cyclictest30-21ksoftirqd/222:15:002
1671733995620,32cyclictest0-21swapper/021:52:380
1671733995617,4cyclictest0-21swapper/023:15:020
167173399561,16cyclictest1-21systemd21:05:010
1671733995517,6cyclictest12-21ksoftirqd/022:25:250
1671733995517,6cyclictest12-21ksoftirqd/022:25:250
1671733995513,9cyclictest306-21in:imuxsock23:17:000
167173399542,16cyclictest1682602-21munin-run22:40:000
1671733995420,8cyclictest12-21ksoftirqd/021:15:010
167173399541,49cyclictest0-21swapper/019:53:020
1671733995325,24cyclictest1679612-21latency_hist21:40:020
167173399532,16cyclictest272-21dbus-daemon20:55:000
167173399532,11cyclictest235-21systemd-journal00:25:000
1671748995235,12cyclictest1-21systemd23:45:003
1671748995218,26cyclictest37-21ksoftirqd/322:15:023
1671748995218,26cyclictest37-21ksoftirqd/322:15:023
1671748995214,5cyclictest131rcu_preempt20:25:013
1671738995234,14cyclictest235-21systemd-journal20:05:001
167174899511,9cyclictest1674804-21true20:10:013
167173399513,17cyclictest1684951-21munin-run23:25:010
1671733995123,25cyclictest12-21ksoftirqd/021:10:010
1671733995118,8cyclictest12-21ksoftirqd/021:33:280
1671748995031,13cyclictest1-21systemd21:50:003
1671748995012,6cyclictest131rcu_preempt20:30:023
1671745995034,11cyclictest235-21systemd-journal20:55:002
167174599501,4cyclictest131rcu_preempt23:00:002
167174599501,4cyclictest131rcu_preempt23:00:002
167173399506,8cyclictest1671737-21latency_hist19:10:020
167173399503,13cyclictest1672478-21grep19:25:000
1671733995022,24cyclictest1680975-21latency_hist22:05:030
1671733995016,4cyclictest1674543-21latency_hist20:05:020
1671733995015,12cyclictest1672166-21sshd19:17:170
1671733995014,7cyclictest1689028-21date00:40:020
1671748994935,9cyclictest1683633-21latency_hist23:00:003
1671748994935,9cyclictest1683633-21latency_hist23:00:003
1671748994926,17cyclictest37-21ksoftirqd/320:20:003
1671745994936,9cyclictest1687505-21sshd00:12:292
1671745994933,12cyclictest1682601-21run-parts22:40:002
1671745994932,13cyclictest272-21dbus-daemon21:15:012
1671733994930,15cyclictest235-21systemd-journal00:20:010
1671733994913,7cyclictest1683858-21date23:00:020
167174899483,5cyclictest131rcu_preempt19:25:003
1671748994831,13cyclictest1677903-21cron21:10:003
167174599485,4cyclictest131rcu_preempt00:35:002
167173399483,8cyclictest1675310-21modprobe20:20:000
167173399483,14cyclictest235-21systemd-journal22:20:000
1671733994819,4cyclictest1672690-21kworker/u8:219:30:000
1671733994816,8cyclictest12-21ksoftirqd/019:47:270
1671733994813,7cyclictest1678975-21cut21:30:010
1671733994813,7cyclictest1678975-21cut21:30:010
167173399481,2cyclictest0-21swapper/023:55:020
1671733994810,9cyclictest1-21systemd21:00:000
167174899474,7cyclictest131rcu_preempt00:20:013
167174599474,5cyclictest0-21swapper/221:45:022
167174599474,5cyclictest0-21swapper/221:45:022
1671738994734,9cyclictest1-21systemd23:35:011
1671738994731,11cyclictest272-21dbus-daemon19:45:011
1671738994731,11cyclictest272-21dbus-daemon19:45:011
1671733994717,7cyclictest235-21systemd-journal20:15:010
1671733994717,7cyclictest235-21systemd-journal20:15:010
167174899462,5cyclictest131rcu_preempt20:50:023
167174899462,5cyclictest131rcu_preempt20:50:023
167174599464,6cyclictest131rcu_preempt23:55:022
1671745994634,7cyclictest306-21in:imuxsock00:25:022
167174599462,3cyclictest131rcu_preempt21:05:012
1671738994626,14cyclictest272-21dbus-daemon00:00:001
1671738994623,17cyclictest272-21dbus-daemon00:05:001
1671733994617,10cyclictest1675998-21date20:30:020
1671733994615,5cyclictest12-21ksoftirqd/019:15:010
1671733994613,9cyclictest272-21dbus-daemon00:30:000
167173399461,19cyclictest0-21swapper/021:23:580
167174899458,5cyclictest131rcu_preempt00:00:013
1671748994529,11cyclictest1677388-21run-parts21:00:003
1671748994525,14cyclictest235-21systemd-journal20:15:003
1671748994525,14cyclictest235-21systemd-journal20:15:003
1671748994511,6cyclictest131rcu_preempt19:55:003
1671748994511,5cyclictest131rcu_preempt23:40:013
167174899451,14cyclictest37-21ksoftirqd/322:05:023
1671745994531,11cyclictest30-21ksoftirqd/220:05:012
1671738994519,17cyclictest272-21dbus-daemon20:15:011
1671738994519,17cyclictest272-21dbus-daemon20:15:011
167173399454,5cyclictest131rcu_preempt20:40:000
1671733994516,4cyclictest1681838-21/usr/sbin/munin22:25:010
167173399451,4cyclictest0-21swapper/022:59:240
1671748994429,12cyclictest272-21dbus-daemon00:25:013
1671748994423,16cyclictest1-21systemd23:35:013
1671748994412,5cyclictest131rcu_preempt19:15:003
167174899441,10cyclictest37-21ksoftirqd/321:30:003
167174899441,10cyclictest37-21ksoftirqd/321:30:003
167174599446,9cyclictest30-21ksoftirqd/221:17:002
167174599445,4cyclictest131rcu_preempt22:55:002
167174599444,4cyclictest131rcu_preempt00:20:002
167174599443,4cyclictest131rcu_preempt00:40:002
167173899446,6cyclictest131rcu_preempt21:23:281
1671738994431,10cyclictest1-21systemd21:10:011
167173399449,11cyclictest12-21ksoftirqd/022:50:000
1671733994418,9cyclictest12-21ksoftirqd/023:35:010
1671733994416,6cyclictest16950irq/224-800000019:30:400
167173399441,24cyclictest0-21swapper/021:15:110
167173399441,21cyclictest0-21swapper/022:43:280
167173399441,21cyclictest0-21swapper/022:43:280
167174899434,10cyclictest37-21ksoftirqd/300:10:013
167174899434,10cyclictest37-21ksoftirqd/300:10:013
1671748994332,8cyclictest37-21ksoftirqd/323:50:003
1671748994330,10cyclictest37-21ksoftirqd/322:00:013
1671748994330,10cyclictest37-21ksoftirqd/322:00:013
167174599434,4cyclictest131rcu_preempt19:20:012
1671745994331,9cyclictest1-21systemd00:30:002
1671745994323,14cyclictest408-21lldpd22:30:012
1671745994323,14cyclictest408-21lldpd22:30:002
1671738994329,10cyclictest1-21systemd22:55:001
1671738994310,6cyclictest131rcu_preempt21:50:001
1671733994336,3cyclictest0-21swapper/023:10:020
167174899427,11cyclictest131rcu_preempt22:50:013
167174899425,5cyclictest131rcu_preempt21:05:023
1671748994231,8cyclictest37-21ksoftirqd/320:00:013
1671748994228,11cyclictest37-21ksoftirqd/323:55:013
1671748994227,13cyclictest37-21ksoftirqd/321:55:003
1671748994212,6cyclictest131rcu_preempt22:05:013
167174599423,3cyclictest131rcu_preempt23:50:012
1671745994224,13cyclictest272-21dbus-daemon21:10:012
167174599421,3cyclictest131rcu_preempt19:55:012
1671745994211,19cyclictest131rcu_preempt21:45:282
1671738994232,5cyclictest306-21in:imuxsock23:40:011
1671738994229,10cyclictest1-21systemd21:55:001
1671738994212,20cyclictest1672469-21modprobe19:25:001
167173399423,7cyclictest281-21systemd-logind00:35:020
167174899419,6cyclictest131rcu_preempt23:15:033
167174899414,8cyclictest37-21ksoftirqd/300:15:003
167174899413,8cyclictest37-21ksoftirqd/300:35:013
1671748994129,9cyclictest37-21ksoftirqd/321:10:033
1671748994129,7cyclictest306-21in:imuxsock22:40:003
1671748994124,12cyclictest1-21systemd20:55:003
1671748994123,13cyclictest37-21ksoftirqd/322:55:013
1671748994110,6cyclictest131rcu_preempt20:45:013
1671748994110,5cyclictest131rcu_preempt00:05:013
1671745994129,9cyclictest30-21ksoftirqd/223:25:022
167174599412,4cyclictest131rcu_preempt19:10:022
167174599412,3cyclictest131rcu_preempt19:45:012
167174599412,3cyclictest131rcu_preempt19:45:012
1671745994120,16cyclictest272-21dbus-daemon23:35:012
1671738994128,9cyclictest235-21systemd-journal23:48:301
1671738994127,10cyclictest1-21systemd23:25:011
1671738994122,14cyclictest235-21systemd-journal00:40:001
1671733994121,16cyclictest0-21swapper/020:10:000
1671733994120,4cyclictest0-21swapper/020:49:510
1671733994120,4cyclictest0-21swapper/020:49:500
1671733994118,4cyclictest0-21swapper/019:43:300
1671733994118,4cyclictest0-21swapper/019:43:300
167174899406,6cyclictest131rcu_preempt21:20:013
167174899405,6cyclictest131rcu_preempt23:25:023
1671748994030,5cyclictest1684670-21sh23:20:003
1671748994020,15cyclictest1-21systemd22:35:003
1671745994030,7cyclictest30-21ksoftirqd/220:15:032
1671745994026,11cyclictest30-21ksoftirqd/220:30:012
1671745994021,14cyclictest272-21dbus-daemon22:20:002
167173899405,29cyclictest272-21dbus-daemon21:45:021
167173899405,29cyclictest272-21dbus-daemon21:45:011
1671738994029,8cyclictest23-21ksoftirqd/119:15:021
1671738994024,11cyclictest235-21systemd-journal22:35:001
167174899399,12cyclictest37-21ksoftirqd/300:25:293
167174899393,10cyclictest37-21ksoftirqd/320:40:003
1671748993930,7cyclictest37-21ksoftirqd/321:35:013
1671748993928,8cyclictest1672166-21sshd19:17:173
1671748993927,9cyclictest37-21ksoftirqd/321:40:003
1671748993925,11cyclictest37-21ksoftirqd/319:45:013
1671748993925,11cyclictest37-21ksoftirqd/319:45:013
1671748993910,5cyclictest131rcu_preempt20:05:013
167174599395,4cyclictest131rcu_preempt20:40:002
167174599394,4cyclictest131rcu_preempt21:00:002
167174599393,4cyclictest131rcu_preempt20:00:012
1671745993928,6cyclictest306-21in:imuxsock21:40:002
1671745993925,10cyclictest235-21systemd-journal19:14:172
1671738993928,8cyclictest1675285-21sshd20:17:251
1671738993928,7cyclictest306-21in:imuxsock22:05:001
1671738993926,10cyclictest235-21systemd-journal22:15:001
1671738993926,10cyclictest235-21systemd-journal22:15:001
1671738993925,11cyclictest272-21dbus-daemon21:20:001
1671738993924,11cyclictest1679485-21systemctl21:40:001
167174899386,8cyclictest37-21ksoftirqd/321:25:003
167174899385,10cyclictest37-21ksoftirqd/323:05:013
1671748993826,8cyclictest235-21systemd-journal22:40:023
1671748993826,8cyclictest235-21systemd-journal22:40:023
1671748993825,10cyclictest37-21ksoftirqd/322:25:023
167174899381,7cyclictest0-21swapper/319:50:003
167174599383,4cyclictest131rcu_preempt20:15:002
1671745993829,7cyclictest30-21ksoftirqd/223:05:012
1671745993827,8cyclictest30-21ksoftirqd/222:00:022
1671745993827,8cyclictest30-21ksoftirqd/222:00:022
1671745993827,8cyclictest235-21systemd-journal20:35:002
1671745993826,9cyclictest1685342-21ls23:30:022
1671738993828,7cyclictest272-21dbus-daemon22:50:011
1671738993828,7cyclictest0-21swapper/119:10:021
1671738993826,8cyclictest23-21ksoftirqd/123:30:001
167173899381,3cyclictest131rcu_preempt19:50:001
167174899375,6cyclictest131rcu_preempt19:30:013
1671748993722,11cyclictest235-21systemd-journal23:05:023
1671748993719,14cyclictest272-21dbus-daemon19:35:013
167174599373,3cyclictest131rcu_preempt22:50:012
1671745993728,7cyclictest30-21ksoftirqd/219:25:022
1671738993726,8cyclictest1688564-21sshd00:33:271
1671738993723,11cyclictest23-21ksoftirqd/120:57:321
1671738993713,16cyclictest272-21dbus-daemon19:20:011
1671733993723,10cyclictest272-21dbus-daemon22:00:010
1671733993723,10cyclictest272-21dbus-daemon22:00:010
167174899365,7cyclictest37-21ksoftirqd/323:30:003
1671748993622,11cyclictest1682071-21sshd22:29:283
1671748993622,11cyclictest1682071-21sshd22:29:283
1671748993620,12cyclictest37-21ksoftirqd/319:40:003
167174599362,5cyclictest131rcu_preempt19:35:012
1671745993623,9cyclictest30-21ksoftirqd/223:20:002
1671745993621,11cyclictest1-21systemd22:45:022
1671745993621,11cyclictest1-21systemd22:45:012
1671745993619,13cyclictest1-21systemd23:15:012
1671745993619,12cyclictest272-21dbus-daemon20:45:012
1671738993626,8cyclictest23-21ksoftirqd/123:45:011
1671738993626,5cyclictest1681044-21sh22:10:001
1671738993625,8cyclictest23-21ksoftirqd/122:39:281
1671738993624,9cyclictest23-21ksoftirqd/123:20:001
1671738993624,7cyclictest1687780-21cron00:20:001
1671738993619,13cyclictest272-21dbus-daemon20:10:011
1671738993618,13cyclictest272-21dbus-daemon21:05:011
167174899358,5cyclictest131rcu_preempt20:31:263
167174599354,4cyclictest131rcu_preempt19:40:002
1671745993527,5cyclictest1672753-21ls19:30:012
1671745993526,5cyclictest306-21in:imuxsock21:55:012
1671745993526,5cyclictest1673931-21latency_hist19:50:022
1671745993520,11cyclictest272-21dbus-daemon21:35:002
1671745993515,14cyclictest272-21dbus-daemon00:10:012
1671745993515,14cyclictest272-21dbus-daemon00:10:012
167173899359,5cyclictest131rcu_preempt21:28:281
167173899359,5cyclictest131rcu_preempt21:28:281
167173899353,4cyclictest131rcu_preempt19:55:001
1671738993524,9cyclictest23-21ksoftirqd/123:15:001
1671738993520,11cyclictest1677144-21systemd-run20:55:001
1671738993519,12cyclictest272-21dbus-daemon23:55:011
167174899347,5cyclictest131rcu_preempt19:10:023
167174899346,19cyclictest1681554-21modprobe22:20:003
1671745993423,9cyclictest30-21ksoftirqd/220:10:012
167174599342,11cyclictest131rcu_preempt22:20:282
1671745993421,10cyclictest1686113-21sed23:45:022
167173899346,21cyclictest1688066-21modprobe00:25:001
167173899344,14cyclictest0-21swapper/123:05:011
1671738993427,4cyclictest1682223-21latency_hist22:30:021
1671738993427,4cyclictest1682223-21latency_hist22:30:021
1671738993425,5cyclictest1683614-21cron23:00:001
1671738993425,5cyclictest1683614-21cron23:00:001
1671738993424,5cyclictest306-21in:imuxsock22:25:001
1671733993428,4cyclictest0-21swapper/022:31:170
1671745993322,8cyclictest1675457-21sed20:20:022
167173899333,4cyclictest131rcu_preempt19:30:001
167173899333,25cyclictest1675851-21cut20:30:001
1671738993326,4cyclictest1676723-21latency_hist20:45:011
1671738993324,6cyclictest23-21ksoftirqd/120:32:291
1671738993320,9cyclictest23-21ksoftirqd/120:45:551
1671738993320,9cyclictest23-21ksoftirqd/120:45:551
1671738993319,7cyclictest306-21in:imuxsock20:40:001
1671738993318,11cyclictest235-21systemd-journal19:40:001
1671745993217,12cyclictest30-21ksoftirqd/223:10:012
167173899324,5cyclictest131rcu_preempt20:24:191
1671738993224,5cyclictest1687530-21latency_hist00:15:021
1671738993222,5cyclictest1673208-21kworker/u8:020:00:001
167173899315,3cyclictest131rcu_preempt22:00:011
167173899315,3cyclictest131rcu_preempt22:00:011
167173899314,21cyclictest1679226-21cut21:35:001
1671738993116,11cyclictest1687490-21cut00:10:021
1671738993022,5cyclictest306-21in:imuxsock00:30:011
1671738993022,5cyclictest1683076-21latency_hist22:45:021
1671738993022,5cyclictest1678378-21latency_hist21:15:021
1671738993018,9cyclictest23-21ksoftirqd/123:10:031
1671738992920,7cyclictest23-21ksoftirqd/100:10:011
1671738992918,8cyclictest23-21ksoftirqd/119:35:011
1671738992910,12cyclictest1682864-21modprobe22:45:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional