You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Jan 01, 2026 00:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2397320997723,5cyclictest131rcu_preempt23:25:012
239732599708,5cyclictest131rcu_preempt21:10:023
2397320997028,37cyclictest2407173-21sh22:20:012
2397320997028,37cyclictest2407173-21sh22:20:012
2397320997022,11cyclictest2398838-21cron19:40:002
239732099639,33cyclictest291rcuc/219:45:012
2397320996321,5cyclictest2403067-21latency_hist21:00:012
2397320996224,4cyclictest2406981-21latency_hist22:15:032
2397320996220,7cyclictest2413380-21modprobe00:20:002
2397325995848,6cyclictest2412293-21cron00:00:003
2397320995721,8cyclictest30-21ksoftirqd/223:40:012
2397320995640,12cyclictest2414400-21cat00:40:012
2397320995621,9cyclictest30-21ksoftirqd/221:25:012
2397310995635,16cyclictest1-21systemd00:05:000
2397320995514,12cyclictest275-21dbus-daemon22:40:002
2397320995427,8cyclictest30-21ksoftirqd/223:35:012
2397320995425,5cyclictest2406625-21latency_hist22:10:012
239731099542,31cyclictest131rcu_preempt21:40:020
2397320995334,14cyclictest238-21systemd-journal00:30:002
2397320995334,13cyclictest238-21systemd-journal19:20:012
2397320995324,8cyclictest30-21ksoftirqd/219:29:322
2397320995316,8cyclictest309-21in:imuxsock21:58:332
2397325995237,11cyclictest1-21systemd23:50:003
2397325995212,11cyclictest131rcu_preempt23:20:003
2397320995221,9cyclictest30-21ksoftirqd/220:35:022
2397320995221,9cyclictest30-21ksoftirqd/220:35:022
239731099524,3cyclictest131rcu_preempt22:30:010
239732599518,6cyclictest131rcu_preempt23:10:013
2397325995132,14cyclictest1-21systemd23:15:003
2397320995135,12cyclictest1-21systemd23:20:002
2397315995131,15cyclictest275-21dbus-daemon21:00:001
239732599508,10cyclictest37-21ksoftirqd/300:37:423
2397315995034,12cyclictest238-21systemd-journal21:45:011
2397315995033,12cyclictest238-21systemd-journal22:50:001
239731099502,42cyclictest2399377-21munin-run19:50:000
2397310995010,33cyclictest12-21ksoftirqd/021:10:000
239732599494,5cyclictest131rcu_preempt21:05:023
239732599494,3cyclictest131rcu_preempt22:50:023
2397315994931,13cyclictest238-21systemd-journal22:30:011
2397325994837,8cyclictest37-21ksoftirqd/319:25:013
2397325994836,8cyclictest1-21systemd19:55:013
2397320994822,4cyclictest2407919-21latency_hist22:35:022
2397320994822,4cyclictest2407919-21latency_hist22:35:022
239731599489,5cyclictest131rcu_preempt00:20:011
239731599482,3cyclictest131rcu_preempt00:30:031
239731099483,3cyclictest131rcu_preempt21:20:020
239732599474,4cyclictest131rcu_preempt22:30:023
2397325994734,9cyclictest275-21dbus-daemon23:45:013
2397325994728,5cyclictest37-21ksoftirqd/300:30:013
2397325994718,9cyclictest37-21ksoftirqd/320:50:023
239732099472,3cyclictest131rcu_preempt20:15:012
239731599475,4cyclictest131rcu_preempt00:05:001
2397310994732,11cyclictest1-21systemd22:50:010
2397310994728,14cyclictest275-21dbus-daemon23:50:000
239731099472,40cyclictest2414394-21cron00:40:000
239731099472,12cyclictest2411254-21cron23:40:010
239731099471,15cyclictest2406063-21sshd21:55:320
239732599466,36cyclictest2405074-21munin-plugin-st21:40:003
239732599464,4cyclictest131rcu_preempt23:05:013
2397325994610,6cyclictest131rcu_preempt00:35:033
2397320994633,10cyclictest1-21systemd21:30:012
2397320994633,10cyclictest1-21systemd21:30:012
239731599464,3cyclictest131rcu_preempt21:15:011
239731599463,3cyclictest131rcu_preempt23:25:011
2397315994621,5cyclictest23-21ksoftirqd/120:00:011
2397315994621,5cyclictest23-21ksoftirqd/120:00:011
239731099463,3cyclictest131rcu_preempt22:05:020
2397310994628,13cyclictest275-21dbus-daemon23:00:000
239731099462,3cyclictest131rcu_preempt00:34:380
239732599455,3cyclictest131rcu_preempt20:25:003
2397325994520,15cyclictest37-21ksoftirqd/320:20:013
2397320994534,7cyclictest238-21systemd-journal00:33:522
2397320994528,13cyclictest238-21systemd-journal20:00:002
2397320994528,13cyclictest238-21systemd-journal20:00:002
239731599455,4cyclictest131rcu_preempt00:40:011
239731599454,3cyclictest131rcu_preempt19:20:021
239731599453,3cyclictest131rcu_preempt20:30:001
2397315994533,7cyclictest309-21in:imuxsock21:40:011
2397315994526,14cyclictest275-21dbus-daemon20:15:011
239731599452,3cyclictest131rcu_preempt22:25:011
239731599452,3cyclictest131rcu_preempt21:30:021
239731599452,3cyclictest131rcu_preempt21:30:021
239731599452,3cyclictest131rcu_preempt20:25:021
239731599451,5cyclictest131rcu_preempt19:35:011
2397310994531,10cyclictest2407894-21sshd22:31:370
2397310994531,10cyclictest2407894-21sshd22:31:370
239732599445,6cyclictest131rcu_preempt21:50:023
239732599445,5cyclictest131rcu_preempt21:15:003
239732599445,31cyclictest2409171-21kworker/u8:123:30:023
239732599444,4cyclictest131rcu_preempt21:26:223
239732599444,4cyclictest131rcu_preempt21:26:223
239732599444,3cyclictest131rcu_preempt19:20:023
239732599443,3cyclictest131rcu_preempt20:45:023
2397325994428,12cyclictest2406116-21tr22:00:003
239732599442,3cyclictest131rcu_preempt21:35:013
239732599441,5cyclictest131rcu_preempt22:25:013
239731599444,3cyclictest131rcu_preempt23:11:421
239731599444,3cyclictest131rcu_preempt22:02:331
239731599444,3cyclictest131rcu_preempt19:55:011
239731599444,3cyclictest131rcu_preempt19:50:001
239731599444,3cyclictest131rcu_preempt19:40:021
239731599442,3cyclictest131rcu_preempt20:05:011
239731599441,4cyclictest131rcu_preempt23:46:171
239731599441,3cyclictest131rcu_preempt22:45:021
239731099447,33cyclictest0-21swapper/020:20:030
239731099444,3cyclictest131rcu_preempt23:05:000
2397310994435,7cyclictest12-21ksoftirqd/000:30:020
2397310994428,12cyclictest238-21systemd-journal20:12:330
2397310994424,15cyclictest275-21dbus-daemon23:55:000
2397310994422,15cyclictest1-21systemd22:20:020
2397310994422,15cyclictest1-21systemd22:20:010
239732599435,3cyclictest131rcu_preempt19:35:023
239732599434,3cyclictest131rcu_preempt22:20:023
239732599434,3cyclictest131rcu_preempt22:20:013
239732599433,3cyclictest131rcu_preempt22:05:023
239732599433,3cyclictest131rcu_preempt21:45:023
239732599432,3cyclictest131rcu_preempt00:10:013
239732599431,3cyclictest131rcu_preempt00:20:033
239732099432,9cyclictest1-21systemd21:05:012
239732099432,3cyclictest131rcu_preempt20:05:012
239731599434,4cyclictest131rcu_preempt21:07:331
239731599434,4cyclictest131rcu_preempt20:20:021
239731599434,4cyclictest131rcu_preempt00:35:001
239731599434,3cyclictest131rcu_preempt23:40:021
239731599434,3cyclictest131rcu_preempt00:25:021
239731599433,3cyclictest131rcu_preempt20:10:021
2397315994323,5cyclictest23-21ksoftirqd/120:45:001
239731599431,3cyclictest131rcu_preempt23:20:001
239731599431,3cyclictest131rcu_preempt21:50:031
239731599431,3cyclictest131rcu_preempt19:45:001
239731599431,3cyclictest131rcu_preempt00:10:021
2397310994336,4cyclictest16950irq/224-800000021:00:000
2397310994336,4cyclictest16950irq/224-800000019:55:000
2397310994335,4cyclictest16950irq/224-800000023:20:000
2397310994330,9cyclictest238-21systemd-journal20:35:010
2397310994330,9cyclictest238-21systemd-journal20:35:010
2397310994328,12cyclictest2404289-21sshd21:22:330
2397310994327,12cyclictest1-21systemd21:45:010
239732599424,3cyclictest131rcu_preempt22:55:013
239732599423,4cyclictest131rcu_preempt22:45:003
2397325994227,11cyclictest275-21dbus-daemon21:25:013
2397325994226,12cyclictest2401764-21cat20:35:023
2397325994226,12cyclictest2401764-21cat20:35:023
239732599422,3cyclictest131rcu_preempt22:07:333
2397325994220,5cyclictest37-21ksoftirqd/323:25:023
239732599421,4cyclictest131rcu_preempt20:02:333
239732599421,3cyclictest131rcu_preempt22:35:013
239732599421,3cyclictest131rcu_preempt22:35:013
239732599421,3cyclictest131rcu_preempt20:10:013
2397320994233,5cyclictest2398546-21sh19:35:002
2397320994231,7cyclictest309-21in:imuxsock20:40:002
2397320994228,10cyclictest275-21dbus-daemon21:45:022
2397320994228,10cyclictest2407405-21cron22:25:012
239732099421,6cyclictest2399346-21kworker/2:219:55:022
239731599428,25cyclictest2404030-21modprobe21:20:001
239731599425,3cyclictest131rcu_preempt22:15:011
239731599422,9cyclictest23-21ksoftirqd/121:35:001
2397315994226,12cyclictest2402737-21systemd-run20:55:011
2397315994226,12cyclictest2402737-21systemd-run20:55:011
239731599422,3cyclictest131rcu_preempt20:34:341
239731599422,3cyclictest131rcu_preempt20:34:341
2397315994221,5cyclictest23-21ksoftirqd/122:10:001
2397315994219,5cyclictest23-21ksoftirqd/122:00:011
239731599421,4cyclictest131rcu_preempt00:15:001
239731599421,3cyclictest131rcu_preempt23:05:011
239731599421,3cyclictest131rcu_preempt22:20:031
239731599421,3cyclictest131rcu_preempt22:20:021
239731599421,3cyclictest131rcu_preempt20:50:011
239731099423,6cyclictest2401958-21munin-run20:40:000
2397310994236,4cyclictest16950irq/224-800000000:20:020
2397310994235,4cyclictest16950irq/224-800000020:30:000
2397310994234,5cyclictest16950irq/224-800000023:15:010
2397310994231,7cyclictest2412306-21cron00:00:000
239731099422,3cyclictest131rcu_preempt00:25:010
2397310994223,14cyclictest1-21systemd23:10:010
239732599414,3cyclictest131rcu_preempt00:23:443
239732599413,3cyclictest131rcu_preempt19:48:333
239732599412,3cyclictest131rcu_preempt20:39:333
239732599411,3cyclictest131rcu_preempt23:55:023
239732599411,3cyclictest131rcu_preempt21:17:333
239732099412,3cyclictest131rcu_preempt23:55:012
239731599414,3cyclictest131rcu_preempt19:30:001
239731599413,3cyclictest131rcu_preempt22:40:001
239731599413,3cyclictest131rcu_preempt19:25:011
2397315994126,10cyclictest1-21systemd00:00:011
239731599412,3cyclictest131rcu_preempt21:05:011
2397315994121,5cyclictest23-21ksoftirqd/123:10:011
239731599411,3cyclictest131rcu_preempt22:55:021
2397310994129,9cyclictest1-21systemd20:55:010
2397310994129,9cyclictest1-21systemd20:55:010
2397310994128,9cyclictest1-21systemd19:15:000
2397310994123,14cyclictest2405571-21sshd21:46:320
2397310994110,27cyclictest1-21systemd21:50:330
239732599404,3cyclictest131rcu_preempt00:12:423
2397325994030,6cyclictest309-21in:imuxsock23:00:003
239732599402,3cyclictest131rcu_preempt21:00:013
239732599402,3cyclictest131rcu_preempt19:15:003
239732599401,3cyclictest131rcu_preempt19:45:013
2397320994030,5cyclictest2405068-21sendmail21:40:002
2397320994026,10cyclictest0-21swapper/222:05:012
239731599404,3cyclictest131rcu_preempt23:45:011
239731599404,3cyclictest131rcu_preempt23:35:011
239731599403,3cyclictest131rcu_preempt23:53:251
239731599403,3cyclictest131rcu_preempt20:40:011
2397315994028,9cyclictest2407901-21sshd22:33:321
2397315994028,9cyclictest2407901-21sshd22:33:321
239731599402,3cyclictest131rcu_preempt22:55:031
2397315994019,15cyclictest1-21systemd21:25:011
2397315994014,11cyclictest0-21swapper/123:30:021
239731099409,5cyclictest131rcu_preempt21:35:010
2397310994034,4cyclictest16950irq/224-800000019:30:000
2397310994031,6cyclictest2406139-21kworker/0:122:24:320
239731099401,5cyclictest1-21systemd20:00:010
239731099401,5cyclictest1-21systemd20:00:010
239731099401,35cyclictest2402205-21sshd20:42:330
2397325993916,5cyclictest37-21ksoftirqd/319:59:343
2397325993916,5cyclictest37-21ksoftirqd/319:59:343
239732599391,4cyclictest131rcu_preempt22:40:003
2397325993914,6cyclictest37-21ksoftirqd/323:40:013
239732599391,3cyclictest131rcu_preempt21:55:013
2397320993928,7cyclictest30-21ksoftirqd/221:15:012
2397320993927,8cyclictest1-21systemd22:50:012
2397320993917,16cyclictest238-21systemd-journal21:35:002
2397315993927,9cyclictest2405896-21cat21:55:011
2397315993918,4cyclictest23-21ksoftirqd/119:12:331
239731099393,3cyclictest131rcu_preempt19:40:000
2397310993928,8cyclictest238-21systemd-journal00:07:420
2397310993927,9cyclictest12-21ksoftirqd/020:50:010
2397310993927,8cyclictest2400129-21sshd20:01:330
239731099391,6cyclictest2408183-21munin-run22:40:000
239731099391,34cyclictest0-21swapper/019:20:020
239732599385,3cyclictest131rcu_preempt19:40:003
239732599384,4cyclictest131rcu_preempt00:01:423
239732599384,3cyclictest131rcu_preempt20:26:333
239732599382,3cyclictest131rcu_preempt23:35:013
2397320993828,7cyclictest2403482-21sshd21:06:192
2397320993827,7cyclictest30-21ksoftirqd/220:20:032
2397320993827,7cyclictest2399354-21sshd19:47:342
239732099382,3cyclictest131rcu_preempt20:25:022
2397320993820,13cyclictest2409223-21tr23:00:002
239732099381,4cyclictest131rcu_preempt22:45:012
239732099381,3cyclictest131rcu_preempt00:10:012
2397310993831,4cyclictest16950irq/224-800000023:41:490
2397310993828,7cyclictest2408471-21latency_hist22:45:010
239731099382,6cyclictest2403275-21munin-run21:05:010
2397310993821,9cyclictest0-21swapper/019:10:010
2397310993815,20cyclictest12-21ksoftirqd/022:15:010
2397310993814,8cyclictest12-21ksoftirqd/021:30:020
2397310993814,8cyclictest12-21ksoftirqd/021:30:020
239732599371,3cyclictest131rcu_preempt20:15:023
2397320993729,5cyclictest2409722-21munin-plugin-st23:10:002
2397320993723,9cyclictest275-21dbus-daemon20:45:012
2397310993729,4cyclictest2398047-21munin-plugin-st19:25:010
2397310993724,9cyclictest12-21ksoftirqd/020:10:010
2397310993724,10cyclictest12-21ksoftirqd/000:15:020
239731099372,3cyclictest131rcu_preempt19:45:000
239732599363,3cyclictest131rcu_preempt19:25:333
239732599362,4cyclictest131rcu_preempt20:55:023
239732599362,4cyclictest131rcu_preempt20:55:023
239732099369,7cyclictest131rcu_preempt21:55:022
2397320993624,8cyclictest2412326-21tr00:00:002
2397320993621,13cyclictest30-21ksoftirqd/200:15:012
2397310993628,4cyclictest2408965-21unin-run22:55:010
2397310993626,8cyclictest12-21ksoftirqd/023:25:010
239731099362,3cyclictest131rcu_preempt23:30:020
2397310993622,5cyclictest12-21ksoftirqd/021:15:000
2397310993621,11cyclictest2406792-21cat22:10:030
2397325993514,5cyclictest37-21ksoftirqd/322:10:323
2397320993528,5cyclictest30-21ksoftirqd/223:50:012
2397320993523,9cyclictest30-21ksoftirqd/223:00:022
2397320993519,12cyclictest238-21systemd-journal21:50:022
2397320993516,12cyclictest238-21systemd-journal20:55:002
2397320993516,12cyclictest238-21systemd-journal20:55:002
239732099343,3cyclictest131rcu_preempt22:55:022
2397320993427,4cyclictest2397518-21sh19:15:002
2397320993424,7cyclictest309-21in:imuxsock23:12:422
2397320993412,16cyclictest238-21systemd-journal23:45:002
2397310993429,3cyclictest16950irq/224-800000020:25:000
2397310993425,6cyclictest2398548-21modprobe19:35:000
239731099341,6cyclictest0-21swapper/023:33:420
239732099335,3cyclictest131rcu_preempt20:30:022
239732099333,3cyclictest131rcu_preempt19:25:022
2397320993326,4cyclictest2397253-21latency_hist19:10:012
2397320993324,5cyclictest309-21in:imuxsock20:50:012
2397320993322,9cyclictest30-21ksoftirqd/200:05:012
2397320993321,8cyclictest0-21swapper/221:20:012
239732599323,3cyclictest131rcu_preempt19:10:013
2397320993224,5cyclictest2410887-21latency_hist23:30:032
2397320993223,7cyclictest30-21ksoftirqd/222:30:022
2397320993220,10cyclictest30-21ksoftirqd/220:10:012
2397315993121,7cyclictest2397409-21date19:10:011
2397320993023,4cyclictest2413794-21latency_hist00:25:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional