You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 14:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Nov 22, 2025 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3762152998379,2cyclictest0-21swapper/020:35:020
376215899729,14cyclictest131rcu_preempt22:35:031
3762152996830,3cyclictest0-21swapper/022:40:020
3762163996619,12cyclictest30-21ksoftirqd/221:05:022
376215899653,13cyclictest23-21ksoftirqd/122:00:001
3762163996419,11cyclictest30-21ksoftirqd/220:04:592
3762163996418,11cyclictest30-21ksoftirqd/222:30:012
3762163996315,12cyclictest30-21ksoftirqd/219:20:022
3762163996313,9cyclictest30-21ksoftirqd/221:50:012
3762163996213,11cyclictest30-21ksoftirqd/219:15:022
376216399619,15cyclictest30-21ksoftirqd/220:20:022
3762163996110,13cyclictest30-21ksoftirqd/221:40:022
3762163995918,13cyclictest30-21ksoftirqd/220:55:012
3762163995812,11cyclictest30-21ksoftirqd/223:55:012
3762163995812,11cyclictest30-21ksoftirqd/223:55:012
3762158995826,10cyclictest23-21ksoftirqd/122:40:001
376216399571,12cyclictest3777200-40mandb00:00:012
376216399571,12cyclictest3777200-40mandb00:00:002
376216399571,11cyclictest30-21ksoftirqd/221:25:002
3762163995710,9cyclictest30-21ksoftirqd/222:12:132
3762158995718,11cyclictest23-21ksoftirqd/120:34:331
376216399566,11cyclictest30-21ksoftirqd/223:45:012
376216399562,14cyclictest30-21ksoftirqd/220:50:002
3762163995610,9cyclictest30-21ksoftirqd/222:10:022
376215899562,9cyclictest23-21ksoftirqd/100:35:011
3762158995611,6cyclictest131rcu_preempt23:30:021
376216399558,8cyclictest30-21ksoftirqd/220:10:022
376216399558,10cyclictest30-21ksoftirqd/220:15:022
376216399558,10cyclictest30-21ksoftirqd/220:15:012
3762163995510,8cyclictest30-21ksoftirqd/200:25:022
376215899551,13cyclictest23-21ksoftirqd/119:15:011
376216399545,14cyclictest30-21ksoftirqd/200:19:592
376216399545,10cyclictest30-21ksoftirqd/223:10:012
376216399542,14cyclictest30-21ksoftirqd/223:40:022
376216399542,14cyclictest30-21ksoftirqd/223:40:022
3762163995410,9cyclictest30-21ksoftirqd/200:30:022
3762163995410,9cyclictest30-21ksoftirqd/200:30:022
3762163995410,7cyclictest30-21ksoftirqd/219:55:022
376215899549,6cyclictest131rcu_preempt19:25:001
376215899547,6cyclictest131rcu_preempt21:50:021
376215899543,10cyclictest23-21ksoftirqd/100:00:021
376215899543,10cyclictest23-21ksoftirqd/100:00:021
376216799533,3cyclictest131rcu_preempt20:00:023
376216799533,3cyclictest131rcu_preempt20:00:013
376216399539,8cyclictest30-21ksoftirqd/222:55:022
376215899537,11cyclictest23-21ksoftirqd/120:50:011
376215899535,9cyclictest23-21ksoftirqd/121:10:011
376215899535,8cyclictest131rcu_preempt21:15:021
3762158995315,6cyclictest131rcu_preempt23:20:011
376216399529,11cyclictest30-21ksoftirqd/223:30:022
376216399522,9cyclictest30-21ksoftirqd/222:50:022
376216399522,9cyclictest30-21ksoftirqd/222:50:012
376216399522,13cyclictest30-21ksoftirqd/222:00:022
376216399522,10cyclictest30-21ksoftirqd/222:35:012
376216399521,14cyclictest30-21ksoftirqd/200:39:592
376215899528,12cyclictest23-21ksoftirqd/120:51:581
376215899526,12cyclictest23-21ksoftirqd/122:45:001
376215899525,8cyclictest23-21ksoftirqd/123:50:001
376216399515,8cyclictest30-21ksoftirqd/223:20:032
376216399515,7cyclictest30-21ksoftirqd/221:20:022
376216399515,10cyclictest30-21ksoftirqd/220:30:022
376216399515,10cyclictest30-21ksoftirqd/220:30:022
376216399514,9cyclictest30-21ksoftirqd/220:40:022
376216399514,12cyclictest30-21ksoftirqd/200:15:022
376216399512,14cyclictest30-21ksoftirqd/223:20:012
376216399512,12cyclictest30-21ksoftirqd/200:05:022
3762163995113,10cyclictest30-21ksoftirqd/223:05:022
3762163995113,10cyclictest30-21ksoftirqd/223:05:022
376216399511,12cyclictest30-21ksoftirqd/220:00:022
376216399511,12cyclictest30-21ksoftirqd/220:00:012
376215899517,8cyclictest23-21ksoftirqd/120:45:011
376215899517,10cyclictest23-21ksoftirqd/119:20:031
376215899512,7cyclictest23-21ksoftirqd/121:19:591
376216399506,12cyclictest30-21ksoftirqd/219:25:022
376216399504,5cyclictest1-21systemd21:10:012
376215899505,8cyclictest23-21ksoftirqd/122:55:011
376215899501,4cyclictest131rcu_preempt21:00:011
3762152995013,20cyclictest16850irq/134-800000023:42:280
376216399498,8cyclictest30-21ksoftirqd/222:20:022
376216399498,8cyclictest30-21ksoftirqd/222:20:012
376216399497,10cyclictest0-21swapper/219:35:012
376216399497,10cyclictest0-21swapper/219:35:002
376216399496,6cyclictest30-21ksoftirqd/223:00:022
376216399491,7cyclictest0-21swapper/223:35:002
376215899499,5cyclictest131rcu_preempt21:43:321
376215899493,7cyclictest23-21ksoftirqd/122:03:341
376215899492,8cyclictest23-21ksoftirqd/100:30:021
376215899492,8cyclictest23-21ksoftirqd/100:30:021
376215899492,5cyclictest131rcu_preempt00:04:591
3762158994918,11cyclictest23-21ksoftirqd/120:05:001
376215299498,30cyclictest19050irq/34-mmc120:25:010
376216799486,6cyclictest131rcu_preempt21:15:013
376216399485,13cyclictest30-21ksoftirqd/223:50:022
376216399483,11cyclictest30-21ksoftirqd/220:45:012
376216399481,8cyclictest0-21swapper/221:00:022
376216399481,14cyclictest30-21ksoftirqd/221:35:022
376215899489,5cyclictest131rcu_preempt00:07:331
376215899488,7cyclictest23-21ksoftirqd/122:20:011
376215899488,7cyclictest23-21ksoftirqd/122:20:001
376215899487,5cyclictest131rcu_preempt23:20:031
376215899486,9cyclictest23-21ksoftirqd/123:05:021
376215899486,9cyclictest23-21ksoftirqd/123:05:021
376215899485,7cyclictest23-21ksoftirqd/120:19:591
3762158994831,13cyclictest272-21dbus-daemon21:30:011
3762158994830,13cyclictest23-21ksoftirqd/120:15:011
3762158994830,13cyclictest23-21ksoftirqd/120:15:011
3762158994813,5cyclictest131rcu_preempt20:40:001
3762167994710,6cyclictest131rcu_preempt19:55:013
376216399477,8cyclictest30-21ksoftirqd/221:45:012
376216399476,6cyclictest30-21ksoftirqd/222:05:032
376216399473,12cyclictest30-21ksoftirqd/222:40:002
376216399472,11cyclictest30-21ksoftirqd/223:15:022
376216399471,7cyclictest3770788-21sed21:55:012
376215899476,5cyclictest23-21ksoftirqd/123:15:011
376215899476,5cyclictest131rcu_preempt20:25:021
376215899472,8cyclictest23-21ksoftirqd/122:25:001
3762158994715,21cyclictest131rcu_preempt22:10:021
3762152994736,8cyclictest12-21ksoftirqd/023:55:010
3762152994736,8cyclictest12-21ksoftirqd/023:55:010
3762152994733,10cyclictest1-21systemd23:35:010
3762167994635,6cyclictest3773016-21sendmail22:40:003
3762167994632,10cyclictest235-21systemd-journal22:20:003
3762167994632,10cyclictest235-21systemd-journal22:20:003
3762167994620,16cyclictest37-21ksoftirqd/322:10:033
376216399466,10cyclictest30-21ksoftirqd/219:40:022
376216399461,8cyclictest0-21swapper/221:14:592
376215899468,7cyclictest131rcu_preempt19:55:011
376215899468,4cyclictest131rcu_preempt22:30:001
376215899466,5cyclictest131rcu_preempt19:40:021
376215899466,11cyclictest23-21ksoftirqd/123:00:001
376215899464,8cyclictest23-21ksoftirqd/120:30:021
376215899464,8cyclictest23-21ksoftirqd/120:30:021
376215899464,8cyclictest23-21ksoftirqd/100:25:011
3762158994638,5cyclictest23-21ksoftirqd/123:10:021
376215899463,5cyclictest131rcu_preempt21:05:011
376215899462,8cyclictest23-21ksoftirqd/122:10:031
376215899462,5cyclictest131rcu_preempt19:55:031
376215899462,5cyclictest131rcu_preempt19:55:021
3762158994613,6cyclictest131rcu_preempt00:20:001
3762158994611,10cyclictest131rcu_preempt23:35:021
3762158994610,6cyclictest131rcu_preempt00:40:001
3762152994628,13cyclictest235-21systemd-journal21:35:010
376216799456,6cyclictest131rcu_preempt20:10:023
376216399453,10cyclictest30-21ksoftirqd/200:08:462
376216399452,11cyclictest30-21ksoftirqd/219:45:012
376215899458,5cyclictest131rcu_preempt22:50:021
376215899458,5cyclictest131rcu_preempt22:50:021
376215899456,5cyclictest131rcu_preempt19:10:021
3762158994518,6cyclictest23-21ksoftirqd/121:25:021
376215899451,5cyclictest131rcu_preempt21:40:021
3762158994515,9cyclictest23-21ksoftirqd/119:40:021
3762152994532,9cyclictest3775104-21systemctl23:20:010
3762152994529,12cyclictest1-21systemd23:00:010
3762152994527,14cyclictest12-21ksoftirqd/020:00:010
3762152994527,14cyclictest12-21ksoftirqd/020:00:010
3762152994514,6cyclictest131rcu_preempt19:15:010
376216799445,8cyclictest37-21ksoftirqd/320:59:333
376216799444,4cyclictest131rcu_preempt23:10:023
376216399445,9cyclictest30-21ksoftirqd/221:30:002
376216399445,8cyclictest30-21ksoftirqd/222:25:022
376216399441,8cyclictest0-21swapper/200:31:432
376216399441,5cyclictest0-21swapper/219:25:002
376215899446,6cyclictest131rcu_preempt21:35:001
376215899443,6cyclictest131rcu_preempt23:40:001
376215899443,6cyclictest131rcu_preempt23:39:591
376215899443,10cyclictest23-21ksoftirqd/123:45:021
376215299448,6cyclictest131rcu_preempt21:40:010
376215299446,6cyclictest131rcu_preempt22:35:030
3762152994429,11cyclictest1-21systemd21:50:000
3762152994427,11cyclictest1-21systemd23:25:010
376215299441,5cyclictest131rcu_preempt19:45:020
376216799438,5cyclictest131rcu_preempt23:50:023
376216399433,7cyclictest30-21ksoftirqd/220:25:012
376216399432,6cyclictest30-21ksoftirqd/219:10:012
3762158994330,9cyclictest235-21systemd-journal21:50:031
376215299436,4cyclictest131rcu_preempt22:05:010
3762152994336,4cyclictest16950irq/224-800000000:34:080
3762152994335,5cyclictest16950irq/224-800000000:40:010
3762152994326,12cyclictest272-21dbus-daemon20:15:010
3762152994326,12cyclictest272-21dbus-daemon20:15:000
3762152994326,11cyclictest235-21systemd-journal23:05:010
3762152994326,11cyclictest235-21systemd-journal23:05:000
376216799425,5cyclictest131rcu_preempt19:10:023
376216799425,31cyclictest3775092-21modprobe23:20:013
3762167994230,8cyclictest272-21dbus-daemon19:15:013
3762167994228,10cyclictest1-21systemd20:15:013
3762167994228,10cyclictest1-21systemd20:15:003
376216399424,8cyclictest30-21ksoftirqd/220:34:592
376216399421,7cyclictest3764182-21munin-run19:50:012
376215899427,5cyclictest131rcu_preempt19:35:011
376215899427,5cyclictest131rcu_preempt19:35:001
376215899427,5cyclictest131rcu_preempt00:15:011
376215899424,5cyclictest131rcu_preempt19:29:591
376215899421,3cyclictest131rcu_preempt20:10:011
3762158994213,6cyclictest131rcu_preempt23:55:021
3762158994213,6cyclictest131rcu_preempt23:55:021
376215299426,32cyclictest3773288-21latency_hist22:45:020
376215299424,3cyclictest131rcu_preempt21:20:020
3762152994235,4cyclictest16950irq/224-800000019:55:000
3762152994230,9cyclictest1-21systemd23:15:010
3762152994227,12cyclictest272-21dbus-daemon19:39:590
3762152994227,11cyclictest1-21systemd22:15:000
376215299422,3cyclictest131rcu_preempt23:50:020
376215299422,35cyclictest3776119-21cron23:40:000
376215299422,35cyclictest3776119-21cron23:39:590
3762152994216,10cyclictest3766739-21sshd20:36:560
376216799413,4cyclictest131rcu_preempt21:10:003
3762167994131,6cyclictest3769663-21latency_hist21:35:023
3762167994129,10cyclictest37-21ksoftirqd/322:35:003
3762167994127,9cyclictest1-21systemd22:30:013
3762167994121,17cyclictest37-21ksoftirqd/321:55:023
376215899412,7cyclictest23-21ksoftirqd/119:50:001
376215299414,4cyclictest131rcu_preempt23:30:010
3762152994130,8cyclictest3768338-21cron21:10:000
3762152994126,12cyclictest3768578-21sshd21:13:330
3762152994126,11cyclictest3777927-21sshd00:12:330
3762152994126,11cyclictest3767796-21modprobe20:59:590
3762152994123,14cyclictest1-21systemd23:59:590
3762152994123,14cyclictest1-21systemd00:00:000
376215299411,12cyclictest3778476-21sshd00:22:440
376216799405,9cyclictest37-21ksoftirqd/300:34:593
376216799405,4cyclictest131rcu_preempt23:40:003
376216799405,4cyclictest131rcu_preempt23:39:593
376216799404,7cyclictest37-21ksoftirqd/320:20:023
376216799403,6cyclictest131rcu_preempt21:47:323
3762167994026,10cyclictest1-21systemd22:24:593
3762167994025,11cyclictest272-21dbus-daemon00:05:003
3762167994024,12cyclictest272-21dbus-daemon22:05:013
3762167994024,11cyclictest272-21dbus-daemon19:40:003
376216799402,3cyclictest131rcu_preempt23:30:013
3762167994021,15cyclictest37-21ksoftirqd/300:15:003
3762167994016,7cyclictest361rcuc/319:35:023
3762167994016,7cyclictest361rcuc/319:35:013
376215299404,4cyclictest131rcu_preempt21:55:030
3762152994033,4cyclictest16850irq/134-800000021:27:060
3762152994025,10cyclictest235-21systemd-journal22:30:000
376215299401,4cyclictest131rcu_preempt00:10:020
376215299401,3cyclictest131rcu_preempt22:00:000
376216799399,6cyclictest131rcu_preempt23:15:023
3762167993929,6cyclictest271-21cron00:40:003
376216799392,3cyclictest131rcu_preempt23:25:013
3762167993923,12cyclictest272-21dbus-daemon23:59:593
3762167993923,12cyclictest272-21dbus-daemon00:00:003
3762167993918,4cyclictest37-21ksoftirqd/321:30:013
376215299397,6cyclictest131rcu_preempt22:20:010
376215299397,6cyclictest131rcu_preempt22:20:010
376215299395,3cyclictest131rcu_preempt19:25:010
376215299394,4cyclictest131rcu_preempt20:30:010
376215299394,4cyclictest131rcu_preempt20:30:000
3762152993932,4cyclictest16950irq/224-800000022:55:000
3762152993926,10cyclictest3773526-21sshd22:48:340
3762152993926,10cyclictest3773526-21sshd22:48:330
3762152993924,11cyclictest408-21lldpd23:06:050
3762152993924,11cyclictest12-21ksoftirqd/020:50:010
3762152993924,11cyclictest1-21systemd20:45:000
376215299392,32cyclictest3771438-21sshd22:07:120
376216799389,3cyclictest131rcu_preempt23:55:013
376216799389,3cyclictest131rcu_preempt23:55:013
376216799388,3cyclictest131rcu_preempt22:00:013
376216799383,4cyclictest131rcu_preempt21:20:323
3762167993828,6cyclictest306-21in:imuxsock23:45:013
376216799382,4cyclictest131rcu_preempt20:25:013
3762167993822,12cyclictest3767311-21systemd-run20:50:013
376216399382,7cyclictest30-21ksoftirqd/222:45:012
376215299385,4cyclictest131rcu_preempt00:15:030
3762152993826,9cyclictest235-21systemd-journal19:32:260
3762152993826,9cyclictest235-21systemd-journal19:32:250
3762152993824,10cyclictest12-21ksoftirqd/020:10:000
376215299381,3cyclictest131rcu_preempt21:22:040
376216799376,3cyclictest131rcu_preempt22:47:243
376216799376,3cyclictest131rcu_preempt22:47:233
376216799375,3cyclictest131rcu_preempt20:35:003
376216799374,4cyclictest131rcu_preempt00:20:003
3762167993725,8cyclictest3762593-21sshd19:18:313
376216799371,3cyclictest131rcu_preempt20:30:023
376216799371,3cyclictest131rcu_preempt20:30:013
376215299376,3cyclictest131rcu_preempt22:25:000
3762152993726,7cyclictest3777551-21grep00:05:010
3762152993723,10cyclictest408-21lldpd19:15:300
376215299371,3cyclictest0-21swapper/019:49:480
376216799367,3cyclictest131rcu_preempt20:36:523
376216799366,3cyclictest131rcu_preempt23:00:013
376216799366,3cyclictest131rcu_preempt22:52:253
376216799365,3cyclictest131rcu_preempt21:45:003
376216799365,3cyclictest131rcu_preempt00:20:093
376216799363,3cyclictest131rcu_preempt21:05:003
376216799363,3cyclictest131rcu_preempt19:26:313
376215299364,3cyclictest131rcu_preempt21:05:010
376215299362,5cyclictest131rcu_preempt20:20:000
376215299362,3cyclictest131rcu_preempt20:55:020
376215299361,31cyclictest0-21swapper/000:28:080
376215299361,31cyclictest0-21swapper/000:28:080
376216799356,3cyclictest131rcu_preempt21:17:043
376216799355,3cyclictest131rcu_preempt23:32:333
376216799355,3cyclictest131rcu_preempt00:10:023
376216799352,3cyclictest131rcu_preempt20:05:023
3762167993520,11cyclictest1-21systemd22:15:003
376215299353,5cyclictest131rcu_preempt20:00:310
376215299352,3cyclictest131rcu_preempt19:10:020
376216799345,4cyclictest131rcu_preempt20:41:313
376216799343,4cyclictest131rcu_preempt20:55:013
3762167993425,6cyclictest306-21in:imuxsock00:30:013
3762167993425,6cyclictest306-21in:imuxsock00:30:013
3762152993422,9cyclictest3770186-21sh21:45:010
3762152993421,9cyclictest306-21in:imuxsock19:30:010
376216799334,4cyclictest131rcu_preempt19:21:453
376216799333,3cyclictest131rcu_preempt23:05:023
376216799333,3cyclictest131rcu_preempt23:05:013
376216799332,3cyclictest131rcu_preempt21:40:013
3762167993320,9cyclictest3763905-21sshd19:41:473
3762167993216,4cyclictest37-21ksoftirqd/319:50:023
376216799313,4cyclictest131rcu_preempt22:45:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional