You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Thu Dec 04, 2025 00:45:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1053840991093,9cyclictest30-21ksoftirqd/223:40:032
1053833991068,17cyclictest131rcu_preempt19:10:020
1053833997968,3cyclictest0-21swapper/022:25:030
105383399695,22cyclictest0-21swapper/023:40:020
1053844996414,11cyclictest37-21ksoftirqd/319:20:003
105384499627,10cyclictest37-21ksoftirqd/321:40:023
1053840996120,11cyclictest30-21ksoftirqd/219:40:002
1053840996116,5cyclictest131rcu_preempt23:30:012
105384099611,15cyclictest30-21ksoftirqd/222:50:012
1053833996127,24cyclictest151rcuc/023:55:010
105384499605,9cyclictest37-21ksoftirqd/319:30:393
105384099607,11cyclictest30-21ksoftirqd/220:35:032
105383899604,10cyclictest23-21ksoftirqd/122:40:021
105384499598,9cyclictest37-21ksoftirqd/321:00:003
105384499598,9cyclictest37-21ksoftirqd/321:00:003
105384499595,7cyclictest37-21ksoftirqd/300:15:013
1053844995940,15cyclictest1-21systemd23:55:003
105384099599,14cyclictest30-21ksoftirqd/200:00:022
105384099596,10cyclictest30-21ksoftirqd/200:20:002
105383899596,28cyclictest1063271-21kworker/u8:022:20:031
1053833995952,4cyclictest16950irq/224-800000020:20:010
105384499585,11cyclictest37-21ksoftirqd/320:45:013
1053844995815,6cyclictest131rcu_preempt21:15:013
105384099588,12cyclictest30-21ksoftirqd/221:05:002
105384099586,7cyclictest30-21ksoftirqd/221:55:012
105383899585,9cyclictest23-21ksoftirqd/122:00:001
1053838995813,7cyclictest131rcu_preempt23:20:031
1053833995818,14cyclictest1069037-40mandb00:00:020
105384499574,9cyclictest37-21ksoftirqd/323:50:023
1053844995711,6cyclictest131rcu_preempt22:55:043
105384099578,9cyclictest30-21ksoftirqd/222:05:012
1053840995714,5cyclictest131rcu_preempt20:50:032
1053840995712,6cyclictest131rcu_preempt23:35:012
1053838995711,6cyclictest131rcu_preempt23:45:011
105384499566,6cyclictest37-21ksoftirqd/322:20:013
1053844995640,11cyclictest1-21systemd22:25:013
1053844995615,5cyclictest131rcu_preempt20:40:023
1053844995613,6cyclictest131rcu_preempt23:00:003
1053844995613,6cyclictest131rcu_preempt22:05:013
1053844995612,9cyclictest37-21ksoftirqd/323:10:023
1053844995611,6cyclictest131rcu_preempt20:20:013
1053844995610,6cyclictest131rcu_preempt22:35:013
105384099569,6cyclictest131rcu_preempt22:10:032
105384099569,6cyclictest131rcu_preempt22:10:022
105384099567,5cyclictest30-21ksoftirqd/222:35:032
105384099563,9cyclictest30-21ksoftirqd/222:40:002
1053840995614,5cyclictest131rcu_preempt22:45:002
105383899565,6cyclictest23-21ksoftirqd/119:19:421
1053838995632,11cyclictest23-21ksoftirqd/119:45:001
1053838995611,6cyclictest131rcu_preempt00:40:021
1053833995625,8cyclictest12-21ksoftirqd/022:15:010
105384499557,11cyclictest37-21ksoftirqd/320:15:063
105384499555,10cyclictest37-21ksoftirqd/320:25:023
1053844995536,8cyclictest37-21ksoftirqd/321:20:023
1053844995525,17cyclictest37-21ksoftirqd/319:45:003
105384099554,9cyclictest30-21ksoftirqd/219:30:382
105383899557,6cyclictest131rcu_preempt23:00:001
1053838995516,8cyclictest23-21ksoftirqd/119:30:011
105383399555,29cyclictest16950irq/224-800000019:30:380
105384499549,5cyclictest131rcu_preempt22:10:023
105384499549,5cyclictest131rcu_preempt22:10:013
105384499548,8cyclictest37-21ksoftirqd/322:50:023
105384499546,10cyclictest37-21ksoftirqd/300:10:013
105384499546,10cyclictest37-21ksoftirqd/300:10:003
1053844995410,5cyclictest131rcu_preempt00:05:033
105384099546,5cyclictest131rcu_preempt00:10:032
105384099544,9cyclictest30-21ksoftirqd/221:30:002
1053840995425,9cyclictest30-21ksoftirqd/221:00:012
1053840995425,9cyclictest30-21ksoftirqd/221:00:002
1053840995415,4cyclictest131rcu_preempt22:30:012
1053840995410,5cyclictest131rcu_preempt20:25:022
105383899548,5cyclictest131rcu_preempt20:40:011
105383899542,5cyclictest131rcu_preempt21:40:011
105384499539,6cyclictest131rcu_preempt21:50:003
105384499539,11cyclictest37-21ksoftirqd/321:10:013
105384499536,7cyclictest37-21ksoftirqd/322:30:023
105384099537,10cyclictest30-21ksoftirqd/200:05:022
105384099536,9cyclictest30-21ksoftirqd/219:25:012
105384099536,8cyclictest30-21ksoftirqd/220:15:052
105384099536,7cyclictest30-21ksoftirqd/200:25:032
105384099536,7cyclictest30-21ksoftirqd/200:25:022
105384099535,9cyclictest30-21ksoftirqd/223:50:012
105384099534,8cyclictest30-21ksoftirqd/223:22:412
105384099532,23cyclictest30-21ksoftirqd/221:45:022
105384099532,12cyclictest30-21ksoftirqd/200:40:002
1053840995316,6cyclictest131rcu_preempt22:55:012
1053840995313,6cyclictest131rcu_preempt19:45:012
105383899539,5cyclictest131rcu_preempt21:45:021
1053838995313,12cyclictest23-21ksoftirqd/119:25:001
1053838995312,6cyclictest131rcu_preempt21:50:011
1053838995311,6cyclictest131rcu_preempt20:50:011
1053844995232,9cyclictest37-21ksoftirqd/319:30:013
1053844995224,10cyclictest37-21ksoftirqd/323:45:013
1053844995213,6cyclictest131rcu_preempt20:50:013
1053844995211,5cyclictest131rcu_preempt20:35:013
1053844995211,4cyclictest131rcu_preempt21:45:023
105384099529,6cyclictest131rcu_preempt19:19:422
105384099529,5cyclictest131rcu_preempt21:11:322
105384099527,6cyclictest30-21ksoftirqd/220:30:012
105384099527,4cyclictest131rcu_preempt21:50:022
105384099524,13cyclictest30-21ksoftirqd/220:10:012
1053840995215,5cyclictest131rcu_preempt19:50:012
1053840995213,11cyclictest30-21ksoftirqd/220:05:002
105383899529,6cyclictest131rcu_preempt23:55:011
105383899525,5cyclictest131rcu_preempt23:40:011
105383899521,5cyclictest131rcu_preempt00:20:011
105383899521,10cyclictest1067788-21cat23:35:021
1053838995210,5cyclictest131rcu_preempt20:55:021
1053833995245,3cyclictest12-21ksoftirqd/000:10:030
1053833995245,3cyclictest12-21ksoftirqd/000:10:020
1053833995237,10cyclictest1-21systemd19:25:490
1053833995234,13cyclictest1-21systemd21:20:010
1053833995214,33cyclictest1059459-21sort20:55:010
105384499519,9cyclictest37-21ksoftirqd/321:25:013
105384499518,5cyclictest131rcu_preempt00:20:033
105384499517,8cyclictest37-21ksoftirqd/319:25:013
105384499517,7cyclictest37-21ksoftirqd/320:55:023
105384499517,5cyclictest131rcu_preempt00:30:023
105384499516,9cyclictest37-21ksoftirqd/300:00:033
105384499516,10cyclictest37-21ksoftirqd/300:21:283
105384499516,10cyclictest37-21ksoftirqd/300:21:273
1053844995135,12cyclictest272-21dbus-daemon20:00:013
1053844995111,5cyclictest131rcu_preempt23:15:023
105384099519,6cyclictest131rcu_preempt00:35:022
105384099518,11cyclictest30-21ksoftirqd/222:00:022
105384099517,7cyclictest30-21ksoftirqd/222:15:012
105384099516,5cyclictest30-21ksoftirqd/221:35:022
105384099511,9cyclictest30-21ksoftirqd/223:55:022
105384099511,11cyclictest1067174-21cut23:20:022
105383899513,5cyclictest131rcu_preempt20:25:011
105383899512,4cyclictest131rcu_preempt00:15:001
105383899511,5cyclictest131rcu_preempt21:34:591
1053833995112,6cyclictest131rcu_preempt20:30:000
1053844995013,6cyclictest131rcu_preempt23:30:013
1053844995012,6cyclictest131rcu_preempt22:00:003
1053844995012,5cyclictest131rcu_preempt20:30:003
1053844995011,5cyclictest131rcu_preempt22:42:163
105384099508,5cyclictest131rcu_preempt20:36:502
105384099508,11cyclictest30-21ksoftirqd/221:40:012
105384099507,7cyclictest30-21ksoftirqd/219:57:362
105384099506,10cyclictest30-21ksoftirqd/221:25:012
105383899507,6cyclictest131rcu_preempt23:30:021
105383899502,4cyclictest131rcu_preempt20:10:011
105383899502,12cyclictest1070532-21latency_hist00:25:031
1053838995010,6cyclictest131rcu_preempt22:05:021
1053833995034,11cyclictest306-21in:imuxsock00:20:000
105383399501,20cyclictest131rcu_preempt23:00:010
105384499496,7cyclictest37-21ksoftirqd/322:40:003
105384499494,10cyclictest37-21ksoftirqd/320:10:023
1053844994913,9cyclictest131rcu_preempt19:15:033
1053844994913,6cyclictest131rcu_preempt00:40:013
105384099497,8cyclictest30-21ksoftirqd/220:17:002
105384099497,7cyclictest30-21ksoftirqd/221:20:032
105384099497,6cyclictest30-21ksoftirqd/219:15:002
105384099495,9cyclictest30-21ksoftirqd/200:30:002
105384099494,8cyclictest30-21ksoftirqd/223:04:592
105384099494,12cyclictest30-21ksoftirqd/223:10:002
105384099493,10cyclictest30-21ksoftirqd/200:10:002
105384099493,10cyclictest30-21ksoftirqd/200:10:002
105384099492,11cyclictest30-21ksoftirqd/222:25:002
105384099491,11cyclictest1066866-21latency_hist23:15:032
1053840994910,5cyclictest131rcu_preempt20:45:012
1053840994910,5cyclictest131rcu_preempt19:25:362
105383899496,7cyclictest131rcu_preempt22:50:021
105383899496,6cyclictest131rcu_preempt20:20:021
105383899496,5cyclictest131rcu_preempt20:35:021
105383899492,5cyclictest131rcu_preempt20:45:001
1053838994913,6cyclictest131rcu_preempt19:15:011
1053838994910,5cyclictest131rcu_preempt00:05:021
105383399492,12cyclictest1065272-21cat22:45:020
105384499488,7cyclictest37-21ksoftirqd/319:10:023
105384499487,9cyclictest37-21ksoftirqd/323:25:013
105384499486,11cyclictest37-21ksoftirqd/319:40:013
105384499483,4cyclictest131rcu_preempt23:35:023
1053844994810,6cyclictest131rcu_preempt20:05:033
1053844994810,5cyclictest131rcu_preempt23:40:023
105384099486,5cyclictest131rcu_preempt23:45:012
1053840994814,6cyclictest131rcu_preempt19:10:022
105384099481,15cyclictest1065898-21sendmail23:00:012
105383899488,6cyclictest131rcu_preempt22:10:021
105383899488,6cyclictest131rcu_preempt22:10:021
105383899482,5cyclictest131rcu_preempt19:55:011
105383899481,8cyclictest23-21ksoftirqd/100:20:151
105383899481,5cyclictest131rcu_preempt22:30:011
1053838994815,9cyclictest23-21ksoftirqd/121:55:011
105383899481,10cyclictest131rcu_preempt21:25:011
1053838994810,6cyclictest131rcu_preempt21:00:011
1053838994810,6cyclictest131rcu_preempt21:00:001
105384499477,7cyclictest37-21ksoftirqd/323:20:023
105384499476,8cyclictest37-21ksoftirqd/321:05:013
105384499475,8cyclictest37-21ksoftirqd/319:50:013
1053844994713,5cyclictest131rcu_preempt19:55:003
1053844994712,6cyclictest131rcu_preempt00:35:013
105384499471,10cyclictest37-21ksoftirqd/321:25:033
105384099479,5cyclictest131rcu_preempt22:15:032
105384099475,8cyclictest30-21ksoftirqd/219:55:012
105384099472,7cyclictest30-21ksoftirqd/221:10:022
1053840994716,5cyclictest131rcu_preempt20:55:002
105383899479,5cyclictest131rcu_preempt20:30:011
105383899477,8cyclictest23-21ksoftirqd/100:35:001
105383899477,6cyclictest131rcu_preempt19:10:031
105383399471,9cyclictest131rcu_preempt21:00:030
105383399471,3cyclictest131rcu_preempt19:45:000
105384499469,5cyclictest131rcu_preempt21:34:593
105383899468,6cyclictest131rcu_preempt19:50:031
105383899466,7cyclictest131rcu_preempt00:00:031
105383899463,9cyclictest23-21ksoftirqd/121:30:021
105383899462,8cyclictest23-21ksoftirqd/122:15:011
105383399466,8cyclictest12-21ksoftirqd/021:55:020
105383399463,3cyclictest131rcu_preempt23:15:000
1053833994631,11cyclictest235-21systemd-journal23:48:510
105384499457,9cyclictest37-21ksoftirqd/322:12:103
105384499457,7cyclictest37-21ksoftirqd/321:55:033
105384499455,5cyclictest131rcu_preempt23:05:013
105383899459,6cyclictest131rcu_preempt00:30:011
105383899459,5cyclictest131rcu_preempt23:05:021
105383899458,6cyclictest131rcu_preempt20:00:001
105383899457,4cyclictest131rcu_preempt23:45:391
105383899457,4cyclictest131rcu_preempt23:15:001
105383899454,6cyclictest131rcu_preempt20:10:031
105383899454,6cyclictest131rcu_preempt19:35:031
1053838994520,11cyclictest23-21ksoftirqd/119:40:001
1053833994532,6cyclictest12-21ksoftirqd/019:40:010
1053838994434,7cyclictest23-21ksoftirqd/121:10:391
105383899442,5cyclictest131rcu_preempt22:55:021
105383899441,6cyclictest131rcu_preempt21:10:011
1053838994415,23cyclictest131rcu_preempt21:05:001
105383399448,32cyclictest12-21ksoftirqd/020:14:380
105383399446,7cyclictest12-21ksoftirqd/021:45:030
105383899432,5cyclictest131rcu_preempt21:20:031
105383899432,4cyclictest131rcu_preempt23:25:001
1053833994336,4cyclictest16950irq/224-800000020:00:010
1053833994333,6cyclictest16850irq/134-800000023:42:280
1053833994332,8cyclictest12-21ksoftirqd/000:25:010
1053833994332,8cyclictest12-21ksoftirqd/000:25:000
1053833994330,10cyclictest235-21systemd-journal20:41:390
105383399432,37cyclictest363-21snmpd00:35:070
1053838994232,7cyclictest23-21ksoftirqd/120:05:011
1053838994230,8cyclictest1063820-21cron22:20:001
105383899422,6cyclictest131rcu_preempt23:10:011
1053838994210,6cyclictest131rcu_preempt22:44:591
105383399428,31cyclictest131rcu_preempt22:00:020
1053833994235,4cyclictest16950irq/224-800000019:25:000
1053833994226,12cyclictest1065366-21sshd22:48:390
1053833994226,11cyclictest272-21dbus-daemon19:20:010
1053833994220,3cyclictest0-21swapper/021:30:030
105383899416,9cyclictest23-21ksoftirqd/100:10:021
105383899416,9cyclictest23-21ksoftirqd/100:10:011
1053838994129,8cyclictest23-21ksoftirqd/122:30:391
1053833994131,6cyclictest19050irq/34-mmc121:00:010
1053833994131,6cyclictest19050irq/34-mmc121:00:010
105383399411,4cyclictest131rcu_preempt20:50:020
105383399403,33cyclictest0-21swapper/022:25:020
105383399402,5cyclictest131rcu_preempt22:20:010
1053833994024,12cyclictest1060400-21sed21:10:030
105383399402,33cyclictest235-21systemd-journal00:29:040
105383399401,8cyclictest1057863-21systemd-run20:25:010
105383399401,4cyclictest131rcu_preempt19:55:010
105383399401,35cyclictest1054004-21sshd19:13:370
105383399394,3cyclictest131rcu_preempt20:30:370
105383399392,5cyclictest1-21systemd20:05:010
1053833993923,12cyclictest1060428-21sshd21:14:380
1053833993921,13cyclictest1064590-21sshd22:34:380
105383399388,6cyclictest131rcu_preempt20:10:000
105383399386,7cyclictest12-21ksoftirqd/023:25:020
1053833993829,6cyclictest19050irq/34-mmc122:50:250
1053833993827,8cyclictest12-21ksoftirqd/022:05:020
1053833993825,10cyclictest1062235-21sshd21:45:380
105383399382,3cyclictest131rcu_preempt19:46:360
105383399381,33cyclictest0-21swapper/000:05:020
1053833993728,6cyclictest12-21ksoftirqd/022:10:030
1053833993728,6cyclictest12-21ksoftirqd/022:10:020
1053833993721,5cyclictest131rcu_preempt00:15:000
105383399371,4cyclictest131rcu_preempt21:25:000
105383399371,33cyclictest1066919-21sshd23:18:400
105383399363,3cyclictest131rcu_preempt21:40:010
105383399363,20cyclictest19050irq/34-mmc123:10:020
105383399362,8cyclictest1058617-21munin-run20:40:000
105383399362,5cyclictest1-21systemd00:35:000
1053833993528,4cyclictest0-21swapper/023:26:130
1053833993517,7cyclictest131rcu_preempt21:35:020
105383399351,5cyclictest1-21systemd23:35:010
1053833993422,9cyclictest12-21ksoftirqd/023:05:000
1053833993317,5cyclictest131rcu_preempt22:37:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional