You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 01:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Nov 21, 2025 12:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3701258998617,30cyclictest0-21swapper/308:55:033
3701243996627,9cyclictest12-21ksoftirqd/010:05:020
3701251995937,16cyclictest235-21systemd-journal09:10:001
3701251995937,16cyclictest235-21systemd-journal09:10:001
3701243995816,11cyclictest12-21ksoftirqd/009:25:020
3701243995812,6cyclictest0-21swapper/009:40:010
3701243995812,6cyclictest0-21swapper/009:40:010
3701243995721,9cyclictest12-21ksoftirqd/007:35:020
3701243995716,12cyclictest12-21ksoftirqd/008:50:020
370125899568,6cyclictest131rcu_preempt11:10:033
3701258995447,5cyclictest361rcuc/308:00:013
3701256995441,8cyclictest3708991-21sendmail09:40:002
3701256995441,8cyclictest3708991-21sendmail09:40:002
370124399547,12cyclictest12-21ksoftirqd/007:25:020
3701243995414,8cyclictest12-21ksoftirqd/007:15:000
3701243995316,32cyclictest1-21systemd10:15:010
3701256995241,8cyclictest3710029-21sendmail09:59:592
3701256995236,12cyclictest272-21dbus-daemon10:55:012
370125199521,3cyclictest131rcu_preempt12:20:021
3701256995110,7cyclictest131rcu_preempt09:25:022
370124399516,9cyclictest12-21ksoftirqd/012:10:020
370124399514,10cyclictest12-21ksoftirqd/007:45:000
370124399512,5cyclictest0-21swapper/008:25:030
3701256995034,11cyclictest272-21dbus-daemon10:45:002
3701256995017,11cyclictest30-21ksoftirqd/212:15:012
370124399506,9cyclictest12-21ksoftirqd/010:45:020
370124399506,8cyclictest12-21ksoftirqd/009:30:020
3701243995040,6cyclictest3713187-21sendmail10:59:590
3701243995012,7cyclictest0-21swapper/010:00:010
3701258994935,11cyclictest3716317-21tr12:00:013
370124399496,7cyclictest12-21ksoftirqd/007:20:030
370124399495,6cyclictest12-21ksoftirqd/011:15:010
3701243994936,10cyclictest3717300-21sshd12:15:300
3701243994915,3cyclictest0-21swapper/012:15:020
3701243994911,4cyclictest19050irq/34-mmc111:05:010
370125899487,15cyclictest37-21ksoftirqd/308:25:003
370125899484,3cyclictest131rcu_preempt11:25:023
3701258994829,16cyclictest1-21systemd12:15:013
370125699483,26cyclictest30-21ksoftirqd/212:00:022
3701256994831,11cyclictest1-21systemd09:45:002
370125699481,9cyclictest3705122-21latency_hist08:25:012
370125699481,8cyclictest3701709-21munin-run07:20:012
3701256994812,6cyclictest131rcu_preempt12:40:022
370124399488,4cyclictest131rcu_preempt12:00:020
370124399486,6cyclictest12-21ksoftirqd/009:00:020
370124399486,10cyclictest12-21ksoftirqd/007:40:010
370124399485,8cyclictest12-21ksoftirqd/008:00:000
370124399484,5cyclictest12-21ksoftirqd/008:20:010
370124399484,10cyclictest12-21ksoftirqd/008:45:020
370124399482,13cyclictest12-21ksoftirqd/008:05:020
370124399482,12cyclictest3709781-21systemd-run09:55:010
370124399481,11cyclictest3709268-21munin-run09:45:000
370125899471,3cyclictest131rcu_preempt07:55:003
3701256994736,8cyclictest30-21ksoftirqd/211:55:002
3701256994720,7cyclictest30-21ksoftirqd/212:20:022
3701256994715,6cyclictest131rcu_preempt08:05:012
3701256994710,6cyclictest131rcu_preempt08:17:002
3701251994731,11cyclictest1-21systemd09:20:011
3701251994728,15cyclictest1-21systemd08:00:011
370124399479,4cyclictest131rcu_preempt11:25:120
370124399474,9cyclictest12-21ksoftirqd/012:40:010
370124399474,8cyclictest12-21ksoftirqd/010:30:010
370124399473,6cyclictest12-21ksoftirqd/010:50:000
370124399472,8cyclictest12-21ksoftirqd/008:15:020
3701243994721,8cyclictest12-21ksoftirqd/007:49:070
370124399472,15cyclictest12-21ksoftirqd/012:30:010
370124399471,7cyclictest0-21swapper/008:35:000
3701243994713,7cyclictest0-21swapper/009:10:000
3701243994713,7cyclictest0-21swapper/009:10:000
370125699464,7cyclictest30-21ksoftirqd/208:50:022
3701256994636,7cyclictest30-21ksoftirqd/209:55:032
3701256994634,8cyclictest30-21ksoftirqd/211:30:012
3701256994628,13cyclictest272-21dbus-daemon08:45:012
370125699461,7cyclictest3707791-21wc09:15:012
3701256994610,6cyclictest131rcu_preempt10:50:002
370125199463,3cyclictest131rcu_preempt07:35:031
370124399464,9cyclictest12-21ksoftirqd/012:35:020
370124399464,9cyclictest12-21ksoftirqd/011:25:030
370125899454,3cyclictest131rcu_preempt07:40:013
3701258994527,14cyclictest3712113-21systemctl10:40:003
3701258994527,14cyclictest3712113-21systemctl10:40:003
3701258994515,5cyclictest131rcu_preempt10:55:003
370125699451,11cyclictest30-21ksoftirqd/211:50:002
3701251994532,9cyclictest272-21dbus-daemon07:50:011
370124399454,12cyclictest12-21ksoftirqd/010:20:020
370124399454,10cyclictest12-21ksoftirqd/009:35:010
3701243994538,4cyclictest16950irq/224-800000011:35:020
370124399451,15cyclictest3710558-21systemd-run10:10:010
3701258994430,10cyclictest3711055-21cron10:20:003
3701258994427,14cyclictest235-21systemd-journal09:35:003
370125699448,26cyclictest3700584-21kworker/u8:207:40:022
370125199444,3cyclictest131rcu_preempt10:05:011
3701251994427,13cyclictest272-21dbus-daemon12:40:001
370124399445,8cyclictest12-21ksoftirqd/007:30:020
370124399445,10cyclictest12-21ksoftirqd/011:10:030
370124399444,6cyclictest12-21ksoftirqd/011:45:040
370124399443,8cyclictest12-21ksoftirqd/011:20:010
3701243994425,16cyclictest19050irq/34-mmc110:54:580
3701243994411,9cyclictest0-21swapper/008:25:010
3701243994410,9cyclictest0-21swapper/010:35:010
3701258994331,8cyclictest235-21systemd-journal11:45:003
3701258994328,11cyclictest235-21systemd-journal09:20:003
3701258994327,12cyclictest272-21dbus-daemon11:35:013
370125899432,3cyclictest131rcu_preempt07:50:023
370125699434,10cyclictest30-21ksoftirqd/211:25:002
370125699433,7cyclictest30-21ksoftirqd/208:29:352
370125699433,4cyclictest131rcu_preempt09:26:312
3701256994334,6cyclictest30-21ksoftirqd/211:20:022
3701256994332,7cyclictest3707414-21cron09:10:002
3701256994332,7cyclictest3707414-21cron09:10:002
3701256994331,9cyclictest30-21ksoftirqd/208:10:002
370125699431,6cyclictest131rcu_preempt09:05:012
3701256994314,7cyclictest30-21ksoftirqd/211:45:022
3701251994331,8cyclictest3708454-21sshd09:27:311
3701251994323,15cyclictest272-21dbus-daemon09:40:011
3701251994323,15cyclictest272-21dbus-daemon09:40:011
370124399433,9cyclictest12-21ksoftirqd/011:40:010
3701243994334,6cyclictest151rcuc/010:40:000
3701243994334,6cyclictest151rcuc/010:40:000
3701243994332,7cyclictest3705869-21modprobe08:40:000
3701243994331,8cyclictest0-21swapper/012:25:020
3701243994327,11cyclictest272-21dbus-daemon09:15:000
370125899422,3cyclictest131rcu_preempt09:40:043
3701256994233,5cyclictest30-21ksoftirqd/207:50:012
3701256994228,9cyclictest306-21in:imuxsock07:24:312
3701256994227,11cyclictest1-21systemd12:30:012
3701256994212,6cyclictest131rcu_preempt08:35:012
370125199423,3cyclictest131rcu_preempt08:45:021
3701251994229,8cyclictest235-21systemd-journal07:20:011
3701251994225,13cyclictest272-21dbus-daemon09:55:011
370125199421,3cyclictest131rcu_preempt10:10:021
370124399425,8cyclictest12-21ksoftirqd/011:45:010
3701243994228,11cyclictest12-21ksoftirqd/009:20:000
3701243994212,5cyclictest0-21swapper/009:50:020
370125899413,3cyclictest131rcu_preempt10:00:013
3701258994126,11cyclictest235-21systemd-journal10:10:043
3701258994124,12cyclictest37-21ksoftirqd/309:25:003
370125699419,10cyclictest30-21ksoftirqd/209:00:022
370125699417,6cyclictest131rcu_preempt07:35:022
3701256994131,5cyclictest306-21in:imuxsock07:55:002
3701256994128,10cyclictest235-21systemd-journal10:11:312
370125699412,7cyclictest30-21ksoftirqd/212:10:022
370125699411,8cyclictest30-21ksoftirqd/207:30:022
3701256994110,5cyclictest131rcu_preempt10:40:012
3701256994110,5cyclictest131rcu_preempt10:40:002
3701251994127,10cyclictest272-21dbus-daemon10:00:011
3701251994126,11cyclictest1-21systemd12:10:011
3701251994125,12cyclictest272-21dbus-daemon07:45:011
3701251994118,16cyclictest235-21systemd-journal10:25:011
3701251994118,16cyclictest235-21systemd-journal10:25:001
3701243994130,7cyclictest3716047-21munin-plugin-st11:55:000
3701258994030,8cyclictest37-21ksoftirqd/312:25:023
3701258994028,9cyclictest235-21systemd-journal11:15:003
3701258994026,10cyclictest235-21systemd-journal12:40:023
3701258994024,11cyclictest1-21systemd11:00:003
3701258994022,13cyclictest235-21systemd-journal10:30:003
370125699408,6cyclictest131rcu_preempt12:05:012
370125699403,5cyclictest131rcu_preempt10:05:012
3701256994029,7cyclictest3711597-21munin-plugin-st10:29:592
3701251994030,5cyclictest306-21in:imuxsock07:25:001
3701251994028,9cyclictest3701452-21pluginstate07:15:011
3701251994025,10cyclictest306-21in:imuxsock07:35:031
370125199401,3cyclictest131rcu_preempt10:59:591
370124399409,27cyclictest151rcuc/007:52:310
370124399406,6cyclictest12-21ksoftirqd/008:55:010
370124399403,9cyclictest12-21ksoftirqd/008:10:010
3701243994028,10cyclictest12-21ksoftirqd/012:05:010
370124399402,33cyclictest3711313-21sshd10:20:050
370124399402,33cyclictest3711313-21sshd10:20:050
3701258993930,5cyclictest3709771-21munin-plugin-st09:55:003
3701258993928,8cyclictest37-21ksoftirqd/307:20:033
370125899392,6cyclictest37-21ksoftirqd/307:10:013
3701258993925,10cyclictest235-21systemd-journal07:15:003
3701258993923,12cyclictest272-21dbus-daemon08:20:003
3701258993923,12cyclictest235-21systemd-journal07:20:043
3701258993921,13cyclictest3709000-21systemctl09:40:003
3701258993921,13cyclictest3709000-21systemctl09:40:003
3701258993911,5cyclictest131rcu_preempt09:30:013
370125699398,7cyclictest131rcu_preempt11:05:002
370125699393,6cyclictest131rcu_preempt11:35:012
3701256993929,6cyclictest3713187-21sendmail11:00:002
3701256993926,10cyclictest30-21ksoftirqd/209:20:012
370125199395,4cyclictest131rcu_preempt10:35:021
3701251993927,9cyclictest23-21ksoftirqd/109:25:031
3701251993926,10cyclictest3702367-21wc07:30:021
3701251993921,12cyclictest235-21systemd-journal11:40:001
370124399399,4cyclictest131rcu_preempt09:05:010
3701258993826,10cyclictest37-21ksoftirqd/308:35:003
370125699387,5cyclictest131rcu_preempt08:40:002
370125699385,10cyclictest30-21ksoftirqd/207:42:302
370125699384,6cyclictest131rcu_preempt07:10:012
370125699382,6cyclictest131rcu_preempt08:00:012
370125699382,4cyclictest131rcu_preempt11:40:022
3701256993813,6cyclictest131rcu_preempt07:10:022
3701256993812,5cyclictest131rcu_preempt12:20:312
3701256993812,5cyclictest131rcu_preempt10:25:022
3701256993812,5cyclictest131rcu_preempt10:25:022
3701256993811,6cyclictest131rcu_preempt09:47:312
370125199389,22cyclictest0-21swapper/109:15:001
370125199384,4cyclictest131rcu_preempt12:00:011
370125199382,8cyclictest3704554-21munin-run08:15:001
370125199382,3cyclictest131rcu_preempt12:25:011
3701251993821,7cyclictest23-21ksoftirqd/108:20:031
3701251993821,12cyclictest1-21systemd08:20:001
3701243993827,6cyclictest3701267-21date07:10:010
3701258993729,5cyclictest3706132-21latency_hist08:45:023
3701258993726,6cyclictest37-21ksoftirqd/308:15:003
3701258993725,7cyclictest37-21ksoftirqd/309:10:003
3701258993725,7cyclictest37-21ksoftirqd/309:10:003
3701258993722,11cyclictest306-21in:imuxsock10:45:023
3701258993719,13cyclictest0-21swapper/312:20:003
370125699376,11cyclictest30-21ksoftirqd/211:15:022
3701256993728,6cyclictest30-21ksoftirqd/212:35:022
370125699371,5cyclictest0-21swapper/208:15:022
3701256993713,6cyclictest131rcu_preempt10:10:012
3701251993729,5cyclictest3711569-21sshd10:26:311
3701251993728,7cyclictest23-21ksoftirqd/108:55:021
3701251993726,9cyclictest23-21ksoftirqd/110:40:011
3701251993726,9cyclictest23-21ksoftirqd/110:40:001
3701251993726,8cyclictest3717844-21sshd12:25:301
3701251993725,9cyclictest1-21systemd08:35:011
3701251993724,8cyclictest3714215-21cron11:20:011
370125899364,9cyclictest37-21ksoftirqd/310:21:343
370125899364,9cyclictest37-21ksoftirqd/310:21:343
370125899363,3cyclictest131rcu_preempt12:05:013
3701258993623,10cyclictest3715283-21tr11:40:003
3701256993626,7cyclictest3711086-21sendmail10:20:012
370125699361,5cyclictest0-21swapper/208:55:002
3701256993612,4cyclictest131rcu_preempt11:10:022
3701251993629,4cyclictest3714747-21cron11:29:591
3701251993627,7cyclictest23-21ksoftirqd/111:40:031
3701251993626,7cyclictest23-21ksoftirqd/112:15:011
3701251993626,7cyclictest23-21ksoftirqd/111:52:311
3701251993619,12cyclictest3705842-21cron08:40:001
3701251993618,14cyclictest235-21systemd-journal11:35:001
370125199361,3cyclictest131rcu_preempt11:15:001
370125199361,3cyclictest131rcu_preempt08:30:021
3701258993527,6cyclictest37-21ksoftirqd/307:30:013
3701258993525,7cyclictest37-21ksoftirqd/312:30:013
3701258993525,7cyclictest37-21ksoftirqd/309:50:023
3701258993524,8cyclictest37-21ksoftirqd/307:44:063
3701258993524,7cyclictest3712380-21/usr/sbin/munin10:45:013
3701258993522,10cyclictest3716951-21sed12:10:023
3701258993515,5cyclictest131rcu_preempt11:55:013
370125899351,3cyclictest131rcu_preempt08:05:013
370125699359,5cyclictest131rcu_preempt09:35:022
3701256993527,6cyclictest30-21ksoftirqd/210:35:002
370125199359,5cyclictest131rcu_preempt07:52:311
370125199355,6cyclictest131rcu_preempt08:10:011
370125199355,3cyclictest131rcu_preempt08:50:021
370125199354,14cyclictest0-21swapper/109:00:011
3701251993527,5cyclictest3701183-21latency_hist07:10:011
370125199352,4cyclictest131rcu_preempt11:05:021
3701251993522,9cyclictest23-21ksoftirqd/109:50:011
3701251993520,12cyclictest23-21ksoftirqd/112:05:001
3701251993519,12cyclictest272-21dbus-daemon10:45:001
370125899346,9cyclictest37-21ksoftirqd/308:30:023
370125899346,8cyclictest37-21ksoftirqd/312:31:193
370125899344,3cyclictest131rcu_preempt09:00:013
3701258993427,4cyclictest3702547-21latency_hist07:35:023
3701258993416,14cyclictest1-21systemd10:10:013
370125199343,6cyclictest131rcu_preempt10:54:581
370125199343,3cyclictest131rcu_preempt12:35:011
3701251993422,10cyclictest23-21ksoftirqd/111:10:011
370125199342,10cyclictest131rcu_preempt09:45:001
3701251993416,13cyclictest235-21systemd-journal10:15:001
370125899335,6cyclictest131rcu_preempt10:30:083
3701258993323,5cyclictest306-21in:imuxsock09:15:013
3701258993322,7cyclictest3704451-21date08:10:023
3701258993319,11cyclictest37-21ksoftirqd/311:05:013
370125899331,4cyclictest131rcu_preempt11:50:023
370125899331,3cyclictest131rcu_preempt11:20:003
370125199339,5cyclictest131rcu_preempt11:24:231
3701251993324,6cyclictest23-21ksoftirqd/111:50:011
3701251993318,11cyclictest235-21systemd-journal09:05:001
370125199331,5cyclictest3711076-21munin-run10:20:001
370125899322,6cyclictest131rcu_preempt09:04:593
3701258993224,4cyclictest3710298-21sh10:05:003
3701258993222,5cyclictest37-21ksoftirqd/308:50:013
3701258993214,11cyclictest0-21swapper/308:40:003
370125199322,7cyclictest0-21swapper/108:05:021
3701251993220,8cyclictest235-21systemd-journal10:50:001
370125899311,4cyclictest131rcu_preempt11:30:013
370125199311,4cyclictest131rcu_preempt09:35:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional