You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-17 - 20:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Thu Jul 17, 2025 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
526341997833,11cyclictest235-21systemd-journal07:13:371
526341997335,5cyclictest530943-21sendmail08:40:001
526336997311,48cyclictest12-21ksoftirqd/008:00:020
526341996946,16cyclictest235-21systemd-journal11:19:591
52633699662,16cyclictest12-21ksoftirqd/008:35:000
52633699637,7cyclictest12-21ksoftirqd/007:55:000
52633699633,9cyclictest12-21ksoftirqd/009:00:010
52633699628,8cyclictest12-21ksoftirqd/010:15:010
526348996153,6cyclictest361rcuc/312:30:033
526345996019,8cyclictest30-21ksoftirqd/210:50:022
526345996011,11cyclictest30-21ksoftirqd/208:00:022
52633699603,9cyclictest12-21ksoftirqd/012:20:000
52633699596,14cyclictest12-21ksoftirqd/011:40:000
52633699595,7cyclictest12-21ksoftirqd/010:40:010
52634599576,7cyclictest30-21ksoftirqd/208:10:002
52634599574,8cyclictest30-21ksoftirqd/211:20:002
526345995747,6cyclictest30-21ksoftirqd/208:20:012
526345995717,13cyclictest30-21ksoftirqd/207:27:382
526345995715,7cyclictest30-21ksoftirqd/210:45:012
52633699575,11cyclictest12-21ksoftirqd/011:35:000
526345995615,10cyclictest30-21ksoftirqd/209:00:022
52633699569,5cyclictest131rcu_preempt11:10:010
52633699564,7cyclictest12-21ksoftirqd/008:40:000
52634899558,6cyclictest131rcu_preempt07:30:013
52634899557,31cyclictest363-21snmpd10:25:023
52634599554,9cyclictest30-21ksoftirqd/207:50:012
52634599554,9cyclictest30-21ksoftirqd/207:50:002
526345995539,12cyclictest1-21systemd07:40:012
52634599552,14cyclictest30-21ksoftirqd/208:40:012
52633699558,12cyclictest12-21ksoftirqd/008:50:000
526336995511,9cyclictest12-21ksoftirqd/009:50:020
526336995511,9cyclictest12-21ksoftirqd/009:50:020
52634899541,5cyclictest131rcu_preempt11:00:003
52634599544,12cyclictest30-21ksoftirqd/211:00:012
52634599542,9cyclictest30-21ksoftirqd/210:20:012
52633699546,12cyclictest12-21ksoftirqd/010:10:020
52633699542,11cyclictest12-21ksoftirqd/012:10:000
526336995410,13cyclictest12-21ksoftirqd/008:55:030
52634599537,11cyclictest30-21ksoftirqd/208:30:012
52634599536,13cyclictest30-21ksoftirqd/207:15:002
52634599535,7cyclictest30-21ksoftirqd/212:30:022
52633699538,9cyclictest12-21ksoftirqd/010:35:020
52633699536,10cyclictest12-21ksoftirqd/009:59:590
52634599529,13cyclictest30-21ksoftirqd/208:55:022
52634599526,7cyclictest30-21ksoftirqd/209:45:012
52634599523,11cyclictest30-21ksoftirqd/207:35:012
526345995211,9cyclictest30-21ksoftirqd/210:15:002
526345995211,10cyclictest30-21ksoftirqd/209:25:022
526345995211,10cyclictest30-21ksoftirqd/209:25:012
52633699527,7cyclictest12-21ksoftirqd/011:05:010
52633699527,6cyclictest131rcu_preempt11:00:020
52633699527,5cyclictest131rcu_preempt12:00:030
52633699524,7cyclictest12-21ksoftirqd/009:45:020
526336995242,6cyclictest12-21ksoftirqd/008:10:000
52633699524,11cyclictest12-21ksoftirqd/012:25:020
52633699524,10cyclictest12-21ksoftirqd/009:04:390
526336995236,13cyclictest235-21systemd-journal10:50:010
52633699522,8cyclictest12-21ksoftirqd/010:30:020
526336995215,32cyclictest16850irq/134-800000008:25:380
526336995210,5cyclictest131rcu_preempt11:20:010
52634899518,5cyclictest131rcu_preempt10:50:023
52634599517,9cyclictest30-21ksoftirqd/211:25:002
52634599516,8cyclictest30-21ksoftirqd/209:35:032
526345995134,13cyclictest1-21systemd11:40:012
526345995134,12cyclictest534589-21cron09:50:012
526345995134,12cyclictest534589-21cron09:50:012
52634599513,11cyclictest30-21ksoftirqd/208:05:022
52634599513,11cyclictest30-21ksoftirqd/208:05:012
52634599512,13cyclictest30-21ksoftirqd/208:25:002
52633699518,7cyclictest12-21ksoftirqd/009:55:000
52633699517,9cyclictest12-21ksoftirqd/009:15:000
52633699515,5cyclictest131rcu_preempt07:40:010
52633699514,10cyclictest12-21ksoftirqd/008:17:010
526336995111,5cyclictest131rcu_preempt11:45:020
526336995111,5cyclictest131rcu_preempt11:45:010
526336995111,5cyclictest131rcu_preempt07:35:010
526336995110,5cyclictest131rcu_preempt09:10:020
526348995024,7cyclictest37-21ksoftirqd/310:30:013
52634599507,9cyclictest30-21ksoftirqd/210:00:022
52634599505,7cyclictest30-21ksoftirqd/210:09:592
52634599505,5cyclictest30-21ksoftirqd/211:05:002
52634599501,8cyclictest30-21ksoftirqd/208:35:012
52634599501,5cyclictest0-21swapper/212:10:002
52633699506,7cyclictest12-21ksoftirqd/010:55:010
526336995012,6cyclictest131rcu_preempt11:50:010
52633699501,10cyclictest12-21ksoftirqd/010:44:590
526348994911,6cyclictest131rcu_preempt12:15:013
526348994911,6cyclictest131rcu_preempt12:15:013
52634599498,7cyclictest30-21ksoftirqd/209:40:002
52634599496,5cyclictest30-21ksoftirqd/207:10:022
52634599496,5cyclictest30-21ksoftirqd/207:10:012
52634599495,8cyclictest30-21ksoftirqd/212:25:012
52634599493,11cyclictest30-21ksoftirqd/209:05:002
52634599493,10cyclictest30-21ksoftirqd/208:50:012
52633699498,6cyclictest12-21ksoftirqd/012:35:020
52633699493,8cyclictest12-21ksoftirqd/011:55:010
526336994914,5cyclictest131rcu_preempt07:20:010
52634899488,6cyclictest131rcu_preempt10:05:023
52634899488,6cyclictest131rcu_preempt10:05:013
52634899488,5cyclictest131rcu_preempt09:25:013
52634899488,5cyclictest131rcu_preempt09:25:003
526348994830,14cyclictest235-21systemd-journal12:25:013
526348994830,14cyclictest235-21systemd-journal12:00:013
52634899482,6cyclictest131rcu_preempt08:30:013
526348994820,18cyclictest37-21ksoftirqd/309:10:033
52634899481,5cyclictest131rcu_preempt11:50:013
526348994815,5cyclictest131rcu_preempt07:40:013
52634599486,7cyclictest30-21ksoftirqd/210:25:022
52634599486,6cyclictest30-21ksoftirqd/212:20:022
52634599483,10cyclictest30-21ksoftirqd/212:15:002
52634599483,10cyclictest30-21ksoftirqd/212:15:002
52634599483,10cyclictest30-21ksoftirqd/211:45:032
52634599483,10cyclictest30-21ksoftirqd/211:45:032
52634599482,8cyclictest30-21ksoftirqd/209:30:022
52634599482,12cyclictest30-21ksoftirqd/211:30:032
52634599482,12cyclictest30-21ksoftirqd/211:30:032
52634599481,11cyclictest541371-21modprobe12:00:012
526341994837,6cyclictest530436-21munin-plugin-st08:30:011
52633699487,8cyclictest12-21ksoftirqd/009:25:010
52633699487,8cyclictest12-21ksoftirqd/009:25:000
52633699487,5cyclictest131rcu_preempt07:25:030
52633699487,11cyclictest12-21ksoftirqd/008:25:010
52633699486,9cyclictest12-21ksoftirqd/009:20:020
52633699483,7cyclictest12-21ksoftirqd/007:50:020
52633699483,7cyclictest12-21ksoftirqd/007:50:010
52633699482,42cyclictest235-21systemd-journal08:15:000
526336994818,6cyclictest12-21ksoftirqd/012:15:020
526336994818,6cyclictest12-21ksoftirqd/012:15:020
526336994813,6cyclictest131rcu_preempt08:45:010
526336994813,6cyclictest131rcu_preempt08:45:010
526336994810,5cyclictest131rcu_preempt10:05:030
526336994810,5cyclictest131rcu_preempt10:05:030
52634899477,9cyclictest37-21ksoftirqd/308:15:013
52634899471,9cyclictest0-21swapper/309:20:023
526348994711,5cyclictest131rcu_preempt11:10:003
52634599477,9cyclictest30-21ksoftirqd/211:10:012
52634599477,7cyclictest30-21ksoftirqd/210:40:012
52634599473,9cyclictest30-21ksoftirqd/207:25:032
52634599473,6cyclictest30-21ksoftirqd/207:20:012
52634599473,12cyclictest30-21ksoftirqd/212:35:022
526341994733,7cyclictest306-21in:imuxsock07:20:011
52633699477,10cyclictest12-21ksoftirqd/007:44:590
52633699475,6cyclictest131rcu_preempt12:30:030
526336994711,5cyclictest131rcu_preempt09:40:000
526336994711,5cyclictest131rcu_preempt08:05:020
526336994711,5cyclictest131rcu_preempt08:05:010
52634899465,9cyclictest37-21ksoftirqd/309:40:003
52634899464,11cyclictest0-21swapper/309:00:003
526348994636,7cyclictest37-21ksoftirqd/310:20:003
526348994633,9cyclictest235-21systemd-journal07:45:003
52634899461,8cyclictest534411-21latency_hist09:45:023
526348994617,6cyclictest131rcu_preempt07:55:003
52634599467,8cyclictest30-21ksoftirqd/211:55:022
52634599467,8cyclictest30-21ksoftirqd/208:15:012
52634599467,6cyclictest30-21ksoftirqd/211:15:012
52634599465,7cyclictest30-21ksoftirqd/209:10:032
52634599465,6cyclictest131rcu_preempt08:45:032
52634599465,6cyclictest131rcu_preempt08:45:032
526345994630,12cyclictest235-21systemd-journal09:10:492
526345994614,10cyclictest30-21ksoftirqd/210:30:012
52634199464,8cyclictest131rcu_preempt09:19:591
52633699467,5cyclictest131rcu_preempt11:25:000
52633699466,7cyclictest12-21ksoftirqd/007:30:000
52633699461,6cyclictest0-21swapper/009:35:010
52633699461,6cyclictest0-21swapper/009:35:010
52634899457,5cyclictest131rcu_preempt10:44:593
52634899456,6cyclictest131rcu_preempt10:12:393
52634899453,10cyclictest37-21ksoftirqd/312:19:593
52634899451,5cyclictest131rcu_preempt07:15:013
526348994512,11cyclictest37-21ksoftirqd/310:10:013
52634599454,8cyclictest30-21ksoftirqd/212:40:002
52634599452,6cyclictest30-21ksoftirqd/209:55:012
526341994532,9cyclictest1-21systemd07:50:011
526341994532,9cyclictest1-21systemd07:50:011
526341994531,8cyclictest306-21in:imuxsock12:30:011
526341994530,11cyclictest272-21dbus-daemon10:10:001
526341994525,14cyclictest272-21dbus-daemon11:10:011
52633699458,5cyclictest131rcu_preempt07:15:010
52633699454,7cyclictest12-21ksoftirqd/011:30:020
52633699454,7cyclictest12-21ksoftirqd/011:30:020
52633699451,15cyclictest543386-21sshd12:35:380
52633699451,12cyclictest12-21ksoftirqd/012:00:000
526336994510,5cyclictest131rcu_preempt10:20:010
52634899448,7cyclictest131rcu_preempt11:25:003
52634899446,6cyclictest131rcu_preempt11:40:003
526348994416,18cyclictest131rcu_preempt12:05:003
526348994415,6cyclictest131rcu_preempt09:05:023
52634599444,8cyclictest30-21ksoftirqd/210:55:002
52634599444,7cyclictest30-21ksoftirqd/209:35:022
52634599444,11cyclictest30-21ksoftirqd/211:35:012
52634199444,4cyclictest131rcu_preempt09:15:011
52634199442,3cyclictest131rcu_preempt12:40:021
52634199442,3cyclictest131rcu_preempt10:30:021
52633699445,10cyclictest12-21ksoftirqd/010:25:010
52634899437,6cyclictest131rcu_preempt10:55:013
526348994332,6cyclictest535151-21unin-run10:00:003
526348994330,10cyclictest526750-21sshd07:15:383
52634899432,6cyclictest131rcu_preempt08:25:013
52634899431,7cyclictest539009-21munin-run11:15:003
526348994314,7cyclictest37-21ksoftirqd/308:39:593
52634899431,20cyclictest0-21swapper/308:09:593
52634599437,6cyclictest30-21ksoftirqd/210:35:022
526345994333,7cyclictest30-21ksoftirqd/207:40:032
52634599431,9cyclictest30-21ksoftirqd/211:50:002
526345994311,7cyclictest131rcu_preempt10:05:022
526345994311,7cyclictest131rcu_preempt10:05:022
526345994310,4cyclictest131rcu_preempt07:55:002
526341994332,6cyclictest306-21in:imuxsock12:35:011
526341994329,9cyclictest531458-21cron08:50:001
526341994326,12cyclictest235-21systemd-journal08:52:381
52634199432,5cyclictest131rcu_preempt09:00:011
52634199431,3cyclictest131rcu_preempt08:00:011
52633699436,5cyclictest131rcu_preempt11:15:020
52634899425,6cyclictest131rcu_preempt11:55:023
526348994228,11cyclictest529894-21cron08:20:013
52634899422,5cyclictest131rcu_preempt12:35:023
52634899422,5cyclictest131rcu_preempt11:30:023
52634899422,5cyclictest131rcu_preempt11:30:023
526348994225,9cyclictest37-21ksoftirqd/309:50:023
526348994225,9cyclictest37-21ksoftirqd/309:50:023
52634899421,6cyclictest131rcu_preempt12:40:003
526348994215,6cyclictest131rcu_preempt08:45:013
526348994215,6cyclictest131rcu_preempt08:45:003
52634599421,7cyclictest533071-21systemd-run09:20:012
52634199425,22cyclictest542382-21modprobe12:20:001
526341994232,7cyclictest534568-21sshd09:45:381
526341994232,7cyclictest534568-21sshd09:45:381
526341994225,12cyclictest272-21dbus-daemon10:25:001
526341994219,14cyclictest537182-21modprobe10:39:591
52634199421,4cyclictest131rcu_preempt08:20:021
526341994212,6cyclictest131rcu_preempt08:35:011
52633699424,6cyclictest12-21ksoftirqd/009:30:010
52634899415,8cyclictest37-21ksoftirqd/310:40:013
526348994134,5cyclictest361rcuc/308:00:023
52634899412,5cyclictest131rcu_preempt10:35:023
52634899412,5cyclictest131rcu_preempt07:35:023
52634899411,6cyclictest131rcu_preempt11:05:013
526348994112,5cyclictest131rcu_preempt11:34:393
526345994110,5cyclictest131rcu_preempt12:05:002
52634199418,7cyclictest131rcu_preempt09:05:011
526341994119,15cyclictest272-21dbus-daemon09:25:011
526341994119,15cyclictest272-21dbus-daemon09:25:011
52634199411,5cyclictest131rcu_preempt12:00:011
52634899406,7cyclictest131rcu_preempt09:30:013
52634899403,8cyclictest0-21swapper/307:47:383
52634899403,8cyclictest0-21swapper/307:47:373
52634899403,7cyclictest37-21ksoftirqd/309:55:023
52634899402,5cyclictest131rcu_preempt09:35:013
52634899402,5cyclictest131rcu_preempt09:35:013
526348994012,18cyclictest37-21ksoftirqd/307:22:573
52634899401,17cyclictest0-21swapper/308:55:013
526341994030,6cyclictest306-21in:imuxsock09:30:001
526341994029,8cyclictest539968-21sed11:30:021
526341994029,8cyclictest539968-21sed11:30:021
52634199402,3cyclictest131rcu_preempt11:05:011
52634199402,3cyclictest131rcu_preempt09:45:021
526341994011,5cyclictest131rcu_preempt09:40:001
52634899392,6cyclictest131rcu_preempt08:35:003
52634899391,4cyclictest0-21swapper/311:15:383
52634199396,6cyclictest541089-21grep11:55:001
52634199393,5cyclictest131rcu_preempt10:00:021
526341993922,13cyclictest235-21systemd-journal09:32:381
526341993922,13cyclictest235-21systemd-journal09:32:371
52634199391,6cyclictest131rcu_preempt08:10:001
52634199391,3cyclictest131rcu_preempt12:10:021
52633699394,5cyclictest131rcu_preempt07:10:020
52633699394,5cyclictest131rcu_preempt07:10:010
52634899389,9cyclictest131rcu_preempt08:46:383
52634899383,8cyclictest0-21swapper/311:45:023
52634899383,8cyclictest0-21swapper/311:45:023
526341993829,5cyclictest531215-21sh08:45:011
526341993829,5cyclictest531215-21sh08:45:011
526341993828,7cyclictest527289-21sshd07:25:381
52634199382,6cyclictest536160-21munin-plugin-st10:20:011
526341993821,10cyclictest272-21dbus-daemon11:00:011
52634199381,3cyclictest131rcu_preempt09:55:011
52634899379,6cyclictest131rcu_preempt12:10:013
52634899372,3cyclictest131rcu_preempt07:10:033
52634899372,3cyclictest131rcu_preempt07:10:023
526348993720,9cyclictest37-21ksoftirqd/309:15:013
52634199372,6cyclictest131rcu_preempt07:10:031
52634199372,6cyclictest131rcu_preempt07:10:031
526341993723,10cyclictest542118-21cron12:15:001
526341993723,10cyclictest542118-21cron12:15:001
526341993722,11cyclictest23-21ksoftirqd/110:50:011
526341993722,10cyclictest235-21systemd-journal12:25:011
526348993611,6cyclictest131rcu_preempt08:05:023
526348993611,6cyclictest131rcu_preempt08:05:013
526341993628,5cyclictest527829-21sendmail07:40:001
526341993624,9cyclictest23-21ksoftirqd/111:45:011
526341993624,9cyclictest23-21ksoftirqd/111:45:011
526341993624,8cyclictest306-21in:imuxsock07:31:391
52634199355,4cyclictest131rcu_preempt08:15:021
52634199354,7cyclictest0-21swapper/111:15:021
52634199353,4cyclictest131rcu_preempt11:40:001
526341993525,8cyclictest23-21ksoftirqd/111:25:001
526341993525,6cyclictest528603-21sh07:55:001
52634199344,3cyclictest131rcu_preempt07:25:001
52634199343,4cyclictest131rcu_preempt08:25:021
52634199342,5cyclictest131rcu_preempt08:05:011
52634199342,5cyclictest131rcu_preempt08:05:011
526341993425,5cyclictest306-21in:imuxsock07:44:391
526341993424,7cyclictest537946-21sshd10:52:381
526341993420,10cyclictest272-21dbus-daemon11:50:001
52634199341,5cyclictest131rcu_preempt09:10:011
52634199334,23cyclictest537116-21grep10:35:031
52634199323,3cyclictest131rcu_preempt10:05:011
52634199323,3cyclictest131rcu_preempt10:05:011
526341993223,5cyclictest540041-21cron11:35:001
526341993220,9cyclictest537453-21sshd10:44:381
52634199315,3cyclictest131rcu_preempt10:15:001
52634199311,3cyclictest131rcu_preempt12:05:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional