You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-01 - 07:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Oct 01, 2023 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2882435999086,2cyclictest0-21swapper/023:35:020
2882435996912,14cyclictest366-21snmpd00:11:110
288243599681,62cyclictest0-21swapper/023:59:400
2882435996810,3cyclictest131rcu_preempt21:55:020
2882435996763,2cyclictest0-21swapper/019:35:020
2882435996622,36cyclictest304-21in:imuxsock21:10:000
2882435996619,4cyclictest2883888-21sshd19:37:380
2882435996617,13cyclictest274-21dbus-daemon20:35:000
2882435996616,8cyclictest2886994-21cron20:40:000
2882435996524,37cyclictest273-21cron21:30:000
2882435996520,2cyclictest0-21swapper/022:35:010
288243599642,21cyclictest2897891-21cron00:10:000
288243599636,13cyclictest2892188-21modprobe22:20:010
288243599635,17cyclictest2884926-21cron20:00:000
288243599633,12cyclictest2899474-21munin-plugin-st00:40:000
288243599623,16cyclictest2887287-21cut20:45:000
288243599623,14cyclictest2896311-21cron23:39:590
288243599623,14cyclictest235-21systemd-journal20:05:000
288243599621,25cyclictest0-21swapper/020:10:240
288243599621,25cyclictest0-21swapper/020:10:240
288243599614,11cyclictest307-21rs:main0
2882435996121,34cyclictest18650irq/34-mmc123:20:030
2882442996027,9cyclictest30-21ksoftirqd/220:40:002
288243599604,11cyclictest2894301-21cat23:00:020
288243599604,11cyclictest2894301-21cat23:00:020
288243599603,14cyclictest2893219-21modprobe22:40:000
288243599603,14cyclictest2893219-21modprobe22:40:000
2882435996022,13cyclictest274-21dbus-daemon19:30:010
2882442995916,10cyclictest30-21ksoftirqd/222:40:012
2882442995916,10cyclictest30-21ksoftirqd/222:40:012
2882435995916,6cyclictest0-21swapper/000:30:000
2882442995813,5cyclictest131rcu_preempt21:55:002
2882435995815,9cyclictest235-21systemd-journal19:50:010
2882435995814,9cyclictest2890801-21sed21:50:020
2882435995721,6cyclictest2886251-21latency_hist20:25:020
288243599563,16cyclictest2885488-21latency_hist20:10:010
288243599562,13cyclictest2882506-21latency_hist19:10:020
2882435995620,10cyclictest274-21dbus-daemon19:15:010
2882444995535,14cyclictest1-21systemd19:25:013
288243599554,10cyclictest2899213-21cut00:35:000
2882435995523,28cyclictest2890123-21latency_hist21:40:020
2882435995523,28cyclictest2890123-21latency_hist21:40:020
288243599552,13cyclictest2897647-21modprobe00:05:000
2882435995520,4cyclictest2892449-21sshd22:22:390
2882435995520,4cyclictest2892449-21sshd22:22:390
288243599551,23cyclictest0-21swapper/021:15:010
2882444995421,10cyclictest37-21ksoftirqd/320:20:003
2882442995412,5cyclictest131rcu_preempt20:35:002
2882435995418,6cyclictest2898680-21cron00:25:000
2882435995413,9cyclictest304-21in:imuxsock20:20:000
2882442995310,6cyclictest131rcu_preempt20:15:012
2882442995310,6cyclictest131rcu_preempt20:15:012
288243599532,14cyclictest2894552-21cat23:05:010
2882435995319,5cyclictest2891153-21sendmail22:00:000
2882435995319,5cyclictest2891153-21sendmail22:00:000
2882435995318,10cyclictest304-21in:imuxsock22:10:000
2882435995311,8cyclictest0-21swapper/023:50:010
288244299526,5cyclictest131rcu_preempt00:10:002
288243899526,6cyclictest131rcu_preempt00:00:011
2882438995233,13cyclictest1-21systemd23:25:001
2882438995213,9cyclictest131rcu_preempt21:14:591
2882435995227,19cyclictest274-21dbus-daemon23:25:000
2882435995220,8cyclictest12-21ksoftirqd/022:55:020
2882435995216,11cyclictest274-21dbus-daemon22:30:010
288244499516,6cyclictest131rcu_preempt20:40:003
288244299519,5cyclictest131rcu_preempt20:54:592
288244299517,6cyclictest131rcu_preempt00:20:022
288244299517,6cyclictest131rcu_preempt00:20:022
2882442995121,5cyclictest30-21ksoftirqd/220:50:012
2882442995110,6cyclictest131rcu_preempt00:00:002
2882435995123,24cyclictest12-21ksoftirqd/020:30:030
2882435995117,9cyclictest12-21ksoftirqd/021:40:430
2882435995111,13cyclictest12-21ksoftirqd/019:45:020
288244499504,7cyclictest37-21ksoftirqd/319:40:003
2882444995030,15cyclictest1-21systemd20:10:013
2882444995029,15cyclictest235-21systemd-journal00:25:003
288244499502,5cyclictest2897363-21/usr/sbin/munin00:00:003
2882442995035,10cyclictest1-21systemd19:55:012
2882442995035,10cyclictest1-21systemd19:55:012
2882442995010,6cyclictest131rcu_preempt20:10:002
288243899502,5cyclictest131rcu_preempt21:10:001
288243599503,19cyclictest2898441-21sendmail-msp00:20:000
288243599503,19cyclictest2898441-21sendmail-msp00:20:000
288244499499,5cyclictest131rcu_preempt19:45:003
2882444994929,15cyclictest1-21systemd23:55:013
2882444994929,15cyclictest1-21systemd23:55:013
288244299495,6cyclictest131rcu_preempt23:25:002
2882442994912,5cyclictest131rcu_preempt23:40:002
2882442994910,7cyclictest131rcu_preempt20:45:012
2882444994835,8cyclictest2885207-21cron20:05:003
288244299488,5cyclictest131rcu_preempt23:00:012
288244299488,5cyclictest131rcu_preempt23:00:012
288244299483,7cyclictest30-21ksoftirqd/221:36:582
288244299483,7cyclictest30-21ksoftirqd/221:36:582
2882442994810,6cyclictest131rcu_preempt22:25:002
2882442994810,6cyclictest131rcu_preempt22:25:002
288243899481,4cyclictest131rcu_preempt23:20:021
2882435994819,4cyclictest2891615-21latency_hist22:05:020
288244299478,7cyclictest30-21ksoftirqd/219:50:002
288244299477,6cyclictest131rcu_preempt21:09:592
288244299476,6cyclictest30-21ksoftirqd/220:20:012
288244299472,5cyclictest131rcu_preempt21:00:002
2882442994714,7cyclictest30-21ksoftirqd/222:33:392
2882442994713,5cyclictest131rcu_preempt20:00:002
2882438994732,11cyclictest1-21systemd00:15:001
288243899471,5cyclictest131rcu_preempt20:15:011
288243899471,5cyclictest131rcu_preempt20:15:011
2882438994712,6cyclictest131rcu_preempt20:15:381
288243599476,9cyclictest0-21swapper/020:55:010
2882435994715,8cyclictest12-21ksoftirqd/022:15:010
288244499463,6cyclictest131rcu_preempt21:10:003
288244299468,5cyclictest131rcu_preempt23:05:002
288244299467,4cyclictest131rcu_preempt23:33:402
288244299466,6cyclictest131rcu_preempt21:15:012
288244299466,11cyclictest30-21ksoftirqd/220:30:022
288244299464,6cyclictest131rcu_preempt19:20:012
2882442994611,6cyclictest131rcu_preempt22:20:032
2882442994611,6cyclictest131rcu_preempt21:16:572
288243899468,6cyclictest131rcu_preempt20:00:021
2882438994632,11cyclictest1-21systemd22:25:001
2882438994632,11cyclictest1-21systemd22:25:001
2882438994632,10cyclictest274-21dbus-daemon21:40:011
2882438994632,10cyclictest274-21dbus-daemon21:40:011
2882438994628,13cyclictest274-21dbus-daemon19:55:011
2882438994628,13cyclictest274-21dbus-daemon19:55:011
288243899462,5cyclictest131rcu_preempt22:00:021
288243899462,5cyclictest131rcu_preempt22:00:021
2882438994615,6cyclictest131rcu_preempt23:45:001
2882435994633,8cyclictest2895961-21wc23:30:020
288244499451,5cyclictest131rcu_preempt23:30:023
288243899456,5cyclictest131rcu_preempt20:45:001
2882438994530,11cyclictest274-21dbus-daemon20:05:011
2882435994528,10cyclictest2888025-21cron21:00:000
288244499448,6cyclictest131rcu_preempt19:30:013
288244499445,5cyclictest131rcu_preempt23:00:013
288244499445,5cyclictest131rcu_preempt23:00:013
288244499445,32cyclictest2896059-21cron23:35:003
288244499444,6cyclictest131rcu_preempt00:40:003
2882444994427,12cyclictest274-21dbus-daemon00:35:013
288244499442,5cyclictest131rcu_preempt23:50:023
288244299447,5cyclictest131rcu_preempt19:40:002
288244299447,4cyclictest131rcu_preempt20:23:392
288244299446,6cyclictest131rcu_preempt21:24:592
288244299445,5cyclictest131rcu_preempt00:05:012
288244299445,10cyclictest30-21ksoftirqd/221:30:012
288244299444,7cyclictest30-21ksoftirqd/222:51:392
2882442994428,12cyclictest235-21systemd-journal22:45:022
288243899446,6cyclictest131rcu_preempt23:40:001
2882438994432,8cyclictest2891404-21sshd22:02:391
2882438994423,10cyclictest23-21ksoftirqd/123:04:231
2882438994414,9cyclictest131rcu_preempt20:50:021
2882438994410,4cyclictest131rcu_preempt19:47:381
2882435994412,28cyclictest0-21swapper/023:15:010
2882444994330,10cyclictest235-21systemd-journal23:25:013
2882444994329,9cyclictest235-21systemd-journal22:25:003
2882444994329,9cyclictest235-21systemd-journal22:25:003
2882444994328,11cyclictest274-21dbus-daemon21:20:003
2882444994315,18cyclictest37-21ksoftirqd/319:10:463
288244499430,6cyclictest131rcu_preempt21:55:003
288244299438,6cyclictest131rcu_preempt22:00:002
288244299438,6cyclictest131rcu_preempt22:00:002
288244299438,5cyclictest131rcu_preempt23:55:002
288244299438,5cyclictest131rcu_preempt23:55:002
288244299433,5cyclictest131rcu_preempt21:05:022
288244299432,5cyclictest131rcu_preempt19:41:482
2882442994310,7cyclictest30-21ksoftirqd/200:15:002
2882442994310,5cyclictest131rcu_preempt21:50:012
2882442994310,4cyclictest131rcu_preempt22:00:392
288243899438,6cyclictest131rcu_preempt00:22:401
2882438994313,5cyclictest131rcu_preempt20:35:001
2882435994327,12cyclictest12-21ksoftirqd/021:19:590
288244499425,5cyclictest131rcu_preempt23:40:003
2882444994231,8cyclictest2898928-21sshd00:27:503
2882444994230,9cyclictest2887769-21sshd20:54:393
288244299427,7cyclictest30-21ksoftirqd/200:35:012
288244299426,5cyclictest131rcu_preempt22:11:392
288244299426,5cyclictest131rcu_preempt00:25:022
288244299425,9cyclictest30-21ksoftirqd/200:37:402
288244299425,31cyclictest2891680-21grep22:10:002
288244299424,7cyclictest30-21ksoftirqd/223:20:012
288244299422,5cyclictest131rcu_preempt23:30:022
288243899426,6cyclictest131rcu_preempt21:05:011
288243899425,5cyclictest131rcu_preempt19:15:001
288243899423,5cyclictest131rcu_preempt00:40:021
2882438994218,20cyclictest221rcuc/123:00:021
2882438994218,20cyclictest221rcuc/123:00:021
2882438994217,19cyclictest131rcu_preempt19:25:021
288243899421,5cyclictest131rcu_preempt00:10:001
288243599421,4cyclictest0-21swapper/022:45:490
288243599421,23cyclictest0-21swapper/019:51:120
288243599421,23cyclictest0-21swapper/019:51:120
288244499413,5cyclictest131rcu_preempt00:15:003
2882444994115,7cyclictest131rcu_preempt21:45:003
2882444994115,6cyclictest131rcu_preempt21:04:593
2882444994111,6cyclictest131rcu_preempt20:30:023
288244299419,5cyclictest131rcu_preempt21:45:002
288244299416,7cyclictest30-21ksoftirqd/223:10:012
2882442994115,6cyclictest131rcu_preempt23:41:402
2882442994111,6cyclictest131rcu_preempt21:31:572
288243899419,5cyclictest131rcu_preempt19:35:011
288243899417,5cyclictest131rcu_preempt19:40:021
2882438994132,5cyclictest23-21ksoftirqd/123:50:011
288243899411,4cyclictest131rcu_preempt20:30:031
2882438994111,6cyclictest131rcu_preempt21:45:011
2882435994126,11cyclictest366-21snmpd21:30:400
288243599411,22cyclictest0-21swapper/023:07:500
288244499408,6cyclictest131rcu_preempt22:40:003
288244499408,6cyclictest131rcu_preempt22:40:003
2882444994031,5cyclictest2888024-21cron21:00:003
2882444994028,9cyclictest2898030-21cat00:10:013
2882444994028,9cyclictest1-21systemd21:30:003
2882444994028,10cyclictest2893984-21sshd22:53:403
2882444994026,10cyclictest37-21ksoftirqd/320:00:003
288244499401,5cyclictest131rcu_preempt22:00:003
288244499401,5cyclictest131rcu_preempt22:00:003
288244499401,5cyclictest131rcu_preempt20:45:013
2882444994014,17cyclictest37-21ksoftirqd/320:45:363
2882444994011,4cyclictest131rcu_preempt19:50:013
288244299408,5cyclictest131rcu_preempt22:42:082
288244299408,4cyclictest131rcu_preempt19:25:012
288244299406,8cyclictest30-21ksoftirqd/220:01:502
288244299406,5cyclictest131rcu_preempt19:32:382
288244299402,10cyclictest30-21ksoftirqd/223:15:002
288243899408,5cyclictest131rcu_preempt21:18:391
288243899404,5cyclictest131rcu_preempt22:50:001
2882438994030,7cyclictest2889587-21sshd21:26:381
288243899401,7cyclictest1-21systemd20:23:391
288243899401,6cyclictest23-21ksoftirqd/123:32:391
2882438994012,5cyclictest131rcu_preempt21:50:391
2882438994010,6cyclictest131rcu_preempt22:20:001
288243599404,3cyclictest131rcu_preempt23:55:000
288243599404,3cyclictest131rcu_preempt23:55:000
288244499399,6cyclictest131rcu_preempt19:50:393
288244499399,6cyclictest131rcu_preempt19:50:393
288244499399,4cyclictest131rcu_preempt20:12:393
288244499399,4cyclictest131rcu_preempt20:12:393
288244499395,8cyclictest37-21ksoftirqd/320:30:393
288244499394,6cyclictest131rcu_preempt23:20:023
2882444993928,8cyclictest37-21ksoftirqd/321:20:403
2882444993927,9cyclictest274-21dbus-daemon21:50:013
2882444993926,9cyclictest1-21systemd21:35:013
288244499391,4cyclictest131rcu_preempt23:45:003
288244299398,4cyclictest131rcu_preempt19:15:002
288244299397,5cyclictest131rcu_preempt22:27:392
288243899399,5cyclictest131rcu_preempt22:51:391
288243899394,4cyclictest131rcu_preempt23:52:401
288243899394,4cyclictest131rcu_preempt23:52:401
288243899392,5cyclictest131rcu_preempt20:40:001
288243899391,7cyclictest131rcu_preempt22:10:021
2882438993915,6cyclictest131rcu_preempt22:32:071
288243599392,33cyclictest2883146-21sshd19:24:380
2882435993923,11cyclictest12-21ksoftirqd/021:05:000
288243599392,16cyclictest2896602-21/usr/sbin/munin23:45:010
2882435993920,13cyclictest235-21systemd-journal20:45:030
288244499388,4cyclictest131rcu_preempt23:05:003
288244499386,4cyclictest131rcu_preempt00:03:403
288244499384,6cyclictest131rcu_preempt21:40:023
288244499384,6cyclictest131rcu_preempt21:40:023
2882444993828,8cyclictest37-21ksoftirqd/322:34:373
2882444993828,7cyclictest2882884-21sendmail19:20:013
2882444993825,10cyclictest2893695-21sed22:45:023
2882444993824,10cyclictest37-21ksoftirqd/321:15:013
2882444993821,11cyclictest274-21dbus-daemon23:10:013
288244499381,5cyclictest131rcu_preempt22:15:013
288244299387,5cyclictest131rcu_preempt00:27:402
288243899382,5cyclictest131rcu_preempt20:55:011
288243899382,5cyclictest131rcu_preempt19:30:011
2882438993811,19cyclictest131rcu_preempt22:13:391
2882435993825,10cyclictest366-21snmpd21:20:390
288243599381,2cyclictest0-21swapper/022:45:010
288244499371,5cyclictest131rcu_preempt22:50:003
288244499371,4cyclictest131rcu_preempt00:20:033
288244499371,4cyclictest131rcu_preempt00:20:023
288244299374,9cyclictest30-21ksoftirqd/219:28:382
2882442993725,9cyclictest30-21ksoftirqd/223:50:012
288243899378,5cyclictest131rcu_preempt23:30:011
288243899378,5cyclictest131rcu_preempt23:10:001
288243899373,5cyclictest131rcu_preempt21:25:001
2882438993727,7cyclictest2890632-21sshd21:49:391
2882438993727,6cyclictest304-21in:imuxsock21:35:011
2882438993726,8cyclictest23-21ksoftirqd/119:45:001
2882438993724,10cyclictest2895242-21cut23:15:031
2882438993722,11cyclictest2893238-21tr22:40:001
2882438993722,11cyclictest2893238-21tr22:40:001
288243899371,6cyclictest131rcu_preempt00:20:001
288243899371,6cyclictest131rcu_preempt00:20:001
2882438993714,17cyclictest131rcu_preempt22:42:091
288244499367,4cyclictest131rcu_preempt20:21:513
2882444993626,8cyclictest37-21ksoftirqd/319:35:033
2882444993624,10cyclictest37-21ksoftirqd/322:20:013
2882444993622,10cyclictest274-21dbus-daemon22:10:013
288244299365,4cyclictest131rcu_preempt19:10:012
288243899367,5cyclictest131rcu_preempt00:02:401
2882438993622,10cyclictest23-21ksoftirqd/100:30:001
2882438993613,7cyclictest131rcu_preempt19:20:011
288243899357,5cyclictest131rcu_preempt20:56:381
2882438993525,7cyclictest2882455-21cut19:10:021
2882444993415,15cyclictest131rcu_preempt22:00:473
2882444993414,16cyclictest131rcu_preempt23:13:393
2882444993410,6cyclictest131rcu_preempt22:25:033
288243899334,5cyclictest131rcu_preempt20:07:411
2882438993313,6cyclictest131rcu_preempt00:35:001
288244499321,4cyclictest131rcu_preempt19:10:023
2882438993223,6cyclictest235-21systemd-journal22:25:391
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional