You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 07:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Wed Apr 24, 2024 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
341505997934,40cyclictest235-21systemd-journal22:12:542
34149799782,5cyclictest348729-21munin-run21:30:020
341505997518,51cyclictest235-21systemd-journal23:35:002
341497997529,42cyclictest1-21systemd20:35:010
34149799722,5cyclictest355876-21latency_hist23:45:020
341505997120,9cyclictest304-21in:imuxsock22:10:012
341497997016,44cyclictest16750irq/134-800000020:59:490
34150599697,51cyclictest347140-21munin-run21:00:002
34150799682,17cyclictest344522-21sshd20:07:483
34150799682,17cyclictest344522-21sshd20:07:483
341505996826,37cyclictest304-21in:imuxsock20:45:482
341505996826,37cyclictest304-21in:imuxsock20:45:482
341505996824,39cyclictest304-21in:imuxsock21:02:482
341505996821,42cyclictest351723-21grep22:25:012
341497996819,45cyclictest349739-21cron21:50:000
341505996728,35cyclictest356216-21sshd23:53:132
34150599665,15cyclictest355442-21cron23:40:012
34150599665,15cyclictest355442-21cron23:40:002
34149799661,4cyclictest356541-21munin-run00:00:010
34149799661,4cyclictest356541-21munin-run00:00:010
341497996612,27cyclictest12-21ksoftirqd/000:15:020
34150599655,17cyclictest347849-21cat21:10:022
34150599654,18cyclictest345046-21cron20:20:002
34149799653,18cyclictest355203-21munin-run23:35:000
34149799652,5cyclictest350802-21munin-run22:10:010
34150599645,18cyclictest353642-21modprobe23:05:002
34150599645,15cyclictest343186-21latency_hist19:40:022
34150599644,15cyclictest344024-21munin-run20:00:002
34150599644,14cyclictest354946-21sh23:30:002
34149799642,10cyclictest346310-21cat20:40:030
34150799633,16cyclictest344024-21munin-run20:00:003
34150799632,54cyclictest347144-21cut21:00:003
341507996321,22cyclictest37-21ksoftirqd/300:35:013
341507996311,48cyclictest350765-21sshd22:07:533
34150599636,13cyclictest307-21rs:main2
34150599635,20cyclictest235-21systemd-journal00:00:002
34150599635,20cyclictest235-21systemd-journal00:00:002
34150599633,17cyclictest354835-21cat23:25:022
341501996318,11cyclictest23-21ksoftirqd/121:55:001
34150599626,14cyclictest353383-21modprobe23:00:002
34150599625,17cyclictest353127-21cut22:55:002
34150599624,12cyclictest308-21systemd-logind19:25:012
34150599623,23cyclictest349677-21grep21:45:022
34149799622,5cyclictest343020-21munin-run19:40:010
34150599616,13cyclictest343782-21cat19:55:022
34150599616,13cyclictest343782-21cat19:55:022
34150599615,18cyclictest350307-21wc22:00:012
34150599615,13cyclictest345340-21grep20:25:002
34150599615,13cyclictest345340-21grep20:25:002
34150599613,17cyclictest349231-21systemctl21:40:012
341505996127,29cyclictest348776-21grep21:30:012
34149799613,17cyclictest357824-21munin-run00:25:000
341497996128,28cyclictest1-21systemd00:20:000
34149799612,13cyclictest347425-21munin-run21:05:010
34149799611,5cyclictest356236-21latency_hist23:55:020
341507996031,25cyclictest235-21systemd-journal21:55:003
34150799602,20cyclictest353125-21munin-run22:55:013
34150799602,19cyclictest355850-21latency_hist23:45:013
34150599606,12cyclictest346394-21grep20:45:012
34150599605,13cyclictest354420-21sendmail23:20:002
341505996026,30cyclictest274-21dbus-daemon20:10:022
341505996026,30cyclictest274-21dbus-daemon20:10:022
34150599602,20cyclictest355748-21unin-run23:45:002
341505996014,8cyclictest342740-21systemd-run19:34:592
34149799602,7cyclictest341933-21cron19:20:000
34149799601,17cyclictest347923-21munin-run21:15:000
34150799593,16cyclictest350298-21munin-run22:00:013
341507995931,24cyclictest356550-21wc00:00:013
341507995931,24cyclictest356550-21wc00:00:013
341507995927,29cyclictest352602-21sshd22:44:483
341507995917,37cyclictest341805-21latency_hist19:15:023
341507995916,36cyclictest0-21swapper/322:25:023
34150599595,13cyclictest344353-21latency_hist20:05:022
34150599594,16cyclictest352335-21modprobe22:39:592
34150599594,12cyclictest307-21rs:main2
34150599594,12cyclictest307-21rs:main2
34150599593,26cyclictest352973-21wc22:50:022
34150599593,14cyclictest358230-21latency_hist00:30:032
34150599593,14cyclictest358230-21latency_hist00:30:022
34150599592,16cyclictest356166-21grep23:50:022
34150199593,3cyclictest131rcu_preempt19:50:011
34149799593,9cyclictest355444-21cron23:40:000
34149799593,9cyclictest355444-21cron23:40:000
34149799593,16cyclictest352087-21munin-run22:35:000
341507995818,34cyclictest304-21in:imuxsock19:35:003
34150599585,10cyclictest308-21systemd-logind00:35:002
34150599584,15cyclictest308-21systemd-logind19:45:012
34150599584,15cyclictest308-21systemd-logind19:45:012
34150599584,14cyclictest346079-21cron20:40:002
34150599584,14cyclictest345592-21munin-run20:30:002
34150599584,12cyclictest357031-21munin-plugin-st00:10:002
34150599584,12cyclictest346878-21munin-run20:55:002
34150599583,16cyclictest235-21systemd-journal21:15:012
34150599583,12cyclictest341824-21grep19:15:022
34150599582,16cyclictest348184-21cron21:20:002
34150599582,16cyclictest348184-21cron21:20:002
34150599582,13cyclictest351306-21latency_hist22:20:002
34150599582,13cyclictest351306-21latency_hist22:20:002
341505995815,9cyclictest0-21swapper/219:20:002
34149799582,7cyclictest342239-21munin-run19:25:010
34149799582,3cyclictest0-21swapper/023:10:020
34149799582,3cyclictest0-21swapper/023:10:020
34149799582,13cyclictest345064-21latency_hist20:20:000
341497995819,35cyclictest351313-21/usr/sbin/munin22:20:010
341497995819,35cyclictest351313-21/usr/sbin/munin22:20:010
34150799574,10cyclictest346142-21cat20:40:023
34150799573,22cyclictest346393-21cat20:45:013
34150799573,15cyclictest342442-21sshd19:25:473
34150799572,15cyclictest354699-21munin-plugin-st23:25:003
34150799572,11cyclictest48-21kcompactd000:15:383
341507995720,33cyclictest1-21systemd19:25:013
34150799571,14cyclictest343737-21sshd19:52:483
34150799571,14cyclictest343737-21sshd19:52:483
341507995710,42cyclictest274-21dbus-daemon19:20:003
34150599577,12cyclictest351837-21munin-plugin-st22:30:002
34150599576,8cyclictest352263-21latency_hist22:35:022
34150599575,12cyclictest354256-21grep23:15:012
34150599573,17cyclictest358605-21munin-plugin-st00:40:002
34150599572,20cyclictest1-21systemd19:30:012
34150599572,18cyclictest343513-21munin-plugin-st19:50:002
34149799573,21cyclictest353387-21cut23:00:000
34149799572,7cyclictest344607-21wc20:10:020
34149799572,7cyclictest344607-21wc20:10:020
34149799571,15cyclictest349345-21wc21:40:020
34150799563,11cyclictest355195-21modprobe23:35:003
34150799562,8cyclictest358582-21cron00:40:003
34150599566,17cyclictest350713-21grep22:05:022
34150599566,17cyclictest350713-21grep22:05:022
34150599563,11cyclictest357532-21cron00:19:592
34149799562,9cyclictest352972-21grep22:50:020
34149799562,5cyclictest358070-21modprobe00:29:490
34149799562,5cyclictest358070-21modprobe00:29:480
34149799562,19cyclictest1-21systemd22:25:000
34149799562,17cyclictest348477-21munin-run21:25:000
34149799562,14cyclictest346515-21sort20:45:030
34149799562,13cyclictest352632-21munin-run22:45:010
34150799553,11cyclictest357379-21latency_hist00:15:023
34150799552,15cyclictest357837-21munin-run00:25:013
34150799552,15cyclictest349500-21munin-run21:45:003
34150799551,12cyclictest345485-21sed20:25:023
34150799551,12cyclictest345485-21sed20:25:023
34150799551,12cyclictest1-21systemd22:05:023
34150799551,12cyclictest1-21systemd22:05:013
34150599556,22cyclictest348462-21munin-plugin-st21:25:002
34149799552,17cyclictest358622-21sendmail-msp00:40:000
34149799552,17cyclictest344280-21cron20:05:000
34149799552,15cyclictest344003-21cron20:00:000
34150799542,8cyclictest355443-21cron23:40:013
34150799542,8cyclictest355443-21cron23:40:003
34150799542,11cyclictest353384-21systemctl23:00:003
34150799541,7cyclictest346605-21sshd20:48:483
34150799541,7cyclictest346605-21sshd20:48:483
34150799541,6cyclictest351041-21munin-run22:15:003
34150799541,12cyclictest345039-21sshd20:18:483
34150599545,11cyclictest1-21systemd21:35:012
341505995416,10cyclictest274-21dbus-daemon20:35:012
34149799547,41cyclictest0-21swapper/022:00:000
34149799545,6cyclictest354122-21sshd23:10:490
34149799541,6cyclictest343630-21cat19:50:020
34150799535,43cyclictest0-21swapper/321:40:003
34150799534,9cyclictest346903-21sort20:55:013
34150799534,5cyclictest344293-21munin-run20:05:003
34150799533,7cyclictest1-21systemd21:05:013
34150799532,6cyclictest344799-21munin-run20:15:003
34150799532,17cyclictest352971-21cat22:50:023
341507995319,30cyclictest349011-21sed21:35:013
341507995316,32cyclictest341551-21cat19:10:023
341507995316,32cyclictest273-21cron22:35:003
34150599536,9cyclictest344821-21wc20:15:012
34149799532,6cyclictest348969-21munin-run21:35:000
34149799532,10cyclictest352344-21munin-plugin-st22:40:010
34149799531,7cyclictest354395-21sshd23:18:070
341497995314,33cyclictest356781-21cut00:05:000
34149799531,13cyclictest1-21systemd23:50:020
34149799531,11cyclictest345644-21latency_hist20:30:010
34150799522,8cyclictest354421-21cron23:20:003
34150799522,8cyclictest235-21systemd-journal23:30:003
34150799522,4cyclictest48-21kcompactd019:40:263
34150799522,4cyclictest48-21kcompactd019:40:263
341507995217,30cyclictest0-21swapper/322:17:003
341507995217,30cyclictest0-21swapper/322:17:003
34150799521,15cyclictest345843-21cut20:35:003
34150599525,8cyclictest349759-21true21:50:012
341501995211,22cyclictest340835-21kworker/u8:219:10:021
34149799524,42cyclictest357169-21cat00:10:030
341497995231,15cyclictest1-21systemd23:25:010
34149799522,8cyclictest345308-21sshd20:21:480
34149799522,8cyclictest345308-21sshd20:21:480
34149799522,5cyclictest350515-21sshd22:03:490
34149799522,5cyclictest350515-21sshd22:03:480
34149799522,15cyclictest349678-21wc21:45:020
34150799513,7cyclictest274-21dbus-daemon22:30:013
34150799513,40cyclictest0-21swapper/319:50:013
34150799512,8cyclictest1-21systemd00:10:023
34150799512,6cyclictest1917536-21snmpd23:02:103
34150799511,7cyclictest352308-21sshd22:36:483
341507995117,30cyclictest356833-21latency_hist00:05:013
34150599515,16cyclictest350002-21munin-plugin-st21:55:002
34150599515,12cyclictest352642-21wc22:45:012
34150599515,11cyclictest357303-21grep00:15:012
341501995141,7cyclictest23-21ksoftirqd/123:55:031
341501995141,7cyclictest23-21ksoftirqd/123:55:031
34149799518,38cyclictest0-21swapper/019:30:000
34149799514,41cyclictest0-21swapper/022:30:000
34149799513,8cyclictest354933-21sshd23:28:490
34149799513,7cyclictest344774-21sshd20:12:480
341497995134,13cyclictest274-21dbus-daemon20:50:010
341497995134,13cyclictest274-21dbus-daemon20:50:010
34149799512,7cyclictest348186-21cron21:20:000
34149799512,7cyclictest348186-21cron21:20:000
34149799512,6cyclictest235-21systemd-journal22:55:010
34149799512,6cyclictest1-21systemd23:05:010
34149799511,7cyclictest346876-21modprobe20:55:000
34150799502,6cyclictest1917536-21snmpd21:06:083
34150799502,10cyclictest235-21systemd-journal23:15:003
34150799501,7cyclictest355958-21sshd23:47:493
341507995015,31cyclictest37-21ksoftirqd/321:25:013
341507995013,32cyclictest304-21in:imuxsock00:28:013
341507995013,32cyclictest304-21in:imuxsock00:28:003
341507995012,32cyclictest343107-21latency_hist19:40:023
341501995029,17cyclictest351585-21munin-run22:25:001
34149799505,10cyclictest235-21systemd-journal21:55:000
34149799503,5cyclictest274-21dbus-daemon19:35:000
34149799502,7cyclictest358329-21cron00:34:590
34150799492,7cyclictest1-21systemd21:30:013
34150799492,6cyclictest345569-21sshd20:28:483
34150799492,5cyclictest1917536-21snmpd21:47:553
34150799491,13cyclictest1917536-21snmpd23:08:433
34150799491,13cyclictest1917536-21snmpd23:08:433
34149799492,5cyclictest1-21systemd19:15:010
34149799491,9cyclictest343774-21systemd-run19:55:010
34149799491,9cyclictest343774-21systemd-run19:55:010
34150799482,7cyclictest235-21systemd-journal21:20:013
34150799482,7cyclictest235-21systemd-journal21:20:013
34150799481,6cyclictest356240-21munin-run23:55:013
34149799482,5cyclictest1917536-21snmpd19:40:420
34149799482,5cyclictest1917536-21snmpd19:40:420
34150799472,6cyclictest347909-21cron21:15:003
341501994734,9cyclictest1-21systemd23:10:021
341501994734,9cyclictest1-21systemd23:10:011
34149799472,14cyclictest347679-21true21:10:000
341501994633,10cyclictest235-21systemd-journal23:35:001
34149799462,6cyclictest1-21systemd22:13:480
341501994528,12cyclictest23-21ksoftirqd/100:35:021
341501994527,13cyclictest274-21dbus-daemon22:05:011
341501994527,13cyclictest274-21dbus-daemon22:05:011
341501994431,8cyclictest350787-21munin-plugin-st22:10:001
341501994430,11cyclictest354937-21cron23:30:001
341501994430,11cyclictest235-21systemd-journal20:39:191
341501994429,11cyclictest355439-21sshd23:39:481
341501994429,11cyclictest355439-21sshd23:39:481
341501994427,12cyclictest235-21systemd-journal23:20:001
341501994422,17cyclictest352873-21munin-run22:50:001
34149799441,5cyclictest341574-21wc19:10:020
34150199434,6cyclictest131rcu_preempt19:20:011
341501994331,8cyclictest1-21systemd20:30:001
341501994329,10cyclictest274-21dbus-daemon22:00:011
341501994326,11cyclictest274-21dbus-daemon22:45:001
341501994314,24cyclictest344722-21cat20:10:021
341501994314,24cyclictest344722-21cat20:10:021
34150199423,8cyclictest23-21ksoftirqd/120:22:481
34150199423,8cyclictest23-21ksoftirqd/120:22:481
341501994235,4cyclictest345100-21munin-run20:20:011
341501994229,10cyclictest235-21systemd-journal19:51:181
341501994229,10cyclictest235-21systemd-journal19:51:181
341501994227,11cyclictest351005-21sshd22:10:491
341501994226,11cyclictest341664-21sshd19:13:481
34150199422,26cyclictest23-21ksoftirqd/120:05:021
341501994219,18cyclictest352311-21sshd22:37:481
341501994213,25cyclictest357090-21latency_hist00:10:011
341501994131,6cyclictest343271-21munin-run19:45:011
341501994131,6cyclictest343271-21munin-run19:45:011
341501994126,10cyclictest304-21in:imuxsock00:35:021
341501994123,14cyclictest348141-21sshd21:15:481
341501994123,14cyclictest348141-21sshd21:15:481
341501994121,15cyclictest347896-21sshd21:12:401
341501994113,23cyclictest353540-21cat23:00:021
34150199409,7cyclictest0-21swapper/121:27:451
341501994031,6cyclictest273-21cron21:10:001
341501994031,5cyclictest355986-21munin-run23:50:001
341501994029,8cyclictest342449-21sshd19:27:211
341501994026,10cyclictest342226-21munin-run19:25:001
34150199402,33cyclictest356442-21date23:55:031
341501994022,14cyclictest357507-21cron00:16:591
341501994018,17cyclictest356524-40mandb00:00:041
341501993927,10cyclictest23-21ksoftirqd/120:00:011
341501993926,9cyclictest358045-21sshd00:26:481
341501993926,9cyclictest358045-21sshd00:26:481
341501993924,10cyclictest235-21systemd-journal19:40:011
341501993918,16cyclictest342730-21grep19:35:001
341501993914,21cyclictest348550-21latency_hist21:25:021
341501993914,18cyclictest347141-21modprobe21:00:001
341501993831,4cyclictest349766-21munin-run21:50:011
341501993830,5cyclictest355736-21munin-run23:45:011
341501993830,5cyclictest352091-21sh22:35:021
341501993823,11cyclictest353636-21sshd23:04:481
341501993823,11cyclictest351275-21sshd22:17:471
341501993823,11cyclictest351275-21sshd22:17:471
341501993820,13cyclictest1-21systemd20:45:011
341501993815,19cyclictest349546-21latency_hist21:45:011
341501993814,18cyclictest346022-21sort20:35:021
341501993719,13cyclictest1917536-21snmpd21:35:241
341501993715,17cyclictest354851-21latency_hist23:25:021
341501993713,20cyclictest344812-21munin-run20:15:011
341505993629,5cyclictest30-21ksoftirqd/219:10:022
341501993625,7cyclictest351808-21sshd22:27:491
341501993624,9cyclictest23-21ksoftirqd/121:35:021
341501993621,12cyclictest346590-21sshd20:45:481
341501993621,12cyclictest346590-21sshd20:45:481
341505993520,11cyclictest274-21dbus-daemon00:05:002
341501993528,4cyclictest353129-21sh22:55:011
341501993520,10cyclictest274-21dbus-daemon20:55:001
341501993519,11cyclictest1917536-21snmpd21:02:201
341501993421,10cyclictest23-21ksoftirqd/100:25:011
341501993320,9cyclictest1917536-21snmpd23:11:071
341501993220,9cyclictest48-21kcompactd000:10:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional