You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-20 - 08:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Sep 20, 2024 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
924957991021,79cyclictest0-21swapper/021:15:020
924957996626,10cyclictest12-21ksoftirqd/000:20:300
924961996529,33cyclictest939472-21wc23:50:011
924961996416,42cyclictest272-21dbus-daemon23:55:001
92496199633,18cyclictest935037-21munin-run22:25:021
924957996329,7cyclictest12-21ksoftirqd/023:30:010
924964996221,36cyclictest934755-21systemctl22:20:002
92496199621,10cyclictest927736-21sshd20:04:301
92496499612,16cyclictest932399-21sshd21:33:302
924964996120,37cyclictest925708-21latency_hist19:25:022
924964996120,37cyclictest925708-21latency_hist19:25:012
92496199611,7cyclictest928776-21sshd20:23:301
92496199611,6cyclictest928669-21cat20:20:021
924964996022,34cyclictest931114-21latency_hist21:10:022
92496199602,9cyclictest936038-21sshd22:43:281
92496199602,6cyclictest932963-21munin-run21:45:001
92496199602,5cyclictest929298-21munin-run20:35:001
92496199602,12cyclictest927985-21modprobe20:08:301
924964995946,9cyclictest938884-21sendmail23:40:012
92496199591,5cyclictest936308-21munin-run22:50:011
92496199591,4cyclictest937603-21munin-run23:15:011
92496199591,20cyclictest934756-21munin-run22:20:001
924964995815,13cyclictest272-21dbus-daemon22:50:012
924964995815,12cyclictest272-21dbus-daemon00:35:022
924964995815,12cyclictest272-21dbus-daemon00:35:012
924964995813,5cyclictest0-21swapper/219:50:012
924961995829,25cyclictest938250-21latency_hist23:25:021
924961995826,28cyclictest23-21ksoftirqd/119:55:001
92496199582,5cyclictest931139-21munin-run21:10:001
92496199582,5cyclictest363-21snmpd22:30:571
924961995817,37cyclictest272-21dbus-daemon23:10:001
92496199581,4cyclictest926969-21munin-run19:50:011
924957995854,2cyclictest0-21swapper/000:10:010
92496499573,16cyclictest939923-21munin-run00:00:002
92496499573,16cyclictest939923-21munin-run00:00:002
924964995725,27cyclictest30-21ksoftirqd/200:23:272
92496499572,18cyclictest925161-21munin-run19:15:002
924964995717,6cyclictest0-21swapper/222:39:592
924964995716,4cyclictest929883-21latency_hist20:45:012
924964995713,11cyclictest938334-21cat23:25:032
92496499571,16cyclictest933224-21wc21:50:012
924964995710,8cyclictest30-21ksoftirqd/220:04:592
92496199572,17cyclictest935275-21munin-run22:30:001
92496199572,17cyclictest935275-21munin-run22:30:001
92496199571,9cyclictest925939-21munin-run19:29:591
92496199571,11cyclictest363-21snmpd20:48:361
92496199571,10cyclictest926165-21sshd19:32:391
92496499561,51cyclictest30-21ksoftirqd/220:55:022
92496499561,18cyclictest926732-21munin-run19:45:012
92496199563,6cyclictest939929-40ogrotate00:00:011
92496199563,6cyclictest939929-40ogrotate00:00:001
92496199562,4cyclictest941269-21apt-config00:23:301
92496199562,13cyclictest930302-21sshd20:52:301
92496199561,5cyclictest929558-21munin-run20:40:001
92496199561,11cyclictest363-21snmpd00:17:441
92495799563,17cyclictest927212-21modprobe19:55:010
924957995617,16cyclictest1-21systemd22:05:010
924964995522,28cyclictest30-21ksoftirqd/222:05:022
92496499552,15cyclictest931625-21cron21:20:012
92496499552,15cyclictest931625-21cron21:20:012
92496499552,13cyclictest927992-21cron20:10:002
924964995516,8cyclictest936682-21date22:55:012
924961995528,24cyclictest272-21dbus-daemon22:10:021
92496199551,6cyclictest942068-21munin-run00:40:011
92496199551,6cyclictest941560-21munin-run00:30:001
924961995511,37cyclictest272-21dbus-daemon19:40:021
924961995511,37cyclictest272-21dbus-daemon19:40:011
92496199551,12cyclictest936796-21cron23:00:011
924964995425,26cyclictest940200-21latency_hist00:05:022
924964995421,28cyclictest935020-21sh22:25:002
92496499541,19cyclictest933452-21munin-run21:55:012
92496499541,19cyclictest933452-21munin-run21:55:002
92496499541,16cyclictest932958-21systemd-run21:45:012
92496199546,42cyclictest0-21swapper/121:55:021
92496199546,42cyclictest0-21swapper/121:55:021
92496199543,6cyclictest932388-21modprobe21:31:311
92496199542,5cyclictest927483-21munin-run20:00:011
924961995417,30cyclictest306-21in:imuxsock22:05:001
92496199541,6cyclictest363-21snmpd21:00:391
92496199541,5cyclictest363-21snmpd21:16:411
92496199541,5cyclictest363-21snmpd21:16:401
92496199541,12cyclictest938360-21sshd23:28:301
924967995332,17cyclictest1-21systemd23:30:003
924967995331,18cyclictest272-21dbus-daemon23:59:593
924967995331,18cyclictest272-21dbus-daemon00:00:003
92496499533,13cyclictest272-21dbus-daemon20:40:012
92496499532,18cyclictest938650-21cat23:35:012
92496499532,12cyclictest929077-21latency_hist20:30:012
924964995313,9cyclictest940629-21wc00:10:012
92496199533,5cyclictest0-21swapper/122:54:401
924961995327,23cyclictest925621-21latency_hist19:20:021
92496199532,5cyclictest363-21snmpd23:01:351
924961995317,32cyclictest939233-21cat23:45:021
92496199531,5cyclictest940217-21munin-run00:05:001
92496199531,17cyclictest937887-21systemd-run23:20:011
92496199531,13cyclictest940441-21cron00:10:001
92496199531,12cyclictest363-21snmpd22:36:121
924957995317,32cyclictest0-21swapper/020:35:030
924967995224,18cyclictest1-21systemd20:25:003
924964995222,26cyclictest30-21ksoftirqd/223:05:022
924964995213,4cyclictest0-21swapper/200:20:012
92496199522,6cyclictest925663-21sshd19:22:381
92496199522,6cyclictest925663-21sshd19:22:371
92496199522,5cyclictest363-21snmpd21:25:431
92496199522,4cyclictest925021-21cat19:10:021
92496199522,4cyclictest925021-21cat19:10:011
924961995219,29cyclictest1-21systemd23:35:011
92496199521,8cyclictest933169-21sshd21:46:311
92496199521,5cyclictest363-21snmpd00:31:321
92496199521,5cyclictest363-21snmpd00:31:321
92496199521,4cyclictest928257-21munin-run20:15:011
92496199521,4cyclictest928257-21munin-run20:15:001
92496199521,13cyclictest363-21snmpd21:23:201
924957995225,24cyclictest19050irq/34-mmc119:25:000
924957995225,24cyclictest19050irq/34-mmc119:25:000
92496799512,39cyclictest272-21dbus-daemon23:10:013
92496499512,19cyclictest942069-21grep00:39:592
92496499512,17cyclictest932672-21latency_hist21:40:002
924964995115,11cyclictest30-21ksoftirqd/221:11:292
92496199513,5cyclictest0-21swapper/120:56:221
92496199512,5cyclictest363-21snmpd21:58:271
92496199511,12cyclictest363-21snmpd21:35:161
924967995026,20cyclictest272-21dbus-daemon22:30:023
924967995026,20cyclictest272-21dbus-daemon22:30:013
92496499507,10cyclictest0-21swapper/219:35:022
92496499502,36cyclictest930062-21kworker/u8:320:50:022
924964995019,9cyclictest939172-21systemctl23:45:002
924964995015,5cyclictest306-21in:imuxsock00:15:012
924964995014,8cyclictest934347-21latency_hist22:10:022
92496199502,4cyclictest0-21swapper/119:11:041
92496199502,14cyclictest940673-21sshd00:11:291
92496799495,32cyclictest935528-21grep22:34:593
924967994930,14cyclictest235-21systemd-journal00:30:033
924967994930,14cyclictest235-21systemd-journal00:30:023
924967994918,19cyclictest0-21swapper/323:00:003
92496499497,10cyclictest30-21ksoftirqd/219:37:302
92496499497,10cyclictest30-21ksoftirqd/219:37:302
924964994923,23cyclictest930584-21latency_hist21:00:012
92496499492,15cyclictest936938-21wc23:00:022
924964994917,12cyclictest30-21ksoftirqd/219:57:312
92496199492,5cyclictest0-21swapper/119:42:251
92496199492,4cyclictest929796-21modprobe20:41:311
92496199492,4cyclictest363-21snmpd23:35:091
92496799485,28cyclictest927472-21modprobe20:00:003
924967994828,10cyclictest306-21in:imuxsock21:35:003
924967994820,18cyclictest272-21dbus-daemon21:05:023
92496499483,16cyclictest934501-21munin-run22:15:012
924964994830,14cyclictest272-21dbus-daemon19:20:012
924964994813,5cyclictest1-21systemd21:25:002
92495799488,8cyclictest12-21ksoftirqd/022:30:290
924967994731,12cyclictest235-21systemd-journal20:35:023
924967994730,12cyclictest272-21dbus-daemon22:45:003
92496499472,15cyclictest925926-21cron19:29:592
924964994713,5cyclictest939412-21latency_hist23:50:022
92496199477,10cyclictest272-21dbus-daemon20:30:021
92496199472,4cyclictest363-21snmpd22:10:031
92496199471,6cyclictest931382-21/usr/sbin/munin21:15:011
924957994731,12cyclictest934751-21cat22:20:000
924967994631,11cyclictest235-21systemd-journal20:17:003
924967994630,9cyclictest306-21in:imuxsock22:50:013
924967994623,17cyclictest235-21systemd-journal22:19:593
924967994623,15cyclictest272-21dbus-daemon23:50:013
924967994620,16cyclictest272-21dbus-daemon00:10:003
924967994531,9cyclictest941542-21sh00:30:013
924967994531,11cyclictest37-21ksoftirqd/322:15:013
924967994526,14cyclictest272-21dbus-daemon19:40:023
924967994526,14cyclictest272-21dbus-daemon19:40:013
924967994524,17cyclictest925427-21tr19:20:013
924967994523,15cyclictest235-21systemd-journal00:19:593
924967994520,17cyclictest235-21systemd-journal00:15:013
92496499452,16cyclictest933714-21systemctl21:59:592
92496499452,15cyclictest932180-21munin-run21:30:012
924964994515,9cyclictest941747-21cut00:30:032
924957994528,12cyclictest235-21systemd-journal00:17:000
92495799452,38cyclictest932657-21cron21:40:010
92496799446,25cyclictest940196-21modprobe00:04:593
924967994425,14cyclictest931100-21cron21:09:593
924967994425,13cyclictest306-21in:imuxsock21:40:023
924964994430,10cyclictest1-21systemd20:20:002
924957994428,9cyclictest235-21systemd-journal21:30:010
924957994424,15cyclictest306-21in:imuxsock19:40:010
924957994424,15cyclictest306-21in:imuxsock19:40:000
92496799439,22cyclictest235-21systemd-journal22:40:003
92496799438,20cyclictest932665-21modprobe21:40:013
92496799435,26cyclictest925940-21grep19:30:003
92496799434,28cyclictest235-21systemd-journal21:15:003
924967994322,15cyclictest1-21systemd23:55:003
924967994315,19cyclictest272-21dbus-daemon20:30:013
92496499432,16cyclictest937346-21true23:10:012
92495799438,30cyclictest235-21systemd-journal20:40:000
924957994330,9cyclictest937858-21systemctl23:20:010
924957994330,10cyclictest12-21ksoftirqd/000:35:000
924957994330,10cyclictest12-21ksoftirqd/000:35:000
924957994328,12cyclictest1-21systemd21:35:000
924957994323,14cyclictest272-21dbus-daemon20:20:010
92495799431,8cyclictest931644-21systemctl21:20:010
92495799431,8cyclictest931644-21systemctl21:20:000
92496799424,30cyclictest935027-21cut22:25:003
924967994227,9cyclictest306-21in:imuxsock19:55:013
924967994225,12cyclictest235-21systemd-journal21:59:593
924957994228,11cyclictest12-21ksoftirqd/023:55:000
924957994228,10cyclictest930848-21cron21:05:010
924957994225,12cyclictest12-21ksoftirqd/020:30:010
924967994130,7cyclictest37-21ksoftirqd/320:05:013
924967994129,9cyclictest37-21ksoftirqd/320:10:033
924967994127,10cyclictest306-21in:imuxsock23:25:023
924967994122,13cyclictest1-21systemd19:15:003
924964994126,11cyclictest937837-21cron23:20:002
92495799414,12cyclictest235-21systemd-journal20:00:010
924957994134,4cyclictest16950irq/224-800000023:00:000
924957994134,4cyclictest16950irq/224-800000022:10:000
924957994133,5cyclictest930585-21sendmail21:00:000
924957994133,5cyclictest19050irq/34-mmc100:40:000
924957994133,5cyclictest16950irq/224-800000021:09:590
924957994127,9cyclictest12-21ksoftirqd/023:59:590
924957994127,9cyclictest12-21ksoftirqd/000:00:000
924957994118,17cyclictest272-21dbus-daemon21:50:000
924967994027,10cyclictest37-21ksoftirqd/322:08:103
924967994025,11cyclictest1-21systemd21:50:003
92496799402,4cyclictest131rcu_preempt23:15:003
924967994021,13cyclictest1-21systemd20:35:023
924967994018,16cyclictest272-21dbus-daemon20:15:013
924967994018,16cyclictest272-21dbus-daemon20:15:013
924964994010,6cyclictest131rcu_preempt20:30:302
924957994036,2cyclictest0-21swapper/019:15:030
924957994033,4cyclictest16950irq/224-800000022:40:010
924957994030,7cyclictest12-21ksoftirqd/000:15:020
924957994026,11cyclictest12-21ksoftirqd/023:05:020
924957994025,12cyclictest940189-21sshd00:04:280
924957994025,11cyclictest926158-21sshd19:31:290
924957994024,12cyclictest272-21dbus-daemon22:55:010
92495799402,34cyclictest938625-21sh23:35:010
92496799399,17cyclictest930575-21modprobe21:00:003
92496799395,26cyclictest926896-21cat19:45:013
924967993928,8cyclictest938613-21sshd23:34:303
924967993927,6cyclictest37-21ksoftirqd/321:20:013
924967993927,6cyclictest37-21ksoftirqd/321:20:003
924967993925,10cyclictest235-21systemd-journal23:43:293
924957993925,11cyclictest363-21snmpd19:40:230
92495799392,4cyclictest1-21systemd20:45:000
924957993924,11cyclictest363-21snmpd23:06:040
924957993922,13cyclictest272-21dbus-daemon23:15:000
924967993830,5cyclictest937085-21latency_hist23:05:013
924967993824,9cyclictest306-21in:imuxsock22:53:223
924967993824,10cyclictest937938-21cat23:20:023
924967993819,13cyclictest1-21systemd00:40:013
924957993829,5cyclictest925134-21cron19:14:590
924957993828,8cyclictest12-21ksoftirqd/020:55:010
924957993826,9cyclictest938848-21sshd23:35:300
92495799382,32cyclictest1-21systemd20:15:010
92495799382,32cyclictest1-21systemd20:15:010
92496799373,25cyclictest933453-21grep21:55:013
92496799373,25cyclictest933453-21grep21:55:003
924967993724,9cyclictest235-21systemd-journal21:25:003
924967993712,15cyclictest941314-21latency_hist00:25:023
92496499372,6cyclictest131rcu_preempt23:55:012
924957993730,4cyclictest0-21swapper/021:24:300
92495799372,5cyclictest935025-21munin-run22:25:000
924957993721,12cyclictest272-21dbus-daemon21:55:010
924957993721,12cyclictest272-21dbus-daemon21:55:000
924957993718,14cyclictest235-21systemd-journal20:10:010
92495799371,7cyclictest926938-21sshd19:46:300
924967993625,8cyclictest37-21ksoftirqd/320:55:013
924967993617,14cyclictest1-21systemd23:40:013
924957993628,5cyclictest936237-21latency_hist22:45:020
924957993627,6cyclictest12-21ksoftirqd/022:50:010
924957993626,6cyclictest12850irq/18-4d00000020:25:010
924967993527,5cyclictest1-21systemd20:50:013
92495799359,16cyclictest924955-21ls19:10:020
92495799359,16cyclictest924955-21ls19:10:010
92496799349,15cyclictest929819-21modprobe20:45:003
92496799344,24cyclictest235-21systemd-journal22:05:013
924967993424,6cyclictest927076-21latency_hist19:50:013
924964993424,8cyclictest30-21ksoftirqd/220:25:022
924964993421,10cyclictest938474-21date23:30:022
924964993419,11cyclictest272-21dbus-daemon19:55:012
924964993418,13cyclictest235-21systemd-journal22:30:012
924964993418,13cyclictest235-21systemd-journal22:30:002
924957993427,4cyclictest16950irq/224-800000022:30:020
924957993427,4cyclictest16950irq/224-800000022:30:010
924957993425,4cyclictest12850irq/18-4d00000023:50:020
924957993422,9cyclictest272-21dbus-daemon20:05:000
924957993416,14cyclictest272-21dbus-daemon22:15:020
924967993321,9cyclictest37-21ksoftirqd/321:30:023
924964993325,5cyclictest928435-21latency_hist20:15:032
924964993325,5cyclictest928435-21latency_hist20:15:022
924964993318,11cyclictest930926-21sort21:05:012
924957993328,3cyclictest0-21swapper/023:45:010
924957993325,5cyclictest925935-21latency_hist19:30:020
924957993324,6cyclictest932948-21munin-plugin-st21:44:590
924957993322,7cyclictest306-21in:imuxsock23:23:290
924957993317,12cyclictest933870-21grep22:00:010
92495799331,29cyclictest0-21swapper/020:49:340
92496799322,15cyclictest925696-21true19:25:013
92496799322,15cyclictest925696-21true19:25:013
924964993221,8cyclictest924978-21sed19:10:012
924964993221,8cyclictest924978-21sed19:10:012
924964993221,8cyclictest30-21ksoftirqd/222:45:022
924964993219,10cyclictest937775-21sed23:15:022
924967993120,9cyclictest37-21ksoftirqd/319:10:013
924967993120,9cyclictest37-21ksoftirqd/319:10:013
924957993121,7cyclictest12-21ksoftirqd/000:30:030
92496799306,16cyclictest926176-21cron19:35:003
924964992714,10cyclictest1-21systemd22:35:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional