You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-22 - 12:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jan 22, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3749836991129,4cyclictest131rcu_preempt20:55:022
3749836997822,49cyclictest3753915-21cron20:30:002
3749836997822,49cyclictest3753915-21cron20:30:002
3749836997613,19cyclictest3759627-21cron22:20:002
374982499742,10cyclictest3761461-21munin-run22:55:010
3749836997124,43cyclictest3764249-21grep23:45:032
3749836997124,43cyclictest3764249-21grep23:45:022
3749836996918,46cyclictest272-21dbus-daemon00:15:002
374982499682,6cyclictest3754401-21sshd20:36:210
374982499681,13cyclictest3754688-21sshd20:42:210
3749836996718,44cyclictest272-21dbus-daemon20:05:012
3749836996718,44cyclictest272-21dbus-daemon20:05:012
3749836996718,12cyclictest3763774-21sshd23:39:222
374982499673,15cyclictest3752611-21sshd20:02:210
374982499673,15cyclictest3752611-21sshd20:02:210
374982499671,9cyclictest3764791-21sshd23:55:210
374982499671,9cyclictest3764791-21sshd23:55:210
374982499671,13cyclictest3766904-21sshd00:37:210
3749836996521,14cyclictest272-21dbus-daemon22:15:012
3749836996416,38cyclictest306-21in:imuxsock23:20:002
374982499642,7cyclictest1-21systemd23:36:210
374982499642,5cyclictest3760720-21munin-run22:40:020
374982499642,15cyclictest3766652-21sshd00:32:360
374982499641,11cyclictest3752085-21sshd19:54:220
3749836996322,5cyclictest3753397-21latency_hist20:20:032
374982499632,14cyclictest3759127-21munin-run22:10:000
374982499632,13cyclictest3760425-21munin-run22:35:000
374982499632,13cyclictest3760425-21munin-run22:35:000
374982499632,12cyclictest3759351-21sshd22:12:040
374982499631,14cyclictest3751294-21sshd19:37:210
374983699625,20cyclictest3751843-21modprobe19:50:002
374983699625,20cyclictest3751843-21modprobe19:50:002
374983699623,17cyclictest3766919-21cron00:40:012
374983699622,18cyclictest3759128-21grep22:10:012
3749836996218,6cyclictest30-21ksoftirqd/219:13:212
374982499622,15cyclictest3754183-21munin-run20:35:000
374982499622,14cyclictest3764589-21munin-run23:55:020
374982499622,14cyclictest3758356-21munin-run21:55:010
374982499622,10cyclictest3752334-21sshd19:58:210
374982499613,5cyclictest3756235-21sshd21:11:530
374982499612,5cyclictest3763009-21sshd23:22:200
374982499612,4cyclictest3755435-21sshd20:56:210
374982499612,14cyclictest3758573-21sshd21:58:210
374982499612,13cyclictest363-21snmpd22:46:060
374982499611,14cyclictest363-21snmpd00:17:570
3749836996022,11cyclictest3760942-21sshd22:44:222
374983699602,15cyclictest3751060-21modprobe19:35:002
374983699602,13cyclictest3764924-21wc00:00:032
374983699602,13cyclictest3764924-21wc00:00:022
374982499603,4cyclictest0-21swapper/023:50:010
3749824996027,29cyclictest1-21systemd23:29:210
374982499602,5cyclictest3759921-21munin-run22:25:000
374982499602,4cyclictest3757569-21munin-run21:40:000
374982499602,16cyclictest235-21systemd-journal19:30:000
374982499601,6cyclictest3751652-21cat19:45:020
374982499601,5cyclictest3762464-21sshd23:11:210
374982499601,5cyclictest3756776-21sshd21:21:550
374982499601,13cyclictest3755984-21sshd21:09:210
374982499601,12cyclictest3752898-21munin-run20:10:000
374982499601,11cyclictest3765597-21sshd00:14:220
3749824996010,46cyclictest1-21systemd20:25:000
374983699599,8cyclictest0-21swapper/220:44:592
374983699595,11cyclictest3762500-21munin-plugin-st23:15:002
374983699594,18cyclictest3754426-21modprobe20:40:002
374982499592,11cyclictest3762246-21munin-run23:10:000
3749824995914,40cyclictest363-21snmpd23:02:090
374982499591,12cyclictest3765376-21munin-run00:10:000
374983699584,9cyclictest3761987-21modprobe23:05:002
374983699584,19cyclictest3759934-21systemd-run22:25:012
374983699584,11cyclictest235-21systemd-journal20:10:002
374983699583,16cyclictest3757309-21munin-run21:35:012
374983699582,18cyclictest3757843-21cut21:45:002
374983699582,18cyclictest3757843-21cut21:45:002
3749836995821,33cyclictest3765382-21latency_hist00:10:022
3749836995820,8cyclictest30-21ksoftirqd/223:25:022
3749836995816,6cyclictest3751035-21kworker/u8:119:40:002
374983299589,25cyclictest23-21ksoftirqd/121:50:021
374982499582,5cyclictest3765127-21munin-run00:05:010
374982499582,16cyclictest3759637-21modprobe22:20:010
374982499582,12cyclictest3766157-21munin-run00:25:000
374982499582,10cyclictest3750003-21sshd19:14:210
374983699575,20cyclictest3758084-21modprobe21:49:592
374983699575,11cyclictest272-21dbus-daemon21:20:002
374983699573,12cyclictest3760427-21cut22:35:012
374983699573,12cyclictest3760427-21cut22:35:002
374983699572,17cyclictest3754322-21wc20:35:012
374983699572,16cyclictest3758980-21latency_hist22:05:012
3749836995719,10cyclictest3765292-21wc00:05:012
3749824995753,2cyclictest0-21swapper/020:30:030
3749824995753,2cyclictest0-21swapper/020:30:020
374982499572,8cyclictest3760924-21sshd22:41:210
374982499572,6cyclictest363-21snmpd21:18:030
374982499572,6cyclictest363-21snmpd00:29:580
374982499572,5cyclictest3764050-21sshd23:43:210
374982499572,5cyclictest3764050-21sshd23:43:210
374982499571,13cyclictest3758852-21sshd22:02:210
374982499571,10cyclictest3755178-21sshd20:50:210
374983699568,12cyclictest0-21swapper/220:00:002
374983699564,13cyclictest3765864-21cron00:20:012
374983699564,11cyclictest3755988-21cron21:10:012
374983699563,12cyclictest235-21systemd-journal19:20:002
374983699562,15cyclictest235-21systemd-journal22:40:002
374982499566,4cyclictest0-21swapper/020:15:170
374982499562,8cyclictest3757803-21sshd21:40:220
374982499562,8cyclictest3757803-21sshd21:40:210
374982499562,5cyclictest3758097-21systemd-run21:50:000
374982499562,5cyclictest3755748-21munin-run21:05:010
374982499561,13cyclictest3754946-21sshd20:49:220
374982499561,12cyclictest3760155-21sshd22:29:210
374983699559,7cyclictest0-21swapper/221:40:002
374983699554,13cyclictest3755022-21cat20:50:012
374983699553,14cyclictest3766495-21latency_hist00:30:012
374983699553,12cyclictest3756265-21munin-run21:15:002
374983699552,18cyclictest3766219-21grep00:25:012
374983699552,15cyclictest1-21systemd21:05:002
3749832995513,9cyclictest23-21ksoftirqd/122:50:021
374982499552,5cyclictest3753117-21modprobe20:13:220
374982499552,47cyclictest3749818-21cyclictest19:10:010
374982499552,15cyclictest3763505-21sshd23:30:210
374982499551,10cyclictest3762791-21systemd-run23:20:020
374983699545,10cyclictest281-21systemd-logind23:10:002
3749836995422,9cyclictest30-21ksoftirqd/221:55:012
374983699542,15cyclictest3766677-21munin-run00:35:002
3749836995415,10cyclictest30-21ksoftirqd/223:55:002
374983299548,8cyclictest3756965-21sed21:25:031
374982499542,6cyclictest3761686-21sshd22:57:140
374982499542,5cyclictest3750291-21munin-run19:20:010
374982499541,4cyclictest3751051-21sshd19:34:220
374983999537,9cyclictest37-21ksoftirqd/300:00:013
374983999537,9cyclictest37-21ksoftirqd/300:00:013
374983699533,13cyclictest3761724-21grep23:00:012
3749836995321,28cyclictest3752170-21latency_hist19:55:012
374982499532,5cyclictest363-21snmpd19:20:240
374982499531,13cyclictest363-21snmpd21:25:560
3749839995236,11cyclictest1-21systemd00:10:003
3749839995228,18cyclictest272-21dbus-daemon21:40:013
374983699525,14cyclictest3755355-21grep20:55:032
374983699523,20cyclictest3758605-21munin-plugin-st22:00:002
374983699522,17cyclictest3761313-21latency_hist22:50:012
3749832995238,9cyclictest235-21systemd-journal23:00:001
374983999514,11cyclictest37-21ksoftirqd/321:45:013
374983999514,11cyclictest37-21ksoftirqd/321:45:013
374983699517,11cyclictest3753129-21modprobe20:15:002
374983699515,14cyclictest3757076-21latency_hist21:30:012
374983699512,8cyclictest3763533-21modprobe23:34:592
374983299515,9cyclictest23-21ksoftirqd/120:05:011
374983299515,9cyclictest23-21ksoftirqd/120:05:011
3749832995136,9cyclictest272-21dbus-daemon00:00:031
3749832995136,9cyclictest272-21dbus-daemon00:00:021
374983999506,6cyclictest37-21ksoftirqd/321:30:033
374983999503,11cyclictest37-21ksoftirqd/322:20:023
374983699503,17cyclictest3750915-21wc19:30:022
374983699502,13cyclictest3753676-21modprobe20:25:002
374983299503,6cyclictest23-21ksoftirqd/100:35:011
374982499504,40cyclictest0-21swapper/021:35:000
374982499502,5cyclictest3751818-21sshd19:46:210
374982499502,5cyclictest3751818-21sshd19:46:210
3749839994938,6cyclictest37-21ksoftirqd/321:15:013
3749836994931,13cyclictest272-21dbus-daemon22:55:002
374983999489,12cyclictest37-21ksoftirqd/321:00:003
374983999484,10cyclictest37-21ksoftirqd/323:21:213
3749839994834,10cyclictest1-21systemd19:20:013
3749832994827,13cyclictest23-21ksoftirqd/123:35:021
3749836994733,9cyclictest272-21dbus-daemon23:30:012
374983299478,7cyclictest23-21ksoftirqd/121:50:011
3749832994730,11cyclictest235-21systemd-journal00:25:011
3749832994729,15cyclictest272-21dbus-daemon19:35:011
3749832994710,9cyclictest23-21ksoftirqd/123:45:001
3749832994710,9cyclictest23-21ksoftirqd/123:45:001
3749839994632,9cyclictest272-21dbus-daemon21:20:003
3749839994621,9cyclictest37-21ksoftirqd/300:02:223
3749839994616,6cyclictest131rcu_preempt19:35:013
3749839994612,5cyclictest40850lldpd19:46:333
3749839994612,5cyclictest40850lldpd19:46:333
374983699464,14cyclictest3756826-21grep21:25:012
3749832994613,8cyclictest0-21swapper/119:55:001
3749839994529,12cyclictest272-21dbus-daemon20:45:003
3749839994528,12cyclictest1-21systemd23:30:013
3749839994527,12cyclictest235-21systemd-journal22:15:003
374983999451,8cyclictest1-21systemd19:40:003
3749839994511,6cyclictest131rcu_preempt23:45:023
3749839994511,6cyclictest131rcu_preempt23:45:023
374983299456,8cyclictest23-21ksoftirqd/119:40:021
374983299453,9cyclictest23-21ksoftirqd/120:40:021
374983299452,11cyclictest23-21ksoftirqd/122:40:001
374983299451,29cyclictest3755177-21kworker/u8:021:15:021
374983999445,8cyclictest37-21ksoftirqd/322:50:013
374983999442,5cyclictest131rcu_preempt21:25:023
3749839994411,6cyclictest131rcu_preempt23:40:003
374983299448,9cyclictest0-21swapper/122:45:001
374983299446,7cyclictest0-21swapper/121:40:011
3749839994333,8cyclictest361rcuc/322:30:003
3749839994333,7cyclictest37-21ksoftirqd/321:10:023
3749839994332,8cyclictest37-21ksoftirqd/300:20:003
3749839994313,6cyclictest131rcu_preempt23:32:223
3749832994332,5cyclictest3764574-21munin-plugin-st23:55:001
3749832994329,10cyclictest23-21ksoftirqd/120:15:011
374983999427,7cyclictest37-21ksoftirqd/322:25:033
3749839994224,14cyclictest235-21systemd-journal20:00:003
3749832994229,10cyclictest3752897-21systemd-run20:10:001
374983999415,6cyclictest131rcu_preempt23:20:023
3749839994129,10cyclictest37-21ksoftirqd/320:55:003
374983299414,6cyclictest23-21ksoftirqd/121:15:011
3749832994128,9cyclictest235-21systemd-journal21:30:001
3749832994125,11cyclictest272-21dbus-daemon00:05:001
3749832994123,13cyclictest1-21systemd20:35:001
3749832994112,8cyclictest23-21ksoftirqd/119:13:061
374983999408,8cyclictest37-21ksoftirqd/321:55:013
3749839994031,7cyclictest37-21ksoftirqd/323:00:023
3749839994031,6cyclictest37-21ksoftirqd/321:50:023
3749839994027,9cyclictest0-21swapper/319:45:013
3749839994022,13cyclictest1-21systemd22:10:003
3749839994012,5cyclictest131rcu_preempt20:15:023
374983299404,6cyclictest23-21ksoftirqd/100:16:291
3749832994033,4cyclictest3753948-21latency_hist20:30:011
3749832994033,4cyclictest3753948-21latency_hist20:30:011
3749832994029,7cyclictest23-21ksoftirqd/121:05:011
3749832994028,7cyclictest3758861-21cron22:05:001
3749839993929,7cyclictest37-21ksoftirqd/320:20:003
3749839993928,9cyclictest37-21ksoftirqd/319:15:013
3749839993928,9cyclictest361rcuc/323:55:023
3749839993928,8cyclictest3755726-21sshd21:03:223
3749839993928,8cyclictest37-21ksoftirqd/320:40:013
3749839993928,8cyclictest37-21ksoftirqd/300:33:433
3749839993926,9cyclictest37-21ksoftirqd/319:55:013
3749839993924,11cyclictest37-21ksoftirqd/322:55:013
3749839993923,11cyclictest235-21systemd-journal19:30:003
374983999391,9cyclictest0-21swapper/320:24:213
374983299393,8cyclictest23-21ksoftirqd/122:25:011
3749832993930,6cyclictest3765445-21sed00:10:011
374983999384,10cyclictest37-21ksoftirqd/323:10:003
3749839993830,6cyclictest37-21ksoftirqd/320:31:213
3749839993830,4cyclictest3764328-21sh23:50:003
3749839993829,6cyclictest37-21ksoftirqd/300:15:023
374983999382,4cyclictest131rcu_preempt23:15:013
374983999381,10cyclictest0-21swapper/320:05:003
374983999381,10cyclictest0-21swapper/320:05:003
3749832993830,4cyclictest208-21jbd2/mmcblk1p2-19:23:041
3749832993826,10cyclictest23-21ksoftirqd/120:55:021
3749832993824,11cyclictest23-21ksoftirqd/120:45:011
374983999379,5cyclictest131rcu_preempt22:05:033
3749839993730,4cyclictest3758602-21/usr/sbin/munin22:00:013
3749839993728,5cyclictest306-21in:imuxsock22:40:013
3749839993727,8cyclictest37-21ksoftirqd/320:30:003
3749839993727,8cyclictest37-21ksoftirqd/320:30:003
3749839993727,7cyclictest361rcuc/322:45:003
3749839993727,6cyclictest306-21in:imuxsock00:25:003
3749839993725,10cyclictest37-21ksoftirqd/320:50:003
3749839993710,23cyclictest361rcuc/322:56:213
3749836993727,8cyclictest30-21ksoftirqd/219:23:222
374983299375,23cyclictest3752893-21kworker/u8:120:20:021
3749832993729,6cyclictest23-21ksoftirqd/122:28:211
3749832993728,5cyclictest3756009-21modprobe21:10:001
3749832993726,8cyclictest23-21ksoftirqd/119:45:011
3749832993724,10cyclictest23-21ksoftirqd/120:00:001
3749832993712,6cyclictest0-21swapper/121:59:211
374983999369,5cyclictest131rcu_preempt22:35:023
374983999368,5cyclictest131rcu_preempt19:10:013
374983999366,8cyclictest37-21ksoftirqd/320:10:013
374983299364,6cyclictest23-21ksoftirqd/123:05:021
3749832993628,4cyclictest3766937-21munin-plugin-st00:40:001
3749832993624,8cyclictest3761435-21sshd22:52:131
3749832993624,8cyclictest3757990-21date21:45:021
3749832993624,8cyclictest3757990-21date21:45:021
3749832993623,9cyclictest272-21dbus-daemon23:35:001
3749832993623,9cyclictest23-21ksoftirqd/119:28:221
3749839993527,6cyclictest361rcuc/321:35:003
3749839993526,7cyclictest37-21ksoftirqd/319:25:003
3749839993525,6cyclictest3766928-21modprobe00:40:003
374983299352,6cyclictest23-21ksoftirqd/100:12:321
3749832993524,7cyclictest3759806-21cat22:20:021
3749832993522,9cyclictest3751965-21sed19:50:021
3749832993522,9cyclictest3751965-21sed19:50:021
3749832993518,12cyclictest235-21systemd-journal23:23:211
3749839993424,7cyclictest37-21ksoftirqd/300:28:213
3749836993420,11cyclictest30-21ksoftirqd/223:50:002
374983299346,7cyclictest0-21swapper/123:18:221
374983299343,6cyclictest131rcu_preempt21:35:001
3749832993426,5cyclictest616-21sendmail-mta20:23:411
3749832993419,11cyclictest23-21ksoftirqd/120:45:211
3749832993418,12cyclictest272-21dbus-daemon23:30:021
374983299341,7cyclictest23-21ksoftirqd/120:57:221
3749832993417,12cyclictest272-21dbus-daemon23:15:021
374983999335,9cyclictest37-21ksoftirqd/322:35:003
3749836993322,7cyclictest3751698-21cat19:45:032
374983299334,24cyclictest3750330-21latency_hist19:20:021
3749836993217,8cyclictest3760170-21munin-plugin-st22:30:002
374983299326,7cyclictest23-21ksoftirqd/122:30:451
374983299326,7cyclictest23-21ksoftirqd/122:30:441
374983299323,8cyclictest0-21swapper/122:09:051
374983299323,6cyclictest0-21swapper/123:06:141
3749832993219,10cyclictest23-21ksoftirqd/123:45:221
374983299321,4cyclictest0-21swapper/100:28:361
3749832993123,5cyclictest1-21systemd22:15:021
374983699304,18cyclictest3749908-21latency_hist19:10:022
3749832992519,4cyclictest3749886-21latency_hist19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional