You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-16 - 14:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue Apr 16, 2024 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
35441809910782,23cyclictest0-21swapper/019:15:020
3544180999790,3cyclictest0-21swapper/020:10:030
354418099971,3cyclictest0-21swapper/022:05:020
3544180998679,2cyclictest0-21swapper/022:25:010
3544180998577,3cyclictest0-21swapper/023:05:020
354418099731,70cyclictest0-21swapper/021:25:010
3544190997014,18cyclictest3558770-21sed23:50:012
3544190996824,15cyclictest3553758-21sshd22:12:242
3544190996617,13cyclictest3558974-21latency_hist23:55:022
3544190996517,19cyclictest3554881-21date22:35:022
3544190996416,14cyclictest3550745-21cut21:15:012
3544190996413,16cyclictest3557737-21latency_hist23:30:022
3544190996413,16cyclictest3557737-21latency_hist23:30:022
3544190996314,19cyclictest3556820-21cat23:10:022
3544190996313,12cyclictest3560016-21latency_hist00:15:022
3544190996213,17cyclictest3546296-21sed19:50:022
3544185996039,15cyclictest235-21systemd-journal23:25:001
3544190995929,20cyclictest274-21dbus-daemon20:40:012
3544190995912,16cyclictest3555721-21latency_hist22:50:022
3544190995912,16cyclictest3555721-21latency_hist22:50:012
3544194995820,5cyclictest131rcu_preempt19:12:023
3544190995740,12cyclictest3548287-21apt-get20:29:502
3544190995737,16cyclictest3547460-21sshd20:11:072
3544180995753,2cyclictest0-21swapper/022:20:020
3544180995753,2cyclictest0-21swapper/022:20:020
3544180995751,4cyclictest16850irq/224-800000023:10:000
3544190995648,4cyclictest3546964-21sshd20:02:112
354419499554,7cyclictest37-21ksoftirqd/320:20:023
3544190995532,13cyclictest3559216-21cron00:00:002
354419499544,7cyclictest37-21ksoftirqd/323:00:013
3544190995435,15cyclictest3552225-21sshd21:43:072
3544190995433,16cyclictest3545931-21sshd19:42:082
3544180995437,13cyclictest12-21ksoftirqd/000:00:010
3544180995410,6cyclictest131rcu_preempt21:15:020
354419499537,11cyclictest37-21ksoftirqd/321:35:013
354419499535,9cyclictest37-21ksoftirqd/323:55:013
3544194995332,16cyclictest1-21systemd23:40:013
3544190995330,16cyclictest235-21systemd-journal22:30:012
3544190995329,17cyclictest274-21dbus-daemon19:20:012
3544190995329,16cyclictest274-21dbus-daemon22:25:002
3544190995236,12cyclictest3549621-21munin-run20:55:012
3544190995233,12cyclictest304-21in:imuxsock20:49:592
354418599524,38cyclictest221rcuc/123:10:011
354418099522,6cyclictest131rcu_preempt00:07:360
3544180995225,24cyclictest12-21ksoftirqd/023:35:010
354419499516,10cyclictest37-21ksoftirqd/321:55:003
354419499511,7cyclictest3559264-40mandb00:00:013
3544194995115,6cyclictest131rcu_preempt20:45:073
3544190995137,10cyclictest3549852-21sshd20:58:072
3544190995136,11cyclictest3560781-21sshd00:26:092
3544190995136,11cyclictest3553531-21munin-run22:10:002
3544190995135,13cyclictest1917536-21snmpd23:16:162
3544190995135,13cyclictest1917536-21snmpd23:16:162
3544190995135,12cyclictest3555042-21sshd22:35:062
3544190995135,12cyclictest3553261-21sshd22:04:072
3544190995135,12cyclictest3548559-21sshd20:32:072
3544190995135,11cyclictest3545641-21sshd19:35:072
3544190995133,15cyclictest1917536-21snmpd00:36:322
3544194995032,14cyclictest1-21systemd20:45:003
3544194995012,6cyclictest131rcu_preempt22:35:023
3544190995040,6cyclictest3549380-21kworker/2:022:00:002
3544190995036,10cyclictest3552746-21munin-run21:55:002
3544190995035,11cyclictest3561068-21/usr/sbin/munin00:35:012
3544190995035,11cyclictest3561068-21/usr/sbin/munin00:35:012
3544190995035,10cyclictest3544904-21sshd19:24:072
3544190995034,12cyclictest3544381-21munin-run19:15:002
3544190995025,19cyclictest3558205-21systemctl23:40:002
3544190995022,21cyclictest3556041-21cat22:55:022
3544185995031,15cyclictest235-21systemd-journal22:20:011
3544185995031,15cyclictest235-21systemd-journal22:20:011
3544180995024,22cyclictest12-21ksoftirqd/022:40:000
354419499492,10cyclictest37-21ksoftirqd/321:25:013
3544194994911,6cyclictest131rcu_preempt22:50:013
3544194994911,6cyclictest131rcu_preempt22:50:003
3544194994911,5cyclictest131rcu_preempt20:25:013
3544190994935,10cyclictest3548008-21sshd20:22:132
3544190994934,11cyclictest3547219-21sshd20:08:062
3544190994934,10cyclictest3554052-21munin-run22:20:012
3544190994934,10cyclictest3554052-21munin-run22:20:012
3544190994934,10cyclictest3550432-21munin-run21:10:012
3544190994934,10cyclictest3546718-21munin-run20:00:012
3544190994929,16cyclictest1917536-21snmpd00:15:472
3544185994936,9cyclictest1-21systemd21:35:001
3544180994936,8cyclictest304-21in:imuxsock19:29:080
354418099492,5cyclictest131rcu_preempt00:35:020
354418099492,5cyclictest131rcu_preempt00:35:010
3544180994925,19cyclictest131rcu_preempt22:15:010
354419499489,10cyclictest37-21ksoftirqd/300:35:093
354419499487,9cyclictest37-21ksoftirqd/321:15:023
354419499484,10cyclictest37-21ksoftirqd/323:25:013
3544194994812,5cyclictest131rcu_preempt19:50:003
3544190994835,9cyclictest3552542-21cat21:50:022
3544190994835,9cyclictest3552542-21cat21:50:012
3544190994835,10cyclictest3550128-21sshd21:02:182
3544190994835,10cyclictest3545425-21munin-run19:35:012
3544190994834,10cyclictest3556396-21munin-run23:05:012
3544190994834,10cyclictest3546458-21munin-run19:55:002
3544190994833,11cyclictest3556869-21sshd23:11:072
3544190994828,16cyclictest1917536-21snmpd21:25:172
3544190994827,16cyclictest1917536-21snmpd00:08:082
3544180994838,7cyclictest12-21ksoftirqd/021:40:010
3544180994816,10cyclictest12-21ksoftirqd/000:20:010
3544180994815,12cyclictest12-21ksoftirqd/022:10:010
3544180994811,13cyclictest12-21ksoftirqd/019:45:010
354419499473,9cyclictest37-21ksoftirqd/320:35:003
3544190994734,10cyclictest3550908-21sshd21:18:072
3544190994733,9cyclictest3551222-21munin-run21:25:022
3544190994732,12cyclictest3549090-21sshd20:42:072
3544190994732,11cyclictest3560561-21munin-run00:25:002
3544190994732,11cyclictest3559529-21munin-run00:05:022
3544190994732,11cyclictest3559529-21munin-run00:05:022
3544190994726,16cyclictest3558451-21sshd23:42:312
3544180994732,11cyclictest3555370-21systemctl22:45:010
3544180994729,14cyclictest3551436-21sshd21:27:220
3544180994714,12cyclictest12-21ksoftirqd/020:25:010
354419499466,9cyclictest37-21ksoftirqd/319:20:013
3544194994627,14cyclictest274-21dbus-daemon23:45:013
3544194994614,5cyclictest131rcu_preempt00:15:013
3544194994612,5cyclictest131rcu_preempt22:15:013
3544194994611,7cyclictest131rcu_preempt20:06:073
3544194994611,6cyclictest131rcu_preempt23:35:003
354419499461,16cyclictest0-21swapper/319:53:373
3544194994611,5cyclictest131rcu_preempt00:30:463
3544194994611,5cyclictest131rcu_preempt00:30:463
3544190994639,4cyclictest3557440-21/usr/sbin/munin23:24:592
3544190994634,9cyclictest3555334-21sshd22:42:272
3544190994632,10cyclictest1917536-21snmpd20:16:132
3544190994623,18cyclictest3551693-21modprobe21:33:072
3544180994642,2cyclictest0-21swapper/022:55:020
3544180994628,11cyclictest274-21dbus-daemon21:35:010
3544180994623,20cyclictest354886350sendmail-msp20:40:010
354419499458,7cyclictest131rcu_preempt20:55:073
3544194994536,6cyclictest37-21ksoftirqd/321:45:003
354419499452,8cyclictest37-21ksoftirqd/320:40:023
354419499452,6cyclictest0-21swapper/319:25:033
354419499451,9cyclictest0-21swapper/320:11:073
354419499451,6cyclictest0-21swapper/319:58:073
3544194994513,6cyclictest131rcu_preempt22:25:013
3544190994537,4cyclictest3545174-21munin-run19:29:592
3544190994532,9cyclictest48-21kcompactd022:55:432
3544190994529,12cyclictest1917536-21snmpd21:37:162
3544185994535,5cyclictest304-21in:imuxsock21:30:001
3544180994526,14cyclictest401-21lldpd19:55:020
354418099451,40cyclictest12-21ksoftirqd/000:25:000
3544180994511,18cyclictest131rcu_preempt00:30:020
354419499446,5cyclictest131rcu_preempt21:26:073
354419499443,10cyclictest37-21ksoftirqd/320:02:103
3544194994429,11cyclictest37-21ksoftirqd/323:20:023
3544194994429,11cyclictest37-21ksoftirqd/323:20:013
3544190994431,10cyclictest1917536-21snmpd23:31:192
3544185994433,8cyclictest3554304-21sshd22:24:071
3544185994430,10cyclictest274-21dbus-daemon21:45:001
354418099444,6cyclictest131rcu_preempt00:15:010
354418099442,9cyclictest12-21ksoftirqd/020:05:000
354418099441,7cyclictest131rcu_preempt21:09:590
3544180994416,5cyclictest3393576-1kworker/0:0H19:20:020
354418099441,39cyclictest3556157-21systemd-run23:00:010
354419499439,6cyclictest131rcu_preempt23:27:303
354419499439,6cyclictest131rcu_preempt23:27:293
354419499438,5cyclictest131rcu_preempt00:28:163
354419499434,10cyclictest37-21ksoftirqd/322:39:083
3544194994331,8cyclictest361rcuc/320:30:003
3544194994331,10cyclictest37-21ksoftirqd/319:39:593
354419499432,8cyclictest37-21ksoftirqd/320:55:023
3544194994324,14cyclictest235-21systemd-journal21:40:013
354419499431,9cyclictest37-21ksoftirqd/321:10:003
3544185994323,14cyclictest235-21systemd-journal19:40:001
354418099438,26cyclictest18650irq/34-mmc120:50:020
354418099433,7cyclictest131rcu_preempt23:30:020
354418099433,7cyclictest131rcu_preempt23:30:010
354418099431,38cyclictest3552973-21sshd21:57:060
3544180994313,20cyclictest131rcu_preempt23:25:010
3544180994310,5cyclictest131rcu_preempt19:35:060
354419499426,9cyclictest37-21ksoftirqd/322:45:013
354419499425,4cyclictest131rcu_preempt21:45:593
354419499425,4cyclictest131rcu_preempt21:45:583
354419499422,9cyclictest37-21ksoftirqd/319:45:013
354419499421,9cyclictest37-21ksoftirqd/300:25:013
3544194994214,6cyclictest131rcu_preempt21:02:183
3544194994212,5cyclictest131rcu_preempt00:20:023
3544185994228,10cyclictest1-21systemd22:35:011
3544185994227,11cyclictest274-21dbus-daemon00:05:021
3544185994227,11cyclictest274-21dbus-daemon00:05:011
3544185994226,12cyclictest1-21systemd20:55:001
3544185994216,20cyclictest23-21ksoftirqd/119:45:011
354418099427,14cyclictest12-21ksoftirqd/023:50:000
3544180994232,8cyclictest12-21ksoftirqd/020:15:000
3544180994230,8cyclictest12-21ksoftirqd/019:35:010
354418099422,9cyclictest12-21ksoftirqd/023:55:000
3544180994229,10cyclictest235-21systemd-journal20:20:000
354418099422,6cyclictest131rcu_preempt19:50:000
354419499412,8cyclictest37-21ksoftirqd/323:50:003
354419499411,5cyclictest131rcu_preempt23:10:023
3544194994111,5cyclictest131rcu_preempt22:00:073
354418599411,26cyclictest3548001-21kworker/u8:120:50:021
354418099418,8cyclictest12-21ksoftirqd/020:57:160
3544180994133,5cyclictest16850irq/224-800000020:00:000
354418099411,9cyclictest12-21ksoftirqd/000:35:430
3544180994114,21cyclictest131rcu_preempt21:05:020
3544180994113,6cyclictest131rcu_preempt20:55:020
354418099411,2cyclictest0-21swapper/000:05:020
354418099411,2cyclictest0-21swapper/000:05:010
3544180994112,6cyclictest131rcu_preempt20:45:000
354419499404,5cyclictest131rcu_preempt00:06:263
3544194994031,6cyclictest37-21ksoftirqd/319:30:083
3544190994023,14cyclictest3544195-21latency_hist19:10:022
354418599405,20cyclictest3548001-21kworker/u8:120:25:001
3544185994029,8cyclictest23-21ksoftirqd/122:00:001
3544185994026,11cyclictest3557170-21systemctl23:20:011
3544185994026,11cyclictest3557170-21systemctl23:20:011
3544185994026,10cyclictest3559177-21sed23:55:021
3544185994025,12cyclictest23-21ksoftirqd/119:55:031
3544185994021,15cyclictest1-21systemd00:00:011
354418599401,6cyclictest131rcu_preempt19:10:021
3544185994010,5cyclictest131rcu_preempt22:55:001
354418099409,6cyclictest131rcu_preempt21:52:080
354418099406,6cyclictest12-21ksoftirqd/021:50:010
354418099406,6cyclictest12-21ksoftirqd/021:50:000
354418099403,5cyclictest3554575-21munin-run22:30:000
3544194993928,8cyclictest37-21ksoftirqd/322:09:083
3544194993928,8cyclictest37-21ksoftirqd/300:00:093
3544194993928,8cyclictest37-21ksoftirqd/300:00:093
3544194993927,9cyclictest37-21ksoftirqd/321:16:073
3544194993925,9cyclictest361rcuc/321:59:593
354419499391,6cyclictest149-21pr/ttyS222:26:543
3544194993911,5cyclictest131rcu_preempt23:02:283
3544185993929,8cyclictest23-21ksoftirqd/120:32:081
3544185993928,8cyclictest23-21ksoftirqd/122:26:061
3544185993927,9cyclictest3553755-21sshd22:12:071
3544185993927,9cyclictest235-21systemd-journal19:16:481
3544185993926,10cyclictest23-21ksoftirqd/123:15:011
3544185993917,8cyclictest23-21ksoftirqd/100:22:091
354418599391,3cyclictest131rcu_preempt21:25:021
3544180993931,6cyclictest12-21ksoftirqd/022:30:380
3544180993930,6cyclictest12-21ksoftirqd/021:18:510
354418099392,5cyclictest131rcu_preempt23:20:000
354418099392,5cyclictest131rcu_preempt23:20:000
3544180993925,10cyclictest3552381-21wc21:45:020
3544180993921,12cyclictest274-21dbus-daemon22:50:010
3544180993921,12cyclictest274-21dbus-daemon22:50:000
354418099391,35cyclictest151rcuc/023:45:020
3544194993825,10cyclictest37-21ksoftirqd/323:15:023
3544185993830,5cyclictest304-21in:imuxsock21:05:001
3544185993829,6cyclictest3559766-21munin-plugin-st00:10:001
3544185993829,5cyclictest304-21in:imuxsock20:45:011
3544185993827,9cyclictest23-21ksoftirqd/121:10:031
3544185993826,8cyclictest0-21swapper/122:10:001
354418599381,6cyclictest0-21swapper/120:29:501
354418599381,3cyclictest131rcu_preempt00:15:011
354418099383,5cyclictest131rcu_preempt23:36:070
3544180993830,5cyclictest3544921-21/usr/sbin/munin19:25:000
354418599373,5cyclictest131rcu_preempt20:20:011
3544185993725,9cyclictest23-21ksoftirqd/100:30:021
3544185993725,8cyclictest3556102-21cron23:00:001
3544185993725,8cyclictest23-21ksoftirqd/100:40:001
3544185993720,6cyclictest131rcu_preempt20:14:591
354418599371,6cyclictest131rcu_preempt23:40:001
354418599371,30cyclictest0-21swapper/100:31:441
354418599371,30cyclictest0-21swapper/100:31:441
3544180993730,4cyclictest0-21swapper/020:35:010
354418099371,4cyclictest131rcu_preempt19:10:010
3544180993712,20cyclictest131rcu_preempt20:29:500
354419499366,5cyclictest131rcu_preempt19:10:013
3544194993629,4cyclictest3554044-21sendmail22:20:013
3544194993629,4cyclictest3554044-21sendmail22:20:013
3544194993621,11cyclictest37-21ksoftirqd/319:25:003
3544185993627,6cyclictest23-21ksoftirqd/123:45:021
3544185993625,9cyclictest23-21ksoftirqd/121:50:001
3544185993625,9cyclictest23-21ksoftirqd/121:50:001
354418599362,4cyclictest131rcu_preempt19:50:011
3544185993620,12cyclictest3544542-21sed19:15:021
354418599361,6cyclictest131rcu_preempt22:40:001
3544194993523,9cyclictest3555838-21sshd22:52:273
354418599353,5cyclictest131rcu_preempt19:59:591
3544185993526,6cyclictest3557755-21latency_hist23:30:021
3544185993526,6cyclictest3557755-21latency_hist23:30:011
3544185993526,4cyclictest23-21ksoftirqd/123:04:021
3544185993523,8cyclictest235-21systemd-journal23:48:081
3544185993522,9cyclictest274-21dbus-daemon21:15:011
3544185993522,10cyclictest23-21ksoftirqd/119:35:001
3544185993521,10cyclictest23-21ksoftirqd/120:02:111
3544185993517,5cyclictest131rcu_preempt22:45:081
3544185993517,5cyclictest131rcu_preempt22:45:081
354418599348,6cyclictest23-21ksoftirqd/119:25:031
354418599345,6cyclictest131rcu_preempt21:40:011
354418599343,7cyclictest0-21swapper/119:30:001
3544185993421,10cyclictest23-21ksoftirqd/100:20:011
3544185993414,6cyclictest131rcu_preempt21:52:071
354418099341,29cyclictest0-21swapper/023:10:420
354418599338,13cyclictest23-21ksoftirqd/123:34:071
3544185993318,11cyclictest3549861-21cron21:00:011
354418599331,5cyclictest131rcu_preempt22:42:261
354418599331,3cyclictest131rcu_preempt20:10:031
354418599323,5cyclictest131rcu_preempt21:19:061
3544185993019,9cyclictest23-21ksoftirqd/122:05:011
3544185992919,7cyclictest3548816-21sshd20:38:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional