You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 03:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Fri Apr 26, 2024 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6544419912347,12cyclictest0-21swapper/008:10:010
654441999680,10cyclictest0-21swapper/010:15:010
65444199802,75cyclictest0-21swapper/009:40:020
654441997874,2cyclictest0-21swapper/011:25:020
654441997369,2cyclictest0-21swapper/012:20:030
654441997369,2cyclictest0-21swapper/010:00:020
65444199711,66cyclictest0-21swapper/011:20:030
654441996863,3cyclictest0-21swapper/009:30:020
65444199678,30cyclictest664945-21cat10:30:020
654441996760,4cyclictest657481-1kworker/0:1H08:15:030
654441996627,11cyclictest0-21swapper/007:15:010
654441996615,17cyclictest666531-21wc11:00:020
65444199658,18cyclictest656281-21ls07:45:010
654441996459,3cyclictest0-21swapper/010:50:010
654441996211,16cyclictest657126-21date08:00:020
65444199617,45cyclictest0-21swapper/007:39:590
65444199608,19cyclictest669339-21sed11:55:010
654441995955,2cyclictest0-21swapper/011:45:020
654441995918,7cyclictest12-21ksoftirqd/012:10:010
654451995839,14cyclictest235-21systemd-journal09:20:002
654441995852,4cyclictest0-21swapper/009:25:020
65445199577,33cyclictest235-21systemd-journal11:05:022
65445199577,33cyclictest235-21systemd-journal11:05:012
654441995613,31cyclictest667111-21munin-run11:14:590
654446995515,6cyclictest131rcu_preempt07:55:001
654441995535,15cyclictest235-21systemd-journal09:20:010
65444199552,46cyclictest654904-21sendmail07:20:000
65444699541,37cyclictest653966-21kworker/u8:207:50:021
65444699541,37cyclictest653966-21kworker/u8:207:50:011
654451995312,6cyclictest131rcu_preempt09:50:012
654441995310,18cyclictest668130-21sshd11:32:500
654441995247,3cyclictest0-21swapper/010:00:010
654441995211,10cyclictest12-21ksoftirqd/009:25:010
65445699511,5cyclictest131rcu_preempt08:35:013
654451995134,12cyclictest274-21dbus-daemon07:45:002
654451995115,6cyclictest131rcu_preempt11:20:012
654441995143,4cyclictest16850irq/224-800000011:10:010
654441995133,12cyclictest304-21in:imuxsock11:39:590
654441995132,13cyclictest1-21systemd10:25:000
654441995131,16cyclictest235-21systemd-journal10:40:030
654441995130,13cyclictest235-21systemd-journal08:25:000
654441995127,5cyclictest658799-21munin-run08:35:010
654441995125,16cyclictest235-21systemd-journal10:20:010
654441995125,16cyclictest235-21systemd-journal10:20:010
654451995042,5cyclictest661946-21kworker/2:210:07:012
654451995041,6cyclictest667913-21kworker/2:011:39:552
654441995041,4cyclictest0-21swapper/012:20:020
654441995041,4cyclictest0-21swapper/012:20:020
654441995030,16cyclictest665273-21sshd10:39:490
654441995013,8cyclictest12-21ksoftirqd/008:50:020
65445199497,7cyclictest30-21ksoftirqd/212:39:492
654451994941,5cyclictest667913-21kworker/2:012:31:342
654451994941,5cyclictest667913-21kworker/2:011:46:592
654451994941,5cyclictest663247-21kworker/2:110:50:532
654451994941,5cyclictest663247-21kworker/2:110:43:302
654451994941,5cyclictest653674-21kworker/2:008:35:002
654451994938,7cyclictest30-21ksoftirqd/208:40:002
654451994937,8cyclictest30-21ksoftirqd/207:15:022
65444699492,9cyclictest131rcu_preempt08:30:011
654441994928,11cyclictest304-21in:imuxsock10:55:010
654456994826,11cyclictest655253-21cat07:25:023
65445699481,8cyclictest663216-21modprobe10:00:003
65445699481,7cyclictest37-21ksoftirqd/309:10:003
65445199487,9cyclictest30-21ksoftirqd/211:15:012
654451994841,4cyclictest667913-21kworker/2:012:08:162
654451994841,4cyclictest661946-21kworker/2:210:56:072
654451994841,4cyclictest653674-21kworker/2:007:22:572
654451994840,5cyclictest667913-21kworker/2:012:20:022
654451994840,5cyclictest667913-21kworker/2:011:58:472
654451994840,5cyclictest667913-21kworker/2:011:52:322
654451994840,5cyclictest663247-21kworker/2:110:28:172
654451994840,5cyclictest663247-21kworker/2:110:19:062
654451994840,5cyclictest663247-21kworker/2:110:19:052
654451994840,5cyclictest661946-21kworker/2:210:03:572
654451994840,5cyclictest661946-21kworker/2:209:40:032
654451994840,5cyclictest658073-21kworker/2:309:11:222
654451994840,5cyclictest653674-21kworker/2:009:50:292
654451994840,5cyclictest653674-21kworker/2:009:50:282
654451994840,5cyclictest653674-21kworker/2:009:29:232
654451994840,5cyclictest653674-21kworker/2:008:45:112
654451994840,5cyclictest653674-21kworker/2:008:40:022
654451994840,5cyclictest653674-21kworker/2:008:15:142
654451994840,5cyclictest653674-21kworker/2:008:02:072
654451994840,5cyclictest653674-21kworker/2:008:02:062
654451994840,5cyclictest653674-21kworker/2:007:35:302
654451994839,6cyclictest663247-21kworker/2:110:35:412
654451994839,5cyclictest667913-21kworker/2:012:16:252
654451994839,5cyclictest667913-21kworker/2:012:16:242
654451994838,6cyclictest667913-21kworker/2:012:11:402
654451994838,6cyclictest667913-21kworker/2:012:03:312
654451994838,6cyclictest667913-21kworker/2:012:03:312
654451994812,5cyclictest131rcu_preempt09:40:012
65444699489,6cyclictest131rcu_preempt08:40:001
65444699489,5cyclictest131rcu_preempt11:40:011
654446994831,13cyclictest274-21dbus-daemon12:30:011
65444699482,27cyclictest671279-21kworker/u8:212:40:011
65445699471,41cyclictest235-21systemd-journal08:13:493
654451994740,4cyclictest667913-21kworker/2:011:42:482
654451994740,4cyclictest663247-21kworker/2:110:20:242
654451994740,4cyclictest653674-21kworker/2:009:05:152
654451994740,4cyclictest653674-21kworker/2:008:55:322
654451994740,4cyclictest653674-21kworker/2:008:52:122
654451994740,4cyclictest653674-21kworker/2:008:14:152
654451994740,4cyclictest653674-21kworker/2:007:25:162
654451994739,5cyclictest667913-21kworker/2:012:28:132
654451994739,5cyclictest663247-21kworker/2:110:30:262
654451994739,5cyclictest661946-21kworker/2:211:30:032
654451994739,5cyclictest661946-21kworker/2:211:26:042
654451994739,5cyclictest661946-21kworker/2:211:23:042
654451994739,5cyclictest661946-21kworker/2:211:09:102
654451994739,5cyclictest661946-21kworker/2:210:12:252
654451994739,5cyclictest653674-21kworker/2:009:57:032
654451994739,5cyclictest653674-21kworker/2:009:20:162
654451994739,5cyclictest653674-21kworker/2:009:00:102
654451994739,5cyclictest653674-21kworker/2:009:00:092
654451994739,5cyclictest653674-21kworker/2:008:07:452
654451994739,5cyclictest653674-21kworker/2:007:55:402
654451994739,5cyclictest653674-21kworker/2:007:45:152
654451994739,5cyclictest653674-21kworker/2:007:45:152
654451994739,5cyclictest653674-21kworker/2:007:30:162
654451994739,5cyclictest653674-21kworker/2:007:30:152
654451994738,5cyclictest653674-21kworker/2:007:50:572
65444699477,5cyclictest131rcu_preempt09:45:011
654441994743,2cyclictest0-21swapper/009:05:020
654441994743,2cyclictest0-21swapper/009:05:010
654441994728,12cyclictest12-21ksoftirqd/008:39:590
654456994631,11cyclictest1-21systemd10:40:023
654456994628,8cyclictest37-21ksoftirqd/307:19:493
654456994622,20cyclictest37-21ksoftirqd/309:55:013
654456994622,20cyclictest37-21ksoftirqd/309:55:003
65445699461,12cyclictest658298-21munin-run08:25:003
654451994639,4cyclictest661946-21kworker/2:210:47:532
654451994639,4cyclictest653674-21kworker/2:009:30:402
654451994638,5cyclictest653674-21kworker/2:008:25:542
654451994638,5cyclictest653674-21kworker/2:008:20:052
654451994638,5cyclictest653674-21kworker/2:007:16:382
654446994625,18cyclictest23-21ksoftirqd/109:38:491
65444199469,34cyclictest0-21swapper/007:55:010
65444199464,38cyclictest0-21swapper/008:58:370
654441994633,10cyclictest235-21systemd-journal09:10:480
654441994629,13cyclictest668917-21sshd11:49:490
654441994628,13cyclictest304-21in:imuxsock09:45:020
65444699458,6cyclictest131rcu_preempt11:00:001
654446994534,8cyclictest23-21ksoftirqd/108:10:001
654446994532,10cyclictest235-21systemd-journal11:10:491
654446994527,12cyclictest274-21dbus-daemon10:05:011
654446994515,9cyclictest131rcu_preempt11:20:011
654446994513,7cyclictest131rcu_preempt08:20:001
654446994512,5cyclictest131rcu_preempt11:54:501
654441994529,12cyclictest659832-21munin-run08:54:590
654441994529,10cyclictest0-21swapper/008:30:010
654441994527,14cyclictest671281-21sshd12:30:480
654441994527,12cyclictest660629-21sendmail-mta09:10:000
654441994526,14cyclictest671603-21/usr/sbin/munin12:40:000
654441994526,11cyclictest235-21systemd-journal08:15:010
654441994524,15cyclictest1917536-21snmpd11:01:360
654441994524,15cyclictest1917536-21snmpd11:01:350
65445699443,5cyclictest131rcu_preempt07:39:593
654456994418,21cyclictest37-21ksoftirqd/312:00:023
65445699441,28cyclictest659810-21kworker/u8:309:25:023
65444699445,5cyclictest131rcu_preempt12:08:491
654446994426,14cyclictest1-21systemd10:50:011
65444699441,4cyclictest131rcu_preempt09:31:231
654441994440,2cyclictest0-21swapper/010:05:030
654441994428,13cyclictest669683-21sshd12:00:440
654441994428,13cyclictest669683-21sshd12:00:440
654441994424,15cyclictest235-21systemd-journal12:15:000
654456994328,12cyclictest667716-21cat11:25:003
65445699431,6cyclictest131rcu_preempt10:50:023
65444699438,7cyclictest23-21ksoftirqd/110:45:001
65444699435,9cyclictest23-21ksoftirqd/107:59:231
654446994328,12cyclictest1-21systemd12:25:001
65444699431,6cyclictest131rcu_preempt10:55:011
65444199434,37cyclictest0-21swapper/011:22:260
654441994330,10cyclictest669481-21munin-run12:00:010
654441994328,12cyclictest274-21dbus-daemon09:45:010
654441994327,11cyclictest655684-21munin-run07:35:010
654441994327,11cyclictest655684-21munin-run07:35:000
654441994326,13cyclictest659341-21munin-run08:45:010
654441994326,13cyclictest655413-21sshd07:29:430
654441994321,13cyclictest0-21swapper/010:35:010
654456994230,9cyclictest274-21dbus-daemon10:20:013
654456994230,9cyclictest274-21dbus-daemon10:20:013
654456994227,12cyclictest37-21ksoftirqd/307:55:003
654456994221,15cyclictest274-21dbus-daemon10:35:013
65445699421,5cyclictest131rcu_preempt08:05:013
65445699421,5cyclictest131rcu_preempt08:05:003
65444699426,6cyclictest131rcu_preempt08:32:481
65444699423,35cyclictest149-21pr/ttyS209:02:571
65444699423,35cyclictest149-21pr/ttyS209:02:571
654446994229,9cyclictest1-21systemd07:15:001
654446994219,19cyclictest23-21ksoftirqd/110:09:301
654446994218,20cyclictest23-21ksoftirqd/109:49:251
654441994221,17cyclictest671078-21unin-run12:30:000
654441994221,16cyclictest1917536-21snmpd07:49:060
654441994221,16cyclictest1917536-21snmpd07:49:050
65445699414,6cyclictest131rcu_preempt07:30:013
654456994129,9cyclictest669016-21sed11:50:023
654456994128,10cyclictest658059-21systemd-run08:20:023
654456994126,11cyclictest654601-21date07:10:023
654456994118,19cyclictest361rcuc/309:35:023
65444699416,5cyclictest131rcu_preempt11:25:001
65444699414,9cyclictest131rcu_preempt12:18:491
65444699414,9cyclictest131rcu_preempt12:18:481
65444699414,5cyclictest131rcu_preempt11:05:071
654446994118,15cyclictest23-21ksoftirqd/111:40:511
65444699411,6cyclictest0-21swapper/107:25:021
65444699411,5cyclictest131rcu_preempt09:55:011
65444699411,5cyclictest131rcu_preempt09:55:011
654446994115,8cyclictest131rcu_preempt07:35:011
654446994115,8cyclictest131rcu_preempt07:35:001
65444699411,4cyclictest131rcu_preempt10:35:021
654446994110,5cyclictest131rcu_preempt07:30:001
654456994024,11cyclictest235-21systemd-journal11:39:593
654456994011,18cyclictest37-21ksoftirqd/307:48:503
654456994011,18cyclictest37-21ksoftirqd/307:48:493
65444699409,6cyclictest131rcu_preempt11:25:491
65444699408,6cyclictest131rcu_preempt10:11:491
65444699408,5cyclictest131rcu_preempt11:46:571
65444699408,5cyclictest131rcu_preempt08:14:531
65444699407,5cyclictest131rcu_preempt10:38:051
65444699406,5cyclictest131rcu_preempt07:40:001
65444699405,7cyclictest23-21ksoftirqd/109:25:101
65444699404,32cyclictest149-21pr/ttyS208:20:131
65444699403,33cyclictest149-21pr/ttyS210:20:481
654446994028,9cyclictest656195-21sshd07:43:491
654446994022,14cyclictest23-21ksoftirqd/110:30:011
654446994019,5cyclictest23-21ksoftirqd/111:34:461
654446994018,18cyclictest23-21ksoftirqd/109:24:481
654446994011,7cyclictest131rcu_preempt09:15:021
654441994030,7cyclictest12-21ksoftirqd/007:25:020
654441994028,9cyclictest657217-21sshd08:00:500
654441994028,9cyclictest657217-21sshd08:00:490
65444199402,15cyclictest662952-21latency_hist09:55:020
65444199402,15cyclictest662952-21latency_hist09:55:010
65445699398,21cyclictest37-21ksoftirqd/308:40:503
65445699396,5cyclictest131rcu_preempt10:00:173
65445699393,6cyclictest131rcu_preempt10:30:023
654456993930,5cyclictest37-21ksoftirqd/308:00:013
654456993928,8cyclictest657505-21/usr/sbin/munin08:10:003
654456993928,8cyclictest37-21ksoftirqd/308:50:483
654456993927,9cyclictest235-21systemd-journal12:10:003
654456993925,10cyclictest1-21systemd10:15:003
654456993925,10cyclictest1-21systemd09:05:013
654456993925,10cyclictest1-21systemd09:05:003
654456993922,6cyclictest1-21systemd09:15:013
654456993920,14cyclictest1-21systemd11:00:013
65445699391,6cyclictest131rcu_preempt11:35:023
65444699399,4cyclictest131rcu_preempt08:44:121
65444699398,5cyclictest131rcu_preempt08:54:491
65444699398,4cyclictest131rcu_preempt11:56:591
65444699397,5cyclictest131rcu_preempt11:01:271
65444699397,5cyclictest131rcu_preempt11:01:271
65444699396,6cyclictest131rcu_preempt07:15:171
65444699393,5cyclictest131rcu_preempt08:55:311
654446993921,4cyclictest23-21ksoftirqd/108:49:131
65444699391,5cyclictest131rcu_preempt09:59:591
654446993910,5cyclictest131rcu_preempt12:01:051
654446993910,5cyclictest131rcu_preempt12:01:051
654456993828,7cyclictest671552-21sshd12:39:493
654456993826,9cyclictest661358-21cat09:20:033
654456993826,8cyclictest668647-21sshd11:40:413
65445699382,5cyclictest131rcu_preempt08:30:013
65445699382,13cyclictest1-21systemd08:50:013
65445699381,6cyclictest0-21swapper/307:34:453
65445699381,6cyclictest0-21swapper/307:34:443
65445699381,4cyclictest131rcu_preempt12:11:123
65444699387,4cyclictest131rcu_preempt09:07:101
654446993828,7cyclictest23-21ksoftirqd/110:20:011
654446993828,7cyclictest23-21ksoftirqd/110:20:001
654446993820,4cyclictest23-21ksoftirqd/109:19:491
654446993817,17cyclictest23-21ksoftirqd/112:13:451
65445699377,20cyclictest37-21ksoftirqd/307:42:293
65445699374,5cyclictest131rcu_preempt11:55:013
65445699373,6cyclictest131rcu_preempt09:45:013
65445699372,5cyclictest131rcu_preempt11:20:013
65445699372,5cyclictest131rcu_preempt09:40:013
654456993724,10cyclictest274-21dbus-daemon11:10:003
654456993723,10cyclictest1-21systemd09:30:003
654456993717,16cyclictest131rcu_preempt09:47:503
654456993716,18cyclictest361rcuc/312:05:013
654456993716,18cyclictest361rcuc/312:05:013
65445699371,5cyclictest131rcu_preempt10:45:003
654456993713,15cyclictest666612-21systemd-run11:05:013
654456993713,15cyclictest666612-21systemd-run11:05:003
65444699377,12cyclictest23-21ksoftirqd/108:05:021
65444699377,12cyclictest23-21ksoftirqd/108:05:011
654441993714,17cyclictest654551-21latency_hist07:10:020
654456993621,11cyclictest37-21ksoftirqd/312:29:593
65445699361,5cyclictest0-21swapper/312:20:233
654456993611,10cyclictest37-21ksoftirqd/311:15:013
654456993610,6cyclictest131rcu_preempt10:55:013
654446993624,10cyclictest23-21ksoftirqd/112:35:031
65445699358,12cyclictest131rcu_preempt12:17:003
65445699358,12cyclictest131rcu_preempt12:17:003
65445699353,5cyclictest131rcu_preempt08:55:473
654456993525,7cyclictest663712-21sshd10:06:503
65445699351,5cyclictest131rcu_preempt10:25:013
654456993515,15cyclictest131rcu_preempt08:38:493
654456993512,5cyclictest37-21ksoftirqd/311:30:013
65445699342,5cyclictest131rcu_preempt12:35:023
654451993320,10cyclictest30-21ksoftirqd/207:10:012
654446993124,5cyclictest654385-21latency_hist07:10:021
654456992919,7cyclictest37-21ksoftirqd/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional