You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 14:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 20, 2024 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
257499692,46cyclictest30910-21ssh23:15:182
257499669,18cyclictest131rcu_preempt21:40:142
257499532,34cyclictest22613-21ssh00:30:132
257499532,34cyclictest22613-21ssh00:30:132
256099492,3cyclictest12978-21apt-get21:25:141
2586994732,8cyclictest26866irq/58-eth023:35:153
257499461,36cyclictest0-21swapper/200:05:142
256099462,3cyclictest0-21swapper/120:35:141
256099451,2cyclictest0-21swapper/123:30:151
256099451,2cyclictest0-21swapper/123:15:181
255199453,29cyclictest12985-21ssh21:25:130
256099442,3cyclictest0-21swapper/123:55:131
256099441,12cyclictest5554-21apt-get22:20:121
2574994318,20cyclictest630-21Async19:56:242
2551994315,21cyclictest12-21ksoftirqd/022:35:140
256099422,4cyclictest0-21swapper/119:15:151
256099421,3cyclictest22418-21ssh23:10:111
256099411,4cyclictest22112-21ssh21:50:131
256099411,4cyclictest22112-21ssh21:50:131
256099411,3cyclictest14031-21apt-get21:45:161
255199413,31cyclictest22103-21ssh21:50:130
255199413,31cyclictest22103-21ssh21:50:130
258699394,21cyclictest26866irq/58-eth022:15:153
256099391,2cyclictest8044-21apt-get23:40:131
256099391,2cyclictest32169-21apt-get23:35:161
256099381,3cyclictest0-21swapper/121:55:151
256099381,3cyclictest0-21swapper/120:15:141
258699371,22cyclictest26866irq/58-eth021:35:143
257499373,22cyclictest24744-21rm23:50:142
257499372,28cyclictest0-21swapper/200:35:172
257499372,26cyclictest261rcuc/221:55:162
256099371,4cyclictest0-21swapper/122:25:141
256099371,3cyclictest5830-21apt-get22:40:141
256099371,3cyclictest0-21swapper/121:40:141
256099371,3cyclictest0-21swapper/119:40:171
256099371,2cyclictest29833-21ssh21:35:131
256099370,4cyclictest21995-21apt-get22:50:131
258699365,18cyclictest26866irq/58-eth022:35:153
257499362,24cyclictest30028-21ssh22:55:132
256099363,30cyclictest13911-21ssh23:05:131
256099362,3cyclictest0-21swapper/120:25:171
256099362,2cyclictest29910-21apt-get22:15:151
256099361,8cyclictest0-21swapper/120:50:131
256099361,3cyclictest0-21swapper/122:00:141
256099361,3cyclictest0-21swapper/119:50:191
256099361,3cyclictest0-21swapper/119:35:151
257499359,6cyclictest27-21ksoftirqd/222:20:132
256099356,25cyclictest23895-21cut19:30:241
256099354,22cyclictest21871-21ssh00:05:551
256099352,2cyclictest4649-21apt-get21:20:161
256099352,2cyclictest32051-21apt-get00:15:161
256099352,2cyclictest0-21swapper/123:50:141
256099350,3cyclictest17782-21apt-get20:00:031
255199353,23cyclictest32059-21ssh00:15:180
258699342,23cyclictest0-21swapper/323:05:133
256099341,3cyclictest0-21swapper/120:45:161
256099341,12cyclictest14707-21ssh00:01:461
256099340,7cyclictest17716-21find22:05:311
258699336,19cyclictest26866irq/58-eth022:30:153
258699335,16cyclictest26866irq/58-eth000:30:123
258699335,16cyclictest26866irq/58-eth000:30:123
257499332,27cyclictest261rcuc/221:15:142
256099336,7cyclictest131rcu_preempt00:10:131
256099332,5cyclictest0-21swapper/119:20:111
256099331,9cyclictest4924-21diskstats20:20:161
256099331,9cyclictest32182-21users20:10:321
256099331,3cyclictest0-21swapper/123:00:141
256099330,10cyclictest21565-21ssh23:46:081
256099330,10cyclictest15745-21irqstats00:25:201
258699323,19cyclictest26866irq/58-eth022:45:163
257499323,19cyclictest22046irq/40-dwc2_hso20:05:142
257499323,19cyclictest22046irq/40-dwc2_hso20:05:142
256099325,24cyclictest25982-21tr20:40:221
256099322,4cyclictest0-21swapper/119:25:141
256099321,9cyclictest23335-21ssh21:30:221
256099321,9cyclictest12673-21ssh23:21:551
256099321,8cyclictest30990-21cut00:35:241
256099321,4cyclictest0-21swapper/122:45:161
256099321,10cyclictest26088-21usage_Anybus22:30:331
258699311,19cyclictest26866irq/58-eth022:40:143
257499312,17cyclictest22046irq/40-dwc2_hso22:40:142
2574993114,10cyclictest27-21ksoftirqd/223:30:152
256099315,23cyclictest15214-21cat21:05:011
256099313,25cyclictest28466-21ssh21:15:121
256099311,3cyclictest30017-21apt-get22:55:141
256099311,10cyclictest25762-21find22:10:321
256099310,8cyclictest2782-21df19:10:181
256099310,10cyclictest20221-21ssh21:10:121
2551993119,8cyclictest32449-21apt-get20:15:130
258699305,19cyclictest26866irq/58-eth023:30:153
258699305,12cyclictest26866irq/58-eth023:15:173
258699303,17cyclictest26866irq/58-eth021:25:143
257499303,21cyclictest22046irq/40-dwc2_hso00:15:172
257499302,24cyclictest5855-21ssh22:00:152
257499302,18cyclictest22046irq/40-dwc2_hso22:45:162
256099304,23cyclictest0-21swapper/121:05:291
256099301,8cyclictest7303-21ssh00:20:161
256099301,7cyclictest26474-21xargs00:30:301
256099301,7cyclictest26474-21xargs00:30:301
256099301,6cyclictest26167-21sed20:05:311
256099301,6cyclictest26167-21sed20:05:311
256099301,6cyclictest13547-21if_eth019:55:281
256099301,3cyclictest0-21swapper/119:45:161
256099301,2cyclictest14365-21apt-get20:30:151
256099301,26cyclictest2572-21ls19:10:011
256099301,25cyclictest6278-21irqcore20:55:201
256099300,7cyclictest32346-21cut22:35:231
2586992913,10cyclictest26866irq/58-eth000:16:543
257499293,22cyclictest6882-21ssh23:20:152
257499293,18cyclictest22046irq/40-dwc2_hso20:00:152
256099298,18cyclictest15350-21/usr/sbin/munin23:25:281
255199292,22cyclictest14103-21ssh23:23:540
255199291,22cyclictest4977-21ssh23:17:540
258699287,10cyclictest26866irq/58-eth022:50:133
258699285,13cyclictest0-21swapper/322:55:133
257499284,16cyclictest22046irq/40-dwc2_hso19:10:162
257499284,16cyclictest22046irq/40-dwc2_hso00:10:132
257499283,16cyclictest0-21swapper/223:00:142
257499282,21cyclictest22046irq/40-dwc2_hso22:50:152
257499282,19cyclictest0-21swapper/219:25:152
257499282,19cyclictest0-21swapper/200:25:142
255199283,6cyclictest12-21ksoftirqd/022:15:010
255199283,21cyclictest4660-21ssh21:20:180
257499273,18cyclictest0-21swapper/221:25:142
257499273,14cyclictest22046irq/40-dwc2_hso00:00:132
257499272,15cyclictest32191-21ssh23:35:162
257499272,11cyclictest15387-21ssh23:25:152
255199273,13cyclictest21953-21ssh22:30:140
257499268,13cyclictest0-21swapper/221:45:162
257499266,10cyclictest0-21swapper/222:15:162
257499264,13cyclictest22046irq/40-dwc2_hso19:45:162
257499263,15cyclictest22046irq/40-dwc2_hso20:25:162
257499262,9cyclictest0-21swapper/219:40:172
257499262,5cyclictest0-21swapper/221:50:132
257499262,5cyclictest0-21swapper/221:50:132
255199264,6cyclictest12-21ksoftirqd/022:29:380
255199263,13cyclictest29927-21sh22:15:160
255199261,21cyclictest32192-21ssh23:35:160
255199261,19cyclictest0-21swapper/020:35:140
2586992514,7cyclictest26866irq/58-eth023:50:143
2586992510,11cyclictest26866irq/58-eth023:20:143
257499252,8cyclictest27539-21ssh23:10:552
255199254,9cyclictest12-21ksoftirqd/023:55:120
255199254,4cyclictest12-21ksoftirqd/019:55:330
255199253,7cyclictest13172-21ssh23:04:310
255199252,5cyclictest12-21ksoftirqd/023:00:010
255199252,18cyclictest8064-21ssh23:40:130
2551992519,4cyclictest151rcuc/021:16:550
2586992411,8cyclictest26866irq/58-eth021:15:133
257499246,9cyclictest21493-21cron22:10:012
257499246,10cyclictest22046irq/40-dwc2_hso20:15:132
255199245,4cyclictest12-21ksoftirqd/023:25:200
255199244,5cyclictest12-21ksoftirqd/023:50:350
255199242,17cyclictest24473-21ssh00:10:140
255199242,17cyclictest16367-21rm23:45:150
255199241,8cyclictest12-21ksoftirqd/000:01:400
2551992418,4cyclictest151rcuc/022:21:470
2551992418,4cyclictest151rcuc/021:35:380
2551992418,4cyclictest151rcuc/019:50:340
255199241,7cyclictest12-21ksoftirqd/019:30:330
255199241,4cyclictest12-21ksoftirqd/000:35:250
255199241,18cyclictest21413-21ssh21:30:140
258699239,9cyclictest26866irq/58-eth021:30:143
258699232,11cyclictest0-21swapper/320:05:143
258699232,11cyclictest0-21swapper/320:05:143
257499234,13cyclictest22046irq/40-dwc2_hso21:35:132
257499232,8cyclictest0-21swapper/220:45:172
257499232,16cyclictest0-21swapper/223:08:552
2574992314,6cyclictest868-21snmpd23:40:212
255199238,11cyclictest21540-21cron19:30:010
255199234,6cyclictest12-21ksoftirqd/021:44:050
255199234,5cyclictest12-21ksoftirqd/022:00:230
255199234,5cyclictest12-21ksoftirqd/020:10:010
255199234,5cyclictest12-21ksoftirqd/020:10:010
255199234,5cyclictest12-21ksoftirqd/019:10:250
255199234,4cyclictest12-21ksoftirqd/023:34:440
255199233,8cyclictest131rcu_preempt23:06:140
255199233,5cyclictest12-21ksoftirqd/023:12:300
255199232,5cyclictest12-21ksoftirqd/020:20:200
255199232,4cyclictest12-21ksoftirqd/022:46:460
255199231,7cyclictest12-21ksoftirqd/000:34:250
255199231,7cyclictest12-21ksoftirqd/000:34:250
2586992213,6cyclictest0-21swapper/300:35:233
257499229,5cyclictest26330-21idleruntime-cro19:35:012
257499224,14cyclictest14596-21idleruntime20:30:182
2574992211,9cyclictest0-21swapper/222:10:162
2574992211,8cyclictest21783-21irqcore21:30:192
257499221,15cyclictest22046irq/40-dwc2_hso23:55:122
255199228,10cyclictest151rcuc/022:40:330
255199224,5cyclictest12-21ksoftirqd/019:20:310
255199224,5cyclictest12-21ksoftirqd/019:10:010
255199224,4cyclictest12-21ksoftirqd/019:35:160
255199223,4cyclictest12-21ksoftirqd/000:05:190
255199222,16cyclictest15318-21apt-get21:05:160
255199221,8cyclictest12-21ksoftirqd/022:50:180
2551992217,4cyclictest151rcuc/021:11:100
2551992217,4cyclictest151rcuc/000:20:470
2551992217,3cyclictest151rcuc/021:00:160
2551992216,4cyclictest151rcuc/021:45:110
2551992216,4cyclictest151rcuc/020:55:310
2551992214,4cyclictest7477-21apt-get19:15:150
2551992213,3cyclictest13885-21apt-get22:05:130
255199221,18cyclictest27801-21uniq20:10:150
258699211,12cyclictest0-21swapper/321:50:133
258699211,12cyclictest0-21swapper/321:50:133
257499214,9cyclictest29677-21kthreadcore19:35:262
257499212,9cyclictest4040-21kthreadcore20:50:292
257499211,17cyclictest8796-21ssh21:20:352
257499211,17cyclictest28127-21latency_hist21:15:012
257499211,14cyclictest19293-21cut20:35:182
257499211,10cyclictest9334-21tr19:15:252
257499211,10cyclictest27572-21awk20:40:302
257499211,10cyclictest11233-21tail00:20:322
257499210,18cyclictest12359-21cut19:20:152
255199219,5cyclictest12-21ksoftirqd/021:55:320
255199214,3cyclictest131rcu_preempt00:25:210
2551992115,3cyclictest18130-21apt-get20:00:150
255199211,16cyclictest2232-21ls19:40:270
258699209,6cyclictest0-21swapper/322:00:523
257499207,3cyclictest131rcu_preempt19:50:222
257499201,16cyclictest20430-21tail23:45:332
257499201,15cyclictest22046irq/40-dwc2_hso22:35:142
257499201,10cyclictest4987-21fschecks_time20:20:172
255199203,4cyclictest12-21ksoftirqd/019:45:340
255199203,3cyclictest12-21ksoftirqd/020:30:170
258699199,3cyclictest0-21swapper/322:22:233
258699198,7cyclictest26866irq/58-eth021:10:143
258699198,7cyclictest0-21swapper/323:00:133
258699194,11cyclictest0-21swapper/300:10:143
258699193,9cyclictest0-21swapper/319:15:153
258699193,11cyclictest0-21swapper/323:14:543
258699192,11cyclictest0-21swapper/320:35:153
258699192,11cyclictest0-21swapper/319:25:153
258699191,6cyclictest0-21swapper/323:45:143
2586991911,4cyclictest0-21swapper/321:43:563
258699191,12cyclictest0-21swapper/323:40:163
258699191,11cyclictest0-21swapper/319:55:223
2586991910,6cyclictest0-21swapper/323:30:003
2586991910,6cyclictest0-21swapper/322:25:363
2586991910,6cyclictest0-21swapper/300:06:533
2586991910,6cyclictest0-21swapper/300:02:543
257499196,10cyclictest17628-21timerwakeupswit22:25:312
257499191,15cyclictest22788-21cut22:30:212
257499191,15cyclictest17813-21idleruntime-cro20:00:012
257499191,10cyclictest9165-21ntp_192.168.11420:55:262
255199194,3cyclictest131rcu_preempt20:45:210
255199191,5cyclictest12-21ksoftirqd/020:30:010
258699189,6cyclictest26866irq/58-eth022:05:123
258699189,6cyclictest0-21swapper/323:56:173
258699189,6cyclictest0-21swapper/321:55:533
258699182,10cyclictest0-21swapper/321:05:173
258699182,10cyclictest0-21swapper/320:20:143
258699182,10cyclictest0-21swapper/319:20:133
258699181,13cyclictest0-21swapper/320:15:303
2586991810,6cyclictest0-21swapper/321:47:303
2586991810,6cyclictest0-21swapper/300:21:423
2586991810,5cyclictest0-21swapper/322:10:223
257499183,10cyclictest10552-21acpi21:00:122
257499181,9cyclictest2537-21cyclictest19:10:012
255199184,3cyclictest131rcu_preempt20:50:260
258699179,6cyclictest0-21swapper/321:20:063
258699179,6cyclictest0-21swapper/300:26:163
258699172,10cyclictest0-21swapper/321:00:143
258699172,10cyclictest0-21swapper/320:30:153
258699172,10cyclictest0-21swapper/319:40:173
258699172,10cyclictest0-21swapper/319:35:153
258699172,10cyclictest0-21swapper/319:10:183
258699171,9cyclictest0-21swapper/320:45:013
258699162,9cyclictest0-21swapper/319:50:193
258699161,8cyclictest0-21swapper/319:45:163
2551991611,3cyclictest0-21swapper/020:40:010
258699152,7cyclictest0-21swapper/320:50:123
257499159,4cyclictest27-21ksoftirqd/221:05:262
258699149,3cyclictest0-21swapper/320:45:303
258699149,3cyclictest0-21swapper/320:25:323
258699149,3cyclictest0-21swapper/319:30:203
258699141,8cyclictest0-21swapper/320:00:153
258699141,7cyclictest0-21swapper/320:10:293
258699131,7cyclictest0-21swapper/320:55:143
258699101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional