You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-14 - 03:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Mon Oct 14, 2024 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
110332239161,26sleep00-21swapper/019:09:410
108362239162,25sleep20-21swapper/219:07:092
110362232164,54sleep30-21swapper/319:09:453
107532202164,25sleep10-21swapper/119:06:031
1208821680,4sleep11115099cyclictest21:35:211
989521350,4sleep11115099cyclictest21:30:201
11155991080,106cyclictest0-21swapper/223:33:432
111559910298,3cyclictest0-21swapper/200:07:212
111559910096,3cyclictest0-21swapper/220:45:502
11155997559,2cyclictest0-21swapper/223:47:122
197532720,4sleep081ktimersoftd/000:20:180
300902620,2sleep10-21swapper/123:30:181
87152580,4sleep21115599cyclictest22:40:262
168392540,3sleep31116199cyclictest20:35:013
181522520,3sleep31116199cyclictest21:50:163
45112510,4sleep230-21rcuc/221:20:132
1114399378,13cyclictest2671-21munin-run22:30:010
1115599352,26cyclictest19577-21kworker/2:223:21:572
1115599352,26cyclictest19577-21kworker/2:200:01:072
1115599352,25cyclictest16288-21kworker/2:021:53:322
1115599342,26cyclictest19577-21kworker/2:222:48:002
1115599342,25cyclictest16288-21kworker/2:021:40:342
1115599342,25cyclictest16288-21kworker/2:021:26:362
11155993418,8cyclictest19577-21kworker/2:222:28:142
1115599341,20cyclictest19577-21kworker/2:223:53:152
1115599332,24cyclictest19577-21kworker/2:223:27:522
1115599332,20cyclictest16288-21kworker/2:021:33:312
11155993317,4cyclictest16288-21kworker/2:020:08:142
11155993219,3cyclictest16288-21kworker/2:019:29:342
11155993218,3cyclictest16288-21kworker/2:020:03:502
11155993218,3cyclictest16288-21kworker/2:020:03:492
1115599321,25cyclictest19577-21kworker/2:221:55:062
1115599321,22cyclictest19577-21kworker/2:223:57:492
1115599312,20cyclictest19577-21kworker/2:223:16:442
1115599312,20cyclictest19577-21kworker/2:223:16:442
1115599312,20cyclictest19577-21kworker/2:222:00:502
1115599312,13cyclictest16288-21kworker/2:019:15:112
1115599312,12cyclictest19577-21kworker/2:200:10:492
11155993119,2cyclictest19577-21kworker/2:200:29:552
1115599311,10cyclictest19577-21kworker/2:223:01:422
1115599302,21cyclictest19577-21kworker/2:222:51:282
1115599302,19cyclictest16288-21kworker/2:020:12:382
1115599302,14cyclictest19577-21kworker/2:200:36:272
1115599301,12cyclictest16288-21kworker/2:021:08:502
1115599301,12cyclictest16288-21kworker/2:021:08:492
1115099301,6cyclictest86750irq/128-enp1s0-23:48:501
1115599292,19cyclictest19577-21kworker/2:222:59:042
1115599292,13cyclictest19577-21kworker/2:222:17:122
1115599292,12cyclictest16288-21kworker/2:021:36:002
1115599292,11cyclictest16288-21kworker/2:019:46:362
11155992919,2cyclictest16288-21kworker/2:020:34:382
1115599291,26cyclictest19577-21kworker/2:223:36:002
1115599291,20cyclictest19577-21kworker/2:200:24:152
1115599282,9cyclictest16288-21kworker/2:021:11:002
1115599282,16cyclictest19577-21kworker/2:223:06:242
1115599282,11cyclictest16288-21kworker/2:021:01:072
1115599282,11cyclictest16288-21kworker/2:020:42:502
1115599281,20cyclictest19577-21kworker/2:223:44:562
1115099281,20cyclictest0-21swapper/100:31:011
1114399283,12cyclictest8029-21munin-run20:15:010
1114399283,12cyclictest16827-21munin-run20:35:010
1114399281,12cyclictest17446-21awk21:50:010
1116199271,24cyclictest0-21swapper/323:15:253
1116199271,24cyclictest0-21swapper/323:15:253
1116199271,23cyclictest8967-21ntp_states23:55:213
1115599272,18cyclictest19577-21kworker/2:222:23:002
1115599272,11cyclictest19577-21kworker/2:200:31:132
1115599272,11cyclictest16288-21kworker/2:019:34:182
11155992719,3cyclictest16288-21kworker/2:020:17:102
11155992719,2cyclictest19577-21kworker/2:223:12:402
11155992716,3cyclictest19577-21kworker/2:222:34:142
11155992711,3cyclictest16288-21kworker/2:020:37:262
1115599271,11cyclictest16288-21kworker/2:019:58:502
1115099271,20cyclictest0-21swapper/122:48:391
177792260,4sleep3401ktimersoftd/300:15:193
1116199269,5cyclictest15655-21expr21:45:123
1116199261,23cyclictest401ktimersoftd/300:35:153
11155992618,3cyclictest19577-21kworker/2:200:16:052
1115599261,11cyclictest19577-21kworker/2:222:37:242
1115599261,11cyclictest16288-21kworker/2:020:25:262
1115099261,16cyclictest14634-21snmpd20:38:321
1115099261,14cyclictest0-21swapper/123:35:031
1114399262,12cyclictest7678-21munin-run23:55:010
11143992617,6cyclictest2112250/usr/sbin/munin00:25:130
1116199251,22cyclictest0-21swapper/323:05:173
1116199251,22cyclictest0-21swapper/321:05:233
1116199251,22cyclictest0-21swapper/321:05:233
1116199251,21cyclictest20559-21cpuspeed_turbos23:10:153
11161992510,3cyclictest29100-21ntp_states21:00:233
1115599252,12cyclictest16288-21kworker/2:021:47:392
11155992518,2cyclictest16288-21kworker/2:020:20:542
1115599251,11cyclictest19577-21kworker/2:222:05:102
1115599251,10cyclictest16288-21kworker/2:019:51:112
11155992510,3cyclictest16288-21kworker/2:020:57:092
11155992510,3cyclictest16288-21kworker/2:020:52:382
1115099256,11cyclictest14256-21sshd20:25:281
1115099254,15cyclictest23704-40dnf22:01:031
11150992519,3cyclictest30632-21munin-run22:20:021
11150992514,4cyclictest1294-21/usr/sbin/munin23:40:241
1115099251,19cyclictest0-21swapper/121:11:081
1115099251,17cyclictest0-21swapper/121:50:211
1114399255,17cyclictest25297-21munin-run00:35:010
1114399251,13cyclictest12112-21awk00:05:000
1114399251,11cyclictest8874-21gltestperf20:15:160
1116199242,19cyclictest3858-21/usr/sbin/munin20:05:183
1116199241,21cyclictest401ktimersoftd/322:45:213
1116199241,21cyclictest401ktimersoftd/322:20:113
1116199241,21cyclictest401ktimersoftd/320:55:243
1116199241,21cyclictest401ktimersoftd/320:10:223
11161992411,2cyclictest26247-21awk22:10:013
1116199241,11cyclictest14237-21/usr/sbin/munin19:15:193
1116199240,22cyclictest0-21swapper/319:20:263
1115599245,3cyclictest19577-21kworker/2:222:11:572
1115599245,3cyclictest19577-21kworker/2:222:11:572
11155992414,2cyclictest16288-21kworker/2:019:40:222
11155992411,3cyclictest16288-21kworker/2:019:37:102
11150992414,3cyclictest17373-21taskset21:49:261
1115099241,17cyclictest0-21swapper/121:41:301
1115099241,17cyclictest0-21swapper/121:00:271
1115099241,17cyclictest0-21swapper/100:36:081
1115099241,16cyclictest0-21swapper/100:15:191
1115099241,14cyclictest0-21swapper/121:25:131
1115099240,19cyclictest0-21swapper/121:21:281
1114399245,16cyclictest19646-21basename21:55:000
1114399241,11cyclictest23406-21munin-run20:50:000
1116199239,3cyclictest16380-21sshd00:13:363
1116199234,16cyclictest41-21ksoftirqd/323:25:123
1116199231,9cyclictest0-21swapper/323:49:283
1116199231,20cyclictest401ktimersoftd/322:30:173
1116199231,20cyclictest401ktimersoftd/321:10:273
1116199231,11cyclictest27059-21sshd19:43:293
1116199231,11cyclictest11346-21/usr/sbin/munin19:10:223
1116199231,10cyclictest15109-21gunzip22:55:223
1116199231,10cyclictest0-21swapper/320:45:263
1115599239,3cyclictest16288-21kworker/2:021:15:172
11155992310,3cyclictest16288-21kworker/2:019:21:502
1115099239,4cyclictest3637-21munin-run20:05:011
1115099239,4cyclictest3637-21munin-run20:05:011
1115099239,2cyclictest0-21swapper/120:50:151
1115099234,10cyclictest0-21swapper/121:16:541
1115099232,8cyclictest19035-21kworker/1:121:05:261
1115099232,8cyclictest19035-21kworker/1:121:05:261
11150992317,3cyclictest6518-21irqstats23:50:181
11150992317,3cyclictest0-21swapper/100:00:181
1115099231,19cyclictest28678-21/usr/sbin/munin19:45:241
1115099231,16cyclictest0-21swapper/123:10:461
1115099231,16cyclictest0-21swapper/120:45:471
1115099231,16cyclictest0-21swapper/120:23:071
1115099231,16cyclictest0-21swapper/119:10:281
1114399233,2cyclictest0-21swapper/023:02:530
1114399233,16cyclictest13477-21cat19:15:020
1114399231,12cyclictest24053-21awk22:05:000
1114399230,10cyclictest20612-21awk19:30:010
1116199228,3cyclictest14634-21snmpd00:26:273
1116199227,3cyclictest9166-21memory20:15:193
1116199223,17cyclictest7059-21sadc22:40:013
1116199221,19cyclictest401ktimersoftd/319:25:253
1116199221,10cyclictest814-21sshd19:39:293
1116199221,10cyclictest30122-21grep23:30:183
1116199221,10cyclictest29715-21ntpq22:15:213
1116199221,10cyclictest20505-21smart_sda20:40:253
1116199221,10cyclictest13598-21switchtime00:05:233
1115599228,2cyclictest16288-21kworker/2:019:14:292
1115099225,9cyclictest86750irq/128-enp1s0-23:25:221
1115099225,7cyclictest197552grep00:20:191
1115099224,6cyclictest602-21/usr/sbin/munin22:25:281
1115099224,2cyclictest86850irq/129-enp1s0-23:00:231
1115099223,17cyclictest0-21swapper/122:58:391
11150992218,3cyclictest0-21swapper/123:21:301
1115099221,3cyclictest9576-21switchtime20:15:251
1115099221,3cyclictest26034-21irqstats19:40:181
1115099221,16cyclictest10725-21kworker/u8:122:33:581
1115099221,16cyclictest0-21swapper/122:12:281
1115099221,16cyclictest0-21swapper/122:12:281
1115099221,16cyclictest0-21swapper/119:21:571
1115099221,15cyclictest0-21swapper/122:40:101
1115099221,15cyclictest0-21swapper/122:39:491
1115099221,15cyclictest0-21swapper/120:31:191
1115099221,15cyclictest0-21swapper/100:27:191
1115099221,13cyclictest0-21swapper/122:21:531
1114399227,6cyclictest0-21swapper/021:36:480
1114399221,7cyclictest32208-21diskmemload22:55:180
1114399221,6cyclictest0-21swapper/023:25:000
11143992216,3cyclictest27668-21grep23:25:150
1116199217,4cyclictest0-21swapper/322:41:053
1116199217,2cyclictest13913-21smart_sda20:25:243
1116199216,4cyclictest86550irq/126-enp1s0-20:35:233
1116199212,3cyclictest0-21swapper/320:51:133
1116199211,7cyclictest780-21rngd23:03:393
1116199211,11cyclictest0-21swapper/319:30:213
1116199211,10cyclictest14634-21snmpd21:41:323
1116199210,20cyclictest0-21swapper/322:10:233
1116199210,20cyclictest0-21swapper/322:10:233
1115099216,9cyclictest0-21swapper/119:25:211
1115099215,9cyclictest0-21swapper/123:56:501
1115099215,10cyclictest0-21swapper/100:05:231
11150992117,3cyclictest0-21swapper/120:55:221
1115099211,18cyclictest0-21swapper/123:05:061
1115099211,12cyclictest0-21swapper/122:08:071
1115099211,11cyclictest6403-21diskstats20:10:131
1115099210,15cyclictest0-21swapper/120:42:021
1114399212,11cyclictest581-21systemd-journal21:29:290
1114399212,11cyclictest27417-21/usr/sbin/munin19:45:180
98912201,3sleep010128-21tail22:45:130
143322201,3sleep00-21swapper/021:40:220
1116199205,3cyclictest27837-21sort19:45:163
1116199204,6cyclictest41-21ksoftirqd/323:51:413
1116199202,4cyclictest1042-21fschecks_count22:25:153
1116199201,9cyclictest14634-21snmpd19:53:283
11161992014,3cyclictest7330-21tail21:25:153
1116199201,10cyclictest0-21swapper/300:33:293
1115099206,8cyclictest0-21swapper/119:30:021
1115099205,9cyclictest0-21swapper/122:50:261
11150992017,2cyclictest0-21swapper/100:10:241
1115099201,18cyclictest0-21swapper/121:57:051
1115099201,18cyclictest0-21swapper/119:18:101
1115099201,17cyclictest0-21swapper/119:39:151
1115099201,13cyclictest0-21swapper/123:17:161
1115099201,13cyclictest0-21swapper/123:17:161
1114399207,3cyclictest0-21swapper/023:08:150
1114399206,6cyclictest0-21swapper/022:23:420
1114399206,5cyclictest0-21swapper/022:16:210
1114399204,6cyclictest0-21swapper/000:08:480
1114399202,15cyclictest7960-21hddtemp_smartct22:40:150
1114399201,4cyclictest0-21swapper/022:11:270
1114399201,4cyclictest0-21swapper/022:11:270
11143992014,3cyclictest26186-21gltestperf00:35:150
1114399201,10cyclictest0-21swapper/023:36:510
11143992010,8cyclictest1997-21sshd21:14:290
1114399200,9cyclictest0-21swapper/023:59:120
1116199195,3cyclictest86850irq/129-enp1s0-00:00:263
1116199193,3cyclictest2045-21latency_hist21:15:013
1116199191,9cyclictest0-21swapper/320:22:183
1116199191,15cyclictest842-21lldpd22:51:203
1116199191,15cyclictest14634-21snmpd19:58:283
1116199191,15cyclictest11677-21grep21:35:163
1116199191,14cyclictest20446-21grep21:55:153
1115099194,9cyclictest0-21swapper/120:06:001
11150991915,3cyclictest0-21swapper/119:56:411
11150991915,3cyclictest0-21swapper/119:51:491
1114399194,9cyclictest0-21swapper/023:46:540
1114399193,7cyclictest86550irq/126-enp1s0-00:15:130
1114399192,8cyclictest842-21lldpd20:05:180
1114399191,5cyclictest0-21swapper/021:16:210
11143991913,4cyclictest24250-21/usr/sbin/munin19:35:230
1114399191,10cyclictest25223-21/usr/sbin/munin19:40:150
1114399191,10cyclictest19857-21/usr/sbin/munin21:55:240
1114399191,10cyclictest14634-21snmpd22:33:550
1114399191,10cyclictest12140-21sshd20:21:290
1114399191,10cyclictest0-21swapper/000:11:160
206172181,2sleep30-21swapper/300:21:353
1116199184,3cyclictest32208-21diskmemload23:36:233
1116199181,9cyclictest0-21swapper/321:24:473
1116199181,7cyclictest0-21swapper/321:34:243
1116199181,5cyclictest2390-21open_files23:40:213
1116199181,5cyclictest23753-21sshd22:01:293
1114399183,12cyclictest6449-21turbostat21:25:010
1114399182,3cyclictest0-21swapper/023:30:170
1114399181,9cyclictest17049-21/usr/sbin/munin20:35:220
1114399181,9cyclictest11694-21/usr/sbin/munin22:50:220
1114399181,9cyclictest0-21swapper/023:40:480
1114399181,9cyclictest0-21swapper/023:17:080
1114399181,9cyclictest0-21swapper/023:17:080
1114399181,9cyclictest0-21swapper/019:34:280
1114399181,8cyclictest0-21swapper/019:21:240
11143991811,4cyclictest611-21sensors_temp19:55:240
1114399181,14cyclictest5512-21grep22:35:160
1114399181,10cyclictest0-21swapper/021:30:540
1116199171,5cyclictest25492-21head23:20:183
1116199171,15cyclictest0-21swapper/320:02:533
1116199171,15cyclictest0-21swapper/320:02:533
1114399176,3cyclictest29613-21/usr/sbin/munin19:50:200
1114399171,8cyclictest0-21swapper/023:12:050
1114399171,6cyclictest14840-21grep19:15:160
11143991712,4cyclictest0-21swapper/022:05:140
1114399171,15cyclictest0-21swapper/021:03:510
1114399161,6cyclictest0-21swapper/020:40:360
1114399161,5cyclictest0-21swapper/020:53:290
11143991611,3cyclictest31859-21seq21:05:410
11143991611,3cyclictest31859-21seq21:05:410
11143991611,3cyclictest26301-21fschecks_time20:55:150
11143991610,4cyclictest0-21swapper/020:00:210
11143991610,4cyclictest0-21swapper/020:00:210
11143991610,3cyclictest13708-21ntpq20:25:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional