You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-06 - 05:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i7-M620 @2667 MHz, Linux 4.19.106-rt45+ (Profile)

Latency plot of shadow in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Tue Jun 06, 2023 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1600221390,1sleep01005799cyclictest23:11:390
10005211894,19sleep00-21swapper/019:09:400
99652115104,6sleep20-21swapper/219:09:092
9839210790,12sleep10-21swapper/119:07:321
9826210689,12sleep30-21swapper/319:07:223
30372780,1sleep0101ktimersoftd/020:15:260
61432700,0sleep00-21swapper/023:42:170
6762660,0sleep10-21swapper/121:24:501
213282630,5sleep31006099cyclictest23:20:013
1006099210,19cyclictest0-21swapper/321:45:183
110122200,0sleep2301ktimersoftd/221:37:532
1005899200,18cyclictest0-21swapper/120:55:131
1005899200,18cyclictest0-21swapper/120:55:131
10059991714,2cyclictest8090-21kworker/u8:022:19:062
1005899171,3cyclictest0-21swapper/119:45:301
1006099161,13cyclictest30848-21diskstats20:05:203
10059991613,2cyclictest24270-21kworker/u8:000:22:042
1005999161,13cyclictest28933-21which22:45:242
1005999161,13cyclictest25871-21ssh23:25:242
10060991512,2cyclictest0-21swapper/323:43:573
1006099150,13cyclictest0-21swapper/322:35:163
1006099150,13cyclictest0-21swapper/300:21:563
1005999150,13cyclictest25245-21ps22:40:322
1005999150,13cyclictest22120-21diskmemload21:11:362
1005899152,6cyclictest0-21swapper/123:20:001
1005899151,3cyclictest25460-21grep19:50:151
1005899151,12cyclictest18800-21apt-get00:00:191
10058991510,3cyclictest18639-21sendmail-mta21:00:021
10058991510,3cyclictest18639-21sendmail-mta21:00:021
1005899150,3cyclictest0-21swapper/123:49:081
1005899150,13cyclictest25734-21if_eth023:25:231
1005899150,13cyclictest0-21swapper/123:55:281
1005899150,13cyclictest0-21swapper/122:49:341
154592140,3sleep215466-21ssh00:39:362
133102140,1sleep30-21swapper/322:25:173
113532140,1sleep00-21swapper/021:38:410
10060991411,2cyclictest0-21swapper/322:01:463
10060991411,2cyclictest0-21swapper/321:14:263
1006099140,2cyclictest0-21swapper/322:47:463
1006099140,13cyclictest0-21swapper/322:06:563
1006099140,12cyclictest9391-21munin-node3
1006099140,12cyclictest22120-21diskmemload21:21:463
1006099140,12cyclictest0-21swapper/323:11:353
1006099140,12cyclictest0-21swapper/322:17:123
1006099140,12cyclictest0-21swapper/321:35:333
1006099140,12cyclictest0-21swapper/300:10:183
1005999141,3cyclictest23402-21ssh22:38:352
10059991411,2cyclictest29995-21kworker/u8:022:55:052
10059991411,2cyclictest22933-21kworker/u8:022:11:042
10059991411,2cyclictest22120-21diskmemload23:17:572
10059991411,2cyclictest16975-21kworker/u8:223:22:042
10059991411,2cyclictest0-21swapper/221:31:062
10059991411,2cyclictest0-21swapper/221:21:362
1005999141,11cyclictest0-21swapper/219:30:232
10059991410,2cyclictest0-21swapper/221:00:012
10059991410,2cyclictest0-21swapper/221:00:002
1005999140,3cyclictest0-21swapper/200:18:022
1005999140,2cyclictest0-21swapper/222:25:262
1005999140,13cyclictest0-21swapper/221:50:372
1005999140,13cyclictest0-21swapper/221:05:262
1005999140,12cyclictest0-21swapper/221:56:322
1005999140,12cyclictest0-21swapper/219:35:322
1005999140,12cyclictest0-21swapper/200:00:492
1005899149,3cyclictest4774-21awk23:40:261
1005899142,7cyclictest0-21swapper/121:45:011
1005899140,2cyclictest0-21swapper/122:03:301
1005899140,12cyclictest22120-21diskmemload23:50:481
1005899140,12cyclictest22120-21diskmemload21:25:421
1005899140,12cyclictest0-21swapper/123:31:441
1005899140,12cyclictest0-21swapper/123:02:581
1005899140,12cyclictest0-21swapper/121:59:561
10057991411,2cyclictest18251-21kworker/u8:100:04:040
1005799140,12cyclictest23126-21latency_hist21:55:000
1005799140,12cyclictest22120-21diskmemload21:43:250
1005799140,12cyclictest10184-21processes20:35:190
1005799140,12cyclictest0-21swapper/022:56:580
1005799140,12cyclictest0-21swapper/021:45:230
1005799140,12cyclictest0-21swapper/020:20:230
315742130,2sleep022120-21diskmemload00:16:220
123972130,2sleep022120-21diskmemload22:23:420
1006099138,3cyclictest3933-21runrttasks00:35:273
1006099138,3cyclictest1222-21cat23:35:333
1006099131,10cyclictest16873-21ssh22:30:023
10060991310,2cyclictest23217-21apt-get19:45:163
10060991310,2cyclictest22120-21diskmemload23:32:083
10060991310,2cyclictest22120-21diskmemload22:11:153
10060991310,2cyclictest0-21swapper/323:55:383
10060991310,2cyclictest0-21swapper/323:50:033
10060991310,2cyclictest0-21swapper/323:45:483
10060991310,2cyclictest0-21swapper/321:58:163
10060991310,2cyclictest0-21swapper/321:43:343
10060991310,2cyclictest0-21swapper/321:32:203
10060991310,2cyclictest0-21swapper/300:01:053
1006099130,2cyclictest22120-21diskmemload22:20:243
1006099130,2cyclictest0-21swapper/323:00:253
1006099130,2cyclictest0-21swapper/300:33:263
1006099130,2cyclictest0-21swapper/300:16:173
1006099130,12cyclictest0-21swapper/323:28:363
1006099130,12cyclictest0-21swapper/323:20:193
1006099130,12cyclictest0-21swapper/322:59:143
1006099130,12cyclictest0-21swapper/321:53:113
1006099130,12cyclictest0-21swapper/320:45:003
1006099130,11cyclictest27786-21ssh21:16:123
1006099130,11cyclictest22120-21diskmemload00:05:523
1006099130,11cyclictest0-21swapper/320:20:183
10059991310,2cyclictest22120-21diskmemload23:39:042
10059991310,2cyclictest0-21swapper/223:55:232
10059991310,2cyclictest0-21swapper/223:50:162
10059991310,2cyclictest0-21swapper/223:45:302
10059991310,2cyclictest0-21swapper/223:41:102
10059991310,2cyclictest0-21swapper/223:32:222
10059991310,2cyclictest0-21swapper/222:09:262
10059991310,2cyclictest0-21swapper/220:30:282
1005999130,2cyclictest0-21swapper/223:04:022
1005999130,2cyclictest0-21swapper/222:00:362
1005999130,2cyclictest0-21swapper/219:20:222
1005999130,12cyclictest22120-21diskmemload21:15:122
1005999130,12cyclictest0-21swapper/222:50:202
1005999130,12cyclictest0-21swapper/221:45:162
1005999130,12cyclictest0-21swapper/221:25:272
1005999130,11cyclictest12204-21ssh21:40:062
1005999130,11cyclictest0-21swapper/200:25:502
1005899138,3cyclictest12848-21ntp_states19:15:191
1005899137,5cyclictest0-21swapper/121:37:321
1005899132,6cyclictest0-21swapper/122:54:591
1005899132,6cyclictest0-21swapper/122:10:001
1005899131,6cyclictest0-21swapper/100:25:001
10058991310,2cyclictest22120-21diskmemload22:29:041
10058991310,2cyclictest22120-21diskmemload21:46:071
10058991310,2cyclictest0-21swapper/123:35:341
10058991310,2cyclictest0-21swapper/122:55:231
10058991310,2cyclictest0-21swapper/122:44:151
10058991310,2cyclictest0-21swapper/122:07:551
10058991310,2cyclictest0-21swapper/100:21:541
10058991310,2cyclictest0-21swapper/100:13:171
10058991310,2cyclictest0-21swapper/100:05:201
1005899130,2cyclictest22120-21diskmemload22:33:271
1005899130,2cyclictest0-21swapper/123:15:201
1005899130,2cyclictest0-21swapper/122:15:221
1005899130,2cyclictest0-21swapper/121:15:171
1005899130,2cyclictest0-21swapper/120:45:221
1005899130,2cyclictest0-21swapper/119:10:121
1005899130,2cyclictest0-21swapper/100:31:501
1005899130,1cyclictest0-21swapper/123:05:591
1005899130,12cyclictest0-21swapper/123:15:001
1005899130,12cyclictest0-21swapper/122:39:261
1005899130,12cyclictest0-21swapper/121:50:141
1005899130,12cyclictest0-21swapper/121:10:001
1005899130,12cyclictest0-21swapper/100:16:171
1005899130,11cyclictest22120-21diskmemload21:31:061
10057991310,2cyclictest617-21kworker/u8:200:29:040
10057991310,2cyclictest22940-21ssh23:20:390
10057991310,2cyclictest22120-21diskmemload22:36:320
10057991310,2cyclictest22120-21diskmemload21:11:110
10057991310,2cyclictest0-21swapper/023:35:240
10057991310,2cyclictest0-21swapper/023:32:580
10057991310,2cyclictest0-21swapper/022:10:040
10057991310,2cyclictest0-21swapper/021:21:430
10057991310,2cyclictest0-21swapper/020:05:000
10057991310,2cyclictest0-21swapper/000:36:240
10057991310,2cyclictest0-21swapper/000:34:220
10057991310,2cyclictest0-21swapper/000:21:200
10057991310,2cyclictest0-21swapper/000:11:270
1005799130,2cyclictest22120-21diskmemload22:50:240
1005799130,2cyclictest10485-21cpu23:05:180
1005799130,2cyclictest0-21swapper/023:57:550
1005799130,2cyclictest0-21swapper/022:19:480
1005799130,2cyclictest0-21swapper/021:28:010
1005799130,2cyclictest0-21swapper/020:05:240
1005799130,12cyclictest7257-21kworker/u8:221:01:050
1005799130,12cyclictest7257-21kworker/u8:221:01:040
1005799130,12cyclictest17046-21ssh22:30:200
1005799130,12cyclictest0-21swapper/023:53:530
1005799130,12cyclictest0-21swapper/023:01:470
1005799130,12cyclictest0-21swapper/022:03:420
1005799130,12cyclictest0-21swapper/019:55:270
1005799130,12cyclictest0-21swapper/019:30:280
1005799130,12cyclictest0-21swapper/000:06:480
1005799130,11cyclictest22120-21diskmemload23:46:270
1005799130,11cyclictest22120-21diskmemload21:57:070
1005799130,11cyclictest0-21swapper/023:27:530
294682120,2sleep20-21swapper/220:00:222
1006099129,2cyclictest0-21swapper/320:55:193
1006099129,2cyclictest0-21swapper/320:55:183
1006099129,2cyclictest0-21swapper/320:35:003
1006099129,2cyclictest0-21swapper/319:40:283
1006099122,6cyclictest0-21swapper/321:00:293
1006099122,6cyclictest0-21swapper/321:00:293
1006099120,2cyclictest0-21swapper/322:44:423
1006099120,2cyclictest0-21swapper/300:26:013
1006099120,11cyclictest0-21swapper/323:08:553
1006099120,11cyclictest0-21swapper/322:50:213
1006099120,11cyclictest0-21swapper/321:28:043
1005999129,2cyclictest9396-21apt-get20:35:132
1005999129,2cyclictest19520-21cut21:00:272
1005999129,2cyclictest19520-21cut21:00:272
1005999120,11cyclictest22120-21diskmemload22:30:062
1005999120,11cyclictest22120-21diskmemload22:22:552
1005999120,11cyclictest0-21swapper/223:11:362
1005999120,11cyclictest0-21swapper/223:06:402
1005999120,11cyclictest0-21swapper/220:50:292
1005999120,11cyclictest0-21swapper/220:45:262
1005999120,11cyclictest0-21swapper/219:45:172
1005999120,11cyclictest0-21swapper/219:25:322
1005999120,11cyclictest0-21swapper/200:31:572
1005999120,11cyclictest0-21swapper/200:11:322
1005999120,11cyclictest0-21swapper/200:06:072
1005899129,2cyclictest0-21swapper/120:20:241
1005899127,3cyclictest19375-21wc19:35:001
1005899127,3cyclictest0-21swapper/119:20:241
1005899121,9cyclictest0-21swapper/119:40:181
1005899121,3cyclictest6234-21iostat_ios20:25:301
1005899121,3cyclictest0-21swapper/122:20:161
1005899120,2cyclictest0-21swapper/120:50:281
1005899120,2cyclictest0-21swapper/100:35:091
1005899120,11cyclictest0-21swapper/120:35:131
1005799129,2cyclictest15975-21df19:25:180
1005799129,2cyclictest0-21swapper/022:45:540
1005799129,2cyclictest0-21swapper/022:43:090
1005799128,2cyclictest32543-21grep20:10:120
1005799120,2cyclictest22120-21diskmemload21:32:010
1005799120,2cyclictest0-21swapper/021:16:430
1005799120,11cyclictest0-21swapper/023:15:570
1005799120,11cyclictest0-21swapper/022:25:080
1005799120,11cyclictest0-21swapper/021:05:250
1005799120,11cyclictest0-21swapper/020:45:330
1005799120,11cyclictest0-21swapper/019:45:320
1005799120,11cyclictest0-21swapper/019:40:370
1006099112,6cyclictest0-21swapper/319:23:583
1006099112,5cyclictest0-21swapper/319:35:003
1006099111,5cyclictest0-21swapper/320:55:003
1006099110,10cyclictest0-21swapper/319:35:343
1006099110,10cyclictest0-21swapper/319:10:383
1005999115,4cyclictest1606-21sh20:15:002
1005899111,8cyclictest0-21swapper/120:40:171
1005899111,8cyclictest0-21swapper/120:15:161
1005899111,3cyclictest32439-21munin-node1
1005899110,9cyclictest0-21swapper/119:55:141
1005799118,2cyclictest0-21swapper/019:20:270
1005799110,10cyclictest7257-21kworker/u8:220:51:050
121052100,1sleep00-21swapper/020:40:230
1006099102,6cyclictest0-21swapper/319:55:253
1006099100,7cyclictest0-21swapper/320:19:583
1006099100,7cyclictest0-21swapper/320:01:593
1005999107,2cyclictest0-21swapper/219:40:202
1005999107,2cyclictest0-21swapper/219:17:322
1005999105,3cyclictest10672-21tune2fs19:10:152
1005999101,7cyclictest30937-21cut20:05:212
1005899107,2cyclictest0-21swapper/120:30:201
1005899106,2cyclictest0-21swapper/119:25:271
1005899105,3cyclictest20493-21munin-node1
1005899105,3cyclictest0-21swapper/120:05:251
1005799107,2cyclictest0-21swapper/022:05:590
1005799100,9cyclictest7257-21kworker/u8:220:55:040
1005799100,9cyclictest7257-21kworker/u8:220:55:040
1005799100,9cyclictest7257-21kworker/u8:220:30:040
1005799100,9cyclictest3474-21kworker/u8:120:27:040
461290,1sleep30-21swapper/320:10:173
100609996,2cyclictest0-21swapper/319:50:183
100609994,3cyclictest0-21swapper/320:45:213
100609992,6cyclictest0-21swapper/321:07:583
100599996,2cyclictest3474-21kworker/u8:120:29:032
100599996,2cyclictest0-21swapper/219:55:242
100599994,3cyclictest3933-21runrttasks20:20:342
100589994,3cyclictest19579-21cpuspeed_turbos19:35:191
100589991,6cyclictest0-21swapper/120:00:161
100579996,2cyclictest0-21swapper/019:35:270
100579995,3cyclictest3933-21runrttasks19:13:500
100579990,3cyclictest121162munin-node0
100609981,6cyclictest0-21swapper/320:25:393
100609981,3cyclictest0-21swapper/319:25:183
100609980,7cyclictest0-21swapper/319:15:223
100599984,2cyclictest295-21jbd2/sda1-819:50:072
100599983,3cyclictest11437-21df_abs20:40:132
100599983,3cyclictest0-21swapper/220:15:122
100579980,7cyclictest0-21swapper/019:51:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional