You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 05:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Thu Apr 25, 2024 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
158002930,2sleep00-21swapper/020:34:500
2091099863,74cyclictest0-21swapper/319:04:273
2090999864,26cyclictest0-21swapper/219:37:152
2090999854,28cyclictest0-21swapper/222:39:532
2091099833,72cyclictest2078-21head17:59:593
2091099833,21cyclictest15906-21needreboot20:34:523
20910998315,11cyclictest0-21swapper/322:06:243
2090999835,25cyclictest0-21swapper/222:19:382
20909998329,11cyclictest11238-21sadc19:24:292
20910998227,48cyclictest27698-21cut17:44:433
2091099815,68cyclictest0-21swapper/320:44:263
2091099813,25cyclictest0-21swapper/320:29:263
20910998124,51cyclictest19380-21fschecks_time18:34:543
2090999815,27cyclictest0-21swapper/217:55:072
20909998017,13cyclictest0-21swapper/219:50:032
20910997928,42cyclictest0-21swapper/321:44:373
20910997924,48cyclictest30006-21open_files18:54:583
20910997922,50cyclictest0-21swapper/319:39:293
2090999794,27cyclictest0-21swapper/221:30:022
2090999794,27cyclictest0-21swapper/217:34:452
20910997830,41cyclictest6974-21needreboot22:24:533
2091099783,67cyclictest8238-21sed19:15:083
20910997826,45cyclictest14755-21fschecks_time19:29:503
20910997824,8cyclictest159401munin-node18:30:033
20910997824,47cyclictest29773-21aten_r1power_en21:04:353
20910997822,50cyclictest0-21swapper/318:29:323
2091099774,19cyclictest0-21swapper/321:29:443
20910997712,12cyclictest0-21swapper/319:40:053
2090999774,27cyclictest0-21swapper/221:02:282
20903997720,39cyclictest0-21swapper/017:40:530
2091099764,21cyclictest0-21swapper/321:09:503
2091099763,28cyclictest27404-21needreboot18:49:593
20910997628,42cyclictest2219-21grep19:04:483
20910997628,42cyclictest2219-21grep19:04:483
20910997625,43cyclictest0-21swapper/318:19:313
20910997625,43cyclictest0-21swapper/318:19:313
20910997624,46cyclictest21885-21fschecks_time17:24:523
20910997624,46cyclictest21885-21fschecks_time17:24:523
20910997621,9cyclictest3427-21cpuspeed_turbos22:19:353
20910997621,11cyclictest31289-21cut17:54:493
20910997617,9cyclictest4515-21fschecks_count19:09:463
2090999765,26cyclictest0-21swapper/219:32:152
2090999762,25cyclictest0-21swapper/222:00:562
2091099754,21cyclictest0-21swapper/318:09:483
20910997530,39cyclictest12830-21cut19:24:583
2091099753,64cyclictest0-21swapper/320:09:493
2091099753,22cyclictest0-21swapper/321:59:423
2091099753,22cyclictest0-21swapper/320:59:353
2091099753,22cyclictest0-21swapper/319:23:463
2091099753,21cyclictest0-21swapper/319:44:373
20910997523,45cyclictest15023-21unixbench_multi22:39:583
20910997522,9cyclictest8333-21needreboot20:19:543
20910997518,9cyclictest21133-21aten_r1power_vo18:39:413
20910997517,9cyclictest11019-21cpuspeed_turbos22:34:363
2091099744,26cyclictest0-21swapper/321:34:363
2091099743,11cyclictest0-21swapper/320:54:263
2091099743,11cyclictest0-21swapper/320:54:263
20910997425,42cyclictest9158-21gltestperf22:29:463
20910997425,42cyclictest9158-21gltestperf22:29:463
20910997422,8cyclictest16021-21cpuspeed_turbos22:44:333
20910997421,10cyclictest30654-21cpuspeed_turbos22:09:333
20910997418,8cyclictest25078-21fschecks_time17:34:483
20910997418,8cyclictest16137-21gltestperf21:39:433
20910997418,8cyclictest16137-21gltestperf21:39:433
20910997330,37cyclictest1410-21gltestperf22:14:453
2091099733,63cyclictest9002-21snmpd20:57:273
2091099733,62cyclictest0-21swapper/317:44:333
2091099733,22cyclictest0-21swapper/318:05:593
20910997317,8cyclictest4141-21switchtime21:14:583
20910997315,8cyclictest31006-21unixbench_multi20:00:103
20910997315,8cyclictest31006-21unixbench_multi20:00:103
20910997312,11cyclictest0-21swapper/320:04:403
2091099724,18cyclictest0-21swapper/321:49:353
20910997234,32cyclictest26989-21grep19:54:433
20910997234,32cyclictest26989-21grep19:54:433
2091099723,22cyclictest1286-21runrttasks22:54:103
20910997218,9cyclictest20633-21gltestperf20:44:463
20910997218,9cyclictest20633-21gltestperf20:44:463
20910997216,8cyclictest6801-21timerwakeupswit21:19:583
2091099713,61cyclictest0-21swapper/320:14:513
2091099713,60cyclictest0-21swapper/318:24:323
2091099713,26cyclictest24230-21tail19:49:443
20910997123,42cyclictest23684-21head18:44:443
20910997118,9cyclictest23214-21df_abs21:54:373
20910997023,9cyclictest12107-21aten_r1power_fr20:29:363
20910997016,9cyclictest8270-21grep21:24:413
20903997018,45cyclictest0-21swapper/021:55:020
20903997018,33cyclictest0-21swapper/020:29:370
20903997017,35cyclictest0-21swapper/017:44:450
20903996921,32cyclictest6991-21munin-node20:19:470
20903996916,31cyclictest0-21swapper/019:54:380
20903996916,31cyclictest0-21swapper/019:54:380
20903996823,12cyclictest20901-21awk17:24:340
20903996822,27cyclictest25498-21fschecks_time20:54:460
2091099674,8cyclictest24237-21chrt17:34:353
20903996718,43cyclictest0-21swapper/019:24:410
2090399668,49cyclictest0-21swapper/022:14:020
20903996624,35cyclictest2591-21grep18:00:080
20903996619,32cyclictest0-21swapper/018:05:010
20903996617,42cyclictest0-21swapper/020:09:550
20903996617,32cyclictest0-21swapper/018:15:010
20903996617,32cyclictest0-21swapper/018:15:010
20903996616,42cyclictest0-21swapper/022:48:040
20903996616,35cyclictest0-21swapper/020:24:510
2091099653,13cyclictest0-21swapper/317:50:073
20903996521,37cyclictest12382-21munin-run21:34:240
20903996520,28cyclictest0-21swapper/021:27:020
20903996519,40cyclictest0-21swapper/018:40:130
20903996519,30cyclictest0-21swapper/018:54:310
20903996518,30cyclictest0-21swapper/018:34:520
20903996514,44cyclictest0-21swapper/021:09:460
20903996419,29cyclictest0-21swapper/021:04:360
20903996418,28cyclictest0-21swapper/022:34:310
20903996417,40cyclictest0-21swapper/022:04:370
20903996417,40cyclictest0-21swapper/021:19:200
20903996416,40cyclictest0-21swapper/017:28:120
20903996416,40cyclictest0-21swapper/017:28:120
20903996416,30cyclictest0-21swapper/018:14:320
20903996339,19cyclictest0-21swapper/021:01:440
20903996322,12cyclictest4341-21latency22:19:500
20903996321,34cyclictest1286-21runrttasks19:34:440
20903996319,29cyclictest0-21swapper/018:54:390
20903996317,25cyclictest19725-21sshd21:46:340
20903996316,40cyclictest0-21swapper/022:44:210
20903996316,31cyclictest0-21swapper/019:44:560
20903996217,38cyclictest0-21swapper/020:04:520
20903996217,29cyclictest0-21swapper/018:59:550
20903996216,39cyclictest0-21swapper/020:39:590
20903996216,29cyclictest0-21swapper/019:10:320
2090399618,35cyclictest0-21swapper/018:20:040
20903996116,39cyclictest0-21swapper/021:34:380
20903996115,32cyclictest0-21swapper/017:51:590
20903996115,29cyclictest0-21swapper/022:29:220
20903996112,30cyclictest0-21swapper/019:24:300
20903996110,34cyclictest0-21swapper/021:59:490
20910996015,7cyclictest20955-21latency_hist17:24:343
2090399609,34cyclictest0-21swapper/020:49:340
2090399609,34cyclictest0-21swapper/020:49:340
2090399607,47cyclictest0-21swapper/020:14:370
20903996017,28cyclictest0-21swapper/020:46:320
20903996017,28cyclictest0-21swapper/020:46:320
20903996016,37cyclictest0-21swapper/022:14:380
20903996015,30cyclictest0-21swapper/018:31:450
2090999595,47cyclictest0-21swapper/222:09:322
20909995928,25cyclictest964-21latency_hist19:04:302
20907995941,10cyclictest26236-21gltestperf21:59:441
2090399599,44cyclictest0-21swapper/022:49:560
2090399596,37cyclictest0-21swapper/018:46:490
20903995918,35cyclictest7331-21munin-run21:24:240
20903995916,28cyclictest0-21swapper/019:04:550
20903995916,28cyclictest0-21swapper/019:04:550
20903995914,38cyclictest30800-21cpuspeed_turbos17:54:440
20903995910,32cyclictest0-21swapper/017:35:030
20907995838,12cyclictest22497-21tr20:49:381
20907995838,12cyclictest22497-21tr20:49:381
20903995832,19cyclictest7500-21if_err_eth019:14:540
20903995817,15cyclictest0-21swapper/018:24:460
20903995816,27cyclictest0-21swapper/021:49:330
20903995814,37cyclictest17427-21munin-run21:44:240
20903995814,37cyclictest17427-21munin-run21:44:240
20903995723,27cyclictest1286-21runrttasks19:43:220
20903995716,26cyclictest13999-21munin-node19:30:070
20903995714,15cyclictest0-21swapper/020:00:020
20903995714,15cyclictest0-21swapper/020:00:020
242299560,6rtkit-daemon2421-21rtkit-daemon21:14:392
20907995629,10cyclictest9955-21gltestperf19:19:551
2090399559,32cyclictest0-21swapper/019:50:130
20903995521,21cyclictest9002-21snmpd17:32:250
2090799546,18cyclictest0-21swapper/117:39:341
20907995437,10cyclictest18627-21tail21:44:431
20903995423,15cyclictest0-21swapper/022:31:430
20903995423,15cyclictest0-21swapper/022:31:430
20909995323,11cyclictest21913-21latency_hist20:49:272
20909995323,11cyclictest21913-21latency_hist20:49:272
20907995332,12cyclictest5407-21gltestperf20:14:461
242299520,5rtkit-daemon2421-21rtkit-daemon19:59:422
242299520,5rtkit-daemon2421-21rtkit-daemon19:59:422
2090999524,13cyclictest0-21swapper/222:14:472
2090799525,28cyclictest0-21swapper/121:19:461
2090799525,27cyclictest0-21swapper/117:25:001
2090799525,27cyclictest0-21swapper/117:25:001
2090799524,28cyclictest0-21swapper/121:14:531
2090799524,28cyclictest0-21swapper/118:19:561
2090799524,27cyclictest0-21swapper/119:25:041
2090799523,30cyclictest0-21swapper/120:04:471
20907995219,13cyclictest0-21swapper/118:00:001
20907995219,13cyclictest0-21swapper/117:34:421
56142513,39sleep10-21swapper/122:24:311
242299510,5rtkit-daemon2421-21rtkit-daemon20:49:402
242299510,5rtkit-daemon2421-21rtkit-daemon20:49:402
2090999514,27cyclictest0-21swapper/217:59:492
20909995122,12cyclictest0-21swapper/221:43:332
20909995122,12cyclictest0-21swapper/221:43:332
20909995114,32cyclictest0-21swapper/219:09:512
2090799514,29cyclictest0-21swapper/119:55:061
2090799514,29cyclictest0-21swapper/119:55:061
2090799514,29cyclictest0-21swapper/119:39:391
2090799514,27cyclictest0-21swapper/122:19:471
2090799514,27cyclictest0-21swapper/121:40:001
2090799514,27cyclictest0-21swapper/121:40:001
2090799514,27cyclictest0-21swapper/121:24:381
2090799514,27cyclictest0-21swapper/120:44:511
2090799514,27cyclictest0-21swapper/120:44:511
2090799514,27cyclictest0-21swapper/119:59:581
2090799514,27cyclictest0-21swapper/119:59:581
2090799514,27cyclictest0-21swapper/119:09:521
2090799514,27cyclictest0-21swapper/118:25:071
20907995117,13cyclictest0-21swapper/121:30:001
182292512,40sleep10-21swapper/122:49:261
160282513,9sleep10-21swapper/119:31:131
45502503,40sleep20-21swapper/220:14:392
2090999505,27cyclictest0-21swapper/218:54:432
2090999504,28cyclictest0-21swapper/221:09:342
2090999504,27cyclictest0-21swapper/222:04:572
20909995020,13cyclictest0-21swapper/220:30:012
2090799504,28cyclictest0-21swapper/122:44:411
2090799504,28cyclictest0-21swapper/122:39:461
2090799504,28cyclictest0-21swapper/120:19:401
2090799504,26cyclictest0-21swapper/118:42:551
32832493,9sleep10-21swapper/120:09:581
242299490,5rtkit-daemon2421-21rtkit-daemon21:51:082
2090999495,26cyclictest0-21swapper/219:44:542
2090999495,15cyclictest0-21swapper/222:34:222
2090999495,15cyclictest0-21swapper/222:34:222
2090999494,39cyclictest0-21swapper/217:54:322
20909994933,11cyclictest0-21swapper/220:29:062
20909994923,11cyclictest0-21swapper/219:04:572
20909994923,11cyclictest0-21swapper/219:04:572
2090799495,25cyclictest1286-21runrttasks18:34:501
2090799494,28cyclictest0-21swapper/121:54:381
2090799494,27cyclictest0-21swapper/122:19:221
2090799494,27cyclictest0-21swapper/122:09:521
2090799494,27cyclictest0-21swapper/121:04:381
2090799494,27cyclictest0-21swapper/120:34:431
2090799494,25cyclictest0-21swapper/121:12:301
2090799494,25cyclictest0-21swapper/117:51:591
20907994912,28cyclictest0-21swapper/118:14:551
20907994912,28cyclictest0-21swapper/118:14:551
127622493,39sleep20-21swapper/222:36:532
79582483,39sleep10-21swapper/118:12:341
2090999485,25cyclictest0-21swapper/222:25:472
2090999484,27cyclictest0-21swapper/220:37:242
2090999484,27cyclictest0-21swapper/220:09:552
2090999484,26cyclictest0-21swapper/218:34:432
20909994834,9cyclictest9184-21grep18:14:552
20909994834,9cyclictest9184-21grep18:14:552
20909994831,10cyclictest11558-21grep18:19:522
2090999483,41cyclictest0-21swapper/218:54:302
2090999483,26cyclictest26626-21munin-node19:54:552
2090999483,26cyclictest26626-21munin-node19:54:552
20909994827,13cyclictest0-21swapper/221:19:422
20909994820,12cyclictest0-21swapper/217:48:202
20909994819,13cyclictest0-21swapper/221:34:532
2090799484,27cyclictest0-21swapper/118:29:431
2090799484,26cyclictest0-21swapper/121:52:361
2090799484,26cyclictest0-21swapper/120:59:551
2090799484,26cyclictest0-21swapper/120:55:021
2090799484,26cyclictest0-21swapper/118:49:591
2090799484,26cyclictest0-21swapper/118:44:491
2090799484,22cyclictest27476-21aten_r1power_fr17:44:401
20907994831,11cyclictest14496-21unixbench_singl21:34:591
20907994819,12cyclictest0-21swapper/122:04:511
20907994817,13cyclictest0-21swapper/122:29:421
20907994817,13cyclictest0-21swapper/122:29:421
86882473,9sleep10-21swapper/119:19:201
54162473,10sleep10-21swapper/118:07:181
35052472,38sleep10-21swapper/119:07:481
35052472,38sleep10-21swapper/119:07:481
305222472,36sleep10-21swapper/117:54:361
241052472,10sleep10-21swapper/119:49:431
2090999475,27cyclictest0-21swapper/218:39:412
2090999474,26cyclictest0-21swapper/219:14:582
20909994732,8cyclictest17791-21grep18:30:112
20909994717,13cyclictest0-21swapper/218:12:192
20909994717,12cyclictest0-21swapper/217:33:542
2090799474,26cyclictest0-21swapper/122:35:491
20907994731,10cyclictest13036-21gltestperf20:29:451
2090799473,27cyclictest0-21swapper/120:25:011
2090799473,27cyclictest0-21swapper/118:59:301
20907994711,12cyclictest0-21swapper/119:45:541
176702473,37sleep20-21swapper/222:45:042
96902463,36sleep20-21swapper/221:27:212
2090999464,13cyclictest1286-21runrttasks21:59:022
2090999462,39cyclictest0-21swapper/217:24:462
2090999462,39cyclictest0-21swapper/217:24:462
20909994619,9cyclictest24670-21munin-node20:54:482
2090799464,25cyclictest0-21swapper/120:40:471
187932462,36sleep10-21swapper/119:39:191
2090999455,33cyclictest0-21swapper/219:39:372
2090999454,24cyclictest0-21swapper/218:04:542
20909994531,9cyclictest0-21swapper/218:45:082
20909994530,8cyclictest30709-21grep21:04:462
20909994528,12cyclictest0-21swapper/220:44:262
20909994527,13cyclictest0-21swapper/220:09:272
2090799454,24cyclictest0-21swapper/118:59:401
205122454,34sleep20-21swapper/217:20:092
236722442,9sleep10-21swapper/117:30:021
20909994426,13cyclictest0-21swapper/222:49:452
20909994425,14cyclictest0-21swapper/218:24:542
2090999442,37cyclictest0-21swapper/220:19:402
135082443,34sleep20-21swapper/219:26:152
270012423,34sleep20-21swapper/217:42:362
2090999427,30cyclictest19070-21ntp_offset21:44:532
2090799387,19cyclictest0-21swapper/117:24:351
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional