You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 15:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Sat Apr 20, 2024 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2422991110,5rtkit-daemon2421-21rtkit-daemon06:13:521
726521020,4sleep1201ktimersoftd/108:58:371
32075999437,11cyclictest32648-21cpuspeed_turbos05:28:572
32075999437,11cyclictest24976-21grep10:34:102
3207599925,32cyclictest0-21swapper/210:18:592
3207599913,27cyclictest0-21swapper/206:13:592
32074998427,50cyclictest0-21swapper/106:19:591
32074998335,41cyclictest7916-21tr05:43:571
32074998335,41cyclictest7916-21tr05:43:571
32074998227,48cyclictest19145-21munin-node09:19:131
32074998027,47cyclictest27614-21telnet09:38:441
242299800,5rtkit-daemon2421-21rtkit-daemon10:42:413
242299800,4rtkit-daemon2421-21rtkit-daemon10:46:193
242299800,4rtkit-daemon2421-21rtkit-daemon10:46:193
242299790,5rtkit-daemon2421-21rtkit-daemon10:37:123
242299790,4rtkit-daemon2421-21rtkit-daemon10:56:183
242299780,4rtkit-daemon2421-21rtkit-daemon10:50:413
32074997733,24cyclictest4352-21runrttasks08:50:131
32074997727,30cyclictest0-21swapper/110:38:131
32074997630,39cyclictest9002-21snmpd06:56:461
32074997626,44cyclictest0-21swapper/108:23:491
32074997535,34cyclictest24399-21unixbench_multi09:29:121
32074997530,39cyclictest5201-21basename09:58:341
32074997530,39cyclictest5201-21basename09:58:341
32074997530,38cyclictest25767-21aten_r1power_vo10:38:431
32074997527,42cyclictest15022-21latency_hist05:58:441
32074997428,15cyclictest0-21swapper/108:43:371
32074997425,43cyclictest0-21swapper/108:08:381
32074997324,29cyclictest0-21swapper/105:28:451
32074997320,47cyclictest0-21swapper/108:45:301
32074997227,39cyclictest28732-21grep06:28:561
32074997227,11cyclictest28172-21munin-run06:28:421
32074997226,39cyclictest20856-21open_files08:19:091
32074997226,39cyclictest20856-21open_files08:19:091
32074997226,27cyclictest4283-21munin-node07:49:051
32074997225,41cyclictest0-21swapper/110:22:321
32074997225,28cyclictest11145-21http_loadtime05:49:031
32074997225,28cyclictest11145-21http_loadtime05:49:031
32074997154,12cyclictest1286-21runrttasks07:24:181
32074997154,12cyclictest1286-21runrttasks07:24:181
32074997127,27cyclictest28617-21df_abs10:43:471
32074997127,27cyclictest28617-21df_abs10:43:471
32074997126,25cyclictest16117-21iostat_ios09:14:041
32074997027,36cyclictest0-21swapper/106:39:001
32074997027,36cyclictest0-21swapper/106:39:001
32074997027,12cyclictest28783-21munin-node08:34:151
32074997026,38cyclictest21658-21kworker/u8:210:14:021
32074997019,30cyclictest0-21swapper/105:59:161
32074996930,32cyclictest0-21swapper/108:29:401
32074996926,38cyclictest10623-21aten2.4-expect10:08:451
32074996919,44cyclictest0-21swapper/110:53:551
32074996919,32cyclictest0-21swapper/109:12:591
32074996918,44cyclictest0-21swapper/105:33:451
32074996826,11cyclictest3003-21latency07:44:031
32074996744,14cyclictest32625-21users10:49:091
32074996726,34cyclictest0-21swapper/109:23:561
32074996721,30cyclictest0-21swapper/107:08:521
32074996719,31cyclictest0-21swapper/105:34:001
32074996718,39cyclictest0-21swapper/108:59:161
32074996641,18cyclictest32069-21runrttasks09:44:121
32074996640,17cyclictest11470-21runrttasks08:01:431
32074996628,30cyclictest0-21swapper/107:54:571
32074996626,19cyclictest0-21swapper/108:13:471
32074996623,36cyclictest16256-21unixbench_singl08:09:161
32074996623,36cyclictest16256-21unixbench_singl08:09:161
32074996623,26cyclictest19554-21fschecks_count07:14:011
32074996619,41cyclictest0-21swapper/110:06:471
32074996541,16cyclictest7601-21runrttasks10:02:141
32074996538,18cyclictest1286-21runrttasks07:39:171
32074996524,16cyclictest0-21swapper/106:08:441
32074996521,29cyclictest0-21swapper/110:29:181
32074996440,15cyclictest15600-21unixbench-2d07:04:151
32074996422,35cyclictest0-21swapper/106:33:521
32074996421,37cyclictest0-21swapper/106:53:421
32074996418,30cyclictest0-21swapper/109:34:121
3207599634,41cyclictest0-21swapper/206:18:532
32074996341,17cyclictest20361-21runrttasks06:10:581
32074996341,17cyclictest20082-21taskset10:25:081
32074996316,41cyclictest0-21swapper/109:03:441
32074996316,41cyclictest0-21swapper/109:03:441
242299630,4rtkit-daemon2421-21rtkit-daemon10:53:562
242299630,4rtkit-daemon2421-21rtkit-daemon10:39:042
32074996239,16cyclictest5606-21grep05:38:581
32074996236,20cyclictest1286-21runrttasks06:59:541
32074996236,19cyclictest4885-21runrttasks06:44:081
32074996221,36cyclictest29567-21munin-node07:33:551
32074996216,30cyclictest0-21swapper/107:18:501
3207099623,52cyclictest25120-21aten2.4-expect09:33:460
32074996114,40cyclictest0-21swapper/109:48:451
242299610,4rtkit-daemon2421-21rtkit-daemon10:53:262
32074996027,16cyclictest95250irq/46-eth007:28:481
32074996027,16cyclictest95250irq/46-eth007:28:481
260672604,13sleep00-21swapper/008:29:100
32076995939,12cyclictest26958-21unixbench_singl09:34:133
3207599594,36cyclictest0-21swapper/207:23:572
3207599594,36cyclictest0-21swapper/207:23:572
3207699585,22cyclictest0-21swapper/308:48:363
3207099583,49cyclictest0-21swapper/008:24:140
208022582,9sleep00-21swapper/006:13:470
327602563,11sleep30-21swapper/309:48:463
32076995613,27cyclictest0-21swapper/306:24:383
3207099567,43cyclictest41ktimersoftd/009:08:590
3207699555,31cyclictest0-21swapper/308:53:363
3207699545,30cyclictest0-21swapper/310:19:273
3207699545,30cyclictest0-21swapper/307:43:523
3207699544,29cyclictest0-21swapper/307:13:523
3207699543,45cyclictest17470-21munin-node09:19:183
32075995438,9cyclictest4600-21tr07:48:522
3207099544,44cyclictest0-21swapper/009:53:520
3207099544,44cyclictest0-21swapper/009:53:520
3207099544,44cyclictest0-21swapper/008:53:440
3207099544,44cyclictest0-21swapper/007:33:400
3207099544,44cyclictest0-21swapper/007:33:400
3207099543,45cyclictest0-21swapper/006:28:540
3207099543,42cyclictest0-21swapper/006:18:580
178172542,43sleep00-21swapper/006:03:560
65002532,9sleep30-21swapper/307:52:463
3207699535,30cyclictest0-21swapper/309:54:043
3207699535,30cyclictest0-21swapper/309:54:043
3207699534,41cyclictest0-21swapper/306:13:553
3207699534,29cyclictest0-21swapper/307:59:093
32076995339,8cyclictest23717-21ls08:24:143
32076995333,12cyclictest2927-21timerwakeupswit06:39:173
32076995333,12cyclictest2927-21timerwakeupswit06:39:173
32076995328,14cyclictest1286-21runrttasks07:13:393
3207599534,33cyclictest0-21swapper/207:14:172
3207599534,28cyclictest0-21swapper/208:18:572
3207599534,28cyclictest0-21swapper/208:18:572
32075995315,11cyclictest0-21swapper/209:59:032
3207099534,44cyclictest0-21swapper/008:47:310
3207099534,43cyclictest0-21swapper/009:49:020
3207099534,43cyclictest0-21swapper/009:03:560
3207099534,43cyclictest0-21swapper/009:03:560
3207099534,43cyclictest0-21swapper/008:03:490
3207099534,43cyclictest0-21swapper/007:33:500
3207099533,44cyclictest0-21swapper/008:12:330
3207099533,44cyclictest0-21swapper/008:12:330
3207099533,44cyclictest0-21swapper/007:18:550
303602532,43sleep00-21swapper/010:47:240
303602532,43sleep00-21swapper/010:47:240
253932533,42sleep00-21swapper/010:38:330
242299530,4rtkit-daemon2421-21rtkit-daemon10:47:262
242299530,4rtkit-daemon2421-21rtkit-daemon10:47:262
3207699524,40cyclictest0-21swapper/308:13:453
3207699524,31cyclictest0-21swapper/308:13:393
3207699524,30cyclictest0-21swapper/306:29:053
3207699524,29cyclictest0-21swapper/309:29:063
3207699524,29cyclictest0-21swapper/309:13:363
3207699524,29cyclictest0-21swapper/307:03:583
3207699524,28cyclictest0-21swapper/307:38:403
3207699523,31cyclictest0-21swapper/306:53:583
3207699523,27cyclictest10602-21df_abs05:48:573
3207699523,27cyclictest10602-21df_abs05:48:573
32076995219,14cyclictest0-21swapper/309:07:583
32076995219,14cyclictest0-21swapper/309:07:583
3207599524,29cyclictest0-21swapper/208:13:332
3207599524,29cyclictest0-21swapper/208:13:332
3207599524,29cyclictest0-21swapper/206:24:152
3207599523,34cyclictest0-21swapper/209:44:092
32075995224,9cyclictest22320-21idleruntime-cro09:28:342
3207099524,42cyclictest0-21swapper/005:58:530
3207099524,41cyclictest0-21swapper/007:39:040
3207099523,43cyclictest12763-21latency_hist10:13:340
3207099523,43cyclictest0-21swapper/010:19:140
3207099523,43cyclictest0-21swapper/005:46:300
3207099523,43cyclictest0-21swapper/005:46:300
211522522,10sleep00-21swapper/007:18:410
3207699515,38cyclictest20363-21runrttasks10:28:323
3207699515,38cyclictest0-21swapper/307:39:073
3207699514,29cyclictest0-21swapper/308:04:043
3207699514,29cyclictest0-21swapper/306:34:133
3207699514,28cyclictest0-21swapper/307:29:113
3207699514,28cyclictest0-21swapper/307:29:113
3207699514,23cyclictest0-21swapper/309:13:403
32076995124,9cyclictest16675-21munin-run08:13:383
32076995119,16cyclictest0-21swapper/307:57:263
32076995115,20cyclictest1286-21runrttasks06:21:153
3207599514,37cyclictest0-21swapper/207:18:512
3207599513,29cyclictest0-21swapper/210:28:332
32075995121,13cyclictest0-21swapper/208:29:152
3207099513,43cyclictest0-21swapper/010:23:460
3207099513,42cyclictest0-21swapper/010:13:430
316732513,11sleep00-21swapper/009:44:070
147382512,9sleep20-21swapper/205:56:282
112872512,9sleep00-21swapper/007:59:200
3207699504,29cyclictest0-21swapper/309:43:483
3207699504,28cyclictest0-21swapper/306:44:063
3207699503,39cyclictest0-21swapper/306:58:533
32076995020,13cyclictest0-21swapper/309:38:453
32076995019,13cyclictest0-21swapper/310:33:123
32076995018,14cyclictest0-21swapper/310:13:383
3207599504,30cyclictest0-21swapper/207:39:042
3207599504,28cyclictest0-21swapper/208:53:582
3207599504,28cyclictest0-21swapper/208:38:532
32075995034,9cyclictest6952-21unixbench_singl05:39:192
32075995033,11cyclictest29300-21cpuspeed_turbos07:33:512
32075995032,11cyclictest12013-21grep05:49:192
32075995032,11cyclictest12013-21grep05:49:192
32075995018,14cyclictest0-21swapper/209:38:552
3207099504,41cyclictest0-21swapper/007:49:050
3207099504,40cyclictest8060-21aten2.4-expect10:03:430
3207099504,39cyclictest0-21swapper/009:33:340
3207099504,38cyclictest0-21swapper/007:54:040
3207099503,42cyclictest0-21swapper/005:34:080
3207099503,41cyclictest0-21swapper/009:23:460
3207099503,40cyclictest20577-21latency_hist06:13:430
176632502,40sleep20-21swapper/206:03:462
3207699495,27cyclictest0-21swapper/305:58:453
3207699494,27cyclictest0-21swapper/310:09:283
3207699494,25cyclictest0-21swapper/309:23:453
32076994930,11cyclictest17041-21unixbench_singl05:59:163
3207599495,27cyclictest0-21swapper/206:48:592
3207599494,28cyclictest3123-21tune2fs08:48:592
3207599494,28cyclictest0-21swapper/209:48:582
3207599494,28cyclictest0-21swapper/207:44:042
3207599494,27cyclictest0-21swapper/209:19:012
3207599494,27cyclictest0-21swapper/209:03:502
3207599494,27cyclictest0-21swapper/209:03:502
3207599494,27cyclictest0-21swapper/206:44:202
3207599494,27cyclictest0-21swapper/206:34:132
3207599494,26cyclictest0-21swapper/209:08:442
3207599494,26cyclictest0-21swapper/208:43:452
3207599493,29cyclictest16996-21unixbench_multi05:59:162
3207599493,28cyclictest0-21swapper/210:08:522
32075994921,21cyclictest0-21swapper/207:28:482
32075994921,21cyclictest0-21swapper/207:28:482
3207099494,38cyclictest0-21swapper/010:29:070
3207099494,38cyclictest0-21swapper/006:59:160
32070994933,9cyclictest8079-21tail06:49:190
3207099493,27cyclictest0-21swapper/007:13:410
257962492,41sleep00-21swapper/007:24:170
257962492,41sleep00-21swapper/007:24:170
59652482,40sleep00-21swapper/006:48:220
3207699484,26cyclictest0-21swapper/308:58:443
3207699483,30cyclictest0-21swapper/307:24:093
3207699483,30cyclictest0-21swapper/307:24:093
32076994819,14cyclictest0-21swapper/305:34:013
32076994816,26cyclictest1286-21runrttasks10:01:473
3207599484,27cyclictest0-21swapper/207:54:092
3207599484,27cyclictest0-21swapper/207:09:072
3207599484,27cyclictest0-21swapper/206:53:532
3207599484,26cyclictest0-21swapper/209:54:082
3207599484,26cyclictest0-21swapper/209:54:082
32075994833,9cyclictest21933-21tr10:29:012
3207599483,40cyclictest0-21swapper/208:59:052
3207599483,28cyclictest0-21swapper/209:34:072
3207599483,27cyclictest0-21swapper/210:13:462
32075994819,12cyclictest0-21swapper/208:13:552
3207099484,36cyclictest0-21swapper/010:38:420
3207099484,13cyclictest0-21swapper/009:38:530
3207099483,39cyclictest0-21swapper/010:53:430
3207099483,39cyclictest0-21swapper/008:38:490
3207099483,39cyclictest0-21swapper/005:53:520
32070994818,12cyclictest0-21swapper/006:39:150
32070994818,12cyclictest0-21swapper/006:39:150
247542482,39sleep20-21swapper/209:32:412
158882482,40sleep00-21swapper/009:14:000
132572482,9sleep20-21swapper/206:59:402
62232472,9sleep30-21swapper/306:48:453
3207699477,24cyclictest0-21swapper/308:53:443
3207699474,34cyclictest0-21swapper/306:13:433
3207699474,28cyclictest0-21swapper/307:18:573
32076994731,10cyclictest26202-21wc08:29:123
32076994730,10cyclictest19155-21proc_pri06:04:143
3207599475,13cyclictest0-21swapper/208:34:152
3207599474,37cyclictest0-21swapper/208:04:012
3207599474,26cyclictest0-21swapper/207:58:572
3207599474,26cyclictest0-21swapper/205:34:192
32075994731,10cyclictest16851-21head09:14:162
3207599473,32cyclictest0-21swapper/205:44:002
3207599473,32cyclictest0-21swapper/205:44:002
3207099478,33cyclictest41ktimersoftd/008:33:480
3207099473,40cyclictest0-21swapper/008:49:050
3207099473,40cyclictest0-21swapper/005:39:220
3207099473,38cyclictest0-21swapper/008:18:370
3207099473,38cyclictest0-21swapper/006:53:500
231392473,37sleep20-21swapper/208:24:042
20262472,39sleep30-21swapper/305:30:503
100192472,9sleep30-21swapper/310:05:513
94212462,38sleep00-21swapper/009:00:300
52762464,35sleep35425-21cpuspeed_turbos05:38:563
40162462,9sleep00-21swapper/007:48:190
323982462,38sleep00-21swapper/006:34:110
3207699464,27cyclictest0-21swapper/308:38:373
3207699464,27cyclictest0-21swapper/305:43:553
3207699464,27cyclictest0-21swapper/305:43:553
3207599464,26cyclictest0-21swapper/207:04:012
3207599463,26cyclictest0-21swapper/206:38:502
3207599463,26cyclictest0-21swapper/206:38:502
3207099464,36cyclictest0-21swapper/006:24:130
3207099464,35cyclictest0-21swapper/005:28:530
3207099463,36cyclictest0-21swapper/010:49:010
319332464,6sleep00-21swapper/005:27:240
74352452,38sleep00-21swapper/010:00:030
3207699454,14cyclictest0-21swapper/308:18:533
3207699454,14cyclictest0-21swapper/308:18:533
3207599454,25cyclictest0-21swapper/206:11:312
3207599453,25cyclictest0-21swapper/210:04:052
3207599452,39cyclictest0-21swapper/206:30:452
3207099453,35cyclictest0-21swapper/008:18:480
3207099453,35cyclictest0-21swapper/008:18:480
194962452,9sleep00-21swapper/009:19:570
32070994431,9cyclictest0-21swapper/007:07:310
32070994421,7cyclictest12397-21latency_hist05:53:440
32070994421,7cyclictest12397-21latency_hist05:53:440
3207699434,31cyclictest0-21swapper/308:38:493
3207599424,31cyclictest0-21swapper/205:28:452
32076994125,9cyclictest32242-21latency_hist05:28:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional