You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 12:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Fri May 02, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
845899964,29cyclictest0-21swapper/221:47:032
845899935,27cyclictest0-21swapper/222:22:342
845599937,79cyclictest0-21swapper/020:42:470
845599936,13cyclictest0-21swapper/018:07:280
845899925,29cyclictest0-21swapper/220:52:322
845899925,28cyclictest0-21swapper/218:27:412
8458999221,13cyclictest0-21swapper/217:52:362
845799914,10cyclictest13088-21munin-node18:12:341
845899905,27cyclictest0-21swapper/217:07:482
845899904,27cyclictest0-21swapper/220:02:392
845599905,9cyclictest27525-21sleep17:37:130
845799886,11cyclictest0-21swapper/118:27:121
845799885,12cyclictest0-21swapper/120:52:061
8455998810,71cyclictest0-21swapper/019:27:080
845899874,28cyclictest0-21swapper/221:27:262
845899874,28cyclictest0-21swapper/221:02:402
845899874,26cyclictest0-21swapper/219:17:422
845899864,27cyclictest0-21swapper/220:12:322
845799864,11cyclictest29808-21munin-run19:57:081
845799864,11cyclictest0-21swapper/117:27:141
845599864,10cyclictest15599-21latency_hist17:12:150
845899854,28cyclictest0-21swapper/221:47:272
845799853,9cyclictest11957-21munin-run18:12:121
845799852,11cyclictest0-21swapper/118:17:251
845799852,11cyclictest0-21swapper/118:17:251
845599854,12cyclictest0-21swapper/018:03:240
845599853,11cyclictest0-21swapper/022:12:370
8455998519,47cyclictest0-21swapper/019:57:380
845799844,10cyclictest17865-21munin-run19:32:081
845599847,13cyclictest1263-21runrttasks19:10:340
845599846,11cyclictest0-21swapper/017:02:280
845599844,73cyclictest0-21swapper/019:02:250
845599844,73cyclictest0-21swapper/019:02:250
845599844,11cyclictest0-21swapper/020:17:370
845899835,41cyclictest0-21swapper/220:17:252
845899834,36cyclictest0-21swapper/217:47:142
845599837,11cyclictest0-21swapper/019:17:100
845599836,11cyclictest0-21swapper/017:22:290
845599834,11cyclictest0-21swapper/020:37:280
845599834,10cyclictest1632-21nscd22:27:010
845599833,12cyclictest0-21swapper/021:17:340
845599833,12cyclictest0-21swapper/021:17:340
845799825,13cyclictest5264-21cat19:02:411
845799825,13cyclictest5264-21cat19:02:411
845799825,10cyclictest0-21swapper/120:22:201
845799824,12cyclictest0-21swapper/122:12:151
8457998211,8cyclictest7150irq/45-ahci20:02:081
845599826,11cyclictest0-21swapper/020:22:400
845599824,8cyclictest15358-21latency_hist21:42:040
845599824,11cyclictest0-21swapper/018:42:360
845599824,11cyclictest0-21swapper/017:12:350
845599823,11cyclictest0-21swapper/020:31:160
845899814,27cyclictest0-21swapper/218:37:372
845799814,8cyclictest26171-21awk20:57:051
845799813,9cyclictest7618-21python19:07:461
845799813,12cyclictest0-21swapper/122:02:141
845599814,8cyclictest21688-21strings19:37:350
845599814,56cyclictest27412-21latency_hist19:52:090
845599814,11cyclictest0-21swapper/019:47:080
845599814,11cyclictest0-21swapper/019:47:080
845599814,11cyclictest0-21swapper/017:47:360
845599814,11cyclictest0-21swapper/017:27:370
845599813,9cyclictest18099-21aten_r1power_po21:47:160
845599813,11cyclictest16368-21fschecks_time21:42:220
845799805,9cyclictest2910-21df_abs22:22:191
845799804,9cyclictest19437-21cpu20:42:201
845599805,11cyclictest0-21swapper/018:12:260
845599804,10cyclictest0-21swapper/020:47:370
845599803,9cyclictest3519-21if_eth020:07:300
845599803,12cyclictest0-21swapper/017:57:400
845599803,10cyclictest1263-21runrttasks21:52:160
845799796,65cyclictest1131-21taskset17:47:381
845799795,66cyclictest0-21swapper/117:42:401
845799795,11cyclictest0-21swapper/118:02:121
845799794,7cyclictest1059-21cat18:57:101
845799793,10cyclictest17933-21ls20:37:321
845599794,11cyclictest0-21swapper/017:37:280
845599794,11cyclictest0-21swapper/017:37:280
845599793,9cyclictest6058-21aten2.4-expect21:22:170
845599793,9cyclictest22789-21latency18:32:400
845599793,11cyclictest0-21swapper/020:14:530
8455997912,47cyclictest0-21swapper/018:27:300
845799785,9cyclictest31567-21cstates21:07:211
845799785,7cyclictest10862-21latency_hist17:02:151
845799785,63cyclictest4091-21python17:52:461
845799785,63cyclictest0-21swapper/117:07:151
845799784,9cyclictest1424-21sendmail_mailqu20:02:381
845799784,12cyclictest0-21swapper/121:47:041
845799784,10cyclictest21810-21munin-node21:52:341
8457997820,51cyclictest0-21swapper/121:57:351
8457997820,51cyclictest0-21swapper/121:57:351
845599787,62cyclictest1263-21runrttasks17:43:040
845599786,66cyclictest0-21swapper/018:22:360
845599786,12cyclictest0-21swapper/019:27:230
845599784,55cyclictest16726-21latency_hist18:22:120
845599784,55cyclictest16726-21latency_hist18:22:120
845599783,11cyclictest0-21swapper/018:57:490
845799774,64cyclictest0-21swapper/121:47:321
845799774,50cyclictest0-21swapper/118:02:311
845799774,11cyclictest0-21swapper/120:17:201
845799773,9cyclictest26286-21hwlatdetect17:32:391
845799773,12cyclictest0-21swapper/119:22:101
845599777,60cyclictest1263-21runrttasks20:52:260
845599777,59cyclictest1263-21runrttasks17:52:430
845599775,66cyclictest0-21swapper/022:05:320
845599773,11cyclictest0-21swapper/021:09:150
845799767,58cyclictest15457-21runrttasks17:10:271
845799764,51cyclictest0-21swapper/121:17:411
845799764,51cyclictest0-21swapper/121:17:411
845799764,12cyclictest0-21swapper/117:19:121
845799763,11cyclictest0-21swapper/121:37:241
845599767,60cyclictest1263-21runrttasks20:57:060
845599767,57cyclictest1263-21runrttasks19:52:210
845599766,11cyclictest0-21swapper/020:06:150
845599763,66cyclictest0-21swapper/019:32:230
8455997611,43cyclictest0-21swapper/016:57:390
845799756,60cyclictest3351-21runrttasks21:16:211
845799755,62cyclictest8028-21runrttasks21:25:071
845799755,61cyclictest23869-21runrttasks18:37:091
845799754,59cyclictest10542-21runrttasks19:15:061
845799754,50cyclictest0-21swapper/122:10:001
845799753,9cyclictest16869-21cut17:12:381
845799753,7cyclictest16641-21latency_hist20:37:071
8457997519,49cyclictest29371-21fschecks_count21:02:241
8457997518,48cyclictest26716-21switchtime19:47:401
845599757,58cyclictest1263-21runrttasks22:18:430
845599757,57cyclictest1263-21runrttasks21:32:410
845599757,57cyclictest1263-21runrttasks18:49:220
845599755,62cyclictest23762-21if_eth021:57:250
845599755,62cyclictest23762-21if_eth021:57:250
845599755,12cyclictest0-21swapper/020:32:210
8455997531,38cyclictest24760-21grep18:37:320
845799748,56cyclictest6926-21runrttasks20:15:271
845799747,60cyclictest24969-21runrttasks17:30:101
845799746,8cyclictest1247-21date18:57:111
845799746,58cyclictest1948-21runrttasks22:20:371
845799744,50cyclictest0-21swapper/121:27:261
845799744,49cyclictest0-21swapper/121:01:371
845799744,12cyclictest0-21swapper/119:22:301
845799744,11cyclictest0-21swapper/119:42:331
845799744,11cyclictest0-21swapper/119:42:331
845799744,11cyclictest0-21swapper/119:35:151
845799744,10cyclictest0-21swapper/118:42:361
845599748,52cyclictest27302-21latency_hist22:07:040
845599745,60cyclictest1263-21runrttasks21:30:230
845599744,63cyclictest0-21swapper/018:52:360
8455997423,44cyclictest19753-21timerandwakeup17:17:470
8455997419,49cyclictest1263-21runrttasks21:15:360
8455997417,50cyclictest12642-21unixbench_multi19:17:480
845799735,59cyclictest4636-21runrttasks20:11:441
845799735,57cyclictest24118-21runrttasks18:37:251
845799733,10cyclictest30026-21sort18:47:401
8457997322,45cyclictest1263-21runrttasks19:37:501
8457997321,45cyclictest20222-21gltestperf18:27:381
8455997319,48cyclictest30524-21grep21:02:400
845799723,61cyclictest12071-21munin-node20:27:311
8457997220,45cyclictest28279-21df_abs17:37:311
8457997220,45cyclictest28279-21df_abs17:37:311
8457997219,46cyclictest11611-21gltestperf21:32:261
845599723,9cyclictest7150irq/45-ahci16:57:150
845599723,9cyclictest7150irq/45-ahci16:57:150
227799690,4rtkit-daemon2276-21rtkit-daemon21:54:453
845799674,10cyclictest1263-21runrttasks16:57:161
845799674,10cyclictest1263-21runrttasks16:57:161
227799640,4rtkit-daemon2276-21rtkit-daemon19:22:063
227799610,7rtkit-daemon0-21swapper/218:22:233
845999599,31cyclictest26160-21munin-run20:57:053
8459995917,26cyclictest3533-21memory22:22:273
79682592,50sleep20-21swapper/221:24:202
227799580,8rtkit-daemon0-21swapper/321:12:392
227799580,7rtkit-daemon0-21swapper/320:31:402
227799580,5rtkit-daemon8867-21python18:02:463
845999575,33cyclictest0-21swapper/317:12:283
8459995714,35cyclictest3939-21fgrep20:07:383
8459995612,26cyclictest0-21swapper/318:52:433
227799560,6rtkit-daemon0-21swapper/221:12:543
227799560,5rtkit-daemon2449-21sendmail19:40:272
227799560,5rtkit-daemon2276-21rtkit-daemon19:43:503
227799560,5rtkit-daemon2276-21rtkit-daemon19:43:503
227799560,5rtkit-daemon2276-21rtkit-daemon19:08:443
227799560,5rtkit-daemon2276-21rtkit-daemon17:31:183
227799560,5rtkit-daemon2276-21rtkit-daemon17:21:403
227799560,4rtkit-daemon16292-21diskstats19:27:282
227799560,12rtkit-daemon2276-21rtkit-daemon17:37:473
227799560,12rtkit-daemon2276-21rtkit-daemon17:37:473
189842562,45sleep20-21swapper/219:32:322
845999558,12cyclictest0-21swapper/318:27:483
845999556,37cyclictest0-21swapper/318:13:083
845999555,30cyclictest0-21swapper/320:07:073
8459995512,36cyclictest30047-21taskset22:12:163
8458995533,13cyclictest27515-21sadc17:37:132
227799550,6rtkit-daemon2276-21rtkit-daemon18:11:013
227799550,5rtkit-daemon2276-21rtkit-daemon22:21:593
227799550,5rtkit-daemon2276-21rtkit-daemon22:00:593
227799550,5rtkit-daemon2276-21rtkit-daemon22:00:593
227799550,5rtkit-daemon2276-21rtkit-daemon21:47:303
227799550,5rtkit-daemon2276-21rtkit-daemon21:03:323
227799550,5rtkit-daemon2276-21rtkit-daemon19:37:503
227799550,5rtkit-daemon2276-21rtkit-daemon18:48:443
227799550,5rtkit-daemon2276-21rtkit-daemon18:41:523
845999546,30cyclictest0-21swapper/319:02:343
845999546,30cyclictest0-21swapper/319:02:343
8459995413,31cyclictest0-21swapper/317:42:343
845899544,25cyclictest0-21swapper/220:42:192
8458995436,11cyclictest28531-21grep19:52:312
8458995413,24cyclictest27795-21rt-features20:57:352
227799540,6rtkit-daemon2276-21rtkit-daemon18:34:153
227799540,6rtkit-daemon2276-21rtkit-daemon18:19:233
227799540,6rtkit-daemon2276-21rtkit-daemon18:19:233
227799540,6rtkit-daemon0-21swapper/322:08:442
227799540,5rtkit-daemon2276-21rtkit-daemon21:42:083
227799540,5rtkit-daemon2276-21rtkit-daemon21:18:313
227799540,5rtkit-daemon2276-21rtkit-daemon21:18:313
227799540,5rtkit-daemon2276-21rtkit-daemon20:48:023
227799540,5rtkit-daemon2276-21rtkit-daemon20:41:253
227799540,5rtkit-daemon2276-21rtkit-daemon20:34:103
227799540,5rtkit-daemon2276-21rtkit-daemon20:17:263
227799540,5rtkit-daemon2276-21rtkit-daemon20:13:113
227799540,5rtkit-daemon2276-21rtkit-daemon19:58:423
227799540,5rtkit-daemon2276-21rtkit-daemon19:47:203
227799540,5rtkit-daemon2276-21rtkit-daemon19:28:503
227799540,5rtkit-daemon2276-21rtkit-daemon19:16:433
227799540,4rtkit-daemon2276-21rtkit-daemon21:29:313
227799540,4rtkit-daemon2276-21rtkit-daemon17:25:183
227799540,4rtkit-daemon1086-21lldpd18:51:592
845999537,39cyclictest0-21swapper/322:03:583
845899535,28cyclictest0-21swapper/218:52:402
845899535,28cyclictest0-21swapper/218:32:392
845899534,28cyclictest0-21swapper/220:07:392
845899534,26cyclictest0-21swapper/219:01:192
845899534,25cyclictest0-21swapper/221:55:122
845899534,25cyclictest0-21swapper/221:17:152
845899534,25cyclictest0-21swapper/221:17:152
845899534,25cyclictest0-21swapper/219:49:182
8458995323,13cyclictest0-21swapper/219:02:382
8458995323,13cyclictest0-21swapper/219:02:382
8458995319,27cyclictest1263-21runrttasks18:42:272
227799530,5rtkit-daemon2276-21rtkit-daemon21:38:533
227799530,5rtkit-daemon2276-21rtkit-daemon21:22:163
227799530,5rtkit-daemon2276-21rtkit-daemon20:59:173
227799530,5rtkit-daemon2276-21rtkit-daemon20:42:253
227799530,5rtkit-daemon2276-21rtkit-daemon19:33:353
227799530,5rtkit-daemon2276-21rtkit-daemon19:23:513
227799530,5rtkit-daemon2276-21rtkit-daemon19:00:523
227799530,5rtkit-daemon2276-21rtkit-daemon17:53:093
227799530,5rtkit-daemon2276-21rtkit-daemon17:49:023
227799530,5rtkit-daemon2276-21rtkit-daemon17:36:253
227799530,4rtkit-daemon2276-21rtkit-daemon21:07:323
227799530,4rtkit-daemon2276-21rtkit-daemon20:28:333
227799530,4rtkit-daemon2276-21rtkit-daemon20:24:333
227799530,4rtkit-daemon2276-21rtkit-daemon19:54:273
227799530,4rtkit-daemon2276-21rtkit-daemon18:43:153
845899525,28cyclictest0-21swapper/219:07:292
845899525,28cyclictest0-21swapper/217:37:362
845899525,28cyclictest0-21swapper/217:37:362
845899525,27cyclictest0-21swapper/220:22:402
845899524,29cyclictest0-21swapper/218:17:112
845899524,28cyclictest0-21swapper/217:12:412
845899524,27cyclictest0-21swapper/220:47:392
227799520,4rtkit-daemon2276-21rtkit-daemon22:08:223
227799520,4rtkit-daemon2276-21rtkit-daemon21:32:163
227799520,4rtkit-daemon2276-21rtkit-daemon17:59:163
8459995112,32cyclictest13605-21tail17:07:283
8459995111,32cyclictest11061-21munin-node17:02:513
845899515,28cyclictest0-21swapper/217:27:512
845899514,27cyclictest0-21swapper/221:37:372
845899514,27cyclictest0-21swapper/219:27:092
845899514,27cyclictest0-21swapper/218:17:492
845899514,27cyclictest0-21swapper/218:17:492
8458995120,12cyclictest0-21swapper/219:57:352
8458995119,12cyclictest0-21swapper/221:07:292
845999504,27cyclictest0-21swapper/316:57:163
845999504,27cyclictest0-21swapper/316:57:163
845899504,27cyclictest0-21swapper/222:17:262
845899504,27cyclictest0-21swapper/218:07:372
165002502,10sleep20-21swapper/220:35:512
845999496,29cyclictest0-21swapper/316:57:293
845899495,26cyclictest0-21swapper/219:12:492
845899494,24cyclictest0-21swapper/217:57:282
845899494,23cyclictest0-21swapper/217:02:282
845899493,28cyclictest0-21swapper/217:17:482
8458994919,13cyclictest0-21swapper/218:27:062
845899484,27cyclictest0-21swapper/220:37:242
845899484,26cyclictest0-21swapper/221:32:192
845899484,26cyclictest0-21swapper/219:42:242
845899484,26cyclictest0-21swapper/219:42:242
8458994820,12cyclictest0-21swapper/216:57:542
8458994718,12cyclictest0-21swapper/222:02:282
845899464,27cyclictest0-21swapper/217:22:432
845899464,25cyclictest1550-21mailstats17:47:462
8458994632,8cyclictest8475-21latency18:02:392
8458994614,12cyclictest0-21swapper/222:12:162
845899453,26cyclictest0-21swapper/221:58:592
845899453,26cyclictest0-21swapper/221:58:592
845899383,19cyclictest8432-21latency_hist16:57:162
845899383,19cyclictest8432-21latency_hist16:57:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional