You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 14:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 14, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29056999437,10cyclictest19006-21grep12:21:262
2904999943,42cyclictest5317-21taskset08:31:290
29049999423,12cyclictest0-21swapper/008:56:150
29049999423,12cyclictest0-21swapper/008:56:150
243799910,4rtkit-daemon2436-21rtkit-daemon07:06:300
2905699904,28cyclictest0-21swapper/209:10:422
29056999035,11cyclictest22095-21wc12:26:362
29049998833,10cyclictest693-21tr08:21:350
29049998732,11cyclictest28364-21tr08:11:320
29056998536,10cyclictest8062-21grep11:56:432
2904999853,43cyclictest0-21swapper/010:50:590
2904999853,43cyclictest0-21swapper/007:36:290
2904999834,38cyclictest0-21swapper/012:00:060
29052997122,24cyclictest9339-21forks12:01:261
29052996215,11cyclictest0-21swapper/111:41:281
2905699604,31cyclictest0-21swapper/209:16:312
2905299604,12cyclictest0-21swapper/108:00:211
243799600,4rtkit-daemon2436-21rtkit-daemon10:33:211
181892600,5sleep318187-21cat12:21:083
181892600,5sleep318187-21cat12:21:083
189792593,48sleep20-21swapper/210:06:332
2905699584,36cyclictest0-21swapper/208:41:282
2904999583,47cyclictest0-21swapper/012:11:250
243799580,4rtkit-daemon2436-21rtkit-daemon07:55:131
2905799574,37cyclictest0-21swapper/307:20:103
29057995739,11cyclictest16872-21gltestperf10:01:313
2905299574,13cyclictest0-21swapper/108:11:181
2904999573,48cyclictest0-21swapper/011:46:200
2905299565,12cyclictest0-21swapper/107:31:191
2905299565,12cyclictest0-21swapper/107:31:191
2905299563,29cyclictest0-21swapper/109:41:141
29052995621,16cyclictest0-21swapper/110:21:261
29049995618,14cyclictest0-21swapper/009:51:400
243799560,6rtkit-daemon2436-21rtkit-daemon08:26:410
29057995539,10cyclictest15692-21hwlatdetect11:06:333
29056995533,10cyclictest0-21swapper/207:56:322
2905299554,12cyclictest0-21swapper/109:00:071
29052995519,16cyclictest0-21swapper/107:21:371
2904999553,46cyclictest0-21swapper/010:41:330
243799550,5rtkit-daemon2436-21rtkit-daemon08:46:410
243799550,5rtkit-daemon2436-21rtkit-daemon08:36:480
122012553,44sleep312276-21if_eth007:36:393
29056995436,11cyclictest17680-21sed10:01:442
29056995432,8cyclictest0-21swapper/207:06:482
2905699543,26cyclictest0-21swapper/207:11:312
2905299545,29cyclictest0-21swapper/108:50:391
2905299544,30cyclictest0-21swapper/110:11:451
2905299544,30cyclictest0-21swapper/110:11:451
2905299544,30cyclictest0-21swapper/108:51:521
2905299544,30cyclictest0-21swapper/108:51:521
2905299544,12cyclictest0-21swapper/111:46:211
2904999543,45cyclictest0-21swapper/010:21:250
243892544,42sleep30-21swapper/312:31:373
243799540,6rtkit-daemon2436-21rtkit-daemon09:06:320
243799540,5rtkit-daemon2436-21rtkit-daemon11:51:260
243799540,5rtkit-daemon2436-21rtkit-daemon08:01:350
202972545,9sleep00-21swapper/010:08:090
8232534,9sleep00-21swapper/011:41:480
2905699536,29cyclictest0-21swapper/210:36:362
2905699535,39cyclictest0-21swapper/209:36:262
2905699535,27cyclictest0-21swapper/208:21:482
2905699534,43cyclictest0-21swapper/207:21:532
29056995332,13cyclictest0-21swapper/209:41:462
29056995330,11cyclictest0-21swapper/209:26:282
29056995330,11cyclictest0-21swapper/208:16:522
29056995330,10cyclictest0-21swapper/208:51:472
29056995330,10cyclictest0-21swapper/208:51:472
29056995329,10cyclictest0-21swapper/207:16:442
29056995322,13cyclictest0-21swapper/209:11:282
2905299535,30cyclictest0-21swapper/109:06:301
2905299535,29cyclictest0-21swapper/112:31:331
2905299535,27cyclictest0-21swapper/111:06:361
2905299534,31cyclictest0-21swapper/112:21:071
2905299534,31cyclictest0-21swapper/112:21:071
2905299534,29cyclictest1369-21runrttasks07:36:451
2905299534,28cyclictest0-21swapper/111:26:341
2904999534,41cyclictest0-21swapper/011:36:290
2904999534,41cyclictest0-21swapper/011:36:290
2904999533,44cyclictest0-21swapper/012:26:190
2904999533,44cyclictest0-21swapper/011:31:350
29049995324,13cyclictest0-21swapper/010:26:420
29049995323,13cyclictest0-21swapper/011:21:100
243799530,5rtkit-daemon2436-21rtkit-daemon09:03:100
243799530,5rtkit-daemon2436-21rtkit-daemon08:09:350
243799530,5rtkit-daemon2436-21rtkit-daemon07:11:440
228292534,9sleep00-21swapper/007:59:010
225862537,38sleep00-21swapper/010:11:550
225862537,38sleep00-21swapper/010:11:550
2905799526,28cyclictest0-21swapper/309:41:263
2905799525,29cyclictest0-21swapper/310:31:353
2905799525,28cyclictest0-21swapper/308:11:403
2905799524,28cyclictest0-21swapper/311:46:093
29057995235,11cyclictest818-21grep09:26:483
29057995230,13cyclictest3270-21unixbench_multi11:46:423
29057995221,13cyclictest0-21swapper/308:21:403
29057995216,29cyclictest0-21swapper/308:46:473
2905699524,40cyclictest0-21swapper/212:11:382
2905699523,26cyclictest0-21swapper/211:05:532
29056995229,16cyclictest0-21swapper/209:00:172
29056995227,11cyclictest0-21swapper/212:06:302
29056995221,12cyclictest0-21swapper/209:21:322
2905299525,28cyclictest0-21swapper/111:56:351
2905299525,28cyclictest0-21swapper/108:30:271
2905299524,29cyclictest0-21swapper/112:11:251
2905299524,29cyclictest0-21swapper/110:56:501
2905299524,28cyclictest0-21swapper/110:26:271
2905299524,28cyclictest0-21swapper/107:16:441
29052995219,14cyclictest0-21swapper/109:26:401
2904999523,43cyclictest0-21swapper/009:43:370
243799520,5rtkit-daemon2436-21rtkit-daemon10:40:591
243799520,5rtkit-daemon2436-21rtkit-daemon07:01:373
237022523,42sleep323799-21munin-node09:06:493
2905799515,28cyclictest0-21swapper/312:06:193
2905799515,28cyclictest0-21swapper/307:21:493
2905799515,12cyclictest0-21swapper/312:21:143
2905799514,28cyclictest0-21swapper/311:11:423
29057995121,12cyclictest0-21swapper/311:36:103
29057995110,26cyclictest0-21swapper/311:36:213
29057995110,26cyclictest0-21swapper/311:36:213
2905699515,39cyclictest0-21swapper/209:01:362
2905699515,30cyclictest0-21swapper/208:06:342
2905699514,40cyclictest0-21swapper/209:56:132
2905699514,25cyclictest0-21swapper/211:21:232
2905699514,25cyclictest0-21swapper/210:46:252
2905699513,25cyclictest0-21swapper/207:31:432
29056995118,13cyclictest0-21swapper/211:06:362
2905299515,27cyclictest0-21swapper/110:10:531
2905299515,27cyclictest0-21swapper/109:15:421
2905299514,29cyclictest0-21swapper/107:46:531
2905299514,28cyclictest0-21swapper/111:26:101
2905299514,28cyclictest0-21swapper/109:31:351
2905299514,28cyclictest0-21swapper/109:31:351
2905299514,28cyclictest0-21swapper/108:46:161
2905299514,28cyclictest0-21swapper/108:36:171
2904999513,42cyclictest0-21swapper/011:11:020
29049995123,12cyclictest0-21swapper/009:11:310
243799510,5rtkit-daemon2436-21rtkit-daemon08:58:480
180682513,9sleep20-21swapper/207:49:242
98442504,39sleep00-21swapper/009:46:360
94392502,9sleep20-21swapper/210:52:242
94392502,9sleep20-21swapper/210:52:242
2905799505,28cyclictest0-21swapper/310:41:453
2905799505,28cyclictest0-21swapper/307:06:453
2905799505,27cyclictest0-21swapper/310:26:433
2905799504,28cyclictest0-21swapper/312:26:333
2905799504,28cyclictest0-21swapper/312:11:413
2905799504,28cyclictest0-21swapper/311:51:313
2905799504,28cyclictest0-21swapper/309:56:433
2905799504,28cyclictest0-21swapper/308:26:363
2905799504,28cyclictest0-21swapper/308:01:523
2905799504,28cyclictest0-21swapper/307:11:483
2905799504,27cyclictest0-21swapper/309:06:163
2905799504,25cyclictest1243-21snmpd07:46:503
2905799504,25cyclictest0-21swapper/309:36:263
29057995032,11cyclictest6126-21hwlatdetect10:46:343
2905799503,26cyclictest0-21swapper/307:28:023
2905799503,26cyclictest0-21swapper/307:28:023
29057995018,14cyclictest0-21swapper/311:16:383
2905699504,28cyclictest0-21swapper/210:56:262
2905699504,25cyclictest0-21swapper/209:31:272
2905699504,25cyclictest0-21swapper/209:31:272
2905699504,25cyclictest0-21swapper/208:46:282
2905699503,27cyclictest0-21swapper/210:41:422
29056995026,9cyclictest0-21swapper/207:44:512
29056995021,14cyclictest0-21swapper/207:51:362
29056995020,12cyclictest0-21swapper/210:30:562
2905299505,27cyclictest0-21swapper/110:42:541
2905299505,26cyclictest0-21swapper/107:41:471
2905299504,28cyclictest0-21swapper/110:46:301
2905299504,27cyclictest0-21swapper/108:11:291
2905299504,26cyclictest0-21swapper/110:16:321
29052995019,14cyclictest0-21swapper/107:31:381
2904999504,40cyclictest0-21swapper/009:21:450
2904999503,41cyclictest0-21swapper/012:21:140
2904999503,41cyclictest0-21swapper/012:06:290
2904999503,41cyclictest0-21swapper/011:26:250
2904999503,41cyclictest0-21swapper/011:01:340
2904999503,41cyclictest0-21swapper/010:36:250
2904999503,41cyclictest0-21swapper/007:31:390
264712502,9sleep30-21swapper/309:15:193
155182503,42sleep10-21swapper/109:58:061
120952504,9sleep00-21swapper/008:44:570
105752502,42sleep00-21swapper/010:56:340
83212492,9sleep30-21swapper/311:58:443
2905799495,27cyclictest0-21swapper/308:40:373
2905799494,28cyclictest0-21swapper/312:01:363
2905799494,28cyclictest0-21swapper/310:56:403
2905799494,27cyclictest0-21swapper/310:51:503
2905799494,27cyclictest0-21swapper/310:51:503
2905799494,27cyclictest0-21swapper/308:51:283
2905799494,27cyclictest0-21swapper/308:51:283
2905799494,27cyclictest0-21swapper/307:56:503
2905799494,25cyclictest0-21swapper/308:56:293
29057994919,13cyclictest0-21swapper/307:51:323
2905699495,28cyclictest0-21swapper/210:26:132
2905699495,28cyclictest0-21swapper/208:01:392
2905699495,26cyclictest0-21swapper/211:16:302
2905699494,29cyclictest0-21swapper/210:16:502
2905699494,28cyclictest0-21swapper/210:31:262
2905699494,25cyclictest0-21swapper/208:26:492
2905699494,25cyclictest0-21swapper/207:36:392
2905699494,24cyclictest0-21swapper/212:31:212
2905699494,24cyclictest0-21swapper/211:36:232
2905699494,24cyclictest0-21swapper/211:36:232
2905699494,24cyclictest0-21swapper/209:46:262
29056994927,11cyclictest0-21swapper/207:26:522
29056994927,11cyclictest0-21swapper/207:26:522
29056994921,13cyclictest0-21swapper/208:31:492
29056994918,12cyclictest0-21swapper/212:01:252
29056994917,13cyclictest0-21swapper/209:56:472
2905299494,26cyclictest0-21swapper/109:25:441
29052994915,14cyclictest0-21swapper/109:01:441
2904999494,40cyclictest0-21swapper/007:51:470
2904999494,37cyclictest0-21swapper/012:01:220
2904999493,40cyclictest0-21swapper/012:31:160
2904999493,40cyclictest0-21swapper/012:16:290
2904999493,40cyclictest0-21swapper/012:16:290
2904999493,40cyclictest0-21swapper/011:21:240
2904999493,40cyclictest0-21swapper/010:51:480
2904999493,40cyclictest0-21swapper/010:51:480
2904999493,40cyclictest0-21swapper/010:01:260
2904999493,40cyclictest0-21swapper/009:26:290
2904999493,40cyclictest0-21swapper/007:16:470
2904999493,26cyclictest0-21swapper/011:16:100
261672492,9sleep30-21swapper/311:27:283
24072492,9sleep30-21swapper/310:39:323
225522493,9sleep10-21swapper/112:27:361
214572493,40sleep10-21swapper/111:18:151
307162483,39sleep10-21swapper/107:06:501
2905799484,27cyclictest0-21swapper/310:21:443
2905799484,27cyclictest0-21swapper/308:31:373
29057994832,10cyclictest12879-21unixbench-2d09:51:473
29057994830,11cyclictest19070-21fschecks_count10:06:333
29057994830,11cyclictest11476-21fgrep08:41:463
29057994827,14cyclictest0-21swapper/310:11:313
29057994827,14cyclictest0-21swapper/310:11:313
29057994826,8cyclictest31571-21missed_timers08:16:413
2905699485,27cyclictest1243-21snmpd11:51:092
2905699485,27cyclictest0-21swapper/208:40:592
2905699484,24cyclictest0-21swapper/211:26:252
2905699484,23cyclictest0-21swapper/211:51:212
2905699484,13cyclictest0-21swapper/210:11:232
2905699484,13cyclictest0-21swapper/210:11:232
2905699483,38cyclictest17240-21aten2.4-expect11:11:222
2905699483,29cyclictest0-21swapper/211:31:272
2905299485,26cyclictest0-21swapper/110:01:241
2905299485,14cyclictest0-21swapper/111:36:351
2905299485,14cyclictest0-21swapper/111:36:351
2905299484,26cyclictest0-21swapper/111:31:191
2905299484,25cyclictest0-21swapper/109:46:261
2905299484,14cyclictest0-21swapper/108:21:571
29052994831,11cyclictest28524-21unixbench_singl09:16:491
29052994825,14cyclictest0-21swapper/112:06:251
29052994819,9cyclictest8166-21diskstats10:51:331
29052994819,9cyclictest8166-21diskstats10:51:331
29052994815,11cyclictest8726-21latency08:36:411
2904999484,39cyclictest0-21swapper/007:45:140
2904999483,39cyclictest0-21swapper/010:16:260
2904999483,39cyclictest0-21swapper/009:36:250
2904999483,39cyclictest0-21swapper/007:46:350
2904999483,39cyclictest0-21swapper/007:21:320
86562473,38sleep00-21swapper/007:31:040
86562473,38sleep00-21swapper/007:31:040
2905799475,26cyclictest0-21swapper/307:43:013
2905799475,25cyclictest0-21swapper/309:35:563
2905799475,25cyclictest0-21swapper/309:35:563
2905799474,26cyclictest0-21swapper/311:21:243
2905799474,26cyclictest0-21swapper/309:25:443
2905799474,26cyclictest0-21swapper/309:19:583
2905699475,14cyclictest0-21swapper/208:11:422
2905699474,24cyclictest0-21swapper/212:16:212
2905699474,24cyclictest0-21swapper/212:16:212
29052994728,12cyclictest31216-21gltestperf08:16:361
29052994717,12cyclictest0-21swapper/109:41:371
2904999474,36cyclictest0-21swapper/008:16:340
2904999473,38cyclictest0-21swapper/009:16:270
29049994723,9cyclictest32484-21awk10:36:110
29057994617,10cyclictest9099-21cpuspeed_turbos09:46:273
2905699464,27cyclictest0-21swapper/211:41:432
2905299464,35cyclictest0-21swapper/108:01:501
2905299464,24cyclictest0-21swapper/109:51:261
29052994631,10cyclictest4810-21gltestperf11:51:301
29052994618,9cyclictest19308-21iostat12:21:301
29052994617,12cyclictest0-21swapper/111:11:271
29052994614,14cyclictest0-21swapper/107:11:331
2904999463,37cyclictest0-21swapper/009:56:440
2904999463,37cyclictest0-21swapper/009:31:280
2904999463,37cyclictest0-21swapper/009:31:280
276082462,9sleep30-21swapper/308:09:043
231552462,9sleep30-21swapper/310:16:223
2905799454,13cyclictest0-21swapper/311:01:403
29052994528,10cyclictest13219-21gltestperf11:01:311
2905799425,31cyclictest0-21swapper/307:31:303
2905699402,24cyclictest0-21swapper/207:06:202
2904999383,30cyclictest0-21swapper/007:06:200
2905299333,25cyclictest0-21swapper/107:06:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional