You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 14:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Thu Dec 25, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1872499954,12cyclictest0-21swapper/311:23:363
18724999219,50cyclictest0-21swapper/312:13:363
285342910,6sleep128535-21munin-node10:38:441
1872499914,15cyclictest0-21swapper/311:10:353
1872499904,65cyclictest31650-21hostname11:53:323
1872499894,62cyclictest0-21swapper/311:48:263
1872499894,62cyclictest0-21swapper/311:48:263
1872499894,13cyclictest0-21swapper/310:33:293
1872499894,13cyclictest0-21swapper/309:28:193
1872499884,67cyclictest0-21swapper/310:48:403
1872499884,11cyclictest0-21swapper/312:13:123
1872499875,57cyclictest0-21swapper/310:53:273
1872499874,58cyclictest0-21swapper/309:38:333
1872499874,13cyclictest0-21swapper/311:13:403
1872499874,12cyclictest0-21swapper/312:28:123
18724998722,44cyclictest0-21swapper/308:23:223
1872499864,58cyclictest0-21swapper/308:53:303
1872499864,58cyclictest0-21swapper/308:53:303
1872499864,14cyclictest0-21swapper/307:38:353
1872499864,13cyclictest0-21swapper/310:38:393
1872499863,58cyclictest1344-21diskstats09:43:343
18724998610,64cyclictest0-21swapper/311:38:293
18724998516,48cyclictest0-21swapper/307:12:463
1872499844,63cyclictest0-21swapper/309:58:373
1872499844,59cyclictest22458-21ntp_states11:33:423
1872499844,57cyclictest0-21swapper/309:18:323
1872499844,11cyclictest0-21swapper/308:43:543
1872499834,62cyclictest0-21swapper/307:07:403
1872499834,53cyclictest21091-21turbostat.cron07:03:253
1872499834,12cyclictest0-21swapper/309:13:463
1872499834,12cyclictest0-21swapper/307:33:353
1872499834,12cyclictest0-21swapper/307:33:353
1872499834,11cyclictest16770-21cpuspeed_turbos09:08:333
1872499833,12cyclictest0-21swapper/310:43:213
1872499825,61cyclictest0-21swapper/309:53:313
1872499825,55cyclictest0-21swapper/307:13:453
1872499824,59cyclictest0-21swapper/311:33:143
1872499823,53cyclictest0-21swapper/310:08:503
1872499817,51cyclictest0-21swapper/307:53:333
1872499814,60cyclictest0-21swapper/310:28:193
1872499814,60cyclictest0-21swapper/309:28:303
1872499813,58cyclictest0-21swapper/308:12:503
18724998115,51cyclictest2006-21yum-updatesd-he08:38:323
1872499804,60cyclictest0-21swapper/308:03:233
1872499804,53cyclictest0-21swapper/309:33:453
1872499804,15cyclictest15232-21munin-node10:13:483
1872499804,12cyclictest0-21swapper/310:23:243
1872499804,12cyclictest0-21swapper/310:23:243
1872499803,57cyclictest31799-21awk08:33:213
18724998012,45cyclictest0-21swapper/308:33:323
1872499796,66cyclictest0-21swapper/311:03:163
1872499796,62cyclictest0-21swapper/309:53:193
1872499794,58cyclictest0-21swapper/311:18:263
1872499794,56cyclictest0-21swapper/308:03:353
1872499794,12cyclictest0-21swapper/307:25:143
18724997911,59cyclictest0-21swapper/311:58:363
1872499785,51cyclictest0-21swapper/309:03:373
1872499784,59cyclictest11413-21uname12:18:373
1872499784,55cyclictest0-21swapper/308:48:333
1872499784,54cyclictest0-21swapper/312:03:253
1872499784,51cyclictest0-21swapper/310:08:173
1872499784,16cyclictest0-21swapper/311:43:433
18724997814,40cyclictest0-21swapper/307:43:343
18724997811,11cyclictest121850irq/46-eth008:58:383
18724997811,11cyclictest121850irq/46-eth008:58:383
1872499775,56cyclictest0-21swapper/307:53:233
1872499774,53cyclictest0-21swapper/311:03:393
1872499774,53cyclictest0-21swapper/311:03:393
1872499765,9cyclictest0-21swapper/310:20:363
1872499749,7cyclictest29118-21grep07:18:443
1872499734,12cyclictest2341-21sshd07:28:513
1872499733,61cyclictest22477-21munin-node08:13:413
1872499733,61cyclictest22477-21munin-node08:13:413
1872499724,13cyclictest0-21swapper/308:23:373
189412690,3sleep041ktimersoftd/009:13:320
1872499664,12cyclictest0-21swapper/306:58:253
1872499664,12cyclictest0-21swapper/306:58:253
248599630,5rtkit-daemon2484-21rtkit-daemon06:58:250
248599630,5rtkit-daemon2484-21rtkit-daemon06:58:250
318922614,49sleep10-21swapper/107:23:471
211202600,3sleep021168-21latency08:08:480
18722995926,13cyclictest0-21swapper/109:28:191
161472583,19sleep10-21swapper/111:23:111
248599570,6rtkit-daemon2484-21rtkit-daemon10:38:390
248599570,5rtkit-daemon2484-21rtkit-daemon09:58:180
248599570,5rtkit-daemon2484-21rtkit-daemon07:38:540
248599560,5rtkit-daemon2484-21rtkit-daemon08:03:450
248599550,6rtkit-daemon2484-21rtkit-daemon10:16:550
18723995538,10cyclictest21486-21grep11:33:292
18723995536,12cyclictest12822-21cut11:13:382
263032542,44sleep00-21swapper/008:18:510
248599540,5rtkit-daemon2484-21rtkit-daemon09:43:410
248599540,5rtkit-daemon2484-21rtkit-daemon08:38:440
248599540,5rtkit-daemon2484-21rtkit-daemon08:33:510
248599540,5rtkit-daemon2484-21rtkit-daemon08:28:370
248599540,5rtkit-daemon2484-21rtkit-daemon07:45:010
248599540,5rtkit-daemon2484-21rtkit-daemon07:33:460
248599540,5rtkit-daemon2484-21rtkit-daemon07:33:460
1872399544,42cyclictest0-21swapper/207:13:252
1872399544,32cyclictest0-21swapper/212:13:452
1872299544,12cyclictest0-21swapper/109:38:391
18721995413,11cyclictest0-21swapper/007:58:430
248599530,6rtkit-daemon2484-21rtkit-daemon09:23:350
248599530,5rtkit-daemon2484-21rtkit-daemon10:33:470
248599530,5rtkit-daemon2484-21rtkit-daemon10:28:320
248599530,5rtkit-daemon2484-21rtkit-daemon10:18:400
248599530,5rtkit-daemon2484-21rtkit-daemon10:03:400
248599530,5rtkit-daemon2484-21rtkit-daemon09:50:480
248599530,5rtkit-daemon2484-21rtkit-daemon09:36:490
248599530,5rtkit-daemon2484-21rtkit-daemon08:48:360
248599530,5rtkit-daemon2484-21rtkit-daemon07:13:470
1872399536,27cyclictest8947-21gltestperf07:43:412
1872399534,31cyclictest0-21swapper/207:18:562
1872399534,30cyclictest0-21swapper/210:13:362
1872399534,25cyclictest0-21swapper/207:06:532
18723995335,11cyclictest13596-21gltestperf12:23:312
18722995322,24cyclictest0-21swapper/112:13:361
18721995336,10cyclictest21589-21tr09:18:350
111682534,42sleep10-21swapper/108:54:441
111682534,42sleep10-21swapper/108:54:441
100492533,44sleep10-21swapper/110:01:011
272312522,43sleep00-21swapper/011:43:410
248599520,5rtkit-daemon2484-21rtkit-daemon09:43:190
248599520,5rtkit-daemon2484-21rtkit-daemon08:58:500
248599520,5rtkit-daemon2484-21rtkit-daemon08:58:500
248599520,5rtkit-daemon2484-21rtkit-daemon08:55:280
248599520,5rtkit-daemon2484-21rtkit-daemon08:55:280
248599520,5rtkit-daemon2484-21rtkit-daemon07:53:230
248599520,5rtkit-daemon2484-21rtkit-daemon07:28:470
243402523,10sleep10-21swapper/110:30:271
18723995213,24cyclictest20026-21aten_r1power_cu10:23:272
18723995213,24cyclictest20026-21aten_r1power_cu10:23:272
1872299524,11cyclictest0-21swapper/111:38:361
18722995237,9cyclictest21221-21tail10:23:411
18722995237,9cyclictest21221-21tail10:23:411
18721995236,9cyclictest28792-21tr07:18:400
99792513,9sleep10-21swapper/107:44:301
256142513,42sleep00-21swapper/011:41:560
248599510,5rtkit-daemon2484-21rtkit-daemon08:23:370
232282514,40sleep10-21swapper/108:13:401
232282514,40sleep10-21swapper/108:13:401
1872399518,36cyclictest31724-21wc09:38:392
1872399514,29cyclictest0-21swapper/211:38:372
1872399514,29cyclictest0-21swapper/208:48:322
1872399514,28cyclictest0-21swapper/208:38:362
1872299515,29cyclictest0-21swapper/110:33:311
1872299515,13cyclictest0-21swapper/108:03:221
1872299514,41cyclictest0-21swapper/107:38:401
113752512,9sleep00-21swapper/011:13:140
317352503,12sleep20-21swapper/211:53:342
305212503,41sleep10-21swapper/107:21:571
248599500,5rtkit-daemon2484-21rtkit-daemon10:08:400
208502504,40sleep10-21swapper/109:16:121
207442502,10sleep00-21swapper/006:58:590
195382504,39sleep10-21swapper/108:04:171
1872399504,29cyclictest0-21swapper/207:38:412
1872399504,28cyclictest0-21swapper/211:43:402
1872399504,28cyclictest0-21swapper/209:38:192
18723995020,13cyclictest0-21swapper/208:18:522
18723995011,22cyclictest927-21chrt07:28:342
1872299505,27cyclictest0-21swapper/111:53:321
1872299505,27cyclictest0-21swapper/109:34:021
1872299505,27cyclictest0-21swapper/108:48:571
1872299504,27cyclictest0-21swapper/106:58:361
1872299504,26cyclictest0-21swapper/107:07:401
184362503,10sleep10-21swapper/109:10:561
160942502,8sleep20-21swapper/209:06:332
88472493,40sleep00-21swapper/011:06:250
88472493,40sleep00-21swapper/011:06:250
248599490,5rtkit-daemon2484-21rtkit-daemon07:13:250
244632492,9sleep20-21swapper/208:16:152
244632492,9sleep20-21swapper/208:16:152
206622493,9sleep10-21swapper/111:29:391
1872399495,36cyclictest0-21swapper/208:58:372
1872399495,36cyclictest0-21swapper/208:58:372
1872399495,27cyclictest0-21swapper/210:18:422
1872399494,28cyclictest0-21swapper/211:08:352
1872399494,27cyclictest0-21swapper/211:58:292
1872399494,27cyclictest0-21swapper/211:48:322
1872399494,27cyclictest0-21swapper/211:48:322
1872399494,27cyclictest0-21swapper/210:58:312
1872399494,27cyclictest0-21swapper/209:53:362
1872399494,27cyclictest0-21swapper/209:28:362
1872399494,27cyclictest0-21swapper/208:38:212
1872399494,27cyclictest0-21swapper/207:33:402
1872399494,27cyclictest0-21swapper/207:33:402
1872399494,24cyclictest0-21swapper/209:58:302
18723994933,10cyclictest30453-21grep08:28:412
18723994932,10cyclictest23156-21tune2fs10:28:342
18723994931,11cyclictest21259-21cat08:08:492
18723994931,10cyclictest6293-21tune2fs12:08:312
18723994930,11cyclictest9647-21grep08:53:332
18723994930,11cyclictest9647-21grep08:53:332
18723994922,12cyclictest0-21swapper/210:53:302
18723994920,12cyclictest0-21swapper/206:58:532
1872299495,27cyclictest0-21swapper/109:43:461
1872299495,27cyclictest0-21swapper/108:43:011
1872299494,27cyclictest0-21swapper/110:58:161
1872299494,27cyclictest0-21swapper/110:48:361
18722994922,8cyclictest0-21swapper/110:08:381
18722994921,11cyclictest0-21swapper/112:23:481
18722994916,12cyclictest0-21swapper/111:33:421
18722994916,11cyclictest21257-21munin-node09:18:471
1872199493,40cyclictest0-21swapper/012:23:420
160522493,40sleep00-21swapper/009:05:590
159452493,39sleep00-21swapper/011:20:320
103802493,39sleep10-21swapper/112:18:201
64382482,10sleep00-21swapper/011:01:050
40372482,39sleep00-21swapper/010:55:530
293572483,10sleep10-21swapper/111:48:381
293572483,10sleep10-21swapper/111:48:381
248599480,4rtkit-daemon2484-21rtkit-daemon07:23:470
1872399484,27cyclictest0-21swapper/211:03:442
1872399484,27cyclictest0-21swapper/211:03:442
1872399484,27cyclictest0-21swapper/210:33:442
1872399484,27cyclictest0-21swapper/209:13:492
1872399484,27cyclictest0-21swapper/209:08:482
1872399484,27cyclictest0-21swapper/208:43:362
1872399484,26cyclictest0-21swapper/211:28:152
1872399484,26cyclictest0-21swapper/211:18:372
1872399484,26cyclictest0-21swapper/208:23:532
1872399484,26cyclictest0-21swapper/207:48:532
1872399484,25cyclictest0-21swapper/210:38:272
1872399483,28cyclictest0-21swapper/207:23:362
1872399483,27cyclictest0-21swapper/209:23:462
1872399483,27cyclictest0-21swapper/209:18:422
18723994813,21cyclictest13939-21iostat_ios07:53:432
1872299484,39cyclictest0-21swapper/108:18:441
1872299484,27cyclictest26122-21telnet09:28:311
1872299484,27cyclictest0-21swapper/107:50:501
1872299484,26cyclictest0-21swapper/110:03:471
1872299484,26cyclictest0-21swapper/108:28:511
1872299484,24cyclictest0-21swapper/111:03:491
1872299484,24cyclictest0-21swapper/111:03:491
1872299483,26cyclictest0-21swapper/107:58:231
18722994827,13cyclictest16749-21cpuspeed_turbos11:23:281
18722994819,13cyclictest0-21swapper/107:13:511
18722994819,12cyclictest0-21swapper/110:58:261
18721994834,9cyclictest11313-21hwlatdetect12:18:340
101172483,9sleep10-21swapper/111:08:361
246102472,40sleep00-21swapper/008:18:110
246102472,40sleep00-21swapper/008:18:110
232642472,39sleep00-21swapper/011:37:280
208322472,9sleep00-21swapper/011:31:520
1872399474,26cyclictest0-21swapper/212:18:362
1872399474,26cyclictest0-21swapper/211:28:412
1872399474,26cyclictest0-21swapper/210:48:282
1872399474,26cyclictest0-21swapper/210:43:532
1872399474,26cyclictest0-21swapper/210:08:552
1872399474,26cyclictest0-21swapper/210:03:342
1872399474,26cyclictest0-21swapper/209:43:422
18723994732,9cyclictest3661-21df12:03:282
1872399473,37cyclictest15847-21sed07:58:382
1872399473,27cyclictest0-21swapper/209:48:332
1872299473,29cyclictest0-21swapper/107:08:431
1872299473,26cyclictest0-21swapper/111:58:341
18722994715,15cyclictest0-21swapper/109:03:311
18721994732,10cyclictest32303-21tr10:48:310
18721994721,11cyclictest0-21swapper/011:58:330
75712463,37sleep10-21swapper/112:10:461
295602463,37sleep10-21swapper/110:43:211
280282463,37sleep00-21swapper/009:31:260
219852463,9sleep10-21swapper/108:10:021
1872299464,37cyclictest0-21swapper/110:13:451
1872299464,34cyclictest0-21swapper/111:13:261
1872299464,25cyclictest0-21swapper/110:18:481
1872299463,25cyclictest0-21swapper/111:43:431
18722994628,11cyclictest4796-21ls12:03:451
18722994619,8cyclictest29448-21cat08:28:221
18722994618,9cyclictest3867-21diskstats09:48:361
18721994632,9cyclictest16632-21head09:08:320
18721994630,11cyclictest0-21swapper/008:43:330
18721994630,10cyclictest29109-21tune2fs11:48:330
18721994630,10cyclictest29109-21tune2fs11:48:330
183732462,10sleep00-21swapper/011:26:040
17112463,37sleep10-21swapper/107:28:421
15622463,37sleep10-21swapper/108:35:531
65852453,36sleep10-21swapper/108:47:261
234212453,9sleep00-21swapper/007:07:460
1872399454,20cyclictest0-21swapper/207:18:242
18723994532,8cyclictest17754-21aten_r1power_en08:03:342
18722994530,10cyclictest6057-21tr09:53:341
18721994523,10cyclictest0-21swapper/011:14:240
18721994519,14cyclictest0-21swapper/012:03:250
33542443,10sleep10-21swapper/107:33:341
33542443,10sleep10-21swapper/107:33:341
326422442,37sleep00-21swapper/011:55:010
18722994414,12cyclictest0-21swapper/108:58:331
18722994414,12cyclictest0-21swapper/108:58:331
18721994426,7cyclictest8080-21munin-node09:58:380
248599430,4rtkit-daemon2484-21rtkit-daemon10:25:400
248599430,4rtkit-daemon2484-21rtkit-daemon10:25:400
1872199434,33cyclictest0-21swapper/010:43:360
1872199433,35cyclictest0-21swapper/012:08:350
18721994320,10cyclictest0-21swapper/012:13:360
248599420,4rtkit-daemon2484-21rtkit-daemon07:53:380
1872399372,22cyclictest0-21swapper/206:58:262
1872399372,22cyclictest0-21swapper/206:58:262
186092356,22sleep10-21swapper/106:57:271
186092356,22sleep10-21swapper/106:57:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional