You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-16 - 22:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Thu Oct 16, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1672999954,28cyclictest0-21swapper/208:51:322
1672999954,27cyclictest0-21swapper/209:46:402
1672999954,27cyclictest0-21swapper/209:46:402
16729999319,12cyclictest0-21swapper/207:36:412
16729999319,12cyclictest0-21swapper/207:36:412
1672999915,26cyclictest0-21swapper/206:36:502
1672999914,26cyclictest0-21swapper/205:51:512
1672999913,26cyclictest0-21swapper/205:46:422
1672999905,27cyclictest0-21swapper/206:31:432
16729998910,23cyclictest0-21swapper/210:16:302
1672999884,27cyclictest0-21swapper/206:02:012
16729998819,12cyclictest0-21swapper/206:10:442
1672999875,23cyclictest0-21swapper/206:11:342
1672999875,23cyclictest0-21swapper/206:11:342
1672999874,75cyclictest0-21swapper/207:36:022
16729998719,12cyclictest0-21swapper/209:11:472
223099860,4rtkit-daemon2229-21rtkit-daemon05:43:143
1672999864,26cyclictest0-21swapper/209:51:462
1672999864,25cyclictest0-21swapper/210:21:232
1672999864,12cyclictest0-21swapper/208:41:172
223099840,5rtkit-daemon2229-21rtkit-daemon10:21:263
1672999843,26cyclictest0-21swapper/206:21:352
16729998419,12cyclictest0-21swapper/207:01:402
223099830,5rtkit-daemon2229-21rtkit-daemon10:01:343
1672999834,27cyclictest0-21swapper/208:01:522
1672999834,27cyclictest0-21swapper/205:21:482
1672999834,25cyclictest0-21swapper/208:41:272
1672999834,22cyclictest0-21swapper/204:59:162
1672999833,26cyclictest0-21swapper/209:16:392
223099820,5rtkit-daemon2229-21rtkit-daemon09:51:423
223099820,5rtkit-daemon2229-21rtkit-daemon07:55:093
223099820,5rtkit-daemon2229-21rtkit-daemon05:46:593
1672999825,25cyclictest0-21swapper/207:15:582
223099810,5rtkit-daemon2229-21rtkit-daemon06:26:353
1672999815,25cyclictest0-21swapper/208:57:182
1672999814,28cyclictest0-21swapper/209:31:332
16729998129,45cyclictest0-21swapper/207:06:442
223099800,4rtkit-daemon2229-21rtkit-daemon09:06:293
223099800,4rtkit-daemon2229-21rtkit-daemon09:06:293
1672999804,26cyclictest0-21swapper/207:51:532
1672999804,12cyclictest0-21swapper/208:51:152
16729998011,8cyclictest106250irq/46-eth008:11:552
223099790,5rtkit-daemon2229-21rtkit-daemon09:05:363
223099790,5rtkit-daemon2229-21rtkit-daemon07:31:323
223099790,5rtkit-daemon2229-21rtkit-daemon05:26:523
223099790,5rtkit-daemon2229-21rtkit-daemon05:26:523
1672999795,17cyclictest0-21swapper/205:56:512
1672999794,25cyclictest0-21swapper/206:47:182
1672999794,22cyclictest0-21swapper/205:26:382
1672999794,22cyclictest0-21swapper/205:26:382
1672999794,13cyclictest0-21swapper/206:51:342
1672999794,13cyclictest0-21swapper/206:51:342
1672999793,25cyclictest0-21swapper/205:11:382
16729997911,12cyclictest0-21swapper/210:11:292
223099780,4rtkit-daemon2229-21rtkit-daemon10:06:343
223099780,4rtkit-daemon2229-21rtkit-daemon08:51:373
223099780,4rtkit-daemon2229-21rtkit-daemon06:41:493
1672999784,29cyclictest0-21swapper/207:16:532
1672999784,24cyclictest1160-21snmpd07:56:472
1672999784,13cyclictest0-21swapper/208:23:462
1672999784,13cyclictest0-21swapper/208:23:462
1672999783,24cyclictest0-21swapper/210:01:262
1672999783,12cyclictest0-21swapper/207:41:352
16729997824,9cyclictest1265-21runrttasks07:26:422
16729997821,11cyclictest22558-21crond08:31:172
16729997816,13cyclictest0-21swapper/206:16:342
1672999777,11cyclictest0-21swapper/210:00:592
1672999775,22cyclictest0-21swapper/206:41:552
1672999774,32cyclictest0-21swapper/205:01:482
1672999774,14cyclictest0-21swapper/205:16:532
1672999774,12cyclictest0-21swapper/208:11:072
1672999774,12cyclictest0-21swapper/205:06:432
16729997722,10cyclictest1265-21runrttasks05:41:382
223099760,6rtkit-daemon0-21swapper/306:55:263
223099760,6rtkit-daemon0-21swapper/306:55:263
223099760,4rtkit-daemon2229-21rtkit-daemon07:06:413
1672999765,26cyclictest0-21swapper/206:59:502
1672999764,9cyclictest0-21swapper/205:41:242
1672999764,32cyclictest0-21swapper/206:26:522
1672999764,26cyclictest0-21swapper/209:01:372
1672999754,30cyclictest0-21swapper/209:41:152
1672999754,12cyclictest0-21swapper/209:21:462
223099740,4rtkit-daemon2229-21rtkit-daemon09:59:493
1672999744,23cyclictest0-21swapper/209:06:302
1672999744,23cyclictest0-21swapper/209:06:302
1672999744,23cyclictest0-21swapper/207:46:312
1672999744,14cyclictest0-21swapper/209:26:372
16729997420,10cyclictest23397-21grep07:21:542
16729997410,12cyclictest0-21swapper/205:31:532
1672999734,23cyclictest0-21swapper/208:16:322
1672999734,20cyclictest3617-21munin-node10:06:502
16729997321,8cyclictest23485-21sort08:31:362
223099720,6rtkit-daemon2229-21rtkit-daemon05:21:453
223099720,5rtkit-daemon2229-21rtkit-daemon09:46:353
223099720,5rtkit-daemon2229-21rtkit-daemon09:46:353
223099720,5rtkit-daemon2229-21rtkit-daemon08:56:373
223099720,5rtkit-daemon2229-21rtkit-daemon06:06:433
223099710,5rtkit-daemon2229-21rtkit-daemon07:36:473
223099710,5rtkit-daemon2229-21rtkit-daemon07:36:473
1672999719,12cyclictest0-21swapper/209:46:142
223099700,6rtkit-daemon0-21swapper/309:35:503
223099700,5rtkit-daemon2229-21rtkit-daemon05:16:453
223099690,5rtkit-daemon2229-21rtkit-daemon09:11:513
223099690,5rtkit-daemon2229-21rtkit-daemon08:16:083
223099650,4rtkit-daemon2229-21rtkit-daemon08:44:003
223099650,4rtkit-daemon2229-21rtkit-daemon07:15:033
223099640,4rtkit-daemon2229-21rtkit-daemon09:23:213
223099630,4rtkit-daemon2229-21rtkit-daemon08:34:223
1672499614,50cyclictest0-21swapper/005:21:250
52002573,10sleep30-21swapper/305:36:553
150992573,48sleep00-21swapper/008:12:300
65902563,45sleep00-21swapper/009:01:510
1673099564,27cyclictest0-21swapper/308:26:323
1672499563,16cyclictest0-21swapper/006:16:520
99682553,46sleep00-21swapper/009:11:310
260792552,46sleep00-21swapper/007:30:020
16724995540,9cyclictest8497-21aten2.4-expect08:01:300
129882553,45sleep10-21swapper/108:11:061
223099540,5rtkit-daemon2229-21rtkit-daemon04:55:092
16730995421,13cyclictest0-21swapper/306:16:513
80532533,44sleep00-21swapper/007:58:560
1673099535,28cyclictest0-21swapper/308:21:503
1673099535,28cyclictest0-21swapper/308:21:503
1673099535,28cyclictest0-21swapper/307:02:053
1673099534,29cyclictest0-21swapper/308:46:403
1673099534,28cyclictest0-21swapper/310:16:243
16727995322,14cyclictest0-21swapper/105:46:191
165432534,8sleep116596-21latency_hist09:26:151
152682532,10sleep00-21swapper/007:06:410
56402523,42sleep10-21swapper/105:38:021
4872524,13sleep00-21swapper/006:36:380
296132523,9sleep10-21swapper/108:45:001
236102523,10sleep10-21swapper/107:23:591
235522522,44sleep00-21swapper/005:07:220
182332523,42sleep10-21swapper/104:56:551
1673099525,28cyclictest0-21swapper/307:56:373
1673099525,28cyclictest0-21swapper/304:56:453
1673099524,29cyclictest0-21swapper/309:26:353
1673099524,29cyclictest0-21swapper/308:16:513
1673099524,28cyclictest0-21swapper/307:21:543
16730995221,13cyclictest0-21swapper/306:56:383
16727995236,11cyclictest9962-21cpuspeed_turbos09:11:301
16724995237,9cyclictest9315-21wc10:16:340
154182523,9sleep10-21swapper/106:00:481
97392512,9sleep00-21swapper/006:56:330
68622512,42sleep10-21swapper/109:04:301
320752513,42sleep00-21swapper/008:50:430
259042513,10sleep10-21swapper/109:43:441
203432513,42sleep320351-21iostat_ios05:01:483
19282513,42sleep00-21swapper/008:53:570
1673099514,28cyclictest0-21swapper/307:41:413
1673099514,28cyclictest0-21swapper/306:46:453
1673099514,28cyclictest0-21swapper/306:40:503
1673099514,28cyclictest0-21swapper/306:11:413
1673099514,28cyclictest0-21swapper/306:11:413
1673099514,27cyclictest0-21swapper/305:11:263
1673099513,35cyclictest0-21swapper/309:41:263
16730995120,13cyclictest0-21swapper/308:36:283
1672799513,12cyclictest0-21swapper/106:36:421
16727995117,29cyclictest0-21swapper/105:46:541
16727995115,29cyclictest20481-21aten_r1power_fr08:26:311
16724995136,10cyclictest0-21swapper/009:56:340
129962513,9sleep10-21swapper/105:55:191
129902513,42sleep113063-21munin-node07:01:421
102692513,10sleep10-21swapper/110:17:421
6302502,42sleep00-21swapper/005:27:170
6302502,42sleep00-21swapper/005:27:170
56092502,9sleep00-21swapper/007:53:160
44532502,41sleep00-21swapper/008:59:130
43972503,10sleep10-21swapper/108:58:301
320932503,9sleep00-21swapper/006:35:250
237392503,9sleep10-21swapper/105:09:491
1673099504,28cyclictest0-21swapper/308:06:383
1673099504,28cyclictest0-21swapper/308:01:483
1673099504,28cyclictest0-21swapper/307:26:463
1673099504,27cyclictest0-21swapper/306:06:233
1673099504,27cyclictest0-21swapper/305:11:473
16730995020,11cyclictest0-21swapper/309:16:193
1672799503,41cyclictest22906-21sshd09:36:431
1672799503,41cyclictest18103-21munin-node06:06:531
1672799503,40cyclictest5942-21cat10:11:141
1672499505,18cyclictest0-21swapper/004:56:260
164972502,41sleep00-21swapper/009:25:410
80112492,9sleep00-21swapper/010:14:220
33812492,9sleep00-21swapper/005:34:470
315712493,9sleep10-21swapper/109:56:301
306902492,9sleep10-21swapper/109:53:391
283712493,39sleep10-21swapper/107:33:501
259082493,9sleep00-21swapper/005:12:010
224562492,10sleep00-21swapper/006:14:090
224562492,10sleep00-21swapper/006:14:090
211162493,40sleep10-21swapper/107:17:341
1673099494,36cyclictest0-21swapper/306:21:473
1673099494,28cyclictest0-21swapper/305:31:573
1673099494,18cyclictest0-21swapper/307:16:313
16730994932,10cyclictest30371-21cpuspeed_turbos06:31:363
1673099492,41cyclictest0-21swapper/305:57:343
1672799493,40cyclictest0-21swapper/109:21:151
16727994921,11cyclictest0-21swapper/106:43:161
1672499494,20cyclictest0-21swapper/009:35:360
1672499493,40cyclictest0-21swapper/007:36:310
1672499493,40cyclictest0-21swapper/007:36:310
1672499493,39cyclictest0-21swapper/006:52:190
1672499493,39cyclictest0-21swapper/006:52:190
16724994925,19cyclictest0-21swapper/008:21:410
16724994925,19cyclictest0-21swapper/008:21:410
127582493,40sleep10-21swapper/110:24:021
78612482,9sleep10-21swapper/110:12:261
77432483,38sleep10-21swapper/106:51:381
77432483,38sleep10-21swapper/106:51:381
49022483,38sleep00-21swapper/006:46:340
258892483,39sleep10-21swapper/107:27:331
223232482,39sleep00-21swapper/008:28:140
200482482,9sleep10-21swapper/108:24:441
200482482,9sleep10-21swapper/108:24:441
1673099484,26cyclictest0-21swapper/309:36:323
1672799488,34cyclictest22629-21crond06:16:221
1672799488,34cyclictest22629-21crond06:16:221
1672799484,26cyclictest0-21swapper/106:50:511
16727994833,10cyclictest0-21swapper/109:46:441
16727994833,10cyclictest0-21swapper/109:46:441
16727994831,10cyclictest30356-21tr06:31:361
1672799483,39cyclictest31185-21munin-node07:41:551
1672799482,40cyclictest0-21swapper/109:26:451
16724994832,10cyclictest4279-21tune2fs10:06:320
1672499483,40cyclictest0-21swapper/005:37:000
1672499483,38cyclictest13416-21telnet05:56:360
6082472,10sleep00-21swapper/007:42:520
258042472,10sleep00-21swapper/009:42:260
247702472,10sleep00-21swapper/008:34:020
189662472,10sleep10-21swapper/107:15:411
178142472,38sleep00-21swapper/006:05:500
16730994729,10cyclictest1213-21aten_r1power_en07:46:313
1672799473,38cyclictest0-21swapper/108:16:301
16727994728,13cyclictest0-21swapper/108:51:341
16727994727,14cyclictest0-21swapper/109:31:301
16724994731,11cyclictest0-21swapper/008:06:480
1672499473,39cyclictest0-21swapper/008:42:360
1672499473,26cyclictest0-21swapper/007:11:320
16724994720,8cyclictest18945-21latency_hist09:31:150
16724994716,24cyclictest28870-21tail09:51:280
150732473,38sleep10-21swapper/108:12:111
117862473,38sleep10-21swapper/107:00:291
84632462,9sleep10-21swapper/108:01:231
79652463,37sleep10-21swapper/107:57:481
271742463,9sleep10-21swapper/108:39:151
233952462,37sleep00-21swapper/009:37:060
197582463,37sleep00-21swapper/005:01:410
1673099464,26cyclictest0-21swapper/305:55:423
16727994632,9cyclictest5464-21unixbench_singl07:51:551
1672799463,15cyclictest0-21swapper/107:46:401
16727994617,24cyclictest0-21swapper/109:06:451
16727994617,24cyclictest0-21swapper/109:06:451
16727994610,28cyclictest0-21swapper/106:21:351
1672499464,12cyclictest0-21swapper/006:41:420
16724994624,10cyclictest0-21swapper/006:21:550
16724994623,11cyclictest0-21swapper/008:36:390
16724994618,8cyclictest20249-21latency_hist06:11:230
73472453,9sleep00-21swapper/005:41:480
307262452,38sleep10-21swapper/105:22:341
237902452,37sleep00-21swapper/007:26:200
176492452,36sleep00-21swapper/008:19:320
1672799456,33cyclictest25829-21unixbench_multi05:11:591
16727994531,9cyclictest525-21grep05:26:591
16727994531,9cyclictest525-21grep05:26:591
16727994531,10cyclictest16241-21sed07:06:541
1672799453,37cyclictest0-21swapper/105:01:381
1672799453,23cyclictest29954-21munin-node08:46:471
16727994527,12cyclictest0-21swapper/108:31:291
16727994525,15cyclictest0-21swapper/106:26:481
16727994520,10cyclictest0-21swapper/110:01:251
1672499454,35cyclictest0-21swapper/005:46:520
1672499453,37cyclictest0-21swapper/004:56:540
1672499453,35cyclictest0-21swapper/005:21:350
16724994529,11cyclictest0-21swapper/007:31:440
16724994528,11cyclictest0-21swapper/007:46:320
309262443,10sleep10-21swapper/107:41:041
309262443,10sleep10-21swapper/107:41:041
1672799443,34cyclictest0-21swapper/106:16:371
16727994424,14cyclictest0-21swapper/105:31:391
1672799442,37cyclictest0-21swapper/105:16:371
1672499444,12cyclictest0-21swapper/007:16:340
1672499443,34cyclictest0-21swapper/010:21:300
16724994429,11cyclictest0-21swapper/007:01:530
1672499442,38cyclictest0-21swapper/009:16:440
176062432,35sleep10-21swapper/106:03:051
1672499433,35cyclictest0-21swapper/009:46:420
1672499433,35cyclictest0-21swapper/009:46:420
1672499432,36cyclictest0-21swapper/006:26:380
1673099424,32cyclictest6706-21fschecks_time10:11:313
1672499424,31cyclictest0-21swapper/010:01:440
1672499423,34cyclictest0-21swapper/009:06:310
1672499423,34cyclictest0-21swapper/009:06:310
16724994225,12cyclictest0-21swapper/005:51:350
164902394,28sleep30-21swapper/304:53:513
1672799338,11cyclictest0-21swapper/104:56:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional