You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-25 - 23:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Tue Mar 25, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1764799934,51cyclictest0-21swapper/008:57:500
1764799925,67cyclictest0-21swapper/008:22:350
1764799894,67cyclictest0-21swapper/009:37:470
1764799874,64cyclictest0-21swapper/010:12:320
1764799873,69cyclictest0-21swapper/009:32:470
1764799873,69cyclictest0-21swapper/009:32:470
1764799864,74cyclictest0-21swapper/010:37:140
1764799864,73cyclictest0-21swapper/009:17:440
1764799864,64cyclictest0-21swapper/007:52:210
1764799863,64cyclictest0-21swapper/010:57:310
1764799855,73cyclictest0-21swapper/007:22:460
1764799855,73cyclictest0-21swapper/007:22:460
1764799854,74cyclictest0-21swapper/008:52:440
1764799854,74cyclictest0-21swapper/008:52:440
1764799854,74cyclictest0-21swapper/005:42:540
1764799854,73cyclictest0-21swapper/010:17:360
1764799854,73cyclictest0-21swapper/010:17:360
1764799854,65cyclictest0-21swapper/005:52:390
1764799854,65cyclictest0-21swapper/005:52:390
1764799853,74cyclictest1138-21snmpd10:46:400
1764799853,66cyclictest0-21swapper/007:52:290
1764799845,71cyclictest0-21swapper/005:42:260
1764799845,64cyclictest0-21swapper/007:17:530
1764799844,73cyclictest0-21swapper/006:12:580
1764799844,63cyclictest0-21swapper/009:52:160
1764799844,59cyclictest0-21swapper/006:32:240
1764799843,65cyclictest0-21swapper/009:52:230
1764799843,64cyclictest0-21swapper/010:37:260
1764799834,72cyclictest0-21swapper/006:07:590
1764799834,60cyclictest0-21swapper/008:37:330
1764799833,72cyclictest0-21swapper/006:37:560
1764799833,72cyclictest0-21swapper/006:37:560
1764799833,62cyclictest0-21swapper/007:11:090
1764799833,59cyclictest28396-21cat07:02:220
1764799824,71cyclictest0-21swapper/010:22:460
1764799824,71cyclictest0-21swapper/010:22:460
1764799824,71cyclictest0-21swapper/007:47:210
1764799824,71cyclictest0-21swapper/006:02:120
1764799824,62cyclictest0-21swapper/006:22:440
1764799815,68cyclictest0-21swapper/009:07:420
1764799814,70cyclictest0-21swapper/010:02:360
1764799814,68cyclictest0-21swapper/005:48:250
1764799814,62cyclictest0-21swapper/010:49:270
1764799814,58cyclictest0-21swapper/007:57:450
1764799813,71cyclictest1263-21runrttasks09:27:160
1764799813,70cyclictest8235-21munin-node07:27:510
1764799813,63cyclictest6994-21df_abs08:42:320
1764799804,60cyclictest0-21swapper/008:27:360
1764799804,59cyclictest0-21swapper/009:57:330
1764799804,58cyclictest0-21swapper/008:03:100
1764799804,58cyclictest0-21swapper/008:03:100
1764799803,62cyclictest0-21swapper/009:02:410
1764799803,52cyclictest0-21swapper/007:32:360
1764799797,42cyclictest90401sshd06:20:140
1764799795,46cyclictest0-21swapper/008:37:160
1764799794,68cyclictest0-21swapper/009:28:080
1764799794,68cyclictest0-21swapper/009:12:290
1764799794,68cyclictest0-21swapper/007:07:220
1764799794,60cyclictest0-21swapper/006:52:210
1764799793,60cyclictest0-21swapper/006:57:210
1764799785,65cyclictest0-21swapper/005:32:530
1764799784,66cyclictest1263-21runrttasks08:12:420
1764799783,60cyclictest0-21swapper/007:13:170
1764799783,57cyclictest0-21swapper/007:37:520
1764799782,66cyclictest0-21swapper/009:42:480
1764799774,62cyclictest0-21swapper/008:17:350
1764799764,52cyclictest0-21swapper/006:02:500
1764799763,43cyclictest19044-21aten2.4-expect06:42:340
1764799748,59cyclictest8050-21awk10:12:150
1764799743,56cyclictest0-21swapper/006:32:400
1764799723,57cyclictest0-21swapper/008:08:170
1764799717,44cyclictest18610-21kworker/0:210:52:210
1764799715,44cyclictest0-21swapper/010:27:550
1764799713,59cyclictest17648-21basename05:32:250
1764799713,58cyclictest0-21swapper/008:48:240
1764799713,58cyclictest0-21swapper/008:48:240
2900426312,42sleep10-21swapper/105:53:011
2900426312,42sleep10-21swapper/105:53:011
122102593,12sleep20-21swapper/210:17:472
122102593,12sleep20-21swapper/210:17:472
21992563,45sleep30-21swapper/309:52:453
1765099565,30cyclictest0-21swapper/108:02:471
1765099565,30cyclictest0-21swapper/108:02:471
17650995639,10cyclictest14570-21cpuspeed_turbos09:02:321
91972552,8sleep11138-21snmpd06:22:161
307162554,44sleep130748-21latency_hist07:07:221
134292553,11sleep20-21swapper/210:22:312
134292553,11sleep20-21swapper/210:22:312
107152552,46sleep10-21swapper/106:22:521
1765099544,31cyclictest0-21swapper/106:37:561
1765099544,31cyclictest0-21swapper/106:37:561
1765099544,26cyclictest0-21swapper/106:27:361
70292532,43sleep17104-21fschecks_count08:42:341
4672532,10sleep10-21swapper/108:24:531
288632533,11sleep30-21swapper/305:52:593
288632533,11sleep30-21swapper/305:52:593
1765299533,43cyclictest32189-21munin-node06:02:523
1765099534,30cyclictest0-21swapper/105:42:471
1765099534,25cyclictest0-21swapper/110:52:211
1765099534,25cyclictest0-21swapper/105:37:381
17650995335,12cyclictest21745-21tr06:47:391
1765099533,44cyclictest0-21swapper/106:46:391
17650995324,22cyclictest5281-21fschecks_count06:12:451
17650995322,13cyclictest0-21swapper/107:17:591
146792533,44sleep10-21swapper/106:32:441
59482522,9sleep30-21swapper/308:38:533
227799520,5rtkit-daemon2276-21rtkit-daemon05:42:402
202952522,10sleep10-21swapper/107:53:571
1765299524,41cyclictest1715-21perl07:12:413
1765199524,41cyclictest0-21swapper/210:58:022
1765199523,42cyclictest1138-21snmpd07:41:342
1765099525,28cyclictest0-21swapper/106:57:571
1765099524,29cyclictest0-21swapper/110:12:451
17650995221,13cyclictest0-21swapper/110:02:161
17650995221,13cyclictest0-21swapper/109:32:471
17650995221,13cyclictest0-21swapper/109:32:471
17650995220,13cyclictest0-21swapper/105:58:021
268502512,10sleep10-21swapper/105:48:141
258672512,10sleep10-21swapper/108:11:261
229612512,43sleep10-21swapper/110:46:561
1765299513,42cyclictest1138-21snmpd08:06:323
1765299513,42cyclictest1138-21snmpd08:06:323
17652995119,15cyclictest0-21swapper/307:57:153
17651995123,8cyclictest3214-21latency_hist07:17:232
1765199512,44cyclictest0-21swapper/207:27:182
1765199512,44cyclictest0-21swapper/207:27:182
1765099516,26cyclictest0-21swapper/106:07:591
1765099515,28cyclictest0-21swapper/110:37:421
1765099515,28cyclictest0-21swapper/108:12:331
1765099515,28cyclictest0-21swapper/107:37:481
1765099515,27cyclictest0-21swapper/109:22:411
1765099515,26cyclictest0-21swapper/110:02:361
1765099515,23cyclictest0-21swapper/105:32:371
1765099514,29cyclictest0-21swapper/107:22:581
1765099514,29cyclictest0-21swapper/107:22:581
1765099514,28cyclictest0-21swapper/106:02:511
1765099514,26cyclictest0-21swapper/107:12:341
17650995121,13cyclictest0-21swapper/110:57:441
17650995119,14cyclictest0-21swapper/107:42:401
17650995119,13cyclictest0-21swapper/109:52:171
127262512,9sleep30-21swapper/310:21:153
127262512,9sleep30-21swapper/310:21:153
126242512,11sleep10-21swapper/110:19:561
126242512,11sleep10-21swapper/110:19:561
53112502,10sleep30-21swapper/307:18:453
259212503,41sleep20-21swapper/206:57:112
227799500,5rtkit-daemon2276-21rtkit-daemon05:33:562
227799500,5rtkit-daemon2276-21rtkit-daemon05:32:262
216592502,42sleep10-21swapper/109:20:591
17652995035,9cyclictest22237-21head06:47:443
1765299503,41cyclictest15299-21sed09:02:413
17651995033,10cyclictest30532-21grep09:42:352
17651995021,8cyclictest8036-21latency_hist10:12:162
1765099505,28cyclictest0-21swapper/109:12:331
1765099504,29cyclictest0-21swapper/108:57:411
1765099504,28cyclictest0-21swapper/109:42:451
1765099504,28cyclictest0-21swapper/109:37:401
1765099504,28cyclictest0-21swapper/108:52:391
1765099504,28cyclictest0-21swapper/108:52:391
1765099504,27cyclictest0-21swapper/108:17:421
1765099503,41cyclictest0-21swapper/110:47:361
17650995020,13cyclictest0-21swapper/109:52:421
88372492,39sleep10-21swapper/108:47:261
88372492,39sleep10-21swapper/108:47:261
5412492,10sleep30-21swapper/308:25:513
5162492,41sleep10-21swapper/107:10:491
1765299494,40cyclictest0-21swapper/309:32:473
1765299494,40cyclictest0-21swapper/309:32:473
17651994922,7cyclictest2657-21latency_hist09:57:162
17651994920,7cyclictest13992-21latency_hist09:02:192
17651994920,24cyclictest0-21swapper/208:52:292
17651994920,24cyclictest0-21swapper/208:52:292
1765099494,35cyclictest0-21swapper/110:22:361
1765099494,35cyclictest0-21swapper/110:22:361
1765099494,26cyclictest0-21swapper/109:07:231
1765099494,26cyclictest0-21swapper/108:32:271
17650994931,11cyclictest24642-21gltestperf06:52:451
17650994920,9cyclictest24791-21idleruntime-cro09:32:161
136562492,10sleep30-21swapper/308:58:003
117642493,39sleep10-21swapper/107:32:491
323482482,10sleep30-21swapper/309:49:393
207872483,36sleep10-21swapper/107:57:241
196802482,39sleep30-21swapper/305:33:083
17652994831,12cyclictest0-21swapper/306:07:563
17652994830,13cyclictest0-21swapper/308:52:443
17652994830,13cyclictest0-21swapper/308:52:443
17652994820,7cyclictest31963-21awk06:02:253
1765199486,27cyclictest0-21swapper/208:42:442
17651994823,8cyclictest18176-21latency_hist07:52:212
115762482,10sleep20-21swapper/206:26:562
71742472,10sleep30-21swapper/310:04:023
46712472,38sleep30-21swapper/306:12:303
42762472,10sleep20-21swapper/206:10:192
319212472,37sleep20-21swapper/206:02:092
280462472,38sleep20-21swapper/208:13:412
280042472,9sleep30-21swapper/308:13:103
250032472,39sleep20-21swapper/208:03:052
250032472,39sleep20-21swapper/208:03:052
238222472,10sleep30-21swapper/309:23:033
178672472,38sleep10-21swapper/107:48:251
1765299474,25cyclictest15269-21iostat_ios06:32:503
1765299473,38cyclictest12829-21kworker/3:210:37:563
1765199478,32cyclictest0-21swapper/208:12:202
1765199474,37cyclictest0-21swapper/205:47:492
17651994732,9cyclictest17242-21grep06:37:412
17651994732,9cyclictest17242-21grep06:37:412
1765199473,37cyclictest0-21swapper/209:02:282
17651994728,13cyclictest0-21swapper/208:17:332
17651994723,8cyclictest16393-21latency_hist06:37:242
17651994720,8cyclictest22948-21latency_hist08:02:212
17651994717,14cyclictest0-21swapper/209:17:352
17650994714,12cyclictest0-21swapper/110:37:151
139412472,39sleep20-21swapper/206:31:402
139222472,39sleep30-21swapper/306:31:253
10302473,38sleep10-21swapper/108:28:521
101392472,38sleep30-21swapper/310:13:373
60032462,11sleep10-21swapper/108:39:361
281332462,10sleep30-21swapper/310:59:123
270622462,10sleep20-21swapper/209:36:002
270622462,10sleep20-21swapper/209:36:002
258312462,10sleep30-21swapper/310:55:173
17652994632,9cyclictest29228-21fschecks_count07:02:413
17652994620,8cyclictest28509-21cat07:02:233
1765199468,12cyclictest0-21swapper/207:02:572
1765199465,35cyclictest0-21swapper/207:32:472
17651994628,13cyclictest0-21swapper/210:32:142
17651994619,12cyclictest0-21swapper/206:57:362
17651994619,11cyclictest0-21swapper/206:03:022
1765099464,37cyclictest0-21swapper/110:10:081
1765099464,25cyclictest0-21swapper/107:27:481
156392462,11sleep10-21swapper/110:29:561
79982452,36sleep30-21swapper/310:11:523
258592452,10sleep30-21swapper/308:11:203
232572452,9sleep30-21swapper/310:47:513
204002452,9sleep20-21swapper/207:55:192
190282452,10sleep20-21swapper/209:12:502
1765299456,8cyclictest20874-21awk09:17:443
1765299454,26cyclictest0-21swapper/309:37:293
17652994531,9cyclictest17749-21unixbench_multi10:32:433
17652994530,9cyclictest7665-21sort06:17:423
17652994530,9cyclictest30597-21gltestperf09:42:363
17652994530,9cyclictest1492-21taskset08:32:193
1765299453,37cyclictest1263-21runrttasks09:12:303
17652994518,10cyclictest16625-21munin-node06:37:533
17652994518,10cyclictest16625-21munin-node06:37:533
17651994531,9cyclictest322332sleep208:22:472
1765199453,36cyclictest0-21swapper/208:32:182
1765199453,28cyclictest0-21swapper/207:42:412
17651994527,13cyclictest0-21swapper/206:52:232
17651994522,17cyclictest1263-21runrttasks07:11:352
17651994522,12cyclictest0-21swapper/207:17:452
1765199452,37cyclictest0-21swapper/210:42:142
17651994517,9cyclictest9652-21wc07:27:522
17651994517,12cyclictest0-21swapper/205:37:422
173742452,37sleep30-21swapper/307:45:043
157032452,10sleep30-21swapper/310:30:463
48202442,9sleep20-21swapper/208:37:392
37382442,10sleep20-21swapper/208:36:102
239682442,9sleep20-21swapper/209:24:432
226782442,9sleep20-21swapper/210:43:152
226662442,36sleep30-21swapper/310:43:083
222522442,9sleep30-21swapper/305:40:343
208842442,35sleep30-21swapper/306:43:423
181672442,37sleep20-21swapper/210:36:422
1765299445,33cyclictest0-21swapper/308:17:333
17652994430,9cyclictest23396-21uniq05:42:443
1765299443,34cyclictest8594-21cpuspeed_turbos07:27:363
17652994419,8cyclictest15174-21munin-run07:42:213
17651994429,9cyclictest5400-21grep06:12:462
1765199442,37cyclictest0-21swapper/210:56:082
17651994417,7cyclictest7429-21latency_hist10:07:162
17651994414,25cyclictest8431-21head10:12:292
1765099444,11cyclictest0-21swapper/105:32:261
127102442,10sleep30-21swapper/307:36:303
90712432,9sleep20-21swapper/208:50:292
90712432,9sleep20-21swapper/208:50:292
47142432,35sleep20-21swapper/209:58:042
210582432,9sleep20-21swapper/206:45:592
1765299434,12cyclictest2796-21munin-node08:32:383
17652994318,9cyclictest8238-21users08:42:503
1765199434,33cyclictest0-21swapper/209:37:402
1765199433,36cyclictest0-21swapper/206:17:562
17651994317,13cyclictest0-21swapper/209:49:392
170182434,6sleep30-21swapper/305:27:543
169322432,9sleep30-21swapper/309:11:423
235162422,9sleep20-21swapper/210:51:132
17652994228,9cyclictest21143-21cpuspeed_turbos07:57:353
17652994225,12cyclictest0-21swapper/309:57:453
1765299422,36cyclictest0-21swapper/305:48:563
1765199427,13cyclictest0-21swapper/205:57:092
1765199427,13cyclictest0-21swapper/205:57:092
113782422,34sleep30-21swapper/306:24:193
1765299418,28cyclictest0-21swapper/307:07:483
1765299414,32cyclictest0-21swapper/306:57:233
1765299412,35cyclictest0-21swapper/307:24:503
1765299412,35cyclictest0-21swapper/307:24:503
1765199412,34cyclictest0-21swapper/209:07:442
1765299403,33cyclictest0-21swapper/307:49:223
17652994025,10cyclictest13203-21aten_r1power_vo10:22:283
17652994025,10cyclictest13203-21aten_r1power_vo10:22:283
17652994022,13cyclictest0-21swapper/309:32:183
1765199403,31cyclictest0-21swapper/209:32:162
1765299392,13cyclictest0-21swapper/308:52:143
1765299392,13cyclictest0-21swapper/308:52:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional