You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Sat Apr 27, 2024 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2422991180,6rtkit-daemon2421-21rtkit-daemon17:55:192
2422991180,6rtkit-daemon2421-21rtkit-daemon17:55:192
1405499964,72cyclictest0-21swapper/320:07:583
1405499943,61cyclictest9241-21gltestperf22:32:553
14054999013,51cyclictest0-21swapper/322:47:423
242299890,4rtkit-daemon2421-21rtkit-daemon18:58:172
242299880,6rtkit-daemon0-21swapper/319:37:453
1405499874,63cyclictest0-21swapper/321:43:083
1405499864,74cyclictest0-21swapper/317:43:203
1405499864,63cyclictest31116-21sort21:08:043
1405499843,73cyclictest0-21swapper/322:19:363
1405499834,72cyclictest0-21swapper/321:32:483
1405499834,71cyclictest0-21swapper/319:58:033
1405499834,60cyclictest0-21swapper/322:07:453
1405499824,70cyclictest0-21swapper/319:14:003
1405499824,70cyclictest0-21swapper/318:08:063
1405499824,61cyclictest0-21swapper/320:27:523
1405499824,59cyclictest0-21swapper/321:13:003
1405499824,57cyclictest0-21swapper/322:00:193
1405499823,60cyclictest2761-21munin-node19:08:023
242299810,4rtkit-daemon2421-21rtkit-daemon22:29:391
1405499814,59cyclictest0-21swapper/321:25:583
1405499813,70cyclictest0-21swapper/322:42:293
1405499813,60cyclictest0-21swapper/321:02:113
1405499813,60cyclictest0-21swapper/318:03:103
1405499813,55cyclictest0-21swapper/321:37:443
14054998116,42cyclictest16788-21munin-node17:27:553
14054998114,49cyclictest1896-21munin-node20:12:573
242299800,4rtkit-daemon2421-21rtkit-daemon18:15:033
1405499804,70cyclictest0-21swapper/317:37:423
1405499804,51cyclictest0-21swapper/319:33:033
1405499794,68cyclictest0-21swapper/322:06:003
1405499794,68cyclictest0-21swapper/321:52:583
1405499794,67cyclictest0-21swapper/322:47:293
1405499794,50cyclictest0-21swapper/320:42:353
1405499794,50cyclictest0-21swapper/320:42:353
1405499793,61cyclictest0-21swapper/322:28:043
14054997916,45cyclictest0-21swapper/318:19:193
242299780,4rtkit-daemon2421-21rtkit-daemon22:08:091
1405499787,45cyclictest0-21swapper/320:47:553
1405499784,68cyclictest0-21swapper/317:47:503
1405499784,62cyclictest0-21swapper/319:42:463
1405499784,59cyclictest0-21swapper/318:30:273
1405499784,51cyclictest0-21swapper/317:37:593
1405499784,47cyclictest0-21swapper/321:17:563
1405499783,60cyclictest0-21swapper/320:43:023
1405499783,53cyclictest9002-21snmpd18:34:273
1405499775,64cyclictest0-21swapper/317:22:543
1405499774,65cyclictest0-21swapper/318:57:543
1405499774,51cyclictest0-21swapper/319:54:343
1405499774,51cyclictest0-21swapper/319:05:143
1405499773,9cyclictest28474-21tail18:52:593
1405499773,56cyclictest0-21swapper/321:47:423
1405499773,32cyclictest4508-21uname22:22:583
14054997714,47cyclictest0-21swapper/320:17:483
14054997711,43cyclictest95250irq/46-eth020:03:003
1405499764,51cyclictest0-21swapper/317:52:523
1405499764,51cyclictest0-21swapper/317:52:523
1405499764,49cyclictest0-21swapper/321:02:593
1405499764,48cyclictest0-21swapper/318:22:473
1405499763,67cyclictest0-21swapper/318:48:063
1405499753,64cyclictest0-21swapper/322:16:163
1405499753,64cyclictest0-21swapper/322:16:163
14054997516,44cyclictest0-21swapper/319:22:503
1405499744,63cyclictest0-21swapper/318:37:573
1405499744,63cyclictest0-21swapper/318:37:573
1405499743,64cyclictest0-21swapper/319:19:233
1405499738,47cyclictest0-21swapper/321:29:503
1405499738,47cyclictest0-21swapper/321:29:503
1405499735,48cyclictest0-21swapper/319:47:383
1405499734,61cyclictest1286-21runrttasks20:22:553
1405499734,53cyclictest0-21swapper/317:57:593
1405499724,61cyclictest0-21swapper/319:28:483
1405499724,61cyclictest0-21swapper/319:28:483
1405499724,60cyclictest0-21swapper/320:53:013
1405499713,8cyclictest13601-21proc_pri20:33:063
1405499713,43cyclictest0-21swapper/318:42:493
1405299696,17cyclictest0-21swapper/119:42:381
242299680,6rtkit-daemon0-21swapper/221:42:331
1405399665,51cyclictest0-21swapper/218:08:032
1405499654,55cyclictest0-21swapper/317:22:433
1405399653,53cyclictest18599-21grep21:47:492
242299640,7rtkit-daemon0-21swapper/022:40:461
1405399644,51cyclictest0-21swapper/217:33:192
1405099644,50cyclictest0-21swapper/022:02:520
1405399624,49cyclictest0-21swapper/221:33:022
242299600,7rtkit-daemon0-21swapper/022:15:322
242299600,7rtkit-daemon0-21swapper/022:15:322
242299600,6rtkit-daemon0-21swapper/319:42:452
242299600,5rtkit-daemon0-21swapper/320:47:352
168092603,11sleep00-21swapper/021:43:090
1405399604,33cyclictest0-21swapper/218:32:402
242299590,6rtkit-daemon0-21swapper/320:50:122
1405299597,38cyclictest0-21swapper/121:07:561
14050995937,14cyclictest0-21swapper/020:40:160
14050995937,14cyclictest0-21swapper/020:40:160
242299580,6rtkit-daemon0-21swapper/022:15:021
242299580,6rtkit-daemon0-21swapper/022:15:021
242299580,4rtkit-daemon2421-21rtkit-daemon22:36:530
14050995839,12cyclictest0-21swapper/019:51:410
14050995836,15cyclictest0-21swapper/018:44:260
242299570,7rtkit-daemon0-21swapper/022:43:232
242299570,4rtkit-daemon2421-21rtkit-daemon22:13:320
242299570,4rtkit-daemon2421-21rtkit-daemon22:13:320
242299570,4rtkit-daemon2421-21rtkit-daemon22:09:390
1405399574,14cyclictest0-21swapper/218:43:122
14052995741,11cyclictest0-21swapper/120:23:131
14050995740,10cyclictest13969-21tail20:33:120
14050995726,24cyclictest0-21swapper/018:13:030
242299560,4rtkit-daemon2421-21rtkit-daemon22:42:380
242299560,4rtkit-daemon2421-21rtkit-daemon22:40:530
1405399566,21cyclictest0-21swapper/221:12:332
1405399564,30cyclictest0-21swapper/217:44:132
14050995636,13cyclictest0-21swapper/021:19:200
258992553,12sleep20-21swapper/220:57:572
242299550,7rtkit-daemon0-21swapper/021:44:252
242299550,4rtkit-daemon2421-21rtkit-daemon22:29:540
149062553,13sleep20-21swapper/219:33:002
1405399554,32cyclictest0-21swapper/220:32:512
1405399553,36cyclictest0-21swapper/220:54:582
14053995513,18cyclictest0-21swapper/219:07:392
14050995537,12cyclictest27645-21grep17:47:580
114542553,14sleep00-21swapper/018:18:130
84182544,43sleep20-21swapper/219:17:572
14053995437,11cyclictest20433-21df18:37:542
14053995437,11cyclictest20433-21df18:37:542
1405299545,41cyclictest23376-21gltestperf18:42:571
14050995435,13cyclictest0-21swapper/020:03:130
295862532,44sleep00-21swapper/018:55:340
1405399539,27cyclictest0-21swapper/221:12:452
1405399534,30cyclictest0-21swapper/219:17:372
1405399534,28cyclictest0-21swapper/222:07:572
1405399534,28cyclictest0-21swapper/217:28:182
1405399533,30cyclictest0-21swapper/221:58:012
1405299537,39cyclictest32702-21gltestperf20:07:541
1405099534,31cyclictest0-21swapper/019:13:130
242299520,6rtkit-daemon2421-21rtkit-daemon22:32:461
229162523,11sleep20-21swapper/217:38:052
1405399526,26cyclictest0-21swapper/219:37:382
1405399524,28cyclictest0-21swapper/222:33:042
1405399524,28cyclictest0-21swapper/222:23:012
1405399524,28cyclictest0-21swapper/219:08:102
14053995234,11cyclictest18351-21sed18:33:012
1405399523,28cyclictest0-21swapper/220:07:352
14053995220,13cyclictest0-21swapper/221:02:522
14053995216,29cyclictest6033-21proc_pri20:18:102
1405299525,38cyclictest0-21swapper/121:24:161
14052995222,12cyclictest0-21swapper/118:08:061
1405099524,41cyclictest0-21swapper/020:27:340
1405099524,29cyclictest0-21swapper/018:07:410
1405099524,13cyclictest0-21swapper/019:07:530
14050995236,11cyclictest0-21swapper/019:33:070
138632523,10sleep10-21swapper/118:23:101
70732512,10sleep20-21swapper/220:22:462
243262513,10sleep10-21swapper/120:55:141
24032513,41sleep00-21swapper/019:06:300
1405399517,27cyclictest0-21swapper/218:02:482
1405399517,25cyclictest0-21swapper/222:27:422
1405399514,29cyclictest0-21swapper/220:12:472
1405399514,28cyclictest0-21swapper/222:17:432
1405399514,28cyclictest0-21swapper/220:07:552
1405399514,27cyclictest0-21swapper/222:37:422
1405399514,27cyclictest0-21swapper/220:37:502
1405399514,27cyclictest0-21swapper/220:37:502
1405399514,27cyclictest0-21swapper/219:52:552
1405399514,15cyclictest0-21swapper/218:23:122
14053995134,10cyclictest8593-21grep18:13:072
14053995120,13cyclictest0-21swapper/221:52:572
14053995118,13cyclictest0-21swapper/221:22:562
14053995117,14cyclictest0-21swapper/221:38:092
1405299519,35cyclictest25081-21munin-run22:02:301
1405299514,39cyclictest0-21swapper/121:37:321
1405299514,39cyclictest0-21swapper/117:47:411
14052995134,10cyclictest28294-21head21:02:551
1405099514,29cyclictest0-21swapper/021:53:090
14050995130,8cyclictest25905-21timerandwakeup19:53:120
14050995125,8cyclictest27226-21latency_hist18:52:400
14050995123,10cyclictest27290-21diskstats19:57:530
118872513,9sleep10-21swapper/118:20:161
315192503,10sleep10-21swapper/120:07:331
286852502,10sleep20-21swapper/219:58:442
236732503,10sleep10-21swapper/119:49:121
168822503,11sleep10-21swapper/118:29:311
1405399504,28cyclictest0-21swapper/219:30:072
1405399504,28cyclictest0-21swapper/219:30:072
1405399504,27cyclictest0-21swapper/217:58:072
1405399504,26cyclictest0-21swapper/220:27:422
1405399503,28cyclictest0-21swapper/219:23:032
1405399503,27cyclictest0-21swapper/218:52:502
14053995026,10cyclictest0-21swapper/219:50:372
1405299504,28cyclictest0-21swapper/118:47:571
14052995035,9cyclictest29743-21tr17:52:551
14052995035,9cyclictest29743-21tr17:52:551
14052995031,12cyclictest3709-21gltestperf19:07:591
14052995027,7cyclictest22685-21cat21:57:311
1405099504,29cyclictest0-21swapper/018:38:090
1405099504,29cyclictest0-21swapper/018:38:090
1405099504,28cyclictest0-21swapper/021:58:030
1405099504,28cyclictest0-21swapper/018:58:120
1405099504,28cyclictest0-21swapper/017:28:030
14050995031,13cyclictest0-21swapper/020:18:110
1405099503,41cyclictest0-21swapper/021:00:190
1405099503,23cyclictest25120-21cut17:42:580
14050995020,17cyclictest0-21swapper/018:12:400
295022493,10sleep10-21swapper/118:54:301
244462492,42sleep00-21swapper/020:56:460
217902493,10sleep10-21swapper/120:50:061
195462493,10sleep10-21swapper/118:36:201
192182492,40sleep00-21swapper/017:32:430
166302493,10sleep10-21swapper/120:38:371
166302493,10sleep10-21swapper/120:38:371
1405399494,29cyclictest0-21swapper/217:23:052
1405399494,26cyclictest0-21swapper/222:02:422
1405399494,26cyclictest0-21swapper/217:47:492
14053994933,10cyclictest11126-21grep18:18:052
1405399493,27cyclictest0-21swapper/222:47:552
1405299498,34cyclictest201ktimersoftd/119:02:571
14052994936,7cyclictest0-21swapper/119:27:241
14052994932,12cyclictest0-21swapper/117:57:471
14052994930,11cyclictest27197-21unixbench_singl22:03:071
14052994927,14cyclictest0-21swapper/120:17:591
1405299492,38cyclictest0-21swapper/117:22:441
1405099495,37cyclictest0-21swapper/019:22:570
1405099495,36cyclictest0-21swapper/022:17:380
1405099494,39cyclictest0-21swapper/021:28:040
1405099494,39cyclictest0-21swapper/021:28:040
1405099494,38cyclictest0-21swapper/022:22:530
1405099494,38cyclictest0-21swapper/020:07:530
1405099494,38cyclictest0-21swapper/019:42:500
1405099493,39cyclictest1286-21runrttasks18:29:110
14050994928,14cyclictest0-21swapper/017:57:140
14050994928,14cyclictest0-21swapper/017:57:140
139462494,39sleep00-21swapper/017:21:470
27872483,10sleep10-21swapper/122:20:391
147412482,41sleep00-21swapper/021:39:310
1405399484,29cyclictest0-21swapper/221:18:022
1405299485,37cyclictest0-21swapper/119:53:591
1405299484,24cyclictest21060-21kernelversion18:38:041
1405299484,24cyclictest21060-21kernelversion18:38:041
14052994831,11cyclictest2330-21tr20:12:481
14052994831,10cyclictest28558-21unixbench_multi19:58:131
14052994831,10cyclictest25683-21gltestperf20:57:531
1405299483,36cyclictest2621-21df_abs18:02:531
14052994829,11cyclictest3835-21cut22:22:491
14052994828,13cyclictest0-21swapper/120:28:121
14052994825,7cyclictest201ktimersoftd/119:32:381
14052994825,7cyclictest201ktimersoftd/119:32:381
14052994810,8cyclictest18019-21grep19:38:061
1405099484,38cyclictest0-21swapper/021:37:320
1405099484,27cyclictest0-21swapper/021:23:080
1405099483,28cyclictest0-21swapper/017:23:040
14050994829,13cyclictest0-21swapper/020:13:450
172012473,10sleep10-21swapper/121:43:391
1405399474,27cyclictest0-21swapper/218:47:512
1405299477,23cyclictest4887-21munin-run21:22:321
1405299474,26cyclictest0-21swapper/121:27:391
1405299474,26cyclictest0-21swapper/121:27:391
1405299474,26cyclictest0-21swapper/117:34:111
14052994728,11cyclictest16135-21cpuspeed_turbos22:47:441
14052994710,31cyclictest0-21swapper/119:13:111
1405099475,35cyclictest0-21swapper/022:48:040
1405099474,27cyclictest0-21swapper/019:18:370
14050994732,10cyclictest0-21swapper/019:27:450
14050994732,10cyclictest0-21swapper/019:27:450
14050994732,10cyclictest0-21swapper/018:23:110
16752462,11sleep00-21swapper/018:00:360
160822463,10sleep10-21swapper/119:33:561
143982463,11sleep10-21swapper/120:37:241
14053994622,11cyclictest0-21swapper/221:27:582
14053994622,11cyclictest0-21swapper/221:27:582
1405299464,34cyclictest0-21swapper/118:57:591
1405299464,34cyclictest0-21swapper/118:12:481
1405099466,24cyclictest0-21swapper/021:02:520
1405099464,36cyclictest0-21swapper/021:12:560
1405099464,26cyclictest0-21swapper/020:29:400
1405099464,11cyclictest18789-21cat20:43:110
14050994632,8cyclictest29959-21yum-updatesd-he21:07:500
1405099463,25cyclictest17708-21cpuspeed_turbos18:32:530
14050994626,13cyclictest0-21swapper/021:48:030
14050994623,11cyclictest0-21swapper/019:38:060
238692452,37sleep00-21swapper/017:39:200
20292453,10sleep10-21swapper/121:13:561
1405299454,33cyclictest0-21swapper/120:43:131
1405299454,33cyclictest0-21swapper/117:27:511
1405099454,14cyclictest0-21swapper/020:49:430
1405299444,33cyclictest0-21swapper/121:48:071
1405299444,32cyclictest0-21swapper/122:42:381
1405299434,33cyclictest0-21swapper/119:18:061
1405299427,22cyclictest0-21swapper/117:22:431
1405299424,14cyclictest0-21swapper/117:37:511
14052994213,13cyclictest0-21swapper/117:49:141
139832424,30sleep20-21swapper/217:22:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional