You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-04 - 03:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Thu Sep 04, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15792991074,52cyclictest0-21swapper/220:31:462
1578399974,41cyclictest0-21swapper/023:06:090
15783999437,14cyclictest0-21swapper/018:26:440
1578399913,27cyclictest0-21swapper/021:07:500
1578399913,27cyclictest0-21swapper/021:07:500
58982850,4chrt5895-21grep20:36:350
223099770,7rtkit-daemon0-21swapper/119:01:411
223099770,7rtkit-daemon0-21swapper/119:01:411
223099740,7rtkit-daemon0-21swapper/119:16:331
1579299744,62cyclictest0-21swapper/222:56:442
223099690,5rtkit-daemon18-21rcuc/118:11:351
223099690,4rtkit-daemon2229-21rtkit-daemon21:27:133
223099690,4rtkit-daemon2229-21rtkit-daemon20:46:443
223099680,7rtkit-daemon0-21swapper/121:33:201
223099670,4rtkit-daemon2229-21rtkit-daemon21:22:283
223099660,5rtkit-daemon2229-21rtkit-daemon20:21:382
223099660,5rtkit-daemon2229-21rtkit-daemon20:21:382
223099650,4rtkit-daemon2229-21rtkit-daemon22:16:483
223099650,4rtkit-daemon2229-21rtkit-daemon20:42:073
223099650,4rtkit-daemon2229-21rtkit-daemon20:21:533
223099650,4rtkit-daemon2229-21rtkit-daemon20:21:533
223099650,4rtkit-daemon2229-21rtkit-daemon19:51:543
223099650,4rtkit-daemon2229-21rtkit-daemon18:02:063
223099650,4rtkit-daemon2229-21rtkit-daemon18:01:213
269922640,4sleep027036-21switchtime17:56:500
223099640,4rtkit-daemon2229-21rtkit-daemon19:17:033
223099640,11rtkit-daemon2229-21rtkit-daemon23:01:242
223099630,4rtkit-daemon2229-21rtkit-daemon20:36:523
223099620,9rtkit-daemon0-21swapper/019:16:482
223099610,7rtkit-daemon0-21swapper/020:01:461
223099610,6rtkit-daemon0-21swapper/022:21:411
223099610,6rtkit-daemon0-21swapper/021:36:501
1579299614,12cyclictest0-21swapper/220:11:292
1579099616,41cyclictest0-21swapper/119:33:271
223099600,6rtkit-daemon0-21swapper/019:07:031
223099600,5rtkit-daemon0-21swapper/018:06:361
1579299604,49cyclictest0-21swapper/221:31:322
1579299603,49cyclictest0-21swapper/218:06:472
228472594,47sleep30-21swapper/320:06:163
223099590,6rtkit-daemon0-21swapper/022:41:401
223099590,6rtkit-daemon0-21swapper/022:11:411
223099590,6rtkit-daemon0-21swapper/019:36:321
223099590,6rtkit-daemon0-21swapper/017:42:441
223099590,5rtkit-daemon0-21swapper/018:56:411
15783995938,13cyclictest0-21swapper/018:26:200
223099580,6rtkit-daemon0-21swapper/017:40:521
223099580,4rtkit-daemon2229-21rtkit-daemon17:59:582
1579299584,12cyclictest0-21swapper/217:51:492
1579299583,10cyclictest0-21swapper/218:38:072
15792995818,21cyclictest16562-21vmstat18:41:542
15783995836,14cyclictest0-21swapper/021:12:500
223099570,6rtkit-daemon0-21swapper/020:11:231
223099570,4rtkit-daemon2229-21rtkit-daemon22:16:332
15793995735,14cyclictest0-21swapper/318:16:373
15793995735,14cyclictest0-21swapper/318:16:373
1579299574,13cyclictest0-21swapper/221:46:412
1578399574,36cyclictest0-21swapper/018:46:520
15783995736,14cyclictest0-21swapper/021:36:440
15783995636,13cyclictest0-21swapper/019:41:460
15783995636,13cyclictest0-21swapper/019:41:460
223099550,5rtkit-daemon2229-21rtkit-daemon21:06:443
223099550,5rtkit-daemon2229-21rtkit-daemon21:06:443
281512544,43sleep30-21swapper/319:06:503
254672544,9sleep125512-21ntp_offset21:16:401
223099540,4rtkit-daemon16018-21open_inodes20:56:443
1579399545,12cyclictest0-21swapper/321:38:373
1579399544,27cyclictest0-21swapper/322:31:243
15793995434,12cyclictest26318-21gltestperf20:11:383
15793995422,13cyclictest0-21swapper/317:41:413
1579099545,12cyclictest0-21swapper/122:33:191
1579099544,12cyclictest0-21swapper/121:51:131
15790995422,13cyclictest0-21swapper/118:41:461
15783995436,12cyclictest0-21swapper/019:16:310
108172543,9sleep10-21swapper/121:56:021
108172543,9sleep10-21swapper/121:56:021
1579399535,29cyclictest0-21swapper/322:26:473
1579399535,29cyclictest0-21swapper/317:46:383
15793995333,12cyclictest31076-21gltestperf22:36:303
15793995333,12cyclictest31076-21gltestperf22:36:303
1579299534,28cyclictest0-21swapper/221:16:422
15792995335,10cyclictest2340-21gltestperf21:36:332
1579099535,42cyclictest18-21rcuc/122:54:541
1579099534,28cyclictest0-21swapper/121:21:311
1579099534,27cyclictest0-21swapper/121:46:131
15790995311,24cyclictest0-21swapper/118:16:311
15790995311,24cyclictest0-21swapper/118:16:311
24432522,9sleep20-21swapper/220:30:212
1579399524,28cyclictest0-21swapper/319:36:393
15793995232,12cyclictest8986-21cpuspeed_turbos21:51:263
15793995232,12cyclictest8986-21cpuspeed_turbos21:51:263
15793995230,14cyclictest0-21swapper/322:41:353
15793995220,13cyclictest0-21swapper/322:21:383
1579299524,29cyclictest0-21swapper/222:11:292
1579299524,28cyclictest0-21swapper/221:51:232
1579299524,28cyclictest0-21swapper/221:51:232
1579299524,28cyclictest0-21swapper/220:16:302
1579299524,28cyclictest0-21swapper/219:46:422
1579299524,28cyclictest0-21swapper/219:01:532
1579299524,28cyclictest0-21swapper/219:01:532
1579299524,28cyclictest0-21swapper/217:36:432
1579299524,27cyclictest0-21swapper/222:42:292
1579299524,27cyclictest0-21swapper/221:01:152
1579099525,28cyclictest0-21swapper/118:26:411
1579099524,28cyclictest0-21swapper/122:36:361
1579099524,28cyclictest0-21swapper/122:36:361
1579099524,28cyclictest0-21swapper/121:26:251
1579099524,28cyclictest0-21swapper/121:01:341
1579099524,28cyclictest0-21swapper/118:31:501
1579099524,24cyclictest0-21swapper/119:51:301
15790995221,13cyclictest0-21swapper/118:26:211
1578399526,39cyclictest0-21swapper/022:41:200
1578399524,42cyclictest0-21swapper/019:51:450
15783995237,10cyclictest0-21swapper/020:16:520
15783995233,12cyclictest0-21swapper/022:16:230
15783995230,9cyclictest0-21swapper/021:52:000
15783995230,9cyclictest0-21swapper/021:52:000
154162522,42sleep015501-21tail20:56:350
149042524,42sleep314980-21sed22:01:443
144682526,9sleep00-21swapper/019:46:380
130592522,9sleep00-21swapper/021:59:070
260912512,9sleep30-21swapper/321:17:223
203492513,9sleep00-21swapper/017:43:560
201602512,9sleep00-21swapper/022:13:220
178972512,9sleep00-21swapper/017:38:000
1579399515,28cyclictest0-21swapper/318:41:443
1579399515,28cyclictest0-21swapper/318:11:463
1579399514,29cyclictest0-21swapper/319:26:453
1579399514,28cyclictest0-21swapper/319:46:183
1579399514,28cyclictest0-21swapper/319:46:183
15793995136,11cyclictest0-21swapper/319:01:413
15793995136,11cyclictest0-21swapper/319:01:413
15793995126,7cyclictest4881-21cat22:51:103
15793995124,8cyclictest24100-21cat21:16:143
1579299514,28cyclictest0-21swapper/222:26:272
1579299514,28cyclictest0-21swapper/222:01:472
1579299514,28cyclictest0-21swapper/219:21:482
1579299514,28cyclictest0-21swapper/219:06:302
1579299514,28cyclictest0-21swapper/218:31:482
1579299514,27cyclictest0-21swapper/222:06:262
1579299514,27cyclictest0-21swapper/221:21:482
1579299514,27cyclictest0-21swapper/219:36:192
1579299514,27cyclictest0-21swapper/218:21:332
1579299514,27cyclictest0-21swapper/218:01:382
1579299513,41cyclictest1160-21snmpd18:18:042
1579299513,41cyclictest1160-21snmpd18:18:042
1579099515,27cyclictest0-21swapper/120:46:351
1579099515,27cyclictest0-21swapper/120:21:501
1579099515,27cyclictest0-21swapper/120:21:501
1579099514,28cyclictest0-21swapper/120:06:451
15790995119,14cyclictest0-21swapper/120:51:441
1578399514,40cyclictest0-21swapper/019:36:300
1578399514,40cyclictest0-21swapper/019:31:320
1578399514,39cyclictest0-21swapper/017:46:330
1578399514,29cyclictest0-21swapper/022:56:310
15783995129,14cyclictest0-21swapper/020:51:290
130852512,9sleep30-21swapper/318:36:393
93642502,9sleep20-21swapper/220:42:062
70332502,41sleep00-21swapper/022:53:530
49602503,6sleep04970-21munin-run18:21:210
49602503,6sleep04970-21munin-run18:21:210
47452502,42sleep00-21swapper/020:34:100
296612503,7sleep329756-21fschecks_count19:11:343
274222503,41sleep10-21swapper/122:29:191
226962503,9sleep10-21swapper/122:20:081
1579399504,29cyclictest0-21swapper/318:21:303
1579399504,28cyclictest0-21swapper/322:14:573
1579399504,28cyclictest0-21swapper/318:46:483
1579399504,27cyclictest0-21swapper/323:06:103
1579399504,27cyclictest0-21swapper/319:31:323
1579399504,26cyclictest0-21swapper/320:06:273
1579399504,21cyclictest0-21swapper/318:31:393
15793995033,10cyclictest514-21cpuspeed_turbos20:26:273
15793995019,13cyclictest0-21swapper/321:31:423
1579299504,27cyclictest0-21swapper/220:41:162
1579299504,12cyclictest0-21swapper/218:26:552
15792995012,23cyclictest0-21swapper/219:26:302
1579099504,29cyclictest0-21swapper/119:26:301
1579099504,27cyclictest0-21swapper/120:33:221
1579099504,27cyclictest0-21swapper/117:51:481
15790995018,13cyclictest0-21swapper/119:21:511
1578399505,28cyclictest0-21swapper/020:46:290
1578399505,28cyclictest0-21swapper/020:21:290
1578399505,28cyclictest0-21swapper/020:21:290
1578399505,28cyclictest0-21swapper/018:06:480
1578399504,38cyclictest0-21swapper/020:06:270
1578399504,29cyclictest0-21swapper/022:31:270
1578399504,29cyclictest0-21swapper/022:21:270
1578399504,28cyclictest0-21swapper/021:26:400
1578399504,28cyclictest0-21swapper/021:21:140
1578399504,28cyclictest0-21swapper/018:31:370
1578399504,28cyclictest0-21swapper/018:11:550
15783995021,12cyclictest0-21swapper/019:01:460
15783995021,12cyclictest0-21swapper/019:01:460
155262504,7sleep30-21swapper/317:33:243
155262504,7sleep30-21swapper/317:33:243
35162493,10sleep30-21swapper/319:23:443
223099490,5rtkit-daemon2229-21rtkit-daemon22:31:182
223099490,5rtkit-daemon2229-21rtkit-daemon21:06:362
223099490,5rtkit-daemon2229-21rtkit-daemon21:06:362
1579399494,27cyclictest0-21swapper/321:45:483
15793994927,15cyclictest28742-21gltestperf20:16:373
1579299495,38cyclictest0-21swapper/217:47:592
1579299494,27cyclictest0-21swapper/221:26:432
1579299494,27cyclictest0-21swapper/219:51:342
1579299494,27cyclictest0-21swapper/219:41:422
1579299494,27cyclictest0-21swapper/219:41:422
1579299494,26cyclictest0-21swapper/218:46:492
1579299493,27cyclictest0-21swapper/220:01:362
1579299493,27cyclictest0-21swapper/219:11:282
1579099497,10cyclictest0-21swapper/122:48:101
1579099495,13cyclictest0-21swapper/122:01:371
1579099494,28cyclictest0-21swapper/120:36:291
1579099494,28cyclictest0-21swapper/119:11:381
15790994932,10cyclictest20374-21gltestperf18:51:421
15790994918,14cyclictest0-21swapper/123:02:321
1578399495,37cyclictest0-21swapper/021:26:140
1578399495,27cyclictest0-21swapper/020:41:500
1578399494,38cyclictest0-21swapper/018:56:360
1578399494,28cyclictest0-21swapper/019:26:400
1578399494,28cyclictest0-21swapper/018:41:500
1578399494,27cyclictest0-21swapper/021:46:120
1578399494,27cyclictest0-21swapper/017:51:380
15783994932,10cyclictest19917-21cpuspeed_turbos18:51:370
1578399493,28cyclictest0-21swapper/020:01:520
15783994910,22cyclictest0-21swapper/020:11:370
143582492,41sleep00-21swapper/018:39:060
141642492,9sleep20-21swapper/220:52:302
132112493,7sleep213213-21systemd-cgroups22:01:072
312322482,10sleep00-21swapper/019:16:190
214312483,40sleep10-21swapper/121:08:501
214312483,40sleep10-21swapper/121:08:501
1579399484,28cyclictest0-21swapper/322:07:473
1579399484,25cyclictest0-21swapper/319:56:273
15793994832,10cyclictest5341-21cpuspeed_turbos22:51:263
15793994831,10cyclictest17906-21gltestperf21:01:343
15793994830,11cyclictest14876-21proc_pri19:46:453
15793994819,12cyclictest0-21swapper/318:53:103
1579299484,38cyclictest0-21swapper/219:31:402
1579299484,26cyclictest0-21swapper/218:51:372
1579299483,27cyclictest17110-21aten_r1power_cu21:01:262
15792994817,12cyclictest0-21swapper/221:41:362
1579099488,13cyclictest0-21swapper/121:56:321
1579099488,10cyclictest0-21swapper/118:40:091
15790994838,6cyclictest201ktimersoftd/120:16:331
15790994830,11cyclictest14786-21aten_r1power_fr20:56:281
1578399485,24cyclictest0-21swapper/019:21:320
1578399484,27cyclictest0-21swapper/021:01:490
1578399483,28cyclictest0-21swapper/022:46:380
156052483,6sleep00-21swapper/017:34:250
156052483,6sleep00-21swapper/017:34:250
322002472,9sleep00-21swapper/022:39:220
322002472,9sleep00-21swapper/022:39:220
1579399475,33cyclictest0-21swapper/321:56:293
15793994732,9cyclictest16888-21cut17:36:413
15793994717,13cyclictest0-21swapper/322:56:433
15793994715,14cyclictest0-21swapper/320:51:473
1579299474,27cyclictest0-21swapper/221:13:332
1579299474,27cyclictest0-21swapper/220:51:152
1579299474,26cyclictest0-21swapper/220:06:362
1579299474,24cyclictest2087-21tail18:11:552
1579299473,27cyclictest0-21swapper/222:51:432
1579299473,22cyclictest2831-21aten2.4-expect22:46:222
1579299473,21cyclictest0-21swapper/222:21:242
1579099474,25cyclictest0-21swapper/117:46:341
15790994730,11cyclictest16716-21gltestperf22:06:281
1579099473,24cyclictest1122-21lldpd19:46:441
1579099470,9cyclictest745-21diskstats20:26:291
1578399475,26cyclictest0-21swapper/019:56:350
1578399475,24cyclictest0-21swapper/022:26:260
1578399474,26cyclictest0-21swapper/022:01:280
1578399474,25cyclictest0-21swapper/018:01:380
1578399473,38cyclictest0-21swapper/020:26:410
1578399473,26cyclictest0-21swapper/022:06:370
130682473,10sleep10-21swapper/119:43:171
130682473,10sleep10-21swapper/119:43:171
94522463,37sleep10-21swapper/120:43:141
239732462,9sleep30-21swapper/318:59:573
238792463,37sleep10-21swapper/121:14:381
202652463,37sleep10-21swapper/119:58:441
1579399467,21cyclictest0-21swapper/321:46:253
1579399464,25cyclictest0-21swapper/318:08:023
1579299464,12cyclictest0-21swapper/217:36:232
1579299464,12cyclictest0-21swapper/217:36:232
1579299463,25cyclictest0-21swapper/217:41:392
15783994619,11cyclictest0-21swapper/021:34:130
1579399454,34cyclictest0-21swapper/320:31:283
1579299452,9cyclictest0-21swapper/222:36:242
1579299452,9cyclictest0-21swapper/222:36:242
15790994530,10cyclictest18860-21unixbench_multi18:46:541
1578399455,25cyclictest0-21swapper/019:06:300
1578399454,24cyclictest0-21swapper/021:46:250
252282443,9sleep30-21swapper/317:55:203
1579299444,24cyclictest0-21swapper/218:56:322
15792994415,13cyclictest0-21swapper/219:56:252
297552433,34sleep10-21swapper/118:02:071
15793994329,10cyclictest0-21swapper/318:28:173
1579099439,28cyclictest0-21swapper/117:56:311
93742413,9sleep10-21swapper/122:58:131
15790993916,10cyclictest0-21swapper/117:36:231
15790993916,10cyclictest0-21swapper/117:36:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional