You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 11:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Sun Nov 23, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
509721040,4sleep2281ktimersoftd/222:28:132
148672920,6sleep214868-21aten2.4-expect22:48:242
148672920,6sleep214868-21aten2.4-expect22:48:242
135299923,43cyclictest0-21swapper/022:46:150
1352999215,31cyclictest1369-21runrttasks22:13:330
1352999138,9cyclictest8462-21grep20:18:370
135299884,25cyclictest0-21swapper/022:53:420
1352998420,7cyclictest27593-21latency_hist00:23:090
135299743,38cyclictest0-21swapper/019:58:550
58072730,9sleep1135699cyclictest19:03:541
243799610,5rtkit-daemon2436-21rtkit-daemon20:33:222
243799610,5rtkit-daemon2436-21rtkit-daemon20:29:522
243799600,4rtkit-daemon2436-21rtkit-daemon22:41:322
243799600,4rtkit-daemon2436-21rtkit-daemon19:20:552
243799600,4rtkit-daemon2436-21rtkit-daemon19:20:552
1357996014,27cyclictest0-21swapper/219:54:492
243799590,4rtkit-daemon2436-21rtkit-daemon22:04:342
24002595,46sleep20-21swapper/223:28:312
24002595,46sleep20-21swapper/223:28:312
135799594,38cyclictest0-21swapper/220:48:532
69042573,46sleep37033-21df_abs19:08:363
204102574,45sleep30-21swapper/321:52:083
243799560,7rtkit-daemon2436-21rtkit-daemon21:23:502
243799560,5rtkit-daemon2436-21rtkit-daemon00:23:282
135799566,31cyclictest0-21swapper/223:48:482
1357995637,12cyclictest32088-21grep19:58:522
1357995634,16cyclictest9229-21cpuspeed_turbos19:13:362
135699563,46cyclictest0-21swapper/100:08:241
271382554,44sleep20-21swapper/200:18:452
243799550,6rtkit-daemon2436-21rtkit-daemon23:08:242
243799550,6rtkit-daemon2436-21rtkit-daemon19:23:332
243799550,5rtkit-daemon2436-21rtkit-daemon23:53:292
243799550,5rtkit-daemon2436-21rtkit-daemon21:08:362
243799550,5rtkit-daemon2436-21rtkit-daemon20:38:452
243799550,5rtkit-daemon2436-21rtkit-daemon20:18:382
243799550,5rtkit-daemon2436-21rtkit-daemon20:13:382
243799550,5rtkit-daemon2436-21rtkit-daemon19:48:542
169272553,9sleep316931-21crond20:38:163
1357995539,9cyclictest6817-21grep19:08:352
135799553,36cyclictest20737-21munin-node21:53:492
135699553,45cyclictest0-21swapper/121:03:381
10352554,6sleep00-21swapper/018:54:520
243799540,5rtkit-daemon2436-21rtkit-daemon23:13:242
243799540,5rtkit-daemon2436-21rtkit-daemon23:03:392
243799540,5rtkit-daemon2436-21rtkit-daemon22:13:412
243799540,5rtkit-daemon2436-21rtkit-daemon20:23:382
243799540,5rtkit-daemon2436-21rtkit-daemon19:43:542
229132542,45sleep00-21swapper/019:42:590
219412544,13sleep30-21swapper/320:48:283
135799545,29cyclictest0-21swapper/221:13:282
135799544,31cyclictest0-21swapper/220:53:452
1357995438,9cyclictest18866-21df21:48:302
1357995437,11cyclictest9058-21tr23:43:262
1357995434,11cyclictest18093-21crond21:48:132
1357995422,13cyclictest0-21swapper/219:03:202
1357995421,14cyclictest0-21swapper/222:08:332
1357995421,14cyclictest0-21swapper/222:08:332
1357995421,14cyclictest0-21swapper/221:38:422
1357995421,13cyclictest0-21swapper/220:03:362
1357995421,13cyclictest0-21swapper/220:03:362
243799530,5rtkit-daemon2436-21rtkit-daemon23:41:452
243799530,5rtkit-daemon2436-21rtkit-daemon22:43:402
243799530,5rtkit-daemon2436-21rtkit-daemon21:33:502
243799530,5rtkit-daemon2436-21rtkit-daemon21:03:512
240432533,43sleep10-21swapper/123:08:111
142922536,9sleep10-21swapper/120:30:001
135899535,41cyclictest0-21swapper/321:28:363
135899535,41cyclictest0-21swapper/321:28:363
1358995335,10cyclictest20154-21fschecks_count20:43:353
1358995335,10cyclictest10793-21sed22:38:343
135799534,29cyclictest0-21swapper/221:58:482
135799534,29cyclictest0-21swapper/220:48:162
135799534,29cyclictest0-21swapper/219:38:322
135799534,22cyclictest0-21swapper/222:33:392
1357995338,9cyclictest16855-21tail19:28:412
1357995322,13cyclictest0-21swapper/223:58:312
1357995322,13cyclictest0-21swapper/223:23:212
1357995322,13cyclictest0-21swapper/219:33:312
1357995318,13cyclictest0-21swapper/220:58:412
1356995337,10cyclictest30802-21head23:18:431
1356995314,31cyclictest0-21swapper/120:13:331
135299534,39cyclictest0-21swapper/021:42:360
135299533,40cyclictest0-21swapper/022:58:320
1352995319,28cyclictest4264-21rt-features20:08:490
273642524,10sleep10-21swapper/122:04:231
243799520,5rtkit-daemon2436-21rtkit-daemon22:29:183
243799520,5rtkit-daemon2436-21rtkit-daemon22:18:332
243799520,5rtkit-daemon2436-21rtkit-daemon21:18:362
13642523,10sleep10-21swapper/120:03:441
13642523,10sleep10-21swapper/120:03:441
135799524,29cyclictest0-21swapper/223:18:402
135799524,28cyclictest0-21swapper/200:03:392
1357995221,11cyclictest3572-21tune2fs20:08:382
1357995220,12cyclictest0-21swapper/221:30:052
1357995220,12cyclictest0-21swapper/221:30:052
135699525,37cyclictest0-21swapper/122:39:091
24432512,9sleep30-21swapper/322:22:143
1358995134,10cyclictest26979-21cut23:13:273
135899512,44cyclictest0-21swapper/322:11:493
135899512,44cyclictest0-21swapper/322:11:493
1358995116,27cyclictest8729-21aten_r1power_cu23:43:233
135799515,28cyclictest0-21swapper/200:13:372
135799515,28cyclictest0-21swapper/200:08:282
135799514,29cyclictest0-21swapper/222:28:362
135799514,28cyclictest0-21swapper/223:33:282
135799514,28cyclictest0-21swapper/222:58:442
135799513,27cyclictest0-21swapper/219:03:372
135799513,26cyclictest0-21swapper/222:53:272
135299513,42cyclictest0-21swapper/023:58:370
129402513,41sleep10-21swapper/123:49:091
129052513,12sleep30-21swapper/322:43:303
276772503,41sleep10-21swapper/119:52:321
136542503,40sleep313679-21ntp_states20:28:453
135899504,11cyclictest0-21swapper/321:10:023
135899503,29cyclictest0-21swapper/320:08:543
135899503,24cyclictest0-21swapper/319:35:013
135899502,43cyclictest0-21swapper/319:16:183
135699506,27cyclictest0-21swapper/119:34:461
310372493,40sleep331156-21perl19:58:383
285292495,38sleep10-21swapper/123:14:321
252062492,9sleep30-21swapper/322:02:193
1358994932,11cyclictest16266-21cpuspeed_turbos23:58:243
1356994933,10cyclictest19418-21latency21:48:381
135699493,37cyclictest0-21swapper/123:38:171
1356994911,8cyclictest0-21swapper/122:38:121
135299494,41cyclictest0-21swapper/023:19:070
97752483,38sleep39785-21munin-run20:23:173
233362482,39sleep30-21swapper/319:43:353
191212482,10sleep30-21swapper/322:56:203
135899483,27cyclictest0-21swapper/323:48:263
135699484,26cyclictest0-21swapper/120:24:541
1356994820,12cyclictest0-21swapper/122:28:251
1356994816,26cyclictest15271-21tail21:38:481
1356994810,31cyclictest0-21swapper/121:13:521
135299483,37cyclictest0-21swapper/019:19:160
135299483,37cyclictest0-21swapper/019:19:160
96252472,9sleep30-21swapper/322:37:143
50532472,9sleep30-21swapper/323:33:363
36812472,38sleep00-21swapper/021:17:030
263942472,40sleep00-21swapper/023:12:450
263452473,38sleep10-21swapper/123:12:061
250742473,38sleep00-21swapper/019:45:010
24752473,38sleep10-21swapper/122:22:241
243799470,5rtkit-daemon2436-21rtkit-daemon22:23:483
230252472,38sleep00-21swapper/000:13:100
204072474,10sleep10-21swapper/100:07:551
199022473,7sleep320027-21unixbench_singl00:03:423
167982473,9sleep30-21swapper/322:52:093
167982473,9sleep30-21swapper/322:52:093
135899474,12cyclictest0-21swapper/321:53:463
1358994730,10cyclictest338-21aten_r1power_en20:03:323
1358994730,10cyclictest338-21aten_r1power_en20:03:323
1358994730,10cyclictest23410-21cpuspeed_turbos00:13:233
135899473,26cyclictest0-21swapper/322:58:243
135699474,37cyclictest0-21swapper/119:22:311
135699474,37cyclictest0-21swapper/119:22:311
1356994723,7cyclictest3941-21cat19:03:211
1356994719,9cyclictest1385-21cat23:28:111
1356994718,9cyclictest16985-21cat20:38:171
135299473,38cyclictest0-21swapper/019:33:460
111642472,9sleep30-21swapper/321:33:183
215112463,37sleep10-21swapper/123:01:221
206692462,9sleep30-21swapper/300:08:223
194432463,36sleep10-21swapper/120:43:181
178672463,37sleep10-21swapper/121:45:211
178432463,9sleep30-21swapper/321:45:023
166412462,39sleep00-21swapper/022:50:080
166412462,39sleep00-21swapper/022:50:080
158982463,37sleep10-21swapper/123:58:201
1358994633,8cyclictest12305-21kernelversion19:18:453
1358994633,8cyclictest12305-21kernelversion19:18:453
1358994631,9cyclictest22141-21ps19:38:483
1358994631,10cyclictest28195-21tr00:23:233
135899463,26cyclictest0-21swapper/300:18:293
135899463,25cyclictest0-21swapper/319:48:453
1358994629,11cyclictest29432-21cpuspeed_turbos21:03:313
1358994629,10cyclictest4417-21cpuspeed_turbos19:03:353
135699464,25cyclictest0-21swapper/122:53:421
1356994632,9cyclictest6749-21cpuspeed_turbos21:23:321
135699463,9cyclictest0-21swapper/123:32:321
135699463,9cyclictest0-21swapper/123:32:321
135699463,38cyclictest0-21swapper/123:33:411
1356994620,11cyclictest0-21swapper/120:58:311
1356994612,28cyclictest28581-21fschecks_count19:53:381
135299468,31cyclictest0-21swapper/021:02:370
135299466,10cyclictest0-21swapper/023:56:510
135299464,37cyclictest0-21swapper/023:38:310
135299464,36cyclictest0-21swapper/021:03:370
135299464,22cyclictest0-21swapper/023:24:550
135299463,37cyclictest23842-21taskset21:58:320
135299463,36cyclictest0-21swapper/020:38:290
135299463,21cyclictest0-21swapper/021:57:350
1352994626,14cyclictest0-21swapper/022:03:250
1352994623,18cyclictest0-21swapper/023:30:040
1352994623,18cyclictest0-21swapper/023:30:040
1352994619,10cyclictest16935-21latency_hist20:38:170
114622463,37sleep00-21swapper/020:23:500
105802466,9sleep10-21swapper/123:44:301
84122453,37sleep10-21swapper/119:10:361
59612452,38sleep00-21swapper/019:04:460
58892453,35sleep10-21swapper/121:19:461
52242452,37sleep00-21swapper/021:18:400
309442453,10sleep30-21swapper/323:19:543
278982452,37sleep00-21swapper/022:08:210
278982452,37sleep00-21swapper/022:08:210
263822452,9sleep30-21swapper/323:12:363
1358994525,13cyclictest0-21swapper/319:28:393
1358994519,10cyclictest15267-21head23:53:423
1358994516,13cyclictest0-21swapper/321:43:153
135699455,35cyclictest1369-21runrttasks19:29:451
135699455,32cyclictest0-21swapper/123:53:201
135699454,34cyclictest0-21swapper/122:48:121
135699454,25cyclictest0-21swapper/100:23:321
135699454,11cyclictest0-21swapper/120:53:171
1356994530,9cyclictest32225-21fschecks_time21:08:351
1356994529,10cyclictest24043-21tail19:43:421
1356994516,12cyclictest0-21swapper/119:23:471
1356994514,7cyclictest24957-21cut20:53:351
135299455,36cyclictest0-21swapper/021:30:210
135299455,36cyclictest0-21swapper/021:30:210
135299455,34cyclictest0-21swapper/020:48:290
135299454,35cyclictest0-21swapper/000:23:270
135299453,37cyclictest0-21swapper/020:55:370
135299453,36cyclictest0-21swapper/023:43:240
135299453,10cyclictest0-21swapper/023:06:530
1352994519,12cyclictest0-21swapper/020:44:280
60082442,36sleep30-21swapper/321:21:173
322022443,36sleep10-21swapper/122:15:151
135899447,21cyclictest5244-21munin-node20:13:413
135899443,34cyclictest0-21swapper/320:53:353
135899443,25cyclictest0-21swapper/319:24:443
1358994423,7cyclictest19339-21systemd-cgroups20:43:163
1358994419,10cyclictest26161-21fschecks_count22:03:323
1358994415,13cyclictest0-21swapper/318:58:353
135699445,33cyclictest14721-21aten_r1power_cu22:48:221
135699445,33cyclictest14721-21aten_r1power_cu22:48:221
135699445,14cyclictest0-21swapper/120:13:191
135699444,33cyclictest0-21swapper/121:38:151
135699443,35cyclictest0-21swapper/121:53:381
135699443,11cyclictest0-21swapper/100:16:001
1356994428,10cyclictest24407-21grep21:58:411
1356994426,10cyclictest4406-21timerwakeupswit22:23:481
1356994425,14cyclictest0-21swapper/121:28:361
1356994425,14cyclictest0-21swapper/121:28:361
135299444,36cyclictest0-21swapper/023:52:030
135299444,33cyclictest0-21swapper/019:28:330
135299444,13cyclictest0-21swapper/022:40:170
135299443,36cyclictest0-21swapper/022:28:330
135299443,36cyclictest0-21swapper/019:23:510
135299443,21cyclictest0-21swapper/021:49:110
1352994429,11cyclictest12932-21head21:33:500
10182442,36sleep00-21swapper/021:10:300
94292432,36sleep00-21swapper/022:34:410
1358994329,9cyclictest22236-21tr23:03:263
1358994321,8cyclictest1243-21snmpd22:15:133
135899430,8cyclictest7532-21memory21:23:433
135799435,12cyclictest0-21swapper/218:58:212
135699435,31cyclictest0-21swapper/119:13:331
135299435,31cyclictest0-21swapper/021:48:140
135299435,31cyclictest0-21swapper/019:13:430
135299434,35cyclictest0-21swapper/022:22:340
135299433,10cyclictest0-21swapper/021:23:400
1352994325,12cyclictest0-21swapper/019:48:400
135299432,22cyclictest2786-21munin-node22:23:250
36452422,35sleep00-21swapper/019:00:390
284932422,35sleep00-21swapper/023:14:020
189782423,33sleep10-21swapper/120:38:551
135899423,12cyclictest0-21swapper/320:23:453
1358994226,10cyclictest2385-21fschecks_time21:13:363
1358994220,9cyclictest26753-21munin-node20:58:333
1358994214,8cyclictest1476-21telnet23:28:223
1358994214,8cyclictest1476-21telnet23:28:223
135699425,30cyclictest0-21swapper/119:38:411
135699425,13cyclictest0-21swapper/122:08:471
135699425,13cyclictest0-21swapper/122:08:471
1356994222,13cyclictest25783-21cpuspeed_turbos00:18:231
1356994213,11cyclictest1506-21cat18:58:211
135299424,12cyclictest0-21swapper/020:28:340
135299423,35cyclictest0-21swapper/019:53:540
135299423,34cyclictest0-21swapper/000:03:510
1352994212,24cyclictest6505-21tail19:08:320
135899410,8cyclictest28556-21diskstats19:53:373
135699414,32cyclictest0-21swapper/120:23:181
135299415,30cyclictest0-21swapper/020:03:280
135299415,30cyclictest0-21swapper/020:03:280
135299413,11cyclictest0-21swapper/023:33:400
135899409,24cyclictest0-21swapper/323:38:323
1358994026,9cyclictest32542-21latency23:23:333
135699404,31cyclictest0-21swapper/119:58:311
1352994025,11cyclictest0-21swapper/000:11:500
1352994016,19cyclictest7398-21latency_hist20:18:180
135899364,25cyclictest1337-21munin-run18:58:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional