You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 09:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 21, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12198999337,49cyclictest18871-21grep19:10:441
12198998932,11cyclictest31148-21gltestperf20:45:361
1219899893,37cyclictest3992-21needreboot20:55:341
12198998834,8cyclictest16182-21cpuspeed_turbos22:30:201
12198998829,9cyclictest27587-21hwlatdetect21:45:291
12198998633,10cyclictest25125-21tail21:40:281
12198998534,10cyclictest5089-21taskset00:20:261
12198998518,11cyclictest28776-21basename22:55:281
12198998326,12cyclictest0-21swapper/123:25:041
12198998126,12cyclictest0-21swapper/122:10:071
12198997421,11cyclictest0-21swapper/100:01:361
243799660,5rtkit-daemon2436-21rtkit-daemon23:54:522
281552607,7sleep00-21swapper/021:45:410
243799590,4rtkit-daemon2436-21rtkit-daemon21:54:493
1219899593,24cyclictest0-21swapper/120:30:421
1219899593,24cyclictest0-21swapper/120:30:421
1219699594,48cyclictest0-21swapper/021:33:000
12200995829,9cyclictest17213-21telnet21:25:213
12199995828,25cyclictest0-21swapper/220:15:292
12199995823,9cyclictest7255-21munin-run23:20:032
1219699584,35cyclictest0-21swapper/019:20:350
75782572,10sleep20-21swapper/221:05:212
1220099573,46cyclictest0-21swapper/319:10:133
1219999574,36cyclictest0-21swapper/223:45:332
12198995725,10cyclictest10538-21unixbench_singl20:00:481
12196995721,30cyclictest6096-21latency_hist22:10:070
59192564,10sleep00-21swapper/019:52:050
243799560,7rtkit-daemon2436-21rtkit-daemon20:13:463
243799560,5rtkit-daemon2436-21rtkit-daemon23:25:232
243799560,5rtkit-daemon2436-21rtkit-daemon23:25:232
243799560,5rtkit-daemon2436-21rtkit-daemon22:05:413
191172563,11sleep00-21swapper/021:28:160
1219999565,32cyclictest0-21swapper/200:15:022
1219999565,32cyclictest0-21swapper/200:15:022
121282563,45sleep312139-21cut22:20:293
243799550,6rtkit-daemon2436-21rtkit-daemon20:54:223
216472554,14sleep20-21swapper/221:35:072
140842554,44sleep00-21swapper/023:30:500
12199995526,24cyclictest0-21swapper/200:15:232
12198995530,10cyclictest0-21swapper/121:00:111
12198995528,9cyclictest18373-21telnet20:20:271
1219699554,27cyclictest0-21swapper/023:15:200
285942543,9sleep00-21swapper/019:31:290
264042543,9sleep00-21swapper/019:29:030
243799540,6rtkit-daemon2436-21rtkit-daemon19:48:173
243799540,6rtkit-daemon2436-21rtkit-daemon19:20:333
243799540,5rtkit-daemon2436-21rtkit-daemon22:50:173
243799540,5rtkit-daemon2436-21rtkit-daemon21:40:353
195742542,44sleep10-21swapper/123:45:171
1219999545,13cyclictest0-21swapper/220:05:312
1219999544,12cyclictest0-21swapper/222:35:212
12199995425,24cyclictest0-21swapper/222:40:332
12199995425,24cyclictest0-21swapper/222:40:332
56932533,9sleep00-21swapper/000:20:380
288632532,9sleep30-21swapper/322:55:313
243799530,6rtkit-daemon2436-21rtkit-daemon23:42:002
243799530,5rtkit-daemon2436-21rtkit-daemon23:20:312
243799530,5rtkit-daemon2436-21rtkit-daemon21:45:423
243799530,5rtkit-daemon2436-21rtkit-daemon21:08:593
243799530,5rtkit-daemon2436-21rtkit-daemon20:40:303
243799530,5rtkit-daemon2436-21rtkit-daemon20:00:393
23952533,43sleep00-21swapper/023:09:300
178132533,10sleep30-21swapper/322:32:183
173852533,44sleep00-21swapper/020:15:420
144402534,11sleep00-21swapper/021:19:290
12200995321,27cyclictest0-21swapper/320:49:413
12199995337,9cyclictest6734-21tr19:55:252
1219899535,38cyclictest0-21swapper/123:30:361
1219899534,42cyclictest0-21swapper/122:50:041
12198995332,9cyclictest30224-21munin-node23:00:371
1219699535,25cyclictest27657-21sadc20:40:100
1219699534,43cyclictest0-21swapper/022:00:450
1219699534,42cyclictest0-21swapper/022:15:260
1219699533,37cyclictest0-21swapper/020:30:300
1219699533,37cyclictest0-21swapper/020:30:300
12196995324,11cyclictest0-21swapper/000:27:430
12196995317,21cyclictest27964-21tail22:55:200
70422522,10sleep30-21swapper/323:17:273
54582522,42sleep10-21swapper/119:50:451
243799520,5rtkit-daemon2436-21rtkit-daemon22:02:343
243799520,5rtkit-daemon2436-21rtkit-daemon19:15:403
189572524,42sleep10-21swapper/121:26:111
12200995235,11cyclictest24872-21grep20:30:473
12200995235,11cyclictest24872-21grep20:30:473
1219999524,29cyclictest0-21swapper/222:14:592
1219999524,13cyclictest0-21swapper/221:50:372
12198995230,8cyclictest32144-21sort21:55:241
12198995230,8cyclictest32144-21sort21:55:241
82282513,10sleep00-21swapper/019:56:040
322752512,10sleep20-21swapper/223:03:522
30192513,41sleep00-21swapper/020:55:240
26232513,42sleep12697-21missed_timers22:00:371
243799510,5rtkit-daemon2436-21rtkit-daemon22:45:173
243799510,5rtkit-daemon2436-21rtkit-daemon20:59:513
237332512,9sleep30-21swapper/321:36:173
14742512,9sleep20-21swapper/219:45:222
1219999516,27cyclictest0-21swapper/221:55:172
1219999516,27cyclictest0-21swapper/221:55:172
1219999515,28cyclictest0-21swapper/221:45:422
1219999515,22cyclictest26-21rcuc/220:20:482
1219999514,41cyclictest0-21swapper/223:35:212
1219999514,29cyclictest0-21swapper/219:35:322
1219999514,22cyclictest17331-21smartctl22:30:372
12199995136,9cyclictest24690-21grep19:25:282
12199995123,12cyclictest0-21swapper/219:00:422
12199995121,13cyclictest0-21swapper/221:20:392
12199995120,12cyclictest0-21swapper/219:16:562
12198995138,8cyclictest29547-21grep20:40:431
12198995125,12cyclictest0-21swapper/123:55:021
1219699515,39cyclictest0-21swapper/023:50:160
1219699514,26cyclictest0-21swapper/023:40:200
12196995121,13cyclictest0-21swapper/023:00:250
12196995121,11cyclictest0-21swapper/023:55:250
10662514,40sleep20-21swapper/220:50:302
82362503,40sleep00-21swapper/022:12:000
323422503,9sleep00-21swapper/020:49:110
311802502,9sleep30-21swapper/319:39:063
308492502,9sleep30-21swapper/300:06:073
300182503,41sleep00-21swapper/020:45:000
286302502,9sleep30-21swapper/300:03:173
243799500,5rtkit-daemon2436-21rtkit-daemon00:20:212
188832506,38sleep10-21swapper/123:41:101
165672503,10sleep00-21swapper/023:37:060
1219999505,29cyclictest0-21swapper/222:15:292
1219999505,29cyclictest0-21swapper/219:10:282
1219999505,28cyclictest0-21swapper/220:30:352
1219999505,28cyclictest0-21swapper/220:30:352
1219999505,27cyclictest0-21swapper/223:30:382
1219999504,29cyclictest0-21swapper/221:15:292
1219999504,28cyclictest0-21swapper/219:30:232
1219999504,27cyclictest0-21swapper/219:45:132
1219999504,27cyclictest0-21swapper/219:45:132
1219999503,24cyclictest28178-21tr22:55:212
12199995020,14cyclictest0-21swapper/221:40:262
12199995019,14cyclictest0-21swapper/200:25:352
12199995019,13cyclictest0-21swapper/221:00:082
12198995033,11cyclictest17724-21grep20:15:451
12198995024,12cyclictest0-21swapper/122:50:271
12198995023,12cyclictest0-21swapper/120:55:091
1219699505,27cyclictest0-21swapper/022:42:290
1219699505,27cyclictest0-21swapper/022:42:290
1219699505,27cyclictest0-21swapper/021:05:200
1219699505,26cyclictest0-21swapper/019:15:300
1219699505,26cyclictest0-21swapper/000:02:400
1219699504,27cyclictest0-21swapper/019:35:270
12196995022,11cyclictest0-21swapper/022:27:260
12196995019,10cyclictest20419-21latency_hist22:40:050
11352504,10sleep00-21swapper/021:59:390
11352504,10sleep00-21swapper/021:59:390
94772494,38sleep10-21swapper/121:07:051
286822492,10sleep30-21swapper/319:32:373
285132492,9sleep20-21swapper/200:01:472
243799490,5rtkit-daemon2436-21rtkit-daemon21:55:343
243799490,5rtkit-daemon2436-21rtkit-daemon21:55:343
243799490,5rtkit-daemon2436-21rtkit-daemon18:59:563
243799490,5rtkit-daemon2436-21rtkit-daemon18:59:563
214712494,39sleep10-21swapper/119:17:041
212642492,9sleep30-21swapper/323:46:003
177562493,39sleep30-21swapper/319:10:293
140772493,39sleep30-21swapper/321:15:423
131222493,41sleep00-21swapper/020:07:320
12200994929,8cyclictest31393-21munin-node00:10:263
12200994929,8cyclictest31393-21munin-node00:10:263
12200994921,13cyclictest0-21swapper/321:00:113
1219999495,27cyclictest0-21swapper/222:25:332
1219999495,27cyclictest0-21swapper/221:00:362
1219999495,27cyclictest0-21swapper/200:05:172
1219999494,29cyclictest0-21swapper/220:10:302
1219999494,28cyclictest0-21swapper/222:05:222
1219999494,28cyclictest0-21swapper/221:35:282
1219999494,28cyclictest0-21swapper/220:00:302
1219999494,27cyclictest0-21swapper/222:20:212
1219999494,27cyclictest0-21swapper/220:25:212
1219999494,27cyclictest0-21swapper/220:25:212
1219999494,27cyclictest0-21swapper/200:00:022
1219999494,26cyclictest0-21swapper/221:10:222
12199994920,12cyclictest0-21swapper/219:52:012
12198994935,9cyclictest8419-21crond00:30:001
12198994934,9cyclictest29993-21gltestperf19:35:311
1219699495,27cyclictest0-21swapper/021:52:200
1219699495,27cyclictest0-21swapper/019:40:370
1219699495,27cyclictest0-21swapper/019:40:370
1219699495,26cyclictest0-21swapper/022:52:300
1219699495,26cyclictest0-21swapper/020:22:050
1219699494,37cyclictest0-21swapper/022:30:320
12196994921,11cyclictest0-21swapper/020:25:210
12196994921,11cyclictest0-21swapper/020:25:210
12196994920,11cyclictest0-21swapper/019:46:020
60982483,38sleep30-21swapper/319:54:213
302622482,9sleep20-21swapper/220:45:142
243799480,4rtkit-daemon2436-21rtkit-daemon20:07:463
213752482,38sleep30-21swapper/321:31:373
213212484,37sleep10-21swapper/121:30:551
203962482,9sleep30-21swapper/322:39:553
12200994823,9cyclictest16974-21cat23:40:043
1219999484,28cyclictest0-21swapper/222:50:192
1219999484,27cyclictest0-21swapper/220:40:282
1219999484,27cyclictest0-21swapper/219:05:412
1219999484,26cyclictest0-21swapper/222:02:222
12199994820,13cyclictest0-21swapper/223:12:342
12199994819,12cyclictest0-21swapper/220:40:022
12199994818,13cyclictest0-21swapper/219:20:292
1219899483,39cyclictest0-21swapper/123:40:031
12198994828,8cyclictest7003-21sort19:55:281
12198994819,24cyclictest25061-21fschecks_time19:25:321
1219699486,37cyclictest0-21swapper/019:05:310
1219699485,25cyclictest0-21swapper/023:22:350
1219699484,39cyclictest0-21swapper/023:45:320
1219699484,26cyclictest0-21swapper/021:10:250
12196994833,10cyclictest29370-21tr00:05:180
1219699483,40cyclictest0-21swapper/022:20:400
12196994821,10cyclictest8821-21head20:00:260
12196994819,12cyclictest0-21swapper/021:42:190
12196994818,12cyclictest1243-21snmpd21:37:180
105502482,10sleep30-21swapper/322:16:023
70222473,39sleep00-21swapper/021:01:120
325332473,8sleep332631-21memory19:40:353
325332473,8sleep332631-21memory19:40:353
251352473,38sleep00-21swapper/022:49:230
223792470,3sleep3361ktimersoftd/322:40:393
223792470,3sleep3361ktimersoftd/322:40:393
19552470,6sleep20-21swapper/223:05:362
12200994727,8cyclictest1243-21snmpd23:23:573
12200994721,12cyclictest0-21swapper/321:10:233
1219999474,26cyclictest0-21swapper/221:25:292
12198994718,24cyclictest0-21swapper/122:35:271
1219699475,25cyclictest0-21swapper/023:27:350
1219699475,25cyclictest0-21swapper/023:27:350
1219699475,22cyclictest1104-21cut20:50:290
1219699474,25cyclictest0-21swapper/020:10:280
1219699473,26cyclictest0-21swapper/021:25:090
1219699473,26cyclictest0-21swapper/019:00:260
1219699473,25cyclictest0-21swapper/023:10:260
12196994720,11cyclictest0-21swapper/000:13:560
12196994720,11cyclictest0-21swapper/000:13:560
118032472,9sleep30-21swapper/323:27:183
118032472,9sleep30-21swapper/323:27:183
117292473,38sleep10-21swapper/123:26:191
117292473,38sleep10-21swapper/123:26:191
59342462,10sleep30-21swapper/300:23:483
33262463,10sleep00-21swapper/000:15:550
263612462,10sleep30-21swapper/323:59:503
243799460,6rtkit-daemon2436-21rtkit-daemon22:10:343
227752463,10sleep30-21swapper/320:28:593
227752463,10sleep30-21swapper/320:28:593
202462462,9sleep30-21swapper/320:22:063
142912462,9sleep30-21swapper/319:01:463
12200994633,8cyclictest29978-21kworker/3:023:10:193
12200994632,9cyclictest2317-21grep00:15:203
12200994630,10cyclictest16726-21grep20:15:303
12198994620,12cyclictest0-21swapper/100:20:021
119372464,6sleep00-21swapper/018:57:210
119372464,6sleep00-21swapper/018:57:210
263182452,10sleep30-21swapper/319:27:583
243799450,4rtkit-daemon2436-21rtkit-daemon19:56:313
1220099459,29cyclictest0-21swapper/323:05:233
12200994526,7cyclictest14512-21cat23:35:033
12200994525,8cyclictest6273-21munin-node00:25:263
12200994522,10cyclictest30224-21munin-node23:00:233
1219999455,12cyclictest0-21swapper/222:47:302
1219899454,35cyclictest0-21swapper/119:00:131
1219899454,35cyclictest0-21swapper/119:00:131
1219899453,36cyclictest0-21swapper/122:25:051
12198994522,7cyclictest8573-21cat22:15:061
1219699455,24cyclictest0-21swapper/019:10:240
243799440,4rtkit-daemon2436-21rtkit-daemon22:26:183
243799440,4rtkit-daemon2436-21rtkit-daemon21:22:063
243799440,4rtkit-daemon2436-21rtkit-daemon20:35:303
132322442,36sleep10-21swapper/120:08:581
1220099449,22cyclictest0-21swapper/323:42:213
12200994423,10cyclictest0-21swapper/323:50:383
12198994423,10cyclictest0-21swapper/120:35:401
12198994422,7cyclictest17087-21cat19:10:141
37072432,35sleep10-21swapper/119:49:361
1219899434,32cyclictest0-21swapper/121:10:221
1219899433,35cyclictest0-21swapper/123:10:351
1219899433,35cyclictest0-21swapper/100:05:161
1219899433,34cyclictest0-21swapper/122:45:051
1219899433,34cyclictest0-21swapper/122:45:051
1219899433,33cyclictest0-21swapper/100:15:021
1219899433,33cyclictest0-21swapper/100:15:021
12198994329,9cyclictest21926-21tr20:25:351
12198994329,9cyclictest21926-21tr20:25:351
12198994323,10cyclictest0-21swapper/119:30:431
12198994321,18cyclictest0-21swapper/122:27:281
12198994321,10cyclictest0-21swapper/119:44:301
12198994321,10cyclictest0-21swapper/119:44:301
1219899423,34cyclictest0-21swapper/120:10:231
1219899423,23cyclictest0-21swapper/121:15:301
1219899423,12cyclictest0-21swapper/119:01:011
12198994226,10cyclictest21945-21aten_r1power_en21:35:201
12198994221,11cyclictest0-21swapper/121:20:341
1219899413,34cyclictest0-21swapper/121:50:321
1219899413,32cyclictest0-21swapper/123:55:211
12198994121,9cyclictest0-21swapper/123:09:581
12198994121,10cyclictest0-21swapper/123:15:141
12198994119,8cyclictest24118-21awk19:25:121
12198994116,8cyclictest10914-21cat22:20:051
12199994016,9cyclictest0-21swapper/219:00:142
12199994016,9cyclictest0-21swapper/219:00:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional