You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 14:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Sun Nov 30, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
868821020,4sleep226-21rcuc/211:46:272
280719910029,64cyclictest29892-21grep09:06:450
2807199894,33cyclictest0-21swapper/011:56:290
2807199884,27cyclictest0-21swapper/008:06:390
2807199873,27cyclictest11877-21unixbench_multi11:51:410
2807199863,12cyclictest0-21swapper/010:50:210
2807199863,12cyclictest0-21swapper/010:50:210
2807199853,27cyclictest0-21swapper/010:31:240
28071998432,46cyclictest3487-21cpuspeed_turbos07:06:330
28071998432,46cyclictest3487-21cpuspeed_turbos07:06:330
2807199833,36cyclictest7054-21grep07:11:490
2807199824,11cyclictest0-21swapper/011:26:290
2807199824,11cyclictest0-21swapper/011:26:290
2807199823,29cyclictest12083-21grep07:21:530
28071998220,11cyclictest24117-21tune2fs08:56:330
2807199814,70cyclictest0-21swapper/010:46:090
2807199814,27cyclictest0-21swapper/010:06:440
2807199814,27cyclictest0-21swapper/008:45:580
2807199814,26cyclictest0-21swapper/011:36:280
2807199814,23cyclictest0-21swapper/009:11:300
2807199814,12cyclictest0-21swapper/011:16:290
28071998133,42cyclictest9947-21head08:26:350
28071998131,9cyclictest846-21cpuspeed_turbos09:16:260
28071998131,9cyclictest846-21cpuspeed_turbos09:16:260
2807199813,27cyclictest0-21swapper/009:26:350
28071998123,10cyclictest30181-21grep10:16:300
2603828110,17sleep20-21swapper/206:46:322
28071998026,10cyclictest12214-21tune2fs08:31:330
2807199794,29cyclictest0-21swapper/006:56:310
2807199794,12cyclictest0-21swapper/009:21:230
2807199794,12cyclictest0-21swapper/007:46:390
2807199793,32cyclictest0-21swapper/010:36:430
2807199793,12cyclictest0-21swapper/010:11:290
28071997929,9cyclictest21473-21grep12:11:380
28071997929,9cyclictest21473-21grep12:11:380
28071997920,7cyclictest1243-21snmpd08:16:000
2807199784,8cyclictest20797-21crond08:51:130
2807199783,29cyclictest0-21swapper/011:31:220
2807199783,21cyclictest0-21swapper/007:51:370
2807199783,21cyclictest0-21swapper/007:51:370
28071997822,10cyclictest1369-21runrttasks07:01:520
28071997818,8cyclictest13804-21munin-node10:51:210
2807199774,11cyclictest0-21swapper/009:36:370
2807199773,33cyclictest0-21swapper/012:16:370
2807199773,31cyclictest0-21swapper/011:01:250
28071997726,9cyclictest13067-21tr07:26:310
2807199754,25cyclictest0-21swapper/006:51:490
2807199753,27cyclictest8564-21tune2fs09:31:300
28071997522,8cyclictest21047-21munin-node08:51:330
28071997518,8cyclictest1243-21snmpd09:46:140
28071997517,9cyclictest1369-21runrttasks10:30:390
2807199745,29cyclictest0-21swapper/007:16:460
2807199744,30cyclictest0-21swapper/008:16:410
28071997417,8cyclictest1243-21snmpd10:56:250
2807199733,23cyclictest0-21swapper/012:06:070
2807199733,23cyclictest0-21swapper/011:46:300
2807199733,23cyclictest0-21swapper/007:56:300
28071997320,8cyclictest29444-21munin-node08:01:450
2807199724,31cyclictest0-21swapper/011:45:530
2807199724,30cyclictest0-21swapper/012:06:240
2807199723,23cyclictest0-21swapper/010:01:290
2807199723,23cyclictest0-21swapper/010:01:290
2807199723,12cyclictest0-21swapper/007:31:400
28071997218,12cyclictest0-21swapper/009:01:260
28071997216,10cyclictest19335-21sshd07:37:290
2807199714,11cyclictest0-21swapper/010:21:190
2807199703,29cyclictest0-21swapper/009:56:320
28071996915,9cyclictest19434-21sshd09:54:240
2807199683,29cyclictest0-21swapper/011:21:320
28071996723,10cyclictest0-21swapper/008:36:280
28071996723,10cyclictest0-21swapper/008:36:280
28071996718,8cyclictest21993-21yum-updatesd-he07:45:490
215372634,12sleep20-21swapper/211:06:262
294772623,9sleep329552-21sendmail_mailqu11:21:393
2807999614,49cyclictest0-21swapper/311:56:413
28078996133,12cyclictest0-21swapper/210:31:342
243799610,4rtkit-daemon2436-21rtkit-daemon07:27:071
2420926118,9sleep20-21swapper/212:20:262
243799600,6rtkit-daemon2436-21rtkit-daemon09:31:241
243799600,4rtkit-daemon2436-21rtkit-daemon06:55:451
2807999585,46cyclictest0-21swapper/310:41:333
2807999584,46cyclictest0-21swapper/307:01:383
2807999584,45cyclictest1243-21snmpd10:46:233
2807999584,45cyclictest1243-21snmpd10:46:233
243799580,4rtkit-daemon2436-21rtkit-daemon11:42:191
243799580,4rtkit-daemon2436-21rtkit-daemon08:50:561
77002574,46sleep20-21swapper/207:16:172
2807999575,45cyclictest0-21swapper/308:11:283
28076995712,27cyclictest23873-21unixbench-2d10:01:451
28076995712,27cyclictest23873-21unixbench-2d10:01:451
2807999564,45cyclictest0-21swapper/308:51:143
2807999563,46cyclictest0-21swapper/311:26:193
2807999563,46cyclictest0-21swapper/311:26:193
2807899564,43cyclictest0-21swapper/209:06:322
28076995624,10cyclictest26525-21fschecks_time11:16:281
28076995623,14cyclictest0-21swapper/111:11:401
2807199563,45cyclictest0-21swapper/009:45:530
243799560,4rtkit-daemon2436-21rtkit-daemon07:12:001
138442560,7chrt0-21swapper/111:56:341
123492563,7sleep112431-21iostat10:46:311
123492563,7sleep112431-21iostat10:46:311
28076995522,13cyclictest0-21swapper/111:51:251
28076995521,14cyclictest0-21swapper/107:46:431
290222543,10sleep10-21swapper/110:14:511
2807699549,25cyclictest21556-21fschecks_count11:06:261
2807699544,31cyclictest0-21swapper/112:14:181
2807699544,31cyclictest0-21swapper/112:14:181
2807699544,29cyclictest0-21swapper/110:26:281
28076995422,13cyclictest0-21swapper/112:01:171
28076995422,13cyclictest0-21swapper/107:31:371
243799540,6rtkit-daemon2436-21rtkit-daemon06:56:381
243799540,5rtkit-daemon2436-21rtkit-daemon07:45:581
199262543,11sleep20-21swapper/211:01:392
181702542,10sleep30-21swapper/310:59:023
146472542,10sleep20-21swapper/207:28:352
109852542,9sleep20-21swapper/210:43:412
51052533,12sleep10-21swapper/107:08:341
51052533,12sleep10-21swapper/107:08:341
325982533,11sleep10-21swapper/111:30:191
325982533,11sleep10-21swapper/111:30:191
28079995339,9cyclictest32696-21sed08:06:363
2807699537,28cyclictest0-21swapper/108:01:371
2807699535,29cyclictest0-21swapper/109:26:431
2807699535,29cyclictest0-21swapper/108:56:391
2807699535,29cyclictest0-21swapper/108:31:311
2807699535,29cyclictest0-21swapper/108:26:391
2807699535,29cyclictest0-21swapper/108:16:491
2807699535,28cyclictest0-21swapper/110:31:211
2807699535,27cyclictest0-21swapper/111:36:311
2807699534,28cyclictest0-21swapper/109:06:391
28076995319,14cyclictest0-21swapper/107:56:531
243799530,5rtkit-daemon2436-21rtkit-daemon08:24:271
222312533,42sleep322338-21cpuspeed12:16:203
183882532,10sleep30-21swapper/308:45:583
97712523,10sleep10-21swapper/107:17:131
302212522,10sleep30-21swapper/309:09:243
2807999524,42cyclictest0-21swapper/308:26:313
28079995235,11cyclictest31285-21cut06:56:373
2807699525,29cyclictest0-21swapper/110:41:331
2807699525,28cyclictest0-21swapper/111:21:271
2807699525,28cyclictest0-21swapper/110:51:401
2807699525,28cyclictest0-21swapper/110:16:221
2807699525,28cyclictest0-21swapper/109:56:421
2807699525,28cyclictest0-21swapper/109:16:411
2807699525,28cyclictest0-21swapper/109:16:411
2807699525,27cyclictest0-21swapper/112:06:441
2807699525,27cyclictest0-21swapper/110:36:351
2807699525,27cyclictest0-21swapper/109:01:231
2807699525,27cyclictest0-21swapper/108:11:261
2807699525,26cyclictest0-21swapper/110:56:251
2807699524,42cyclictest0-21swapper/111:01:251
2807699524,29cyclictest0-21swapper/108:51:261
2807699524,29cyclictest0-21swapper/108:06:381
2807699524,28cyclictest0-21swapper/109:36:251
2807699524,27cyclictest1179-21lldpd08:37:331
2807699524,27cyclictest1179-21lldpd08:37:331
28076995231,12cyclictest743-21cpuspeed_turbos11:31:201
28076995219,12cyclictest0-21swapper/107:01:341
28076995216,13cyclictest0-21swapper/109:11:381
243799520,5rtkit-daemon2436-21rtkit-daemon09:51:391
243799520,5rtkit-daemon2436-21rtkit-daemon08:43:491
116522523,10sleep10-21swapper/107:21:481
50362512,10sleep20-21swapper/211:39:392
28079995136,9cyclictest24306-21gltestperf08:56:353
28079995135,11cyclictest4208-21head11:36:333
28079995124,8cyclictest18743-21munin-node12:06:363
2807699515,27cyclictest0-21swapper/110:06:171
2807699515,26cyclictest0-21swapper/109:41:131
2807699514,29cyclictest0-21swapper/112:16:231
2807699514,28cyclictest0-21swapper/109:21:261
28076995120,12cyclictest0-21swapper/110:21:341
28076995119,13cyclictest0-21swapper/111:46:351
27212512,10sleep30-21swapper/309:19:313
27212512,10sleep30-21swapper/309:19:313
266012513,11sleep10-21swapper/107:53:331
266012513,11sleep10-21swapper/107:53:331
217562512,10sleep30-21swapper/312:14:383
217562512,10sleep30-21swapper/312:14:383
206632514,7sleep220764-21http_loadtime07:41:372
79392502,10sleep30-21swapper/307:16:303
76132502,10sleep30-21swapper/309:31:083
39082503,10sleep30-21swapper/310:30:003
2807999505,30cyclictest0-21swapper/311:16:273
2807999505,30cyclictest0-21swapper/307:21:463
2807999504,25cyclictest4888-21cut10:31:303
2807999503,43cyclictest0-21swapper/307:26:523
2807899504,21cyclictest0-21swapper/209:16:292
2807899504,21cyclictest0-21swapper/209:16:292
28078995026,11cyclictest0-21swapper/209:51:242
2807699504,23cyclictest17473-21runrttasks07:36:231
265182503,10sleep20-21swapper/207:52:292
265182503,10sleep20-21swapper/207:52:292
263952502,42sleep30-21swapper/310:06:513
241982502,9sleep30-21swapper/310:04:173
241982502,9sleep30-21swapper/310:04:173
86172492,10sleep30-21swapper/308:23:173
326252493,10sleep30-21swapper/309:14:363
291112492,9sleep20-21swapper/210:16:012
2807999493,39cyclictest0-21swapper/310:21:303
28078994936,8cyclictest8367-21unixbench_singl10:36:432
28078994933,10cyclictest26570-21tune2fs09:01:302
28078994923,21cyclictest0-21swapper/210:52:532
28078994922,23cyclictest0-21swapper/208:47:182
2807699494,27cyclictest0-21swapper/109:51:111
19972492,10sleep30-21swapper/311:31:383
167792492,9sleep30-21swapper/312:01:573
146592492,10sleep30-21swapper/309:44:263
123002492,10sleep20-21swapper/207:24:032
100082493,10sleep20-21swapper/207:20:202
74262482,10sleep30-21swapper/307:12:443
2807999485,28cyclictest0-21swapper/307:35:533
2807999485,27cyclictest0-21swapper/308:41:033
2807999485,27cyclictest0-21swapper/308:41:033
2807999485,27cyclictest0-21swapper/307:55:563
2807999485,27cyclictest0-21swapper/307:55:563
28079994832,10cyclictest7101-21grep11:41:413
2807999482,11cyclictest0-21swapper/307:56:303
2807899484,37cyclictest0-21swapper/207:01:182
28078994834,9cyclictest4339-21aten_r1power_cu08:16:262
28078994833,9cyclictest9566-21grep09:31:442
28078994833,9cyclictest25060-21unixbench_multi11:11:412
2807899483,38cyclictest0-21swapper/208:21:382
28078994812,28cyclictest26375-21grep11:16:262
280532482,39sleep20-21swapper/207:56:402
247962484,38sleep324832-21switchtime11:11:393
146932482,40sleep20-21swapper/209:44:532
99192472,9sleep30-21swapper/309:35:073
2807999476,22cyclictest3328-21aten2.4-expect07:06:313
2807999476,22cyclictest3328-21aten2.4-expect07:06:313
2807999475,26cyclictest0-21swapper/309:56:163
2807999475,26cyclictest0-21swapper/309:06:073
2807999474,28cyclictest0-21swapper/307:46:443
2807999474,12cyclictest0-21swapper/308:01:313
28079994730,11cyclictest28721-21sed10:11:463
28079994720,9cyclictest11628-21telnet08:31:273
28078994734,8cyclictest14688-21gltestperf08:36:342
28078994734,8cyclictest14688-21gltestperf08:36:342
2807899473,39cyclictest11843-21df10:46:242
2807899473,39cyclictest11843-21df10:46:242
24212472,10sleep20-21swapper/211:32:062
194442472,9sleep30-21swapper/307:38:553
177872472,38sleep20-21swapper/212:06:232
304362462,10sleep20-21swapper/206:56:142
28162462,9sleep20-21swapper/207:04:492
2807999464,24cyclictest18441-21iostat09:51:333
28079994617,11cyclictest0-21swapper/311:01:253
28079994615,10cyclictest17362-21cat09:51:123
2807899465,35cyclictest0-21swapper/211:41:292
28078994633,8cyclictest5795-21grep09:26:262
28078994633,8cyclictest16836-21unixbench_singl07:31:502
28078994632,9cyclictest26265-21grep10:06:442
28078994631,9cyclictest19866-21head09:56:222
277132464,6sleep00-21swapper/006:47:030
252692462,10sleep20-21swapper/208:57:102
231582462,38sleep20-21swapper/208:55:492
231062462,9sleep20-21swapper/210:01:342
231062462,9sleep20-21swapper/210:01:342
192982462,10sleep20-21swapper/207:37:012
122432462,10sleep30-21swapper/311:55:003
326192453,36sleep20-21swapper/209:14:312
314862452,37sleep30-21swapper/310:20:533
28079994530,9cyclictest11906-21grep09:36:463
28079994527,13cyclictest0-21swapper/310:36:223
2807899454,34cyclictest0-21swapper/210:16:402
2807899454,26cyclictest0-21swapper/207:46:442
28078994531,9cyclictest29343-21head11:21:352
2807899453,26cyclictest0-21swapper/210:26:272
2807199453,27cyclictest0-21swapper/011:11:400
217342452,10sleep30-21swapper/307:42:263
11382452,9sleep20-21swapper/210:21:532
80472443,35sleep08120-21fgrep08:21:450
51882442,36sleep20-21swapper/207:09:402
51882442,36sleep20-21swapper/207:09:402
2807999444,8cyclictest23117-21munin-run11:11:073
2807999444,34cyclictest0-21swapper/310:56:093
2807899444,35cyclictest0-21swapper/208:41:412
2807899443,37cyclictest0-21swapper/208:12:202
28078994428,11cyclictest16663-21grep12:01:382
2807199444,34cyclictest0-21swapper/011:06:220
216582443,10sleep20-21swapper/212:13:202
216582443,10sleep20-21swapper/212:13:202
111152442,9sleep20-21swapper/208:29:452
292682432,35sleep30-21swapper/306:51:403
2807999433,22cyclictest21047-21munin-node08:51:303
2807899439,29cyclictest18349-21awk11:01:082
2807899434,35cyclictest0-21swapper/211:56:332
2807899432,35cyclictest0-21swapper/209:21:222
279302434,6sleep10-21swapper/106:49:531
169592433,10sleep20-21swapper/209:48:162
2807999424,33cyclictest0-21swapper/308:16:353
2807899424,32cyclictest0-21swapper/208:31:272
28078994228,9cyclictest32695-21grep08:06:362
28078994226,10cyclictest30318-21gltestperf08:01:352
2807999414,33cyclictest0-21swapper/309:21:393
28078994127,9cyclictest10588-21cpuspeed_turbos09:36:282
2807999403,32cyclictest0-21swapper/311:46:393
2807899404,31cyclictest0-21swapper/211:51:232
28078994024,10cyclictest30609-21aten2.4-expect11:26:202
28078994024,10cyclictest30609-21aten2.4-expect11:26:202
2807999303,23cyclictest28060-21munin-run06:51:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional