You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-05 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Tue Dec 05, 2023 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1623991050,16rtkit-daemon0-21swapper/010:22:190
327330210061,14sleep30-21swapper/307:09:163
32737229876,14sleep20-21swapper/207:09:482
32595229859,13sleep10-21swapper/107:05:201
162399970,9rtkit-daemon1622-21rtkit-daemon10:15:581
32754499922,86cyclictest0-21swapper/212:17:592
32754499922,86cyclictest0-21swapper/209:16:592
32713328464,13sleep00-21swapper/007:06:440
162399820,35rtkit-daemon427545-21users11:50:393
162399820,26rtkit-daemon1622-21rtkit-daemon10:09:323
327540997971,5cyclictest0-21swapper/112:17:591
327540997871,5cyclictest0-21swapper/109:16:591
32754099783,44cyclictest0-21swapper/110:46:231
327540996962,5cyclictest0-21swapper/108:02:591
162399690,10rtkit-daemon1622-21rtkit-daemon10:40:050
4038762670,14sleep0131ksoftirqd/010:45:300
327544996724,14cyclictest880726-21firefox12:38:042
162399670,9rtkit-daemon1622-21rtkit-daemon11:29:110
4091722650,5sleep1409168-40lsb_release11:00:241
3345552640,9chrt881251-21Softwar~cThread07:25:373
32753299643,54cyclictest1753-21gnome-shell11:01:590
32754099638,15cyclictest26-21ksoftirqd/108:31:591
327544996127,13cyclictest1753-21gnome-shell08:01:592
32754499612,54cyclictest880898-21Xwayland07:16:222
32754499612,54cyclictest880898-21Xwayland07:16:222
32754799602,53cyclictest1753-21gnome-shell08:17:593
327544996031,7cyclictest1753-21gnome-shell10:16:582
32754499602,16cyclictest1753-21gnome-shell11:34:582
327547995937,17cyclictest1753-21gnome-shell08:36:593
32754799591,18cyclictest1753-21gnome-shell09:06:593
32754799591,17cyclictest1753-21gnome-shell09:33:063
32754799591,17cyclictest1753-21gnome-shell09:33:063
32754499593,48cyclictest34-21ksoftirqd/209:31:592
32754499593,48cyclictest34-21ksoftirqd/209:31:582
32753299591,54cyclictest1753-21gnome-shell11:53:590
32754799571,51cyclictest1753-21gnome-shell11:37:593
327540995727,26cyclictest420868-21grep11:35:001
32754099572,12cyclictest880726-21firefox11:46:221
162399570,9rtkit-daemon1622-21rtkit-daemon08:31:512
162399570,8rtkit-daemon1622-21rtkit-daemon08:42:292
32754799563,11cyclictest881331-21Privileged14:31:443
32754799562,50cyclictest1753-21gnome-shell09:56:593
32754799562,50cyclictest1753-21gnome-shell09:56:593
32754799561,51cyclictest1753-21gnome-shell08:03:583
32754099561,50cyclictest1753-21gnome-shell08:27:591
32753299563,47cyclictest881253-21Renderer10:16:230
32753299561,51cyclictest1753-21gnome-shell12:05:590
32754799553,11cyclictest1753-21gnome-shell10:16:233
32754499552,47cyclictest1753-21gnome-shell09:04:592
32754499551,49cyclictest1753-21gnome-shell09:51:592
32754099556,46cyclictest231irq_work/112:12:591
32754099556,46cyclictest231irq_work/112:12:591
32754099552,48cyclictest1753-21gnome-shell10:55:591
32754099552,10cyclictest881253-21Renderer07:16:221
32754099552,10cyclictest881253-21Renderer07:16:221
327540995511,5cyclictest0-21swapper/109:31:591
327540995511,5cyclictest0-21swapper/109:31:591
327532995537,12cyclictest1753-21gnome-shell12:16:230
32754799547,44cyclictest42-21ksoftirqd/307:46:593
32754799542,11cyclictest1753-21gnome-shell11:42:593
327547995419,30cyclictest42-21ksoftirqd/308:53:593
32753299542,9cyclictest881335-21WRScene~ilder#110:31:210
32753299542,48cyclictest1753-21gnome-shell08:19:590
327547995346,5cyclictest327527-21cyclictest09:52:593
32754799533,12cyclictest0-21swapper/310:46:233
32754799531,48cyclictest1753-21gnome-shell07:20:593
327544995336,13cyclictest1753-21gnome-shell09:09:582
327544995336,12cyclictest1753-21gnome-shell07:56:592
32754099532,46cyclictest1753-21gnome-shell11:50:591
327540995321,27cyclictest344935-21ntp_states07:55:361
327540995315,33cyclictest390568-21cat10:05:391
32753299537,44cyclictest327527-21cyclictest08:28:580
32753299532,9cyclictest1576137-21update-manager09:33:060
32753299532,9cyclictest1576137-21update-manager09:33:060
32754799521,47cyclictest1753-21gnome-shell07:53:593
32754799521,47cyclictest1753-21gnome-shell07:37:593
32754499525,7cyclictest0-21swapper/207:35:592
32754499521,46cyclictest1753-21gnome-shell09:22:592
32754499521,11cyclictest1753-21gnome-shell09:36:592
32754099524,46cyclictest231irq_work/110:27:581
32754099522,47cyclictest880898-21Xwayland10:31:221
327540995219,28cyclictest359397-21seq08:38:351
32754099521,45cyclictest432349-21cat12:05:301
327540995214,33cyclictest333549-21cat07:25:021
162399520,25rtkit-daemon0-21swapper/312:15:503
162399520,25rtkit-daemon0-21swapper/310:51:393
162399520,25rtkit-daemon0-21swapper/310:51:393
32754499514,8cyclictest34-21ksoftirqd/211:38:592
32754499513,8cyclictest0-21swapper/211:46:232
327544995137,9cyclictest1753-21gnome-shell08:07:592
32754499512,44cyclictest0-21swapper/211:58:582
32754099513,45cyclictest231irq_work/108:20:591
32754099512,8cyclictest231irq_work/110:21:591
32754099512,45cyclictest231irq_work/111:24:591
32754099512,45cyclictest231irq_work/110:41:591
32754099512,45cyclictest1753-21gnome-shell12:30:581
32754099512,45cyclictest0-21swapper/111:15:591
32754099512,45cyclictest0-21swapper/111:15:581
327540995115,31cyclictest348046-21seq08:05:311
32754499502,7cyclictest0-21swapper/212:32:592
32754499502,7cyclictest0-21swapper/212:28:592
32754499502,6cyclictest0-21swapper/207:47:592
32754499502,44cyclictest0-21swapper/209:57:592
32754499502,44cyclictest0-21swapper/209:57:582
32754499502,44cyclictest0-21swapper/209:48:592
32754099503,44cyclictest231irq_work/111:55:591
32754099502,45cyclictest231irq_work/109:20:591
32754099501,8cyclictest231irq_work/108:54:591
327540995018,27cyclictest438632-21seq12:25:151
327540995015,31cyclictest381739-21wc09:40:431
327540995015,30cyclictest335644-21tr07:30:231
32754099501,45cyclictest231irq_work/112:24:591
32753299501,45cyclictest1753-21gnome-shell07:30:580
327547994937,7cyclictest42-21ksoftirqd/308:07:593
32754799492,7cyclictest327527-21cyclictest09:14:593
32754799491,44cyclictest1753-21gnome-shell09:18:593
32754499492,7cyclictest0-21swapper/207:52:592
32754499492,7cyclictest0-21swapper/207:10:592
32754499492,7cyclictest0-21swapper/207:10:592
32754499492,44cyclictest0-21swapper/207:32:592
32754499492,43cyclictest0-21swapper/207:43:582
32754099494,39cyclictest1753-21gnome-shell11:06:591
327540994940,6cyclictest231irq_work/109:01:591
32754099493,10cyclictest1827-21JS14:31:441
32754099492,45cyclictest231irq_work/108:56:591
327540994922,23cyclictest337168-21/usr/sbin/munin07:35:371
327540994922,22cyclictest343188-21ntpq07:50:331
327540994920,24cyclictest352068-21sed08:15:441
327540994915,29cyclictest405469-21expr10:50:181
327540994915,29cyclictest405469-21expr10:50:171
32754099491,45cyclictest231irq_work/107:27:591
32754099491,44cyclictest231irq_work/109:26:591
327540994913,31cyclictest389011-21Indexed~#14162410:01:221
327540994912,33cyclictest328224-21irqstats07:10:311
327540994912,33cyclictest328224-21irqstats07:10:311
327532994937,8cyclictest1753-21gnome-shell11:46:220
32753299492,5cyclictest0-21swapper/011:42:590
4405742480,5chrt0-21swapper/312:30:213
4161662480,24sleep2331rcuc/211:20:212
327547994839,6cyclictest327527-21cyclictest12:38:593
327547994834,9cyclictest881331-21Privileged14:31:443
327547994826,18cyclictest961-21snmpd11:13:573
32754799482,5cyclictest0-21swapper/312:21:593
32754499482,7cyclictest0-21swapper/208:16:592
32754499482,6cyclictest0-21swapper/208:38:582
32754499482,6cyclictest0-21swapper/207:25:592
327544994823,9cyclictest400688-21strings10:35:362
327540994839,6cyclictest231irq_work/111:41:581
32754099482,42cyclictest0-21swapper/112:00:591
32754099482,42cyclictest0-21swapper/108:47:581
327540994814,29cyclictest339342-21expr07:40:231
32753299482,7cyclictest113-1kworker/0:1H09:16:220
32753299482,5cyclictest0-21swapper/008:38:590
32753299482,44cyclictest0-21swapper/007:50:590
327547994726,17cyclictest961-21snmpd12:04:063
327547994726,17cyclictest961-21snmpd11:28:583
32754799472,42cyclictest0-21swapper/311:22:583
32754499472,7cyclictest0-21swapper/210:20:592
32754499472,5cyclictest0-21swapper/208:24:592
32754499472,42cyclictest0-21swapper/211:13:592
32754499472,42cyclictest0-21swapper/210:08:592
327540994738,6cyclictest231irq_work/111:35:581
327540994738,6cyclictest231irq_work/110:12:591
32754099472,5cyclictest0-21swapper/108:12:591
327540994718,24cyclictest340690-21sshd07:45:161
327540994714,28cyclictest442548-21smartctl12:35:301
327540994713,29cyclictest881251-21Softwar~cThread09:07:421
32753299472,43cyclictest0-21swapper/011:56:590
32753299472,43cyclictest0-21swapper/011:08:590
32753299472,43cyclictest0-21swapper/007:55:580
32753299472,42cyclictest0-21swapper/008:03:590
32754799462,5cyclictest0-21swapper/312:11:593
32754799462,5cyclictest0-21swapper/312:11:583
32754799462,42cyclictest0-21swapper/309:36:593
32754799462,42cyclictest0-21swapper/308:29:583
32754499462,5cyclictest0-21swapper/211:01:222
32754499462,5cyclictest0-21swapper/210:50:592
32754499462,5cyclictest0-21swapper/210:50:592
32754499462,41cyclictest0-21swapper/210:28:592
32754099468,12cyclictest380105-21cat09:35:391
327540994639,4cyclictest0-21swapper/111:28:591
327540994612,30cyclictest384796-21wc09:50:301
327540994612,29cyclictest881251-21Softwar~cThread09:45:231
327532994640,4cyclictest0-21swapper/011:17:590
327532994640,4cyclictest0-21swapper/011:17:580
327532994640,4cyclictest0-21swapper/007:45:590
32753299462,42cyclictest0-21swapper/010:27:590
32753299462,41cyclictest0-21swapper/009:59:590
32753299462,41cyclictest0-21swapper/009:59:590
327532994621,21cyclictest371696-21unin-run09:15:010
32753299461,5cyclictest141rcu_preempt07:40:220
32753299461,40cyclictest1753-21gnome-shell12:01:590
3654122450,7sleep342-21ksoftirqd/308:55:283
327547994538,4cyclictest0-21swapper/307:58:593
32754799452,41cyclictest0-21swapper/312:09:593
32754799452,41cyclictest0-21swapper/310:41:593
327547994524,17cyclictest961-21snmpd09:23:243
32754499453,37cyclictest1753-21gnome-shell09:14:592
32754099457,12cyclictest370530-21expr09:10:201
32754099457,11cyclictest360590-21proc_pri08:40:361
327540994538,4cyclictest0-21swapper/111:10:591
32754099451,41cyclictest0-21swapper/109:56:021
32754099451,41cyclictest0-21swapper/109:56:021
327532994539,4cyclictest0-21swapper/011:37:590
327532994539,4cyclictest0-21swapper/011:22:580
327532994539,4cyclictest0-21swapper/010:52:590
327532994539,4cyclictest0-21swapper/010:52:590
327532994526,15cyclictest347896-21expr08:05:250
32753299452,5cyclictest0-21swapper/008:31:220
3658782440,9sleep013-21ksoftirqd/008:55:380
327547994438,4cyclictest0-21swapper/308:46:593
327547994432,8cyclictest1753-21gnome-shell08:44:593
327547994426,6cyclictest42-21ksoftirqd/308:20:393
32754799442,40cyclictest0-21swapper/312:25:593
327544994438,4cyclictest0-21swapper/208:26:592
327532994438,4cyclictest0-21swapper/009:36:590
327532994438,4cyclictest0-21swapper/007:29:590
327532994437,4cyclictest0-21swapper/009:52:590
327532994411,11cyclictest881251-21Softwar~cThread09:02:210
162399440,31rtkit-daemon0-21swapper/311:06:153
327547994337,4cyclictest0-21swapper/307:11:593
327547994337,4cyclictest0-21swapper/307:11:593
327547994336,5cyclictest0-21swapper/307:17:593
327547994336,5cyclictest0-21swapper/307:17:593
327547994324,6cyclictest42-21ksoftirqd/309:45:343
327547994323,16cyclictest961-21snmpd10:28:473
32754499432,7cyclictest0-21swapper/210:03:592
327532994337,4cyclictest0-21swapper/012:20:590
327532994337,4cyclictest0-21swapper/010:07:580
327532994323,15cyclictest13-21ksoftirqd/010:00:210
162399430,31rtkit-daemon880726-21firefox09:28:530
327547994236,4cyclictest0-21swapper/309:27:583
327547994235,4cyclictest0-21swapper/310:32:593
327547994230,8cyclictest1753-21gnome-shell10:23:593
327547994230,8cyclictest1753-21gnome-shell10:13:583
327547994225,6cyclictest42-21ksoftirqd/311:30:153
327547994222,7cyclictest42-21ksoftirqd/311:55:283
327547994221,18cyclictest42-21ksoftirqd/310:35:433
327547994217,21cyclictest388000-40lsb_release10:00:203
327544994234,5cyclictest1576137-21update-manager12:08:262
327544994224,13cyclictest438-20systemd-journal07:21:162
327532994236,4cyclictest0-21swapper/012:25:590
327532994234,5cyclictest131ksoftirqd/007:25:010
327532994225,12cyclictest438-20systemd-journal10:55:150
32753299422,36cyclictest270290-21kworker/0:107:19:590
32753299422,36cyclictest270290-21kworker/0:107:19:590
162399420,31rtkit-daemon0-21swapper/212:23:212
327547994124,6cyclictest42-21ksoftirqd/308:31:133
327547994124,6cyclictest42-21ksoftirqd/307:40:413
327547994122,7cyclictest42-21ksoftirqd/308:10:333
327547994118,19cyclictest42-21ksoftirqd/311:17:153
327547994118,19cyclictest42-21ksoftirqd/311:17:153
32754799411,12cyclictest1753-21gnome-shell09:40:593
327544994135,4cyclictest0-21swapper/208:13:592
327544994130,7cyclictest1753-21gnome-shell08:52:592
327532994135,4cyclictest0-21swapper/009:49:580
327532994135,4cyclictest0-21swapper/009:05:590
327532994130,7cyclictest1753-21gnome-shell09:23:580
327547994023,6cyclictest42-21ksoftirqd/309:00:203
327547994023,6cyclictest42-21ksoftirqd/307:30:543
327547994022,6cyclictest42-21ksoftirqd/310:55:353
327532994034,4cyclictest0-21swapper/011:11:590
327532994034,4cyclictest0-21swapper/007:10:590
327532994034,4cyclictest0-21swapper/007:10:590
327532994022,13cyclictest433831-21df_inode12:10:230
327532994022,13cyclictest433831-21df_inode12:10:230
327532993922,6cyclictest13-21ksoftirqd/011:30:380
32753299391,33cyclictest1753-21gnome-shell08:21:590
327544993818,15cyclictest1-21systemd10:15:012
327532993832,4cyclictest0-21swapper/008:11:590
327532993826,8cyclictest0-21swapper/012:32:590
327544993731,4cyclictest0-21swapper/211:54:592
327532993724,9cyclictest382123-21sshd09:43:470
327532993723,9cyclictest881625-21Isolated14:31:440
32753299372,31cyclictest0-21swapper/010:11:590
32753299372,31cyclictest0-21swapper/007:37:590
327532993719,13cyclictest13-21ksoftirqd/008:45:010
327544993630,4cyclictest0-21swapper/211:26:592
327544993625,7cyclictest0-21swapper/212:00:262
327544993621,11cyclictest407534-21wc10:55:292
327544993621,10cyclictest438-20systemd-journal10:45:002
327544993620,12cyclictest855-21dbus-daemon09:41:562
327544993620,12cyclictest399837-21sed10:35:022
327544993619,13cyclictest410795-21cat11:05:192
327544993619,12cyclictest362562-21wc08:45:422
327544993618,13cyclictest881251-21Softwar~cThread12:12:502
327544993618,13cyclictest881251-21Softwar~cThread12:12:502
327532993619,5cyclictest13-21ksoftirqd/008:45:360
327544993524,6cyclictest0-21swapper/210:46:212
327544993520,11cyclictest415649-21cron11:20:002
327544993520,11cyclictest415649-21cron11:20:002
327544993519,12cyclictest422866-21basename11:40:202
327532993525,7cyclictest0-21swapper/010:36:580
327544993419,11cyclictest375673-21expr09:25:222
327532993423,6cyclictest13-21ksoftirqd/008:50:240
327544993217,11cyclictest365186-21tr08:55:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional