You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-23 - 04:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Thu Oct 23, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9893279918401,1832cyclictest14432932sleep300:30:243
9893279915871,1581cyclictest11887922sleep321:54:563
9893279915701,1564cyclictest14469602sleep300:35:013
9893279915580,1553cyclictest12360242sleep322:23:423
9893279915541,1548cyclictest13344282sleep323:25:073
9893279915510,1546cyclictest13481962sleep323:31:483
9893279915331,1527cyclictest11647902sleep321:39:573
9893279915270,1522cyclictest11652492sleep321:40:133
9893279915031,1497cyclictest11301852sleep321:16:483
9893279914841,1478cyclictest12847122sleep322:53:483
9893279914771,1471cyclictest10637862sleep320:15:483
9893279914651,1459cyclictest11471782sleep321:27:423
9893279914411,1435cyclictest13743672sleep323:49:483
9893279914351,1429cyclictest11786682sleep321:46:423
9893279914281,1422cyclictest10383072sleep319:54:563
9893279914151,1409cyclictest11056332sleep320:59:573
9893279914021,1397cyclictest10173392sleep319:33:483
9893279913281,1322cyclictest10844852sleep320:36:483
9893279913121,1307cyclictest9942742sleep319:12:483
989327994091,404cyclictest1196834-21turbostat22:04:553
988950213055,70sleep30-21swapper/319:06:593
117416621200,3sleep2421rcuc/221:45:122
103273621160,3sleep134-21ksoftirqd/119:45:281
98915028757,10sleep20-21swapper/219:09:352
98412828555,10sleep10-21swapper/119:05:011
98893727754,17sleep00-21swapper/019:06:490
989327992219,1cyclictest0-21swapper/320:10:003
13144072221,7sleep01314418-21/usr/sbin/munin23:10:300
98932799210,17cyclictest1415222-21taskset00:14:533
98932799201,16cyclictest1058572-21chrt20:10:353
98932799201,15cyclictest1333182-21taskset23:23:593
98932799201,15cyclictest0-21swapper/322:16:023
98932799200,1cyclictest0-21swapper/322:09:573
12011912201,4sleep01201234-21ntpq22:00:180
98932799194,12cyclictest0-21swapper/300:05:203
989327991916,1cyclictest0-21swapper/323:35:183
98932799191,14cyclictest0-21swapper/323:15:183
989327991910,6cyclictest1380481-21chrt23:51:443
98932799188,6cyclictest1001662-21turbostat19:20:013
989327991815,1cyclictest0-21swapper/323:55:193
989327991812,3cyclictest0-21swapper/300:28:093
98932799181,14cyclictest0-21swapper/323:42:173
98932799181,14cyclictest0-21swapper/322:37:183
98932799181,14cyclictest0-21swapper/321:56:233
98932799181,13cyclictest0-21swapper/322:40:143
98932799181,13cyclictest0-21swapper/321:30:383
98932799180,15cyclictest1048637-21sleep20:05:003
11562722180,1sleep00-21swapper/021:34:220
98932799174,10cyclictest0-21swapper/300:15:183
989327991714,1cyclictest0-21swapper/321:00:173
989327991714,1cyclictest0-21swapper/300:20:153
989327991713,2cyclictest0-21swapper/300:00:183
98932799171,14cyclictest0-21swapper/321:21:173
98932799171,13cyclictest0-21swapper/323:00:283
98932799171,13cyclictest0-21swapper/322:45:023
98932799171,13cyclictest0-21swapper/322:25:113
98932799171,13cyclictest0-21swapper/321:10:063
98932799171,12cyclictest0-21swapper/322:55:063
12024522171,3sleep11115918-21diskmemload22:01:161
98932799168,4cyclictest1095251-21taskset20:48:503
98932799162,10cyclictest1069599-21chrt20:25:133
989327991613,1cyclictest0-21swapper/321:10:003
98932799161,2cyclictest0-21swapper/322:34:033
98932799161,11cyclictest0-21swapper/323:10:083
98932799160,14cyclictest0-21swapper/323:08:323
98932799160,13cyclictest0-21swapper/322:10:153
98932799160,12cyclictest57-1kworker/3:0H-kblockd19:25:363
12745672160,11sleep01274582-21ssh22:45:530
12678492161,4sleep21267863-21ssh22:43:222
98932799151,11cyclictest0-21swapper/319:55:133
98932799150,13cyclictest0-21swapper/320:51:233
14431162151,9sleep11443239-21collect200:30:231
13557562151,9sleep21355770-21ssh23:36:132
98932799141,9cyclictest1002175-21taskset19:20:013
989327991411,2cyclictest0-21swapper/319:45:133
989327991411,1cyclictest0-21swapper/320:20:123
98932799140,11cyclictest0-21swapper/320:40:193
98932799140,11cyclictest0-21swapper/319:40:203
13885172141,9sleep01388527-21ssh23:56:310
13826762141,8sleep11382738-21munin-run23:55:001
13644792141,9sleep21115918-21diskmemload23:42:062
12286512141,3sleep01228662-21ssh22:19:470
10749792141,9sleep01075153-21cat20:30:170
10648632141,3sleep21064899-21kthreadcore20:20:172
10078162141,5sleep21009319-21cut19:25:142
98932799131,8cyclictest1078695-21taskset20:30:233
98932799130,1cyclictest0-21swapper/319:35:203
13385202131,7sleep01338574-21ntpq23:25:210
12840082131,8sleep21284019-21ssh22:52:432
11951362131,3sleep21195146-21ssh21:57:242
11805562131,8sleep01180567-21ssh21:49:370
11573052131,4sleep11157434-21gltestperf21:35:151
11483622131,5sleep01115918-21diskmemload21:29:330
11246952131,7sleep11124880-21head21:15:151
14066542121,7sleep21115918-21diskmemload00:09:122
13816812121,3sleep00-21swapper/023:53:310
13343782120,6sleep21333848-21apt-get23:25:032
13338312121,7sleep01333975-21date23:25:010
13081812121,7sleep21308195-21ssh23:07:432
12676982122,6sleep11115918-21diskmemload22:43:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional