You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 06:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Thu Nov 27, 2025 00:43:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281139022240,2sleep02811392-21find23:40:210
2439809212860,64sleep30-21swapper/319:09:303
25947312880,9sleep00-21swapper/021:30:150
243951628065,9sleep20-21swapper/219:05:492
243948928060,13sleep00-21swapper/019:05:270
243961527661,10sleep10-21swapper/119:07:011
2439979993228,2cyclictest0-21swapper/300:15:263
2439979993127,1cyclictest0-21swapper/321:35:013
27980582300,2sleep12798057-21ssh23:34:281
243997999291,23cyclictest2589696-21chrt21:25:373
2439979992824,2cyclictest0-21swapper/323:25:193
2439979992724,1cyclictest0-21swapper/321:10:003
2439979992613,9cyclictest2765382-21sleep23:15:003
24847142251,10sleep02484902-21expr19:50:110
2439979992522,1cyclictest0-21swapper/300:25:143
243997999251,20cyclictest2580717-21taskset21:20:253
2439979992420,2cyclictest0-21swapper/321:55:113
2439979992419,3cyclictest0-21swapper/323:40:133
2439979992411,9cyclictest2824125-21taskset23:50:173
27087222230,2sleep30-21swapper/322:40:103
25809852230,3sleep02559572-21diskmemload21:20:250
243997999232,16cyclictest2609620-21chrt21:40:133
2439979992319,2cyclictest0-21swapper/323:55:183
243997999231,17cyclictest2819466-21chrt23:45:213
243997999231,12cyclictest2655508-21chrt22:05:353
28642152221,15sleep12864422-21expr00:15:151
28394402220,9sleep134-21ksoftirqd/123:59:461
26424042221,15sleep22642418-21apt-get22:00:142
243997999222,14cyclictest2886736-21taskset00:30:103
243997999222,14cyclictest2665613-21taskset22:13:283
243997999222,14cyclictest2633893-21taskset21:55:003
243997999222,12cyclictest2683645-21taskset22:25:073
2439979992219,1cyclictest0-21swapper/321:40:003
2439979992219,1cyclictest0-21swapper/319:20:243
2439979992218,2cyclictest0-21swapper/323:05:153
2439979992218,2cyclictest0-21swapper/323:05:153
2439979992218,2cyclictest0-21swapper/300:35:253
2439979992217,3cyclictest0-21swapper/322:22:113
243997999221,14cyclictest2648203-21taskset22:01:543
243997999212,13cyclictest2794108-21taskset23:30:183
243997999212,13cyclictest2697378-21taskset22:31:243
2439979992118,2cyclictest0-21swapper/322:55:003
2439979992118,1cyclictest0-21swapper/300:10:003
2439979992117,2cyclictest0-21swapper/322:39:563
243997999211,15cyclictest2721135-21chrt22:45:243
27828022201,8sleep02784110-21kthreadcore23:25:190
243997999202,6cyclictest26750252chrt22:19:543
2439979992016,2cyclictest0-21swapper/323:01:333
2439979992016,2cyclictest0-21swapper/321:15:133
2439979992016,2cyclictest0-21swapper/321:15:123
243997999201,3cyclictest0-21swapper/319:52:093
2439979992012,4cyclictest28739362sleep300:20:183
243997999201,15cyclictest2781079-21chrt23:23:563
243997999201,14cyclictest2617475-21chrt21:45:093
243997999201,14cyclictest2565402-21chrt21:11:303
28704342191,11sleep02559572-21diskmemload00:18:180
26547672191,5sleep12654823-21ssh22:05:211
24624422191,13sleep22462503-21/usr/sbin/munin19:25:212
243997999192,12cyclictest2770945-21taskset23:16:023
243997999192,11cyclictest2799385-21taskset23:35:153
2439979991916,1cyclictest0-21swapper/320:05:163
2439979991916,1cyclictest0-21swapper/319:55:183
2439979991915,2cyclictest0-21swapper/320:45:163
2439979991915,2cyclictest0-21swapper/320:34:593
243997999191,14cyclictest0-21swapper/322:55:063
243997999191,10cyclictest2511088-21taskset20:15:073
243997999191,10cyclictest2511088-21taskset20:15:063
28835152181,5sleep12883563-21ntpq00:25:211
28788432181,5sleep02878851-21ssh00:24:280
28217502180,2sleep20-21swapper/223:47:582
27380962181,2sleep00-21swapper/022:55:510
26804042181,3sleep00-21swapper/022:20:320
25762042182,11sleep12576284-21cpuspeed_turbos21:20:121
243997999188,7cyclictest2521232-21chrt20:23:083
243997999182,11cyclictest2839977-21taskset00:00:083
2439979991815,1cyclictest0-21swapper/300:10:243
243997999181,10cyclictest2483276-21taskset19:45:163
27328672171,4sleep12559572-21diskmemload22:55:041
27161842171,4sleep02559572-21diskmemload22:44:410
27156882171,5sleep22559572-21diskmemload22:43:552
26976612171,3sleep00-21swapper/022:31:510
25978462171,2sleep20-21swapper/221:30:392
25915012171,4sleep22559572-21diskmemload21:28:262
25906992170,2sleep12559572-21diskmemload21:27:091
2439979991714,2cyclictest0-21swapper/320:50:113
2439979991713,2cyclictest0-21swapper/319:15:213
2439979991711,3cyclictest2527282-21turbostat20:39:553
243997999171,13cyclictest0-21swapper/319:40:203
243997999171,10cyclictest2549083-21taskset21:00:003
28855892160,6sleep02559572-21diskmemload00:28:210
28507812161,10sleep22852205-21/usr/sbin/munin00:05:182
28220122161,3sleep10-21swapper/123:48:211
28111372162,7sleep12811190-21ntpq23:40:181
27802372160,8sleep00-21swapper/023:22:420
27711872161,10sleep22771195-21rm23:16:272
26656252161,9sleep02559572-21diskmemload22:13:300
261340321610,3sleep0221rcuc/021:40:180
26065332161,4sleep22559572-21diskmemload21:36:232
26049792161,3sleep00-21swapper/021:35:180
25803982161,10sleep22580458-21ntpq21:20:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional