You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 07:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Mon Dec 01, 2025 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275662234160,60sleep00-21swapper/019:05:400
275892233164,57sleep30-21swapper/319:05:593
276492203164,25sleep10-21swapper/119:06:461
259402203166,24sleep20-21swapper/219:05:062
445921300,3sleep02799599cyclictest20:40:180
110002560,1sleep10-21swapper/119:40:261
235892550,4sleep341-21ksoftirqd/323:50:213
57952520,4sleep339-21rcuc/321:55:283
61012490,1sleep10-21swapper/119:30:181
28006994744,2cyclictest0-21swapper/220:24:292
2800299371,24cyclictest12256-21diskstats19:45:131
2801199321,23cyclictest0-21swapper/323:15:143
2800299321,6cyclictest0-21swapper/121:59:011
2801199302,14cyclictest26587-40dnf23:57:343
28011992923,4cyclictest0-21swapper/322:25:223
28011992923,4cyclictest0-21swapper/322:25:223
28002992910,4cyclictest28882-21systemd-journal00:03:221
2799599291,13cyclictest1320-21awk20:35:010
144442290,4sleep121-21rcuc/123:30:161
144442290,4sleep121-21rcuc/123:30:161
28011992821,4cyclictest15183-21ntp_states19:50:243
2801199281,14cyclictest26626-21grep20:15:263
2801199281,14cyclictest26626-21grep20:15:263
267272281,23sleep026738-21/usr/sbin/munin22:45:160
28002992711,4cyclictest4317-21sed23:05:281
2799599272,13cyclictest8285-21munin-run00:35:010
27995992710,14cyclictest1977-21idleruntime-cro21:50:010
96262260,2chrt0-21swapper/119:40:021
28011992611,3cyclictest22327-21df22:35:163
2800699266,10cyclictest0-21swapper/220:06:162
2800699263,16cyclictest29708-21kworker/2:121:50:272
28002992620,3cyclictest12016-40dnf20:56:231
2800299261,15cyclictest0-21swapper/123:50:051
2800299261,11cyclictest0-21swapper/123:38:311
2800299261,11cyclictest0-21swapper/100:34:261
2800299260,17cyclictest0-21swapper/122:35:271
2801199251,17cyclictest0-21swapper/321:45:183
2801199251,17cyclictest0-21swapper/321:27:023
2801199251,15cyclictest0-21swapper/322:00:203
2801199251,13cyclictest1462-21runrttasks00:17:363
2801199251,12cyclictest0-21swapper/322:30:153
2800699250,5cyclictest0-21swapper/221:25:012
2800699250,5cyclictest0-21swapper/221:25:012
28002992519,3cyclictest6938-21diskstats22:00:141
2801199249,3cyclictest32053-21smart_sda22:55:253
2801199242,11cyclictest39-21rcuc/300:10:223
28011992419,3cyclictest14634-21snmpd19:49:273
28011992418,3cyclictest28595-21/usr/sbin/munin22:50:263
2801199241,12cyclictest0-21swapper/300:35:063
2801199241,11cyclictest21942-21gunzip20:05:233
2801199241,11cyclictest15739-21diskstats22:20:143
2801199241,10cyclictest21282-21taskset21:20:013
2800699243,12cyclictest0-21swapper/219:17:432
2800299242,12cyclictest30163-21munin-run19:15:011
28002992418,3cyclictest28228-21latency20:20:211
28002992415,5cyclictest0-21swapper/121:26:341
2800299241,20cyclictest19142-21memory23:40:181
2800299241,17cyclictest0-21swapper/123:15:141
2800299241,17cyclictest0-21swapper/122:57:361
2800299241,17cyclictest0-21swapper/121:21:211
2800299241,17cyclictest0-21swapper/121:21:211
2800299241,14cyclictest0-21swapper/100:11:241
2800299241,11cyclictest10763-21latency_hist22:10:001
2800299241,10cyclictest0-21swapper/121:15:401
2800299240,16cyclictest86750irq/128-enp1s0-22:26:361
2800299240,16cyclictest86750irq/128-enp1s0-22:26:361
2799599241,5cyclictest0-21swapper/022:55:200
2799599241,5cyclictest0-21swapper/022:05:260
2801199239,3cyclictest6648-21df_inode00:30:163
2801199239,3cyclictest27197-21irqstats22:45:193
2801199239,3cyclictest10630-21memory19:40:203
2801199238,3cyclictest9608-21irqstats22:05:193
2801199231,9cyclictest0-21swapper/319:18:093
28011992319,3cyclictest0-21swapper/323:35:213
28011992319,2cyclictest28198-21/usr/sbin/munin19:10:203
28011992319,2cyclictest16729-21diskmemload23:30:173
28011992319,2cyclictest16729-21diskmemload23:30:173
2801199231,5cyclictest30298-21cpuspeed_turbos21:40:133
2801199231,13cyclictest0-21swapper/322:18:193
2801199231,12cyclictest0-21swapper/300:02:353
2801199231,11cyclictest11963-21df23:25:143
2801199231,10cyclictest3649-21irqstats23:05:203
2801199231,10cyclictest2514-21gltestperf00:20:153
2801199231,10cyclictest0-21swapper/323:20:163
2801199231,10cyclictest0-21swapper/321:54:133
2800699237,9cyclictest0-21swapper/221:45:462
2800299238,3cyclictest26723-21df22:45:151
28002992320,2cyclictest0-21swapper/123:46:431
28002992320,2cyclictest0-21swapper/120:45:551
28002992319,3cyclictest0-21swapper/122:41:501
28002992319,3cyclictest0-21swapper/120:04:271
28002992318,2cyclictest5185-21cat19:30:011
28002992317,3cyclictest2543-21/usr/sbin/munin21:50:151
2800299231,16cyclictest0-21swapper/121:30:231
2800299231,12cyclictest13111-21unixbench-2d23:25:261
2800299231,11cyclictest0-21swapper/121:39:111
2800299231,10cyclictest603-21strings00:15:161
2800299231,10cyclictest15821-21ntp_states21:05:231
2800299231,10cyclictest15554-21cpuspeed_turbos22:20:121
2800299230,10cyclictest12961-21sh22:15:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional