You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 17:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Sat Nov 29, 2025 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169172236163,23sleep30-21swapper/307:07:233
170222230156,22sleep10-21swapper/107:08:461
170932204166,26sleep00-21swapper/007:09:370
168162202165,25sleep20-21swapper/207:06:052
1942721750,5sleep31723699cyclictest08:25:183
264572640,4sleep226459-21fschecks_count09:55:182
184602620,8sleep011-21rcuc/012:05:010
71312600,2sleep011-21rcuc/009:10:190
321692540,1sleep20-21swapper/211:20:242
75552510,3sleep3401ktimersoftd/308:00:013
36572480,2sleep339-21rcuc/311:30:173
181202420,2sleep00-21swapper/009:35:200
17229993834,3cyclictest0-21swapper/212:16:192
1721899338,12cyclictest16038-21idleruntime-cro12:00:000
1721899338,12cyclictest16038-21idleruntime-cro12:00:000
1721899328,11cyclictest5996-21awk09:10:010
1722999283,11cyclictest25077-40dnf08:40:022
1721899287,12cyclictest25099-21munin-run08:40:000
1721899287,12cyclictest25099-21munin-run08:40:000
113672280,3sleep29-21rcu_preempt09:20:172
1721899279,15cyclictest13861-21idleruntime-cro11:55:000
1722499261,20cyclictest0-21swapper/111:27:301
17224992611,13cyclictest28584-21kworker/u8:211:41:331
1721899261,12cyclictest8812-21munin-run10:30:000
1721899260,18cyclictest0-21swapper/011:48:290
1722499251,18cyclictest0-21swapper/112:28:411
1722499251,17cyclictest836-21runrttasks07:49:041
1721899251,5cyclictest0-21swapper/011:24:480
263572241,3sleep126391-21ntp_states08:40:231
17236992417,4cyclictest28159-21cut08:45:163
1723699241,11cyclictest13435-21/usr/sbin/munin10:40:143
17229992411,4cyclictest0-21swapper/209:35:152
1722499242,19cyclictest27712-21irqstats07:30:181
1722499242,19cyclictest27712-21irqstats07:30:181
17224992414,3cyclictest0-21swapper/109:00:291
1722499241,17cyclictest0-21swapper/108:07:091
1722499241,15cyclictest0-21swapper/109:40:041
1722499240,17cyclictest0-21swapper/108:26:231
1721899241,12cyclictest13221-21awk10:40:000
8512231,2sleep3891-21latency_hist07:45:003
1723699237,7cyclictest0-21swapper/309:30:273
1723699233,11cyclictest16273-21/usr/sbin/munin12:00:293
17236992315,5cyclictest25288-21latency_hist08:40:013
1722999233,5cyclictest9083-40dnf11:40:332
1722499231,16cyclictest0-21swapper/109:59:491
1722499231,15cyclictest0-21swapper/107:42:271
1721899237,3cyclictest6158-21tail10:20:270
1721899232,18cyclictest3077-21/usr/sbin/munin11:30:190
1721899231,11cyclictest24486-21awk07:25:010
1721899231,11cyclictest14634-21snmpd07:48:300
119762231,3sleep112048-21irqstats10:35:201
267082220,5sleep011-21rcuc/011:10:130
225272225,2sleep222689-21cstates07:20:152
17236992215,4cyclictest31646-21head11:20:173
1723699221,2cyclictest0-21swapper/311:40:333
1722999221,4cyclictest27722-21kworker/u8:010:11:232
1722499229,10cyclictest19365-21cat07:14:561
1722499221,18cyclictest18487-21/usr/sbin/munin12:05:221
1722499221,18cyclictest13563-21timerwakeupswit08:10:281
1722499221,18cyclictest10213-21ps10:30:241
1722499221,16cyclictest0-21swapper/107:25:361
1722499221,15cyclictest0-21swapper/110:10:301
1722499221,15cyclictest0-21swapper/109:50:341
1722499221,15cyclictest0-21swapper/108:55:321
1722499221,15cyclictest0-21swapper/108:32:181
1722499220,16cyclictest0-21swapper/110:04:111
1721899221,7cyclictest4323-21needreboot07:50:200
256632210,1sleep00-21swapper/011:05:250
231202211,2sleep10-21swapper/108:35:131
231202211,2sleep10-21swapper/108:35:131
193022211,3sleep019428-21users10:50:270
193022211,3sleep019428-21users10:50:270
182792210,3sleep30-21swapper/309:35:243
177882210,3sleep117792-21gltestperf09:35:181
1723699218,4cyclictest22883-21/usr/sbin/munin12:15:233
1723699212,17cyclictest15839-21sed08:15:263
17236992115,3cyclictest10723-21kworker/u8:007:19:223
1723699211,11cyclictest28621-21fschecks_count10:00:163
1723699211,11cyclictest14634-21snmpd09:02:593
1722999216,6cyclictest179842grep10:50:142
1722999216,6cyclictest179842grep10:50:142
17229992114,4cyclictest3412-21df11:30:142
17229992112,2cyclictest86850irq/129-enp1s0-09:10:212
1722499219,3cyclictest30808-21latency_hist11:20:021
1722499218,4cyclictest0-21swapper/109:25:121
17224992114,2cyclictest0-21swapper/107:36:411
17224992113,3cyclictest0-21swapper/109:06:431
1722499211,18cyclictest0-21swapper/110:46:111
1722499211,18cyclictest0-21swapper/110:15:201
1722499211,15cyclictest0-21swapper/110:20:471
1722499211,15cyclictest0-21swapper/109:48:021
1722499211,15cyclictest0-21swapper/108:49:191
1722499211,15cyclictest0-21swapper/108:22:431
1722499211,15cyclictest0-21swapper/108:16:021
1722499211,11cyclictest17350-21ntp_states12:00:241
17224992110,3cyclictest13841-21latency_hist11:55:021
17224992110,3cyclictest10942-21smart_sda11:45:281
1721899216,8cyclictest0-21swapper/008:59:290
1721899214,6cyclictest14199-21mii-tool10:40:180
1721899213,8cyclictest9934-21/usr/sbin/munin08:05:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional