You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 14:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Sat Jul 12, 2025 00:43:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1268429782,10sleep10-21swapper/119:08:561
1246829582,8sleep30-21swapper/319:06:093
1261129265,8sleep00-21swapper/019:07:590
1252428070,6sleep20-21swapper/219:06:522
275232700,0sleep00-21swapper/019:40:000
154662540,0sleep10-21swapper/121:39:171
12845993332,1cyclictest0-21swapper/123:42:491
12845992927,1cyclictest0-21swapper/121:48:291
12854992726,1cyclictest0-21swapper/323:04:093
12845992726,1cyclictest0-21swapper/123:14:091
12845992523,1cyclictest0-21swapper/123:27:491
1285499242,21cyclictest25056-21latency_hist19:35:003
12845992322,1cyclictest0-21swapper/123:38:291
1285499224,16cyclictest19862-21grep22:50:173
12845992120,1cyclictest0-21swapper/123:30:491
1284099210,1cyclictest0-21swapper/022:37:510
1284099210,1cyclictest0-21swapper/022:37:510
1284099202,14cyclictest20347-21/usr/sbin/munin21:20:250
1284099200,19cyclictest0-21swapper/000:36:580
1284099200,0cyclictest0-21swapper/021:42:110
12854991917,1cyclictest17603-21ssh23:10:443
1285099191,11cyclictest0-21swapper/222:25:122
1284599196,3cyclictest224-21systemd-journal21:20:011
1284599190,2cyclictest0-21swapper/100:27:561
12850991818,0cyclictest0-21swapper/221:48:152
1285099180,18cyclictest0-21swapper/223:12:532
12840991814,3cyclictest13-21ksoftirqd/023:06:470
1285499170,13cyclictest32387-21apache_accesses00:35:133
1285099172,6cyclictest0-21swapper/223:30:002
1285099170,1cyclictest0-21swapper/221:42:562
12840991713,3cyclictest26727-21ssh22:09:540
77192160,0sleep10-21swapper/121:55:321
1285499169,6cyclictest9162-21cron20:10:003
1285499160,15cyclictest0-21swapper/322:55:143
12850991613,3cyclictest0-21swapper/223:30:382
1285099160,16cyclictest0-21swapper/221:13:142
1285099160,12cyclictest0-21swapper/222:30:082
12840991612,3cyclictest25312sleep023:23:400
1284099160,15cyclictest0-21swapper/022:29:320
1285099151,1cyclictest0-21swapper/222:07:412
1284599150,4cyclictest0-21swapper/121:20:211
1284099153,6cyclictest636-21grep21:00:000
12840991515,0cyclictest0-21swapper/021:13:380
112162150,1sleep00-21swapper/021:35:290
1285499141,2cyclictest0-21swapper/323:44:123
1285499140,13cyclictest0-21swapper/321:38:293
12850991414,0cyclictest0-21swapper/220:35:142
12850991412,1cyclictest4548-21ssh00:10:292
12850991412,1cyclictest32-21ksoftirqd/221:37:022
12850991411,2cyclictest37850irq/131-snd_hda22:21:552
12850991410,3cyclictest32047-21sort21:50:222
12850991410,2cyclictest15894-21sh21:16:232
1285099140,14cyclictest0-21swapper/200:21:062
1285099140,14cyclictest0-21swapper/200:21:062
1285099140,0cyclictest0-21swapper/222:02:492
1284599149,3cyclictest20951-21tail19:25:181
12845991410,3cyclictest581-21cron21:00:001
12845991410,3cyclictest20690-21ssh22:50:221
12845991410,2cyclictest8978-21cut23:05:161
12845991410,2cyclictest26663-21grep22:55:151
1284599140,13cyclictest1789-21grep00:35:281
1284599140,13cyclictest0-21swapper/119:45:171
1284099140,1cyclictest0-21swapper/019:40:170
1284099140,13cyclictest27604-21wc23:40:250
1284099140,12cyclictest28449-21tr00:05:140
1285499139,3cyclictest691-21grep19:50:183
12854991310,2cyclictest7374-21grep22:40:303
12854991310,2cyclictest5868-21grep21:10:143
12854991310,2cyclictest27878-21munin-plugin-st00:05:003
12854991310,2cyclictest23024-21sed20:35:273
12854991310,2cyclictest19722-21wc22:05:003
1285499130,11cyclictest23029-21sed00:25:273
1285099138,4cyclictest19768-21head23:35:222
1285099137,2cyclictest17763-21uname20:25:202
1285099133,6cyclictest1549-21idleruntime-cro22:15:002
12850991311,2cyclictest32-21ksoftirqd/200:15:152
12850991310,2cyclictest32-21ksoftirqd/221:25:252
12850991310,2cyclictest23900-21grep20:40:162
12850991310,2cyclictest21129-21sh20:35:002
12850991310,1cyclictest32-21ksoftirqd/223:09:162
1284599139,3cyclictest2552-21sed21:00:281
1284599138,3cyclictest16722-21grep20:25:141
1284599138,3cyclictest12971-21tail20:15:191
1284599134,3cyclictest29159-21turbostat23:20:001
1284599133,4cyclictest1552-21dump-pmu-power22:15:001
1284599132,2cyclictest224-21systemd-journal22:01:121
12845991313,0cyclictest0-21swapper/122:36:571
12845991313,0cyclictest0-21swapper/122:36:571
12845991311,2cyclictest0-21swapper/119:50:151
12845991310,2cyclictest6612-21tail21:10:181
12845991310,2cyclictest4124-21wc23:25:001
12845991310,2cyclictest24433-21sed19:30:261
12845991310,2cyclictest18998-21tr19:20:191
1284099139,3cyclictest7131-21rm22:40:290
1284099139,3cyclictest642-21grep23:45:140
1284099139,2cyclictest6449-21sh21:55:210
1284099139,2cyclictest17999-21cut19:20:130
1284099134,8cyclictest11726-21packagekitd22:47:030
1284099134,1cyclictest13005-21apt-get22:00:150
1284099132,8cyclictest1419-21apache222:34:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional