You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-10 - 15:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Thu Jul 10, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28749913031237,37cyclictest16915-21munin-run08:00:011
28749912341212,19cyclictest0-21swapper/107:25:131
28739911931169,20cyclictest0-21swapper/007:25:100
25042564517,31sleep00-21swapper/007:22:340
24462464418,31sleep10-21swapper/107:22:011
287399254223,26cyclictest16928-21watchsnmpd08:00:010
1567121370,3sleep00-21swapper/010:20:440
144372880,7sleep00-21swapper/012:00:260
305492750,5sleep00-21swapper/012:50:370
91212710,4sleep041ktimersoftd/010:15:040
153732530,2sleep115374-21sh10:55:051
287499516,25cyclictest0-21swapper/112:04:431
26502430,4sleep1211ktimersoftd/109:35:001
2874994228,11cyclictest22776-21ssh11:02:021
2873994221,12cyclictest8842-21ssh09:40:280
2873994217,7cyclictest3-21ksoftirqd/012:09:010
287399414,5cyclictest1739-21head12:20:310
2873994127,4cyclictest25237-21sshd08:20:210
2873994117,8cyclictest3-21ksoftirqd/009:47:560
2873994018,7cyclictest3-21ksoftirqd/011:49:280
2873994017,8cyclictest3-21ksoftirqd/012:41:450
2873994017,8cyclictest3-21ksoftirqd/011:53:280
237932400,5sleep023791-21irqstats12:10:210
2873993915,12cyclictest6203-21packagekitd10:08:460
2873993816,7cyclictest3-21ksoftirqd/012:49:130
2873993816,7cyclictest3-21ksoftirqd/009:20:490
2873993815,7cyclictest3-21ksoftirqd/011:09:450
142952381,33sleep114308-21sh11:26:421
287499377,15cyclictest40950irq/93-i91509:45:171
2873993719,5cyclictest2104-21swap10:40:300
2873993712,15cyclictest3-21ksoftirqd/008:25:070
279092371,32sleep127920-21sh11:07:311
287499369,5cyclictest40950irq/93-i91511:10:151
2873993621,5cyclictest10954-21idleruntime-cro09:10:010
2873993620,5cyclictest8925-21df11:55:130
2873993620,5cyclictest8925-21df11:55:130
2873993620,5cyclictest18040-21/usr/sbin/munin09:50:190
2873993619,6cyclictest31669-21runrttasks11:10:580
2873993617,10cyclictest0-21swapper/010:35:280
2873993614,7cyclictest3-21ksoftirqd/010:46:290
2873993613,12cyclictest7915-21fschecks_time07:35:170
2873993613,12cyclictest26084-21ssh10:31:520
287399361,22cyclictest14812-21ssh10:54:280
318422351,30sleep131885-21ssh11:45:181
287499355,26cyclictest0-21swapper/112:51:371
287499353,6cyclictest0-21swapper/110:03:241
2874993513,18cyclictest0-21swapper/112:19:221
2873993520,5cyclictest25350-21ssh11:38:440
2873993518,5cyclictest0-21swapper/011:42:190
2873993515,5cyclictest0-21swapper/009:17:000
2873993511,9cyclictest28961-21ssh09:28:070
2873993511,13cyclictest11590-21ssh11:24:450
2873993510,7cyclictest2834-21timerwakeupswit08:45:270
287499348,22cyclictest0-21swapper/112:07:191
287499341,15cyclictest15677-21ssh09:14:501
287399349,8cyclictest3-21ksoftirqd/012:36:110
287399349,8cyclictest3-21ksoftirqd/012:36:100
2873993423,6cyclictest29894-21runrttasks08:32:540
2873993419,6cyclictest565-21/usr/sbin/munin08:40:260
2873993416,3cyclictest0-21swapper/007:52:250
2873993413,7cyclictest3-21ksoftirqd/010:59:010
2873993412,7cyclictest3-21ksoftirqd/009:02:540
2873993412,6cyclictest3-21ksoftirqd/011:26:130
2873993410,8cyclictest6388-21ssh11:19:130
62172331,28sleep16231-21ssh10:11:021
2874993321,8cyclictest927-21if_eth012:20:201
2874993321,8cyclictest26750-21ssh10:32:521
2874993319,11cyclictest22-21ksoftirqd/112:48:421
2873993317,5cyclictest31653-21munin-run10:05:010
2873993317,5cyclictest17411-21hddtemp_smartct11:30:180
2873993316,5cyclictest30546-21df08:35:140
2873993316,5cyclictest23331-21iostat08:15:200
2873993313,4cyclictest5896-21iostat12:25:200
2873993313,12cyclictest0-21swapper/010:26:100
2873993312,3cyclictest0-21swapper/009:31:520
2873993312,11cyclictest30205-21ssh12:16:440
2873993311,7cyclictest3-21ksoftirqd/012:31:330
267652331,4sleep12376-21NetworkManager09:59:571
232192331,28sleep123230-21ssh11:35:381
108702331,28sleep10-21swapper/112:30:261
319152321,3sleep131928-21ssh10:38:221
287499327,21cyclictest0-21swapper/112:13:301
2874993216,13cyclictest22-21ksoftirqd/111:17:041
2874993213,15cyclictest22-21ksoftirqd/112:41:401
287499321,15cyclictest21132-21ssh09:53:441
287399329,14cyclictest0-21swapper/007:34:290
287399328,8cyclictest12091-21ntp_states07:45:250
287399328,5cyclictest22886-21ssh11:02:110
2873993216,5cyclictest19277-21df08:05:190
2873993215,6cyclictest3356-21diskstats09:35:160
2873993215,5cyclictest10220-21/usr/sbin/munin07:40:230
2873993213,15cyclictest4568-21wc10:10:130
2873993211,6cyclictest3-21ksoftirqd/008:04:320
287499317,20cyclictest0-21swapper/110:27:351
287499315,20cyclictest0-21swapper/110:17:551
287499313,14cyclictest0-21swapper/109:42:121
2874993120,8cyclictest19206-21cut08:05:171
2874993111,15cyclictest22-21ksoftirqd/109:26:191
2873993118,5cyclictest0-21swapper/008:10:190
2873993115,5cyclictest22950-21iostat09:55:190
2873993115,5cyclictest11913-21irqstats09:10:210
287399311,18cyclictest4483-21processes08:50:280
287499303,23cyclictest0-21swapper/109:18:311
287499302,24cyclictest0-21swapper/111:57:231
287499302,24cyclictest0-21swapper/111:57:231
2874993020,6cyclictest17469-21chrt11:30:201
2874993016,5cyclictest22-21ksoftirqd/110:49:381
287499301,4cyclictest81rcu_preempt11:53:281
2874993013,14cyclictest22-21ksoftirqd/110:42:141
287499301,24cyclictest21854-21ntp_states09:20:251
287399302,16cyclictest0-21swapper/008:55:120
85342291,5sleep10-21swapper/112:28:111
45862291,24sleep14636-21switchtime08:50:301
287499299,16cyclictest0-21swapper/110:50:151
287499292,22cyclictest0-21swapper/109:39:441
287499292,17cyclictest0-21swapper/110:06:071
2874992920,6cyclictest11697-21fschecks_time07:45:161
2874992917,9cyclictest25447-21ntpq08:20:251
2874992916,5cyclictest22-21ksoftirqd/111:24:071
287499291,14cyclictest15199-21df_inode12:35:161
287499291,14cyclictest15199-21df_inode12:35:161
2874992910,14cyclictest14749-21irqstats10:20:211
100142291,5sleep10-21swapper/109:05:251
87822281,5sleep10-21swapper/107:35:391
70152281,5sleep10-21swapper/108:58:271
287499284,18cyclictest7954-21iostat_ios09:00:221
2874992816,9cyclictest22-21ksoftirqd/111:40:081
240342281,5sleep10-21swapper/108:15:431
71752271,5sleep10-21swapper/107:33:541
319222271,5sleep10-21swapper/108:39:271
2874992718,6cyclictest22-21ksoftirqd/108:48:021
283812271,5sleep10-21swapper/108:30:091
278732271,5sleep10-21swapper/108:26:231
287499269,12cyclictest0-21swapper/108:40:271
2874992617,5cyclictest17141-21aten_r4power_en08:00:091
287499242,8cyclictest0-21swapper/108:10:201
287499242,17cyclictest9674-21df_inode07:40:141
2874992314,5cyclictest22-21ksoftirqd/107:50:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional