You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-27 - 08:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Mon Oct 27, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27169210375,9sleep10-21swapper/119:09:041
27099210389,9sleep30-21swapper/319:08:103
2595128666,14sleep20-21swapper/219:05:192
2718328166,10sleep00-21swapper/019:09:150
266472680,0sleep00-21swapper/023:05:440
2733099320,0cyclictest0-21swapper/123:35:291
27328992624,1cyclictest13-21ksoftirqd/022:40:220
2733899240,1cyclictest0-21swapper/300:03:523
2733499240,24cyclictest0-21swapper/222:34:222
2733899231,2cyclictest0-21swapper/300:25:093
2733099230,23cyclictest0-21swapper/122:43:231
27334992221,1cyclictest15540-21kworker/2:221:26:592
2733899211,2cyclictest32472-21cron00:20:003
2732899211,16cyclictest13-21ksoftirqd/000:10:230
2732899210,1cyclictest0-21swapper/019:25:240
123572210,0sleep30-21swapper/323:19:043
2733499200,20cyclictest0-21swapper/200:24:522
2733499200,20cyclictest0-21swapper/200:24:522
2733099200,3cyclictest141rcu_preempt23:28:361
2733099200,20cyclictest0-21swapper/122:09:541
2733899190,1cyclictest0-21swapper/319:35:143
2733899190,16cyclictest0-21swapper/322:37:083
2733499195,11cyclictest26651-21/usr/sbin/munin00:15:142
2733099190,1cyclictest0-21swapper/122:01:461
2733899184,9cyclictest27158-21cron22:00:003
27334991818,0cyclictest0-21swapper/222:36:002
2733499180,18cyclictest0-21swapper/221:31:242
2733499180,17cyclictest0-21swapper/222:02:432
2732899187,6cyclictest13406-21sendmail-msp23:20:010
27328991813,2cyclictest13-21ksoftirqd/022:54:400
27328991812,3cyclictest13-21ksoftirqd/023:22:550
2732899180,17cyclictest0-21swapper/021:20:130
27338991716,1cyclictest0-21swapper/322:03:083
2733499175,11cyclictest0-21swapper/222:41:512
27334991717,0cyclictest0-21swapper/200:13:422
2733499170,0cyclictest0-21swapper/222:18:292
2732899174,7cyclictest23955-21tr22:20:000
2732899170,16cyclictest0-21swapper/021:38:230
2733899160,16cyclictest0-21swapper/321:17:403
2733499163,4cyclictest224-21systemd-journal22:05:002
2733499160,16cyclictest0-21swapper/223:51:252
2733099163,7cyclictest224-21systemd-journal20:45:001
2733099162,10cyclictest13672-21ssh22:35:141
2733099160,0cyclictest0-21swapper/100:34:271
2732899165,2cyclictest141rcu_preempt23:55:030
2732899164,6cyclictest10580-21apt-get20:50:000
2732899164,3cyclictest13-21ksoftirqd/021:47:330
2732899164,2cyclictest13-21ksoftirqd/022:20:180
27328991615,1cyclictest0-21swapper/023:40:260
27328991612,1cyclictest13-21ksoftirqd/023:30:320
27328991610,3cyclictest5825-21cron23:15:010
2733899153,1cyclictest141rcu_preempt22:44:323
27338991511,2cyclictest16990-21grep21:30:133
2733899150,14cyclictest0-21swapper/321:53:083
27334991514,1cyclictest5101-21kworker/2:021:24:192
27334991514,1cyclictest2509-21kworker/2:022:47:392
2733499150,2cyclictest0-21swapper/223:00:592
2732899159,5cyclictest0-21swapper/022:10:000
27328991514,1cyclictest13-21ksoftirqd/000:23:300
27328991514,1cyclictest13-21ksoftirqd/000:23:300
27328991511,2cyclictest13-21ksoftirqd/023:04:560
27328991511,1cyclictest161rcuc/023:47:410
27328991510,4cyclictest496-21dbus-daemon00:07:230
2733899148,3cyclictest30444-21tr19:15:153
2733899141,8cyclictest23262-21kworker/u8:222:19:553
27338991410,3cyclictest14968-21wc23:20:203
27338991410,2cyclictest18871-21grep00:35:173
27338991410,2cyclictest10433-21sed21:25:183
2733899140,14cyclictest0-21swapper/321:47:043
2733499142,3cyclictest224-21systemd-journal23:40:002
27334991412,2cyclictest0-21swapper/221:37:012
27334991411,2cyclictest849-21wc23:55:232
27334991411,2cyclictest18474-21grep19:55:282
27334991410,3cyclictest389-21grep20:25:302
27334991410,3cyclictest12358-21tail23:40:282
2733099142,5cyclictest30881-21cron20:25:001
2732899149,5cyclictest13-21ksoftirqd/021:51:380
2732899149,3cyclictest27180-21sh22:00:000
2732899144,4cyclictest13-21ksoftirqd/022:46:560
2732899142,6cyclictest13-21ksoftirqd/021:33:480
27328991413,1cyclictest161rcuc/021:40:160
27328991411,2cyclictest13-21ksoftirqd/021:00:000
27328991410,3cyclictest3212-21cron20:34:590
27328991410,3cyclictest20661-21wc20:00:270
2732899140,9cyclictest0-21swapper/019:13:010
2732899140,4cyclictest0-21swapper/023:51:490
2732899140,14cyclictest13-21ksoftirqd/000:18:050
2733899139,2cyclictest31920-21cut23:55:153
2733899131,3cyclictest18974-21munin-run20:00:003
27338991313,0cyclictest0-21swapper/323:07:353
27338991312,1cyclictest0-21swapper/300:30:583
27338991310,2cyclictest32492-21grep22:25:183
27338991310,2cyclictest31869-21tr23:10:163
27338991310,2cyclictest30883-21sh20:25:003
27338991310,2cyclictest28611-21tr23:30:163
27338991310,2cyclictest20781-21cut00:10:183
27338991310,2cyclictest10224-21grep22:55:153
2733899130,12cyclictest2588-21ls21:20:143
2733499139,3cyclictest8536-21grep00:00:252
2733499139,3cyclictest25128-21cut22:20:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional