You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 11:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Mon Nov 17, 2025 00:43:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4813210980,10sleep10-21swapper/119:08:201
4689210561,39sleep30-21swapper/319:06:453
480229075,10sleep00-21swapper/019:08:110
400228565,15sleep20-21swapper/219:05:242
503499330,1cyclictest0-21swapper/322:51:093
503499330,1cyclictest0-21swapper/322:47:093
503499330,1cyclictest0-21swapper/321:47:493
503099330,3cyclictest23562-21fschecks_time00:15:192
503499320,32cyclictest0-21swapper/322:20:493
503499320,1cyclictest0-21swapper/323:44:093
503499320,1cyclictest0-21swapper/321:51:093
503499320,1cyclictest0-21swapper/300:01:293
5034992725,1cyclictest0-21swapper/319:50:153
503099272,1cyclictest0-21swapper/221:40:192
503099241,2cyclictest0-21swapper/221:55:202
503099232,3cyclictest12078-21awk23:20:002
503099220,1cyclictest0-21swapper/223:37:232
502299225,10cyclictest224-21systemd-journal20:35:000
503099215,8cyclictest37850irq/131-snd_hda21:25:172
5028992113,5cyclictest224-21systemd-journal00:23:411
5028992113,5cyclictest224-21systemd-journal00:23:411
313222210,1sleep30-21swapper/321:10:183
503499202,16cyclictest0-21swapper/321:40:153
503499200,1cyclictest0-21swapper/321:58:493
503099202,3cyclictest29471-21latency_hist22:45:002
502299205,12cyclictest27910-21python23:30:300
503499193,15cyclictest3453-21kworker/u8:023:27:363
503499191,16cyclictest0-21swapper/300:21:083
503499191,16cyclictest0-21swapper/300:21:083
503099191,6cyclictest37850irq/131-snd_hda23:27:282
503099190,19cyclictest0-21swapper/222:34:232
502899190,16cyclictest0-21swapper/122:00:161
503499183,6cyclictest4509-21cat00:25:163
503099182,1cyclictest19495-21ntp_offset23:00:232
503099180,18cyclictest0-21swapper/222:00:532
503099179,7cyclictest27351-21fgrep21:30:242
503099178,1cyclictest6225-21grep22:50:282
503099175,2cyclictest224-21systemd-journal21:55:002
503099173,4cyclictest224-21systemd-journal23:55:002
503099172,1cyclictest0-21swapper/219:35:232
5030991714,3cyclictest0-21swapper/200:22:472
5030991714,3cyclictest0-21swapper/200:22:472
503099170,3cyclictest0-21swapper/221:35:222
503099170,0cyclictest0-21swapper/223:07:312
5028991715,2cyclictest25-21ksoftirqd/122:12:541
502299179,5cyclictest20220-21tune2fs23:25:180
502299171,3cyclictest141rcu_preempt22:49:500
503499168,7cyclictest20679-21munin-run23:50:003
5034991614,1cyclictest0-21swapper/300:17:203
503099168,2cyclictest9918-21rm21:18:062
503099168,2cyclictest8479-21expr22:05:182
503099168,2cyclictest4901-21ssh23:14:362
503099168,2cyclictest19248-21ssh00:11:422
503099168,2cyclictest14138-21ssh23:45:082
503099168,1cyclictest4116-21cat00:00:272
503099163,2cyclictest31652-21idleruntime-cro20:05:002
5030991615,1cyclictest0-21swapper/222:29:212
503099160,10cyclictest1061-21ssh22:47:182
502899163,11cyclictest19973-21cron21:50:001
37462160,0sleep10-21swapper/100:00:231
503499151,9cyclictest0-21swapper/322:44:503
503099158,2cyclictest16982-21ssh21:46:542
503099158,1cyclictest7073-21ssh00:26:312
503099158,1cyclictest5073-21cut21:15:012
503099158,1cyclictest30127-21cat21:05:282
503099157,2cyclictest9104-21cat20:25:002
503099157,2cyclictest31445-21ssh22:21:302
503099157,2cyclictest30676-21rm23:56:122
503099157,2cyclictest19006-21ssh22:12:402
503099157,2cyclictest18885-21ssh00:36:102
503099157,2cyclictest14826-21ssh23:20:512
503099156,2cyclictest21227-21grep19:40:272
503099153,3cyclictest224-21systemd-journal00:08:222
5030991513,1cyclictest0-21swapper/220:40:162
503099151,2cyclictest224-21systemd-journal19:45:002
503099150,9cyclictest14820-21ssh21:21:032
503099150,1cyclictest0-21swapper/222:35:152
502899154,8cyclictest30660-21diskmemload23:37:461
502899153,5cyclictest0-21swapper/120:23:591
502899150,2cyclictest0-21swapper/121:35:321
502299155,5cyclictest224-21systemd-journal22:00:000
5022991514,1cyclictest0-21swapper/022:11:300
5022991511,2cyclictest19287-21wc23:00:210
5022991510,3cyclictest22384-21wc20:50:210
503499144,9cyclictest32630-21/usr/sbin/munin21:35:263
503499143,11cyclictest0-21swapper/300:06:283
5034991410,3cyclictest9247-21grep19:15:273
503499140,10cyclictest0-21swapper/320:05:173
503099148,1cyclictest21974-21diskstats20:50:172
503099148,1cyclictest14504-21ssh00:33:242
503099148,1cyclictest112172sleep220:27:542
503099148,1cyclictest10750-21ssh23:41:552
503099147,3cyclictest15160-21tr20:35:232
503099147,2cyclictest9312-21grep19:15:272
503099147,1cyclictest27957-21ssh23:30:302
503099147,1cyclictest24931-21egrep19:50:162
503099141,1cyclictest224-21systemd-journal22:56:372
503099141,11cyclictest0-21swapper/220:45:262
503099140,3cyclictest15445-21grep19:30:192
5028991414,0cyclictest0-21swapper/121:27:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional