You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-05 - 01:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun May 04, 2025 12:44:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7138210378,8sleep30-21swapper/307:06:363
7250210287,10sleep10-21swapper/107:08:031
719229064,8sleep00-21swapper/007:07:180
709728164,13sleep20-21swapper/207:06:022
270612540,0sleep20-21swapper/211:15:502
7488993130,1cyclictest0-21swapper/108:58:081
748399275,10cyclictest0-21swapper/009:15:140
7490992626,0cyclictest0-21swapper/212:36:152
749399255,13cyclictest418-21diskmemload11:10:193
749099240,24cyclictest0-21swapper/211:54:382
748899240,1cyclictest0-21swapper/109:29:541
749399230,23cyclictest0-21swapper/310:00:413
749399230,23cyclictest0-21swapper/310:00:413
7488992320,2cyclictest7744-21snmpd10:04:081
7488992320,2cyclictest7744-21snmpd10:04:081
748899231,5cyclictest26352-21sed12:35:171
749099220,22cyclictest0-21swapper/211:22:072
7483992017,2cyclictest7744-21snmpd09:48:580
7493991917,1cyclictest7744-21snmpd10:07:323
748399198,6cyclictest23851-21awk09:54:590
7483991916,2cyclictest7744-21snmpd12:19:380
749099182,14cyclictest23517-21pmu-power07:40:312
749099180,18cyclictest0-21swapper/210:03:022
749099180,18cyclictest0-21swapper/210:03:022
749099180,15cyclictest0-21swapper/211:26:442
748899180,2cyclictest0-21swapper/110:42:471
749399177,8cyclictest0-21swapper/309:00:153
7493991716,1cyclictest418-21diskmemload10:16:343
7493991715,1cyclictest7744-21snmpd10:34:333
749099170,16cyclictest0-21swapper/211:00:442
749099170,16cyclictest0-21swapper/210:32:082
748899175,11cyclictest5027-21munin-run09:40:001
748399174,7cyclictest31348-21ssh09:35:000
748399173,7cyclictest1075-21sendmail-msp12:40:000
749399160,1cyclictest0-21swapper/310:23:293
748899168,3cyclictest24631-21apache_accesses11:55:121
748399166,8cyclictest19617-21hwlatdetect11:10:180
748399163,6cyclictest29138-21munin-run07:55:000
7483991613,2cyclictest7744-21snmpd09:37:160
749399150,1cyclictest0-21swapper/312:18:003
7490991512,3cyclictest0-21swapper/209:30:442
7490991512,2cyclictest7744-21snmpd10:36:322
7490991511,2cyclictest24944-21tr07:45:192
749099150,15cyclictest0-21swapper/212:21:172
7488991512,2cyclictest7744-21snmpd12:24:391
7488991511,3cyclictest390-21awk12:20:001
7488991510,4cyclictest24459-21tr11:35:141
748899150,1cyclictest0-21swapper/111:08:431
748399154,6cyclictest31530-21tr08:00:000
748399151,7cyclictest9898-21munin-run07:15:000
749399147,5cyclictest1054-21awk12:40:003
749399142,2cyclictest141rcu_preempt11:49:393
7493991410,3cyclictest9395-21grep07:10:283
7493991410,3cyclictest16119-21basename08:35:143
7493991410,3cyclictest12163-21sed10:10:283
749399140,12cyclictest19258-21cut09:50:263
749099148,5cyclictest2346-21grep12:00:302
7490991413,1cyclictest0-21swapper/207:10:032
7490991412,1cyclictest23411-21apt-get08:50:152
7490991410,3cyclictest25408-21cut11:35:212
748899148,4cyclictest15423-21munin-run11:30:001
7488991413,1cyclictest0-21swapper/109:55:421
7488991411,2cyclictest31824-21cron10:55:001
7488991410,3cyclictest10220-21wc08:20:391
748399143,6cyclictest418-21diskmemload11:15:240
7483991412,1cyclictest21213-21ssh09:25:260
7483991410,3cyclictest15790-21taskset08:35:010
7483991410,2cyclictest21549-21grep07:35:290
749399139,3cyclictest2861-21grep12:20:273
749399139,3cyclictest1817-21grep11:40:273
7493991310,2cyclictest7183-21cat11:25:003
7493991310,2cyclictest22933-21sort07:40:213
7493991310,2cyclictest15799-21apt08:35:013
7493991310,2cyclictest12276-21cat11:05:003
749399130,12cyclictest6796-21tr09:15:173
749099139,3cyclictest28883-21taskset10:25:162
749099139,3cyclictest16132-21grep10:15:152
749099139,3cyclictest10571-21tr10:10:162
7490991310,2cyclictest4835-21tail08:10:192
7490991310,2cyclictest26594-21grep11:55:272
7490991310,2cyclictest26194-21cut10:50:152
7490991310,2cyclictest12786-21tr07:20:172
7490991310,2cyclictest10567-21grep08:20:432
749099130,12cyclictest30127-21grep07:55:182
749099130,12cyclictest19646-21wc09:50:302
748899139,3cyclictest11069-21sed10:10:181
748899139,2cyclictest31926-21grep08:00:161
748899138,3cyclictest19960-21wc08:40:261
748899135,4cyclictest12773-21cut09:20:161
7488991311,1cyclictest1287-21minissdpd11:19:301
7488991310,2cyclictest5069-21wc08:10:201
7488991310,2cyclictest29666-21grep07:55:161
7488991310,2cyclictest23508-21grep07:40:311
748899130,13cyclictest0-21swapper/109:41:531
748899130,12cyclictest7637-21sed11:00:241
748399139,3cyclictest8360-21apache_accesses10:35:140
748399139,3cyclictest14494-21sed11:05:300
748399139,2cyclictest7203-21ls09:40:310
748399139,2cyclictest23934-21grep07:40:350
748399134,5cyclictest0-21swapper/010:10:150
7483991312,1cyclictest0-21swapper/007:20:160
7483991310,2cyclictest7744-21snmpd11:48:460
749399129,2cyclictest7744-21snmpd11:15:003
749399129,2cyclictest7744-21snmpd10:59:183
749399129,2cyclictest30589-21ls07:55:233
749399129,2cyclictest26761-21sort10:50:193
749399129,2cyclictest26544-21tail08:55:183
749399129,2cyclictest25051-21grep09:55:243
749399129,2cyclictest24877-21grep07:45:193
749399129,2cyclictest10404-21tr07:15:163
749399129,2cyclictest10175-21ls08:20:383
749399128,3cyclictest434-21uname08:00:233
749399123,3cyclictest0-21swapper/309:25:143
749399120,11cyclictest8621-21awk08:20:003
749399120,11cyclictest27745-21grep07:50:203
749399120,11cyclictest24127-21cron11:55:013
749099129,2cyclictest6165-21df09:40:212
749099129,2cyclictest28905-21head09:00:192
749099129,2cyclictest26037-21grep12:15:182
749099129,2cyclictest17180-21grep12:10:162
749099129,2cyclictest16792-21grep11:30:182
749099129,2cyclictest11880-21sed07:15:272
749099128,3cyclictest6750-21cut09:15:162
749099128,3cyclictest21760-21grep08:45:182
749099127,2cyclictest11293-21grep12:25:272
749099121,3cyclictest5013-21ssh09:40:002
749099121,1cyclictest32621-21/usr/sbin/munin11:40:162
748899129,2cyclictest9081-21ssh12:25:021
748899129,2cyclictest820-21cut11:20:231
748899129,2cyclictest574-21cut12:00:181
748899129,2cyclictest31043-21tr09:05:161
748899129,2cyclictest21525-21sed07:35:291
748899129,2cyclictest12787-21tr07:20:171
748899128,3cyclictest21818-21ssh09:52:411
748899128,3cyclictest21139-21grep08:45:161
748899128,2cyclictest19613-21cron07:35:001
748899124,2cyclictest224-21systemd-journal11:15:001
748899123,7cyclictest18937-21ssh12:30:231
748899120,11cyclictest2500-21ssh10:56:161
748899120,0cyclictest0-21swapper/112:05:431
748399129,2cyclictest7925-21cron10:35:010
748399129,2cyclictest6059-21ssh10:05:280
748399129,2cyclictest32546-21grep10:55:150
748399129,2cyclictest30355-21grep09:05:000
748399129,2cyclictest26857-21grep10:50:190
748399129,2cyclictest26489-21head08:55:170
748399129,2cyclictest18404-21ls12:10:240
748399129,2cyclictest18396-21sed11:50:270
748399129,2cyclictest17908-21cut12:30:170
748399129,2cyclictest17422-21ssh10:15:240
748399129,2cyclictest15525-21wc11:30:000
748399128,3cyclictest9634-21cut12:05:170
748399128,3cyclictest5418-21gzip08:10:230
748399128,2cyclictest8802-21wc08:20:000
748399127,2cyclictest224-21systemd-journal12:25:000
748399121,3cyclictest0-21swapper/011:31:170
748399120,2cyclictest0-21swapper/011:59:530
748399120,10cyclictest25015-21ssh10:49:350
748399120,10cyclictest0-21swapper/011:40:180
749399119,1cyclictest381rcuc/308:45:153
749399118,2cyclictest919-21expr12:00:183
749399118,2cyclictest8560-21ssh12:05:023
749399118,2cyclictest6197-21ssh09:40:213
749399118,2cyclictest31470-21ssh11:39:433
749399118,2cyclictest28498-21cat10:25:163
749399118,2cyclictest27280-21ssh11:55:593
749399118,2cyclictest24435-21chrt10:48:473
749399118,2cyclictest23924-21sh12:33:553
749399118,2cyclictest1782-21basename08:05:143
749399118,2cyclictest17069-21cut12:10:153
749399118,2cyclictest16775-21head11:30:183
749399118,2cyclictest16409-21ssh09:22:253
749399118,2cyclictest15848-21uname07:25:213
749399118,2cyclictest15564-21cat10:40:473
749399118,2cyclictest13166-21sh12:26:523
749399118,2cyclictest12542-21ssh11:27:423
749399118,2cyclictest12530-21grep09:45:183
749399118,2cyclictest11579-21tr08:25:163
749399117,3cyclictest2590-21wc09:10:303
749399117,2cyclictest4246-21grep08:10:163
749399112,7cyclictest0-21swapper/311:05:163
749399111,9cyclictest26029-21taskset09:30:163
749399111,7cyclictest31934-21aten2_r7power_v09:35:163
749099119,1cyclictest393-21cron09:10:002
749099118,2cyclictest8625-21ssh12:05:052
749099118,2cyclictest26485-21ssh09:56:042
749099118,2cyclictest20697-21cat07:35:202
749099118,2cyclictest20459-21grep10:45:182
749099118,2cyclictest17720-21awk12:30:152
749099118,2cyclictest17703-21sed08:35:262
749099118,2cyclictest17126-21sed07:29:592
749099118,2cyclictest14843-21sh08:30:212
749099118,2cyclictest14587-21grep10:40:392
749099118,2cyclictest13830-21rm11:48:222
749099118,2cyclictest12868-21cut09:20:172
749099117,3cyclictest8417-21chrt08:17:522
749099117,3cyclictest27884-21apt08:59:592
749099117,2cyclictest9981-21cat10:10:002
749099110,8cyclictest0-21swapper/211:11:092
749099110,3cyclictest0-21swapper/210:21:572
749099110,2cyclictest0-21swapper/211:05:182
748899118,2cyclictest8860-21tr07:10:221
748899118,2cyclictest6773-21ssh11:44:211
748899118,2cyclictest6072-21head10:05:281
748899118,2cyclictest5204-21ssh09:13:161
748899118,2cyclictest29032-21tail10:25:181
748899118,2cyclictest28838-21tune2fs09:00:181
748899118,2cyclictest2797-21ssh10:30:171
748899118,2cyclictest27028-21sh09:30:221
748899118,2cyclictest25512-21sh08:55:001
748899118,2cyclictest22231-21expr10:20:161
748899118,2cyclictest19729-21ssh12:11:031
748899118,2cyclictest16723-21chrt07:26:101
748899118,2cyclictest16431-21apache_volume11:50:131
748899118,2cyclictest16185-21cut10:15:151
748899118,2cyclictest15921-21cat08:35:011
748899118,2cyclictest15611-21ssh09:47:421
748899118,2cyclictest12943-21sed08:25:281
748899118,2cyclictest12852-21ssh11:47:341
748899117,3cyclictest8879-21ssh10:35:171
748899110,11cyclictest0-21swapper/109:17:241
748899110,10cyclictest21168-21ssh10:45:251
748399118,2cyclictest6597-21cut11:00:170
748399118,2cyclictest3737-21ssh10:04:590
748399118,2cyclictest3737-21ssh10:04:580
748399118,2cyclictest30470-21ssh11:38:550
748399118,2cyclictest2932-21sed08:05:220
748399118,2cyclictest28242-21ssh09:58:060
748399118,2cyclictest24183-21sed08:50:180
748399118,2cyclictest22196-21grep10:20:160
748399118,2cyclictest2145-21ntp_states09:10:250
748399118,2cyclictest16394-21ssh10:41:390
748399118,2cyclictest15972-21ssh09:21:560
748399118,2cyclictest15158-21grep07:25:170
748399118,2cyclictest13426-21dump-pmu-power08:30:010
748399118,2cyclictest11192-21cat07:15:220
748399117,3cyclictest1025-21grep08:00:290
748399115,5cyclictest32015-21ps09:05:220
748399110,5cyclictest0-21swapper/012:00:180
748399110,10cyclictest6312-21ssh11:24:300
749399108,1cyclictest8989-21df10:35:183
749399108,1cyclictest23820-21expr08:50:163
749399108,1cyclictest21626-21vmstat07:35:303
749399108,1cyclictest12928-21cat07:20:173
749399107,2cyclictest31577-21basename09:05:193
749399107,2cyclictest18732-21grep08:40:163
749099108,2cyclictest3045-21ssh09:10:482
749099108,1cyclictest3378-21awk08:05:272
749099108,1cyclictest32610-21expr10:55:152
749099108,1cyclictest13059-21chrt08:25:292
749099108,1cyclictest12123-21cat09:45:162
749099107,2cyclictest432-21cut08:00:222
749099107,2cyclictest20724-21ssh09:25:222
749099107,2cyclictest19153-21sed07:30:292
749099101,3cyclictest1-21systemd07:52:372
749099100,9cyclictest202342sleep208:41:002
748899108,2cyclictest19344-21ssh11:31:341
748899108,1cyclictest6685-21cat08:15:201
748899108,1cyclictest11912-21chrt07:15:361
748899107,2cyclictest24931-21grep07:45:181
748399108,1cyclictest30580-21ssh10:25:550
748399108,1cyclictest20489-21taskset08:44:180
748399108,1cyclictest16873-21ls08:35:180
748399108,1cyclictest10804-21chrt08:22:020
748399107,2cyclictest21105-21grep08:45:160
748399107,2cyclictest17802-21grep07:30:170
748399107,2cyclictest11927-21ssh12:25:520
748399105,4cyclictest26272-21sed07:45:290
74939995,3cyclictest18244-21grep07:30:193
74889997,1cyclictest28391-21readlink07:50:251
74889997,1cyclictest16083-21cut08:35:141
74889992,3cyclictest224-21systemd-journal08:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional