You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-09 - 00:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot3s.osadl.org (updated Fri Dec 08, 2023 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
105491690,47ptp4l0-21swapper/007:09:550
105491570,34ptp4l0-21swapper/107:05:271
790425046,2sleep20-21swapper/207:05:582
750925047,1sleep30-21swapper/307:05:303
9112440,0sleep3401ktimersoftd/311:55:273
264512410,0sleep30-21swapper/307:40:253
105491300,0ptp4l0-21swapper/110:50:201
105491260,0ptp4l0-21swapper/008:20:230
105491200,0ptp4l0-21swapper/108:35:021
847999190,16cyclictest0-21swapper/112:00:331
105491180,0ptp4l0-21swapper/008:05:310
105491180,0ptp4l0-21swapper/008:00:520
105491170,0ptp4l0-21swapper/111:20:291
105491170,0ptp4l0-21swapper/010:50:260
105491170,0ptp4l0-21swapper/010:35:230
105491170,0ptp4l0-21swapper/008:10:310
105491160,1ptp4l31349-21apt-get07:50:201
105491160,1ptp4l19857-21apt-get07:30:011
105491160,1ptp4l10918-21apt-get12:15:210
105491160,0ptp4l0-21swapper/111:55:251
105491160,0ptp4l0-21swapper/109:25:311
105491160,0ptp4l0-21swapper/107:25:341
105491160,0ptp4l0-21swapper/009:05:330
105491150,2ptp4l3942-21cut12:00:270
105491150,2ptp4l0-21swapper/210:15:402
105491150,1ptp4l6946-21apt-get09:05:201
105491150,1ptp4l4934-21memory10:00:301
105491150,1ptp4l4934-21memory10:00:291
105491150,1ptp4l32636-21ls09:50:351
105491150,1ptp4l31076-21memory11:50:300
105491150,1ptp4l27580-21apt-get10:45:211
105491150,1ptp4l22382-21/usr/sbin/munin08:30:351
105491150,1ptp4l21885-21snmp_o5powerctl09:30:310
105491150,1ptp4l2130-21snmp_o5powerctl10:55:330
105491150,1ptp4l20589-21irqstats12:35:261
105491150,1ptp4l19295-21date12:35:011
105491150,1ptp4l18999-21users10:25:351
105491150,1ptp4l14329-21apt-get07:20:010
105491150,1ptp4l14090-21snmp_o5powerctl08:15:350
105491150,1ptp4l14076-21ls09:15:321
105491150,1ptp4l13920-21smartctl09:15:310
105491150,1ptp4l12302-21ntp_states12:15:301
105491150,1ptp4l10538-21ls11:10:341
105491150,0ptp4l0-21swapper/107:20:201
105491150,0ptp4l0-21swapper/011:05:200
105491150,0ptp4l0-21swapper/009:48:530
105491150,0ptp4l0-21swapper/008:45:310
105491150,0ptp4l0-21swapper/008:25:240
105491150,0ptp4l0-21swapper/007:14:160
105491140,1ptp4l9962-21snmp_o5powerctl12:10:410
105491140,1ptp4l847999cyclictest10:35:211
105491140,1ptp4l6675-21hddtemp_smartct11:05:251
105491140,1ptp4l5681-21ntp_states08:01:001
105491140,1ptp4l553-21ls08:50:331
105491140,1ptp4l3176-21/usr/sbin/munin07:55:310
105491140,1ptp4l30163-21snmp_o5powerctl07:45:320
105491140,1ptp4l29344-21if_enp4s008:45:251
105491140,1ptp4l28593-21ntp_states11:45:301
105491140,1ptp4l26427-21ps09:40:351
105491140,1ptp4l24664-21snmp_o5powerctl07:35:511
105491140,1ptp4l22049-21snmp_o5powerctl08:30:340
105491140,1ptp4l21006-21ntp_states10:30:260
105491140,1ptp4l20846-21grep10:30:241
105491140,1ptp4l17983-21ntp_states11:25:290
105491140,1ptp4l1603-21/usr/sbin/munin08:55:220
105491140,1ptp4l16012-21snmp_o5powerctl10:20:361
105491140,1ptp4l15426-21timerwakeupswit12:20:351
105491140,1ptp4l13561-21/usr/sbin/munin12:20:330
105491140,1ptp4l11998-21latency_hist09:15:011
105491140,1ptp4l1165-21irqrtprio11:55:270
105491140,0ptp4l1160-21apt-get08:55:021
105491140,0ptp4l0-21swapper/112:10:401
105491140,0ptp4l0-21swapper/111:00:011
105491140,0ptp4l0-21swapper/110:15:331
105491140,0ptp4l0-21swapper/110:05:311
105491140,0ptp4l0-21swapper/109:35:211
105491140,0ptp4l0-21swapper/109:20:361
105491140,0ptp4l0-21swapper/108:40:321
105491140,0ptp4l0-21swapper/108:25:311
105491140,0ptp4l0-21swapper/108:05:021
105491140,0ptp4l0-21swapper/107:55:231
105491140,0ptp4l0-21swapper/107:45:331
105491140,0ptp4l0-21swapper/107:40:021
105491140,0ptp4l0-21swapper/107:10:351
105491140,0ptp4l0-21swapper/012:30:140
105491140,0ptp4l0-21swapper/012:05:320
105491140,0ptp4l0-21swapper/011:30:310
105491140,0ptp4l0-21swapper/010:40:320
105491140,0ptp4l0-21swapper/010:25:210
105491140,0ptp4l0-21swapper/010:15:200
105491140,0ptp4l0-21swapper/009:35:190
105491140,0ptp4l0-21swapper/009:20:220
105491140,0ptp4l0-21swapper/007:51:230
105491140,0ptp4l0-21swapper/007:25:290
105491140,0ptp4l0-21swapper/007:15:200
105491130,0ptp4l9954-21strings11:10:272
105491130,0ptp4l16581-21latency_hist12:30:010
105491130,0ptp4l0-21swapper/212:20:282
105491130,0ptp4l0-21swapper/212:15:332
105491130,0ptp4l0-21swapper/212:00:292
105491130,0ptp4l0-21swapper/211:35:322
105491130,0ptp4l0-21swapper/211:05:222
105491130,0ptp4l0-21swapper/210:25:272
105491130,0ptp4l0-21swapper/209:30:292
105491130,0ptp4l0-21swapper/209:05:352
105491130,0ptp4l0-21swapper/208:55:322
105491130,0ptp4l0-21swapper/208:40:332
105491130,0ptp4l0-21swapper/208:10:272
105491130,0ptp4l0-21swapper/207:15:442
105491130,0ptp4l0-21swapper/112:05:241
105491130,0ptp4l0-21swapper/111:50:221
105491130,0ptp4l0-21swapper/111:40:251
105491130,0ptp4l0-21swapper/111:35:181
105491130,0ptp4l0-21swapper/111:30:301
105491130,0ptp4l0-21swapper/111:25:211
105491130,0ptp4l0-21swapper/111:15:251
105491130,0ptp4l0-21swapper/110:55:231
105491130,0ptp4l0-21swapper/110:40:241
105491130,0ptp4l0-21swapper/110:10:251
105491130,0ptp4l0-21swapper/109:55:231
105491130,0ptp4l0-21swapper/109:45:211
105491130,0ptp4l0-21swapper/109:30:221
105491130,0ptp4l0-21swapper/109:00:201
105491130,0ptp4l0-21swapper/108:20:201
105491130,0ptp4l0-21swapper/108:15:241
105491130,0ptp4l0-21swapper/108:10:241
105491130,0ptp4l0-21swapper/107:15:011
105491130,0ptp4l0-21swapper/012:35:250
105491130,0ptp4l0-21swapper/011:45:230
105491130,0ptp4l0-21swapper/011:40:320
105491130,0ptp4l0-21swapper/011:35:200
105491130,0ptp4l0-21swapper/011:20:220
105491130,0ptp4l0-21swapper/011:15:270
105491130,0ptp4l0-21swapper/011:10:240
105491130,0ptp4l0-21swapper/011:00:230
105491130,0ptp4l0-21swapper/010:45:240
105491130,0ptp4l0-21swapper/010:20:310
105491130,0ptp4l0-21swapper/010:10:300
105491130,0ptp4l0-21swapper/010:05:250
105491130,0ptp4l0-21swapper/010:00:240
105491130,0ptp4l0-21swapper/010:00:230
105491130,0ptp4l0-21swapper/009:55:340
105491130,0ptp4l0-21swapper/009:50:210
105491130,0ptp4l0-21swapper/009:40:310
105491130,0ptp4l0-21swapper/009:25:230
105491130,0ptp4l0-21swapper/009:10:260
105491130,0ptp4l0-21swapper/009:00:190
105491130,0ptp4l0-21swapper/008:50:220
105491130,0ptp4l0-21swapper/008:40:280
105491130,0ptp4l0-21swapper/008:35:240
105491130,0ptp4l0-21swapper/007:40:260
105491130,0ptp4l0-21swapper/007:35:170
105491130,0ptp4l0-21swapper/007:30:210
105491120,0ptp4l0-21swapper/211:45:332
105491120,0ptp4l0-21swapper/209:15:282
105491120,0ptp4l0-21swapper/208:50:292
105491120,0ptp4l0-21swapper/208:15:222
105491110,0ptp4l0-21swapper/212:05:212
105491110,0ptp4l0-21swapper/210:50:182
105491110,0ptp4l0-21swapper/210:35:392
848099103,6cyclictest9392-21kworker/2:112:10:412
105491100,0ptp4l0-21swapper/212:35:162
105491100,0ptp4l0-21swapper/207:45:232
84819997,1cyclictest9170-21taskset08:05:433
84819997,1cyclictest8503-21taskset10:06:503
84819997,1cyclictest3667-21expr11:00:223
84819997,1cyclictest30433-21cut10:50:193
84819997,1cyclictest24646-21chrt10:39:593
84819997,1cyclictest218432sleep310:32:593
84819997,1cyclictest18935-21cut07:25:333
84819997,1cyclictest1802-21cat09:55:253
84819997,1cyclictest17728-21cut08:25:193
84819997,1cyclictest17112-21chrt07:23:593
84819997,1cyclictest16343-21chrt12:27:093
84819997,1cyclictest15188-21find12:20:323
84819997,1cyclictest15182-21wc11:20:273
84819997,1cyclictest14239-21grep09:15:343
84819997,1cyclictest119332sleep309:14:213
84819997,1cyclictest11854-21chrt08:11:163
84819997,1cyclictest11346-21expr12:15:223
84819992,2cyclictest0-21swapper/311:25:203
84819990,8cyclictest11254-21taskset10:13:113
84809997,1cyclictest5257-21cut08:00:552
84809997,1cyclictest302042sleep209:45:392
84809997,1cyclictest29818-21taskset11:50:152
84809997,1cyclictest27964-21chrt07:41:392
84809997,1cyclictest25140-21seq07:37:262
84809997,1cyclictest14298-21chrt10:20:172
29268290,0sleep20-21swapper/208:45:262
10549190,0ptp4l0-21swapper/212:32:562
10549190,0ptp4l0-21swapper/212:25:152
10549190,0ptp4l0-21swapper/211:56:012
10549190,0ptp4l0-21swapper/211:43:402
10549190,0ptp4l0-21swapper/211:31:372
10549190,0ptp4l0-21swapper/211:28:052
10549190,0ptp4l0-21swapper/211:20:362
10549190,0ptp4l0-21swapper/211:15:422
10549190,0ptp4l0-21swapper/211:00:042
10549190,0ptp4l0-21swapper/210:57:562
10549190,0ptp4l0-21swapper/210:46:222
10549190,0ptp4l0-21swapper/210:44:382
10549190,0ptp4l0-21swapper/210:11:202
10549190,0ptp4l0-21swapper/210:05:592
10549190,0ptp4l0-21swapper/210:01:312
10549190,0ptp4l0-21swapper/210:01:302
10549190,0ptp4l0-21swapper/209:58:142
10549190,0ptp4l0-21swapper/209:50:392
10549190,0ptp4l0-21swapper/209:44:362
10549190,0ptp4l0-21swapper/209:22:312
10549190,0ptp4l0-21swapper/209:00:172
10549190,0ptp4l0-21swapper/208:25:072
10549190,0ptp4l0-21swapper/208:21:152
10549190,0ptp4l0-21swapper/208:05:392
10549190,0ptp4l0-21swapper/207:56:012
10549190,0ptp4l0-21swapper/207:51:402
10549190,0ptp4l0-21swapper/207:29:522
10549190,0ptp4l0-21swapper/207:22:302
10549190,0ptp4l0-21swapper/207:11:512
10549190,0ptp4l0-21swapper/112:25:031
84819986,1cyclictest9092-21expr11:10:233
84819986,1cyclictest84882sleep307:10:063
84819986,1cyclictest8031-21tr09:05:263
84819986,1cyclictest77872sleep312:07:233
84819986,1cyclictest664-21cut10:55:233
84819986,1cyclictest6344-21seq08:01:203
84819986,1cyclictest6071-21hostname11:05:213
84819986,1cyclictest5703-21head10:00:373
84819986,1cyclictest5703-21head10:00:363
84819986,1cyclictest4298-21dpkg09:00:183
84819986,1cyclictest4033-21cut12:00:273
84819986,1cyclictest31252-21cut08:50:203
84819986,1cyclictest31160-21dpkg07:50:013
84819986,1cyclictest31103-21cat09:50:213
84819986,1cyclictest30237-21cat11:50:223
84819986,1cyclictest29849-21cut08:45:283
84819986,1cyclictest29451-21taskset11:48:173
84819986,1cyclictest28806-21expr07:45:243
84819986,1cyclictest28660-21fschecks_count09:45:233
84819986,1cyclictest27846-21expr10:45:233
84819986,1cyclictest26404-21interrupts09:40:343
84819986,1cyclictest25994-21cut08:40:233
84819986,1cyclictest25154-21cut10:40:253
84819986,1cyclictest24974-21ls11:40:243
84819986,1cyclictest24532-21snmp_o5powerctl09:35:313
84819986,1cyclictest24103-21cut07:35:243
84819986,1cyclictest23597-21expr08:35:223
84819986,1cyclictest22729-21sort07:35:013
84819986,1cyclictest22392-21chrt09:32:303
84819986,1cyclictest213472sleep311:31:253
84819986,1cyclictest20329-21seq08:30:143
84819986,1cyclictest19751-21expr12:35:223
84819986,1cyclictest17892-21tr10:25:283
84819986,1cyclictest17811-21expr09:25:213
84819986,1cyclictest17169-21expr12:30:163
84819986,1cyclictest1690-21cut07:55:233
84819986,1cyclictest16388-21chrt10:20:513
84819986,1cyclictest15842sleep308:55:073
84819986,1cyclictest15598-21grep08:20:213
84819986,1cyclictest15409-21entropy09:20:253
84819986,1cyclictest14747-21taskset08:19:273
84819986,1cyclictest14024-21taskset07:16:183
84819986,1cyclictest12053-21cat11:15:253
84819986,1cyclictest10523-21taskset12:13:223
84809986,1cyclictest23042-21grep09:35:212
84809986,1cyclictest20486-21hostname08:30:242
84809986,1cyclictest20327-21wc10:30:222
84809986,1cyclictest20177-21cp07:30:022
84809986,1cyclictest18694-21cut09:25:252
10549180,0ptp4l0-21swapper/209:10:462
10549180,0ptp4l0-21swapper/208:36:522
84819976,1cyclictest12387-21awk10:15:243
84819975,1cyclictest23619-21ls11:35:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional