You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 17:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i3-6100U @2300 MHz, Linux 5.15.79-rt54 (Profile)

Latency plot of shadow in rack #7, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack7slot7s.osadl.org (updated Mon Mar 20, 2023 12:44:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5879210475,9sleep10-21swapper/107:09:491
595029581,10sleep30-21swapper/307:09:593
554728864,6sleep20-21swapper/207:05:322
574828270,8sleep00-21swapper/007:08:060
604899290,25cyclictest0-21swapper/211:17:362
604599280,3cyclictest0-21swapper/110:19:361
604899270,26cyclictest0-21swapper/211:50:082
604899260,26cyclictest0-21swapper/209:11:002
604899260,25cyclictest11801-21cut07:20:192
48432250,1sleep20-21swapper/210:33:322
604899230,23cyclictest0-21swapper/209:42:232
604899221,20cyclictest0-21swapper/208:10:142
604899220,1cyclictest0-21swapper/212:18:472
604299222,2cyclictest141rcu_preempt10:32:290
605199210,1cyclictest0-21swapper/310:51:043
604899219,11cyclictest1089-21munin-node11:35:002
6042992121,0cyclictest0-21swapper/008:40:150
604299210,0cyclictest0-21swapper/009:37:360
604299207,6cyclictest7178-21ssh09:35:000
6045991914,1cyclictest25-21ksoftirqd/110:33:361
604299197,6cyclictest22631-21munin-run11:45:000
605199180,16cyclictest14785-21cron10:40:003
6048991816,1cyclictest24957-21timerandwakeup11:25:282
604899181,16cyclictest2420-21ssh10:51:362
604899180,17cyclictest0-21swapper/212:26:342
604899180,17cyclictest0-21swapper/211:45:172
604899180,14cyclictest0-21swapper/211:22:282
6051991716,1cyclictest0-21swapper/312:22:373
604899170,17cyclictest0-21swapper/210:36:402
604899170,16cyclictest0-21swapper/211:12:242
604899170,16cyclictest0-21swapper/209:20:042
604299177,5cyclictest14751-21perf11:00:010
605199160,16cyclictest0-21swapper/309:33:043
605199160,0cyclictest0-21swapper/310:59:403
604899169,1cyclictest141rcu_preempt07:50:272
604899163,2cyclictest141rcu_preempt11:58:502
6048991614,1cyclictest31193-21diskmemload12:33:572
604899160,16cyclictest0-21swapper/212:09:132
604899160,16cyclictest0-21swapper/210:55:302
6045991614,1cyclictest0-21swapper/110:13:101
6045991611,2cyclictest25-21ksoftirqd/111:45:001
604599160,2cyclictest141rcu_preempt11:46:181
6051991514,1cyclictest0-21swapper/311:42:063
6051991514,1cyclictest0-21swapper/311:14:003
6051991510,2cyclictest39-21ksoftirqd/309:39:433
6048991513,1cyclictest0-21swapper/212:23:242
6048991511,2cyclictest0-21swapper/211:33:492
604899150,15cyclictest0-21swapper/209:15:062
604899150,15cyclictest0-21swapper/209:15:052
6042991511,4cyclictest5938-21apache212:18:430
605199143,6cyclictest23414-21aten2_r7power_p12:05:213
6051991410,3cyclictest6551-21grep11:55:013
604899147,6cyclictest0-21swapper/210:00:192
604899142,10cyclictest0-21swapper/212:13:412
604899140,13cyclictest29919-21cat08:00:012
604899140,13cyclictest25556-21sed09:45:302
604899140,13cyclictest0-21swapper/209:34:212
604599147,3cyclictest25538-21sed11:05:331
604599142,7cyclictest4735-21munin-run08:14:591
6045991410,3cyclictest24173-21grep07:45:231
604599140,13cyclictest31152-21wc08:00:311
604599140,13cyclictest0-21swapper/112:18:391
604299141,1cyclictest0-21swapper/011:46:300
6042991411,2cyclictest32082-21df12:30:210
604299140,2cyclictest0-21swapper/011:36:020
604299140,10cyclictest7204-21/usr/sbin/munin12:35:190
605199139,3cyclictest8724-21cat09:15:283
605199139,3cyclictest8724-21cat09:15:273
605199139,3cyclictest24623-21cat12:25:303
605199139,3cyclictest20447-21wc07:40:003
605199139,3cyclictest16897-21sh08:39:593
605199139,3cyclictest1036-21grep10:10:303
605199131,3cyclictest31193-21diskmemload09:22:013
605199131,12cyclictest0-21swapper/309:40:403
6051991310,2cyclictest7391-21grep12:15:183
6051991310,2cyclictest31142-21cat11:10:003
6051991310,2cyclictest27518-21sh10:27:103
6051991310,2cyclictest16068-21grep07:30:163
6051991310,2cyclictest14683-21cron12:40:003
6048991310,2cyclictest9256-21sort07:15:222
6048991310,2cyclictest8704-21head12:35:332
6048991310,2cyclictest24921-21wc10:05:262
6048991310,2cyclictest14990-21sh11:40:092
604899130,2cyclictest0-21swapper/210:20:162
604899130,1cyclictest0-21swapper/209:37:462
604899130,12cyclictest32759-21grep10:10:232
604899130,0cyclictest0-21swapper/210:45:452
604599139,3cyclictest6654-21chrt09:34:351
604599139,3cyclictest6499-21cron12:15:011
604599139,3cyclictest31456-21munin-plugin-st09:30:011
604599139,3cyclictest23985-21grep11:25:181
604599139,3cyclictest15243-21grep11:20:161
604599139,2cyclictest23803-21cut10:05:181
604599138,3cyclictest736-21wc11:50:321
604599136,2cyclictest25-21ksoftirqd/107:40:161
604599133,8cyclictest0-21swapper/109:05:201
604599133,6cyclictest9264-21users10:35:311
6045991311,2cyclictest62050irq/128-enp2s0-09:14:231
6045991311,1cyclictest25-21ksoftirqd/109:40:421
6045991310,2cyclictest29379-21grep07:55:311
6045991310,2cyclictest25851-21cat08:55:271
6045991310,2cyclictest15486-21cut10:40:161
6045991310,2cyclictest15486-21cut10:40:161
6042991311,1cyclictest197172sleep009:42:120
6042991310,2cyclictest7292-21cut11:55:170
6042991310,2cyclictest6162-21grep08:15:210
6042991310,2cyclictest32763-21strings10:10:230
6042991310,2cyclictest22887-21sh10:45:000
6042991310,2cyclictest22887-21sh10:45:000
6042991310,2cyclictest14816-21sh10:20:000
605199129,2cyclictest6555-21grep07:10:153
605199129,2cyclictest32321-21grep12:30:233
605199129,2cyclictest30795-21munin-plugin-st11:30:003
605199129,2cyclictest28305-21cut07:55:213
605199129,2cyclictest24128-21grep11:45:253
605199129,2cyclictest22608-21grep08:50:163
605199129,2cyclictest19308-21perf08:45:003
605199129,2cyclictest17257-21sed10:40:313
605199129,2cyclictest17257-21sed10:40:303
605199120,11cyclictest24987-21ntpq09:45:253
604899129,2cyclictest9190-21sed10:15:302
604899129,2cyclictest6978-21grep07:10:182
604899129,2cyclictest32636-21sed09:50:272
604899129,2cyclictest28371-21chrt11:07:502
604899129,2cyclictest24542-21grep08:55:162
604899129,2cyclictest17411-21head11:00:392
604899129,2cyclictest15978-21head12:00:232
604899129,2cyclictest14178-21grep07:25:182
604899129,2cyclictest1333-21cut08:05:212
604899129,2cyclictest11186-21tr08:25:222
604899128,3cyclictest25614-21cat09:25:282
604899125,1cyclictest16045-21/usr/sbin/munin10:40:212
604899125,1cyclictest16045-21/usr/sbin/munin10:40:202
604899123,2cyclictest7294-21apt-get09:55:152
6048991212,0cyclictest0-21swapper/208:30:142
604899120,11cyclictest8900-21sed08:20:282
604599129,2cyclictest6721-21sed08:15:281
604599129,2cyclictest19802-21tr08:45:201
604599129,2cyclictest19310-21awk08:45:001
604599129,2cyclictest19228-21grep07:35:211
604599129,2cyclictest15700-21grep10:20:171
604599128,3cyclictest17895-21cron07:35:001
604599127,3cyclictest25-21ksoftirqd/111:56:041
604599122,6cyclictest14603-21cron11:19:591
6045991210,1cyclictest25-21ksoftirqd/107:10:151
6045991210,1cyclictest0-21swapper/112:30:511
604599120,2cyclictest0-21swapper/110:49:561
604599120,12cyclictest0-21swapper/111:31:451
604599120,11cyclictest14543-21perf08:35:001
604299129,2cyclictest31939-21grep11:10:220
604299129,2cyclictest31843-21tr10:50:200
604299129,2cyclictest31070-21cat12:30:000
604299129,2cyclictest30965-21basename10:10:000
604299129,2cyclictest28438-21head07:55:200
604299129,2cyclictest24889-21cat11:25:270
604299129,2cyclictest24279-21grep10:45:260
604299129,2cyclictest23666-21head07:45:180
604299129,2cyclictest15785-21cut10:00:220
604299129,2cyclictest11319-21grep07:20:160
604299128,3cyclictest24200-21grep10:25:190
604299128,3cyclictest13428-21apt-get07:25:140
604299125,6cyclictest26457-21awk09:00:000
604299121,8cyclictest10051-21apt-get08:25:150
604299121,10cyclictest18802-21runrttasks09:21:210
6042991210,1cyclictest5923-21apache209:52:270
604299120,11cyclictest32020-21df12:10:250
101302120,0sleep30-21swapper/309:56:113
605199119,2cyclictest20140-21ssh11:02:503
605199119,1cyclictest9859-21cut07:15:273
605199119,1cyclictest15621-21cut10:00:203
605199118,2cyclictest9046-21ssh11:55:313
605199118,2cyclictest7985-21cat11:35:203
605199118,2cyclictest7564-21tr11:15:173
605199118,2cyclictest4747-21awk08:14:593
605199118,2cyclictest3693-21ssh09:52:323
605199118,2cyclictest32543-21ssh11:30:293
605199118,2cyclictest31802-21grep08:00:383
605199118,2cyclictest27616-21taskset09:26:533
605199118,2cyclictest26412-21sh09:00:013
605199118,2cyclictest25185-21wc10:05:293
605199118,2cyclictest23182-21ssh10:45:083
605199118,2cyclictest2154-21ssh12:11:313
605199118,2cyclictest21190-21grep08:45:343
605199118,2cyclictest16149-21grep10:20:193
605199118,2cyclictest15017-21ssh11:20:073
605199118,2cyclictest13121-21fw_packets08:30:183
605199118,2cyclictest12110-21ssh10:17:503
605199118,2cyclictest1062-21grep10:30:323
605199113,8cyclictest1378-21ntpd12:00:573
605199113,4cyclictest224-21systemd-journal08:14:593
605199110,10cyclictest20675-21aten2_r7power_v07:40:143
605199110,10cyclictest1511-21seq09:10:393
604899119,1cyclictest219412sleep208:50:132
604899119,1cyclictest20823-21tr07:40:162
604899118,2cyclictest26477-21ssh10:26:192
604899118,2cyclictest20675-21head08:45:262
604899118,2cyclictest17156-21chrt08:40:122
604899117,3cyclictest7262-21cat08:20:002
604599119,2cyclictest25-21ksoftirqd/109:46:401
604599119,1cyclictest169132sleep112:20:301
604599118,2cyclictest9989-21ssh09:15:571
604599118,2cyclictest9989-21ssh09:15:571
604599118,2cyclictest9386-21ssh12:35:421
604599118,2cyclictest8775-21wc11:35:281
604599118,2cyclictest8664-21ssh10:55:271
604599118,2cyclictest32361-21tail10:50:231
604599118,2cyclictest31613-21ssh11:10:181
604599118,2cyclictest28606-21chrt09:02:301
604599118,2cyclictest26402-21tr07:50:211
604599118,2cyclictest24187-21sed12:05:251
604599118,2cyclictest23635-21grep08:50:271
604599118,2cyclictest15668-21ssh09:20:111
604599118,2cyclictest15400-21cat12:00:201
604599118,2cyclictest15108-21ssh11:00:071
604599118,2cyclictest11152-21cut07:20:151
604599116,2cyclictest25-21ksoftirqd/112:26:381
604599110,10cyclictest25647-21sh10:25:391
604599110,10cyclictest16291-21head10:00:231
604299118,2cyclictest9554-21ssh10:35:440
604299118,2cyclictest8358-21cat11:15:230
604299118,2cyclictest7833-21tr09:15:220
604299118,2cyclictest7833-21tr09:15:210
604299118,2cyclictest7141-21ssh09:55:080
604299118,2cyclictest31658-21wc11:30:240
604299118,2cyclictest30932-21ssh11:50:010
604299118,2cyclictest3023-21ssh09:11:530
604299118,2cyclictest25801-21sed09:25:290
604299118,2cyclictest25044-21open_inodes09:45:250
604299118,2cyclictest21671-21sh08:49:590
604299118,2cyclictest18436-21tr07:35:170
604299118,2cyclictest17568-21ssh11:20:470
604299118,2cyclictest17323-21ssh12:00:390
604299118,2cyclictest16286-21grep11:00:260
604299118,2cyclictest16261-21cut10:20:200
604299118,2cyclictest15833-21grep07:30:140
604299118,2cyclictest13137-21head08:30:170
604299115,2cyclictest224-21systemd-journal08:10:000
604299114,4cyclictest0-21swapper/007:10:140
604299111,4cyclictest23413-21aten2_r7power_p12:05:220
604299110,10cyclictest23972-21seq08:54:160
605199108,1cyclictest95762sleep308:22:303
605199108,1cyclictest29514-21expr09:05:223
605199108,1cyclictest27005-21cat09:00:153
605199108,1cyclictest26909-21date07:50:273
605199108,1cyclictest249802sleep307:49:203
605199108,1cyclictest100122sleep308:25:093
605199107,2cyclictest13561-21grep07:25:153
605199107,2cyclictest11420-21cat07:20:173
604899108,1cyclictest27903-21cut09:00:232
604899108,1cyclictest22965-21cut07:45:142
604899107,2cyclictest29566-21df09:05:222
604899107,2cyclictest14842-21basename08:35:132
604899102,2cyclictest224-21systemd-journal07:31:312
604899100,9cyclictest18169-21apache_processe07:35:132
604599108,1cyclictest9702-21cat07:15:251
604599108,1cyclictest7782-21grep08:20:171
604599108,1cyclictest2507-21ssh09:51:341
604599108,1cyclictest14870-21awk08:35:131
604599107,2cyclictest7767-21apt-get09:35:181
604599107,2cyclictest15468-21chrt07:29:231
604599107,2cyclictest11749-21grep08:25:291
604599104,4cyclictest912-21cut08:05:181
604599102,8cyclictest25-21ksoftirqd/109:55:201
604299108,2cyclictest23808-21grep11:05:200
604299108,1cyclictest9089-21cat07:15:190
604299108,1cyclictest8016-21expr08:20:180
604299108,1cyclictest20857-21cstates07:40:160
604299108,1cyclictest16737-21chrt08:38:010
604299108,1cyclictest15121-21rm12:20:150
604299107,2cyclictest29753-21grep09:05:230
604299107,2cyclictest28033-21sendmail_mailst09:00:240
604299107,2cyclictest26442-21grep07:50:220
604299100,6cyclictest4519-21sshd08:12:250
604299100,10cyclictest30482-21ls08:00:270
60519996,2cyclictest884-21grep08:05:173
60489997,1cyclictest301002sleep208:00:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional