You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 17:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i3-2310E @2100 MHz, Linux 5.10.47-rt46 (Profile)

Latency plot of shadow in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack8slot2s.osadl.org (updated Mon Mar 20, 2023 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
647199152143,8cyclictest0-21swapper/108:12:371
647099138134,3cyclictest6510-21apt-get07:12:290
647199123115,7cyclictest0-21swapper/107:12:171
647399108104,3cyclictest0-21swapper/307:12:293
647099103100,2cyclictest0-21swapper/008:12:370
5873210140,31sleep20-21swapper/207:08:052
317132960,1sleep031717-21sendmail_mailqu08:02:490
613629148,32sleep30-21swapper/307:11:313
601529144,35sleep00-21swapper/007:09:550
598429149,31sleep10-21swapper/107:09:321
79422890,1sleep00-21swapper/009:20:160
6472998381,1cyclictest0-21swapper/207:12:292
72772810,0sleep20-21swapper/210:15:052
199792810,0sleep00-21swapper/011:48:140
6473997270,1cyclictest0-21swapper/308:12:373
6472995148,2cyclictest3670-21strings08:12:372
29112320,5sleep228-21ksoftirqd/209:42:512
6471993026,2cyclictest4928-21ssh12:02:591
195172300,1sleep30-21swapper/310:52:463
195172300,1sleep30-21swapper/310:52:463
647099290,1cyclictest0-21swapper/009:52:380
647299270,1cyclictest0-21swapper/209:52:382
6471992718,6cyclictest30643-21ssh10:34:541
57972260,0sleep30-21swapper/312:32:183
6470992522,2cyclictest19994-21kworker/u8:012:12:190
258692250,0sleep20-21swapper/210:02:472
246042250,1sleep30-21swapper/312:20:233
6473992314,6cyclictest5733-21ssh12:04:003
6472992220,1cyclictest0-21swapper/211:47:362
647299221,1cyclictest0-21swapper/210:27:312
647299220,21cyclictest0-21swapper/210:17:322
647299220,1cyclictest0-21swapper/211:17:322
6471992215,1cyclictest298442sleep107:58:051
6471992118,1cyclictest7490-21sensors11:37:411
647199210,20cyclictest0-21swapper/111:12:361
647399200,19cyclictest0-21swapper/309:42:353
647299200,19cyclictest0-21swapper/212:32:352
647099200,19cyclictest0-21swapper/009:22:320
68442190,0sleep00-21swapper/011:10:060
647099181,10cyclictest6026-21ssh10:42:170
309752180,1sleep16468-21cyclictest09:39:451
647199174,8cyclictest0-21swapper/111:49:441
6471991611,3cyclictest15126-21ssh09:53:271
6471991611,2cyclictest6344-21ssh09:18:151
647099160,15cyclictest0-21swapper/007:32:300
647399155,5cyclictest0-21swapper/311:42:363
647399153,9cyclictest673-21egrep10:37:333
647399153,9cyclictest0-21swapper/309:36:583
647199159,1cyclictest1609-21sed10:37:391
647199157,3cyclictest30702-21chrt11:30:351
647099155,8cyclictest7025-21basename08:22:160
169582150,0chrt0-21swapper/109:27:391
647399148,4cyclictest12153-21ssh12:09:443
647399145,3cyclictest0-21swapper/308:43:113
647399144,5cyclictest19870-21ssh12:16:543
647299144,5cyclictest0-21swapper/211:03:492
647199147,5cyclictest20286-21systemctl11:22:171
647199145,7cyclictest32552-21ssh11:32:271
647199145,4cyclictest27510-21/usr/sbin/munin12:22:391
647199144,5cyclictest16798-21cstates08:42:311
647099147,5cyclictest27868-21cron07:57:170
647099144,5cyclictest0-21swapper/010:24:180
647099142,6cyclictest11950-21cut07:22:320
647099141,7cyclictest16211-21perf07:32:170
647399138,4cyclictest0-21swapper/311:22:043
647399137,4cyclictest13475-21tr10:47:463
647399134,8cyclictest0-21swapper/310:22:373
647399134,4cyclictest0-21swapper/311:08:073
647399133,5cyclictest31830-21ssh10:36:243
647299137,4cyclictest22164-21ls12:17:442
647299135,5cyclictest0-21swapper/210:53:222
647299135,5cyclictest0-21swapper/210:53:222
647299134,8cyclictest0-21swapper/210:48:182
647299134,4cyclictest0-21swapper/208:10:012
647299131,10cyclictest11610-21cron08:32:172
647199137,4cyclictest20687-21tr12:17:321
647199137,4cyclictest14147-21apt-get12:12:171
647199134,8cyclictest0-21swapper/110:58:491
647199134,8cyclictest0-21swapper/110:45:411
647199134,8cyclictest0-21swapper/110:25:311
647199134,8cyclictest0-21swapper/110:18:251
647199134,8cyclictest0-21swapper/110:11:531
647199130,7cyclictest21578-21ssh11:22:361
647099134,4cyclictest0-21swapper/010:42:380
647099134,4cyclictest0-21swapper/009:04:300
142132130,0sleep10-21swapper/110:47:551
647399125,5cyclictest3536-21latency_hist09:17:173
647399125,5cyclictest3536-21latency_hist09:17:173
647399124,6cyclictest1261-21missed_timers12:27:463
647399124,6cyclictest1261-21missed_timers12:27:463
647399123,8cyclictest0-21swapper/311:04:163
647399123,4cyclictest14796-21ssh10:22:093
647399121,6cyclictest9294-21ssh10:44:033
647399121,5cyclictest24917-21/usr/sbin/munin10:57:403
647299128,3cyclictest0-21swapper/209:37:252
647299126,4cyclictest10614-21head09:22:352
647299124,7cyclictest0-21swapper/209:51:382
647299124,3cyclictest18225-21ssh12:14:512
647299124,2cyclictest0-21swapper/211:23:332
647299124,2cyclictest0-21swapper/209:28:132
647199127,3cyclictest441-21tr12:27:381
647199127,3cyclictest441-21tr12:27:381
647199127,3cyclictest20315-21grep09:57:441
647199126,4cyclictest16210-21cron08:42:171
647199125,6cyclictest0-21swapper/108:55:091
647199124,5cyclictest0-21swapper/112:37:451
647199124,5cyclictest0-21swapper/110:52:411
647199124,5cyclictest0-21swapper/110:52:411
647199124,5cyclictest0-21swapper/109:42:591
647199124,5cyclictest0-21swapper/109:33:211
647199124,5cyclictest0-21swapper/109:05:511
647199124,4cyclictest0-21swapper/108:19:571
647199124,2cyclictest0-21swapper/112:01:111
647199123,8cyclictest0-21swapper/111:08:471
647199123,4cyclictest11111-21grep09:22:391
647199123,4cyclictest0-21swapper/109:51:171
647199122,8cyclictest0-21swapper/108:47:401
647199122,7cyclictest23325-21cat07:47:171
647199122,5cyclictest30946-21df_inode09:12:331
647199122,5cyclictest30946-21df_inode09:12:331
647199122,5cyclictest0-21swapper/107:18:021
647099127,3cyclictest570-21runrttasks10:13:500
647099126,4cyclictest24075-21seq10:29:020
647099126,4cyclictest11485-21cron11:42:170
647099125,5cyclictest21315-21cron09:32:170
647099124,3cyclictest0-21swapper/009:15:570
647099124,3cyclictest0-21swapper/009:15:570
647099121,3cyclictest23937-21munin-run10:02:160
647099120,2cyclictest0-21swapper/011:02:430
647099120,11cyclictest0-21swapper/011:31:130
81992110,0sleep00-21swapper/012:33:110
647399116,4cyclictest27325-21kworker/u8:011:32:383
647399116,4cyclictest12420-21head07:22:353
647399116,3cyclictest32035-21apt-get11:32:163
647399116,3cyclictest18117-21sh09:57:163
647399116,3cyclictest16553-21grep09:27:353
647399115,5cyclictest0-21swapper/312:26:243
647399115,5cyclictest0-21swapper/309:07:513
647399115,4cyclictest6809-21iwlist11:37:353
647399115,4cyclictest5362-21ntpq09:17:453
647399115,4cyclictest31039-21head08:02:423
647399115,4cyclictest30057-21head11:57:353
647399115,4cyclictest0-21swapper/308:41:173
647399114,5cyclictest26268-21awk07:52:333
647399114,5cyclictest0-21swapper/310:02:323
647399113,4cyclictest5960-21tr08:17:383
647399112,4cyclictest0-21swapper/309:57:363
647399111,5cyclictest0-21swapper/307:32:353
647399110,6cyclictest32766-21ssh10:08:413
647299115,5cyclictest0-21swapper/207:42:342
647299115,4cyclictest8360-21cat08:22:432
647299115,4cyclictest589-21lldpd11:08:072
647299115,4cyclictest27503-21expr10:32:352
647299115,4cyclictest22123-21tr08:52:362
647299115,4cyclictest18860-21tr08:47:352
647299115,4cyclictest1482-21grep12:27:492
647299115,4cyclictest1482-21grep12:27:492
647299115,4cyclictest0-21swapper/212:40:462
647299114,6cyclictest0-21swapper/212:24:532
647299114,5cyclictest28323-21wc09:07:372
647299114,5cyclictest0-21swapper/208:40:202
647299114,2cyclictest141rcu_preempt11:12:402
647299114,1cyclictest0-21swapper/210:26:532
647299113,7cyclictest0-21swapper/208:59:512
647299113,6cyclictest0-21swapper/208:00:512
647299113,2cyclictest0-21swapper/212:07:432
647299113,2cyclictest0-21swapper/212:03:432
647299113,2cyclictest0-21swapper/211:53:372
647299113,2cyclictest0-21swapper/211:39:492
647299113,2cyclictest0-21swapper/211:35:312
647299113,2cyclictest0-21swapper/209:59:292
647299113,2cyclictest0-21swapper/209:32:412
647299112,4cyclictest15871-21chrt07:28:202
647299111,5cyclictest7933-21ssh09:20:162
647299110,5cyclictest16621-21ssh11:46:242
647199116,4cyclictest0-21swapper/107:32:171
647199115,5cyclictest0-21swapper/108:11:551
647199115,4cyclictest16606-21grep12:12:521
647199115,3cyclictest0-21swapper/111:45:171
647199115,3cyclictest0-21swapper/108:33:141
647199114,5cyclictest31633-21grep11:02:541
647199114,5cyclictest232-21systemd-journal10:13:491
647199114,5cyclictest11628-21latency_hist08:32:171
647199114,5cyclictest0-21swapper/111:55:051
647199113,5cyclictest0-21swapper/109:09:511
647199113,5cyclictest0-21swapper/108:05:591
647199113,4cyclictest0-21swapper/107:47:371
647199113,3cyclictest12255-21df07:22:331
647199111,5cyclictest57952sh12:32:181
647199111,5cyclictest30083-21diskmemload10:27:591
647099119,1cyclictest0-21swapper/008:32:410
647099116,4cyclictest10927-21awk07:17:460
647099116,3cyclictest30198-21munin-plugin-st08:02:170
647099116,3cyclictest232-21systemd-journal11:47:170
647099116,3cyclictest17103-21cut08:42:330
647099115,4cyclictest19061-21grep08:47:360
647099114,5cyclictest581-21http09:42:170
647099113,6cyclictest24927-21cut10:02:390
647099113,3cyclictest14829-21apt-get11:17:310
647099112,3cyclictest0-21swapper/008:24:320
647099111,7cyclictest570-21runrttasks12:17:320
647099111,6cyclictest0-21swapper/009:48:220
647099111,5cyclictest2644-21ssh10:11:380
647099111,2cyclictest0-21swapper/011:22:210
647399108,1cyclictest0-21swapper/310:31:303
647399107,2cyclictest570-21runrttasks09:02:303
647399107,2cyclictest3836-21cpuspeed_turbos10:12:323
647399106,2cyclictest6831-21sh09:47:223
647399106,2cyclictest24090-21sh11:52:373
647399105,4cyclictest43450irq/33-enp3s0-r07:47:183
647399105,4cyclictest0-21swapper/311:47:263
647399105,3cyclictest9486-21/usr/sbin/munin11:12:383
647399105,3cyclictest18988-21tr07:37:323
647399105,3cyclictest0-21swapper/308:29:353
647399105,3cyclictest0-21swapper/307:42:303
647399104,5cyclictest0-21swapper/312:37:313
647399104,5cyclictest0-21swapper/308:07:343
647399104,4cyclictest28282-21cpu07:57:343
647399104,1cyclictest0-21swapper/309:23:013
647399103,5cyclictest575-21snmpd11:26:263
647399103,5cyclictest23819-21df_inode08:57:333
647399103,5cyclictest13130-21systemctl08:32:403
647399102,4cyclictest7266-21apt-get08:22:323
647399101,7cyclictest19229-21diskstats08:47:373
647399100,8cyclictest10255-21if_enp4s007:17:393
647399100,4cyclictest27874-21cpuspeed_turbos09:37:363
647299107,2cyclictest29763-21ssh11:57:332
647299107,2cyclictest24838-21python07:47:412
647299107,2cyclictest2098-21ssh09:15:312
647299107,2cyclictest2098-21ssh09:15:312
647299105,4cyclictest0-21swapper/210:42:332
647299105,4cyclictest0-21swapper/210:40:512
647299105,4cyclictest0-21swapper/210:10:472
647299105,3cyclictest19011-21cut07:37:322
647299105,3cyclictest12867-21tail07:22:392
647299104,5cyclictest0-21swapper/207:54:362
647299104,4cyclictest5529-21tail08:17:342
647299104,4cyclictest31021-21head08:02:422
647299104,4cyclictest27108-21cut11:27:342
647299104,4cyclictest16911-21df08:42:322
647299103,5cyclictest0-21swapper/207:35:152
647299102,2cyclictest0-21swapper/210:58:592
647299101,7cyclictest0-21swapper/208:32:342
647299101,7cyclictest0-21swapper/207:17:332
647199104,5cyclictest0-21swapper/107:52:231
647199103,5cyclictest575-21snmpd08:22:291
647199103,5cyclictest17368-21latency07:32:371
647199103,5cyclictest0-21swapper/108:57:351
647199101,1cyclictest0-21swapper/107:38:571
647099106,3cyclictest0-21swapper/009:08:000
647099106,2cyclictest25447-21sh09:02:170
647099106,2cyclictest2379-21munin-run08:12:160
647099106,2cyclictest20026-21sh10:52:530
647099106,2cyclictest20026-21sh10:52:530
647099105,4cyclictest3842-21interrupts12:02:400
647099105,4cyclictest0-21swapper/012:27:270
647099105,4cyclictest0-21swapper/012:27:270
647099105,3cyclictest28052-21kernelversion10:32:410
647099105,3cyclictest25711-21latency_hist07:52:170
647099105,3cyclictest24959-21wc11:52:450
647099105,3cyclictest0-21swapper/009:44:160
647099105,3cyclictest0-21swapper/007:41:150
647099104,5cyclictest0-21swapper/008:52:340
647099104,4cyclictest597-21gdbus09:32:450
647099104,4cyclictest29277-21cron11:02:170
647099104,4cyclictest25267-21gdbus11:32:170
647099104,4cyclictest232-21systemd-journal08:30:140
647099104,4cyclictest15537-21switchtime08:37:410
647099103,6cyclictest0-21swapper/010:49:510
647099103,5cyclictest575-21snmpd12:24:450
647099103,5cyclictest575-21snmpd12:07:480
647099103,5cyclictest575-21snmpd10:17:320
647099102,5cyclictest30083-21diskmemload11:57:180
647099102,5cyclictest13043-21ssh12:37:460
647099101,7cyclictest11229-21ssh11:13:160
647099101,4cyclictest0-21swapper/007:42:480
64739994,3cyclictest22049-21missed_timers08:52:363
64739993,4cyclictest575-21snmpd07:27:293
64729993,4cyclictest26077-21cat09:02:322
64719993,5cyclictest0-21swapper/110:02:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional