You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-02 - 19:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot2s.osadl.org (updated Sat Dec 02, 2023 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30004991430,1cyclictest0-21swapper/307:12:413
3000199133130,2cyclictest0-21swapper/007:30:430
1196121100,1sleep111957-21ssh10:03:051
2898821080,0sleep00-21swapper/009:18:030
29590210645,31sleep30-21swapper/307:10:503
30003999997,1cyclictest0-21swapper/207:30:432
2890129841,43sleep20-21swapper/207:08:122
2960729047,31sleep00-21swapper/007:11:030
2762728846,31sleep10-21swapper/107:07:431
75552820,0sleep30-21swapper/310:28:163
30004997977,1cyclictest0-21swapper/307:30:433
149442750,1sleep30-21swapper/312:03:443
30002997062,7cyclictest0-21swapper/107:12:411
87692280,1sleep00-21swapper/012:30:540
34482280,1sleep20-21swapper/210:25:072
3000399281,1cyclictest0-21swapper/207:47:392
135372280,1sleep10-21swapper/112:35:161
81252270,1sleep00-21swapper/009:28:130
3000199267,18cyclictest0-21swapper/010:28:010
212502260,1sleep20-21swapper/207:58:142
105352250,0sleep10-21swapper/107:38:011
3000399230,21cyclictest5888-21ssh10:28:012
3000399230,1cyclictest0-21swapper/210:33:022
210362230,1sleep00-21swapper/009:08:050
3000499220,21cyclictest0-21swapper/308:32:573
30002992119,1cyclictest0-21swapper/109:42:531
168342200,0sleep10-21swapper/109:37:361
3000299194,6cyclictest0-21swapper/111:03:051
3000199194,8cyclictest28131-21perf11:17:370
3000299188,2cyclictest28799-21grep09:48:011
3000299185,5cyclictest16159-21tail10:08:001
3000399170,16cyclictest0-21swapper/209:27:592
3000299179,1cyclictest5380-21apt-get07:27:541
3000299175,4cyclictest0-21swapper/108:35:341
116082170,0sleep30-21swapper/310:33:013
3000299160,15cyclictest0-21swapper/108:18:001
3000199162,7cyclictest17584-21munin-run12:07:370
3000499154,6cyclictest34-21ksoftirqd/311:14:073
3000299152,5cyclictest0-21swapper/112:03:531
3000199155,8cyclictest8146-21munin-run08:42:370
178112150,2sleep3331rcuc/311:08:043
3000499148,4cyclictest6592-21ssh09:57:443
3000499148,4cyclictest27626-21taskset12:16:513
3000499145,6cyclictest0-21swapper/310:08:283
3000499141,9cyclictest0-21swapper/312:09:083
3000399144,5cyclictest0-21swapper/212:33:542
3000299144,6cyclictest0-21swapper/109:40:401
3000299144,5cyclictest23187-21ssh11:13:031
3000299144,5cyclictest0-21swapper/109:28:061
3000199147,5cyclictest1098-21cron08:27:370
3000199145,4cyclictest20709-21ssh12:09:230
3000199143,7cyclictest0-21swapper/009:48:030
61732130,4sleep06187-21ssh09:57:280
3000499136,5cyclictest16538-21kernelversion12:38:123
3000499135,6cyclictest0-21swapper/311:03:003
3000499134,4cyclictest4356-21ssh12:27:273
3000399133,5cyclictest0-21swapper/209:41:102
3000399132,6cyclictest10249-21ssh11:59:412
3000299135,3cyclictest0-21swapper/108:00:241
3000299133,5cyclictest17151-21ssh11:07:451
3000199138,3cyclictest11057-21tr08:47:590
3000199136,5cyclictest32060-21cron10:22:360
3000199136,5cyclictest240-21systemd-journal08:57:370
3000199134,7cyclictest19888-21munin-run09:07:370
3000199134,4cyclictest0-21swapper/010:08:040
3000199134,4cyclictest0-21swapper/009:41:330
3000199131,7cyclictest0-21swapper/007:17:370
3000199130,9cyclictest20940-21ssh11:10:330
3000199130,7cyclictest14528-21awk07:47:370
299242130,1chrt0-21swapper/110:49:351
3000499128,3cyclictest0-21swapper/311:47:353
3000499127,3cyclictest0-21swapper/309:14:483
3000499125,5cyclictest240-21systemd-journal10:42:373
3000499124,6cyclictest0-21swapper/308:16:223
3000499124,3cyclictest9248-21ssh09:29:333
3000499123,4cyclictest0-21swapper/310:12:463
3000499122,5cyclictest0-21swapper/308:58:043
3000499121,9cyclictest12241-21turbostat.cron12:02:373
3000499121,6cyclictest7431-21ssh12:28:513
3000499121,4cyclictest23018-21grep10:43:063
3000399129,2cyclictest0-21swapper/211:54:522
3000399129,2cyclictest0-21swapper/210:48:422
3000399126,4cyclictest4314-21head08:33:012
3000399125,5cyclictest0-21swapper/209:49:132
3000299128,2cyclictest12231-21cron12:02:371
3000299126,4cyclictest13450-21sed10:33:181
3000299124,7cyclictest0-21swapper/111:36:581
3000299124,7cyclictest0-21swapper/108:43:591
3000299124,6cyclictest6555-1kworker/u9:209:13:001
3000299124,3cyclictest0-21swapper/108:13:071
3000299123,8cyclictest0-21swapper/110:52:421
3000299123,4cyclictest0-21swapper/108:23:441
3000299122,8cyclictest0-21swapper/110:43:441
3000299121,6cyclictest0-21swapper/110:23:291
3000299121,4cyclictest24116-21sort12:13:061
3000299120,10cyclictest14503-21apt-get07:47:371
3000199124,3cyclictest0-21swapper/007:13:060
3000199122,6cyclictest0-21swapper/010:48:040
3000199122,5cyclictest0-21swapper/008:03:570
3000199120,10cyclictest5493-21perf10:57:370
3000499116,4cyclictest17763-21kworker/3:110:25:443
3000499116,4cyclictest17763-21kworker/3:109:05:363
3000499116,3cyclictest22220-21diskmemload11:48:063
3000499115,5cyclictest17763-21kworker/3:111:00:563
3000499115,5cyclictest17763-21kworker/3:109:36:103
3000499115,4cyclictest28651-21aten2.4-expect11:17:593
3000499115,4cyclictest2737-21grep11:23:043
3000499115,4cyclictest240-21systemd-journal12:22:373
3000499115,4cyclictest240-21systemd-journal09:07:373
3000499115,4cyclictest13403-21tr08:52:593
3000499115,4cyclictest0-21swapper/307:38:453
3000499114,6cyclictest0-21swapper/310:03:103
3000499114,6cyclictest0-21swapper/308:08:273
3000499114,5cyclictest514-21dbus-daemon08:03:083
3000499112,6cyclictest4277-21grep07:23:113
3000499112,4cyclictest0-21swapper/307:52:003
3000499111,9cyclictest0-21swapper/309:52:113
3000499111,8cyclictest27715-21perf09:47:373
3000499111,5cyclictest0-21swapper/307:58:043
3000499111,2cyclictest0-21swapper/312:33:143
3000499110,9cyclictest3455-21dump-pmu-power08:32:373
3000399118,2cyclictest11419-21sed08:48:022
3000399117,2cyclictest25455-21fschecks_count11:43:032
3000399117,2cyclictest10485-21cron08:47:362
3000399116,4cyclictest514-21dbus-daemon12:23:022
3000399116,4cyclictest17031-21sed08:58:132
3000399116,3cyclictest7056-21latency10:58:112
3000399116,3cyclictest17599-21idleruntime-cro12:07:372
3000399115,5cyclictest0-21swapper/212:14:442
3000399115,5cyclictest0-21swapper/208:38:522
3000399115,5cyclictest0-21swapper/207:13:292
3000399115,4cyclictest722-21awk08:23:512
3000399115,4cyclictest514-21dbus-daemon10:22:362
3000399115,4cyclictest15352-21http10:07:372
3000399115,4cyclictest12706-21ssh11:32:102
3000399115,4cyclictest0-21swapper/210:13:312
3000399114,5cyclictest547-21snmpd12:28:452
3000399114,5cyclictest514-21dbus-daemon12:38:162
3000399114,5cyclictest3499-21cat11:52:382
3000399114,5cyclictest20441-21ssh12:08:592
3000399114,5cyclictest20332-21cut11:38:092
3000399114,5cyclictest0-21swapper/211:16:582
3000399114,5cyclictest0-21swapper/209:36:362
3000399114,5cyclictest0-21swapper/209:16:182
3000399114,5cyclictest0-21swapper/208:14:282
3000399114,4cyclictest0-21swapper/209:17:542
3000399112,8cyclictest0-21swapper/208:12:132
3000399110,9cyclictest31214-21sendmail-msp08:22:372
3000399110,9cyclictest19894-21idleruntime-cro09:07:372
3000299116,3cyclictest4619-21if_enp3s011:53:071
3000299116,3cyclictest4251-21head07:23:111
3000299116,3cyclictest12165-21switchtime08:48:111
3000299115,5cyclictest0-21swapper/112:10:411
3000299115,4cyclictest6722-21grep08:38:021
3000299115,4cyclictest25430-21grep11:43:031
3000299115,4cyclictest240-21systemd-journal11:39:081
3000299114,6cyclictest0-21swapper/110:18:421
3000299114,5cyclictest1877-21tune2fs08:28:011
3000299114,5cyclictest0-21swapper/112:29:061
3000299114,5cyclictest0-21swapper/110:38:231
3000299114,5cyclictest0-21swapper/110:31:201
3000299114,5cyclictest0-21swapper/108:59:411
3000299114,2cyclictest0-21swapper/111:27:461
3000299113,5cyclictest0-21swapper/108:07:451
3000299113,2cyclictest0-21swapper/111:47:381
3000299113,2cyclictest0-21swapper/109:52:461
3000299112,5cyclictest547-21snmpd09:25:031
3000199117,2cyclictest869-21open_files12:23:010
3000199117,2cyclictest20227-21latency_hist12:42:370
3000199116,4cyclictest0-21swapper/007:53:180
3000199116,3cyclictest26522-21latency_hist10:47:370
3000199116,3cyclictest23648-21sh09:13:040
3000199116,3cyclictest22220-21diskmemload11:38:110
3000199116,3cyclictest20824-21munin-plugin-st10:42:370
3000199116,3cyclictest11571-21sh09:32:380
3000199115,4cyclictest7970-21grep07:33:000
3000199115,4cyclictest30373-21sed11:18:140
3000199115,4cyclictest28121-21cron12:17:370
3000199115,4cyclictest16105-21sed08:58:030
3000199114,5cyclictest12232-21cron12:02:370
3000199113,6cyclictest884-21ssh09:22:450
3000199113,6cyclictest15905-21ssh10:36:260
3000199113,3cyclictest0-21swapper/009:45:310
3000199112,4cyclictest0-21swapper/012:37:270
3000199111,7cyclictest0-21swapper/008:27:570
3000199111,5cyclictest0-21swapper/011:26:580
221482110,1chrt22-21ksoftirqd/108:02:581
200512110,5sleep228-21ksoftirqd/210:11:322
3000499106,3cyclictest31156-21kworker/3:008:47:563
3000499106,3cyclictest21784-21sh09:41:543
3000499106,3cyclictest17763-21kworker/3:111:37:523
3000499106,3cyclictest17763-21kworker/3:109:17:523
3000499106,2cyclictest29076-21sh08:18:013
3000499105,4cyclictest31156-21kworker/3:008:43:463
3000499105,4cyclictest27278-21kworker/3:107:42:523
3000499105,4cyclictest27278-21kworker/3:107:32:483
3000499105,3cyclictest7690-21grep08:38:123
3000499105,3cyclictest0-21swapper/308:22:433
3000499104,4cyclictest32502-21cut07:17:373
3000499104,4cyclictest32462-21ssh10:52:393
3000499104,4cyclictest27698-21/usr/sbin/munin10:48:023
3000499103,5cyclictest547-21snmpd11:32:583
3000499103,5cyclictest27514-21grep10:18:013
3000499102,4cyclictest0-21swapper/309:53:493
3000499102,3cyclictest0-21swapper/309:23:053
3000499101,3cyclictest17784-21df_inode07:52:593
3000499101,2cyclictest0-21swapper/311:28:133
3000499100,8cyclictest4442-21gltestperf11:53:063
3000399105,3cyclictest29782-21needreboot11:18:082
3000399104,5cyclictest0-21swapper/212:20:242
3000399104,5cyclictest0-21swapper/211:09:092
3000399104,5cyclictest0-21swapper/211:06:392
3000399104,5cyclictest0-21swapper/210:39:352
3000399104,5cyclictest0-21swapper/209:42:582
3000399104,5cyclictest0-21swapper/209:26:422
3000399104,5cyclictest0-21swapper/209:10:572
3000399104,5cyclictest0-21swapper/207:38:032
3000399104,5cyclictest0-21swapper/207:35:212
3000399104,4cyclictest4770-21vmstat07:23:162
3000399104,4cyclictest1737-21cpuspeed_turbos09:53:052
3000399104,4cyclictest13553-21basename11:32:582
3000399101,3cyclictest22909-21irqstats08:03:042
3000399101,1cyclictest0-21swapper/209:59:322
3000399100,9cyclictest0-21swapper/210:45:122
3000299105,4cyclictest0-21swapper/112:19:451
3000299105,3cyclictest17982-21gltestperf07:53:011
3000299104,5cyclictest0-21swapper/110:58:001
3000299104,5cyclictest0-21swapper/108:52:411
3000299104,5cyclictest0-21swapper/107:32:441
3000299104,4cyclictest22696-21pmu-power10:13:091
3000299103,6cyclictest0-21swapper/111:17:591
3000299103,5cyclictest547-21snmpd12:42:121
3000299103,5cyclictest547-21snmpd09:58:191
3000299103,5cyclictest14966-21acpi07:48:011
3000299103,4cyclictest547-21snmpd09:07:081
3000299102,5cyclictest514-21dbus-daemon07:17:371
3000299101,8cyclictest0-21swapper/112:23:341
3000299101,7cyclictest0-21swapper/109:08:001
3000299101,5cyclictest1845-21ssh11:22:391
3000299100,9cyclictest0-21swapper/109:17:441
3000199106,3cyclictest530-21runrttasks12:19:000
3000199106,3cyclictest530-21runrttasks07:26:010
3000199106,3cyclictest12-21ksoftirqd/010:17:370
3000199106,2cyclictest32456-21sh10:22:490
3000199106,2cyclictest31025-21df11:48:000
3000199106,2cyclictest14926-21gdbus07:47:380
3000199105,4cyclictest10942-21cut10:02:570
3000199105,3cyclictest8920-21cut08:43:010
3000199105,3cyclictest12312-21ssh11:03:070
3000199105,3cyclictest0-21swapper/010:57:540
3000199104,4cyclictest3867-21tail08:32:590
3000199104,4cyclictest24518-21aten2.4-expect08:07:580
3000199104,2cyclictest0-21swapper/007:58:120
3000199103,5cyclictest547-21snmpd10:00:130
3000199100,8cyclictest45-21kcompactd011:52:380
300039995,2cyclictest324-21grep07:17:382
300039994,4cyclictest13065-21gltestperf07:43:012
300039994,3cyclictest2421-21tr11:23:032
300039994,3cyclictest2048-21/usr/sbin/munin08:28:022
300039993,4cyclictest547-21snmpd07:53:032
300039993,4cyclictest14236-21awk08:53:072
300039992,5cyclictest0-21swapper/210:52:452
300019995,3cyclictest0-21swapper/011:42:400
300019995,3cyclictest0-21swapper/008:12:580
300019995,2cyclictest10577-21fschecks_count07:38:020
300019994,3cyclictest7888-21sh11:28:020
300019993,4cyclictest29894-21cat08:18:070
300019992,5cyclictest14163-21cut11:33:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional