You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-11-28 - 10:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot2s.osadl.org (updated Tue Nov 28, 2023 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2820399148140,7cyclictest0-21swapper/119:27:491
2820299124122,1cyclictest0-21swapper/019:27:490
27707211145,32sleep10-21swapper/119:09:411
27844211045,31sleep20-21swapper/219:11:272
170421090,0sleep30-21swapper/322:22:563
27813210744,30sleep00-21swapper/019:11:040
12122970,1sleep20-21swapper/223:19:072
12122970,1sleep20-21swapper/223:19:072
2764029148,32sleep30-21swapper/319:08:513
28205999086,1cyclictest0-21swapper/319:27:493
28203998779,7cyclictest0-21swapper/119:12:401
236552860,0sleep00-21swapper/022:42:330
205192860,2sleep320518-21hddtemp_smartct23:08:023
263772830,0sleep20-21swapper/222:43:562
40372690,0sleep00-21swapper/022:24:060
28202996361,1cyclictest0-21swapper/019:12:400
28205993720,16cyclictest0-21swapper/320:12:533
28205993025,4cyclictest0-21swapper/319:12:403
162302290,4sleep228-21ksoftirqd/222:35:222
121402290,0sleep30-21swapper/323:58:003
28204992725,1cyclictest0-21swapper/219:27:492
293522260,1sleep20-21swapper/223:15:582
2820299241,21cyclictest547-21snmpd20:02:540
5102230,1sleep221018-21diskmemload23:48:012
28203992220,1cyclictest0-21swapper/123:37:501
28203992220,1cyclictest0-21swapper/119:42:531
28204992018,1cyclictest0-21swapper/221:32:362
22552200,1sleep30-21swapper/321:23:083
2820299171,8cyclictest13772-21basename20:57:330
2820399161,9cyclictest6806-21pmu-power23:53:041
2820299161,8cyclictest27199-21munin-run20:17:330
2820499155,5cyclictest0-21swapper/220:28:342
2820299155,5cyclictest15726-21ssh21:35:360
2820299151,7cyclictest21579-21munin-run00:07:330
2820599147,5cyclictest20548-21grep21:08:093
2820599144,8cyclictest9129-21ssh23:55:313
2820599144,5cyclictest0-21swapper/321:46:423
2820599144,5cyclictest0-21swapper/321:46:423
2820599144,5cyclictest0-21swapper/320:33:043
2820599144,5cyclictest0-21swapper/319:58:063
2820599144,5cyclictest0-21swapper/300:17:463
2820499140,8cyclictest11806-21cpuspeed_turbos22:02:572
2820399149,3cyclictest27190-21latency_hist20:17:331
2820399149,3cyclictest23506-21tail21:13:071
2820399147,5cyclictest11328-21cron21:32:331
2820399141,8cyclictest20279-21ssh23:37:061
2820399140,12cyclictest29261-21memory22:18:041
2820399140,10cyclictest23985-21ssh00:39:301
25052140,5sleep122-21ksoftirqd/123:49:081
2820599134,5cyclictest0-21swapper/322:00:363
2820599134,4cyclictest0-21swapper/300:03:523
2820599133,5cyclictest0-21swapper/323:34:123
2820499138,3cyclictest0-21swapper/221:39:092
2820499137,4cyclictest6822-21head21:27:582
2820499135,6cyclictest27417-21grep23:42:582
2820499135,3cyclictest0-21swapper/222:12:162
2820499134,7cyclictest30240-21ssh21:19:382
2820499133,5cyclictest0-21swapper/219:17:592
2820499132,9cyclictest24898-21ssh00:40:502
2820499132,6cyclictest29647-21ssh22:47:562
2820499132,6cyclictest12527-21if_err_enp4s020:53:012
2820499131,5cyclictest21018-21diskmemload23:41:422
2820399135,7cyclictest7070-21aten2.4_r8power20:42:541
2820399135,7cyclictest0-21swapper/123:21:121
2820399135,7cyclictest0-21swapper/123:21:121
2820399135,7cyclictest0-21swapper/121:02:381
2820399134,4cyclictest0-21swapper/121:11:591
2820299138,3cyclictest0-21swapper/019:27:290
2820299137,4cyclictest547-21snmpd00:26:050
2820299135,3cyclictest0-21swapper/022:12:370
2820299134,8cyclictest0-21swapper/023:05:290
2820299134,6cyclictest0-21swapper/019:47:380
2820299133,5cyclictest0-21swapper/021:57:000
60882120,4sleep228-21ksoftirqd/222:26:522
2820599128,2cyclictest11146-21grep19:42:543
2820599126,4cyclictest11428-21head00:27:593
2820599125,5cyclictest32272-21ssh22:49:173
2820599125,5cyclictest0-21swapper/322:07:223
2820599125,4cyclictest0-21swapper/322:43:413
2820599124,7cyclictest0-21swapper/322:13:373
2820599124,7cyclictest0-21swapper/300:23:103
2820599123,7cyclictest240-21systemd-journal22:27:533
2820599123,4cyclictest1162-21ntp_states23:48:063
2820599122,5cyclictest0-21swapper/322:18:023
2820599120,3cyclictest1751-21ssh23:19:513
2820599120,3cyclictest1751-21ssh23:19:513
2820599120,10cyclictest10177-21sed19:38:093
2820499128,3cyclictest0-21swapper/222:39:192
2820499126,4cyclictest240-21systemd-journal19:17:282
2820499125,5cyclictest0-21swapper/221:53:482
2820499124,6cyclictest1891-21processes21:23:062
2820499124,6cyclictest0-21swapper/222:29:322
2820499124,6cyclictest0-21swapper/222:00:022
2820499123,4cyclictest28806-21ls21:48:062
2820499122,5cyclictest912-21ssh22:22:332
2820499122,5cyclictest0-21swapper/200:13:072
2820499121,2cyclictest0-21swapper/223:07:592
2820499120,1cyclictest0-21swapper/221:42:472
2820499120,1cyclictest0-21swapper/221:42:472
2820399128,2cyclictest11616-21cpuspeed_turbos23:57:561
2820399127,4cyclictest0-21swapper/121:25:501
2820399126,4cyclictest23474-21grep00:08:061
2820399125,6cyclictest0-21swapper/123:02:061
2820399125,6cyclictest0-21swapper/122:43:461
2820399125,6cyclictest0-21swapper/122:15:381
2820399125,5cyclictest23569-21ssh21:43:151
2820399125,5cyclictest23569-21ssh21:43:151
2820399125,5cyclictest23404-21cron22:42:331
2820399125,5cyclictest0-21swapper/119:23:411
2820399124,7cyclictest203-21jbd2/sda1-822:03:261
2820399124,7cyclictest0-21swapper/120:48:041
2820399123,4cyclictest0-21swapper/123:43:001
2820399123,4cyclictest0-21swapper/123:06:021
2820399121,6cyclictest0-21swapper/120:39:191
2820299128,2cyclictest21587-21grep00:37:590
2820299124,3cyclictest21018-21diskmemload22:08:400
2820299123,4cyclictest30831-21ssh21:20:260
2820299123,4cyclictest0-21swapper/019:41:470
2820299122,5cyclictest17043-21ssh22:36:280
2820299122,5cyclictest14174-21ssh00:30:290
2820299121,2cyclictest0-21swapper/022:49:070
254022120,0sleep20-21swapper/221:15:382
2820599117,3cyclictest40550irq/33-enp3s0-r23:28:083
2820599117,2cyclictest21099-21turbostat.cron21:42:333
2820599116,4cyclictest28416-21awk21:48:023
2820599116,3cyclictest18683-21idleruntime-cro21:07:333
2820599116,3cyclictest18255-21ntp_kernel_pll_22:08:033
2820599115,5cyclictest26265-21ls23:42:343
2820599115,5cyclictest0-21swapper/321:54:543
2820599115,4cyclictest0-21swapper/322:53:263
2820599115,4cyclictest0-21swapper/321:31:293
2820599114,6cyclictest22982-21ssh22:41:593
2820599114,5cyclictest21018-21diskmemload23:00:233
2820599114,5cyclictest0-21swapper/323:44:483
2820599114,4cyclictest0-21swapper/321:17:383
2820599113,5cyclictest0-21swapper/300:11:253
2820599112,4cyclictest0-21swapper/319:17:343
2820599111,5cyclictest17573-21ssh23:05:143
2820499117,3cyclictest26323-21kworker/u8:022:52:562
2820499116,4cyclictest547-21snmpd00:22:062
2820499116,4cyclictest15497-21apt-get23:32:552
2820499115,5cyclictest0-21swapper/219:24:262
2820499115,5cyclictest0-21swapper/200:02:102
2820499115,4cyclictest17586-21cron19:57:332
2820499114,6cyclictest0-21swapper/223:01:272
2820499114,5cyclictest0-21swapper/219:50:482
2820499114,5cyclictest0-21swapper/200:03:042
2820499113,6cyclictest22139-21aten2.4-expect00:07:562
2820499113,2cyclictest0-21swapper/223:53:382
2820499113,2cyclictest0-21swapper/223:06:022
2820399116,4cyclictest607-21gdbus00:22:331
2820399115,5cyclictest7556-1kworker/u9:222:33:011
2820399115,5cyclictest0-21swapper/119:51:421
2820399115,4cyclictest5864-21tr00:22:591
2820399115,4cyclictest521-21systemd-logind22:53:051
2820399115,4cyclictest29722-21cat20:22:341
2820399115,4cyclictest28669-21head00:13:101
2820399115,4cyclictest0-21swapper/120:09:541
2820399114,5cyclictest6272-21grep21:57:561
2820399114,1cyclictest0-21swapper/100:05:521
2820399113,7cyclictest1624-21grep22:22:561
2820399113,6cyclictest612-21rs:main1
2820399113,6cyclictest24802-21cat23:12:281
2820399111,9cyclictest0-21swapper/122:07:341
2820299117,2cyclictest263002sleep023:13:050
2820299116,3cyclictest6710-21sed21:57:580
2820299116,3cyclictest6011-21apt-key19:32:340
2820299116,3cyclictest30568-21sh22:18:520
2820299116,3cyclictest29742-21sh23:44:420
2820299116,3cyclictest2503-21sh00:18:460
2820299116,3cyclictest24368-21sh00:08:520
2820299116,3cyclictest12731-21sendmail_mailst23:58:060
2820299115,4cyclictest9639-21grep20:47:560
2820299115,4cyclictest514-21dbus-daemon19:17:330
2820299115,4cyclictest32677-21grep23:47:590
2820299115,4cyclictest30342-21grep20:22:560
2820299115,4cyclictest27742-21grep20:17:550
2820299115,3cyclictest0-21swapper/020:57:580
2820299114,6cyclictest15019-21perf23:32:330
2820299114,5cyclictest547-21snmpd23:41:030
2820299114,5cyclictest0-21swapper/023:07:380
2820299113,5cyclictest0-21swapper/022:28:250
2820299113,5cyclictest0-21swapper/021:29:050
2820299112,4cyclictest0-21swapper/023:18:000
2820299112,4cyclictest0-21swapper/023:18:000
2820299112,4cyclictest0-21swapper/023:01:050
2820299111,8cyclictest0-21swapper/021:47:330
2820299111,8cyclictest0-21swapper/021:47:330
2820299111,8cyclictest0-21swapper/019:47:330
2820299111,6cyclictest15501-21sh23:32:550
2820299111,4cyclictest7870-21cron22:57:330
2820299110,9cyclictest21007-21grep21:12:330
2820599106,3cyclictest27614-21cpu20:17:553
2820599106,2cyclictest5968-21mailstats20:38:103
2820599106,2cyclictest21018-21diskmemload00:14:483
2820599105,4cyclictest21954-21sort20:03:073
2820599105,4cyclictest0-21swapper/319:55:003
2820599105,3cyclictest13515-21sendmail_mailtr21:33:073
2820599104,5cyclictest547-21snmpd22:36:343
2820599104,5cyclictest0-21swapper/320:27:463
2820599104,4cyclictest7904-21kernelversion20:43:003
2820599104,4cyclictest16262-21wc21:02:333
2820599104,2cyclictest21282-21sh00:37:573
2820599103,5cyclictest23030-21grep20:07:563
2820599103,2cyclictest17237-21ntpq00:33:043
2820599102,4cyclictest0-21swapper/320:47:593
2820599101,5cyclictest23048-21/usr/sbin/munin21:13:043
2820599100,8cyclictest5594-21grep23:23:053
2820599100,8cyclictest30163-21aten2.4_r8power20:22:563
2820599100,8cyclictest12757-21needreboot20:53:033
2820499106,3cyclictest7070-21aten2.4_r8power20:42:542
2820499106,3cyclictest11688-21grep23:28:112
2820499105,3cyclictest9136-21gltestperf19:38:002
2820499105,3cyclictest6867-21wc00:23:082
2820499105,3cyclictest6627-21sh23:23:472
2820499105,3cyclictest0-21swapper/221:07:412
2820499104,4cyclictest607-21gdbus20:58:072
2820499104,4cyclictest20425-21apt-get20:02:542
2820499104,3cyclictest0-21swapper/220:19:092
2820499104,2cyclictest0-21swapper/219:59:032
2820499103,6cyclictest0-21swapper/219:45:072
2820499103,5cyclictest240-21systemd-journal20:42:322
2820499101,7cyclictest9392-21telnet20:47:552
2820499101,7cyclictest11500-21cut00:27:592
2820499101,7cyclictest0-21swapper/220:12:542
2820499101,4cyclictest0-21swapper/222:16:302
2820499100,9cyclictest0-21swapper/200:33:082
2820499100,4cyclictest6110-21/usr/sbin/munin19:33:042
2820499100,4cyclictest2352-21telnet20:32:542
2820399105,3cyclictest20764-21munin-run00:37:331
2820399105,3cyclictest20660-21cpuspeed_turbos20:02:561
2820399105,3cyclictest14652-21grep20:57:571
2820399105,3cyclictest12707-21sh23:29:261
2820399105,3cyclictest0-21swapper/119:32:561
2820399104,4cyclictest521-21systemd-logind21:18:061
2820399103,5cyclictest547-21snmpd20:22:361
2820399103,5cyclictest547-21snmpd19:57:541
2820399103,3cyclictest0-21swapper/123:22:581
2820399103,3cyclictest0-21swapper/121:53:021
2820399103,1cyclictest0-21swapper/122:28:181
2820399103,1cyclictest0-21swapper/121:48:001
2820399102,3cyclictest0-21swapper/119:55:241
2820399101,7cyclictest4366-21cron20:37:331
2820399101,7cyclictest12339-21tr21:32:571
2820399101,7cyclictest10556-21apt19:42:331
2820399101,4cyclictest0-21swapper/121:38:001
2820399101,2cyclictest0-21swapper/120:52:541
2820399100,9cyclictest28458-21ssh23:14:461
2820399100,8cyclictest451-21fschecks_time20:27:571
2820299105,3cyclictest9408-21latency_hist23:27:340
2820299105,3cyclictest23198-21getsmi21:13:060
2820299105,3cyclictest1143-21pmu-power20:28:040
2820299105,3cyclictest0-21swapper/020:01:390
2820299104,5cyclictest3646-21mailstats20:33:050
2820299104,5cyclictest0-21swapper/023:54:370
2820299104,5cyclictest0-21swapper/000:34:320
2820299104,4cyclictest547-21snmpd21:24:320
2820299104,4cyclictest514-21dbus-daemon21:48:050
2820299104,4cyclictest18428-21aten2.4_r8power22:37:560
2820299104,4cyclictest12509-21sort22:03:030
2820299103,5cyclictest547-21snmpd21:40:240
2820299103,5cyclictest547-21snmpd20:42:490
2820299103,5cyclictest547-21snmpd20:37:370
2820299101,4cyclictest0-21swapper/000:12:340
282059996,2cyclictest6199-21aten2.4_r8power19:32:533
282059991,7cyclictest0-21swapper/319:47:533
282059990,5cyclictest25093-21ssh23:12:343
282059990,5cyclictest1652-21cstates19:22:563
282049994,3cyclictest30097-21telnet20:22:552
282049993,4cyclictest22798-21head20:07:552
282049993,4cyclictest17144-21cut21:02:562
282039993,5cyclictest547-21snmpd00:27:441
282039992,5cyclictest31286-21acpi19:17:521
282029994,3cyclictest23019-21cpuspeed_turbos20:07:560
282029993,3cyclictest0-21swapper/021:02:350
282029992,5cyclictest547-21snmpd19:52:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional