You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 10:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Sat Apr 27, 2024 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
140974821040,1sleep10-21swapper/100:15:011
13614052460,1sleep10-21swapper/123:40:131
14390432300,1sleep20-21swapper/200:35:182
1008049992522,2cyclictest491-21avahi-daemon23:35:000
1008059992422,1cyclictest2609-21cron23:00:003
1008052992419,4cyclictest1081909-21cron20:10:001
1008052992320,2cyclictest0-21swapper/100:00:001
1008052992319,2cyclictest0-21swapper/121:20:001
1008052992319,2cyclictest0-21swapper/121:20:001
100804999232,20cyclictest1070108-21apt-get20:00:100
1008052992219,2cyclictest1294684-21kthreadcore22:50:211
1008052992218,2cyclictest0-21swapper/100:20:001
100804999219,6cyclictest512-21rs:main0
1008057992017,2cyclictest1166685-21basename21:20:002
1008057992017,2cyclictest1166685-21basename21:19:592
1008052992016,2cyclictest510-21in:imuxsock19:25:011
1008059991917,1cyclictest1388740-21sh00:00:003
1008057991915,2cyclictest1360971-21cron23:39:592
1008052991916,2cyclictest512-21rs:main1
1008052991916,2cyclictest0-21swapper/100:05:001
1008049991915,3cyclictest1105567-21systemd-run20:30:000
269698180,1rtkit-daemon0-21swapper/322:14:463
269698180,11rtkit-daemon1423399-21sed00:25:000
1008059991815,2cyclictest503-21polkitd19:45:013
1008057991816,1cyclictest1339859-21turbostat.cron23:24:592
1008057991815,2cyclictest1052088-21idleruntime-cro19:45:012
1008057991814,3cyclictest1346854-21sh23:30:002
1008057991814,2cyclictest1409749-21latency_hist00:15:012
1008057991814,2cyclictest1200140-21ntp_kernel_pll_21:40:232
1008052991815,2cyclictest1239383-21kthreadcore22:10:171
1008052991814,3cyclictest510-21in:imuxsock21:05:001
1008052991811,5cyclictest353-21systemd-journal21:30:001
1008059991714,2cyclictest1222452-21cron22:00:003
1008059991713,3cyclictest1088301-21fschecks_count20:15:133
1008057991713,3cyclictest1105274-21sshd20:26:482
100805299179,2cyclictest191rcu_preempt23:00:221
1008052991712,4cyclictest1375754-21sed23:50:161
1008052991712,3cyclictest1438641-21kthreadcore00:35:181
1008049991714,2cyclictest1360983-21awk23:40:000
1008049991712,4cyclictest0-21swapper/019:40:010
1008059991613,2cyclictest0-21swapper/300:35:163
1008059991612,3cyclictest1430410-21cpuspeed_turbos00:30:113
1008059991612,3cyclictest1033638-21unixbench_singl19:25:253
1008059991612,2cyclictest1147420-21fschecks_count21:05:163
1008057991614,1cyclictest1413547-21kthreadcore00:15:172
1008057991613,2cyclictest1131322-21kthreadcore20:50:182
1008057991612,3cyclictest0-21swapper/219:10:252
1008057991611,3cyclictest1265664-21kthreadcore22:30:192
100805799160,8cyclictest1436951-21sh00:35:002
100805299166,8cyclictest1436943-21munin-run00:35:001
1008052991613,2cyclictest1082161-21/usr/sbin/munin20:10:151
1008052991612,3cyclictest1092815-21ntp_kernel_err20:15:211
1008052991612,3cyclictest1008703-21sed19:10:171
1008052991612,2cyclictest0-21swapper/121:30:161
1008052991611,3cyclictest0-21swapper/123:25:181
1008049991613,2cyclictest533-21gdbus00:00:000
1008049991613,2cyclictest0-21swapper/023:45:000
1008049991613,1cyclictest0-21swapper/020:20:000
1008049991612,3cyclictest1437427-21fschecks_count00:35:140
1008049991612,3cyclictest1200632-21unixbench_singl21:40:290
1008049991610,4cyclictest1257037-21munin-run22:25:010
12070082150,3sleep0171ktimers/021:45:240
1008059991512,2cyclictest1291573-21apt22:50:013
1008059991512,2cyclictest1236262-21/usr/sbin/munin22:10:013
1008059991512,2cyclictest0-21swapper/300:05:003
1008059991511,3cyclictest1082259-21cpuspeed_turbos20:10:103
1008059991511,2cyclictest0-21swapper/320:55:003
1008059991511,2cyclictest0-21swapper/320:35:003
100805999150,4cyclictest1364997-21cut23:40:203
1008057991512,2cyclictest1332930-21idleruntime-cro23:20:002
1008057991512,2cyclictest1117561-21/usr/sbin/munin20:40:162
1008057991511,3cyclictest1070305-21fschecks_count20:00:132
1008057991511,3cyclictest0-21swapper/223:35:002
1008057991511,3cyclictest0-21swapper/200:10:002
1008057991511,2cyclictest0-21swapper/223:09:592
1008057991510,3cyclictest1057041-21needreboot19:45:182
100805299158,5cyclictest353-21systemd-journal19:50:011
100805299152,2cyclictest191rcu_preempt23:35:241
1008052991513,1cyclictest1336936-21kthreadcore23:20:201
1008052991512,2cyclictest1370830-21kthreadcore23:45:161
1008052991512,2cyclictest1326248-21/usr/sbin/munin23:15:181
1008052991512,2cyclictest1107341-21kthreadcore20:30:161
1008052991512,2cyclictest1016279-21cpuspeed_turbos19:15:111
1008052991511,3cyclictest1352447-21unixbench_multi23:30:261
100804999159,4cyclictest495-21dbus-daemon23:10:000
1008049991512,2cyclictest1099369-21kworker/0:2+kacpid21:15:390
1008049991512,2cyclictest1099369-21kworker/0:2+kacpid21:15:390
1008049991512,2cyclictest0-21swapper/019:45:010
1008049991511,3cyclictest17250irq/127-ahci[0000:00:17.0]21:15:000
1008049991511,3cyclictest1435674-21unixbench_singl00:30:260
1008049991511,3cyclictest1269591-21unixbench_singl22:30:300
1008049991511,3cyclictest1016098-21fschecks_count19:15:030
10023212153,7sleep11002354-21apt-get19:05:111
100805999149,3cyclictest1007959-21kworker/u8:3-rpciod21:30:013
100805999148,4cyclictest353-21systemd-journal19:55:013
100805999141,2cyclictest13538161idleruntime-cro23:35:003
1008059991411,2cyclictest1340237-21cpuspeed_turbos23:25:143
1008059991411,2cyclictest1333449-21fschecks_count23:20:163
1008059991411,2cyclictest0-21swapper/322:00:253
1008059991411,2cyclictest0-21swapper/321:45:153
1008059991411,2cyclictest0-21swapper/321:00:003
1008059991411,2cyclictest0-21swapper/300:20:003
1008059991410,3cyclictest1257400-21cpuspeed_turbos22:25:113
1008059991410,3cyclictest1198157-21tr21:40:213
100805999140,5cyclictest1265200-21irqrtprio22:30:183
100805999140,4cyclictest1246177-21kthreadcore22:15:163
100805799149,3cyclictest1099649-21sh20:25:002
1008057991411,2cyclictest1292339-21irqcore22:50:172
1008057991411,2cyclictest1287847-21kthreadcore22:45:182
1008057991411,2cyclictest1278367-21gltestperf22:40:142
1008057991411,2cyclictest1154117-21irqrtprio21:10:202
1008057991411,2cyclictest1152836-21diskmemload21:50:012
1008057991411,2cyclictest0-21swapper/221:30:152
1008057991410,3cyclictest1422072-21unixbench_singl00:20:322
1008057991410,3cyclictest1229391-21cron22:05:002
1008057991410,2cyclictest0-21swapper/222:00:002
1008057991410,2cyclictest0-21swapper/219:15:152
100805799140,5cyclictest1250728-21gltestperf22:20:142
100805299149,4cyclictest1046148-21apt19:40:011
100805299147,5cyclictest353-21systemd-journal21:25:001
1008052991412,1cyclictest1194955-21cron21:40:001
1008052991411,2cyclictest1271397-21gltestperf22:35:161
1008052991411,2cyclictest1075750-21idleruntime-cro20:05:011
1008052991410,3cyclictest1396233-21cpuspeed_turbos00:05:091
1008052991410,3cyclictest1278138-21cpuspeed_turbos22:40:121
1008052991410,3cyclictest1153503-21gltestperf21:10:171
1008052991410,3cyclictest1028608-21fschecks_count19:25:131
1008052991410,3cyclictest0-21swapper/122:30:171
1008052991410,2cyclictest0-21swapper/120:40:001
1008052991410,2cyclictest0-21swapper/119:45:011
1008049991412,1cyclictest1001573-21kworker/0:0+kacpid20:10:160
100804999141,12cyclictest1188037-21turbostat21:35:000
1008049991411,2cyclictest1052428-21cpuspeed_turbos19:45:100
1008049991410,3cyclictest1310960-21unixbench_multi23:00:280
1008049991410,2cyclictest1147290-21cpuspeed_turbos21:05:140
100805999139,3cyclictest1250480-21cpuspeed_turbos22:20:123
100805999139,3cyclictest1075287-21unixbench_multi20:00:263
100805999136,5cyclictest495-21dbus-daemon19:40:003
1008059991310,2cyclictest1312917-21gltestperf23:05:153
1008059991310,2cyclictest1023833-21sort19:20:163
1008059991310,2cyclictest0-21swapper/321:20:003
1008059991310,2cyclictest0-21swapper/321:19:593
100805999130,4cyclictest1409599-21munin-run00:15:003
1008057991311,1cyclictest0-21swapper/223:55:002
1008057991310,2cyclictest1382182-21cpuspeed_turbos23:55:112
1008057991310,2cyclictest1191180-21kthreadcore21:35:192
1008057991310,2cyclictest0-21swapper/222:40:012
1008057991310,2cyclictest0-21swapper/220:55:212
100805799130,4cyclictest1169763-21cut21:20:212
100805799130,3cyclictest739-21runrttasks22:25:142
100805799130,3cyclictest0-21swapper/222:10:252
100805799130,12cyclictest0-21swapper/219:35:242
100805299138,4cyclictest353-21systemd-journal21:59:541
100805299137,1cyclictest11274381kthreadcore20:45:191
100805299136,5cyclictest1325390-21sshd23:12:481
100805299132,2cyclictest0-21swapper/120:30:001
1008052991311,1cyclictest0-21swapper/120:50:181
1008052991310,2cyclictest1298988-21gltestperf22:55:161
1008052991310,2cyclictest0-21swapper/122:00:151
100805299130,4cyclictest1152999-21latency_hist21:10:011
100805299130,2cyclictest191rcu_preempt23:10:001
100804999139,3cyclictest171ktimers/021:20:120
100804999139,3cyclictest1094355-21gltestperf20:20:120
100804999139,2cyclictest0-21swapper/023:20:160
100804999138,3cyclictest1034061-21cron19:30:010
100804999131,3cyclictest353-21systemd-journal21:55:010
1008049991311,1cyclictest1099369-21kworker/0:2+kacpid21:25:110
1008049991311,1cyclictest1099369-21kworker/0:2+kacpid00:27:140
1008049991310,2cyclictest1408430-21unixbench_singl00:10:280
1008049991310,2cyclictest1389941-21irqrtprio00:00:170
1008049991310,2cyclictest1319768-21gltestperf23:10:150
1008049991310,2cyclictest1146494-21unixbench_multi21:00:240
1008049991310,2cyclictest1058340-21cpuspeed_turbos19:50:100
100804999130,12cyclictest353-21systemd-journal23:20:000
11476342120,2sleep21147678-21sort21:05:192
100805999129,2cyclictest1008314-21cpuspeed_turbos19:10:133
100805999129,2cyclictest0-21swapper/321:50:173
100805999129,2cyclictest0-21swapper/319:50:013
100805999129,1cyclictest0-21swapper/321:20:013
100805999129,1cyclictest0-21swapper/319:30:133
100805999128,2cyclictest0-21swapper/323:45:253
100805999128,2cyclictest0-21swapper/323:10:153
100805999128,2cyclictest0-21swapper/320:35:143
100805999127,4cyclictest1278542-21irqcore22:40:163
100805999127,3cyclictest0-21swapper/319:15:233
100805799129,2cyclictest1305327-21munin-plugin-st23:00:002
100805799129,2cyclictest1088171-21cpuspeed_turbos20:15:112
100805799129,2cyclictest1059650-21kthreadcore19:50:152
100805799128,3cyclictest1129155-21cron20:50:002
100805799128,3cyclictest1027269-21grep19:20:212
100805799127,4cyclictest1311650-21sshd23:02:422
100805799126,4cyclictest1389601-21sed00:00:162
100805799122,3cyclictest1008044-21cyclictest21:28:242
1008057991210,1cyclictest1244558-21irqrtprio22:15:152
1008057991210,1cyclictest0-21swapper/222:05:292
100805299129,2cyclictest1422015-21unixbench_multi00:20:321
100805299129,2cyclictest1209494-21irqcore21:50:181
100805299129,2cyclictest1064158-21/usr/sbin/munin19:55:201
100805299129,2cyclictest0-21swapper/122:45:131
100805299129,1cyclictest0-21swapper/100:25:151
100805299126,4cyclictest1250518-21cpuspeed_turbos22:20:121
100805299126,4cyclictest1202557-21smartctl21:45:161
1008052991210,1cyclictest1232919-21kthreadcore22:05:221
1008052991210,1cyclictest1094666-21irqcore20:20:131
100805299120,2cyclictest0-21swapper/122:25:261
100804999129,2cyclictest1375604-21gltestperf23:50:160
100804999129,2cyclictest1160560-21kworker/u8:0-rpciod21:35:180
100804999129,2cyclictest1034672-21cpuspeed_turbos19:30:100
100804999129,2cyclictest0-21swapper/023:45:160
100804999129,2cyclictest0-21swapper/020:00:010
100804999128,3cyclictest353-21systemd-journal21:59:480
100804999128,3cyclictest1304047-21sed22:55:290
100804999126,4cyclictest1-21systemd19:25:010
100804999126,2cyclictest211rcuc/022:05:010
100804999121,2cyclictest1250078-21idleruntime-cro22:20:010
1008049991210,1cyclictest1129897-21irqcore20:50:150
1008049991210,1cyclictest1129164-21idleruntime-cro20:50:000
1008049991210,1cyclictest0-21swapper/022:07:080
1008049991210,1cyclictest0-21swapper/020:35:180
100804999120,10cyclictest1260311-21tr22:25:170
269698110,4rtkit-daemon2764-21sendmail-mta20:25:353
100805999118,2cyclictest491ktimers/300:07:563
100805999118,2cyclictest0-21swapper/323:50:133
100805999118,2cyclictest0-21swapper/323:37:483
100805999118,2cyclictest0-21swapper/323:00:133
100805999118,2cyclictest0-21swapper/321:35:203
100805999118,2cyclictest0-21swapper/320:10:013
100805999117,3cyclictest1186862-21sshd21:30:493
100805999117,3cyclictest1123783-21gltestperf20:45:133
100805999116,4cyclictest353-21systemd-journal20:25:003
100805999116,4cyclictest1275575-21tr22:35:203
100805999110,3cyclictest0-21swapper/300:23:183
100805799119,1cyclictest1364009-21kthreadcore23:40:202
100805799118,2cyclictest1105958-21cpuspeed_turbos20:30:112
100805799118,2cyclictest1068854-21needreboot19:55:232
100805799118,2cyclictest0-21swapper/223:10:212
100805799118,2cyclictest0-21swapper/219:35:012
100805799118,2cyclictest0-21swapper/200:25:172
100805799117,3cyclictest1076349-21cpuspeed_turbos20:05:112
100805799117,2cyclictest0-21swapper/223:45:252
100805799116,4cyclictest495-21dbus-daemon21:50:012
100805799115,4cyclictest1034122-21pluginstate19:30:012
100805299118,2cyclictest1243515-21cpuspeed_turbos22:15:111
100805299113,2cyclictest191rcu_preempt21:40:151
100805299111,7cyclictest1136268-21grep20:55:181
100804999118,2cyclictest6950irq/9-acpi22:35:030
100804999118,2cyclictest1099369-21kworker/0:2+kacpi_notify00:05:530
100804999116,4cyclictest1106151-21gltestperf20:30:130
100804999113,7cyclictest1140788-21sshd20:57:200
100804999112,7cyclictest1278067-21apt-get22:40:110
100804999111,8cyclictest1123241-21munin-run20:45:000
100804999110,4cyclictest1289959-21python322:45:240
100804999110,3cyclictest1014086-21cat19:15:000
100804999110,2cyclictest59324-21gdbus20:05:010
100804999110,1cyclictest0-21swapper/022:50:120
100805999108,1cyclictest0-21swapper/322:50:183
100805999108,1cyclictest0-21swapper/321:10:143
100805999107,2cyclictest0-21swapper/300:25:183
100805999106,3cyclictest1141448-21cut21:00:103
100805999105,4cyclictest1328283-21cut23:15:183
100805999101,2cyclictest191rcu_preempt19:55:183
100805799107,2cyclictest1116517-21sendmail_mailqu20:35:262
100805799105,4cyclictest1143189-21cut21:00:162
100805799105,4cyclictest1086158-21cut20:10:172
100805299105,4cyclictest1117777-21cut20:40:121
100805299101,2cyclictest191rcu_preempt19:50:101
100804999108,1cyclictest1340686-21irqcore23:25:180
100804999100,1cyclictest1413770-21tr00:15:170
10080599996,2cyclictest0-21swapper/320:40:113
10080599985,2cyclictest1007907-21latency_hist19:10:003
10080579985,2cyclictest0-21swapper/219:10:012
10080499970,6cyclictest0-21swapper/019:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional