You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-07 - 16:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat Sep 07, 2024 12:44:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10803210187,9sleep30-21swapper/307:05:333
1081629986,9sleep10-21swapper/107:05:431
1099427858,9sleep00-21swapper/007:08:030
1086927350,21sleep20-21swapper/207:06:242
217752570,0sleep20-21swapper/210:11:282
1125099340,33cyclictest0-21swapper/311:31:083
1125099330,32cyclictest0-21swapper/309:27:293
11250993231,1cyclictest0-21swapper/311:04:483
11250993231,1cyclictest0-21swapper/309:43:093
11250993130,1cyclictest0-21swapper/312:06:283
1124799310,20cyclictest0-21swapper/209:45:562
11250993029,1cyclictest0-21swapper/309:56:093
11250993029,1cyclictest0-21swapper/309:49:293
11250993029,1cyclictest0-21swapper/309:22:093
11250993029,1cyclictest0-21swapper/309:22:093
11243993027,2cyclictest25-21ksoftirqd/111:20:011
11250992928,1cyclictest0-21swapper/312:18:293
11250992928,1cyclictest0-21swapper/311:05:493
11250992827,1cyclictest0-21swapper/311:55:283
11250992726,1cyclictest0-21swapper/309:37:293
11250992625,1cyclictest0-21swapper/312:21:093
11250992625,1cyclictest0-21swapper/311:48:293
11250992625,1cyclictest0-21swapper/311:39:493
11250992624,1cyclictest22461-21ssh10:36:483
1124799261,24cyclictest0-21swapper/210:08:062
1124799260,25cyclictest0-21swapper/210:49:392
25122250,2sleep02511-21turbostat11:09:590
210662250,0sleep00-21swapper/011:42:250
11250992524,1cyclictest0-21swapper/310:21:493
11250992524,1cyclictest0-21swapper/309:31:093
11250992523,1cyclictest0-21swapper/310:42:293
1124799250,25cyclictest0-21swapper/210:22:542
11250992423,1cyclictest0-21swapper/311:29:293
11250992423,1cyclictest0-21swapper/310:04:093
11250992322,1cyclictest0-21swapper/311:14:493
11247992311,2cyclictest0-21swapper/212:35:392
1124399230,2cyclictest0-21swapper/109:37:011
1124799220,22cyclictest0-21swapper/209:19:582
11250992120,1cyclictest0-21swapper/312:35:293
11250992120,1cyclictest0-21swapper/312:02:093
11250992120,1cyclictest0-21swapper/311:23:283
11250992120,1cyclictest0-21swapper/310:51:083
11250992120,1cyclictest0-21swapper/309:11:293
1125099210,2cyclictest141rcu_preempt12:26:043
74102200,0sleep00-21swapper/012:36:040
1124799200,20cyclictest0-21swapper/210:34:112
1124799200,18cyclictest0-21swapper/211:12:122
1124399200,1cyclictest0-21swapper/111:55:141
11238992018,2cyclictest16747-21apache209:27:130
264492190,0sleep20-21swapper/209:50:192
11247991918,1cyclictest0-21swapper/209:55:502
11247991910,3cyclictest13625-21munin-plugin-st07:15:002
1124799190,19cyclictest0-21swapper/212:10:192
1123899190,19cyclictest0-21swapper/009:37:150
1124799189,2cyclictest25340-21wc07:35:282
1124799182,4cyclictest0-21swapper/207:40:132
11247991818,0cyclictest0-21swapper/212:23:282
11247991810,2cyclictest8957-21tr11:55:232
11247991810,2cyclictest16002-21cron07:20:002
11247991810,2cyclictest13302-21sh12:19:592
1124799180,18cyclictest0-21swapper/211:15:092
1124399180,1cyclictest0-21swapper/111:28:321
1124399180,17cyclictest0-21swapper/111:10:431
11250991716,1cyclictest0-21swapper/309:18:093
11247991715,1cyclictest0-21swapper/209:10:142
11247991710,2cyclictest26587-21uname10:15:212
1124799170,16cyclictest0-21swapper/209:26:422
1124399175,11cyclictest16698-21apache210:45:351
1124399171,8cyclictest224-21systemd-journal08:19:591
11243991715,1cyclictest25-21ksoftirqd/110:50:121
11243991714,2cyclictest25-21ksoftirqd/111:10:001
1124399170,17cyclictest0-21swapper/109:53:071
1123899176,1cyclictest141rcu_preempt09:43:290
11238991713,3cyclictest13-21ksoftirqd/010:50:110
11243991610,4cyclictest25-21ksoftirqd/112:05:341
1123899169,4cyclictest13-21ksoftirqd/011:15:170
1124799154,5cyclictest30263-21unixbench_singl07:45:272
1124799150,1cyclictest0-21swapper/211:36:522
1124799150,15cyclictest0-21swapper/211:29:082
1124399157,3cyclictest141rcu_preempt11:40:171
1124399151,6cyclictest4634-21diskmemload10:21:261
11243991511,3cyclictest25-21ksoftirqd/110:00:101
1124399150,3cyclictest0-21swapper/110:35:311
1123899151,7cyclictest13343-21basename12:20:000
1125099149,3cyclictest28687-21grep07:45:153
1125099145,6cyclictest6001-21apt-get10:25:143
1125099142,2cyclictest0-21swapper/309:05:123
1124399142,2cyclictest141rcu_preempt11:33:581
1124399141,8cyclictest25-21ksoftirqd/110:14:541
11243991414,0cyclictest0-21swapper/112:10:051
11243991410,1cyclictest25-21ksoftirqd/111:35:021
1123899141,8cyclictest22636-21munin-run12:04:590
11238991413,1cyclictest0-21swapper/010:02:050
11238991410,3cyclictest1260-21tail10:20:260
1123899140,1cyclictest0-21swapper/012:09:030
1125099139,3cyclictest12008-21grep08:15:233
1125099139,3cyclictest10000-21grep08:10:263
1125099133,9cyclictest32194-21/usr/sbin/munin11:50:273
11250991310,2cyclictest2666-21grep07:55:263
1125099130,2cyclictest0-21swapper/307:30:123
1124799139,3cyclictest24440-21wc08:40:272
11247991311,2cyclictest0-21swapper/209:35:502
11247991310,2cyclictest6409-21grep08:05:172
11247991310,2cyclictest18712-21grep11:40:242
11247991310,2cyclictest17499-21grep07:20:242
11247991310,2cyclictest16697-21awk12:00:222
1124399139,3cyclictest25444-21tr09:25:171
1124399139,3cyclictest18307-21tr08:30:161
1124399138,3cyclictest11441-21wc09:15:001
1124399138,1cyclictest25-21ksoftirqd/110:05:081
1124399135,4cyclictest224-21systemd-journal12:19:591
11243991312,1cyclictest0-21swapper/110:31:171
11243991310,2cyclictest4745-21wc09:35:001
11243991310,2cyclictest3076-21cron08:00:011
11243991310,2cyclictest30390-21tr08:55:151
11243991310,2cyclictest241rcuc/112:20:291
11243991310,2cyclictest11961-21ssh09:40:161
1124399130,9cyclictest0-21swapper/112:28:061
1123899139,3cyclictest31693-21tail07:50:160
11238991310,2cyclictest5913-21sed09:10:170
11238991310,2cyclictest2708-21ssh11:30:270
11238991310,2cyclictest24879-21uname07:35:220
11238991310,2cyclictest23636-21grep12:25:250
11238991310,2cyclictest19353-21head09:45:250
1125099129,2cyclictest25437-21awk07:35:283
1125099129,2cyclictest20190-21ls08:34:593
1125099129,2cyclictest18770-21cron10:35:003
11250991211,1cyclictest0-21swapper/310:06:093
1125099120,11cyclictest17318-21grep08:25:263
1124799129,2cyclictest8345-21cut11:35:002
1124799129,2cyclictest6083-21cut10:25:162
1124799129,2cyclictest32640-21tail07:50:262
1124799129,2cyclictest25741-21tr11:45:202
1124799129,2cyclictest14823-21grep08:20:282
1124799129,2cyclictest11484-21grep08:15:172
1124799129,2cyclictest11340-21taskset10:04:252
1124799128,3cyclictest19465-21ntpq09:20:222
1124799128,3cyclictest19465-21ntpq09:20:212
1124799123,8cyclictest0-21swapper/211:20:122
1124799121,10cyclictest0-21swapper/209:44:052
11247991210,1cyclictest23042-21/usr/sbin/munin11:00:272
11247991210,1cyclictest14000-21apt-get10:55:142
11247991210,1cyclictest0-21swapper/210:51:112
1124799120,2cyclictest0-21swapper/211:51:122
1124799120,11cyclictest0-21swapper/211:06:132
1124399129,2cyclictest5593-21cut12:35:181
1124399129,2cyclictest26669-21grep07:40:141
1124399129,2cyclictest226992sleep107:30:261
1124399129,2cyclictest1147-21grep09:55:251
1124399128,3cyclictest7454-21grep08:05:281
1124399128,3cyclictest32156-21ls10:45:001
1124399128,3cyclictest29560-21ssh11:48:061
1124399128,3cyclictest14249-21grep10:55:151
1124399122,2cyclictest4634-21diskmemload09:47:411
1124399120,11cyclictest8652-21tr08:10:151
1124399120,11cyclictest15575-21head07:15:261
1123899129,2cyclictest9849-21head11:35:190
1123899129,2cyclictest7874-21grep11:55:150
1123899129,2cyclictest4634-21diskmemload11:10:120
1123899129,2cyclictest27397-21uname10:40:220
1123899129,2cyclictest27218-21tail10:15:270
1123899129,2cyclictest25957-21grep11:25:150
1123899129,2cyclictest1709-21grep11:50:260
1123899129,2cyclictest1017-21ssh12:32:130
1123899126,6cyclictest0-21swapper/011:20:350
1123899124,4cyclictest224-21systemd-journal12:25:000
11238991211,1cyclictest0-21swapper/012:11:490
1123899120,2cyclictest0-21swapper/010:27:450
1125099118,2cyclictest865-21head09:00:173
1125099118,2cyclictest31609-21head12:10:163
1125099118,2cyclictest31021-21uname08:55:193
1125099118,2cyclictest29156-21ssh12:30:033
1125099118,2cyclictest28929-21grep08:50:203
1125099118,2cyclictest27464-21ssh10:15:353
1125099118,2cyclictest27373-21chrt08:49:313
1125099118,2cyclictest26419-21cat09:50:203
1125099118,2cyclictest23652-21sed08:40:183
1125099118,2cyclictest2361-21ssh10:45:433
1125099118,2cyclictest19767-21ssh11:41:193
1125099118,2cyclictest19203-21wc10:10:293
1125099118,2cyclictest14481-21ssh11:17:243
1125099118,2cyclictest14041-21grep10:55:143
1125099118,2cyclictest11851-21ls07:10:143
1125099117,3cyclictest21423-21wc08:35:223
1124799118,2cyclictest9702-21wc08:10:222
1124799118,2cyclictest476-21expr09:00:152
1124799118,2cyclictest30806-21sh09:30:062
1124799118,2cyclictest30033-21ssh12:30:172
1124799118,2cyclictest29411-21head08:50:242
1124799118,2cyclictest28340-21ssh10:40:312
1124799118,2cyclictest24858-21grep12:05:262
1124799118,2cyclictest22063-21grep12:25:142
1124799118,2cyclictest21566-21ssh10:35:502
1124799118,2cyclictest20709-21sshd07:28:032
1124799118,2cyclictest20709-21sshd07:28:022
1124799118,2cyclictest20205-21munin-run08:35:002
1124799118,2cyclictest16758-21cut08:25:202
1124799110,11cyclictest0-21swapper/208:00:152
1124399118,2cyclictest6648-21uniq10:25:191
1124399118,2cyclictest4179-21grep09:05:251
1124399118,2cyclictest3339-21ssh12:34:141
1124399118,2cyclictest32699-21grep09:00:151
1124399118,2cyclictest32561-21tr11:50:141
1124399118,2cyclictest25-21ksoftirqd/109:20:011
1124399118,2cyclictest24517-21grep07:35:191
1124399118,2cyclictest23010-21sed11:00:271
1124399118,2cyclictest23003-21wget08:40:171
1124399118,2cyclictest19108-21cat11:20:181
1124399118,2cyclictest18121-21sh09:20:101
1124399118,2cyclictest18121-21sh09:20:091
1124399118,2cyclictest15727-21expr12:00:151
1124399117,2cyclictest28296-21cut08:50:151
1124399110,4cyclictest0-21swapper/107:45:131
1124399110,10cyclictest25464-21tr10:15:141
1124399110,10cyclictest13949-21head08:20:191
1123899118,2cyclictest5501-21turbostat.cron08:05:000
1123899118,2cyclictest480-21ntpq09:55:190
1123899118,2cyclictest4612-21sh10:48:010
1123899118,2cyclictest3268-21grep09:05:160
1123899118,2cyclictest31260-21cat09:30:150
1123899118,2cyclictest28643-21cut07:45:150
1123899118,2cyclictest28410-21grep08:50:160
1123899118,2cyclictest27548-21ssh11:04:480
1123899118,2cyclictest25913-21grep09:50:160
1123899118,2cyclictest24202-21ssh10:38:340
1123899118,2cyclictest2175-21head07:55:220
1123899118,2cyclictest20797-21cat10:10:400
1123899118,2cyclictest18733-21sort08:30:180
1123899118,2cyclictest17254-21rm09:19:300
1123899118,2cyclictest16710-21ssh10:32:560
1123899118,2cyclictest15993-21cat07:19:550
1123899118,2cyclictest13928-21ssh10:55:100
1123899118,2cyclictest13662-21date07:15:000
1123899118,2cyclictest12483-21cut10:05:150
1123899116,4cyclictest0-21swapper/008:29:590
1125099108,1cyclictest3724-21expr08:00:153
1125099108,1cyclictest205252sleep307:25:403
1125099108,1cyclictest205252sleep307:25:393
1125099108,1cyclictest15429-21wc07:15:253
1125099107,2cyclictest7743-21taskset08:08:153
1125099107,2cyclictest26148-21cat07:40:113
1125099107,2cyclictest17105-21grep07:20:213
1125099107,2cyclictest13508-21expr08:20:173
1124799108,1cyclictest300572sleep208:55:032
1124799108,1cyclictest26259-21wc08:45:202
1124799107,2cyclictest2875-21taskset07:57:482
1124799107,2cyclictest21614-21tr07:30:132
1124799106,3cyclictest21239-21head08:35:202
1124799101,5cyclictest224-21systemd-journal09:06:552
1124399108,1cyclictest3344-21cat08:00:111
1124399108,1cyclictest27118-21taskset08:46:461
1124399108,1cyclictest16247-21grep08:25:161
1124399107,2cyclictest19311-21df_abs07:25:191
1124399107,2cyclictest19311-21df_abs07:25:191
1124399107,2cyclictest16454-21egrep07:20:171
1124399107,2cyclictest11616-21grep07:10:131
1124399100,6cyclictest22538-21sshd08:39:441
1123899108,1cyclictest27275-21chrt08:48:470
1123899108,1cyclictest22910-21aten2_r7power_e08:40:160
1123899108,1cyclictest20951-21sed08:35:170
1123899108,1cyclictest18526-21cut09:20:160
1123899108,1cyclictest18526-21cut09:20:150
1123899108,1cyclictest13524-21expr08:20:170
1123899107,2cyclictest8589-21cat08:10:150
1123899107,2cyclictest30347-21grep08:55:150
1123899107,2cyclictest28522-21cut07:45:010
1123899107,2cyclictest25257-21tail11:45:160
1123899107,2cyclictest22094-21expr07:30:170
1123899107,2cyclictest17152-21sed07:20:200
1123899107,2cyclictest1197-21ntp_kernel_err09:00:210
1123899103,6cyclictest7908-21munin-run08:10:000
112509997,1cyclictest31417-21expr07:50:153
112439997,1cyclictest31129-21grep07:50:131
112389997,1cyclictest11178-21ls08:15:160
112389996,2cyclictest18874-21tr07:25:180
112389996,2cyclictest18874-21tr07:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional