You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-18 - 19:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Thu Apr 18, 2024 12:43:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2322210087,9sleep10-21swapper/107:09:271
234129682,10sleep30-21swapper/307:09:423
209528467,6sleep20-21swapper/207:06:312
226627967,8sleep00-21swapper/007:08:420
124772570,0sleep00-21swapper/007:30:180
245199310,0cyclictest0-21swapper/111:42:051
245899250,0cyclictest0-21swapper/311:57:523
245899241,20cyclictest0-21swapper/312:11:553
2447992318,4cyclictest161rcuc/011:50:110
245699210,21cyclictest0-21swapper/212:33:102
244799213,2cyclictest141rcu_preempt11:57:040
244799210,4cyclictest13-21ksoftirqd/010:14:010
245899206,8cyclictest224-21systemd-journal11:10:003
245899200,1cyclictest0-21swapper/312:09:563
245699200,20cyclictest0-21swapper/211:28:232
2447992013,7cyclictest13-21ksoftirqd/008:50:150
244799200,1cyclictest0-21swapper/009:00:270
245899190,1cyclictest27397-21diskmemload11:17:383
245899190,1cyclictest0-21swapper/310:40:383
245699190,1cyclictest0-21swapper/211:30:202
245699190,19cyclictest0-21swapper/211:00:132
245699190,17cyclictest0-21swapper/212:18:282
244799194,1cyclictest141rcu_preempt09:57:030
245899181,14cyclictest0-21swapper/312:25:103
245899180,18cyclictest0-21swapper/311:46:183
245899180,18cyclictest0-21swapper/309:59:183
2456991817,1cyclictest37850irq/131-snd_hda09:37:082
245699180,2cyclictest0-21swapper/208:05:132
245699180,18cyclictest0-21swapper/210:04:022
2451991817,1cyclictest0-21swapper/110:29:201
245199180,17cyclictest0-21swapper/110:55:131
244799186,2cyclictest1979-21ssh11:24:170
245899170,17cyclictest0-21swapper/310:45:373
245899170,0cyclictest0-21swapper/311:21:573
245699170,3cyclictest16551-21cstates11:10:132
245699170,1cyclictest0-21swapper/209:40:122
244799171,16cyclictest161rcuc/010:42:070
244799171,15cyclictest7644-21apt-get12:15:150
244799170,6cyclictest0-21swapper/009:50:060
244799170,6cyclictest0-21swapper/009:50:050
2447991614,1cyclictest30691-21rm11:45:130
2447991611,5cyclictest1254-21kworker/u8:009:06:370
2458991512,2cyclictest39-21ksoftirqd/307:25:113
245699152,2cyclictest0-21swapper/210:07:362
2456991513,2cyclictest32-21ksoftirqd/210:10:202
2456991511,3cyclictest562-21wc08:10:282
2456991511,3cyclictest15356-21ssh12:20:192
2456991510,3cyclictest3592-21sh08:20:012
244799155,9cyclictest161rcuc/009:13:210
244799153,10cyclictest0-21swapper/010:54:270
2447991511,2cyclictest161rcuc/010:31:460
2447991511,1cyclictest23758-21ssh11:40:140
245899143,7cyclictest9103-21cron10:15:003
2458991413,1cyclictest0-21swapper/310:36:443
2458991410,2cyclictest18340-21grep11:35:313
245899140,14cyclictest0-21swapper/311:33:593
245899140,14cyclictest0-21swapper/311:28:403
245699142,11cyclictest0-21swapper/211:40:292
2456991413,1cyclictest32-21ksoftirqd/209:45:372
2456991411,2cyclictest23014-21wc12:25:182
2456991410,3cyclictest24506-21grep11:15:302
245699140,2cyclictest0-21swapper/212:03:402
245699140,13cyclictest22779-21sed10:25:162
245699140,13cyclictest0-21swapper/209:55:182
245199146,7cyclictest10779-21ntp_kernel_pll_09:50:221
245199146,7cyclictest10779-21ntp_kernel_pll_09:50:211
2451991413,1cyclictest25-21ksoftirqd/111:24:511
2451991413,1cyclictest25-21ksoftirqd/111:00:131
2451991413,1cyclictest0-21swapper/109:13:241
2451991412,1cyclictest0-21swapper/112:05:241
2451991411,2cyclictest28953-21grep10:30:151
2451991410,3cyclictest23910-21grep10:00:261
244799147,6cyclictest31210-21awk08:10:010
244799146,2cyclictest141rcu_preempt10:36:260
244799145,4cyclictest13-21ksoftirqd/010:00:140
244799143,3cyclictest13-21ksoftirqd/008:55:130
2447991410,3cyclictest27397-21diskmemload12:39:030
245899139,3cyclictest3591-21sh08:20:003
245899139,3cyclictest14300-21cron12:19:593
245899139,2cyclictest24585-21tr07:55:143
245899139,2cyclictest24585-21tr07:55:133
245899139,2cyclictest20525-21wc07:45:193
245899139,2cyclictest16523-21grep11:10:133
245899136,3cyclictest224-21systemd-journal10:58:313
2458991312,1cyclictest0-21swapper/309:36:193
245899131,12cyclictest0-21swapper/311:41:183
2458991310,2cyclictest22198-21tr07:50:163
2458991310,2cyclictest20613-21grep12:00:153
2458991310,2cyclictest11388-21grep10:15:303
245899130,12cyclictest0-21swapper/309:33:413
245699139,3cyclictest27878-21grep09:10:132
245699139,2cyclictest28422-21head08:00:312
245699138,3cyclictest21124-21tune2fs08:55:152
245699137,6cyclictest37850irq/131-snd_hda12:08:512
245699131,8cyclictest224-21systemd-journal07:50:002
245699131,11cyclictest27397-21diskmemload10:52:182
2456991310,2cyclictest2909-21cut10:35:162
2456991310,2cyclictest14522-21cron07:35:002
2456991310,2cyclictest1434-21tr09:15:172
245699130,9cyclictest224-21systemd-journal07:40:002
245699130,8cyclictest0-21swapper/210:48:002
245699130,13cyclictest0-21swapper/211:49:182
245199138,3cyclictest24729-21chrt09:01:311
2451991310,2cyclictest5801-21cron09:20:001
2451991310,2cyclictest3136-21sh12:34:481
2451991310,2cyclictest15005-21grep08:40:291
245199130,11cyclictest30703-21grep09:40:171
244799139,3cyclictest161rcuc/011:03:130
244799135,7cyclictest32424-21apache209:22:540
244799135,7cyclictest13-21ksoftirqd/007:45:140
244799134,8cyclictest558-21lldpd12:06:080
244799134,2cyclictest841-21ssh12:32:250
2447991311,2cyclictest13-21ksoftirqd/011:07:450
2447991311,2cyclictest13-21ksoftirqd/010:16:360
2447991311,1cyclictest4584-21tr08:20:200
2447991311,1cyclictest27397-21diskmemload10:45:120
2447991310,2cyclictest16514-21grep08:45:190
2447991310,2cyclictest14085-21head09:25:260
244799130,13cyclictest0-21swapper/012:12:540
244799130,12cyclictest16870-21grep11:10:160
245899129,2cyclictest7919-21tail08:25:313
245899129,2cyclictest770-21grep08:10:293
245899129,2cyclictest29947-21grep12:30:193
245899129,2cyclictest26936-21grep08:00:153
245899129,2cyclictest24008-21head09:00:213
245899129,2cyclictest18719-21chrt08:50:193
245899128,3cyclictest18737-21wc07:40:283
245899128,3cyclictest10943-21grep09:50:243
245899128,3cyclictest10943-21grep09:50:233
245899121,11cyclictest27397-21diskmemload09:41:483
2458991211,1cyclictest0-21swapper/311:53:573
245899120,11cyclictest27397-21diskmemload12:36:153
245699129,2cyclictest5734-21grep12:35:332
245699129,2cyclictest24035-21cron07:55:012
245699129,2cyclictest12383-21cron11:55:012
245699128,3cyclictest2642-21taskset07:10:022
245699128,3cyclictest25022-21grep07:55:162
245699128,3cyclictest25022-21grep07:55:152
245699124,2cyclictest224-21systemd-journal10:43:292
245699120,9cyclictest18921-21fschecks_count09:30:212
245699120,2cyclictest0-21swapper/210:33:592
245699120,1cyclictest0-21swapper/209:24:312
245199129,2cyclictest29359-21grep08:05:141
245199129,2cyclictest27383-21cron09:09:591
245199129,2cyclictest24229-21wc07:55:011
245199129,2cyclictest22101-21apt-config11:15:011
245199129,2cyclictest21815-21wc10:49:591
245199129,2cyclictest2143-21uniq08:15:151
245199129,2cyclictest21155-21grep08:55:151
245199129,2cyclictest19734-21cut09:30:291
245199129,2cyclictest18286-21grep11:35:311
245199129,2cyclictest15532-21grep12:20:201
245199129,2cyclictest13372-21ssh09:25:201
245199129,2cyclictest11394-21sed10:15:301
245199129,2cyclictest10762-21grep07:25:141
245199129,2cyclictest10019-21tr11:30:141
245199128,3cyclictest32476-21cat08:10:191
245199128,3cyclictest24442-21grep11:15:301
245199128,1cyclictest1135-21ssh12:10:181
245199126,2cyclictest9099-21cron10:15:011
245199125,3cyclictest224-21systemd-journal09:40:001
2451991210,2cyclictest20519-21uniq12:00:161
245199120,12cyclictest0-21swapper/110:42:131
245199120,11cyclictest5321-21tr07:15:141
244799129,2cyclictest29512-21cut10:55:210
244799129,2cyclictest19794-21ssh12:00:110
244799129,2cyclictest17134-21sort11:35:180
244799129,2cyclictest15098-21cut12:20:170
244799129,2cyclictest13-21ksoftirqd/011:33:250
244799129,2cyclictest10765-21head07:25:140
244799122,9cyclictest3376-21/usr/sbin/munin09:45:240
245899118,2cyclictest8908-21grep07:20:223
245899118,2cyclictest8896-21ssh09:21:213
245899118,2cyclictest7213-21chrt11:02:413
245899118,2cyclictest3363-21cut07:10:193
245899118,2cyclictest31585-21taskset10:31:493
245899118,2cyclictest30288-21sh09:11:053
245899118,2cyclictest2664-21ssh10:09:553
245899118,2cyclictest25624-21ssh10:27:173
245899118,2cyclictest2534-21head09:15:263
245899118,2cyclictest21263-21grep08:55:173
245899118,2cyclictest16810-21sed12:20:333
245899118,2cyclictest14529-21sh07:34:593
245899117,3cyclictest5554-21sed08:20:313
245899112,1cyclictest29086-21apt-get08:05:133
245899111,5cyclictest224-21systemd-journal07:20:003
245899110,1cyclictest0-21swapper/310:52:193
245899110,10cyclictest9772-21ntpq08:30:213
245699118,2cyclictest9364-21grep07:20:282
245699118,2cyclictest31520-21ssh11:21:092
245699118,2cyclictest29037-21cat10:55:172
245699118,2cyclictest26052-21sed09:05:172
245699118,2cyclictest19787-21grep08:50:302
245699118,2cyclictest17266-21sh11:57:592
245699118,2cyclictest16520-21sort10:20:202
245699118,2cyclictest16246-21rm11:35:062
245699118,2cyclictest16065-21seq08:45:152
245699118,2cyclictest14224-21taskset09:25:382
245699118,2cyclictest14036-21ssh11:07:562
245699118,2cyclictest13620-21tr08:40:152
245699118,2cyclictest12614-21grep08:35:292
245699118,2cyclictest10403-21grep10:15:182
245699117,3cyclictest5986-21sleep08:25:012
245699113,6cyclictest10122-21rm09:50:162
245699113,6cyclictest10122-21rm09:50:152
245699111,9cyclictest11272-21pmu-power07:25:212
245199119,1cyclictest9363-21cat12:15:301
245199119,1cyclictest4992-21ssh10:36:001
245199118,2cyclictest9842-21grep08:30:231
245199118,2cyclictest7564-21dpkg07:20:011
245199118,2cyclictest6482-21ssh09:46:561
245199118,2cyclictest6013-21expr11:50:131
245199118,2cyclictest4308-21cat11:25:211
245199118,2cyclictest29716-21tune2fs10:05:181
245199118,2cyclictest27434-21grep08:00:181
245199118,2cyclictest22923-21cat12:25:181
245199118,2cyclictest18445-21ssh10:21:241
245199118,2cyclictest17334-21grep07:40:121
245199118,2cyclictest16466-21grep07:35:321
245199118,2cyclictest13790-21basename11:55:191
245199118,2cyclictest11173-21ntpq11:05:201
245199115,2cyclictest224-21systemd-journal08:00:011
245199115,2cyclictest224-21systemd-journal08:00:011
245199110,10cyclictest16889-21rm09:55:201
245199110,10cyclictest14491-21taskset07:34:401
244799118,3cyclictest13-21ksoftirqd/007:14:030
244799118,2cyclictest6119-21cat07:15:210
244799118,2cyclictest30236-21tr10:05:210
244799118,2cyclictest2691-21ls08:15:210
244799118,2cyclictest23861-21uname11:15:230
244799118,2cyclictest22987-21sed07:50:220
244799118,2cyclictest21949-21rm10:25:130
244799118,2cyclictest20525-21sh10:23:580
244799118,2cyclictest18879-21entropy09:30:210
244799118,2cyclictest1587-21expr09:15:170
244799118,2cyclictest15589-21head07:35:200
244799118,2cyclictest11099-21cut08:35:150
244799117,3cyclictest161rcuc/009:40:060
244799114,5cyclictest161rcuc/011:25:270
244799110,9cyclictest0-21swapper/012:25:120
245899108,1cyclictest5536-21ssh09:45:383
245899108,1cyclictest15881-21grep10:20:163
245899108,1cyclictest157652sleep308:45:133
245899108,1cyclictest13458-21cut08:40:143
245899107,2cyclictest14620-21ssh09:26:043
245899106,2cyclictest26054-21tail09:05:173
2458991010,0cyclictest0-21swapper/310:00:213
245699108,1cyclictest6173-21cut07:15:202
245699108,1cyclictest10662-21chrt08:34:192
245699107,2cyclictest24713-21taskset09:01:182
245699107,2cyclictest1114-21sh12:10:182
245699106,3cyclictest6901-21grep08:25:212
245699101,8cyclictest18910-21unixbench_singl07:40:282
245199108,1cyclictest6188-21ssh09:20:101
245199108,1cyclictest5752-21chrt08:22:111
245199108,1cyclictest3688-21ssh12:35:101
245199108,1cyclictest21593-21taskset07:49:101
245199108,1cyclictest18451-21cat08:50:171
245199107,2cyclictest6986-21head08:25:211
245199107,2cyclictest31661-21chrt11:45:201
245199107,2cyclictest22014-21sh10:50:011
245199107,2cyclictest16701-21uname08:45:201
245199100,9cyclictest2665-21cat07:10:121
244799108,1cyclictest6700-21expr08:25:180
244799108,1cyclictest25952-21ssh09:35:290
244799108,1cyclictest18609-21wc07:40:280
244799107,2cyclictest32267-21cat08:10:180
244799107,2cyclictest27031-21expr08:00:140
244799105,2cyclictest0-21swapper/008:30:200
24589997,1cyclictest12756-21sshd08:35:403
24589996,2cyclictest15891-21grep07:35:243
24519996,2cyclictest12189-21awk08:35:251
24479997,1cyclictest7816-21grep07:20:140
24479997,1cyclictest24890-21dnf07:55:160
24479997,1cyclictest24890-21dnf07:55:160
24479997,1cyclictest13694-21cat08:40:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional