You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 21:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Apr 25, 2024 12:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14893210764,38sleep10-21swapper/107:07:371
15017210087,9sleep30-21swapper/307:09:113
1496128464,16sleep20-21swapper/207:08:302
1500328270,8sleep00-21swapper/007:09:010
279992640,0sleep30-21swapper/310:55:023
1517299260,26cyclictest0-21swapper/210:43:152
1517299260,26cyclictest0-21swapper/209:32:152
1517299216,10cyclictest839-21apache_volume11:40:142
1517299210,21cyclictest0-21swapper/209:19:332
1517299210,0cyclictest0-21swapper/212:05:112
15169992120,1cyclictest0-21swapper/109:39:201
15167992121,0cyclictest0-21swapper/011:28:500
1517299200,20cyclictest0-21swapper/212:31:162
1517299200,19cyclictest0-21swapper/212:10:222
1517299190,19cyclictest0-21swapper/212:39:262
1517299190,19cyclictest0-21swapper/210:18:092
1516999194,14cyclictest0-21swapper/111:45:151
15169991918,1cyclictest0-21swapper/110:18:261
15167991917,1cyclictest0-21swapper/010:39:550
133432190,0sleep30-21swapper/311:07:283
1517299182,2cyclictest15513-21sed11:50:012
15169991817,0cyclictest0-21swapper/111:24:021
15177991716,1cyclictest0-21swapper/309:27:083
15177991711,5cyclictest5240-21apache212:12:413
15172991713,2cyclictest0-21swapper/210:58:022
15172991713,1cyclictest141rcu_preempt09:39:352
1517299170,3cyclictest0-21swapper/211:24:152
1517299170,14cyclictest0-21swapper/210:45:222
1516799172,14cyclictest0-21swapper/007:10:120
15167991716,1cyclictest0-21swapper/009:00:110
15167991714,1cyclictest0-21swapper/008:14:140
1516799170,3cyclictest0-21swapper/012:05:130
1516799170,0cyclictest0-21swapper/010:28:480
1517799160,16cyclictest0-21swapper/311:38:443
1517299166,8cyclictest500-21in:imuxsock10:33:242
1517299165,5cyclictest224-21systemd-journal09:23:122
15172991616,0cyclictest0-21swapper/211:12:252
15172991614,2cyclictest32-21ksoftirqd/209:30:002
1517299161,1cyclictest7758-21diskmemload11:31:572
1517299160,5cyclictest7758-21diskmemload11:09:522
1517299160,15cyclictest0-21swapper/211:15:242
1517299160,14cyclictest0-21swapper/210:36:062
1516999165,10cyclictest0-21swapper/108:30:121
1516799165,7cyclictest19497-21cron09:40:000
1516799164,6cyclictest4147-21perf07:55:000
1516799160,2cyclictest0-21swapper/009:21:590
15177991514,1cyclictest0-21swapper/312:02:083
15172991513,2cyclictest0-21swapper/209:54:012
1517299151,14cyclictest0-21swapper/210:53:152
1517299150,1cyclictest0-21swapper/210:01:232
1517299150,15cyclictest0-21swapper/210:27:072
1517299150,15cyclictest0-21swapper/209:45:462
1517299150,12cyclictest0-21swapper/208:10:142
15169991511,2cyclictest23229-21ssh10:05:281
15169991510,3cyclictest24422-21uname11:55:201
1517799144,7cyclictest0-21swapper/309:00:103
1517799141,5cyclictest11279-21/usr/sbin/munin11:25:263
15177991414,0cyclictest0-21swapper/310:04:393
15177991413,1cyclictest39-21ksoftirqd/309:44:133
15177991413,1cyclictest0-21swapper/311:43:503
15177991411,2cyclictest17614-21sed10:25:173
1517799140,1cyclictest0-21swapper/310:52:173
15172991412,1cyclictest32-21ksoftirqd/210:12:312
1517299140,1cyclictest0-21swapper/211:37:332
1517299140,14cyclictest0-21swapper/210:05:092
1517299140,13cyclictest0-21swapper/212:00:452
1516999145,7cyclictest7744-21snmpd11:01:361
15169991410,3cyclictest22817-21tr07:25:171
1516999140,2cyclictest0-21swapper/112:09:481
1516999140,10cyclictest0-21swapper/110:20:051
1516799147,7cyclictest30377-21/usr/sbin/munin12:00:230
1516799140,4cyclictest0-21swapper/009:17:590
1516799140,12cyclictest0-21swapper/009:55:200
1517799139,3cyclictest27144-21ls07:35:003
1517799139,3cyclictest14725-21head08:15:203
1517799139,3cyclictest14711-21tail09:35:343
1517799139,3cyclictest13326-21cat10:44:593
1517799136,6cyclictest8090-21ssh11:45:013
1517799131,7cyclictest0-21swapper/310:49:333
15177991310,2cyclictest28538-21head07:35:243
15177991310,2cyclictest24888-21turbostat.cron11:15:003
15177991310,2cyclictest1829-21sed11:20:143
15177991310,2cyclictest14469-21sendmail10:00:003
1517299139,3cyclictest24011-21cut11:55:162
1517299138,4cyclictest20695-21taskset08:28:052
1517299131,4cyclictest141rcu_preempt07:25:262
1517299131,1cyclictest7758-21diskmemload11:27:532
15172991310,2cyclictest9362-21wc09:10:202
15172991310,2cyclictest5554-21wc09:05:002
15172991310,2cyclictest28766-21wc12:20:352
15172991310,2cyclictest16113-21grep07:10:172
15172991310,2cyclictest14442-21cron10:00:002
15172991310,2cyclictest10458-21head08:05:232
15172991310,2cyclictest10458-21head08:05:222
1517299130,11cyclictest0-21swapper/207:30:112
1516999139,3cyclictest27026-21cron07:35:001
1516999139,3cyclictest16593-21cut12:35:141
1516999139,3cyclictest14636-21ssh10:45:211
1516999139,3cyclictest11318-21cron08:10:001
1516999139,3cyclictest11318-21cron08:10:001
1516999136,6cyclictest28569-21/usr/sbin/munin10:10:191
15169991310,2cyclictest791-21turbostat.cron11:20:001
15169991310,2cyclictest6662-21wc09:05:201
15169991310,2cyclictest21722-21head10:50:181
15169991310,2cyclictest17503-21wc11:50:281
15169991310,2cyclictest17097-21tail08:20:181
15167991310,2cyclictest9812-21grep11:45:280
15167991310,2cyclictest24006-21head11:55:160
15167991310,2cyclictest22105-21head10:50:220
15167991310,2cyclictest14916-21cron09:15:000
1516799130,1cyclictest0-21swapper/012:18:250
1516799130,11cyclictest0-21swapper/010:30:450
1517799129,2cyclictest8514-21tail08:00:343
1517799129,2cyclictest5964-21cut09:30:173
1517799129,2cyclictest48982sleep310:15:393
1517799129,2cyclictest3343-21fgrep07:50:233
1517799129,2cyclictest31477-21grep08:50:203
1517799129,2cyclictest31315-21grep07:40:273
1517799129,2cyclictest26661-21head08:40:153
1517799129,2cyclictest16155-21ssh09:15:173
1517799129,2cyclictest1309-21sed07:45:303
1517799120,2cyclictest0-21swapper/311:18:483
1517299129,2cyclictest8530-21tail08:00:342
1517299129,2cyclictest6591-21wc09:05:202
1517299129,2cyclictest497-21cron08:55:002
1517299129,2cyclictest4118-21sed11:00:332
1517299129,2cyclictest318-21tail07:45:182
1517299129,2cyclictest31795-21cron07:45:002
1517299129,2cyclictest2685-21grep12:25:192
1517299129,2cyclictest2270-21tr07:50:132
1517299129,2cyclictest21440-21tr09:40:262
1517299129,2cyclictest20955-21cut07:20:182
1517299129,2cyclictest19103-21wc07:15:252
1517299129,2cyclictest17818-21grep11:50:312
1517299129,2cyclictest15110-21tr08:15:242
1517299128,3cyclictest32-21ksoftirqd/208:20:162
1517299128,3cyclictest30253-21taskset08:48:382
1516999129,2cyclictest9937-21chrt09:10:301
1516999129,2cyclictest9695-21chrt11:05:181
1516999129,2cyclictest6566-21turbostat.cron08:00:001
1516999129,2cyclictest3421-21tail09:50:301
1516999129,2cyclictest31735-21head09:25:261
1516999129,2cyclictest19524-21sed08:25:171
1516999129,2cyclictest18709-21tail10:25:281
1516999129,2cyclictest17250-21grep09:15:291
1516999129,2cyclictest15628-21tr07:10:131
1516999128,3cyclictest7206-21grep09:30:301
1516999128,3cyclictest27086-21sed08:40:211
1516999128,3cyclictest20885-21grep09:40:191
1516999128,2cyclictest9765-21grep11:25:141
1516999128,2cyclictest31472-21cut08:50:201
1516999127,3cyclictest20956-21tail07:20:181
1516999120,11cyclictest17912-21grep11:10:191
1516799129,2cyclictest9274-21tr12:30:150
1516799129,2cyclictest4200-21sed11:00:330
1516799129,2cyclictest30432-21tail07:40:170
1516799129,2cyclictest29482-21ssh10:55:210
1516799129,2cyclictest24647-21cron07:30:000
1516799129,2cyclictest2364-21head09:50:190
1516799129,2cyclictest19006-21ssh11:10:290
1516799129,2cyclictest16679-21cat11:09:590
1516799129,2cyclictest13233-21grep12:10:200
1516799125,6cyclictest6571-21munin-run08:00:000
1516799120,9cyclictest0-21swapper/010:41:420
1517799118,2cyclictest9433-21expr12:30:163
1517799118,2cyclictest6145-21head12:05:173
1517799118,2cyclictest4115-21ssh12:25:453
1517799118,2cyclictest31411-21chrt10:11:303
1517799118,2cyclictest2951-21uniq11:00:183
1517799118,2cyclictest2646-21taskset08:56:473
1517799118,2cyclictest25946-21sh10:30:283
1517799118,2cyclictest25498-21taskset08:38:473
1517799118,2cyclictest24965-21rm09:20:373
1517799118,2cyclictest24825-21cat11:55:263
1517799118,2cyclictest23658-21rm12:17:263
1517799118,2cyclictest21849-21grep07:20:293
1517799118,2cyclictest21532-21expr08:30:133
1517799118,2cyclictest17825-21cat11:30:003
1517799118,2cyclictest1686-21ssh10:36:153
1517799118,2cyclictest16813-21expr12:35:153
1517799118,2cyclictest16449-21ssh11:50:173
1517799118,2cyclictest1404-21ssh09:50:093
1517799118,2cyclictest12448-21ssh10:21:113
1517799118,2cyclictest10176-21grep09:10:313
1517799117,3cyclictest20845-21cron08:30:003
1517799113,7cyclictest0-21swapper/312:20:313
1517799110,2cyclictest0-21swapper/309:45:023
1517299118,2cyclictest27210-21wc08:40:222
1517299118,2cyclictest25737-21sh12:19:202
1517299118,2cyclictest2374-21wc08:55:302
1517299118,2cyclictest21361-21tr08:30:132
1517299117,3cyclictest14126-21ssh10:22:532
1516999119,1cyclictest273092sleep107:35:131
1516999118,2cyclictest9491-21cat12:30:151
1516999118,2cyclictest4220-21ssh11:41:441
1516999118,2cyclictest31414-21sed07:40:281
1516999118,2cyclictest29824-21sh12:21:181
1516999118,2cyclictest26529-21ssh10:30:581
1516999118,2cyclictest26382-21sh11:35:181
1516999118,2cyclictest23601-21grep09:20:161
1516999118,2cyclictest20657-21cut12:15:211
1516999118,2cyclictest19085-21turbostat12:15:001
1516999118,2cyclictest18837-21grep07:15:211
1516999118,2cyclictest1711-21taskset07:49:381
1516999118,2cyclictest16727-21grep10:00:271
1516999118,2cyclictest12370-21cut08:10:211
1516999118,2cyclictest10368-21taskset09:55:311
1516999117,3cyclictest28941-21cut08:45:171
1516999117,3cyclictest23326-21chrt11:33:071
1516999110,10cyclictest7965-21ssh10:40:261
1516799119,2cyclictest15379-21python10:45:300
1516799119,1cyclictest32106-21tr07:45:140
1516799119,1cyclictest26168-21sort11:15:180
1516799119,1cyclictest20016-21cat07:20:000
1516799118,2cyclictest6869-21head09:05:230
1516799118,2cyclictest6445-21rm10:17:070
1516799118,2cyclictest4719-21chrt09:30:000
1516799118,2cyclictest32076-21grep08:50:270
1516799118,2cyclictest30213-21ssh09:46:560
1516799118,2cyclictest25907-21apache_accesses11:35:140
1516799118,2cyclictest24213-21rm11:33:570
1516799118,2cyclictest22938-21grep10:05:250
1516799118,2cyclictest22341-21ssh09:40:470
1516799118,2cyclictest21013-21cat08:30:010
1516799118,2cyclictest17069-21tune2fs08:20:170
1516799118,2cyclictest1639-21ssh12:25:030
1516799118,2cyclictest15568-21sort10:00:160
1516799118,2cyclictest13403-21sh10:22:100
1516799114,4cyclictest7744-21snmpd08:55:150
1516799112,8cyclictest30395-21awk08:50:000
1516799112,6cyclictest7744-21snmpd11:22:560
1516799110,10cyclictest7312-21expr08:00:200
1516799110,10cyclictest287122sleep007:35:280
1517799108,1cyclictest28247-21basename08:45:093
1517799108,1cyclictest21529-21ssh10:05:083
1517799108,1cyclictest19807-21taskset07:19:073
1517799108,1cyclictest16369-21chrt08:20:113
1517799108,1cyclictest11982-21expr08:10:193
1517799107,2cyclictest9625-21expr08:05:153
1517799107,2cyclictest9625-21expr08:05:153
1517799107,2cyclictest24542-21taskset07:28:403
1517799107,2cyclictest16180-21expr07:10:173
1517799104,2cyclictest224-21systemd-journal08:00:003
1517299108,1cyclictest29251-21taskset07:38:082
1517299108,1cyclictest23730-21cut08:35:182
1517299107,2cyclictest4978-21cat07:55:132
1516999108,1cyclictest87182sleep108:02:061
1516999108,1cyclictest3283-21cut07:50:231
1516999108,1cyclictest32490-21taskset12:00:311
1516999108,1cyclictest2339-21timerwakeupswit08:55:291
1516999107,2cyclictest4842-21grep09:00:561
1516999107,2cyclictest4135-21taskset12:25:461
1516999107,2cyclictest31508-21cut10:35:141
1516999107,2cyclictest28141-21ssh10:55:091
1516999107,2cyclictest26494-21cat09:45:011
1516799108,1cyclictest5702-21expr09:30:130
1516799108,1cyclictest28429-21ssh10:10:020
1516799108,1cyclictest27547-21sed08:40:260
1516799108,1cyclictest27025-21ssh12:20:170
1516799108,1cyclictest24878-21date07:30:000
1516799108,1cyclictest20148-21cat07:20:110
1516799108,1cyclictest16962-21ssh12:35:150
1516799108,1cyclictest14721-21grep08:15:200
1516799107,2cyclictest21770-21sort08:30:160
1516799107,2cyclictest2087-21grep11:40:230
1516799107,2cyclictest16142-21cstates11:50:150
1516799100,9cyclictest23610-21aten2_r7power_v08:35:170
151779998,1cyclictest6086-21expr09:05:153
151699997,1cyclictest24855-21cut08:35:281
151699996,2cyclictest13955-21basename08:15:101
151679997,2cyclictest10460-21awk08:05:230
151679997,2cyclictest10460-21awk08:05:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional