You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 06:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Apr 20, 2024 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224229885,9sleep30-21swapper/319:09:263
194229379,10sleep10-21swapper/119:05:351
226927966,8sleep00-21swapper/019:09:470
202227565,6sleep20-21swapper/219:06:362
265612670,1sleep026559-21ssh22:15:340
198162660,0sleep10-21swapper/121:48:181
2380993332,1cyclictest0-21swapper/223:30:162
2377993026,3cyclictest94-21kswapd021:42:341
238099290,28cyclictest0-21swapper/200:15:022
238099290,23cyclictest0-21swapper/221:20:042
238099260,2cyclictest4036-21/usr/sbin/munin22:45:332
238299231,0cyclictest0-21swapper/300:35:243
238099230,22cyclictest0-21swapper/222:56:342
2382992217,4cyclictest14516-21latency_hist21:45:013
238299220,1cyclictest0-21swapper/323:46:023
238099220,18cyclictest0-21swapper/222:07:462
2382992119,1cyclictest0-21swapper/319:25:123
2380992118,0cyclictest0-21swapper/223:02:272
237799213,12cyclictest1291-21runrttasks23:23:031
238299200,0cyclictest0-21swapper/322:07:383
238099202,18cyclictest27553-21diskmemload23:22:422
238099200,19cyclictest0-21swapper/221:42:592
238099200,16cyclictest0-21swapper/200:00:012
238099190,19cyclictest0-21swapper/223:27:232
237599194,8cyclictest9945-21sendmail-msp21:20:000
237599190,1cyclictest0-21swapper/021:44:290
238099180,3cyclictest21656-21sed21:49:592
237799181,16cyclictest0-21swapper/123:00:131
2375991817,1cyclictest0-21swapper/021:13:000
238299174,3cyclictest224-21systemd-journal21:30:003
238299170,17cyclictest0-21swapper/322:33:293
238299170,17cyclictest0-21swapper/321:50:293
238299170,0cyclictest0-21swapper/322:21:493
238299170,0cyclictest0-21swapper/322:10:293
238099170,1cyclictest0-21swapper/200:11:032
2377991716,1cyclictest0-21swapper/122:17:391
2382991615,1cyclictest0-21swapper/321:19:293
2380991614,1cyclictest0-21swapper/220:05:182
238099160,1cyclictest0-21swapper/223:11:192
238099160,14cyclictest0-21swapper/222:40:582
2377991615,1cyclictest0-21swapper/100:13:531
237599165,6cyclictest26158-21awk19:59:590
237599160,1cyclictest0-21swapper/021:29:520
238099159,5cyclictest25654-21tail19:55:362
238099150,15cyclictest0-21swapper/223:35:182
2377991512,2cyclictest27538-21cron21:10:001
237799151,10cyclictest12396-21latency_hist00:25:001
237599151,7cyclictest25177-21munin-run21:05:000
2375991510,2cyclictest13-21ksoftirqd/022:45:500
2382991412,1cyclictest0-21swapper/321:58:573
2382991411,2cyclictest7668-21wc20:25:253
2382991411,1cyclictest0-21swapper/323:20:533
2382991410,3cyclictest2307-21ssh23:30:153
2382991410,3cyclictest10269-21sh22:04:593
2382991410,2cyclictest9021-21grep20:30:153
2382991410,2cyclictest20175-21sh00:05:153
2382991410,2cyclictest14568-21grep19:35:143
238299140,12cyclictest0-21swapper/323:10:223
238099144,5cyclictest27553-21diskmemload22:25:242
238099142,4cyclictest224-21systemd-journal22:19:552
2380991411,2cyclictest442-21wc20:10:252
2380991410,3cyclictest26977-21grep00:35:252
238099140,14cyclictest0-21swapper/223:16:172
238099140,13cyclictest0-21swapper/223:40:442
238099140,13cyclictest0-21swapper/200:31:372
2377991411,2cyclictest9951-21ls22:50:211
2377991410,3cyclictest6701-21tr20:25:161
2377991410,3cyclictest20098-21cut00:30:191
2377991410,2cyclictest2643-21wc20:15:221
237599142,6cyclictest26740-21ssh23:25:010
2375991411,2cyclictest28544-21ls23:25:220
237599140,12cyclictest25698-21tr21:05:170
237599140,12cyclictest20911-21tr22:35:140
237599140,11cyclictest0-21swapper/021:53:110
238299139,3cyclictest11749-21ssh21:20:243
2382991310,2cyclictest7302-21wc19:20:003
2382991310,2cyclictest7302-21wc19:20:003
2382991310,2cyclictest39-21ksoftirqd/300:02:283
2382991310,2cyclictest21713-21wc21:50:003
2382991310,2cyclictest13455-21cat00:00:013
238299130,1cyclictest0-21swapper/322:40:213
238299130,1cyclictest0-21swapper/321:37:093
238299130,13cyclictest0-21swapper/323:05:433
238299130,12cyclictest5271-21tr20:20:213
2380991312,1cyclictest62050irq/128-enp2s0-00:25:212
2380991310,2cyclictest7257-21grep00:20:172
2380991310,2cyclictest5353-21wc20:20:212
2380991310,2cyclictest31178-21tail21:55:322
2380991310,2cyclictest26502-21wc21:05:242
2380991310,2cyclictest23880-21sed21:50:262
2380991310,2cyclictest22776-21awk21:00:002
2380991310,2cyclictest11833-21grep20:35:172
238099130,13cyclictest0-21swapper/223:50:512
237799139,3cyclictest3366-21wc23:30:251
237799139,3cyclictest16604-21chrt19:40:001
237799132,6cyclictest0-21swapper/122:35:111
237799131,9cyclictest24925-21cron21:30:001
2377991310,2cyclictest6945-21grep23:10:261
2377991310,2cyclictest26274-21apt-key19:59:591
2377991310,2cyclictest19956-21grep20:50:331
2377991310,2cyclictest19354-21grep22:10:281
2377991310,2cyclictest13815-21grep22:30:151
2377991310,2cyclictest12874-21cron22:30:001
237799130,12cyclictest0-21swapper/121:59:541
237599137,5cyclictest27399-21/usr/sbin/munin22:40:150
237599136,4cyclictest13863-21needreboot00:25:210
237599135,7cyclictest19696-21python20:50:310
237599131,4cyclictest224-21systemd-journal23:15:010
2375991313,0cyclictest0-21swapper/022:00:480
2375991310,2cyclictest8247-21grep00:20:280
2375991310,2cyclictest19182-21cat19:45:010
2375991310,2cyclictest12376-21grep19:30:180
2375991310,2cyclictest10400-21grep19:25:170
238299129,2cyclictest8437-21sh22:50:003
238299129,2cyclictest3537-21cut19:10:213
238299129,2cyclictest32747-21taskset23:28:453
238299129,2cyclictest24613-21basename22:15:153
238299129,2cyclictest22316-21grep19:50:243
238299129,2cyclictest21884-21sed20:55:223
238299129,2cyclictest12412-21grep19:30:173
238299128,3cyclictest12727-21wc20:35:283
238299128,2cyclictest10745-21grep22:50:293
238299126,5cyclictest24951-21perf00:35:013
2382991210,1cyclictest0-21swapper/322:28:543
238299120,1cyclictest0-21swapper/323:15:403
238299120,11cyclictest0-21swapper/321:13:553
238099129,2cyclictest7909-21wc23:55:192
238099129,2cyclictest7116-21grep20:25:192
238099129,2cyclictest5207-21grep19:15:152
238099129,2cyclictest5207-21grep19:15:152
238099129,2cyclictest4459-21ssh21:15:232
238099129,2cyclictest25855-21expr21:30:152
238099129,2cyclictest23286-21grep19:50:362
238099129,2cyclictest13858-21tr22:30:152
238099129,2cyclictest12400-21grep19:30:182
238099128,3cyclictest7527-21awk21:40:002
238099128,3cyclictest11468-21grep19:25:272
238099120,12cyclictest0-21swapper/222:51:212
237799129,2cyclictest7525-21uname23:55:161
237799129,2cyclictest4656-21grep20:20:141
237799129,2cyclictest29347-21grep22:40:311
237799129,2cyclictest27901-21grep23:25:161
237799129,2cyclictest26858-21grep20:00:121
237799129,2cyclictest26246-21sed21:30:171
237799129,2cyclictest22365-21seq21:50:141
237799129,2cyclictest17622-21grep20:45:291
237799129,2cyclictest15651-21sh20:45:001
237799129,2cyclictest13257-21cron20:40:011
237799128,3cyclictest23691-21wc23:45:011
237799128,3cyclictest17495-21uniq19:40:211
237799121,1cyclictest27553-21diskmemload22:24:461
237799120,9cyclictest9904-21latency_hist21:20:001
237799120,1cyclictest0-21swapper/121:39:501
237799120,12cyclictest0-21swapper/123:18:311
237599129,2cyclictest9548-21head20:30:170
237599129,2cyclictest7656-21cat23:55:170
237599129,2cyclictest6482-21tr20:25:160
237599129,2cyclictest4811-21cut20:20:150
237599129,2cyclictest377-21tr22:20:220
237599129,2cyclictest3759-21ntp_states19:10:230
237599129,2cyclictest2988-21tail23:30:210
237599129,2cyclictest29754-21sed20:05:230
237599129,2cyclictest29538-21tr21:55:190
237599129,2cyclictest19196-21grep00:30:130
237599129,2cyclictest18486-21sh21:46:590
237599129,2cyclictest14409-21sed22:30:170
237599128,3cyclictest28260-21grep20:00:250
237599122,3cyclictest141rcu_preempt19:50:210
237599120,1cyclictest0-21swapper/023:02:080
238299119,1cyclictest263052sleep321:30:173
238299118,2cyclictest9268-21ssh23:35:013
238299118,2cyclictest29873-21sh00:12:153
238299118,2cyclictest29229-21grep20:05:193
238299118,2cyclictest28477-21taskset20:02:003
238299118,2cyclictest28271-21ssh23:03:553
238299118,2cyclictest25704-21tr21:05:173
238299118,2cyclictest24304-21ssh22:37:123
238299118,2cyclictest19184-21ssh22:56:583
238299118,2cyclictest17828-21uname19:40:233
238299118,2cyclictest15404-21chrt20:41:543
238299118,2cyclictest1532-21ssh23:50:513
238299118,2cyclictest12944-21cat00:25:133
238299118,2cyclictest11349-21ssh00:23:583
238299118,2cyclictest1085-21taskset20:14:563
238299117,3cyclictest26156-21lockfile-create19:59:593
238299113,3cyclictest0-21swapper/323:41:173
238099118,2cyclictest597-21ssh21:13:232
238099118,2cyclictest5794-21tail22:00:312
238099118,2cyclictest31538-21sh23:05:242
238099118,2cyclictest28303-21head20:00:262
238099118,2cyclictest25192-21basename21:05:012
238099118,2cyclictest24689-21cut23:45:172
238099118,2cyclictest22339-21sh22:13:222
238099118,2cyclictest20958-21seq22:35:142
238099118,2cyclictest20912-21grep19:45:292
238099118,2cyclictest18903-21ssh21:25:182
238099118,2cyclictest18036-21awk19:40:252
238099118,2cyclictest17366-21cat20:45:282
238099118,2cyclictest1645-21tr20:15:132
238099118,2cyclictest15459-21wc19:35:202
238099118,2cyclictest14167-21cut20:40:152
238099114,6cyclictest20170-21grep00:05:152
238099114,6cyclictest1071-21munin-node22:20:462
237799118,2cyclictest9368-21ssh22:04:081
237799118,2cyclictest776-21ssh21:13:331
237799118,2cyclictest6703-21sed19:15:291
237799118,2cyclictest6703-21sed19:15:281
237799118,2cyclictest603-21expr00:15:181
237799118,2cyclictest4725-21perf19:15:001
237799118,2cyclictest3676-21ssh22:45:301
237799118,2cyclictest31712-21ssh23:05:251
237799118,2cyclictest31567-21wget23:50:131
237799118,2cyclictest29224-21tr20:05:191
237799118,2cyclictest24778-21head21:00:281
237799118,2cyclictest24774-21ssh23:45:171
237799118,2cyclictest22648-21taskset20:58:331
237799118,2cyclictest21170-21taskset19:47:361
237799118,2cyclictest17310-21ssh00:02:101
237799118,2cyclictest15068-21ssh00:26:051
237799118,2cyclictest12233-21grep19:30:171
237799118,2cyclictest12229-21ssh23:36:111
237799118,2cyclictest11853-21cron19:30:001
237799118,2cyclictest11300-21head21:20:201
237799118,2cyclictest10696-21chrt20:32:411
237799118,2cyclictest10638-21ssh22:05:081
237799117,3cyclictest32482-21wc20:10:171
237799114,3cyclictest224-21systemd-journal23:00:001
237799113,4cyclictest224-21systemd-journal00:06:511
237599119,1cyclictest16545-21grep22:55:160
237599119,1cyclictest1608-21grep20:15:130
237599119,1cyclictest14604-21tr19:35:150
237599118,2cyclictest9782-21grep23:35:150
237599118,2cyclictest7370-21apache_accesses19:20:090
237599118,2cyclictest6970-21seq19:18:360
237599118,2cyclictest6970-21seq19:18:350
237599118,2cyclictest4478-21chrt23:09:560
237599118,2cyclictest32436-21ssh00:15:000
237599118,2cyclictest27496-21sh21:30:330
237599118,2cyclictest26784-21ssh23:46:110
237599118,2cyclictest25660-21cut00:35:160
237599118,2cyclictest22290-21ssh23:43:550
237599118,2cyclictest21276-21ssh22:12:160
237599118,2cyclictest19155-21rm00:04:290
237599118,2cyclictest1849-21ls21:35:200
237599118,2cyclictest14465-21head23:15:320
237599118,2cyclictest13793-21grep20:40:130
237599118,2cyclictest1325-21ssh23:50:410
237599118,2cyclictest12134-21ssh22:51:440
237599118,2cyclictest12087-21wc22:05:220
237599118,2cyclictest11839-21tr21:20:250
237599117,3cyclictest23788-21ssh00:07:290
237599113,6cyclictest15926-21apt-get20:45:140
238299108,1cyclictest250062sleep321:03:043
238299108,1cyclictest20206-21cut19:45:213
238299108,1cyclictest2009-21grep20:15:153
238299108,1cyclictest18368-21cut20:50:203
238299108,1cyclictest1273-21uname00:15:223
238299107,2cyclictest9077-21grep19:20:253
238299102,4cyclictest224-21systemd-journal20:46:083
238099108,1cyclictest7731-21expr19:20:112
238099108,1cyclictest108212sleep220:34:182
238099107,2cyclictest18851-21grep20:50:222
238099102,2cyclictest4707-21cron19:15:002
237799108,1cyclictest9024-21sed19:20:261
237799107,2cyclictest25487-21ssh00:35:061
237799107,2cyclictest22238-21fschecks_count19:50:241
237599108,2cyclictest343-21sensors20:10:210
237599108,2cyclictest22528-21taskset20:56:590
237599108,1cyclictest6024-21aten2_r7power_e22:25:130
237599108,1cyclictest27440-21uname00:10:180
237599108,1cyclictest132262sleep020:39:420
237599102,4cyclictest224-21systemd-journal19:45:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional