You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-11 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot3s.osadl.org (updated Mon Dec 11, 2023 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
105491660,31ptp4l0-21swapper/107:05:301
105491650,58ptp4l0-21swapper/007:08:090
105491520,35ptp4l0-21swapper/207:06:092
2989825146,2sleep30-21swapper/307:06:213
64622420,0sleep00-21swapper/010:25:270
164342420,0sleep10-21swapper/110:45:011
105491230,0ptp4l0-21swapper/007:25:360
105491220,1ptp4l3549-21unixbench_singl08:15:381
3044499182,3cyclictest121rcu_preempt12:15:293
105491180,0ptp4l0-21swapper/008:35:230
30442991717,0cyclictest0-21swapper/107:40:251
105491170,0ptp4l0-21swapper/107:15:301
105491160,1ptp4l121rcu_preempt07:15:310
105491160,0ptp4l0-21swapper/112:25:291
105491160,0ptp4l0-21swapper/112:00:281
105491160,0ptp4l0-21swapper/110:18:521
105491160,0ptp4l0-21swapper/109:20:321
105491160,0ptp4l0-21swapper/108:40:311
105491160,0ptp4l0-21swapper/107:10:371
105491160,0ptp4l0-21swapper/008:20:360
3044499151,0cyclictest121rcu_preempt11:40:343
105491150,1ptp4l7282-21ntp_states10:25:311
105491150,1ptp4l4637-21ntp_states10:20:311
105491150,1ptp4l28632-21snmp_o5powerctl12:05:380
105491150,1ptp4l27681-21snmp_o5powerctl08:00:371
105491150,1ptp4l25673-21ls11:00:281
105491150,1ptp4l23334-21apt-get07:55:201
105491150,1ptp4l21172-21apt-get11:55:010
105491150,1ptp4l13525-21memory07:35:271
105491150,1ptp4l13022-21ls10:35:371
105491150,1ptp4l12251-21ntp_states12:35:271
105491150,1ptp4l11173-21snmp_o5powerctl08:30:350
105491150,1ptp4l10820-21apt-get11:35:181
105491150,0ptp4l0-21swapper/109:40:191
105491150,0ptp4l0-21swapper/107:34:101
105491150,0ptp4l0-21swapper/011:45:210
105491150,0ptp4l0-21swapper/011:32:230
105491150,0ptp4l0-21swapper/010:55:190
105491150,0ptp4l0-21swapper/010:51:420
105491150,0ptp4l0-21swapper/010:37:360
105491150,0ptp4l0-21swapper/010:10:190
105491150,0ptp4l0-21swapper/009:45:190
105491150,0ptp4l0-21swapper/009:00:230
105491150,0ptp4l0-21swapper/008:19:560
105491150,0ptp4l0-21swapper/007:10:020
30444991413,0cyclictest41-21ksoftirqd/308:20:003
105491140,1ptp4l9570-21/usr/sbin/munin12:30:310
105491140,1ptp4l8779-21date09:30:001
105491140,1ptp4l3834-21irqstats12:20:280
105491140,1ptp4l3690-21irqrtprio11:20:250
105491140,1ptp4l337-21ls12:15:230
105491140,1ptp4l30293-21hddtemp_smartct12:10:271
105491140,1ptp4l29380-21python10:05:470
105491140,1ptp4l28948-21cstates08:05:250
105491140,1ptp4l28948-21cstates08:05:240
105491140,1ptp4l28201-21fschecks_count10:05:241
105491140,1ptp4l26064-21irqrtprio10:00:281
105491140,1ptp4l25-21ksoftirqd/110:55:191
105491140,1ptp4l22145-21smartctl07:50:320
105491140,1ptp4l15523-21python10:40:300
105491140,1ptp4l13582-21telnet11:40:210
105491140,1ptp4l11311-21munin-run09:35:000
105491140,0ptp4l0-21swapper/210:00:152
105491140,0ptp4l0-21swapper/209:20:402
105491140,0ptp4l0-21swapper/208:01:182
105491140,0ptp4l0-21swapper/112:15:241
105491140,0ptp4l0-21swapper/111:34:191
105491140,0ptp4l0-21swapper/110:50:321
105491140,0ptp4l0-21swapper/110:45:201
105491140,0ptp4l0-21swapper/109:55:331
105491140,0ptp4l0-21swapper/109:45:041
105491140,0ptp4l0-21swapper/109:05:271
105491140,0ptp4l0-21swapper/108:50:361
105491140,0ptp4l0-21swapper/108:45:371
105491140,0ptp4l0-21swapper/108:30:271
105491140,0ptp4l0-21swapper/108:20:341
105491140,0ptp4l0-21swapper/012:35:340
105491140,0ptp4l0-21swapper/011:52:490
105491140,0ptp4l0-21swapper/011:10:330
105491140,0ptp4l0-21swapper/011:00:330
105491140,0ptp4l0-21swapper/010:00:400
105491140,0ptp4l0-21swapper/009:56:090
105491140,0ptp4l0-21swapper/009:40:290
105491140,0ptp4l0-21swapper/009:25:260
105491140,0ptp4l0-21swapper/009:19:450
105491140,0ptp4l0-21swapper/008:55:470
105491140,0ptp4l0-21swapper/008:50:360
105491140,0ptp4l0-21swapper/008:49:150
105491140,0ptp4l0-21swapper/008:10:590
30444991313,0cyclictest41-21ksoftirqd/307:30:333
105491130,1ptp4l10001-21expr07:30:240
105491130,0ptp4l0-21swapper/212:15:332
105491130,0ptp4l0-21swapper/212:10:322
105491130,0ptp4l0-21swapper/211:55:342
105491130,0ptp4l0-21swapper/210:45:362
105491130,0ptp4l0-21swapper/210:40:222
105491130,0ptp4l0-21swapper/209:25:332
105491130,0ptp4l0-21swapper/207:38:512
105491130,0ptp4l0-21swapper/112:30:031
105491130,0ptp4l0-21swapper/112:20:211
105491130,0ptp4l0-21swapper/112:05:371
105491130,0ptp4l0-21swapper/111:55:201
105491130,0ptp4l0-21swapper/111:50:251
105491130,0ptp4l0-21swapper/111:45:241
105491130,0ptp4l0-21swapper/111:40:311
105491130,0ptp4l0-21swapper/111:25:181
105491130,0ptp4l0-21swapper/111:20:231
105491130,0ptp4l0-21swapper/111:15:231
105491130,0ptp4l0-21swapper/111:10:271
105491130,0ptp4l0-21swapper/111:05:301
105491130,0ptp4l0-21swapper/110:30:291
105491130,0ptp4l0-21swapper/110:10:201
105491130,0ptp4l0-21swapper/109:51:061
105491130,0ptp4l0-21swapper/109:35:211
105491130,0ptp4l0-21swapper/109:30:271
105491130,0ptp4l0-21swapper/109:15:131
105491130,0ptp4l0-21swapper/109:10:271
105491130,0ptp4l0-21swapper/109:00:241
105491130,0ptp4l0-21swapper/108:55:391
105491130,0ptp4l0-21swapper/108:35:231
105491130,0ptp4l0-21swapper/108:25:251
105491130,0ptp4l0-21swapper/108:10:331
105491130,0ptp4l0-21swapper/108:05:311
105491130,0ptp4l0-21swapper/108:05:301
105491130,0ptp4l0-21swapper/107:50:241
105491130,0ptp4l0-21swapper/107:45:251
105491130,0ptp4l0-21swapper/107:25:321
105491130,0ptp4l0-21swapper/107:20:111
105491130,0ptp4l0-21swapper/012:25:250
105491130,0ptp4l0-21swapper/012:10:280
105491130,0ptp4l0-21swapper/012:00:250
105491130,0ptp4l0-21swapper/011:35:220
105491130,0ptp4l0-21swapper/011:25:010
105491130,0ptp4l0-21swapper/011:15:240
105491130,0ptp4l0-21swapper/011:05:300
105491130,0ptp4l0-21swapper/010:45:290
105491130,0ptp4l0-21swapper/010:30:210
105491130,0ptp4l0-21swapper/010:20:210
105491130,0ptp4l0-21swapper/010:15:240
105491130,0ptp4l0-21swapper/009:51:040
105491130,0ptp4l0-21swapper/009:35:270
105491130,0ptp4l0-21swapper/009:20:240
105491130,0ptp4l0-21swapper/009:10:290
105491130,0ptp4l0-21swapper/009:05:250
105491130,0ptp4l0-21swapper/008:40:290
105491130,0ptp4l0-21swapper/008:25:260
105491130,0ptp4l0-21swapper/008:00:070
105491130,0ptp4l0-21swapper/007:55:220
105491130,0ptp4l0-21swapper/007:45:060
105491130,0ptp4l0-21swapper/007:40:260
105491130,0ptp4l0-21swapper/007:35:250
105491130,0ptp4l0-21swapper/007:20:320
30444991210,1cyclictest41-21ksoftirqd/309:55:223
30443991210,1cyclictest5081-21ls10:20:352
105491120,0ptp4l0-21swapper/212:25:242
105491120,0ptp4l0-21swapper/210:55:242
105491120,0ptp4l0-21swapper/207:25:232
105491120,0ptp4l0-21swapper/207:10:232
3044499119,1cyclictest41-21ksoftirqd/311:50:343
3044399113,6cyclictest0-21swapper/208:55:412
3044499109,0cyclictest41-21ksoftirqd/308:20:323
30444991010,0cyclictest41-21ksoftirqd/311:00:253
30444991010,0cyclictest41-21ksoftirqd/310:40:333
3044399106,3cyclictest0-21swapper/212:20:302
3044399100,9cyclictest31057-21tail11:10:272
105491100,1ptp4l0-21swapper/210:26:112
105491100,1ptp4l0-21swapper/208:19:302
105491100,0ptp4l0-21swapper/212:31:492
105491100,0ptp4l0-21swapper/209:35:212
105491100,0ptp4l0-21swapper/209:18:322
304449999,0cyclictest41-21ksoftirqd/311:25:283
304449999,0cyclictest41-21ksoftirqd/309:35:213
304449998,0cyclictest41-21ksoftirqd/312:35:203
304449998,0cyclictest41-21ksoftirqd/311:25:023
304449997,1cyclictest9359-21chrt07:29:353
304449997,1cyclictest866-21tail08:10:583
304449997,1cyclictest5384-21chrt10:21:493
304449997,1cyclictest4069-21expr09:20:253
304449997,1cyclictest3359-21cat09:20:013
304449997,1cyclictest32180-21snmp_o5powerctl07:10:383
304449997,1cyclictest30482-21cat12:10:293
304449997,1cyclictest25146-21turbostat09:00:023
304449997,1cyclictest10729-21cut11:35:003
304449996,2cyclictest2337-21tail07:15:243
304439999,0cyclictest311rcuc/212:35:252
304439997,1cyclictest9556-21cut10:30:262
304439997,1cyclictest31456-21tr10:10:272
304439997,1cyclictest27943-21cut12:05:312
304439997,1cyclictest24011-21cat09:55:362
304439997,1cyclictest21379-21chrt10:53:062
304439997,1cyclictest18912-21egrep11:50:222
304439997,1cyclictest1450-21grep11:15:292
304439997,1cyclictest10953-21taskset09:30:372
304439992,4cyclictest24954-21head12:00:252
304439992,1cyclictest33-21ksoftirqd/208:45:402
304439990,8cyclictest30005-21chrt10:09:102
10549190,1ptp4l0-21swapper/211:43:302
10549190,0ptp4l0-21swapper/308:38:313
10549190,0ptp4l0-21swapper/308:06:233
10549190,0ptp4l0-21swapper/308:06:223
10549190,0ptp4l0-21swapper/211:47:392
10549190,0ptp4l0-21swapper/211:35:592
10549190,0ptp4l0-21swapper/211:32:032
10549190,0ptp4l0-21swapper/211:26:262
10549190,0ptp4l0-21swapper/211:22:122
10549190,0ptp4l0-21swapper/211:05:192
10549190,0ptp4l0-21swapper/211:04:212
10549190,0ptp4l0-21swapper/210:36:172
10549190,0ptp4l0-21swapper/210:17:532
10549190,0ptp4l0-21swapper/209:53:242
10549190,0ptp4l0-21swapper/209:46:152
10549190,0ptp4l0-21swapper/209:43:092
10549190,0ptp4l0-21swapper/209:10:402
10549190,0ptp4l0-21swapper/209:06:532
10549190,0ptp4l0-21swapper/209:02:432
10549190,0ptp4l0-21swapper/208:50:072
10549190,0ptp4l0-21swapper/208:43:282
10549190,0ptp4l0-21swapper/208:38:122
10549190,0ptp4l0-21swapper/208:33:142
10549190,0ptp4l0-21swapper/208:27:022
10549190,0ptp4l0-21swapper/208:20:492
10549190,0ptp4l0-21swapper/208:10:222
10549190,0ptp4l0-21swapper/208:09:132
10549190,0ptp4l0-21swapper/208:09:132
10549190,0ptp4l0-21swapper/207:56:512
10549190,0ptp4l0-21swapper/207:51:462
10549190,0ptp4l0-21swapper/207:45:282
10549190,0ptp4l0-21swapper/207:40:162
10549190,0ptp4l0-21swapper/207:30:442
10549190,0ptp4l0-21swapper/207:22:332
10549190,0ptp4l0-21swapper/207:19:542
304449987,0cyclictest41-21ksoftirqd/310:05:213
304449986,1cyclictest9601-21aten_r9power_vo08:30:253
304449986,1cyclictest9117-21cat09:30:243
304449986,1cyclictest8772-21tr10:30:203
304449986,1cyclictest7987-21dpkg12:30:023
304449986,1cyclictest7202-21expr08:25:243
304449986,1cyclictest7132-21chrt09:25:333
304449986,1cyclictest6257-21cut12:25:273
304449986,1cyclictest5898-21apt10:25:233
304449986,1cyclictest4482-21cut07:20:213
304449986,1cyclictest41-21ksoftirqd/312:00:223
304449986,1cyclictest41-21ksoftirqd/310:35:343
304449986,1cyclictest32120-21sort10:10:343
304449986,1cyclictest3013-21cat12:20:243
304449986,1cyclictest30059-21expr11:10:223
304449986,1cyclictest28323-21chrt08:04:173
304449986,1cyclictest28026-21seq09:05:153
304449986,1cyclictest26984-21sh12:05:253
304449986,1cyclictest26452-21cut09:00:293
304449986,1cyclictest25913-21if_err_enp4s010:00:273
304449986,1cyclictest23112-21cat07:55:023
304449986,1cyclictest22397-21cat11:55:243
304449986,1cyclictest20170-21grep09:51:053
304449986,1cyclictest20054-21seq08:50:103
304449986,1cyclictest19331-21grep10:50:203
304449986,1cyclictest18608-21cat07:45:253
304449986,1cyclictest18152-21hostname08:45:283
304449986,1cyclictest17361-21chrt07:42:243
304449986,1cyclictest17185-21sed10:45:263
304449986,1cyclictest17070-21taskset08:43:323
304449986,1cyclictest168212sleep309:45:013
304449986,1cyclictest16110-21acpi11:45:213
304449986,1cyclictest14533-21expr09:40:223
304449986,1cyclictest12495-21cat07:35:203
304449986,1cyclictest12420-21wc11:35:293
304449986,1cyclictest1198-21tail10:15:213
304449986,1cyclictest1027-21sed11:15:263
10549180,0ptp4l0-21swapper/307:50:173
304449975,1cyclictest31052-21cat09:10:263
304449975,1cyclictest27873-21cat11:05:253
304449975,1cyclictest21974-21wc10:55:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional