You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-07 - 15:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot4s.osadl.org (updated Thu Dec 07, 2023 12:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
277532169151,12sleep30-21swapper/307:05:423
2842499135133,1cyclictest0-21swapper/207:14:402
2658721260,0sleep30-21swapper/309:00:013
2842499121118,2cyclictest30323-21kworker/u16:1+events_unbound10:35:262
279432118105,9sleep20-21swapper/207:05:552
279672115102,10sleep10-21swapper/107:06:131
22941210894,10sleep00-21swapper/007:05:150
2842499103100,2cyclictest16301-21kworker/u16:3+events_unbound07:37:302
28421999895,2cyclictest30323-21kworker/u16:1+events_unbound10:05:341
28424998874,12cyclictest30323-21kworker/u16:1+events_unbound10:12:412
28421998883,2cyclictest0-21swapper/107:14:391
28424998076,3cyclictest16301-21kworker/u16:3+events_unbound07:15:382
28424997975,3cyclictest31244-21kworker/u16:2+events_unbound09:10:352
28424997975,3cyclictest16301-21kworker/u16:3+events_unbound09:25:382
28424997971,3cyclictest30323-21kworker/u16:1+events_unbound11:25:362
28424997874,3cyclictest31244-21kworker/u16:2+events_unbound10:45:362
28424997871,3cyclictest31244-21kworker/u16:2+events_unbound10:30:362
28424997761,12cyclictest31244-21kworker/u16:2+events_unbound08:04:412
28424997672,3cyclictest31244-21kworker/u16:2+events_unbound11:40:342
28424997571,3cyclictest31244-21kworker/u16:2+events_unbound11:20:322
28424997571,3cyclictest30323-21kworker/u16:1+events_unbound11:00:362
28421997569,5cyclictest31244-21kworker/u16:2+events_unbound08:10:231
28424997470,3cyclictest16301-21kworker/u16:3+events_unbound08:15:342
28424997064,3cyclictest16301-21kworker/u16:3+events_unbound08:30:372
28424996965,3cyclictest31244-21kworker/u16:2+events_unbound12:15:332
28424996962,3cyclictest30323-21kworker/u16:1+events_unbound10:40:322
28424996864,3cyclictest31244-21kworker/u16:2+events_unbound11:55:382
28424996864,3cyclictest16301-21kworker/u16:3+events_unbound07:20:412
28424996763,3cyclictest31244-21kworker/u16:2+events_unbound09:05:382
28424996763,3cyclictest31244-21kworker/u16:2+events_unbound08:10:312
28424996761,5cyclictest30323-21kworker/u16:1+events_unbound11:10:212
28424996561,3cyclictest31244-21kworker/u16:2+events_unbound11:30:342
28424996561,3cyclictest30323-21kworker/u16:1+events_unbound12:30:342
28424996561,3cyclictest16301-21kworker/u16:3+events_unbound09:00:322
28424996460,3cyclictest31244-21kworker/u16:2+events_unbound08:50:352
28424996460,3cyclictest30323-21kworker/u16:1+events_unbound11:50:342
28424996460,3cyclictest16301-21kworker/u16:3+events_unbound09:20:332
28421996360,2cyclictest31244-21kworker/u16:2+events_unbound10:15:331
28424996258,3cyclictest31244-21kworker/u16:2+events_unbound12:05:002
28424996258,3cyclictest31244-21kworker/u16:2+events_unbound10:50:362
28424996258,3cyclictest31244-21kworker/u16:2+events_unbound09:40:332
28424996258,3cyclictest30323-21kworker/u16:1+events_unbound12:20:342
28424996258,3cyclictest16301-21kworker/u16:3+events_unbound09:55:362
28421996252,6cyclictest31244-21kworker/u16:2+events_unbound10:25:281
28421996155,2cyclictest31244-21kworker/u16:2+events_unbound08:20:211
28424996056,3cyclictest30323-21kworker/u16:1+events_unbound12:35:362
28424996056,3cyclictest16301-21kworker/u16:3+events_unbound09:30:352
28424996056,3cyclictest16301-21kworker/u16:3+events_unbound08:35:242
28424995956,2cyclictest16301-21kworker/u16:3+events_unbound08:40:022
28424995955,3cyclictest31244-21kworker/u16:2+events_unbound07:55:342
28424995854,3cyclictest31244-21kworker/u16:2+events_unbound08:25:032
28424995854,3cyclictest16301-21kworker/u16:3+events_unbound08:45:382
28424995854,3cyclictest16301-21kworker/u16:3+events_unbound07:25:212
28424995753,3cyclictest30323-21kworker/u16:1+events_unbound12:00:022
28424995652,3cyclictest31244-21kworker/u16:2+events_unbound10:55:372
28424995652,3cyclictest30323-21kworker/u16:1+events_unbound10:05:332
28424995652,3cyclictest16301-21kworker/u16:3+events_unbound08:05:342
28424995550,4cyclictest30323-21kworker/u16:1+events_unbound12:25:212
28424995349,3cyclictest31244-21kworker/u16:2+events_unbound07:50:332
28424995349,3cyclictest31244-21kworker/u16:2+events_unbound07:30:242
28424995248,3cyclictest31244-21kworker/u16:2+events_unbound10:25:352
28424995248,3cyclictest31244-21kworker/u16:2+events_unbound10:20:362
28424995248,3cyclictest31244-21kworker/u16:2+events_unbound07:45:382
28424995147,3cyclictest31244-21kworker/u16:2+events_unbound11:45:212
28424995147,3cyclictest31244-21kworker/u16:2+events_unbound10:15:282
28424995147,3cyclictest31244-21kworker/u16:2+events_unbound09:15:202
28424995147,3cyclictest16301-21kworker/u16:3+events_unbound08:55:202
28417995146,4cyclictest31244-21kworker/u16:2+events_unbound10:35:020
28417995041,8cyclictest0-21swapper/007:14:390
28424994844,3cyclictest31244-21kworker/u16:2+events_unbound09:45:362
28421994846,2cyclictest31244-21kworker/u16:2+events_unbound07:30:351
28424994743,3cyclictest31244-21kworker/u16:2+events_unbound12:10:242
28424994743,3cyclictest31244-21kworker/u16:2+events_unbound09:35:252
28424994740,3cyclictest31244-21kworker/u16:2+events_unbound10:00:332
28424994642,3cyclictest31244-21kworker/u16:2+events_unbound11:05:252
28424994642,3cyclictest31244-21kworker/u16:2+events_unbound09:50:232
28421994643,3cyclictest30323-21kworker/u16:1+events_unbound11:35:331
28421994643,2cyclictest31244-21kworker/u16:2+events_unbound12:30:321
28421994541,4cyclictest31244-21kworker/u16:2+events_unbound07:22:391
28417994540,4cyclictest16301-21kworker/u16:3+events_unbound08:40:370
28424994440,3cyclictest31244-21kworker/u16:2+events_unbound07:40:022
28424994339,3cyclictest31244-21kworker/u16:2+events_unbound11:35:212
28424994339,3cyclictest31244-21kworker/u16:2+events_unbound11:15:232
28424994138,2cyclictest31244-21kworker/u16:2+events_unbound08:20:022
28421994036,3cyclictest16301-21kworker/u16:3+events_unbound09:20:191
28421994035,3cyclictest30323-21kworker/u16:1+events_unbound11:30:211
28421994035,3cyclictest30323-21kworker/u16:1+events_unbound11:10:261
28417994036,3cyclictest31244-21kworker/u16:2+events_unbound07:50:020
28417993935,3cyclictest16301-21kworker/u16:3+events_unbound09:00:020
28421993831,5cyclictest16301-21kworker/u16:3+events_unbound07:25:191
28421993730,6cyclictest31244-21kworker/u16:2+events_unbound08:41:411
28421993718,15cyclictest30323-21kworker/u16:1+events_unbound12:19:161
28417993734,2cyclictest31244-21kworker/u16:2+events_unbound09:10:260
28421993632,3cyclictest31244-21kworker/u16:2+events_unbound09:40:241
28421993631,4cyclictest30323-21kworker/u16:1+events_unbound12:00:211
28421993529,5cyclictest30323-21kworker/u16:1+events_unbound11:50:201
28421993529,4cyclictest31244-21kworker/u16:2+events_unbound10:20:231
28421993522,12cyclictest31244-21kworker/u16:2+events_unbound07:16:211
28417993530,4cyclictest30323-21kworker/u16:1+events_unbound11:50:280
28421993430,3cyclictest31244-21kworker/u16:2+events_unbound08:55:261
28421993429,4cyclictest30323-21kworker/u16:1+events_unbound11:40:221
28421993428,4cyclictest16301-21kworker/u16:3+events_unbound08:15:321
28417993429,4cyclictest31244-21kworker/u16:2+events_unbound07:55:320
28421993329,3cyclictest31244-21kworker/u16:2+events_unbound09:35:341
28417993329,3cyclictest30323-21kworker/u16:1+events_unbound12:20:010
28417993328,4cyclictest31244-21kworker/u16:2+events_unbound10:25:370
28421993225,5cyclictest30323-21kworker/u16:1+events_unbound11:20:171
28417993228,3cyclictest31244-21kworker/u16:2+events_unbound09:40:310
28421993127,3cyclictest31244-21kworker/u16:2+events_unbound11:00:011
28421993127,3cyclictest30323-21kworker/u16:1+events_unbound12:25:251
28421993118,10cyclictest0-21swapper/110:50:351
28421993026,3cyclictest31244-21kworker/u16:2+events_unbound12:10:221
28421993015,11cyclictest16301-21kworker/u16:3+events_unbound08:54:291
28421992925,3cyclictest31244-21kworker/u16:2+events_unbound10:55:271
28421992917,9cyclictest31244-21kworker/u16:2+events_unbound09:59:521
28421992916,12cyclictest16301-21kworker/u16:3+events_unbound08:31:431
28421992914,12cyclictest31244-21kworker/u16:2+events_unbound12:36:201
28421992823,4cyclictest31244-21kworker/u16:2+events_unbound07:50:201
28417992826,1cyclictest16301-21kworker/u16:3+events_unbound08:35:370
28417992823,4cyclictest30323-21kworker/u16:1+events_unbound11:10:340
28417992822,4cyclictest31244-21kworker/u16:2+events_unbound08:50:250
28421992723,3cyclictest31244-21kworker/u16:2+events_unbound09:05:311
28421992723,3cyclictest31244-21kworker/u16:2+events_unbound07:55:361
28421992723,3cyclictest30323-21kworker/u16:1+events_unbound10:30:011
28421992622,3cyclictest31244-21kworker/u16:2+events_unbound09:10:311
28421992615,6cyclictest12573-21cat09:15:251
28417992622,3cyclictest30323-21kworker/u16:1+events_unbound11:15:310
2842199259,9cyclictest31244-21kworker/u16:2+events_unbound11:57:561
28421992521,3cyclictest31244-21kworker/u16:2+events_unbound07:45:231
28421992521,3cyclictest16301-21kworker/u16:3+events_unbound08:35:261
28421992520,3cyclictest31244-21kworker/u16:2+events_unbound08:25:281
28421992510,11cyclictest16301-21kworker/u16:3+events_unbound09:52:461
2841799259,15cyclictest0-21swapper/008:15:190
28421992420,3cyclictest31244-21kworker/u16:2+events_unbound11:05:271
28421992415,8cyclictest843-21ntpq09:30:321
2842199241,15cyclictest13744-21proc_pri10:10:331
28417992419,4cyclictest31244-21kworker/u16:2+events_unbound12:30:260
2842199234,12cyclictest31244-21kworker/u16:2+events_unbound11:25:131
2842199233,19cyclictest15146-21/usr/sbin/munin12:05:351
28421992319,3cyclictest16301-21kworker/u16:3+events_unbound08:00:351
28417992319,3cyclictest30323-21kworker/u16:1+events_unbound10:10:330
28417992318,4cyclictest16301-21kworker/u16:3+events_unbound09:55:280
28421992218,3cyclictest16301-21kworker/u16:3+events_unbound09:25:241
28421992218,3cyclictest16301-21kworker/u16:3+events_unbound08:45:241
28417992218,3cyclictest16301-21kworker/u16:3+events_unbound08:05:260
2842199218,10cyclictest30323-21kworker/u16:1+events_unbound12:24:581
28421992117,3cyclictest31244-21kworker/u16:2+events_unbound09:45:301
28421992117,3cyclictest30323-21kworker/u16:1+events_unbound10:40:191
28417992117,3cyclictest31244-21kworker/u16:2+events_unbound08:20:250
28421992012,6cyclictest16846-21/usr/sbin/munin10:45:221
2842199201,12cyclictest31753-21grep09:00:321
2842199197,11cyclictest0-21swapper/110:00:001
2842199192,8cyclictest0-21swapper/107:42:331
2842199187,8cyclictest31244-21kworker/u16:2+events_unbound11:47:581
2842199187,8cyclictest0-21swapper/111:15:021
2842199187,11cyclictest0-21swapper/110:39:021
28417991815,2cyclictest9542-21sendmail07:20:000
28417991815,2cyclictest27184-21cat12:20:010
28417991814,3cyclictest31244-21kworker/u16:2+events_unbound07:35:250
28417991814,3cyclictest31244-21kworker/u16:2+events_unbound07:30:330
28417991814,3cyclictest30323-21kworker/u16:1+events_unbound10:40:340
2842199176,11cyclictest0-21swapper/108:06:071
2842199173,9cyclictest26919-21aten2_r5power_v07:35:211
28417991712,4cyclictest31244-21kworker/u16:2+events_unbound09:45:250
28417991612,3cyclictest31244-21kworker/u16:2+events_unbound11:25:230
28417991611,4cyclictest31244-21kworker/u16:2+events_unbound10:45:220
2841799160,11cyclictest27641-21memory08:00:320
2841799158,4cyclictest141ktimers/010:55:250
2841799154,6cyclictest9940-21cpu08:45:240
2841799152,8cyclictest21615-21latency_hist12:15:010
28417991511,3cyclictest31244-21kworker/u16:2+events_unbound09:50:330
28417991511,3cyclictest30323-21kworker/u16:1+events_unbound11:30:250
28417991510,4cyclictest31244-21kworker/u16:2+events_unbound11:40:320
28417991510,4cyclictest31244-21kworker/u16:2+events_unbound08:25:340
195042150,1chrt0-21swapper/312:10:303
2841799144,9cyclictest8970-21munin-run10:10:000
2841799143,6cyclictest7585-21sed09:40:000
28417991411,2cyclictest3311-21cron10:05:000
28417991410,3cyclictest16301-21kworker/u16:3+events_unbound09:25:280
2841799139,3cyclictest31244-21kworker/u16:2+events_unbound07:40:470
2841799138,3cyclictest16301-21kworker/u16:3+events_unbound08:30:310
2841799133,5cyclictest0-21swapper/011:55:270
2841799132,6cyclictest29389-21cron11:25:000
2841799132,6cyclictest25469-21sed11:50:000
28417991310,2cyclictest505-21rs:main0
28417991310,2cyclictest23516-21irqrtprio10:50:270
2841799130,4cyclictest11711-21smartctl11:05:340
2841799129,2cyclictest20910-21cron07:30:010
2841799129,2cyclictest1818-21df_inode11:00:250
2841799129,2cyclictest15895-21irqrtprio10:15:270
2841799128,3cyclictest31244-21kworker/u16:2+events_unbound09:15:240
2841799126,4cyclictest16301-21kworker/u16:3+events_unbound09:30:250
2841799124,8cyclictest0-21swapper/010:20:190
2841799122,8cyclictest0-21swapper/012:35:260
2841799122,8cyclictest0-21swapper/008:12:020
2841799121,6cyclictest0-21swapper/009:07:340
2841799120,11cyclictest15225-21threads12:05:360
2841799114,5cyclictest0-21swapper/010:30:270
2841799114,5cyclictest0-21swapper/008:58:300
2841799113,8cyclictest0-21swapper/012:00:060
2841799112,8cyclictest0-21swapper/009:20:240
2841799110,2cyclictest690-21apt-get12:25:210
2841799110,10cyclictest8533-21/usr/sbin/munin11:35:220
2841799102,6cyclictest10425-21ntp_states07:45:340
4718290,1sleep30-21swapper/308:40:253
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
none08:51:277
9820270,5sleep30-21swapper/308:45:233
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
none08:51:276
9220260,4sleep30-21swapper/310:10:193
6341260,4sleep30-21swapper/312:30:203
6110260,4sleep30-21swapper/309:10:193
28141260,4sleep30-21swapper/310:55:243
26820260,4sleep30-21swapper/309:00:173
21791260,4sleep30-21swapper/312:15:203
19561260,4sleep30-21swapper/308:25:033
19001260,4sleep30-21swapper/309:50:193
15498260,4sleep30-21swapper/307:25:193
14868260,4sleep30-21swapper/310:15:203
1391260,4sleep30-21swapper/311:00:213
12021260,4sleep30-21swapper/311:35:283
10435260,4sleep30-21swapper/312:05:023
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
none08:51:275
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional