You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-19 - 01:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 18, 2024 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309212241165,24sleep20-21swapper/207:07:102
309442237163,60sleep00-21swapper/007:07:280
308112203166,24sleep10-21swapper/107:05:441
307932203166,24sleep30-21swapper/307:05:283
2400621690,5sleep03123899cyclictest10:30:200
2400621690,5sleep03123899cyclictest10:30:200
642821340,2sleep29-21rcu_preempt11:05:092
31249991080,106cyclictest0-21swapper/210:31:432
31249991080,106cyclictest0-21swapper/210:31:432
88492550,1sleep30-21swapper/307:30:153
267832480,4sleep121-21rcuc/110:35:261
3124999471,36cyclictest14859-21kworker/2:209:39:002
3124399451,34cyclictest0-21swapper/112:15:111
31238994010,14cyclictest14667-21munin-run11:25:010
3124399391,12cyclictest0-21swapper/109:25:471
3124399391,12cyclictest0-21swapper/109:25:471
3124399381,29cyclictest0-21swapper/112:31:251
3124999373,7cyclictest836-21runrttasks09:10:022
3124399371,29cyclictest0-21swapper/107:15:251
3124399371,29cyclictest0-21swapper/107:15:251
3124399371,12cyclictest0-21swapper/107:58:131
3124999365,3cyclictest0-21swapper/211:50:122
31249993613,3cyclictest29004-21/usr/sbin/munin09:30:262
31249993612,3cyclictest8916-21/usr/sbin/munin12:25:162
31249993612,3cyclictest6720-21/usr/sbin/munin12:20:132
3124399360,18cyclictest0-21swapper/111:32:051
3123899360,13cyclictest0-21swapper/010:19:020
3123899351,12cyclictest0-21swapper/011:11:580
3123899351,11cyclictest0-21swapper/008:44:460
31249993412,3cyclictest32488-21/usr/sbin/munin12:05:142
3124399341,15cyclictest0-21swapper/111:45:131
3123899341,19cyclictest28459-21taskset09:26:060
3123899341,19cyclictest28459-21taskset09:26:060
3123899341,16cyclictest0-21swapper/011:59:580
3123899341,14cyclictest0-21swapper/007:30:540
3123899340,2cyclictest9978-21kworker/0:107:10:110
31249993312,3cyclictest20601-21pluginstate10:25:002
31249993311,2cyclictest836-21runrttasks09:42:222
31249993311,2cyclictest836-21runrttasks07:39:342
3124399331,14cyclictest0-21swapper/112:07:271
3124399331,14cyclictest0-21swapper/109:32:511
3124399331,14cyclictest0-21swapper/109:22:291
3124399331,12cyclictest0-21swapper/110:47:331
3124399331,12cyclictest0-21swapper/110:16:191
3124399331,10cyclictest0-21swapper/111:43:291
3123899331,15cyclictest6372-21kworker/0:212:25:220
3123899331,14cyclictest3591-21kworker/0:111:03:280
3123899331,13cyclictest0-21swapper/011:15:020
3123899330,13cyclictest11756-21kworker/0:210:07:480
3124999324,3cyclictest10778-21wc10:00:202
3124999321,6cyclictest19943-21sa109:10:002
3124999321,5cyclictest16428-21/usr/sbin/munin10:15:122
3124999321,5cyclictest15276-21/usr/sbin/munin10:10:182
3124999321,5cyclictest0-21swapper/212:17:232
3124999321,4cyclictest28340-21/usr/sbin/munin08:15:152
31249993211,2cyclictest836-21runrttasks12:13:172
3124399321,13cyclictest0-21swapper/108:05:251
3124399321,12cyclictest0-21swapper/109:58:191
3124399320,12cyclictest0-21swapper/110:32:471
3124399320,12cyclictest0-21swapper/110:32:471
3123899321,13cyclictest0-21swapper/009:53:340
3123899320,14cyclictest16787-21kworker/0:011:30:100
3123899320,10cyclictest7100-21kworker/0:310:20:360
235352320,4sleep123537-21pmu-power09:15:211
3124999316,5cyclictest12741-21cat10:05:152
3124999311,6cyclictest8300-21/usr/sbin/munin11:10:262
3124999311,5cyclictest5779-21diskstats09:50:132
3124999311,4cyclictest0-21swapper/210:25:492
31249993113,3cyclictest7320-21cat07:25:182
31249993112,2cyclictest836-21runrttasks10:36:332
31249993111,2cyclictest836-21runrttasks08:47:102
31249993110,3cyclictest22631-21cat08:00:162
3124399319,3cyclictest0-21swapper/107:44:541
3124399318,3cyclictest0-21swapper/111:56:131
3124399311,15cyclictest0-21swapper/111:29:421
3123899311,12cyclictest0-21swapper/009:09:100
3123899311,11cyclictest1806-21kworker/0:112:15:460
3123899311,11cyclictest0-21swapper/011:44:540
3123899311,11cyclictest0-21swapper/010:46:540
3123899311,11cyclictest0-21swapper/008:24:540
3123899311,11cyclictest0-21swapper/008:24:540
3123899311,10cyclictest0-21swapper/008:13:420
3123899310,10cyclictest21146-21kworker/0:011:39:220
3123899310,10cyclictest21146-21kworker/0:011:39:220
3124999309,4cyclictest20855-21grep11:35:262
3124999309,4cyclictest20855-21grep11:35:262
3124999302,15cyclictest32385-21wc07:10:172
3124999301,4cyclictest10567-21cat07:35:012
31249993013,3cyclictest10276-21perf11:19:552
31249993011,3cyclictest24922-21uname08:05:182
31249993011,3cyclictest23967-21expr11:45:102
31249993011,3cyclictest22717-21seq11:40:202
31249993011,3cyclictest14889-21/usr/sbin/munin11:25:122
3124399308,3cyclictest0-21swapper/112:39:411
3124399301,14cyclictest0-21swapper/108:42:531
3124399301,12cyclictest0-21swapper/109:49:011
3123899301,16cyclictest10465-21kworker/0:007:45:200
3123899301,14cyclictest22549-21kworker/0:010:27:400
3123899301,13cyclictest0-21swapper/011:46:140
3123899301,12cyclictest31477-21kworker/0:210:55:160
3123899301,11cyclictest0-21swapper/010:10:060
3123899301,11cyclictest0-21swapper/009:45:500
3123899300,17cyclictest6372-21kworker/0:212:24:580
3123899300,15cyclictest0-21swapper/009:40:540
3124999299,4cyclictest31411-21latency08:20:162
3124999299,4cyclictest31411-21latency08:20:162
3124999295,3cyclictest12050irq/123-ahci[0012:00:012
3124999291,6cyclictest0-21swapper/209:22:282
3124999291,4cyclictest0-21swapper/210:54:472
31249992910,4cyclictest0-21swapper/210:45:172
31249992910,3cyclictest20874-21cat07:55:222
3124399291,19cyclictest0-21swapper/107:27:101
3124399291,14cyclictest0-21swapper/108:27:411
3124399291,14cyclictest0-21swapper/107:12:541
3124399291,13cyclictest0-21swapper/110:51:571
3124399291,13cyclictest0-21swapper/109:50:051
3124399291,12cyclictest1526-21crond09:05:011
3124399291,11cyclictest10611-21sed08:45:251
3124399291,11cyclictest0-21swapper/112:12:371
3124399290,25cyclictest0-21swapper/112:20:211
3124399290,14cyclictest0-21swapper/110:40:111
3123899291,14cyclictest0-21swapper/007:50:200
3123899291,12cyclictest19255-21kworker/0:308:00:000
3123899291,12cyclictest0-21swapper/012:39:340
3123899291,10cyclictest0-21swapper/011:05:420
3123899290,8cyclictest616-21kworker/0:207:15:560
3123899290,8cyclictest616-21kworker/0:207:15:550
3123899290,16cyclictest0-21swapper/009:34:300
3123899290,13cyclictest6633-21kworker/0:108:39:000
3123899290,13cyclictest17509-21kworker/0:009:22:280
3123899290,13cyclictest0-21swapper/008:15:180
3123899290,12cyclictest15287-21kworker/0:208:57:360
3124999281,6cyclictest9138-21cat08:45:022
3124999281,6cyclictest0-21swapper/212:02:292
3124999281,6cyclictest0-21swapper/210:42:272
3124999281,5cyclictest32742-21/usr/sbin/munin08:25:282
3124999280,5cyclictest0-21swapper/209:25:172
3124999280,5cyclictest0-21swapper/209:25:172
3124399281,17cyclictest0-21swapper/110:07:571
3124399281,15cyclictest0-21swapper/110:13:411
3124399281,15cyclictest0-21swapper/107:51:501
3124399281,15cyclictest0-21swapper/107:32:541
3124399281,14cyclictest0-21swapper/111:50:211
3124399281,10cyclictest0-21swapper/112:27:091
3124399281,10cyclictest0-21swapper/108:32:571
3123899287,2cyclictest31952-21kworker/0:212:10:180
3123899281,17cyclictest0-21swapper/010:35:580
3123899281,15cyclictest0-21swapper/011:54:060
3123899281,14cyclictest0-21swapper/009:15:500
3123899281,13cyclictest0-21swapper/010:54:140
31238992810,3cyclictest0-21swapper/012:32:140
3123899280,14cyclictest10465-21kworker/0:007:35:220
3124999272,5cyclictest581-21systemd-journal07:45:102
3124999271,7cyclictest792-21avahi-daemon11:22:002
3124999271,7cyclictest11362-21/usr/sbin/munin08:50:092
3124999271,5cyclictest3870-21/usr/sbin/munin11:00:182
3124999271,4cyclictest0-21swapper/209:48:432
31249992711,3cyclictest17733-21expr11:30:142
31249992711,3cyclictest14291-21timerandwakeup07:40:242
3124999271,12cyclictest0-21swapper/207:20:132
31249992710,3cyclictest11498-21cut12:30:132
3124399276,3cyclictest0-21swapper/111:35:421
3124399276,3cyclictest0-21swapper/111:35:411
3124399271,14cyclictest0-21swapper/108:01:571
3124399271,12cyclictest0-21swapper/111:12:381
3123899271,14cyclictest0-21swapper/009:59:580
3123899271,10cyclictest26965-21kworker/0:210:40:340
3123899270,14cyclictest28426-21kworker/0:209:37:540
3123899270,11cyclictest0-21swapper/012:07:420
3123899270,11cyclictest0-21swapper/010:03:100
3123899270,10cyclictest0-21swapper/009:05:020
3125499263,17cyclictest41-21ksoftirqd/308:45:063
3124999261,6cyclictest14626-21ntp_states08:55:192
3124999260,2cyclictest29916-1kworker/u9:209:59:112
3124399261,16cyclictest0-21swapper/108:16:211
3124399261,12cyclictest0-21swapper/109:11:571
3124399260,11cyclictest0-21swapper/111:06:241
3125499255,7cyclictest0-21swapper/309:50:583
3125499251,12cyclictest0-21swapper/308:04:423
3124999259,4cyclictest18885-21unixbench_singl07:50:242
3124999251,6cyclictest3498-21taskset10:58:332
3124399251,16cyclictest0-21swapper/107:39:151
3124399251,13cyclictest0-21swapper/108:13:171
3124399251,12cyclictest0-21swapper/107:47:021
3124399250,6cyclictest0-21swapper/111:17:101
3123899259,3cyclictest0-21swapper/009:12:380
3123899251,3cyclictest0-21swapper/011:26:140
3123899251,13cyclictest0-21swapper/012:03:020
3123899250,2cyclictest32321-21kworker/0:208:34:180
3123899250,2cyclictest32321-21kworker/0:208:26:360
3124999247,2cyclictest38952sleep207:19:422
3124999247,2cyclictest38952sleep207:19:422
3123899241,13cyclictest22002-21expr08:00:120
3123899241,12cyclictest0-21swapper/008:46:220
3125499231,7cyclictest0-21swapper/312:08:003
31254992311,5cyclictest16340-21taskset09:00:153
3124999235,3cyclictest0-21swapper/209:03:412
3124999231,12cyclictest0-21swapper/209:18:492
3124999231,10cyclictest0-21swapper/212:38:352
3124399238,3cyclictest0-21swapper/110:29:011
3124399236,3cyclictest0-21swapper/112:01:171
3124399236,3cyclictest0-21swapper/109:35:411
3124399231,15cyclictest0-21swapper/110:20:151
3124399231,14cyclictest0-21swapper/110:02:291
3124399231,13cyclictest0-21swapper/109:07:411
3124399231,13cyclictest0-21swapper/107:21:161
3124399231,12cyclictest0-21swapper/108:37:251
3124399231,10cyclictest0-21swapper/108:53:411
3123899237,4cyclictest8872-21kworker/0:308:50:360
3123899231,9cyclictest0-21swapper/007:23:180
3123899231,11cyclictest10753-21kworker/0:107:42:160
3123899230,3cyclictest8035-21kworker/0:307:30:000
3125499224,6cyclictest0-21swapper/311:01:113
3125499224,10cyclictest15174-21missed_timers10:10:173
3125499221,6cyclictest0-21swapper/311:40:063
3125499221,5cyclictest0-21swapper/311:46:453
3125499221,5cyclictest0-21swapper/311:07:513
3125499221,5cyclictest0-21swapper/310:55:283
3125499221,4cyclictest0-21swapper/312:38:573
3125499221,4cyclictest0-21swapper/312:30:113
3125499221,14cyclictest0-21swapper/311:32:353
3125499221,12cyclictest0-21swapper/310:21:163
3125499221,11cyclictest0-21swapper/310:38:283
3125499220,8cyclictest0-21swapper/311:14:243
3124399229,3cyclictest0-21swapper/109:40:291
3124399221,12cyclictest0-21swapper/110:59:261
3124399221,12cyclictest0-21swapper/108:20:291
3124399221,12cyclictest0-21swapper/108:20:291
3124399220,5cyclictest0-21swapper/111:22:161
3123899228,3cyclictest0-21swapper/008:06:220
57002211,17sleep15716-21grep11:03:141
3125499219,3cyclictest29274-21kworker/u8:310:50:133
3125499216,3cyclictest17735-21sshd07:50:143
3125499212,10cyclictest32742-21/usr/sbin/munin08:25:273
3125499212,10cyclictest14634-21snmpd10:41:003
3125499211,9cyclictest0-21swapper/311:25:183
3125499211,5cyclictest0-21swapper/311:37:583
3125499211,5cyclictest0-21swapper/311:37:583
3125499211,5cyclictest0-21swapper/311:23:453
3125499211,5cyclictest0-21swapper/309:48:593
3125499211,4cyclictest19945-40dnf09:10:033
3125499211,4cyclictest0-21swapper/312:29:503
3125499211,4cyclictest0-21swapper/311:19:263
3125499211,4cyclictest0-21swapper/310:16:053
3125499211,10cyclictest23974-21users09:15:263
3124999211,11cyclictest0-21swapper/208:39:272
3124399211,10cyclictest0-21swapper/108:58:371
3125499209,3cyclictest24605-21/usr/sbin/munin09:20:113
3125499208,4cyclictest0-21swapper/312:10:173
3125499208,4cyclictest0-21swapper/310:03:123
3125499205,12cyclictest19937-21diskmemload09:55:133
3125499201,4cyclictest0-21swapper/312:02:393
3125499201,4cyclictest0-21swapper/309:35:173
3125499201,12cyclictest0-21swapper/309:42:473
3125499201,11cyclictest6720-21/usr/sbin/munin12:20:193
3125499201,10cyclictest825-21lldpd08:35:283
3125499201,10cyclictest0-21swapper/310:26:273
3125499201,10cyclictest0-21swapper/309:25:043
3125499201,10cyclictest0-21swapper/309:25:043
3125499191,9cyclictest19937-21diskmemload12:18:063
31254991914,3cyclictest23691-21cut10:30:163
31254991914,3cyclictest23691-21cut10:30:163
3125499191,11cyclictest0-21swapper/309:30:243
3125499191,11cyclictest0-21swapper/308:10:163
3125499191,10cyclictest5216-21ntpq07:20:193
3125499191,10cyclictest27467-21awk11:50:263
3125499191,10cyclictest1969-21/usr/sbin/munin07:15:173
3125499191,10cyclictest1969-21/usr/sbin/munin07:15:173
3125499191,10cyclictest0-21swapper/311:55:543
3125499191,10cyclictest0-21swapper/310:45:203
3125499191,10cyclictest0-21swapper/310:06:353
67022181,3sleep30-21swapper/307:25:143
3125499188,3cyclictest0-21swapper/307:45:223
3125499185,4cyclictest31999-21cat08:20:243
3125499185,4cyclictest31999-21cat08:20:243
3125499185,2cyclictest32191-21if_enp2s007:10:143
3125499181,9cyclictest18813-21kernelversion09:05:183
3125499181,8cyclictest13294-21df_inode07:40:133
3125499181,15cyclictest0-21swapper/308:30:223
3125499181,10cyclictest19539-21/usr/sbin/munin07:55:133
3124999184,3cyclictest0-21swapper/208:13:542
3125499175,3cyclictest12293-21sed08:50:173
3125499172,14cyclictest0-21swapper/307:39:123
31254991712,3cyclictest28708-21grep08:15:133
31254991712,3cyclictest13893-21sh08:55:113
3125499171,13cyclictest23941-21/usr/sbin/munin08:05:263
31249991710,4cyclictest3725-21pmu-power08:30:202
31254991610,3cyclictest7307-21sh08:40:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional