You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 23:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot6.osadl.org (updated Sat Apr 27, 2024 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2585421590,5sleep12918299cyclictest10:27:491
1791911090,4ptp4l401ktimersoftd/309:25:043
74872660,3sleep2311rcuc/211:48:072
261602630,2sleep30-21swapper/313:25:523
179191620,1ptp4l401ktimersoftd/312:22:533
179191620,1ptp4l401ktimersoftd/312:11:463
27332590,4sleep22918699cyclictest13:37:192
179191590,1ptp4l401ktimersoftd/312:51:223
179191590,1ptp4l401ktimersoftd/312:34:323
114692580,3sleep30-21swapper/313:08:143
101922580,2sleep10-21swapper/114:23:151
72322560,3sleep2321ktimersoftd/209:48:032
25182550,2sleep00-21swapper/012:21:090
58602540,5sleep35861-21irqrtprio14:18:113
216222530,3sleep30-21swapper/314:37:553
280792520,2sleep20-21swapper/214:06:262
179191520,1ptp4l401ktimersoftd/313:33:343
179191490,1ptp4l401ktimersoftd/313:05:073
179191480,2ptp4l16281-21kworker/3:214:02:453
179191460,1ptp4l401ktimersoftd/311:55:143
179191410,1ptp4l401ktimersoftd/313:02:523
2901523921,8sleep00-21swapper/009:27:020
179191390,1ptp4l401ktimersoftd/312:22:203
29186993813,21cyclictest12218-21cat13:48:072
184891380,1getstats0-21swapper/310:06:013
179191380,1ptp4l401ktimersoftd/313:44:353
179191380,1ptp4l401ktimersoftd/313:22:473
179191380,1ptp4l401ktimersoftd/312:29:543
179191380,1ptp4l401ktimersoftd/311:37:423
179191380,1ptp4l401ktimersoftd/310:48:083
179191380,14ptp4l401ktimersoftd/311:18:033
179191380,0ptp4l401ktimersoftd/312:17:053
179191380,0ptp4l401ktimersoftd/312:04:003
290692373,21sleep229071-21sshd09:27:452
179191370,1ptp4l401ktimersoftd/314:49:213
179191370,1ptp4l401ktimersoftd/314:26:493
179191370,1ptp4l401ktimersoftd/313:13:073
179191370,1ptp4l401ktimersoftd/312:57:293
179191370,0ptp4l401ktimersoftd/311:43:593
179191370,0ptp4l401ktimersoftd/309:49:383
179191370,0ptp4l401ktimersoftd/309:37:493
29186993611,22cyclictest5100-21bc14:18:022
29186993611,22cyclictest15432-21expr11:58:012
179191360,1ptp4l401ktimersoftd/314:57:373
179191360,1ptp4l401ktimersoftd/314:29:393
179191360,1ptp4l401ktimersoftd/314:15:363
179191360,1ptp4l401ktimersoftd/314:10:313
179191360,0ptp4l401ktimersoftd/312:01:303
29186993513,19cyclictest2664-21/usr/sbin/munin11:43:232
29186993513,19cyclictest25222-21latency_hist12:47:522
29186993510,22cyclictest22401-21gltestperf14:38:092
29186993510,22cyclictest16493-21unixbench_singl13:13:252
179191350,1ptp4l401ktimersoftd/314:07:383
179191350,1ptp4l401ktimersoftd/313:38:183
179191350,0ptp4l401ktimersoftd/311:37:583
179191350,0ptp4l401ktimersoftd/311:37:583
2918699349,21cyclictest16820-21wc10:08:002
2918699347,3cyclictest11357-21cat11:03:142
2918699343,8cyclictest17900-21cut12:38:142
2918699343,26cyclictest26720-21apt-get11:33:022
2918699342,5cyclictest0-21swapper/212:13:022
29186993412,19cyclictest12180-21sort12:32:532
29186993410,21cyclictest4890-21grep10:48:212
179191340,1ptp4l401ktimersoftd/314:45:123
179191340,1ptp4l401ktimersoftd/313:31:093
179191340,1ptp4l401ktimersoftd/313:31:093
2918699337,3cyclictest4922-21cut14:17:542
2918699332,5cyclictest0-21swapper/212:58:052
2918699332,28cyclictest2544-21grep14:53:142
2918699331,4cyclictest29821-21/usr/sbin/munin09:27:592
29186993313,17cyclictest26739-21if_ens314:43:152
29186993311,18cyclictest8061-21expr13:43:062
29186993310,19cyclictest21468-21sort10:18:012
2918699330,29cyclictest10537-21rm13:08:052
179191330,1ptp4l401ktimersoftd/313:53:493
179191330,1ptp4l401ktimersoftd/310:21:583
179191330,1ptp4l401ktimersoftd/310:08:173
179191330,0ptp4l401ktimersoftd/309:36:383
2918699329,3cyclictest21254-21sh13:58:132
2918699329,3cyclictest20262-21latency_hist13:57:542
2918699327,4cyclictest15342-21cat11:13:012
2918699322,7cyclictest5584-21apt-get10:53:032
2918699321,4cyclictest0-21swapper/209:52:572
2918699321,4cyclictest0-21swapper/209:52:572
29186993210,19cyclictest10199-21cut13:07:522
2918699320,5cyclictest0-21swapper/211:38:042
2918699320,5cyclictest0-21swapper/211:38:042
29176993222,5cyclictest9-21ksoftirqd/012:38:020
179191320,1ptp4l401ktimersoftd/310:37:303
179191320,1ptp4l401ktimersoftd/310:25:273
179191320,0ptp4l401ktimersoftd/311:52:363
99292310,3sleep129168-21cyclictest12:28:371
2918699319,19cyclictest1701-21sed10:43:122
2918699318,3cyclictest31246-21kernelversion14:48:152
2918699312,9cyclictest19020-21apt-get10:13:042
2918699312,4cyclictest14284-21apt-get10:02:592
2918699311,7cyclictest7187-21ssh13:42:212
2918699311,3cyclictest0-21swapper/211:57:512
2918699311,3cyclictest0-21swapper/211:23:022
2918699311,27cyclictest26656-21if_enp1s010:28:092
29186993112,16cyclictest31882-21ptp4l09:32:482
29186993111,17cyclictest28133-21irqrtprio13:28:152
29186993111,17cyclictest28133-21irqrtprio13:28:152
179191310,1ptp4l401ktimersoftd/311:05:173
179191310,0ptp4l401ktimersoftd/309:29:333
4932300,1sleep00-21swapper/011:38:450
4932300,1sleep00-21swapper/011:38:450
29195993025,3cyclictest41-21ksoftirqd/312:43:103
29195993023,5cyclictest41-21ksoftirqd/310:13:023
2918699309,3cyclictest12092-21cstates09:58:032
2918699308,3cyclictest8415-21grep10:58:072
2918699308,3cyclictest6283-21sh12:24:492
2918699308,3cyclictest23856-21mailstats11:28:162
2918699306,3cyclictest23953-21cat10:23:012
2918699303,6cyclictest4806-21apt-get09:42:592
2918699303,23cyclictest0-21swapper/212:06:502
29186993011,15cyclictest25389-21/usr/sbin/munin12:48:052
29186993010,17cyclictest18296-21if_enp1s0.114:33:122
29186993010,17cyclictest1779-21runrttasks13:20:392
2918699300,2cyclictest0-21swapper/212:08:042
29176993025,3cyclictest9-21ksoftirqd/011:57:500
281012306,10sleep10-21swapper/109:23:121
29195992923,3cyclictest41-21ksoftirqd/314:42:553
2918699298,3cyclictest18393-21irqrtprio11:18:082
2918699296,3cyclictest3690-21ptp4l-jitter09:38:122
2918699291,9cyclictest29827-21/usr/sbin/munin12:53:192
2918699290,10cyclictest0-21swapper/212:18:062
29182992923,3cyclictest25-21ksoftirqd/114:46:401
179191290,1ptp4l401ktimersoftd/311:30:223
29195992823,3cyclictest41-21ksoftirqd/310:28:023
2918699287,3cyclictest31558-21irqrtprio10:38:102
2918699285,4cyclictest29883-21irqrtprio14:08:152
2918699281,4cyclictest11791-21idleruntime-cro12:32:512
2918699280,8cyclictest0-21swapper/214:28:072
2918699280,4cyclictest25668-21ssh13:24:472
29182992822,3cyclictest25-21ksoftirqd/114:08:251
29182992822,3cyclictest25-21ksoftirqd/112:24:151
29182992821,4cyclictest25-21ksoftirqd/113:18:051
29182992814,4cyclictest25-21ksoftirqd/114:39:581
29176992822,3cyclictest9-21ksoftirqd/011:17:490
29176992821,4cyclictest9-21ksoftirqd/010:26:310
29176992821,4cyclictest9-21ksoftirqd/010:02:550
179191280,1ptp4l401ktimersoftd/310:39:153
29195992722,3cyclictest41-21ksoftirqd/312:38:033
2918699277,3cyclictest321ktimersoftd/210:33:142
2918699271,5cyclictest9873-21hddtemp_smartct14:23:142
29182992722,3cyclictest25-21ksoftirqd/112:38:061
29182992722,3cyclictest25-21ksoftirqd/112:08:171
29182992720,4cyclictest25-21ksoftirqd/110:50:411
29182992720,4cyclictest25-21ksoftirqd/110:47:491
29182992720,4cyclictest25-21ksoftirqd/110:33:471
29182992720,4cyclictest25-21ksoftirqd/109:58:461
29176992721,3cyclictest9-21ksoftirqd/012:36:310
179191270,1ptp4l401ktimersoftd/309:46:223
177282271,22sleep117817-21awk13:53:251
29195992620,3cyclictest41-21ksoftirqd/309:57:563
2918699261,4cyclictest13230-21fschecks_count11:08:032
29182992621,3cyclictest25-21ksoftirqd/109:56:251
29182992621,3cyclictest25-21ksoftirqd/109:56:251
29182992620,3cyclictest25-21ksoftirqd/111:50:351
29182992620,3cyclictest25-21ksoftirqd/111:33:161
29182992619,4cyclictest25-21ksoftirqd/113:40:151
29182992619,4cyclictest25-21ksoftirqd/110:29:451
2918299261,3cyclictest121rcu_preempt09:40:081
29176992620,3cyclictest9-21ksoftirqd/014:41:170
29176992620,3cyclictest9-21ksoftirqd/013:50:550
29176992620,3cyclictest9-21ksoftirqd/009:47:570
29176992619,4cyclictest9-21ksoftirqd/011:48:200
29176992619,4cyclictest9-21ksoftirqd/011:00:400
29176992613,3cyclictest9-21ksoftirqd/013:53:190
29176992612,3cyclictest9-21ksoftirqd/014:57:530
179191260,9ptp4l401ktimersoftd/310:46:563
168882260,4sleep116891-21unixbench_multi11:13:211
29182992520,3cyclictest25-21ksoftirqd/109:32:511
29182992518,4cyclictest25-21ksoftirqd/113:35:461
29182992518,4cyclictest25-21ksoftirqd/110:40:331
29182992518,4cyclictest25-21ksoftirqd/109:49:311
29182992518,4cyclictest25-21ksoftirqd/109:44:211
2918299250,3cyclictest121rcu_preempt13:59:361
29176992520,3cyclictest9-21ksoftirqd/014:03:100
29176992519,3cyclictest9-21ksoftirqd/010:16:340
29176992518,4cyclictest9-21ksoftirqd/009:59:250
29176992518,4cyclictest9-21ksoftirqd/009:41:470
2917699250,8cyclictest29597-21cstates09:27:560
179191250,1ptp4l401ktimersoftd/313:48:283
179191250,0ptp4l401ktimersoftd/310:59:443
33952240,2sleep10-21swapper/111:43:091
29195992417,4cyclictest41-21ksoftirqd/311:17:173
29182992419,3cyclictest25-21ksoftirqd/112:48:231
29182992418,4cyclictest25-21ksoftirqd/111:00:331
29182992418,3cyclictest25-21ksoftirqd/113:49:431
29182992418,3cyclictest25-21ksoftirqd/111:09:321
29182992417,4cyclictest25-21ksoftirqd/113:23:051
29182992417,4cyclictest25-21ksoftirqd/111:05:461
29176992419,3cyclictest9-21ksoftirqd/009:56:460
29176992419,3cyclictest9-21ksoftirqd/009:56:460
29176992418,3cyclictest9-21ksoftirqd/012:56:530
29176992417,4cyclictest9-21ksoftirqd/012:02:220
29176992414,5cyclictest14369-21latency_hist13:12:520
179191240,2ptp4l401ktimersoftd/310:53:143
179191240,1ptp4l401ktimersoftd/311:23:213
179191240,1ptp4l401ktimersoftd/311:07:593
179191240,1ptp4l401ktimersoftd/309:52:583
179191240,1ptp4l401ktimersoftd/309:52:583
98512230,18sleep10-21swapper/113:43:581
2918299234,15cyclictest18941-21/usr/sbin/munin10:13:111
29182992318,3cyclictest25-21ksoftirqd/114:48:061
29182992317,3cyclictest25-21ksoftirqd/112:33:471
29182992317,3cyclictest25-21ksoftirqd/111:30:031
29182992317,3cyclictest25-21ksoftirqd/110:57:351
29182992317,3cyclictest20131-21tr11:23:031
29182992317,3cyclictest17797-21grep14:33:061
29182992316,4cyclictest25-21ksoftirqd/111:42:481
29182992316,4cyclictest25-21ksoftirqd/111:42:481
29182992316,4cyclictest14677-21grep10:03:031
29176992318,3cyclictest9-21ksoftirqd/013:02:100
29176992318,3cyclictest9-21ksoftirqd/011:11:130
29176992317,4cyclictest31509-21turbostat12:17:510
29176992317,3cyclictest9-21ksoftirqd/013:28:420
29176992317,3cyclictest9-21ksoftirqd/013:28:420
29176992317,3cyclictest9-21ksoftirqd/012:43:110
29176992317,3cyclictest9-21ksoftirqd/010:49:310
29176992317,3cyclictest23004-21tail10:18:200
29176992316,4cyclictest9-21ksoftirqd/014:31:130
29176992316,4cyclictest9-21ksoftirqd/011:26:410
29176992316,4cyclictest9-21ksoftirqd/011:04:490
29176992316,4cyclictest9-21ksoftirqd/010:33:540
29182992215,4cyclictest21507-21grep12:43:071
29176992217,3cyclictest9-21ksoftirqd/011:46:130
29176992216,3cyclictest9-21ksoftirqd/013:14:460
29176992216,3cyclictest9-21ksoftirqd/010:39:070
29176992215,4cyclictest9-21ksoftirqd/010:45:340
29176992215,4cyclictest32388-21/usr/sbin/munin09:33:110
29176992215,4cyclictest25891-21grep12:48:050
2918299214,12cyclictest21406-21apt-get10:18:001
29182992116,3cyclictest25-21ksoftirqd/114:53:141
29182992116,3cyclictest25-21ksoftirqd/114:32:231
29182992116,3cyclictest25-21ksoftirqd/113:10:161
29182992115,3cyclictest16308-21sed11:58:101
29182992114,4cyclictest4931-21grep14:17:541
29182992114,4cyclictest25-21ksoftirqd/114:20:541
2918299210,3cyclictest121rcu_preempt13:02:341
2917699214,3cyclictest121rcu_preempt13:21:510
29176992115,3cyclictest9-21ksoftirqd/014:49:370
29176992115,3cyclictest31522-21/usr/sbin/munin13:33:080
29176992114,4cyclictest9-21ksoftirqd/010:53:530
29176992113,5cyclictest9-21ksoftirqd/013:03:090
29182992014,3cyclictest31191-21ptp4l12:53:171
29182992013,4cyclictest9927-21sh13:07:241
29182992013,4cyclictest30481-21sh12:15:311
29182992013,4cyclictest25-21ksoftirqd/113:29:271
29182992013,4cyclictest25-21ksoftirqd/113:29:271
29182992013,4cyclictest25-21ksoftirqd/112:20:421
29182992013,4cyclictest22088-21diskmemload13:16:471
29182992013,4cyclictest22088-21diskmemload12:05:401
29182992013,4cyclictest18852-21grep10:12:481
2918299201,16cyclictest18997-21snmp_o5powerctl11:18:151
29176992014,3cyclictest9-21ksoftirqd/009:42:580
29176992014,3cyclictest8738-21kernelversion13:43:130
29176992014,3cyclictest23442-21sh12:07:570
29176992014,3cyclictest23346-21cpuspeed_turbos13:23:050
29176992014,3cyclictest21176-21grep12:03:260
29176992014,3cyclictest10525-21sh14:23:190
29176992013,4cyclictest5860-21sh12:23:470
29176992013,4cyclictest5548-21grep14:18:080
29176992013,4cyclictest4007-21sh14:16:190
29176992013,4cyclictest27595-21cut11:33:090
29176992013,4cyclictest22394-21/usr/sbin/munin11:28:020
29176992013,4cyclictest22088-21diskmemload14:36:330
29176992013,4cyclictest18880-21ptp4l11:18:130
29182991913,3cyclictest30742-21tr09:28:071
29182991913,3cyclictest11427-21fschecks_count11:53:091
29182991912,4cyclictest24983-21cpuspeed_turbos14:03:071
2917699198,8cyclictest3924-21fschecks_count13:38:080
2917699194,9cyclictest20727-21apt-get13:58:060
2917699193,11cyclictest16650-21apt-get10:07:590
2917699193,10cyclictest29119-21apt-get14:08:060
29176991914,3cyclictest9-21ksoftirqd/014:43:080
29176991913,3cyclictest8301-21entropy12:28:070
29176991913,3cyclictest26289-21expr10:28:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional