You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-09 - 09:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot4.osadl.org (updated Sun Feb 09, 2025 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41527642322279,28sleep10-21swapper/119:07:021
41528162261189,27sleep00-21swapper/019:07:350
123699590,4rtkit-daemon1235-21rtkit-daemon23:10:541
123699590,4rtkit-daemon1235-21rtkit-daemon22:42:110
123699570,4rtkit-daemon1235-21rtkit-daemon22:25:361
123699550,5rtkit-daemon1235-21rtkit-daemon00:08:201
123699540,4rtkit-daemon1235-21rtkit-daemon23:36:451
415323299523,10cyclictest115307-21apt-get23:40:151
123699520,4rtkit-daemon1235-21rtkit-daemon21:37:050
123699520,17rtkit-daemon88343-21timerandwakeup23:00:250
1018842510,1chrt101221-21/usr/sbin/munin23:20:211
123699490,4rtkit-daemon1235-21rtkit-daemon19:47:031
123699470,4rtkit-daemon1235-21rtkit-daemon19:11:141
123699470,4rtkit-daemon1235-21rtkit-daemon00:10:481
123699470,3rtkit-daemon1235-21rtkit-daemon21:10:040
4153232994221,19cyclictest33578-21ssh21:43:511
123699420,3rtkit-daemon0-21swapper/021:17:460
415323299402,13cyclictest44739-21apt-get22:00:121
123699390,5rtkit-daemon4187954-21cstates20:35:161
415323299371,18cyclictest4171883-21wc19:55:021
123699370,4rtkit-daemon0-21swapper/021:52:011
4153232993616,4cyclictest65687-21sh22:30:111
123699360,4rtkit-daemon1235-21rtkit-daemon21:57:110
123699360,3rtkit-daemon0-21swapper/022:45:150
123699360,3rtkit-daemon0-21swapper/021:51:190
123699360,3rtkit-daemon0-21swapper/021:44:250
123699360,3rtkit-daemon0-21swapper/020:43:371
415323299353,9cyclictest150191-21apt-get00:30:081
123699350,2rtkit-daemon0-21swapper/000:31:130
4153232993432,1cyclictest0-21swapper/100:00:181
4153232993414,5cyclictest141rcu_preempt21:55:151
415323299341,13cyclictest107666-21/usr/sbin/munin23:30:221
123699340,4rtkit-daemon86022-21ssh22:58:140
123699340,4rtkit-daemon1235-21rtkit-daemon21:22:290
123699340,4rtkit-daemon1235-21rtkit-daemon00:29:061
123699340,3rtkit-daemon0-21swapper/019:15:411
123699340,3rtkit-daemon0-21swapper/000:38:430
415323299333,8cyclictest16236-21apt-get21:20:131
4153232993330,2cyclictest67350irq/132-enp2s019:56:431
415322599339,7cyclictest13-21ksoftirqd/022:20:000
123699330,5rtkit-daemon13-21ksoftirqd/019:21:031
123699330,4rtkit-daemon90067-21ssh23:03:321
123699330,4rtkit-daemon1235-21rtkit-daemon19:30:300
123699330,4rtkit-daemon1235-21rtkit-daemon00:05:400
123699330,3rtkit-daemon0-21swapper/123:55:240
123699330,3rtkit-daemon0-21swapper/021:08:091
123699330,3rtkit-daemon0-21swapper/020:53:590
123699330,3rtkit-daemon0-21swapper/020:07:501
123699320,4rtkit-daemon1235-21rtkit-daemon21:46:300
123699320,4rtkit-daemon1235-21rtkit-daemon20:33:300
123699320,3rtkit-daemon0-21swapper/119:50:260
123699320,3rtkit-daemon0-21swapper/022:09:441
123699320,3rtkit-daemon0-21swapper/021:25:381
123699320,1rtkit-daemon1235-21rtkit-daemon22:35:241
415322599318,5cyclictest13-21ksoftirqd/023:50:080
123699310,5rtkit-daemon902-21snmpd23:48:181
123699310,4rtkit-daemon1235-21rtkit-daemon22:15:481
123699310,4rtkit-daemon1235-21rtkit-daemon20:07:020
123699310,3rtkit-daemon0-21swapper/122:54:210
123699310,3rtkit-daemon0-21swapper/023:05:161
415323299303,8cyclictest97490-21apt-get23:15:131
415323299302,9cyclictest8134-21apt-get21:10:151
415322599309,4cyclictest13-21ksoftirqd/023:41:590
4153225993027,2cyclictest67350irq/132-enp2s022:32:020
4153225993026,2cyclictest104413-21rm23:24:120
123699300,4rtkit-daemon1235-21rtkit-daemon23:09:490
123699300,4rtkit-daemon1235-21rtkit-daemon21:01:240
123699300,3rtkit-daemon0-21swapper/021:36:011
415323299297,6cyclictest84588-21unixbench_multi22:55:261
415322599297,5cyclictest13-21ksoftirqd/022:10:130
4153225992926,2cyclictest2676428-1kworker/0:2H00:15:300
4153225992914,9cyclictest4156173-21apt-get19:15:140
4153225992914,8cyclictest111474-21apt-get23:35:130
123699290,4rtkit-daemon1235-21rtkit-daemon20:24:120
123699290,3rtkit-daemon0-21swapper/123:30:060
123699290,3rtkit-daemon0-21swapper/019:30:591
415323299288,6cyclictest4180338-21gltestperf20:15:161
4153225992811,2cyclictest141rcu_preempt21:31:160
123699280,4rtkit-daemon1235-21rtkit-daemon21:33:251
123699280,3rtkit-daemon1235-21rtkit-daemon20:36:430
123699280,3rtkit-daemon0-21swapper/123:26:310
123699280,3rtkit-daemon0-21swapper/119:47:590
123699280,3rtkit-daemon0-21swapper/119:37:080
123699280,3rtkit-daemon0-21swapper/100:00:480
415322599277,18cyclictest13-21ksoftirqd/000:20:280
4153225992719,6cyclictest13-21ksoftirqd/022:25:000
1541572270,5sleep126-21ksoftirqd/100:35:151
123699270,3rtkit-daemon0-21swapper/100:11:020
123699270,1rtkit-daemon1235-21rtkit-daemon23:55:171
123699270,1rtkit-daemon1235-21rtkit-daemon19:59:470
415323299260,6cyclictest0-21swapper/120:49:071
415322599265,6cyclictest13-21ksoftirqd/020:25:110
4153232992518,4cyclictest554-21/usr/sbin/munin20:50:271
415323299250,6cyclictest0-21swapper/123:29:151
415322599256,8cyclictest13-21ksoftirqd/023:15:060
415322599252,15cyclictest4179979-21apt-get20:15:110
415322599251,5cyclictest7844-21diskmemload00:25:210
4153225992510,2cyclictest141rcu_preempt22:05:010
123699250,4rtkit-daemon1235-21rtkit-daemon22:38:590
123699250,4rtkit-daemon1235-21rtkit-daemon20:11:231
123699250,20rtkit-daemon1235-21rtkit-daemon22:49:241
123699250,1rtkit-daemon1235-21rtkit-daemon22:08:390
415323299242,15cyclictest79542-21apt-get22:50:111
415323299240,8cyclictest0-21swapper/119:30:001
415323299240,7cyclictest0-21swapper/122:12:031
415322599249,5cyclictest0-21swapper/023:14:430
415322599247,2cyclictest13-21ksoftirqd/022:28:130
415322599242,15cyclictest4174138-21apt-get20:00:150
123699240,1rtkit-daemon1235-21rtkit-daemon22:41:431
4153232992314,7cyclictest5436-21idleruntime-cro21:05:001
4153232992314,7cyclictest4166923-21systemctl19:40:261
415323299230,6cyclictest0-21swapper/121:15:501
415322599232,14cyclictest1709-21apt-get20:55:160
4153225992312,6cyclictest6071-21apt-get21:05:220
4153225992310,6cyclictest4158149-21apt-get19:20:140
123699230,1rtkit-daemon1235-21rtkit-daemon22:24:021
123699230,1rtkit-daemon1235-21rtkit-daemon20:28:311
415323299220,5cyclictest0-21swapper/120:00:361
415322599228,5cyclictest0-21swapper/021:26:120
4153225992215,5cyclictest13-21ksoftirqd/019:45:000
123699220,1rtkit-daemon1235-21rtkit-daemon19:37:501
415323299218,2cyclictest1414912ssh00:15:381
415322599213,10cyclictest4160098-21apt-get19:25:150
415322599212,14cyclictest4191724-21apt-get20:45:140
415322599212,14cyclictest4178028-21apt-get20:10:100
415322599212,13cyclictest4189780-21apt-get20:40:160
415322599210,11cyclictest13-21ksoftirqd/023:48:020
123699210,4rtkit-daemon1235-21rtkit-daemon19:13:010
415323299203,9cyclictest143711-21apt-get00:20:121
415323299202,16cyclictest4187610-21latency_hist20:34:591
4153232992012,6cyclictest124152-21ssh23:52:231
415323299192,15cyclictest3457-21latency_hist21:00:011
4153232991911,4cyclictest7844-21diskmemload21:45:121
415323299180,17cyclictest0-21swapper/120:20:261
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:253
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
none10:37:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional