You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-19 - 09:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot4.osadl.org (updated Wed Jun 19, 2024 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21469262259218,27sleep10-21swapper/119:07:231
21470882256188,52sleep00-21swapper/019:09:080
214728199113109,3cyclictest2207241-21kworker/u8:021:26:000
22155822590,5sleep00-21swapper/021:35:230
22946942550,4sleep02294676-21awk23:25:250
23323462520,4sleep10-21swapper/100:18:411
159199500,4rtkit-daemon1590-21rtkit-daemon23:03:280
159199500,4rtkit-daemon1590-21rtkit-daemon20:16:030
159199500,3rtkit-daemon1590-21rtkit-daemon23:31:301
159199460,3rtkit-daemon1590-21rtkit-daemon22:33:450
159199450,5rtkit-daemon2209728-21ssh21:28:311
22140952440,3sleep00-21swapper/021:34:110
159199420,8rtkit-daemon0-21swapper/123:42:361
159199400,5rtkit-daemon2239673-21cron22:10:001
159199390,3rtkit-daemon0-21swapper/000:22:230
21563402360,3chrt2155666-21http19:30:150
159199350,5rtkit-daemon0-21swapper/120:34:441
159199350,4rtkit-daemon1590-21rtkit-daemon23:35:551
159199350,4rtkit-daemon1590-21rtkit-daemon21:14:021
159199350,3rtkit-daemon0-21swapper/123:52:460
159199350,3rtkit-daemon0-21swapper/120:25:381
159199350,3rtkit-daemon0-21swapper/019:27:520
159199340,4rtkit-daemon1590-21rtkit-daemon23:09:190
159199340,3rtkit-daemon0-21swapper/123:49:081
159199340,3rtkit-daemon0-21swapper/121:31:581
159199340,3rtkit-daemon0-21swapper/100:05:380
159199340,3rtkit-daemon0-21swapper/022:02:140
159199340,3rtkit-daemon0-21swapper/021:57:120
159199340,3rtkit-daemon0-21swapper/021:11:280
159199330,4rtkit-daemon1590-21rtkit-daemon21:45:581
159199330,4rtkit-daemon1590-21rtkit-daemon19:20:180
159199330,3rtkit-daemon0-21swapper/121:22:190
159199330,3rtkit-daemon0-21swapper/020:42:130
214728199322,23cyclictest2149954-21apt-get19:15:140
159199320,4rtkit-daemon1590-21rtkit-daemon19:58:580
159199320,3rtkit-daemon0-21swapper/122:11:220
2147281993113,4cyclictest13-21ksoftirqd/023:31:090
159199310,3rtkit-daemon1590-21rtkit-daemon20:51:140
159199310,3rtkit-daemon0-21swapper/023:10:411
159199310,3rtkit-daemon0-21swapper/020:07:381
159199310,3rtkit-daemon0-21swapper/000:11:070
214728199304,2cyclictest141rcu_preempt23:20:000
159199300,4rtkit-daemon1590-21rtkit-daemon00:29:030
159199300,3rtkit-daemon1590-21rtkit-daemon22:35:090
159199300,3rtkit-daemon1590-21rtkit-daemon20:46:200
159199300,3rtkit-daemon1590-21rtkit-daemon19:54:550
159199300,3rtkit-daemon1590-21rtkit-daemon19:35:320
159199300,3rtkit-daemon0-21swapper/123:43:140
159199300,3rtkit-daemon0-21swapper/023:24:041
159199300,3rtkit-daemon0-21swapper/023:06:151
159199300,3rtkit-daemon0-21swapper/023:04:121
159199300,3rtkit-daemon0-21swapper/022:58:401
159199300,3rtkit-daemon0-21swapper/021:56:001
159199300,3rtkit-daemon0-21swapper/020:00:101
23463312290,4chrt0-21swapper/000:37:530
22614632290,25sleep02261537-21df22:40:190
159199290,4rtkit-daemon2201635-21ssh21:16:540
159199290,3rtkit-daemon1590-21rtkit-daemon21:04:410
159199290,3rtkit-daemon0-21swapper/020:59:341
2147286992820,6cyclictest2318870-21apt-get00:00:101
159199280,4rtkit-daemon1590-21rtkit-daemon19:25:511
159199280,3rtkit-daemon0-21swapper/121:47:340
159199280,3rtkit-daemon0-21swapper/100:31:350
159199280,3rtkit-daemon0-21swapper/021:05:031
159199280,3rtkit-daemon0-21swapper/021:01:331
159199280,24rtkit-daemon2186474-21cut20:45:241
21852862270,14chrt26-21ksoftirqd/120:41:211
214728699270,6cyclictest0-21swapper/100:38:401
2147281992719,4cyclictest2195489-21diskmemload22:59:050
2147281992713,3cyclictest13-21ksoftirqd/000:00:340
2147286992612,8cyclictest63950irq/130-enp2s019:30:131
214728699260,6cyclictest0-21swapper/119:57:451
214728199268,2cyclictest141rcu_preempt22:29:250
214728199268,2cyclictest141rcu_preempt20:35:320
214728199267,4cyclictest2264745-21apt-get22:45:110
214728199267,4cyclictest2264745-21apt-get22:45:110
214728199267,4cyclictest13-21ksoftirqd/000:15:160
214728199265,2cyclictest141rcu_preempt23:55:280
214728199262,17cyclictest2282946-21apt-get23:10:160
159199260,3rtkit-daemon0-21swapper/019:41:291
159199260,21rtkit-daemon2147278-21cyclictest00:13:301
159199260,1rtkit-daemon1590-21rtkit-daemon21:36:251
214728699259,9cyclictest2311692-21apt-get23:50:121
214728699258,11cyclictest2196248-1kworker/u9:023:25:121
2147286992511,9cyclictest769347-1kworker/u9:121:50:121
214728699250,6cyclictest0-21swapper/120:40:011
214728199259,2cyclictest141rcu_preempt23:37:380
214728199259,2cyclictest141rcu_preempt22:16:410
214728199252,2cyclictest141rcu_preempt21:40:340
214728199252,16cyclictest2169878-21apt-get20:05:130
159199250,20rtkit-daemon2147278-21cyclictest22:54:251
159199250,1rtkit-daemon1590-21rtkit-daemon22:50:430
214728699249,9cyclictest2196248-1kworker/u9:023:15:131
2147286992416,6cyclictest2250323-21cron22:24:591
214728699240,6cyclictest0-21swapper/122:10:241
214728699240,6cyclictest0-21swapper/120:24:191
214728699240,6cyclictest0-21swapper/120:13:171
214728699240,6cyclictest0-21swapper/100:21:091
214728199248,2cyclictest141rcu_preempt22:23:300
214728199248,2cyclictest141rcu_preempt19:10:110
214728199247,4cyclictest13-21ksoftirqd/023:23:000
214728199246,16cyclictest141rcu_preempt23:45:240
214728199242,15cyclictest2176030-21apt-get20:20:110
2147281992410,8cyclictest2160114-21apt-get19:40:150
2147281992410,2cyclictest141rcu_preempt22:06:020
2147281992410,12cyclictest161rcuc/021:00:010
159199240,1rtkit-daemon1590-21rtkit-daemon19:46:160
159199240,19rtkit-daemon2147278-21cyclictest20:27:390
214728699230,5cyclictest0-21swapper/122:26:151
2147281992317,4cyclictest161rcuc/020:15:010
159199230,4rtkit-daemon1590-21rtkit-daemon19:15:221
159199230,1rtkit-daemon1590-21rtkit-daemon22:15:551
214728699229,8cyclictest2297852-1kworker/u9:223:55:121
214728699222,14cyclictest2218489-21apt-get21:40:151
214728699222,14cyclictest2164010-21apt-get19:50:141
214728199229,3cyclictest13-21ksoftirqd/021:05:290
159199220,3rtkit-daemon1590-21rtkit-daemon00:27:011
159199220,17rtkit-daemon1590-21rtkit-daemon21:50:350
214728699211,14cyclictest2261308-21apt-get22:40:171
214728199218,7cyclictest13-21ksoftirqd/020:00:150
214728199212,17cyclictest2181762-21cat20:35:000
159199210,3rtkit-daemon1590-21rtkit-daemon22:00:151
214728699204,8cyclictest1591046-21snapd00:05:071
214728699202,13cyclictest2199566-21apt-get21:15:131
159199200,13rtkit-daemon0-21swapper/121:23:011
214728699192,12cyclictest2257722-21apt-get22:35:141
214728699192,12cyclictest2162060-21apt-get19:45:141
2147286991914,3cyclictest2265689-21proc_pri22:45:271
2147286991914,3cyclictest2265689-21proc_pri22:45:271
2147286991912,5cyclictest2159877-21cron19:40:001
159199190,3rtkit-daemon0-21swapper/100:31:141
214728699188,7cyclictest2147744-21perl19:10:191
2147286991813,3cyclictest2188629-21ntp_offset20:50:261
2147286991811,5cyclictest2255045-21ntpq22:30:321
2147286991712,3cyclictest2152157-21fschecks_count19:20:171
214728699151,5cyclictest2174039-21/usr/sbin/munin20:15:301
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional