You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-30 - 07:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot4.osadl.org (updated Sat Sep 30, 2023 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34630002329284,27sleep10-21swapper/119:05:111
34643702319275,28sleep00-21swapper/019:05:560
3464854992040,2cyclictest3568445-21fwupd22:28:531
346485299152141,5cyclictest3555284-21kworker/u8:322:28:530
159199680,4rtkit-daemon1590-21rtkit-daemon21:16:421
159199680,3rtkit-daemon0-21swapper/123:22:520
159199670,4rtkit-daemon1590-21rtkit-daemon19:11:371
159199650,3rtkit-daemon0-21swapper/122:59:350
346485299646,3cyclictest0-21swapper/000:19:020
3464852996421,25cyclictest3499969-21gltestperf20:35:280
159199640,4rtkit-daemon1590-21rtkit-daemon23:23:561
159199640,4rtkit-daemon1590-21rtkit-daemon00:20:120
159199640,3rtkit-daemon0-21swapper/120:13:390
3464852996225,26cyclictest3653073-21ls00:35:020
159199620,4rtkit-daemon1590-21rtkit-daemon23:54:160
159199620,3rtkit-daemon1590-21rtkit-daemon00:13:010
159199620,3rtkit-daemon0-21swapper/123:19:120
159199610,3rtkit-daemon1590-21rtkit-daemon21:58:030
346485299601,8cyclictest3514619-21ntp_states21:10:300
346485299600,4cyclictest3631193-21cat00:00:280
159199600,3rtkit-daemon0-21swapper/123:04:470
159199600,3rtkit-daemon0-21swapper/121:07:470
35646902590,2sleep10-21swapper/122:22:591
3464852995925,20cyclictest3613977-21grep23:35:310
3464852995918,24cyclictest3496571-21processes20:25:340
346485299591,5cyclictest3627186-21apt-get23:55:210
159199590,3rtkit-daemon0-21swapper/123:33:240
159199590,3rtkit-daemon0-21swapper/121:33:280
159199590,3rtkit-daemon0-21swapper/120:31:510
159199590,3rtkit-daemon0-21swapper/119:11:240
159199590,3rtkit-daemon0-21swapper/022:54:141
159199590,3rtkit-daemon0-21swapper/020:57:421
3464852995813,8cyclictest3483320-21apt-get19:55:000
3464852995811,2cyclictest0-21swapper/020:05:280
3464852995811,2cyclictest0-21swapper/020:05:270
159199580,3rtkit-daemon1590-21rtkit-daemon22:58:541
159199580,3rtkit-daemon0-21swapper/120:50:240
159199580,3rtkit-daemon0-21swapper/019:18:051
159199580,3rtkit-daemon0-21swapper/000:00:391
346485299571,6cyclictest3513470-21diskmemload21:45:330
3464852995715,32cyclictest3493802-21cpuspeed_turbos20:20:210
346485299571,4cyclictest3559241-21diskstats22:15:240
346485299571,4cyclictest3509739-40apt-check21:00:190
159199570,3rtkit-daemon0-21swapper/121:16:500
159199570,3rtkit-daemon0-21swapper/120:59:090
159199570,3rtkit-daemon0-21swapper/119:41:430
159199570,3rtkit-daemon0-21swapper/100:36:380
3464852995615,10cyclictest3504303-21pmu-power20:45:350
159199560,5rtkit-daemon1590-21rtkit-daemon21:00:181
159199560,3rtkit-daemon0-21swapper/123:28:130
159199560,3rtkit-daemon0-21swapper/122:47:500
159199560,3rtkit-daemon0-21swapper/122:34:030
159199560,3rtkit-daemon0-21swapper/121:53:180
159199560,3rtkit-daemon0-21swapper/121:23:530
159199560,3rtkit-daemon0-21swapper/119:21:120
159199560,3rtkit-daemon0-21swapper/019:43:431
346485299559,3cyclictest3646740-21df_inode00:25:210
346485299555,10cyclictest3480815-21timerwakeupswit19:45:370
346485299555,10cyclictest3480815-21timerwakeupswit19:45:360
3464852995513,3cyclictest3620272-21ssh23:45:150
3464852995512,3cyclictest0-21swapper/000:08:260
159199550,3rtkit-daemon1590-21rtkit-daemon23:12:480
159199550,3rtkit-daemon0-21swapper/020:22:021
159199550,2rtkit-daemon0-21swapper/020:00:591
346485299540,2cyclictest3619754-21ssh23:44:240
3464854995327,8cyclictest3651988-21sh00:32:251
3464854995326,9cyclictest3596163-21sh23:10:051
3464852995320,17cyclictest3485147-21unixbench_multi19:55:410
159199530,3rtkit-daemon1590-21rtkit-daemon22:06:160
159199530,3rtkit-daemon0-21swapper/121:29:270
159199530,3rtkit-daemon0-21swapper/022:16:071
159199530,3rtkit-daemon0-21swapper/020:15:411
36449612520,2sleep10-21swapper/100:21:391
3464852995218,6cyclictest3475740-21gltestperf19:35:250
3464852995211,5cyclictest3471757-21grep19:25:250
346485299520,2cyclictest3502159-21missed_timers20:40:320
159199520,3rtkit-daemon0-21swapper/022:00:030
159199520,3rtkit-daemon0-21swapper/000:10:141
3464854995126,7cyclictest3630165-21sh23:59:521
346485299514,2cyclictest3466944-21missed_timers19:15:020
3464852995114,3cyclictest3474645-21sed19:30:370
3464854995026,8cyclictest3595632-21sh23:09:151
3464854995023,9cyclictest3642961-21latency_hist00:20:011
346485299501,3cyclictest0-21swapper/023:09:200
159199500,3rtkit-daemon0-21swapper/123:25:451
159199500,3rtkit-daemon0-21swapper/021:48:451
159199500,3rtkit-daemon0-21swapper/021:10:421
159199500,3rtkit-daemon0-21swapper/000:05:371
3464854994930,2cyclictest3600938-21grep23:15:371
3464854994913,16cyclictest3646974-21gltestperf00:25:261
346485299496,2cyclictest0-21swapper/022:53:250
346485299495,5cyclictest3486563-21needreboot20:00:310
159199490,3rtkit-daemon0-21swapper/122:33:261
34732072480,5sleep1251rcuc/119:30:011
346485499487,4cyclictest0-21swapper/120:50:381
3464854994823,8cyclictest3592101-21sh23:03:511
3464854994815,16cyclictest3653584-21tar00:35:041
3464852994816,9cyclictest3492160-21gltestperf20:15:290
159199480,3rtkit-daemon0-21swapper/119:53:441
159199480,3rtkit-daemon0-21swapper/021:55:261
3464854994724,6cyclictest3620759-21gltestperf23:45:271
3464854994724,6cyclictest3617331-21/usr/sbin/munin23:40:331
3464854994719,6cyclictest3613674-21gltestperf23:35:281
3464854994717,8cyclictest3479707-21sort19:45:221
3464854994717,8cyclictest3479707-21sort19:45:211
3464854994714,3cyclictest26-21ksoftirqd/120:00:001
159199470,3rtkit-daemon0-21swapper/119:23:521
3464854994622,8cyclictest3611982-21sh23:32:361
3464854994615,9cyclictest3503816-21gltestperf20:45:281
3464854994612,9cyclictest3624616-21sh23:50:331
159199460,3rtkit-daemon1590-21rtkit-daemon20:05:051
159199460,3rtkit-daemon1590-21rtkit-daemon20:05:041
3464854994523,2cyclictest3496015-21grep20:25:261
3464854994516,14cyclictest3489851-21sort20:10:221
3464852994512,10cyclictest2378633-1kworker/u9:022:35:230
159199450,1rtkit-daemon1590-21rtkit-daemon21:34:311
346485499448,5cyclictest3498347-21ntpq20:30:331
346485499446,2cyclictest0-21swapper/120:35:341
3464854994413,2cyclictest0-21swapper/120:40:401
159199440,4rtkit-daemon0-21swapper/121:29:481
159199440,3rtkit-daemon1590-21rtkit-daemon21:23:121
3464854994315,12cyclictest3475443-21cpuspeed_turbos19:35:201
3464854994313,2cyclictest0-21swapper/119:35:001
346485299436,17cyclictest3513470-21diskmemload21:40:080
159199420,3rtkit-daemon0-21swapper/122:24:410
159199420,3rtkit-daemon0-21swapper/022:35:141
159199420,3rtkit-daemon0-21swapper/022:01:071
159199410,3rtkit-daemon0-21swapper/122:14:320
159199410,24rtkit-daemon0-21swapper/121:05:091
159199380,1rtkit-daemon1590-21rtkit-daemon22:44:350
159199380,16rtkit-daemon0-21swapper/121:36:541
346485299338,6cyclictest3533041-21diskmemload21:35:440
159199320,16rtkit-daemon0-21swapper/121:54:001
3464854993121,5cyclictest3580187-21grep22:45:311
346485499312,13cyclictest3552104-21apt-get22:05:191
3464854993019,6cyclictest420575-21snmpd22:10:141
3464854993018,6cyclictest809-21runrttasks22:43:481
346485499301,6cyclictest3513470-21diskmemload21:43:271
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional