You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-03-02 - 01:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot4.osadl.org (updated Fri Mar 01, 2024 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33994822315274,27sleep00-21swapper/007:09:540
33993282291191,60sleep10-21swapper/107:08:161
339962099211204,3cyclictest3463276-21mailstats09:30:371
3399617999280,7cyclictest3463107-21fwupd09:30:360
159199710,3rtkit-daemon1590-21rtkit-daemon09:12:311
34022082600,4sleep00-21swapper/007:15:050
159199590,4rtkit-daemon1590-21rtkit-daemon08:04:220
34898722560,1chrt3489871-21ssh10:06:581
159199560,5rtkit-daemon1590-21rtkit-daemon11:03:061
34956562530,4sleep126-21ksoftirqd/110:15:271
159199530,4rtkit-daemon1590-21rtkit-daemon12:10:521
159199510,4rtkit-daemon1590-21rtkit-daemon07:59:160
159199500,4rtkit-daemon1590-21rtkit-daemon12:23:011
159199500,4rtkit-daemon1590-21rtkit-daemon10:01:540
35560742490,1chrt3556073-21/usr/sbin/munin11:40:331
159199490,4rtkit-daemon1590-21rtkit-daemon07:15:491
159199480,5rtkit-daemon1590-21rtkit-daemon09:20:160
159199480,4rtkit-daemon1590-21rtkit-daemon10:06:460
159199480,19rtkit-daemon0-21swapper/009:28:550
34230462470,4sleep03423011-21needreboot08:05:400
34425002460,4sleep00-21swapper/008:55:280
159199460,7rtkit-daemon1590-21rtkit-daemon10:27:071
159199460,4rtkit-daemon1590-21rtkit-daemon10:53:271
3399617994523,7cyclictest3431879-21cron08:30:000
3399617994421,11cyclictest3513353-21needreboot10:40:310
159199440,3rtkit-daemon1590-21rtkit-daemon12:30:171
159199430,3rtkit-daemon1590-21rtkit-daemon12:15:341
34735062420,3sleep1251rcuc/109:45:281
339961799403,5cyclictest13-21ksoftirqd/010:25:370
3399617994024,8cyclictest3473007-21cron09:45:000
3399617994023,8cyclictest3519174-21cron10:50:000
3399617994011,3cyclictest141rcu_preempt10:10:210
3399617994011,2cyclictest3526829-21ssh11:00:190
159199400,3rtkit-daemon1590-21rtkit-daemon08:23:020
159199400,3rtkit-daemon0-21swapper/107:27:510
35131832390,3sleep1251rcuc/110:40:291
3399617993924,8cyclictest3401483-21cron07:15:000
159199390,6rtkit-daemon3538055-21df11:15:261
159199390,3rtkit-daemon0-21swapper/107:36:260
35333872380,3chrt0-21swapper/111:08:561
339962099389,4cyclictest0-21swapper/109:59:091
3399620993819,3cyclictest0-21swapper/107:55:281
339961799388,2cyclictest141rcu_preempt10:51:300
3399617993816,11cyclictest3498365-21cron10:19:590
339962099379,4cyclictest0-21swapper/108:10:011
339962099379,2cyclictest0-21swapper/107:30:321
3399620993722,9cyclictest3526193-21python311:00:001
339961799378,6cyclictest3448476-21latency09:10:270
3399617993725,5cyclictest3434889-21pmu-power08:35:350
3399617993718,5cyclictest3480068-21cron09:55:000
3399617993717,6cyclictest3426594-21gltestperf08:15:270
3399617993717,6cyclictest3426594-21gltestperf08:15:260
3399617993717,5cyclictest3512309-21cron10:40:000
339961799371,3cyclictest141rcu_preempt09:35:230
159199370,26rtkit-daemon3372530-21gmain07:49:030
339962099368,5cyclictest3441036-21wc08:50:431
339962099367,3cyclictest141rcu_preempt08:35:341
3399620993620,9cyclictest3512312-21cron10:40:001
3399620993614,2cyclictest0-21swapper/110:03:361
3399620993613,2cyclictest0-21swapper/110:49:231
3399620993611,9cyclictest3447701-1kworker/u9:109:25:231
3399620993610,3cyclictest141rcu_preempt10:15:011
3399617993623,5cyclictest3454819-21cron09:20:000
159199360,4rtkit-daemon1590-21rtkit-daemon09:55:260
339962099359,4cyclictest0-21swapper/110:24:231
339962099359,2cyclictest141rcu_preempt07:55:011
3399620993523,2cyclictest3407445-21grep07:25:511
3399620993515,1cyclictest0-21swapper/109:52:201
3399620993510,2cyclictest0-21swapper/109:00:211
339961799357,3cyclictest141rcu_preempt09:45:240
159199350,3rtkit-daemon0-21swapper/112:38:311
159199350,3rtkit-daemon0-21swapper/111:59:221
159199350,3rtkit-daemon0-21swapper/008:21:261
35070362346,2sleep13507042-21sh10:30:521
339962099348,4cyclictest0-21swapper/107:35:341
339962099347,5cyclictest3439188-21grep08:45:341
339962099347,4cyclictest3470293-21grep09:40:321
3399620993422,6cyclictest3466342-21gltestperf09:35:301
3399620993422,3cyclictest3446307-21kernelversion09:05:321
3399620993422,2cyclictest3404618-21grep07:20:341
3399620993419,2cyclictest0-21swapper/108:15:341
3399620993419,2cyclictest0-21swapper/108:15:341
3399620993414,2cyclictest0-21swapper/107:10:421
3399620993412,2cyclictest141rcu_preempt09:00:021
3399620993411,2cyclictest0-21swapper/108:25:331
339961799346,3cyclictest161rcuc/007:20:400
3399617993418,9cyclictest3417825-21cron07:55:010
3399617993414,4cyclictest3523462-21hwlatdetect10:55:290
3399617993414,4cyclictest3498805-21cpuspeed_turbos10:20:250
3399617993412,5cyclictest3424621-21grep08:10:250
159199340,4rtkit-daemon3576125-21ssh12:09:110
159199340,3rtkit-daemon0-21swapper/108:42:181
159199340,3rtkit-daemon0-21swapper/009:09:370
35072282331,27sleep03447514-21diskmemload10:31:180
339962099338,2cyclictest0-21swapper/108:00:031
339962099333,3cyclictest141rcu_preempt09:21:341
3399620993325,3cyclictest3451908-21grep09:15:251
3399620993317,2cyclictest3414858-21head07:45:341
339961799336,3cyclictest141rcu_preempt08:45:180
3399617993321,2cyclictest3433839-21idleruntime-cro08:35:000
3399617993318,8cyclictest3409874-21cron07:35:010
159199330,3rtkit-daemon1590-21rtkit-daemon11:39:391
159199330,3rtkit-daemon0-21swapper/112:25:360
159199330,3rtkit-daemon0-21swapper/111:53:330
339962099322,4cyclictest3432474-21grep08:30:321
3399620993216,2cyclictest0-21swapper/108:15:001
3399620993215,3cyclictest0-21swapper/107:40:241
339961799327,2cyclictest3436648-21missed_timers08:40:290
339961799325,5cyclictest3440510-21grep08:50:380
3399617993222,3cyclictest3413048-21grep07:40:390
3399617993119,4cyclictest3444231-21gltestperf09:00:260
159199310,4rtkit-daemon1590-21rtkit-daemon11:48:380
159199310,3rtkit-daemon0-21swapper/112:12:400
339961799301,10cyclictest3552187-21diskstats11:35:260
159199300,3rtkit-daemon1590-21rtkit-daemon11:17:490
339962099299,3cyclictest141rcu_preempt12:10:011
3399617992924,3cyclictest1855330-20systemd-journal11:05:010
159199290,1rtkit-daemon1590-21rtkit-daemon12:00:201
159199280,4rtkit-daemon0-21swapper/112:31:270
159199280,3rtkit-daemon0-21swapper/111:31:440
159199280,3rtkit-daemon0-21swapper/011:14:361
339961799277,7cyclictest13-21ksoftirqd/012:39:050
339961799267,9cyclictest141rcu_preempt12:15:280
3399617992610,2cyclictest141rcu_preempt12:00:180
339962099252,16cyclictest3544707-21apt-get11:25:201
3399620992517,3cyclictest3463187-21gdbus11:30:421
3399620992512,8cyclictest2370405-1kworker/u9:011:20:191
339961799252,6cyclictest13-21ksoftirqd/011:21:260
339961799252,16cyclictest3584232-21apt-get12:20:180
339961799252,16cyclictest3565996-21apt-get11:55:250
339961799246,7cyclictest13-21ksoftirqd/011:28:360
3399617992411,11cyclictest13-21ksoftirqd/011:15:010
3399617992310,3cyclictest13-21ksoftirqd/011:40:480
339962099210,6cyclictest0-21swapper/111:54:111
3399620992011,5cyclictest3447514-21diskmemload11:49:041
339962099170,4cyclictest0-21swapper/112:27:251
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional