You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-02 - 16:44

x86 Intel Celeron N3350 @1100 MHz, Linux 5.10.90-rt60 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot4.osadl.org (updated Fri Dec 02, 2022 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32391602331289,27sleep10-21swapper/107:07:531
32390802274232,28sleep00-21swapper/007:07:000
33007052710,1chrt3300706-21/usr/sbin/munin09:30:220
323950699658,27cyclictest0-21swapper/110:42:031
3239506996514,24cyclictest3268587-21fschecks_time08:20:181
323950699647,25cyclictest3377734-21munin-run11:35:001
323950699636,29cyclictest3388341-21awk11:50:301
3239506996318,23cyclictest3413434-21head12:30:311
3239506996310,29cyclictest3369621-21which11:20:241
32532082610,6sleep03253206-21awk07:40:250
323950699611,34cyclictest3395060-21users12:00:411
3239506996113,26cyclictest3256593-21cut07:50:181
3239506996111,28cyclictest3339254-21sh10:31:231
323950699605,34cyclictest3300316-21seq09:30:101
323950699605,31cyclictest3364750-21sh11:12:441
323950699605,28cyclictest0-21swapper/109:17:001
323950699604,27cyclictest3331814-21grep10:20:181
323950699604,25cyclictest3309397-21idleruntime-cro09:45:001
3239506996011,24cyclictest3366630-21ps11:15:261
3239506996010,30cyclictest3287505-21seq09:08:271
3239506996010,25cyclictest3324903-21latency_hist10:10:011
323950699599,30cyclictest3246413-21cat07:25:151
323950699599,29cyclictest3352587-21sh10:53:541
323950699598,28cyclictest3262146-21apt-get08:05:111
323950699597,29cyclictest3261215-21sensors08:00:281
323950699596,30cyclictest3282093-21ls08:55:171
323950699594,25cyclictest3243189-21sh07:15:231
323950699591,38cyclictest3405963-21apt-get12:20:151
323950699591,38cyclictest3318869-21apt-get10:00:211
3239506995912,27cyclictest3289420-21taskset09:10:521
3239506995910,30cyclictest3317141-21seq09:55:401
3239506995910,29cyclictest3386699-21ssh11:49:131
323950699589,29cyclictest3311330-21seq09:46:311
323950699588,30cyclictest3359066-21taskset11:04:161
323950699588,29cyclictest3383874-21ssh11:44:571
323950699588,28cyclictest3329677-21taskset10:15:311
323950699588,28cyclictest3329677-21taskset10:15:311
323950699586,27cyclictest3399960-21sort12:10:151
323950699581,37cyclictest3409036-21apt-get12:25:161
323950699581,37cyclictest3252355-21apt-get07:40:141
323950699581,37cyclictest3250253-21apt-get07:35:141
323950699581,36cyclictest3273932-21seq08:34:521
3239506995811,30cyclictest3417681-21ssh12:38:521
3239506995811,29cyclictest3250077-21sh07:35:011
3239506995811,28cyclictest3393013-21runrttasks11:59:581
3239506995810,29cyclictest3299391-21seq09:27:381
3239506995810,28cyclictest3349412-21sh10:48:211
3239506995810,28cyclictest3254774-21basename07:45:191
3239506995810,27cyclictest3360121-21uname11:05:191
323950699579,28cyclictest3314875-21sh09:53:151
323950699579,26cyclictest3270998-21ntp_kernel_pll_08:25:221
323950699577,24cyclictest3259048-21cut07:55:241
323950699576,30cyclictest3342163-21taskset10:36:101
323950699574,32cyclictest3372320-21cat11:25:191
323950699571,36cyclictest3334544-21apt-get10:25:141
3239506995710,28cyclictest3281592-21seq08:53:051
3239506995710,27cyclictest3398610-21chrt12:06:561
323950699569,29cyclictest3327217-21seq10:12:221
323950699568,29cyclictest3277765-21taskset08:43:541
323950699567,30cyclictest3285457-21seq09:02:251
323950699567,28cyclictest3354878-21sh10:55:441
323950699563,31cyclictest3379993-21seq11:37:441
323950699561,36cyclictest3294299-21apt-get09:20:151
3239506995610,30cyclictest3279742-21seq08:49:091
323950699558,29cyclictest3266450-21sed08:15:181
323950699558,27cyclictest3274524-21head08:35:181
323950699556,28cyclictest3404745-21chrt12:16:571
323950699552,28cyclictest3245146-21grep07:20:231
323950699550,35cyclictest3303411-21apt-get09:35:131
323950699540,34cyclictest3265610-21seq08:12:351
323950699480,27cyclictest3241275-21taskset07:14:081
3239502994520,18cyclictest3287673-21diskmemload11:00:210
3239502994412,18cyclictest3363426-21cut11:10:210
3239502994216,10cyclictest0-21swapper/007:35:520
323950299404,26cyclictest3287673-21diskmemload09:45:300
3239502994025,9cyclictest3287673-21diskmemload12:35:200
3239502994019,14cyclictest3300096-21cron09:30:000
3239502993920,6cyclictest3287673-21diskmemload10:45:170
159199390,4rtkit-daemon3394761-21mailstats12:00:390
159199390,3rtkit-daemon0-21swapper/110:38:480
323950299388,4cyclictest0-21swapper/008:00:000
159199380,5rtkit-daemon3351011-21grep10:50:240
159199380,3rtkit-daemon0-21swapper/108:23:550
159199380,3rtkit-daemon0-21swapper/009:08:310
323950299379,15cyclictest3287673-21diskmemload09:41:250
323950299378,5cyclictest3332384-21grep10:20:250
323950299377,6cyclictest3273369-21timerwakeupswit08:30:290
323950299374,2cyclictest3377711-21ssh11:35:010
323950299371,3cyclictest3337379-21munin-run10:30:000
159199370,3rtkit-daemon0-21swapper/108:57:390
159199370,3rtkit-daemon0-21swapper/012:15:240
3239502993625,3cyclictest3407047-21switchtime12:20:260
3239502993624,3cyclictest3245103-21cut07:20:210
3239502993617,2cyclictest0-21swapper/010:56:520
3239502993610,2cyclictest0-21swapper/011:55:230
323950299358,4cyclictest3397656-21pmu-power12:05:300
323950299358,2cyclictest141rcu_preempt09:50:250
323950299357,5cyclictest3256881-21missed_timers07:50:220
323950299357,4cyclictest3246686-21fschecks_time07:25:190
323950299357,3cyclictest0-21swapper/011:40:240
323950299356,5cyclictest3400356-21hwlatdetect12:10:210
323950299356,5cyclictest3264381-21grep08:10:160
323950299355,6cyclictest3295473-21timerandwakeup09:20:270
323950299354,6cyclictest3274805-21missed_timers08:35:210
3239502993525,2cyclictest3324725-21http10:10:020
3239502993523,2cyclictest3344438-21head10:40:200
323950299351,2cyclictest0-21swapper/009:10:260
3239502993510,5cyclictest13-21ksoftirqd/011:25:010
323950299350,11cyclictest33901752grep11:55:010
323950299349,4cyclictest3325302-21kworker/u8:110:19:500
323950299349,4cyclictest3325302-21kworker/u8:110:19:500
323950299349,2cyclictest0-21swapper/012:27:500
323950299347,5cyclictest3266703-21missed_timers08:15:220
323950299346,6cyclictest3255314-21mailstats07:45:240
323950299345,19cyclictest3287673-21diskmemload12:30:560
323950299344,5cyclictest3316765-21mailstats09:55:320
3239502993420,3cyclictest3366106-21grep11:15:210
3239502993413,4cyclictest3338783-21timerandwakeup10:30:280
3239502993411,5cyclictest3271411-21timerwakeupswit08:25:270
323950299338,5cyclictest3359698-21tr11:05:140
323950299336,4cyclictest3280580-21kernelversion08:50:220
323950299334,4cyclictest3243102-21needreboot07:15:200
3239502993324,2cyclictest3260599-21cut08:00:210
3239502993323,3cyclictest3304257-21ssh09:35:250
3239502993314,5cyclictest3249293-21mailstats07:30:250
3239502993314,3cyclictest3320112-21unixbench-2d10:00:350
3239502993313,4cyclictest3372382-21kernelversion11:25:210
3239502993313,4cyclictest3284259-21fschecks_time09:00:170
323950299330,24cyclictest3287673-21diskmemload11:45:090
323950299328,2cyclictest3279202-21timerandwakeup08:45:260
323950299325,3cyclictest3322584-21ls10:05:200
3239502993215,2cyclictest3291479-21tr09:15:190
3239502993215,2cyclictest3277403-21grep08:40:300
323950299319,5cyclictest3378617-21missed_timers11:35:250
323950299314,5cyclictest3262847-21cut08:05:220
323950299290,18cyclictest0-21swapper/007:10:220
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional