You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-09 - 17:13

x86 Intel Celeron N3350 @1100 MHz, Linux 5.10.90-rt60 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot4.osadl.org (updated Wed Dec 07, 2022 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
312036024750,8sleep10-21swapper/107:05:181
31217722246173,58sleep00-21swapper/007:08:500
159199860,6rtkit-daemon1590-21rtkit-daemon10:20:170
32448952770,1chrt3244898-21proc_pri11:10:351
32046412700,8chrt3204639-21ssh10:05:180
159199700,5rtkit-daemon1590-21rtkit-daemon11:57:000
31838462680,5sleep013-21ksoftirqd/009:30:280
159199670,3rtkit-daemon0-21swapper/007:22:371
159199660,3rtkit-daemon0-21swapper/008:13:001
159199630,3rtkit-daemon0-21swapper/010:12:131
159199630,3rtkit-daemon0-21swapper/009:59:141
32816802610,4sleep00-21swapper/012:10:260
159199610,3rtkit-daemon0-21swapper/010:54:201
159199610,3rtkit-daemon0-21swapper/009:23:281
159199610,3rtkit-daemon0-21swapper/008:28:201
159199610,3rtkit-daemon0-21swapper/008:18:561
159199610,3rtkit-daemon0-21swapper/008:07:571
159199580,3rtkit-daemon0-21swapper/007:47:111
3121980995718,21cyclictest3154528-21awk08:30:000
32765632560,4sleep00-21swapper/012:02:070
312198299561,4cyclictest3142687-21apt-get08:00:181
312198299556,3cyclictest3292822-21cron12:30:001
3121982995521,10cyclictest3242566-21ssh11:06:311
3121980995521,9cyclictest3219405-21sh10:29:220
159199550,3rtkit-daemon0-21swapper/011:32:310
159199550,3rtkit-daemon0-21swapper/008:52:380
159199550,3rtkit-daemon0-21swapper/007:11:471
159199550,3rtkit-daemon0-21swapper/007:10:580
32956822540,10sleep126-21ksoftirqd/112:34:281
32259102540,4chrt0-21swapper/110:40:231
159199540,6rtkit-daemon3173892-21/usr/sbin/munin09:15:291
159199540,3rtkit-daemon0-21swapper/011:15:530
159199540,3rtkit-daemon0-21swapper/009:20:260
159199530,3rtkit-daemon0-21swapper/012:37:450
159199530,3rtkit-daemon0-21swapper/011:21:200
312198299527,2cyclictest141rcu_preempt10:16:211
3121982995219,4cyclictest26-21ksoftirqd/112:40:001
312198099529,3cyclictest141rcu_preempt12:30:150
3121980995226,12cyclictest3133324-21open_files07:35:290
3121980995221,11cyclictest3128787-21cstates07:25:170
159199520,3rtkit-daemon0-21swapper/011:26:300
159199520,3rtkit-daemon0-21swapper/011:06:160
159199520,3rtkit-daemon0-21swapper/010:47:510
159199520,3rtkit-daemon0-21swapper/009:48:171
159199520,3rtkit-daemon0-21swapper/009:04:101
312198099516,12cyclictest3284425-21gltestperf12:15:230
159199510,3rtkit-daemon0-21swapper/012:19:171
159199510,3rtkit-daemon0-21swapper/012:06:321
159199510,3rtkit-daemon0-21swapper/009:40:061
159199510,3rtkit-daemon0-21swapper/008:34:510
312198299503,7cyclictest3156263-21anacron08:32:561
3121982995027,6cyclictest3214609-21pmu-power10:20:311
312198299502,2cyclictest3287836-21needreboot12:20:271
3121982995018,13cyclictest3170495-21diskmemload09:11:321
3121982995018,10cyclictest3255226-21ssh11:26:571
3121982995016,13cyclictest3170495-21diskmemload10:31:471
3121982995016,10cyclictest3170495-21diskmemload11:31:261
3121982995013,2cyclictest141rcu_preempt11:16:021
3121980995027,6cyclictest3180486-21gltestperf09:25:200
3121980995026,9cyclictest3175249-21unixbench_singl09:15:310
3121980995026,7cyclictest3278315-21fschecks_time12:05:260
3121980995023,7cyclictest3168723-21/usr/sbin/munin09:05:330
159199500,2rtkit-daemon0-21swapper/008:35:120
3121982994914,10cyclictest3170495-21diskmemload12:01:231
3121980994927,6cyclictest3238936-21swap11:00:300
3121980994926,6cyclictest3211441-21ntp_kernel_err10:15:280
3121980994921,11cyclictest3200661-21sh09:59:040
3121980994919,15cyclictest3131792-21unixbench_multi07:30:290
3121980994918,6cyclictest3226513-21missed_timers10:40:340
3121980994918,6cyclictest3186665-21missed_timers09:35:400
3121980994917,7cyclictest3134690-21cpuspeed_turbos07:40:170
3121980994916,8cyclictest3188939-21sh09:40:040
3121980994914,2cyclictest141rcu_preempt10:55:010
159199490,3rtkit-daemon0-21swapper/011:11:350
159199490,3rtkit-daemon0-21swapper/010:36:541
159199490,3rtkit-daemon0-21swapper/008:45:041
159199490,3rtkit-daemon0-21swapper/008:12:190
3121982994829,6cyclictest3252002-21diskmemload11:21:251
3121982994820,7cyclictest3272483-21ntp_states11:55:271
3121982994814,17cyclictest3170495-21diskmemload11:03:401
3121982994813,15cyclictest1090681-21ntpd09:50:251
3121980994825,6cyclictest3266067-21idleruntime11:45:260
3121980994825,6cyclictest3266067-21idleruntime11:45:260
3121980994825,5cyclictest3236112-21unixbench-2d10:55:330
3121980994823,9cyclictest3173402-21taskset09:14:160
3121980994819,7cyclictest3151422-21load08:20:290
3121980994819,12cyclictest3224496-21sh10:36:380
3121980994817,7cyclictest3288566-21sh12:20:490
3121980994817,6cyclictest3194143-21sh09:47:370
3121980994817,6cyclictest3140577-21cpuspeed_turbos07:55:150
3121980994816,12cyclictest3142716-21runrttasks08:00:240
3121980994816,10cyclictest3170495-21diskmemload09:50:290
3121980994814,9cyclictest3290385-21fschecks_count12:25:240
159199480,3rtkit-daemon0-21swapper/010:28:211
159199480,3rtkit-daemon0-21swapper/008:06:330
3121982994726,7cyclictest3236013-21ssh10:55:311
3121982994717,10cyclictest3170495-21diskmemload10:46:281
3121982994716,16cyclictest3170495-21diskmemload12:11:201
3121980994725,6cyclictest3158943-21fschecks_count08:40:180
3121980994724,8cyclictest3165606-21timerandwakeup08:55:280
3121980994724,6cyclictest3263063-21idleruntime11:40:260
3121980994724,6cyclictest3219816-21sh10:30:070
3121980994722,6cyclictest3209437-21sh10:11:500
3121980994722,6cyclictest3160788-21cstates08:45:170
3121980994720,7cyclictest3260106-21irqstats11:35:280
3121980994717,6cyclictest3136838-21fschecks_time07:45:190
3121980994715,7cyclictest3203592-21sh10:02:360
312198299466,6cyclictest3205033-21grep10:05:231
3121982994625,7cyclictest3164809-21tune2fs08:55:191
3121982994619,3cyclictest3267124-21diskmemload11:46:111
3121982994619,3cyclictest3267124-21diskmemload11:46:101
3121982994616,8cyclictest3170495-21diskmemload09:31:331
3121982994613,14cyclictest3157862-21unixbench-2d08:35:321
3121980994623,9cyclictest3167053-21taskset09:00:230
3121980994623,7cyclictest3270325-21diskmemload11:51:360
3121980994613,17cyclictest3138457-21/usr/sbin/munin07:50:290
3121982994525,8cyclictest3133170-21missed_timers07:35:271
3121982994523,4cyclictest3261282-21diskmemload11:36:491
3121982994521,12cyclictest3201630-21apt-esm-hook10:00:291
3121982994518,8cyclictest3170495-21diskmemload09:39:101
3121982994513,16cyclictest3181309-21unixbench-2d09:25:301
3121982994512,20cyclictest3131036-21idleruntime07:30:201
3121982994510,12cyclictest3264353-21diskmemload11:42:001
3121980994523,5cyclictest3150687-21latency_hist08:20:010
3121980994510,24cyclictest3128265-21taskset07:23:370
312198299449,2cyclictest0-21swapper/108:50:221
3121982994426,2cyclictest3268782-21cstates11:50:171
3121982994411,2cyclictest0-21swapper/108:20:301
3121980994422,5cyclictest3124703-21cpu07:15:150
312198299436,5cyclictest3125907-21grep07:15:291
312198299434,4cyclictest26-21ksoftirqd/107:29:581
312198299433,4cyclictest3158757-21grep08:40:141
3121982994315,6cyclictest3138642-21cstates07:50:201
3121982994315,5cyclictest3134933-21grep07:40:211
312198299412,6cyclictest3141108-21grep07:55:221
312198299409,3cyclictest0-21swapper/109:05:231
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional