You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-02-05 - 00:00

x86 Intel Celeron N3350 @1100 MHz, Linux 5.10.90-rt60 (Profile)

Latency plot of system in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot4.osadl.org (updated Sat Feb 04, 2023 12:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3140752210830,9sleep0161rcuc/007:05:210
31418962275213,43sleep10-21swapper/107:07:421
314221099154139,4cyclictest3211405-21kworker/u8:110:27:320
3142213991100,1cyclictest0-21swapper/110:27:321
159199950,3rtkit-daemon1590-21rtkit-daemon07:38:521
159199850,18rtkit-daemon0-21swapper/108:12:150
314221399820,5cyclictest3262673-21tr10:45:241
159199780,3rtkit-daemon1590-21rtkit-daemon12:11:430
159199780,1rtkit-daemon1590-21rtkit-daemon08:16:050
159199780,1rtkit-daemon1590-21rtkit-daemon08:04:200
159199770,3rtkit-daemon1590-21rtkit-daemon10:03:060
159199770,3rtkit-daemon1590-21rtkit-daemon09:12:080
159199750,3rtkit-daemon1590-21rtkit-daemon11:28:160
159199750,1rtkit-daemon1590-21rtkit-daemon08:58:140
159199740,4rtkit-daemon1590-21rtkit-daemon08:06:120
159199730,4rtkit-daemon1590-21rtkit-daemon11:34:390
159199730,3rtkit-daemon1590-21rtkit-daemon11:19:140
159199730,3rtkit-daemon1590-21rtkit-daemon09:29:180
159199730,3rtkit-daemon1590-21rtkit-daemon08:21:020
159199720,1rtkit-daemon1590-21rtkit-daemon07:25:240
314221399710,2cyclictest3190150-21diskmemload09:15:521
159199710,3rtkit-daemon1590-21rtkit-daemon11:46:121
159199710,3rtkit-daemon1590-21rtkit-daemon07:47:120
159199710,1rtkit-daemon1590-21rtkit-daemon08:29:060
159199700,3rtkit-daemon1590-21rtkit-daemon11:09:150
159199700,15rtkit-daemon0-21swapper/107:34:020
314221399690,3cyclictest3190150-21diskmemload09:11:131
159199690,3rtkit-daemon1590-21rtkit-daemon09:40:110
159199690,3rtkit-daemon1590-21rtkit-daemon09:15:340
159199680,4rtkit-daemon1590-21rtkit-daemon11:04:050
159199680,3rtkit-daemon1590-21rtkit-daemon10:57:570
159199680,1rtkit-daemon1590-21rtkit-daemon10:11:210
314221399670,3cyclictest3148003-21timerwakeupswit07:20:331
159199670,4rtkit-daemon1590-21rtkit-daemon09:28:301
159199670,3rtkit-daemon1590-21rtkit-daemon08:43:050
159199670,3rtkit-daemon1590-21rtkit-daemon08:36:251
159199670,1rtkit-daemon1590-21rtkit-daemon12:24:360
159199670,1rtkit-daemon1590-21rtkit-daemon11:11:050
159199670,1rtkit-daemon1590-21rtkit-daemon09:24:270
159199670,1rtkit-daemon1590-21rtkit-daemon09:03:330
159199670,16rtkit-daemon0-21swapper/110:39:120
159199660,4rtkit-daemon1590-21rtkit-daemon07:10:190
159199660,3rtkit-daemon1590-21rtkit-daemon09:57:350
159199660,3rtkit-daemon1590-21rtkit-daemon07:36:360
3142213996513,26cyclictest3184754-21fschecks_count08:55:181
159199650,3rtkit-daemon1590-21rtkit-daemon10:23:250
159199650,2rtkit-daemon1590-21rtkit-daemon12:05:590
3142210996416,6cyclictest3178487-21grep08:40:010
159199640,3rtkit-daemon1590-21rtkit-daemon09:45:130
159199640,1rtkit-daemon1590-21rtkit-daemon12:32:220
159199640,1rtkit-daemon1590-21rtkit-daemon08:31:060
159199640,17rtkit-daemon0-21swapper/109:09:160
314221399633,2cyclictest3297711-21sh11:33:401
314221399632,12cyclictest3155149-21update-motd-upd07:40:211
314221399630,4cyclictest3250695-21apt-get10:30:121
314221399630,3cyclictest3206308-21df09:30:161
159199630,1rtkit-daemon1590-21rtkit-daemon08:53:450
314221399620,6cyclictest3256051-21sh10:36:281
314221399620,4cyclictest3333408-21cat12:20:291
314221399620,2cyclictest1-21systemd07:35:011
314221099621,19cyclictest3190150-21diskmemload12:20:020
159199620,4rtkit-daemon1590-21rtkit-daemon10:46:550
159199620,3rtkit-daemon1590-21rtkit-daemon12:27:231
159199620,3rtkit-daemon1590-21rtkit-daemon11:42:390
159199620,1rtkit-daemon1590-21rtkit-daemon11:55:241
159199620,1rtkit-daemon1590-21rtkit-daemon11:20:550
159199620,1rtkit-daemon1590-21rtkit-daemon07:41:090
314221399614,4cyclictest3164219-21unixbench_multi08:00:291
3142213996111,22cyclictest3189641-21unixbench_multi09:05:281
3142210996117,11cyclictest3190150-21diskmemload11:54:290
3142210996113,2cyclictest3306316-21tune2fs11:45:170
314221399601,2cyclictest3164971-21cpu08:05:151
314221399600,4cyclictest3331609-21/usr/sbin/munin12:20:011
314221399600,4cyclictest3287431-21apt-get11:20:121
314221399600,2cyclictest3220482-21diskmemload09:49:241
159199600,18rtkit-daemon0-21swapper/110:05:550
314221399599,3cyclictest0-21swapper/110:56:071
314221399590,1cyclictest0-21swapper/110:09:331
159199590,16rtkit-daemon0-21swapper/112:25:130
314221399588,5cyclictest3186434-21seq09:00:061
314221399586,2cyclictest3299542-21awk11:35:271
314221399582,2cyclictest3179299-21missed_timers08:40:231
314221399580,6cyclictest3343601-21gltestperf12:35:211
314221399580,2cyclictest3223952-21ssh09:53:411
314221399580,1cyclictest0-21swapper/111:13:561
3142210995821,6cyclictest3162807-21munin-run08:00:010
3142210995814,7cyclictest3343216-21cpu12:35:140
314221099581,2cyclictest0-21swapper/007:50:240
159199580,1rtkit-daemon1590-21rtkit-daemon07:16:530
159199580,16rtkit-daemon0-21swapper/108:46:190
314221399578,5cyclictest3340884-21vmstat12:30:291
314221399575,6cyclictest3168089-21sed08:10:281
314221399573,4cyclictest3173086-21cut08:25:251
314221399571,9cyclictest3174707-21/usr/sbin/munin08:30:221
314221399571,8cyclictest3284788-21/usr/sbin/munin11:15:271
314221399571,5cyclictest3190150-21diskmemload12:09:251
314221099576,2cyclictest0-21swapper/007:20:290
314221399566,4cyclictest0-21swapper/109:23:471
314221399561,3cyclictest0-21swapper/109:57:041
314221399560,8cyclictest3259232-21sed10:40:251
314221399560,5cyclictest3170118-21unixbench_singl08:15:311
314221399560,4cyclictest3320203-21python312:05:021
314221399560,3cyclictest0-21swapper/111:41:401
314221399560,2cyclictest3294743-21latency_hist11:30:021
3142210995621,9cyclictest3190150-21diskmemload09:52:400
3142210995610,2cyclictest3317756-21ntp_kernel_pll_12:00:240
314221399556,4cyclictest3190150-21diskmemload12:10:051
314221399556,1cyclictest0-21swapper/109:42:471
3142213995528,9cyclictest4000952-21snmpd07:15:021
3142213995511,19cyclictest4000952-21snmpd11:53:011
314221099557,5cyclictest3299017-21sh11:35:200
3142210995513,5cyclictest3211405-21kworker/u8:110:34:450
3142210995511,3cyclictest3315862-21ssh11:58:030
314221399544,2cyclictest3158402-21head07:45:281
314221399540,2cyclictest3190150-21diskmemload11:02:451
314221399540,1cyclictest3190150-21diskmemload10:18:081
159199540,3rtkit-daemon0-21swapper/007:11:351
314221399532,2cyclictest3182400-21latency_hist08:50:001
314221399531,8cyclictest3276438-21apt-get11:05:131
314221399530,4cyclictest3170815-21acpi08:20:111
314221399530,2cyclictest3190150-21diskmemload10:53:131
314221399530,1cyclictest3190150-21diskmemload09:37:101
3142210995318,8cyclictest3190150-21diskmemload10:19:500
314221399523,4cyclictest3183111-21latency08:50:201
314221399522,4cyclictest3149081-21fschecks_count07:25:251
314221399521,3cyclictest3161048-21apt-get07:55:141
314221399521,2cyclictest3228365-21apt-get10:00:121
314221399520,4cyclictest3160183-21timerandwakeup07:50:271
314221399520,1cyclictest0-21swapper/110:21:011
3142210995216,9cyclictest3190150-21diskmemload09:31:450
314221399510,7cyclictest3237200-21ssh10:10:291
3142210994922,7cyclictest3213298-21diskmemload09:39:520
314221099485,2cyclictest0-21swapper/010:50:170
3142210994822,9cyclictest3190150-21diskmemload10:44:400
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:533
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
none09:06:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional