You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-09 - 00:47

x86 Intel Core i3-2310E @2100 MHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot2.osadl.org (updated Mon Aug 08, 2022 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3188099141139,1cyclictest0-21swapper/307:18:253
31412210741,31sleep20-21swapper/207:13:092
220712950,0sleep30-21swapper/309:16:283
3136629344,39sleep10-21swapper/107:12:321
3134129248,32sleep00-21swapper/007:12:150
3138829047,32sleep30-21swapper/307:12:503
291302810,0sleep20-21swapper/208:20:512
184682690,1sleep20-21swapper/212:17:552
31879994441,2cyclictest0-21swapper/207:15:382
31878994133,7cyclictest0-21swapper/107:15:381
80542320,1sleep30-21swapper/310:05:353
39572320,1sleep00-21swapper/012:04:540
272772320,0sleep00-21swapper/010:23:040
180232320,1sleep00-21swapper/012:17:170
71092310,3sleep228-21ksoftirqd/209:33:392
26332310,0sleep20-21swapper/211:00:582
7292300,0chrt0-21swapper/209:57:402
297082300,0sleep30-21swapper/311:26:293
31878992924,3cyclictest5588-21latency_hist11:35:351
22712280,1sleep30-21swapper/311:00:553
301422260,1sleep00-21swapper/010:25:560
71952250,1chrt7194-21df_inode11:05:523
31878992420,3cyclictest31966-21kworker/u8:010:59:211
31878992417,1cyclictest141rcu_preempt10:00:511
31877992422,1cyclictest0-21swapper/007:15:380
102232230,0chrt0-21swapper/309:36:003
31879992018,1cyclictest0-21swapper/207:45:482
3187799206,7cyclictest19509-21cron08:00:340
117452200,0chrt0-21swapper/310:08:153
31880991915,2cyclictest19724-21ssh09:45:373
31880991915,2cyclictest19724-21ssh09:45:373
31880991614,1cyclictest0-21swapper/308:40:373
3188099160,15cyclictest0-21swapper/312:35:503
3187999164,6cyclictest0-21swapper/210:53:482
31879991612,2cyclictest22424-21ssh11:20:502
3187899165,6cyclictest0-21swapper/111:15:461
3187899165,6cyclictest0-21swapper/108:45:351
3187899162,11cyclictest19680-21diskmemload10:49:241
265622160,1sleep00-21swapper/010:53:060
3187899155,5cyclictest17209-21ssh10:13:351
3187899154,6cyclictest3802-21sh11:01:571
3187899154,6cyclictest30532-21cat08:25:351
278782150,1sleep20-21swapper/209:22:202
3188099145,7cyclictest30559-21missed_timers10:26:003
3188099145,6cyclictest8296-21ssh09:35:153
3188099145,4cyclictest0-21swapper/312:40:373
3188099144,5cyclictest0-21swapper/311:57:203
3188099144,5cyclictest0-21swapper/311:45:563
3188099141,8cyclictest13315-21/usr/sbin/munin11:11:013
3187999149,3cyclictest22203-21cpuspeed_turbos08:05:522
3187899144,8cyclictest0-21swapper/109:26:311
3187899144,5cyclictest12950-21ssh10:09:591
3187899144,5cyclictest0-21swapper/109:20:591
3187899141,7cyclictest23816-21latency_hist10:20:351
3187799146,5cyclictest0-21swapper/010:57:480
122572140,1chrt12256-21python07:41:181
3188099137,4cyclictest19518-21df_inode10:15:563
3188099136,5cyclictest16729-21sed12:16:013
3188099134,8cyclictest0-21swapper/309:58:013
3188099132,6cyclictest801-21ssh12:31:393
3188099132,4cyclictest30342-21grep10:56:063
3187999138,3cyclictest7228-21tr08:45:532
3187999136,6cyclictest27601-1kworker/u9:212:25:492
3187999135,3cyclictest0-21swapper/208:35:412
3187999134,6cyclictest1039-21ssh11:00:302
3187999133,8cyclictest0-21swapper/211:55:382
3187999133,5cyclictest0-21swapper/211:33:372
3187999133,5cyclictest0-21swapper/209:45:562
3187999133,5cyclictest0-21swapper/209:45:562
3187999132,6cyclictest0-21swapper/209:17:542
3187999131,9cyclictest0-21swapper/209:39:432
3187999130,6cyclictest14258-21ssh09:40:372
3187899135,6cyclictest0-21swapper/107:35:451
3187899135,3cyclictest0-21swapper/109:13:421
3187899134,8cyclictest0-21swapper/109:02:431
3187899134,8cyclictest0-21swapper/109:00:051
3187899134,4cyclictest31803-21ssh12:30:511
3187899134,4cyclictest11077-21ssh10:38:351
3187899134,4cyclictest0-21swapper/111:07:311
3187899134,4cyclictest0-21swapper/110:31:501
3187899133,8cyclictest0-21swapper/108:07:311
3187899133,8cyclictest0-21swapper/107:45:491
3187899131,2cyclictest0-21swapper/110:55:231
3187799135,7cyclictest0-21swapper/008:01:270
3187799134,8cyclictest0-21swapper/012:39:210
3187799134,4cyclictest0-21swapper/012:10:590
3187799134,4cyclictest0-21swapper/008:50:550
3187799130,7cyclictest238-21systemd-journal12:25:340
3188099128,2cyclictest26739-21ssh09:21:043
3188099127,3cyclictest0-21swapper/308:11:583
3188099126,5cyclictest4229-21kworker/3:212:20:533
3188099126,4cyclictest9304-21uniq07:35:543
3188099126,4cyclictest3151-21dump-pmu-power09:30:343
3188099126,4cyclictest12854-21basename07:45:343
3188099125,5cyclictest538-21dbus-daemon08:25:343
3188099124,7cyclictest0-21swapper/312:26:393
3188099124,7cyclictest0-21swapper/312:13:153
3188099120,10cyclictest0-21swapper/308:35:533
3187999127,3cyclictest6373-21turbostat07:30:342
3187999127,3cyclictest15337-21sh09:05:342
3187999126,4cyclictest18533-21grep11:16:012
3187999125,6cyclictest0-21swapper/212:38:062
3187999125,6cyclictest0-21swapper/208:02:372
3187999125,5cyclictest21228-21cstates12:20:512
3187999124,6cyclictest0-21swapper/211:12:122
3187999124,6cyclictest0-21swapper/210:05:342
3187999124,6cyclictest0-21swapper/208:40:522
3187999124,4cyclictest8395-21ssh10:05:372
3187999124,3cyclictest0-21swapper/210:13:292
3187999124,3cyclictest0-21swapper/208:50:442
3187999124,2cyclictest0-21swapper/208:18:072
3187999123,4cyclictest4668-21ssh10:31:592
3187999123,4cyclictest0-21swapper/210:25:352
3187999122,5cyclictest0-21swapper/211:40:582
3187899127,4cyclictest0-21swapper/109:41:511
3187899127,3cyclictest27295-21sendmail11:25:341
3187899127,3cyclictest27131-21cat08:15:571
3187899127,3cyclictest16011-21grep07:50:521
3187899126,4cyclictest4266-21cat07:25:341
3187899126,4cyclictest26719-21grep09:51:041
3187899125,5cyclictest0-21swapper/108:25:591
3187899124,5cyclictest5299-21expr12:05:531
3187899124,5cyclictest0-21swapper/110:25:451
3187899124,3cyclictest0-21swapper/108:37:381
3187899123,8cyclictest0-21swapper/112:19:051
3187899123,8cyclictest0-21swapper/111:11:211
3187899123,8cyclictest0-21swapper/109:48:411
3187899123,8cyclictest0-21swapper/109:48:411
3187899123,8cyclictest0-21swapper/109:36:151
3187899122,5cyclictest0-21swapper/108:00:531
3187899120,6cyclictest28317-21fschecks_time11:25:551
3187899120,10cyclictest0-21swapper/107:26:001
3187799127,4cyclictest9847-21apt-config12:10:340
3187799126,4cyclictest14470-21uname09:00:540
3187799125,6cyclictest23836-21awk10:20:350
3187799125,5cyclictest0-21swapper/009:05:420
3187799124,5cyclictest0-21swapper/012:30:450
3187799124,3cyclictest0-21swapper/007:52:360
3187799123,4cyclictest0-21swapper/011:41:500
3187799123,4cyclictest0-21swapper/009:30:240
3187799122,5cyclictest0-21swapper/008:21:160
3187799121,2cyclictest0-21swapper/007:26:260
71952110,0sleep30-21swapper/308:45:533
3188099117,2cyclictest23578-21ntp_offset11:20:593
3188099116,3cyclictest29952-21sh09:55:343
3188099115,4cyclictest833-21chrt11:30:503
3188099115,4cyclictest31940-21sort08:25:573
3188099115,4cyclictest18422-21idleruntime09:10:553
3188099114,5cyclictest8853-21ntpq10:36:043
3188099114,5cyclictest26907-21ssh10:53:363
3188099114,5cyclictest0-21swapper/309:00:353
3188099114,5cyclictest0-21swapper/307:30:563
3188099114,5cyclictest0-21swapper/307:23:083
3188099111,8cyclictest17364-21turbostat07:55:343
3188099111,5cyclictest6165-21cut11:35:523
3188099111,5cyclictest11703-21df_inode08:55:503
3188099110,6cyclictest19795-21grep10:46:033
3188099110,5cyclictest7270-21ssh12:07:063
3187999117,3cyclictest791-21ntp_offset12:00:592
3187999117,2cyclictest11220-21tr07:40:532
3187999116,4cyclictest238-21systemd-journal10:43:552
3187999116,3cyclictest6445-21unixbench_multi12:06:032
3187999116,3cyclictest1267-21grep08:30:542
3187999115,4cyclictest8355-21sed10:35:592
3187999115,4cyclictest6894-21grep07:30:482
3187999115,4cyclictest6483-21grep11:35:552
3187999115,4cyclictest31721-21head12:30:512
3187999114,5cyclictest28028-21cpuspeed_turbos11:25:522
3187999113,3cyclictest0-21swapper/212:10:582
3187999113,3cyclictest0-21swapper/210:48:422
3187999113,3cyclictest0-21swapper/209:10:562
3187999112,6cyclictest0-21swapper/212:43:162
3187999110,5cyclictest28153-21ssh09:52:562
3187899115,5cyclictest0-21swapper/112:26:191
3187899115,5cyclictest0-21swapper/109:57:371
3187899115,5cyclictest0-21swapper/108:46:381
3187899115,4cyclictest9476-21cat12:40:531
3187899115,4cyclictest20752-21sh11:49:561
3187899115,4cyclictest18139-21fschecks_time07:55:531
3187899115,4cyclictest0-21swapper/110:42:121
3187899115,4cyclictest0-21swapper/110:20:411
3187899115,3cyclictest22-21ksoftirqd/112:00:371
3187899114,5cyclictest25163-21ntp_kernel_pll_08:11:011
3187899114,5cyclictest0-21swapper/111:38:531
3187899113,5cyclictest0-21swapper/112:36:311
3187899112,8cyclictest0-21swapper/109:17:251
3187899112,8cyclictest0-21swapper/108:32:131
3187899112,6cyclictest0-21swapper/111:40:531
3187899112,1cyclictest0-21swapper/108:53:531
3187899111,8cyclictest590-21snmpd07:33:231
3187899111,5cyclictest19680-21diskmemload11:56:091
3187899111,5cyclictest17536-21awk09:10:341
3187899110,8cyclictest10341-21/usr/sbin/munin12:10:541
3187799115,4cyclictest7587-21sed08:45:560
3187799115,4cyclictest238-21systemd-journal11:03:250
3187799115,4cyclictest18226-21cut09:10:540
3187799115,4cyclictest0-21swapper/011:54:440
3187799115,4cyclictest0-21swapper/011:35:500
3187799114,5cyclictest9633-21ssh09:35:540
3187799114,5cyclictest28887-21ssh09:23:480
3187799112,4cyclictest0-21swapper/007:31:140
3187799111,9cyclictest0-21swapper/011:25:590
3188099106,2cyclictest43750irq/33-enp3s0-r10:33:543
3188099105,4cyclictest20816-21tr08:00:563
3188099105,4cyclictest0-21swapper/310:15:143
3188099105,3cyclictest0-21swapper/309:06:303
3188099104,5cyclictest4229-21kworker/3:207:47:433
3188099104,4cyclictest18486-21uname11:16:003
3188099104,4cyclictest17818-21tr07:55:503
3188099104,4cyclictest15116-21gltestperf09:40:563
3188099103,2cyclictest9222-21awk08:50:503
3188099102,3cyclictest0-21swapper/308:15:563
3188099100,8cyclictest32567-21forks12:00:533
3188099100,8cyclictest21493-21runrttasks11:50:413
3188099100,8cyclictest11042-21ssh11:40:433
3188099100,4cyclictest24368-21cpu10:20:533
3188099100,4cyclictest13232-21tr10:40:563
3187999105,3cyclictest2481-21grep07:20:542
3187999105,3cyclictest24736-21cut08:10:572
3187999105,3cyclictest19680-21diskmemload11:50:512
3187999105,3cyclictest19528-21sh10:15:562
3187999105,3cyclictest0-21swapper/211:05:362
3187999105,3cyclictest0-21swapper/210:21:052
3187999105,3cyclictest0-21swapper/207:37:572
3187999104,5cyclictest0-21swapper/211:45:412
3187999104,4cyclictest17896-21tr07:55:502
3187999104,4cyclictest17129-21head09:06:012
3187999103,5cyclictest590-21snmpd07:51:242
3187999101,7cyclictest0-21swapper/208:25:492
3187999101,3cyclictest32257-21grep09:26:032
3187899104,4cyclictest238-21systemd-journal09:33:441
3187899104,4cyclictest23200-21grep11:51:031
3187899104,4cyclictest21973-21missed_timers12:20:571
3187799107,2cyclictest19680-21diskmemload09:15:510
3187799106,3cyclictest11204-21grep07:40:530
3187799106,2cyclictest8941-21cpu07:35:510
3187799106,2cyclictest7414-21in:imuxsock07:48:260
3187799106,2cyclictest19680-21diskmemload10:10:570
3187799105,4cyclictest26848-21apt11:55:500
3187799105,4cyclictest0-21swapper/011:13:100
3187799105,4cyclictest0-21swapper/010:00:040
3187799105,4cyclictest0-21swapper/008:28:430
3187799105,4cyclictest0-21swapper/008:05:460
3187799105,3cyclictest8875-21open_files10:36:050
3187799105,3cyclictest603-21runrttasks10:00:540
3187799105,3cyclictest0-21swapper/012:41:520
3187799104,5cyclictest0-21swapper/009:31:540
3187799104,4cyclictest23052-21tr11:20:530
3187799104,4cyclictest20239-21sh11:49:130
3187799104,4cyclictest15611-21ntp_offset09:41:010
3187799104,4cyclictest14585-21grep10:41:080
3187799104,4cyclictest0-21swapper/008:17:220
3187799103,5cyclictest0-21swapper/008:35:050
3187799103,4cyclictest0-21swapper/010:32:470
3187799102,3cyclictest0-21swapper/011:06:330
3187799101,7cyclictest19722-21apt-get09:45:370
3187799101,7cyclictest19722-21apt-get09:45:370
3187799101,4cyclictest0-21swapper/009:51:100
3187799101,1cyclictest0-21swapper/012:26:490
318809995,3cyclictest0-21swapper/308:05:433
318809995,2cyclictest5831-21timerandwakeup07:26:103
318809993,4cyclictest817-21uniq08:30:503
318799990,7cyclictest11258-21/usr/sbin/munin08:55:492
318779996,2cyclictest1443-21if_err_enp5s011:30:540
318779995,3cyclictest12-21ksoftirqd/008:55:520
318779994,4cyclictest5577-21cat08:40:580
318779994,4cyclictest25073-21missed_timers08:11:000
318779994,4cyclictest0-21swapper/011:15:370
318779994,4cyclictest0-21swapper/010:45:390
318779993,4cyclictest2657-21sort08:35:490
318779992,5cyclictest0-21swapper/007:20:530
318779990,8cyclictest0-21swapper/010:06:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional