You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-05-30 - 11:13

x86 Intel Core i3-2310E @2100 MHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot2.osadl.org (updated Tue May 30, 2023 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1088699119110,8cyclictest0-21swapper/119:25:081
1088799114112,1cyclictest0-21swapper/219:25:082
8658210945,49sleep20-21swapper/219:05:082
10560210745,32sleep00-21swapper/019:09:440
108869910091,8cyclictest0-21swapper/119:10:171
10887999694,1cyclictest0-21swapper/219:10:172
1050629349,32sleep30-21swapper/319:09:023
1030829048,31sleep10-21swapper/119:06:311
276912850,0sleep20-21swapper/223:15:412
295962830,0sleep20-21swapper/222:45:352
135782830,0sleep30-21swapper/323:35:303
10888998277,2cyclictest17803-21apt-get19:25:083
10888996258,2cyclictest10919-21apt-get19:10:173
10885995047,2cyclictest0-21swapper/019:25:080
10886994034,1cyclictest141rcu_preempt22:40:251
10886993110,20cyclictest0-21swapper/122:15:241
231922300,0sleep00-21swapper/022:05:530
10885993028,1cyclictest0-21swapper/019:10:170
289352240,1sleep30-21swapper/323:17:223
10885992416,4cyclictest7561-21ntp_states21:50:420
10885992217,3cyclictest29446-21ssh23:18:100
1088599205,8cyclictest11412-21perf20:25:060
10887991816,1cyclictest0-21swapper/220:25:242
1088599183,8cyclictest7057-21munin-run20:15:060
1088799164,9cyclictest17106-21rm00:11:112
1088799161,5cyclictest141rcu_preempt21:54:322
1088699166,1cyclictest22-21ksoftirqd/121:58:501
1088699162,10cyclictest141rcu_preempt22:48:281
1088799157,3cyclictest0-21swapper/200:21:212
1088699156,1cyclictest22-21ksoftirqd/123:39:481
1088699156,1cyclictest22-21ksoftirqd/123:18:551
1088699155,4cyclictest22-21ksoftirqd/100:10:221
1088699153,3cyclictest0-21swapper/123:40:271
1088699151,3cyclictest141rcu_preempt21:00:311
10886991510,3cyclictest22-21ksoftirqd/122:40:061
1088599153,6cyclictest18733-21basename21:30:050
1088599150,14cyclictest0-21swapper/022:50:250
1088899148,4cyclictest31691-21/usr/sbin/munin23:20:313
1088899148,4cyclictest24463-21grep23:45:293
1088899144,5cyclictest0-21swapper/322:40:323
1088899144,5cyclictest0-21swapper/319:42:523
1088799148,1cyclictest28-21ksoftirqd/223:46:062
1088799145,4cyclictest0-21swapper/220:40:332
1088799145,4cyclictest0-21swapper/220:40:332
1088799143,4cyclictest28-21ksoftirqd/200:39:322
1088699149,3cyclictest11889-21grep20:25:251
1088699146,1cyclictest22-21ksoftirqd/122:00:531
1088699145,8cyclictest22-21ksoftirqd/122:53:041
1088699145,8cyclictest22-21ksoftirqd/122:12:511
1088699145,8cyclictest22-21ksoftirqd/100:30:531
1088699145,2cyclictest22-21ksoftirqd/123:53:431
1088699145,1cyclictest22-21ksoftirqd/122:57:331
1088699145,1cyclictest22-21ksoftirqd/121:40:401
1088699145,1cyclictest22-21ksoftirqd/100:07:101
1088699144,8cyclictest22-21ksoftirqd/100:25:071
1088699143,9cyclictest22-21ksoftirqd/121:20:051
1088699140,8cyclictest141rcu_preempt20:31:171
1088599146,3cyclictest14460-21kworker/0:100:40:060
1088599145,8cyclictest0-21swapper/021:44:020
1088599144,8cyclictest0-21swapper/022:31:340
1088599140,9cyclictest14144-21ssh21:57:260
1088899137,5cyclictest0-21swapper/323:12:523
1088899136,2cyclictest34-21ksoftirqd/323:40:313
1088899133,5cyclictest0-21swapper/322:31:483
1088899133,5cyclictest0-21swapper/321:17:333
1088899130,7cyclictest20447-21ssh22:04:203
1088799138,3cyclictest22985-21cron23:45:062
1088799137,1cyclictest28-21ksoftirqd/223:05:332
1088799135,1cyclictest28-21ksoftirqd/221:45:212
1088799134,5cyclictest141rcu_preempt19:20:372
1088799134,4cyclictest0-21swapper/222:58:522
1088699137,4cyclictest4528-21head22:20:351
1088699137,4cyclictest4528-21head22:20:351
1088699137,1cyclictest22-21ksoftirqd/119:30:271
1088699136,6cyclictest22-21ksoftirqd/123:12:051
1088699136,1cyclictest22-21ksoftirqd/123:05:361
1088699136,1cyclictest22-21ksoftirqd/121:35:241
1088699136,1cyclictest22-21ksoftirqd/121:25:421
1088699136,1cyclictest22-21ksoftirqd/121:20:351
1088699136,1cyclictest22-21ksoftirqd/100:35:081
1088699135,7cyclictest22-21ksoftirqd/123:56:551
1088699135,7cyclictest22-21ksoftirqd/123:56:551
1088699135,3cyclictest0-21swapper/120:02:021
1088699135,2cyclictest22-21ksoftirqd/123:45:271
1088699135,2cyclictest22-21ksoftirqd/121:52:411
1088699135,2cyclictest22-21ksoftirqd/121:00:061
1088699135,2cyclictest22-21ksoftirqd/100:16:371
1088699135,2cyclictest22-21ksoftirqd/100:00:461
1088699135,1cyclictest22-21ksoftirqd/123:25:561
1088699135,1cyclictest22-21ksoftirqd/123:01:111
1088699134,8cyclictest22-21ksoftirqd/121:32:251
1088699134,7cyclictest22-21ksoftirqd/123:20:311
1088699134,3cyclictest22-21ksoftirqd/119:20:061
1088699134,1cyclictest22-21ksoftirqd/123:30:191
1088699133,2cyclictest141rcu_preempt21:10:311
1088699131,2cyclictest141rcu_preempt22:05:251
1088599135,8cyclictest0-21swapper/023:08:120
1088599134,4cyclictest4951-21memory21:15:340
1088599133,7cyclictest12248-21ssh00:05:460
1088599131,9cyclictest4969-21sh21:48:400
1088599131,4cyclictest25086-21ssh23:45:380
299502120,4sleep228-21ksoftirqd/222:13:412
1088899127,3cyclictest31990-21grep21:10:313
1088899126,4cyclictest29402-21cut19:50:293
1088899125,5cyclictest0-21swapper/319:18:433
1088899124,6cyclictest24916-21ssh22:08:323
1088899123,8cyclictest0-21swapper/321:52:553
1088899123,5cyclictest5579-21df_inode20:10:263
1088899122,5cyclictest0-21swapper/321:40:313
1088899120,2cyclictest18117-21/usr/sbin/munin20:40:263
1088899120,2cyclictest18117-21/usr/sbin/munin20:40:263
1088799128,3cyclictest28-21ksoftirqd/223:40:062
1088799128,2cyclictest4153-21cpuspeed_turbos21:15:262
1088799128,2cyclictest10128-21grep23:30:402
1088799127,3cyclictest31071-21acpi23:20:252
1088799127,3cyclictest26493-21cron21:00:062
1088799127,3cyclictest24238-21sh21:35:262
1088799127,2cyclictest28-21ksoftirqd/222:30:512
1088799126,4cyclictest5586-21perf22:55:062
1088799126,4cyclictest17855-21cron20:40:062
1088799126,2cyclictest28-21ksoftirqd/220:10:322
1088799125,5cyclictest579-21snmpd23:02:342
1088799125,1cyclictest28-21ksoftirqd/221:58:482
1088799125,1cyclictest28-21ksoftirqd/200:25:272
1088799124,3cyclictest0-21swapper/220:53:212
1088799123,6cyclictest0-21swapper/221:12:542
1088799123,4cyclictest0-21swapper/220:48:182
1088799122,7cyclictest75-21kswapd000:30:262
1088799122,6cyclictest28-21ksoftirqd/223:55:002
1088799122,2cyclictest141rcu_preempt22:40:262
1088799121,5cyclictest141rcu_preempt19:45:382
1088799121,10cyclictest0-21swapper/222:05:062
1088799120,5cyclictest2740-21latency_hist20:05:062
1088699126,4cyclictest21577-21awk20:45:381
1088699126,4cyclictest16671-21awk20:35:301
1088699125,1cyclictest22-21ksoftirqd/122:30:161
1088699125,1cyclictest22-21ksoftirqd/122:27:591
1088699125,1cyclictest22-21ksoftirqd/121:45:281
1088699125,1cyclictest22-21ksoftirqd/120:10:301
1088699125,1cyclictest22-21ksoftirqd/100:20:071
1088699124,6cyclictest22-21ksoftirqd/119:20:321
1088699124,2cyclictest22-21ksoftirqd/121:05:301
1088699124,2cyclictest22-21ksoftirqd/119:35:311
1088699123,4cyclictest0-21swapper/120:52:241
1088599125,5cyclictest26635-21ssh22:42:560
1088599124,6cyclictest16545-21smartctl19:20:340
1088599124,5cyclictest0-21swapper/023:30:420
1088599124,5cyclictest0-21swapper/023:12:380
1088599124,5cyclictest0-21swapper/023:00:460
1088599124,5cyclictest0-21swapper/021:33:420
1088599124,5cyclictest0-21swapper/021:09:360
1088599124,5cyclictest0-21swapper/021:01:560
1088599124,5cyclictest0-21swapper/020:41:460
1088599124,5cyclictest0-21swapper/020:41:460
1088599124,5cyclictest0-21swapper/019:36:480
1088599124,5cyclictest0-21swapper/000:22:040
1088599124,3cyclictest0-21swapper/021:35:160
1088599124,2cyclictest0-21swapper/020:47:500
1088599124,2cyclictest0-21swapper/020:37:180
1088599124,2cyclictest0-21swapper/019:32:260
1088599123,7cyclictest23010-21awk23:45:060
1088599122,6cyclictest26527-21basename21:00:060
1088599122,5cyclictest9207-21sh20:20:060
1088599122,5cyclictest32425-21sh23:54:160
1088599121,6cyclictest21879-21ssh00:16:280
230972110,1sleep20-21swapper/200:18:432
146422110,1sleep00-21swapper/023:35:470
1088899116,4cyclictest233-21systemd-journal22:46:283
1088899116,3cyclictest1607-21grep22:50:303
1088899116,2cyclictest0-21swapper/322:36:543
1088899115,4cyclictest14225-21df20:30:243
1088899114,5cyclictest22202-21perf19:35:063
1088899114,5cyclictest0-21swapper/321:28:463
1088899114,5cyclictest0-21swapper/321:09:153
1088899114,4cyclictest0-21swapper/322:57:043
1088899113,7cyclictest0-21swapper/300:03:413
1088899113,6cyclictest25009-21cat21:35:313
1088899112,1cyclictest0-21swapper/323:27:013
1088899111,8cyclictest16108-21gdbus22:00:063
1088899111,5cyclictest0-21swapper/300:12:273
1088899111,2cyclictest141rcu_preempt21:00:233
1088899110,10cyclictest0-21swapper/323:54:363
1088799118,2cyclictest15910-21ssh21:25:532
1088799118,2cyclictest0-21swapper/222:07:172
1088799116,3cyclictest29391-21uniq21:05:292
1088799116,3cyclictest22701-21grep19:35:262
1088799116,3cyclictest14829-21sh00:10:062
1088799115,1cyclictest28-21ksoftirqd/223:26:052
1088799114,6cyclictest0-21swapper/222:22:322
1088799114,6cyclictest0-21swapper/222:22:322
1088799114,5cyclictest516-21dbus-daemon19:30:372
1088799114,5cyclictest14944-21grep20:30:332
1088799114,1cyclictest28-21ksoftirqd/221:24:532
1088799111,6cyclictest9198-21cron20:20:062
1088799111,5cyclictest20047-21grep21:30:302
1088799111,4cyclictest141rcu_preempt22:28:142
1088699115,4cyclictest3056-21cut20:05:251
1088699115,1cyclictest22-21ksoftirqd/120:40:251
1088699115,1cyclictest22-21ksoftirqd/120:40:251
1088699115,1cyclictest22-21ksoftirqd/119:55:321
1088699115,1cyclictest22-21ksoftirqd/119:50:311
1088699115,1cyclictest22-21ksoftirqd/119:45:261
1088699114,1cyclictest22-21ksoftirqd/120:15:341
1088599115,5cyclictest0-21swapper/022:25:500
1088599114,5cyclictest588-21ssh21:10:440
1088599114,5cyclictest30025-21awk19:50:350
1088599114,5cyclictest233-21systemd-journal20:55:060
1088599114,5cyclictest1226-21grep23:55:250
1088599114,5cyclictest1226-21grep23:55:250
1088599114,4cyclictest0-21swapper/022:20:300
1088599114,4cyclictest0-21swapper/022:20:300
1088599114,4cyclictest0-21swapper/021:20:240
1088599114,4cyclictest0-21swapper/020:27:180
1088599114,4cyclictest0-21swapper/019:56:480
1088599114,2cyclictest0-21swapper/023:24:540
1088599113,7cyclictest0-21swapper/000:04:020
1088599113,5cyclictest0-21swapper/020:31:360
1088599113,5cyclictest0-21swapper/000:32:160
1088599112,8cyclictest22071-21ssh22:38:320
1088599112,4cyclictest7039-21memory22:55:320
1088599112,4cyclictest0-21swapper/000:28:300
1088599111,6cyclictest30837-21diskmemload22:18:160
1088599111,5cyclictest25055-21/usr/sbin/munin19:40:280
1088599110,7cyclictest27654-21ps22:10:380
1088599110,6cyclictest31189-21ssh22:47:210
1088899106,3cyclictest8774-21acpi22:25:243
1088899106,2cyclictest20224-21needreboot21:30:313
1088899105,4cyclictest0-21swapper/322:12:443
1088899105,4cyclictest0-21swapper/321:21:043
1088899105,4cyclictest0-21swapper/320:03:263
1088899105,4cyclictest0-21swapper/319:47:103
1088899105,3cyclictest11713-21head23:00:273
1088899104,5cyclictest0-21swapper/323:59:023
1088899104,5cyclictest0-21swapper/323:59:023
1088899104,5cyclictest0-21swapper/323:33:383
1088899104,5cyclictest0-21swapper/322:20:443
1088899104,5cyclictest0-21swapper/322:20:443
1088899104,5cyclictest0-21swapper/322:16:473
1088899104,5cyclictest0-21swapper/319:56:353
1088899104,5cyclictest0-21swapper/319:22:043
1088899104,5cyclictest0-21swapper/300:31:333
1088899104,4cyclictest5716-21cpu00:35:263
1088899104,4cyclictest3117-21tr20:05:253
1088899104,4cyclictest0-21swapper/323:05:103
1088899104,4cyclictest0-21swapper/319:37:043
1088899103,5cyclictest0-21swapper/300:21:333
1088899102,7cyclictest0-21swapper/320:50:513
1088899102,3cyclictest34-21ksoftirqd/300:25:083
1088899101,7cyclictest12090-21grep00:05:363
1088899101,4cyclictest0-21swapper/320:20:533
1088899101,2cyclictest20933-21ntp_states00:15:363
1088899100,9cyclictest0-21swapper/320:55:263
1088799107,1cyclictest9371-21ssh00:03:482
1088799106,2cyclictest30837-21diskmemload22:15:382
1088799105,3cyclictest28276-21head21:00:382
1088799105,3cyclictest13733-21aten2_r8power_c19:15:232
1088799105,3cyclictest0-21swapper/220:07:592
1088799104,4cyclictest1305-21tr23:55:252
1088799104,4cyclictest1305-21tr23:55:252
1088799103,5cyclictest579-21snmpd23:10:102
1088799103,5cyclictest579-21snmpd20:23:322
1088799102,2cyclictest141rcu_preempt22:35:332
1088799102,1cyclictest141rcu_preempt19:55:262
1088799101,3cyclictest28-21ksoftirqd/221:43:442
1088699104,4cyclictest0-21swapper/119:43:461
1088699104,1cyclictest22-21ksoftirqd/120:20:331
1088599105,4cyclictest0-21swapper/022:01:170
1088599105,3cyclictest0-21swapper/019:45:280
1088599104,5cyclictest0-21swapper/020:05:100
1088599104,5cyclictest0-21swapper/019:15:240
1088599104,5cyclictest0-21swapper/000:10:160
1088599104,4cyclictest0-21swapper/020:00:080
1088599102,2cyclictest0-21swapper/023:25:440
108889996,2cyclictest20969-21if_eno120:45:303
108889995,3cyclictest0-21swapper/320:35:233
108889995,2cyclictest7748-21cat20:15:303
108889991,2cyclictest0-21swapper/321:45:303
108879994,3cyclictest24893-21tr19:40:262
108879990,8cyclictest28-21ksoftirqd/219:50:252
108889984,2cyclictest11895-21cpuspeed_turbos20:25:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional