You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-11 - 19:46
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8.osadl.org (updated Wed Sep 11, 2024 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41290312980,1sleep00-21swapper/010:33:400
40446792410,1chrt4044677-21hddtemp_smartct09:40:130
40338492350,1sleep10-21swapper/109:30:201
3846719992623,1cyclictest0-21swapper/311:25:003
3846719992623,1cyclictest0-21swapper/310:25:003
3846717992614,6cyclictest4028505-21idleruntime-cro09:30:002
39028132250,1chrt0-21swapper/007:50:190
3846717992521,3cyclictest500-21avahi-daemon07:25:002
3846709992521,3cyclictest501-21dbus-daemon10:20:000
3846712992421,2cyclictest0-21swapper/111:10:001
3846709992421,2cyclictest508-21polkitd11:15:000
3846717992317,4cyclictest4096020-21tail10:10:192
3846712992315,7cyclictest130117-21sendmail12:40:001
3846712992314,2cyclictest3867405-21latency_hist07:25:001
3846712992312,2cyclictest0-21swapper/110:55:011
3846712992311,1cyclictest0-21swapper/110:25:121
3846717992219,2cyclictest4191765-21latency_hist11:14:592
3846712992211,2cyclictest0-21swapper/110:55:171
3846712992211,2cyclictest0-21swapper/110:45:151
3846719992118,1cyclictest0-21swapper/311:40:003
384671999211,17cyclictest44977-21turbostat11:45:003
3846712992118,2cyclictest3972186-21cron08:50:011
3846712992111,2cyclictest4091204-21cpuspeed_turbos10:10:121
3846709992119,1cyclictest3984765-21sh09:00:010
3846719992017,1cyclictest0-21swapper/308:30:143
3846719992016,2cyclictest114653-21turbostat12:30:003
3846717992018,2cyclictest3846705-21cyclictest08:15:012
3846717992016,3cyclictest4183856-21idleruntime-cro11:10:002
3846712992011,2cyclictest0-21swapper/109:15:001
3846717991916,2cyclictest0-21swapper/212:40:012
3846717991916,2cyclictest0-21swapper/210:05:002
3846717991915,2cyclictest0-21swapper/211:30:002
3846717991914,4cyclictest0-21swapper/209:10:012
3846712991916,2cyclictest7019-21irqrtprio11:20:171
3846712991916,2cyclictest3947506-21cut08:30:011
3846712991915,2cyclictest0-21swapper/112:30:221
3846712991910,3cyclictest4067953-21fschecks_count09:55:141
3846712991910,3cyclictest4067953-21fschecks_count09:55:141
3846709991916,2cyclictest4115755-21sed10:25:170
3846709991916,2cyclictest3852855-21latency_hist07:15:000
3846719991816,1cyclictest0-21swapper/310:20:013
3846719991815,1cyclictest0-21swapper/312:25:003
3846719991815,1cyclictest0-21swapper/311:15:003
3846717991815,2cyclictest4044043-21latency_hist09:39:592
3846717991815,2cyclictest37170-21idleruntime-cro11:40:002
3846717991813,4cyclictest3997100-21diskmemload10:45:192
384671799181,16cyclictest4138084-21irqcore10:40:152
384671799180,6cyclictest91107-21ssh12:14:092
3846712991814,3cyclictest522-21in:imuxsock09:05:011
3846709991816,1cyclictest0-21swapper/011:20:000
3846709991815,2cyclictest89551-21unixbench_multi12:10:200
3846709991815,2cyclictest4114016-21cron10:25:000
3846709991815,2cyclictest13529-21sh11:25:000
3846709991814,2cyclictest3917386-21irqcore08:05:160
3846719991714,1cyclictest0-21swapper/312:37:233
3846719991714,1cyclictest0-21swapper/312:15:153
3846719991714,1cyclictest0-21swapper/310:45:183
384671999170,2cyclictest40444752sleep309:40:133
3846717991714,2cyclictest7929-21kthreadcore11:20:182
3846717991714,2cyclictest522-21in:imuxsock11:20:002
3846717991714,2cyclictest502-21NetworkManager11:47:132
3846717991714,2cyclictest502-21NetworkManager10:18:422
3846717991714,2cyclictest4111658-21ntp_kernel_err10:20:202
3846717991714,2cyclictest3935179-21idleruntime-cro08:20:002
3846717991714,2cyclictest3874111-21irqcore07:30:172
3846717991713,3cyclictest0-21swapper/207:55:002
3846717991713,2cyclictest0-21swapper/209:20:002
3846717991711,4cyclictest0-21swapper/209:20:202
3846712991714,2cyclictest508-21polkitd09:10:011
3846712991714,2cyclictest502-21NetworkManager12:18:421
3846712991711,5cyclictest501-21dbus-daemon07:20:001
3846709991714,2cyclictest4130336-21irqcore10:35:150
3846709991714,2cyclictest4011003-21unixbench-2d09:15:240
3846709991714,2cyclictest3997372-21kworker/u8:0-nfsiod10:40:370
3846709991714,2cyclictest3004-21cron11:30:000
3846719991613,1cyclictest0-21swapper/310:14:393
3846719991613,1cyclictest0-21swapper/309:30:003
3846719991613,1cyclictest0-21swapper/307:50:143
3846719991613,1cyclictest0-21swapper/307:35:223
384671799167,6cyclictest1-21systemd08:05:002
384671799167,6cyclictest1-21systemd08:04:592
384671799164,4cyclictest4065552-21unixbench_multi09:50:242
3846717991613,2cyclictest769-21runrttasks09:00:172
3846717991613,2cyclictest4067732-21/usr/sbin/munin09:55:202
3846717991613,2cyclictest4067732-21/usr/sbin/munin09:55:202
3846717991613,2cyclictest3846705-21cyclictest11:35:002
3846717991613,2cyclictest0-21swapper/209:30:172
3846717991612,3cyclictest53722-21irqcore11:50:152
3846717991612,3cyclictest3855453-21gltestperf07:15:132
3846717991612,3cyclictest104528-21needreboot12:20:172
3846717991612,2cyclictest0-21swapper/209:45:232
3846712991614,1cyclictest3846705-21cyclictest09:35:221
3846712991612,3cyclictest4011026-21unixbench_multi09:15:251
3846712991612,3cyclictest3561480-21systemd-journal11:50:071
3846712991612,3cyclictest2115290-21kworker/u8:0-ext4-rsv-conversion07:45:001
3846709991613,2cyclictest4176743-21tr11:05:150
3846709991613,2cyclictest3990502-21unixbench_multi09:00:240
3846709991613,2cyclictest3936818-21kthreadcore08:20:150
3846709991613,2cyclictest3856087-21irqrtprio07:15:160
3846709991613,2cyclictest23397-21kthreadcore11:30:180
3846709991612,3cyclictest4039467-21kthreadcore09:35:170
3846709991612,3cyclictest3941805-21cut08:25:130
3846709991612,3cyclictest3904899-21grep07:55:140
3846709991612,2cyclictest3934680-21unixbench_multi08:15:250
3846719991512,1cyclictest0-21swapper/312:05:163
3846719991512,1cyclictest0-21swapper/312:00:183
3846719991512,1cyclictest0-21swapper/311:58:303
3846719991512,1cyclictest0-21swapper/311:58:303
3846719991512,1cyclictest0-21swapper/311:33:293
3846719991512,1cyclictest0-21swapper/310:55:223
3846719991512,1cyclictest0-21swapper/309:50:253
3846719991512,1cyclictest0-21swapper/309:35:133
3846719991512,1cyclictest0-21swapper/309:20:133
3846719991512,1cyclictest0-21swapper/309:04:073
3846719991512,1cyclictest0-21swapper/308:35:173
384671999150,4cyclictest41437202sleep310:41:423
384671999150,1cyclictest0-21swapper/308:24:553
384671799159,4cyclictest3872981-21grep07:25:242
384671799158,5cyclictest122363-21cron12:35:002
3846717991512,2cyclictest4123515-21kthreadcore10:30:172
3846717991511,3cyclictest43034-21unixbench-2d11:40:222
3846717991511,3cyclictest3980268-21kthreadcore08:55:172
3846717991511,3cyclictest3969256-21kthreadcore08:45:182
3846717991511,3cyclictest3954321-21idleruntime08:35:162
3846717991511,3cyclictest3892445-21fschecks_count07:45:152
384671299151,2cyclictest0-21swapper/107:50:161
3846712991512,2cyclictest3973699-21irqrtprio08:50:151
3846712991512,2cyclictest0-21swapper/110:30:161
3846712991511,3cyclictest4171248-21kthreadcore11:00:171
3846712991511,3cyclictest3928496-21unixbench-2d08:10:231
384671299150,15cyclictest0-21swapper/111:35:221
3846709991513,1cyclictest99059-21kworker/0:1+kacpid12:36:090
3846709991513,1cyclictest4011953-21kworker/0:2+kacpid10:55:290
3846709991513,1cyclictest4011953-21kworker/0:2+kacpid10:14:210
3846709991513,1cyclictest4011953-21kworker/0:2+kacpid09:58:020
3846709991513,1cyclictest4011953-21kworker/0:2+kacpid09:58:010
3846709991513,1cyclictest3840337-21kworker/0:1+kacpid08:30:110
3846709991513,1cyclictest3840337-21kworker/0:1+kacpid08:00:510
3846709991513,1cyclictest3840337-21kworker/0:1+kacpid08:00:510
3846709991512,2cyclictest4150440-21netstat10:45:210
3846709991512,2cyclictest3965614-21unixbench_singl08:40:280
3846709991512,2cyclictest13303-21kworker/0:0+kacpid12:19:100
3846709991512,2cyclictest0-21swapper/009:30:000
3846709991512,1cyclictest99784-21fschecks_time12:20:120
3846709991510,4cyclictest3885246-21grep07:35:230
774322140,1sleep10-21swapper/112:05:161
3846719991411,1cyclictest0-21swapper/312:30:133
3846719991411,1cyclictest0-21swapper/312:10:153
3846719991411,1cyclictest0-21swapper/311:54:293
3846719991411,1cyclictest0-21swapper/311:45:193
3846719991411,1cyclictest0-21swapper/311:25:193
3846719991411,1cyclictest0-21swapper/311:00:133
3846719991411,1cyclictest0-21swapper/310:00:203
3846719991411,1cyclictest0-21swapper/309:58:183
3846719991411,1cyclictest0-21swapper/309:58:183
3846719991411,1cyclictest0-21swapper/309:47:063
3846719991411,1cyclictest0-21swapper/309:10:163
3846719991411,1cyclictest0-21swapper/308:50:013
3846719991411,1cyclictest0-21swapper/308:10:133
3846719991411,1cyclictest0-21swapper/307:25:203
3846719991410,1cyclictest0-21swapper/307:15:173
384671799147,5cyclictest547-21gdbus08:00:002
384671799147,5cyclictest3561480-21systemd-journal07:40:002
384671799147,5cyclictest114633-21cron12:30:002
3846717991411,2cyclictest79293-21kthreadcore12:05:182
3846717991411,2cyclictest66261-21timerwakeupswit11:55:232
3846717991411,2cyclictest66261-21timerwakeupswit11:55:232
3846717991411,2cyclictest500-21avahi-daemon12:18:422
3846717991411,2cyclictest4161092-21fschecks_count10:55:162
3846717991411,2cyclictest4160602-21latency_hist10:55:012
3846717991411,2cyclictest4119221-21latency10:25:192
3846717991411,2cyclictest4005066-21latency_hist09:15:002
3846717991410,3cyclictest3952838-21proc_pri08:30:222
3846717991410,2cyclictest75976-21munin-run12:04:592
3846717991410,2cyclictest0-21swapper/211:00:182
3846717991410,2cyclictest0-21swapper/208:55:002
3846717991410,2cyclictest0-21swapper/207:40:152
384671299149,3cyclictest4089570-21sshd10:07:071
384671299148,4cyclictest0-21swapper/110:35:141
3846712991411,2cyclictest4102511-21kthreadcore10:15:181
3846712991411,2cyclictest4075615-21cpuspeed_turbos10:00:121
3846712991411,2cyclictest4049892-21unixbench_singl09:40:221
3846712991411,2cyclictest3960494-21gltestperf08:40:141
3846712991411,2cyclictest3825-21unixbench-2d11:15:261
3846712991411,2cyclictest2108677-21kworker/u8:3-events_unbound08:00:001
3846712991411,2cyclictest14948-21irqrtprio11:25:171
3846712991410,3cyclictest4138305-21irqcore10:40:151
3846712991410,3cyclictest4015082-21kthreadcore09:20:171
3846712991410,2cyclictest0-21swapper/109:50:011
384671299140,2cyclictest84174-21df_inode12:10:101
384671299140,2cyclictest3934234-21netstat08:15:211
3846709991412,1cyclictest99059-21kworker/0:1+kacpid12:31:530
3846709991412,1cyclictest99059-21kworker/0:1+kacpid12:26:230
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid10:05:110
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid10:00:090
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid09:51:320
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid09:45:270
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid09:30:010
3846709991412,1cyclictest4011953-21kworker/0:2+kacpid09:21:540
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid09:10:040
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid09:06:100
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid08:51:040
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid08:45:040
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid08:11:010
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid07:47:320
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid07:40:100
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid07:30:070
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid07:27:180
3846709991412,1cyclictest3840337-21kworker/0:1+kacpid07:21:160
3846709991412,1cyclictest13303-21kworker/0:0+kacpid12:09:380
3846709991412,1cyclictest13303-21kworker/0:0+kacpid11:56:490
3846709991412,1cyclictest13303-21kworker/0:0+kacpid11:56:490
3846709991412,1cyclictest13303-21kworker/0:0+kacpid11:46:060
3846709991412,1cyclictest13303-21kworker/0:0+kacpid11:40:350
3846709991412,1cyclictest13303-21kworker/0:0+kacpid11:35:290
3846709991411,2cyclictest69991-21kthreadcore12:00:180
3846709991411,2cyclictest4154263-21irqrtprio10:50:130
3846709991411,2cyclictest4011953-21kworker/0:2+kacpid11:00:010
3846709991411,2cyclictest3840337-21kworker/0:1+kacpid08:37:550
3846709991411,2cyclictest0-21swapper/011:50:150
384671999131,8cyclictest4153094-21taskset10:50:103
3846719991311,1cyclictest0-21swapper/311:15:173
3846719991311,1cyclictest0-21swapper/308:50:173
3846719991311,1cyclictest0-21swapper/308:40:263
3846719991311,1cyclictest0-21swapper/308:05:003
3846719991311,1cyclictest0-21swapper/308:05:003
3846719991311,1cyclictest0-21swapper/307:45:183
3846719991310,1cyclictest0-21swapper/310:29:213
3846719991310,1cyclictest0-21swapper/309:30:183
3846719991310,1cyclictest0-21swapper/309:15:523
3846719991310,1cyclictest0-21swapper/309:05:183
3846719991310,1cyclictest0-21swapper/307:56:523
3846719991310,1cyclictest0-21swapper/307:40:173
3846719991310,1cyclictest0-21swapper/307:30:523
384671799139,3cyclictest3941322-21cron08:25:002
384671799139,2cyclictest0-21swapper/209:40:152
384671799138,4cyclictest3850752-21tr07:10:192
3846717991311,1cyclictest3943242-21kthreadcore08:25:162
3846717991310,2cyclictest4135357-21unixbench-2d10:35:232
3846717991310,2cyclictest3963370-21kthreadcore08:40:182
3846717991310,2cyclictest3808427-21kworker/u8:1-rpciod10:05:142
384671299139,3cyclictest3935538-21cpuspeed_turbos08:20:111
384671299139,3cyclictest3881180-21kthreadcore07:35:171
384671299139,3cyclictest3873741-21cpuspeed_turbos07:30:141
384671299139,3cyclictest3868876-21kthreadcore07:25:171
384671299139,3cyclictest100118-21irqcore12:20:141
384671299139,2cyclictest0-21swapper/108:35:011
384671299138,4cyclictest3997100-21diskmemload11:10:171
384671299138,3cyclictest68476-21ssh12:00:011
384671299134,3cyclictest3916676-21grep08:05:001
384671299134,3cyclictest3916676-21grep08:05:001
3846712991310,2cyclictest60743-21acpi11:55:111
3846712991310,2cyclictest60743-21acpi11:55:101
3846712991310,2cyclictest3921002-21kthreadcore08:05:191
3846712991310,2cyclictest107604-21gltestperf12:25:151
3846712991310,2cyclictest0-21swapper/111:30:131
3846712991310,2cyclictest0-21swapper/109:25:221
384671299130,12cyclictest3895217-21kthreadcore07:45:191
384671999129,1cyclictest0-21swapper/311:05:133
384671999129,1cyclictest0-21swapper/310:35:143
384671999129,1cyclictest0-21swapper/310:35:013
384671999129,1cyclictest0-21swapper/310:06:563
384671999129,1cyclictest0-21swapper/308:25:173
384671999129,1cyclictest0-21swapper/308:05:163
384671999129,1cyclictest0-21swapper/307:20:143
384671799129,2cyclictest0-21swapper/208:05:182
384671299129,2cyclictest45323-21ssh11:45:141
384671299129,2cyclictest4107816-21irqrtprio10:20:161
384671299128,2cyclictest0-21swapper/107:10:251
384671299126,4cyclictest3980830-21tr08:55:171
3846712991211,1cyclictest0-21swapper/111:40:101
384671299120,6cyclictest4067544-21cat09:55:011
384671999110,9cyclictest3929002-21turbostat08:19:543
384671299111,8cyclictest3846586-21latency_hist07:10:011
384671299110,1cyclictest0-21swapper/108:35:161
38451462110,7sleep03845611-21grep07:05:190
384671999108,1cyclictest0-21swapper/308:55:243
384671999100,7cyclictest3846866-21chrt07:10:103
38467179990,6cyclictest3846586-21latency_hist07:10:012
3840709260,2sleep30-21swapper/307:05:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional