You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 10:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackeslot2.osadl.org (updated Sat Apr 27, 2024 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
202023421030,1sleep12020236-21processes23:00:251
17375072910,2sleep00-21swapper/019:40:210
17817432900,2sleep10-21swapper/120:20:031
18689682860,2sleep20-21swapper/221:25:312
1699051994227,7cyclictest0-21swapper/200:30:002
1699051993117,9cyclictest0-21swapper/223:15:102
18082082300,2chrt0-21swapper/220:40:192
169905199272,16cyclictest0-21swapper/222:40:012
2358491260,3ptp4l0-21swapper/319:10:123
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-023:01:243
2358491250,5ptp4l164790irq/127-enp1s0-TxRx-000:31:263
2358491250,4ptp4l164790irq/127-enp1s0-TxRx-022:13:023
2358491250,4ptp4l164790irq/127-enp1s0-TxRx-021:54:273
2358491250,4ptp4l164790irq/127-enp1s0-TxRx-021:54:273
2358491250,3ptp4l0-21swapper/319:05:103
19280322250,2chrt0-21swapper/322:05:163
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-023:20:393
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-021:35:123
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-020:21:363
2358491240,5ptp4l164790irq/127-enp1s0-TxRx-000:20:483
2358491240,4ptp4l164790irq/127-enp1s0-TxRx-023:18:443
188603522420,2sleep20-21swapper/221:37:472
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-023:54:143
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-023:26:333
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-022:47:063
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-022:18:033
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-020:26:413
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-000:28:173
2358491230,5ptp4l164790irq/127-enp1s0-TxRx-000:16:393
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-022:54:103
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-022:30:583
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-021:48:063
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-021:32:063
2358491230,4ptp4l164790irq/127-enp1s0-TxRx-021:11:073
2358491230,3ptp4l0-21swapper/321:25:133
2358491230,3ptp4l0-21swapper/319:20:103
2358491230,2ptp4l0-21swapper/323:45:383
2358491230,12ptp4l164790irq/127-enp1s0-TxRx-022:02:133
217381222319,2sleep20-21swapper/200:37:152
216491322319,2sleep20-21swapper/200:30:342
214878922319,2sleep20-21swapper/200:20:312
214067922319,2sleep20-21swapper/200:15:302
212693622319,2sleep20-21swapper/200:09:532
211705022319,2sleep20-21swapper/200:01:202
208537522319,2sleep20-21swapper/223:41:562
206934222319,2sleep20-21swapper/223:31:542
203753922319,2sleep20-21swapper/223:12:102
200528122319,2sleep20-21swapper/222:51:342
197475122319,2sleep20-21swapper/222:34:142
196534022319,2sleep20-21swapper/222:26:302
195220222319,2sleep20-21swapper/222:20:152
194095122319,2sleep20-21swapper/222:11:052
193275022319,2sleep20-21swapper/222:05:362
191710722319,2sleep20-21swapper/221:56:102
190811622319,2sleep20-21swapper/221:50:252
190811622319,2sleep20-21swapper/221:50:252
190301922319,2sleep20-21swapper/221:49:452
186021022319,2sleep20-21swapper/221:20:232
185309622319,2sleep20-21swapper/221:15:502
184513822319,2sleep20-21swapper/221:10:492
174053522319,2sleep20-21swapper/219:40:282
169881622318,3sleep20-21swapper/219:09:222
2358491220,6ptp4l164790irq/127-enp1s0-TxRx-022:27:293
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-021:42:253
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-021:16:093
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-000:37:243
2358491220,4ptp4l164790irq/127-enp1s0-TxRx-000:07:053
206069722218,2sleep20-21swapper/223:25:402
205388822218,2sleep20-21swapper/223:22:562
202988022218,2sleep20-21swapper/223:07:452
183331922219,2sleep20-21swapper/221:01:162
182149422219,2sleep20-21swapper/220:50:582
179870222219,2sleep20-21swapper/220:33:462
179273922219,2sleep20-21swapper/220:25:462
178125222219,2sleep20-21swapper/220:16:112
177546722219,2sleep20-21swapper/220:10:422
176991722219,2sleep20-21swapper/220:08:132
176410422219,2sleep20-21swapper/220:01:482
173539322219,2sleep20-21swapper/219:37:032
2358491210,6ptp4l164790irq/127-enp1s0-TxRx-020:30:103
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-023:41:363
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-023:32:193
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-022:22:043
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-021:23:173
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-020:45:013
2358491210,4ptp4l164790irq/127-enp1s0-TxRx-019:45:103
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-023:55:233
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-023:11:133
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-021:58:393
2358491210,3ptp4l164790irq/127-enp1s0-TxRx-021:00:563
2358491210,2ptp4l0-21swapper/320:35:323
2358491210,11ptp4l164790irq/127-enp1s0-TxRx-023:35:123
23582912115,2phc2sys0-21swapper/320:08:113
23582912115,2phc2sys0-21swapper/319:31:213
23582912115,2phc2sys0-21swapper/319:28:153
178715722118,2sleep20-21swapper/220:22:522
175271722118,2sleep20-21swapper/219:52:502
174681122118,2sleep20-21swapper/219:45:492
172977822118,2sleep20-21swapper/219:33:472
172395922118,2sleep20-21swapper/219:27:182
171830122118,2sleep20-21swapper/219:23:292
170458822118,2sleep20-21swapper/219:13:162
1699050992119,1cyclictest2143445-21cat00:20:021
1699050992115,5cyclictest2083968-21grep23:40:231
1699049992118,2cyclictest2007083-21idleruntime-cro22:55:000
2358491200,5ptp4l164790irq/127-enp1s0-TxRx-020:15:553
2358491200,4ptp4l164790irq/127-enp1s0-TxRx-021:07:303
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-023:08:433
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-022:56:143
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-022:36:353
2358491200,3ptp4l164790irq/127-enp1s0-TxRx-019:50:163
2358491200,2ptp4l0-21swapper/322:42:443
2358491200,2ptp4l0-21swapper/300:01:003
169905099208,3cyclictest2163415-21kthreadcore00:30:221
1699050992017,2cyclictest1992841-21kthreadcore22:45:181
1699050992016,3cyclictest1705262-21kthreadcore19:15:021
169904999202,7cyclictest1170104-21kworker/u8:2+nfsiod21:26:370
1699049992017,2cyclictest1943294-21systemd-run22:15:010
1699049992017,2cyclictest1804652-21sh20:40:010
1699049992017,2cyclictest1485-21polkitd21:45:030
1699049992017,2cyclictest1454-1atopacctd00:30:010
2358491190,4ptp4l164790irq/127-enp1s0-TxRx-020:50:073
2358491190,4ptp4l164790irq/127-enp1s0-TxRx-019:42:143
2358491190,3ptp4l541ktimers/319:15:123
2358491190,3ptp4l164790irq/127-enp1s0-TxRx-020:10:213
2358491190,3ptp4l164790irq/127-enp1s0-TxRx-019:55:133
2358491190,3ptp4l164790irq/127-enp1s0-TxRx-000:11:563
23582911917,1phc2sys0-21swapper/320:00:203
1699050991913,5cyclictest2032890-21kthreadcore23:10:201
1699050991912,3cyclictest34-21ksoftirqd/100:00:001
1699050991911,4cyclictest191rcu_preempt23:30:151
1699050991911,1cyclictest191rcu_preempt21:23:111
1699049991916,2cyclictest1810324-21wc20:45:000
2358491180,4ptp4l164790irq/127-enp1s0-TxRx-020:55:573
2358491180,2ptp4l0-21swapper/319:36:253
1699051991815,1cyclictest0-21swapper/223:50:012
1699051991814,2cyclictest0-21swapper/223:35:092
169905099185,8cyclictest2152916-21kthreadcore00:25:211
169905099185,3cyclictest0-21swapper/100:05:021
1699050991812,5cyclictest2009792-21kthreadcore22:55:211
1699050991811,3cyclictest34-21ksoftirqd/100:10:001
1699049991815,2cyclictest25776-21rtkit-daemon22:36:560
1699049991815,2cyclictest1833646-21latency_hist21:05:000
2358491170,2ptp4l0-21swapper/320:45:093
1699051991714,1cyclictest0-21swapper/222:45:002
1699051991713,2cyclictest0-21swapper/200:15:022
169905199171,12cyclictest0-21swapper/219:20:012
169905099179,1cyclictest191rcu_preempt22:38:431
169905099178,2cyclictest191rcu_preempt20:25:131
169905099178,2cyclictest191rcu_preempt19:50:101
169905099177,4cyclictest254-21systemd-journal22:25:001
169905099177,3cyclictest1882130-21kthreadcore21:35:211
169905099176,1cyclictest471rcuog/222:33:511
169905099175,7cyclictest1866397-21kthreadcore21:25:191
169905099174,4cyclictest2143826-21cpuspeed00:20:161
1699050991714,2cyclictest1943621-21/usr/sbin/munin22:15:191
1699050991714,2cyclictest1837025-21kthreadcore21:05:221
1699050991714,2cyclictest1465-21gmain21:05:011
1699050991713,3cyclictest1769295-21mailstats20:05:251
1699050991713,2cyclictest191rcu_preempt21:45:131
1699050991713,1cyclictest191rcu_preempt21:57:141
1699050991711,5cyclictest2072509-21irqrtprio23:35:171
1699050991711,2cyclictest191rcu_preempt20:50:191
1699050991711,1cyclictest191rcu_preempt23:28:221
1699050991710,1cyclictest191rcu_preempt22:44:211
1699050991710,1cyclictest191rcu_preempt22:12:041
169905099170,4cyclictest2094953-21munin-run23:50:001
169904999176,3cyclictest1704828-21kworker/u8:3-nfsiod00:12:100
1699049991714,2cyclictest1882309-21sed21:35:210
1699049991714,2cyclictest1502-21systemd-logind22:50:010
1699049991714,2cyclictest0-21swapper/019:25:000
1699049991713,3cyclictest1704828-21kworker/u8:3+nfsiod22:07:570
169904999170,14cyclictest1705192-21munin-run19:15:010
169905199161,8cyclictest0-21swapper/222:50:012
1699051991614,1cyclictest0-21swapper/222:55:202
1699051991613,1cyclictest0-21swapper/222:04:592
1699051991613,1cyclictest0-21swapper/221:10:002
169905099169,1cyclictest191rcu_preempt22:05:391
169905099169,1cyclictest191rcu_preempt21:10:201
169905099168,3cyclictest34-21ksoftirqd/121:00:011
169905099168,1cyclictest191rcu_preempt23:08:001
169905099168,1cyclictest191rcu_preempt00:37:551
169905099167,2cyclictest191rcu_preempt21:30:201
169905099165,6cyclictest191rcu_preempt21:50:231
169905099165,6cyclictest191rcu_preempt21:50:221
169905099165,3cyclictest0-21swapper/123:15:101
169905099164,6cyclictest0-21swapper/119:45:171
1699050991613,2cyclictest1502-21systemd-logind19:35:001
1699050991613,2cyclictest0-21swapper/119:20:001
1699050991612,3cyclictest1454-1atopacctd22:00:191
1699050991611,4cyclictest254-21systemd-journal20:40:011
1699050991611,1cyclictest191rcu_preempt23:20:171
1699050991611,1cyclictest191rcu_preempt21:41:421
1699050991610,2cyclictest191rcu_preempt20:30:101
169904999169,6cyclictest2124514-21unixbench-2d00:05:250
169904999161,3cyclictest1464-21dbus-daemon20:30:000
1699049991613,2cyclictest25781-21sendmail-mta22:41:140
1699049991613,2cyclictest25776-21rtkit-daemon21:48:210
1699049991613,2cyclictest2160241-21gltestperf00:30:150
1699049991613,2cyclictest2110889-40updatedb.mlocat00:00:000
1699049991613,2cyclictest2094914-21cron23:50:000
1699049991613,2cyclictest2074727-21kthreadcore23:35:200
1699049991613,2cyclictest2066490-21kthreadcore23:30:190
1699049991613,2cyclictest2015990-21cut23:00:150
1699049991613,2cyclictest1920346-21irqcore22:00:180
1699049991613,2cyclictest171ktimers/020:15:150
1699049991612,3cyclictest1822408-21kworker/u8:1+nfsiod22:16:220
1699049991610,5cyclictest2044121-21pmu-power23:15:240
1699049991610,2cyclictest191rcu_preempt00:16:520
169905199158,4cyclictest0-21swapper/220:55:132
1699051991513,1cyclictest0-21swapper/221:30:202
1699051991512,1cyclictest0-21swapper/223:59:292
1699051991511,2cyclictest0-21swapper/223:00:352
1699051991511,2cyclictest0-21swapper/221:40:202
1699051991510,3cyclictest1758615-21turbostat20:00:002
169905099159,2cyclictest191rcu_preempt20:40:191
169905099158,3cyclictest191rcu_preempt22:50:271
169905099157,1cyclictest191rcu_preempt20:45:111
169905099157,1cyclictest191rcu_preempt19:55:211
169905099156,4cyclictest1770881-21fschecks_time20:10:121
169905099155,2cyclictest191rcu_preempt19:20:181
169905099154,6cyclictest1849572-21kthreadcore21:15:331
169905099154,6cyclictest1782009-21cpuspeed_turbos20:20:111
169905099153,6cyclictest1723476-21python319:25:261
1699050991512,2cyclictest1963121-21kthreadcore22:25:191
1699050991512,2cyclictest1730366-21tr19:35:101
1699050991512,2cyclictest1454-1atopacctd23:50:041
169905099150,5cyclictest1-21systemd00:15:021
169904999158,1cyclictest581rcuop/321:34:030
169904999155,3cyclictest0-21swapper/020:55:190
169904999152,8cyclictest1967329-21kworker/u8:2+rpciod22:57:500
1699049991512,2cyclictest2169390-21kthreadcore00:35:190
1699049991512,2cyclictest2151525-21latency_hist00:25:010
1699049991512,2cyclictest1919161-21systemd-run22:00:000
1699049991512,2cyclictest1821937-21apt-key20:55:010
1699049991512,2cyclictest1797478-21kthreadcore20:30:190
1699049991512,2cyclictest1772138-21kthreadcore20:10:160
1699049991512,2cyclictest1731356-21kthreadcore19:35:180
1699049991511,3cyclictest1927155-21kworker/u8:0+nfsiod23:29:160
1699049991511,3cyclictest1927155-21kworker/u8:0+nfsiod23:14:240
1699049991511,3cyclictest1927155-21kworker/u8:0+nfsiod22:31:290
1699049991511,3cyclictest1822408-21kworker/u8:1+nfsiod21:16:280
1699049991511,3cyclictest1822408-21kworker/u8:1+nfsiod21:10:560
1699049991511,3cyclictest1170104-21kworker/u8:2+nfsiod21:20:080
1699049991511,2cyclictest1699153-21latency_hist19:10:010
1699049991511,2cyclictest0-21swapper/000:05:010
169905199146,1cyclictest0-21swapper/220:45:122
1699051991412,1cyclictest0-21swapper/222:18:162
1699051991410,2cyclictest0-21swapper/220:35:232
169905099146,3cyclictest1760087-21kthreadcore20:00:171
169905099145,7cyclictest321rcuc/119:10:011
1699050991411,2cyclictest1736466-21fschecks_time19:40:161
169904999147,1cyclictest581rcuop/320:06:380
1699049991411,2cyclictest1960752-21kernelversion22:25:170
1699049991411,2cyclictest1952664-21irqrtprio22:20:170
1699049991411,2cyclictest1782399-21fschecks_time20:20:130
1699049991411,2cyclictest1759432-21fschecks_time20:00:130
1699049991411,2cyclictest1709003-21kthreadcore19:15:160
1699049991411,2cyclictest0-21swapper/023:05:100
1699051991311,1cyclictest0-21swapper/223:50:032
169904999134,3cyclictest0-21swapper/021:05:170
169904999134,3cyclictest0-21swapper/019:30:190
1699049991310,2cyclictest2095905-21gltestperf23:50:150
1699049991310,2cyclictest2047611-21cpuspeed_turbos23:20:150
1699049991310,2cyclictest1814008-21kthreadcore20:45:200
1699049991310,2cyclictest1746535-21timerandwakeup19:45:250
1699049991310,2cyclictest1559-21gdbus20:00:020
1699049991310,2cyclictest0-21swapper/023:45:020
1699049991310,2cyclictest0-21swapper/021:55:010
1699049991310,2cyclictest0-21swapper/021:55:000
169904999123,8cyclictest1718939-21grep19:25:110
169904999115,4cyclictest1748760-21kthreadcore19:50:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional