You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot6.osadl.org (updated Sat Jun 28, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2137321590,4sleep1309499cyclictest13:46:521
151891910,3ptp4l401ktimersoftd/309:52:503
30322760,4sleep2309899cyclictest13:23:122
323862700,5sleep132388-21phc2sys-jitter14:38:151
323862700,5sleep132388-21phc2sys-jitter14:38:151
151891670,1ptp4l401ktimersoftd/312:18:513
26952640,2sleep20-21swapper/214:42:342
26952640,2sleep20-21swapper/214:42:342
151891620,1ptp4l401ktimersoftd/312:32:013
151891600,1ptp4l401ktimersoftd/313:31:363
151891590,1ptp4l401ktimersoftd/314:11:193
104312590,4sleep3310799cyclictest14:49:133
151891580,1ptp4l401ktimersoftd/315:01:593
151891580,1ptp4l401ktimersoftd/314:31:063
151891580,1ptp4l401ktimersoftd/313:20:173
151891580,1ptp4l401ktimersoftd/310:20:463
3094995447,4cyclictest25-21ksoftirqd/113:33:011
3094995346,4cyclictest25-21ksoftirqd/110:47:571
151891510,1ptp4l401ktimersoftd/311:55:103
3094994940,4cyclictest25-21ksoftirqd/113:17:591
151891480,1ptp4l401ktimersoftd/312:23:573
3094994739,4cyclictest25-21ksoftirqd/111:47:591
3094994537,4cyclictest25-21ksoftirqd/111:27:581
151891430,8ptp4l401ktimersoftd/312:09:153
309499403,8cyclictest25-21ksoftirqd/112:38:021
309899392,34cyclictest6843-21sed10:03:002
3098993915,21cyclictest8016-21/usr/sbin/munin14:48:132
20682393,22sleep12171-21ptp4l-jitter09:53:041
3098993821,14cyclictest33-21ksoftirqd/210:12:182
3098993815,20cyclictest19876-21sed13:43:202
3098993811,4cyclictest22149-21expr13:08:022
3094993826,9cyclictest25-21ksoftirqd/113:03:031
298423816,11sleep09-21ksoftirqd/009:57:400
151891380,1ptp4l401ktimersoftd/312:38:133
151891380,1ptp4l401ktimersoftd/312:13:473
151891380,15ptp4l401ktimersoftd/314:07:313
3094993726,5cyclictest25-21ksoftirqd/110:07:581
151891370,2ptp4l20995-21kworker/3:112:43:563
151891370,1ptp4l401ktimersoftd/313:55:513
151891370,1ptp4l401ktimersoftd/313:49:043
151891370,1ptp4l401ktimersoftd/313:00:083
151891370,1ptp4l401ktimersoftd/313:00:083
151891370,1ptp4l401ktimersoftd/312:35:443
313082360,3sleep2321ktimersoftd/212:38:112
309899368,8cyclictest27968-21grep15:08:102
309899363,6cyclictest0-21swapper/212:52:592
309899362,11cyclictest13976-21rm12:57:592
309899362,11cyclictest13976-21rm12:57:592
3098993619,14cyclictest33-21ksoftirqd/211:35:002
3098993619,14cyclictest33-21ksoftirqd/210:32:092
3098993614,19cyclictest9679-21pluginstate11:17:432
3098993614,19cyclictest9679-21pluginstate11:17:432
3094993630,3cyclictest25-21ksoftirqd/115:27:491
3094993630,3cyclictest25-21ksoftirqd/112:47:451
151891360,1ptp4l401ktimersoftd/315:12:333
151891360,1ptp4l401ktimersoftd/313:40:543
151891360,1ptp4l401ktimersoftd/311:21:353
151891360,1ptp4l401ktimersoftd/310:37:183
3098993513,3cyclictest15368-21sh14:54:282
309899351,31cyclictest32226-21tail13:58:242
3098993511,3cyclictest11574-21sed13:33:182
3098993511,3cyclictest10627-21irqrtprio11:18:042
3098993510,4cyclictest5281-21sed15:18:092
3098993510,4cyclictest27102-21basename14:33:022
3094993530,3cyclictest25-21ksoftirqd/113:27:471
3094993530,3cyclictest25-21ksoftirqd/112:52:451
3094993529,3cyclictest25-21ksoftirqd/114:07:481
3094993523,5cyclictest25-21ksoftirqd/110:32:571
151891350,2ptp4l21155-21kworker/3:211:48:483
151891350,2ptp4l21155-21kworker/3:210:26:353
151891350,1ptp4l401ktimersoftd/314:47:333
151891350,1ptp4l401ktimersoftd/313:59:083
151891350,11ptp4l401ktimersoftd/311:58:143
309899349,3cyclictest26166-21cut12:33:042
309899343,9cyclictest10694-21apt-get14:13:042
309899343,6cyclictest5881-21apt-get12:48:032
3098993415,16cyclictest33-21ksoftirqd/211:02:122
3098993413,3cyclictest3287-21phc2sys-jitter14:03:152
3098993413,3cyclictest29486-21sh13:17:192
3098993410,4cyclictest14989-21sed12:18:232
3094993429,3cyclictest25-21ksoftirqd/113:17:461
3094993429,3cyclictest25-21ksoftirqd/112:32:451
3094993426,5cyclictest25-21ksoftirqd/115:07:481
3094993425,4cyclictest25-21ksoftirqd/111:23:011
151891340,2ptp4l5034-21kworker/3:215:19:503
151891340,1ptp4l401ktimersoftd/312:49:533
151891340,12ptp4l401ktimersoftd/315:07:003
151891340,12ptp4l401ktimersoftd/312:06:403
151891340,11ptp4l401ktimersoftd/313:33:043
151891340,11ptp4l401ktimersoftd/313:15:583
151891340,0ptp4l401ktimersoftd/310:08:043
309899339,4cyclictest3384-21sed11:03:062
309899339,4cyclictest3384-21sed11:03:062
309899336,9cyclictest24101-21cron11:47:442
309899331,5cyclictest0-21swapper/211:37:582
3098993312,3cyclictest19046-21mailstats14:58:142
3098993311,4cyclictest15779-21timerwakeupswit13:38:232
3098993310,4cyclictest47550irq/101-ens3-Tx12:09:092
3094993328,3cyclictest25-21ksoftirqd/115:02:481
3094993328,3cyclictest25-21ksoftirqd/114:32:481
3094993325,4cyclictest25-21ksoftirqd/113:28:021
239712330,4sleep1309499cyclictest13:48:241
151891330,1ptp4l401ktimersoftd/314:57:203
151891330,1ptp4l401ktimersoftd/314:36:133
151891330,1ptp4l401ktimersoftd/313:23:273
151891330,1ptp4l401ktimersoftd/311:12:083
151891330,1ptp4l401ktimersoftd/311:12:083
151891330,1ptp4l401ktimersoftd/310:38:403
151891330,0ptp4l401ktimersoftd/314:13:333
309899322,5cyclictest18825-21apt-get14:23:032
309899322,4cyclictest0-21swapper/213:48:052
309899322,11cyclictest26740-21uname10:43:062
3098993212,3cyclictest3702-21cut09:58:022
3098993212,3cyclictest30669-21sendmail_mailtr11:58:092
3098993212,3cyclictest28895-21diskmemload14:11:112
3098993212,3cyclictest15479-21cat11:28:032
3098993212,3cyclictest1505-21runrttasks13:56:142
3098993211,3cyclictest29104-21latency_hist11:57:452
3098993210,3cyclictest17277-21/usr/sbin/munin12:23:162
3094993227,3cyclictest25-21ksoftirqd/112:07:441
3094993226,3cyclictest25-21ksoftirqd/114:48:041
3094993226,3cyclictest25-21ksoftirqd/110:12:551
3094993225,3cyclictest25-21ksoftirqd/114:17:471
3094993221,9cyclictest25-21ksoftirqd/110:58:131
3094993220,9cyclictest25-21ksoftirqd/114:08:091
3094993219,3cyclictest25-21ksoftirqd/113:08:201
151891320,1ptp4l401ktimersoftd/315:24:273
151891320,1ptp4l401ktimersoftd/311:43:273
151891320,1ptp4l401ktimersoftd/311:40:343
151891320,1ptp4l401ktimersoftd/311:03:403
151891320,1ptp4l401ktimersoftd/311:03:403
151891320,1ptp4l401ktimersoftd/310:58:273
151891320,1ptp4l401ktimersoftd/310:42:553
151891320,0ptp4l401ktimersoftd/312:57:023
309899319,4cyclictest1505-21runrttasks15:16:442
309899318,4cyclictest3600-21grep14:43:052
309899318,4cyclictest10776-21unixbench_multi12:13:182
309899318,3cyclictest0-21swapper/215:23:122
309899312,7cyclictest23214-21apt-get10:38:002
3098993110,3cyclictest28895-21diskmemload12:06:002
3094993125,3cyclictest25-21ksoftirqd/114:37:481
3094993125,3cyclictest25-21ksoftirqd/113:02:451
3094993125,3cyclictest25-21ksoftirqd/113:02:451
3094993125,3cyclictest25-21ksoftirqd/112:27:441
3094993125,3cyclictest25-21ksoftirqd/111:58:051
22852316,12sleep20-21swapper/209:53:102
151891310,8ptp4l401ktimersoftd/313:43:523
151891310,1ptp4l401ktimersoftd/314:26:143
151891310,1ptp4l401ktimersoftd/311:15:063
151891310,1ptp4l401ktimersoftd/311:15:063
151891310,0ptp4l401ktimersoftd/313:05:373
309899309,4cyclictest47550irq/101-ens3-Tx12:32:412
309899309,4cyclictest12486-21/usr/sbin/munin11:23:052
309899308,4cyclictest47550irq/101-ens3-Tx13:03:162
3098993013,3cyclictest26774-21awk14:32:472
3098993011,3cyclictest28895-21diskmemload13:20:122
3094993025,3cyclictest25-21ksoftirqd/112:33:041
3094993025,3cyclictest25-21ksoftirqd/111:57:441
151891300,1ptp4l401ktimersoftd/313:08:073
151891300,1ptp4l401ktimersoftd/311:24:403
151891300,1ptp4l401ktimersoftd/310:56:183
3107992923,3cyclictest41-21ksoftirqd/310:03:003
3094992924,3cyclictest25-21ksoftirqd/113:38:231
3094992924,3cyclictest25-21ksoftirqd/111:43:041
3094992924,3cyclictest25-21ksoftirqd/110:43:121
3094992923,3cyclictest25-21ksoftirqd/115:08:071
3094992923,3cyclictest25-21ksoftirqd/114:18:181
3094992923,3cyclictest25-21ksoftirqd/113:58:111
3094992923,3cyclictest25-21ksoftirqd/113:57:471
3094992923,3cyclictest25-21ksoftirqd/112:17:451
3094992923,3cyclictest17931-21cron10:27:421
3094992913,7cyclictest25-21ksoftirqd/111:07:431
3094992913,7cyclictest25-21ksoftirqd/111:07:431
3089992923,4cyclictest9-21ksoftirqd/013:43:130
3089992923,3cyclictest9-21ksoftirqd/012:47:450
151891290,1ptp4l401ktimersoftd/315:15:323
36212280,3sleep10-21swapper/114:43:051
309899289,3cyclictest28895-21diskmemload15:04:002
3094992824,2cyclictest25-21ksoftirqd/110:57:431
3094992823,3cyclictest25-21ksoftirqd/111:22:431
3094992822,3cyclictest25-21ksoftirqd/115:22:491
3094992822,3cyclictest25-21ksoftirqd/114:53:091
3094992822,3cyclictest25-21ksoftirqd/110:28:031
3094992822,3cyclictest25-21ksoftirqd/110:02:431
3089992823,3cyclictest9-21ksoftirqd/013:36:200
3089992822,3cyclictest9-21ksoftirqd/012:18:160
3089992821,4cyclictest9-21ksoftirqd/014:23:100
3089992821,4cyclictest9-21ksoftirqd/011:48:150
151891280,1ptp4l401ktimersoftd/310:01:003
105152280,2sleep00-21swapper/014:49:220
309899278,3cyclictest18425-21latency_hist14:22:472
3098992721,4cyclictest33-21ksoftirqd/211:07:502
3098992721,4cyclictest33-21ksoftirqd/211:07:502
3098992720,4cyclictest33-21ksoftirqd/210:56:022
3094992722,3cyclictest25-21ksoftirqd/115:13:181
3094992722,3cyclictest25-21ksoftirqd/112:53:011
3094992722,3cyclictest25-21ksoftirqd/112:22:451
3094992722,3cyclictest25-21ksoftirqd/112:12:451
3094992721,3cyclictest25-21ksoftirqd/114:23:101
3094992721,3cyclictest25-21ksoftirqd/111:33:061
3094992721,3cyclictest25-21ksoftirqd/110:17:551
3089992722,3cyclictest9-21ksoftirqd/010:02:420
3089992721,3cyclictest9-21ksoftirqd/011:02:070
3107992620,3cyclictest41-21ksoftirqd/311:27:573
3107992619,4cyclictest41-21ksoftirqd/310:14:443
309899260,2cyclictest0-21swapper/210:51:452
3094992621,3cyclictest25-21ksoftirqd/111:38:091
3094992620,3cyclictest25-21ksoftirqd/111:17:071
3094992620,3cyclictest25-21ksoftirqd/111:17:071
3094992620,3cyclictest25-21ksoftirqd/110:38:071
3089992621,3cyclictest9-21ksoftirqd/014:47:480
3089992621,3cyclictest9-21ksoftirqd/013:23:030
3089992621,3cyclictest9-21ksoftirqd/011:53:060
3089992619,4cyclictest9-21ksoftirqd/014:00:520
3089992619,4cyclictest9-21ksoftirqd/012:07:140
3089992619,4cyclictest9-21ksoftirqd/010:49:300
3089992619,4cyclictest9-21ksoftirqd/010:14:130
151891260,1ptp4l401ktimersoftd/310:48:263
151891260,0ptp4l401ktimersoftd/311:34:003
3098992519,3cyclictest33-21ksoftirqd/210:21:372
309899251,5cyclictest0-21swapper/212:43:082
309899250,14cyclictest0-21swapper/213:31:062
309899250,14cyclictest0-21swapper/211:49:212
3094992520,3cyclictest25-21ksoftirqd/111:07:581
3094992520,3cyclictest25-21ksoftirqd/111:07:581
3089992520,3cyclictest9-21ksoftirqd/010:38:070
3089992519,3cyclictest9-21ksoftirqd/014:52:530
3089992519,3cyclictest9-21ksoftirqd/013:08:080
3089992519,3cyclictest9-21ksoftirqd/013:02:550
3089992519,3cyclictest9-21ksoftirqd/011:58:050
3089992518,4cyclictest9-21ksoftirqd/011:32:090
151891250,1ptp4l401ktimersoftd/314:19:453
50302240,1sleep00-21swapper/012:07:580
3098992418,3cyclictest33-21ksoftirqd/210:33:022
3089992419,3cyclictest9-21ksoftirqd/012:13:070
3089992419,3cyclictest9-21ksoftirqd/011:12:580
3089992419,3cyclictest9-21ksoftirqd/011:12:580
3089992418,3cyclictest9-21ksoftirqd/015:23:050
3089992418,3cyclictest9-21ksoftirqd/014:11:280
3089992418,3cyclictest9-21ksoftirqd/011:34:490
3089992418,3cyclictest9-21ksoftirqd/010:54:030
151891240,0ptp4l401ktimersoftd/310:27:543
309899230,14cyclictest0-21swapper/210:13:432
3094992318,3cyclictest25-21ksoftirqd/109:58:041
3089992318,3cyclictest9-21ksoftirqd/015:12:490
3089992318,3cyclictest9-21ksoftirqd/014:33:210
3089992318,3cyclictest9-21ksoftirqd/014:32:010
3089992318,3cyclictest9-21ksoftirqd/012:58:030
3089992318,3cyclictest9-21ksoftirqd/012:58:030
3089992318,3cyclictest9-21ksoftirqd/011:07:430
3089992318,3cyclictest9-21ksoftirqd/011:07:430
3089992317,3cyclictest9-21ksoftirqd/012:35:020
3089992317,3cyclictest14498-21fschecks_time13:38:080
3089992316,4cyclictest9-21ksoftirqd/014:03:010
3089992316,4cyclictest9-21ksoftirqd/010:27:260
151891230,1ptp4l401ktimersoftd/314:38:173
151891230,1ptp4l401ktimersoftd/314:38:173
10932231,18sleep028895-21diskmemload13:21:550
3098992216,4cyclictest33-21ksoftirqd/210:23:102
308999223,16cyclictest22873-21proc_pri12:28:110
3089992217,3cyclictest9-21ksoftirqd/015:02:490
3089992217,3cyclictest9-21ksoftirqd/012:27:450
3089992217,3cyclictest9-21ksoftirqd/011:18:010
3089992217,3cyclictest9-21ksoftirqd/010:33:010
3089992217,3cyclictest9-21ksoftirqd/010:04:570
3089992216,3cyclictest9-21ksoftirqd/011:46:590
3089992216,3cyclictest9-21ksoftirqd/011:41:000
3089992215,4cyclictest9-21ksoftirqd/013:56:390
3089992215,4cyclictest14616-21sh11:27:430
3089992214,4cyclictest15383-21if_err_enp1s0.114:18:080
3089992116,3cyclictest9-21ksoftirqd/015:18:090
3089992116,3cyclictest9-21ksoftirqd/015:13:150
3089992115,3cyclictest9-21ksoftirqd/010:09:190
3089992115,3cyclictest2276-21sh14:41:460
3089992115,3cyclictest2276-21sh14:41:460
3089992115,3cyclictest12362-21sh14:13:210
3089992015,3cyclictest9-21ksoftirqd/013:13:070
3089992014,4cyclictest665-21sh12:41:060
3089992014,4cyclictest25494-21/usr/sbin/munin10:43:020
3089992014,3cyclictest8779-21sh13:30:540
3089992014,3cyclictest5172-21/usr/sbin/munin11:08:090
3089992014,3cyclictest5172-21/usr/sbin/munin11:08:090
3089992014,3cyclictest26050-21sh15:06:270
3089992013,4cyclictest6866-21sed12:48:110
3089992013,4cyclictest22517-21df13:48:080
3089992013,4cyclictest15566-21latency_hist10:22:430
308999200,2cyclictest121rcu_preempt12:53:000
3089991913,3cyclictest18457-21cat10:27:570
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional