You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 16:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Tue Dec 02, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2333521740,4sleep21589199cyclictest10:38:342
357021510,4sleep01588199cyclictest12:48:400
1871521440,4sleep31589899cyclictest11:51:143
177391990,4ptp4l401ktimersoftd/308:22:343
177391770,1ptp4l401ktimersoftd/311:54:233
177391740,12ptp4l401ktimersoftd/313:44:153
177391730,1ptp4l401ktimersoftd/313:09:053
15891996552,7cyclictest33-21ksoftirqd/211:13:192
15891996452,6cyclictest33-21ksoftirqd/213:48:252
15891996450,6cyclictest33-21ksoftirqd/210:13:192
15891996350,6cyclictest33-21ksoftirqd/210:33:182
15891996249,7cyclictest33-21ksoftirqd/209:08:222
112862600,6sleep01588199cyclictest11:03:370
177391590,1ptp4l401ktimersoftd/313:41:113
15891995947,6cyclictest33-21ksoftirqd/211:58:202
304272570,2sleep10-21swapper/113:20:331
177391570,1ptp4l401ktimersoftd/312:26:523
177391570,1ptp4l401ktimersoftd/308:52:343
15891995747,4cyclictest33-21ksoftirqd/211:28:212
177391550,1ptp4l401ktimersoftd/312:12:133
15891995541,7cyclictest33-21ksoftirqd/209:48:182
19092540,2sleep30-21swapper/312:48:113
1588199532,12cyclictest18442-21snmp_o5powerctl13:43:360
93642520,2sleep00-21swapper/012:18:250
277492520,2sleep30-21swapper/308:43:413
216462520,4sleep221640-21ssh12:33:202
15891995240,6cyclictest33-21ksoftirqd/210:23:202
15891995239,7cyclictest33-21ksoftirqd/208:48:192
15891995137,7cyclictest33-21ksoftirqd/208:43:182
1588199511,13cyclictest26158-21awk11:58:390
154662514,36sleep10-21swapper/108:18:531
1588199500,12cyclictest31076-21sed10:48:280
1588199500,11cyclictest5129-21switchtime13:28:450
1508925029,9sleep20-21swapper/208:18:342
1588199491,20cyclictest18525-21ssh10:33:250
177391480,2ptp4l1125-21kworker/3:011:39:043
1588199480,15cyclictest7441-21hddtemp_smartct10:18:230
1588199480,13cyclictest5371-21irqrtprio10:13:260
1588199470,16cyclictest9018-21diskmemload10:32:490
1588199461,8cyclictest7349-21apt-get11:38:220
1588199451,17cyclictest9416-21/usr/sbin/munin13:33:360
1588199450,9cyclictest6315-21timedrift12:13:400
1588199450,16cyclictest9018-21diskmemload12:24:370
1588199450,11cyclictest9018-21diskmemload13:18:510
1588199450,11cyclictest19567-21/usr/sbin/munin11:13:280
1588199442,14cyclictest3523-1kworker/u9:113:08:530
1588199441,9cyclictest20076-21apt-get11:53:200
1588199441,8cyclictest25961-21apt-get12:38:200
1588199441,8cyclictest15710-21apt-get11:48:220
1588199441,8cyclictest11471-21apt-get11:43:200
1588199441,8cyclictest10827-21python10:23:320
1588199441,11cyclictest31395-21/usr/sbin/munin11:28:370
1588199440,9cyclictest3134-21snmp_o5powerctl10:53:410
177391430,13ptp4l12013-21kworker/3:112:54:583
1588199431,9cyclictest26816-21munin-run11:23:070
1588199431,9cyclictest26816-21munin-run11:23:070
1588199431,9cyclictest19374-21python09:33:270
1588199431,9cyclictest19374-21python09:33:270
1588199431,16cyclictest9018-21diskmemload13:03:040
1588199430,13cyclictest9018-21diskmemload10:39:450
15891994234,4cyclictest33-21ksoftirqd/211:08:202
1588199421,9cyclictest30119-21snmp_o5powerctl12:03:340
1588199421,8cyclictest30046-21cstates09:58:220
1588199421,8cyclictest16660-21if_enp1s008:23:200
1588199421,8cyclictest13280-21apt-get09:23:180
1588199420,12cyclictest23133-21perl09:43:210
1588199420,11cyclictest3179-21cat11:33:220
1588199411,8cyclictest27784-21diskstats09:53:220
1588199411,8cyclictest10611-21munin-run09:18:040
1588199410,16cyclictest9018-21diskmemload13:13:310
1588199410,10cyclictest32353-21open_files08:53:250
1589199409,6cyclictest33-21ksoftirqd/212:14:172
15891994028,9cyclictest33-21ksoftirqd/210:48:382
15891994018,7cyclictest33-21ksoftirqd/211:04:482
15891994015,4cyclictest33-21ksoftirqd/213:33:382
15891994014,4cyclictest33-21ksoftirqd/212:03:242
1588199401,8cyclictest25750-21apt-get10:43:220
1588199400,8cyclictest9018-21diskmemload13:50:440
1588199400,8cyclictest9018-21diskmemload13:24:010
1588199400,8cyclictest9018-21diskmemload11:23:440
1588199400,8cyclictest2979-21sshd08:59:230
1588199400,8cyclictest17075-21ssh11:11:100
1588199400,17cyclictest15565-21/usr/sbin/munin09:28:310
1588199400,16cyclictest9018-21diskmemload12:28:320
177391390,1ptp4l401ktimersoftd/313:38:083
177391390,0ptp4l401ktimersoftd/313:29:113
1588199391,8cyclictest7871-21snmp_o5powerctl12:53:410
1588199391,8cyclictest18865-21diskstats08:28:060
177391380,1ptp4l401ktimersoftd/313:52:013
177391380,1ptp4l401ktimersoftd/313:27:013
177391380,1ptp4l401ktimersoftd/312:59:063
177391380,1ptp4l401ktimersoftd/312:35:383
177391380,1ptp4l401ktimersoftd/312:28:253
177391380,1ptp4l401ktimersoftd/311:36:073
177391380,1ptp4l401ktimersoftd/310:56:443
177391380,1ptp4l401ktimersoftd/310:38:323
15891993833,3cyclictest33-21ksoftirqd/211:28:072
15891993832,3cyclictest33-21ksoftirqd/211:43:272
15891993825,11cyclictest33-21ksoftirqd/213:03:252
15891993823,3cyclictest33-21ksoftirqd/208:33:152
1589199381,3cyclictest121rcu_preempt08:33:042
1588199381,8cyclictest25558-21ssh12:38:060
1588199380,8cyclictest30583-21ssh12:43:250
177391370,2ptp4l12013-21kworker/3:112:41:323
177391370,1ptp4l401ktimersoftd/313:14:013
177391370,1ptp4l401ktimersoftd/313:05:223
177391370,1ptp4l401ktimersoftd/312:06:193
177391370,1ptp4l401ktimersoftd/311:25:293
177391370,1ptp4l401ktimersoftd/311:22:483
177391370,1ptp4l401ktimersoftd/311:22:483
177391370,1ptp4l401ktimersoftd/311:13:443
177391370,1ptp4l401ktimersoftd/311:01:043
177391370,1ptp4l401ktimersoftd/310:23:253
15891993732,3cyclictest33-21ksoftirqd/211:53:082
15891993732,3cyclictest33-21ksoftirqd/209:13:202
15891993732,3cyclictest33-21ksoftirqd/208:28:042
15891993731,3cyclictest33-21ksoftirqd/212:58:102
15891993731,3cyclictest33-21ksoftirqd/212:53:102
15891993731,3cyclictest33-21ksoftirqd/212:28:262
15891993726,4cyclictest33-21ksoftirqd/213:38:272
177391360,0ptp4l401ktimersoftd/312:45:403
177391360,0ptp4l401ktimersoftd/312:19:323
177391360,0ptp4l401ktimersoftd/311:12:353
15891993631,3cyclictest33-21ksoftirqd/213:43:402
15891993631,3cyclictest33-21ksoftirqd/213:28:102
15891993631,3cyclictest33-21ksoftirqd/212:23:282
15891993630,3cyclictest33-21ksoftirqd/213:08:272
15891993630,3cyclictest33-21ksoftirqd/212:18:292
15891993630,3cyclictest33-21ksoftirqd/211:18:392
15891993630,3cyclictest33-21ksoftirqd/211:18:392
15891993630,3cyclictest33-21ksoftirqd/210:58:262
142732363,19sleep014369-21/usr/sbin/munin08:18:240
177391350,1ptp4l401ktimersoftd/312:02:293
177391350,0ptp4l401ktimersoftd/311:45:103
177391350,0ptp4l401ktimersoftd/310:51:563
177391350,0ptp4l401ktimersoftd/309:57:453
15891993530,3cyclictest33-21ksoftirqd/212:43:272
15891993530,3cyclictest33-21ksoftirqd/209:58:062
15891993530,3cyclictest33-21ksoftirqd/208:58:212
15891993529,3cyclictest33-21ksoftirqd/213:18:112
15891993529,3cyclictest33-21ksoftirqd/212:08:392
15891993529,3cyclictest33-21ksoftirqd/210:33:072
15891993529,3cyclictest33-21ksoftirqd/209:28:222
15891993523,3cyclictest33-21ksoftirqd/211:33:242
1588199350,8cyclictest9018-21diskmemload11:00:580
177391340,2ptp4l1125-21kworker/3:011:05:343
177391340,1ptp4l401ktimersoftd/311:30:113
177391340,1ptp4l401ktimersoftd/309:38:293
177391340,1ptp4l401ktimersoftd/309:25:223
177391340,0ptp4l401ktimersoftd/309:17:433
15891993429,3cyclictest33-21ksoftirqd/210:53:232
15891993429,3cyclictest33-21ksoftirqd/210:48:072
15891993428,4cyclictest33-21ksoftirqd/212:58:242
15891993428,3cyclictest33-21ksoftirqd/213:23:112
15891993428,3cyclictest33-21ksoftirqd/210:08:062
15891993421,10cyclictest33-21ksoftirqd/211:53:382
177391330,1ptp4l401ktimersoftd/310:35:393
177391330,10ptp4l401ktimersoftd/310:28:463
15891993328,3cyclictest33-21ksoftirqd/211:38:252
15891993328,3cyclictest33-21ksoftirqd/209:58:242
15891993327,3cyclictest33-21ksoftirqd/209:38:212
15891993327,3cyclictest33-21ksoftirqd/209:23:352
15891993327,3cyclictest33-21ksoftirqd/209:18:262
15891993327,3cyclictest33-21ksoftirqd/208:38:202
177391320,1ptp4l401ktimersoftd/313:20:093
177391320,1ptp4l401ktimersoftd/310:45:573
177391320,1ptp4l401ktimersoftd/310:16:043
177391320,1ptp4l401ktimersoftd/308:26:433
15891993227,3cyclictest33-21ksoftirqd/213:28:272
15891993227,3cyclictest33-21ksoftirqd/209:03:162
15891993227,3cyclictest33-21ksoftirqd/208:53:182
15891993226,3cyclictest33-21ksoftirqd/212:38:272
15891993226,3cyclictest33-21ksoftirqd/210:23:062
15891993226,3cyclictest33-21ksoftirqd/210:08:382
1588199321,9cyclictest7498-21python09:08:340
177391310,1ptp4l401ktimersoftd/312:13:303
177391310,1ptp4l401ktimersoftd/310:09:163
177391310,1ptp4l401ktimersoftd/309:36:393
177391310,1ptp4l401ktimersoftd/309:36:393
177391310,1ptp4l401ktimersoftd/308:59:073
177391310,0ptp4l401ktimersoftd/309:48:463
177391310,0ptp4l401ktimersoftd/308:29:113
15891993126,3cyclictest33-21ksoftirqd/209:43:282
15891993126,3cyclictest33-21ksoftirqd/209:33:212
15891993126,3cyclictest33-21ksoftirqd/209:33:212
177391300,0ptp4l401ktimersoftd/309:04:283
177391300,0ptp4l401ktimersoftd/308:40:073
1588799304,15cyclictest22007-21gltestperf13:48:301
1588799300,3cyclictest121rcu_preempt12:12:351
1588199291,15cyclictest16314-21snmp_o5powerctl13:03:380
1588199291,12cyclictest784-21sed10:03:260
1588199291,10cyclictest21957-21perl08:33:200
1588199290,16cyclictest9018-21diskmemload13:42:440
177391280,1ptp4l401ktimersoftd/309:18:223
1588799283,14cyclictest5770-21sh10:58:191
15887992822,3cyclictest25-21ksoftirqd/111:47:561
15887992822,3cyclictest25-21ksoftirqd/109:08:241
15887992821,4cyclictest25-21ksoftirqd/110:37:371
1588199281,8cyclictest26667-21fschecks_time08:43:230
15887992721,3cyclictest25-21ksoftirqd/110:44:101
15887992721,3cyclictest25-21ksoftirqd/109:37:501
15887992721,3cyclictest25-21ksoftirqd/109:37:501
15887992720,4cyclictest25-21ksoftirqd/109:47:021
15881992721,3cyclictest9-21ksoftirqd/009:05:260
15881992720,4cyclictest9-21ksoftirqd/008:50:450
177391260,1ptp4l401ktimersoftd/310:21:283
15887992620,3cyclictest25-21ksoftirqd/112:57:211
15887992620,3cyclictest25-21ksoftirqd/111:29:371
15887992619,4cyclictest25-21ksoftirqd/111:48:231
15887992619,4cyclictest25-21ksoftirqd/110:17:371
15887992619,4cyclictest25-21ksoftirqd/109:30:401
177391250,1ptp4l401ktimersoftd/310:04:453
177391250,1ptp4l401ktimersoftd/309:43:273
15898992519,3cyclictest41-21ksoftirqd/308:55:523
1588799253,12cyclictest7988-21ssh13:33:241
15887992520,3cyclictest25-21ksoftirqd/113:29:031
15887992519,4cyclictest25-21ksoftirqd/110:04:371
15887992519,3cyclictest25-21ksoftirqd/108:50:471
15887992518,4cyclictest25-21ksoftirqd/108:25:171
1588199251,8cyclictest25280-21snmp_o5powerctl08:38:300
15881992518,4cyclictest9-21ksoftirqd/009:48:420
15881992515,5cyclictest101ktimersoftd/009:38:170
1588199251,11cyclictest0-21swapper/012:08:220
177391240,1ptp4l401ktimersoftd/309:28:283
177391240,1ptp4l401ktimersoftd/309:08:233
177391240,1ptp4l401ktimersoftd/308:33:173
15887992419,3cyclictest25-21ksoftirqd/110:58:061
15887992418,3cyclictest25-21ksoftirqd/113:43:351
15887992418,3cyclictest25-21ksoftirqd/110:12:121
15887992418,3cyclictest25-21ksoftirqd/109:54:221
15887992418,3cyclictest25-21ksoftirqd/109:04:461
15887992418,3cyclictest25-21ksoftirqd/108:54:241
15887992417,4cyclictest25-21ksoftirqd/113:00:371
15887992417,4cyclictest25-21ksoftirqd/109:25:021
15881992418,3cyclictest9-21ksoftirqd/010:12:510
177391230,1ptp4l401ktimersoftd/309:58:323
1588799232,11cyclictest9018-21diskmemload12:43:221
15887992317,4cyclictest11437-21if_enp1s009:18:251
15887992317,3cyclictest25-21ksoftirqd/112:41:421
15887992317,3cyclictest25-21ksoftirqd/112:15:521
15887992317,3cyclictest25-21ksoftirqd/110:00:581
15887992317,3cyclictest25-21ksoftirqd/109:51:271
15887992316,4cyclictest25-21ksoftirqd/113:16:071
15887992316,4cyclictest25-21ksoftirqd/109:39:441
15887992216,4cyclictest25-21ksoftirqd/111:16:391
15887992216,3cyclictest25-21ksoftirqd/110:21:301
15887992214,5cyclictest8794-21df_inode09:13:181
15887992213,5cyclictest21432-21latency_hist12:33:091
15887992212,3cyclictest14170-21irqrtprio12:23:311
1588799221,18cyclictest11168-21snmp_o5powerctl11:03:351
172492210,2sleep10-21swapper/113:04:511
15887992115,3cyclictest25-21ksoftirqd/110:52:391
15887992114,4cyclictest25-21ksoftirqd/108:35:121
1588799202,16cyclictest18783-21kworker/1:113:09:061
15887992015,3cyclictest25-21ksoftirqd/113:39:591
15887992014,4cyclictest25-21ksoftirqd/113:27:191
15887992014,3cyclictest25-21ksoftirqd/110:28:521
15887992014,3cyclictest20758-21grep08:28:321
15887992013,4cyclictest3016-21kernelversion12:48:301
15887992013,4cyclictest2791-21grep08:58:331
15887992013,4cyclictest23005-21expr11:18:221
15887992013,4cyclictest23005-21expr11:18:221
15887992013,4cyclictest1760-21runrttasks12:05:371
15887992013,4cyclictest11164-21grep10:23:391
15887992012,4cyclictest9018-21diskmemload11:53:221
1588199209,7cyclictest11894-21cut09:18:270
1588799192,15cyclictest7605-21kworker/1:012:33:111
1588799191,6cyclictest24220-21ssh11:58:101
1588799191,4cyclictest10357-21/usr/sbin/munin12:18:331
15887991914,3cyclictest3017-21/usr/sbin/munin11:33:231
15887991913,3cyclictest24752-21cut08:38:231
15887991912,4cyclictest26681-21fschecks_time08:43:231
15887991912,4cyclictest25-21ksoftirqd/110:39:371
1588799191,16cyclictest18783-21kworker/1:111:39:001
1588799182,14cyclictest18783-21kworker/1:111:23:291
15887991812,3cyclictest15469-21sh11:08:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional