You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 00:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Sun Nov 30, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1365621710,5sleep11110999cyclictest08:26:571
1773911060,39ptp4l401ktimersoftd/308:20:353
109732820,1sleep20-21swapper/213:06:002
117332800,2sleep10-21swapper/110:30:381
140282700,3sleep00-21swapper/010:32:220
140282700,3sleep00-21swapper/010:32:220
11114996452,6cyclictest33-21ksoftirqd/212:37:152
121022620,3sleep01110499cyclictest13:07:190
11114996046,7cyclictest33-21ksoftirqd/209:02:102
11114995946,7cyclictest33-21ksoftirqd/213:17:172
177391580,0ptp4l401ktimersoftd/311:18:383
97582560,1sleep00-21swapper/011:44:480
11114995640,8cyclictest33-21ksoftirqd/212:57:152
11114995543,7cyclictest33-21ksoftirqd/213:42:172
293972540,1sleep20-21swapper/212:49:182
210282520,1sleep20-21swapper/210:42:112
11114994937,6cyclictest33-21ksoftirqd/210:37:122
11114994937,6cyclictest33-21ksoftirqd/210:37:122
107192494,33sleep20-21swapper/208:18:142
82452470,1sleep00-21swapper/010:27:000
11114994635,5cyclictest33-21ksoftirqd/210:22:122
11114994233,5cyclictest33-21ksoftirqd/208:32:142
11114994129,6cyclictest33-21ksoftirqd/209:42:112
11104994115,4cyclictest10212-21grep11:07:190
11118994024,13cyclictest41-21ksoftirqd/312:46:263
11114994028,7cyclictest33-21ksoftirqd/212:27:162
177391390,1ptp4l401ktimersoftd/313:43:173
177391390,1ptp4l401ktimersoftd/311:29:163
177391390,1ptp4l401ktimersoftd/310:27:263
11104993917,4cyclictest9-21ksoftirqd/009:04:580
177391380,1ptp4l401ktimersoftd/312:17:563
177391380,1ptp4l401ktimersoftd/312:17:563
177391380,1ptp4l401ktimersoftd/312:05:373
177391380,1ptp4l401ktimersoftd/311:26:033
177391380,1ptp4l401ktimersoftd/311:02:153
11114993827,6cyclictest33-21ksoftirqd/209:52:122
1100223815,12sleep09-21ksoftirqd/008:21:550
177391370,1ptp4l401ktimersoftd/313:39:243
177391370,1ptp4l401ktimersoftd/313:31:263
177391370,1ptp4l401ktimersoftd/313:04:183
177391370,1ptp4l401ktimersoftd/312:58:493
177391370,1ptp4l401ktimersoftd/311:45:253
177391370,1ptp4l401ktimersoftd/311:16:023
177391370,1ptp4l401ktimersoftd/311:00:103
177391370,1ptp4l401ktimersoftd/310:47:513
177391370,1ptp4l401ktimersoftd/310:14:513
177391370,1ptp4l401ktimersoftd/308:30:063
177391370,0ptp4l401ktimersoftd/312:23:563
177391370,0ptp4l401ktimersoftd/310:41:183
177391370,0ptp4l401ktimersoftd/310:41:183
11118993713,20cyclictest10732-21latency_hist11:47:023
11118993712,22cyclictest3065-21expr11:37:163
11114993726,6cyclictest33-21ksoftirqd/209:37:122
177391360,1ptp4l401ktimersoftd/313:24:463
177391360,1ptp4l401ktimersoftd/313:08:393
177391360,1ptp4l401ktimersoftd/312:40:273
177391360,1ptp4l401ktimersoftd/311:33:183
177391360,0ptp4l401ktimersoftd/311:58:493
177391360,0ptp4l401ktimersoftd/310:42:503
11118993611,22cyclictest1407-21sed09:07:153
11114993631,3cyclictest33-21ksoftirqd/213:32:042
11104993616,3cyclictest9-21ksoftirqd/008:37:000
177391350,0ptp4l401ktimersoftd/313:33:133
177391350,0ptp4l401ktimersoftd/310:53:163
1111899351,30cyclictest26750-21head08:52:173
11118993512,20cyclictest17486-21date09:41:593
11118993510,22cyclictest5846-21expr09:17:123
11114993531,3cyclictest33-21ksoftirqd/210:52:002
11114993529,3cyclictest33-21ksoftirqd/212:12:212
11114993529,3cyclictest33-21ksoftirqd/212:12:212
1111899349,21cyclictest4306-21cat09:12:183
11114993425,5cyclictest33-21ksoftirqd/208:37:112
1111899336,4cyclictest15999-21if_err_enp1s0.113:47:243
1111899332,5cyclictest0-21swapper/310:02:123
1111899332,4cyclictest0-21swapper/309:27:133
11114993327,3cyclictest33-21ksoftirqd/211:37:182
3692320,3sleep00-21swapper/012:52:410
1111899329,20cyclictest26836-21/usr/sbin/munin12:47:223
1111899329,19cyclictest10474-21if_enp1s0.111:07:203
1111899325,3cyclictest25039-21grep13:22:053
1111899323,9cyclictest23888-21apt-get08:47:123
1111899323,9cyclictest23888-21apt-get08:47:123
1111899322,6cyclictest4438-21apt-get10:22:123
1111899322,6cyclictest0-21swapper/309:22:113
1111899321,6cyclictest10586-21apt-get12:27:163
1111499323,3cyclictest121rcu_preempt12:02:182
11114993227,3cyclictest33-21ksoftirqd/211:07:012
11114993227,3cyclictest33-21ksoftirqd/210:57:192
11114993226,3cyclictest33-21ksoftirqd/211:52:012
11114993224,4cyclictest33-21ksoftirqd/211:17:162
11114993221,9cyclictest33-21ksoftirqd/211:12:332
11114993213,17cyclictest33-21ksoftirqd/211:07:202
1111899317,6cyclictest29899-21sendmail_mailqu08:57:203
1111899316,8cyclictest13298-21gltestperf09:32:163
1111899313,8cyclictest16792-21apt-get08:32:143
11118993110,18cyclictest31365-21fschecks_count12:52:213
11118993110,18cyclictest16801-21fschecks_time13:12:213
11114993126,3cyclictest33-21ksoftirqd/213:47:242
11114993126,3cyclictest33-21ksoftirqd/212:12:022
11114993126,3cyclictest33-21ksoftirqd/212:12:022
11114993126,3cyclictest33-21ksoftirqd/210:37:002
11114993126,3cyclictest33-21ksoftirqd/210:37:002
11114993126,3cyclictest33-21ksoftirqd/210:32:002
11114993120,6cyclictest33-21ksoftirqd/213:07:182
1111499311,5cyclictest17779-21mailstats13:12:282
11114993111,8cyclictest33-21ksoftirqd/212:52:402
90332305,10sleep10-21swapper/108:17:121
3182523023,3sleep09-21ksoftirqd/010:53:300
1853913022,3phc2sys41-21ksoftirqd/308:37:183
1111899306,4cyclictest18783-21latency_hist11:57:023
1111899305,4cyclictest228532chrt08:42:213
1111899305,4cyclictest228532chrt08:42:213
1111899305,4cyclictest0-21swapper/312:12:213
1111899305,4cyclictest0-21swapper/312:12:213
1111899303,7cyclictest17777-21apt-get09:42:113
1111899301,4cyclictest0-21swapper/310:32:113
1111899301,4cyclictest0-21swapper/310:32:113
11114993025,3cyclictest33-21ksoftirqd/212:22:222
11114993025,3cyclictest33-21ksoftirqd/211:57:182
11114993025,3cyclictest33-21ksoftirqd/211:32:252
11114993025,3cyclictest33-21ksoftirqd/211:22:312
11114993025,3cyclictest33-21ksoftirqd/209:36:592
11114993025,3cyclictest33-21ksoftirqd/209:07:172
11114993025,3cyclictest33-21ksoftirqd/208:46:572
11114993025,3cyclictest33-21ksoftirqd/208:46:572
11114993024,3cyclictest33-21ksoftirqd/213:22:362
11114993024,3cyclictest33-21ksoftirqd/211:52:202
11114993024,3cyclictest33-21ksoftirqd/208:51:572
11114993024,3cyclictest33-21ksoftirqd/208:51:572
11104993023,4cyclictest9-21ksoftirqd/010:59:200
1111899299,3cyclictest15176-21if_ens312:32:223
1111899293,3cyclictest0-21swapper/308:22:103
1111899292,7cyclictest24899-21apt-get09:57:133
1111899292,6cyclictest22535-21apt-get09:52:123
1111899292,5cyclictest30999-21apt-get09:02:113
1111899290,10cyclictest30983-21latency_hist12:12:023
1111899290,10cyclictest30983-21latency_hist12:12:023
11114992924,3cyclictest33-21ksoftirqd/210:57:012
11114992924,3cyclictest33-21ksoftirqd/208:31:572
11114992924,2cyclictest33-21ksoftirqd/210:07:292
11114992922,4cyclictest33-21ksoftirqd/210:18:022
11109992924,3cyclictest25-21ksoftirqd/113:32:041
1111899286,4cyclictest31088-21timerandwakeup10:07:293
1111899281,5cyclictest2078-21cut10:17:143
11114992824,2cyclictest33-21ksoftirqd/211:32:022
11114992823,3cyclictest33-21ksoftirqd/213:37:362
11114992823,3cyclictest33-21ksoftirqd/213:32:182
11114992823,3cyclictest33-21ksoftirqd/212:42:172
11114992823,3cyclictest33-21ksoftirqd/209:47:182
11114992823,3cyclictest33-21ksoftirqd/209:17:182
11114992821,4cyclictest33-21ksoftirqd/209:30:212
11114992821,3cyclictest33-21ksoftirqd/212:32:172
11109992821,4cyclictest25-21ksoftirqd/111:26:311
11109992821,4cyclictest25-21ksoftirqd/110:06:541
137142270,4sleep14036-21diskmemload10:32:191
137142270,4sleep14036-21diskmemload10:32:191
1111899271,5cyclictest20256-21grep09:47:133
11114992722,3cyclictest33-21ksoftirqd/209:22:182
11114992721,3cyclictest33-21ksoftirqd/212:19:312
11114992721,3cyclictest33-21ksoftirqd/212:19:312
11114992721,3cyclictest33-21ksoftirqd/210:02:182
11114992720,4cyclictest33-21ksoftirqd/208:26:382
11109992721,3cyclictest25-21ksoftirqd/110:40:051
11109992721,3cyclictest25-21ksoftirqd/110:40:051
11109992720,4cyclictest25-21ksoftirqd/111:48:231
11109992720,4cyclictest25-21ksoftirqd/111:21:411
11109992713,8cyclictest6981-21ssh11:02:251
1110999270,3cyclictest121rcu_preempt12:07:201
1110999270,3cyclictest121rcu_preempt12:07:201
11104992722,3cyclictest9-21ksoftirqd/012:57:230
11104992721,3cyclictest31579-21sed13:27:340
11104992720,4cyclictest9-21ksoftirqd/010:16:360
11104992720,4cyclictest9-21ksoftirqd/009:56:570
11104992720,4cyclictest9-21ksoftirqd/008:34:030
11114992619,4cyclictest33-21ksoftirqd/208:54:442
1111499260,7cyclictest7266-21ssh11:42:192
11109992620,3cyclictest25-21ksoftirqd/109:44:231
11109992620,3cyclictest25-21ksoftirqd/108:50:101
11109992620,3cyclictest25-21ksoftirqd/108:50:101
11109992620,3cyclictest25-21ksoftirqd/108:38:431
11109992620,3cyclictest25-21ksoftirqd/108:34:351
11109992619,4cyclictest29023-21ssh10:51:441
11109992619,4cyclictest25-21ksoftirqd/111:52:511
11109992619,4cyclictest25-21ksoftirqd/110:08:461
11104992620,3cyclictest9-21ksoftirqd/013:43:570
11104992620,3cyclictest9-21ksoftirqd/009:40:580
11104992620,3cyclictest9-21ksoftirqd/009:09:360
11104992620,3cyclictest9-21ksoftirqd/008:59:470
11104992619,4cyclictest9-21ksoftirqd/011:56:000
11104992619,4cyclictest9-21ksoftirqd/008:44:550
11104992619,4cyclictest9-21ksoftirqd/008:44:550
11114992518,4cyclictest33-21ksoftirqd/208:58:222
11109992519,4cyclictest25-21ksoftirqd/108:31:571
11109992518,4cyclictest25-21ksoftirqd/112:13:321
11109992518,4cyclictest25-21ksoftirqd/112:13:321
11109992518,4cyclictest25-21ksoftirqd/110:13:441
11109992518,4cyclictest25-21ksoftirqd/109:34:091
11109992518,4cyclictest25-21ksoftirqd/109:25:591
11109992518,4cyclictest25-21ksoftirqd/109:17:111
11109992518,4cyclictest25-21ksoftirqd/109:03:301
11104992520,3cyclictest9-21ksoftirqd/009:16:460
11104992519,4cyclictest9-21ksoftirqd/008:27:520
11104992519,3cyclictest9-21ksoftirqd/012:49:160
11104992518,4cyclictest9-21ksoftirqd/009:20:070
11104992518,4cyclictest9-21ksoftirqd/008:23:230
74322240,5sleep11110999cyclictest10:25:081
33222240,5sleep11110999cyclictest10:57:481
11114992419,3cyclictest33-21ksoftirqd/210:01:592
11114992418,3cyclictest33-21ksoftirqd/210:15:242
11109992418,3cyclictest25-21ksoftirqd/109:48:401
11109992418,3cyclictest25-21ksoftirqd/109:08:111
11109992417,5cyclictest25-21ksoftirqd/112:46:461
11109992417,4cyclictest25-21ksoftirqd/113:37:511
11109992417,4cyclictest25-21ksoftirqd/111:12:581
11104992419,3cyclictest9-21ksoftirqd/012:12:330
11104992419,3cyclictest9-21ksoftirqd/012:12:330
11104992419,3cyclictest9-21ksoftirqd/011:47:160
11104992418,4cyclictest9-21ksoftirqd/010:08:050
11104992417,4cyclictest9-21ksoftirqd/011:25:410
11104992417,4cyclictest9-21ksoftirqd/010:21:340
11104992417,4cyclictest9-21ksoftirqd/010:00:220
11104992417,4cyclictest9-21ksoftirqd/009:32:360
11104992417,4cyclictest9-21ksoftirqd/008:55:390
1110499240,20cyclictest29605-21cut13:27:040
11114992317,3cyclictest33-21ksoftirqd/209:12:182
11109992317,3cyclictest25-21ksoftirqd/111:37:281
11109992317,3cyclictest25-21ksoftirqd/110:00:421
11109992316,4cyclictest25-21ksoftirqd/110:44:511
11109992316,4cyclictest25-21ksoftirqd/108:42:031
11109992316,4cyclictest25-21ksoftirqd/108:42:031
1110499238,5cyclictest9-21ksoftirqd/012:07:200
1110499238,5cyclictest9-21ksoftirqd/012:07:200
11104992317,4cyclictest9-21ksoftirqd/012:19:450
11104992317,4cyclictest9-21ksoftirqd/012:19:450
11104992317,3cyclictest9-21ksoftirqd/013:32:160
11104992317,3cyclictest9-21ksoftirqd/012:30:340
11104992317,3cyclictest9-21ksoftirqd/009:28:250
11104992317,3cyclictest9-21ksoftirqd/008:48:160
11104992317,3cyclictest9-21ksoftirqd/008:48:160
1110999225,11cyclictest14568-21ssh12:32:161
1110999225,10cyclictest18994-21apt-get11:57:161
1110999222,17cyclictest17805-21kworker/1:012:20:061
1110999222,17cyclictest17805-21kworker/1:012:20:061
11109992216,3cyclictest25-21ksoftirqd/112:03:541
11109992216,3cyclictest25-21ksoftirqd/108:58:131
11109992215,4cyclictest25-21ksoftirqd/113:45:421
11109992215,4cyclictest25-21ksoftirqd/113:03:461
11109992215,4cyclictest25-21ksoftirqd/109:12:421
11109992215,4cyclictest22543-21apt12:42:031
11109992215,4cyclictest2101-21tr10:17:141
11104992217,3cyclictest9-21ksoftirqd/011:32:260
11104992217,3cyclictest9-21ksoftirqd/010:29:540
1110499221,18cyclictest15345-21apt-get13:47:180
11109992116,3cyclictest25-21ksoftirqd/111:28:391
11109992115,3cyclictest17805-21kworker/1:013:34:501
11109992115,3cyclictest17805-21kworker/1:013:10:011
11109992114,4cyclictest25-21ksoftirqd/109:52:401
11109992114,4cyclictest15507-21grep09:37:161
11109992113,4cyclictest26853-21timerandwakeup13:22:361
11109992113,4cyclictest1760-21runrttasks12:22:381
1110999211,17cyclictest17805-21kworker/1:012:59:291
11109992110,6cyclictest4036-21diskmemload12:47:171
11104992116,3cyclictest9-21ksoftirqd/013:05:380
11104992116,3cyclictest9-21ksoftirqd/011:27:110
11104992115,3cyclictest9-21ksoftirqd/013:38:210
11104992115,3cyclictest9-21ksoftirqd/012:01:590
11104992114,4cyclictest9-21ksoftirqd/012:24:390
11104992114,4cyclictest9-21ksoftirqd/010:02:070
11104992114,4cyclictest3294-21gltestperf11:37:190
11104992114,4cyclictest22300-21sed09:52:000
1110499211,2cyclictest0-21swapper/013:17:340
1110499211,17cyclictest6585-21memory11:02:200
1110499211,17cyclictest0-21swapper/012:44:030
1110499210,19cyclictest19668-21ssh10:39:440
1110499210,19cyclictest19668-21ssh10:39:440
11109992013,4cyclictest21149-21df_inode13:17:201
11109992012,4cyclictest31512-21gltestperf12:52:221
11109992012,4cyclictest10193-21entropy11:07:181
11104992016,2cyclictest0-21swapper/012:38:530
11104992014,3cyclictest9-21ksoftirqd/013:13:560
11104992013,4cyclictest9-21ksoftirqd/009:23:570
11104992013,4cyclictest20314-21uptime11:17:340
11104992013,4cyclictest15344-21open_inodes11:12:240
11104992012,5cyclictest18026-21df09:42:130
1110499201,17cyclictest4036-21diskmemload12:02:320
11109991914,3cyclictest17805-21kworker/1:012:27:281
11109991913,3cyclictest31970-21sh10:53:501
11109991913,3cyclictest31017-21/usr/sbin/munin11:32:251
11109991913,3cyclictest17734-21rt-features13:12:281
11109991912,4cyclictest26477-21expr08:52:141
11104991913,3cyclictest9-21ksoftirqd/010:43:040
1110499190,17cyclictest0-21swapper/012:34:470
11109991812,3cyclictest12716-21latency_hist09:31:591
11109991811,4cyclictest10068-21sh11:45:341
11109991810,4cyclictest15285-21acpi13:47:171
11104991813,3cyclictest26850-21sh10:47:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional