You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-04 - 12:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Sun Jan 04, 2026 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1588321470,4sleep20-21swapper/221:52:002
1588321470,4sleep20-21swapper/221:52:002
2428521440,2sleep10-21swapper/122:00:391
3081221350,2sleep130814-21ssh22:40:381
664221190,2sleep0151rcuc/023:22:120
664221190,2sleep0151rcuc/023:22:120
2524521100,10sleep012-21ksoftirqd/020:30:370
2524521100,10sleep012-21ksoftirqd/020:30:370
1398921090,3chrt0-21swapper/123:30:251
1398921090,3chrt0-21swapper/123:30:251
1739121030,3sleep00-21swapper/000:40:090
1739121030,3sleep00-21swapper/000:40:090
1739121030,3sleep00-21swapper/000:40:090
179702950,2sleep129537-21cyclictest21:54:531
179702950,2sleep129537-21cyclictest21:54:531
101202950,3sleep10-21swapper/123:59:231
101202950,3sleep10-21swapper/123:59:231
30992920,4sleep10-21swapper/121:39:411
30992920,4sleep10-21swapper/121:39:411
30992920,4sleep10-21swapper/121:39:411
23792890,2sleep30-21swapper/323:50:413
23792890,2sleep30-21swapper/323:50:413
20972870,3sleep00-21swapper/021:38:200
20972870,3sleep00-21swapper/021:38:200
20972870,3sleep00-21swapper/021:38:200
203992830,2sleep00-21swapper/021:24:210
203992830,2sleep00-21swapper/021:24:210
268082820,3sleep10-21swapper/123:10:201
268082820,3sleep10-21swapper/123:10:201
171202750,3sleep10-21swapper/123:00:191
171202750,3sleep10-21swapper/123:00:191
247522700,3sleep00-21swapper/022:00:460
308932670,2sleep230904-21sh23:14:202
308932670,2sleep230904-21sh23:14:202
153572670,4chrt0-21swapper/100:37:211
153572670,4chrt0-21swapper/100:37:211
153572670,4chrt0-21swapper/100:37:211
97792650,2chrt0-21swapper/021:13:240
138402640,3sleep00-21swapper/023:30:200
138402640,3sleep00-21swapper/023:30:200
126032630,10sleep333-21ksoftirqd/323:28:423
126032630,10sleep333-21ksoftirqd/323:28:423
12322560,3sleep30-21swapper/320:55:313
12322560,3sleep30-21swapper/320:55:313
12322560,3sleep30-21swapper/320:55:313
271682550,3chrt0-21swapper/300:15:573
271682550,3chrt0-21swapper/300:15:573
174602550,3sleep00-21swapper/023:00:300
149812530,3chrt0-21swapper/323:30:393
149812530,3chrt0-21swapper/323:30:393
131272530,3chrt0-21swapper/021:16:090
131272530,3chrt0-21swapper/021:16:090
294932520,3sleep228095-21ls19:10:472
294932520,3sleep228095-21ls19:10:472
276322520,2chrt0-21swapper/323:10:343
276322520,2chrt0-21swapper/323:10:343
213852520,2sleep221408-21sort21:25:292
213852520,2sleep221408-21sort21:25:292
93362480,2sleep00-21swapper/021:45:350
93362480,2sleep00-21swapper/021:45:350
303572470,1sleep20-21swapper/222:06:512
303572470,1sleep20-21swapper/222:06:512
27072460,4chrt0-21swapper/322:11:423
254922460,3chrt0-21swapper/121:29:331
254922460,3chrt0-21swapper/121:29:331
128852460,2sleep00-21swapper/022:22:280
82732440,3sleep30-21swapper/323:56:523
82732440,3sleep30-21swapper/323:56:523
73912440,2sleep30-21swapper/300:29:553
73912440,2sleep30-21swapper/300:29:553
241182440,3sleep10-21swapper/123:40:301
241182440,3sleep10-21swapper/123:40:301
78392430,3sleep227-21ksoftirqd/200:30:222
78392430,3sleep227-21ksoftirqd/200:30:222
38162430,4sleep10-21swapper/123:19:561
38162430,4sleep10-21swapper/123:19:561
38162430,4sleep10-21swapper/123:19:561
236162430,2sleep10-21swapper/122:33:471
236162430,2sleep10-21swapper/122:33:471
189292430,4chrt0-21swapper/323:01:143
189292430,4chrt0-21swapper/323:01:143
255452420,3sleep30-21swapper/322:01:583
126482420,3sleep00-21swapper/000:01:190
126482420,3sleep00-21swapper/000:01:190
30912410,3sleep00-21swapper/021:00:300
30912410,3sleep00-21swapper/021:00:300
227742390,2sleep00-21swapper/000:11:400
227742390,2sleep00-21swapper/000:11:400
227742390,2sleep00-21swapper/000:11:400
160032390,3sleep00-21swapper/022:25:380
160032390,3sleep00-21swapper/022:25:380
3812380,3sleep30-21swapper/322:10:183
3812380,3sleep30-21swapper/322:10:183
212132380,1sleep00-21swapper/021:57:370
212132380,1sleep00-21swapper/021:57:370
212132380,1sleep00-21swapper/021:57:370
176002380,4chrt0-21swapper/322:27:143
176002380,4chrt0-21swapper/322:27:143
243462370,2sleep26153-21diskmemload00:13:522
243462370,2sleep26153-21diskmemload00:13:522
243462370,2sleep26153-21diskmemload00:13:522
205662360,3sleep10-21swapper/100:10:201
107462360,2sleep010757-21diskmemload00:32:460
107462360,2sleep010757-21diskmemload00:32:460
325382350,3sleep30-21swapper/323:49:163
325382350,3sleep30-21swapper/323:49:163
313082350,3sleep10-21swapper/100:20:371
294882351,27sleep30-21swapper/319:10:443
294882351,27sleep30-21swapper/319:10:443
293392345,20sleep129472-21/usr/sbin/munin19:10:421
293392345,20sleep129472-21/usr/sbin/munin19:10:421
143392340,3sleep20-21swapper/220:00:332
143392340,3sleep20-21swapper/220:00:332
31112330,2sleep23124-21sh21:39:432
31112330,2sleep23124-21sh21:39:432
31112330,2sleep23124-21sh21:39:432
229112330,3sleep022905-21ssh23:05:370
229112330,3sleep022905-21ssh23:05:370
227932330,2sleep222804-21ssh23:39:152
227932330,2sleep222804-21ssh23:39:152
227932330,2sleep222804-21ssh23:39:152
145582330,2sleep20-21swapper/200:03:582
145582330,2sleep20-21swapper/200:03:582
137342330,2sleep30-21swapper/300:02:503
137342330,2sleep30-21swapper/300:02:503
68352320,3sleep00-21swapper/021:43:060
29538993211,9cyclictest12-21ksoftirqd/023:04:350
29538993211,9cyclictest12-21ksoftirqd/023:04:350
161092320,3sleep3321rcuc/300:38:233
161092320,3sleep3321rcuc/300:38:233
161092320,3sleep3321rcuc/300:38:233
135722320,3sleep30-21swapper/321:50:243
135722320,3sleep30-21swapper/321:50:243
76172310,2sleep00-21swapper/022:16:470
76172310,2sleep00-21swapper/022:16:470
33962310,3sleep30-21swapper/322:45:383
179242310,1sleep06153-21diskmemload23:34:190
179242310,1sleep06153-21diskmemload23:34:190
244252300,2sleep20-21swapper/222:00:402
44372290,3sleep20-21swapper/223:20:302
44372290,3sleep20-21swapper/223:20:302
2953899299,10cyclictest12-21ksoftirqd/019:56:350
2953899299,10cyclictest12-21ksoftirqd/019:56:350
2953899297,14cyclictest12-21ksoftirqd/019:41:130
2953899297,14cyclictest12-21ksoftirqd/019:41:130
2953899283,16cyclictest12-21ksoftirqd/022:59:440
121852280,4sleep30-21swapper/321:15:363
121852280,4sleep30-21swapper/321:15:363
69232270,3sleep20-21swapper/223:55:382
69232270,3sleep20-21swapper/223:55:382
38672270,2sleep20-21swapper/222:45:532
2953899276,9cyclictest12-21ksoftirqd/020:16:190
2953899276,9cyclictest12-21ksoftirqd/020:16:190
2953899276,10cyclictest12-21ksoftirqd/019:12:580
2953899276,10cyclictest12-21ksoftirqd/019:12:580
193122270,1sleep20-21swapper/221:22:512
193122270,1sleep20-21swapper/221:22:512
156042270,3sleep129537-21cyclictest21:19:281
156042270,3sleep129537-21cyclictest21:19:281
2953899269,10cyclictest12-21ksoftirqd/019:24:410
2953899269,10cyclictest12-21ksoftirqd/019:24:410
2953899268,10cyclictest12-21ksoftirqd/021:07:030
2953899268,10cyclictest12-21ksoftirqd/021:07:030
2953899267,11cyclictest12-21ksoftirqd/020:37:130
2953899267,11cyclictest12-21ksoftirqd/020:37:130
2953899267,11cyclictest12-21ksoftirqd/020:37:130
2953899261,20cyclictest7117-21ls22:50:170
71142250,3sleep00-21swapper/000:29:320
71142250,3sleep00-21swapper/000:29:320
29540992512,10cyclictest13181-21ssh21:16:132
29540992512,10cyclictest13181-21ssh21:16:132
2953899258,4cyclictest131rcu_preempt20:28:500
2953899258,4cyclictest131rcu_preempt20:28:500
2953899258,10cyclictest12-21ksoftirqd/020:07:510
2953899258,10cyclictest12-21ksoftirqd/020:07:510
2953899257,7cyclictest12-21ksoftirqd/023:59:120
2953899257,7cyclictest12-21ksoftirqd/023:59:120
2953899257,11cyclictest12-21ksoftirqd/020:41:480
2953899256,9cyclictest12-21ksoftirqd/022:36:300
2953899256,9cyclictest12-21ksoftirqd/022:36:300
2953899256,9cyclictest12-21ksoftirqd/022:36:300
2953899256,7cyclictest12-21ksoftirqd/022:34:390
2953899256,7cyclictest12-21ksoftirqd/022:34:390
2953899256,7cyclictest12-21ksoftirqd/021:33:400
2953899256,7cyclictest12-21ksoftirqd/021:33:400
2953899255,11cyclictest12-21ksoftirqd/020:22:270
2953899255,11cyclictest12-21ksoftirqd/020:22:270
2953899254,15cyclictest12-21ksoftirqd/021:51:380
2953899254,15cyclictest12-21ksoftirqd/021:51:380
29538992510,4cyclictest12-21ksoftirqd/023:52:370
29538992510,4cyclictest12-21ksoftirqd/023:52:370
29538992510,4cyclictest12-21ksoftirqd/022:14:030
84652240,3sleep30-21swapper/322:17:573
84652240,3sleep30-21swapper/322:17:573
61202240,2sleep30-21swapper/321:42:093
2953999240,2cyclictest131rcu_preempt22:00:171
2953999240,2cyclictest131rcu_preempt22:00:171
2953999240,2cyclictest131rcu_preempt22:00:171
2953899249,7cyclictest12-21ksoftirqd/000:16:450
2953899249,7cyclictest12-21ksoftirqd/000:16:450
2953899248,4cyclictest12-21ksoftirqd/022:09:450
2953899248,4cyclictest12-21ksoftirqd/022:09:450
2953899247,9cyclictest12-21ksoftirqd/020:54:120
2953899247,9cyclictest12-21ksoftirqd/020:54:120
2953899247,9cyclictest12-21ksoftirqd/019:46:220
2953899247,9cyclictest12-21ksoftirqd/019:46:220
2953899247,8cyclictest12-21ksoftirqd/019:39:190
2953899247,8cyclictest12-21ksoftirqd/019:39:190
2953899247,8cyclictest12-21ksoftirqd/019:39:190
2953899246,4cyclictest12-21ksoftirqd/021:25:430
2953899246,4cyclictest12-21ksoftirqd/021:25:430
2953899246,11cyclictest12-21ksoftirqd/023:44:360
2953899246,11cyclictest12-21ksoftirqd/023:44:360
2953899244,7cyclictest12-21ksoftirqd/022:45:210
2953899241,17cyclictest12-21ksoftirqd/023:46:110
2953899241,17cyclictest12-21ksoftirqd/023:46:110
108012240,3sleep00-21swapper/019:50:340
108012240,3sleep00-21swapper/019:50:340
29540992310,10cyclictest26273-21/usr/sbin/munin20:35:322
29540992310,10cyclictest26273-21/usr/sbin/munin20:35:322
29540992310,10cyclictest26273-21/usr/sbin/munin20:35:322
29539992312,9cyclictest9279-21switchtime19:45:391
29539992312,9cyclictest9279-21switchtime19:45:391
2953899239,7cyclictest12-21ksoftirqd/000:07:060
2953899236,7cyclictest12-21ksoftirqd/019:16:090
2953899236,7cyclictest12-21ksoftirqd/019:16:090
29538992316,5cyclictest28850irq/49-vc409:14:540
29538992310,6cyclictest12-21ksoftirqd/000:23:250
29540992212,7cyclictest0-21swapper/220:15:172
2954099221,19cyclictest13963-21ls00:35:402
2954099221,19cyclictest13963-21ls00:35:402
2954099221,19cyclictest13963-21ls00:35:402
2953999228,4cyclictest131rcu_preempt19:15:401
2953999228,4cyclictest131rcu_preempt19:15:401
2953899228,7cyclictest12-21ksoftirqd/020:04:560
2953899228,7cyclictest12-21ksoftirqd/020:04:560
2953899227,7cyclictest12-21ksoftirqd/023:35:450
2953899227,7cyclictest12-21ksoftirqd/023:35:450
2953899227,7cyclictest12-21ksoftirqd/023:35:450
2953899226,7cyclictest12-21ksoftirqd/023:14:180
2953899226,7cyclictest12-21ksoftirqd/023:14:180
2953899225,3cyclictest131rcu_preempt19:30:320
2953899225,3cyclictest131rcu_preempt19:30:320
2953899224,7cyclictest12-21ksoftirqd/022:52:590
2953899224,7cyclictest12-21ksoftirqd/022:52:590
29540992115,5cyclictest15011-21ssh22:57:332
29540992115,5cyclictest15011-21ssh22:57:332
29540992114,5cyclictest7312-21rm21:43:462
29540992112,7cyclictest0-21swapper/222:20:392
2954099211,17cyclictest25373-21ls22:35:312
2954099211,17cyclictest25373-21ls22:35:312
2954099211,17cyclictest25373-21ls22:35:312
2954099211,16cyclictest5961-21ssh22:15:322
2954099211,16cyclictest5961-21ssh22:15:322
2953999211,6cyclictest23095-21ssh00:12:071
2953999211,6cyclictest23095-21ssh00:12:071
2953999211,6cyclictest23095-21ssh00:12:071
2953999211,18cyclictest11622-21ls00:00:331
2953999211,18cyclictest11622-21ls00:00:331
2953999211,16cyclictest28156-21ls20:40:321
2953899215,8cyclictest12-21ksoftirqd/019:28:540
2953899215,8cyclictest12-21ksoftirqd/019:28:540
2953899215,6cyclictest12-21ksoftirqd/023:16:030
2953899215,6cyclictest12-21ksoftirqd/023:16:030
2953899215,6cyclictest12-21ksoftirqd/023:16:030
2953899215,10cyclictest12-21ksoftirqd/020:45:340
71372200,2sleep10-21swapper/121:43:311
2954099209,9cyclictest31716-21latency_hist19:20:182
2954099209,9cyclictest31716-21latency_hist19:20:182
2954099205,12cyclictest22882-21grep20:25:292
2954099205,12cyclictest22882-21grep20:25:292
29540992013,5cyclictest6153-21diskmemload00:34:072
29540992013,5cyclictest6153-21diskmemload00:34:072
29540992012,6cyclictest17344-21/usr/sbin/munin23:00:332
29540992012,6cyclictest17344-21/usr/sbin/munin23:00:332
2954099201,15cyclictest30075-21grep20:45:302
2954099201,14cyclictest7946-21/usr/sbin/munin22:50:332
2954099201,14cyclictest7946-21/usr/sbin/munin22:50:332
2954099201,10cyclictest29746-21grep23:45:402
2954099201,10cyclictest29746-21grep23:45:402
29540992010,7cyclictest0-21swapper/220:55:292
29540992010,7cyclictest0-21swapper/220:55:292
29540992010,7cyclictest0-21swapper/220:55:292
2954099200,5cyclictest10768-21ssh21:14:452
2954099200,13cyclictest22638-21seq22:32:252
2954099200,13cyclictest22638-21seq22:32:252
2953999208,9cyclictest302-21grep20:50:411
2953999208,9cyclictest302-21grep20:50:411
29539992013,5cyclictest4950-21sendmail-mta19:35:301
29539992013,5cyclictest4950-21sendmail-mta19:35:301
29539992013,5cyclictest4950-21sendmail-mta19:35:301
2953899206,8cyclictest12-21ksoftirqd/020:57:260
2953899206,8cyclictest12-21ksoftirqd/020:57:260
2953899206,8cyclictest12-21ksoftirqd/020:57:260
7302191,3sleep10-21swapper/119:21:351
7302191,3sleep10-21swapper/119:21:351
56642190,3sleep10-21swapper/100:27:321
56642190,3sleep10-21swapper/100:27:321
299352190,3sleep30-21swapper/321:33:493
299352190,3sleep30-21swapper/321:33:493
2954199191,14cyclictest0-21swapper/323:05:393
2954199191,14cyclictest0-21swapper/323:05:393
2954099196,10cyclictest1281-21dbus-daemon00:20:172
2954099196,10cyclictest1281-21dbus-daemon00:20:172
2954099191,9cyclictest32523-21ntp_kernel_pll_19:20:352
2954099191,9cyclictest32523-21ntp_kernel_pll_19:20:352
2954099191,6cyclictest20445-1kworker/u9:200:20:332
29540991912,5cyclictest0-21swapper/223:25:302
29540991912,5cyclictest0-21swapper/223:25:302
29540991911,6cyclictest0-21swapper/220:05:402
29540991911,6cyclictest0-21swapper/220:05:402
29540991911,6cyclictest0-21swapper/219:55:172
29540991911,6cyclictest0-21swapper/219:55:172
2954099191,15cyclictest1817-21needreboot19:25:332
2954099191,15cyclictest1817-21needreboot19:25:332
29540991910,7cyclictest1567-21which22:10:392
2954099190,14cyclictest5821-21latency_hist23:55:252
2954099190,14cyclictest5821-21latency_hist23:55:252
2953999198,9cyclictest3048-21cut22:45:341
2953999198,9cyclictest28536-21cut22:05:321
2953999198,9cyclictest28536-21cut22:05:321
2953999197,10cyclictest15672-21cat22:25:331
2953999197,10cyclictest15672-21cat22:25:331
2953999196,11cyclictest30463-21ntp_kernel_err20:45:341
2953999193,7cyclictest10261-21ssh22:20:221
2953999193,7cyclictest10261-21ssh22:20:221
2953999192,12cyclictest22592-21latency_hist20:25:211
2953999192,12cyclictest22592-21latency_hist20:25:211
29539991911,6cyclictest7635-21sh22:50:311
29539991911,6cyclictest7635-21sh22:50:311
2953999191,16cyclictest2072-21grep20:55:401
2953999191,16cyclictest2072-21grep20:55:401
2953999191,16cyclictest2072-21grep20:55:401
29539991911,6cyclictest18398-21unixbench-2d20:10:401
2953999191,13cyclictest12393-21ls19:55:321
2953999191,13cyclictest12393-21ls19:55:321
2953999190,17cyclictest1123-21latency22:10:331
2953999190,11cyclictest29600-21ssh00:19:171
2953999190,11cyclictest29600-21ssh00:19:171
187572191,3sleep30-21swapper/320:13:173
2954199181,13cyclictest0-21swapper/322:35:393
2954199181,13cyclictest0-21swapper/322:35:393
2954199181,13cyclictest0-21swapper/322:35:393
2954199181,12cyclictest0-21swapper/321:25:393
2954199181,12cyclictest0-21swapper/321:25:393
2954199181,11cyclictest0-21swapper/321:05:293
2954199181,11cyclictest0-21swapper/321:05:293
2954199181,11cyclictest0-21swapper/320:00:303
2954199181,11cyclictest0-21swapper/320:00:303
2954099188,8cyclictest24164-21fw_packets23:40:302
2954099188,8cyclictest24164-21fw_packets23:40:302
2954099182,14cyclictest3014-21fschecks_time21:00:302
2954099182,14cyclictest3014-21fschecks_time21:00:302
2954099181,9cyclictest695-21runrttasks20:21:102
2954099181,9cyclictest695-21runrttasks20:21:102
2954099181,8cyclictest4606-21fschecks_time23:20:312
2954099181,8cyclictest4606-21fschecks_time23:20:312
2954099181,3cyclictest0-21swapper/200:05:452
29540991812,4cyclictest0-21swapper/220:35:172
29540991812,4cyclictest0-21swapper/220:35:172
2954099181,15cyclictest4878-21ls21:05:302
2954099181,15cyclictest4878-21ls21:05:302
29540991811,5cyclictest17197-21ssh23:33:172
29540991811,5cyclictest17197-21ssh23:33:172
29540991810,6cyclictest9888-21sshd19:49:032
29540991810,6cyclictest9888-21sshd19:49:032
2954099180,16cyclictest4849-21find19:35:292
2954099180,16cyclictest4849-21find19:35:292
2954099180,16cyclictest4849-21find19:35:292
2953999189,7cyclictest695-21runrttasks19:50:301
2953999189,7cyclictest695-21runrttasks19:50:301
2953999188,8cyclictest3351-21fw_packets19:30:311
2953999188,8cyclictest3351-21fw_packets19:30:311
2953999181,3cyclictest131rcu_preempt21:05:351
2953999181,3cyclictest131rcu_preempt21:05:351
29539991811,5cyclictest31004-21sh21:35:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional