You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 21:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot4.osadl.org (updated Thu Nov 20, 2025 12:46:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3263123610,2sleep331120-21kworker/u8:407:05:383
3263123610,2sleep331120-21kworker/u8:407:05:383
3275723520,4sleep132756-21ntp_states07:05:481
3275723520,4sleep132756-21ntp_states07:05:481
103323300,3sleep20-21swapper/207:10:002
103323300,3sleep20-21swapper/207:09:592
111398304297,4cyclictest4702-21kworker/u8:12407:22:152
110998242233,6cyclictest4615-21kworker/u8:3707:22:151
109898213199,11cyclictest10-21ksoftirqd/007:22:150
111898191183,5cyclictest4581-21kworker/u8:207:22:153
111398153146,4cyclictest11566-21kworker/u8:5709:11:532
111398153146,4cyclictest11566-21kworker/u8:5709:11:522
111398116109,4cyclictest7070-21kworker/u8:1310:07:082
1109989383,5cyclictest32620-21kworker/u8:108:37:111
1109989083,4cyclictest11567-21kworker/u8:5809:11:521
1109989083,4cyclictest11567-21kworker/u8:5809:11:521
192862740,3sleep30-21swapper/309:20:233
1109987063,4cyclictest28900-21kworker/u8:4708:26:491
294992690,3sleep10-21swapper/109:40:461
229342650,15sleep20-21swapper/208:05:442
132682640,3sleep10-21swapper/107:40:271
1098986349,11cyclictest10-21ksoftirqd/008:37:110
16822590,3sleep30-21swapper/307:10:253
1113985952,4cyclictest32054-21kworker/u8:3312:12:052
1113985950,4cyclictest1067-21kworker/u8:2111:12:392
1098985848,6cyclictest1050ksoftirqd/009:11:520
1098985848,6cyclictest1050ksoftirqd/009:11:520
69072560,3sleep16889-21apt-get11:10:181
238762560,4sleep2241rcuc/211:45:282
80802550,2sleep10-21swapper/112:20:231
285542550,3sleep30-21swapper/310:46:073
285542550,3sleep30-21swapper/310:46:073
211942530,4sleep121154-21ssh09:25:021
15312530,8sleep2241rcuc/208:35:352
1109985345,5cyclictest7100-21kworker/u8:4210:07:081
109898521,26cyclictest209352diskmemload09:24:240
32442470,3sleep11613-21bootmenu12:10:351
295772470,3sleep20-21swapper/210:50:172
109898464,4cyclictest8353-21taskset08:55:330
109898462,38cyclictest0-21swapper/011:10:590
109898443,37cyclictest0-21swapper/009:45:020
109898441,39cyclictest0-21swapper/010:40:030
109898441,34cyclictest23708-21ssh09:30:150
109898434,34cyclictest13000-21irqrtprio07:40:250
109898433,36cyclictest17314-21ssh11:30:470
109898433,36cyclictest17314-21ssh11:30:470
1098984329,11cyclictest10-21ksoftirqd/008:26:490
109898432,31cyclictest26900-21cron10:45:000
109898431,8cyclictest18306-21ssh10:25:380
109898431,7cyclictest9261-21ssh11:15:170
1118984233,5cyclictest32654-21kworker/u8:3609:11:533
1118984233,5cyclictest32654-21kworker/u8:3609:11:523
1118984212,8cyclictest4079-21ntpq07:15:403
1118984212,8cyclictest4079-21ntpq07:15:403
1113984234,5cyclictest11535-21kworker/u8:2409:07:462
109898424,3cyclictest119650irq/29-4f00000.12:10:020
109898424,34cyclictest25513-21latency_hist08:15:020
109898424,34cyclictest25476-21kworker/u8:1211:50:000
109898424,32cyclictest1723-21irqrtprio07:10:260
109898423,4cyclictest6623-21latency_hist11:10:020
109898423,4cyclictest26751-21switchtime08:15:590
109898423,3cyclictest0-21swapper/008:30:120
109898422,36cyclictest16495-21taskset07:50:180
109898421,3cyclictest119650irq/29-4f00000.12:25:520
109898421,35cyclictest2676-21runrttasks08:40:140
109898421,35cyclictest119650irq/29-4f00000.12:10:270
109898416,31cyclictest30539-21proc_pri10:50:500
109898416,31cyclictest1-21init10:49:390
109898416,31cyclictest1-21init10:49:390
109898414,3cyclictest119650irq/29-4f00000.09:59:590
109898414,33cyclictest8683-21idleruntime-cro07:30:010
109898414,33cyclictest8683-21idleruntime-cro07:30:010
109898414,33cyclictest8632-21runrttasks12:20:330
109898413,32cyclictest0-21swapper/011:20:160
109898412,34cyclictest119650irq/29-4f00000.07:30:380
109898411,7cyclictest0-21swapper/010:00:300
109898411,6cyclictest0-21swapper/010:15:210
109898411,6cyclictest0-21swapper/009:15:500
109898411,36cyclictest0-21swapper/011:50:330
109898411,36cyclictest0-21swapper/011:50:330
288482400,7sleep225-21ksoftirqd/209:40:272
1118984033,4cyclictest32625-21kworker/u8:608:37:113
109898409,27cyclictest12786-21aten2.4-expect12:30:200
109898409,27cyclictest12786-21aten2.4-expect12:30:200
109898406,28cyclictest0-21swapper/009:35:190
109898405,4cyclictest17155-21diskmemload10:25:000
109898404,3cyclictest0-21swapper/008:05:150
109898404,32cyclictest21096-21ssh11:40:160
109898404,32cyclictest10656-21ssh10:10:290
109898404,32cyclictest10656-21ssh10:10:280
109898404,31cyclictest15578-21irqrtprio12:35:260
109898402,34cyclictest4922-21modprobe07:20:000
109898402,34cyclictest4922-21modprobe07:20:000
109898401,7cyclictest0-21swapper/009:45:190
109898401,6cyclictest119650irq/29-4f00000.08:45:230
109898401,6cyclictest0-21swapper/011:36:030
109898401,6cyclictest0-21swapper/011:36:030
109898401,35cyclictest15870-1kworker/u9:009:25:150
109898401,33cyclictest0-21swapper/008:00:150
109898401,33cyclictest0-21swapper/008:00:150
299922390,11sleep225-21ksoftirqd/211:58:502
111898395,10cyclictest2473-21ssh09:52:493
109898399,26cyclictest14774-21sh11:25:360
109898398,27cyclictest1537-21snmpd12:00:280
109898394,3cyclictest32167-21ssh10:55:270
109898394,31cyclictest29404-21sh11:55:500
109898393,32cyclictest6022-21diskmemload12:15:310
109898391,7cyclictest0-21swapper/009:05:430
109898391,6cyclictest0-21swapper/009:50:430
109898391,3cyclictest0-21swapper/008:20:210
109898391,3cyclictest0-21swapper/008:20:210
109898391,34cyclictest0-21swapper/011:00:470
73602380,3chrt0-21swapper/207:25:242
73602380,3chrt0-21swapper/207:25:242
111898386,12cyclictest12483-21date10:15:023
111898386,12cyclictest12483-21date10:15:023
111898385,11cyclictest5401-21ntpq08:45:363
110998387,14cyclictest121rcu_preempt11:20:401
109898384,30cyclictest12370-21munin-plugin-st07:40:000
109898383,31cyclictest19835-21aten2_rfpower_c10:30:190
109898382,3cyclictest0-21swapper/008:55:010
109898382,32cyclictest12325-21kworker/0:009:00:260
109898381,4cyclictest7884-21taskset10:05:140
109898381,32cyclictest0-21swapper/007:55:110
111898379,9cyclictest1568-21runrttasks08:50:143
111898379,9cyclictest13932-21taskset07:41:263
111898374,12cyclictest16587-21df07:50:193
109898371,7cyclictest15472-21awk07:45:500
6902360,22sleep225-21ksoftirqd/209:50:082
153732360,23sleep225-21ksoftirqd/211:26:152
111898368,11cyclictest20434-21seq11:38:083
111898368,11cyclictest20434-21seq11:38:083
111898368,10cyclictest13419-21seq11:23:283
111898368,10cyclictest10190-21runrttasks07:31:063
1118983610,8cyclictest21459-21seq08:02:273
1118983610,8cyclictest21459-21seq08:02:273
111398362,5cyclictest0-21swapper/209:25:142
83242350,5sleep21613-21bootmenu08:55:282
77652350,22sleep225-21ksoftirqd/208:52:242
231232350,9sleep119-21ksoftirqd/111:44:541
197292350,4sleep10-21swapper/111:36:231
197292350,4sleep10-21swapper/111:36:231
188272350,2sleep10-21swapper/109:20:011
123402350,4sleep112336-21idleruntime09:05:231
111898358,9cyclictest16858-21runrttasks10:23:403
111898357,10cyclictest23605-21runrttasks10:36:283
111898356,8cyclictest0-21swapper/311:57:323
111898355,11cyclictest7255-21ssh10:02:563
111898355,10cyclictest6312-21ntpq12:15:423
1118983510,8cyclictest11890-21runrttasks12:27:093
111398359,4cyclictest121rcu_preempt10:45:462
111398359,4cyclictest121rcu_preempt10:45:452
111398352,5cyclictest121rcu_preempt12:15:172
109898354,26cyclictest1170-21latency_hist07:10:010
109898354,26cyclictest1170-21latency_hist07:10:010
284262340,18sleep127062-21kworker/u8:508:20:471
284262340,18sleep127062-21kworker/u8:508:20:471
111898348,9cyclictest24305-21ssh11:45:423
111898347,9cyclictest9658-21runrttasks10:08:083
111898346,9cyclictest5374-21sed10:00:023
111898344,9cyclictest6264-21ssh11:09:263
111398347,6cyclictest121rcu_preempt11:35:142
111398347,6cyclictest121rcu_preempt11:35:132
111398341,5cyclictest121rcu_preempt10:20:142
111898338,8cyclictest2426-21ssh12:09:043
111898335,9cyclictest29688-21df_inode08:25:363
111898334,9cyclictest26278-21df_inode09:35:183
111898334,9cyclictest19375-21ssh10:29:083
111898334,9cyclictest10276-21cpu09:00:193
111398331,8cyclictest121rcu_preempt10:25:142
111398331,5cyclictest121rcu_preempt12:20:122
110998331,5cyclictest5528-21ssh10:00:161
251812320,9sleep225-21ksoftirqd/209:31:492
239542320,18sleep11613-21bootmenu10:38:221
111898325,8cyclictest29527-21ssh10:50:133
111898324,8cyclictest10680-21ssh11:16:233
111398325,6cyclictest1094-21cyclictest11:05:402
110998326,4cyclictest121rcu_preempt09:25:511
110998326,11cyclictest121rcu_preempt08:30:251
98862310,3sleep20-21swapper/211:15:322
63182310,3chrt0-21swapper/208:50:022
326432310,10sleep1121rcu_preempt09:48:331
261632310,3sleep20-21swapper/211:50:322
261632310,3sleep20-21swapper/211:50:322
111898318,6cyclictest8592-21runrttasks07:29:173
111898318,6cyclictest8592-21runrttasks07:29:173
111898317,6cyclictest165672kworker/u8:1012:38:533
111898314,8cyclictest27799-21diskstats08:20:253
111898314,8cyclictest27799-21diskstats08:20:253
111898314,11cyclictest1966-21aten2_rfpower_e11:00:193
1118983118,10cyclictest21080-21apt-get11:40:153
1118983116,11cyclictest29807-21ssh09:41:033
1118983116,11cyclictest22252-21/usr/sbin/munin08:05:253
111398313,6cyclictest367-21ntp_states10:55:462
111398313,4cyclictest121rcu_preempt11:35:012
111398313,4cyclictest121rcu_preempt11:35:012
111398312,4cyclictest121rcu_preempt07:45:202
111398312,14cyclictest31135-21hddtemp_smartct09:45:292
1113983120,6cyclictest25-21ksoftirqd/210:40:322
111398311,5cyclictest121rcu_preempt09:35:252
110998312,6cyclictest15313-21ntpq07:45:381
59472300,4sleep119-21ksoftirqd/108:47:441
50352300,3sleep20-21swapper/209:58:592
38532300,3sleep20-21swapper/211:04:162
241052300,2sleep118304-21kworker/u8:2609:30:261
192022300,2sleep115843-21kworker/u8:607:55:391
19092300,3sleep20-21swapper/212:06:082
177132300,3sleep20-21swapper/207:52:172
144092300,3sleep20-21swapper/212:32:512
144092300,3sleep20-21swapper/212:32:512
111898308,8cyclictest1568-21runrttasks08:10:523
111898302,9cyclictest13652-21diskmemload11:31:583
111898302,9cyclictest13652-21diskmemload11:31:573
1118983017,9cyclictest6889-21apt-get11:10:133
1118983016,10cyclictest16149-21cat11:30:013
1118983014,12cyclictest18675-21idleruntime-cro09:20:003
1118983014,12cyclictest1445-21cron12:20:023
1118983013,12cyclictest12677-21grep09:05:283
111898301,17cyclictest0-21swapper/311:00:013
111398302,4cyclictest121rcu_preempt12:29:592
111398301,6cyclictest121rcu_preempt08:25:542
111398301,5cyclictest121rcu_preempt12:00:162
110998306,7cyclictest121rcu_preempt08:05:221
110998305,10cyclictest121rcu_preempt12:05:181
110998304,12cyclictest121rcu_preempt10:14:461
110998304,12cyclictest121rcu_preempt10:14:461
110998301,6cyclictest12465-21sed07:40:021
110862300,3sleep20-21swapper/209:00:502
327532290,2sleep2121rcu_preempt08:33:032
298452290,2sleep1181rcuc/111:58:051
135602290,4sleep225-21ksoftirqd/211:24:112
1118982916,9cyclictest26714-21ssh10:43:593
1118982914,11cyclictest25788-21taskset08:15:223
1118982912,13cyclictest20903-21ssh10:30:463
1118982911,14cyclictest18455-21hddtemp_smartct07:55:193
111398294,3cyclictest121rcu_preempt08:40:252
111398293,5cyclictest121rcu_preempt10:00:382
111398293,3cyclictest121rcu_preempt10:30:492
111398293,3cyclictest121rcu_preempt07:35:322
111398293,3cyclictest121rcu_preempt07:10:262
111398292,4cyclictest121rcu_preempt12:40:012
111398291,6cyclictest121rcu_preempt08:10:472
111398291,4cyclictest121rcu_preempt10:15:172
111398291,4cyclictest121rcu_preempt09:20:252
110998295,5cyclictest121rcu_preempt07:10:261
110998291,6cyclictest771-21ssh10:55:571
110998291,13cyclictest121rcu_preempt09:50:501
33222280,3sleep10-21swapper/109:55:191
269432280,3sleep30-21swapper/311:50:533
269432280,3sleep30-21swapper/311:50:533
121722280,2chrt25-21ksoftirqd/210:14:232
121722280,2chrt25-21ksoftirqd/210:14:232
111898286,5cyclictest1568-21runrttasks08:58:143
1118982814,11cyclictest31411-21sshd12:00:403
1118982814,11cyclictest21937-21/usr/sbin/munin09:25:283
1118982814,10cyclictest30864-21cut09:45:213
1118982814,10cyclictest2919-21/usr/sbin/munin08:40:183
1118982814,10cyclictest24974-21/usr/sbin/munin09:30:483
1118982813,11cyclictest12850-21/usr/sbin/munin10:15:203
1118982813,10cyclictest31790-21sed08:30:243
1118982812,12cyclictest11561-21ntpq07:35:353
111398283,5cyclictest121rcu_preempt08:00:322
111398283,5cyclictest121rcu_preempt08:00:322
111398283,4cyclictest121rcu_preempt07:15:352
111398283,4cyclictest121rcu_preempt07:15:352
111398283,3cyclictest121rcu_preempt08:25:012
111398283,3cyclictest121rcu_preempt08:25:002
111398283,3cyclictest121rcu_preempt07:55:252
111398282,4cyclictest23113-21ssh10:35:432
111398282,4cyclictest121rcu_preempt11:40:172
111398282,11cyclictest121rcu_preempt09:15:182
110998286,5cyclictest121rcu_preempt10:50:031
110998283,4cyclictest121rcu_preempt09:35:231
110998282,5cyclictest121rcu_preempt09:00:001
110998282,4cyclictest121rcu_preempt10:15:371
86222270,12sleep14697-21kworker/u8:11907:29:381
86222270,12sleep14697-21kworker/u8:11907:29:381
48052270,2sleep14687-21kworker/u8:10907:18:331
48052270,2sleep14687-21kworker/u8:10907:18:331
269442270,3sleep20-21swapper/208:16:202
1118982714,10cyclictest15586-21ls07:45:523
1118982713,11cyclictest2591-21apt-get12:10:123
1118982713,11cyclictest13106-21grep12:30:263
1118982713,11cyclictest13106-21grep12:30:263
111398272,4cyclictest121rcu_preempt07:30:332
111398271,3cyclictest121rcu_preempt07:40:322
110998275,6cyclictest121rcu_preempt11:00:501
110998274,4cyclictest121rcu_preempt12:30:471
110998274,4cyclictest121rcu_preempt12:30:471
110998274,4cyclictest121rcu_preempt11:25:261
110998273,4cyclictest121rcu_preempt10:21:131
1109982721,4cyclictest23330-21kworker/u8:4608:12:261
110998271,5cyclictest21070-21ssh10:30:481
110998271,10cyclictest121rcu_preempt11:05:271
110998262,5cyclictest121rcu_preempt11:50:331
110998262,5cyclictest121rcu_preempt11:50:331
110998261,7cyclictest7350-21processes08:50:491
110998261,6cyclictest18298-21ntpq10:25:371
110998261,4cyclictest121rcu_preempt12:35:481
110998261,3cyclictest109702kworker/u8:311:45:551
286242250,12sleep119-21ksoftirqd/110:46:301
286242250,12sleep119-21ksoftirqd/110:46:301
270822250,12sleep119-21ksoftirqd/108:17:481
180382250,3sleep1121rcu_preempt11:32:481
180382250,3sleep1121rcu_preempt11:32:471
111992250,12sleep119-21ksoftirqd/112:25:531
110998254,4cyclictest121rcu_preempt12:15:281
110998254,4cyclictest121rcu_preempt12:00:361
110998251,7cyclictest1094-21cyclictest11:15:511
110998251,5cyclictest121rcu_preempt08:40:131
114022240,11sleep119-21ksoftirqd/109:01:201
110998244,4cyclictest121rcu_preempt10:40:431
110998243,9cyclictest121rcu_preempt07:30:361
110998242,9cyclictest121rcu_preempt07:50:191
110998241,3cyclictest0-21swapper/108:00:281
110998241,3cyclictest0-21swapper/108:00:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional