You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 12:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot4.osadl.org (updated Sat Nov 29, 2025 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2431323020,1sleep323028-21kworker/u8:419:05:283
2439722720,1sleep2241rcuc/219:05:362
2482422530,2sleep10-21swapper/119:06:221
2527098166160,4cyclictest24590-21kworker/u8:7220:37:053
2526998155147,5cyclictest19371-21kworker/u8:1521:56:572
721421330,5sleep37170-21ssh00:31:003
2526498132121,8cyclictest10-21ksoftirqd/021:56:570
2526998120113,5cyclictest24893-21kworker/u8:2019:12:062
2527098106101,3cyclictest26495-21kworker/u8:5020:42:113
252699810497,4cyclictest7426-21kworker/u8:619:52:012
25264989588,4cyclictest1050ksoftirqd/019:12:060
25266989285,5cyclictest24093-21kworker/u8:8021:56:571
25269988778,5cyclictest20767-21kworker/u8:520:32:022
25264988371,9cyclictest10-21ksoftirqd/019:52:020
25269988073,5cyclictest24589-21kworker/u8:7120:37:042
219842620,3sleep220990-21kworker/u8:322:50:192
25264985953,3cyclictest1050ksoftirqd/020:32:020
25264985947,9cyclictest10-21ksoftirqd/020:37:050
25266985751,4cyclictest24940-21kworker/u8:6719:12:051
25270984741,3cyclictest10769-21kworker/u8:223:56:493
25270984511,5cyclictest121rcu_preempt21:50:163
23412430,2sleep32342-21ssh00:22:353
9122420,6sleep1191ksoftirqd/123:11:581
268872420,2sleep10-21swapper/123:00:031
2526698421,22cyclictest1158-21/usr/sbin/munin20:55:391
25266984135,4cyclictest7491-21kworker/u8:7319:52:021
2526698411,19cyclictest25017-21/usr/sbin/munin21:50:251
253292400,3sleep28691-21kworker/u8:822:55:502
25270984035,3cyclictest19176-21kworker/u8:321:56:573
2526698402,20cyclictest4223-21/usr/sbin/munin19:40:091
25266984014,23cyclictest1029-21irqrtprio00:20:241
2526698401,24cyclictest21453-21/usr/sbin/munin21:40:451
2526698401,18cyclictest18345-21ntpq20:15:311
2526698391,22cyclictest11561-21/usr/sbin/munin23:35:271
25270983831,3cyclictest12127-21kworker/u8:1221:36:283
2526698389,11cyclictest5851-21apt-config00:30:031
2526698388,12cyclictest2081-21runrttasks21:17:091
2526698388,12cyclictest2081-21runrttasks21:17:091
2526698381,24cyclictest4020-21ssh22:12:081
2526698381,20cyclictest2081-21runrttasks19:26:151
2526698381,16cyclictest9485-21find00:35:441
2526698376,14cyclictest1022-21irqrtprio19:30:221
2526698371,30cyclictest29819-21apt-get22:00:141
2526698371,24cyclictest9173-21ssh22:24:001
2526698371,21cyclictest32021-21chrt20:51:491
2526698371,21cyclictest32021-21chrt20:51:491
2526698371,20cyclictest12198-21seq21:22:361
2526698371,19cyclictest2976-21/usr/sbin/munin19:35:241
2526698371,17cyclictest31571-21ntp_states00:15:411
2526698371,16cyclictest10542-21ssh22:25:331
2526698361,17cyclictest4119-21seq21:05:181
325952350,10sleep331-21ksoftirqd/319:29:323
64492340,2sleep10-21swapper/123:25:011
26532340,11sleep225-21ksoftirqd/223:15:302
2527098343,5cyclictest121rcu_preempt23:50:173
25270983429,3cyclictest9349-21kworker/u8:520:16:443
2526698341,28cyclictest16580-21/usr/sbin/munin23:45:201
2526698341,18cyclictest23167-21taskset20:30:091
2526698341,17cyclictest16928-21taskset21:32:261
2526698341,15cyclictest6965-21ntpq19:45:361
2526698341,14cyclictest27380-21ssh00:07:001
2526698341,14cyclictest16238-21ssh22:37:181
2526698341,14cyclictest0-21swapper/123:15:361
2526698341,13cyclictest12851-21gcc20:00:491
25269983327,3cyclictest3689-21kworker/u8:5019:41:462
2526698332,15cyclictest26171-21taskset22:58:151
2526698331,28cyclictest21934-21ntpq23:55:331
2526698331,16cyclictest29528-21vmstat00:10:431
2526698331,16cyclictest24853-21taskset00:01:091
2526698331,16cyclictest24853-21taskset00:01:091
2526698331,16cyclictest20746-21ssh23:54:431
2526698331,16cyclictest1071-21ssh22:05:541
2526698331,15cyclictest5410-21ssh00:29:521
2526698331,15cyclictest24274-21seq21:47:371
2526698331,15cyclictest24274-21seq21:47:371
2526698331,12cyclictest6457-21ssh22:17:291
2526698323,19cyclictest264482kworker/u8:020:48:201
2526698321,28cyclictest15108-21timerandwakeup23:40:421
2526698321,15cyclictest22772-21taskset20:28:111
2526698321,15cyclictest22772-21taskset20:28:111
2526698321,13cyclictest29976-21gcc19:20:281
2526698321,13cyclictest13235-21ntpq22:30:521
2526698321,12cyclictest25352-21diskstats20:35:231
2526698321,11cyclictest14745-21ssh21:28:301
234722320,10sleep331-21ksoftirqd/300:00:143
234722320,10sleep331-21ksoftirqd/300:00:143
2527098317,5cyclictest121rcu_preempt22:20:153
2527098316,4cyclictest121rcu_preempt22:55:023
2526698319,19cyclictest22078-21/usr/sbin/munin22:50:221
2526698311,17cyclictest7916-21taskset23:25:531
2526698311,12cyclictest18910-21ssh22:43:531
2526698311,12cyclictest18910-21ssh22:43:531
25264983122,5cyclictest10-21ksoftirqd/022:15:140
277652300,6sleep2251ksoftirqd/220:40:282
2527098307,15cyclictest121rcu_preempt22:05:003
2527098303,3cyclictest121rcu_preempt22:30:293
2526998302,5cyclictest25-21ksoftirqd/220:20:262
2526698309,18cyclictest21079-21latency_hist20:25:021
2526698306,18cyclictest268322kworker/u8:1219:16:321
2526698302,23cyclictest7236-21chrt21:11:431
2526698302,23cyclictest7236-21chrt21:11:431
2526698301,25cyclictest27291-21/usr/sbin/munin20:40:211
2526698301,25cyclictest18897-21grep21:35:451
2526698301,25cyclictest10988-21/usr/sbin/munin19:55:421
2526698301,15cyclictest2028-21apt-get21:00:121
2526698301,14cyclictest16977-21runrttasks20:10:571
25266983011,15cyclictest29759-21if_err_eth123:05:281
25264983023,4cyclictest10-21ksoftirqd/023:10:170
155942300,8sleep225-21ksoftirqd/223:42:352
91892290,2sleep30-21swapper/323:30:183
61312290,2sleep20-21swapper/223:24:022
25270982914,3cyclictest31-21ksoftirqd/322:05:193
2526698291,25cyclictest8979-21awk23:30:021
2526698291,24cyclictest19400-21apt-get22:45:161
2526698291,24cyclictest19400-21apt-get22:45:151
2526698291,24cyclictest14976-21/usr/sbin/munin20:05:491
96662280,2sleep30-21swapper/319:55:013
72382280,3chrt0-21swapper/221:11:442
72382280,3chrt0-21swapper/221:11:442
25270982813,3cyclictest31-21ksoftirqd/319:30:303
25270982812,5cyclictest31-21ksoftirqd/300:05:263
239772280,3sleep323970-21ntpq20:30:323
222202280,16sleep30-21swapper/321:44:433
214152280,16sleep220994-21kworker/u8:722:49:272
214152280,16sleep220994-21kworker/u8:722:49:262
104422280,1sleep39685-21kworker/u8:1921:20:013
104422280,1sleep39685-21kworker/u8:1921:20:013
87142270,2sleep20-21swapper/221:15:262
87142270,2sleep20-21swapper/221:15:252
296372274,5sleep2121rcu_preempt20:45:322
2527098277,3cyclictest121rcu_preempt21:20:423
2527098277,13cyclictest31-21ksoftirqd/322:50:293
2527098275,3cyclictest121rcu_preempt20:05:483
2527098271,4cyclictest0-21swapper/319:55:423
25270982713,3cyclictest31-21ksoftirqd/321:45:293
25270982713,3cyclictest31-21ksoftirqd/321:45:293
25270982712,3cyclictest31-21ksoftirqd/323:10:183
25270982710,3cyclictest31-21ksoftirqd/322:43:103
25270982710,3cyclictest31-21ksoftirqd/322:43:103
2526998277,4cyclictest121rcu_preempt00:35:302
2526998275,5cyclictest25-21ksoftirqd/222:45:002
2526998275,5cyclictest25-21ksoftirqd/222:45:002
2526998271,4cyclictest17726-21tail20:15:202
54292260,2sleep30-21swapper/300:29:593
321212260,2sleep3311ksoftirqd/320:52:503
321212260,2sleep3311ksoftirqd/320:52:503
309782260,14sleep223412-21kworker/u8:1723:07:082
2527098267,4cyclictest121rcu_preempt21:10:023
2527098265,5cyclictest121rcu_preempt22:35:313
2527098264,3cyclictest121rcu_preempt00:38:423
2527098261,5cyclictest13946-21/usr/sbin/munin23:40:253
2527098261,5cyclictest121rcu_preempt23:45:353
25270982614,3cyclictest31-21ksoftirqd/323:25:203
25270982614,3cyclictest31-21ksoftirqd/322:46:223
25270982614,3cyclictest31-21ksoftirqd/322:46:223
25270982613,3cyclictest31-21ksoftirqd/320:10:323
25270982612,3cyclictest31-21ksoftirqd/319:40:283
25270982611,8cyclictest31-21ksoftirqd/321:30:213
25270982611,3cyclictest31-21ksoftirqd/323:09:083
25270982611,3cyclictest31-21ksoftirqd/322:27:283
2526998267,4cyclictest121rcu_preempt22:10:382
2526998265,4cyclictest121rcu_preempt21:25:162
2526998264,5cyclictest25-21ksoftirqd/221:10:022
2526998263,4cyclictest121rcu_preempt23:00:042
2526998262,3cyclictest121rcu_preempt22:25:292
2526998261,7cyclictest11497-21awk20:00:002
2526998261,5cyclictest18445-21modprobe23:50:012
2526998261,4cyclictest0-21swapper/221:50:262
2526998261,4cyclictest0-21swapper/220:00:132
2526498264,8cyclictest118150irq/29-4f00000.00:25:160
25264982619,4cyclictest10-21ksoftirqd/023:20:160
207502260,2sleep30-21swapper/320:21:383
17022260,2sleep20-21swapper/222:07:232
2527098259,3cyclictest31-21ksoftirqd/319:15:173
2527098255,3cyclictest121rcu_preempt21:25:163
2527098251,3cyclictest121rcu_preempt23:00:353
2527098251,3cyclictest121rcu_preempt22:15:373
2527098251,3cyclictest121rcu_preempt20:00:453
25270982513,3cyclictest31-21ksoftirqd/321:10:333
25270982513,3cyclictest31-21ksoftirqd/321:10:323
25270982512,3cyclictest31-21ksoftirqd/323:37:303
2527098250,5cyclictest0-21swapper/320:55:183
2527098250,12cyclictest0-21swapper/300:15:073
2526998256,3cyclictest121rcu_preempt00:05:082
2526998251,4cyclictest24676-21kworker/2:120:55:002
2526998251,4cyclictest24676-21kworker/2:120:55:002
25269982514,3cyclictest96792kworker/u8:1421:20:342
217522250,1sleep219371-21kworker/u8:1521:42:132
7782240,2sleep20-21swapper/223:11:232
73972240,2sleep20-21swapper/219:46:442
67522240,7sleep2241rcuc/222:19:012
299022240,1sleep329825-21kworker/u8:100:12:383
2527098244,5cyclictest121rcu_preempt20:25:173
2527098244,5cyclictest121rcu_preempt20:25:173
2527098243,4cyclictest121rcu_preempt20:45:133
2527098241,5cyclictest5111-21ntpq23:20:353
25270982413,3cyclictest31-21ksoftirqd/319:35:183
25270982412,4cyclictest31-21ksoftirqd/322:10:383
25270982411,3cyclictest31-21ksoftirqd/323:19:393
2527098240,3cyclictest121rcu_preempt19:20:203
2526998242,4cyclictest121rcu_preempt00:20:402
2526998242,3cyclictest121rcu_preempt00:13:092
2526998241,3cyclictest121rcu_preempt23:30:562
2526998241,3cyclictest0-21swapper/200:30:022
2526998241,3cyclictest0-21swapper/200:03:142
2526998241,3cyclictest0-21swapper/200:03:142
2526998240,3cyclictest0-21swapper/219:25:222
2526498242,8cyclictest309422/usr/sbin/munin20:50:200
2526498242,8cyclictest309422/usr/sbin/munin20:50:190
25264982413,6cyclictest3959-21sendmail23:20:000
160682240,2sleep28696-21kworker/u8:1422:36:232
35242230,12sleep225-21ksoftirqd/221:03:482
304562230,1sleep224950-21kworker/u8:7719:20:592
2527098231,5cyclictest121rcu_preempt19:10:393
2527098231,3cyclictest121rcu_preempt19:45:233
2526998233,3cyclictest121rcu_preempt22:20:232
2526998231,5cyclictest121rcu_preempt19:35:302
2526998231,4cyclictest121rcu_preempt23:25:362
2526998231,4cyclictest0-21swapper/223:35:392
2526998231,3cyclictest121rcu_preempt19:15:202
2526998231,3cyclictest0-21swapper/221:30:232
25264982317,3cyclictest10-21ksoftirqd/022:35:310
2526498231,6cyclictest6733-21ps21:10:390
2526498231,6cyclictest6733-21ps21:10:380
153622230,2sleep20-21swapper/220:08:472
33152220,2sleep30-21swapper/321:01:113
2526998222,3cyclictest121rcu_preempt20:55:182
2526998222,3cyclictest121rcu_preempt00:30:122
2526998221,5cyclictest28960-21kworker/u8:422:30:342
2526998221,4cyclictest24733-21ssh21:50:002
2526998221,4cyclictest24733-21ssh21:50:002
2526998220,3cyclictest121rcu_preempt21:35:362
2526998220,3cyclictest121rcu_preempt00:15:292
2526498225,3cyclictest32703-21aten2_rfpower_v22:05:340
2526498223,9cyclictest25666-21kernelversion21:50:260
25264982216,3cyclictest118150irq/29-4f00000.19:20:400
213242220,10sleep225-21ksoftirqd/220:25:152
213242220,10sleep225-21ksoftirqd/220:25:152
2526998213,3cyclictest121rcu_preempt22:00:282
2526998211,6cyclictest19553-21ntpq23:50:342
2526998211,3cyclictest121rcu_preempt23:55:142
2526998210,3cyclictest121rcu_preempt19:35:012
2526498218,10cyclictest11473-21cron19:59:590
2526498215,7cyclictest17724-21aten2_rfpower_v20:15:190
25264982116,3cyclictest118150irq/29-4f00000.20:42:180
25264982116,3cyclictest118150irq/29-4f00000.00:10:410
25264982113,6cyclictest12085-21aten2_rfpower_f20:00:310
25264982111,3cyclictest10-21ksoftirqd/021:27:560
2526998202,3cyclictest121rcu_preempt20:10:192
2526498207,3cyclictest19831-21irqrtprio20:20:240
2526498206,6cyclictest121rcu_preempt00:35:270
2526498205,9cyclictest0-21swapper/021:06:580
2526498203,5cyclictest156322ssh23:42:520
2526498203,11cyclictest0-21swapper/022:45:120
2526498203,11cyclictest0-21swapper/022:45:120
2526498202,10cyclictest23879-21cron22:54:590
25264982015,3cyclictest25562-21ssh00:05:010
25264982015,3cyclictest25562-21ssh00:05:010
25264982015,3cyclictest118150irq/29-4f00000.21:05:020
25264982015,3cyclictest10-21ksoftirqd/022:55:440
25264982013,5cyclictest5499-21diskmemload23:25:310
25264982012,6cyclictest8922-21sh21:15:320
25264982012,6cyclictest8922-21sh21:15:310
25264982012,6cyclictest27735-21latency23:00:390
25264982012,6cyclictest2044-21snmpd20:06:470
2526498199,8cyclictest7624-21awk22:20:250
2526498197,10cyclictest10-21ksoftirqd/023:45:140
25264981914,3cyclictest31484-21ntpq00:15:340
25264981914,3cyclictest2044-21snmpd22:27:170
25264981914,3cyclictest2044-21snmpd19:40:200
25264981914,3cyclictest12495-21ntpq23:35:410
25264981914,3cyclictest118150irq/29-4f00000.21:48:510
25264981914,3cyclictest118150irq/29-4f00000.21:48:510
25264981914,3cyclictest10-21ksoftirqd/000:20:160
25264981913,3cyclictest10-21ksoftirqd/000:05:370
25264981912,5cyclictest2081-21runrttasks20:25:310
25264981912,5cyclictest2081-21runrttasks20:25:310
25264981912,5cyclictest10-21ksoftirqd/020:55:150
25264981911,6cyclictest2081-21runrttasks22:30:420
25264981911,5cyclictest18064-21mailstats22:40:410
25264981911,5cyclictest18064-21mailstats22:40:400
25264981910,6cyclictest30184-21if_eth022:00:240
25264981910,5cyclictest15653-21if_eth021:30:240
25264981910,3cyclictest10-21ksoftirqd/000:30:250
2526498189,7cyclictest21725-21latency23:55:260
25264981813,3cyclictest3105-21/usr/sbin/munin19:35:250
25264981813,3cyclictest19550-21/usr/sbin/munin23:50:330
25264981813,3cyclictest18090-21ssh21:35:270
25264981813,3cyclictest10-21ksoftirqd/021:20:240
25264981813,3cyclictest10-21ksoftirqd/020:10:330
25264981813,3cyclictest10-21ksoftirqd/019:15:140
25264981812,3cyclictest10-21ksoftirqd/019:25:350
25264981811,4cyclictest10-21ksoftirqd/023:30:150
2526498181,12cyclictest0-21swapper/019:35:000
25264981810,6cyclictest29670-21sh23:05:220
25264981810,6cyclictest2760-21forks22:10:250
25264981712,3cyclictest7749-21latency_hist19:50:010
25264981712,3cyclictest10-21ksoftirqd/020:45:170
25264981710,5cyclictest20231-21telnet21:40:170
2526498156,7cyclictest25367-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional