You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 20:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Sat Nov 29, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1425721690,4sleep3829399cyclictest11:14:583
1935721670,4sleep1828299cyclictest09:51:251
988721640,3sleep3829399cyclictest11:09:363
2991921600,3sleep1828299cyclictest13:26:521
1773911210,8ptp4l26505-21kworker/3:008:16:573
84262650,3sleep10-21swapper/111:46:431
58182630,4sleep2311rcuc/211:41:592
167302620,2sleep00-21swapper/008:36:410
149422570,2sleep20-21swapper/211:53:232
8293995626,9cyclictest41-21ksoftirqd/312:36:303
131112560,1sleep10-21swapper/113:43:381
8293995524,5cyclictest41-21ksoftirqd/311:56:333
8293995524,5cyclictest41-21ksoftirqd/311:56:333
8293995522,8cyclictest41-21ksoftirqd/311:46:283
8293995522,8cyclictest41-21ksoftirqd/310:11:453
829399546,4cyclictest12089-21munin-run11:51:283
261992540,2sleep10-21swapper/111:28:171
8293995323,10cyclictest41-21ksoftirqd/312:26:423
8293995321,8cyclictest41-21ksoftirqd/312:20:083
278062530,2sleep10-21swapper/110:51:471
203842530,2sleep30-21swapper/311:21:483
283782520,3sleep3829399cyclictest11:31:443
829399517,8cyclictest9881-21cron09:31:253
829399514,9cyclictest25504-21fschecks_time12:06:463
8293995122,9cyclictest41-21ksoftirqd/313:18:313
8293995122,10cyclictest41-21ksoftirqd/312:58:063
8293995122,10cyclictest41-21ksoftirqd/311:18:563
116672510,2sleep20-21swapper/211:50:382
829399506,10cyclictest41-21ksoftirqd/313:51:313
829399506,10cyclictest41-21ksoftirqd/312:01:553
829399506,10cyclictest41-21ksoftirqd/312:01:553
8293995023,4cyclictest41-21ksoftirqd/308:33:023
8293995023,4cyclictest41-21ksoftirqd/308:33:023
8293995022,9cyclictest41-21ksoftirqd/311:56:103
8293995022,5cyclictest41-21ksoftirqd/309:12:323
8293995021,7cyclictest41-21ksoftirqd/311:39:573
8293995021,5cyclictest41-21ksoftirqd/313:15:123
8293995021,10cyclictest41-21ksoftirqd/313:41:443
8293995021,10cyclictest41-21ksoftirqd/308:56:403
8293995019,10cyclictest41-21ksoftirqd/312:44:063
8293995017,9cyclictest41-21ksoftirqd/313:38:143
60372504,34sleep20-21swapper/208:16:262
829399496,5cyclictest41-21ksoftirqd/309:26:253
829399493,7cyclictest41-21ksoftirqd/310:46:273
8293994922,8cyclictest41-21ksoftirqd/312:53:033
8293994921,9cyclictest41-21ksoftirqd/313:26:473
8293994921,6cyclictest41-21ksoftirqd/310:06:263
8293994921,5cyclictest41-21ksoftirqd/308:46:353
8293994920,10cyclictest41-21ksoftirqd/308:41:493
8293994918,8cyclictest41-21ksoftirqd/312:37:183
829399486,8cyclictest26505-21kworker/3:009:06:423
829399485,11cyclictest41-21ksoftirqd/308:26:273
829399485,11cyclictest41-21ksoftirqd/308:26:273
829399484,11cyclictest41-21ksoftirqd/313:31:563
8293994821,8cyclictest41-21ksoftirqd/310:36:113
8293994821,8cyclictest41-21ksoftirqd/308:39:403
8293994820,8cyclictest41-21ksoftirqd/311:00:413
8293994820,5cyclictest41-21ksoftirqd/309:38:363
8293994819,9cyclictest41-21ksoftirqd/311:02:103
8293994819,8cyclictest41-21ksoftirqd/310:25:553
8293994819,7cyclictest41-21ksoftirqd/310:17:313
8293994819,7cyclictest41-21ksoftirqd/310:17:313
8293994817,7cyclictest41-21ksoftirqd/309:01:393
828299482,39cyclictest22616-21apt-get10:46:411
829399476,8cyclictest41-21ksoftirqd/313:06:543
8293994733,3cyclictest41-21ksoftirqd/309:31:453
8293994717,7cyclictest41-21ksoftirqd/310:53:423
829399470,7cyclictest41-21ksoftirqd/312:21:473
829399467,7cyclictest41-21ksoftirqd/311:26:473
829399466,8cyclictest41-21ksoftirqd/310:26:573
829399465,8cyclictest41-21ksoftirqd/308:21:463
829399465,8cyclictest41-21ksoftirqd/308:21:463
829399465,7cyclictest41-21ksoftirqd/310:46:453
8293994618,8cyclictest41-21ksoftirqd/308:52:563
829399455,7cyclictest27117-21awk12:46:483
829399452,6cyclictest20972-21/usr/sbin/munin09:51:503
8293994520,9cyclictest41-21ksoftirqd/310:09:083
8293994518,9cyclictest41-21ksoftirqd/313:24:483
8293994517,9cyclictest41-21ksoftirqd/313:05:543
56922450,2sleep20-21swapper/213:36:452
829399437,6cyclictest18129-21idleruntime-cro10:41:273
829399434,8cyclictest22148-21tr09:56:403
8293994334,3cyclictest41-21ksoftirqd/309:41:473
829399422,7cyclictest41-21ksoftirqd/309:21:253
829399420,7cyclictest41-21ksoftirqd/309:46:483
829399411,8cyclictest41-21ksoftirqd/312:11:593
810124122,9sleep10-21swapper/108:20:291
828799405,13cyclictest30126-21wc10:11:472
8282994013,24cyclictest25718-21cron08:56:241
8287993932,4cyclictest10594-21sh08:26:232
8287993932,4cyclictest10594-21sh08:26:232
828799381,6cyclictest0-21swapper/209:16:382
193412380,13sleep0101ktimersoftd/010:41:460
828799372,5cyclictest0-21swapper/210:06:392
828799372,14cyclictest12272-21needreboot11:11:492
8287993720,14cyclictest33-21ksoftirqd/209:37:412
8287993714,20cyclictest14741-21idleruntime-cro13:46:312
8282993720,14cyclictest25-21ksoftirqd/108:35:091
8282993720,14cyclictest25-21ksoftirqd/108:35:091
828299371,5cyclictest0-21swapper/112:31:421
8282993712,22cyclictest848-21sh10:58:281
8282993711,22cyclictest7723-21wc10:26:571
8282993711,22cyclictest12836-21fschecks_count11:51:461
828299370,7cyclictest0-21swapper/111:06:241
828799362,7cyclictest0-21swapper/210:41:392
828799362,6cyclictest0-21swapper/210:16:412
828799362,6cyclictest0-21swapper/210:16:412
8287993613,4cyclictest21918-21date12:41:302
8287993612,3cyclictest28597-21ls09:01:402
8287993610,4cyclictest14134-21ls08:31:402
8287993610,4cyclictest14134-21ls08:31:402
8282993620,13cyclictest25-21ksoftirqd/109:21:301
8282993614,19cyclictest1760-21runrttasks12:16:061
8282993613,20cyclictest11586-21df13:06:461
8282993610,22cyclictest25943-21/usr/sbin/munin08:56:451
8287993514,4cyclictest2819-21sh11:39:282
8287993511,3cyclictest23846-21cat11:26:422
828799350,5cyclictest2911-21ptp4l-jitter10:21:492
828299356,3cyclictest0-21swapper/111:41:441
828299352,6cyclictest30814-21apt-get12:51:451
828299352,4cyclictest0-21swapper/113:21:441
8282993514,18cyclictest28310-21/usr/sbin/munin09:01:461
8282993510,21cyclictest32592-21fschecks_count11:36:461
8282993510,21cyclictest12566-21basename09:36:391
827599354,11cyclictest1583-21ssh10:21:380
809523514,11sleep09-21ksoftirqd/008:20:240
828799349,4cyclictest6393-21cat09:21:472
828799342,13cyclictest0-21swapper/213:06:552
8287993417,14cyclictest33-21ksoftirqd/209:48:012
828799341,4cyclictest4205-21ssh13:34:472
8287993412,3cyclictest29210-21sh12:11:302
8287993411,4cyclictest20190-21cat13:16:462
828299349,22cyclictest27919-21grep12:46:541
828299343,7cyclictest22248-21apt-get12:41:421
828299342,5cyclictest0-21swapper/109:26:381
828299342,5cyclictest0-21swapper/109:11:381
8282993412,19cyclictest9207-21/usr/sbin/munin12:26:471
8282993411,20cyclictest30416-21ptp4l-jitter10:11:501
828799338,4cyclictest124482chrt12:30:122
828799336,7cyclictest22146-21grep09:56:402
828799333,9cyclictest30936-21ssh10:56:412
828799332,6cyclictest10334-21apt-get10:31:392
8287993313,4cyclictest27055-21sh12:07:182
8287993312,3cyclictest6755-21/usr/sbin/munin11:06:522
8287993312,3cyclictest30805-21sh11:34:072
8287993312,3cyclictest19864-21sh11:59:532
8287993312,3cyclictest19864-21sh11:59:532
8287993311,3cyclictest22157-21sh12:41:402
8287993311,3cyclictest16064-21sh12:33:262
828799330,4cyclictest3878-21/usr/sbin/munin12:56:562
828799330,4cyclictest17254-21sh11:17:122
828299339,3cyclictest28911-21cat12:11:241
828299337,4cyclictest4437-21rm12:20:581
828299337,3cyclictest0-21swapper/112:01:461
828299337,3cyclictest0-21swapper/112:01:461
828299332,4cyclictest0-21swapper/111:31:421
828299332,4cyclictest0-21swapper/111:11:411
8282993313,17cyclictest20115-21sed10:41:581
8282993312,18cyclictest4372-21sh12:57:051
8282993311,19cyclictest10388-21ls09:31:391
828799327,9cyclictest0-21swapper/213:49:162
8287993215,14cyclictest33-21ksoftirqd/208:51:002
8287993210,3cyclictest14292-21/usr/sbin/munin10:36:452
828299328,3cyclictest857-21seq10:17:301
828299328,3cyclictest857-21seq10:17:301
828299327,3cyclictest20137-21grep08:41:461
828299325,6cyclictest16051-21irqrtprio11:16:481
828299322,6cyclictest5072-21apt-get12:21:431
828299322,3cyclictest0-21swapper/108:26:371
828299322,3cyclictest0-21swapper/108:26:371
828299321,6cyclictest19631-21apt-get11:21:421
8282993212,17cyclictest6271-21if_enp1s0.113:36:511
8282993210,19cyclictest46950irq/101-ens3-Tx10:36:461
8282993210,19cyclictest3034-21runrttasks10:21:531
828299320,10cyclictest1197-21diskmemload10:31:541
828799319,3cyclictest10710-21latency_hist13:06:312
828799316,9cyclictest0-21swapper/212:03:042
828799316,9cyclictest0-21swapper/212:03:042
8287993113,3cyclictest4973-21sh12:21:402
828299318,3cyclictest27805-21missed_timers10:06:471
828299318,3cyclictest18397-21rm13:14:081
828299317,4cyclictest23120-21seq08:48:491
828299317,3cyclictest18935-21rm13:50:261
828299317,3cyclictest102442chrt08:21:571
828299317,3cyclictest102442chrt08:21:571
828299316,5cyclictest23937-21rm13:21:101
828299316,3cyclictest20138-21sed09:51:431
828299313,4cyclictest0-21swapper/110:01:251
828299312,9cyclictest3314-21rm13:33:001
828299311,10cyclictest25784-21snmp_o5powerctl10:01:591
828299310,9cyclictest18866-21ssh11:57:381
828299310,9cyclictest18866-21ssh11:57:381
828799309,3cyclictest16495-21expr08:36:382
8287993011,3cyclictest31915-21processes09:06:472
828799301,12cyclictest465-21sh12:52:472
828299306,3cyclictest7369-21cat11:06:481
828299306,3cyclictest4219-21phc2sys-jitter09:16:481
828299301,9cyclictest6501-21rm13:01:321
827599304,18cyclictest22253-21ssh12:41:420
828799291,4cyclictest7516-21snmp_o5powerctl10:26:512
828799290,12cyclictest932-21ssh12:16:392
828299296,20cyclictest0-21swapper/109:41:431
828299291,5cyclictest18151-21grep12:36:461
828299291,4cyclictest0-21swapper/109:06:371
8275992924,3cyclictest9-21ksoftirqd/012:01:420
8275992924,3cyclictest9-21ksoftirqd/012:01:420
8287992822,3cyclictest33-21ksoftirqd/209:41:552
8287992821,4cyclictest33-21ksoftirqd/209:52:022
8275992823,3cyclictest9-21ksoftirqd/008:41:250
828799273,7cyclictest2527-21apt-get11:01:412
8287992720,4cyclictest33-21ksoftirqd/213:31:272
8287992720,4cyclictest33-21ksoftirqd/209:33:442
8287992720,4cyclictest33-21ksoftirqd/208:43:012
828299277,3cyclictest16709-21perl08:36:411
8275992721,4cyclictest9-21ksoftirqd/010:20:150
8275992721,4cyclictest9-21ksoftirqd/010:20:150
8275992721,3cyclictest9-21ksoftirqd/012:14:330
8287992621,3cyclictest33-21ksoftirqd/208:57:532
8287992620,3cyclictest33-21ksoftirqd/211:24:442
8287992619,4cyclictest33-21ksoftirqd/210:03:062
8275992620,4cyclictest9-21ksoftirqd/011:02:020
8275992620,4cyclictest9-21ksoftirqd/011:00:570
8275992620,3cyclictest9-21ksoftirqd/012:06:420
8275992620,3cyclictest9-21ksoftirqd/009:08:370
8275992620,3cyclictest9-21ksoftirqd/008:58:410
8275992619,4cyclictest9-21ksoftirqd/010:02:280
2130422619,3sleep09-21ksoftirqd/013:16:570
8287992518,4cyclictest33-21ksoftirqd/208:27:322
8287992518,4cyclictest33-21ksoftirqd/208:27:322
827599253,12cyclictest10396-21ssh13:41:450
8275992520,3cyclictest9-21ksoftirqd/013:06:450
8275992520,3cyclictest9-21ksoftirqd/009:18:200
8275992519,3cyclictest9-21ksoftirqd/010:48:290
8275992519,3cyclictest9-21ksoftirqd/010:07:150
8275992519,3cyclictest9-21ksoftirqd/009:54:010
8275992519,3cyclictest9-21ksoftirqd/009:43:230
8275992519,3cyclictest9-21ksoftirqd/009:25:140
8275992518,4cyclictest9-21ksoftirqd/012:31:110
8275992518,4cyclictest9-21ksoftirqd/011:18:200
8275992518,4cyclictest9-21ksoftirqd/011:08:480
8287992419,3cyclictest33-21ksoftirqd/209:14:332
8275992419,3cyclictest9-21ksoftirqd/013:31:520
8275992419,3cyclictest9-21ksoftirqd/011:31:310
8275992419,3cyclictest9-21ksoftirqd/009:02:460
8275992417,4cyclictest9-21ksoftirqd/010:54:470
8275992417,4cyclictest9-21ksoftirqd/008:29:550
8275992417,4cyclictest9-21ksoftirqd/008:29:550
8275992417,4cyclictest26184-21grep13:22:020
8275992414,4cyclictest9-21ksoftirqd/011:29:400
827599240,12cyclictest323582grep12:52:010
8287992317,4cyclictest33-21ksoftirqd/209:30:352
828799230,14cyclictest0-21swapper/213:14:592
8275992318,3cyclictest9-21ksoftirqd/013:04:350
8275992318,3cyclictest9-21ksoftirqd/011:51:000
8275992317,4cyclictest9-21ksoftirqd/012:24:040
8275992317,3cyclictest9-21ksoftirqd/011:14:250
8275992317,3cyclictest9-21ksoftirqd/009:27:110
8275992317,3cyclictest9-21ksoftirqd/008:26:240
8275992317,3cyclictest9-21ksoftirqd/008:26:240
8275992316,4cyclictest9-21ksoftirqd/010:30:500
8275992316,4cyclictest9-21ksoftirqd/009:58:230
8275992316,4cyclictest9-21ksoftirqd/009:38:070
8275992216,3cyclictest9-21ksoftirqd/009:48:060
8275992215,4cyclictest9-21ksoftirqd/009:14:310
8275992215,4cyclictest9-21ksoftirqd/008:48:570
8275992215,4cyclictest30389-21sh12:51:300
8287992116,3cyclictest33-21ksoftirqd/213:23:522
8287992115,3cyclictest28293-21unixbench_singl12:47:012
8287992114,4cyclictest29258-21sh10:53:362
8287992114,4cyclictest23919-21diskstats08:51:402
8287992114,4cyclictest227-21systemd-journal10:51:272
827599212,16cyclictest16976-21sed13:47:020
8275992116,3cyclictest9-21ksoftirqd/012:57:480
8275992115,4cyclictest9-21ksoftirqd/013:37:010
8275992115,3cyclictest9-21ksoftirqd/011:22:010
827599211,17cyclictest11769-21snmp_o5powerctl10:31:530
8275992015,3cyclictest14383-21grep08:31:420
8275992015,3cyclictest14383-21grep08:31:420
8275992014,3cyclictest24112-21cut08:51:420
8275992014,3cyclictest18004-21sh12:36:430
8275992014,3cyclictest16700-21/usr/sbin/munin11:56:570
8275992014,3cyclictest16700-21/usr/sbin/munin11:56:570
8275992014,3cyclictest16134-21awk10:36:570
8275992013,4cyclictest29103-21/usr/sbin/munin10:11:480
827599200,17cyclictest950-21/usr/sbin/munin12:16:530
8275991914,3cyclictest9-21ksoftirqd/013:15:270
8275991913,3cyclictest12422-21/usr/sbin/munin11:51:460
8275991912,4cyclictest10518-21fschecks_count09:31:410
827599191,16cyclictest0-21swapper/011:46:020
8275991911,5cyclictest1197-21diskmemload13:30:370
827599190,2cyclictest0-21swapper/012:31:310
827599190,17cyclictest0-21swapper/011:37:540
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional