You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 05:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Tue Dec 02, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1563921610,5sleep3342899cyclictest00:58:133
2361921590,4sleep0340899cyclictest01:07:540
1491221540,4sleep3342899cyclictest23:48:593
1773911050,3ptp4l401ktimersoftd/320:18:403
3428997764,5cyclictest41-21ksoftirqd/321:08:033
32982634,9sleep00-21swapper/020:22:420
32022635,11sleep10-21swapper/120:21:261
138922610,2sleep10-21swapper/100:23:061
3428995847,5cyclictest41-21ksoftirqd/323:23:033
341599587,12cyclictest25763-21/usr/sbin/munin21:08:031
125312580,3sleep1241ktimersoftd/100:21:291
3428995729,3cyclictest41-21ksoftirqd/300:48:063
177391570,1ptp4l401ktimersoftd/321:13:023
177391570,1ptp4l401ktimersoftd/301:48:543
341599567,17cyclictest24019-21irqrtprio00:33:111
3428995548,4cyclictest41-21ksoftirqd/300:37:533
204622550,2sleep20-21swapper/201:03:142
3428995442,6cyclictest41-21ksoftirqd/320:28:003
3428995428,4cyclictest41-21ksoftirqd/322:53:153
3428995428,4cyclictest41-21ksoftirqd/322:53:153
243322540,2sleep00-21swapper/001:43:150
177391540,1ptp4l401ktimersoftd/322:33:273
177391540,1ptp4l401ktimersoftd/300:01:403
3428995328,4cyclictest41-21ksoftirqd/323:43:223
3428995244,4cyclictest41-21ksoftirqd/300:07:563
3428995243,4cyclictest41-21ksoftirqd/321:23:013
3428995227,4cyclictest41-21ksoftirqd/300:38:223
3428995227,4cyclictest41-21ksoftirqd/300:38:223
150412520,3sleep00-21swapper/023:49:130
3428995126,4cyclictest41-21ksoftirqd/323:28:123
3428995126,3cyclictest41-21ksoftirqd/321:07:483
3428995125,4cyclictest41-21ksoftirqd/301:43:083
3428995044,3cyclictest41-21ksoftirqd/320:58:113
3428995027,7cyclictest41-21ksoftirqd/323:18:113
3428995026,4cyclictest41-21ksoftirqd/320:23:103
3428995026,4cyclictest41-21ksoftirqd/300:53:123
3428995025,4cyclictest41-21ksoftirqd/322:08:203
3428995025,4cyclictest41-21ksoftirqd/321:53:193
3428995024,4cyclictest41-21ksoftirqd/323:00:433
3428995024,4cyclictest41-21ksoftirqd/300:16:023
3428995022,4cyclictest41-21ksoftirqd/300:02:553
341599507,10cyclictest20776-21snmp_o5powerctl01:03:191
3428994939,5cyclictest41-21ksoftirqd/322:18:033
3428994938,3cyclictest41-21ksoftirqd/301:22:543
3428994924,4cyclictest41-21ksoftirqd/321:03:063
3428994922,4cyclictest41-21ksoftirqd/323:55:383
3428994922,4cyclictest41-21ksoftirqd/322:43:313
3428994917,10cyclictest41-21ksoftirqd/301:28:113
142592490,2sleep30-21swapper/322:41:023
3428994823,4cyclictest41-21ksoftirqd/321:58:063
3428994823,4cyclictest41-21ksoftirqd/321:33:133
3428994822,4cyclictest41-21ksoftirqd/323:34:103
3428994821,4cyclictest41-21ksoftirqd/301:40:593
3428994723,4cyclictest41-21ksoftirqd/321:48:153
3428994722,7cyclictest41-21ksoftirqd/301:34:423
3428994722,4cyclictest41-21ksoftirqd/320:42:473
3428994721,7cyclictest41-21ksoftirqd/323:17:253
3428994721,4cyclictest41-21ksoftirqd/323:04:323
3428994721,4cyclictest41-21ksoftirqd/300:47:443
3428994714,11cyclictest41-21ksoftirqd/323:08:123
3428994636,3cyclictest41-21ksoftirqd/300:23:153
3428994635,3cyclictest41-21ksoftirqd/301:08:253
3428994621,7cyclictest41-21ksoftirqd/322:32:343
3428994621,7cyclictest41-21ksoftirqd/300:30:183
3428994621,4cyclictest41-21ksoftirqd/322:48:343
3428994535,3cyclictest41-21ksoftirqd/320:53:053
3428994534,3cyclictest41-21ksoftirqd/321:42:493
3428994523,7cyclictest41-21ksoftirqd/300:21:063
3428994521,7cyclictest41-21ksoftirqd/301:13:273
3428994519,4cyclictest41-21ksoftirqd/321:43:123
3428994518,4cyclictest41-21ksoftirqd/322:23:013
341599457,8cyclictest27605-21irqstats22:18:091
3428994433,3cyclictest41-21ksoftirqd/321:32:493
3428994425,10cyclictest41-21ksoftirqd/301:04:113
341599441,17cyclictest0-21swapper/100:44:551
3428994333,3cyclictest41-21ksoftirqd/320:33:043
3428994314,7cyclictest41-21ksoftirqd/301:24:433
3428994311,10cyclictest41-21ksoftirqd/323:38:103
341599436,8cyclictest15932-21munin-run20:47:471
342899423,3cyclictest41-21ksoftirqd/322:17:503
342899415,3cyclictest121rcu_preempt20:43:023
3428994130,4cyclictest41-21ksoftirqd/321:22:483
342899406,4cyclictest41-21ksoftirqd/320:48:013
342899401,5cyclictest20238-21taskset22:03:083
341599399,17cyclictest17279-21/usr/sbin/munin23:18:081
341599368,14cyclictest28782-21diskmemload23:15:191
341599358,14cyclictest19385-21interrupts01:38:121
341599352,19cyclictest0-21swapper/101:19:071
341599352,19cyclictest0-21swapper/100:12:571
65342340,2sleep10-21swapper/121:33:121
341599348,8cyclictest16366-21/usr/sbin/munin00:58:221
341599348,12cyclictest5789-21cat20:27:471
341599347,16cyclictest0-21swapper/122:41:051
341599346,14cyclictest30659-21ssh01:13:541
341599337,8cyclictest28782-21diskmemload00:53:071
341599336,14cyclictest1051-21ssh23:34:361
12492336,13sleep20-21swapper/220:18:002
341599329,20cyclictest23436-21phc2sys-jitter23:58:141
341599327,13cyclictest31242-21ssh22:23:431
341599326,14cyclictest17799-21ssh01:36:281
3415993220,6cyclictest30683-21apt-get21:18:051
3415993214,14cyclictest0-21swapper/100:28:051
3419993125,3cyclictest33-21ksoftirqd/221:58:042
341999310,2cyclictest121rcu_preempt01:49:362
341599317,9cyclictest16230-21apt-get20:48:011
341599317,6cyclictest5286-21ssh22:31:551
341599316,13cyclictest13674-21ssh01:31:591
341599314,12cyclictest28782-21diskmemload23:55:141
341599313,13cyclictest0-21swapper/120:53:081
3415993125,4cyclictest25-21ksoftirqd/123:12:511
341599312,11cyclictest0-21swapper/122:09:081
341599308,8cyclictest872-21ssh00:08:171
341599307,9cyclictest24839-21ssh23:26:101
341599306,9cyclictest0-21swapper/121:58:041
341599305,16cyclictest26321-21irqstats22:53:121
341599305,16cyclictest26321-21irqstats22:53:121
341599304,13cyclictest0-21swapper/122:33:041
341599303,17cyclictest0-21swapper/121:23:171
341599301,10cyclictest0-21swapper/100:49:381
3419992923,3cyclictest33-21ksoftirqd/223:17:362
3415992924,3cyclictest25-21ksoftirqd/101:08:171
3415992918,6cyclictest3293-21apt-get23:38:051
3415992918,6cyclictest23350-21apt-get21:03:001
3415992918,6cyclictest23350-21apt-get21:03:001
3415992913,11cyclictest0-21swapper/121:38:021
3408992922,4cyclictest9-21ksoftirqd/020:44:050
3419992822,4cyclictest33-21ksoftirqd/222:20:062
3419992821,4cyclictest33-21ksoftirqd/221:39:512
341999280,3cyclictest121rcu_preempt23:59:232
341599283,11cyclictest0-21swapper/120:38:091
3415992815,9cyclictest30115-21apt-get22:58:041
3415992811,14cyclictest23446-21rm22:50:101
3415992811,14cyclictest15620-21gpgv22:42:511
3415992810,15cyclictest28189-21tail01:48:111
3408992822,4cyclictest9-21ksoftirqd/020:29:080
3408992822,4cyclictest32451-1kworker/u9:023:03:080
3408992822,3cyclictest9-21ksoftirqd/022:48:460
3408992821,4cyclictest9-21ksoftirqd/021:22:270
3408992821,4cyclictest9-21ksoftirqd/020:38:150
245552280,4sleep09-21ksoftirqd/023:25:400
3419992722,3cyclictest33-21ksoftirqd/220:52:462
3419992721,4cyclictest33-21ksoftirqd/221:31:202
3419992720,4cyclictest33-21ksoftirqd/221:49:062
3419992720,4cyclictest33-21ksoftirqd/220:53:302
3419992719,5cyclictest1855-21grep22:28:052
341599275,19cyclictest0-21swapper/121:43:171
341599272,10cyclictest0-21swapper/120:33:101
3415992717,5cyclictest2997-21apt-get21:28:021
3415992717,5cyclictest2418-21apt-get23:03:031
3415992714,9cyclictest24479-21snmp_o5powerctl01:43:171
3408992720,4cyclictest9-21ksoftirqd/020:50:390
3419992620,4cyclictest33-21ksoftirqd/221:16:072
3419992619,4cyclictest33-21ksoftirqd/201:38:472
3419992619,4cyclictest33-21ksoftirqd/200:38:212
3419992619,4cyclictest33-21ksoftirqd/200:38:212
341599268,15cyclictest0-21swapper/100:03:041
341599262,10cyclictest0-21swapper/122:03:111
341599262,10cyclictest0-21swapper/121:53:091
3415992614,8cyclictest8044-21snmp_o5powerctl20:28:141
3415992614,8cyclictest6824-21snmp_o5powerctl01:23:271
3415992614,8cyclictest25703-21snmp_o5powerctl22:13:161
3415992612,9cyclictest0-21swapper/120:58:001
3415992610,11cyclictest0-21swapper/123:28:041
3408992621,3cyclictest9-21ksoftirqd/020:58:300
3408992620,3cyclictest9-21ksoftirqd/021:37:430
3408992620,3cyclictest9-21ksoftirqd/020:37:060
3408992619,4cyclictest9-21ksoftirqd/021:05:130
3408992619,4cyclictest9-21ksoftirqd/021:05:130
274462260,4sleep233-21ksoftirqd/200:37:522
854722518,3sleep09-21ksoftirqd/000:17:520
341999252,3cyclictest121rcu_preempt01:14:212
3419992520,3cyclictest33-21ksoftirqd/221:23:452
3419992519,4cyclictest33-21ksoftirqd/222:05:162
3419992519,3cyclictest33-21ksoftirqd/221:00:202
3419992519,3cyclictest33-21ksoftirqd/220:40:372
3419992519,3cyclictest33-21ksoftirqd/220:35:192
3419992518,4cyclictest33-21ksoftirqd/222:17:292
3419992518,4cyclictest33-21ksoftirqd/221:55:092
3415992520,3cyclictest25-21ksoftirqd/100:37:571
3415992520,3cyclictest25-21ksoftirqd/100:37:571
3415992515,5cyclictest12736-21apt-get23:48:061
3415992514,5cyclictest12490-21apt-get21:48:021
3415992511,12cyclictest30177-21http21:17:481
3408992519,3cyclictest9-21ksoftirqd/000:32:520
3408992518,4cyclictest9-21ksoftirqd/022:15:300
3408992517,4cyclictest29308-21grep22:23:030
3419992419,3cyclictest33-21ksoftirqd/200:22:522
3419992418,4cyclictest33-21ksoftirqd/201:44:052
3419992418,3cyclictest33-21ksoftirqd/223:04:062
3419992417,4cyclictest33-21ksoftirqd/223:20:562
3419992417,4cyclictest33-21ksoftirqd/221:22:092
3419992417,4cyclictest33-21ksoftirqd/221:07:522
340899246,15cyclictest9-21ksoftirqd/022:36:500
3408992419,3cyclictest9-21ksoftirqd/022:44:120
3408992418,3cyclictest9-21ksoftirqd/020:55:470
3408992418,3cyclictest9-21ksoftirqd/000:20:000
3408992417,4cyclictest9-21ksoftirqd/022:03:380
3408992417,4cyclictest9-21ksoftirqd/021:42:020
341999233,11cyclictest675-21ssh01:18:062
3419992317,4cyclictest33-21ksoftirqd/223:23:042
3419992317,4cyclictest33-21ksoftirqd/221:04:182
3419992317,4cyclictest33-21ksoftirqd/221:04:182
3419992316,4cyclictest3998-21sed23:38:112
3419992316,4cyclictest33-21ksoftirqd/220:45:382
3415992311,8cyclictest7916-21taskset23:43:031
3408992318,3cyclictest9-21ksoftirqd/021:09:070
3408992317,3cyclictest9-21ksoftirqd/021:14:450
3408992317,3cyclictest9-21ksoftirqd/020:23:340
3408992317,3cyclictest9-21ksoftirqd/000:36:380
3408992316,4cyclictest9-21ksoftirqd/022:22:160
3408992316,4cyclictest9-21ksoftirqd/001:36:560
3419992217,3cyclictest33-21ksoftirqd/200:10:422
3419992216,4cyclictest33-21ksoftirqd/223:33:372
3419992216,3cyclictest33-21ksoftirqd/223:54:242
3419992216,3cyclictest33-21ksoftirqd/223:31:032
3419992216,3cyclictest25276-21ssh22:53:002
3419992216,3cyclictest25276-21ssh22:53:002
3419992215,4cyclictest9110-21ssh23:08:272
341999221,4cyclictest11454-21snmp_o5powerctl01:28:222
3408992216,3cyclictest9-21ksoftirqd/022:10:030
3408992216,3cyclictest9-21ksoftirqd/021:47:550
3408992216,3cyclictest9-21ksoftirqd/001:17:010
3408992215,4cyclictest9-21ksoftirqd/023:32:340
3408992214,4cyclictest3469-21gltestperf21:28:060
340899221,3cyclictest2266-21ssh23:36:450
341999219,6cyclictest321ktimersoftd/201:08:082
341999212,13cyclictest64-21kswapd000:23:042
3419992116,3cyclictest33-21ksoftirqd/201:37:082
3419992116,3cyclictest33-21ksoftirqd/201:01:492
3419992116,3cyclictest33-21ksoftirqd/200:52:342
3419992115,4cyclictest33-21ksoftirqd/222:11:592
3419992114,4cyclictest5874-21hddtemp_smartct21:33:072
3419992114,4cyclictest4326-21grep20:23:052
3419992114,4cyclictest32406-21ssh22:25:532
3419992114,4cyclictest20823-21sh00:29:292
3419992114,4cyclictest14956-21ssh23:49:032
3419992114,4cyclictest11104-21memory21:43:112
3419992113,4cyclictest20579-21sh22:48:022
341999211,18cyclictest5381-21ssh00:13:152
341999211,18cyclictest0-21swapper/201:26:182
341999211,17cyclictest10779-21ssh22:37:522
3408992116,3cyclictest9-21ksoftirqd/000:44:330
3408992115,3cyclictest9847-21sh23:09:480
3408992115,3cyclictest9-21ksoftirqd/021:59:420
3408992115,3cyclictest9-21ksoftirqd/021:44:430
3408992115,3cyclictest29886-21sh01:49:130
340899211,4cyclictest11968-21apt-get23:13:030
3408992114,4cyclictest8027-21sh01:25:040
3408992114,4cyclictest3211-21/usr/sbin/munin23:38:100
3408992114,4cyclictest28782-21diskmemload00:58:270
3408992114,4cyclictest28782-21diskmemload00:51:050
3408992114,4cyclictest28782-21diskmemload00:42:510
3408992114,4cyclictest28782-21diskmemload00:42:510
3408992114,4cyclictest21845-21latency_hist23:57:530
3408992114,4cyclictest12444-21snmp_o5powerctl22:38:200
3408992110,3cyclictest28434-21ssh22:55:330
3408992110,3cyclictest28434-21ssh22:55:330
3419992015,3cyclictest0-21swapper/222:43:042
3419992014,3cyclictest31534-21apt-config00:07:522
341999201,17cyclictest0-21swapper/222:34:302
341999201,16cyclictest13396-21ssh00:56:272
341999201,11cyclictest47050irq/102-ens3-Tx23:43:052
341999200,17cyclictest8043-21snmp_o5powerctl20:28:132
3408992015,3cyclictest9-21ksoftirqd/023:00:550
3408992015,3cyclictest9-21ksoftirqd/021:27:250
3408992015,3cyclictest16627-21/usr/sbin/munin23:18:080
3408992014,3cyclictest9-21ksoftirqd/000:05:590
3408992014,3cyclictest22246-21sh01:41:590
3408992014,3cyclictest13873-21cstates00:23:060
3408992014,3cyclictest12828-21sh01:30:180
3408992013,4cyclictest931-21df01:18:080
3408992013,4cyclictest23210-21wc23:58:120
3408992012,5cyclictest46850irq/100-ens3-Tx00:08:050
340899201,16cyclictest8529-21hddtemp_smartct23:43:090
3419991914,3cyclictest30562-21tail22:58:082
341999191,15cyclictest32742-21apt-get00:43:072
340899191,3cyclictest14200-21munin-run00:57:530
3408991913,3cyclictest28782-21diskmemload01:09:490
3408991912,4cyclictest2366-21sed22:28:080
3408991812,3cyclictest14800-21/usr/sbin/munin21:53:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional