You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-18 - 14:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot6.osadl.org (updated Tue Mar 18, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
586421620,4sleep21168099cyclictest01:46:342
151891960,4ptp4l401ktimersoftd/320:59:523
181842700,7sleep0101ktimersoftd/001:21:350
265732610,2sleep20-21swapper/201:31:292
265732610,2sleep20-21swapper/201:31:292
291852590,2sleep10-21swapper/121:36:011
136712570,2sleep30-21swapper/301:16:323
44032550,5sleep04408-21irqrtprio02:26:270
136222550,2sleep00-21swapper/023:53:180
1167599559,13cyclictest27216-21awk02:11:381
24162520,2sleep00-21swapper/001:42:550
39162510,4sleep1241ktimersoftd/100:21:421
20642510,2sleep10-21swapper/123:41:191
1167599518,18cyclictest30763-21cut21:41:161
1167599517,18cyclictest18658-21mailstats21:11:241
1167599514,16cyclictest0-21swapper/100:50:401
11675995110,16cyclictest32310-21seq23:37:331
1167599508,18cyclictest25595-21sed22:36:311
1167599486,17cyclictest25596-21ssh23:28:471
11680994134,4cyclictest11684-21latency_hist21:01:012
11675994112,18cyclictest4604-21apt-get01:06:201
11688994029,6cyclictest41-21ksoftirqd/301:11:193
1168899401,36cyclictest29430-21grep22:46:223
151891390,1ptp4l401ktimersoftd/301:46:333
1167599399,16cyclictest14956-21basename21:06:151
151891380,1ptp4l401ktimersoftd/323:17:143
151891380,1ptp4l401ktimersoftd/302:21:263
151891380,1ptp4l401ktimersoftd/300:26:053
151891380,1ptp4l401ktimersoftd/300:16:063
1168899380,4cyclictest0-21swapper/302:11:253
1167599389,9cyclictest8009-21snmp_o5powerctl23:06:301
1140323819,9sleep10-21swapper/120:58:551
151891370,1ptp4l401ktimersoftd/323:32:293
151891370,1ptp4l401ktimersoftd/323:21:513
151891370,1ptp4l401ktimersoftd/321:33:113
151891370,1ptp4l401ktimersoftd/302:30:143
151891370,1ptp4l401ktimersoftd/301:37:333
151891370,1ptp4l401ktimersoftd/300:27:323
151891370,0ptp4l401ktimersoftd/301:21:483
151891370,0ptp4l401ktimersoftd/300:50:583
11688993718,15cyclictest31671-21head21:41:273
1167599379,17cyclictest19910-21timedrift23:21:451
1167599379,15cyclictest7308-21apt-get22:01:191
1167599379,15cyclictest7308-21apt-get22:01:191
1167599379,15cyclictest4407-21apt-get01:46:201
11675993710,8cyclictest28561-21apt-get22:46:081
151891360,1ptp4l401ktimersoftd/302:01:543
151891360,1ptp4l401ktimersoftd/300:08:563
151891360,0ptp4l401ktimersoftd/300:40:163
1167599369,9cyclictest18272-21apt-get00:01:121
1167599369,15cyclictest25700-21df00:51:201
11675993610,11cyclictest26493-21apt-get22:41:161
151891350,1ptp4l401ktimersoftd/321:21:213
1168899357,4cyclictest1787-21sed23:01:043
11688993515,17cyclictest32305-21/usr/sbin/munin01:41:223
11688993512,19cyclictest5675-21cat21:56:223
11688993511,20cyclictest6396-21cat23:46:163
11688993511,20cyclictest6396-21cat23:46:163
11688993511,20cyclictest5620-21latency01:06:263
1167599359,9cyclictest16841-21hddtemp_smartct01:21:191
1167599358,14cyclictest19722-21apt-get21:16:151
1167599358,13cyclictest1710-21diskmemload01:44:391
1167599356,18cyclictest0-21swapper/123:01:161
1167599356,18cyclictest0-21swapper/121:21:231
1167599356,12cyclictest0-21swapper/102:26:301
11675993523,6cyclictest9203-21apt-get00:31:151
11675993523,6cyclictest25328-21apt-get01:31:191
11675993523,6cyclictest25328-21apt-get01:31:191
11675993510,11cyclictest22068-21grep02:06:311
171982340,4sleep21168099cyclictest23:59:392
1168899349,4cyclictest25724-21unixbench_multi22:36:323
1168899347,3cyclictest8837-21kworker/3:001:01:183
11688993413,18cyclictest2628-21if_enp1s0.123:41:233
11688993410,21cyclictest22288-21date02:06:363
11675993415,13cyclictest14035-21cat21:06:011
11675993413,18cyclictest27263-21/usr/sbin/munin23:31:251
11675993410,8cyclictest9646-21hddtemp_smartct01:11:231
151891330,1ptp4l401ktimersoftd/323:29:563
150791330,1getstats0-21swapper/301:31:403
150791330,1getstats0-21swapper/301:31:403
11688993313,17cyclictest3491-21sh23:01:243
11688993312,18cyclictest22181-21sh01:26:313
11675993321,6cyclictest8917-21apt-get01:51:201
11675993310,8cyclictest16564-21diskstats02:01:241
1168899323,7cyclictest24662-21apt-get21:26:143
11688993215,14cyclictest41-21ksoftirqd/322:31:153
11688993213,16cyclictest41-21ksoftirqd/321:07:533
11675993216,11cyclictest0-21swapper/123:11:151
1151523211,11sleep09-21ksoftirqd/021:00:240
1168899319,4cyclictest31811-21sh01:00:223
1168899319,4cyclictest26359-21irqstats00:51:253
1168899318,3cyclictest13203-21mailstats21:01:243
1168899318,3cyclictest11575-21cat23:51:243
1168899317,5cyclictest0-21swapper/300:01:193
1168899317,3cyclictest17892-21sendmail_mailst00:41:243
1168899315,3cyclictest9929-21entropy22:06:183
1168899312,6cyclictest6576-21apt-get23:06:163
1168899311,9cyclictest0-21swapper/322:26:033
11688993110,4cyclictest28604-21sh00:11:283
1167599318,9cyclictest1123-21processes21:46:241
1167599318,8cyclictest21772-21ssh01:26:271
1167599315,13cyclictest0-21swapper/123:19:421
272072300,2sleep00-21swapper/000:51:360
1168899308,3cyclictest9485-21sh00:31:173
1168899306,3cyclictest27600-21phc2sys22:41:253
1168899303,9cyclictest19289-21cpuspeed_turbos22:26:173
11688993011,16cyclictest1710-21diskmemload23:15:593
11688993010,3cyclictest15536-21phc2sys22:16:253
11680993023,4cyclictest33-21ksoftirqd/221:46:012
1167599307,9cyclictest28214-21irqstats00:11:251
1167599305,11cyclictest0-21swapper/121:26:221
11675993015,10cyclictest0-21swapper/102:21:151
11675993014,11cyclictest0-21swapper/122:51:211
11669993023,4cyclictest9-21ksoftirqd/000:31:150
1168899297,3cyclictest4361-21taskset21:55:273
1168899293,5cyclictest0-21swapper/322:51:253
11680992923,4cyclictest33-21ksoftirqd/200:11:002
11680992922,4cyclictest33-21ksoftirqd/223:46:582
11680992922,4cyclictest33-21ksoftirqd/223:46:582
11680992922,4cyclictest33-21ksoftirqd/202:00:242
1167599296,9cyclictest1033-21snmpd01:04:261
1167599293,9cyclictest0-21swapper/102:19:221
11675992915,9cyclictest16557-21sh00:41:141
11675992915,10cyclictest0-21swapper/122:31:161
11675992914,12cyclictest8843-21idleruntime-cro00:31:051
11669992910,11cyclictest22704-21ssh23:26:160
1168899286,2cyclictest0-21swapper/323:58:053
1168899285,4cyclictest30349-21rm02:17:593
1168899285,3cyclictest7439-21cpuspeed_turbos22:01:213
1168899285,3cyclictest7439-21cpuspeed_turbos22:01:213
1168899282,7cyclictest32385-21apt-get21:46:143
1168899281,6cyclictest9886-21latency01:51:273
11680992823,3cyclictest33-21ksoftirqd/223:23:552
1167599284,9cyclictest0-21swapper/101:36:111
11675992816,8cyclictest12762-21users23:51:371
11675992815,9cyclictest3005-21/usr/sbin/munin21:51:201
11675992811,14cyclictest1-21systemd23:56:591
1168899278,2cyclictest0-21swapper/323:37:153
1168899277,3cyclictest17829-21sed21:11:183
1168899277,3cyclictest14754-21sh01:58:533
1168899274,4cyclictest12136-21cpuspeed_turbos22:11:153
1168899272,4cyclictest21061-21snmp_o5powerctl21:16:283
11680992721,3cyclictest33-21ksoftirqd/221:46:082
11680992721,3cyclictest33-21ksoftirqd/221:31:142
11680992720,4cyclictest33-21ksoftirqd/223:39:552
11680992720,4cyclictest33-21ksoftirqd/222:08:142
11680992720,4cyclictest33-21ksoftirqd/221:52:482
11680992720,4cyclictest33-21ksoftirqd/201:02:222
11675992716,8cyclictest23290-21hddtemp_smartct00:06:221
11675992716,7cyclictest10311-21ps22:06:211
11675992715,8cyclictest16297-21wc00:41:061
11675992715,8cyclictest14842-21grep22:16:201
11675992712,12cyclictest15441-21rm01:20:041
11675992712,11cyclictest1873-21sed23:01:041
11675992711,13cyclictest6349-21ssh23:46:121
11675992711,13cyclictest6349-21ssh23:46:121
11669992722,3cyclictest9-21ksoftirqd/002:06:310
11669992722,3cyclictest9-21ksoftirqd/000:11:270
11669992721,3cyclictest9-21ksoftirqd/002:21:160
11669992720,4cyclictest9-21ksoftirqd/021:26:450
11669992720,4cyclictest9-21ksoftirqd/021:03:320
11669992720,4cyclictest23816-21cron22:36:030
11680992620,3cyclictest33-21ksoftirqd/222:58:352
11680992620,3cyclictest33-21ksoftirqd/222:55:142
11680992619,4cyclictest33-21ksoftirqd/222:34:392
11680992619,4cyclictest33-21ksoftirqd/221:17:272
11680992615,9cyclictest33-21ksoftirqd/201:40:162
11675992615,8cyclictest5143-21grep21:56:191
11675992615,8cyclictest235-21systemd-journal02:01:071
11675992614,9cyclictest29729-21awk21:41:011
11675992614,9cyclictest19090-21/usr/sbin/munin22:26:151
11675992614,8cyclictest12431-21fschecks_time22:11:171
11675992613,9cyclictest31614-21ssh00:16:111
11669992621,3cyclictest9-21ksoftirqd/002:16:080
11669992620,3cyclictest9-21ksoftirqd/022:23:310
11669992620,3cyclictest9-21ksoftirqd/022:20:570
11669992619,4cyclictest9-21ksoftirqd/022:56:540
11669992619,4cyclictest9-21ksoftirqd/021:55:550
11669992619,4cyclictest9-21ksoftirqd/021:11:010
11669992618,6cyclictest9-21ksoftirqd/001:39:140
1168099252,19cyclictest0-21swapper/202:01:212
11680992519,3cyclictest33-21ksoftirqd/222:12:342
11680992518,4cyclictest33-21ksoftirqd/223:19:042
11680992518,4cyclictest33-21ksoftirqd/223:04:502
11680992518,4cyclictest33-21ksoftirqd/222:20:062
11680992518,4cyclictest33-21ksoftirqd/221:57:342
11680992518,4cyclictest33-21ksoftirqd/221:23:462
11680992518,4cyclictest33-21ksoftirqd/200:46:162
11680992517,5cyclictest33-21ksoftirqd/222:50:412
1168099251,21cyclictest0-21swapper/201:51:272
11675992515,7cyclictest16953-21cut22:21:201
11675992513,8cyclictest29489-21latency_hist00:56:071
11669992520,3cyclictest9-21ksoftirqd/000:06:230
11669992519,3cyclictest9-21ksoftirqd/022:52:080
11669992519,3cyclictest9-21ksoftirqd/022:47:170
11669992519,3cyclictest9-21ksoftirqd/022:43:570
11669992519,3cyclictest9-21ksoftirqd/022:31:030
11669992519,3cyclictest9-21ksoftirqd/000:51:060
11669992519,3cyclictest9-21ksoftirqd/000:18:280
11669992518,4cyclictest9-21ksoftirqd/022:02:180
11669992518,4cyclictest9-21ksoftirqd/022:02:180
11669992518,4cyclictest9-21ksoftirqd/021:36:400
11669992518,4cyclictest9-21ksoftirqd/021:33:070
11680992418,3cyclictest33-21ksoftirqd/223:26:122
11680992418,3cyclictest33-21ksoftirqd/201:46:032
11680992417,4cyclictest33-21ksoftirqd/222:43:432
11680992417,4cyclictest33-21ksoftirqd/222:40:542
11680992417,4cyclictest33-21ksoftirqd/222:29:422
11680992417,4cyclictest33-21ksoftirqd/221:13:422
11680992417,4cyclictest33-21ksoftirqd/221:01:092
11680992417,4cyclictest33-21ksoftirqd/200:31:272
1168099241,20cyclictest3446-21snmp_o5powerctl00:21:272
1168099241,20cyclictest1710-21diskmemload00:53:462
11669992419,3cyclictest9-21ksoftirqd/023:21:350
11669992418,3cyclictest9-21ksoftirqd/022:13:190
11669992418,3cyclictest9-21ksoftirqd/022:07:100
11669992418,3cyclictest9-21ksoftirqd/002:18:240
11669992418,3cyclictest9-21ksoftirqd/001:01:150
11669992417,4cyclictest9-21ksoftirqd/021:18:020
11669992417,4cyclictest9-21ksoftirqd/000:31:050
11680992317,3cyclictest33-21ksoftirqd/201:19:572
11680992317,3cyclictest33-21ksoftirqd/200:56:402
11680992316,4cyclictest33-21ksoftirqd/223:51:402
11680992316,4cyclictest33-21ksoftirqd/223:31:202
11680992316,4cyclictest33-21ksoftirqd/222:22:432
11680992316,4cyclictest33-21ksoftirqd/222:01:032
11680992316,4cyclictest33-21ksoftirqd/222:01:032
11680992316,4cyclictest33-21ksoftirqd/200:39:222
11680992315,5cyclictest1268-21grep02:21:352
11680992314,5cyclictest29012-21unixbench_multi00:11:362
1168099231,20cyclictest0-21swapper/200:27:262
11680992311,9cyclictest9129-21apt-get01:11:192
1168099230,21cyclictest0-21swapper/221:06:342
1166999232,12cyclictest27354-21ssh23:31:160
11669992318,3cyclictest9-21ksoftirqd/021:46:210
11669992318,3cyclictest9-21ksoftirqd/002:02:240
11669992318,3cyclictest9-21ksoftirqd/001:06:260
11669992317,3cyclictest9-21ksoftirqd/023:46:240
11669992317,3cyclictest9-21ksoftirqd/023:46:240
11669992317,3cyclictest9-21ksoftirqd/022:36:450
11669992316,4cyclictest9-21ksoftirqd/021:56:400
11669992316,4cyclictest9-21ksoftirqd/021:22:270
35422220,2sleep00-21swapper/000:21:290
323572220,2sleep20-21swapper/200:17:412
151891220,1ptp4l401ktimersoftd/321:36:183
11680992215,4cyclictest16249-21/usr/sbin/munin01:21:272
1168099221,4cyclictest19714-21snmp_o5powerctl00:01:262
11680992213,6cyclictest11662-21cyclictest02:06:202
1168099220,20cyclictest0-21swapper/223:41:112
1166999226,12cyclictest16610-21cpuspeed_turbos00:41:150
1166999222,12cyclictest2086-21ssh23:01:130
1166999222,11cyclictest4420-21ssh01:46:200
11669992217,3cyclictest9-21ksoftirqd/023:41:240
11669992216,3cyclictest9-21ksoftirqd/021:41:110
11669992215,4cyclictest9-21ksoftirqd/023:10:350
11669992215,4cyclictest9-21ksoftirqd/021:11:510
1166999221,4cyclictest14999-21apt-get23:56:070
11680992115,3cyclictest11557-21sed23:11:202
11680992114,4cyclictest3937-21fschecks_count02:26:242
11680992114,4cyclictest33-21ksoftirqd/221:27:472
11680992114,4cyclictest33-21ksoftirqd/202:16:462
1168099211,18cyclictest0-21swapper/201:30:042
1168099211,17cyclictest1710-21diskmemload23:07:132
1168099210,18cyclictest18866-21ssh00:42:312
11669992116,3cyclictest9-21ksoftirqd/023:37:500
11669992116,3cyclictest9-21ksoftirqd/023:11:230
11669992116,3cyclictest9-21ksoftirqd/000:57:220
11669992115,3cyclictest9-21ksoftirqd/001:53:590
11669992115,3cyclictest9-21ksoftirqd/000:05:390
11669992111,6cyclictest9-21ksoftirqd/001:31:200
11669992111,6cyclictest9-21ksoftirqd/001:31:200
11669992110,9cyclictest17880-21sleep23:21:030
11680992015,3cyclictest33-21ksoftirqd/201:09:372
1168099201,16cyclictest0-21swapper/202:13:552
11669992015,3cyclictest9-21ksoftirqd/001:26:300
11669992015,3cyclictest10848-21grep01:11:360
11669992014,3cyclictest15186-21sh00:39:050
11669992013,4cyclictest13734-21sh01:16:400
11680991915,2cyclictest0-21swapper/221:36:242
11669991913,3cyclictest15148-21sh01:59:410
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional