You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-18 - 03:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Tue Jun 18, 2024 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1529821310,0sleep00-21swapper/000:12:050
35622970,3sleep10-21swapper/122:00:291
187932960,2sleep118792-21bash23:40:441
233312920,0sleep323329-21sshd23:31:523
94992900,2sleep30-21swapper/323:05:473
179152900,2sleep20-21swapper/222:45:062
909828926,51sleep30-21swapper/319:09:503
54752890,1sleep20-21swapper/200:01:332
153252880,2sleep215329-21id23:12:362
37302870,0sleep00-21swapper/000:04:390
162972870,3sleep1936999cyclictest22:51:021
171782860,0sleep00-21swapper/023:28:120
317762840,1sleep20-21swapper/220:15:282
153862840,2sleep10-21swapper/122:29:181
906128333,46sleep10-21swapper/119:09:251
220592830,1sleep022060-21sshd22:02:150
195112830,2sleep20-21swapper/223:03:462
119322810,2sleep311929-21ls00:05:223
197112800,2sleep20-21swapper/222:20:262
93832780,2sleep29382-21bash00:27:262
262922780,1sleep30-21swapper/322:11:533
262052770,2sleep3431ktimersoftd/322:30:173
262052770,2sleep3431ktimersoftd/322:30:163
66162760,2sleep30-21swapper/300:11:113
81582750,1sleep08147-21sshd22:31:380
81582750,1sleep08147-21sshd22:31:380
6282750,2sleep30-21swapper/300:17:013
292682750,2sleep329267-21sshd00:23:023
286542710,2sleep228644-21sshd00:10:142
192582710,2sleep10-21swapper/122:05:101
158912710,2sleep3937199cyclictest22:57:143
149552710,1sleep20-21swapper/223:53:062
3242700,0sleep0326-21bash21:41:360
14432700,1sleep20-21swapper/221:44:552
3692690,67sleep227150irq/26-ahci[00021:29:212
302082680,2sleep1936999cyclictest22:15:201
191492650,4sleep2937099cyclictest21:12:372
273702640,0sleep00-21swapper/021:22:370
707526323,9sleep20-21swapper/219:05:092
254712600,2sleep325472-21sshd21:25:283
148792600,1sleep3937199cyclictest22:47:513
896525833,13sleep00-21swapper/019:08:110
91572523,46sleep2144950irq/27-eth022:50:242
115802400,0sleep111579-21bash00:34:061
115802400,0sleep111579-21bash00:34:061
224312390,3sleep20-21swapper/200:35:112
192532341,31sleep2144950irq/27-eth022:14:212
63192330,3sleep20-21swapper/221:23:422
936599301,5cyclictest11854-21sshd22:41:190
87582290,3sleep10-21swapper/100:20:551
284912290,26sleep2144950irq/27-eth022:30:292
284912290,26sleep2144950irq/27-eth022:30:282
139962280,0sleep113998-21bash23:49:541
138432280,5sleep1936999cyclictest00:27:521
936599270,18cyclictest21295-21bash23:16:110
53782260,1sleep10-21swapper/122:12:591
193832260,2sleep219380-21sshd21:15:362
77712250,3sleep11927-21nfsd00:17:431
309902250,2sleep20-21swapper/200:16:472
208352240,1sleep220830-21sshd23:47:182
206952240,1sleep220690-21bash21:52:532
937099235,3cyclictest0-21swapper/222:09:152
321052230,2sleep332096-21sshd21:57:023
135642230,2sleep113558-21rm22:22:561
937099224,9cyclictest14607-21sshd22:38:312
937099224,9cyclictest14607-21sshd22:38:312
129862220,1sleep00-21swapper/022:47:400
937099212,16cyclictest32384-21sshd23:35:492
937099212,13cyclictest19754-21cp23:34:432
9370992117,3cyclictest14147-21sshd23:15:322
9370992111,9cyclictest152450irq/28-eth1-rx-20:53:382
244302210,0sleep00-21swapper/019:55:170
101042210,2sleep110105-21bash21:27:061
100802210,1sleep010075-21sshd22:53:380
937099203,14cyclictest1636-21sshd23:29:522
937099203,12cyclictest5283-21sshd00:30:152
937099203,12cyclictest5283-21sshd00:30:152
937099193,13cyclictest11029-21sshd00:21:092
937099193,10cyclictest0-21swapper/220:02:242
53802190,2sleep30-21swapper/323:39:343
265772190,4sleep20-21swapper/221:37:582
163722190,17sleep2144950irq/27-eth023:59:332
9371991814,2cyclictest27260-21sshd21:35:003
937099182,3cyclictest1861-21sshd22:27:592
937099182,15cyclictest152450irq/28-eth1-rx-00:09:112
937099182,10cyclictest10766-21sshd21:30:152
937099181,0cyclictest0-21swapper/222:02:472
936999183,0cyclictest0-21swapper/121:56:231
936999181,16cyclictest22-21ksoftirqd/100:12:391
936999180,3cyclictest152450irq/28-eth1-rx-22:34:361
936999180,3cyclictest152450irq/28-eth1-rx-22:34:361
936599185,7cyclictest14723-21cp22:57:070
936599184,9cyclictest18264-21sshd22:20:190
86782180,1sleep08674-21sshd21:45:290
179072180,3sleep1936999cyclictest21:46:221
143822180,15sleep114549-21sshd21:39:581
937199175,10cyclictest4264-21bash23:11:303
937199174,12cyclictest10325-21sshd22:19:383
9371991713,3cyclictest6734-21id00:36:443
937199171,14cyclictest7063-21sshd00:30:233
937199171,14cyclictest7063-21sshd00:30:233
937099178,7cyclictest12045-21sshd22:19:482
937099177,5cyclictest0-21swapper/220:37:212
937099173,11cyclictest6304-21sshd21:45:172
937099173,10cyclictest144950irq/27-eth023:21:562
937099172,14cyclictest152450irq/28-eth1-rx-21:05:272
937099172,14cyclictest152450irq/28-eth1-rx-20:06:182
937099170,12cyclictest15479-21sshd22:57:102
936999176,3cyclictest36-21rcuop/200:38:151
936999175,11cyclictest32360-21bash22:55:391
936999174,12cyclictest29092-21sshd21:16:351
9369991713,2cyclictest0-21swapper/123:57:591
9369991711,5cyclictest144950irq/27-eth021:10:421
9369991710,6cyclictest22474-21bash23:19:291
936999170,8cyclictest0-21swapper/121:33:381
936999170,16cyclictest22-21ksoftirqd/122:35:551
936999170,16cyclictest22-21ksoftirqd/122:35:541
936599170,13cyclictest0-21swapper/022:11:460
36062170,0sleep33603-21cp23:20:383
112412170,3sleep21926-21nfsd21:58:102
937199165,2cyclictest28594-21sshd21:13:313
9371991611,4cyclictest9180-21sshd22:53:323
9371991611,4cyclictest1927-21nfsd23:57:593
9371991611,4cyclictest11791-21bash23:27:413
937199161,13cyclictest18487-21id21:43:233
9371991610,5cyclictest28672-21sshd21:16:333
937099165,10cyclictest152450irq/28-eth1-rx-19:29:192
937099164,9cyclictest152450irq/28-eth1-rx-19:36:542
937099163,8cyclictest0-21swapper/221:04:332
937099162,8cyclictest1625-21sshd23:05:112
937099161,14cyclictest152450irq/28-eth1-rx-22:40:032
937099161,14cyclictest152450irq/28-eth1-rx-20:11:392
937099161,14cyclictest152450irq/28-eth1-rx-19:32:232
937099161,14cyclictest152450irq/28-eth1-rx-19:12:582
936999169,6cyclictest15969-21sshd21:40:051
936999169,2cyclictest9166-21sshd23:33:391
9369991612,3cyclictest0-21swapper/123:10:041
936999160,3cyclictest14831-21sshd23:21:441
936599168,4cyclictest17287-21id21:18:350
936599168,4cyclictest0-21swapper/021:55:050
936599165,7cyclictest4748-21kworker/0:323:50:050
936599164,8cyclictest0-21swapper/022:36:120
936599164,8cyclictest0-21swapper/022:36:120
936599161,12cyclictest0-21swapper/000:17:110
9365991611,1cyclictest0-21swapper/023:48:020
936599160,4cyclictest19397-21bash21:34:120
936599160,4cyclictest13148-21bash23:56:010
936599160,4cyclictest0-21swapper/000:28:350
936599160,4cyclictest0-21swapper/000:06:580
936599160,3cyclictest22271-21sshd23:07:040
936599160,15cyclictest1972-21nscd21:54:180
208042160,2sleep320807-21bash22:26:403
937199159,5cyclictest0-21swapper/319:56:043
937199158,6cyclictest0-21swapper/322:41:513
937199155,1cyclictest32486-21sshd23:45:173
937199154,10cyclictest15006-21sshd21:36:483
937199153,10cyclictest0-21swapper/322:05:113
937199151,5cyclictest6376-21cp21:51:273
9371991513,1cyclictest0-21swapper/322:02:513
9371991513,1cyclictest0-21swapper/321:05:263
937199151,13cyclictest933-21sshd21:23:123
9371991511,3cyclictest25190-21sshd00:00:203
9371991511,3cyclictest17236-21munin-node23:50:213
937199151,13cyclictest0-21swapper/323:17:183
9371991510,4cyclictest0-21swapper/319:28:413
937199150,14cyclictest0-21swapper/319:10:193
937099158,6cyclictest11335-21sshd23:40:082
937099158,1cyclictest17459-21kernelversion20:55:182
937099155,9cyclictest152450irq/28-eth1-rx-20:33:142
937099154,10cyclictest152450irq/28-eth1-rx-19:54:392
937099152,8cyclictest144950irq/27-eth019:45:052
937099152,5cyclictest144950irq/27-eth020:40:172
937099150,14cyclictest152450irq/28-eth1-rx-20:47:192
937099150,14cyclictest152450irq/28-eth1-rx-20:47:192
936999159,5cyclictest23738-21df_inode00:00:131
936999159,5cyclictest144950irq/27-eth020:00:471
936999159,4cyclictest8683-21sshd22:47:151
9369991512,2cyclictest25438-21sshd23:50:531
9369991511,3cyclictest21503-21bash23:28:381
9369991510,3cyclictest13697-21sshd23:06:111
936999150,9cyclictest22-21ksoftirqd/123:00:371
936999150,14cyclictest4942-21sshd21:20:251
936999150,14cyclictest26882-21sshd21:50:201
936999150,14cyclictest22-21ksoftirqd/100:07:301
936999150,14cyclictest10956-21bash22:41:141
936999150,13cyclictest0-21swapper/121:05:201
936599158,3cyclictest10761-21sshd23:36:530
936599155,9cyclictest2106-21kworker/0:000:20:570
936599152,9cyclictest0-21swapper/023:33:050
936599152,9cyclictest0-21swapper/022:17:010
936599152,12cyclictest0-21swapper/023:10:140
936599150,4cyclictest0-21swapper/022:06:070
936599150,4cyclictest0-21swapper/020:59:410
936599150,3cyclictest11970-21sshd21:11:550
936599150,2cyclictest3018-21sshd23:02:090
936599150,14cyclictest26549-21sshd00:35:300
936599150,14cyclictest0-21swapper/023:20:530
936599150,11cyclictest12082-21sshd23:40:110
937199148,5cyclictest152450irq/28-eth1-rx-22:35:133
937199148,5cyclictest152450irq/28-eth1-rx-22:35:133
937199148,5cyclictest0-21swapper/319:41:533
9371991412,1cyclictest0-21swapper/321:45:063
9371991411,2cyclictest23368-21bash23:00:583
937199141,12cyclictest0-21swapper/323:42:593
937199141,12cyclictest0-21swapper/321:03:113
937199141,12cyclictest0-21swapper/320:37:193
937199141,12cyclictest0-21swapper/320:18:413
937199141,12cyclictest0-21swapper/320:02:003
937199141,12cyclictest0-21swapper/319:51:113
937199141,12cyclictest0-21swapper/319:45:433
937199141,12cyclictest0-21swapper/319:35:313
937199141,11cyclictest17090-21sshd00:25:033
937199140,13cyclictest0-21swapper/322:20:093
937199140,12cyclictest28252-21sshd20:09:303
937099145,9cyclictest152450irq/28-eth1-rx-19:16:582
937099145,8cyclictest0-21swapper/219:55:552
937099144,9cyclictest152450irq/28-eth1-rx-20:20:392
937099144,9cyclictest152450irq/28-eth1-rx-19:41:542
937099144,9cyclictest152450irq/28-eth1-rx-19:20:592
9369991410,3cyclictest0-21swapper/119:44:021
936999140,8cyclictest22-21ksoftirqd/123:38:001
936999140,13cyclictest2006-21snmpd19:51:111
936999140,13cyclictest19648-21idleruntime21:00:131
936599147,4cyclictest0-21swapper/021:00:570
936599140,2cyclictest3262-21sshd22:25:040
936599140,1cyclictest6998-21sshd21:26:480
936599140,13cyclictest8903-21sshd00:33:490
936599140,13cyclictest8903-21sshd00:33:480
936599140,10cyclictest32528-21sshd21:38:320
937199132,10cyclictest25-21rcuop/120:45:093
937199132,10cyclictest25-21rcuop/120:45:083
937199131,4cyclictest0-21swapper/319:30:203
937199131,4cyclictest0-21swapper/319:18:053
937199131,11cyclictest29697-21runrttasks20:10:333
937199131,11cyclictest14249-21munin-node19:20:133
937199131,11cyclictest0-21swapper/320:55:153
937199131,11cyclictest0-21swapper/320:50:093
937199131,11cyclictest0-21swapper/320:41:393
937199131,11cyclictest0-21swapper/320:30:073
937199131,11cyclictest0-21swapper/320:28:023
937199130,11cyclictest1383-21iostat20:20:173
937099132,10cyclictest1927-21nfsd20:25:212
936999137,5cyclictest0-21swapper/119:35:061
936999131,4cyclictest1994-21snmpd20:35:061
9369991311,1cyclictest0-21swapper/120:07:021
936999130,5cyclictest0-21swapper/119:46:551
936999130,4cyclictest2017-21switchtime20:20:261
936999130,3cyclictest29838-21munin-node20:10:381
936999130,2cyclictest152450irq/28-eth1-rx-19:23:091
936999130,12cyclictest1994-21snmpd20:33:101
936999130,12cyclictest152450irq/28-eth1-rx-20:40:231
936999130,12cyclictest0-21swapper/119:19:391
936999130,12cyclictest0-21swapper/119:13:421
936999130,11cyclictest17098-21df19:30:141
936999130,11cyclictest15192-21irqstats20:50:211
936999130,11cyclictest12404-21df20:45:141
936999130,11cyclictest12404-21df20:45:141
936599137,5cyclictest0-21swapper/019:50:250
936599136,2cyclictest0-21swapper/020:24:590
936599130,12cyclictest0-21swapper/020:10:220
936599130,12cyclictest0-21swapper/019:26:410
936599130,10cyclictest0-21swapper/020:45:470
936599130,10cyclictest0-21swapper/020:45:460
936599130,10cyclictest0-21swapper/020:25:010
936599130,10cyclictest0-21swapper/020:05:070
936599130,10cyclictest0-21swapper/019:30:420
936599130,10cyclictest0-21swapper/019:22:100
936599130,10cyclictest0-21swapper/019:15:240
936999128,3cyclictest1927-21nfsd20:55:001
936999120,4cyclictest24420-21open_files19:55:181
936999120,11cyclictest0-21swapper/120:25:501
936999120,11cyclictest0-21swapper/120:15:051
936999120,11cyclictest0-21swapper/119:25:521
936599121,8cyclictest0-21swapper/020:50:070
9365991210,1cyclictest0-21swapper/020:40:050
936599120,2cyclictest0-21swapper/020:37:400
936599120,11cyclictest0-21swapper/021:05:210
936599120,11cyclictest0-21swapper/020:30:420
936599120,11cyclictest0-21swapper/019:41:390
936599120,11cyclictest0-21swapper/019:35:160
936599120,11cyclictest0-21swapper/019:10:260
936599120,10cyclictest0-21swapper/020:15:010
936599120,10cyclictest0-21swapper/020:00:380
936599120,10cyclictest0-21swapper/019:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional