You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-22 - 12:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Thu May 22, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2602921780,3sleep01578199cyclictest00:27:340
2826121490,4sleep11578299cyclictest21:54:511
1248621400,0sleep012485-21sshd22:23:490
1227521080,2sleep10-21swapper/121:28:551
2464721060,2sleep224645-21id22:06:362
2777821010,3sleep127781-21bash23:17:561
15160210131,42sleep30-21swapper/319:05:343
79032980,3sleep10-21swapper/121:46:471
100132980,1sleep010007-21bash22:32:460
250042960,0sleep00-21swapper/023:51:510
316632950,1sleep031661-21bash22:53:310
4662930,2sleep30-21swapper/323:33:543
251062920,1sleep01848-21nfsd21:51:270
195472900,1sleep20-21swapper/221:35:382
195472900,1sleep20-21swapper/221:35:382
283602880,1sleep328345-21sshd21:51:463
263392880,2sleep326341-21bash21:33:183
105462880,1sleep10-21swapper/100:29:311
316582860,2sleep331640-21sshd00:01:563
322762850,1sleep132274-21bash21:33:521
264562850,1sleep026457-21bash00:07:460
313822840,2sleep331365-21sshd21:15:223
295612840,3sleep10-21swapper/121:42:461
291562840,4sleep11578299cyclictest23:52:161
270032840,2sleep20-21swapper/222:12:572
250162830,2sleep125005-21sshd21:14:521
190192830,1sleep31848-21nfsd22:58:313
174482830,2sleep20-21swapper/223:20:062
1531728336,43sleep10-21swapper/119:07:271
56012800,2sleep20-21swapper/200:39:532
276602790,2sleep30-21swapper/323:11:423
258172790,5sleep21578399cyclictest22:00:332
106312790,1sleep30-21swapper/300:25:483
299372780,1sleep11578299cyclictest23:39:531
168902780,1sleep30-21swapper/323:20:033
219792760,2sleep321976-21sshd22:24:443
116332750,1sleep011628-21bash00:16:030
316632730,2sleep131654-21users22:25:291
82992710,3sleep21578399cyclictest23:50:162
188532710,1sleep00-21swapper/023:17:040
165222710,2sleep216523-21bash23:32:172
149222690,2sleep20-21swapper/222:30:122
89112680,2sleep20-21swapper/223:59:502
320762670,2sleep21847-21nfsd23:49:282
83612660,2sleep11847-21nfsd23:47:071
61542660,1sleep10-21swapper/123:59:331
213762660,1sleep20-21swapper/223:26:322
213762660,1sleep20-21swapper/223:26:312
1546226035,12sleep20-21swapper/219:09:142
280422560,0sleep00-21swapper/023:14:550
1535125633,11sleep00-21swapper/019:07:480
311892550,2sleep10-21swapper/122:53:281
192412480,2sleep20-21swapper/221:29:362
15782993934,3cyclictest2664-21sshd23:30:531
1578199380,1cyclictest0-21swapper/023:40:030
15782993631,4cyclictest1848-21nfsd00:13:381
245632340,3sleep10-21swapper/122:09:431
1578399330,21cyclictest33-21ksoftirqd/223:06:062
15781993328,4cyclictest1959-21sshd22:41:080
67432320,0sleep36742-21sshd21:49:493
175332320,29sleep3135650irq/27-eth000:23:173
1578399312,4cyclictest13571-21sshd00:09:502
1578399310,23cyclictest33-21ksoftirqd/200:18:462
15781993121,4cyclictest469-21sshd21:27:460
272602300,3sleep227253-21sshd22:40:262
1578199300,16cyclictest0-21swapper/022:05:100
193862290,2sleep119383-21cp22:18:211
15784992924,4cyclictest1847-21nfsd22:41:083
120252290,4sleep30-21swapper/322:02:243
230502280,2sleep123049-21id22:37:011
15783992813,4cyclictest135650irq/27-eth023:44:102
1578199281,19cyclictest10279-21kworker/0:123:26:240
1578199281,19cyclictest10279-21kworker/0:123:26:240
1578199280,19cyclictest0-21swapper/000:31:380
112732280,26sleep3135650irq/27-eth023:07:053
284372270,2sleep328430-21rm00:38:473
1578399273,17cyclictest0-21swapper/223:15:022
1578399270,5cyclictest20550-21id22:58:402
48472260,1sleep04845-21sshd23:37:230
186792260,4sleep20-21swapper/200:26:442
15783992623,2cyclictest5555-21sshd21:52:412
15781992618,5cyclictest0-21swapper/000:13:380
1578199260,24cyclictest12194-21sshd00:00:090
1578199260,15cyclictest16336-21sshd23:04:330
293562252,21sleep3135650irq/27-eth000:08:063
1578399254,19cyclictest33-21ksoftirqd/200:32:192
1578399251,22cyclictest33-21ksoftirqd/223:11:532
1578399250,4cyclictest1847-21nfsd22:52:552
1578199250,3cyclictest26521-21sshd23:45:420
82892240,3sleep18285-21sshd00:22:181
256772240,0sleep20-21swapper/221:30:112
1578499248,3cyclictest22326-21sshd22:52:353
1578499246,5cyclictest0-21swapper/321:38:513
1578499246,5cyclictest0-21swapper/321:38:503
1578399243,4cyclictest33-21ksoftirqd/222:35:392
15783992412,7cyclictest10950irq/23-ehci_hcd21:21:152
1578399240,3cyclictest9836-21sshd22:29:432
1578399240,22cyclictest33-21ksoftirqd/221:42:592
1578399240,21cyclictest33-21ksoftirqd/221:56:182
1578399240,21cyclictest135650irq/27-eth000:13:502
1578299245,14cyclictest26950irq/26-ahci[00000:30:411
15781992421,2cyclictest2654-21sshd23:30:540
15781992415,6cyclictest3-21ksoftirqd/023:57:260
63552230,2sleep10-21swapper/100:02:381
303182231,1sleep00-21swapper/021:48:590
245162230,4sleep11578299cyclictest22:21:501
198112230,21sleep1135650irq/27-eth023:07:561
188212230,2sleep30-21swapper/323:26:173
188212230,2sleep30-21swapper/323:26:163
1578499236,2cyclictest27181-21bash21:57:453
15784992317,4cyclictest0-21swapper/321:27:463
1578399238,4cyclictest30957-21sshd22:19:292
1578399234,17cyclictest33-21ksoftirqd/221:49:312
1578399233,18cyclictest33-21ksoftirqd/223:36:432
1578399230,21cyclictest33-21ksoftirqd/223:00:422
1578399230,20cyclictest33-21ksoftirqd/222:46:152
15782992318,3cyclictest1848-21nfsd21:55:371
1578299230,14cyclictest22-21ksoftirqd/123:04:161
1578199237,6cyclictest0-21swapper/022:46:280
15781992316,5cyclictest3-21ksoftirqd/022:25:300
15781992314,6cyclictest3-21ksoftirqd/023:06:030
15781992314,5cyclictest0-21swapper/021:17:150
1578199230,21cyclictest18379-21kworker/0:222:59:080
3532223,17sleep210950irq/23-ehci_hcd00:02:032
15783992212,9cyclictest33-21ksoftirqd/222:20:242
1578399221,19cyclictest33-21ksoftirqd/200:24:422
15782992216,4cyclictest23948-21sshd23:26:471
15782992216,4cyclictest23948-21sshd23:26:471
15782992215,6cyclictest0-21swapper/121:36:131
15782992215,6cyclictest0-21swapper/121:36:131
15782992210,4cyclictest21178-21sshd22:58:441
1578199224,15cyclictest15865-21sshd21:44:330
1578199221,2cyclictest17374-21sshd00:37:290
15781992210,6cyclictest3-21ksoftirqd/022:17:470
1578199220,2cyclictest0-21swapper/000:20:010
211912213,16sleep110950irq/23-ehci_hcd23:23:311
170122212,17sleep3135650irq/27-eth020:21:513
1578499215,4cyclictest8876-21sshd22:14:203
15783992116,3cyclictest14494-21sshd21:16:502
1578299219,4cyclictest21713-21sshd22:49:271
1578199219,6cyclictest1847-21nfsd21:24:300
1578199219,11cyclictest0-21swapper/021:35:010
1578199219,11cyclictest0-21swapper/021:35:010
1578199215,8cyclictest3-21ksoftirqd/021:55:360
1578199211,1cyclictest6517-21sshd22:35:260
1578199210,2cyclictest27258-21sshd21:30:170
1578199210,19cyclictest17311-21sshd21:10:590
321912200,2sleep10-21swapper/100:39:131
1578499207,4cyclictest10950irq/23-ehci_hcd23:46:593
1578499201,4cyclictest0-21swapper/323:44:193
1578399207,9cyclictest33-21ksoftirqd/221:12:222
1578299208,6cyclictest11315-21rm21:22:401
15782992014,3cyclictest1845-21nfsd23:13:261
15782992010,2cyclictest22464-21sshd22:12:301
1578199203,4cyclictest5826-21sshd22:14:020
1578199200,2cyclictest7439-21bash22:01:580
1578199200,1cyclictest13691-21sshd23:22:470
1578199200,16cyclictest0-21swapper/019:10:120
1578499198,8cyclictest1042-21sshd22:38:023
1578499193,4cyclictest24548-21bash21:23:583
1578499193,2cyclictest22488-21bash00:34:333
1578499190,8cyclictest1845-21nfsd00:10:393
1578499190,6cyclictest0-21swapper/322:32:073
1578299197,2cyclictest22164-21sshd22:40:031
15782991915,2cyclictest0-21swapper/121:16:481
15782991914,3cyclictest903-21sshd23:40:131
15782991913,5cyclictest135650irq/27-eth000:19:201
1578199190,9cyclictest0-21swapper/020:25:200
1578199190,16cyclictest0-21swapper/020:35:310
1578499185,5cyclictest10950irq/23-ehci_hcd21:10:533
1578499182,5cyclictest14745-21bash22:27:013
1578499181,5cyclictest135650irq/27-eth021:41:393
15784991814,2cyclictest0-21swapper/323:37:543
1578499181,2cyclictest1846-21nfsd00:15:383
15784991811,5cyclictest2633-21bash23:52:513
15784991811,5cyclictest1848-21nfsd23:57:263
1578499180,6cyclictest25458-21bash23:02:143
15782991812,4cyclictest1848-21nfsd22:32:451
15782991811,5cyclictest3007-21sshd22:01:321
15782991811,5cyclictest1893-21nscd00:07:351
1578199180,16cyclictest0-21swapper/020:24:490
1578499174,1cyclictest1848-21nfsd22:16:363
1578499173,3cyclictest0-21swapper/322:48:583
1578499171,2cyclictest11532-21sshd23:19:333
15781991712,3cyclictest0-21swapper/021:00:220
1578199170,5cyclictest0-21swapper/019:28:400
1578199170,1cyclictest0-21swapper/021:07:490
1578199170,10cyclictest0-21swapper/020:51:270
1578499168,6cyclictest135650irq/27-eth022:05:103
1578299168,1cyclictest15558-21df_inode20:20:131
1578199166,8cyclictest0-21swapper/020:57:360
1578199160,1cyclictest0-21swapper/019:42:280
1578199160,1cyclictest0-21swapper/019:42:280
1578199160,15cyclictest0-21swapper/020:34:140
1578199160,15cyclictest0-21swapper/020:18:020
1578199160,15cyclictest0-21swapper/020:00:160
1578199160,15cyclictest0-21swapper/019:54:000
1578199160,10cyclictest0-21swapper/019:20:130
1578499154,4cyclictest10950irq/23-ehci_hcd19:55:183
1578399156,4cyclictest10950irq/23-ehci_hcd20:35:172
1578299154,10cyclictest10950irq/23-ehci_hcd20:41:581
15782991512,2cyclictest10950irq/23-ehci_hcd19:29:311
15782991511,3cyclictest10950irq/23-ehci_hcd19:41:101
15782991511,3cyclictest10950irq/23-ehci_hcd19:41:091
1578299150,11cyclictest135650irq/27-eth020:25:041
1578199157,7cyclictest0-21swapper/019:15:460
1578199157,4cyclictest0-21swapper/019:57:340
1578199150,7cyclictest0-21swapper/020:45:130
1578199150,7cyclictest0-21swapper/019:31:270
1578199150,1cyclictest0-21swapper/020:09:100
1578199150,14cyclictest0-21swapper/020:44:140
1578199150,12cyclictest0-21swapper/020:12:440
1578199150,12cyclictest0-21swapper/019:45:480
1578499144,2cyclictest0-21swapper/320:25:203
1578399143,10cyclictest10950irq/23-ehci_hcd21:08:122
1578399141,12cyclictest135650irq/27-eth020:46:552
15783991410,3cyclictest10950irq/23-ehci_hcd19:36:492
15783991410,3cyclictest10950irq/23-ehci_hcd19:15:582
1578299143,10cyclictest10950irq/23-ehci_hcd19:52:441
15782991411,2cyclictest10950irq/23-ehci_hcd19:49:271
15782991410,3cyclictest10950irq/23-ehci_hcd20:48:511
15782991410,3cyclictest10950irq/23-ehci_hcd19:31:221
1578299140,3cyclictest10950irq/23-ehci_hcd20:50:131
1578299140,12cyclictest1915-21snmpd19:35:321
1578199146,7cyclictest0-21swapper/019:35:190
1578499139,3cyclictest10950irq/23-ehci_hcd21:04:273
1578499139,3cyclictest0-21swapper/319:42:543
1578499139,3cyclictest0-21swapper/319:42:543
1578499138,4cyclictest0-21swapper/319:18:183
1578499135,2cyclictest135650irq/27-eth019:25:073
1578499132,10cyclictest10950irq/23-ehci_hcd20:39:063
1578499131,11cyclictest10950irq/23-ehci_hcd19:32:463
1578399137,5cyclictest33-21ksoftirqd/219:20:412
1578399137,5cyclictest10950irq/23-ehci_hcd20:00:052
1578399137,2cyclictest10950irq/23-ehci_hcd20:25:202
1578399133,9cyclictest0-21swapper/220:40:522
1578399132,10cyclictest135650irq/27-eth019:56:472
1578399131,3cyclictest25206-21diskstats19:30:122
1578399131,3cyclictest1233-21df21:00:122
1578399131,11cyclictest144550irq/28-eth1-rx-19:27:362
15783991311,1cyclictest0-21swapper/220:17:032
15783991311,1cyclictest0-21swapper/220:05:172
15783991310,2cyclictest30206-21irqstats19:40:172
15783991310,2cyclictest30206-21irqstats19:40:172
15783991310,2cyclictest135650irq/27-eth020:10:162
1578399130,4cyclictest29197-21diskstats20:50:142
1578399130,12cyclictest0-21swapper/220:30:072
1578399130,12cyclictest0-21swapper/220:22:382
15782991311,1cyclictest0-21swapper/120:13:381
15782991311,1cyclictest0-21swapper/120:00:131
1578299130,5cyclictest0-21swapper/119:57:401
1578299130,4cyclictest13227-21df20:15:101
1578299130,12cyclictest2405-21runrttasks20:32:311
1578299130,12cyclictest1848-21nfsd20:37:231
1578299130,12cyclictest0-21swapper/119:13:261
1578299130,11cyclictest955-21munin-node21:00:231
1578299130,11cyclictest20663-21diskstats19:20:121
1578299130,11cyclictest18741-21hddtemp_smartct19:15:161
1578299130,11cyclictest0-21swapper/120:55:161
1578499129,2cyclictest10950irq/23-ehci_hcd20:18:023
1578499127,4cyclictest10950irq/23-ehci_hcd20:40:233
1578499123,0cyclictest0-21swapper/319:20:163
1578499122,3cyclictest10950irq/23-ehci_hcd19:45:213
1578499122,2cyclictest10950irq/23-ehci_hcd19:53:093
1578499121,10cyclictest135650irq/27-eth020:45:133
1578399128,3cyclictest17693-21sshd19:13:172
1578399126,5cyclictest0-21swapper/219:50:392
1578399124,4cyclictest135650irq/27-eth019:45:212
1578399120,11cyclictest0-21swapper/220:54:592
1578299120,3cyclictest0-21swapper/121:05:171
1578299120,11cyclictest0-21swapper/120:05:071
1578499115,5cyclictest0-21swapper/320:05:213
1578499111,9cyclictest0-21swapper/320:34:143
1578499111,9cyclictest0-21swapper/320:00:163
125162110,1sleep30-21swapper/320:12:283
175432100,1sleep30-21swapper/319:11:303
1578499106,3cyclictest3654-21diskstats21:05:173
1578499105,4cyclictest135650irq/27-eth019:35:053
1578499105,4cyclictest0-21swapper/320:56:483
1578499100,9cyclictest0-21swapper/320:50:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional