You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-28 - 00:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackaslot0.osadl.org (updated Tue Feb 27, 2024 12:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
203362980,1sleep228-21rcuc/210:50:432
89762870,1sleep011236-21diskmemload10:29:580
250922650,1sleep225075-21sshd09:21:532
61592630,1sleep06143-21sshd11:31:190
1786526326,19sleep10-21swapper/107:07:141
197462610,1sleep319741-21bash09:14:333
237532600,0sleep08-21rcu_preempt11:48:060
14902590,1sleep30-21swapper/311:03:183
145892590,1sleep31784-21runrttasks10:17:113
145892590,1sleep31784-21runrttasks10:17:113
318952580,0sleep131893-21bash09:32:011
251362580,0sleep30-21swapper/310:34:493
293962570,1sleep11784-21runrttasks11:19:261
276522570,0sleep10-21swapper/109:18:561
6482550,1sleep21547-21nfsd11:16:552
77132540,1sleep37693-21sshd09:16:333
172502540,0sleep00-21swapper/009:34:010
9202530,0sleep20-21swapper/210:19:142
9202530,0sleep20-21swapper/210:19:142
71052530,1sleep337-21rcuc/310:58:173
46092530,1sleep14605-21sshd09:26:081
1775525332,17sleep20-21swapper/207:05:512
159522530,0sleep215941-21sshd11:41:042
315962520,0sleep10-21swapper/112:19:201
46922510,2sleep31834899cyclictest11:28:273
326162510,0sleep00-21swapper/010:52:010
20632510,0sleep32051-21sshd10:29:163
25512500,0sleep00-21swapper/010:42:240
25512500,0sleep00-21swapper/010:42:240
225482500,0sleep00-21swapper/009:24:510
1595025029,18sleep30-21swapper/307:05:123
326822490,1sleep21546-21nfsd10:42:102
326822490,1sleep21546-21nfsd10:42:092
181602490,0sleep20-21swapper/212:29:232
1778524927,18sleep00-21swapper/007:06:130
308212480,2sleep31834899cyclictest12:24:513
280902480,0sleep30-21swapper/311:05:303
177652480,0sleep017744-21sshd10:56:210
105882480,1sleep011236-21diskmemload10:23:400
297162470,0sleep329713-21sshd12:13:423
221932470,0sleep10-21swapper/110:31:111
187232470,0sleep228-21rcuc/209:57:342
113522470,0sleep211305-21sshd10:55:482
287942460,0sleep128792-21sshd11:27:401
182832460,0sleep311236-21diskmemload11:41:163
132112450,0sleep30-21swapper/310:40:153
132112450,0sleep30-21swapper/310:40:153
73552430,0sleep2291ktimersoftd/211:31:262
39712430,2sleep11834699cyclictest12:33:441
161632430,2sleep21834799cyclictest11:07:132
52132370,0sleep35215-21id12:03:253
1834599270,26cyclictest29884-21cp10:12:060
16762250,0sleep01668-21bash11:36:560
1834699212,5cyclictest19377-21sshd09:10:401
1834699212,4cyclictest10139-21sshd11:52:251
18348992014,5cyclictest0-21swapper/312:09:343
1834899200,15cyclictest28024-21sshd12:27:243
18347992018,1cyclictest0-21swapper/209:54:352
1834699201,2cyclictest0-21swapper/112:36:511
1834599200,1cyclictest0-21swapper/012:32:240
1834599200,15cyclictest0-21swapper/012:02:540
198032190,1sleep30-21swapper/311:32:383
1834899198,6cyclictest19573-21sshd09:57:403
18348991916,2cyclictest11236-21diskmemload11:36:113
1834899190,1cyclictest0-21swapper/310:06:083
1834699199,1cyclictest120050irq/25-eth012:23:521
1834699192,3cyclictest26576-21sshd11:39:111
1834699191,1cyclictest0-21swapper/109:49:151
18346991910,5cyclictest25950-21sshd11:56:521
1834699190,3cyclictest12150-21bash10:53:081
1834599193,15cyclictest14900-21sshd09:47:160
1834599190,18cyclictest5817-21sshd11:14:420
1834599190,18cyclictest0-21swapper/009:10:560
133062190,1sleep213282-21sshd11:20:432
131152190,1sleep213091-21sshd11:58:382
119002190,2sleep00-21swapper/011:40:410
60032180,1sleep25999-21rm11:03:422
1834899189,5cyclictest0-21swapper/311:21:573
1834899188,7cyclictest5033-21sshd11:57:533
1834899180,1cyclictest18735-21sshd10:50:353
1834899180,16cyclictest4172-21bash09:26:063
1834799187,2cyclictest0-21swapper/211:28:182
1834799182,15cyclictest13980-21rm10:20:312
18347991812,5cyclictest0-21swapper/212:31:132
1834699184,3cyclictest8085-21cp10:36:151
1834699183,3cyclictest4261-21bash10:29:301
1834699182,13cyclictest31370-21sshd09:38:531
18346991811,4cyclictest32558-21sshd11:30:461
18346991811,4cyclictest0-21swapper/112:10:551
1834699181,13cyclictest25242-21sshd10:11:351
1834699180,13cyclictest0-21swapper/112:25:261
1834599188,9cyclictest24672-21sshd12:15:530
1834599188,9cyclictest12923-21bash10:33:320
1834599188,6cyclictest20936-21sshd11:07:410
1834599186,11cyclictest8374-21sshd12:22:520
1834599180,1cyclictest16920-21sshd09:50:440
1834899178,5cyclictest0-21swapper/311:12:413
1834899176,1cyclictest0-21swapper/309:47:153
1834899171,15cyclictest0-21swapper/309:32:583
1834899171,10cyclictest26197-21sshd12:18:523
1834899170,1cyclictest2258-21sshd10:39:073
1834899170,16cyclictest21813-21sshd09:54:403
1834899170,16cyclictest0-21swapper/310:45:343
1834799172,10cyclictest20853-21sshd11:47:512
1834799170,9cyclictest0-21swapper/210:39:062
1834799170,16cyclictest21590-21sshd11:35:472
1834799170,16cyclictest0-21swapper/209:48:442
1834799170,10cyclictest4174-21bash09:26:062
1834699172,5cyclictest19929-21sshd11:24:111
1834699172,4cyclictest6899-21sshd10:46:061
1834699172,3cyclictest24600-21id10:41:171
1834699172,3cyclictest24600-21id10:41:171
1834699172,12cyclictest0-21swapper/111:40:231
1834699172,11cyclictest4729-21sshd11:06:151
18346991711,4cyclictest21036-21sshd10:07:351
1834699171,13cyclictest0-21swapper/111:48:511
1834699171,13cyclictest0-21swapper/110:58:371
1834699171,11cyclictest492-21jbd2/dm-0-812:06:261
1834599179,7cyclictest0-21swapper/009:37:520
1834599178,8cyclictest19536-21sshd09:40:500
1834599178,8cyclictest0-21swapper/012:06:530
1834599177,9cyclictest25447-21sshd09:25:070
1834599175,7cyclictest13518-21sshd11:50:000
1834599173,2cyclictest0-21swapper/012:10:050
1834599171,2cyclictest0-21swapper/010:46:350
1834599170,1cyclictest0-21swapper/012:26:530
1834599170,1cyclictest0-21swapper/011:01:090
1834599170,1cyclictest0-21swapper/010:35:320
1834599170,16cyclictest32476-21sshd11:25:160
1834599170,16cyclictest26665-21sshd09:15:170
1834599170,16cyclictest1632-21sshd11:58:250
1834599170,16cyclictest0-21swapper/010:06:370
1834899160,1cyclictest13712-21sshd10:20:293
1834899160,1cyclictest0-21swapper/311:15:553
1834899160,1cyclictest0-21swapper/309:20:433
1834899160,15cyclictest2709-21bash11:45:503
1834899160,15cyclictest2670-21cp11:54:503
1834899160,15cyclictest26677-21sshd09:38:233
1834899160,15cyclictest0-21swapper/312:35:483
1834899160,15cyclictest0-21swapper/312:30:223
1834899160,15cyclictest0-21swapper/309:41:493
1834899160,10cyclictest11980-21sshd10:00:103
1834899160,10cyclictest0-21swapper/310:10:343
1834799169,6cyclictest14319-21sshd12:01:242
1834799169,6cyclictest0-21swapper/212:35:482
1834799168,7cyclictest4601-21sshd10:29:322
1834799168,7cyclictest0-21swapper/212:10:532
1834799168,7cyclictest0-21swapper/210:12:052
1834799161,1cyclictest0-21swapper/212:16:262
1834799161,14cyclictest0-21swapper/210:02:092
1834799160,9cyclictest3176-21bash12:05:522
1834799160,1cyclictest28900-21sshd10:08:352
1834799160,1cyclictest21834-21sshd09:31:012
1834799160,1cyclictest21667-21sshd09:18:132
1834799160,1cyclictest17112-21bash09:10:302
1834799160,1cyclictest16059-21sshd11:12:422
1834799160,15cyclictest14013-21id09:40:212
1834799160,15cyclictest12733-21sshd09:36:552
1834799160,10cyclictest17586-21sshd11:50:222
1834799160,10cyclictest1683-21ntpq10:45:342
1834799160,10cyclictest0-21swapper/210:30:312
1834699169,3cyclictest0-21swapper/110:00:391
1834699163,2cyclictest2953-21sshd10:15:551
1834699163,2cyclictest2953-21sshd10:15:541
1834699162,3cyclictest4008-21sshd09:52:561
1834699162,3cyclictest15389-21bash11:01:411
1834699161,3cyclictest19831-21sshd09:24:361
18345991610,5cyclictest0-21swapper/010:02:380
1834599160,15cyclictest29303-21id11:16:360
1834599160,15cyclictest22345-21sshd10:18:010
1834599160,15cyclictest22345-21sshd10:18:010
1834599160,15cyclictest14065-21sshd12:37:200
1834599160,15cyclictest0-21swapper/011:21:530
1834799150,14cyclictest18084-21sshd12:20:532
1834799150,11cyclictest0-21swapper/207:43:442
1834699153,3cyclictest9782-21sshd09:56:391
1834699153,3cyclictest12354-21bash12:01:141
1834699152,3cyclictest13042-21sshd10:20:271
1834699151,5cyclictest0-21swapper/109:40:321
1834699151,3cyclictest32176-21sshd11:11:191
1834599157,7cyclictest4575-21sshd09:56:050
1834599155,6cyclictest0-21swapper/008:15:210
1834799140,10cyclictest0-21swapper/208:10:202
1834599140,13cyclictest1425-21seq08:48:220
1834599140,13cyclictest1425-21seq08:48:220
1834899131,5cyclictest0-21swapper/307:20:203
1834599132,4cyclictest0-21swapper/008:35:210
1834899125,1cyclictest20367-21munin-node08:20:253
1834899121,5cyclictest0-21swapper/307:45:383
1834899120,6cyclictest0-21swapper/307:55:213
1834799128,4cyclictest0-21swapper/207:55:352
136022120,0sleep00-21swapper/008:05:250
1834899115,1cyclictest18655-21cstates07:10:203
1834899111,5cyclictest0-21swapper/308:25:193
1834899111,5cyclictest0-21swapper/308:05:203
1834899110,3cyclictest0-21swapper/308:40:323
1834899110,3cyclictest0-21swapper/308:00:313
18347991110,1cyclictest0-21swapper/208:55:172
1834799110,10cyclictest0-21swapper/209:07:292
1834699111,2cyclictest0-21swapper/107:30:291
18346991110,1cyclictest0-21swapper/108:51:571
1834599113,2cyclictest0-21swapper/007:15:170
1834599112,4cyclictest0-21swapper/007:30:010
1834599112,3cyclictest0-21swapper/008:26:310
1834599112,1cyclictest25191-21diskstats08:30:240
1834599111,1cyclictest0-21swapper/008:10:270
1834599110,7cyclictest0-21swapper/009:09:260
1834599110,6cyclictest0-21swapper/009:00:290
1834599110,6cyclictest0-21swapper/007:35:210
1834599110,5cyclictest0-21swapper/007:40:320
1834599110,1cyclictest0-21swapper/008:50:220
1834599110,1cyclictest0-21swapper/007:25:060
1834599110,1cyclictest0-21swapper/007:23:170
1834599110,1cyclictest0-21swapper/007:10:330
1834899100,5cyclictest0-21swapper/308:10:223
1834899100,5cyclictest0-21swapper/307:40:313
1834799109,1cyclictest0-21swapper/208:48:192
1834799109,1cyclictest0-21swapper/208:48:192
1834799109,1cyclictest0-21swapper/208:28:482
1834799109,1cyclictest0-21swapper/208:21:152
1834799100,1cyclictest0-21swapper/209:01:192
1834799100,10cyclictest0-21swapper/208:35:032
1834699109,1cyclictest0-21swapper/108:04:061
1834699103,5cyclictest0-21swapper/107:35:291
1834699102,3cyclictest11276-21latency_hist09:10:011
1834699100,10cyclictest0-21swapper/107:13:201
1834599100,5cyclictest0-21swapper/007:55:300
1834599100,1cyclictest0-21swapper/008:55:080
1834599100,1cyclictest0-21swapper/008:40:010
1834599100,1cyclictest0-21swapper/008:20:560
1834599100,1cyclictest0-21swapper/008:00:180
1834599100,1cyclictest0-21swapper/007:50:350
1834599100,1cyclictest0-21swapper/007:45:060
183489990,3cyclictest0-21swapper/307:25:263
183479998,1cyclictest0-21swapper/208:34:572
183479991,7cyclictest0-21swapper/208:05:312
183469998,1cyclictest23129-21fschecks_time08:25:221
183469998,1cyclictest0-21swapper/107:46:401
183469992,6cyclictest31908-21cut08:45:261
183469992,6cyclictest31908-21cut08:45:261
183469992,5cyclictest8765-21munin-run09:05:011
183469992,1cyclictest723-40kipmi008:35:281
183469991,3cyclictest7824-21latency_hist07:55:001
183469990,9cyclictest0-21swapper/108:20:421
183489982,4cyclictest120050irq/25-eth008:31:563
183489981,6cyclictest0-21swapper/307:50:203
183489980,1cyclictest0-21swapper/308:49:083
183489980,1cyclictest0-21swapper/308:49:083
183479987,1cyclictest0-21swapper/208:41:212
183469983,4cyclictest4608-21munin-node08:55:231
183469982,4cyclictest0-21swapper/108:30:271
183469982,3cyclictest0-21swapper/108:10:201
183469980,7cyclictest0-21swapper/107:40:301
183469980,5cyclictest0-21swapper/107:20:321
183489975,1cyclictest8040-21munin-node09:00:323
183489975,1cyclictest4980-21perl08:55:253
183489975,1cyclictest2830-21latency08:50:293
183489975,1cyclictest10364-21munin-node09:05:333
183489973,1cyclictest7707-21kworker/3:108:15:263
183489970,7cyclictest0-21swapper/307:30:283
183489970,4cyclictest0-21swapper/307:17:363
183479971,5cyclictest0-21swapper/208:00:262
183479971,5cyclictest0-21swapper/207:45:342
183479971,5cyclictest0-21swapper/207:15:012
183479970,6cyclictest0-21swapper/207:20:242
183479970,6cyclictest0-21swapper/207:10:182
183469973,3cyclictest30711-21grep08:40:281
183469972,5cyclictest0-21swapper/107:55:281
183469972,4cyclictest21365-21fschecks_count07:15:021
183469971,5cyclictest13517-21cut08:05:251
183469971,2cyclictest0-21swapper/108:15:271
183469970,6cyclictest0-21swapper/107:25:211
183489960,1cyclictest0-21swapper/308:35:033
183489960,1cyclictest0-21swapper/307:35:013
183479960,5cyclictest5688-21apache_accesses07:50:202
183479960,5cyclictest30960-21apache_accesses07:35:212
183479960,5cyclictest1855-21apache_volume08:50:202
183479960,5cyclictest17978-21df08:15:222
183479960,5cyclictest0-21swapper/207:25:232
183479950,5cyclictest0-21swapper/207:30:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional