You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 12:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Wed Apr 24, 2024 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3102921370,3sleep01014999cyclictest21:32:210
1415721230,4sleep31015799cyclictest23:25:503
315472920,0sleep03-21ksoftirqd/021:36:020
4802900,1sleep2477-21bash23:39:232
987828841,43sleep30-21swapper/319:09:133
11902880,3sleep10-21swapper/121:47:251
104022870,1sleep010-21rcuc/000:34:560
315092860,2sleep21921-21nscd21:28:432
158682860,2sleep115758-21sshd22:11:231
66612850,2sleep06657-21sshd22:51:210
161372850,4sleep116136-21latency00:15:181
13302850,1sleep31877-21nfsd00:05:423
13302850,1sleep31877-21nfsd00:05:423
118902840,2sleep00-21swapper/021:41:120
792728335,44sleep10-21swapper/119:05:101
52372830,0sleep25236-21sshd22:17:362
27362820,0sleep00-21swapper/023:24:380
221362820,1sleep21876-21nfsd00:08:182
221362820,1sleep21876-21nfsd00:08:172
193702810,2sleep119369-21bash22:41:351
132822810,0sleep00-21swapper/021:26:330
264942800,3sleep31015799cyclictest22:27:313
12042800,1sleep30-21swapper/322:43:193
111422800,2sleep10-21swapper/123:21:471
213062790,0sleep00-21swapper/022:49:240
274742780,2sleep227475-21sshd00:28:532
32862770,2sleep30-21swapper/321:10:293
231232710,3sleep01014999cyclictest22:42:020
126772710,1sleep20-21swapper/222:55:432
989226945,11sleep20-21swapper/219:09:232
318732670,3sleep01014999cyclictest23:01:420
966526541,12sleep00-21swapper/019:06:460
210862650,0sleep121082-21bash21:16:151
184092340,4sleep20-21swapper/200:19:392
89472320,30sleep228750irq/26-ahci[00021:59:292
247952290,2sleep30-21swapper/323:15:493
98222270,1sleep00-21swapper/021:52:050
31562260,2sleep10-21swapper/121:58:481
307152260,2sleep32121-21runrttasks23:42:493
24302260,24sleep1138550irq/27-eth023:58:161
167752260,1sleep016773-21sshd23:56:040
283772250,4sleep11015099cyclictest00:05:111
283772250,4sleep11015099cyclictest00:05:111
149992250,2sleep10-21swapper/123:18:301
291142240,0sleep00-21swapper/022:01:440
203132240,1sleep20-21swapper/223:07:542
190042240,1sleep08-21rcu_preempt22:37:510
106392240,2sleep01014999cyclictest22:07:040
106392240,2sleep01014999cyclictest22:07:040
56702220,3sleep22121-21runrttasks21:21:572
232142220,2sleep10-21swapper/123:26:571
221882220,3sleep10-21swapper/100:28:111
100502220,2sleep20-21swapper/200:14:392
304582210,2sleep330457-21sshd21:21:023
1015799210,19cyclictest44-21ksoftirqd/321:58:593
1015799210,16cyclictest0-21swapper/323:10:033
10151992016,3cyclictest15990-21sshd23:48:352
1014999200,19cyclictest0-21swapper/019:46:530
5592190,1sleep00-21swapper/023:46:470
176222190,17sleep228750irq/26-ahci[00023:33:512
155802190,17sleep3138550irq/27-eth023:59:493
1015199192,8cyclictest0-21swapper/222:52:122
1015199190,8cyclictest30573-21sshd23:57:452
1014999190,2cyclictest20704-21sshd22:26:500
138862180,16sleep1138550irq/27-eth022:26:011
119722180,1sleep00-21swapper/023:36:540
1015199184,8cyclictest20840-21sshd23:19:122
1015199184,8cyclictest18563-21cp23:00:132
1015199183,9cyclictest19041-21memory22:30:212
1015199182,7cyclictest445-21jbd2/dm-0-821:33:422
1015099187,6cyclictest0-21swapper/121:38:391
1014999187,10cyclictest0-21swapper/022:13:340
59222172,13sleep3138550irq/27-eth022:10:163
138722170,15sleep3138550irq/27-eth000:11:073
110002170,15sleep128750irq/26-ahci[00023:48:021
10157991711,4cyclictest15660-21perl22:00:133
1015799170,15cyclictest8541-21sshd22:47:543
1015199174,12cyclictest147450irq/28-eth1-rx-22:21:142
1015199173,8cyclictest0-21swapper/221:36:252
1015199172,9cyclictest0-21swapper/221:49:412
1015199172,8cyclictest0-21swapper/200:23:502
1015199172,1cyclictest2471-21sshd22:06:062
1015199172,1cyclictest2471-21sshd22:06:062
1015199171,13cyclictest0-21swapper/221:41:332
1014999172,4cyclictest32426-21sshd22:31:530
1014999170,16cyclictest28788-21sshd22:20:170
1014999170,16cyclictest20170-21sshd21:19:580
1014999170,15cyclictest0-21swapper/023:53:590
1014999170,15cyclictest0-21swapper/023:53:590
90732160,14sleep310950irq/23-ehci_hcd00:26:313
49562160,7sleep10-21swapper/122:47:291
226212160,14sleep328750irq/26-ahci[00021:31:213
12322160,14sleep3138550irq/27-eth022:54:333
110902160,14sleep2138550irq/27-eth023:29:232
105882162,12sleep1138550irq/27-eth021:52:111
1015799166,5cyclictest0-21swapper/320:05:473
1015799162,11cyclictest4269-21sshd22:21:103
10157991610,1cyclictest1877-21nfsd23:50:283
10157991610,1cyclictest1877-21nfsd23:50:283
1015799160,2cyclictest25159-21sshd23:23:283
1015799160,14cyclictest22306-21sshd21:53:343
1015799160,13cyclictest0-21swapper/320:14:033
1015199165,1cyclictest17928-21diskstats19:30:132
1015199163,8cyclictest147450irq/28-eth1-rx-22:10:012
1015199162,8cyclictest0-21swapper/220:41:522
1015199162,10cyclictest5846-21sshd23:13:432
1015199161,5cyclictest36-21rcuop/222:43:102
1015199161,11cyclictest11597-21sshd21:52:192
1015099162,12cyclictest6367-21sshd22:06:341
1015099162,12cyclictest6367-21sshd22:06:331
10150991612,3cyclictest25111-21sshd21:24:171
1015099160,15cyclictest0-21swapper/123:41:481
1015099160,14cyclictest1877-21nfsd21:10:481
1014999166,9cyclictest0-21swapper/000:00:280
10149991613,2cyclictest0-21swapper/000:23:380
10149991612,1cyclictest0-21swapper/021:59:170
10149991611,4cyclictest1929-21nscd00:15:230
10149991610,1cyclictest0-21swapper/000:10:220
1014999160,15cyclictest9435-21sshd23:29:100
1014999160,13cyclictest0-21swapper/023:10:450
1014999160,12cyclictest0-21swapper/023:16:180
1014999160,12cyclictest0-21swapper/000:05:490
1014999160,12cyclictest0-21swapper/000:05:480
320092150,13sleep1138550irq/27-eth022:02:051
1015799158,6cyclictest0-21swapper/321:39:453
1015799158,5cyclictest8152-21sshd22:59:023
1015799157,6cyclictest21849-21grep23:45:263
1015799153,5cyclictest138550irq/27-eth022:06:053
1015799153,5cyclictest138550irq/27-eth022:06:043
10157991510,4cyclictest1877-21nfsd22:36:023
1015799150,6cyclictest6343-21sshd23:36:133
1015799150,3cyclictest29448-21rm21:46:573
1015799150,14cyclictest16138-21bash21:26:533
1015799150,14cyclictest0-21swapper/300:16:113
1015799150,10cyclictest0-21swapper/322:30:273
1015199157,3cyclictest5516-21sshd21:14:342
1015199153,7cyclictest1874-21nfsd22:38:172
1015199153,11cyclictest0-21swapper/219:53:472
1015199152,9cyclictest30728-21sshd22:01:552
1015199152,12cyclictest0-21swapper/219:36:112
1015199151,13cyclictest0-21swapper/222:45:262
1015199150,13cyclictest32711-21vmstat20:00:272
1015199150,13cyclictest1015-21sshd00:33:452
1015099158,3cyclictest29591-21sshd00:20:551
1015099158,2cyclictest0-21swapper/100:12:061
1015099157,4cyclictest19698-21sshd21:27:181
1015099157,4cyclictest0-21swapper/119:24:381
1015099155,1cyclictest0-21swapper/100:31:171
1015099153,10cyclictest0-21swapper/123:00:211
1015099152,12cyclictest138550irq/27-eth020:05:271
1015099151,7cyclictest23925-21sshd23:30:441
1015099150,3cyclictest31260-21memory23:50:201
1015099150,3cyclictest31260-21memory23:50:191
1015099150,3cyclictest138550irq/27-eth020:46:381
1015099150,13cyclictest16738-21sshd22:15:131
1015099150,13cyclictest0-21swapper/119:35:121
1015099150,12cyclictest0-21swapper/120:02:351
1014999159,3cyclictest0-21swapper/020:06:000
1014999156,8cyclictest3-21ksoftirqd/019:30:000
10149991512,2cyclictest3-21ksoftirqd/019:40:000
1014999150,4cyclictest0-21swapper/022:19:160
1014999150,1cyclictest0-21swapper/022:58:010
1014999150,14cyclictest0-21swapper/023:34:280
38122140,1sleep08-21rcu_preempt23:05:550
230292140,4sleep21015199cyclictest00:04:322
1015799149,4cyclictest138550irq/27-eth023:00:093
1015799147,3cyclictest138550irq/27-eth020:21:503
10157991411,2cyclictest0-21swapper/320:35:423
1015799141,12cyclictest0-21swapper/320:01:133
10157991411,2cyclictest0-21swapper/319:54:003
1015799140,6cyclictest0-21swapper/321:15:193
1015799140,5cyclictest17916-21sshd00:35:453
1015799140,13cyclictest23757-21sshd00:00:383
1015799140,13cyclictest138550irq/27-eth023:30:213
1015799140,13cyclictest0-21swapper/323:06:213
1015799140,13cyclictest0-21swapper/321:08:413
1015799140,12cyclictest25883-21getsmi00:20:293
1015799140,12cyclictest1877-21nfsd20:55:133
1015799140,12cyclictest17558-21sshd22:15:173
1015799140,12cyclictest138550irq/27-eth019:46:483
1015199148,3cyclictest0-21swapper/219:56:072
1015199146,6cyclictest0-21swapper/223:50:042
1015199146,6cyclictest0-21swapper/223:50:032
1015199143,8cyclictest0-21swapper/200:38:382
1015199143,1cyclictest147450irq/28-eth1-rx-23:41:532
1015199142,3cyclictest0-21swapper/220:33:112
1015199142,11cyclictest6334-21sshd21:18:222
1015199142,11cyclictest10950irq/23-ehci_hcd20:07:462
1015199141,12cyclictest0-21swapper/220:24:262
1015199141,11cyclictest0-21swapper/220:15:132
1015199140,13cyclictest138550irq/27-eth019:15:212
1015199140,13cyclictest0-21swapper/220:29:552
1015199140,12cyclictest1875-21nfsd23:22:002
1015199140,10cyclictest0-21swapper/222:26:142
1015099148,4cyclictest19549-21nfsd422:30:241
1015099147,6cyclictest0-21swapper/120:40:281
1015099143,4cyclictest18282-21sshd21:34:391
1015099142,3cyclictest13887-21sshd22:22:181
1015099142,3cyclictest0-21swapper/123:09:591
1015099141,6cyclictest1921-21nscd21:41:311
1015099140,13cyclictest3913-21sshd23:13:301
1015099140,13cyclictest28265-21sshd00:37:071
1015099140,13cyclictest138550irq/27-eth022:50:501
1015099140,12cyclictest978-21sshd23:35:321
1015099140,12cyclictest27421-21sshd00:01:061
1015099140,11cyclictest74252sleep120:19:231
1014999148,5cyclictest0-21swapper/020:17:490
1014999148,3cyclictest0-21swapper/021:05:500
1014999145,8cyclictest3-21ksoftirqd/021:10:140
1014999143,2cyclictest0-21swapper/000:38:240
1014999140,1cyclictest0-21swapper/021:20:030
1014999140,1cyclictest0-21swapper/021:02:060
1014999140,13cyclictest0-21swapper/019:40:290
1014999140,12cyclictest0-21swapper/020:40:120
1015799139,3cyclictest1877-21nfsd20:30:093
1015799138,4cyclictest0-21swapper/319:30:253
1015799138,3cyclictest19703-21iostat20:45:163
1015799137,5cyclictest20948-21munin-node19:35:233
1015799134,4cyclictest10532-21sshd00:30:433
10157991311,1cyclictest0-21swapper/319:42:403
10157991311,1cyclictest0-21swapper/319:29:583
1015799130,4cyclictest1955-21snmpd19:21:193
1015799130,12cyclictest1877-21nfsd20:27:423
1015799130,12cyclictest0-21swapper/320:41:113
1015799130,11cyclictest590-21sshd21:40:003
1015799130,11cyclictest22306-21munin-node20:50:223
1015799130,11cyclictest12724-21df19:15:113
1015199139,3cyclictest138550irq/27-eth020:37:002
1015199132,4cyclictest0-21swapper/219:21:552
1015199132,3cyclictest10950irq/23-ehci_hcd20:52:042
1015199131,3cyclictest0-21swapper/219:40:252
1015199130,3cyclictest138550irq/27-eth020:59:092
1015199130,12cyclictest0-21swapper/221:06:292
1015199130,12cyclictest0-21swapper/221:00:472
1015199130,12cyclictest0-21swapper/220:10:492
1015199130,12cyclictest0-21swapper/219:25:052
1015199130,11cyclictest19619-21hddtemp_smartct20:45:142
1015199130,11cyclictest10543-21df_inode19:10:162
1015099131,7cyclictest14492-21perf19:20:011
10150991311,1cyclictest0-21swapper/122:37:381
10150991311,1cyclictest0-21swapper/120:25:141
10150991310,2cyclictest0-21swapper/121:04:001
1015099130,4cyclictest17595-21munin-node19:30:201
1015099130,12cyclictest1877-21nfsd20:31:021
1015099130,12cyclictest11558-21sshd22:55:351
1015099130,12cyclictest0-21swapper/121:07:501
1015099130,12cyclictest0-21swapper/120:56:301
1015099130,12cyclictest0-21swapper/120:10:231
1015099130,12cyclictest0-21swapper/119:50:291
1015099130,12cyclictest0-21swapper/119:42:061
1015099130,11cyclictest8844-21munin-node20:20:191
1015099130,11cyclictest10908-21sort19:10:181
1014999137,5cyclictest0-21swapper/020:00:040
1014999134,5cyclictest3-21ksoftirqd/021:46:440
1014999131,11cyclictest0-21swapper/023:44:140
1014999130,8cyclictest0-21swapper/000:25:230
1014999130,12cyclictest0-21swapper/020:50:150
1014999130,12cyclictest0-21swapper/020:25:350
1014999130,10cyclictest13782-21switchtime19:15:230
1015799120,3cyclictest10547-21diskstats19:10:153
1015799120,11cyclictest0-21swapper/319:55:223
1015799120,10cyclictest6121-21hddtemp_smartct20:15:173
1015799120,10cyclictest25702-21chrt21:00:063
1015199120,11cyclictest0-21swapper/219:45:222
1015099120,11cyclictest1877-21nfsd20:51:011
1015099120,11cyclictest1877-21nfsd20:37:451
1015099120,11cyclictest138550irq/27-eth019:55:121
1015099120,11cyclictest0-21swapper/119:45:111
1015099120,11cyclictest0-21swapper/119:25:041
1014999127,1cyclictest0-21swapper/020:10:270
1014999126,5cyclictest0-21swapper/019:55:470
1014999126,5cyclictest0-21swapper/019:21:330
1014999120,12cyclictest0-21swapper/020:58:170
1014999120,11cyclictest0-21swapper/020:45:430
1014999120,11cyclictest0-21swapper/020:35:240
1014999120,11cyclictest0-21swapper/020:30:090
1014999120,11cyclictest0-21swapper/020:21:480
1014999120,11cyclictest0-21swapper/019:50:150
1014999120,11cyclictest0-21swapper/019:30:130
1014999120,11cyclictest0-21swapper/019:14:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional