You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 10:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Thu Apr 25, 2024 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281522195178,12sleep00-21swapper/019:09:430
281712191174,12sleep20-21swapper/219:09:572
259522188171,12sleep30-21swapper/319:05:043
279132173159,10sleep10-21swapper/119:06:391
224552690,0sleep30-21swapper/300:25:003
19582440,0sleep10-21swapper/122:16:031
96382170,0sleep10-21swapper/120:50:141
283509990,8cyclictest17882-21perf23:05:000
14570290,0sleep00-21swapper/021:00:150
283509980,1cyclictest19437-21ssh23:25:100
283509980,0cyclictest0-21swapper/023:15:120
283509972,1cyclictest8387-21ssh23:37:160
283509972,1cyclictest498-21systemd-logind21:15:250
283509972,1cyclictest30088-21grep23:50:150
283509972,1cyclictest1498-21ntpd22:26:410
283509971,1cyclictest3602-21snmpd21:08:010
283509971,1cyclictest3602-21snmpd20:44:480
283509971,1cyclictest3602-21snmpd20:37:360
283509971,1cyclictest3602-21snmpd20:34:260
283509971,1cyclictest3602-21snmpd20:29:330
283509971,1cyclictest3602-21snmpd20:20:480
283509971,1cyclictest3602-21snmpd20:13:300
283509971,1cyclictest3602-21snmpd20:08:240
283509971,1cyclictest3602-21snmpd20:03:060
283509971,1cyclictest3602-21snmpd19:35:240
283509971,1cyclictest3602-21snmpd19:34:450
283509971,1cyclictest3602-21snmpd19:29:090
283509971,1cyclictest3602-21snmpd19:23:570
283509971,1cyclictest3602-21snmpd19:15:240
283509971,1cyclictest24849-21uname20:15:190
283509971,1cyclictest21434-21tune2fs23:45:160
283509971,1cyclictest15210-21cut00:20:160
283509971,1cyclictest14728-21awk19:55:200
283509971,1cyclictest12976-21taskset19:50:550
283509971,1cyclictest0-21swapper/020:45:210
283509970,4cyclictest16683-21munin-run22:45:010
283509970,1cyclictest4104-21df21:20:130
283509970,1cyclictest31714-21perl00:30:120
283509970,1cyclictest26068-21diskstats22:50:140
283509970,1cyclictest21507-21apt21:50:010
283509970,1cyclictest17642-21diskstats22:45:130
283509970,1cyclictest12514-21ntp_states20:55:200
6723260,0sleep10-21swapper/100:35:071
283509961,1cyclictest7399-21ntp_states19:40:150
283509961,1cyclictest3602-21snmpd00:07:560
283509961,1cyclictest31091-21ssh23:31:210
283509961,1cyclictest29728-21sh21:35:190
283509961,1cyclictest2541-21if_eth422:55:160
283509961,1cyclictest16678-21sshd21:27:250
283509960,5cyclictest0-21swapper/022:15:110
283509960,5cyclictest0-21swapper/000:10:110
283509960,2cyclictest7012-21cut22:00:130
283509960,2cyclictest20710-21sort21:30:160
283509960,1cyclictest9870-21ntp_states19:45:210
283509960,1cyclictest9277-21df20:50:120
283509960,1cyclictest8944-21users00:35:270
283509960,1cyclictest7963-21diskstats22:20:110
283509960,1cyclictest7963-21diskstats22:20:100
283509960,1cyclictest7478-21ntp_states00:15:240
283509960,1cyclictest5057-21ssh21:40:130
283509960,1cyclictest27885-21irqstats23:10:150
283509960,1cyclictest23954-21perl22:10:160
283509960,1cyclictest23942-21memory00:25:200
283509960,1cyclictest19665-21memory23:05:160
283509960,1cyclictest18915-21ssh21:10:140
283509960,1cyclictest1765-21ntp_states22:35:200
23088260,0sleep10-21swapper/100:25:141
1537260,0sleep10-21swapper/123:14:241
283619950,5cyclictest0-21swapper/323:54:533
283619950,5cyclictest0-21swapper/323:35:143
283619950,5cyclictest0-21swapper/322:31:053
283619950,4cyclictest0-21swapper/319:45:113
283619950,3cyclictest7796-21ssh21:40:563
283579953,2cyclictest24695-21cpuspeed_turbos22:30:142
283579951,2cyclictest0-21swapper/222:00:102
283579951,2cyclictest0-21swapper/220:10:122
283579950,5cyclictest0-21swapper/222:10:112
283579950,4cyclictest7189-21sed22:20:012
283559951,2cyclictest0-21swapper/122:10:101
283559951,2cyclictest0-21swapper/121:40:131
283559951,2cyclictest0-21swapper/121:25:121
283559950,5cyclictest0-21swapper/123:36:311
283559950,5cyclictest0-21swapper/122:42:581
283559950,5cyclictest0-21swapper/100:04:341
283509955,0cyclictest0-21swapper/023:40:030
283509950,1cyclictest7245-21users23:55:220
283509950,1cyclictest31249-21hddtemp_smartct21:55:160
283509950,1cyclictest28977-21df19:10:110
283509950,1cyclictest24858-21df22:30:140
283509950,1cyclictest22820-21hddtemp_smartct21:50:140
283509950,1cyclictest15314-21diskstats22:05:140
283509950,1cyclictest14250-21df_inode00:00:150
283509950,1cyclictest11616-21diskstats23:20:120
19591250,4sleep319605-21ssh22:26:433
15452250,0sleep10-21swapper/119:55:261
283619944,0cyclictest0-21swapper/322:49:593
283619944,0cyclictest0-21swapper/322:36:283
283619943,1cyclictest30550-21sed20:25:223
283619943,1cyclictest19682-21tr23:25:113
283619942,1cyclictest6291-21/usr/sbin/munin20:45:173
283619942,1cyclictest27312-21df23:10:113
283619942,1cyclictest26292-21ssh21:52:113
283619942,1cyclictest23677-21ssh00:25:183
283619942,1cyclictest12460-21aten2_r5power_v23:40:113
283619941,3cyclictest28585-21latency_hist23:49:593
283619941,2cyclictest0-21swapper/322:00:103
283619941,2cyclictest0-21swapper/320:10:123
283619941,1cyclictest4870-21latency20:40:163
283619941,1cyclictest0-21swapper/320:30:123
283619940,4cyclictest11069-21/usr/sbin/munin19:50:303
283619940,4cyclictest0-21swapper/323:07:363
283619940,4cyclictest0-21swapper/321:35:143
283619940,4cyclictest0-21swapper/321:10:143
283619940,4cyclictest0-21swapper/320:55:133
283619940,4cyclictest0-21swapper/320:15:303
283619940,4cyclictest0-21swapper/319:30:073
283619940,4cyclictest0-21swapper/300:31:483
283619940,3cyclictest3524-21perl23:15:143
283619940,3cyclictest3164-21apt-config21:20:013
283619940,3cyclictest30918-21date19:15:013
283619940,3cyclictest19246-21cat23:25:003
283619940,3cyclictest17181-21ssh23:04:293
283619940,3cyclictest13367-21dpkg00:00:003
283619940,1cyclictest8711-21/usr/sbin/munin22:40:193
283619940,1cyclictest1998-21df22:55:123
283619940,1cyclictest16199-21/usr/sbin/munin21:05:213
283619940,1cyclictest13642-21/usr/sbin/munin21:45:213
283619940,1cyclictest12489-21cat21:25:133
283619940,1cyclictest0-21swapper/300:15:133
283619940,0cyclictest0-21swapper/322:08:053
283579943,1cyclictest26080-21cron20:19:592
283579942,2cyclictest32234-21cat20:30:422
283579942,2cyclictest21277-21nscd21:53:572
283579941,3cyclictest3602-21snmpd00:11:202
283579941,3cyclictest17404-21ssh23:42:462
283579941,1cyclictest0-21swapper/219:20:072
283579940,4cyclictest8952-21ssh00:16:092
283579940,4cyclictest2493-21ssh22:35:282
283579940,4cyclictest21757-21ssh21:11:022
283579940,4cyclictest1858-21df_inode20:35:122
283579940,4cyclictest0-21swapper/223:59:212
283579940,4cyclictest0-21swapper/223:52:062
283579940,4cyclictest0-21swapper/223:45:082
283579940,4cyclictest0-21swapper/223:26:502
283579940,4cyclictest0-21swapper/223:22:112
283579940,4cyclictest0-21swapper/223:17:122
283579940,4cyclictest0-21swapper/223:11:172
283579940,4cyclictest0-21swapper/223:08:462
283579940,4cyclictest0-21swapper/223:00:112
283579940,4cyclictest0-21swapper/222:55:172
283579940,4cyclictest0-21swapper/222:20:312
283579940,4cyclictest0-21swapper/222:20:312
283579940,4cyclictest0-21swapper/222:05:042
283579940,4cyclictest0-21swapper/221:30:362
283579940,4cyclictest0-21swapper/221:20:312
283579940,4cyclictest0-21swapper/221:15:142
283579940,4cyclictest0-21swapper/220:43:252
283579940,4cyclictest0-21swapper/200:36:212
283579940,4cyclictest0-21swapper/200:33:252
283579940,4cyclictest0-21swapper/200:25:532
283579940,4cyclictest0-21swapper/200:08:142
283579940,3cyclictest8711-21/usr/sbin/munin22:40:132
283579940,3cyclictest8365-21latency_hist19:45:002
283579940,3cyclictest5423-21cut19:35:012
283579940,3cyclictest4283-21ssh21:39:572
283579940,3cyclictest3774-21cat23:35:002
283579940,3cyclictest31139-21sensors19:15:012
283579940,3cyclictest30322-21ssh21:55:112
283579940,3cyclictest29385-21ssh22:51:492
283579940,3cyclictest27688-21ssh23:30:062
283579940,3cyclictest18445-21diskmemload21:40:012
283579940,3cyclictest18276-21ssh22:45:172
283579940,3cyclictest16781-21hddtemp_smartct22:25:162
283579940,3cyclictest16218-21date20:00:002
283579940,3cyclictest13462-21apt-get21:45:022
283579940,3cyclictest11991-21/usr/sbin/munin21:25:122
283559944,0cyclictest0-21swapper/123:48:441
283559944,0cyclictest0-21swapper/121:19:441
283559944,0cyclictest0-21swapper/100:19:351
283559942,1cyclictest0-21swapper/121:20:101
283559942,1cyclictest0-21swapper/120:40:091
283559942,1cyclictest0-21swapper/120:35:101
283559942,1cyclictest0-21swapper/119:45:111
283559941,2cyclictest0-21swapper/123:55:091
283559941,2cyclictest0-21swapper/123:50:111
283559941,2cyclictest0-21swapper/123:20:101
283559941,2cyclictest0-21swapper/123:05:091
283559941,2cyclictest0-21swapper/123:00:111
283559941,2cyclictest0-21swapper/121:45:131
283559941,2cyclictest0-21swapper/120:25:101
283559941,2cyclictest0-21swapper/100:10:121
283559941,1cyclictest0-21swapper/123:30:101
283559941,1cyclictest0-21swapper/122:20:091
283559941,1cyclictest0-21swapper/122:20:091
283559941,1cyclictest0-21swapper/119:30:071
283559940,4cyclictest0-21swapper/122:30:271
283559940,4cyclictest0-21swapper/121:05:381
283559940,4cyclictest0-21swapper/120:11:291
283559940,2cyclictest0-21swapper/121:30:121
283559940,2cyclictest0-21swapper/119:40:071
283559940,0cyclictest0-21swapper/123:29:581
283619931,1cyclictest31644-21munin-node19:15:023
283619931,1cyclictest18708-21apt-get20:05:113
283619931,1cyclictest0-21swapper/323:55:093
283619931,1cyclictest0-21swapper/323:30:103
283619931,1cyclictest0-21swapper/319:20:073
283619930,3cyclictest27013-21perl20:20:173
283619930,3cyclictest25741-21grep22:50:133
283619930,3cyclictest14215-21cat21:00:133
283619930,3cyclictest0-21swapper/322:20:093
283619930,3cyclictest0-21swapper/322:20:093
283619930,3cyclictest0-21swapper/322:15:113
283619930,3cyclictest0-21swapper/322:10:023
283619930,3cyclictest0-21swapper/321:55:123
283619930,3cyclictest0-21swapper/321:30:023
283619930,3cyclictest0-21swapper/321:15:203
283619930,3cyclictest0-21swapper/320:50:153
283619930,3cyclictest0-21swapper/320:35:103
283619930,3cyclictest0-21swapper/320:01:303
283619930,3cyclictest0-21swapper/319:55:013
283619930,3cyclictest0-21swapper/319:40:073
283619930,3cyclictest0-21swapper/319:38:153
283619930,3cyclictest0-21swapper/319:28:563
283619930,3cyclictest0-21swapper/300:35:063
283619930,3cyclictest0-21swapper/300:05:103
283619930,1cyclictest29919-21ssh00:10:023
283579932,1cyclictest3602-21snmpd19:10:032
283579930,3cyclictest13835-21apt-get00:20:002
283579930,3cyclictest0-21swapper/221:05:062
283579930,3cyclictest0-21swapper/221:00:112
283579930,3cyclictest0-21swapper/220:55:052
283579930,3cyclictest0-21swapper/220:50:102
283579930,3cyclictest0-21swapper/220:44:592
283579930,3cyclictest0-21swapper/220:25:102
283579930,3cyclictest0-21swapper/220:20:112
283579930,3cyclictest0-21swapper/220:05:122
283579930,3cyclictest0-21swapper/219:55:012
283579930,3cyclictest0-21swapper/219:50:132
283579930,3cyclictest0-21swapper/219:45:052
283579930,3cyclictest0-21swapper/219:35:052
283579930,3cyclictest0-21swapper/219:25:052
283579930,3cyclictest0-21swapper/200:00:012
283559933,0cyclictest0-21swapper/122:25:481
283559933,0cyclictest0-21swapper/122:03:151
283559933,0cyclictest0-21swapper/121:37:031
283559933,0cyclictest0-21swapper/120:58:111
283559933,0cyclictest0-21swapper/119:14:181
283559931,1cyclictest0-21swapper/123:40:101
283559931,1cyclictest0-21swapper/122:55:101
283559931,1cyclictest0-21swapper/122:45:111
283559931,1cyclictest0-21swapper/122:35:091
283559931,1cyclictest0-21swapper/122:05:111
283559931,1cyclictest0-21swapper/120:30:121
283559931,1cyclictest0-21swapper/120:15:121
283559931,1cyclictest0-21swapper/120:05:121
283559931,1cyclictest0-21swapper/119:50:141
283559931,1cyclictest0-21swapper/100:30:091
283559930,2cyclictest27100-21chrt21:52:491
283559930,2cyclictest0-21swapper/122:50:111
283559930,2cyclictest0-21swapper/121:55:121
283559930,2cyclictest0-21swapper/121:10:141
283559930,0cyclictest0-21swapper/120:49:101
283559921,1cyclictest0-21swapper/123:15:121
283559921,1cyclictest0-21swapper/120:20:121
283559921,1cyclictest0-21swapper/100:20:011
283559920,2cyclictest16233-21chrt20:00:111
283559920,1cyclictest5602-21taskset19:36:411
283559920,1cyclictest0-21swapper/121:00:111
283559920,1cyclictest0-21swapper/119:20:071
283559920,1cyclictest0-21swapper/100:05:101
283559910,1cyclictest0-21swapper/119:25:011
283559910,1cyclictest0-21swapper/119:15:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional