You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-08 - 17:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot2.osadl.org (updated Sat Feb 08, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
189282202185,12sleep10-21swapper/107:05:351
192592191174,12sleep20-21swapper/207:09:532
168182183166,12sleep00-21swapper/007:05:020
192552182166,12sleep30-21swapper/307:09:503
58102490,0sleep10-21swapper/110:46:231
118432470,0sleep10-21swapper/110:32:321
116642450,0sleep00-21swapper/011:57:270
271702440,0sleep10-21swapper/110:23:241
61322430,0sleep10-21swapper/109:39:161
319892420,0sleep30-21swapper/312:25:403
261312410,0sleep10-21swapper/109:50:011
83832340,0sleep10-21swapper/111:04:271
13572150,0sleep30-21swapper/311:17:463
1946099103,3cyclictest19857-21munin-run08:15:010
1946099102,4cyclictest19538-21munin-run09:30:020
194609991,4cyclictest28113-21munin-run08:35:010
10091290,0sleep10-21swapper/111:05:241
24237280,0sleep30-21swapper/311:13:023
194609971,3cyclictest29764-21munin-run10:25:020
194609970,6cyclictest15143-21/usr/sbin/munin08:05:160
194699960,5cyclictest0-21swapper/210:10:092
194649961,3cyclictest0-21swapper/112:35:121
194609962,4cyclictest13229-21munin-run09:10:020
194609960,3cyclictest9653-21munin-run10:15:010
194609960,3cyclictest30744-21munin-run12:25:000
30332250,0sleep00-21swapper/011:50:150
194779955,0cyclictest11918-21kworker/3:208:37:303
194779955,0cyclictest11918-21kworker/3:208:26:173
194779955,0cyclictest11918-21kworker/3:208:15:053
194779955,0cyclictest11918-21kworker/3:208:03:543
194779955,0cyclictest11918-21kworker/3:207:52:423
194779955,0cyclictest11918-21kworker/3:207:41:313
194779955,0cyclictest11918-21kworker/3:207:30:183
194779955,0cyclictest0-21swapper/309:34:463
194779954,1cyclictest0-21swapper/310:44:533
194779951,3cyclictest30997-21snmpd10:27:333
194779951,3cyclictest20615-21ssh10:36:443
194779950,5cyclictest12316-21ssh12:32:243
194779950,5cyclictest0-21swapper/310:15:133
194699955,0cyclictest0-21swapper/212:18:142
194699950,5cyclictest5996-21sed09:55:202
194699950,5cyclictest0-21swapper/210:41:112
194699950,5cyclictest0-21swapper/207:50:222
194699950,0cyclictest0-21swapper/208:00:232
194649951,3cyclictest0-21swapper/108:50:131
194649951,3cyclictest0-21swapper/108:10:141
194649950,5cyclictest0-21swapper/111:57:311
194649950,5cyclictest0-21swapper/111:37:271
194649950,5cyclictest0-21swapper/110:03:531
194649950,5cyclictest0-21swapper/109:18:381
194609955,0cyclictest0-21swapper/010:42:330
194609954,1cyclictest17413-21munin-run11:10:000
194609951,4cyclictest20280-21munin-run11:45:000
194609950,5cyclictest0-21swapper/011:36:310
194609950,4cyclictest20270-21apt-get08:15:000
194609950,3cyclictest0-21swapper/009:19:230
194609950,0cyclictest0-21swapper/010:53:460
194779944,0cyclictest11918-21kworker/3:210:07:033
194779944,0cyclictest0-21swapper/308:55:183
194779942,2cyclictest25244-21ssh10:55:593
194779942,2cyclictest19736-21ssh10:53:493
194779940,4cyclictest20797-21ssh11:28:373
194779940,4cyclictest13223-21diskmemload12:15:133
194779940,4cyclictest0-21swapper/312:36:593
194779940,4cyclictest0-21swapper/312:06:083
194779940,4cyclictest0-21swapper/312:00:113
194779940,4cyclictest0-21swapper/311:57:263
194779940,4cyclictest0-21swapper/311:54:543
194779940,4cyclictest0-21swapper/311:46:233
194779940,4cyclictest0-21swapper/311:40:123
194779940,4cyclictest0-21swapper/311:00:123
194779940,4cyclictest0-21swapper/310:47:493
194779940,4cyclictest0-21swapper/310:30:073
194779940,4cyclictest0-21swapper/310:12:503
194779940,4cyclictest0-21swapper/310:00:513
194779940,4cyclictest0-21swapper/309:50:053
194779940,4cyclictest0-21swapper/309:42:183
194779940,4cyclictest0-21swapper/309:26:463
194779940,4cyclictest0-21swapper/309:20:113
194779940,4cyclictest0-21swapper/308:45:103
194779940,4cyclictest0-21swapper/308:20:123
194779940,4cyclictest0-21swapper/307:25:123
194779940,4cyclictest0-21swapper/307:15:143
194779940,3cyclictest983-21ssh11:35:123
194779940,3cyclictest31267-21irqstats09:35:173
194779940,3cyclictest28039-21ssh12:23:163
194779940,3cyclictest27525-21ssh10:23:373
194779940,3cyclictest26120-21cut09:50:013
194779940,3cyclictest249542sleep309:15:163
194779940,3cyclictest24458-21/usr/sbin/munin11:30:113
194779940,3cyclictest15411-21ssh11:08:453
194779940,3cyclictest1500-21ssh12:10:063
194779940,3cyclictest13456-21rm09:10:013
194779940,0cyclictest0-21swapper/311:21:083
194699944,0cyclictest0-21swapper/212:00:372
194699944,0cyclictest0-21swapper/211:21:202
194699944,0cyclictest0-21swapper/210:46:432
194699944,0cyclictest0-21swapper/209:51:082
194699944,0cyclictest0-21swapper/209:33:312
194699943,1cyclictest17418-21basename11:10:012
194699942,2cyclictest28455-21ssh09:16:542
194699940,4cyclictest25292-21apt-get07:20:132
194699940,4cyclictest13223-21diskmemload12:24:162
194699940,4cyclictest0-21swapper/212:36:342
194699940,4cyclictest0-21swapper/212:33:112
194699940,4cyclictest0-21swapper/212:26:152
194699940,4cyclictest0-21swapper/212:10:232
194699940,4cyclictest0-21swapper/211:55:382
194699940,4cyclictest0-21swapper/211:30:092
194699940,4cyclictest0-21swapper/211:00:132
194699940,4cyclictest0-21swapper/210:50:152
194699940,4cyclictest0-21swapper/210:35:122
194699940,4cyclictest0-21swapper/210:30:562
194699940,4cyclictest0-21swapper/210:28:202
194699940,4cyclictest0-21swapper/210:20:092
194699940,4cyclictest0-21swapper/210:05:032
194699940,4cyclictest0-21swapper/209:45:022
194699940,4cyclictest0-21swapper/209:35:122
194699940,4cyclictest0-21swapper/209:20:112
194699940,4cyclictest0-21swapper/209:10:182
194699940,4cyclictest0-21swapper/209:09:222
194699940,4cyclictest0-21swapper/208:15:132
194699940,4cyclictest0-21swapper/208:13:292
194699940,3cyclictest763-21ssh11:51:482
194699940,3cyclictest3046-21grep11:35:212
194699940,3cyclictest23258-21irqstats10:55:162
194699940,3cyclictest17799-21ssh11:10:072
194699940,3cyclictest11920-21sh09:25:182
194699940,3cyclictest10894-21cat09:05:012
194699940,3cyclictest0-21swapper/211:15:032
194699940,0cyclictest0-21swapper/210:01:542
194649941,2cyclictest0-21swapper/112:30:021
194649941,2cyclictest0-21swapper/111:15:111
194649941,2cyclictest0-21swapper/110:40:101
194649941,2cyclictest0-21swapper/109:30:071
194649941,2cyclictest0-21swapper/109:00:091
194649941,2cyclictest0-21swapper/108:30:091
194649941,2cyclictest0-21swapper/108:15:121
194649941,2cyclictest0-21swapper/108:00:131
194649941,2cyclictest0-21swapper/107:35:141
194649941,2cyclictest0-21swapper/107:10:121
194649941,1cyclictest0-21swapper/107:45:121
194649940,4cyclictest0-21swapper/110:59:271
194649940,4cyclictest0-21swapper/109:22:261
194609944,0cyclictest0-21swapper/011:26:030
194609944,0cyclictest0-21swapper/011:02:370
194609944,0cyclictest0-21swapper/010:17:050
194609943,1cyclictest18320-21ntp_states12:35:190
194609942,1cyclictest12925-21ntpq12:15:170
194609941,1cyclictest0-21swapper/007:20:120
194609940,4cyclictest0-21swapper/012:30:290
194609940,4cyclictest0-21swapper/012:00:110
194609940,4cyclictest0-21swapper/011:10:070
194609940,4cyclictest0-21swapper/010:30:070
194609940,4cyclictest0-21swapper/009:35:140
194609940,4cyclictest0-21swapper/009:00:190
194609940,3cyclictest615-21turbostat.cron08:45:000
194609940,3cyclictest28453-21sort11:50:010
194609940,3cyclictest27843-21apt-get11:15:120
194609940,3cyclictest24629-21users08:20:210
194609940,3cyclictest22458-21memory07:15:010
194609940,3cyclictest20818-21hddtemp_smartct09:30:110
194609940,3cyclictest1133-21sh09:53:300
194609940,2cyclictest0-21swapper/011:24:390
194609940,1cyclictest865-21/usr/sbin/munin08:45:130
194609940,1cyclictest7910-21/usr/sbin/munin09:40:160
194609940,1cyclictest4085-21/usr/sbin/munin09:55:200
194609940,1cyclictest28338-21/usr/sbin/munin08:35:140
194609940,1cyclictest0-21swapper/010:35:140
194609940,0cyclictest0-21swapper/009:13:300
12018240,0sleep10-21swapper/112:15:131
19668230,0sleep10-21swapper/111:27:541
194779932,1cyclictest30997-21snmpd08:05:013
194779932,1cyclictest0-21swapper/309:05:143
194779930,3cyclictest3944-21fschecks_count08:50:143
194779930,3cyclictest0-21swapper/309:00:093
194779930,3cyclictest0-21swapper/308:40:143
194779930,3cyclictest0-21swapper/308:30:063
194779930,3cyclictest0-21swapper/308:10:013
194779930,3cyclictest0-21swapper/307:55:123
194779930,3cyclictest0-21swapper/307:45:123
194779930,3cyclictest0-21swapper/307:35:143
194779930,3cyclictest0-21swapper/307:20:123
194779930,3cyclictest0-21swapper/307:10:133
194779930,2cyclictest3598-21latency_hist09:55:023
194699930,3cyclictest30035-21sort08:35:192
194699930,3cyclictest28281-21df07:25:142
194699930,3cyclictest15446-21ssh11:25:142
194699930,3cyclictest10693-21df07:55:142
194699930,3cyclictest10185-21apt-get10:15:122
194699930,3cyclictest0-21swapper/212:05:062
194699930,3cyclictest0-21swapper/211:45:122
194699930,3cyclictest0-21swapper/211:40:032
194699930,3cyclictest0-21swapper/209:40:142
194699930,3cyclictest0-21swapper/208:55:072
194699930,3cyclictest0-21swapper/208:50:122
194699930,3cyclictest0-21swapper/208:45:022
194699930,3cyclictest0-21swapper/208:40:132
194699930,3cyclictest0-21swapper/208:30:082
194699930,3cyclictest0-21swapper/208:25:042
194699930,3cyclictest0-21swapper/208:20:122
194699930,3cyclictest0-21swapper/208:05:102
194699930,3cyclictest0-21swapper/207:45:112
194699930,3cyclictest0-21swapper/207:40:102
194699930,3cyclictest0-21swapper/207:35:142
194699930,3cyclictest0-21swapper/207:30:012
194699930,3cyclictest0-21swapper/207:15:132
194699930,3cyclictest0-21swapper/207:10:062
194649933,0cyclictest0-21swapper/112:25:371
194649933,0cyclictest0-21swapper/112:20:471
194649933,0cyclictest0-21swapper/111:14:011
194649933,0cyclictest0-21swapper/110:09:161
194649933,0cyclictest0-21swapper/109:43:481
194649933,0cyclictest0-21swapper/109:29:161
194649931,2cyclictest0-21swapper/110:50:081
194649931,1cyclictest0-21swapper/112:10:071
194649931,1cyclictest0-21swapper/111:20:141
194649931,1cyclictest0-21swapper/110:25:021
194649931,1cyclictest0-21swapper/109:50:081
194649931,1cyclictest0-21swapper/109:10:181
194649931,1cyclictest0-21swapper/109:05:131
194649931,1cyclictest0-21swapper/108:55:121
194649931,1cyclictest0-21swapper/108:45:101
194649931,1cyclictest0-21swapper/108:05:121
194649931,1cyclictest0-21swapper/107:55:131
194649931,1cyclictest0-21swapper/107:40:111
194649931,1cyclictest0-21swapper/107:30:151
194649931,1cyclictest0-21swapper/107:20:121
194649931,1cyclictest0-21swapper/107:15:131
194649930,3cyclictest0-21swapper/108:27:071
194649930,2cyclictest21295-21taskset11:45:311
194649930,2cyclictest187692sleep111:44:021
194649930,2cyclictest0-21swapper/112:00:121
194649930,2cyclictest0-21swapper/110:15:141
194649930,2cyclictest0-21swapper/107:50:121
194649930,1cyclictest0-21swapper/108:20:121
194609933,0cyclictest3-21ksoftirqd/010:25:020
194609932,1cyclictest31560-21ssh12:25:240
194609932,1cyclictest12672-21/usr/sbin/munin08:00:140
194609931,2cyclictest0-21swapper/007:50:120
194609931,1cyclictest0-21swapper/007:45:120
194609931,1cyclictest0-21swapper/007:40:120
194609930,3cyclictest830-21cut07:35:170
194609930,3cyclictest7083-21ntp_states08:55:170
194609930,3cyclictest3977-21rm10:45:120
194609930,3cyclictest2681-21if_eth109:20:150
194609930,3cyclictest24059-21ssh10:05:030
194609930,3cyclictest0-21swapper/011:30:060
194609930,3cyclictest0-21swapper/010:55:130
194609930,3cyclictest0-21swapper/009:45:010
194609930,3cyclictest0-21swapper/008:50:120
194609930,3cyclictest0-21swapper/008:25:150
194609930,3cyclictest0-21swapper/007:55:120
194609930,3cyclictest0-21swapper/007:25:110
194609930,3cyclictest0-21swapper/007:10:130
194609930,1cyclictest13974-21ssh10:00:040
194609930,1cyclictest0-21swapper/012:05:020
194609930,0cyclictest30233-21apt-get07:30:140
194609930,0cyclictest0-21swapper/012:10:100
194649921,1cyclictest277492chrt07:25:041
194649921,1cyclictest0-21swapper/109:55:111
194649920,1cyclictest30820-21chrt08:40:061
194649920,1cyclictest27442-21chrt12:06:051
194649920,1cyclictest0-21swapper/111:50:071
194649920,1cyclictest0-21swapper/111:30:071
194649920,1cyclictest0-21swapper/110:35:061
194649920,1cyclictest0-21swapper/110:10:011
194649920,1cyclictest0-21swapper/108:35:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional