You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-13 - 03:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Wed Jun 12, 2024 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14857809911274,27cyclictest1589357-21latency_hist12:30:010
14857829910393,8cyclictest274-21systemd-journal08:06:491
1485782999786,8cyclictest1534469-21latency_hist09:40:001
1485780999484,8cyclictest1518303-21latency_hist08:50:000
1485782999283,7cyclictest1542955-21latency_hist10:05:001
1485780999180,8cyclictest1549321-21latency_hist10:25:010
1485782998575,7cyclictest1566830-21latency_hist11:20:001
1485780998474,7cyclictest1495911-21latency_hist07:40:010
1485780998273,7cyclictest1570016-21apt-get11:30:030
1485782997870,6cyclictest1489235-21kworker/u8:0+flush-179:007:30:001
1485780997870,6cyclictest1536199-21gpgv09:45:000
1485782997667,7cyclictest1571850-21latency_hist11:35:011
1485780997467,5cyclictest1510350-21latency_hist08:25:000
1485782997369,3cyclictest1584612-21kworker/u8:3+events_unbound12:15:001
1485780997164,5cyclictest1507158-21kworker/u8:2+events_unbound08:18:480
1485780997164,5cyclictest1507158-21kworker/u8:2+events_unbound08:18:480
1485782996862,4cyclictest1582958-21kworker/u8:1+events_unbound12:25:301
1485782996856,10cyclictest1580075-21apt-get12:00:131
1485780996856,10cyclictest1533137-21apt-get09:35:110
1485782996755,9cyclictest1518559-21apt-get08:50:111
1485782996654,9cyclictest274-21systemd-journal08:40:001
1485782996557,6cyclictest1575074-21kworker/u8:2+flush-179:011:55:151
1485780996554,8cyclictest1544543-21latency_hist10:10:010
1485782996252,7cyclictest1497519-21latency_hist07:45:001
1485782996155,4cyclictest1494018-21kworker/u8:2+events_unbound07:35:341
1485782996149,9cyclictest1584834-21apt-get12:15:101
1485780996151,7cyclictest1590944-21latency_hist12:35:010
1485780996151,7cyclictest1590944-21latency_hist12:35:000
1485782996051,7cyclictest1573452-21latency_hist11:40:011
1485780996052,5cyclictest1584612-21kworker/u8:3+flush-179:012:15:130
1485780996050,8cyclictest1510589-21/usr/sbin/munin08:25:320
1485780996050,7cyclictest274-21systemd-journal12:15:000
1485782995956,2cyclictest0-21swapper/108:22:591
1485782995853,4cyclictest1554137-21kworker/u8:1+flush-179:010:45:151
1485782995846,10cyclictest1559113-21apt-get10:55:131
1485780995854,3cyclictest1536055-21kworker/u8:1+flush-179:010:10:310
1485782995749,5cyclictest1499105-21latency_hist07:50:011
1485782995647,7cyclictest274-21systemd-journal09:57:331
1485782995645,8cyclictest274-21systemd-journal08:20:011
1485782995645,8cyclictest274-21systemd-journal08:20:001
1485780995650,4cyclictest1558842-21kworker/u8:2+flush-179:011:05:280
1485780995645,8cyclictest1575074-21kworker/u8:2+flush-179:011:50:200
1485782995545,8cyclictest1563893-21apt-get11:10:111
1485780995547,6cyclictest1515169-21kworker/u8:2+flush-179:009:00:000
1485782995448,4cyclictest1520384-21kworker/u8:1+events_unbound08:55:151
1485782995444,7cyclictest1526512-21latency_hist09:15:001
1485780995448,4cyclictest1560721-21apt-get11:00:130
1485780995444,8cyclictest1512193-21apt-get08:30:150
1485782995245,6cyclictest1554137-21kworker/u8:1+flush-179:010:45:001
1485780995245,4cyclictest1558893-21kworker/u8:3+flush-179:011:25:280
1485780995244,5cyclictest1515169-21kworker/u8:2+flush-179:008:40:180
1485786995147,2cyclictest0-21swapper/209:25:592
1485782995147,3cyclictest1536055-21kworker/u8:1+flush-179:010:10:171
1485782995138,10cyclictest1520384-21kworker/u8:1+flush-179:009:05:011
4521791500,13ptp4l441ktimers/307:10:013
1485782995045,4cyclictest1497509-21kworker/u8:0+events_unbound07:51:291
1485782995044,5cyclictest1526749-21kworker/u8:1+flush-179:009:20:001
1485782995041,6cyclictest1587771-21latency_hist12:25:011
1485782995040,8cyclictest1573712-21apt-get11:40:121
1485782995038,9cyclictest1502591-21apt-get08:00:121
1485780995044,5cyclictest1489308-21kworker/u8:1+flush-179:007:45:380
1485780995043,5cyclictest1554137-21kworker/u8:1+flush-179:010:55:010
1485780995038,9cyclictest1549579-21apt-get10:25:110
1485782994944,4cyclictest1503969-21kworker/u8:3+flush-179:008:30:281
1485782994841,5cyclictest1532928-21kworker/u8:3+flush-179:009:45:311
1485782994840,5cyclictest1523388-21kworker/u8:3+events_unbound09:05:291
1485782994839,7cyclictest1528378-21apt-get09:20:101
1485780994839,6cyclictest1586425-21/usr/sbin/munin12:20:160
1485788994745,1cyclictest0-21swapper/309:26:003
1485780994739,6cyclictest1503969-21kworker/u8:3+flush-179:008:06:480
1485780994738,7cyclictest1531280-21kworker/u8:0+flush-179:009:30:010
1485782994641,4cyclictest1510395-21kworker/u8:1+events_unbound08:40:351
1485782994638,7cyclictest1558842-21kworker/u8:2+flush-179:011:20:151
1485782994637,6cyclictest1554088-21latency_hist10:40:011
1485780994640,4cyclictest1536055-21kworker/u8:1+events_unbound09:50:010
1485780994541,3cyclictest1549559-21kworker/u8:2+flush-179:010:30:260
1485780994536,6cyclictest1489270-21latency_hist07:20:010
1485782994438,4cyclictest1575074-21kworker/u8:2+events_unbound11:50:121
1485782994434,6cyclictest0-21swapper/107:19:591
1485780994441,2cyclictest1523388-21kworker/u8:3+events_unbound09:05:170
1485782994338,4cyclictest1531280-21kworker/u8:0+flush-179:009:50:311
1485782994334,7cyclictest1543194-21/usr/sbin/munin10:05:201
1485780994340,2cyclictest1502672-21kworker/u8:1+events_unbound08:10:310
1485780994338,4cyclictest1571835-21kworker/u8:0+flush-179:011:45:260
1485780994336,5cyclictest1500985-21apt-get07:55:130
148578699420,40cyclictest0-21swapper/207:55:592
1485782994237,4cyclictest1497509-21kworker/u8:0+flush-179:007:55:371
1485782994236,5cyclictest1558893-21kworker/u8:3+events_unbound11:45:311
1485782994133,6cyclictest1563638-21latency_hist11:10:011
148578299411,4cyclictest271rcuc/108:14:591
1485780994135,4cyclictest1515169-21kworker/u8:2+events_unbound08:50:160
1485780994132,6cyclictest1569982-21kworker/u8:1+flush-179:011:35:330
1485780994033,6cyclictest1481612-21kworker/u8:3+events_unbound07:10:310
1485780994031,7cyclictest1537936-21apt-get09:50:010
1485786993931,7cyclictest0-21swapper/209:03:592
148578699390,36cyclictest0-21swapper/208:07:002
1485782993933,4cyclictest1549559-21kworker/u8:2+flush-179:010:25:161
1485782993932,6cyclictest1549559-21kworker/u8:2+events_unbound10:30:181
1485782993928,6cyclictest0-21swapper/110:54:591
1485782993920,18cyclictest1487611-21grep07:15:011
1485780993934,4cyclictest1582958-21kworker/u8:1+flush-179:012:35:360
148578899380,37cyclictest0-21swapper/308:06:593
1485782993834,3cyclictest1516954-21kworker/u8:0+flush-179:008:49:561
1485782993833,4cyclictest1575074-21kworker/u8:2+flush-179:012:30:371
1485782993833,4cyclictest1575074-21kworker/u8:2+flush-179:012:30:361
1485782993833,4cyclictest1492468-21apt-get07:30:041
1485782993829,7cyclictest1534745-21apt-get09:40:141
148578299381,35cyclictest1511403-21ntp_states08:25:311
1485780993834,3cyclictest1558893-21kworker/u8:3+events_unbound11:15:300
1485780993833,3cyclictest658-21Xorg10:16:590
1485780993832,4cyclictest1532918-21systemd-run09:35:000
1485788993735,1cyclictest0-21swapper/307:21:593
1485786993734,2cyclictest0-21swapper/212:29:582
1485786993734,2cyclictest0-21swapper/208:42:002
1485782993734,2cyclictest0-21swapper/109:34:591
1485782993732,4cyclictest1546160-21kworker/u8:3+events_unbound10:15:351
148578299371,32cyclictest0-21swapper/107:20:161
1485780993733,3cyclictest658-21Xorg11:13:590
1485780993733,3cyclictest1557309-21systemd-run10:50:010
1485780993732,3cyclictest613-21gdbus07:51:240
148578099370,34cyclictest1725-21sendmail-mta09:15:140
1485788993634,1cyclictest0-21swapper/307:47:593
148578699360,35cyclictest0-21swapper/209:14:582
148578299361,33cyclictest1570370-21latency_hist11:30:011
1485780993634,2cyclictest211irq_work/009:20:590
1485780993631,4cyclictest1558893-21kworker/u8:3+flush-179:011:40:210
1485780993631,3cyclictest658-21Xorg10:02:590
1485780993630,5cyclictest1502672-21kworker/u8:1+events_unbound08:00:440
1485780993628,6cyclictest1560447-21latency_hist11:00:010
1485780993627,7cyclictest1579811-21latency_hist12:00:010
1485780993627,7cyclictest1567069-21/usr/sbin/munin11:20:170
1485788993532,2cyclictest0-21swapper/312:29:593
148578899350,2cyclictest0-21swapper/311:05:013
148578899350,1cyclictest0-21swapper/311:10:143
1485786993532,1cyclictest0-21swapper/207:44:582
1485786993530,3cyclictest0-21swapper/209:36:002
148578699350,2cyclictest0-21swapper/211:25:132
1485782993530,4cyclictest1558842-21kworker/u8:2+events_unbound11:00:351
1485782993530,4cyclictest1526749-21kworker/u8:1+flush-179:009:25:311
1485782993528,5cyclictest1546160-21kworker/u8:3+events_unbound10:20:281
1485780993531,3cyclictest1546160-21kworker/u8:3+flush-179:010:35:300
1485780993531,2cyclictest658-21Xorg07:30:580
1485780993528,5cyclictest1583117-21latency_hist12:10:010
148578099350,33cyclictest1555866-21date10:45:010
1485788993432,1cyclictest0-21swapper/308:41:593
1485788993428,5cyclictest0-21swapper/309:03:593
148578699340,2cyclictest0-21swapper/212:10:132
148578699340,29cyclictest0-21swapper/207:30:142
1485782993428,5cyclictest1575074-21kworker/u8:2+events_unbound12:40:011
1485780993430,3cyclictest1492450-21cron07:30:010
148578099341,29cyclictest0-21swapper/007:40:140
1485788993330,2cyclictest0-21swapper/307:44:593
1485788993328,4cyclictest0-21swapper/307:30:133
148578899330,2cyclictest0-21swapper/307:15:003
1485786993326,6cyclictest0-21swapper/209:45:012
148578699330,2cyclictest0-21swapper/207:15:012
148578699330,27cyclictest0-21swapper/211:40:132
148578699330,27cyclictest0-21swapper/210:55:142
1485780993329,2cyclictest658-21Xorg12:04:590
148578099330,29cyclictest1541582-21date10:00:010
1485788993230,1cyclictest0-21swapper/309:13:003
1485788993228,2cyclictest0-21swapper/309:35:593
1485788993226,5cyclictest0-21swapper/309:45:023
148578899320,30cyclictest0-21swapper/311:25:133
148578899320,26cyclictest0-21swapper/312:10:133
1485786993229,2cyclictest0-21swapper/210:11:002
1485786993228,2cyclictest0-21swapper/208:17:592
1485786993228,2cyclictest0-21swapper/208:17:582
1485786993225,6cyclictest0-21swapper/211:05:012
148578699320,30cyclictest0-21swapper/212:15:112
148578099321,30cyclictest0-21swapper/009:10:120
148578099321,2cyclictest1522010-21apt-get09:00:120
1485788993125,5cyclictest0-21swapper/310:55:143
1485788993121,7cyclictest0-21swapper/312:21:003
148578899310,2cyclictest0-21swapper/311:40:123
148578899310,29cyclictest0-21swapper/308:30:163
1485786993128,2cyclictest0-21swapper/212:22:002
1485786993128,2cyclictest0-21swapper/210:36:592
1485786993125,5cyclictest0-21swapper/208:50:122
1485786993124,6cyclictest0-21swapper/211:10:142
148578699310,29cyclictest0-21swapper/211:30:032
148578699310,29cyclictest0-21swapper/211:20:132
148578699310,29cyclictest0-21swapper/209:30:132
148578699310,29cyclictest0-21swapper/208:30:152
148578699310,29cyclictest0-21swapper/208:10:122
148578699310,29cyclictest0-21swapper/207:50:122
148578699310,1cyclictest0-21swapper/209:20:202
1485788993028,1cyclictest0-21swapper/308:00:153
1485788993025,4cyclictest0-21swapper/308:50:133
148578899301,27cyclictest4521791ptp4l10:00:013
148578899300,2cyclictest0-21swapper/310:50:133
148578899300,2cyclictest0-21swapper/309:50:013
148578899300,2cyclictest0-21swapper/307:50:113
148578899300,29cyclictest0-21swapper/312:15:113
148578899300,29cyclictest0-21swapper/308:50:003
148578899300,29cyclictest0-21swapper/308:20:143
148578899300,28cyclictest0-21swapper/310:45:013
148578899300,28cyclictest0-21swapper/310:10:133
148578899300,28cyclictest0-21swapper/309:30:143
148578899300,28cyclictest0-21swapper/307:40:003
148578899300,28cyclictest0-21swapper/307:15:023
1485786993027,2cyclictest0-21swapper/210:50:012
1485786993027,2cyclictest0-21swapper/210:40:122
1485786993027,2cyclictest0-21swapper/209:50:012
1485786993027,2cyclictest0-21swapper/208:50:002
1485786993027,2cyclictest0-21swapper/208:00:152
1485786993023,5cyclictest0-21swapper/207:20:582
148578699300,2cyclictest0-21swapper/211:50:112
148578699300,2cyclictest0-21swapper/207:15:032
148578699300,28cyclictest0-21swapper/212:00:142
148578699300,28cyclictest0-21swapper/208:20:152
148578699300,28cyclictest0-21swapper/207:40:002
1485782993025,4cyclictest1582397-21kworker/u8:0+events_unbound12:05:301
1485788992927,1cyclictest0-21swapper/310:35:123
1485788992923,5cyclictest0-21swapper/309:20:203
148578899290,28cyclictest0-21swapper/311:50:123
148578899290,27cyclictest0-21swapper/312:35:143
148578899290,27cyclictest0-21swapper/312:05:153
148578899290,27cyclictest0-21swapper/311:35:123
148578899290,27cyclictest0-21swapper/311:20:123
148578899290,27cyclictest0-21swapper/310:20:143
148578899290,27cyclictest0-21swapper/309:15:133
148578899290,27cyclictest0-21swapper/307:55:153
148578899290,27cyclictest0-21swapper/307:30:023
148578899290,26cyclictest0-21swapper/311:30:033
1485786992926,2cyclictest0-21swapper/212:05:152
1485786992926,2cyclictest0-21swapper/210:45:122
148578699290,28cyclictest0-21swapper/210:00:012
148578699290,27cyclictest0-21swapper/212:35:132
148578699290,27cyclictest0-21swapper/211:45:132
148578699290,27cyclictest0-21swapper/211:35:122
148578699290,27cyclictest0-21swapper/211:15:112
148578699290,27cyclictest0-21swapper/210:20:142
148578699290,27cyclictest0-21swapper/209:45:012
148578699290,27cyclictest0-21swapper/209:15:142
148578699290,27cyclictest0-21swapper/208:58:002
148578699290,21cyclictest0-21swapper/207:30:022
1485780992926,2cyclictest0-21swapper/008:36:590
4521791280,6ptp4l441ktimers/311:50:003
1485788992826,1cyclictest0-21swapper/311:15:113
1485788992825,2cyclictest0-21swapper/311:05:013
148578899280,2cyclictest0-21swapper/311:55:013
148578899280,27cyclictest0-21swapper/310:45:123
148578899280,27cyclictest0-21swapper/308:10:123
148578899280,26cyclictest0-21swapper/312:00:133
148578899280,26cyclictest0-21swapper/310:05:143
148578899280,26cyclictest0-21swapper/309:05:143
1485786992825,2cyclictest0-21swapper/212:30:022
1485786992825,2cyclictest0-21swapper/212:30:022
1485786992825,2cyclictest0-21swapper/211:05:012
148578699280,2cyclictest0-21swapper/211:55:012
148578699280,2cyclictest0-21swapper/209:05:142
148578699280,27cyclictest0-21swapper/210:00:122
148578699280,26cyclictest0-21swapper/207:45:122
1485780992823,3cyclictest1485853-21apt-key07:10:030
4521791270,7ptp4l441ktimers/309:44:543
4521791270,7ptp4l441ktimers/308:59:553
4521791270,6ptp4l441ktimers/312:35:003
4521791270,6ptp4l441ktimers/312:34:593
4521791270,6ptp4l441ktimers/308:30:003
1485788992725,1cyclictest0-21swapper/308:17:593
1485788992725,1cyclictest0-21swapper/308:17:593
148578899270,26cyclictest0-21swapper/310:00:123
148578899270,24cyclictest0-21swapper/308:35:143
148578699270,25cyclictest0-21swapper/210:20:012
148578699270,25cyclictest0-21swapper/208:25:152
148578699270,1cyclictest0-21swapper/210:05:132
148578699270,19cyclictest0-21swapper/208:35:142
1485782992723,3cyclictest1485682-21apt-key07:10:021
4521791260,5ptp4l441ktimers/310:29:563
148578899260,24cyclictest0-21swapper/310:30:103
148578699260,24cyclictest0-21swapper/210:30:112
148578899250,2cyclictest0-21swapper/310:19:593
1485780992520,4cyclictest1490854-21sshd07:24:140
1485786991714,2cyclictest0-21swapper/210:25:592
14857869930,2cyclictest0-21swapper/207:10:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional