You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 03:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Sat Jul 27, 2024 00:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151284525622,29sleep70-21swapper/718:58:257
151301225545,7sleep50-21swapper/519:00:485
151302024736,8sleep40-21swapper/419:00:554
1513433993833,4cyclictest630-21systemd-journal22:22:304
1513441993631,4cyclictest630-21systemd-journal23:37:306
151291923523,8sleep60-21swapper/618:59:296
1513433992927,2cyclictest2083059-21kworker/u16:2+events_unbound00:02:304
1513438992724,2cyclictest1848483-21kworker/u16:3+events_unbound22:17:475
1513438992622,3cyclictest2131909-21latency_hist00:17:315
1513433992421,2cyclictest1986204-21kworker/u16:3+flush-8:023:12:504
1513433992421,2cyclictest1502294-21kworker/u16:4+flush-8:019:12:544
1513433992421,2cyclictest1502294-21kworker/u16:4+flush-8:019:12:544
1513433992321,2cyclictest2054078-21kworker/u16:1+flush-8:023:42:444
1513433992321,2cyclictest1566189-21kworker/u16:3+flush-8:019:32:314
1513438992218,3cyclictest2004855-21latency_hist23:12:305
1513438992117,3cyclictest1653759-21latency_hist20:12:315
1513433992119,2cyclictest1776838-21kworker/u16:0+events_unbound21:37:434
1513433992119,2cyclictest1740648-21kworker/u16:1+events_unbound20:57:454
1513433992119,2cyclictest1673218-21kworker/u16:2+flush-8:021:27:494
1513433992119,2cyclictest1653535-21kworker/u16:1+flush-8:020:27:264
1513441992013,7cyclictest0-21swapper/619:10:256
1513441992013,7cyclictest0-21swapper/619:10:256
1513438992018,2cyclictest2083381-21kworker/u16:3+events_unbound23:52:595
1513433992018,2cyclictest1776838-21kworker/u16:0+flush-8:021:43:074
1513441991917,2cyclictest1888195-21kworker/u16:4+flush-8:022:38:016
1513438991917,2cyclictest1986204-21kworker/u16:3+events_unbound23:42:435
1513433991917,2cyclictest2141610-21kworker/u16:3+events_unbound00:22:304
1513433991917,2cyclictest2131835-21kworker/u16:2+flush-8:000:27:404
1513433991917,2cyclictest2102825-21kworker/u16:0+events_unbound00:05:394
1513433991917,2cyclictest1986204-21kworker/u16:3+flush-8:023:27:314
1513433991917,2cyclictest1986204-21kworker/u16:3+flush-8:023:27:314
1513433991917,2cyclictest1907684-21kworker/u16:2+events_unbound22:47:254
1513433991917,2cyclictest1653535-21kworker/u16:1+flush-8:020:27:544
1513433991917,2cyclictest1653535-21kworker/u16:1+flush-8:020:17:524
1513433991917,2cyclictest1633174-21kworker/u16:4+events_unbound21:35:494
1513433991916,3cyclictest1566188-21kworker/u16:0+flush-8:019:37:314
1513433991915,3cyclictest1907684-21kworker/u16:2+flush-8:023:02:464
151344199180,16cyclictest1523393-21latency_hist19:07:316
1513438991816,2cyclictest2054078-21kworker/u16:1+flush-8:000:27:465
1513433991816,2cyclictest2131835-21kworker/u16:2+events_unbound00:27:054
1513433991816,2cyclictest1986204-21kworker/u16:3+events_unbound23:25:024
1513433991816,2cyclictest1986204-21kworker/u16:3+events_unbound23:25:024
1513433991816,2cyclictest1776838-21kworker/u16:0+flush-8:022:02:424
1513433991816,2cyclictest1776838-21kworker/u16:0+events_unbound21:55:014
1513433991816,2cyclictest1483693-21kworker/u16:1+flush-8:019:47:414
1513433991816,2cyclictest1483693-21kworker/u16:1+events_unbound19:24:574
151344699174,5cyclictest0-21swapper/719:04:247
1513446991716,0cyclictest0-21swapper/722:42:547
151344699170,1cyclictest0-21swapper/722:57:547
1513433991715,2cyclictest2083381-21kworker/u16:3+events_unbound23:52:434
1513433991715,2cyclictest2054078-21kworker/u16:1+flush-8:000:12:504
1513433991715,2cyclictest1995451-21kworker/u16:4+events_unbound23:32:414
1513433991715,2cyclictest1907684-21kworker/u16:2+events_unbound22:30:214
1513433991715,2cyclictest1888195-21kworker/u16:4+events_unbound22:41:414
1513433991715,2cyclictest1776838-21kworker/u16:0+flush-8:021:47:404
1513433991715,2cyclictest1633174-21kworker/u16:4+events_unbound20:02:074
1513433991714,2cyclictest1576190-21kworker/u16:2+flush-8:019:52:564
1513433991714,2cyclictest1135-21snmpd23:48:514
151344699163,6cyclictest0-21swapper/700:19:107
151344699163,5cyclictest0-21swapper/720:13:547
151344699162,6cyclictest0-21swapper/721:29:567
151344699160,1cyclictest0-21swapper/719:42:577
1513441991613,3cyclictest1673218-21kworker/u16:2+flush-8:020:37:306
1513441991613,2cyclictest2013860-21awk23:12:596
151344199160,15cyclictest1784896-21grep21:18:046
151344199160,15cyclictest1784896-21grep21:18:046
1513438991615,1cyclictest1624254-21kworker/u16:0+flush-8:020:02:315
1513438991614,2cyclictest1464948-21kworker/u16:2+events_unbound19:02:505
1513438991613,2cyclictest1740648-21kworker/u16:1+flush-8:021:02:315
1513438991613,2cyclictest1566189-21kworker/u16:3+flush-8:019:53:015
151343899160,15cyclictest0-21swapper/520:22:575
1513433991614,2cyclictest1740648-21kworker/u16:1+events_unbound21:06:444
1513433991614,2cyclictest1633174-21kworker/u16:4+events_unbound20:12:404
1513433991614,2cyclictest1556099-21kworker/u16:2+events_unbound19:22:254
151344699153,5cyclictest0-21swapper/720:57:317
151344199157,8cyclictest0-21swapper/623:05:356
1513441991513,2cyclictest1888195-21kworker/u16:4+events_unbound22:22:406
151344199150,14cyclictest1684409-21irqcore20:27:466
1513438991513,2cyclictest1907684-21kworker/u16:2+events_unbound23:32:305
1513438991513,2cyclictest1907684-21kworker/u16:2+events_unbound23:32:305
1513438991513,2cyclictest1673218-21kworker/u16:2+flush-8:021:22:465
1513438991510,4cyclictest2024406-21latency_hist23:22:305
151343899150,1cyclictest0-21swapper/520:17:485
1513433991513,2cyclictest2054078-21kworker/u16:1+events_unbound23:40:134
1513433991513,2cyclictest2054078-21kworker/u16:1+events_unbound00:12:304
1513433991513,2cyclictest1907684-21kworker/u16:2+flush-8:023:07:514
1513433991513,2cyclictest1691585-21kworker/u16:0+flush-8:020:32:514
1513433991513,2cyclictest1691585-21kworker/u16:0+events_unbound20:57:004
1513433991513,2cyclictest1633174-21kworker/u16:4+flush-8:021:23:234
1513433991513,2cyclictest1566188-21kworker/u16:0+events_unbound19:42:054
1513433991513,2cyclictest1483693-21kworker/u16:1+flush-8:019:07:514
1513433991513,2cyclictest1483693-21kworker/u16:1+flush-8:019:07:514
151344699146,7cyclictest0-21swapper/721:19:527
151344699146,7cyclictest0-21swapper/721:19:527
151344699146,7cyclictest0-21swapper/720:51:587
1513446991413,0cyclictest0-21swapper/720:22:307
1513446991412,1cyclictest0-21swapper/721:07:557
1513446991412,1cyclictest0-21swapper/719:27:317
1513446991411,2cyclictest0-21swapper/719:15:527
1513446991411,2cyclictest0-21swapper/719:15:527
151344699140,6cyclictest0-21swapper/723:29:357
151344699140,6cyclictest0-21swapper/723:29:357
151344699140,6cyclictest0-21swapper/723:14:237
151344699140,6cyclictest0-21swapper/723:07:447
151344699140,6cyclictest0-21swapper/722:33:077
151344699140,6cyclictest0-21swapper/722:13:487
151344699140,6cyclictest0-21swapper/700:04:427
151344699140,1cyclictest0-21swapper/721:23:097
151344699140,13cyclictest0-21swapper/722:17:597
151344199146,7cyclictest0-21swapper/620:11:056
1513441991412,2cyclictest1483693-21kworker/u16:1+flush-8:019:44:266
1513441991411,2cyclictest1135-21snmpd23:51:006
151344199140,1cyclictest0-21swapper/620:57:536
151344199140,14cyclictest0-21swapper/600:27:526
151344199140,14cyclictest0-21swapper/600:07:516
1513438991413,1cyclictest0-21swapper/522:37:485
1513438991412,2cyclictest2083059-21kworker/u16:2+flush-8:023:57:455
1513438991412,2cyclictest1995451-21kworker/u16:4+events_unbound23:12:555
1513438991412,2cyclictest1776838-21kworker/u16:0+flush-8:021:42:505
1513438991410,3cyclictest2102825-21kworker/u16:0+flush-8:000:07:555
151343899140,13cyclictest0-21swapper/500:02:375
1513433991412,2cyclictest1986204-21kworker/u16:3+events_unbound23:22:284
1513433991412,2cyclictest1907684-21kworker/u16:2+events_unbound22:51:084
1513433991412,2cyclictest1858298-21kworker/u16:2+events_unbound21:58:044
1513433991412,2cyclictest1691585-21kworker/u16:0+events_unbound20:46:484
1513433991412,2cyclictest1673218-21kworker/u16:2+flush-8:020:47:464
1513433991412,2cyclictest1624254-21kworker/u16:0+flush-8:020:07:564
1513433991412,2cyclictest1483693-21kworker/u16:1+flush-8:019:43:424
1513433991411,2cyclictest1673218-21kworker/u16:2+flush-8:021:18:184
1513433991411,2cyclictest1673218-21kworker/u16:2+flush-8:021:18:184
1513433991410,3cyclictest1633174-21kworker/u16:4+events_unbound20:02:504
151344699136,6cyclictest0-21swapper/723:47:097
151344699135,7cyclictest0-21swapper/719:50:117
151344699134,8cyclictest0-21swapper/700:31:377
151344699134,7cyclictest0-21swapper/721:04:107
151344699130,6cyclictest0-21swapper/720:43:127
151344699130,5cyclictest0-21swapper/721:45:237
151344699130,12cyclictest0-21swapper/722:02:337
151344699130,12cyclictest0-21swapper/720:41:517
151344699130,12cyclictest0-21swapper/719:21:227
151344199138,4cyclictest2133072-21cat00:17:456
1513441991312,1cyclictest0-21swapper/620:06:576
151344199130,7cyclictest0-21swapper/619:57:456
151344199130,13cyclictest0-21swapper/621:32:436
151344199130,13cyclictest0-21swapper/620:12:576
151343899135,8cyclictest0-21swapper/520:04:235
151343899134,8cyclictest0-21swapper/523:26:445
151343899134,8cyclictest0-21swapper/523:26:445
1513438991312,1cyclictest0-21swapper/523:06:395
1513438991312,1cyclictest0-21swapper/522:46:575
1513438991311,2cyclictest1776838-21kworker/u16:0+flush-8:021:17:315
1513438991311,1cyclictest0-21swapper/521:48:325
1513438991310,2cyclictest1935052-21kthreadcore22:32:515
151343899130,13cyclictest0-21swapper/522:05:005
1513433991311,2cyclictest1975442-21kworker/u16:1+events_unbound22:56:424
1513433991311,2cyclictest1907684-21kworker/u16:2+events_unbound22:24:104
1513433991311,2cyclictest1858298-21kworker/u16:2+events_unbound22:10:584
1513433991311,2cyclictest1633174-21kworker/u16:4+events_unbound21:14:164
151344699129,2cyclictest0-21swapper/700:11:497
151344699120,1cyclictest0-21swapper/720:07:567
151344199129,2cyclictest1135-21snmpd20:25:306
151344199129,2cyclictest1135-21snmpd00:15:526
151344199127,4cyclictest1556313-21latency_hist19:22:316
151344199125,7cyclictest0-21swapper/622:31:286
1513441991211,1cyclictest0-21swapper/623:59:016
1513441991211,1cyclictest0-21swapper/623:45:096
1513441991211,1cyclictest0-21swapper/622:56:266
1513441991211,1cyclictest0-21swapper/621:46:266
1513441991211,1cyclictest0-21swapper/619:32:266
1513441991210,1cyclictest0-21swapper/621:22:446
151344199120,12cyclictest0-21swapper/621:57:576
151343899129,2cyclictest2045303-21irqcore23:32:465
151343899129,2cyclictest1135-21snmpd20:34:545
151343899127,4cyclictest1723147-21cat20:47:465
151343899125,7cyclictest0-21swapper/521:28:175
151343899124,8cyclictest0-21swapper/519:12:225
151343899124,8cyclictest0-21swapper/519:12:225
151343899123,9cyclictest0-21swapper/500:20:585
1513438991211,1cyclictest0-21swapper/522:09:065
1513438991211,1cyclictest0-21swapper/521:06:195
1513438991211,1cyclictest0-21swapper/519:45:105
1513438991210,1cyclictest0-21swapper/519:33:585
151343899120,12cyclictest0-21swapper/523:49:415
151343899120,12cyclictest0-21swapper/500:27:135
1513433991210,2cyclictest1888195-21kworker/u16:4+events_unbound22:14:494
1513433991210,2cyclictest1776838-21kworker/u16:0+events_unbound22:58:354
1513433991210,2cyclictest1776838-21kworker/u16:0+events_unbound22:35:264
1513433991210,2cyclictest1464948-21kworker/u16:2+events_unbound19:04:514
1513433991210,1cyclictest1762302-21irqcore21:07:474
151344699118,2cyclictest0-21swapper/722:57:107
1513446991110,0cyclictest0-21swapper/719:55:317
151344699110,9cyclictest0-21swapper/721:58:007
151344699110,6cyclictest0-21swapper/721:33:117
151344199118,2cyclictest1135-21snmpd20:51:036
151344199118,2cyclictest1135-21snmpd19:54:236
151344199117,3cyclictest1901898-21kthreadcore22:17:546
1513441991110,1cyclictest0-21swapper/623:23:056
1513441991110,1cyclictest0-21swapper/623:23:056
1513441991110,1cyclictest0-21swapper/623:18:026
1513441991110,1cyclictest0-21swapper/623:07:576
1513441991110,1cyclictest0-21swapper/623:02:266
1513441991110,1cyclictest0-21swapper/622:35:476
1513441991110,1cyclictest0-21swapper/622:03:476
1513441991110,1cyclictest0-21swapper/621:50:326
1513441991110,1cyclictest0-21swapper/621:07:306
151344199110,6cyclictest0-21swapper/619:24:076
151344199110,11cyclictest0-21swapper/623:39:106
151344199110,11cyclictest0-21swapper/621:55:066
151344199110,11cyclictest0-21swapper/621:42:206
151344199110,11cyclictest0-21swapper/619:38:256
151344199110,10cyclictest1884342-21kthreadcore22:07:496
151343899119,2cyclictest1975442-21kworker/u16:1+events_unbound22:57:435
151343899119,2cyclictest1776838-21kworker/u16:0+flush-8:022:28:005
151343899119,2cyclictest1483693-21kworker/u16:1+flush-8:019:22:525
151343899119,1cyclictest0-21swapper/520:16:305
151343899119,1cyclictest0-21swapper/519:47:475
151343899118,2cyclictest26695-21JS00:51:085
151343899118,2cyclictest1673218-21kworker/u16:2+events_unbound21:32:545
151343899117,3cyclictest1897603-21latency_hist22:17:305
151343899116,4cyclictest1567229-21cat19:27:475
1513438991110,1cyclictest0-21swapper/523:38:115
1513438991110,1cyclictest0-21swapper/520:56:355
1513438991110,1cyclictest0-21swapper/519:19:285
1513438991110,1cyclictest0-21swapper/519:13:005
1513438991110,1cyclictest0-21swapper/519:13:005
151343399119,2cyclictest1673218-21kworker/u16:2+flush-8:020:37:524
151344699109,0cyclictest0-21swapper/721:56:077
151344699107,2cyclictest0-21swapper/723:21:027
151344699100,9cyclictest0-21swapper/721:37:557
151344699100,9cyclictest0-21swapper/719:29:257
151344699100,8cyclictest0-21swapper/719:40:137
151344699100,5cyclictest0-21swapper/720:59:057
151344699100,5cyclictest0-21swapper/720:26:097
151344199109,1cyclictest0-21swapper/623:28:076
151344199109,1cyclictest0-21swapper/623:28:076
151344199109,1cyclictest0-21swapper/622:49:586
151344199109,1cyclictest0-21swapper/621:27:416
151344199109,1cyclictest0-21swapper/620:18:516
151344199107,3cyclictest2141610-21kworker/u16:3+events_unbound00:27:296
151344199106,3cyclictest1769469-21kthreadcore21:07:526
151344199106,3cyclictest1541086-21cat19:12:526
151344199106,3cyclictest1541086-21cat19:12:526
151344199100,9cyclictest0-21swapper/620:37:546
151343899109,1cyclictest0-21swapper/522:56:345
151343899109,1cyclictest0-21swapper/522:51:315
151343899109,1cyclictest0-21swapper/521:09:215
151343899108,2cyclictest1858298-21kworker/u16:2+flush-8:021:58:015
15134469990,8cyclictest0-21swapper/720:01:087
15134469990,8cyclictest0-21swapper/719:36:497
15134469990,8cyclictest0-21swapper/719:07:467
15134469990,8cyclictest0-21swapper/719:07:467
15134469990,8cyclictest0-21swapper/700:00:427
15134469990,7cyclictest0-21swapper/723:37:507
15134469990,4cyclictest0-21swapper/723:07:237
15134419995,3cyclictest1732367-21cstates20:52:436
15134419990,9cyclictest0-21swapper/619:34:316
15134419990,8cyclictest1771182-21/usr/sbin/munin21:12:436
15134419990,3cyclictest2108005-21kthreadcore00:02:526
15134389998,1cyclictest0-21swapper/522:25:025
15134389997,2cyclictest1673218-21kworker/u16:2+events_unbound20:42:585
15134389997,2cyclictest1653535-21kworker/u16:1+flush-8:020:27:525
15134389996,3cyclictest1633174-21kworker/u16:4+flush-8:021:18:015
15134389996,3cyclictest1633174-21kworker/u16:4+flush-8:021:18:015
15134469987,0cyclictest0-21swapper/723:37:107
15134469987,0cyclictest0-21swapper/722:48:017
15134469985,2cyclictest0-21swapper/720:03:237
15134469985,2cyclictest0-21swapper/700:16:587
15134469981,6cyclictest0-21swapper/721:12:407
15134419985,2cyclictest1952234-21kthreadcore22:42:506
15134419984,4cyclictest27491-21gmain22:12:366
15134419983,4cyclictest1716091-21grep20:42:496
15134419980,7cyclictest2083514-21cat23:52:416
15134389987,1cyclictest0-21swapper/520:40:405
15134389987,1cyclictest0-21swapper/519:39:395
15134389980,7cyclictest0-21swapper/521:52:445
15134469976,0cyclictest0-21swapper/722:27:557
15134469976,0cyclictest0-21swapper/721:47:507
15134469974,2cyclictest0-21swapper/722:38:317
15134469970,6cyclictest16850irq/16-ehci_hcd:usb323:22:427
15134469970,6cyclictest16850irq/16-ehci_hcd:usb323:22:427
15134469970,1cyclictest0-21swapper/700:22:527
15134469970,0cyclictest0-21swapper/720:32:487
15134419973,3cyclictest1607506-21cut19:47:506
15134469960,5cyclictest0-21swapper/722:09:287
15134469960,5cyclictest0-21swapper/720:27:537
15134469960,4cyclictest2083380-21taskset23:52:357
15134469960,2cyclictest19077312sleep722:22:397
15134469950,1cyclictest0-21swapper/723:47:397
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional