You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 02:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Tue May 13, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21793582830,0sleep40-21swapper/404:16:294
2092552993734,2cyclictest2271633-21kworker/u16:3+flush-8:005:36:285
2092568993327,5cyclictest2776301-21latency_hist09:06:287
2092543993229,2cyclictest2415872-21kworker/u16:4+events_unbound06:11:464
2092552993129,2cyclictest2116731-21kworker/u16:3+flush-8:004:06:285
2092552993127,3cyclictest2187921-21kworker/u16:0+events_unbound04:21:465
2092543993027,2cyclictest2187921-21kworker/u16:0+flush-8:004:46:434
2092560992927,2cyclictest2187921-21kworker/u16:0+flush-8:004:41:416
2092552992725,2cyclictest2406197-21kworker/u16:3+events_unbound06:06:575
2092552992724,3cyclictest2187921-21kworker/u16:0+events_unbound05:01:585
2092543992724,2cyclictest2672911-21kworker/u16:4+events_unbound08:31:574
2092543992723,3cyclictest2091048-21kworker/u16:1+events_unbound03:51:464
2092543992624,2cyclictest2518595-21kworker/u16:0+flush-8:009:01:534
2092543992623,2cyclictest2692833-21kworker/u16:5+events_unbound08:51:574
2092560992523,2cyclictest2187921-21kworker/u16:0+flush-8:006:01:586
2092552992523,2cyclictest2415872-21kworker/u16:4+flush-8:006:16:535
2092552992523,2cyclictest2415872-21kworker/u16:4+flush-8:006:16:535
2092552992523,2cyclictest2091048-21kworker/u16:1+flush-8:003:36:545
2092543992522,2cyclictest2187921-21kworker/u16:0+flush-8:004:21:534
2092543992522,2cyclictest2091048-21kworker/u16:1+events_unbound04:26:434
2092543992321,2cyclictest2291092-21kworker/u16:5+flush-8:005:41:534
2092543992321,2cyclictest2091048-21kworker/u16:1+flush-8:003:36:564
2092543992319,3cyclictest2271633-21kworker/u16:3+flush-8:005:36:434
2092568992218,2cyclictest2189054-21munin-run04:21:287
2092543992220,2cyclictest2518595-21kworker/u16:0+flush-8:008:16:584
2092543992218,3cyclictest2219069-21kworker/u16:4+flush-8:004:36:534
2092568992119,1cyclictest2384950-21awk05:56:287
209256899210,19cyclictest2178772-21latency_hist04:16:287
209256899200,18cyclictest2693797-21latency_hist08:26:277
2092543992017,2cyclictest2374298-21kworker/u16:2+flush-8:007:11:544
2092568991913,5cyclictest2570372-21latency_hist07:26:287
209256899190,1cyclictest0-21swapper/707:31:577
2092552991917,2cyclictest2374298-21kworker/u16:2+events_unbound07:16:585
2092552991917,2cyclictest2291092-21kworker/u16:5+flush-8:005:41:285
2092552991916,2cyclictest2672911-21kworker/u16:4+events_unbound08:51:455
2092552991915,3cyclictest2116731-21kworker/u16:3+flush-8:004:11:465
2092543991917,2cyclictest2374298-21kworker/u16:2+events_unbound07:06:294
2092568991815,2cyclictest1313-21snmpd06:47:437
2092568991815,2cyclictest1313-21snmpd04:24:067
2092568991812,5cyclictest2765995-21latency_hist09:01:277
2092543991815,3cyclictest2415872-21kworker/u16:4+flush-8:007:01:424
2092543991815,3cyclictest2415872-21kworker/u16:4+flush-8:007:01:424
2092543991815,2cyclictest2198441-21kworker/u16:2+flush-8:005:21:454
2092568991714,2cyclictest1313-21snmpd06:55:257
2092568991712,4cyclictest2323077-21latency_hist05:26:277
2092543991715,2cyclictest2518595-21kworker/u16:0+flush-8:008:46:534
210245321614,1sleep70-21swapper/703:39:477
2092568991614,1cyclictest2683494-21munin-run08:21:267
2092568991613,2cyclictest1313-21snmpd08:50:017
2092568991613,2cyclictest1313-21snmpd05:40:327
2092568991613,2cyclictest1313-21snmpd04:31:587
2092552991614,2cyclictest2518595-21kworker/u16:0+flush-8:007:46:475
2092552991614,2cyclictest2374298-21kworker/u16:2+flush-8:006:16:275
2092543991614,2cyclictest2363186-21kworker/u16:1+events_unbound05:51:564
2092543991614,2cyclictest2116731-21kworker/u16:3+flush-8:003:46:544
2092543991614,2cyclictest2091048-21kworker/u16:1+flush-8:004:01:574
2092568991514,0cyclictest0-21swapper/707:41:587
209256899150,14cyclictest1313-21snmpd05:34:537
2092560991513,2cyclictest2374298-21kworker/u16:2+events_unbound07:26:476
2092543991513,2cyclictest2187921-21kworker/u16:0+flush-8:005:56:534
2092543991512,2cyclictest2415872-21kworker/u16:4+flush-8:006:46:534
2092568991413,0cyclictest0-21swapper/707:36:557
2092568991411,2cyclictest1313-21snmpd07:00:357
2092568991410,3cyclictest1313-21snmpd06:09:217
2092568991410,3cyclictest1313-21snmpd05:30:327
2092568991410,3cyclictest1313-21snmpd04:06:307
209256899140,13cyclictest643-21systemd-journal08:06:277
2092560991411,2cyclictest2592160-21cstates07:36:436
2092552991412,2cyclictest2374298-21kworker/u16:2+flush-8:007:06:455
2092552991412,2cyclictest2219069-21kworker/u16:4+events_unbound04:56:575
2092552991411,3cyclictest2187921-21kworker/u16:0+flush-8:005:41:595
209255299140,14cyclictest0-21swapper/505:06:545
209255299140,14cyclictest0-21swapper/504:51:515
2092543991413,1cyclictest0-21swapper/407:41:544
2092543991413,1cyclictest0-21swapper/405:10:054
2092543991412,1cyclictest0-21swapper/406:16:464
2092543991412,1cyclictest0-21swapper/406:16:464
2092543991410,3cyclictest2703908-21updatedb08:27:274
209254399140,1cyclictest0-21swapper/404:16:484
209256899139,3cyclictest27394-21gmain06:21:497
209256899136,6cyclictest0-21swapper/708:09:067
2092568991312,0cyclictest0-21swapper/705:01:297
2092568991310,2cyclictest2157186-21ntpq04:01:527
209256899130,11cyclictest2664916-21kthreadcore08:11:467
209256899130,11cyclictest2394872-21chrt05:57:197
2092560991312,1cyclictest0-21swapper/606:21:506
209256099130,13cyclictest0-21swapper/607:31:576
209256099130,13cyclictest0-21swapper/607:18:456
209256099130,13cyclictest0-21swapper/605:01:566
2092552991312,1cyclictest0-21swapper/505:46:525
2092552991311,2cyclictest2415872-21kworker/u16:4+flush-8:007:01:465
2092552991311,2cyclictest2415872-21kworker/u16:4+flush-8:007:01:465
2092552991310,2cyclictest2374298-21kworker/u16:2+events_unbound06:01:305
2092552991310,2cyclictest2224025-21kthreadcore04:36:495
209255299130,13cyclictest0-21swapper/507:26:585
209255299130,12cyclictest0-21swapper/506:56:555
2092543991312,1cyclictest0-21swapper/407:26:554
2092543991312,1cyclictest0-21swapper/406:33:294
209254399130,1cyclictest0-21swapper/408:22:434
209254399130,13cyclictest0-21swapper/405:01:544
209256899129,2cyclictest2621833-21ls07:51:277
209256899129,2cyclictest2572122-21taskset07:26:487
209256899129,2cyclictest1313-21snmpd05:17:447
209256899129,2cyclictest1313-21snmpd04:59:167
209256899129,2cyclictest1313-21snmpd04:55:287
209256899129,2cyclictest0-21swapper/707:06:467
209256899128,3cyclictest1313-21snmpd08:30:137
209256899128,3cyclictest1313-21snmpd05:43:537
209256899128,3cyclictest1313-21snmpd04:46:017
209256899127,4cyclictest2147651-21sshd-session03:59:067
209256899127,4cyclictest2147651-21sshd-session03:59:067
209256899124,7cyclictest2126896-21sshd-session03:47:077
209256899120,4cyclictest2641436-21ntp_kernel_pll_07:56:547
2092560991211,1cyclictest0-21swapper/608:20:136
209256099120,1cyclictest0-21swapper/608:35:126
209256099120,1cyclictest0-21swapper/608:30:006
209256099120,1cyclictest0-21swapper/606:08:176
209256099120,1cyclictest0-21swapper/604:09:536
2092552991211,1cyclictest0-21swapper/505:59:295
2092552991210,2cyclictest2415872-21kworker/u16:4+flush-8:007:21:565
2092552991210,2cyclictest2271633-21kworker/u16:3+events_unbound05:26:415
2092552991210,2cyclictest2138229-21kworker/u16:5+flush-8:004:01:285
2092552991210,2cyclictest2138229-21kworker/u16:5+flush-8:004:01:285
209254399128,3cyclictest2198441-21kworker/u16:2+events_unbound05:11:584
209254399127,4cyclictest2262700-21cat04:56:454
2092543991211,1cyclictest0-21swapper/408:44:574
2092543991211,1cyclictest0-21swapper/407:35:264
2092543991211,1cyclictest0-21swapper/406:58:044
2092543991211,1cyclictest0-21swapper/405:49:014
2092543991211,1cyclictest0-21swapper/405:34:224
2092543991211,1cyclictest0-21swapper/405:16:424
2092543991211,1cyclictest0-21swapper/404:00:244
2092543991211,1cyclictest0-21swapper/404:00:244
209254399120,1cyclictest0-21swapper/406:54:464
209254399120,1cyclictest0-21swapper/404:08:554
209254399120,12cyclictest0-21swapper/407:23:554
209254399120,12cyclictest0-21swapper/405:27:544
209254399120,12cyclictest0-21swapper/404:55:544
209254399120,12cyclictest0-21swapper/404:43:454
209254399120,12cyclictest0-21swapper/404:35:054
209256899118,2cyclictest1313-21snmpd06:31:587
209256899118,2cyclictest0-21swapper/707:16:527
209256899117,3cyclictest27924-21gmain08:42:007
209256899117,3cyclictest1313-21snmpd05:15:567
209256899116,4cyclictest2549534-21sshd-session07:13:057
209256899115,5cyclictest27763-21gvfs-afc-volume05:46:447
2092568991111,0cyclictest0-21swapper/704:28:427
209256899110,3cyclictest2117467-21sendmail03:46:297
209256899110,10cyclictest2454394-21kthreadcore06:26:487
209256899110,10cyclictest2228763-21kthreadcore04:36:527
209256899110,10cyclictest1313-21snmpd08:51:557
209256899110,10cyclictest1313-21snmpd06:42:017
209256899110,10cyclictest1313-21snmpd06:17:307
209256899110,10cyclictest1313-21snmpd06:17:307
2092560991110,1cyclictest0-21swapper/608:57:076
2092560991110,1cyclictest0-21swapper/608:54:266
2092560991110,1cyclictest0-21swapper/608:06:316
2092560991110,1cyclictest0-21swapper/607:25:316
2092560991110,1cyclictest0-21swapper/607:01:346
2092560991110,1cyclictest0-21swapper/607:01:346
2092560991110,1cyclictest0-21swapper/605:55:406
2092560991110,1cyclictest0-21swapper/605:13:346
209256099110,1cyclictest0-21swapper/607:52:576
209256099110,11cyclictest0-21swapper/608:38:406
209256099110,11cyclictest0-21swapper/607:43:016
209256099110,11cyclictest0-21swapper/606:53:036
209256099110,11cyclictest0-21swapper/606:32:246
209256099110,11cyclictest0-21swapper/605:47:006
209256099110,11cyclictest0-21swapper/605:31:526
209256099110,11cyclictest0-21swapper/605:23:406
209256099110,11cyclictest0-21swapper/604:13:216
2092552991110,1cyclictest0-21swapper/508:24:405
2092552991110,1cyclictest0-21swapper/508:11:455
2092552991110,1cyclictest0-21swapper/505:17:525
2092552991110,1cyclictest0-21swapper/504:50:545
2092552991110,1cyclictest0-21swapper/504:18:455
2092552991110,1cyclictest0-21swapper/503:46:355
209255299110,1cyclictest0-21swapper/507:13:155
209255299110,1cyclictest0-21swapper/505:55:325
209255299110,1cyclictest0-21swapper/505:15:585
209255299110,11cyclictest0-21swapper/508:37:455
209255299110,11cyclictest0-21swapper/506:45:505
209255299110,11cyclictest0-21swapper/505:23:235
209255299110,11cyclictest0-21swapper/504:28:125
209254399116,4cyclictest2404686-21/usr/sbin/munin06:01:574
2092543991110,1cyclictest0-21swapper/408:08:424
2092543991110,1cyclictest0-21swapper/406:28:354
2092543991110,1cyclictest0-21swapper/406:24:394
209254399110,11cyclictest0-21swapper/407:53:534
209256899107,2cyclictest1313-21snmpd06:02:307
209256899106,3cyclictest1313-21snmpd06:41:107
209256899106,3cyclictest1313-21snmpd06:11:487
209256899106,3cyclictest1313-21snmpd04:49:347
209256899106,3cyclictest1313-21snmpd03:53:247
209256899105,4cyclictest2716234-21sed08:36:477
209256899100,2cyclictest2622528-21/usr/sbin/munin07:51:587
209256899100,2cyclictest0-21swapper/708:31:587
209256099109,1cyclictest0-21swapper/607:50:166
209256099109,1cyclictest0-21swapper/605:19:056
209256099106,3cyclictest2479145-21cat06:41:446
209256099106,3cyclictest2221518-21cat04:36:466
209255299109,1cyclictest0-21swapper/508:32:335
209255299109,1cyclictest0-21swapper/508:02:365
209255299109,1cyclictest0-21swapper/507:39:545
209255299109,1cyclictest0-21swapper/504:44:265
209255299109,1cyclictest0-21swapper/504:34:125
209255299108,2cyclictest2672911-21kworker/u16:4+events_unbound08:56:465
209254399109,1cyclictest0-21swapper/407:20:564
209254399109,1cyclictest0-21swapper/406:39:194
209254399107,2cyclictest2715349-21/usr/sbin/munin08:36:454
20925689998,0cyclictest0-21swapper/705:06:417
20925689992,6cyclictest27394-21gmain07:01:447
20925689992,6cyclictest27394-21gmain07:01:447
20925609998,1cyclictest0-21swapper/608:48:176
20925609998,1cyclictest0-21swapper/608:02:536
20925609998,1cyclictest0-21swapper/607:06:466
20925609998,1cyclictest0-21swapper/606:16:006
20925609998,1cyclictest0-21swapper/604:54:116
20925609998,1cyclictest0-21swapper/604:22:496
20925609998,1cyclictest0-21swapper/604:16:496
20925609995,3cyclictest2324613-21cat05:26:436
20925529998,1cyclictest0-21swapper/506:55:275
20925529998,1cyclictest0-21swapper/506:30:325
20925529998,1cyclictest0-21swapper/504:06:465
20925529997,2cyclictest2415872-21kworker/u16:4+flush-8:007:42:015
20925529996,2cyclictest2374298-21kworker/u16:2+flush-8:006:46:575
20925529996,2cyclictest0-21swapper/507:31:495
20925439997,2cyclictest2374298-21kworker/u16:2+flush-8:007:51:224
20925439996,3cyclictest2415872-21kworker/u16:4+flush-8:007:36:594
20925609987,1cyclictest0-21swapper/605:07:076
20925609987,1cyclictest0-21swapper/604:35:356
20925609987,1cyclictest0-21swapper/604:04:036
20925609987,1cyclictest0-21swapper/603:50:396
20925609986,1cyclictest0-21swapper/608:41:476
20925609986,1cyclictest0-21swapper/604:56:516
20925609984,3cyclictest2517985-21ntp_states06:56:556
20925609984,3cyclictest2200868-21cat04:26:446
20925609984,3cyclictest2101638-21grep03:36:536
20925609983,4cyclictest2143712-21cut03:56:496
20925609983,4cyclictest2143712-21cut03:56:496
20925609980,8cyclictest0-21swapper/606:26:426
20925609980,7cyclictest0-21swapper/606:36:456
20925529987,1cyclictest0-21swapper/508:18:125
20925529987,1cyclictest0-21swapper/508:10:485
20925529984,3cyclictest2129184-21kthreadcore03:51:465
20925529983,4cyclictest2623009-21expr07:51:435
20925439987,1cyclictest0-21swapper/408:00:204
20925439986,1cyclictest0-21swapper/408:01:464
20925609976,1cyclictest0-21swapper/607:59:156
20925609975,1cyclictest0-21swapper/608:11:466
20925609975,1cyclictest0-21swapper/606:46:406
20925609975,1cyclictest0-21swapper/605:56:456
20925609974,2cyclictest2684522-21cstates08:21:406
20925609974,2cyclictest2245603-21kthreadcore04:46:486
20925609973,3cyclictest2541210-21sed07:11:456
20925609973,3cyclictest2128339-21expr03:51:406
20925609972,4cyclictest2771873-21cut09:01:496
20925609972,4cyclictest2427185-21cat06:16:396
20925609972,4cyclictest2427185-21cat06:16:396
20925609972,4cyclictest2354877-21cat05:41:436
20925609972,4cyclictest2346942-21cut05:36:466
20925529975,1cyclictest0-21swapper/506:21:435
20925529974,2cyclictest2740875-21kthreadcore08:46:495
20925529974,2cyclictest2725735-21cpuspeed_turbos08:41:415
20925529973,3cyclictest2770261-21cut09:01:485
20925529973,3cyclictest2468786-21tune2fs06:36:445
20925529972,4cyclictest2462975-21kthreadcore06:31:495
20925439974,2cyclictest2479040-21fschecks_time06:41:434
20925439973,3cyclictest2761683-21grep08:56:494
20925439973,3cyclictest2414902-21sed06:06:554
20925439972,4cyclictest2668512-21kthreadcore08:11:484
20925689960,5cyclictest2092808-21cat03:36:297
20925529962,3cyclictest2695111-21cut08:26:435
20925529950,4cyclictest2641481-21taskset07:56:565
20925529930,2cyclictest0-21swapper/503:46:295
20925439920,1cyclictest0-21swapper/403:46:294
20925439920,1cyclictest0-21swapper/403:36:294
20925609910,1cyclictest0-21swapper/603:46:296
20925609910,1cyclictest0-21swapper/603:36:296
20925529910,1cyclictest0-21swapper/503:36:295
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional