You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-09 - 01:06

x86 Intel Core i7-3770K @3500 MHz, Linux 5.15.49-rt47 (Profile)

Latency plot of system in rack #0, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a4-7 -p99 -i200 -h400 -t4 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot2.osadl.org (updated Mon Aug 08, 2022 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252628525521,29sleep70-21swapper/707:06:117
3831889530,30rtkit-daemon0-21swapper/607:05:566
252662124133,5sleep40-21swapper/407:09:364
252646723122,6sleep50-21swapper/507:08:015
252703399160,15cyclictest1016-21snmpd11:31:557
2527033991412,1cyclictest2831991-21ntpq09:35:277
2527002991414,0cyclictest0-21swapper/411:12:564
2527024991313,0cyclictest0-21swapper/611:38:326
2527024991313,0cyclictest0-21swapper/611:32:136
2527024991313,0cyclictest0-21swapper/610:15:466
252702499130,13cyclictest0-21swapper/609:59:506
252702499130,0cyclictest0-21swapper/609:50:236
2527012991313,0cyclictest0-21swapper/508:27:385
252701299130,13cyclictest0-21swapper/509:50:055
252701299130,13cyclictest0-21swapper/508:32:225
252701299130,0cyclictest0-21swapper/512:21:365
252701299130,0cyclictest0-21swapper/511:31:375
252701299130,0cyclictest0-21swapper/511:25:195
2527002991313,0cyclictest0-21swapper/412:17:434
2527002991313,0cyclictest0-21swapper/411:29:294
252700299130,13cyclictest0-21swapper/412:03:324
252700299130,13cyclictest0-21swapper/411:56:174
252700299130,13cyclictest0-21swapper/411:19:334
252700299130,0cyclictest0-21swapper/410:19:294
252700299130,0cyclictest0-21swapper/409:30:084
2527033991212,0cyclictest0-21swapper/711:27:147
2527033991212,0cyclictest0-21swapper/709:33:057
252703399120,11cyclictest3220562-21cat12:20:167
252703399120,11cyclictest2725346-21cat08:50:147
252703399120,11cyclictest1016-21snmpd11:38:467
2527024991212,0cyclictest0-21swapper/612:22:316
2527024991212,0cyclictest0-21swapper/611:51:276
2527012991212,0cyclictest0-21swapper/512:18:175
2527012991212,0cyclictest0-21swapper/509:57:115
2527002991212,0cyclictest0-21swapper/412:24:404
2527002991212,0cyclictest0-21swapper/411:32:474
2527033991111,0cyclictest0-21swapper/712:03:207
2527033991111,0cyclictest0-21swapper/711:08:377
2527033991111,0cyclictest0-21swapper/710:19:267
2527033991111,0cyclictest0-21swapper/709:54:037
2527033991111,0cyclictest0-21swapper/709:11:487
2527033991111,0cyclictest0-21swapper/708:20:247
252703399110,2cyclictest2880630-21smart_sda09:55:327
252703399110,10cyclictest3211954-21kthreadcore12:15:227
252703399110,10cyclictest1016-21snmpd08:45:577
2527024991111,0cyclictest0-21swapper/612:15:546
2527024991111,0cyclictest0-21swapper/611:07:196
2527024991111,0cyclictest0-21swapper/609:34:086
2527012991111,0cyclictest0-21swapper/508:44:115
2527002991111,0cyclictest0-21swapper/409:37:144
252700299110,11cyclictest0-21swapper/408:19:404
2527033991010,0cyclictest0-21swapper/711:59:427
2527033991010,0cyclictest0-21swapper/708:43:447
252703399100,9cyclictest3066417-21cat11:15:167
252703399100,7cyclictest38314-10wireplumber07:25:247
252703399100,3cyclictest2760828-21kthreadcore09:05:237
252703399100,1cyclictest38314-10wireplumber08:30:207
2527024991010,0cyclictest0-21swapper/611:16:186
2527024991010,0cyclictest0-21swapper/609:36:306
2527024991010,0cyclictest0-21swapper/608:45:546
2527024991010,0cyclictest0-21swapper/608:23:406
2527012991010,0cyclictest0-21swapper/511:57:475
2527012991010,0cyclictest0-21swapper/511:54:105
2527012991010,0cyclictest0-21swapper/511:07:015
2527012991010,0cyclictest0-21swapper/509:07:505
2527012991010,0cyclictest0-21swapper/508:48:555
2527002991010,0cyclictest0-21swapper/411:52:394
2527002991010,0cyclictest0-21swapper/411:36:064
2527002991010,0cyclictest0-21swapper/408:27:044
25270339990,3cyclictest2659314-21kthreadcore08:15:167
25270339990,2cyclictest3150053-21wc11:50:137
25270339990,2cyclictest2676685-21df_inode08:25:117
25270339990,1cyclictest3053745-21cpuspeed_turbos11:10:137
25270249999,0cyclictest0-21swapper/611:58:436
25270249999,0cyclictest0-21swapper/608:53:186
25270249999,0cyclictest0-21swapper/607:29:076
25270249994,5cyclictest0-21swapper/609:12:056
25270249990,9cyclictest0-21swapper/608:30:316
25270129999,0cyclictest0-21swapper/512:04:445
25270129999,0cyclictest0-21swapper/511:37:555
25270129999,0cyclictest0-21swapper/511:19:005
25270129999,0cyclictest0-21swapper/509:33:515
25270029999,0cyclictest0-21swapper/409:06:104
25270029999,0cyclictest0-21swapper/408:48:594
25270339980,0cyclictest0-21swapper/707:19:587
25270249988,0cyclictest0-21swapper/611:12:596
25270249988,0cyclictest0-21swapper/609:08:086
25270129988,0cyclictest0-21swapper/509:36:135
25270129988,0cyclictest0-21swapper/508:53:395
25270129988,0cyclictest0-21swapper/507:28:315
25270129984,3cyclictest3016807-21grep10:54:585
25270029982,3cyclictest2944307-21munin-run10:24:584
25270339972,3cyclictest0-21swapper/709:44:577
25270339970,6cyclictest950-21containerd11:24:577
25270339970,6cyclictest1-21systemd10:29:587
25270339970,1cyclictest68-21ksoftirqd/707:44:587
25270339970,1cyclictest2908071-21exim410:09:577
25270339970,1cyclictest2908071-21exim410:09:577
25270339970,0cyclictest1-21systemd09:19:577
25270339970,0cyclictest0-21swapper/711:44:577
25270339970,0cyclictest0-21swapper/710:14:587
25270249977,0cyclictest0-21swapper/612:02:206
25270249977,0cyclictest0-21swapper/611:29:146
25270249974,2cyclictest3195839-21munin-run12:09:576
25270129977,0cyclictest0-21swapper/510:15:285
25270029977,0cyclictest0-21swapper/409:56:284
25270029971,3cyclictest2847540-21munin-run09:44:574
25270339961,4cyclictest0-21swapper/709:49:587
25270339961,2cyclictest556-21systemd-journal10:49:587
25270339960,5cyclictest1-21systemd09:24:577
25270339960,5cyclictest0-21swapper/707:39:587
25270339960,5cyclictest0-21swapper/707:14:587
25270339960,3cyclictest556-21systemd-journal12:34:577
25270339960,3cyclictest556-21systemd-journal12:09:577
25270339960,3cyclictest556-21systemd-journal10:59:587
25270339960,3cyclictest556-21systemd-journal10:54:587
25270339960,3cyclictest556-21systemd-journal10:44:587
25270339960,3cyclictest556-21systemd-journal10:24:587
25270339960,3cyclictest556-21systemd-journal10:04:577
25270339960,3cyclictest556-21systemd-journal08:59:577
25270339960,3cyclictest556-21systemd-journal08:04:587
25270339960,3cyclictest556-21systemd-journal07:59:587
25270339960,0cyclictest0-21swapper/711:49:577
25270339960,0cyclictest0-21swapper/708:39:587
25270249960,5cyclictest0-21swapper/610:34:586
25270249960,4cyclictest2536943-21cat07:14:586
25270249960,3cyclictest0-21swapper/607:39:586
25270249960,2cyclictest918-21lldpd08:25:166
25270249960,2cyclictest2559453-21latency_hist07:24:586
25270129966,0cyclictest0-21swapper/509:14:565
25270129966,0cyclictest0-21swapper/508:16:085
25270129961,5cyclictest54-21ksoftirqd/508:21:395
25270029966,0cyclictest0-21swapper/409:13:164
25270029966,0cyclictest0-21swapper/408:32:074
25270029962,3cyclictest2675659-21latency_hist08:24:574
25270029962,2cyclictest794-21systemd-logind07:54:584
25270029962,2cyclictest38246-21llvmpipe-608:40:564
25270029960,3cyclictest2787005-21basename09:19:574
25270339950,4cyclictest3242391-21ssh12:29:577
25270339950,4cyclictest0-21swapper/710:39:587
25270339950,3cyclictest3265384-21ssh12:39:577
25270339950,3cyclictest2753443-21latency_hist09:04:587
25270339950,2cyclictest556-21systemd-journal10:34:587
25270339950,2cyclictest556-21systemd-journal08:14:587
25270339950,2cyclictest556-21systemd-journal08:09:587
25270339950,2cyclictest556-21systemd-journal07:54:587
25270339950,2cyclictest556-21systemd-journal07:34:587
25270339950,0cyclictest0-21swapper/712:14:577
25270249952,2cyclictest3207824-21cut12:14:576
25270249952,2cyclictest3137475-21exim411:44:576
25270249952,2cyclictest2980914-21date10:39:586
25270249952,2cyclictest2646858-21date08:09:586
25270249950,5cyclictest0-21swapper/612:34:576
25270249950,5cyclictest0-21swapper/612:29:576
25270249950,5cyclictest0-21swapper/611:24:576
25270249950,5cyclictest0-21swapper/610:59:586
25270249950,5cyclictest0-21swapper/610:54:586
25270249950,5cyclictest0-21swapper/610:49:576
25270249950,5cyclictest0-21swapper/610:44:576
25270249950,5cyclictest0-21swapper/610:29:576
25270249950,5cyclictest0-21swapper/610:24:576
25270249950,5cyclictest0-21swapper/610:09:586
25270249950,5cyclictest0-21swapper/610:09:586
25270249950,5cyclictest0-21swapper/610:04:576
25270249950,5cyclictest0-21swapper/609:49:586
25270249950,5cyclictest0-21swapper/609:44:576
25270249950,5cyclictest0-21swapper/609:24:586
25270249950,5cyclictest0-21swapper/609:19:576
25270249950,5cyclictest0-21swapper/608:39:586
25270249950,5cyclictest0-21swapper/608:14:596
25270249950,5cyclictest0-21swapper/608:14:586
25270249950,5cyclictest0-21swapper/608:04:586
25270249950,5cyclictest0-21swapper/607:59:586
25270249950,5cyclictest0-21swapper/607:54:586
25270249950,5cyclictest0-21swapper/607:49:586
25270249950,5cyclictest0-21swapper/607:34:586
25270249950,3cyclictest2920302-21latency_hist10:14:586
25270249950,2cyclictest2743344-21latency_hist08:59:576
25270129954,1cyclictest0-21swapper/512:39:575
25270129953,1cyclictest2550077-21latency_hist07:19:585
25270129952,2cyclictest2787252-21latency_hist09:19:585
25270129951,3cyclictest2799294-21date09:24:585
25270129950,5cyclictest0-21swapper/512:29:575
25270129950,5cyclictest0-21swapper/511:09:585
25270129950,5cyclictest0-21swapper/511:04:575
25270129950,5cyclictest0-21swapper/510:59:585
25270129950,5cyclictest0-21swapper/510:29:575
25270129950,5cyclictest0-21swapper/510:24:585
25270129950,5cyclictest0-21swapper/507:59:585
25270129950,5cyclictest0-21swapper/507:54:585
25270129950,5cyclictest0-21swapper/507:44:585
25270129950,5cyclictest0-21swapper/507:39:585
25270029953,1cyclictest2896114-21latency_hist10:04:584
25270029953,1cyclictest0-21swapper/411:05:444
25270029952,2cyclictest2870269-21ssh09:52:544
25270029952,2cyclictest2859919-21cat09:49:584
25270029952,2cyclictest2811549-21date09:29:584
25270029952,2cyclictest2627403-21sed07:59:584
25270029951,3cyclictest2733467-21cut08:54:574
25270029950,2cyclictest2908060-21perf10:09:574
25270029950,2cyclictest2908060-21perf10:09:574
25270339940,3cyclictest3041156-21latency_hist11:04:577
25270339940,1cyclictest556-21systemd-journal07:49:587
25270249942,1cyclictest2753460-21exim409:04:586
25270249941,1cyclictest0-21swapper/608:39:576
25270249940,3cyclictest3265947-21cat12:39:576
25270249940,3cyclictest3149389-21cat11:49:576
25270249940,3cyclictest2811546-21cut09:29:586
25270249940,3cyclictest2598025-21cron07:44:586
25270249940,3cyclictest2550077-21latency_hist07:19:586
25270129942,1cyclictest2608386-21date07:49:585
25270129942,1cyclictest2579040-21exim407:34:585
25270129940,3cyclictest3254098-21exim412:34:575
25270129940,3cyclictest3207150-21latency_hist12:14:575
25270129940,3cyclictest3195829-21sh12:09:575
25270129940,3cyclictest3149423-21latency_hist11:49:575
25270129940,3cyclictest3137458-21turbostat11:44:575
25270129940,3cyclictest3089308-21cut11:24:575
25270129940,3cyclictest3004740-21crond10:49:585
25270129940,3cyclictest2992669-21crond10:44:585
25270129940,3cyclictest2980806-21cat10:39:585
25270129940,3cyclictest2968619-21latency_hist10:34:585
25270129940,3cyclictest2920286-21latency_hist10:14:585
25270129940,3cyclictest2908048-21idleruntime-cro10:09:575
25270129940,3cyclictest2908048-21idleruntime-cro10:09:575
25270129940,3cyclictest2895958-21sh10:04:575
25270129940,3cyclictest2859755-21sed09:49:585
25270129940,3cyclictest2847535-21idleruntime-cro09:44:575
25270129940,3cyclictest2810932-21ssh09:29:585
25270129940,3cyclictest2753420-21sed09:04:585
25270129940,3cyclictest2743104-21cron08:59:575
25270129940,3cyclictest2705029-21cat08:39:575
25270129940,3cyclictest2656388-21perf08:14:585
25270129940,3cyclictest2646643-21munin-run08:09:585
25270129940,3cyclictest2636986-21exim408:04:585
25270129940,3cyclictest2559909-21cat07:24:585
25270129940,3cyclictest2536869-21ls07:14:585
25270029942,1cyclictest2968560-21latency_hist10:34:584
25270029941,2cyclictest3207706-21cat12:14:574
25270029940,3cyclictest3266017-21sed12:39:574
25270029940,3cyclictest3254046-21munin-run12:34:574
25270029940,3cyclictest3242445-21basename12:29:574
25270029940,3cyclictest3195827-21crond12:09:574
25270029940,3cyclictest3149356-21sendmail11:49:574
25270029940,3cyclictest3137446-21date11:44:574
25270029940,3cyclictest3089172-21systemd-run11:24:574
25270029940,3cyclictest3041146-21sendmail11:04:574
25270029940,3cyclictest3028698-21/usr/sbin/munin10:59:584
25270029940,3cyclictest3016808-21munin-run10:54:584
25270029940,3cyclictest3004743-21crond10:49:584
25270029940,3cyclictest2992664-21crond10:44:584
25270029940,3cyclictest2980769-21cat10:39:584
25270029940,3cyclictest2956405-21sendmail10:29:574
25270029940,3cyclictest2920370-21date10:14:584
25270029940,3cyclictest2798988-21/usr/sbin/munin09:24:574
25270029940,3cyclictest2743529-21sed08:59:574
25270029940,3cyclictest2704990-21latency_hist08:39:574
25270029940,3cyclictest2656363-21sendmail08:14:574
25270029940,3cyclictest2637004-21dump-pmu-power08:04:584
25270029940,3cyclictest2608218-21date07:49:584
25270029940,3cyclictest2598382-21perf07:44:584
25270029940,3cyclictest2588714-21sh07:39:584
25270029940,3cyclictest2579050-21idleruntime-cro07:34:584
25270029940,3cyclictest2560126-21chrt07:25:054
25270029940,3cyclictest2559883-21sed07:24:584
25270029940,3cyclictest2550077-21crond07:19:584
25270029940,3cyclictest2536872-21ls07:14:584
25270029940,2cyclictest794-21systemd-logind08:09:584
25270339930,2cyclictest1-21systemd07:24:587
25270249930,2cyclictest3041161-21cut11:04:576
25270029930,2cyclictest2752815-21latency_hist09:04:584
25270339920,1cyclictest0-21swapper/709:29:587
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional