You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 11:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot4.osadl.org (updated Sat Nov 29, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2496042237175,20sleep50-21swapper/519:05:025
2517472235178,19sleep20-21swapper/219:07:562
2517902232175,18sleep70-21swapper/719:08:347
2517372218187,20sleep10-21swapper/119:07:471
2518392215186,18sleep00-21swapper/019:09:150
2517332215185,19sleep60-21swapper/619:07:446
2516612211177,21sleep40-21swapper/419:06:464
2516602211177,22sleep30-21swapper/319:06:453
25210499567,48cyclictest401416-21munin-plugin-st23:45:005
25208599343,30cyclictest285251-21sh20:10:000
252097993231,1cyclictest0-21swapper/321:50:003
252091993230,1cyclictest268816-21latency_hist19:40:012
252091993230,1cyclictest268816-21latency_hist19:40:012
25211099310,30cyclictest395972-21latency_hist23:35:007
25211099310,30cyclictest304112-21sh20:45:007
25211099310,30cyclictest293338-21turbostat.cron20:25:007
252110993028,1cyclictest0-21swapper/722:30:007
25210899300,2cyclictest3820478-21systemd-journal22:35:006
252104993028,1cyclictest0-21swapper/500:00:565
252104993028,1cyclictest0-21swapper/500:00:565
25210499300,2cyclictest336932-21apt-get21:45:565
252097993028,1cyclictest0-21swapper/321:55:543
252097993027,2cyclictest379813-21sh23:05:003
252091993027,2cyclictest1-21systemd19:55:002
25208599300,29cyclictest266605-21apt-get19:35:540
25208599300,29cyclictest266605-21apt-get19:35:540
252110992927,1cyclictest415167-21dump-pmu-power00:10:007
252110992927,1cyclictest415167-21dump-pmu-power00:10:007
252110992927,1cyclictest317573-21sh21:10:017
25211099290,28cyclictest0-21swapper/720:55:567
25211099290,28cyclictest0-21swapper/720:25:567
25211099290,28cyclictest0-21swapper/719:35:557
25211099290,28cyclictest0-21swapper/719:35:557
25211099290,0cyclictest0-21swapper/721:10:557
25210899290,28cyclictest0-21swapper/623:50:006
25210899290,28cyclictest0-21swapper/622:40:556
25210899290,28cyclictest0-21swapper/619:30:556
252104992927,1cyclictest272006-21apt-get19:45:565
25210499290,28cyclictest309970-21apt-get20:55:565
25210499290,28cyclictest0-21swapper/523:00:575
25210299290,1cyclictest358264-21latency_hist22:25:004
25210299290,1cyclictest339149-21munin-run21:50:004
252097992928,1cyclictest430086-21in:imuxsock19:35:003
252097992927,1cyclictest355554-21sh22:20:003
252097992927,1cyclictest331065-21munin-run21:35:003
25209799290,28cyclictest0-21swapper/323:40:563
25209799290,28cyclictest0-21swapper/323:05:563
25209799290,1cyclictest404843-21apt-get23:50:563
252091992926,2cyclictest260748-21dump-pmu-power19:25:002
25209199290,28cyclictest363644-21apt22:35:002
25209199290,1cyclictest0-21swapper/220:35:002
252088992927,1cyclictest283041-21apt-get20:05:551
25208899290,28cyclictest0-21swapper/120:15:571
25208899290,28cyclictest0-21swapper/100:20:571
25208899290,1cyclictest0-21swapper/120:55:571
252085992927,1cyclictest380299-21apt-get23:05:560
252085992927,1cyclictest314888-21latency_hist21:05:010
25208599290,28cyclictest0-21swapper/021:35:570
252110992828,0cyclictest0-21swapper/723:45:567
252108992827,1cyclictest0-21swapper/622:10:566
252108992826,1cyclictest0-21swapper/623:55:566
252108992826,1cyclictest0-21swapper/623:55:566
25210899280,28cyclictest0-21swapper/622:00:016
25210899280,28cyclictest0-21swapper/619:55:006
252104992826,1cyclictest315368-21apt-get21:05:555
25210499280,0cyclictest0-21swapper/519:40:535
252102992827,1cyclictest261225-21apt-get19:25:544
252102992827,1cyclictest0-21swapper/422:55:574
252102992827,1cyclictest0-21swapper/422:35:564
252102992826,1cyclictest0-21swapper/420:30:584
25210299280,27cyclictest0-21swapper/423:44:594
25210299280,27cyclictest0-21swapper/420:35:584
25210299280,27cyclictest0-21swapper/400:35:564
252097992827,1cyclictest0-21swapper/322:00:553
252091992827,0cyclictest0-21swapper/200:35:552
25209199280,28cyclictest0-21swapper/219:55:012
25209199280,0cyclictest0-21swapper/220:00:542
252088992827,1cyclictest385206-21munin-run23:15:001
252088992827,1cyclictest0-21swapper/121:35:581
252088992827,1cyclictest0-21swapper/119:15:001
25208899280,28cyclictest0-21swapper/120:00:541
25208899280,0cyclictest0-21swapper/123:35:571
25208899280,0cyclictest0-21swapper/100:15:561
252085992828,0cyclictest0-21swapper/023:40:560
252085992827,1cyclictest0-21swapper/019:30:550
252085992827,1cyclictest0-21swapper/019:25:540
25208599280,27cyclictest0-21swapper/020:45:570
252110992727,0cyclictest0-21swapper/722:20:567
252110992727,0cyclictest0-21swapper/722:10:567
252110992727,0cyclictest0-21swapper/721:00:557
252110992726,1cyclictest0-21swapper/700:15:567
25211099270,0cyclictest0-21swapper/719:45:567
252108992727,0cyclictest0-21swapper/621:15:566
252108992726,1cyclictest0-21swapper/600:10:566
252108992725,2cyclictest317590-21munin-run21:10:016
25210899270,0cyclictest0-21swapper/623:50:016
25210899270,0cyclictest0-21swapper/620:40:566
25210899270,0cyclictest0-21swapper/619:20:566
25210899270,0cyclictest0-21swapper/619:15:556
25210899270,0cyclictest0-21swapper/600:05:556
25210899270,0cyclictest0-21swapper/600:05:556
252104992727,0cyclictest0-21swapper/522:40:555
252104992727,0cyclictest0-21swapper/520:15:575
252104992725,1cyclictest420539-21sh00:20:005
252104992725,1cyclictest0-21swapper/522:55:575
252104992725,1cyclictest0-21swapper/521:15:555
25210499270,0cyclictest0-21swapper/522:15:565
252102992727,0cyclictest0-21swapper/423:15:554
252102992727,0cyclictest0-21swapper/420:30:004
252102992727,0cyclictest0-21swapper/419:15:554
252102992725,1cyclictest0-21swapper/422:30:554
252102992725,1cyclictest0-21swapper/422:05:554
252097992727,0cyclictest0-21swapper/322:25:553
252097992727,0cyclictest0-21swapper/320:25:563
252097992727,0cyclictest0-21swapper/300:35:553
252097992727,0cyclictest0-21swapper/300:30:563
252097992726,1cyclictest0-21swapper/322:15:003
252097992725,1cyclictest312174-21sh21:00:013
252091992727,0cyclictest0-21swapper/221:55:552
252091992727,0cyclictest0-21swapper/221:15:562
252091992727,0cyclictest0-21swapper/219:50:002
252091992727,0cyclictest0-21swapper/219:15:552
252091992726,1cyclictest0-21swapper/223:50:012
252091992726,1cyclictest0-21swapper/222:00:012
252088992727,0cyclictest0-21swapper/121:25:551
252088992726,1cyclictest377610-21apt-get23:00:571
25208899270,0cyclictest0-21swapper/121:20:551
252085992727,0cyclictest0-21swapper/023:30:550
252085992727,0cyclictest0-21swapper/022:10:000
252085992725,1cyclictest358746-21apt-get22:25:540
252085992722,4cyclictest0-21swapper/022:00:000
252110992626,0cyclictest0-21swapper/723:40:567
252110992626,0cyclictest0-21swapper/721:35:577
252110992623,2cyclictest3820478-21systemd-journal21:45:007
252108992625,1cyclictest0-21swapper/620:55:566
252104992624,1cyclictest3820478-21systemd-journal22:50:005
252102992626,0cyclictest0-21swapper/420:45:574
252102992625,1cyclictest0-21swapper/421:10:554
252102992625,1cyclictest0-21swapper/419:50:574
252102992624,1cyclictest933-21mta-sts-daemon22:00:004
252102992624,1cyclictest388387-21apt-get23:20:564
252097992626,0cyclictest0-21swapper/320:35:583
252097992626,0cyclictest0-21swapper/320:00:553
252097992626,0cyclictest0-21swapper/319:35:553
252097992626,0cyclictest0-21swapper/319:35:553
252097992624,1cyclictest255826-21apt-get19:15:553
252091992626,0cyclictest0-21swapper/223:50:002
252091992626,0cyclictest0-21swapper/222:40:552
252091992626,0cyclictest0-21swapper/220:40:562
252091992626,0cyclictest0-21swapper/219:30:552
252091992626,0cyclictest0-21swapper/200:05:552
252091992626,0cyclictest0-21swapper/200:05:552
252091992625,1cyclictest0-21swapper/223:25:562
252091992625,1cyclictest0-21swapper/222:10:562
252091992625,1cyclictest0-21swapper/220:55:572
252088992626,0cyclictest0-21swapper/119:20:561
252088992625,1cyclictest407081-21awk23:55:001
252088992625,1cyclictest0-21swapper/122:05:551
252085992626,0cyclictest0-21swapper/022:15:550
252085992626,0cyclictest0-21swapper/020:55:570
252085992626,0cyclictest0-21swapper/020:50:560
252085992626,0cyclictest0-21swapper/020:10:560
252085992626,0cyclictest0-21swapper/000:00:000
252085992626,0cyclictest0-21swapper/000:00:000
252085992624,1cyclictest393286-21dump-pmu-power23:30:000
252110992525,0cyclictest0-21swapper/722:30:557
252108992525,0cyclictest0-21swapper/622:05:556
252108992525,0cyclictest0-21swapper/619:35:556
252108992525,0cyclictest0-21swapper/619:35:556
252108992524,1cyclictest0-21swapper/622:45:556
252102992525,0cyclictest0-21swapper/422:40:564
252102992525,0cyclictest0-21swapper/421:15:564
252091992525,0cyclictest0-21swapper/221:25:552
252091992525,0cyclictest0-21swapper/221:00:562
252088992524,1cyclictest291134-21apt-get20:20:581
252088992523,1cyclictest390589-21sh23:25:001
252085992525,0cyclictest0-21swapper/020:20:580
252085992525,0cyclictest0-21swapper/019:20:000
252085992523,1cyclictest0-21swapper/021:20:010
252085992522,2cyclictest0-21swapper/021:30:550
252108992424,0cyclictest0-21swapper/621:25:556
252108992424,0cyclictest0-21swapper/620:45:576
252108992424,0cyclictest0-21swapper/619:55:016
252108992424,0cyclictest0-21swapper/600:00:566
252108992424,0cyclictest0-21swapper/600:00:566
252108992423,1cyclictest0-21swapper/623:10:006
252104992423,0cyclictest0-21swapper/523:05:565
252104992422,1cyclictest417847-21sh00:15:005
252102992424,0cyclictest0-21swapper/422:10:574
252102992424,0cyclictest0-21swapper/421:05:014
252097992424,0cyclictest0-21swapper/321:50:563
252097992423,1cyclictest0-21swapper/323:55:553
252097992423,1cyclictest0-21swapper/323:55:553
252091992424,0cyclictest0-21swapper/223:55:562
252091992424,0cyclictest0-21swapper/223:55:562
252088992424,0cyclictest0-21swapper/121:30:561
252088992422,1cyclictest431320-21cron00:40:001
252088992420,3cyclictest347487-21latency_hist22:05:001
252085992424,0cyclictest0-21swapper/021:45:560
252085992423,1cyclictest252462-21apt-get19:10:580
252085992422,1cyclictest369036-21sh22:45:000
252110992323,0cyclictest0-21swapper/720:10:007
252110992323,0cyclictest0-21swapper/719:50:577
252108992323,0cyclictest0-21swapper/619:40:536
252104992323,0cyclictest0-21swapper/523:45:565
252104992323,0cyclictest0-21swapper/521:40:565
252104992323,0cyclictest0-21swapper/520:35:575
252102992322,1cyclictest0-21swapper/419:55:554
252102992322,1cyclictest0-21swapper/400:35:004
252102992321,1cyclictest280352-21apt-get20:00:544
252097992323,0cyclictest0-21swapper/323:20:563
252097992323,0cyclictest0-21swapper/322:20:563
252097992323,0cyclictest0-21swapper/321:15:563
252097992322,1cyclictest0-21swapper/319:40:533
252097992321,1cyclictest425943-21dump-pmu-power00:30:003
252091992323,0cyclictest0-21swapper/200:10:562
252091992322,1cyclictest0-21swapper/222:05:552
252091992322,1cyclictest0-21swapper/221:10:012
252088992323,0cyclictest0-21swapper/122:45:551
252088992323,0cyclictest0-21swapper/122:40:001
252088992323,0cyclictest0-21swapper/122:20:561
252088992322,1cyclictest0-21swapper/123:40:561
252085992323,0cyclictest0-21swapper/020:40:000
252085992320,2cyclictest374900-21apt-get22:55:570
252110992222,0cyclictest0-21swapper/722:15:557
252110992222,0cyclictest0-21swapper/721:55:547
252108992221,1cyclictest0-21swapper/622:35:576
252108992221,1cyclictest0-21swapper/620:05:556
252108992220,1cyclictest0-21swapper/621:35:586
252104992222,0cyclictest0-21swapper/521:35:575
252104992222,0cyclictest0-21swapper/500:25:575
252102992221,1cyclictest0-21swapper/421:35:574
252097992222,0cyclictest0-21swapper/321:20:553
252097992222,0cyclictest0-21swapper/320:20:583
252097992222,0cyclictest0-21swapper/300:00:563
252097992222,0cyclictest0-21swapper/300:00:563
252097992220,1cyclictest258540-21apt-get19:20:563
252091992221,1cyclictest0-21swapper/200:00:562
252091992221,1cyclictest0-21swapper/200:00:562
252088992222,0cyclictest0-21swapper/120:10:561
252085992222,0cyclictest0-21swapper/019:25:000
252085992222,0cyclictest0-21swapper/000:20:570
252110992121,0cyclictest0-21swapper/723:25:007
252110992121,0cyclictest0-21swapper/721:45:567
252110992121,0cyclictest0-21swapper/719:40:537
252110992121,0cyclictest0-21swapper/700:35:567
252108992121,0cyclictest0-21swapper/619:25:556
252104992121,0cyclictest0-21swapper/520:30:585
252104992121,0cyclictest0-21swapper/500:30:565
252104992119,1cyclictest0-21swapper/500:10:005
252104992119,1cyclictest0-21swapper/500:10:005
252102992121,0cyclictest0-21swapper/420:10:564
252097992121,0cyclictest0-21swapper/323:45:563
252097992121,0cyclictest0-21swapper/320:15:573
252097992121,0cyclictest0-21swapper/300:20:563
252091992121,0cyclictest0-21swapper/222:45:552
252091992120,1cyclictest382538-21awk23:10:002
252091992120,1cyclictest0-21swapper/220:45:572
252088992121,0cyclictest0-21swapper/122:50:561
252088992121,0cyclictest0-21swapper/119:25:551
252110992020,0cyclictest0-21swapper/720:10:567
252110992018,1cyclictest377610-21apt-get23:00:577
252108992018,1cyclictest0-21swapper/623:20:576
252102992020,0cyclictest0-21swapper/423:25:564
252102992018,1cyclictest301906-21apt-get20:40:554
252097992020,0cyclictest0-21swapper/321:10:553
252097992020,0cyclictest0-21swapper/319:55:553
252091992020,0cyclictest0-21swapper/219:40:532
252088992019,1cyclictest0-21swapper/123:30:551
252088992018,2cyclictest320310-21awk21:15:011
252085992019,1cyclictest0-21swapper/020:30:580
252085992017,3cyclictest0-21swapper/022:50:560
25208599200,15cyclictest0-21swapper/023:20:590
252110991919,0cyclictest0-21swapper/723:35:577
252110991918,1cyclictest0-21swapper/720:55:017
252108991917,1cyclictest309484-21sh20:55:016
252108991917,1cyclictest1-21systemd23:20:006
252108991916,2cyclictest374417-21idleruntime-cro22:55:006
252102991919,0cyclictest0-21swapper/423:30:564
252102991919,0cyclictest0-21swapper/400:00:564
252102991919,0cyclictest0-21swapper/400:00:564
25209799190,18cyclictest0-21swapper/323:30:003
252091991918,1cyclictest0-21swapper/220:05:552
252088991918,1cyclictest0-21swapper/119:45:561
252085991919,0cyclictest0-21swapper/000:35:560
252085991918,1cyclictest0-21swapper/022:20:560
252108991818,0cyclictest0-21swapper/621:55:546
252104991818,0cyclictest0-21swapper/500:20:575
252104991816,1cyclictest0-21swapper/520:40:555
252102991818,0cyclictest0-21swapper/423:50:004
252091991818,0cyclictest0-21swapper/222:35:572
252091991818,0cyclictest0-21swapper/219:25:552
252091991817,1cyclictest0-21swapper/221:35:582
252088991818,0cyclictest0-21swapper/100:25:571
252085991816,1cyclictest0-21swapper/020:15:560
252110991715,1cyclictest0-21swapper/720:15:577
252108991717,0cyclictest0-21swapper/620:15:576
25210899170,16cyclictest3256082chrt21:24:136
252104991717,0cyclictest0-21swapper/523:50:565
252104991717,0cyclictest0-21swapper/522:25:555
252104991717,0cyclictest0-21swapper/522:20:565
252104991717,0cyclictest0-21swapper/519:55:555
252104991716,1cyclictest0-21swapper/521:00:555
25210499170,16cyclictest256858-21memory19:15:595
252102991717,0cyclictest0-21swapper/423:55:554
252102991717,0cyclictest0-21swapper/423:55:554
252102991716,1cyclictest0-21swapper/422:45:554
252097991717,0cyclictest0-21swapper/319:30:003
252088991717,0cyclictest0-21swapper/120:50:561
252088991717,0cyclictest0-21swapper/120:40:561
252088991717,0cyclictest0-21swapper/100:05:551
252088991717,0cyclictest0-21swapper/100:05:551
25208599170,17cyclictest0-21swapper/000:05:000
25208599170,17cyclictest0-21swapper/000:05:000
25208599170,16cyclictest0-21swapper/021:26:480
25208599170,14cyclictest0-21swapper/020:03:040
252110991615,1cyclictest366834-21apt-get22:40:557
25211099160,16cyclictest278676-21memory19:55:597
25211099160,15cyclictest0-21swapper/722:48:237
25211099160,15cyclictest0-21swapper/720:35:327
25211099160,15cyclictest0-21swapper/720:34:157
25211099160,15cyclictest0-21swapper/719:10:537
252108991616,0cyclictest0-21swapper/621:55:006
252108991614,1cyclictest356059-21apt-get22:20:566
25210899160,15cyclictest0-21swapper/600:38:266
25210499160,15cyclictest0-21swapper/522:07:445
25210499160,15cyclictest0-21swapper/520:46:555
25210499160,15cyclictest0-21swapper/519:21:025
252102991616,0cyclictest0-21swapper/422:00:564
252102991616,0cyclictest0-21swapper/419:20:564
252102991614,1cyclictest422951-21users00:21:064
252102991613,2cyclictest381019-21perl23:05:584
25210299160,15cyclictest0-21swapper/400:06:034
25210299160,15cyclictest0-21swapper/400:06:034
252097991615,1cyclictest0-21swapper/322:55:003
25209799160,15cyclictest0-21swapper/323:10:063
25209799160,15cyclictest0-21swapper/322:41:463
25209799160,15cyclictest0-21swapper/322:34:393
25209799160,15cyclictest0-21swapper/321:37:023
25209799160,15cyclictest0-21swapper/320:07:223
25209799160,15cyclictest0-21swapper/319:45:103
252091991614,1cyclictest387886-21idleruntime-cro23:20:002
252091991614,1cyclictest0-21swapper/223:30:562
252088991614,1cyclictest0-21swapper/121:05:551
252085991616,0cyclictest0-21swapper/023:00:570
252085991614,0cyclictest0-21swapper/023:50:470
25208599160,15cyclictest0-21swapper/023:46:010
25208599160,15cyclictest0-21swapper/022:30:590
25208599160,14cyclictest0-21swapper/020:29:540
252110991515,0cyclictest0-21swapper/700:13:087
252110991515,0cyclictest0-21swapper/700:00:567
252110991515,0cyclictest0-21swapper/700:00:567
25211099150,1cyclictest259944-21grep19:21:037
25211099150,1cyclictest0-21swapper/721:16:037
25211099150,15cyclictest0-21swapper/722:07:267
25211099150,15cyclictest0-21swapper/719:27:557
25211099150,14cyclictest0-21swapper/722:00:157
25211099150,14cyclictest0-21swapper/720:45:397
25211099150,0cyclictest0-21swapper/722:53:407
25211099150,0cyclictest0-21swapper/721:22:297
25211099150,0cyclictest0-21swapper/719:15:557
252108991514,1cyclictest0-21swapper/620:02:546
25210899150,1cyclictest0-21swapper/623:30:566
25210899150,1cyclictest0-21swapper/623:10:586
25210899150,15cyclictest0-21swapper/620:32:516
25210899150,14cyclictest0-21swapper/623:41:116
25210899150,14cyclictest0-21swapper/623:39:296
25210899150,14cyclictest0-21swapper/623:02:256
25210899150,14cyclictest0-21swapper/622:55:106
25210899150,14cyclictest0-21swapper/622:15:386
25210899150,14cyclictest0-21swapper/621:45:376
25210899150,14cyclictest0-21swapper/621:31:016
25210899150,14cyclictest0-21swapper/621:10:586
25210899150,14cyclictest0-21swapper/621:00:086
25210899150,14cyclictest0-21swapper/620:37:226
25210899150,14cyclictest0-21swapper/620:26:346
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional