You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-05 - 00:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sun May 04, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16691992722,5cyclictest609-21polkitd12:05:001
16691992722,5cyclictest609-21polkitd12:05:001
16691992623,3cyclictest0-21swapper/107:54:481
1669199250,20cyclictest0-21swapper/111:59:481
1669799240,23cyclictest9975-21latency_hist12:24:593
1669399240,23cyclictest26540-21apt-get10:49:492
1669199240,23cyclictest3722-21apt-get11:09:481
1668799240,23cyclictest579-21cron08:50:000
1669799231,21cyclictest10643-21grep10:15:003
1669799230,22cyclictest3078-21sh11:05:003
1669799230,22cyclictest29953-21tr12:00:003
16693992323,0cyclictest27157-21apt-get07:34:492
1669399230,2cyclictest9986-21perf12:25:002
1669399230,22cyclictest28222-21cron08:40:002
1669399230,0cyclictest31287-21apt-get08:49:482
16691992321,1cyclictest0-21swapper/109:35:001
16691992320,2cyclictest0-21swapper/107:20:001
16691992316,6cyclictest12714-21cron11:25:011
1669199230,2cyclictest30411-21sh10:55:001
1669199230,22cyclictest9951-21cron12:25:001
1669199230,22cyclictest18710-21perf10:30:011
1669199230,1cyclictest18561-21apt-get11:39:481
16687992321,1cyclictest31439-21munin-run07:40:000
16687992321,1cyclictest0-21swapper/010:15:000
1668799230,22cyclictest5793-21apt-get12:19:490
16697992220,1cyclictest0-21swapper/310:10:003
16697992220,1cyclictest0-21swapper/308:24:593
1669799220,21cyclictest23467-21idleruntime-cro09:34:593
1669799220,21cyclictest21100-21dump-pmu-power10:35:003
1669799220,21cyclictest0-21swapper/310:04:513
1669799220,21cyclictest0-21swapper/309:55:003
1669799220,21cyclictest0-21swapper/307:34:493
1669799220,0cyclictest23373-21apt-get11:49:493
16693992220,2cyclictest28681-21turbostat.cron09:45:012
16693992220,1cyclictest32317-21cron12:05:002
16693992220,1cyclictest32317-21cron12:04:592
16693992220,1cyclictest18699-21turbostat10:30:002
16693992220,1cyclictest13068-21sh09:14:592
16693992220,1cyclictest11067-21perf08:05:002
1669399220,21cyclictest8565-21apt-get11:19:502
1669399220,21cyclictest22717-21cron11:45:002
1669399220,21cyclictest21733-21apt-get10:39:482
1669399220,21cyclictest15868-21apt-get09:20:022
1669399220,21cyclictest10975-21apt-get11:24:482
1669399220,21cyclictest10682-21sendmail09:09:592
1669399220,21cyclictest0-21swapper/208:14:502
1669399220,1cyclictest0-21swapper/211:30:012
16691992221,1cyclictest6893-21apt-get07:59:501
16691992220,1cyclictest3102-21perf11:04:591
16691992220,1cyclictest24130-21turbostat07:25:001
16691992220,1cyclictest10341-21perf11:20:001
1669199220,21cyclictest4116-21apt-get08:59:481
1669199220,21cyclictest2050-21apt-get07:49:501
1669199220,21cyclictest16928-21apt-get09:24:501
1669199220,21cyclictest0-21swapper/112:39:491
1669199220,21cyclictest0-21swapper/109:15:011
1669199220,1cyclictest0-21swapper/109:45:011
1668799221,1cyclictest612-21gdbus11:39:590
1668799220,21cyclictest8921-21apt-get09:09:480
1668799220,21cyclictest3492-21sendmail-msp10:00:000
1668799220,21cyclictest1317-21apt-get11:04:490
1668799220,21cyclictest0-21swapper/009:44:500
1668799220,1cyclictest4660-21munin-run12:15:010
1668799220,1cyclictest18680-21dump-pmu-power09:25:000
1668799220,1cyclictest0-21swapper/011:59:490
16697992121,0cyclictest0-21swapper/311:19:503
16697992121,0cyclictest0-21swapper/308:29:513
16697992120,1cyclictest0-21swapper/311:09:483
16697992120,0cyclictest0-21swapper/311:24:473
16697992119,1cyclictest21730-21perf07:20:013
16697992119,1cyclictest10636-21apt-get10:15:023
1669799210,20cyclictest0-21swapper/309:29:483
1669799210,1cyclictest10693-21perf09:10:003
1669799210,0cyclictest0-21swapper/310:44:483
16693992121,0cyclictest14113-21apt-get09:19:482
16693992120,1cyclictest25765-21apt-get11:54:492
16693992120,1cyclictest0-21swapper/208:54:482
16693992120,1cyclictest0-21swapper/207:29:492
16693992119,1cyclictest4719-21timerwakeupswit12:15:012
16693992119,1cyclictest23907-21date09:35:012
1669399210,21cyclictest0-21swapper/210:15:012
1669399210,20cyclictest31324-21apt-get10:59:492
1669399210,20cyclictest0-21swapper/208:59:492
1669399210,20cyclictest0-21swapper/208:19:482
1669399210,20cyclictest0-21swapper/208:09:472
1669399210,20cyclictest0-21swapper/207:20:012
1669399210,0cyclictest0-21swapper/212:34:502
1669399210,0cyclictest0-21swapper/209:49:492
16691992121,0cyclictest0-21swapper/108:14:501
16691992120,1cyclictest9305-21apt-get08:04:501
16691992120,1cyclictest19318-21apt-get09:29:491
16691992120,1cyclictest0-21swapper/109:54:531
16691992119,2cyclictest0-21swapper/111:54:481
16691992118,3cyclictest0-21swapper/110:59:491
1669199210,1cyclictest7942-21munin-plugin-st10:09:591
1669199210,0cyclictest24103-21apt-get09:35:021
16687992121,0cyclictest13368-21apt-get11:29:490
16687992121,0cyclictest0-21swapper/012:39:480
16687992121,0cyclictest0-21swapper/012:29:490
16687992121,0cyclictest0-21swapper/008:24:480
16687992121,0cyclictest0-21swapper/008:04:500
16687992120,1cyclictest14088-21apt-get08:14:500
1668799210,20cyclictest20977-21apt-get11:44:470
1668799210,20cyclictest1260-21apt-get08:54:490
1668799210,20cyclictest0-21swapper/008:34:490
1668799210,0cyclictest6529-21apt-get09:04:490
16697992020,0cyclictest0-21swapper/311:39:483
16697992020,0cyclictest0-21swapper/310:24:483
16697992020,0cyclictest0-21swapper/309:19:493
16697992020,0cyclictest0-21swapper/308:04:503
16697992020,0cyclictest0-21swapper/307:39:493
16697992019,1cyclictest16943-21apt-get10:29:483
16697992019,0cyclictest30570-21apt-get12:04:483
16697992019,0cyclictest30570-21apt-get12:04:483
16697992019,0cyclictest0-21swapper/307:29:483
1669799200,0cyclictest0-21swapper/312:39:493
16693992020,0cyclictest14146-21apt-get10:24:482
16693992020,0cyclictest10600-21apt-get12:29:492
16693992020,0cyclictest0-21swapper/212:09:492
16693992020,0cyclictest0-21swapper/211:59:482
16693992020,0cyclictest0-21swapper/211:49:492
16693992020,0cyclictest0-21swapper/210:14:502
16693992020,0cyclictest0-21swapper/210:04:502
16693992020,0cyclictest0-21swapper/209:54:532
16693992020,0cyclictest0-21swapper/209:04:492
16693992020,0cyclictest0-21swapper/208:44:482
16693992018,1cyclictest7945-21perf11:14:592
1669399200,1cyclictest1385-21cron07:45:002
1669399200,0cyclictest0-21swapper/211:09:482
16691992020,0cyclictest0-21swapper/111:29:491
16691992020,0cyclictest0-21swapper/107:39:491
16691992020,0cyclictest0-21swapper/107:34:491
16691992019,1cyclictest504-21apt-get12:09:491
16691992019,1cyclictest17558-21apt-get07:14:491
16691992018,1cyclictest0-21swapper/110:04:591
1669199200,20cyclictest0-21swapper/110:44:481
1669199200,20cyclictest0-21swapper/109:09:481
1669199200,0cyclictest0-21swapper/110:39:481
16687992020,0cyclictest28895-21apt-get08:44:470
16687992020,0cyclictest0-21swapper/011:24:480
16687992020,0cyclictest0-21swapper/011:14:500
16687992020,0cyclictest0-21swapper/010:25:020
16687992020,0cyclictest0-21swapper/009:34:500
16687992018,1cyclictest7943-21munin-run10:10:000
1668799200,20cyclictest0-21swapper/011:30:010
1668799200,20cyclictest0-21swapper/009:29:480
1668799200,0cyclictest12995-21apt-get12:34:500
1668799200,0cyclictest0-21swapper/011:19:500
1668799200,0cyclictest0-21swapper/007:34:490
16697991919,0cyclictest26483-21apt-get08:39:493
16697991919,0cyclictest0-21swapper/312:09:483
16697991919,0cyclictest0-21swapper/309:59:493
16697991919,0cyclictest0-21swapper/307:59:513
16697991918,1cyclictest11322-21apt-get09:14:493
16697991917,1cyclictest11696-21apt-get08:09:483
16693991919,0cyclictest0-21swapper/207:24:482
1669399190,18cyclictest31434-21perf07:40:002
16691991919,0cyclictest21698-21apt-get08:29:531
16691991919,0cyclictest0-21swapper/112:15:021
16691991919,0cyclictest0-21swapper/109:49:491
16691991918,0cyclictest0-21swapper/112:34:501
1669199190,18cyclictest0-21swapper/110:15:021
16687991919,0cyclictest29312-21apt-get09:49:480
16687991919,0cyclictest28932-21apt-get10:54:480
16687991919,0cyclictest0-21swapper/012:09:490
16687991919,0cyclictest0-21swapper/010:49:490
16687991919,0cyclictest0-21swapper/007:24:490
16687991918,0cyclictest0-21swapper/008:29:530
1668799190,0cyclictest0-21swapper/010:24:490
16697991818,0cyclictest0-21swapper/311:34:493
16697991818,0cyclictest0-21swapper/311:14:503
16697991818,0cyclictest0-21swapper/308:49:483
16697991818,0cyclictest0-21swapper/308:34:553
16697991816,1cyclictest0-21swapper/312:15:003
1669799180,1cyclictest0-21swapper/312:35:003
16693991818,0cyclictest15385-21apt-get12:39:492
16693991818,0cyclictest0-21swapper/211:30:022
16693991818,0cyclictest0-21swapper/209:59:492
16693991817,1cyclictest0-21swapper/209:35:022
16693991817,0cyclictest0-21swapper/212:19:502
16691991818,0cyclictest15125-21apt-get11:30:021
16691991818,0cyclictest0-21swapper/112:29:491
16691991818,0cyclictest0-21swapper/111:44:481
16691991818,0cyclictest0-21swapper/111:14:511
16691991818,0cyclictest0-21swapper/108:24:491
16691991817,1cyclictest1684-21apt-get09:59:481
16691991817,0cyclictest19339-21apt-get10:34:481
16687991818,0cyclictest0-21swapper/011:49:490
16687991818,0cyclictest0-21swapper/009:14:490
16687991818,0cyclictest0-21swapper/008:59:490
16687991818,0cyclictest0-21swapper/007:59:510
16687991817,1cyclictest21106-21perf10:35:010
16697991717,0cyclictest0-21swapper/309:44:503
16697991717,0cyclictest0-21swapper/308:14:503
16697991717,0cyclictest0-21swapper/307:54:493
16697991716,1cyclictest0-21swapper/311:44:493
16697991714,2cyclictest24124-21turbostat.cron07:25:003
16693991717,0cyclictest0-21swapper/210:09:532
16693991717,0cyclictest0-21swapper/207:49:502
1669399170,0cyclictest0-21swapper/208:24:492
16691991717,0cyclictest0-21swapper/107:44:491
16691991716,0cyclictest0-21swapper/110:14:511
16691991716,0cyclictest0-21swapper/108:44:481
1669199170,1cyclictest18676-21tr08:20:001
1669199170,16cyclictest25276-21tr08:34:551
16687991717,0cyclictest31324-21apt-get10:59:480
16687991717,0cyclictest0-21swapper/012:04:480
16687991717,0cyclictest0-21swapper/012:04:480
16687991717,0cyclictest0-21swapper/010:44:480
16687991717,0cyclictest0-21swapper/010:19:480
16687991717,0cyclictest0-21swapper/009:19:490
16687991717,0cyclictest0-21swapper/008:09:480
16687991715,1cyclictest5972-21sleep10:05:000
16697991616,0cyclictest0-21swapper/309:24:503
16697991614,1cyclictest34-21ksoftirqd/308:55:013
16697991614,1cyclictest0-21swapper/310:54:573
1669799160,1cyclictest3025-21apt-get08:55:013
1669799160,16cyclictest0-21swapper/312:29:483
1669399164,11cyclictest1686-21apache208:29:532
16693991616,0cyclictest0-21swapper/211:39:482
16693991616,0cyclictest0-21swapper/210:54:502
16693991616,0cyclictest0-21swapper/210:44:482
16693991614,1cyclictest313-21systemd-journal10:35:012
16693991614,1cyclictest0-21swapper/207:14:502
16691991616,0cyclictest0-21swapper/110:49:491
16691991616,0cyclictest0-21swapper/108:09:481
16691991615,1cyclictest2951-21apt-get12:14:521
16691991614,1cyclictest25137-21latency_hist11:49:591
1669199160,14cyclictest0-21swapper/108:36:081
1669199160,0cyclictest0-21swapper/108:54:191
16687991616,0cyclictest0-21swapper/011:09:480
16687991616,0cyclictest0-21swapper/008:19:480
1668799160,3cyclictest0-21swapper/012:24:560
16697991515,0cyclictest0-21swapper/307:44:493
1669799150,14cyclictest0-21swapper/310:49:503
1669799150,14cyclictest0-21swapper/309:04:483
1669199150,1cyclictest0-21swapper/107:25:101
16687991514,1cyclictest23473-21apt-get09:35:020
1668799150,15cyclictest0-21swapper/007:18:200
16697991414,0cyclictest0-21swapper/311:50:183
16697991414,0cyclictest0-21swapper/308:15:003
16697991414,0cyclictest0-21swapper/307:13:283
1669799140,14cyclictest0-21swapper/312:15:513
1669799140,14cyclictest0-21swapper/310:58:563
1669799140,14cyclictest0-21swapper/310:35:553
1669799140,14cyclictest0-21swapper/309:39:543
1669799140,14cyclictest0-21swapper/308:40:173
1669799140,14cyclictest0-21swapper/307:45:493
16693991414,0cyclictest0-21swapper/209:25:442
16693991414,0cyclictest0-21swapper/207:55:002
1669399140,14cyclictest0-21swapper/211:01:162
1669399140,14cyclictest0-21swapper/208:33:472
1669399140,0cyclictest0-21swapper/207:55:252
16691991414,0cyclictest0-21swapper/108:45:151
16691991413,1cyclictest0-21swapper/110:20:301
1669199140,14cyclictest0-21swapper/109:12:221
1669199140,14cyclictest0-21swapper/109:04:421
16687991414,0cyclictest0-21swapper/009:50:180
16687991414,0cyclictest0-21swapper/008:35:000
16687991414,0cyclictest0-21swapper/007:42:080
16687991414,0cyclictest0-21swapper/007:25:480
16687991413,1cyclictest0-21swapper/007:14:490
1668799140,14cyclictest0-21swapper/010:36:060
1668799140,14cyclictest0-21swapper/007:52:210
1668799140,14cyclictest0-21swapper/007:45:310
1668799140,0cyclictest0-21swapper/011:50:040
16697991313,0cyclictest0-21swapper/309:48:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional