You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-27 - 20:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Thu Mar 27, 2025 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162206993123,2cyclictest191rcu_preempt00:30:212
162211993026,3cyclictest174027-21sendmail19:40:003
162206993019,6cyclictest389000-21cut00:35:152
162206992619,1cyclictest191rcu_preempt22:39:292
162206992615,7cyclictest0-21swapper/222:30:532
162201992621,3cyclictest1090808-21systemd-journal19:55:000
162201992611,11cyclictest1090808-21systemd-journal22:35:110
162211992521,3cyclictest344489-21dump-pmu-power23:45:013
162211992521,3cyclictest344489-21dump-pmu-power23:45:003
162211992521,3cyclictest197669-21sh20:40:003
162211992520,3cyclictest0-21swapper/300:00:003
162211992514,9cyclictest0-21swapper/323:05:003
162206992522,2cyclictest352868-21rm23:55:002
162206992519,3cyclictest275581-21munin-plugin-st22:25:002
16220499258,2cyclictest0-21swapper/123:07:411
162204992521,2cyclictest254035-21sendmail22:00:001
162204992520,3cyclictest0-21swapper/121:20:001
162204992518,4cyclictest378014-21sh00:21:071
162201992522,2cyclictest0-21swapper/023:00:000
162206992420,2cyclictest339818-21idleruntime-cro23:40:002
162206992414,8cyclictest0-21swapper/200:20:012
162206992414,7cyclictest1090808-21systemd-journal23:03:112
162204992419,3cyclictest0-21swapper/122:15:001
16220499241,10cyclictest358399-21sort00:00:181
162201992419,3cyclictest189796-21dump-pmu-power20:20:000
162201992414,8cyclictest237430-21ssh21:40:100
162211992320,2cyclictest271015-21idleruntime-cro22:20:003
162211992320,2cyclictest237068-21dump-pmu-power21:40:003
162211992312,8cyclictest221996-21ssh21:20:523
162206992319,3cyclictest421ktimers/223:16:562
16220699231,5cyclictest184036-21latency_hist20:05:012
162206992312,3cyclictest1090808-21systemd-journal21:00:012
16220699231,19cyclictest246083-21ssh21:50:152
162204992320,2cyclictest332256-21unixbench_singl23:30:211
162204992319,2cyclictest199654-21cron20:45:011
162204992318,3cyclictest319160-21kworker/u16:0-nfsiod00:00:001
16220499231,19cyclictest381636-21ssh00:26:311
16221199221,7cyclictest360946-21runrttasks00:02:573
162211992211,8cyclictest726-21gdbus20:40:223
162211992211,6cyclictest252587-21ssh21:57:353
162206992219,2cyclictest227906-21systemd-run21:30:002
162206992217,3cyclictest0-21swapper/222:20:002
162206992216,4cyclictest373523-21ssh00:17:082
162206992215,2cyclictest191810-21kworker/2:2+events_freezable21:14:302
162206992213,8cyclictest411rcuc/222:51:292
162206992212,7cyclictest216341-21grep21:15:202
162206992211,4cyclictest226681-21kworker/u16:2+xprtiod21:41:352
16220699220,8cyclictest178927-21ld19:50:202
16220699220,18cyclictest265839-21sh22:13:532
16220499221,17cyclictest239272-21sh21:41:081
162201992219,2cyclictest327246-21ssh23:25:010
162201992218,3cyclictest712-21tpm2-abrmd21:00:210
16220199221,7cyclictest377217-21mailstats00:20:230
16220199221,7cyclictest222026-21ssh21:20:570
162201992215,4cyclictest230682-21sh21:32:030
162211992117,2cyclictest322660-21apt23:20:003
16221199211,18cyclictest389911-21cat00:35:253
16221199211,18cyclictest209168-21ntpq21:05:183
16221199211,17cyclictest381656-21sh00:26:343
162211992110,7cyclictest373174-21sh00:16:353
16220699218,5cyclictest281135-21kworker/u16:3+xprtiod22:44:322
16220699215,1cyclictest191rcu_preempt23:49:592
16220699215,1cyclictest191rcu_preempt23:49:582
162206992118,2cyclictest249829-21/usr/sbin/munin21:55:142
16220699211,7cyclictest259878-21sh22:06:332
16220699211,7cyclictest188270-21tune2fs20:15:132
162206992117,2cyclictest0-21swapper/200:30:002
162206992110,8cyclictest341297-21ssh23:40:242
162206992110,8cyclictest341297-21ssh23:40:232
16220699210,7cyclictest230354-21ssh21:31:312
16220699210,18cyclictest360829-21sh00:02:462
16220499211,7cyclictest227406-21ssh21:29:131
16220499211,7cyclictest210133-21diskmemload22:47:521
162204992117,2cyclictest0-21swapper/121:30:101
162204992117,2cyclictest0-21swapper/100:35:201
162204992116,3cyclictest0-21swapper/123:00:001
162204992113,5cyclictest266999-21tr22:15:141
16220499211,18cyclictest362606-21sh00:05:151
16220499211,17cyclictest308977-21sh23:03:511
16220499211,16cyclictest233562-21sh21:35:181
16220499211,13cyclictest284019-21grep22:35:141
16220499210,10cyclictest288373-21ssh22:40:121
16220199215,3cyclictest356322-21ssh23:58:010
162201992118,2cyclictest388145-21cut00:35:000
162201992117,2cyclictest352840-21grep23:55:000
162201992117,2cyclictest170089-21systemd-run19:30:000
162201992117,2cyclictest0-21swapper/021:00:000
162201992116,3cyclictest254757-21gltestperf22:00:140
162201992114,4cyclictest344999-21gdbus23:45:010
162201992114,4cyclictest344999-21gdbus23:45:010
16220199211,18cyclictest242871-21sh21:46:410
162201992111,7cyclictest277262-21ssh22:27:260
16220199210,8cyclictest215574-21ssh21:15:120
16221199208,8cyclictest242337-21taskset21:45:503
162211992017,2cyclictest388225-21apt-config00:35:003
162211992016,2cyclictest703-21polkitd20:00:003
162211992016,2cyclictest329015-21sh23:27:323
162211992016,2cyclictest0-21swapper/322:55:153
162211992015,3cyclictest940-21runrttasks23:30:183
162211992014,4cyclictest239852-21ssh21:42:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional