You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 22:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 03, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17759992522,2cyclictest2626-21idleruntime-cro12:05:000
1777499240,23cyclictest18222-21apt-get10:28:113
17759992422,1cyclictest20098-21cron07:15:000
1775999240,23cyclictest5489-21dump-pmu-power07:50:000
1775999240,23cyclictest313-21systemd-journal07:30:000
1775999240,23cyclictest27395-21munin-run10:45:000
17774992322,0cyclictest30279-21apt-get10:53:083
1777499231,21cyclictest25726-21apt-get08:33:103
17768992321,1cyclictest20102-21apt-get07:15:282
17768992320,2cyclictest17390-21idleruntime-cro11:29:592
1776899230,1cyclictest313-21systemd-journal09:25:002
1776499230,2cyclictest2417-21perf08:50:001
1776499230,22cyclictest17367-21grep11:30:001
1775999230,22cyclictest17964-21perf08:15:000
1775999230,0cyclictest0-21swapper/011:10:010
17774992220,1cyclictest15557-21munin-run08:10:003
17774992220,1cyclictest15048-21apt-get12:30:023
17774992219,2cyclictest26928-21basename09:39:593
1777499220,2cyclictest7903-21munin-run07:55:013
1777499220,21cyclictest29816-21apt-get09:48:083
1777499220,21cyclictest29328-21perf09:45:003
1777499220,21cyclictest23017-21apt-get10:38:093
1777499220,21cyclictest0-21swapper/310:00:023
1777499220,21cyclictest0-21swapper/310:00:023
1777499220,21cyclictest0-21swapper/309:58:083
1777499220,21cyclictest0-21swapper/308:28:083
1777499220,1cyclictest313-21systemd-journal09:30:013
17768992220,1cyclictest29779-21apt10:50:002
17768992220,1cyclictest0-21swapper/212:10:002
17768992219,2cyclictest9714-21idleruntime-cro10:10:002
1776899220,21cyclictest30464-21apt-get11:55:032
1776899220,21cyclictest15063-21munin-run12:29:592
1776899220,21cyclictest13084-21apt-get11:23:082
1776899220,21cyclictest0-21swapper/210:28:112
1776899220,21cyclictest0-21swapper/208:48:082
17764992221,1cyclictest10758-21apt-get12:23:081
17764992221,0cyclictest0-21swapper/108:38:081
17764992220,1cyclictest7413-21cron12:14:591
17764992220,1cyclictest27593-21cron08:35:001
1776499220,21cyclictest2597-21apt-get09:58:081
1776499220,21cyclictest19753-21apt-get09:28:081
1776499220,21cyclictest0-21swapper/109:43:091
1776499220,21cyclictest0-21swapper/109:33:081
1776499220,1cyclictest645-21cron07:40:001
1776499220,1cyclictest29852-21perf11:55:011
1776499220,1cyclictest10757-21apt-config08:00:001
17759992220,1cyclictest0-21swapper/012:40:000
1775999220,21cyclictest598-21rs:main0
1775999220,21cyclictest29848-21apt-get11:55:000
1775999220,1cyclictest14929-21idleruntime-cro10:20:010
1775999220,1cyclictest12592-21munin-run11:20:000
1775999220,1cyclictest0-21swapper/010:55:000
17774992121,0cyclictest0-21swapper/310:43:083
17774992120,1cyclictest0-21swapper/311:03:093
17774992120,0cyclictest0-21swapper/309:18:083
17774992119,1cyclictest29763-21cron10:50:003
17774992119,1cyclictest28118-21apt-get08:38:063
17774992119,1cyclictest0-21swapper/311:50:003
17774992119,1cyclictest0-21swapper/309:50:013
1777499210,20cyclictest0-21swapper/309:08:083
1777499210,20cyclictest0-21swapper/307:25:013
1777499210,0cyclictest0-21swapper/308:00:013
17768992120,1cyclictest4578-21apt-get11:05:042
17768992119,1cyclictest7189-21cron09:00:002
17768992119,1cyclictest14442-21perf09:15:002
17768992119,1cyclictest10153-21turbostat.cron11:15:002
1776899210,20cyclictest8391-21apt-get07:58:082
1776899210,20cyclictest654-21munin-run07:40:002
1776899210,20cyclictest22550-21apt-get09:33:092
1776899210,20cyclictest14933-21apt-get09:18:082
1776899210,20cyclictest0-21swapper/210:43:092
1776899210,20cyclictest0-21swapper/209:50:012
1776899210,1cyclictest726-21apt-get12:03:092
1776899210,1cyclictest0-21swapper/208:29:592
1776899210,0cyclictest0-21swapper/208:38:082
1776899210,0cyclictest0-21swapper/207:33:082
17764992121,0cyclictest28685-21apt-get07:33:071
17764992121,0cyclictest1183-21apt-get07:43:081
17764992121,0cyclictest0-21swapper/110:33:091
17764992121,0cyclictest0-21swapper/108:43:081
17764992120,0cyclictest25487-21apt-get10:43:091
17764992120,0cyclictest0-21swapper/110:53:081
17764992119,1cyclictest7987-21apt-get11:10:001
17764992119,1cyclictest15551-21grep08:10:001
1776499210,20cyclictest10264-21perf12:20:001
1776499210,20cyclictest0-21swapper/110:20:021
1776499210,1cyclictest5510-21munin-plugin-st07:50:001
1776499210,1cyclictest2685-21apt-get11:03:091
1776499210,0cyclictest20681-21apt-get11:38:091
1776499210,0cyclictest18287-21apt-get11:33:091
1776499210,0cyclictest0-21swapper/111:55:031
17759992121,0cyclictest31082-21apt-get07:38:080
17759992121,0cyclictest12534-21apt-get09:13:070
17759992121,0cyclictest0-21swapper/012:33:080
17759992121,0cyclictest0-21swapper/012:15:010
17759992120,1cyclictest8391-21apt-get07:58:090
17759992120,1cyclictest0-21swapper/008:35:000
1775999210,21cyclictest0-21swapper/008:38:070
1775999210,20cyclictest2085-21cron09:55:000
1775999210,20cyclictest0-21swapper/011:58:070
1775999210,0cyclictest0-21swapper/009:48:080
1775999210,0cyclictest0-21swapper/009:30:020
17774992020,0cyclictest0-21swapper/311:55:033
17774992020,0cyclictest0-21swapper/311:38:083
17774992020,0cyclictest0-21swapper/311:33:083
17774992020,0cyclictest0-21swapper/311:18:093
17774992020,0cyclictest0-21swapper/310:08:103
17774992020,0cyclictest0-21swapper/308:53:103
17774992020,0cyclictest0-21swapper/308:45:013
17774992020,0cyclictest0-21swapper/308:13:083
17774992020,0cyclictest0-21swapper/307:13:103
17774992018,1cyclictest8341-21apt-get12:18:103
17774992018,1cyclictest7364-21grep11:10:003
17774992018,1cyclictest5318-21apt-get08:58:083
17774992018,1cyclictest0-21swapper/307:23:093
1777499200,20cyclictest0-21swapper/310:33:083
1777499200,0cyclictest0-21swapper/308:23:073
17768992020,0cyclictest0-21swapper/211:43:082
17768992020,0cyclictest0-21swapper/210:58:082
17768992020,0cyclictest0-21swapper/210:53:082
17768992020,0cyclictest0-21swapper/210:03:092
17768992020,0cyclictest0-21swapper/210:03:092
17768992020,0cyclictest0-21swapper/209:28:082
17768992020,0cyclictest0-21swapper/208:13:082
17768992019,1cyclictest15949-21apt-get12:33:082
17768992019,1cyclictest0-21swapper/207:53:082
17768992018,1cyclictest0-21swapper/208:53:102
17768992018,1cyclictest0-21swapper/208:00:002
17768992017,2cyclictest9639-21latency_hist09:05:002
1776899200,20cyclictest0-21swapper/210:10:062
1776899200,20cyclictest0-21swapper/209:48:082
1776899200,0cyclictest14900-21apt-get10:20:012
1776899200,0cyclictest0-21swapper/208:18:092
1776899200,0cyclictest0-21swapper/207:48:092
17764992020,0cyclictest13155-21apt-get12:28:091
17764992020,0cyclictest0-21swapper/112:38:151
17764992020,0cyclictest0-21swapper/111:45:001
17764992020,0cyclictest0-21swapper/111:23:081
17764992020,0cyclictest0-21swapper/107:13:101
17764992019,1cyclictest16045-21apt-get08:13:081
17764992019,1cyclictest13003-21apt-get10:18:091
17764992019,0cyclictest5994-21apt-get07:53:091
17764992019,0cyclictest10667-21apt-get11:18:091
17764992017,2cyclictest22941-21basename07:20:001
1776499200,20cyclictest0-21swapper/109:13:081
1776499200,0cyclictest0-21swapper/110:28:111
17759992020,0cyclictest7813-21apt-get10:08:100
17759992020,0cyclictest5544-21apt-get12:13:100
17759992020,0cyclictest0-21swapper/008:58:070
17759992020,0cyclictest0-21swapper/007:33:080
17759992019,1cyclictest5411-21apt-get10:03:090
17759992019,1cyclictest5411-21apt-get10:03:080
17759992019,0cyclictest0-21swapper/011:43:080
1775999200,20cyclictest0-21swapper/010:58:080
1775999200,1cyclictest313-21systemd-journal11:35:000
1775999200,0cyclictest0-21swapper/011:03:090
1775999200,0cyclictest0-21swapper/010:38:090
1775999200,0cyclictest0-21swapper/008:28:080
17774991919,0cyclictest13155-21apt-get12:28:093
17774991919,0cyclictest0-21swapper/312:23:083
17774991917,1cyclictest0-21swapper/308:00:003
17774991915,3cyclictest0-21swapper/311:23:083
17768991919,0cyclictest18224-21apt-get07:13:112
17768991919,0cyclictest0-21swapper/212:38:152
17768991919,0cyclictest0-21swapper/211:18:092
17768991919,0cyclictest0-21swapper/207:43:082
17768991919,0cyclictest0-21swapper/207:23:092
17768991918,0cyclictest0-21swapper/210:18:082
17768991917,1cyclictest612-21gdbus10:40:002
1776899190,2cyclictest1641-21cron12:25:002
17764991919,0cyclictest23017-21apt-get10:38:091
17764991919,0cyclictest18453-21apt-get08:18:071
17764991919,0cyclictest10132-21apt-get09:08:081
17764991919,0cyclictest0-21swapper/110:58:081
17764991919,0cyclictest0-21swapper/110:03:091
17764991919,0cyclictest0-21swapper/110:03:091
17764991919,0cyclictest0-21swapper/109:38:071
17764991919,0cyclictest0-21swapper/109:23:081
17764991919,0cyclictest0-21swapper/108:28:081
17764991919,0cyclictest0-21swapper/108:23:071
17759991919,0cyclictest10603-21apt-get10:13:090
17759991919,0cyclictest0-21swapper/009:43:090
17759991919,0cyclictest0-21swapper/009:38:070
17759991917,1cyclictest19265-21munin-run09:25:000
17759991917,1cyclictest0-21swapper/008:00:000
17774991818,0cyclictest0-21swapper/312:08:083
17774991818,0cyclictest0-21swapper/309:30:023
17774991818,0cyclictest0-21swapper/308:18:093
17774991818,0cyclictest0-21swapper/307:38:083
17774991817,1cyclictest32675-21apt-get10:58:083
1777499180,17cyclictest0-21swapper/307:43:073
17768991818,0cyclictest0-21swapper/211:53:112
17768991818,0cyclictest0-21swapper/209:08:082
17768991818,0cyclictest0-21swapper/208:33:112
1776899180,0cyclictest0-21swapper/209:58:082
17764991818,0cyclictest32607-21apt-get09:53:081
17764991818,0cyclictest0-21swapper/108:58:081
1775999180,0cyclictest23441-21apt-get07:23:080
1775999180,0cyclictest0-21swapper/009:28:090
17774991717,0cyclictest0-21swapper/309:23:083
17774991717,0cyclictest0-21swapper/309:13:073
17774991711,1cyclictest17710-21perf10:25:003
1776899170,0cyclictest0-21swapper/212:18:102
17764991715,1cyclictest5477-21apt11:08:091
17764991715,1cyclictest2662-21munin-run12:05:001
17764991715,1cyclictest22-21ksoftirqd/111:48:151
1776499170,16cyclictest9720-21perf10:09:591
17759991717,0cyclictest20868-21apt-get08:23:070
17759991717,0cyclictest0-21swapper/010:23:080
17759991717,0cyclictest0-21swapper/009:18:080
17759991716,1cyclictest2924-21apt-get08:53:100
17759991716,1cyclictest12020-21sh09:10:000
17759991716,1cyclictest0-21swapper/012:25:000
17759991715,1cyclictest17376-21awk11:30:000
17774991616,0cyclictest12505-21perf10:15:013
17774991616,0cyclictest0-21swapper/308:48:083
17774991616,0cyclictest0-21swapper/307:15:283
17774991614,1cyclictest0-21swapper/311:10:013
1777499160,14cyclictest781-21lldpd11:26:233
1777499160,0cyclictest0-21swapper/312:39:223
17768991616,0cyclictest0-21swapper/212:13:092
17768991616,0cyclictest0-21swapper/208:43:082
17764991616,0cyclictest0-21swapper/108:00:011
17764991614,1cyclictest0-21swapper/109:48:081
1776499160,15cyclictest0-21swapper/112:30:011
1776499160,15cyclictest0-21swapper/110:13:091
17759991616,0cyclictest0-21swapper/010:33:100
17759991616,0cyclictest0-21swapper/008:18:080
17759991616,0cyclictest0-21swapper/007:53:080
17759991616,0cyclictest0-21swapper/007:43:080
17759991615,1cyclictest7737-21apt-get09:03:070
17759991615,1cyclictest30537-21apt-get08:43:080
1775999160,15cyclictest27145-21cat11:48:200
1775999160,15cyclictest22994-21grep07:19:590
1775999160,15cyclictest20122-21perf10:30:000
17774991515,0cyclictest0-21swapper/311:43:083
17774991515,0cyclictest0-21swapper/309:03:083
17774991514,0cyclictest0-21swapper/312:03:093
17768991515,0cyclictest26281-21apt-get07:28:052
17768991515,0cyclictest0-21swapper/211:03:092
17768991513,1cyclictest0-21swapper/211:38:132
1776899150,3cyclictest0-21swapper/211:33:092
17764991515,0cyclictest0-21swapper/110:48:081
17759991515,0cyclictest0-21swapper/011:23:080
17759991515,0cyclictest0-21swapper/009:58:080
1775999151,4cyclictest0-21swapper/011:38:080
17759991514,1cyclictest13654-21apt-get08:08:100
1775999150,15cyclictest0-21swapper/012:09:550
17774991413,1cyclictest782-21sshd11:52:293
1777499140,14cyclictest0-21swapper/312:10:253
1777499140,14cyclictest0-21swapper/310:15:263
1777499140,14cyclictest0-21swapper/307:30:243
1776899140,14cyclictest0-21swapper/211:45:582
1776899140,14cyclictest0-21swapper/210:30:242
1776899140,14cyclictest0-21swapper/209:36:222
1776899140,14cyclictest0-21swapper/208:21:432
1776899140,14cyclictest0-21swapper/208:05:112
1776899140,0cyclictest0-21swapper/209:40:022
17764991414,0cyclictest0-21swapper/107:25:301
1776499140,0cyclictest0-21swapper/112:05:421
1776499140,0cyclictest0-21swapper/107:22:381
1775999140,14cyclictest0-21swapper/008:45:470
1775999140,0cyclictest0-21swapper/012:26:420
1775999140,0cyclictest0-21swapper/011:05:280
17774991313,0cyclictest0-21swapper/307:45:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional