You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-06 - 19:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Thu Nov 06, 2025 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911040,94ptp4l0-21swapper/107:07:481
2759911040,94ptp4l0-21swapper/107:07:481
94980830,78rtkit-daemon0-21swapper/007:07:470
94980830,78rtkit-daemon0-21swapper/007:07:470
2402926957,9sleep40-21swapper/407:05:226
2402926957,9sleep40-21swapper/407:05:226
2514726756,9sleep50-21swapper/507:09:597
2514726756,9sleep50-21swapper/507:09:597
2488826553,10sleep70-21swapper/707:06:149
2488826553,10sleep70-21swapper/707:06:139
2499526455,7sleep60-21swapper/607:07:498
2499526455,7sleep60-21swapper/607:07:488
2498726452,10sleep110-21swapper/1107:07:423
2498726452,10sleep110-21swapper/1107:07:413
2513526354,7sleep100-21swapper/1007:09:532
2513526354,7sleep100-21swapper/1007:09:532
2550299204,10cyclictest0-21swapper/211:50:004
275991180,1ptp4l21-21ksoftirqd/112:02:431
275991180,1ptp4l21-21ksoftirqd/111:59:551
2553599182,10cyclictest0-21swapper/910:24:5911
2553599181,15cyclictest0-21swapper/912:20:1811
2549799183,9cyclictest0-21swapper/109:30:001
2549199184,8cyclictest0-21swapper/007:15:010
2549199184,8cyclictest0-21swapper/007:15:000
2552999172,8cyclictest0-21swapper/807:55:0110
2552599172,10cyclictest0-21swapper/711:25:009
2549799173,9cyclictest0-21swapper/108:45:001
275991160,0ptp4l0-21swapper/112:15:281
275991160,0ptp4l0-21swapper/107:45:121
275991160,0ptp4l0-21swapper/107:40:181
2552599161,9cyclictest0-21swapper/712:35:019
2552099163,8cyclictest0-21swapper/612:20:008
2552099161,2cyclictest0-21swapper/612:35:178
2550799161,11cyclictest14211-21kworker/u24:009:55:015
2550299163,9cyclictest0-21swapper/209:05:004
93922150,0sleep50-21swapper/508:50:217
275991150,0ptp4l0-21swapper/112:20:261
275991150,0ptp4l0-21swapper/110:35:191
275991150,0ptp4l0-21swapper/109:05:201
2553599153,8cyclictest14211-21kworker/u24:008:10:0011
2552999154,7cyclictest0-21swapper/807:25:2910
2552999153,7cyclictest0-21swapper/808:30:0010
2552599152,9cyclictest0-21swapper/710:10:009
2552599152,8cyclictest0-21swapper/711:15:249
2552599151,8cyclictest0-21swapper/709:34:279
2552099153,8cyclictest0-21swapper/607:20:018
2552099153,8cyclictest0-21swapper/607:20:008
2552099153,7cyclictest0-21swapper/607:25:008
2551199151,11cyclictest179-1kworker/4:1H08:55:006
2550799152,8cyclictest0-21swapper/309:15:005
2550299153,8cyclictest0-21swapper/212:15:284
2550299152,9cyclictest0-21swapper/212:10:234
2550299152,8cyclictest0-21swapper/211:35:274
2550299150,9cyclictest0-21swapper/209:55:014
2549199153,7cyclictest0-21swapper/011:20:000
249972157,6sleep80-21swapper/807:07:5110
249972157,6sleep80-21swapper/807:07:5110
211062150,0sleep00-21swapper/008:25:140
275991140,1ptp4l21-21ksoftirqd/112:10:241
275991140,0ptp4l0-21swapper/112:25:131
275991140,0ptp4l0-21swapper/112:10:001
275991140,0ptp4l0-21swapper/109:55:191
275991140,0ptp4l0-21swapper/109:50:151
275991140,0ptp4l0-21swapper/109:15:141
275991140,0ptp4l0-21swapper/108:25:221
275991140,0ptp4l0-21swapper/108:00:181
275991140,0ptp4l0-21swapper/107:20:221
275991140,0ptp4l0-21swapper/107:20:221
2553599143,7cyclictest0-21swapper/909:35:0111
2553599143,6cyclictest0-21swapper/908:55:0111
2553599142,7cyclictest0-21swapper/911:30:2311
2553599141,7cyclictest14211-21kworker/u24:010:34:5911
25535991412,1cyclictest0-21swapper/912:39:5911
2552999142,7cyclictest0-21swapper/807:45:0010
2552999142,7cyclictest0-21swapper/807:15:0010
2552999142,7cyclictest0-21swapper/807:15:0010
2552599142,8cyclictest0-21swapper/710:20:009
2552599142,6cyclictest0-21swapper/711:00:229
2552099142,8cyclictest0-21swapper/609:00:018
2552099142,7cyclictest0-21swapper/612:30:018
2550799142,7cyclictest0-21swapper/309:50:015
2549199142,8cyclictest0-21swapper/012:35:000
275991130,5ptp4l21-21ksoftirqd/109:05:001
275991130,1ptp4l21-21ksoftirqd/110:30:211
275991130,1ptp4l21-21ksoftirqd/109:10:231
275991130,1ptp4l21-21ksoftirqd/107:50:121
275991130,0ptp4l0-21swapper/111:40:221
275991130,0ptp4l0-21swapper/111:30:141
275991130,0ptp4l0-21swapper/108:30:191
275991130,0ptp4l0-21swapper/108:20:181
2553599133,7cyclictest0-21swapper/908:15:0111
2553599133,6cyclictest0-21swapper/911:30:0011
2553599132,7cyclictest0-21swapper/909:55:2811
2552999132,7cyclictest0-21swapper/811:55:2310
2552599132,7cyclictest0-21swapper/712:00:009
2552099132,7cyclictest0-21swapper/611:50:018
2552099132,7cyclictest0-21swapper/608:30:248
2550799133,7cyclictest0-21swapper/310:15:005
2550799133,6cyclictest0-21swapper/312:30:005
2550799133,6cyclictest0-21swapper/311:05:005
2550799132,7cyclictest0-21swapper/312:00:015
2550299131,7cyclictest261rcuc/212:25:014
2550299130,12cyclictest0-21swapper/211:14:594
2549199133,6cyclictest0-21swapper/012:25:240
2549199132,7cyclictest0-21swapper/008:00:270
2549199132,6cyclictest0-21swapper/008:45:000
225922130,0sleep90-21swapper/908:25:2311
275991120,3ptp4l29979-21sendmail_mailst08:35:231
275991120,1ptp4l21-21ksoftirqd/110:50:561
275991120,0ptp4l0-21swapper/111:45:251
275991120,0ptp4l0-21swapper/111:22:341
275991120,0ptp4l0-21swapper/111:05:261
275991120,0ptp4l0-21swapper/111:04:581
275991120,0ptp4l0-21swapper/110:00:201
275991120,0ptp4l0-21swapper/108:40:201
275991120,0ptp4l0-21swapper/108:13:281
2553999122,8cyclictest0-21swapper/1007:45:242
2553599122,6cyclictest0-21swapper/908:40:0011
2552999122,6cyclictest0-21swapper/812:20:0010
2552999120,7cyclictest0-21swapper/809:50:0010
2552599123,5cyclictest0-21swapper/709:45:019
2552599123,5cyclictest0-21swapper/707:25:019
2552599123,5cyclictest0-21swapper/707:25:009
2552099122,6cyclictest0-21swapper/611:40:238
2552099122,6cyclictest0-21swapper/609:45:238
2552099122,6cyclictest0-21swapper/609:20:018
2552099121,7cyclictest0-21swapper/610:40:278
2550299129,2cyclictest0-21swapper/211:50:144
2550299129,2cyclictest0-21swapper/211:30:234
2549199122,6cyclictest14211-21kworker/u24:010:20:010
2549199122,6cyclictest0-21swapper/009:45:010
275991110,4ptp4l21-21ksoftirqd/110:15:121
275991110,1ptp4l21-21ksoftirqd/111:10:161
275991110,1ptp4l21-21ksoftirqd/108:19:041
275991110,1ptp4l0-21swapper/112:39:591
275991110,1ptp4l0-21swapper/110:25:181
275991110,0ptp4l0-21swapper/111:50:211
275991110,0ptp4l0-21swapper/110:55:241
275991110,0ptp4l0-21swapper/110:10:221
275991110,0ptp4l0-21swapper/110:05:271
275991110,0ptp4l0-21swapper/109:35:141
275991110,0ptp4l0-21swapper/107:55:111
275991110,0ptp4l0-21swapper/107:31:041
2553599111,6cyclictest0-21swapper/912:00:0011
2552999112,6cyclictest0-21swapper/809:00:2510
2552999112,6cyclictest0-21swapper/808:25:0010
2552999110,10cyclictest30411-21cut09:20:1310
2552999110,10cyclictest28130-21grep12:05:1310
2552099112,6cyclictest0-21swapper/609:25:018
2551699110,10cyclictest0-21swapper/510:45:157
2551199110,10cyclictest5319-21phc2sys-jitter12:15:246
2550299118,2cyclictest0-21swapper/210:00:154
2550299113,5cyclictest0-21swapper/211:10:004
2550299112,5cyclictest0-21swapper/210:15:304
2550299110,5cyclictest0-21swapper/207:48:084
2550299110,10cyclictest11017-21irqrtprio09:35:184
2550299110,10cyclictest0-21swapper/210:45:124
2549199112,5cyclictest0-21swapper/007:20:260
2549199112,5cyclictest0-21swapper/007:20:250
275991100,2ptp4l0-21swapper/110:40:231
275991100,1ptp4l8717-40dnf09:34:271
275991100,1ptp4l7136-21switchtime11:35:261
275991100,1ptp4l0-21swapper/112:30:181
275991100,1ptp4l0-21swapper/111:15:141
275991100,0ptp4l0-21swapper/109:40:161
275991100,0ptp4l0-21swapper/109:20:141
275991100,0ptp4l0-21swapper/108:59:381
275991100,0ptp4l0-21swapper/107:15:221
275991100,0ptp4l0-21swapper/107:15:221
2554499107,2cyclictest23804-21latency_hist08:30:013
2553999100,9cyclictest28320-21latency_hist10:00:012
2553999100,9cyclictest0-21swapper/1007:25:202
2553999100,1cyclictest0-21swapper/1009:35:232
2553599102,6cyclictest0-21swapper/908:40:0011
2552999100,1cyclictest0-21swapper/809:55:1910
2552599102,5cyclictest0-21swapper/709:20:019
2552599102,5cyclictest0-21swapper/707:35:019
2551699108,1cyclictest0-21swapper/507:45:017
2551699100,2cyclictest10827-21df11:00:107
2551699100,2cyclictest0-21swapper/508:10:157
2550799100,9cyclictest16832-21/usr/sbin/munin09:00:215
2550799100,5cyclictest0-21swapper/308:50:225
2550299100,9cyclictest0-21swapper/207:25:314
2550299100,2cyclictest8723-21kworker/2:209:55:174
2550299100,2cyclictest0-21swapper/207:40:144
2549199102,5cyclictest0-21swapper/011:40:000
225692101,6sleep90-21swapper/907:05:1611
225692101,6sleep90-21swapper/907:05:1611
9498090,4rtkit-daemon0-21swapper/012:08:150
27599190,0ptp4l0-21swapper/111:25:011
27599190,0ptp4l0-21swapper/110:45:121
27599190,0ptp4l0-21swapper/110:20:011
27599190,0ptp4l0-21swapper/109:45:011
27599190,0ptp4l0-21swapper/108:50:121
27599190,0ptp4l0-21swapper/108:05:141
27599190,0ptp4l0-21swapper/107:35:141
27599190,0ptp4l0-21swapper/107:25:191
27599190,0ptp4l0-21swapper/107:10:001
27599190,0ptp4l0-21swapper/107:10:001
255399998,1cyclictest0-21swapper/1010:35:182
255399997,1cyclictest0-21swapper/1011:15:182
255399996,2cyclictest0-21swapper/1011:49:592
255399992,4cyclictest0-21swapper/1008:50:012
255399990,1cyclictest0-21swapper/1011:35:202
255399990,1cyclictest0-21swapper/1011:10:152
255399990,1cyclictest0-21swapper/1009:00:182
255359990,1cyclictest0-21swapper/911:00:1111
255299998,1cyclictest0-21swapper/809:25:3110
255299990,9cyclictest0-21swapper/807:20:1810
255299990,9cyclictest0-21swapper/807:20:1710
255299990,1cyclictest0-21swapper/808:05:1310
255259992,4cyclictest0-21swapper/712:30:019
255259990,2cyclictest28101-21date10:00:019
255259990,1cyclictest0-21swapper/709:00:199
255209992,6cyclictest0-21swapper/609:00:018
255209990,8cyclictest11842-21idleruntime-cro08:15:008
255209990,2cyclictest1009-21snmpd09:08:078
255169997,1cyclictest14211-21kworker/u24:011:55:017
255169990,8cyclictest0-21swapper/511:20:007
255169990,8cyclictest0-21swapper/507:20:177
255169990,8cyclictest0-21swapper/507:20:177
255119998,1cyclictest0-21swapper/408:25:126
255119998,1cyclictest0-21swapper/407:25:276
255119998,1cyclictest0-21swapper/407:20:006
255119998,1cyclictest0-21swapper/407:20:006
255119996,2cyclictest29076-21turbostat07:15:016
255119996,2cyclictest29076-21turbostat07:15:016
255119992,4cyclictest0-21swapper/412:05:006
255119991,5cyclictest0-21swapper/408:20:006
255119990,9cyclictest0-21swapper/412:20:176
255119990,9cyclictest0-21swapper/408:05:206
255119990,1cyclictest0-21swapper/410:35:206
255119990,1cyclictest0-21swapper/409:30:156
255079992,4cyclictest0-21swapper/311:30:005
255079991,5cyclictest0-21swapper/309:30:015
255079990,9cyclictest0-21swapper/311:35:285
255079990,1cyclictest0-21swapper/308:38:015
255029997,1cyclictest15162-21hddtemp_smartct07:35:164
255029991,7cyclictest0-21swapper/207:20:264
255029991,7cyclictest0-21swapper/207:20:254
255029990,8cyclictest9979-21irqrtprio11:40:184
255029990,8cyclictest8518-21irqrtprio08:50:164
255029990,8cyclictest4723-21/usr/sbin/munin08:05:164
255029990,8cyclictest30426-21sed08:35:254
255029990,8cyclictest2219-21irqrtprio08:00:214
255029990,8cyclictest11869-21cat10:20:124
255029990,8cyclictest0-21swapper/211:55:124
255029990,8cyclictest0-21swapper/211:15:194
255029990,8cyclictest0-21swapper/211:00:204
255029990,8cyclictest0-21swapper/210:40:204
255029990,8cyclictest0-21swapper/210:35:244
255029990,8cyclictest0-21swapper/209:40:164
255029990,8cyclictest0-21swapper/209:15:124
255029990,8cyclictest0-21swapper/208:30:234
255029990,2cyclictest18781-21cpuspeed_turbos09:05:164
255029990,2cyclictest0-21swapper/212:05:124
255029990,1cyclictest0-21swapper/209:10:204
254919991,7cyclictest0-21swapper/010:50:180
254919990,2cyclictest535-21expr10:05:150
27609180,1phc2sys0-21swapper/207:05:334
27609180,1phc2sys0-21swapper/207:05:334
255449987,1cyclictest0-21swapper/1110:35:003
255449980,8cyclictest28479-21fschecks_count12:05:133
255449980,1cyclictest0-21swapper/1111:10:153
255399987,1cyclictest0-21swapper/1009:30:242
255399987,1cyclictest0-21swapper/1008:30:162
255399986,1cyclictest0-21swapper/1007:15:032
255399986,1cyclictest0-21swapper/1007:15:032
255399981,6cyclictest0-21swapper/1009:05:002
255399980,8cyclictest0-21swapper/1012:20:172
255399980,8cyclictest0-21swapper/1010:50:132
255399980,7cyclictest155372sleep1008:18:122
255399980,1cyclictest0-21swapper/1011:30:132
255399980,1cyclictest0-21swapper/1011:20:202
255399980,1cyclictest0-21swapper/1009:40:252
255359987,1cyclictest0-21swapper/910:50:1411
255359987,1cyclictest0-21swapper/907:50:2011
255359980,8cyclictest0-21swapper/912:30:1211
255359980,8cyclictest0-21swapper/912:10:1611
255359980,8cyclictest0-21swapper/910:49:5911
255359980,8cyclictest0-21swapper/907:55:1411
255359980,1cyclictest0-21swapper/911:53:2211
255359980,1cyclictest0-21swapper/907:15:2111
255359980,1cyclictest0-21swapper/907:15:2111
255299987,1cyclictest0-21swapper/811:25:1310
255299987,1cyclictest0-21swapper/810:45:2310
255299987,1cyclictest0-21swapper/810:10:1810
255299987,1cyclictest0-21swapper/809:30:2810
255299987,1cyclictest0-21swapper/808:10:2610
255299986,1cyclictest0-21swapper/812:39:5910
255299984,3cyclictest14074-21awk10:20:2510
255299981,5cyclictest17987-21/usr/sbin/munin11:50:1810
255299981,4cyclictest0-21swapper/811:05:0110
255299980,8cyclictest0-21swapper/812:30:1310
255299980,8cyclictest0-21swapper/811:05:4810
255259987,1cyclictest0-21swapper/707:50:009
255259980,7cyclictest3563-21gnome-settings-07:52:409
255259980,1cyclictest0-21swapper/711:40:219
255259980,1cyclictest0-21swapper/710:10:259
255259980,1cyclictest0-21swapper/708:50:139
255209987,1cyclictest0-21swapper/610:50:198
255209980,5cyclictest0-21swapper/607:35:168
255209980,1cyclictest0-21swapper/609:50:218
255169987,1cyclictest0-21swapper/509:40:157
255169986,1cyclictest0-21swapper/508:25:227
255169980,8cyclictest0-21swapper/512:10:187
255169980,1cyclictest0-21swapper/509:35:147
255119987,1cyclictest0-21swapper/412:30:216
255119987,1cyclictest0-21swapper/409:50:166
255119981,5cyclictest0-21swapper/410:30:226
255119981,5cyclictest0-21swapper/410:00:006
255119981,4cyclictest0-21swapper/411:30:006
255119981,4cyclictest0-21swapper/407:40:006
255119980,8cyclictest0-21swapper/409:05:166
255119980,8cyclictest0-21swapper/408:40:206
255119980,8cyclictest0-21swapper/408:35:196
255119980,2cyclictest20305-21cat07:40:246
255119980,1cyclictest0-21swapper/410:00:216
255119980,1cyclictest0-21swapper/409:35:176
255079987,1cyclictest0-21swapper/309:08:085
255079987,1cyclictest0-21swapper/307:40:195
255079981,4cyclictest0-21swapper/312:30:005
255079981,3cyclictest0-21swapper/309:25:025
255079980,8cyclictest0-21swapper/310:05:155
255079980,8cyclictest0-21swapper/307:30:125
255079980,1cyclictest0-21swapper/311:10:135
255079980,1cyclictest0-21swapper/310:30:195
255029980,8cyclictest0-21swapper/212:25:164
255029980,8cyclictest0-21swapper/210:25:124
255029980,8cyclictest0-21swapper/210:10:194
255029980,8cyclictest0-21swapper/209:45:154
255029980,8cyclictest0-21swapper/208:25:234
255029980,8cyclictest0-21swapper/208:20:124
255029980,8cyclictest0-21swapper/208:10:254
255029980,7cyclictest0-21swapper/209:25:194
255029980,7cyclictest0-21swapper/207:15:154
255029980,7cyclictest0-21swapper/207:15:144
255029980,2cyclictest30765-21/usr/sbin/munin09:20:144
255029980,2cyclictest14097-21/usr/sbin/munin08:15:154
255029980,1cyclictest0-21swapper/212:30:114
255029980,1cyclictest0-21swapper/211:20:174
255029980,1cyclictest0-21swapper/210:55:264
255029980,1cyclictest0-21swapper/210:50:014
255029980,1cyclictest0-21swapper/210:30:174
255029980,1cyclictest0-21swapper/210:05:164
255029980,1cyclictest0-21swapper/209:30:204
255029980,1cyclictest0-21swapper/208:45:154
255029980,1cyclictest0-21swapper/208:40:144
255029980,1cyclictest0-21swapper/207:50:124
254919986,1cyclictest0-21swapper/009:20:180
254919985,2cyclictest26372-21turbostat10:44:550
254919984,3cyclictest2072-21if_err_enp4s011:30:160
255449976,1cyclictest28478-21diskstats08:35:153
255449976,1cyclictest0-21swapper/1112:25:143
255449976,1cyclictest0-21swapper/1112:15:223
255449976,1cyclictest0-21swapper/1112:00:153
255449976,1cyclictest0-21swapper/1111:50:503
255449976,1cyclictest0-21swapper/1111:45:203
255449976,1cyclictest0-21swapper/1111:35:273
255449976,1cyclictest0-21swapper/1111:32:523
255449976,1cyclictest0-21swapper/1111:25:253
255449976,1cyclictest0-21swapper/1111:20:483
255449976,1cyclictest0-21swapper/1111:05:503
255449976,1cyclictest0-21swapper/1110:51:393
255449976,1cyclictest0-21swapper/1110:25:043
255449976,1cyclictest0-21swapper/1110:20:253
255449976,1cyclictest0-21swapper/1110:11:593
255449976,1cyclictest0-21swapper/1110:00:143
255449976,1cyclictest0-21swapper/1109:45:123
255449976,1cyclictest0-21swapper/1109:37:053
255449976,1cyclictest0-21swapper/1109:31:433
255449976,1cyclictest0-21swapper/1109:20:193
255449976,1cyclictest0-21swapper/1109:15:243
255449976,1cyclictest0-21swapper/1109:00:033
255449976,1cyclictest0-21swapper/1108:21:373
255449976,1cyclictest0-21swapper/1108:15:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional