You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 21:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Thu Nov 27, 2025 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911050,95ptp4l0-21swapper/107:07:411
332526855,10sleep50-21swapper/507:05:307
341926454,8sleep40-21swapper/407:06:416
3246126352,8sleep00-21swapper/007:05:010
347126251,9sleep110-21swapper/1107:07:243
346026251,9sleep60-21swapper/607:07:198
333226251,9sleep100-21swapper/1007:05:352
159426249,10sleep70-21swapper/707:05:209
73972610,0sleep60-21swapper/607:55:008
225742600,0sleep20-21swapper/207:30:184
294082580,0sleep00-21swapper/007:40:170
261152560,0sleep40-21swapper/408:15:246
122542560,0sleep40-21swapper/408:00:106
306912540,0sleep30-21swapper/308:25:005
419299440,7cyclictest0-21swapper/1007:30:222
416099222,7cyclictest0-21swapper/408:50:006
416099222,16cyclictest0-21swapper/408:55:016
416099222,15cyclictest0-21swapper/412:20:016
415199203,10cyclictest0-21swapper/208:05:004
416699193,10cyclictest0-21swapper/509:05:007
415199193,10cyclictest0-21swapper/210:05:254
418199182,10cyclictest0-21swapper/811:55:0110
417099183,9cyclictest0-21swapper/607:15:008
417099173,9cyclictest0-21swapper/608:00:008
4170991712,4cyclictest0-21swapper/611:10:178
35962178,7sleep80-21swapper/807:08:5810
168842170,0sleep50-21swapper/508:45:287
419799161,2cyclictest0-21swapper/1109:10:173
418199162,9cyclictest0-21swapper/807:15:0010
417099163,8cyclictest0-21swapper/608:18:348
417099163,7cyclictest0-21swapper/608:10:008
415199163,8cyclictest0-21swapper/210:55:004
415199163,8cyclictest0-21swapper/210:25:014
415199163,8cyclictest0-21swapper/210:25:004
414299163,8cyclictest0-21swapper/009:10:000
414299163,8cyclictest0-21swapper/008:25:010
414299163,8cyclictest0-21swapper/007:30:000
275991160,1ptp4l21-21ksoftirqd/111:25:241
275991160,1ptp4l21-21ksoftirqd/110:35:241
275991160,0ptp4l0-21swapper/112:25:191
275991160,0ptp4l0-21swapper/107:10:251
419799152,8cyclictest0-21swapper/1111:45:003
4192991512,3cyclictest0-21swapper/1010:40:182
418199153,7cyclictest0-21swapper/812:15:0210
417599154,7cyclictest0-21swapper/712:35:009
417599152,9cyclictest0-21swapper/710:05:009
417599152,7cyclictest0-21swapper/709:10:019
417099154,7cyclictest0-21swapper/610:45:008
417099151,9cyclictest0-21swapper/609:20:008
416699153,7cyclictest0-21swapper/512:30:017
414799153,8cyclictest0-21swapper/108:20:001
414799153,8cyclictest0-21swapper/107:50:001
414799153,7cyclictest0-21swapper/107:30:001
414799152,7cyclictest0-21swapper/110:05:001
414299152,7cyclictest0-21swapper/012:05:000
275991150,1ptp4l21-21ksoftirqd/112:05:181
275991150,1ptp4l21-21ksoftirqd/110:55:001
275991150,0ptp4l0-21swapper/108:30:231
138932150,0sleep110-21swapper/1110:50:123
419799142,7cyclictest0-21swapper/1108:35:003
417599143,6cyclictest0-21swapper/709:00:009
417599142,7cyclictest0-21swapper/711:55:249
417599142,7cyclictest0-21swapper/709:15:009
417099143,7cyclictest0-21swapper/612:10:018
417099142,7cyclictest0-21swapper/608:50:018
417099140,13cyclictest24944-21awk09:00:008
416699143,7cyclictest0-21swapper/507:20:007
4160991411,2cyclictest23413-21turbostat09:40:006
414799143,7cyclictest0-21swapper/110:50:011
414799142,8cyclictest0-21swapper/111:50:261
414799141,8cyclictest0-21swapper/111:05:001
275991140,1ptp4l21-21ksoftirqd/112:35:191
275991140,1ptp4l21-21ksoftirqd/111:45:201
195202140,0sleep50-21swapper/512:20:167
419299132,7cyclictest0-21swapper/1007:20:242
4192991312,1cyclictest0-21swapper/1010:05:242
418199133,6cyclictest0-21swapper/809:45:2410
418199132,7cyclictest0-21swapper/812:30:0210
418199132,7cyclictest0-21swapper/811:30:0110
418199132,7cyclictest0-21swapper/808:30:0010
417599133,6cyclictest0-21swapper/707:25:019
417599132,7cyclictest0-21swapper/709:20:009
417599132,6cyclictest0-21swapper/711:10:009
417599132,6cyclictest0-21swapper/710:50:029
417099132,7cyclictest0-21swapper/611:10:008
417099132,6cyclictest0-21swapper/609:40:248
417099131,7cyclictest0-21swapper/611:35:258
416699133,6cyclictest0-21swapper/510:35:007
416699133,6cyclictest0-21swapper/510:35:007
416699132,7cyclictest0-21swapper/509:05:007
416699132,7cyclictest0-21swapper/507:40:007
416699132,6cyclictest0-21swapper/510:25:017
416699132,6cyclictest0-21swapper/510:25:007
415199132,7cyclictest0-21swapper/211:20:284
414799133,7cyclictest0-21swapper/109:25:001
414799132,8cyclictest0-21swapper/107:40:011
414799132,7cyclictest0-21swapper/107:20:011
414299131,7cyclictest0-21swapper/011:15:000
275991130,5ptp4l0-21swapper/108:50:151
275991130,4ptp4l0-21swapper/111:40:131
275991130,1ptp4l21-21ksoftirqd/112:03:591
275991130,1ptp4l21-21ksoftirqd/111:35:161
275991130,1ptp4l21-21ksoftirqd/109:47:441
275991130,0ptp4l0-21swapper/112:30:211
275991130,0ptp4l0-21swapper/110:19:551
275991130,0ptp4l0-21swapper/110:13:231
275991130,0ptp4l0-21swapper/110:13:231
419799122,6cyclictest0-21swapper/1109:50:003
419299122,7cyclictest3969-21pool09:30:002
419299122,7cyclictest0-21swapper/1011:50:012
417599122,6cyclictest0-21swapper/710:20:009
417599122,6cyclictest0-21swapper/707:55:239
417599121,7cyclictest31898-21kworker/u24:112:05:009
417599120,7cyclictest0-21swapper/709:49:569
416699122,6cyclictest0-21swapper/512:35:237
416699122,6cyclictest0-21swapper/512:10:007
416699122,6cyclictest0-21swapper/512:00:017
416699122,6cyclictest0-21swapper/510:15:017
416699122,6cyclictest0-21swapper/510:15:007
416699122,6cyclictest0-21swapper/509:35:017
416699121,6cyclictest0-21swapper/508:25:227
416099122,6cyclictest0-21swapper/410:30:236
416099122,6cyclictest0-21swapper/410:30:236
416099122,6cyclictest0-21swapper/408:05:236
415699123,6cyclictest0-21swapper/311:20:005
415699123,6cyclictest0-21swapper/308:55:245
415699121,6cyclictest0-21swapper/312:20:015
414799121,7cyclictest0-21swapper/109:45:001
275991120,4ptp4l21-21ksoftirqd/110:40:161
275991120,3ptp4l0-21swapper/109:35:211
275991120,0ptp4l0-21swapper/111:55:161
275991120,0ptp4l0-21swapper/111:15:541
275991120,0ptp4l0-21swapper/111:05:251
275991120,0ptp4l0-21swapper/109:15:241
275991120,0ptp4l0-21swapper/109:06:161
275991120,0ptp4l0-21swapper/107:40:161
419799110,10cyclictest16970-21ps10:10:223
419799110,10cyclictest16970-21ps10:10:223
419299112,6cyclictest0-21swapper/1010:30:002
419299112,6cyclictest0-21swapper/1010:30:002
419299112,5cyclictest0-21swapper/1012:00:002
419299110,7cyclictest0-21swapper/1011:20:002
419299110,10cyclictest12067-21cpuspeed_turbos08:00:102
418699113,7cyclictest0-21swapper/911:05:0011
418199110,2cyclictest0-21swapper/811:00:1610
418199110,10cyclictest0-21swapper/810:15:0010
418199110,10cyclictest0-21swapper/810:15:0010
417599112,6cyclictest0-21swapper/712:10:009
417599112,5cyclictest0-21swapper/709:35:009
417599110,5cyclictest30297-21munin-run11:55:009
417599110,2cyclictest0-21swapper/707:45:149
417099111,5cyclictest0-21swapper/611:30:008
416099110,3cyclictest10252-21wget12:10:126
416099110,2cyclictest31981-21perl08:25:136
416099110,10cyclictest9797-21missed_timers07:55:186
415699112,5cyclictest0-21swapper/308:05:005
415699110,10cyclictest2050-21basename09:10:175
415199112,6cyclictest0-21swapper/210:40:004
415199112,6cyclictest0-21swapper/207:40:004
414299112,5cyclictest0-21swapper/009:35:200
275991110,3ptp4l0-21swapper/108:45:281
275991110,2ptp4l4581-21expr11:20:151
275991110,2ptp4l28964-21cut09:05:011
275991110,2ptp4l16664-21sshd08:05:141
275991110,2ptp4l15152-21cat12:15:151
275991110,2ptp4l11905-21irqstats12:10:191
275991110,1ptp4l21-21ksoftirqd/108:20:241
275991110,1ptp4l21-21ksoftirqd/107:55:201
275991110,1ptp4l21-21ksoftirqd/107:20:181
275991110,0ptp4l0-21swapper/111:10:131
275991110,0ptp4l0-21swapper/110:55:131
275991110,0ptp4l0-21swapper/110:30:211
275991110,0ptp4l0-21swapper/110:30:201
275991110,0ptp4l0-21swapper/110:25:141
275991110,0ptp4l0-21swapper/110:25:141
275991110,0ptp4l0-21swapper/110:20:121
275991110,0ptp4l0-21swapper/110:20:111
275991110,0ptp4l0-21swapper/110:00:161
275991110,0ptp4l0-21swapper/109:50:211
275991110,0ptp4l0-21swapper/108:55:581
275991110,0ptp4l0-21swapper/108:40:161
275991110,0ptp4l0-21swapper/107:50:151
275991110,0ptp4l0-21swapper/107:30:231
419299108,1cyclictest0-21swapper/1008:25:002
418699108,1cyclictest0-21swapper/910:20:1611
418699108,1cyclictest0-21swapper/910:20:1611
418699104,3cyclictest0-21swapper/912:25:0011
418699102,7cyclictest0-21swapper/909:20:2611
418699102,6cyclictest0-21swapper/912:10:2511
418699102,5cyclictest0-21swapper/907:15:0011
418699100,1cyclictest11179-21awk08:00:0111
417599102,7cyclictest0-21swapper/709:45:009
417099103,4cyclictest0-21swapper/609:00:138
416699100,5cyclictest0-21swapper/511:20:267
416099100,9cyclictest8583-21irqrtprio10:00:176
416099100,9cyclictest0-21swapper/410:20:006
416099100,2cyclictest13818-21if_enp1s0.109:25:176
415699102,5cyclictest0-21swapper/310:15:005
415699102,5cyclictest0-21swapper/308:45:005
415699101,5cyclictest0-21swapper/311:15:005
415699101,5cyclictest0-21swapper/309:40:005
415699100,9cyclictest15571-21irqrtprio08:45:225
415199102,6cyclictest0-21swapper/212:15:004
415199102,5cyclictest0-21swapper/210:35:004
415199102,5cyclictest0-21swapper/210:35:004
415199101,5cyclictest31898-21kworker/u24:110:00:014
415199100,2cyclictest32261-21users09:05:224
414299102,5cyclictest0-21swapper/011:30:020
414299102,5cyclictest0-21swapper/011:25:000
414299102,5cyclictest0-21swapper/011:00:240
414299101,5cyclictest0-21swapper/012:35:010
275991100,1ptp4l18969-21timerandwakeup109:30:251
275991100,1ptp4l18968-21/usr/sbin/munin12:20:131
275991100,1ptp4l1760-21fschecks_time09:10:151
275991100,0ptp4l0-21swapper/111:30:261
275991100,0ptp4l0-21swapper/109:20:161
275991100,0ptp4l0-21swapper/108:25:201
9498090,0rtkit-daemon0-21swapper/311:38:135
41979990,8cyclictest12597-21grep11:30:143
41979990,8cyclictest0-21swapper/1109:15:203
41979990,1cyclictest0-21swapper/1111:00:133
41929993,5cyclictest0-21swapper/1009:15:002
41869998,1cyclictest0-21swapper/911:45:2011
41869998,1cyclictest0-21swapper/910:05:2311
41869998,1cyclictest0-21swapper/908:20:2411
41869998,1cyclictest0-21swapper/908:18:3311
41869996,2cyclictest0-21swapper/911:55:0111
41869992,6cyclictest0-21swapper/910:50:0111
41869990,9cyclictest0-21swapper/912:35:1111
41869990,9cyclictest0-21swapper/910:15:1711
41869990,9cyclictest0-21swapper/909:15:2311
41869990,9cyclictest0-21swapper/909:10:1611
41869990,9cyclictest0-21swapper/909:00:1411
41869990,9cyclictest0-21swapper/907:35:2511
41869990,8cyclictest0-21swapper/907:50:0011
41869990,1cyclictest0-21swapper/910:35:1511
41869990,1cyclictest0-21swapper/909:30:1911
41869990,1cyclictest0-21swapper/908:25:2011
41819998,1cyclictest0-21swapper/812:15:2410
41819996,2cyclictest18837-21turbostat08:10:0010
41819990,9cyclictest0-21swapper/810:05:1510
41759990,9cyclictest0-21swapper/708:15:149
41759990,1cyclictest0-21swapper/709:35:169
41709990,8cyclictest3086-21/usr/sbin/munin09:55:168
41709990,8cyclictest0-21swapper/611:50:138
41709990,8cyclictest0-21swapper/610:50:178
41709990,1cyclictest0-21swapper/607:25:148
41669997,1cyclictest0-21swapper/509:50:197
41669997,1cyclictest0-21swapper/508:10:117
41669993,4cyclictest0-21swapper/512:00:127
41669990,2cyclictest15249-21hddtemp_smartct12:15:167
41609992,6cyclictest20348-21crond11:00:016
41569998,1cyclictest0-21swapper/312:20:155
41569998,1cyclictest0-21swapper/310:20:215
41569998,1cyclictest0-21swapper/310:20:215
41569992,4cyclictest0-21swapper/310:40:015
41569992,4cyclictest0-21swapper/309:50:005
41569990,9cyclictest0-21swapper/311:00:155
41569990,8cyclictest15928-21fw_forwarded_lo10:10:185
41569990,8cyclictest15928-21fw_forwarded_lo10:10:175
41569990,1cyclictest0-21swapper/312:05:155
41569990,1cyclictest0-21swapper/309:30:175
41519998,1cyclictest0-21swapper/212:00:124
41519998,1cyclictest0-21swapper/210:00:214
41519997,1cyclictest6627-21/usr/sbin/munin07:10:214
41519992,4cyclictest0-21swapper/208:20:224
41519990,9cyclictest0-21swapper/211:35:144
41519990,8cyclictest6777-21expr08:35:124
41429997,1cyclictest0-21swapper/012:10:160
41429991,5cyclictest0-21swapper/009:50:010
41429991,5cyclictest0-21swapper/008:35:000
41429991,5cyclictest0-21swapper/007:35:000
3466291,6sleep90-21swapper/907:07:2211
27599190,1ptp4l3558-21cut09:55:151
27599190,1ptp4l21-21ksoftirqd/108:10:121
27599190,1ptp4l21-21ksoftirqd/108:00:091
27599190,0ptp4l0-21swapper/108:35:111
26017290,0sleep20-21swapper/209:40:204
9498080,0rtkit-daemon0-21swapper/912:28:5011
41979981,4cyclictest0-21swapper/1109:25:243
41979980,1cyclictest0-21swapper/1111:10:173
41929987,1cyclictest0-21swapper/1010:50:132
41929980,7cyclictest0-21swapper/1008:31:192
41929980,7cyclictest0-21swapper/1007:25:162
41929980,1cyclictest0-21swapper/1012:35:142
41929980,1cyclictest0-21swapper/1010:30:462
41929980,1cyclictest0-21swapper/1010:30:462
41869987,1cyclictest0-21swapper/912:15:2511
41869987,1cyclictest0-21swapper/911:40:1311
41869987,1cyclictest0-21swapper/910:50:1211
41869987,1cyclictest0-21swapper/908:45:2511
41869987,1cyclictest0-21swapper/908:00:0011
41869987,1cyclictest0-21swapper/907:40:2011
41869980,8cyclictest0-21swapper/910:40:1211
41869980,1cyclictest0-21swapper/907:25:2311
41869980,1cyclictest0-21swapper/907:20:2511
41819987,1cyclictest0-21swapper/810:25:2010
41819987,1cyclictest0-21swapper/810:25:2010
41819986,1cyclictest0-21swapper/810:45:1610
41819984,3cyclictest17096-21taskset08:47:0810
41819982,3cyclictest0-21swapper/808:50:0010
41819980,8cyclictest0-21swapper/811:30:2110
41819980,8cyclictest0-21swapper/809:30:2610
41819980,1cyclictest0-21swapper/810:50:1110
41819980,1cyclictest0-21swapper/807:25:1610
41759987,1cyclictest0-21swapper/710:45:269
41759980,8cyclictest0-21swapper/711:20:149
41759980,7cyclictest32605-21diskmemload12:39:279
41759980,7cyclictest0-21swapper/708:45:229
41759980,1cyclictest0-21swapper/711:15:209
41759980,1cyclictest0-21swapper/710:57:399
41759980,1cyclictest0-21swapper/710:05:219
41759980,1cyclictest0-21swapper/707:15:189
41709987,1cyclictest0-21swapper/610:25:218
41709987,1cyclictest0-21swapper/610:25:218
41709980,1cyclictest0-21swapper/611:45:008
41709980,1cyclictest0-21swapper/610:20:028
41709980,1cyclictest0-21swapper/610:20:018
41709980,1cyclictest0-21swapper/608:10:158
41709980,1cyclictest0-21swapper/607:49:348
41669987,1cyclictest0-21swapper/511:05:217
41669980,8cyclictest0-21swapper/508:20:137
41669980,7cyclictest32605-21diskmemload10:52:387
41669980,1cyclictest0-21swapper/508:15:247
41609987,1cyclictest0-21swapper/412:30:156
41609987,1cyclictest0-21swapper/409:50:246
41609980,8cyclictest0-21swapper/409:05:106
41609980,1cyclictest0-21swapper/408:40:226
41569987,1cyclictest0-21swapper/311:25:275
41569987,1cyclictest0-21swapper/310:30:145
41569987,1cyclictest0-21swapper/310:30:145
41569983,5cyclictest0-21swapper/307:30:005
41569980,8cyclictest0-21swapper/311:20:185
41569980,8cyclictest0-21swapper/307:25:255
41569980,8cyclictest0-21swapper/307:20:145
41569980,1cyclictest0-21swapper/311:30:215
41569980,1cyclictest0-21swapper/310:25:185
41569980,1cyclictest0-21swapper/310:25:175
41519987,1cyclictest0-21swapper/209:25:164
41519987,1cyclictest0-21swapper/208:45:224
41519981,4cyclictest0-21swapper/212:40:014
41519981,4cyclictest0-21swapper/207:50:014
41519980,8cyclictest0-21swapper/212:30:154
41519980,8cyclictest0-21swapper/211:55:134
41519980,8cyclictest0-21swapper/211:45:264
41519980,8cyclictest0-21swapper/207:45:154
41519980,7cyclictest248742sleep212:29:034
41519980,7cyclictest10215-21smartctl11:25:244
41519980,2cyclictest0-21swapper/209:50:254
41519980,1cyclictest0-21swapper/207:22:334
41429980,8cyclictest0-21swapper/012:25:130
41429980,8cyclictest0-21swapper/008:25:160
41429980,7cyclictest17844-21taskset10:11:450
41429980,7cyclictest17844-21taskset10:11:450
41429980,1cyclictest0-21swapper/008:50:140
41429980,1cyclictest0-21swapper/008:35:130
3418281,5sleep30-21swapper/307:06:415
27609180,7phc2sys0-21swapper/212:23:054
2686280,6sleep23106-21grep07:05:284
41979976,1cyclictest0-21swapper/1112:36:023
41979976,1cyclictest0-21swapper/1112:31:253
41979976,1cyclictest0-21swapper/1112:25:123
41979976,1cyclictest0-21swapper/1111:51:593
41979976,1cyclictest0-21swapper/1111:36:003
41979976,1cyclictest0-21swapper/1110:56:273
41979976,1cyclictest0-21swapper/1109:55:433
41979976,1cyclictest0-21swapper/1109:33:373
41979976,1cyclictest0-21swapper/1109:05:343
41979976,1cyclictest0-21swapper/1108:55:123
41979976,1cyclictest0-21swapper/1108:50:153
41979976,1cyclictest0-21swapper/1108:40:263
41979976,1cyclictest0-21swapper/1108:28:043
41979976,1cyclictest0-21swapper/1108:20:153
41979976,1cyclictest0-21swapper/1108:17:453
41979976,1cyclictest0-21swapper/1108:10:213
41979976,1cyclictest0-21swapper/1108:00:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional