You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 15:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Sat Nov 29, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991950,55ptp4l0-21swapper/107:06:531
94980900,85rtkit-daemon0-21swapper/1007:09:462
252992690,0sleep20-21swapper/210:40:234
252992690,0sleep20-21swapper/210:40:234
2095626759,6sleep50-21swapper/507:07:037
2091626656,8sleep40-21swapper/407:06:276
2105726453,9sleep110-21swapper/1107:08:333
2091926250,10sleep70-21swapper/707:06:299
2091826252,8sleep60-21swapper/607:06:288
2092526151,8sleep00-21swapper/007:06:340
267682600,0sleep30-21swapper/310:45:015
125802600,0sleep70-21swapper/708:20:129
284772590,0sleep80-21swapper/808:40:1610
2158299360,1cyclictest0-21swapper/1010:20:252
2158699231,16cyclictest17843-21diskmemload10:15:003
2155499220,8cyclictest0-21swapper/411:55:006
83402190,1sleep80-21swapper/810:20:1510
2155499192,13cyclictest0-21swapper/408:20:006
2154099192,16cyclictest0-21swapper/109:40:171
275991180,0ptp4l0-21swapper/108:35:141
275991170,0ptp4l0-21swapper/108:30:131
275991170,0ptp4l0-21swapper/108:29:221
275991160,1ptp4l21-21ksoftirqd/111:10:231
275991160,1ptp4l21-21ksoftirqd/109:45:191
275991160,0ptp4l0-21swapper/112:35:211
275991150,0ptp4l0-21swapper/111:35:211
275991150,0ptp4l0-21swapper/111:30:221
2158699153,7cyclictest0-21swapper/1109:20:013
2158699152,8cyclictest0-21swapper/1108:25:003
2157799151,13cyclictest0-21swapper/912:25:1911
2156799151,8cyclictest0-21swapper/709:20:009
2155499153,7cyclictest0-21swapper/411:30:216
2155499152,8cyclictest0-21swapper/408:05:256
21554991510,4cyclictest0-21swapper/409:55:196
2158699143,8cyclictest0-21swapper/1107:25:003
2158699143,7cyclictest0-21swapper/1111:10:013
2158699142,7cyclictest0-21swapper/1109:45:003
2158699142,7cyclictest0-21swapper/1109:30:003
2158299140,13cyclictest0-21swapper/1007:20:012
2156799142,8cyclictest0-21swapper/710:35:009
2156799142,7cyclictest0-21swapper/710:05:009
2156799142,7cyclictest0-21swapper/708:15:009
2156799142,7cyclictest0-21swapper/707:20:009
2155899142,8cyclictest0-21swapper/511:10:257
2155899142,7cyclictest0-21swapper/509:35:007
2155899141,8cyclictest0-21swapper/509:40:007
2155499142,7cyclictest0-21swapper/409:40:236
275991130,1ptp4l21-21ksoftirqd/109:15:411
2158699132,7cyclictest0-21swapper/1111:05:003
2158699132,7cyclictest0-21swapper/1111:05:003
2158699132,6cyclictest0-21swapper/1110:40:253
2158699132,6cyclictest0-21swapper/1110:40:253
2157799130,9cyclictest0-21swapper/911:35:0111
2156799132,6cyclictest0-21swapper/710:25:009
2155899132,7cyclictest0-21swapper/509:05:007
2154999132,6cyclictest0-21swapper/307:55:015
2154099131,8cyclictest0-21swapper/108:00:001
2154099131,7cyclictest0-21swapper/109:10:001
275991120,3ptp4l7980-21tune2fs08:55:151
275991120,1ptp4l21-21ksoftirqd/112:25:131
275991120,1ptp4l21-21ksoftirqd/111:05:131
275991120,1ptp4l21-21ksoftirqd/109:50:161
275991120,0ptp4l0-21swapper/110:11:081
275991120,0ptp4l0-21swapper/108:40:191
275991120,0ptp4l0-21swapper/108:16:071
2158699121,7cyclictest0-21swapper/1111:25:003
21577991210,1cyclictest0-21swapper/910:10:0011
2157799120,6cyclictest1576-21awk07:25:0011
2156799123,5cyclictest0-21swapper/709:10:249
2156799123,5cyclictest0-21swapper/708:30:019
2155899122,7cyclictest0-21swapper/510:45:007
2155899122,7cyclictest0-21swapper/510:45:007
2155499122,6cyclictest0-21swapper/407:45:246
2154999123,5cyclictest0-21swapper/311:30:005
202322120,5sleep30-21swapper/307:05:265
191591120,2getstats27-21ksoftirqd/207:15:174
275991110,3ptp4l0-21swapper/107:15:031
275991110,2ptp4l847-21cpuspeed_turbos08:05:141
275991110,2ptp4l3909-21irqrtprio10:55:191
275991110,2ptp4l3909-21irqrtprio10:55:181
275991110,2ptp4l30566-21irqrtprio08:00:201
275991110,2ptp4l2149-21diskstats09:30:141
275991110,1ptp4l21-21ksoftirqd/111:50:151
275991110,1ptp4l0-21swapper/110:55:001
275991110,1ptp4l0-21swapper/110:55:001
275991110,1ptp4l0-21swapper/110:05:201
275991110,1ptp4l0-21swapper/107:40:161
275991110,1ptp4l0-21swapper/107:25:251
275991110,0ptp4l0-21swapper/112:20:131
275991110,0ptp4l0-21swapper/110:43:111
275991110,0ptp4l0-21swapper/110:43:111
275991110,0ptp4l0-21swapper/110:28:581
275991110,0ptp4l0-21swapper/110:28:581
275991110,0ptp4l0-21swapper/109:00:141
233252110,0sleep50-21swapper/510:40:147
233252110,0sleep50-21swapper/510:40:147
2157799110,1cyclictest0-21swapper/908:45:0111
2157399110,10cyclictest0-21swapper/810:50:0010
2157399110,10cyclictest0-21swapper/810:50:0010
2156799118,2cyclictest1871-21strings10:10:209
2156799111,6cyclictest28783-21kworker/u24:212:35:029
2155899112,6cyclictest0-21swapper/507:45:017
2155499110,10cyclictest0-21swapper/408:35:016
2154599111,6cyclictest0-21swapper/208:50:004
2154099112,6cyclictest0-21swapper/108:50:001
210672112,7sleep80-21swapper/807:08:4210
275991100,2ptp4l21-21ksoftirqd/110:15:161
275991100,2ptp4l21-21ksoftirqd/109:05:201
275991100,1ptp4l86012sleep111:00:241
275991100,1ptp4l86012sleep111:00:241
275991100,1ptp4l6244-21cut08:10:211
275991100,1ptp4l32528-21fw_forwarded_lo12:15:171
275991100,1ptp4l21-21ksoftirqd/110:35:151
275991100,1ptp4l21-21ksoftirqd/110:35:141
275991100,1ptp4l21-21ksoftirqd/107:34:401
275991100,1ptp4l15208-21uptime08:20:251
275991100,1ptp4l0-21swapper/111:55:191
275991100,0ptp4l0-21swapper/112:10:261
275991100,0ptp4l0-21swapper/111:15:121
275991100,0ptp4l0-21swapper/110:45:221
275991100,0ptp4l0-21swapper/110:45:211
275991100,0ptp4l0-21swapper/110:30:151
275991100,0ptp4l0-21swapper/109:20:191
275991100,0ptp4l0-21swapper/108:50:141
275991100,0ptp4l0-21swapper/107:50:161
275991100,0ptp4l0-21swapper/107:35:221
2158699108,1cyclictest0-21swapper/1107:15:183
2158699101,8cyclictest6332-21fschecks_time09:35:173
2158299102,5cyclictest0-21swapper/1008:45:012
2158299100,9cyclictest23731-21/usr/sbin/munin12:05:202
2157799108,1cyclictest0-21swapper/911:24:5511
2157799103,6cyclictest0-21swapper/912:05:0011
2157799103,6cyclictest0-21swapper/910:40:0011
2157799103,6cyclictest0-21swapper/910:40:0011
2157799103,6cyclictest0-21swapper/908:15:0011
2157799102,7cyclictest0-21swapper/911:55:0111
2157799102,7cyclictest0-21swapper/909:25:2611
2157799100,9cyclictest0-21swapper/909:35:1611
2157799100,2cyclictest0-21swapper/908:50:1911
2157399100,1cyclictest0-21swapper/812:05:2810
2157399100,1cyclictest0-21swapper/810:35:2110
2157399100,1cyclictest0-21swapper/810:35:2010
2156799100,9cyclictest8956-21cut08:55:199
2156799100,1cyclictest0-21swapper/710:45:219
2156799100,1cyclictest0-21swapper/710:45:219
2155899107,2cyclictest10179-21turbostat12:30:007
2155499107,2cyclictest23118-21/usr/sbin/munin10:40:156
2155499107,2cyclictest23118-21/usr/sbin/munin10:40:146
2155499102,4cyclictest0-21swapper/409:30:006
2154999103,4cyclictest0-21swapper/311:25:015
2154999102,5cyclictest0-21swapper/312:35:005
2154999102,5cyclictest0-21swapper/312:05:005
2154999102,5cyclictest0-21swapper/308:55:005
2154999101,7cyclictest28783-21kworker/u24:212:15:005
2154599100,9cyclictest0-21swapper/211:45:234
2154599100,6cyclictest0-21swapper/210:00:014
2154599100,6cyclictest0-21swapper/209:50:004
2154599100,5cyclictest0-21swapper/212:20:014
7027290,0sleep40-21swapper/411:00:166
7027290,0sleep40-21swapper/411:00:156
4903290,0sleep00-21swapper/012:20:200
27599190,1ptp4l21-21ksoftirqd/112:05:181
27599190,1ptp4l21-21ksoftirqd/109:35:001
27599190,1ptp4l0-21swapper/111:20:131
27599190,0ptp4l0-21swapper/112:30:131
27599190,0ptp4l0-21swapper/112:00:141
27599190,0ptp4l0-21swapper/111:45:161
27599190,0ptp4l0-21swapper/111:40:171
27599190,0ptp4l0-21swapper/111:25:161
27599190,0ptp4l0-21swapper/110:20:141
27599190,0ptp4l0-21swapper/110:00:151
27599190,0ptp4l0-21swapper/109:55:161
27599190,0ptp4l0-21swapper/109:25:151
27599190,0ptp4l0-21swapper/107:45:151
27599190,0ptp4l0-21swapper/107:20:151
27599190,0ptp4l0-21swapper/107:10:151
215869997,1cyclictest0-21swapper/1108:40:173
215869990,9cyclictest0-21swapper/1109:00:173
215869990,1cyclictest0-21swapper/1107:45:133
215829998,1cyclictest0-21swapper/1008:19:552
215829997,1cyclictest972-21timerwakeupswit10:50:252
215829997,1cyclictest972-21timerwakeupswit10:50:252
215829992,5cyclictest0-21swapper/1007:55:242
215829990,8cyclictest28346-21df12:10:162
215779998,1cyclictest0-21swapper/908:30:2211
215779998,1cyclictest0-21swapper/907:35:1811
215779994,4cyclictest0-21swapper/908:30:0011
215779992,6cyclictest0-21swapper/912:30:2511
215779992,6cyclictest0-21swapper/911:05:0011
215779992,6cyclictest0-21swapper/911:05:0011
215779992,6cyclictest0-21swapper/907:50:0011
215779991,6cyclictest0-21swapper/912:25:0011
215779991,5cyclictest0-21swapper/912:15:2611
215779990,9cyclictest0-21swapper/908:15:1411
215779990,1cyclictest0-21swapper/909:50:1911
215779990,1cyclictest0-21swapper/908:35:2011
215739997,1cyclictest0-21swapper/810:15:1510
215739996,2cyclictest0-21swapper/809:20:1710
215739995,3cyclictest28523-21fschecks_time12:10:1610
215739990,1cyclictest0-21swapper/808:35:1410
215679998,1cyclictest0-21swapper/708:30:129
215679998,1cyclictest0-21swapper/707:25:209
215679996,2cyclictest22389-21/usr/sbin/munin12:00:259
215679992,6cyclictest0-21swapper/710:10:019
215679992,5cyclictest0-21swapper/709:25:009
215679990,9cyclictest0-21swapper/711:20:209
215679990,9cyclictest0-21swapper/711:10:199
215679990,1cyclictest0-21swapper/709:35:009
215639998,1cyclictest0-21swapper/612:05:018
215639997,2cyclictest28783-21kworker/u24:212:20:008
215639997,1cyclictest0-21swapper/608:15:008
215639997,1cyclictest0-21swapper/607:15:248
215639996,2cyclictest0-21swapper/611:50:128
215639990,1cyclictest0-21swapper/609:15:238
215639990,1cyclictest0-21swapper/608:00:208
215589996,2cyclictest26871-21sed11:25:207
215589991,5cyclictest0-21swapper/512:35:017
215589991,5cyclictest0-21swapper/508:20:007
215589991,4cyclictest0-21swapper/511:40:007
215589990,1cyclictest0-21swapper/510:00:247
215549990,8cyclictest23579-21/usr/sbin/munin09:15:176
215549990,8cyclictest0-21swapper/411:55:156
215499992,4cyclictest0-21swapper/308:35:015
215499991,5cyclictest0-21swapper/308:35:005
215499991,5cyclictest0-21swapper/308:05:005
215499991,5cyclictest0-21swapper/307:40:255
215499991,5cyclictest0-21swapper/307:15:245
215459998,1cyclictest0-21swapper/208:15:164
215459996,2cyclictest9423-21turbostat11:05:004
215459996,2cyclictest9423-21turbostat11:05:004
215459990,6cyclictest0-21swapper/212:00:004
215369998,1cyclictest0-21swapper/010:50:250
215369998,1cyclictest0-21swapper/010:50:250
215369998,1cyclictest0-21swapper/009:45:140
215369990,9cyclictest0-21swapper/011:05:200
215369990,8cyclictest0-21swapper/009:40:140
20908291,6sleep90-21swapper/907:06:1911
9498080,0rtkit-daemon0-21swapper/412:15:506
215869987,1cyclictest0-21swapper/1112:20:153
215869987,1cyclictest0-21swapper/1112:00:143
215869987,1cyclictest0-21swapper/1111:55:263
215869987,1cyclictest0-21swapper/1110:05:083
215869981,6cyclictest0-21swapper/1108:50:183
215869980,8cyclictest0-21swapper/1108:10:203
215869980,8cyclictest0-21swapper/1107:55:203
215869980,1cyclictest0-21swapper/1111:37:203
215869980,1cyclictest0-21swapper/1107:50:163
215869980,1cyclictest0-21swapper/1107:25:133
215829986,2cyclictest24475-21ptp4l09:15:212
215829980,1cyclictest0-21swapper/1012:25:142
215829980,1cyclictest0-21swapper/1010:40:262
215829980,1cyclictest0-21swapper/1010:40:262
215829980,1cyclictest0-21swapper/1010:15:162
215829980,1cyclictest0-21swapper/1007:30:232
215829980,1cyclictest0-21swapper/1007:10:272
215779987,1cyclictest0-21swapper/912:05:1811
215779982,5cyclictest0-21swapper/909:50:0011
215779980,1cyclictest0-21swapper/910:50:2411
215779980,1cyclictest0-21swapper/910:50:2411
215779980,1cyclictest0-21swapper/910:00:2111
215779980,1cyclictest0-21swapper/909:01:1911
215739987,1cyclictest0-21swapper/811:25:1510
215739987,1cyclictest0-21swapper/811:15:1610
215739987,1cyclictest0-21swapper/809:05:2710
215739987,1cyclictest0-21swapper/809:00:2610
215739980,8cyclictest0-21swapper/810:50:2510
215739980,8cyclictest0-21swapper/810:50:2510
215739980,8cyclictest0-21swapper/807:10:1210
215739980,7cyclictest29073-21/usr/sbin/munin11:30:1910
215739980,1cyclictest0-21swapper/811:50:1810
215739980,1cyclictest0-21swapper/808:30:1710
215739980,1cyclictest0-21swapper/808:10:1710
215739980,1cyclictest0-21swapper/807:55:2110
215679987,1cyclictest0-21swapper/711:30:199
215679987,1cyclictest0-21swapper/708:45:159
215679987,1cyclictest0-21swapper/708:39:479
215679987,1cyclictest0-21swapper/708:05:189
215679980,8cyclictest0-21swapper/709:35:239
215679980,7cyclictest32068-21pool08:00:289
215679980,1cyclictest0-21swapper/712:10:019
215679980,1cyclictest0-21swapper/712:05:289
215639987,1cyclictest0-21swapper/611:55:248
215639987,1cyclictest0-21swapper/611:20:148
215639987,1cyclictest0-21swapper/610:45:148
215639987,1cyclictest0-21swapper/610:45:148
215639987,1cyclictest0-21swapper/608:50:208
215639987,1cyclictest0-21swapper/608:15:258
215639986,1cyclictest0-21swapper/610:55:158
215639986,1cyclictest0-21swapper/610:55:158
215639986,1cyclictest0-21swapper/610:29:568
215639986,1cyclictest0-21swapper/610:29:568
215639982,4cyclictest3904-21gdbus07:20:128
215639980,8cyclictest0-21swapper/610:00:138
215639980,8cyclictest0-21swapper/609:55:208
215639980,8cyclictest0-21swapper/608:45:238
215639980,8cyclictest0-21swapper/608:20:238
215639980,1cyclictest0-21swapper/612:35:508
215639980,1cyclictest0-21swapper/612:25:138
215639980,1cyclictest0-21swapper/609:40:178
215639980,1cyclictest0-21swapper/609:07:338
215639980,0cyclictest0-21swapper/608:05:188
215589987,1cyclictest0-21swapper/512:00:197
215589980,8cyclictest0-21swapper/509:25:267
215589980,7cyclictest0-21swapper/509:08:517
215589980,1cyclictest0-21swapper/510:20:167
215549980,7cyclictest0-21swapper/411:10:166
215549980,7cyclictest0-21swapper/410:45:176
215549980,7cyclictest0-21swapper/410:45:176
215549980,1cyclictest0-21swapper/409:50:026
215549980,1cyclictest0-21swapper/409:20:186
215549980,1cyclictest0-21swapper/408:55:156
215549980,1cyclictest0-21swapper/407:50:196
215499987,1cyclictest0-21swapper/309:45:175
215499987,1cyclictest0-21swapper/307:30:205
215499985,2cyclictest5751-21perl08:10:185
215499982,4cyclictest0-21swapper/311:55:005
215499982,4cyclictest0-21swapper/307:55:005
215499980,8cyclictest0-21swapper/310:20:175
215499980,1cyclictest0-21swapper/312:07:215
215499980,1cyclictest0-21swapper/311:30:155
215499980,1cyclictest0-21swapper/310:15:215
215499980,1cyclictest0-21swapper/309:50:185
215499980,1cyclictest0-21swapper/308:05:005
215459987,1cyclictest0-21swapper/210:20:014
215459986,2cyclictest644-21fschecks_count10:10:164
215459980,8cyclictest0-21swapper/212:20:164
215459980,8cyclictest0-21swapper/212:05:294
215459980,7cyclictest3852-21chrt08:07:544
215459980,7cyclictest27273-21hddtemp_smartct09:20:164
215459980,7cyclictest18572-21/usr/sbin/munin09:50:194
215459980,5cyclictest0-21swapper/207:35:004
215459980,3cyclictest11857-21/usr/sbin/munin11:05:214
215459980,1cyclictest19929-21diskstats10:35:154
215459980,1cyclictest19929-21diskstats10:35:154
215459980,1cyclictest0-21swapper/211:50:184
215459980,1cyclictest0-21swapper/209:00:254
215459980,1cyclictest0-21swapper/208:10:274
215369987,1cyclictest0-21swapper/009:10:150
215369987,1cyclictest0-21swapper/007:50:190
215369980,8cyclictest0-21swapper/012:00:210
215369980,8cyclictest0-21swapper/011:45:270
215369980,8cyclictest0-21swapper/008:55:010
27609171,4phc2sys1009-21snmpd07:06:224
215869976,1cyclictest0-21swapper/1112:28:293
215869976,1cyclictest0-21swapper/1111:50:303
215869976,1cyclictest0-21swapper/1111:45:123
215869976,1cyclictest0-21swapper/1110:45:223
215869976,1cyclictest0-21swapper/1110:45:213
215869976,1cyclictest0-21swapper/1110:30:433
215869976,1cyclictest0-21swapper/1110:01:283
215869976,1cyclictest0-21swapper/1109:45:073
215869976,1cyclictest0-21swapper/1109:05:243
215869976,1cyclictest0-21swapper/1108:37:523
215869976,1cyclictest0-21swapper/1108:30:153
215869976,1cyclictest0-21swapper/1108:02:313
215869976,1cyclictest0-21swapper/1107:35:153
215869976,1cyclictest0-21swapper/1107:12:513
215869974,2cyclictest31680-21cpuspeed_turbos08:45:133
215869970,7cyclictest0-21swapper/1112:35:223
215869970,7cyclictest0-21swapper/1112:05:013
215869970,7cyclictest0-21swapper/1111:32:553
215869970,7cyclictest0-21swapper/1111:25:033
215869970,7cyclictest0-21swapper/1111:10:163
215869970,7cyclictest0-21swapper/1110:50:323
215869970,7cyclictest0-21swapper/1110:50:323
215869970,7cyclictest0-21swapper/1110:25:203
215869970,7cyclictest0-21swapper/1110:25:203
215869970,7cyclictest0-21swapper/1110:21:393
215869970,7cyclictest0-21swapper/1110:17:133
215869970,7cyclictest0-21swapper/1109:55:533
215869970,7cyclictest0-21swapper/1109:30:043
215869970,7cyclictest0-21swapper/1109:10:013
215869970,7cyclictest0-21swapper/1108:16:213
215869970,7cyclictest0-21swapper/1108:06:363
215869970,1cyclictest0-21swapper/1112:30:063
215869970,1cyclictest0-21swapper/1112:15:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional