You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 23:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Mon Dec 01, 2025 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991920,78ptp4l0-21swapper/107:05:191
788926654,10sleep50-21swapper/507:09:027
772226555,8sleep00-21swapper/007:06:310
779526452,9sleep60-21swapper/607:07:378
778726453,9sleep110-21swapper/1107:07:303
773126455,7sleep70-21swapper/707:06:389
790126352,9sleep40-21swapper/407:09:136
298712630,0sleep00-21swapper/011:45:250
106062620,0sleep90-21swapper/912:05:1211
768026049,9sleep100-21swapper/1007:05:532
147692600,0sleep80-21swapper/807:15:1510
206372590,0sleep30-21swapper/310:10:185
31212550,0sleep30-21swapper/311:10:255
31212550,0sleep30-21swapper/311:10:245
169152550,0sleep80-21swapper/809:25:1910
60732540,0sleep80-21swapper/807:45:1910
833399390,2cyclictest0-21swapper/1009:05:262
830499232,7cyclictest0-21swapper/411:10:006
830499232,7cyclictest0-21swapper/411:10:006
830499232,16cyclictest0-21swapper/407:25:006
130652210,0sleep40-21swapper/411:25:156
830499201,15cyclictest0-21swapper/409:05:006
46312200,0sleep90-21swapper/910:30:2611
833999180,3cyclictest0-21swapper/1108:40:173
830099183,8cyclictest0-21swapper/307:50:005
829599182,10cyclictest0-21swapper/207:35:004
275991170,1ptp4l21-21ksoftirqd/109:45:181
833999160,16cyclictest0-21swapper/1107:39:563
832899163,8cyclictest0-21swapper/908:55:0011
832899162,10cyclictest2995-21kworker/u24:108:15:0011
832899161,3cyclictest0-21swapper/910:50:1611
831499163,8cyclictest0-21swapper/611:00:018
831499161,11cyclictest2995-21kworker/u24:107:45:008
8310991612,2cyclictest0-21swapper/508:15:187
830099163,8cyclictest0-21swapper/307:10:025
830099162,7cyclictest0-21swapper/309:55:245
830099162,7cyclictest0-21swapper/309:10:005
829599162,9cyclictest0-21swapper/212:15:254
829599162,10cyclictest0-21swapper/208:45:004
828799163,8cyclictest0-21swapper/110:25:001
828799162,9cyclictest0-21swapper/109:55:001
8287991613,2cyclictest0-21swapper/107:45:181
325462160,0sleep30-21swapper/308:20:175
275991160,1ptp4l21-21ksoftirqd/109:25:241
275991160,0ptp4l0-21swapper/107:55:161
833399153,7cyclictest0-21swapper/1010:30:012
833399153,7cyclictest0-21swapper/1007:20:232
832899153,6cyclictest0-21swapper/908:50:0011
832399152,8cyclictest0-21swapper/810:25:0010
831999152,8cyclictest0-21swapper/711:20:239
831999150,8cyclictest0-21swapper/711:45:009
831499153,8cyclictest0-21swapper/612:15:008
831099153,7cyclictest0-21swapper/510:00:007
830099153,7cyclictest0-21swapper/308:10:005
830099153,7cyclictest0-21swapper/308:00:005
830099152,8cyclictest0-21swapper/310:40:005
830099152,7cyclictest0-21swapper/309:50:245
828799153,8cyclictest0-21swapper/111:10:001
828799153,8cyclictest0-21swapper/111:10:001
828799152,8cyclictest0-21swapper/108:35:001
828299153,8cyclictest0-21swapper/012:20:010
8282991511,2cyclictest0-21swapper/010:05:200
78662157,6sleep80-21swapper/807:08:3910
275991150,0ptp4l0-21swapper/110:10:221
270762150,0sleep110-21swapper/1111:45:003
8339991412,2cyclictest0-21swapper/1112:15:183
8333991413,1cyclictest0-21swapper/1010:45:182
832899142,8cyclictest0-21swapper/909:25:0011
832899142,7cyclictest0-21swapper/908:05:0111
832899142,7cyclictest0-21swapper/907:55:0011
831999142,8cyclictest0-21swapper/708:55:009
831999142,7cyclictest0-21swapper/712:20:019
831999142,7cyclictest0-21swapper/710:50:009
831999142,7cyclictest0-21swapper/710:05:009
831999142,7cyclictest0-21swapper/709:50:009
831999142,7cyclictest0-21swapper/707:35:009
831999142,7cyclictest0-21swapper/707:20:019
831999140,8cyclictest0-21swapper/707:15:019
831999140,13cyclictest11506-21munin-run07:55:009
831499140,4cyclictest0-21swapper/608:20:008
831099142,8cyclictest0-21swapper/511:55:007
831099142,7cyclictest0-21swapper/511:39:127
830099143,7cyclictest0-21swapper/311:00:005
830099142,8cyclictest0-21swapper/307:30:005
830099141,7cyclictest0-21swapper/311:25:015
829599142,8cyclictest0-21swapper/207:40:234
828799143,5cyclictest0-21swapper/111:35:221
828799142,8cyclictest0-21swapper/110:15:241
828799142,7cyclictest0-21swapper/112:00:241
828299143,7cyclictest0-21swapper/010:35:010
828299143,6cyclictest0-21swapper/008:45:280
828299142,7cyclictest0-21swapper/008:40:000
828299142,7cyclictest0-21swapper/008:00:230
275991140,4ptp4l0-21swapper/110:55:001
275991140,0ptp4l0-21swapper/112:05:181
275991140,0ptp4l0-21swapper/111:57:111
275991140,0ptp4l0-21swapper/107:40:171
833399133,6cyclictest0-21swapper/1008:20:002
832899132,7cyclictest0-21swapper/907:50:0011
832399132,7cyclictest0-21swapper/810:20:0010
831999132,6cyclictest0-21swapper/710:20:249
831499137,5cyclictest0-21swapper/610:10:158
831499134,2cyclictest29615-21aten2_r3power_c09:00:118
831499130,2cyclictest2845-21irqstats12:35:188
831499130,12cyclictest0-21swapper/610:10:008
831099132,6cyclictest0-21swapper/511:00:017
830499132,7cyclictest0-21swapper/408:35:276
830099131,7cyclictest0-21swapper/307:30:275
828799132,8cyclictest0-21swapper/107:10:251
275991130,0ptp4l0-21swapper/111:30:281
275991130,0ptp4l0-21swapper/111:05:581
275991130,0ptp4l0-21swapper/111:05:571
275991130,0ptp4l0-21swapper/110:35:141
275991130,0ptp4l0-21swapper/109:10:111
832899123,6cyclictest0-21swapper/908:20:0011
832899122,6cyclictest0-21swapper/911:45:0011
832399122,7cyclictest0-21swapper/808:05:0110
8319991211,1cyclictest0-21swapper/707:45:189
831999120,11cyclictest6132-21munin-run12:00:019
831499123,2cyclictest15340-21latency_hist08:00:008
831499122,3cyclictest8610-21cut07:50:138
831499122,3cyclictest21669-21cut07:25:158
831499120,7cyclictest0-21swapper/607:20:008
831499120,11cyclictest0-21swapper/609:48:588
831099121,8cyclictest2995-21kworker/u24:110:00:017
831099121,7cyclictest0-21swapper/510:25:267
8310991210,1cyclictest0-21swapper/511:15:007
8310991210,1cyclictest0-21swapper/511:15:007
830499121,6cyclictest0-21swapper/412:30:246
828799122,6cyclictest0-21swapper/112:10:001
828799122,6cyclictest0-21swapper/111:45:001
828799122,6cyclictest0-21swapper/108:15:001
275991120,3ptp4l0-21swapper/107:35:141
275991120,2ptp4l466-21latency_hist09:05:001
275991120,0ptp4l0-21swapper/112:35:581
275991120,0ptp4l0-21swapper/111:03:231
275991120,0ptp4l0-21swapper/111:03:231
275991120,0ptp4l0-21swapper/110:40:321
275991120,0ptp4l0-21swapper/108:02:521
275991120,0ptp4l0-21swapper/107:50:141
275991120,0ptp4l0-21swapper/107:15:021
833999112,5cyclictest0-21swapper/1110:40:003
833999110,3cyclictest16262-21fw_conntrack10:05:183
833999110,10cyclictest13811-21munin-run12:10:003
833399110,2cyclictest14688-21expr08:40:152
833399110,10cyclictest15343-21munin-run11:30:012
832899112,6cyclictest0-21swapper/910:10:0111
832899110,10cyclictest0-21swapper/911:55:0111
832899110,10cyclictest0-21swapper/911:00:1611
832899110,10cyclictest0-21swapper/911:00:1511
832899110,10cyclictest0-21swapper/910:25:0011
832899110,10cyclictest0-21swapper/909:45:2011
832899110,10cyclictest0-21swapper/909:15:2711
831999119,1cyclictest0-21swapper/707:25:019
831999112,6cyclictest0-21swapper/712:40:009
831999112,5cyclictest0-21swapper/709:20:249
831499114,2cyclictest4963-21open_inodes08:25:208
831499114,2cyclictest1020-21runrttasks09:06:478
831499113,7cyclictest0-21swapper/612:00:018
831499113,3cyclictest4670-21fschecks_time09:50:158
831499113,2cyclictest28923-21aten2_r3power_p07:35:128
831499113,2cyclictest19065-21irqrtprio10:50:178
831499113,2cyclictest13799-21cpuspeed10:45:138
831499112,3cyclictest26566-21sed07:30:208
831499111,3cyclictest0-21swapper/608:50:158
831499110,7cyclictest0-21swapper/611:09:148
831499110,7cyclictest0-21swapper/611:09:138
831499110,6cyclictest0-21swapper/611:15:008
831499110,6cyclictest0-21swapper/611:15:008
831499110,6cyclictest0-21swapper/609:30:368
831499110,1cyclictest0-21swapper/610:19:248
831499110,1cyclictest0-21swapper/609:12:028
831499110,1cyclictest0-21swapper/608:30:318
830499112,6cyclictest0-21swapper/411:10:256
830499112,6cyclictest0-21swapper/411:10:246
830499112,5cyclictest14850irq/20-ata_piix12:25:016
829599119,1cyclictest17640-21turbostat08:49:554
829599112,6cyclictest0-21swapper/210:55:004
829599112,6cyclictest0-21swapper/208:30:004
829599112,6cyclictest0-21swapper/208:25:004
829599110,5cyclictest0-21swapper/209:40:254
829599110,3cyclictest20899-21expr09:30:124
829599110,10cyclictest20042-21cat09:30:004
275991110,5ptp4l0-21swapper/109:39:261
275991110,3ptp4l0-21swapper/107:25:241
275991110,2ptp4l24767-21latency_hist11:00:001
275991110,2ptp4l21597-21latency_hist12:20:011
275991110,2ptp4l16110-21latency_hist09:25:001
275991110,2ptp4l12311-21latency08:35:211
275991110,1ptp4l0-21swapper/111:25:161
275991110,1ptp4l0-21swapper/111:15:201
275991110,1ptp4l0-21swapper/111:15:191
275991110,1ptp4l0-21swapper/108:15:191
275991110,0ptp4l0-21swapper/107:22:471
94980100,0rtkit-daemon0-21swapper/209:19:294
833999102,5cyclictest0-21swapper/1109:20:003
833999102,5cyclictest0-21swapper/1108:25:003
833999100,4cyclictest0-21swapper/1110:00:003
833399101,6cyclictest0-21swapper/1007:30:002
832899107,2cyclictest1569-21hddtemp_smartct07:40:1611
831999101,5cyclictest0-21swapper/709:35:019
831999100,9cyclictest29822-21irqstats09:40:189
831999100,9cyclictest18961-21df_inode08:45:189
831999100,5cyclictest0-21swapper/710:54:569
831999100,1cyclictest0-21swapper/712:00:129
831499108,1cyclictest0-21swapper/607:20:178
831499103,6cyclictest0-21swapper/612:30:008
831499103,2cyclictest8761-21if_enp1s009:55:168
831499103,2cyclictest2171-21cpuspeed_turbos10:30:168
831499103,2cyclictest17698-21idleruntime-cro08:45:008
831499103,2cyclictest10213-21latency_hist12:05:018
831499102,3cyclictest25243-21irqstats11:40:208
831499102,2cyclictest31320-21cpuspeed08:20:138
831499101,8cyclictest2995-21kworker/u24:109:25:008
831499100,9cyclictest0-21swapper/609:26:248
831499100,9cyclictest0-21swapper/608:40:008
831499100,7cyclictest0-21swapper/611:25:488
831499100,7cyclictest0-21swapper/610:42:018
831499100,7cyclictest0-21swapper/610:28:528
831499100,7cyclictest0-21swapper/610:24:538
831499100,7cyclictest0-21swapper/610:04:498
831499100,7cyclictest0-21swapper/608:56:068
831499100,6cyclictest0-21swapper/610:37:118
831499100,1cyclictest23815-21cpuspeed_turbos08:10:168
831499100,1cyclictest0-21swapper/612:17:538
831499100,1cyclictest0-21swapper/612:05:378
831499100,1cyclictest0-21swapper/611:31:518
831499100,1cyclictest0-21swapper/611:21:288
831499100,1cyclictest0-21swapper/611:01:098
831499100,1cyclictest0-21swapper/611:01:098
831499100,10cyclictest0-21swapper/609:41:558
831099108,1cyclictest0-21swapper/509:00:007
831099100,9cyclictest0-21swapper/507:40:007
831099100,1cyclictest0-21swapper/507:25:207
830499100,9cyclictest0-21swapper/408:30:256
830499100,2cyclictest0-21swapper/410:55:206
830099107,2cyclictest0-21swapper/307:15:015
830099100,9cyclictest0-21swapper/309:25:235
829599106,3cyclictest30766-21cut11:50:014
829599102,5cyclictest0-21swapper/210:10:004
828799109,1cyclictest0-21swapper/109:55:121
828299107,2cyclictest0-21swapper/007:25:140
828299100,9cyclictest13469-21users12:05:260
76792105,4sleep90-21swapper/907:05:5211
65922103,5sleep30-21swapper/307:05:225
275991100,3ptp4l19784-21/usr/sbin/munin08:05:161
275991100,1ptp4l9778-21df09:15:221
275991100,1ptp4l454-21latency_hist12:35:001
275991100,1ptp4l31431-21cut10:25:171
275991100,1ptp4l28896-21diskstats09:40:141
275991100,1ptp4l27246-21if_p34p112:25:181
275991100,1ptp4l0-21swapper/110:05:001
275991100,1ptp4l0-21swapper/109:10:001
275991100,1ptp4l0-21swapper/108:20:251
275991100,1ptp4l0-21swapper/107:30:201
275991100,0ptp4l0-21swapper/111:20:171
275991100,0ptp4l0-21swapper/108:49:441
275991100,0ptp4l0-21swapper/108:25:141
83399998,1cyclictest0-21swapper/1107:15:033
83399997,1cyclictest0-21swapper/1107:10:133
83399990,2cyclictest7477-21sort12:00:133
83399990,1cyclictest24421-21switchtime08:50:233
83339990,8cyclictest0-21swapper/1012:25:172
83339990,1cyclictest0-21swapper/1012:20:172
83339990,1cyclictest0-21swapper/1008:55:122
83289998,1cyclictest0-21swapper/908:35:1611
83239998,1cyclictest0-21swapper/809:55:1710
83239997,1cyclictest0-21swapper/812:08:0610
83239996,2cyclictest0-21swapper/811:05:1710
83239996,2cyclictest0-21swapper/811:05:1610
83239990,8cyclictest0-21swapper/810:30:2310
83239990,1cyclictest0-21swapper/811:20:0110
83239990,1cyclictest0-21swapper/811:20:0010
83199997,1cyclictest0-21swapper/711:15:209
83199997,1cyclictest0-21swapper/711:15:199
83199991,5cyclictest0-21swapper/710:25:009
83199990,1cyclictest0-21swapper/708:30:019
83149993,2cyclictest31455-21latency12:30:188
83149993,2cyclictest30159-21unixbench-2d11:45:268
83149993,2cyclictest11376-21timerwakeupswit07:10:258
83149990,9cyclictest0-21swapper/608:49:588
83149990,7cyclictest0-21swapper/611:37:558
83149990,6cyclictest0-21swapper/611:17:108
83149990,6cyclictest0-21swapper/611:17:098
83149990,6cyclictest0-21swapper/608:07:558
83109998,1cyclictest0-21swapper/507:50:257
83109990,9cyclictest0-21swapper/508:00:177
83109990,8cyclictest0-21swapper/512:05:197
83049991,5cyclictest14850irq/20-ata_piix10:45:006
83049990,8cyclictest0-21swapper/410:01:106
83049990,8cyclictest0-21swapper/407:45:156
83009997,1cyclictest0-21swapper/311:50:015
83009990,8cyclictest1009-21snmpd07:24:595
83009990,1cyclictest0-21swapper/310:20:125
83009990,1cyclictest0-21swapper/310:15:205
83009990,1cyclictest0-21swapper/309:10:225
83009990,1cyclictest0-21swapper/308:50:195
82959997,1cyclictest20049-21timerandwakeup107:20:234
82959990,8cyclictest27578-21awk09:40:004
82959990,8cyclictest15794-21/usr/sbin/munin07:15:204
82959990,8cyclictest0-21swapper/212:00:184
82959990,3cyclictest25157-21/usr/sbin/munin10:15:234
82959990,3cyclictest11036-21/usr/sbin/munin11:20:244
82959990,2cyclictest26767-21irqstats11:00:214
82959990,2cyclictest26767-21irqstats11:00:204
82959990,2cyclictest24154-21expr08:10:174
82959990,2cyclictest21456-21awk10:55:134
82829996,2cyclictest21280-21aten2_r3power_p10:55:130
82829990,8cyclictest30410-21df12:30:130
82829990,1cyclictest0-21swapper/008:50:240
27599190,0ptp4l0-21swapper/112:20:111
27599190,0ptp4l0-21swapper/111:50:171
27599190,0ptp4l0-21swapper/111:45:141
27599190,0ptp4l0-21swapper/110:45:141
27599190,0ptp4l0-21swapper/110:30:141
27599190,0ptp4l0-21swapper/110:05:191
27599190,0ptp4l0-21swapper/109:30:131
27599190,0ptp4l0-21swapper/108:55:121
27599190,0ptp4l0-21swapper/108:50:121
27599190,0ptp4l0-21swapper/108:40:151
83399987,1cyclictest0-21swapper/1111:20:133
83399980,8cyclictest0-21swapper/1111:55:143
83399980,8cyclictest0-21swapper/1111:10:223
83399980,8cyclictest0-21swapper/1111:10:213
83399980,8cyclictest0-21swapper/1109:25:183
83399980,7cyclictest0-21swapper/1108:04:423
83339987,1cyclictest0-21swapper/1007:55:262
83339980,8cyclictest0-21swapper/1012:05:162
83339980,8cyclictest0-21swapper/1011:55:152
83339980,7cyclictest0-21swapper/1009:21:302
83339980,1cyclictest0-21swapper/1009:00:292
83339980,1cyclictest0-21swapper/1007:47:552
83289987,1cyclictest0-21swapper/910:25:2311
83289987,1cyclictest0-21swapper/909:10:1111
83289987,1cyclictest0-21swapper/908:05:2211
83289987,1cyclictest0-21swapper/907:25:1611
83289980,8cyclictest0-21swapper/911:55:1411
83289980,8cyclictest0-21swapper/910:45:1511
83289980,8cyclictest0-21swapper/910:15:1311
83289980,7cyclictest0-21swapper/907:25:0011
83289980,1cyclictest0-21swapper/912:35:0211
83289980,1cyclictest0-21swapper/912:25:1911
83289980,1cyclictest0-21swapper/911:35:2311
83289980,1cyclictest0-21swapper/909:25:2911
83289980,1cyclictest0-21swapper/908:30:2511
83239986,1cyclictest0-21swapper/811:25:1210
83239980,8cyclictest0-21swapper/811:00:2510
83239980,8cyclictest0-21swapper/811:00:2510
83239980,8cyclictest0-21swapper/807:55:1410
83239980,1cyclictest0-21swapper/812:00:1310
83199987,1cyclictest0-21swapper/711:30:009
83199987,1cyclictest0-21swapper/710:00:009
83199987,1cyclictest0-21swapper/708:20:169
83199987,1cyclictest0-21swapper/707:25:149
83199980,8cyclictest0-21swapper/712:25:229
83199980,8cyclictest0-21swapper/710:40:199
83149983,4cyclictest0-21swapper/608:05:008
83149981,2cyclictest0-21swapper/611:51:268
83149981,2cyclictest0-21swapper/609:15:268
83109987,1cyclictest0-21swapper/512:20:007
83109987,1cyclictest0-21swapper/511:25:167
83109987,1cyclictest0-21swapper/511:05:257
83109987,1cyclictest0-21swapper/511:05:247
83109987,1cyclictest0-21swapper/510:15:187
83109982,5cyclictest0-21swapper/508:38:467
83109980,8cyclictest0-21swapper/510:45:207
83109980,8cyclictest0-21swapper/509:25:287
83109980,8cyclictest0-21swapper/508:10:177
83049987,1cyclictest0-21swapper/410:45:176
83049980,8cyclictest0-21swapper/407:10:266
83049980,1cyclictest0-21swapper/411:50:236
83049980,1cyclictest0-21swapper/409:55:256
83049980,1cyclictest0-21swapper/409:45:236
83009987,1cyclictest0-21swapper/311:30:175
83009987,1cyclictest0-21swapper/311:15:155
83009987,1cyclictest0-21swapper/311:15:155
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional