You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 00:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Mon Jul 06, 2026 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911060,56ptp4l0-21swapper/107:09:161
94080870,82rtkit-daemon0-21swapper/507:07:457
1850127159,10sleep110-21swapper/1107:08:303
1757326956,10sleep40-21swapper/407:05:236
1840826755,10sleep00-21swapper/007:07:070
1839226756,9sleep100-21swapper/1007:06:532
1844326654,10sleep70-21swapper/707:07:389
1833326655,9sleep60-21swapper/607:06:018
135092590,0sleep30-21swapper/311:55:145
237572510,0sleep90-21swapper/908:35:2411
17772470,0sleep110-21swapper/1110:15:123
19000992720,7cyclictest0-21swapper/1008:20:172
1897399222,15cyclictest0-21swapper/409:55:016
1897399212,15cyclictest0-21swapper/409:30:016
1897399212,15cyclictest0-21swapper/409:30:016
1900099202,3cyclictest0-21swapper/1012:35:172
1899599203,9cyclictest0-21swapper/910:45:0111
1898699183,8cyclictest0-21swapper/710:35:239
1897399182,13cyclictest0-21swapper/409:25:016
1897399182,13cyclictest0-21swapper/409:25:016
1895799183,9cyclictest0-21swapper/110:15:011
283491170,0ptp4l0-21swapper/109:40:161
1898699171,2cyclictest0-21swapper/710:10:159
183892171,8sleep80-21swapper/807:06:5110
174492170,0sleep30-21swapper/307:05:215
283491160,1ptp4l21-21ksoftirqd/110:50:211
283491160,0ptp4l0-21swapper/112:25:151
283491160,0ptp4l0-21swapper/109:20:141
283491160,0ptp4l0-21swapper/109:20:141
1899599163,8cyclictest0-21swapper/908:45:2511
1896799163,8cyclictest0-21swapper/312:25:005
283491150,1ptp4l21-21ksoftirqd/107:45:171
283491150,0ptp4l0-21swapper/111:40:141
283491150,0ptp4l0-21swapper/108:45:171
283491150,0ptp4l0-21swapper/107:24:181
1900099153,7cyclictest0-21swapper/1011:04:482
1899599153,7cyclictest0-21swapper/912:15:0111
1899599152,8cyclictest0-21swapper/909:10:2511
1899599152,8cyclictest0-21swapper/909:10:2511
1899599152,8cyclictest0-21swapper/908:35:0011
1899599151,9cyclictest0-21swapper/907:45:0011
1898699153,7cyclictest0-21swapper/709:45:019
1898699153,7cyclictest0-21swapper/709:05:009
1898699153,7cyclictest0-21swapper/709:05:009
1896799152,7cyclictest0-21swapper/307:40:005
1895799153,7cyclictest0-21swapper/110:45:011
1895799153,7cyclictest0-21swapper/108:15:001
1895799152,9cyclictest0-21swapper/110:55:011
62472140,0sleep10-21swapper/108:15:141
283491140,1ptp4l21-21ksoftirqd/108:50:251
283491140,0ptp4l0-21swapper/112:15:201
283491140,0ptp4l0-21swapper/111:50:161
283491140,0ptp4l0-21swapper/107:15:021
1899599143,6cyclictest0-21swapper/911:05:2311
1899599141,8cyclictest0-21swapper/912:20:0111
1895799143,6cyclictest0-21swapper/108:30:011
1895799142,8cyclictest0-21swapper/109:40:001
1895799141,8cyclictest0-21swapper/110:10:011
1895799141,7cyclictest0-21swapper/110:50:011
47412130,0sleep100-21swapper/1010:20:122
283491130,5ptp4l21-21ksoftirqd/111:35:101
283491130,1ptp4l21-21ksoftirqd/111:02:091
283491130,0ptp4l0-21swapper/111:05:141
232372130,0sleep50-21swapper/510:00:177
1900099132,7cyclictest0-21swapper/1011:55:252
1899599130,9cyclictest0-21swapper/907:15:2511
1898699133,6cyclictest0-21swapper/708:40:019
1898699132,6cyclictest0-21swapper/712:35:019
1898299130,12cyclictest0-21swapper/611:10:008
18973991310,2cyclictest19410-21turbostat10:40:006
1896799132,7cyclictest0-21swapper/308:50:015
1896799132,7cyclictest0-21swapper/307:55:005
1896799131,11cyclictest27121-21expr08:00:125
1895799132,6cyclictest0-21swapper/110:05:001
283491120,2ptp4l20316-21perl07:10:221
283491120,1ptp4l21-21ksoftirqd/111:25:241
283491120,1ptp4l21-21ksoftirqd/108:00:181
283491120,0ptp4l0-21swapper/112:30:161
283491120,0ptp4l0-21swapper/112:11:141
283491120,0ptp4l0-21swapper/110:32:301
283491120,0ptp4l0-21swapper/109:10:141
283491120,0ptp4l0-21swapper/109:10:141
283491120,0ptp4l0-21swapper/108:59:321
1900099122,7cyclictest0-21swapper/1012:25:242
1899599122,6cyclictest0-21swapper/908:55:0211
1899199129,2cyclictest20327-21wc12:05:0110
1898699121,6cyclictest0-21swapper/711:25:019
1898699121,6cyclictest0-21swapper/710:30:249
1896299123,6cyclictest0-21swapper/212:10:014
1896299122,7cyclictest0-21swapper/211:40:014
1896299122,7cyclictest0-21swapper/211:10:014
1895799123,8cyclictest0-21swapper/111:20:011
1895299123,6cyclictest0-21swapper/011:45:010
1895299122,6cyclictest0-21swapper/011:00:010
183912126,5sleep90-21swapper/907:06:5211
283491110,2ptp4l32215-21cpuspeed_turbos07:25:121
283491110,2ptp4l21-21ksoftirqd/110:35:131
283491110,1ptp4l31929-21perl09:30:151
283491110,1ptp4l31929-21perl09:30:141
283491110,1ptp4l21-21ksoftirqd/107:50:371
283491110,0ptp4l0-21swapper/110:19:521
283491110,0ptp4l0-21swapper/109:25:101
283491110,0ptp4l0-21swapper/109:25:101
1899599112,5cyclictest0-21swapper/910:55:0011
1899199118,2cyclictest2930-21turbostat12:24:5910
1899199110,2cyclictest0-21swapper/811:35:1410
1898699111,9cyclictest1007-21snmpd07:38:319
1898699111,6cyclictest0-21swapper/711:15:019
1898699110,10cyclictest14646-21unixbench_singl09:05:249
1898699110,10cyclictest14646-21unixbench_singl09:05:249
1898299119,1cyclictest5212-21sed09:35:228
1898299112,8cyclictest501rcuc/612:35:128
1897399110,10cyclictest0-21swapper/411:05:136
1896799112,5cyclictest0-21swapper/312:15:015
1896299112,6cyclictest0-21swapper/209:15:004
1896299112,6cyclictest0-21swapper/209:15:004
1896299112,6cyclictest0-21swapper/208:55:244
1896299112,6cyclictest0-21swapper/208:40:004
1896299112,6cyclictest0-21swapper/208:20:004
1896299112,6cyclictest0-21swapper/208:00:004
1896299112,6cyclictest0-21swapper/207:45:254
1895799113,6cyclictest0-21swapper/107:45:001
1895299113,5cyclictest0-21swapper/008:35:240
1895299111,6cyclictest0-21swapper/007:15:000
1895299110,10cyclictest27270-21latency_hist10:50:010
283491100,1ptp4l21-21ksoftirqd/109:16:231
283491100,1ptp4l21-21ksoftirqd/109:16:231
283491100,1ptp4l21-21ksoftirqd/109:00:231
283491100,1ptp4l21-21ksoftirqd/109:00:231
283491100,1ptp4l16691-21/usr/sbin/munin12:00:241
283491100,1ptp4l13899-21fschecks_time11:55:161
283491100,0ptp4l0-21swapper/111:20:201
283491100,0ptp4l0-21swapper/109:55:191
283491100,0ptp4l0-21swapper/109:45:141
283491100,0ptp4l0-21swapper/108:35:181
283491100,0ptp4l0-21swapper/107:30:131
1900499107,2cyclictest14567-21turbostat12:40:003
1899599107,2cyclictest9031-21fw_forwarded_lo09:00:1511
1899599107,2cyclictest9031-21fw_forwarded_lo09:00:1511
1899599100,2cyclictest0-21swapper/908:40:1511
1899199108,1cyclictest0-21swapper/809:50:0010
1899199107,2cyclictest31455-21turbostat12:20:0010
1898699108,1cyclictest0-21swapper/709:25:009
1898699108,1cyclictest0-21swapper/709:25:009
1898699100,9cyclictest29286-21irqstats10:50:199
1898699100,5cyclictest0-21swapper/710:15:219
1898299109,1cyclictest501rcuc/608:20:208
1898299103,6cyclictest501rcuc/612:00:208
1898299102,1cyclictest501rcuc/611:20:158
1898299100,9cyclictest501rcuc/607:40:128
1898299100,9cyclictest0-21swapper/611:00:128
1898299100,9cyclictest0-21swapper/608:25:198
1897799108,1cyclictest0-21swapper/512:40:017
1897399108,1cyclictest16605-21awk11:15:176
1897399102,5cyclictest0-21swapper/409:40:236
1896799101,5cyclictest0-21swapper/308:05:235
1896299102,5cyclictest0-21swapper/212:35:004
1896299102,5cyclictest0-21swapper/212:30:004
1896299102,5cyclictest0-21swapper/208:50:014
1896299101,6cyclictest0-21swapper/211:00:014
1895299102,5cyclictest0-21swapper/008:00:000
1895299102,5cyclictest0-21swapper/007:35:000
1895299100,6cyclictest0-21swapper/009:10:000
1895299100,6cyclictest0-21swapper/009:10:000
28349190,1ptp4l21-21ksoftirqd/112:35:001
28349190,1ptp4l21-21ksoftirqd/111:45:011
28349190,1ptp4l21-21ksoftirqd/111:30:131
28349190,1ptp4l21-21ksoftirqd/111:10:021
28349190,1ptp4l21-21ksoftirqd/110:25:011
28349190,1ptp4l21-21ksoftirqd/110:20:011
28349190,1ptp4l21-21ksoftirqd/109:05:011
28349190,1ptp4l21-21ksoftirqd/109:05:011
28349190,1ptp4l21-21ksoftirqd/108:40:131
28349190,1ptp4l21-21ksoftirqd/108:25:131
28349190,1ptp4l21-21ksoftirqd/108:20:011
28349190,1ptp4l21-21ksoftirqd/108:05:011
28349190,1ptp4l21-21ksoftirqd/107:55:011
28349190,0ptp4l0-21swapper/112:20:121
28349190,0ptp4l0-21swapper/112:05:141
28349190,0ptp4l0-21swapper/109:50:121
28349190,0ptp4l0-21swapper/107:35:121
190049998,1cyclictest0-21swapper/1110:50:223
190049998,1cyclictest0-21swapper/1110:40:163
190049998,1cyclictest0-21swapper/1108:00:153
190049998,1cyclictest0-21swapper/1107:50:163
190049990,9cyclictest0-21swapper/1111:55:193
190049990,9cyclictest0-21swapper/1108:05:173
190049990,1cyclictest0-21swapper/1109:50:113
190009990,1cyclictest0-21swapper/1012:30:182
190009990,1cyclictest0-21swapper/1008:30:242
189959996,2cyclictest0-21swapper/912:02:0311
189959990,9cyclictest0-21swapper/909:30:1611
189959990,9cyclictest0-21swapper/909:30:1511
189959990,9cyclictest0-21swapper/908:25:1411
189959990,9cyclictest0-21swapper/908:05:1411
189959990,9cyclictest0-21swapper/908:04:1811
189959990,8cyclictest28003-21/usr/sbin/munin07:20:1711
189959990,1cyclictest0-21swapper/907:50:2111
189919998,1cyclictest0-21swapper/808:10:1310
189919997,1cyclictest0-21swapper/808:00:1710
189919997,1cyclictest0-21swapper/807:55:1410
189919993,5cyclictest0-21swapper/812:30:2710
189919993,5cyclictest0-21swapper/808:45:0110
189919992,6cyclictest0-21swapper/811:55:0110
189919992,6cyclictest0-21swapper/807:50:2410
189869997,1cyclictest9058-21/usr/sbin/munin11:50:149
189869996,2cyclictest0-21swapper/710:05:139
189869990,8cyclictest0-21swapper/712:15:149
189829998,1cyclictest501rcuc/609:50:148
189829997,1cyclictest0-21swapper/608:07:418
189829992,1cyclictest501rcuc/610:55:248
189829990,8cyclictest0-21swapper/607:15:148
189829990,6cyclictest0-21swapper/609:45:018
189829990,2cyclictest4597-21cstates08:55:138
189829990,1cyclictest0-21swapper/607:25:008
189779990,8cyclictest0-21swapper/510:30:147
189779990,2cyclictest7459-21chrt12:30:087
189739998,1cyclictest0-21swapper/411:50:176
189739998,1cyclictest0-21swapper/410:05:236
189739998,1cyclictest0-21swapper/408:45:146
189739998,1cyclictest0-21swapper/408:25:126
189739996,2cyclictest26110-21memory07:15:206
189739994,4cyclictest0-21swapper/410:28:116
189739991,5cyclictest0-21swapper/411:04:486
189739990,9cyclictest0-21swapper/410:45:166
189739990,1cyclictest0-21swapper/409:04:256
189739990,1cyclictest0-21swapper/409:04:256
189739990,1cyclictest0-21swapper/408:05:176
189629992,5cyclictest0-21swapper/211:40:014
189629992,5cyclictest0-21swapper/208:05:004
189629991,5cyclictest0-21swapper/208:30:014
189529998,1cyclictest0-21swapper/011:55:010
189529997,1cyclictest26684-21memory08:40:180
189529992,4cyclictest0-21swapper/007:25:000
189529991,5cyclictest0-21swapper/011:30:010
189529991,4cyclictest0-21swapper/011:10:010
189529990,8cyclictest12007-21memory09:45:180
190049987,1cyclictest0-21swapper/1111:50:133
190049987,1cyclictest0-21swapper/1111:35:183
190049987,1cyclictest0-21swapper/1110:35:203
190049987,1cyclictest0-21swapper/1110:30:193
190049987,1cyclictest0-21swapper/1109:05:153
190049987,1cyclictest0-21swapper/1109:05:153
190049980,1cyclictest0-21swapper/1110:05:003
190049980,1cyclictest0-21swapper/1108:10:153
190009980,8cyclictest0-21swapper/1008:50:132
190009980,7cyclictest15006-21diskmemload10:47:162
190009980,1cyclictest0-21swapper/1012:20:242
190009980,1cyclictest0-21swapper/1011:10:122
190009980,1cyclictest0-21swapper/1007:15:252
189959987,1cyclictest0-21swapper/908:20:2511
189959987,1cyclictest0-21swapper/908:15:2611
189959980,1cyclictest0-21swapper/911:35:2011
189959980,1cyclictest0-21swapper/910:15:0911
189959980,1cyclictest0-21swapper/909:50:1611
189959980,1cyclictest0-21swapper/909:44:0611
189919987,1cyclictest0-21swapper/807:20:1510
189919987,1cyclictest0-21swapper/807:15:1410
189919986,1cyclictest225802sleep809:19:1610
189919986,1cyclictest225802sleep809:19:1610
189919983,4cyclictest0-21swapper/810:20:0110
189919983,4cyclictest0-21swapper/808:50:0010
189919981,6cyclictest0-21swapper/811:35:0110
189919981,4cyclictest0-21swapper/810:20:1510
189919980,8cyclictest0-21swapper/808:50:1210
189919980,7cyclictest0-21swapper/810:53:3810
189869986,1cyclictest25945-21/usr/sbin/munin11:30:139
189869980,1cyclictest0-21swapper/708:35:229
189829987,1cyclictest501rcuc/612:11:518
189829987,1cyclictest501rcuc/609:20:168
189829987,1cyclictest501rcuc/609:20:168
189829987,1cyclictest501rcuc/608:40:268
189829986,1cyclictest0-21swapper/610:20:008
189829983,5cyclictest115650irq/18-p34p111:52:148
189829982,5cyclictest115650irq/18-p34p111:45:198
189829982,5cyclictest115650irq/18-p34p111:15:278
189829982,5cyclictest115650irq/18-p34p110:50:018
189829982,5cyclictest115650irq/18-p34p110:41:148
189829982,5cyclictest115650irq/18-p34p110:11:408
189829982,5cyclictest115650irq/18-p34p110:05:038
189829982,5cyclictest115650irq/18-p34p109:45:118
189829982,5cyclictest0-21swapper/612:20:238
189829982,5cyclictest0-21swapper/610:25:018
189829981,1cyclictest501rcuc/610:04:258
189829980,8cyclictest501rcuc/611:10:238
189829980,8cyclictest0-21swapper/611:25:138
189829980,7cyclictest29516-21/usr/sbin/munin12:15:178
189829980,7cyclictest12530-21/usr/sbin/munin10:30:128
189829980,7cyclictest0-21swapper/609:55:188
189829980,7cyclictest0-21swapper/609:20:018
189829980,7cyclictest0-21swapper/609:20:018
189829980,7cyclictest0-21swapper/609:00:148
189829980,7cyclictest0-21swapper/609:00:148
189829980,7cyclictest0-21swapper/608:30:178
189829980,3cyclictest1-21systemd07:39:178
189829980,2cyclictest13679-21df_inode11:55:158
189829980,1cyclictest0-21swapper/610:20:218
189829980,1cyclictest0-21swapper/609:10:178
189829980,1cyclictest0-21swapper/609:10:178
189829980,1cyclictest0-21swapper/608:15:148
189779987,1cyclictest0-21swapper/510:15:107
189779987,1cyclictest0-21swapper/507:35:247
189779980,8cyclictest0-21swapper/509:35:187
189779980,8cyclictest0-21swapper/508:30:187
189779980,8cyclictest0-21swapper/508:25:267
189779980,8cyclictest0-21swapper/508:20:217
189779980,8cyclictest0-21swapper/507:45:137
189779980,8cyclictest0-21swapper/507:40:127
189779980,8cyclictest0-21swapper/507:30:247
189779980,8cyclictest0-21swapper/507:10:297
189779980,1cyclictest0-21swapper/512:01:067
189779980,1cyclictest0-21swapper/511:50:227
189739987,1cyclictest0-21swapper/409:15:226
189739987,1cyclictest0-21swapper/409:15:226
189739987,1cyclictest0-21swapper/408:20:216
189739987,1cyclictest0-21swapper/407:55:196
189739987,1cyclictest0-21swapper/407:10:326
189739981,5cyclictest0-21swapper/407:30:006
189739980,8cyclictest0-21swapper/407:40:206
189739980,7cyclictest3935-21sed11:40:236
189739980,1cyclictest24337-21/usr/sbin/munin12:10:176
189739980,1cyclictest0-21swapper/412:35:006
189679987,1cyclictest0-21swapper/309:00:145
189679987,1cyclictest0-21swapper/309:00:145
189679986,1cyclictest0-21swapper/309:15:145
189679986,1cyclictest0-21swapper/309:15:145
189679981,5cyclictest0-21swapper/308:40:005
189679980,8cyclictest0-21swapper/308:30:195
189679980,1cyclictest0-21swapper/312:25:165
189679980,1cyclictest0-21swapper/310:30:195
189679980,1cyclictest0-21swapper/308:40:245
189629980,1cyclictest0-21swapper/210:20:154
189529987,1cyclictest0-21swapper/012:00:180
189529987,1cyclictest0-21swapper/011:45:120
189529982,4cyclictest0-21swapper/012:23:310
189529982,3cyclictest0-21swapper/011:10:010
189529981,5cyclictest0-21swapper/009:14:530
189529981,5cyclictest0-21swapper/009:14:530
189529981,4cyclictest0-21swapper/009:16:030
189529981,4cyclictest0-21swapper/009:16:030
189529980,8cyclictest0-21swapper/012:30:270
189529980,8cyclictest0-21swapper/007:45:170
17241284,2sleep217437-21open_files07:05:214
190049976,1cyclictest0-21swapper/1112:25:003
190049976,1cyclictest0-21swapper/1112:20:463
190049976,1cyclictest0-21swapper/1112:05:183
190049976,1cyclictest0-21swapper/1112:00:113
190049976,1cyclictest0-21swapper/1111:45:213
190049976,1cyclictest0-21swapper/1111:41:573
190049976,1cyclictest0-21swapper/1111:22:273
190049976,1cyclictest0-21swapper/1111:16:573
190049976,1cyclictest0-21swapper/1111:00:163
190049976,1cyclictest0-21swapper/1110:20:273
190049976,1cyclictest0-21swapper/1109:45:033
190049976,1cyclictest0-21swapper/1109:40:253
190049976,1cyclictest0-21swapper/1109:30:243
190049976,1cyclictest0-21swapper/1109:30:243
190049976,1cyclictest0-21swapper/1108:57:403
190049976,1cyclictest0-21swapper/1108:45:363
190049976,1cyclictest0-21swapper/1108:35:253
190049976,1cyclictest0-21swapper/1108:20:223
190049976,1cyclictest0-21swapper/1108:15:553
190049976,1cyclictest0-21swapper/1107:31:463
190049976,1cyclictest0-21swapper/1107:10:313
190049974,2cyclictest16912-21if_err_enp1s009:10:183
190049974,2cyclictest16912-21if_err_enp1s009:10:183
190049974,2cyclictest10435-21vmstat12:30:283
190049973,3cyclictest29340-21cpuspeed10:10:113
190049970,7cyclictest0-21swapper/1112:10:463
190049970,7cyclictest0-21swapper/1111:30:123
190049970,7cyclictest0-21swapper/1111:10:463
190049970,7cyclictest0-21swapper/1111:09:013
190049970,7cyclictest0-21swapper/1110:56:303
190049970,7cyclictest0-21swapper/1110:45:283
190049970,7cyclictest0-21swapper/1110:25:243
190049970,7cyclictest0-21swapper/1110:05:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional