You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-22 - 16:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Mon Jun 22, 2026 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911160,59ptp4l0-21swapper/107:06:261
94080980,93rtkit-daemon0-21swapper/007:08:250
229126453,9sleep50-21swapper/507:09:107
224026352,9sleep100-21swapper/1007:08:282
218026352,9sleep40-21swapper/407:07:326
210126351,10sleep60-21swapper/607:06:228
206326352,9sleep70-21swapper/707:05:479
229726252,8sleep110-21swapper/1107:09:163
293052490,0sleep60-21swapper/611:10:228
2547992618,8cyclictest0-21swapper/1011:20:192
254099230,22cyclictest0-21swapper/907:30:0011
19172198,8sleep80-21swapper/807:05:2410
252799183,8cyclictest0-21swapper/708:50:009
252799181,10cyclictest0-21swapper/710:25:009
249599183,9cyclictest0-21swapper/110:35:011
249499182,8cyclictest0-21swapper/007:40:010
255199174,8cyclictest0-21swapper/1109:45:003
2551991713,3cyclictest0-21swapper/1109:45:153
254099174,9cyclictest0-21swapper/911:35:0111
254099174,8cyclictest0-21swapper/912:25:0011
249599173,8cyclictest0-21swapper/112:20:241
249599172,9cyclictest0-21swapper/111:40:011
283491160,0ptp4l0-21swapper/111:40:191
283491160,0ptp4l0-21swapper/109:25:001
283491160,0ptp4l0-21swapper/109:25:001
254099163,8cyclictest0-21swapper/909:25:0011
254099163,8cyclictest0-21swapper/909:25:0011
250799162,9cyclictest0-21swapper/411:05:016
249899163,8cyclictest0-21swapper/311:50:015
249899162,9cyclictest0-21swapper/310:30:245
249899162,8cyclictest0-21swapper/312:20:005
249899162,8cyclictest0-21swapper/312:05:015
249599162,9cyclictest0-21swapper/109:20:001
249599162,9cyclictest0-21swapper/109:20:001
249599162,9cyclictest0-21swapper/108:15:001
249499163,8cyclictest0-21swapper/010:30:010
249499163,8cyclictest0-21swapper/009:54:390
249499163,8cyclictest0-21swapper/009:15:000
249499163,8cyclictest0-21swapper/009:15:000
283491150,1ptp4l21-21ksoftirqd/109:55:121
283491150,1ptp4l21-21ksoftirqd/107:10:131
283491150,0ptp4l0-21swapper/110:15:101
283491150,0ptp4l0-21swapper/108:15:131
255199152,8cyclictest0-21swapper/1109:00:003
255199152,8cyclictest0-21swapper/1109:00:003
254799150,1cyclictest0-21swapper/1009:35:212
254099153,7cyclictest0-21swapper/908:15:2311
254099152,8cyclictest0-21swapper/912:15:2311
254099152,8cyclictest0-21swapper/910:10:0011
2540991512,2cyclictest0-21swapper/910:15:1511
254099151,1cyclictest0-21swapper/909:50:1611
254099150,3cyclictest0-21swapper/911:05:1611
252799153,8cyclictest0-21swapper/711:50:019
250799152,8cyclictest0-21swapper/410:00:236
249899153,7cyclictest0-21swapper/309:20:015
249899153,7cyclictest0-21swapper/309:20:015
249899153,7cyclictest0-21swapper/307:15:005
249899152,8cyclictest0-21swapper/309:05:015
249899152,8cyclictest0-21swapper/309:05:005
249499153,8cyclictest0-21swapper/011:10:010
249499153,8cyclictest0-21swapper/009:10:010
249499153,8cyclictest0-21swapper/009:10:010
283491140,1ptp4l21-21ksoftirqd/110:24:501
283491140,1ptp4l21-21ksoftirqd/109:29:391
255199141,8cyclictest0-21swapper/1110:40:013
254099143,7cyclictest0-21swapper/908:00:0111
252799143,7cyclictest0-21swapper/707:45:009
252799142,8cyclictest0-21swapper/710:55:029
252199140,8cyclictest0-21swapper/610:55:018
251499140,1cyclictest18481-21open_inodes08:50:207
249899143,6cyclictest0-21swapper/309:30:015
249899142,7cyclictest0-21swapper/309:35:235
249899142,7cyclictest0-21swapper/308:30:245
249899142,7cyclictest0-21swapper/307:45:245
249599143,7cyclictest0-21swapper/111:55:241
249599142,8cyclictest0-21swapper/111:00:011
249499142,7cyclictest0-21swapper/010:50:010
249499142,7cyclictest0-21swapper/009:00:010
249499142,7cyclictest0-21swapper/009:00:010
249499142,7cyclictest0-21swapper/008:10:250
249499142,7cyclictest0-21swapper/007:35:010
323642130,0sleep30-21swapper/312:00:015
300652130,0sleep80-21swapper/812:35:2110
283491130,1ptp4l21-21ksoftirqd/109:30:081
283491130,1ptp4l21-21ksoftirqd/108:57:531
283491130,1ptp4l21-21ksoftirqd/108:57:531
283491130,1ptp4l21-21ksoftirqd/107:55:141
283491130,0ptp4l0-21swapper/111:50:131
283491130,0ptp4l0-21swapper/109:40:171
258862130,0sleep40-21swapper/407:35:226
254799130,12cyclictest31095-21munin-run09:10:002
254799130,12cyclictest31095-21munin-run09:10:002
254099133,6cyclictest0-21swapper/911:40:0111
254099132,6cyclictest0-21swapper/912:25:2311
252799132,7cyclictest0-21swapper/711:55:019
252799132,7cyclictest0-21swapper/709:50:239
252799132,6cyclictest0-21swapper/711:00:239
252799130,2cyclictest21271-21munin-run08:15:009
251499130,4cyclictest32291cat10:40:017
250799131,7cyclictest0-21swapper/411:20:216
249899133,6cyclictest0-21swapper/311:50:015
249899132,7cyclictest0-21swapper/308:00:255
249899131,7cyclictest0-21swapper/310:20:235
249899131,7cyclictest0-21swapper/309:05:015
249899131,7cyclictest0-21swapper/309:05:015
249599132,7cyclictest0-21swapper/112:30:241
249599132,7cyclictest0-21swapper/109:40:011
249499133,6cyclictest0-21swapper/012:05:010
22392136,6sleep90-21swapper/907:08:2611
283491120,2ptp4l1583-21fw_forwarded_lo12:00:141
283491120,1ptp4l21-21ksoftirqd/110:35:121
283491120,1ptp4l21-21ksoftirqd/110:00:201
283491120,1ptp4l21-21ksoftirqd/108:43:281
283491120,0ptp4l0-21swapper/111:25:561
283491120,0ptp4l0-21swapper/111:24:421
283491120,0ptp4l0-21swapper/110:50:231
283491120,0ptp4l0-21swapper/109:10:241
283491120,0ptp4l0-21swapper/109:10:241
283491120,0ptp4l0-21swapper/108:20:191
283491120,0ptp4l0-21swapper/107:20:171
255199120,11cyclictest0-21swapper/1107:45:143
254799122,6cyclictest0-21swapper/1010:45:232
254099120,11cyclictest19652-21awk08:55:0011
252199122,7cyclictest0-21swapper/611:40:228
252199122,6cyclictest0-21swapper/607:15:008
251499120,5cyclictest9627-21sed07:15:177
251499120,3cyclictest316641irqrtprio09:50:177
251499120,3cyclictest13199-21latency_hist11:35:017
251499120,2cyclictest0-21swapper/511:45:177
250799123,6cyclictest0-21swapper/410:45:016
249499121,7cyclictest0-21swapper/008:25:260
283491110,2ptp4l20830-21hddtemp_smartct07:30:151
283491110,2ptp4l0-21swapper/112:15:001
283491110,1ptp4l21-21ksoftirqd/111:10:011
283491110,1ptp4l21-21ksoftirqd/111:00:121
283491110,1ptp4l21-21ksoftirqd/109:54:451
283491110,1ptp4l21-21ksoftirqd/109:05:011
283491110,1ptp4l21-21ksoftirqd/109:05:011
283491110,1ptp4l0-21swapper/112:25:141
283491110,1ptp4l0-21swapper/110:45:161
283491110,1ptp4l0-21swapper/107:55:001
283491110,0ptp4l0-21swapper/109:04:551
283491110,0ptp4l0-21swapper/109:04:551
264591110,0getstats0-21swapper/112:35:101
254799112,5cyclictest0-21swapper/1010:45:012
254799112,5cyclictest0-21swapper/1009:25:012
254799112,5cyclictest0-21swapper/1009:25:012
254099112,6cyclictest0-21swapper/908:20:2311
254099110,2cyclictest0-21swapper/910:45:1311
254099110,10cyclictest8217-21awk08:40:0011
253499110,2cyclictest0-21swapper/810:25:1310
252799110,1cyclictest0-21swapper/709:05:009
252799110,1cyclictest0-21swapper/709:05:009
252799110,10cyclictest6760-21expr11:25:119
252199112,8cyclictest30143-21unixbench_singl07:40:238
252199112,5cyclictest0-21swapper/612:30:008
252199111,6cyclictest0-21swapper/609:54:398
252199110,3cyclictest24195-21latency09:40:188
252199110,10cyclictest7578-21cut10:00:188
251499116,4cyclictest0-21swapper/508:40:007
251499113,7cyclictest24739-21tail12:30:157
251499112,8cyclictest7662-21grep10:00:197
251499112,8cyclictest29586-21sed10:30:167
251499112,8cyclictest2699-21cat07:50:137
251499112,8cyclictest13855-21grep09:25:247
251499110,5cyclictest9772-21fschecks_time08:40:147
251499110,5cyclictest9053-21sed07:55:257
251499110,5cyclictest31096-21munin-plugin-st09:10:007
251499110,5cyclictest31096-21munin-plugin-st09:10:007
251499110,5cyclictest19908-21cpuspeed_turbos07:30:127
251499110,5cyclictest15615-21cut07:25:017
251499110,4cyclictest26856-21latency_hist07:40:007
251499110,4cyclictest19291-21kernelversion11:40:177
251499110,4cyclictest17858-21sed10:55:247
251499110,2cyclictest2116-21sed09:10:257
251499110,2cyclictest2116-21sed09:10:257
251499110,1cyclictest0-21swapper/508:55:147
251499110,1cyclictest0-21swapper/508:55:137
251499110,11cyclictest0-21swapper/510:10:127
249899112,6cyclictest0-21swapper/310:05:015
249699118,2cyclictest18621-21turbostat11:00:004
249699110,10cyclictest26598-21latency_hist11:10:014
283491100,5ptp4l21-21ksoftirqd/110:42:541
283491100,2ptp4l24205-21diskstats07:35:131
283491100,2ptp4l22143-21df_inode11:45:121
283491100,2ptp4l21-21ksoftirqd/107:15:011
283491100,2ptp4l10131-21diskstats10:05:131
283491100,1ptp4l21-21ksoftirqd/111:17:391
283491100,1ptp4l21-21ksoftirqd/108:50:141
283491100,1ptp4l0-21swapper/109:45:111
283491100,1ptp4l0-21swapper/108:00:131
254799107,2cyclictest30067-21turbostat11:15:002
254799100,2cyclictest19073-21awk07:30:002
254099107,2cyclictest0-21swapper/907:35:1411
254099102,7cyclictest0-21swapper/909:25:2411
253499107,2cyclictest0-21swapper/807:25:0110
252799101,5cyclictest0-21swapper/707:15:249
252199102,5cyclictest0-21swapper/611:35:018
252199102,5cyclictest0-21swapper/609:50:008
252199100,1cyclictest15418-21munin-run12:20:008
251499102,7cyclictest6674-21expr11:25:127
251499102,7cyclictest27326-21egrep07:40:107
251499101,8cyclictest476-21cpuspeed12:00:117
251499101,8cyclictest22284-21expr08:15:127
251499101,8cyclictest18860-21cpuspeed_turbos09:35:127
251499100,5cyclictest3420-21cpuspeed_turbos07:10:137
251499100,5cyclictest2558-21irqrtprio08:30:177
251499100,5cyclictest12704-21cpuspeed08:45:127
251499100,4cyclictest9698-21cpuspeed_turbos10:05:127
251499100,4cyclictest8372-21cut12:10:117
251499100,4cyclictest7909-21cat10:45:117
251499100,4cyclictest7708-21expr09:20:117
251499100,4cyclictest7708-21expr09:20:117
251499100,4cyclictest5283-21cat09:15:187
251499100,4cyclictest5283-21cat09:15:187
251499100,4cyclictest5249-21if_enp4s010:40:157
251499100,4cyclictest32266-21expr07:45:167
251499100,4cyclictest32255-21sed11:15:177
251499100,4cyclictest31904-21switchtime08:25:257
251499100,4cyclictest29607-21ls11:55:137
251499100,4cyclictest27517-21cpu12:35:107
251499100,4cyclictest26811-21aten2_r3power_e11:10:117
251499100,4cyclictest2663-21expr11:20:107
251499100,4cyclictest26346-21cpuspeed09:45:117
251499100,4cyclictest26255-21irqrtprio10:25:187
251499100,4cyclictest24661-21irqrtprio11:05:177
251499100,4cyclictest24006-21aten2_r3power_c09:00:127
251499100,4cyclictest24006-21aten2_r3power_c09:00:127
251499100,4cyclictest23237-21cut09:40:137
251499100,4cyclictest22511-21sed10:20:177
251499100,4cyclictest22015-21ptp4l-jitter12:25:207
251499100,4cyclictest19197-21aten2_r3power_p11:00:127
251499100,4cyclictest18457-21expr08:10:147
251499100,4cyclictest17995-21fschecks_time10:15:137
251499100,4cyclictest1774-21expr09:55:097
251499100,4cyclictest16246-21cut07:25:137
251499100,4cyclictest15717-21fschecks_time09:30:127
251499100,4cyclictest15649-21irqrtprio08:05:187
251499100,4cyclictest14672-21fschecks_count11:35:167
251499100,4cyclictest12494-21cpuspeed_turbos12:15:127
251499100,4cyclictest12406-21tune2fs10:50:137
251499100,4cyclictest10469-21cpu08:00:137
251499100,3cyclictest4913-21cpuspeed_turbos12:05:127
251499100,3cyclictest26369-21gltestperf11:50:147
251499100,3cyclictest25613-21aten2_r3power_c08:20:107
251499100,3cyclictest16096-21sort12:20:117
250799106,3cyclictest1880-21cut11:20:016
249699100,9cyclictest0-21swapper/207:40:134
249499100,9cyclictest0-21swapper/012:15:240
203972100,0sleep00-21swapper/011:00:150
6194290,0sleep40-21swapper/408:35:176
28359190,3phc2sys0-21swapper/207:05:314
28349190,1ptp4l21-21ksoftirqd/112:10:111
28349190,1ptp4l21-21ksoftirqd/112:05:011
28349190,1ptp4l21-21ksoftirqd/111:30:011
28349190,1ptp4l21-21ksoftirqd/111:05:131
28349190,1ptp4l21-21ksoftirqd/110:25:121
28349190,1ptp4l21-21ksoftirqd/110:10:121
28349190,1ptp4l21-21ksoftirqd/108:45:141
28349190,1ptp4l21-21ksoftirqd/108:35:011
28349190,1ptp4l21-21ksoftirqd/108:30:011
28349190,1ptp4l21-21ksoftirqd/108:25:141
28349190,1ptp4l21-21ksoftirqd/108:05:011
28349190,1ptp4l21-21ksoftirqd/107:45:141
28349190,1ptp4l21-21ksoftirqd/107:40:101
28349190,1ptp4l21-21ksoftirqd/107:25:011
25519998,1cyclictest0-21swapper/1111:20:133
25519998,1cyclictest0-21swapper/1110:05:183
25519992,6cyclictest0-21swapper/1112:40:003
25519990,1cyclictest0-21swapper/1111:55:243
25519990,1cyclictest0-21swapper/1107:35:173
25479998,1cyclictest0-21swapper/1012:05:122
25479998,1cyclictest0-21swapper/1007:49:552
25479996,2cyclictest4379-21latency_hist12:05:012
25479992,4cyclictest0-21swapper/1008:50:012
25479990,3cyclictest8214-21munin-run08:40:002
25409997,1cyclictest0-21swapper/909:00:2111
25409997,1cyclictest0-21swapper/909:00:2111
25409990,4cyclictest0-21swapper/909:45:0011
25409990,4cyclictest0-21swapper/907:35:0011
25349998,1cyclictest0-21swapper/809:55:0110
25349997,1cyclictest0-21swapper/809:10:1910
25349997,1cyclictest0-21swapper/809:10:1910
25349997,1cyclictest0-21swapper/807:30:1710
25349990,8cyclictest0-21swapper/809:20:1610
25349990,8cyclictest0-21swapper/809:20:1610
25279990,9cyclictest0-21swapper/711:40:019
25279990,1cyclictest0-21swapper/707:35:239
25219998,1cyclictest501rcuc/607:52:418
25219997,2cyclictest0-21swapper/610:45:208
25219993,6cyclictest0-21swapper/607:25:138
25219993,5cyclictest115650irq/18-p34p112:13:308
25219993,5cyclictest115650irq/18-p34p111:04:528
25219993,5cyclictest115650irq/18-p34p110:43:158
25219992,6cyclictest115650irq/18-p34p111:55:188
25219991,8cyclictest0-21swapper/610:05:128
25219991,7cyclictest8381-21latency07:55:198
25219991,7cyclictest0-21swapper/609:20:148
25219991,7cyclictest0-21swapper/609:20:148
25219991,5cyclictest0-21swapper/611:45:238
25219990,8cyclictest0-21swapper/610:30:018
24989996,2cyclictest0-21swapper/310:35:135
24989990,2cyclictest15397-21taskset12:19:515
24989990,1cyclictest0-21swapper/312:10:165
24949998,1cyclictest0-21swapper/007:20:000
24949997,1cyclictest0-21swapper/009:30:160
24949996,2cyclictest7187-21turbostat10:45:000
24949992,6cyclictest0-21swapper/012:15:000
24949991,1cyclictest0-21swapper/010:15:170
24949990,9cyclictest0-21swapper/010:00:240
2153293,5sleep30-21swapper/307:07:075
25519987,1cyclictest0-21swapper/1108:20:143
25519980,8cyclictest0-21swapper/1110:40:143
25519980,8cyclictest0-21swapper/1108:00:133
25519980,7cyclictest0-21swapper/1111:05:103
25519980,1cyclictest0-21swapper/1110:50:233
25519980,1cyclictest0-21swapper/1109:50:193
25519980,1cyclictest0-21swapper/1109:35:173
25479987,1cyclictest0-21swapper/1011:50:432
25479987,1cyclictest0-21swapper/1009:15:152
25479987,1cyclictest0-21swapper/1009:15:152
25479987,1cyclictest0-21swapper/1008:20:132
25479985,2cyclictest22643-21idleruntime11:45:152
25479980,1cyclictest0-21swapper/1011:15:192
25479980,1cyclictest0-21swapper/1008:05:122
25409980,8cyclictest0-21swapper/911:50:1111
25409980,8cyclictest0-21swapper/911:40:1511
25409980,7cyclictest31091-21diskmemload10:25:2211
25409980,7cyclictest11456-21awk07:20:0011
25409980,1cyclictest31091-21diskmemload12:07:3311
25409980,1cyclictest0-21swapper/909:15:2411
25409980,1cyclictest0-21swapper/909:15:2411
25349987,1cyclictest0-21swapper/811:00:0110
25349987,1cyclictest0-21swapper/809:40:0010
25349987,1cyclictest0-21swapper/808:40:1210
25349985,2cyclictest10999-21awk10:50:0010
25349985,2cyclictest0-21swapper/808:22:0310
25279987,1cyclictest31092-21timerwakeupswit10:30:249
25279987,1cyclictest0-21swapper/712:25:159
25279987,1cyclictest0-21swapper/709:30:139
25279987,1cyclictest0-21swapper/709:20:229
25279987,1cyclictest0-21swapper/709:20:229
25279987,1cyclictest0-21swapper/708:35:189
25279980,8cyclictest0-21swapper/712:05:019
25279980,8cyclictest0-21swapper/711:30:149
25279980,7cyclictest0-21swapper/710:12:199
25279980,7cyclictest0-21swapper/707:55:229
25279980,1cyclictest0-21swapper/710:50:209
25279980,1cyclictest0-21swapper/708:00:179
25219987,1cyclictest501rcuc/612:00:238
25219987,1cyclictest0-21swapper/612:37:448
25219987,1cyclictest0-21swapper/611:15:128
25219987,1cyclictest0-21swapper/610:31:328
25219987,1cyclictest0-21swapper/610:10:128
25219987,1cyclictest0-21swapper/609:36:238
25219987,1cyclictest0-21swapper/608:55:208
25219987,1cyclictest0-21swapper/608:55:208
25219987,1cyclictest0-21swapper/608:50:148
25219987,1cyclictest0-21swapper/608:43:438
25219987,1cyclictest0-21swapper/608:10:168
25219984,4cyclictest0-21swapper/612:31:288
25219983,5cyclictest501rcuc/610:56:438
25219983,5cyclictest501rcuc/609:31:178
25219983,5cyclictest501rcuc/609:19:118
25219983,5cyclictest501rcuc/609:19:118
25219982,5cyclictest115650irq/18-p34p112:05:318
25219982,5cyclictest115650irq/18-p34p110:16:118
25219982,5cyclictest115650irq/18-p34p108:15:108
25219981,7cyclictest0-21swapper/611:50:178
25219981,6cyclictest0-21swapper/610:35:128
25219981,1cyclictest0-21swapper/609:55:178
25219980,8cyclictest0-21swapper/608:30:218
25219980,8cyclictest0-21swapper/607:15:018
25219980,7cyclictest26139-21taskset11:08:278
25219980,7cyclictest11310-21grep09:25:128
25219980,2cyclictest1150-21memory09:10:198
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional