You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Sep 22, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911280,60ptp4l0-21swapper/119:06:151
2272327152,10sleep50-21swapper/519:09:367
2254727052,9sleep40-21swapper/419:06:596
2255426856,10sleep110-21swapper/1119:07:073
2259426756,9sleep100-21swapper/1019:07:412
1945926756,9sleep00-21swapper/019:05:060
2250626654,10sleep70-21swapper/719:06:269
2236826452,9sleep60-21swapper/619:05:248
138742550,0sleep10-21swapper/122:25:151
14772520,0sleep10-21swapper/122:10:151
2309599212,12cyclictest0-21swapper/021:55:000
2309599212,12cyclictest0-21swapper/021:55:000
283491190,0ptp4l0-21swapper/120:30:151
2313999192,3cyclictest0-21swapper/900:20:1711
2313599193,8cyclictest0-21swapper/823:45:2310
2310699193,10cyclictest0-21swapper/222:10:014
264172180,0sleep40-21swapper/423:25:126
2310699183,9cyclictest0-21swapper/223:25:014
283491170,0ptp4l0-21swapper/120:45:131
2313599172,8cyclictest0-21swapper/821:40:0010
2310699174,8cyclictest0-21swapper/221:45:004
2310699174,8cyclictest0-21swapper/221:45:004
2310699172,9cyclictest0-21swapper/220:45:004
2310699172,11cyclictest0-21swapper/222:15:014
2309599172,9cyclictest0-21swapper/000:05:000
2309599171,10cyclictest0-21swapper/020:50:000
283491160,1ptp4l21-21ksoftirqd/120:35:191
283491160,0ptp4l0-21swapper/123:20:141
283491160,0ptp4l0-21swapper/122:45:131
283491160,0ptp4l0-21swapper/119:10:171
2313999162,8cyclictest0-21swapper/919:40:0011
2312999163,8cyclictest0-21swapper/700:25:019
2312599163,8cyclictest0-21swapper/619:50:018
2310699164,8cyclictest0-21swapper/219:10:004
2310699163,9cyclictest0-21swapper/223:00:014
2310699163,9cyclictest0-21swapper/221:05:004
2310699163,8cyclictest0-21swapper/223:32:044
2310699163,8cyclictest0-21swapper/200:30:014
23106991613,2cyclictest0-21swapper/222:40:164
2309599162,8cyclictest0-21swapper/022:40:000
194542167,7sleep80-21swapper/819:05:0210
94080150,2rtkit-daemon0-21swapper/421:40:126
94080150,2rtkit-daemon0-21swapper/421:40:126
283491150,0ptp4l0-21swapper/120:15:161
2314899154,7cyclictest0-21swapper/1122:10:013
2314899152,8cyclictest0-21swapper/1121:40:003
2314899152,7cyclictest0-21swapper/1119:15:003
2314499150,6cyclictest0-21swapper/1019:20:142
2313999153,8cyclictest0-21swapper/900:05:0111
2313999152,9cyclictest0-21swapper/923:10:2311
2313999152,8cyclictest0-21swapper/920:40:0011
2313599153,6cyclictest0-21swapper/821:20:0210
2312999152,8cyclictest0-21swapper/722:10:009
2312999150,3cyclictest0-21swapper/700:35:169
2311999152,8cyclictest0-21swapper/520:40:247
23119991513,2cyclictest0-21swapper/523:50:187
2310699153,7cyclictest0-21swapper/222:00:004
2310699153,7cyclictest0-21swapper/222:00:004
2310699151,9cyclictest0-21swapper/222:25:004
2309599153,7cyclictest0-21swapper/022:00:010
2309599153,7cyclictest0-21swapper/022:00:000
2309599153,7cyclictest0-21swapper/019:35:000
2309599152,9cyclictest0-21swapper/020:35:000
2309599152,8cyclictest0-21swapper/021:50:000
2309599152,8cyclictest0-21swapper/021:50:000
2309599152,8cyclictest0-21swapper/000:00:010
283491140,1ptp4l21-21ksoftirqd/123:25:191
283491140,1ptp4l21-21ksoftirqd/123:05:271
2314899143,7cyclictest0-21swapper/1123:45:013
2314899143,7cyclictest0-21swapper/1122:30:013
2314899143,7cyclictest0-21swapper/1121:35:003
2314899142,7cyclictest0-21swapper/1120:15:243
2314899142,7cyclictest0-21swapper/1120:10:233
2314899142,7cyclictest0-21swapper/1119:35:003
2314499140,13cyclictest0-21swapper/1021:55:202
2314499140,13cyclictest0-21swapper/1021:55:202
2313999143,7cyclictest0-21swapper/919:55:0011
2313999142,7cyclictest0-21swapper/923:55:2411
2313599143,7cyclictest0-21swapper/823:10:0110
2313599143,7cyclictest0-21swapper/822:35:0010
2312999142,8cyclictest0-21swapper/723:55:019
2312599143,7cyclictest0-21swapper/620:45:248
2311999142,7cyclictest0-21swapper/500:05:017
2310999143,7cyclictest0-21swapper/321:30:005
2310699143,7cyclictest0-21swapper/200:00:014
2310699142,8cyclictest0-21swapper/223:45:014
2310099149,4cyclictest19170-21diskmemload23:03:531
2309599143,7cyclictest0-21swapper/023:40:010
2309599143,7cyclictest0-21swapper/023:15:010
2309599143,7cyclictest0-21swapper/022:30:000
2309599142,8cyclictest0-21swapper/022:20:000
2309599142,6cyclictest0-21swapper/020:25:000
283491130,1ptp4l21-21ksoftirqd/122:15:231
283491130,1ptp4l21-21ksoftirqd/119:40:211
283491130,0ptp4l0-21swapper/120:13:241
2314899133,6cyclictest0-21swapper/1122:55:013
2314899133,6cyclictest0-21swapper/1121:30:003
2314899132,7cyclictest0-21swapper/1123:25:013
2314899132,7cyclictest0-21swapper/1119:40:003
2314899132,6cyclictest0-21swapper/1120:10:003
2314899132,5cyclictest0-21swapper/1123:30:013
2314899131,7cyclictest0-21swapper/1120:00:003
2314499132,6cyclictest0-21swapper/1022:15:242
2314499130,12cyclictest0-21swapper/1000:00:222
2313999133,6cyclictest0-21swapper/922:35:2311
2313999133,6cyclictest0-21swapper/900:10:0111
2313999132,6cyclictest0-21swapper/921:10:0111
2313999130,8cyclictest0-21swapper/919:15:0011
2313999130,12cyclictest0-21swapper/923:10:0011
2313599133,6cyclictest0-21swapper/800:30:2610
2313599131,7cyclictest0-21swapper/821:15:0010
2312999132,9cyclictest1024-21head19:20:149
2312999132,6cyclictest0-21swapper/721:35:239
2312599132,7cyclictest0-21swapper/622:30:278
2312599132,6cyclictest0-21swapper/623:40:018
2312599130,12cyclictest0-21swapper/621:35:008
2311999131,7cyclictest0-21swapper/522:20:017
2310699133,7cyclictest0-21swapper/200:00:014
2309599132,6cyclictest0-21swapper/022:05:010
2309599132,6cyclictest0-21swapper/022:05:010
283491120,3ptp4l16539-21cut23:10:171
283491120,2ptp4l5063-21perl00:20:151
283491120,1ptp4l21-21ksoftirqd/122:30:161
283491120,1ptp4l21-21ksoftirqd/119:50:121
283491120,0ptp4l0-21swapper/121:42:121
283491120,0ptp4l0-21swapper/121:42:121
283491120,0ptp4l0-21swapper/119:30:131
283491120,0ptp4l0-21swapper/100:25:251
2314499120,12cyclictest0-21swapper/1021:21:102
2314499120,11cyclictest0-21swapper/1022:05:152
2314499120,11cyclictest0-21swapper/1021:40:002
2314499120,11cyclictest0-21swapper/1020:20:152
2313599129,2cyclictest0-21swapper/819:15:1910
2313599122,6cyclictest0-21swapper/820:30:2510
23129991210,1cyclictest0-21swapper/700:05:019
2312599122,6cyclictest0-21swapper/619:10:008
2311999123,6cyclictest0-21swapper/519:30:017
2311999122,6cyclictest0-21swapper/520:31:297
2310999123,5cyclictest0-21swapper/300:10:015
2310999122,7cyclictest0-21swapper/321:05:005
23106991210,1cyclictest6127-21turbostat21:35:004
2310099127,4cyclictest19170-21diskmemload22:02:221
2310099127,4cyclictest19170-21diskmemload22:02:221
2310099122,7cyclictest0-21swapper/121:25:001
2310099122,6cyclictest0-21swapper/120:25:001
2309599121,6cyclictest0-21swapper/023:35:010
226862126,5sleep90-21swapper/919:09:0411
283491110,2ptp4l24327-21diskstats21:15:131
283491110,2ptp4l22703-21diskstats21:55:141
283491110,2ptp4l22703-21diskstats21:55:141
283491110,2ptp4l21557-21latency21:10:191
283491110,2ptp4l20352-21sed23:15:181
283491110,2ptp4l13502-21taskset23:50:141
283491110,1ptp4l21-21ksoftirqd/121:25:171
283491110,1ptp4l0-21swapper/122:35:131
283491110,1ptp4l0-21swapper/122:05:151
283491110,1ptp4l0-21swapper/119:25:221
283491110,0ptp4l0-21swapper/121:50:011
283491110,0ptp4l0-21swapper/121:50:011
283491110,0ptp4l0-21swapper/120:52:071
283491110,0ptp4l0-21swapper/100:15:201
2314899110,7cyclictest32536-21kworker/u24:120:50:003
23144991110,1cyclictest0-21swapper/1020:50:162
2314499110,11cyclictest0-21swapper/1023:40:142
2314499110,11cyclictest0-21swapper/1019:35:232
2314499110,10cyclictest9793-21grep23:45:122
2313999118,2cyclictest0-21swapper/919:55:1611
2313999110,10cyclictest0-21swapper/919:45:1411
2313599110,10cyclictest24293-21awk00:05:0010
23129991110,1cyclictest561rcuc/720:25:139
2312999110,10cyclictest25975-21cpuspeed_turbos22:00:139
2312999110,10cyclictest25975-21cpuspeed_turbos22:00:139
2312999110,10cyclictest0-21swapper/721:45:239
2312999110,10cyclictest0-21swapper/721:45:239
2312599110,2cyclictest26678-21cat22:40:238
2312599110,2cyclictest10938-21latency_hist19:35:008
2312599110,10cyclictest0-21swapper/622:15:008
2311999111,6cyclictest0-21swapper/522:45:017
2310999113,5cyclictest0-21swapper/323:55:015
2310999112,6cyclictest0-21swapper/323:40:225
2310999112,6cyclictest0-21swapper/319:15:005
2310999112,6cyclictest0-21swapper/300:20:245
2310999110,10cyclictest1259-21cut20:45:155
2310099112,5cyclictest0-21swapper/120:30:011
2310099110,5cyclictest0-21swapper/119:55:001
283491100,2ptp4l31794-21hostname22:50:111
283491100,2ptp4l23790-21grep00:00:251
283491100,1ptp4l2780-21fw_conntrack23:35:141
283491100,1ptp4l25282-21cstates00:05:121
283491100,1ptp4l24926-21df_inode22:40:131
283491100,1ptp4l21-21ksoftirqd/123:55:491
283491100,1ptp4l20874-21latency19:45:181
283491100,1ptp4l12056-21ls19:35:141
283491100,1ptp4l10579-21/usr/sbin/munin22:20:161
283491100,1ptp4l10562-21sensors_fan20:55:221
283491100,1ptp4l0-21swapper/100:30:151
283491100,0ptp4l0-21swapper/123:32:051
283491100,0ptp4l0-21swapper/121:05:231
283491100,0ptp4l0-21swapper/120:40:131
2314899102,5cyclictest0-21swapper/1119:15:243
2314899101,6cyclictest17956-21kworker/u24:023:20:003
2314499104,5cyclictest0-21swapper/1020:35:202
2314499103,6cyclictest15881-21grep00:35:122
2314499100,9cyclictest8247-21cpuspeed_turbos00:25:132
2314499100,9cyclictest7173-21latency_hist19:30:002
2314499100,9cyclictest6967-21grep00:20:262
2314499100,9cyclictest3864-21cut21:30:202
2314499100,9cyclictest32466-21latency_hist20:45:002
2314499100,9cyclictest315-21aten2_r3power_c20:45:122
2314499100,9cyclictest30409-21latency_hist20:00:002
2314499100,9cyclictest2528-21fschecks_count23:35:132
2314499100,9cyclictest25151-21grep00:05:122
2314499100,9cyclictest24599-21gltestperf19:10:182
2314499100,9cyclictest2451-21cpuspeed_turbos20:05:142
2314499100,9cyclictest23962-21grep21:15:132
2314499100,9cyclictest22193-21sed23:20:002
2314499100,9cyclictest20902-21cut21:10:162
2314499100,9cyclictest20251-21tail21:50:202
2314499100,9cyclictest20251-21tail21:50:192
2314499100,9cyclictest20097-21latency_hist22:35:002
2314499100,9cyclictest19194-21grep19:45:122
2314499100,9cyclictest18226-21expr20:25:132
2314499100,9cyclictest15491-21gltestperf21:45:152
2314499100,9cyclictest15491-21gltestperf21:45:152
2314499100,9cyclictest15392-21cpuspeed_turbos23:10:122
2314499100,9cyclictest13160-21fschecks_time21:00:162
2314499100,9cyclictest12768-21grep20:15:252
2314499100,9cyclictest0-21swapper/1020:30:182
2314499100,10cyclictest0-21swapper/1023:59:532
2314499100,10cyclictest0-21swapper/1022:59:032
2314499100,10cyclictest0-21swapper/1020:00:282
2314499100,10cyclictest0-21swapper/1019:15:012
2313599108,1cyclictest0-21swapper/821:45:0110
2313599108,1cyclictest0-21swapper/821:45:0010
2313599108,1cyclictest0-21swapper/820:35:1310
2313599100,9cyclictest13908-21awk20:20:1210
2313599100,5cyclictest0-21swapper/823:20:0010
2312999109,1cyclictest561rcuc/700:00:239
2312999101,1cyclictest561rcuc/723:30:149
2312999100,9cyclictest0-21swapper/722:34:559
2312599100,9cyclictest31100-21grep20:40:208
2312599100,9cyclictest0-21swapper/621:05:018
2312599100,9cyclictest0-21swapper/600:15:258
2312599100,1cyclictest0-21swapper/600:35:168
2311999107,2cyclictest1490-21ps00:15:187
2311699102,5cyclictest14750irq/20-ata_piix23:50:016
2311699102,5cyclictest0-21swapper/421:45:026
2311699102,5cyclictest0-21swapper/421:45:016
2310999109,1cyclictest0-21swapper/322:40:125
2310999102,6cyclictest0-21swapper/319:20:235
2310999102,5cyclictest0-21swapper/322:25:005
2310999102,5cyclictest0-21swapper/320:55:005
2310999102,5cyclictest0-21swapper/319:30:015
2310999102,5cyclictest0-21swapper/300:00:255
2310999101,6cyclictest32536-21kworker/u24:119:45:005
2310999101,6cyclictest27919-21kworker/u24:200:20:015
2310699107,2cyclictest7115-21perl20:10:144
2310699107,2cyclictest0-21swapper/200:05:174
2310699103,7cyclictest0-21swapper/200:25:014
2310699102,7cyclictest0-21swapper/220:00:274
2309599107,2cyclictest11161-21turbostat00:30:010
28349190,1ptp4l21-21ksoftirqd/121:45:121
28349190,1ptp4l21-21ksoftirqd/121:45:121
28349190,1ptp4l21-21ksoftirqd/119:20:111
28349190,1ptp4l21-21ksoftirqd/119:15:011
28349190,0ptp4l0-21swapper/123:45:131
28349190,0ptp4l0-21swapper/123:40:111
28349190,0ptp4l0-21swapper/122:55:011
28349190,0ptp4l0-21swapper/121:35:011
28349190,0ptp4l0-21swapper/121:30:171
28349190,0ptp4l0-21swapper/121:00:141
28349190,0ptp4l0-21swapper/120:05:011
28349190,0ptp4l0-21swapper/120:00:011
28349190,0ptp4l0-21swapper/100:35:151
28349190,0ptp4l0-21swapper/100:10:011
231489997,1cyclictest0-21swapper/1120:40:133
231489997,1cyclictest0-21swapper/1119:50:173
231489990,9cyclictest0-21swapper/1123:34:553
231489990,8cyclictest18222-21fw_conntrack23:55:143
231489990,1cyclictest24352-40dnf20:31:293
231449993,5cyclictest11974-21grep23:05:162
231449990,8cyclictest9190-21cpuspeed_turbos22:20:112
231449990,8cyclictest8576-21fschecks_count19:30:152
231449990,8cyclictest8500-21cpuspeed_turbos20:55:132
231449990,8cyclictest7509-21cat23:00:102
231449990,8cyclictest32703-21cpuspeed_turbos00:15:132
231449990,8cyclictest31801-21cut22:50:112
231449990,8cyclictest31467-21cpuspeed_turbos21:25:112
231449990,8cyclictest31324-21grep23:30:162
231449990,8cyclictest28933-21cut00:10:142
231449990,8cyclictest28620-21expr22:45:142
231449990,8cyclictest26751-21cpuspeed_turbos23:25:142
231449990,8cyclictest25887-21cpuspeed22:00:132
231449990,8cyclictest25887-21cpuspeed22:00:132
231449990,8cyclictest24787-21cat22:40:122
231449990,8cyclictest23884-21fschecks_time19:50:132
231449990,8cyclictest22747-21sort23:20:132
231449990,8cyclictest21198-21fschecks_count22:35:152
231449990,8cyclictest17410-21irqrtprio21:05:172
231449990,8cyclictest13772-21grep23:50:142
231449990,8cyclictest13019-21tr22:25:122
231449990,8cyclictest12517-21grep00:30:162
231449990,8cyclictest10758-21cpuspeed_turbos21:40:132
231449990,8cyclictest10758-21cpuspeed_turbos21:40:122
231449990,8cyclictest0-21swapper/1022:10:192
231449990,8cyclictest0-21swapper/1020:10:152
231449990,8cyclictest0-21swapper/1019:40:122
231399998,1cyclictest0-21swapper/922:40:2111
231399991,8cyclictest0-21swapper/923:35:1611
231399990,1cyclictest0-21swapper/920:30:1611
231359998,1cyclictest0-21swapper/823:05:1610
231359996,2cyclictest8932-21turbostat23:45:0010
231359990,9cyclictest0-21swapper/823:20:1310
231359990,9cyclictest0-21swapper/800:15:2510
231299998,1cyclictest561rcuc/723:40:149
231299998,1cyclictest561rcuc/721:55:209
231299998,1cyclictest561rcuc/721:55:209
231299998,1cyclictest561rcuc/720:15:109
231299996,2cyclictest0-21swapper/722:35:139
231299990,9cyclictest0-21swapper/720:35:129
231299990,9cyclictest0-21swapper/719:35:159
231299990,1cyclictest0-21swapper/723:15:159
231299990,1cyclictest0-21swapper/720:40:199
231259992,6cyclictest26934-21sed21:20:008
231259992,6cyclictest115650irq/18-p34p123:00:198
231259990,8cyclictest0-21swapper/622:40:008
231259990,8cyclictest0-21swapper/620:35:168
231259990,5cyclictest0-21swapper/623:35:008
231259990,3cyclictest6345-21memory20:50:188
231259990,3cyclictest29194-21sed23:25:248
231259990,2cyclictest28919-21tr00:10:148
231259990,1cyclictest0-21swapper/600:00:248
231199990,8cyclictest0-21swapper/522:25:137
231169990,5cyclictest0-21swapper/421:55:006
231169990,5cyclictest0-21swapper/421:55:006
231099992,5cyclictest0-21swapper/322:25:015
231099992,4cyclictest0-21swapper/322:00:025
231099992,4cyclictest0-21swapper/322:00:015
231099991,5cyclictest0-21swapper/323:50:025
231099991,5cyclictest0-21swapper/322:00:015
231099991,5cyclictest0-21swapper/322:00:015
231099990,9cyclictest0-21swapper/319:40:005
230959997,1cyclictest0-21swapper/019:25:240
230959992,5cyclictest0-21swapper/023:02:120
230959990,1cyclictest0-21swapper/019:15:250
9972280,2sleep210255-21diskstats23:45:134
231489980,3cyclictest0-21swapper/1123:04:553
231489980,1cyclictest0-21swapper/1121:55:223
231489980,1cyclictest0-21swapper/1121:55:223
231489980,1cyclictest0-21swapper/1120:25:163
231399987,1cyclictest0-21swapper/922:35:0011
231399987,1cyclictest0-21swapper/922:25:0011
231399987,1cyclictest0-21swapper/921:20:0011
231399981,6cyclictest0-21swapper/919:45:0011
231399981,4cyclictest0-21swapper/923:15:1411
231399980,8cyclictest0-21swapper/920:10:1611
231399980,1cyclictest0-21swapper/923:20:1611
231399980,1cyclictest0-21swapper/922:55:2211
231399980,1cyclictest0-21swapper/921:05:2311
231399980,1cyclictest0-21swapper/920:40:1811
231399980,1cyclictest0-21swapper/920:20:1211
231399980,1cyclictest0-21swapper/900:15:2111
231359987,1cyclictest0-21swapper/821:55:1210
231359987,1cyclictest0-21swapper/821:55:1210
231359987,1cyclictest0-21swapper/800:05:2310
231359986,1cyclictest0-21swapper/822:24:1610
231359986,1cyclictest0-21swapper/819:20:1110
231359980,8cyclictest0-21swapper/822:10:1310
231359980,8cyclictest0-21swapper/821:00:0110
231359980,8cyclictest0-21swapper/820:25:1310
231359980,8cyclictest0-21swapper/819:30:2010
231359980,1cyclictest0-21swapper/822:15:2010
231359980,1cyclictest0-21swapper/821:30:3010
231299987,1cyclictest561rcuc/723:59:539
231299987,1cyclictest561rcuc/723:04:149
231299987,1cyclictest561rcuc/721:21:109
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional