You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 20:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Sat Dec 13, 2025 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27201179910820,1078cyclictest30190722sleep311:00:013
2720117993930,390cyclictest2954421-21turbostat10:14:553
290836521940,2sleep20-21swapper/209:30:222
2720117992017,1cyclictest0-21swapper/307:35:003
2720117991916,1cyclictest0-21swapper/308:40:013
2720117991916,1cyclictest0-21swapper/307:20:003
272011799191,14cyclictest3012463-21taskset10:55:013
272011799186,9cyclictest3063978-21turbostat11:35:013
272011799176,7cyclictest2915453-21turbostat09:39:593
272011799171,12cyclictest3044507-21turbostat11:20:003
272011799171,12cyclictest2883662-21taskset09:15:123
2720117991614,1cyclictest0-21swapper/309:55:003
272011799161,11cyclictest3096328-21sleep12:00:003
272011799161,11cyclictest2720319-21taskset07:10:083
30697222150,5sleep03069794-21ntp_states11:35:210
28568822150,4sleep12857006-21cat08:50:231
272011799158,4cyclictest2812762-21turbostat08:20:003
2720117991513,1cyclictest0-21swapper/311:14:383
2720117991513,1cyclictest0-21swapper/308:08:013
2720117991513,0cyclictest0-21swapper/309:24:053
2720117991512,1cyclictest0-21swapper/311:40:223
2720117991512,1cyclictest0-21swapper/309:09:213
2720117991512,1cyclictest0-21swapper/307:50:193
272011799151,11cyclictest3109348-21chrt12:10:013
272011799151,11cyclictest3018051-21taskset10:55:213
272011799151,11cyclictest2980062-21turbostat10:30:003
272011799151,11cyclictest2825899-21taskset08:30:133
272011799151,10cyclictest2954418-21turbostat10:10:003
272011799151,10cyclictest2935213-21taskset09:55:153
31341712140,5sleep03134228-21ntp_states12:25:210
272011799144,8cyclictest2857386-21taskset08:51:403
272011799144,7cyclictest28056562sleep308:10:233
272011799141,8cyclictest28387152sleep308:40:153
2720117991413,0cyclictest0-21swapper/307:43:033
2720117991412,1cyclictest0-21swapper/310:20:323
2720117991412,1cyclictest0-21swapper/308:25:043
2720117991412,0cyclictest0-21swapper/311:26:063
2720117991411,1cyclictest0-21swapper/310:41:283
2720117991411,1cyclictest0-21swapper/310:38:213
2720117991411,1cyclictest0-21swapper/307:36:513
272011799141,10cyclictest3147900-21turbostat12:40:003
272011799141,10cyclictest3108203-21taskset12:05:203
272011799141,10cyclictest3050347-21taskset11:20:213
272011799141,10cyclictest3025461-21taskset11:05:023
272011799141,10cyclictest2946914-21taskset10:00:283
272011799141,10cyclictest2908693-21taskset09:31:073
272011799141,10cyclictest2787089-21turbostat08:00:003
272011799140,11cyclictest3128044-21chrt12:20:303
31304032130,9sleep13132370-21tr12:25:181
29740952131,8sleep22974210-21cut10:25:162
29739102131,2sleep00-21swapper/010:25:140
29308192131,10sleep22932981-21cut09:50:172
28836352133,6sleep22883657-21apt-get09:15:122
272011799131,9cyclictest3097033-21chrt12:00:153
272011799131,9cyclictest3077526-21taskset11:45:133
272011799131,9cyclictest2961079-21taskset10:15:093
272011799131,9cyclictest2877202-21chrt09:10:193
272011799131,9cyclictest2813048-21taskset08:20:113
272011799131,9cyclictest2793147-21chrt08:00:313
272011799131,9cyclictest2773704-21taskset07:45:583
2720117991312,0cyclictest0-21swapper/309:40:543
2720117991312,0cyclictest0-21swapper/307:20:113
2720117991311,1cyclictest0-21swapper/312:20:003
2720117991310,1cyclictest0-21swapper/309:28:473
2720117991310,1cyclictest0-21swapper/309:03:063
2720117991310,1cyclictest0-21swapper/308:55:183
2720117991310,1cyclictest0-21swapper/307:25:203
272011799130,10cyclictest3089537-21taskset11:50:253
272011799130,10cyclictest3069967-21chrt11:35:233
272011799130,10cyclictest2999580-21taskset10:45:073
272011799130,10cyclictest2850988-21chrt08:46:523
301798120,7rtkit-daemon2970364-21grep10:20:181
28722292120,2sleep00-21swapper/009:05:170
28407832120,1sleep20-21swapper/208:40:262
272011799123,6cyclictest31340312sleep312:25:203
272011799121,9cyclictest2922119-21chrt09:45:093
272011799121,8cyclictest3141132-21taskset12:30:483
2720117991211,0cyclictest0-21swapper/310:30:053
31093612111,5sleep13109394-21apt-get12:10:121
30639602111,3sleep00-21swapper/011:35:010
301798110,7rtkit-daemon3205-21sendmail-mta07:42:500
301798110,7rtkit-daemon3128401-21apt-get12:25:012
301798110,4rtkit-daemon2726644-21munin-node08:42:030
27138312111,6sleep30-21swapper/307:05:073
301798100,7rtkit-daemon636-21snmpd07:57:400
301798100,6rtkit-daemon3083738-21/usr/sbin/munin11:50:132
301798100,6rtkit-daemon2805474-21ntp_kernel_pll_08:10:200
301798100,6rtkit-daemon2744902-21kthreadcore07:25:180
301798100,5rtkit-daemon636-21snmpd11:08:580
301798100,5rtkit-daemon2992058-21ntp_states10:35:221
301798100,5rtkit-daemon2758381-21grep07:35:190
301798100,5rtkit-daemon2720476-21diskstats07:10:090
301798100,3rtkit-daemon636-21snmpd11:45:532
29157682101,4sleep02915771-21apt-get09:40:300
3123832290,6sleep13125922-21cut12:20:161
3051832290,7sleep13052281-21cut11:25:161
3024452290,2sleep20-21swapper/211:00:212
30179890,5rtkit-daemon430-21avahi-daemon12:15:402
30179890,5rtkit-daemon3205-21sendmail-mta07:30:400
30179890,5rtkit-daemon2993673-21cut10:40:181
30179890,5rtkit-daemon2870963-21diskstats09:05:121
30179890,4rtkit-daemon3145184-21cut12:35:172
30179890,4rtkit-daemon2853749-21grep08:50:180
30179890,4rtkit-daemon2826058-21entropy08:30:140
30179890,1rtkit-daemon0-21swapper/008:06:190
3007717290,6sleep00-21swapper/010:50:180
2920974290,5sleep22921050-21ntp_states09:40:402
2882574291,4sleep12882699-21python309:10:341
2813012291,4sleep22813045-21apt-get08:20:112
2787730290,6sleep12787917-21sed08:00:151
2762356290,6sleep22763360-21grep07:40:172
2756097290,7sleep12757962-21grep07:35:181
2743225290,6sleep22744751-21cut07:25:182
2718269291,4sleep22718941-21memory07:05:152
3090185281,4sleep13090331-21df11:55:131
3058508281,5sleep13059464-21kthreadcore11:30:171
30179880,5rtkit-daemon636-21snmpd11:15:400
30179880,4rtkit-daemon667-21lldpd11:40:080
30179880,4rtkit-daemon667-21lldpd10:45:361
30179880,4rtkit-daemon667-21lldpd09:39:061
30179880,4rtkit-daemon638-21ntpd11:55:182
30179880,4rtkit-daemon638-21ntpd08:15:110
30179880,4rtkit-daemon636-21snmpd10:11:201
30179880,4rtkit-daemon636-21snmpd09:46:311
30179880,4rtkit-daemon492-21gmain08:47:380
30179880,4rtkit-daemon448-21wpa_supplicant09:25:181
30179880,4rtkit-daemon448-21wpa_supplicant07:49:230
30179880,4rtkit-daemon447-21systemd-logind07:53:320
30179880,4rtkit-daemon3205-21sendmail-mta07:09:050
30179880,4rtkit-daemon317-21systemd-udevd12:31:332
30179880,4rtkit-daemon2726644-21munin-node12:06:152
30179880,4rtkit-daemon2726644-21munin-node11:11:570
30179880,4rtkit-daemon2726644-21munin-node09:50:221
30179880,4rtkit-daemon2726644-21munin-node09:22:491
30179880,4rtkit-daemon2726644-21munin-node08:25:060
30179880,4rtkit-daemon2726644-21munin-node07:22:210
30179880,1rtkit-daemon0-21swapper/212:00:292
30179880,1rtkit-daemon0-21swapper/109:59:231
30179880,1rtkit-daemon0-21swapper/109:41:351
3008933281,4sleep13011141-21grep10:50:191
2967532281,3sleep02967554-21apt-get10:20:130
2941646281,4sleep02941802-21diskstats10:00:140
2878668281,4sleep02879529-21tr09:10:300
2787531281,4sleep02787727-21cut08:00:140
2736315280,6sleep12737125-21cat07:20:171
2726497281,4sleep22726852-21munin-node07:15:012
3122407270,4sleep23122607-21hddtemp_smartct12:20:142
3122244270,4sleep03122255-21grep12:20:120
3084132271,4sleep03084378-21irqcore11:50:160
3071894270,4sleep23072931-21tr11:40:162
3069873270,5sleep23069963-21switchtime11:35:232
3064517270,4sleep13064666-21df_inode11:35:141
3044483271,3sleep23044568-21systemd-run11:20:012
3019084270,3sleep03019117-21apt-get11:00:100
30179870,4rtkit-daemon638-21ntpd09:18:461
30179870,4rtkit-daemon638-21ntpd07:20:000
30179870,4rtkit-daemon636-21snmpd10:05:341
30179870,4rtkit-daemon636-21snmpd10:00:161
30179870,4rtkit-daemon492-21gmain10:15:421
30179870,4rtkit-daemon492-21gmain09:00:501
30179870,4rtkit-daemon0-21swapper/011:25:410
30179870,3rtkit-daemon638-21ntpd11:20:160
30179870,3rtkit-daemon636-21snmpd12:10:202
30179870,3rtkit-daemon3205-21sendmail-mta10:30:041
30179870,3rtkit-daemon2904783-21kthreadcore09:30:161
30179870,3rtkit-daemon2726644-21munin-node10:55:480
30179870,3rtkit-daemon2726644-21munin-node10:25:121
30179870,3rtkit-daemon2726644-21munin-node08:55:471
30179870,3rtkit-daemon2726644-21munin-node08:35:380
3017906270,4sleep13017964-21ntp_states10:55:201
2992079271,3sleep00-21swapper/010:35:230
2923075270,4sleep22923990-21grep09:45:172
2915432271,3sleep22915692-21sed09:40:002
2847895271,4sleep22850109-21tr08:45:192
2845135270,4sleep12845281-21df08:45:141
2819149270,1sleep10-21swapper/108:24:471
2802425270,1sleep20-21swapper/208:10:182
2780934270,3sleep12780956-21apt-get07:55:101
2776457270,5sleep22777671-21kthreadcore07:50:182
2768325271,3sleep12769318-21irqstats07:45:171
3142127260,3sleep13142890-21irqrtprio12:35:161
3050878260,1sleep10-21swapper/111:21:081
3050284260,3sleep23050341-21ntp_states11:20:212
3037989260,1sleep20-21swapper/211:14:202
3026927260,4sleep10-21swapper/111:05:171
2993575260,3sleep22838499-21kworker/2:2+events10:40:182
2986447260,1sleep00-21swapper/010:34:510
2954650261,2sleep02954683-21apt-get10:10:110
2948254260,1sleep20-21swapper/210:05:082
2947705260,1sleep20-21swapper/210:04:322
2935177261,2sleep20-21swapper/209:55:152
2909711260,3sleep02910707-21irqstats09:35:180
2897884261,2sleep22899684-21kthreadcore09:25:182
2870515260,4sleep02870530-21munin-run09:05:010
2818441260,3sleep02818552-21find08:20:210
2795202261,2sleep20-21swapper/208:05:172
2787918261,3sleep22788933-21kthreadcore08:00:162
2773917260,1sleep20-21swapper/207:48:422
2761872260,4sleep12762355-21sed07:40:161
2748244261,3sleep12748400-21diskstats07:30:141
2737126261,3sleep22735335-21/usr/sbin/munin07:20:172
2720746260,4sleep20-21swapper/207:10:122
2720586260,3sleep12720745-21if_err_eno107:10:111
2716107261,2sleep10-21swapper/107:05:141
3108829250,1sleep00-21swapper/012:06:260
3106095250,2sleep13108026-21latency12:05:191
3096082250,1sleep00-21swapper/011:57:250
3077346250,2sleep10-21swapper/111:45:121
3076992250,1sleep10-21swapper/111:43:421
3059466251,2sleep20-21swapper/211:30:182
3025661250,3sleep23025763-21fschecks_time11:05:142
3024861250,1sleep10-21swapper/111:00:241
3012662250,1sleep20-21swapper/210:55:052
2999139250,1sleep00-21swapper/010:42:330
2992679250,1sleep20-21swapper/210:37:142
2966939250,3sleep20-21swapper/210:16:102
2966205250,2sleep02966351-21ntpq10:15:200
2960785250,1sleep20-21swapper/210:14:302
2941120250,1sleep00-21swapper/009:57:170
2928935251,2sleep00-21swapper/009:50:160
2928164250,1sleep00-21swapper/009:48:400
2904306250,3sleep02906116-21kthreadcore09:30:170
2902349250,1sleep00-21swapper/009:26:590
2890308250,3sleep02890492-21if_eno109:20:150
2883146250,1sleep20-21swapper/209:12:082
2870770250,3sleep22870805-21dpkg09:05:112
2866836251,2sleep20-21swapper/209:00:182
2863781250,1sleep00-21swapper/008:56:270
2858797250,2sleep20-21swapper/208:55:162
2844672250,1sleep10-21swapper/108:42:501
2831997250,1sleep20-21swapper/208:34:462
2825335250,1sleep20-21swapper/208:26:322
2812449250,1sleep20-21swapper/208:16:222
2800221250,2sleep12800364-21/usr/sbin/munin08:10:131
2793749250,2sleep10-21swapper/108:05:121
2780959250,3sleep22781114-21diskstats07:55:112
2774147250,2sleep10-21swapper/107:50:011
2747850250,1sleep10-21swapper/107:28:461
3147533240,3sleep00-21swapper/012:35:380
3135719240,2sleep13135999-21irqcore12:30:151
3121760240,3sleep10-21swapper/112:17:451
3115194240,3sleep00-21swapper/012:10:510
3098018240,1sleep00-21swapper/012:00:170
3089825240,1sleep10-21swapper/111:54:091
3057287240,3sleep20-21swapper/211:26:212
3043791242,1sleep10-21swapper/111:15:231
3035265240,1sleep10-21swapper/111:10:181
3006536240,2sleep20-21swapper/210:50:162
2973461240,1sleep20-21swapper/210:22:312
2891490240,2sleep20-21swapper/209:20:172
2883612240,3sleep00-21swapper/009:15:020
2857645240,2sleep20-21swapper/208:55:012
2832278240,1sleep20-21swapper/208:35:142
2832020240,2sleep12832105-21date08:35:011
2825534240,3sleep10-21swapper/108:29:081
2806387240,3sleep10-21swapper/108:15:011
2760480240,1sleep20-21swapper/207:36:202
3141290231,1sleep00-21swapper/012:32:500
3115764230,1sleep00-21swapper/012:15:040
3096553230,1sleep10-21swapper/112:00:061
3082166230,1sleep00-21swapper/011:45:190
2999890230,1sleep20-21swapper/210:45:152
2999622230,1sleep00-21swapper/010:45:130
2985360230,1sleep20-21swapper/210:30:202
2948252230,1sleep00-21swapper/010:05:060
2837362230,1sleep10-21swapper/108:35:211
2748401230,1sleep20-21swapper/207:30:152
2733948230,1sleep10-21swapper/107:15:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional