You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 22:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Fri Dec 26, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296233421200,3sleep2421rcuc/212:30:182
2618299211759,54sleep30-21swapper/307:06:153
261838628351,10sleep10-21swapper/107:07:211
261370527762,9sleep20-21swapper/207:05:102
261829127658,12sleep00-21swapper/007:06:080
2618873992724,1cyclictest0-21swapper/311:19:593
2618873992421,1cyclictest0-21swapper/310:55:173
2618873992421,1cyclictest0-21swapper/307:35:573
2618873992320,1cyclictest0-21swapper/312:30:003
2618873992219,1cyclictest0-21swapper/311:45:153
2618873992219,1cyclictest0-21swapper/311:45:003
2618873992219,1cyclictest0-21swapper/311:05:103
2618873992219,1cyclictest0-21swapper/310:05:143
2618873992219,1cyclictest0-21swapper/309:55:013
2618873992219,1cyclictest0-21swapper/308:55:203
2618873992218,2cyclictest0-21swapper/308:00:003
2618873992119,1cyclictest0-21swapper/307:30:013
2618873992118,1cyclictest0-21swapper/307:30:103
2618873992117,2cyclictest0-21swapper/312:05:003
261887399209,7cyclictest2726582-21turbostat08:50:013
261887399208,8cyclictest2953190-21sleep12:25:003
2618873992017,1cyclictest0-21swapper/312:35:013
2618873992017,1cyclictest0-21swapper/311:55:163
2618873992017,1cyclictest0-21swapper/308:15:163
2618873992017,1cyclictest0-21swapper/307:10:213
2618873992016,1cyclictest0-21swapper/307:20:003
2618873991916,1cyclictest0-21swapper/309:45:013
2618873991916,1cyclictest0-21swapper/309:15:183
2618873991916,1cyclictest0-21swapper/307:20:163
2618873991915,2cyclictest0-21swapper/311:30:243
2618873991915,2cyclictest0-21swapper/309:30:003
2618873991915,2cyclictest0-21swapper/308:05:003
2618873991910,5cyclictest2663698-21turbostat07:50:013
28835602181,11sleep02883608-21ntpq11:15:220
261887399182,11cyclictest2821977-21taskset10:20:133
2618873991816,1cyclictest0-21swapper/311:50:193
2618873991815,1cyclictest0-21swapper/310:30:173
2618873991815,1cyclictest0-21swapper/309:10:123
2618873991815,1cyclictest0-21swapper/309:05:273
2618873991815,1cyclictest0-21swapper/308:50:173
2618873991814,2cyclictest0-21swapper/310:45:123
2618873991814,2cyclictest0-21swapper/310:40:213
2618873991814,2cyclictest0-21swapper/310:30:003
2618873991814,2cyclictest0-21swapper/308:35:183
261887399181,13cyclictest0-21swapper/312:15:113
261887399181,10cyclictest2803897-21taskset10:00:163
261887399180,11cyclictest2853403-21taskset10:50:073
261887399179,4cyclictest27902182sleep309:50:123
261887399173,11cyclictest2811992-21taskset10:10:143
2618873991715,1cyclictest0-21swapper/311:10:153
2618873991714,1cyclictest0-21swapper/312:11:443
2618873991714,1cyclictest0-21swapper/311:35:133
2618873991714,1cyclictest0-21swapper/311:20:163
2618873991714,1cyclictest0-21swapper/310:15:153
2618873991714,1cyclictest0-21swapper/309:35:173
2618873991714,1cyclictest0-21swapper/307:50:263
2618873991713,2cyclictest0-21swapper/307:45:013
261887399171,12cyclictest0-21swapper/311:00:133
261887399171,12cyclictest0-21swapper/310:35:213
261887399171,10cyclictest2716277-21taskset08:40:013
2618873991614,1cyclictest0-21swapper/309:45:153
261887399161,3cyclictest2890332-21taskset11:25:143
2618873991613,1cyclictest0-21swapper/312:05:233
2618873991613,1cyclictest0-21swapper/309:20:163
2618873991613,1cyclictest0-21swapper/308:20:173
2618873991613,1cyclictest0-21swapper/308:05:143
2618873991612,2cyclictest0-21swapper/309:00:123
29534422151,9sleep12953628-21cat12:25:131
27852292151,9sleep12785332-21/usr/sbin/munin09:45:131
261887399158,3cyclictest0-21swapper/308:10:113
2618873991513,1cyclictest0-21swapper/309:30:133
2618873991513,1cyclictest0-21swapper/308:25:233
2618873991512,1cyclictest0-21swapper/308:30:113
261887399151,12cyclictest0-21swapper/312:35:163
28638992141,9sleep12863918-21apt-get11:00:121
28625162141,8sleep02862605-21ntpq10:55:180
28154222140,9sleep02815471-21ntpq10:10:210
28732212133,2sleep0171rcu_preempt11:05:200
28216702131,8sleep12821727-21grep10:20:001
27167952131,8sleep22716867-21kernelversion08:40:172
26496602131,8sleep12651410-21grep07:35:191
29487252121,7sleep02948791-21kthreadcore12:20:150
29364942121,4sleep02936538-21ntpq12:05:200
29273442121,7sleep12927272-21/usr/sbin/munin12:00:111
28913502124,4sleep22893053-21cut11:25:182
28339482121,7sleep02835682-21cut10:30:170
28059882121,5sleep02806036-21munin-run10:05:000
27782482121,7sleep02778293-21open_files09:35:220
26589872120,8sleep12659046-21grep07:45:131
29322462110,7sleep02932393-21latency_hist12:05:000
29246402111,5sleep02925830-21/usr/sbin/munin11:55:180
28576902111,6sleep12857853-21timerwakeupswit10:50:261
28530762110,1sleep10-21swapper/110:49:091
28429182111,4sleep12842937-21apt-get10:40:121
27439852110,6sleep22745681-21kthreadcore09:05:182
27005842111,5sleep22700840-21df08:25:142
26949852110,2sleep10-21swapper/108:19:201
26688922111,5sleep22668940-21munin-run07:55:002
26433412111,6sleep12644772-21kthreadcore07:30:141
29592322101,5sleep02960539-21kthreadcore12:30:170
29302992101,4sleep22931088-21/usr/sbin/munin12:00:172
28822492101,4sleep12883372-21memory11:15:191
28749402101,5sleep22876325-21cut11:10:152
28626062101,5sleep12862653-21ntpq10:55:191
28600842101,5sleep20-21swapper/210:55:162
28416752101,5sleep014-21ksoftirqd/010:35:220
28379872101,5sleep12838090-21perl10:35:151
28324472101,5sleep22832455-21acpi10:30:112
28262492101,6sleep12826507-21sed10:20:281
28258332101,4sleep22825921-21ntpq10:20:212
28245142101,4sleep02825769-21memory10:20:190
28007322101,5sleep12800813-21wc10:00:001
27746662101,5sleep22774761-21kthreadcore09:35:162
27676972101,4sleep02767744-21ntpq09:25:200
27638112101,4sleep12764018-21diskstats09:25:131
27268432101,4sleep02727099-21df08:50:130
27205962101,4sleep02720670-21find08:40:270
27098762102,5sleep134-21ksoftirqd/108:30:221
27057952101,4sleep22705814-21apt-get08:30:112
26953272100,1sleep10-21swapper/108:20:121
26825532101,5sleep20-21swapper/208:05:182
26691542101,3sleep20-21swapper/207:55:122
26679172102,3sleep02667963-21ntpq07:50:210
26575102101,5sleep22657563-21ntp_states07:40:202
26574032101,4sleep02657460-21ntpq07:40:190
26360212100,7sleep20-21swapper/207:20:212
26194612101,4sleep12619673-21hddtemp_smartct07:10:141
2968513291,4sleep12968758-21unixbench_multi12:35:231
2964186291,4sleep22964190-21find12:35:082
2958825291,4sleep12958948-21diskstats12:30:141
2943540291,4sleep12944616-21kthreadcore12:15:151
2942712291,3sleep22942760-21munin-run12:15:002
2905902291,4sleep12906025-21diskstats11:40:161
2904491291,3sleep02904536-21ntpq11:35:220
2895260291,3sleep12895279-21apt-get11:30:101
2888661291,3sleep02888774-21ntpq11:20:190
2886276291,4sleep22888035-21grep11:20:172
2878700291,4sleep02878945-21users11:10:250
2873084291,3sleep12873124-21ntpq11:05:171
2854336291,4sleep02856006-21cut10:50:170
2828651291,5sleep02830385-21kthreadcore10:25:150
2820558291,4sleep22820659-21ntpq10:15:202
2815719291,4sleep12815794-21timerandwakeup10:10:261
2773238291,4sleep12773498-21sed09:30:261
2764019291,4sleep22764209-21grep09:25:142
2753585291,4sleep12753796-21if_enp1s009:15:151
2742551291,4sleep02742600-21munin-run09:05:000
2741602291,4sleep22741656-21ntpq09:00:212
2738149291,4sleep12739322-21grep09:00:161
2732352291,3sleep22732370-21apt-get08:55:122
2711080291,3sleep02711294-21df_inode08:35:140
2695349291,4sleep22695493-21cstates08:20:132
2689805291,4sleep22689815-21sh08:15:002
2658449291,4sleep12658584-21grep07:45:011
2653125291,4sleep22653340-21cat07:40:012
2652804291,4sleep02652751-31find07:36:060
2637282291,3sleep12637520-21df_inode07:25:131
2628332291,4sleep02630069-21cut07:15:180
2968238281,4sleep02968291-21ld12:35:180
2921235280,1sleep00-21swapper/011:51:490
2917012281,3sleep12917208-21kthreadcore11:50:181
2911809281,4sleep22912924-21kthreadcore11:45:142
2906309281,3sleep02906341-21kthreadcore11:40:180
2889828280,1sleep10-21swapper/111:25:011
2869153281,3sleep22869369-21cstates11:05:102
2864471281,3sleep22865702-21kthreadcore11:00:172
2846900282,2sleep20-21swapper/210:40:212
2841789280,1sleep20-21swapper/210:35:242
2831759280,1sleep20-21swapper/210:25:262
2818091281,3sleep0221rcuc/010:15:180
2807860282,3sleep12809622-21kthreadcore10:05:171
2806817281,3sleep02807859-21grep10:05:160
2790196280,2sleep20-21swapper/209:50:102
2789938281,3sleep02790134-21latency_hist09:50:010
2784392280,1sleep00-21swapper/009:44:480
2758540281,3sleep00-21swapper/009:20:100
2731610280,1sleep20-21swapper/208:52:282
2722047281,4sleep22722129-21kernelversion08:45:132
2706072281,3sleep02706096-21fschecks_count08:30:130
2700300280,1sleep10-21swapper/108:25:001
2690636281,3sleep00-21swapper/008:15:180
2642767281,2sleep02642786-21apt-get07:30:090
2636966280,5sleep00-21swapper/007:24:560
2632590281,3sleep12633486-21cut07:20:151
2953420270,2sleep00-21swapper/012:25:120
2942505270,1sleep00-21swapper/012:12:260
2926862270,1sleep10-21swapper/111:58:281
2915894270,1sleep00-21swapper/011:45:440
2899327271,3sleep02899450-21/usr/sbin/munin11:30:210
2895282271,3sleep20-21swapper/211:30:112
2879190271,3sleep12879191-21sshd11:13:271
2852899270,1sleep00-21swapper/010:46:520
2801016270,2sleep10-21swapper/110:00:121
2800381270,1sleep00-21swapper/009:55:340
2795720270,2sleep20-21swapper/209:55:122
2794846270,1sleep00-21swapper/009:50:450
2790487271,2sleep10-21swapper/109:50:141
2789778270,1sleep20-21swapper/209:48:032
2784411271,3sleep10-21swapper/109:45:001
2784187270,1sleep20-21swapper/209:42:132
2779177270,1sleep10-21swapper/109:40:001
2772797270,2sleep20-21swapper/209:30:192
2753259271,2sleep22753278-21apt-get09:15:122
2751996271,2sleep12752045-21ntpq09:10:181
2749485271,2sleep20-21swapper/209:10:152
2743374271,3sleep12743984-21cat09:05:171
2721286273,1sleep10-21swapper/108:45:011
2680177270,3sleep02680861-21cat08:05:150
2679324270,1sleep10-21swapper/108:04:331
2675270271,2sleep20-21swapper/208:00:172
2674109270,1sleep00-21swapper/007:59:470
2673834270,1sleep10-21swapper/107:56:191
2663765270,4sleep10-21swapper/107:50:021
2659743271,3sleep02661409-21grep07:45:160
2647626270,1sleep20-21swapper/207:33:392
2637804271,2sleep00-21swapper/007:25:160
2637521271,3sleep22637685-21gltestperf07:25:142
2619169270,2sleep20-21swapper/207:10:112
2955595260,2sleep20-21swapper/212:25:182
2941488260,2sleep12941661-21strings12:10:181
2922023260,2sleep20-21swapper/211:55:122
2917210261,2sleep20-21swapper/211:50:192
2915968260,1sleep10-21swapper/111:46:411
2904544261,2sleep10-21swapper/111:35:231
2894157260,3sleep00-21swapper/011:25:240
2885383261,2sleep10-21swapper/111:20:161
2858052260,1sleep20-21swapper/210:50:512
2852974260,1sleep20-21swapper/210:47:512
2847794260,1sleep00-21swapper/010:43:320
2831289261,2sleep10-21swapper/110:25:211
2810281261,2sleep20-21swapper/210:05:222
2802176261,2sleep20-21swapper/210:00:172
2770447261,2sleep00-21swapper/009:30:170
2762776261,2sleep10-21swapper/109:20:241
2753252260,1sleep00-21swapper/009:15:060
2737017260,1sleep00-21swapper/008:56:300
2736945260,1sleep10-21swapper/108:55:351
2725851261,2sleep10-21swapper/108:45:251
2723596261,2sleep00-21swapper/008:45:160
2715834260,1sleep20-21swapper/208:37:522
2715823260,1sleep10-21swapper/108:37:441
2699076261,2sleep00-21swapper/008:20:190
2680862261,2sleep10-21swapper/108:05:171
2663382260,1sleep20-21swapper/207:46:142
2630641262,1sleep20-21swapper/207:15:202
2950826250,1sleep20-21swapper/212:20:172
2944618250,2sleep20-21swapper/212:15:162
2943423250,1sleep00-21swapper/012:15:140
2934759250,1sleep10-21swapper/112:05:171
2910094251,1sleep20-21swapper/211:40:292
2836761250,1sleep10-21swapper/110:30:261
2815252250,1sleep20-21swapper/210:10:192
2758808250,1sleep20-21swapper/209:20:132
2752714250,3sleep00-21swapper/009:11:250
2694936250,3sleep20-21swapper/208:18:412
2685133250,2sleep10-21swapper/108:10:141
2679361250,2sleep00-21swapper/008:05:000
2625414250,1sleep10-21swapper/107:15:031
2623250250,1sleep00-21swapper/007:10:200
2948209240,1sleep10-21swapper/112:20:121
2932503240,1sleep20-21swapper/212:05:102
2900474240,1sleep20-21swapper/211:35:122
2879906240,1sleep20-21swapper/211:15:162
2864189240,1sleep00-21swapper/011:00:150
2742812240,1sleep00-21swapper/009:05:120
2726714240,1sleep10-21swapper/108:50:021
2700913240,1sleep00-21swapper/008:25:160
2685108240,1sleep00-21swapper/008:10:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional