You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 11:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Mon Dec 22, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28196529912461,1242cyclictest29183262sleep320:24:563
28196529910600,1054cyclictest3124264-21sleep23:05:003
29461742180,1sleep10-21swapper/120:45:181
281965299188,7cyclictest31635282sleep323:35:183
2819652991814,2cyclictest2886028-21sleep20:00:003
281965299177,8cyclictest3015189-21turbostat21:44:553
281965299177,7cyclictest2853900-21turbostat19:34:593
2819652991613,1cyclictest0-21swapper/320:42:033
281965299161,11cyclictest3111728-21taskset22:55:143
281965299161,10cyclictest31762632sleep323:45:153
2819652991610,3cyclictest0-21swapper/323:55:003
281965299153,9cyclictest2873778-21chrt19:50:153
2819652991513,1cyclictest0-21swapper/320:30:013
2819652991512,1cyclictest0-21swapper/323:30:223
2819652991512,1cyclictest0-21swapper/321:52:543
2819652991512,1cyclictest0-21swapper/321:20:003
2819652991512,1cyclictest0-21swapper/319:25:253
281965299151,11cyclictest3110667-21taskset22:50:213
281965299151,11cyclictest3073186-21taskset22:25:133
281965299151,11cyclictest3073186-21taskset22:25:133
281965299151,11cyclictest2995921-21turbostat21:25:003
281965299147,4cyclictest28283862sleep319:15:143
281965299144,7cyclictest28593792sleep319:35:213
281965299141,9cyclictest29573092sleep320:55:153
2819652991413,0cyclictest0-21swapper/321:01:013
2819652991412,1cyclictest0-21swapper/321:30:153
281965299141,11cyclictest2860559-21taskset19:40:013
2819652991411,1cyclictest0-21swapper/322:20:003
2819652991411,1cyclictest0-21swapper/321:29:033
2819652991411,1cyclictest0-21swapper/320:52:373
2819652991411,1cyclictest0-21swapper/319:20:543
2819652991411,1cyclictest0-21swapper/319:14:153
2819652991411,1cyclictest0-21swapper/300:32:333
2819652991411,1cyclictest0-21swapper/300:26:493
281965299141,10cyclictest2907994-21taskset20:15:193
281965299141,10cyclictest2892934-21taskset20:05:013
281965299141,10cyclictest2892934-21taskset20:05:003
2819652991410,2cyclictest0-21swapper/322:05:123
30477982130,6sleep13047496-21/usr/sbin/munin22:05:161
281965299134,7cyclictest3155602-21taskset23:25:193
281965299134,7cyclictest3060004-21taskset22:14:313
281965299134,7cyclictest2892445-21turbostat20:05:003
281965299133,8cyclictest3169799-21taskset23:40:013
281965299131,9cyclictest3226700-21taskset00:20:243
281965299131,9cyclictest3189147-21taskset23:55:123
281965299131,9cyclictest2937496-21chrt20:35:383
281965299131,8cyclictest30924472sleep322:40:113
2819652991312,0cyclictest0-21swapper/322:49:133
2819652991312,0cyclictest0-21swapper/300:06:063
2819652991311,1cyclictest0-21swapper/323:20:013
2819652991311,1cyclictest0-21swapper/323:11:023
2819652991311,1cyclictest0-21swapper/323:07:133
2819652991311,1cyclictest0-21swapper/321:55:453
2819652991311,1cyclictest0-21swapper/300:15:103
281965299131,10cyclictest3027871-21taskset21:48:223
281965299131,10cyclictest2950592-21chrt20:49:083
2819652991310,1cyclictest0-21swapper/322:35:073
2819652991310,1cyclictest0-21swapper/321:13:063
2819652991310,1cyclictest0-21swapper/321:08:573
2819652991310,1cyclictest0-21swapper/319:48:513
2819652991310,1cyclictest0-21swapper/300:35:153
281965299130,10cyclictest3014927-21chrt21:36:533
281965299130,10cyclictest2930853-21chrt20:30:453
32265752121,8sleep23226699-21python300:20:242
32209742121,3sleep23221039-21unin-run00:20:002
31698152121,2sleep20-21swapper/223:40:132
301798120,7rtkit-daemon2995697-21sshd21:22:200
281965299121,9cyclictest3046931-21taskset22:01:053
2819652991211,0cyclictest0-21swapper/323:22:543
281965299120,9cyclictest3208389-21chrt00:10:023
281965299120,9cyclictest3195508-21taskset00:00:003
281965299120,9cyclictest2899354-21taskset20:10:023
281965299120,10cyclictest3079506-21taskset22:30:013
281965299120,10cyclictest3072889-21taskset22:24:573
28134142121,5sleep12813447-21apt-get19:05:141
31331622110,4sleep23135221-21grep23:10:202
301798110,7rtkit-daemon2826134-21munin-node21:43:480
301798110,5rtkit-daemon441-21rsyslogd19:50:120
28924302110,3sleep22892447-21apt-get20:05:012
28924302110,3sleep22892447-21apt-get20:05:012
32392972101,5sleep23239350-21ntp_states00:30:222
31426212101,5sleep13142674-21ntp_states23:15:201
301798100,6rtkit-daemon676-21runrttasks23:30:090
301798100,6rtkit-daemon3104206-21/usr/sbin/munin22:45:210
301798100,6rtkit-daemon3034227-21runrttasks21:52:560
301798100,6rtkit-daemon3012986-21grep21:35:210
301798100,6rtkit-daemon2826134-21munin-node23:17:270
301798100,6rtkit-daemon2826134-21munin-node20:18:400
301798100,5rtkit-daemon3053943-21apt-get22:10:130
301798100,3rtkit-daemon636-21snmpd19:43:270
301798100,3rtkit-daemon636-21snmpd00:04:321
301798100,3rtkit-daemon2826134-21munin-node20:34:260
301798100,3rtkit-daemon2826134-21munin-node20:21:530
301798100,3rtkit-daemon2826134-21munin-node00:09:141
301798100,1rtkit-daemon0-21swapper/023:28:310
28416662100,7sleep12842329-21sed19:25:171
3208404291,4sleep13208437-21apt-get00:10:131
3111692291,4sleep23111725-21apt-get22:55:132
3054359290,6sleep13055288-21cut22:10:171
30179890,5rtkit-daemon2826134-21munin-node23:20:130
30179890,5rtkit-daemon2826134-21munin-node22:59:410
30179890,5rtkit-daemon2826134-21munin-node22:20:140
30179890,5rtkit-daemon2826134-21munin-node21:30:140
30179890,4rtkit-daemon3200908-21packagekitd20:10:160
30179890,4rtkit-daemon3105310-21apt-get22:50:110
30179890,4rtkit-daemon2931199-21apt-get20:35:000
30179890,4rtkit-daemon2923180-21tr20:25:180
30179890,4rtkit-daemon2852841-21/usr/sbin/munin19:30:190
30179890,1rtkit-daemon0-21swapper/020:52:200
30179890,1rtkit-daemon0-21swapper/019:56:390
2983562290,2sleep10-21swapper/121:15:161
2963744291,4sleep12963777-21apt-get21:00:131
3234038280,3sleep13234071-21apt-get00:30:131
3214840281,3sleep13214872-21apt-get00:15:131
3202024280,4sleep23202167-21df00:05:142
3175179281,4sleep03175302-21/usr/sbin/munin23:40:230
3130085281,4sleep23130305-21users23:05:232
3118104281,4sleep03118137-21apt-get23:00:130
3104466281,4sleep23104651-21users22:45:242
3085934281,3sleep03085967-21apt-get22:35:120
3079916280,6sleep00-21swapper/022:30:150
3079261281,4sleep13079306-21/usr/sbin/munin22:30:001
3079261281,4sleep13079306-21/usr/sbin/munin22:30:001
30179880,5rtkit-daemon3205-21sendmail-mta23:50:011
30179880,5rtkit-daemon317-21systemd-udevd22:25:332
30179880,5rtkit-daemon317-21systemd-udevd22:25:332
30179880,4rtkit-daemon667-21lldpd21:05:120
30179880,4rtkit-daemon638-21ntpd21:55:170
30179880,4rtkit-daemon638-21ntpd21:46:050
30179880,4rtkit-daemon636-21snmpd23:14:330
30179880,4rtkit-daemon636-21snmpd23:05:120
30179880,4rtkit-daemon636-21snmpd19:25:510
30179880,4rtkit-daemon636-21snmpd19:15:240
30179880,4rtkit-daemon636-21snmpd00:36:161
30179880,4rtkit-daemon636-21snmpd00:27:421
30179880,4rtkit-daemon492-21gmain20:05:360
30179880,4rtkit-daemon492-21gmain20:05:360
30179880,4rtkit-daemon2960761-21kthreadcore20:55:200
30179880,4rtkit-daemon2826134-21munin-node22:08:490
30179880,4rtkit-daemon2780494-21munin-node19:11:170
30179880,4rtkit-daemon2780494-21munin-node19:06:530
30179880,3rtkit-daemon3060335-21apt-get22:15:150
30179880,1rtkit-daemon0-21swapper/023:36:570
30179880,1rtkit-daemon0-21swapper/021:16:370
3009018281,3sleep13009051-21apt-get21:35:151
2968550280,5sleep02968972-21tr21:00:200
2950659281,4sleep22950680-21munin-run20:50:012
2943352281,5sleep12943411-21ntpq20:40:221
2872205280,4sleep12872276-21ntp_states19:45:211
2840353281,4sleep12840598-21awk19:20:251
3243014270,4sleep23245172-21grep00:35:192
3184147270,5sleep23185848-21ps23:50:192
3144258271,3sleep23144458-21hddtemp_smartct23:20:142
3132388270,4sleep13133161-21tr23:10:191
3104939270,1sleep10-21swapper/122:49:001
3093902272,3sleep03095278-21kthreadcore22:40:170
3035386271,3sleep23036258-21kthreadcore21:55:162
30179870,4rtkit-daemon667-21lldpd23:49:141
30179870,4rtkit-daemon492-21gmain21:12:040
30179870,4rtkit-daemon2826134-21munin-node19:35:320
30179870,3rtkit-daemon638-21ntpd23:59:091
30179870,3rtkit-daemon636-21snmpd22:00:180
30179870,3rtkit-daemon636-21snmpd20:42:220
30179870,3rtkit-daemon636-21snmpd19:45:500
30179870,3rtkit-daemon636-21snmpd00:22:121
30179870,3rtkit-daemon2999759-21cut21:25:200
30179870,3rtkit-daemon2826134-21munin-node20:03:520
30179870,3rtkit-daemon2826134-21munin-node19:21:590
3009054273,2sleep20-21swapper/221:35:162
2996162270,3sleep12996195-21apt-get21:25:141
2970193271,3sleep12970225-21apt-get21:05:141
2965197271,4sleep22966408-21cat21:00:172
2945288271,3sleep02946173-21cut20:45:180
2918330270,5sleep22918332-21cron20:24:592
2910995270,4sleep22911054-21ntp_states20:15:232
2905759270,3sleep12905792-21apt-get20:15:141
2901185270,4sleep22903184-21grep20:10:222
2899409270,5sleep12899484-21cut20:10:171
2824041271,3sleep12824962-21missed_timers19:10:191
2818614270,4sleep30-21swapper/319:05:243
3189111261,2sleep20-21swapper/223:55:122
3177710261,2sleep00-21swapper/023:45:200
3150658261,3sleep13150684-21grep23:25:141
3119558260,4sleep10-21swapper/123:00:181
3106768261,2sleep20-21swapper/222:50:172
3092735260,4sleep23092448-21/usr/sbin/munin22:40:152
3087388260,4sleep23088425-21cut22:35:182
3047495261,2sleep23047528-21apt-get22:05:132
3041069260,4sleep1636-21snmpd22:00:031
3035039261,2sleep10-21swapper/121:55:151
3029711261,2sleep10-21swapper/121:50:181
30179860,3rtkit-daemon638-21ntpd23:44:101
30179860,3rtkit-daemon3205-21sendmail-mta22:30:162
3016946260,4sleep13018847-21tr21:40:191
3008688260,1sleep20-21swapper/221:34:082
3001397260,3sleep23001474-21ntp_states21:25:232
2983310260,2sleep22983343-21apt-get21:15:132
2926666260,4sleep12928698-21cut20:30:161
2886466260,3sleep22886666-21/usr/sbin/munin20:00:122
2873577260,4sleep22873777-21hddtemp_smartct19:50:152
2866593260,1sleep10-21swapper/119:43:321
2849301261,2sleep10-21swapper/119:30:181
2834826261,2sleep20-21swapper/219:20:142
3240907250,3sleep03241858-21cut00:35:170
3240122250,1sleep00-21swapper/000:33:590
3233614250,1sleep20-21swapper/200:27:572
3214542250,1sleep20-21swapper/200:14:312
3201394250,1sleep20-21swapper/200:00:412
3182181250,1sleep20-21swapper/223:46:502
3156342250,1sleep20-21swapper/223:26:152
3145580251,2sleep10-21swapper/123:20:171
3143192250,1sleep20-21swapper/223:15:492
3124129250,1sleep20-21swapper/223:03:392
3111944250,2sleep13112133-21if_eno122:55:161
3111147250,1sleep10-21swapper/122:51:201
3095279250,2sleep13097459-21kthreadcore22:40:181
3079239250,3sleep00-21swapper/022:29:430
3079239250,3sleep00-21swapper/022:29:430
3072737250,1sleep20-21swapper/222:22:592
3046930250,1sleep20-21swapper/222:01:052
3034054250,1sleep20-21swapper/221:50:432
3021498250,1sleep20-21swapper/221:43:372
2995579250,1sleep20-21swapper/221:20:502
2992221250,3sleep10-21swapper/121:20:171
2976720250,2sleep10-21swapper/121:10:151
2976119250,1sleep20-21swapper/221:06:472
2962659250,2sleep12962777-21smartctl20:55:251
2939617251,2sleep22941620-21kthreadcore20:40:192
2937022250,3sleep12937144-21sensors_temp20:35:251
2931700250,1sleep20-21swapper/220:35:102
2931111250,1sleep20-21swapper/220:34:062
2885886250,1sleep20-21swapper/219:58:362
2879325250,1sleep10-21swapper/119:52:021
2868422251,2sleep20-21swapper/219:45:172
2860273250,1sleep10-21swapper/119:39:441
2860094250,1sleep20-21swapper/219:37:242
2842330250,3sleep22843493-21grep19:25:182
2834233250,1sleep10-21swapper/119:16:041
2818893250,2sleep20-21swapper/219:05:262
3227657240,1sleep00-21swapper/000:25:150
3227041240,3sleep00-21swapper/000:20:340
3214356240,3sleep00-21swapper/000:12:090
3207847240,3sleep00-21swapper/000:06:120
3195059240,3sleep00-21swapper/023:57:460
3182689240,2sleep00-21swapper/023:50:120
3165078240,2sleep20-21swapper/223:35:212
3161185241,1sleep10-21swapper/123:30:191
3156894240,1sleep23156927-21apt-get23:30:122
3129950240,2sleep10-21swapper/123:05:221
3091892240,3sleep10-21swapper/122:37:101
3080287240,2sleep13081192-21kthreadcore22:30:171
3069536240,2sleep13071680-21kthreadcore22:20:181
3066396240,3sleep10-21swapper/122:19:311
3060649240,2sleep20-21swapper/222:15:192
3059877240,3sleep20-21swapper/222:12:552
3023317240,2sleep10-21swapper/121:45:171
3022049240,2sleep23022141-21/usr/sbin/munin21:45:142
3002757240,2sleep13002578-21/usr/sbin/munin21:30:141
2962778240,1sleep20-21swapper/220:55:262
2952870240,2sleep10-21swapper/120:50:201
2950917240,1sleep20-21swapper/220:50:132
2918588240,1sleep20-21swapper/220:25:122
2918078240,3sleep10-21swapper/120:21:471
2894318240,2sleep12895348-21cat20:05:161
2894318240,2sleep12895348-21cat20:05:161
2887892240,2sleep10-21swapper/120:00:151
2824965240,1sleep20-21swapper/219:10:202
3220941230,1sleep00-21swapper/000:19:340
3197301230,1sleep00-21swapper/000:00:190
3163370230,1sleep10-21swapper/123:35:151
2976834230,1sleep20-21swapper/221:10:162
2918893230,1sleep10-21swapper/120:25:151
2880608230,1sleep10-21swapper/119:55:181
2862201230,1sleep20-21swapper/219:40:172
2847895230,1sleep20-21swapper/219:30:152
2825980230,1sleep20-21swapper/219:15:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional