You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 13:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Fri Nov 21, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26567109912860,1282cyclictest27277242sleep320:04:563
26567109912180,1214cyclictest28344412sleep321:25:013
26567109912080,1204cyclictest27910712sleep320:54:553
26567109911330,1129cyclictest30768152sleep323:54:563
26567109911160,1113cyclictest27593772sleep320:29:563
26567109910504,1041cyclictest2696412-21sleep19:40:013
2656710999801,975cyclictest2662958-21sleep19:15:013
265671099830,79cyclictest3011734-21sleep23:19:553
2656710992522,1cyclictest0-21swapper/320:00:003
265671099177,7cyclictest29718172sleep322:50:133
2656710991715,1cyclictest0-21swapper/322:45:003
2656710991714,1cyclictest0-21swapper/323:25:133
265671099165,8cyclictest2683829-21turbostat19:34:553
2656710991613,1cyclictest0-21swapper/323:05:243
28927792150,2sleep00-21swapper/022:00:170
2656710991513,1cyclictest0-21swapper/323:46:463
2656710991513,1cyclictest0-21swapper/322:39:593
2656710991513,1cyclictest0-21swapper/322:15:053
2656710991513,0cyclictest0-21swapper/320:56:003
2656710991513,0cyclictest0-21swapper/300:32:183
2656710991512,1cyclictest0-21swapper/323:40:173
2656710991512,1cyclictest0-21swapper/322:26:273
2656710991512,1cyclictest0-21swapper/321:55:203
2656710991512,1cyclictest0-21swapper/321:40:143
2656710991512,1cyclictest0-21swapper/319:54:013
265671099151,11cyclictest3118532-21taskset00:20:173
265671099151,11cyclictest2964215-21taskset22:45:163
265671099151,11cyclictest2866771-21taskset21:45:013
265671099151,11cyclictest2753032-21taskset20:23:473
265671099145,6cyclictest28578002sleep321:38:333
265671099145,6cyclictest28334642sleep321:23:143
265671099144,8cyclictest2734510-21taskset20:10:003
265671099141,9cyclictest3004039-21taskset23:10:123
265671099141,9cyclictest3004039-21taskset23:10:113
265671099141,9cyclictest2671536-21taskset19:20:093
2656710991413,0cyclictest0-21swapper/300:11:333
2656710991412,1cyclictest0-21swapper/322:55:503
2656710991412,1cyclictest0-21swapper/320:49:263
2656710991412,0cyclictest0-21swapper/320:30:383
265671099141,10cyclictest3050741-21taskset23:36:173
265671099141,10cyclictest3027067-21taskset23:22:273
265671099141,10cyclictest3027067-21taskset23:22:273
265671099141,10cyclictest2930611-21taskset22:23:403
265671099141,10cyclictest2907316-21chrt22:10:093
265671099141,10cyclictest2683829-21turbostat19:30:013
26505872141,7sleep12650619-21apt-get19:05:121
30299582134,6sleep03031274-21cut23:25:170
29472072131,9sleep02947214-21sshd22:34:560
265671099134,7cyclictest2696639-21taskset19:40:043
265671099134,6cyclictest29878962sleep323:00:083
265671099133,7cyclictest2740787-21taskset20:15:023
265671099131,9cyclictest3141876-21taskset00:35:123
265671099131,9cyclictest3099471-21taskset00:06:023
265671099131,9cyclictest3036546-21taskset23:30:083
265671099131,9cyclictest2892041-21taskset22:00:163
265671099131,9cyclictest2824601-21chrt21:16:243
265671099131,9cyclictest2810737-21taskset21:10:153
265671099131,9cyclictest2705736-21taskset19:45:173
2656710991312,0cyclictest0-21swapper/320:41:423
2656710991311,1cyclictest0-21swapper/321:04:423
2656710991311,1cyclictest0-21swapper/319:15:123
265671099131,10cyclictest2850176-21taskset21:34:303
265671099130,10cyclictest2904623-21taskset22:05:213
265671099130,10cyclictest2875049-21chrt21:50:143
265671099130,10cyclictest2809864-21taskset21:08:553
265671099129,1cyclictest0-21swapper/319:10:003
265671099121,9cyclictest3116288-21taskset00:17:453
265671099121,9cyclictest3083512-21taskset23:56:373
265671099121,10cyclictest3132338-21chrt00:27:213
265671099120,9cyclictest3085265-21taskset00:00:013
265671099120,9cyclictest2939502-21taskset22:30:013
265671099120,9cyclictest2771899-21taskset20:35:533
265671099120,9cyclictest27277402sleep320:05:013
190898120,5rtkit-daemon2888976-21vmstat21:55:280
28341902111,7sleep12834229-21awk21:24:591
27159142110,1sleep10-21swapper/119:55:161
190898110,7rtkit-daemon2858044-21ssh21:39:030
190898110,4rtkit-daemon2873461-21ssh21:47:170
190898110,3rtkit-daemon471rcuog/220:02:282
190898110,3rtkit-daemon3101066-21ssh00:09:432
30858372101,6sleep23086111-21irqcore00:00:162
29555972100,7sleep22955603-21ssh22:40:072
27019742101,7sleep20-21swapper/219:40:202
26565482101,5sleep02656607-21munin-run19:10:010
190898100,6rtkit-daemon3026256-21users23:20:280
190898100,6rtkit-daemon3026256-21users23:20:280
190898100,6rtkit-daemon2995776-21latency_hist23:05:010
190898100,5rtkit-daemon667-21snmpd21:30:150
190898100,5rtkit-daemon2915822-21hddtemp_smartct22:15:160
190898100,5rtkit-daemon2859218-21hddtemp_smartct21:40:150
190898100,5rtkit-daemon2823622-21ntp_states21:15:210
190898100,3rtkit-daemon674-21lldpd22:06:470
190898100,3rtkit-daemon2663087-21munin-node22:22:360
3117173290,7sleep03117181-21ssh00:19:420
3101483291,5sleep13101520-21sh00:10:141
3086890290,6sleep03088684-21grep00:00:190
3051172290,6sleep23051180-21ssh23:37:132
2931733291,5sleep02931799-21/usr/sbin/munin22:25:180
2906324290,3sleep12906332-21ssh22:08:241
2882682291,4sleep02882733-21munin-run21:55:010
2826499291,4sleep02826534-21apt-get21:20:140
2721712291,4sleep12721744-21apt-get20:00:131
2690364291,5sleep22690520-21diskstats19:35:152
19089890,5rtkit-daemon667-21snmpd23:41:340
19089890,5rtkit-daemon438-21wpa_supplicant23:09:130
19089890,5rtkit-daemon2979268-21taskset22:54:270
19089890,5rtkit-daemon2790033-21sed20:50:192
19089890,4rtkit-daemon3123571-21ssh00:20:470
19089890,4rtkit-daemon2970791-21ssh22:48:230
19089890,4rtkit-daemon2664137-21grep19:15:022
19089890,3rtkit-daemon421-21avahi-daemon20:39:502
19089890,3rtkit-daemon2913735-21ssh22:11:520
19089890,2rtkit-daemon2643722-21kworker/u16:0+events_power_efficient19:11:242
3139586280,2sleep13139607-21ssh00:30:261
3060222281,5sleep13060228-21ssh23:44:261
3052760281,4sleep23052918-21df_inode23:40:142
3038815280,2sleep00-21swapper/023:30:180
3004006281,3sleep23004030-21apt-get23:10:122
3004006281,3sleep23004030-21apt-get23:10:122
2995918281,3sleep20-21swapper/223:05:162
2988137280,4sleep22988337-21hddtemp_smartct23:00:152
2937077280,5sleep22937327-21users22:25:272
2906102280,1sleep20-21swapper/222:07:492
2882252280,2sleep12882260-21ssh21:54:061
2866019280,6sleep12866027-21ssh21:43:481
2848079280,5sleep22848342-21grep21:30:252
2809054280,4sleep12809110-21ntp_states21:05:201
2760116281,4sleep02761103-21irqstats20:30:220
2702924280,4sleep12702957-21apt-get19:45:111
2681393281,4sleep02682779-21memory19:25:190
19089880,5rtkit-daemon299-21systemd-journal22:44:551
19089880,5rtkit-daemon1928-21sendmail-mta23:39:440
19089880,5rtkit-daemon1928-21sendmail-mta22:45:401
19089880,5rtkit-daemon1928-21sendmail-mta19:35:002
19089880,4rtkit-daemon667-21snmpd22:39:230
19089880,4rtkit-daemon438-21wpa_supplicant22:57:540
19089880,4rtkit-daemon3149545-21ssh00:39:500
19089880,4rtkit-daemon2663087-21munin-node20:48:572
19089880,4rtkit-daemon1928-21sendmail-mta00:31:500
19089880,3rtkit-daemon2958507-21cut22:40:170
19089880,3rtkit-daemon2835022-21/usr/sbin/munin21:25:150
19089880,3rtkit-daemon2708552-21users19:45:222
19089880,1rtkit-daemon0-21swapper/223:51:472
19089880,1rtkit-daemon0-21swapper/122:29:121
19089880,1rtkit-daemon0-21swapper/023:46:530
19089880,1rtkit-daemon0-21swapper/023:16:570
3143378272,3sleep13145121-21kthreadcore00:35:191
3141123271,4sleep23141131-21ssh00:33:592
3132989270,5sleep13132996-21sh00:28:541
3127887270,4sleep23129988-21cut00:25:172
3124893270,1sleep20-21swapper/200:23:502
3109592270,4sleep13109716-21tr00:15:141
3069858271,3sleep03070683-21kthreadcore23:50:170
3036564270,3sleep23036603-21apt-get23:30:122
3022897270,4sleep23024956-21cut23:20:222
3022897270,4sleep23024956-21cut23:20:212
3020054270,1sleep20-21swapper/223:19:582
3013303270,5sleep13014039-21kthreadcore23:15:161
3004203270,4sleep03004378-21grep23:10:140
3004203270,4sleep03004378-21grep23:10:140
2953148271,3sleep22953389-21users22:35:262
2929949271,3sleep12929957-21ssh22:22:111
2915447271,4sleep22915609-21diskstats22:15:152
2866850270,3sleep22866889-21apt-get21:45:122
2858792270,3sleep22858829-21apt-get21:40:122
2840819270,1sleep10-21swapper/121:26:371
2823144270,4sleep22823504-21grep21:15:202
2818043271,4sleep22818051-21ssh21:15:002
2810329271,4sleep02810481-21df21:10:130
2753415270,4sleep12753427-21grep20:25:131
2728701270,5sleep22729628-21irqrtprio20:05:162
2678204270,5sleep12678316-21irqcore19:25:161
2656857270,3sleep12656890-21apt-get19:10:141
19089870,4rtkit-daemon81364-21packagekitd22:30:081
19089870,4rtkit-daemon81364-21packagekitd21:01:182
19089870,4rtkit-daemon674-21lldpd23:56:282
19089870,4rtkit-daemon671-21ntpd19:05:432
19089870,4rtkit-daemon667-21snmpd19:20:472
19089870,4rtkit-daemon667-21snmpd00:12:132
19089870,4rtkit-daemon421-21avahi-daemon20:55:402
19089870,4rtkit-daemon1928-21sendmail-mta20:41:152
19089870,3rtkit-daemon674-21lldpd00:15:062
19089870,3rtkit-daemon667-21snmpd20:31:192
19089870,3rtkit-daemon667-21snmpd20:21:432
19089870,3rtkit-daemon667-21snmpd19:27:532
19089870,3rtkit-daemon667-21snmpd00:26:320
19089870,3rtkit-daemon299-21systemd-journal20:26:532
19089870,3rtkit-daemon2950521-21kthreadcore22:35:191
19089870,3rtkit-daemon2736564-21grep20:10:212
19089870,3rtkit-daemon2663087-21munin-node21:05:110
19089870,3rtkit-daemon2663087-21munin-node00:36:042
19089870,3rtkit-daemon1928-21sendmail-mta19:53:252
3091504260,1sleep10-21swapper/100:01:161
3078205260,3sleep03079218-21kthreadcore23:55:190
3066197260,3sleep23066413-21grep23:45:232
3042448260,3sleep13042456-21ssh23:30:431
3035478260,4sleep13035484-21ssh23:28:151
3027964260,1sleep10-21swapper/123:24:301
3027964260,1sleep10-21swapper/123:24:291
3009473260,3sleep13009715-21grep23:10:231
3009473260,3sleep13009715-21grep23:10:231
2994491260,1sleep10-21swapper/123:02:261
2986497260,1sleep10-21swapper/122:57:291
2913303260,4sleep12809962-21diskmemload22:10:491
2907758261,3sleep20-21swapper/222:10:212
2896299260,3sleep22896484-21cut22:00:202
2882749260,3sleep12882709-21apt-get21:55:021
2874984260,3sleep22875047-21tr21:50:142
2870277261,2sleep10-21swapper/121:45:191
2850877260,4sleep22850945-21fschecks_count21:35:152
2848847260,3sleep12848855-21ssh21:31:301
2841351260,1sleep20-21swapper/221:27:542
2833169260,1sleep20-21swapper/221:22:372
2810146260,4sleep10-21swapper/121:10:021
2791338261,2sleep00-21swapper/020:55:170
2790956260,1sleep00-21swapper/020:53:300
2752205260,3sleep12752270-21ntpq20:20:221
2742292260,3sleep02744139-21kthreadcore20:15:190
2734941261,3sleep10-21swapper/120:10:201
2715200261,3sleep12715372-21sort19:55:011
2709179260,3sleep02709212-21apt-get19:50:120
2698308260,3sleep12700356-21cut19:40:151
2671033260,1sleep00-21swapper/019:17:060
19089860,3rtkit-daemon2663087-21munin-node20:15:072
19089860,2rtkit-daemon0-21swapper/219:55:182
3095200250,3sleep13097142-21cut00:05:171
3094466251,2sleep00-21swapper/000:05:160
3076255250,3sleep10-21swapper/123:53:391
3063437250,3sleep12873096-21kworker/u16:1+kvfree_rcu_reclaim23:45:181
3051231250,1sleep10-21swapper/123:37:191
3034454250,1sleep20-21swapper/223:25:472
2999676250,3sleep12995921-21/usr/sbin/munin23:05:221
2986788250,3sleep22948706-21kworker/u16:2+rpciod22:58:102
2978084250,1sleep10-21swapper/122:51:401
2972202251,2sleep20-21swapper/222:50:162
2945355250,3sleep20-21swapper/222:30:332
2924262251,2sleep20-21swapper/222:20:192
2922664250,3sleep10-21swapper/122:19:021
2888802250,3sleep20-21swapper/221:55:272
2851771251,2sleep10-21swapper/121:35:181
2797155250,1sleep10-21swapper/120:57:321
2784703250,1sleep00-21swapper/020:48:490
2773546250,2sleep00-21swapper/020:40:190
2765377250,1sleep10-21swapper/120:31:231
2758282250,3sleep02758411-21needreboot20:25:210
2746733250,1sleep10-21swapper/120:18:491
2738855250,3sleep02739529-21cat20:10:230
2730115250,3sleep02732184-21kthreadcore20:05:180
2708820250,1sleep00-21swapper/019:48:430
2702619250,1sleep00-21swapper/019:44:230
2690329251,1sleep10-21swapper/119:35:141
2684073250,2sleep10-21swapper/119:30:131
2671934251,2sleep00-21swapper/019:20:140
2662580250,1sleep00-21swapper/019:10:330
3117823240,2sleep10-21swapper/100:20:161
3079221240,2sleep13081213-21tr23:55:201
2969638240,2sleep20-21swapper/222:45:402
2823666240,2sleep10-21swapper/121:15:231
2805399240,2sleep20-21swapper/221:05:172
2802960240,2sleep10-21swapper/121:00:241
2800036240,2sleep00-21swapper/021:00:190
2786510240,2sleep10-21swapper/120:50:191
2766178240,2sleep10-21swapper/120:35:121
2727454240,3sleep00-21swapper/020:01:280
2696022240,3sleep00-21swapper/019:35:320
2663337240,2sleep12664918-21kthreadcore19:15:031
3107258230,1sleep00-21swapper/000:10:290
2891495230,1sleep10-21swapper/122:00:151
2778984230,1sleep10-21swapper/120:45:181
2774626230,1sleep10-21swapper/120:40:201
2766177230,1sleep00-21swapper/020:35:110
2752962230,1sleep00-21swapper/020:22:530
2728499230,1sleep10-21swapper/120:05:151
2715444230,1sleep00-21swapper/019:55:040
2688998230,1sleep00-21swapper/019:30:200
2671569230,1sleep10-21swapper/119:20:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional