You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 10:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Thu Dec 25, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
38426499913110,1307cyclictest238572sleep323:59:563
38426499913040,1301cyclictest40376292sleep321:39:573
38426499910971,1091cyclictest4147159-21sleep23:05:003
4897821010,1sleep0471rcuog/200:15:200
38632432930,1sleep30-21swapper/319:20:213
3842649992421,1cyclictest0-21swapper/322:50:003
3842649992212,8cyclictest0-21swapper/320:24:213
3842649992111,1cyclictest0-21swapper/323:54:333
3842649992011,7cyclictest0-21swapper/322:53:353
3842649992011,0cyclictest0-21swapper/321:15:173
384264999197,9cyclictest3902581-21turbostat19:59:553
3842649991911,7cyclictest0-21swapper/319:41:473
3842649991910,0cyclictest0-21swapper/323:35:023
301798190,12rtkit-daemon3970021-21grep20:45:170
3842649991815,1cyclictest0-21swapper/320:40:143
3842649991810,1cyclictest0-21swapper/320:36:383
41542492171,7sleep24154800-21cut23:10:162
3842649991714,1cyclictest0-21swapper/323:25:253
3842649991710,0cyclictest0-21swapper/323:21:143
3842649991710,0cyclictest0-21swapper/319:16:173
3842649991710,0cyclictest0-21swapper/300:16:323
3842649991613,2cyclictest0-21swapper/322:35:023
3842649991613,2cyclictest0-21swapper/300:06:183
3842649991613,1cyclictest0-21swapper/321:00:083
3842649991612,2cyclictest0-21swapper/322:33:583
3842649991610,0cyclictest0-21swapper/300:10:553
384264999151,9cyclictest38643262sleep319:25:113
3842649991513,1cyclictest0-21swapper/320:16:163
3842649991512,1cyclictest0-21swapper/321:25:203
3842649991512,1cyclictest0-21swapper/320:50:003
3842649991512,1cyclictest0-21swapper/320:05:223
384264999145,6cyclictest39146272sleep320:00:253
384264999144,7cyclictest4069659-21taskset22:03:323
3842649991412,1cyclictest0-21swapper/322:58:063
3842649991412,1cyclictest0-21swapper/322:20:213
3842649991412,1cyclictest0-21swapper/322:06:523
3842649991412,1cyclictest0-21swapper/321:55:093
3842649991412,1cyclictest0-21swapper/300:25:583
3842649991412,0cyclictest0-21swapper/321:47:513
3842649991412,0cyclictest0-21swapper/320:30:073
3842649991411,2cyclictest0-21swapper/321:44:523
3842649991411,1cyclictest0-21swapper/322:16:463
3842649991411,1cyclictest0-21swapper/319:53:203
3842649991411,1cyclictest0-21swapper/319:49:553
3842649991411,1cyclictest0-21swapper/300:25:013
384264999141,10cyclictest4328-21taskset23:41:253
384264999141,10cyclictest4121185-21taskset22:44:553
384264999141,10cyclictest4055820-21taskset21:50:183
384264999141,10cyclictest3992961-21taskset21:05:113
384264999141,10cyclictest3979423-21taskset20:53:053
384264999141,10cyclictest26970-21taskset00:00:203
384264999141,10cyclictest11029-21turbostat23:50:003
39284982132,8sleep03928531-21apt-get20:15:120
39140352130,1sleep03914471-21/usr/sbin/munin20:00:220
384264999134,7cyclictest3927413-21taskset20:10:213
384264999134,7cyclictest3883236-21taskset19:39:113
384264999131,9cyclictest69178-21taskset00:33:063
384264999131,9cyclictest4160299-21taskset23:15:133
384264999131,9cyclictest4153259-21taskset23:05:433
3842649991312,0cyclictest0-21swapper/323:11:193
3842649991311,1cyclictest0-21swapper/323:30:453
3842649991311,1cyclictest0-21swapper/322:13:573
3842649991311,1cyclictest0-21swapper/321:10:283
384264999131,10cyclictest4031100-21chrt21:33:193
384264999131,10cyclictest3848651-21taskset19:10:273
3842649991310,1cyclictest0-21swapper/320:28:383
384264999130,10cyclictest4101907-21turbostat22:30:003
384264999130,10cyclictest4012191-21taskset21:20:003
384264999130,10cyclictest3979814-21taskset20:55:013
301798130,8rtkit-daemon3957310-21tail20:35:180
301798130,5rtkit-daemon3205-21sendmail-mta22:36:361
39924462121,8sleep00-21swapper/021:05:010
384264999121,9cyclictest75443-21taskset00:36:113
384264999121,8cyclictest3870734-21taskset19:30:023
301798120,5rtkit-daemon171rcu_preempt19:55:180
238612121,8sleep223888-21cut00:00:002
440512111,7sleep244390-21sed00:15:152
41666412110,1sleep20-21swapper/223:20:012
39733942112,5sleep03973427-21apt-get20:50:140
301798110,3rtkit-daemon3849163-21munin-node23:08:431
40957102102,4sleep14095742-21apt-get22:25:111
40571742101,4sleep24057207-21apt-get21:55:112
40507982101,5sleep14050954-21diskstats21:50:131
38422562100,2sleep30-21swapper/319:07:043
301798100,5rtkit-daemon62546-21users00:25:242
301798100,5rtkit-daemon4127626-21munin-run22:50:001
301798100,5rtkit-daemon4018367-21cron21:25:000
301798100,4rtkit-daemon636-21snmpd00:13:142
301798100,3rtkit-daemon636-21snmpd23:51:541
301798100,3rtkit-daemon492-21gmain21:27:500
301798100,3rtkit-daemon3849163-21munin-node20:06:230
4025047290,2sleep00-21swapper/021:30:130
4006191292,5sleep24006768-21irqrtprio21:15:162
30179890,5rtkit-daemon636-21snmpd23:15:131
30179890,5rtkit-daemon636-21snmpd23:10:301
30179890,5rtkit-daemon3205-21sendmail-mta20:44:170
30179890,5rtkit-daemon3205-21sendmail-mta00:00:121
30179890,5rtkit-daemon317-21systemd-udevd23:21:331
30179890,5rtkit-daemon317-21systemd-udevd22:55:321
30179890,5rtkit-daemon317-21systemd-udevd21:55:330
30179890,4rtkit-daemon4081936-21python322:10:210
30179890,2rtkit-daemon3822969-21kworker/0:2+kacpid19:50:170
30179890,1rtkit-daemon0-21swapper/123:33:211
29628290,1sleep00-21swapper/000:00:270
17740291,6sleep017896-21diskstats23:55:140
56775280,4sleep156808-21apt-get00:25:121
50096281,3sleep150128-21apt-get00:20:111
4191936286,1sleep04121233-21kworker/0:1+kacpid23:38:230
4179489281,3sleep04179521-21apt-get23:30:130
4146320281,4sleep24146445-21python323:00:222
4078528280,5sleep14080595-21tr22:10:171
4075376280,5sleep24075482-21/usr/sbin/munin22:05:212
4045327280,5sleep04046334-21kthreadcore21:45:150
4037633281,4sleep24037665-21munin-run21:40:002
4007802280,5sleep04009820-21grep21:15:180
3993437281,4sleep03994311-21sed21:05:170
3979827281,4sleep10-21swapper/120:55:121
3896418281,4sleep13896574-21diskstats19:50:121
3858803280,2sleep23859724-21kthreadcore19:20:172
3841666281,4sleep13841792-21sh19:05:211
30179880,5rtkit-daemon3205-21sendmail-mta22:16:510
30179880,5rtkit-daemon3205-21sendmail-mta22:08:060
30179880,4rtkit-daemon667-21lldpd19:49:530
30179880,4rtkit-daemon667-21lldpd19:41:110
30179880,4rtkit-daemon636-21snmpd23:39:581
30179880,4rtkit-daemon636-21snmpd22:43:381
30179880,4rtkit-daemon636-21snmpd22:23:270
30179880,4rtkit-daemon636-21snmpd20:33:350
30179880,4rtkit-daemon636-21snmpd19:35:270
30179880,4rtkit-daemon636-21snmpd19:20:150
30179880,4rtkit-daemon636-21snmpd19:19:051
30179880,4rtkit-daemon636-21snmpd19:10:241
30179880,4rtkit-daemon636-21snmpd00:35:532
30179880,4rtkit-daemon448-21wpa_supplicant20:56:380
30179880,4rtkit-daemon448-21wpa_supplicant20:27:270
30179880,4rtkit-daemon4004128-21kthreadcore21:10:190
30179880,4rtkit-daemon3864359-21apt-get19:25:090
30179880,4rtkit-daemon3849163-21munin-node23:42:041
30179880,4rtkit-daemon3849163-21munin-node21:44:380
30179880,4rtkit-daemon3849163-21munin-node21:35:020
30179880,4rtkit-daemon3822969-21kworker/0:2+kacpid19:32:030
30179880,4rtkit-daemon317-21systemd-udevd23:01:331
30179880,1rtkit-daemon0-21swapper/123:26:491
30179880,1rtkit-daemon0-21swapper/020:14:460
63374270,4sleep063570-21hddtemp_smartct00:30:150
63170271,3sleep163203-21apt-get00:30:131
56976270,5sleep057175-21hddtemp_smartct00:25:150
4178334270,4sleep24178393-21ntp_states23:25:232
4166701271,3sleep04166734-21apt-get23:20:120
4165623270,1sleep00-21swapper/023:15:230
4121445271,2sleep20-21swapper/222:45:122
4061044270,4sleep14062300-21memory21:55:181
3993157270,4sleep23993287-21grep21:05:152
3986764270,5sleep13987752-21kernelversion21:00:171
3961038271,3sleep23961955-21cut20:40:142
3954173271,2sleep13954205-21apt-get20:35:131
3936187270,5sleep03937320-21kthreadcore20:20:170
3921853270,4sleep13921903-21cat20:10:001
3921042271,3sleep23921167-21python320:05:242
3908153270,5sleep23908199-21ps19:55:232
3902821271,3sleep13902853-21apt-get19:55:141
3891420270,5sleep23892566-21cat19:45:172
3889966270,3sleep13889999-21apt-get19:45:121
30792270,4sleep030981-21basename00:05:150
30179870,4rtkit-daemon638-21ntpd21:51:070
30179870,4rtkit-daemon636-21snmpd23:46:211
30179870,4rtkit-daemon636-21snmpd00:05:101
30179870,4rtkit-daemon3849163-21munin-node00:30:322
30179870,4rtkit-daemon3200908-21packagekitd23:55:091
30179870,3rtkit-daemon638-21ntpd22:25:170
30179870,3rtkit-daemon636-21snmpd22:30:321
30179870,3rtkit-daemon636-21snmpd00:20:022
30179870,3rtkit-daemon4133663-21vmstat22:50:261
30179870,3rtkit-daemon3849163-21munin-node22:02:020
30179870,3rtkit-daemon297-21systemd-journal00:10:191
4894261,2sleep00-21swapper/023:45:120
4185901260,3sleep20-21swapper/223:35:122
4140410260,3sleep04140496-21systemctl23:00:000
4126790260,3sleep04126914-21sh22:45:220
4097167261,2sleep24098868-21ps22:25:162
4095454261,2sleep20-21swapper/222:25:002
4083277260,3sleep24083814-21cut22:15:142
4068039260,3sleep24068794-21sed22:00:202
4038308260,3sleep24038625-21cut21:40:172
4038115260,3sleep14038307-21hddtemp_smartct21:40:161
4024955260,1sleep10-21swapper/121:30:021
4021185260,3sleep14023308-21kthreadcore21:25:171
4018056260,1sleep10-21swapper/121:21:041
4000872260,4sleep14002511-21kthreadcore21:10:181
3968884261,2sleep10-21swapper/120:45:181
3959952260,1sleep20-21swapper/220:35:262
3953917260,4sleep10-21swapper/120:35:001
3948182260,3sleep23948788-21sed20:30:172
3928166260,1sleep20-21swapper/220:14:022
3915438260,4sleep23915649-21ls20:05:002
3909282261,2sleep10-21swapper/120:00:151
3889711260,1sleep20-21swapper/219:45:012
3877330260,3sleep13877457-21grep19:35:141
3857891261,2sleep10-21swapper/119:20:121
3851429261,2sleep20-21swapper/219:15:122
3849075260,3sleep03850579-21grep19:15:020
3843194260,3sleep23843781-21irqrtprio19:10:152
3838131261,2sleep00-21swapper/019:05:160
3836370260,2sleep20-21swapper/219:05:102
36637260,1sleep20-21swapper/200:08:542
75607250,1sleep10-21swapper/100:38:181
4580250,1sleep20-21swapper/223:44:422
4173266250,2sleep00-21swapper/023:25:170
4172479250,1sleep20-21swapper/223:20:422
4155682250,3sleep04157712-21kthreadcore23:10:170
4147734250,3sleep24147386-21/usr/sbin/munin23:05:152
4140951250,2sleep00-21swapper/023:00:120
4133743250,1sleep00-21swapper/022:51:270
4133437250,3sleep24133654-21users22:50:252
4116219250,3sleep04117427-21kthreadcore22:40:170
4115245251,2sleep20-21swapper/222:40:152
4114060250,2sleep04114286-21sed22:35:240
4102129250,1sleep00-21swapper/022:30:020
4095223250,1sleep10-21swapper/122:22:031
4088664250,1sleep10-21swapper/122:15:301
4070713250,3sleep14071649-21grep22:05:161
4065971250,2sleep14068038-21kthreadcore22:00:191
4053676251,2sleep24055819-21grep21:50:182
4048384250,3sleep24049469-21memory21:45:172
4031144250,1sleep20-21swapper/221:33:562
4017538251,2sleep20-21swapper/221:20:212
4011771250,1sleep10-21swapper/121:18:001
3994312250,2sleep13995460-21grep21:05:181
3992203250,1sleep20-21swapper/221:01:542
3979551250,1sleep20-21swapper/220:54:452
3974853250,3sleep13976408-21kthreadcore20:50:191
3947428252,2sleep2431ktimers/220:28:562
3947370250,1sleep10-21swapper/120:28:111
3937323251,2sleep10-21swapper/120:20:181
3929954250,2sleep10-21swapper/120:15:171
3902370250,1sleep20-21swapper/219:52:302
3889440250,1sleep10-21swapper/119:41:311
3876811250,1sleep10-21swapper/119:34:061
3870743250,2sleep20-21swapper/219:30:102
3869112250,2sleep23869514-21/usr/sbin/munin19:25:182
3866983250,3sleep13869111-21cut19:25:171
3844959251,2sleep00-21swapper/019:10:170
25611250,3sleep2421rcuc/200:00:212
17289250,1sleep20-21swapper/223:52:592
55604240,2sleep00-21swapper/000:20:220
4182632240,2sleep24184546-21latency23:30:192
4148854240,2sleep00-21swapper/023:05:170
4134762240,2sleep24134274-21/usr/sbin/munin22:55:162
4110269240,2sleep20-21swapper/222:35:182
4081928240,2sleep24082043-21timerandwakeup22:10:222
4044444241,1sleep10-21swapper/121:45:121
4031481240,1sleep10-21swapper/121:35:111
3966496240,3sleep10-21swapper/120:41:501
3922108240,1sleep10-21swapper/120:10:121
10747240,3sleep20-21swapper/223:46:382
69577230,1sleep00-21swapper/000:35:080
43850230,1sleep10-21swapper/100:15:131
4101929230,1sleep20-21swapper/222:30:012
4018895231,1sleep20-21swapper/221:25:142
4004781230,1sleep20-21swapper/221:10:242
3981283230,1sleep20-21swapper/220:55:162
3967198230,1sleep20-21swapper/220:45:152
3940319230,1sleep20-21swapper/220:20:232
3933829230,1sleep20-21swapper/220:15:222
3877136230,1sleep20-21swapper/219:35:112
37241230,1sleep00-21swapper/000:10:120
2171230,1sleep00-21swapper/023:40:190
11297230,1sleep00-21swapper/023:50:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional