You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 11:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Tue Dec 30, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31392039912202,1212cyclictest3334366-21sleep21:39:593
3139203993280,324cyclictest3502377-21turbostat3
3139203992219,1cyclictest0-21swapper/33
3139203992118,1cyclictest0-21swapper/322:35:173
3139203992017,1cyclictest0-21swapper/323:39:593
3139203991714,1cyclictest0-21swapper/33
3139203991714,1cyclictest0-21swapper/323:30:203
3139203991714,1cyclictest0-21swapper/320:15:163
3139203991714,1cyclictest0-21swapper/319:20:203
34164222160,1sleep03417605-21memory22:40:200
3139203991613,1cyclictest0-21swapper/33
3139203991613,1cyclictest0-21swapper/33
3139203991613,1cyclictest0-21swapper/33
313920399161,11cyclictest3495833-21taskset23:45:013
301798160,5rtkit-daemon3451160-21diskstats2
313920399156,6cyclictest32894122sleep321:05:153
313920399155,7cyclictest35273892sleep300:05:213
3139203991513,1cyclictest0-21swapper/321:26:523
3139203991513,1cyclictest0-21swapper/320:40:063
3139203991512,1cyclictest0-21swapper/33
3139203991512,1cyclictest0-21swapper/322:56:243
3139203991512,1cyclictest0-21swapper/320:36:373
3139203991512,1cyclictest0-21swapper/319:45:153
313920399151,11cyclictest3341043-21taskset21:44:563
313920399151,11cyclictest3139336-21chrt19:10:013
35085802140,8sleep23508581-21sshd23:52:372
313920399145,7cyclictest3444288-21turbostat23:09:553
313920399144,7cyclictest3534549-21taskset00:13:013
313920399144,7cyclictest34576582sleep323:15:163
313920399141,9cyclictest34638452sleep33
313920399141,9cyclictest3167259-21taskset3
3139203991413,0cyclictest0-21swapper/33
3139203991412,1cyclictest0-21swapper/33
3139203991412,1cyclictest0-21swapper/33
3139203991412,1cyclictest0-21swapper/33
3139203991412,1cyclictest0-21swapper/320:22:513
3139203991412,1cyclictest0-21swapper/320:10:333
3139203991412,1cyclictest0-21swapper/320:01:433
3139203991412,0cyclictest0-21swapper/321:17:183
3139203991411,1cyclictest0-21swapper/323:27:233
3139203991411,1cyclictest0-21swapper/321:45:113
3139203991411,1cyclictest0-21swapper/320:55:323
3139203991411,1cyclictest0-21swapper/320:51:423
313920399141,10cyclictest3425682-21taskset3
313920399141,10cyclictest3423657-21taskset22:45:183
313920399141,10cyclictest3380754-21taskset22:15:123
313920399141,10cyclictest3354664-21taskset21:55:023
313920399141,10cyclictest3180749-21taskset19:40:153
313920399141,10cyclictest3173676-21taskset3
313920399141,10cyclictest3150653-21taskset19:15:193
313920399134,6cyclictest3327763-21taskset3
313920399131,9cyclictest3351158-21taskset21:50:173
313920399131,9cyclictest3296438-21taskset21:10:143
313920399131,9cyclictest3244527-21taskset20:30:083
313920399131,9cyclictest3160784-21taskset3
3139203991312,0cyclictest0-21swapper/322:25:163
3139203991312,0cyclictest0-21swapper/319:54:313
3139203991311,1cyclictest0-21swapper/322:42:273
3139203991311,1cyclictest0-21swapper/322:03:483
3139203991311,1cyclictest0-21swapper/300:25:303
3139203991310,1cyclictest0-21swapper/33
3139203991310,1cyclictest0-21swapper/322:09:343
313920399130,6cyclictest0-21swapper/320:05:163
313920399130,10cyclictest3534945-21chrt00:15:013
313920399130,10cyclictest3373936-21taskset22:10:043
301798130,3rtkit-daemon3145717-21munin-node22:26:380
313920399129,1cyclictest0-21swapper/33
313920399124,6cyclictest3502009-21taskset23:45:373
313920399121,9cyclictest3521533-21chrt3
313920399120,9cyclictest3560660-21taskset3
313920399120,9cyclictest3554284-21chrt00:30:123
35611202111,7sleep13561893-21sed00:35:201
301798110,6rtkit-daemon3333646-21awk21:35:250
35604192101,6sleep23560432-21latency_hist2
34830912102,4sleep03483124-21apt-get23:35:130
34574052102,4sleep03457438-21apt-get0
34378582101,5sleep13437861-21cron1
33092722100,7sleep03309439-21/usr/sbin/munin21:20:140
32575312100,1sleep00-21swapper/00
32201012101,7sleep10-21swapper/120:10:191
32140252100,7sleep13216057-21cut20:05:181
31626342101,6sleep13164687-21grep1
31543942100,7sleep00-21swapper/019:20:140
31543942100,7sleep00-21swapper/019:20:140
31333592101,6sleep23133643-21sed19:05:172
31330022102,4sleep03133158-21diskstats0
301798100,6rtkit-daemon636-21snmpd23:03:132
301798100,6rtkit-daemon492-21gmain23:19:091
301798100,5rtkit-daemon3541187-21munin-run00:20:011
301798100,5rtkit-daemon3167256-21apt-get19:30:132
301798100,3rtkit-daemon3205-21sendmail-mta23:44:151
301798100,3rtkit-daemon3145717-21munin-node23:26:461
301798100,3rtkit-daemon3145717-21munin-node00:33:061
3534997291,5sleep03535153-21diskstats00:15:160
3426791290,1sleep13428810-21kthreadcore22:50:181
3417741291,4sleep23417799-21ntp_states22:40:222
3367509291,4sleep13367542-21apt-get1
3333713291,5sleep23333957-21users2
3250669291,2sleep20-21swapper/22
3176471291,2sleep00-21swapper/019:35:200
30179890,5rtkit-daemon448-21wpa_supplicant22:07:000
30179890,5rtkit-daemon448-21wpa_supplicant2
30179890,5rtkit-daemon3508322-21grep23:50:231
30179890,5rtkit-daemon3498978-21grep1
30179890,5rtkit-daemon3205-21sendmail-mta21:42:150
30179890,5rtkit-daemon3205-21sendmail-mta21:11:392
30179890,5rtkit-daemon3205-21sendmail-mta2
30179890,4rtkit-daemon448-21wpa_supplicant2
30179890,4rtkit-daemon3510742-21cut23:55:191
30179890,4rtkit-daemon3323109-21sed21:30:162
30179890,4rtkit-daemon3222831-21cut20:10:202
30179890,3rtkit-daemon3145717-21munin-node2
3541437281,4sleep23541593-21diskstats00:20:142
3515456281,3sleep03515489-21apt-get00:00:130
3444527281,3sleep03444560-21apt-get0
3425481280,5sleep23425681-21hddtemp_smartct22:50:152
3424536280,6sleep03424659-21users22:45:230
3354478281,4sleep23354176-21apt-get2
3348026281,3sleep13348059-21apt-get1
3322170281,4sleep03322299-21cut21:30:160
3281926281,4sleep13282050-21python31
3270678281,4sleep03271669-21irqstats20:50:170
3263801282,2sleep23263834-21apt-get20:45:142
3249972281,4sleep03250089-21ls20:30:210
3244972280,5sleep13245763-21irqrtprio1
3192903281,3sleep13192936-21apt-get1
3180135280,4sleep03180291-21diskstats0
3167417280,5sleep03167535-21gltestperf19:30:150
3145560281,4sleep03145744-21sort19:15:010
3140054280,5sleep23140948-21tr2
30179880,4rtkit-daemon667-21lldpd23:33:171
30179880,4rtkit-daemon636-21snmpd23:35:151
30179880,4rtkit-daemon636-21snmpd22:29:592
30179880,4rtkit-daemon636-21snmpd22:14:050
30179880,4rtkit-daemon636-21snmpd20:35:222
30179880,4rtkit-daemon636-21snmpd20:29:522
30179880,4rtkit-daemon636-21snmpd19:45:182
30179880,4rtkit-daemon636-21snmpd1
30179880,4rtkit-daemon636-21snmpd00:22:211
30179880,4rtkit-daemon636-21snmpd00:05:071
30179880,4rtkit-daemon636-21snmpd0
30179880,4rtkit-daemon448-21wpa_supplicant21:08:082
30179880,4rtkit-daemon430-21avahi-daemon21:25:182
30179880,4rtkit-daemon413-21rpcbind19:15:162
30179880,4rtkit-daemon3145717-21munin-node23:24:031
30179880,4rtkit-daemon3145717-21munin-node22:01:400
30179880,4rtkit-daemon3145717-21munin-node2
30179880,4rtkit-daemon3145717-21munin-node00:02:021
30179880,4rtkit-daemon1-21systemd20:55:002
30179880,4rtkit-daemon0-21swapper/22
30179880,3rtkit-daemon3446184-21cat23:05:172
30179880,3rtkit-daemon3236634-21cat20:20:202
30179880,1rtkit-daemon0-21swapper/22
30179880,1rtkit-daemon0-21swapper/021:54:520
3548042271,4sleep23548250-21cat2
3530004270,5sleep03531480-21kthreadcore00:10:190
3522212270,3sleep03522368-21diskstats00:05:130
3489576270,4sleep03489726-21df_inode23:40:140
3412480271,3sleep13412513-21apt-get22:40:131
3394375271,3sleep13394706-21kthreadcore22:25:181
3368854270,4sleep23370004-21cut22:05:202
3308411270,1sleep00-21swapper/00
3237879270,3sleep13237912-21apt-get20:25:141
3223873271,3sleep03223931-21ntp_states0
3191949270,4sleep03192065-21cat0
3180492270,4sleep23180748-21sed2
3172737271,4sleep13173004-21unixbench_singl19:30:241
3161676271,3sleep03162633-21grep19:25:180
3154415271,3sleep13154428-21apt-get19:20:151
3154415271,3sleep13154428-21apt-get1
3132949270,3sleep30-21swapper/33
30179870,4rtkit-daemon492-21gmain2
30179870,4rtkit-daemon430-21avahi-daemon00:25:441
30179870,4rtkit-daemon3356511-21grep0
30179870,4rtkit-daemon3145717-21munin-node22:30:052
30179870,3rtkit-daemon667-21lldpd21:15:152
30179870,3rtkit-daemon638-21ntpd2
30179870,3rtkit-daemon638-21ntpd19:20:142
30179870,3rtkit-daemon638-21ntpd19:20:132
30179870,3rtkit-daemon636-21snmpd22:15:170
30179870,3rtkit-daemon492-21gmain22:20:170
30179870,3rtkit-daemon430-21avahi-daemon2
30179870,3rtkit-daemon3312100-21ps21:20:162
30179870,3rtkit-daemon3205-21sendmail-mta20:41:502
30179870,3rtkit-daemon3145717-21munin-node20:58:232
30179870,3rtkit-daemon3145717-21munin-node2
3540815260,1sleep20-21swapper/200:16:092
3522174260,4sleep23522178-21find00:05:122
3495674260,3sleep23495812-21store23:45:012
3476700260,1sleep20-21swapper/22
3470292260,3sleep03470445-21df_inode23:25:100
3457589260,4sleep20-21swapper/22
3456304260,3sleep13456379-21ps23:10:211
3418929260,3sleep13419076-21df22:45:121
3328167260,2sleep13328200-21apt-get1
3321495260,1sleep00-21swapper/00
3301350260,3sleep03301409-21ntp_states0
3295570262,2sleep00-21swapper/021:10:000
3294764260,1sleep10-21swapper/121:05:251
3283424260,3sleep13283688-21irqcore21:00:201
3276619260,3sleep03276763-21df0
3267564260,3sleep13268925-21memory20:45:201
3257588260,3sleep13257787-21hddtemp_smartct1
3231482261,2sleep10-21swapper/120:20:131
3230924260,1sleep10-21swapper/120:16:391
3205760260,3sleep03205768-21apt-get20:00:130
3205374260,1sleep00-21swapper/019:58:560
3186510261,2sleep13186543-21apt-get19:45:111
3144524260,1sleep10-21swapper/119:10:211
3549023251,2sleep00-21swapper/000:25:170
3534531250,1sleep20-21swapper/200:12:492
3520890251,2sleep23515457-21/usr/sbin/munin2
3510953251,2sleep00-21swapper/023:55:200
3509024250,2sleep20-21swapper/223:55:142
3508602250,1sleep00-21swapper/00
3488977250,1sleep20-21swapper/223:36:222
3476108250,1sleep20-21swapper/223:25:452
3457087250,1sleep00-21swapper/023:14:120
3450592250,1sleep10-21swapper/123:08:291
3438136250,2sleep13438169-21apt-get23:00:121
3437814250,1sleep00-21swapper/00
3405548250,1sleep10-21swapper/122:32:141
3392869250,1sleep10-21swapper/122:23:541
3387220250,3sleep23387576-21cut22:20:182
3382144250,2sleep13384199-21kthreadcore22:15:151
3380551250,2sleep23380753-21hddtemp_smartct2
3373945250,2sleep20-21swapper/22
3367004250,1sleep10-21swapper/122:01:501
3363767251,2sleep20-21swapper/222:00:192
3353840250,1sleep20-21swapper/221:50:522
3347698250,1sleep20-21swapper/221:49:052
3342405250,3sleep13343355-21grep21:45:151
3340858250,1sleep10-21swapper/121:42:351
3315008250,1sleep10-21swapper/121:23:481
3276085250,1sleep10-21swapper/120:51:541
3269794250,1sleep00-21swapper/020:47:520
3251895251,2sleep00-21swapper/020:35:180
3211807250,1sleep10-21swapper/11
3199295250,2sleep10-21swapper/11
3566123240,2sleep00-21swapper/000:35:270
3560366240,3sleep20-21swapper/22
3554297240,1sleep00-21swapper/000:30:130
3547547240,3sleep00-21swapper/00
3502069240,3sleep00-21swapper/00
3497510240,2sleep23498656-21cut23:45:172
3469315240,3sleep20-21swapper/223:20:242
3426103240,2sleep00-21swapper/022:50:170
3412032240,3sleep00-21swapper/022:37:470
3400733240,2sleep00-21swapper/022:30:170
3379898240,1sleep10-21swapper/122:12:241
3360102240,2sleep10-21swapper/11
3335061240,2sleep20-21swapper/22
3295623240,1sleep10-21swapper/121:10:001
3289091241,2sleep00-21swapper/021:04:110
3252812240,2sleep13254866-21kthreadcore1
3243967240,3sleep00-21swapper/00
3231464240,2sleep00-21swapper/020:20:120
3226519240,2sleep00-21swapper/020:15:190
3179802240,3sleep10-21swapper/11
3154006240,3sleep00-21swapper/019:18:360
3145745240,2sleep10-21swapper/11
3133159240,1sleep10-21swapper/119:05:161
3476693230,1sleep00-21swapper/023:30:080
3463882230,1sleep00-21swapper/023:20:140
3438130230,2sleep0151ktimers/023:00:070
3411909230,1sleep10-21swapper/11
3327011230,1sleep10-21swapper/121:30:211
3315397230,1sleep10-21swapper/11
3303960230,1sleep10-21swapper/11
3211894230,1sleep00-21swapper/00
3197710230,1sleep03198110-21strings0
3180078230,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional