You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-28 - 13:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8.osadl.org (updated Wed Feb 28, 2024 00:44:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3445284994846,0cyclictest0-21swapper/321:10:293
3445284994335,6cyclictest3858209-21turbostat00:20:003
3445284993532,1cyclictest0-21swapper/323:00:013
3445284993432,0cyclictest0-21swapper/323:28:323
37957982330,1chrt0-21swapper/023:35:260
3445284993330,1cyclictest0-21swapper/323:41:323
3445284993330,1cyclictest0-21swapper/322:25:273
3445284993229,1cyclictest0-21swapper/300:13:273
3445284993229,1cyclictest0-21swapper/300:13:273
3445284993128,1cyclictest0-21swapper/322:35:293
3445284993128,1cyclictest0-21swapper/322:25:003
3445284992927,0cyclictest0-21swapper/320:30:003
3445284992926,1cyclictest0-21swapper/323:12:263
3445284992825,1cyclictest0-21swapper/300:36:343
3445284992725,0cyclictest0-21swapper/321:20:223
3445284992725,0cyclictest0-21swapper/320:35:273
3445284992623,1cyclictest0-21swapper/321:36:283
3445284992622,1cyclictest0-21swapper/320:35:003
3445284992523,0cyclictest0-21swapper/323:18:373
3445284992523,0cyclictest0-21swapper/319:20:333
93698240,20rtkit-daemon3737382-21kworker/u8:0+events_unbound00:20:431
3445284992422,0cyclictest0-21swapper/321:55:003
3445284992421,1cyclictest0-21swapper/323:05:263
3445284992421,1cyclictest0-21swapper/322:50:343
3445284992321,0cyclictest0-21swapper/323:35:383
3445284992321,0cyclictest0-21swapper/320:45:013
3445284992320,1cyclictest0-21swapper/322:32:273
3445284992320,1cyclictest0-21swapper/300:00:313
3445284992220,0cyclictest0-21swapper/322:15:003
3445284992220,0cyclictest0-21swapper/319:50:273
3445284992219,1cyclictest0-21swapper/322:00:273
3445284992219,1cyclictest0-21swapper/321:30:363
3445284992219,1cyclictest0-21swapper/319:40:363
3445284992119,0cyclictest0-21swapper/300:25:363
38858732201,8sleep23885879-21ssh00:38:372
3445284992018,0cyclictest0-21swapper/321:15:453
3445284992017,1cyclictest0-21swapper/323:45:253
3445284992017,1cyclictest0-21swapper/323:20:373
3445284992017,1cyclictest0-21swapper/321:25:363
3445284992017,1cyclictest0-21swapper/320:50:283
3445284992017,1cyclictest0-21swapper/300:30:053
3445284991917,0cyclictest0-21swapper/319:18:583
3445284991916,1cyclictest0-21swapper/322:40:283
3445284991916,1cyclictest0-21swapper/300:08:523
3445284991915,1cyclictest35862952sleep321:05:253
3445284991815,1cyclictest0-21swapper/320:45:253
3445284991814,1cyclictest0-21swapper/300:20:313
3445284991715,0cyclictest0-21swapper/322:47:093
3445284991715,0cyclictest0-21swapper/321:55:243
3445284991715,0cyclictest0-21swapper/320:15:273
3445284991714,1cyclictest0-21swapper/322:06:563
3445284991714,1cyclictest0-21swapper/321:40:393
36080182161,12sleep03609887-21tr21:20:250
3445284991614,0cyclictest0-21swapper/323:55:253
3445284991614,0cyclictest0-21swapper/323:00:113
3445284991614,0cyclictest0-21swapper/319:30:293
3445284991613,1cyclictest0-21swapper/323:30:293
3445284991613,1cyclictest0-21swapper/319:55:343
3445284991513,0cyclictest0-21swapper/320:55:263
3445284991512,1cyclictest0-21swapper/319:35:343
93698140,3rtkit-daemon1-21systemd22:21:310
93698140,3rtkit-daemon1-21systemd21:19:472
93698140,2rtkit-daemon3886237-21ssh00:39:521
3445284991411,1cyclictest0-21swapper/321:45:583
3445284991411,1cyclictest0-21swapper/320:00:313
93698130,3rtkit-daemon3759588-21ssh23:07:031
93698130,3rtkit-daemon1-21systemd23:42:271
93698130,3rtkit-daemon1-21systemd23:32:420
93698130,3rtkit-daemon1-21systemd23:29:420
93698130,3rtkit-daemon1-21systemd23:21:330
93698130,3rtkit-daemon1-21systemd22:26:110
93698130,2rtkit-daemon1-21systemd22:14:501
35136842134,5sleep13513706-21apt-get20:05:221
34581702135,5sleep03458301-21netstat19:15:320
3445284991311,0cyclictest0-21swapper/320:20:303
3445284991310,0cyclictest0-21swapper/322:15:243
93698120,3rtkit-daemon355-21systemd-journal23:53:440
93698120,3rtkit-daemon1-21systemd22:58:301
93698120,3rtkit-daemon1-21systemd21:34:022
93698120,3rtkit-daemon1-21systemd21:21:472
93698120,3rtkit-daemon1-21systemd19:25:240
93698120,2rtkit-daemon1-21systemd22:19:130
93698120,2rtkit-daemon1-21systemd21:15:321
93698120,1rtkit-daemon0-21swapper/019:10:360
36415072123,5sleep13641540-21apt-get21:45:191
34764632120,0sleep23476564-21mailstats19:30:362
3445284991210,0cyclictest0-21swapper/319:10:303
34394532122,5sleep10-21swapper/119:05:221
93698110,6rtkit-daemon519-21gdbus21:30:002
93698110,6rtkit-daemon480-21wpa_supplicant20:35:021
93698110,6rtkit-daemon3774324-21ssh23:19:501
93698110,3rtkit-daemon374-21systemd-udevd23:11:361
93698110,3rtkit-daemon1-21systemd23:44:122
93698110,3rtkit-daemon1-21systemd22:36:332
93698110,3rtkit-daemon1-21systemd22:29:522
93698110,3rtkit-daemon1-21systemd21:11:591
93698110,2rtkit-daemon1-21systemd22:16:171
93698110,2rtkit-daemon1-21systemd21:57:192
93698110,2rtkit-daemon1-21systemd20:20:250
93698110,1rtkit-daemon0-21swapper/222:47:352
93698110,1rtkit-daemon0-21swapper/222:40:352
37899932110,0sleep2421rcuc/223:30:242
37113422113,4sleep03711375-21apt-get22:35:240
35256552112,4sleep13525688-21apt-get20:15:241
344528499119,0cyclictest0-21swapper/320:05:243
344528499118,1cyclictest0-21swapper/319:27:273
344528499111,7cyclictest3822718-21taskset23:53:043
93698100,2rtkit-daemon1-21systemd20:25:360
38714042101,5sleep23871406-21ssh00:27:032
35374582103,4sleep1486-21rs:main1
344528499101,6cyclictest3585489-21taskset21:00:363
9369890,3rtkit-daemon3494637-21python319:45:340
3843192291,5sleep23843196-21ssh00:05:592
3758790291,5sleep03758907-21python323:05:370
3731502290,1sleep00-21swapper/022:47:220
3570481290,0sleep13572499-21kthreadcore20:50:291
3569416290,0sleep03569451-21kthreadcore20:50:280
3507345291,5sleep13507349-21cron20:00:001
34452849998,0cyclictest0-21swapper/320:10:333
34452849998,0cyclictest0-21swapper/320:10:323
34452849997,0cyclictest0-21swapper/319:45:253
9369880,4rtkit-daemon690-21ntpd22:05:252
9369880,4rtkit-daemon537-21gmain22:32:512
9369880,4rtkit-daemon537-21gmain21:50:312
9369880,4rtkit-daemon537-21gmain21:37:072
9369880,4rtkit-daemon3591861-21cron21:10:011
9369880,4rtkit-daemon3561539-21chrt20:42:021
9369880,3rtkit-daemon684-21snmpd23:48:370
9369880,3rtkit-daemon3830826-21fschecks_count00:00:230
9369880,3rtkit-daemon3787071-21grep23:25:392
9369880,3rtkit-daemon3506446-21/usr/sbin/munin19:55:350
3802623281,2sleep00-21swapper/023:40:240
3782187281,4sleep13783074-21irqstats23:25:301
3780252280,6sleep13780256-21ssh23:20:491
3753398280,0sleep20-21swapper/223:04:552
3710849280,0sleep00-21swapper/022:34:230
3663908284,2sleep03664896-21cut22:00:330
3662737280,4sleep13662893-21ssh22:00:311
3627577281,3sleep13627608-21apt-get21:35:251
3580075281,3sleep03580107-21apt-get21:00:220
3549954281,4sleep23549990-21fschecks_count20:35:272
3545089280,0sleep2421rcuc/220:30:342
3521055281,5sleep03522849-21kthreadcore20:10:290
3521055281,5sleep03522849-21kthreadcore20:10:290
3512728281,5sleep03512896-21cat20:00:390
9369870,3rtkit-daemon944-21sendmail-mta19:30:110
9369870,3rtkit-daemon944-21sendmail-mta19:20:450
9369870,3rtkit-daemon690-21ntpd21:40:392
9369870,3rtkit-daemon690-21ntpd20:55:341
9369870,3rtkit-daemon690-21ntpd00:16:292
9369870,3rtkit-daemon684-21snmpd23:57:130
9369870,3rtkit-daemon684-21snmpd23:45:382
9369870,3rtkit-daemon684-21snmpd20:35:251
9369870,3rtkit-daemon684-21snmpd00:25:051
9369870,3rtkit-daemon537-21gmain22:01:112
9369870,3rtkit-daemon537-21gmain21:01:001
9369870,3rtkit-daemon537-21gmain20:17:350
9369870,3rtkit-daemon537-21gmain00:20:192
9369870,3rtkit-daemon40542-21packagekitd20:06:220
9369870,3rtkit-daemon3451438-21munin-node21:45:352
9369870,3rtkit-daemon3451438-21munin-node19:41:120
9369870,3rtkit-daemon3451438-21munin-node00:31:131
9369870,3rtkit-daemon1-21systemd23:01:151
9369870,2rtkit-daemon684-21snmpd19:35:230
9369870,2rtkit-daemon3737887-21unixbench_singl22:50:392
9369870,2rtkit-daemon3439842-21hddtemp_smartct19:05:260
9369870,1rtkit-daemon0-21swapper/019:51:380
3879271271,4sleep23879275-21modprobe00:34:562
3856711271,4sleep13591864-21diskmemload00:15:361
3844816271,3sleep23844941-21gltestperf00:10:242
3844816271,3sleep23844941-21gltestperf00:10:242
3831985271,4sleep23832904-21grep00:00:262
3823233270,4sleep13823258-21cut23:55:001
3795842271,3sleep13796038-21grep23:35:281
3795635271,2sleep23795670-21apt-get23:35:262
3773054271,3sleep03773213-21users23:15:390
3767105270,4sleep03767109-21ssh23:14:030
3758693271,3sleep23758750-21ntp_states23:05:352
3739640271,3sleep23739838-21hddtemp_smartct22:55:252
3733881271,3sleep13734744-21grep22:50:311
3723949271,4sleep13724082-21vmstat22:40:381
3693475271,3sleep13695153-21sort22:20:301
3688374271,3sleep23688447-21ntp_states22:15:342
3648976270,1sleep1191rcu_preempt21:50:321
3641502271,3sleep03641506-21/usr/sbin/munin21:45:180
3641157271,3sleep13641162-21ssh21:44:551
3633824271,1sleep00-21swapper/021:38:410
3620183270,0sleep00-21swapper/021:29:390
3613158270,0sleep10-21swapper/121:24:311
3568058271,2sleep20-21swapper/220:50:222
3532163271,3sleep23533028-21sed20:20:292
3494512271,3sleep10-21swapper/119:45:341
3488611271,3sleep13483586-21/usr/sbin/munin19:40:371
3484944271,3sleep23485950-21grep19:40:302
3465743271,3sleep13465938-21hddtemp_smartct19:25:261
3453501271,3sleep23453653-21df_inode19:15:212
9369860,2rtkit-daemon690-21ntpd20:45:171
3858735261,2sleep03858892-21/usr/sbin/munin00:20:240
3774739261,1sleep20-21swapper/223:20:222
3762536261,2sleep20-21swapper/223:10:282
3732597261,3sleep03732709-21diskstats22:50:260
3725563260,4sleep13725601-21dpkg22:45:181
3669973261,3sleep03669998-21ps22:05:300
3640262260,1sleep00-21swapper/021:41:370
3626446260,3sleep03626448-21ssh21:32:030
3620271261,2sleep10-21swapper/121:30:001
3592750260,3sleep03543514-21kworker/u8:1+flush-8:021:10:310
3519664261,2sleep20-21swapper/220:10:232
3519664261,2sleep20-21swapper/220:10:232
3507832261,3sleep13507902-21fschecks_time20:00:281
3506866260,3sleep23506990-21users19:55:402
3460436261,3sleep13461018-21grep19:20:251
3445910261,2sleep20-21swapper/219:10:282
3445447261,2sleep10-21swapper/119:10:231
3439621261,2sleep30-21swapper/319:05:253
9369850,1rtkit-daemon0-21swapper/122:28:141
9369850,1rtkit-daemon0-21swapper/020:30:310
3879932251,2sleep00-21swapper/000:35:230
3871233250,0sleep00-21swapper/000:26:210
3850824251,2sleep00-21swapper/000:13:220
3850824251,2sleep00-21swapper/000:13:220
3845977251,2sleep13846999-21kthreadcore00:10:271
3845977251,2sleep13846999-21kthreadcore00:10:271
3839662251,2sleep13841500-21kthreadcore00:05:301
3837836251,2sleep03837982-21gltestperf00:05:250
3825004251,2sleep20-21swapper/223:55:292
3818028251,2sleep20-21swapper/223:50:292
3772758251,1sleep20-21swapper/223:15:372
3704366251,1sleep10-21swapper/122:30:241
3696712251,2sleep20-21swapper/222:23:472
3682319252,1sleep23543514-21kworker/u8:1+rpciod22:12:012
3677869251,2sleep00-21swapper/022:10:310
3675271251,1sleep10-21swapper/122:06:431
3661135250,0sleep00-21swapper/021:56:040
3655542251,1sleep10-21swapper/121:55:251
3621992251,2sleep10-21swapper/121:30:321
3581434251,2sleep20-21swapper/221:00:282
3575414251,2sleep23576880-21kthreadcore20:55:312
3574510251,2sleep03574612-21irqcore20:55:290
3562084251,2sleep03562190-21expr20:45:230
3556033251,1sleep00-21swapper/020:40:220
3539133251,2sleep13540582-21kthreadcore20:25:291
3526060250,3sleep23526565-21irqcore20:15:282
3509274251,2sleep20-21swapper/220:00:332
3496533251,2sleep10-21swapper/119:50:311
3495589251,1sleep20-21swapper/219:50:232
3490539250,3sleep23489595-21/usr/sbin/munin19:45:262
3477583251,1sleep20-21swapper/219:35:252
3477552251,2sleep10-21swapper/119:35:241
3465544251,2sleep20-21swapper/219:25:232
3461019251,2sleep20-21swapper/219:20:272
3873758241,1sleep00-21swapper/000:30:260
3852012240,2sleep03852098-21ps00:15:270
3830595240,2sleep13830601-21ssh00:00:171
3811014241,1sleep10-21swapper/123:45:301
3795283240,2sleep13795371-21ssh23:35:001
3748129241,1sleep00-21swapper/023:00:320
3739386240,0sleep00-21swapper/022:55:110
3724630240,0sleep00-21swapper/022:42:450
3711828240,1sleep10-21swapper/122:35:291
3654841240,0sleep00-21swapper/021:53:520
3601203241,1sleep00-21swapper/021:15:360
3598763240,2sleep23598765-21ssh21:14:062
3591699240,0sleep20-21swapper/221:08:052
3587472240,2sleep03588216-21kthreadcore21:05:310
3556131240,1sleep20-21swapper/220:40:242
3555427240,0sleep00-21swapper/020:39:220
3525033241,1sleep10-21swapper/120:10:381
3525033241,1sleep10-21swapper/120:10:381
3472875241,1sleep10-21swapper/119:30:311
3444446240,1sleep20-21swapper/219:05:372
3823575230,1sleep10-21swapper/123:55:201
3563440230,1sleep20-21swapper/220:45:292
3537957230,1sleep20-21swapper/220:25:252
3514161230,1sleep20-21swapper/220:05:262
3454825230,1sleep10-21swapper/119:15:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional