You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 12:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Fri Nov 28, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3454662992213,7cyclictest3893611-21turbostat00:10:013
345466299221,17cyclictest3884486-21taskset00:02:233
345466299221,17cyclictest3654123-21taskset21:40:133
345466299211,17cyclictest3646069-21taskset21:35:083
345466299211,16cyclictest3670362-21taskset21:50:153
345466299211,16cyclictest3620057-21chrt21:16:003
345466299211,16cyclictest3605568-21taskset21:07:433
345466299211,16cyclictest3500094-21taskset19:42:503
345466299211,16cyclictest3487635-21sleep19:35:013
345466299211,16cyclictest3460465-21taskset19:10:533
345466299201,16cyclictest3860956-21chrt23:48:173
345466299201,16cyclictest3700281-21taskset22:06:343
345466299201,16cyclictest3645285-21chrt21:33:473
345466299201,15cyclictest3940032-21taskset00:36:373
345466299201,15cyclictest3908411-21taskset00:17:233
345466299201,15cyclictest3852992-21taskset23:43:523
345466299201,15cyclictest3777938-21chrt22:55:193
345466299201,15cyclictest3773097-21chrt22:53:433
345466299201,15cyclictest3756497-21taskset22:42:133
345466299201,15cyclictest3741226-21taskset22:33:433
345466299201,15cyclictest3685681-21taskset21:59:333
345466299201,15cyclictest3612866-21taskset21:12:563
345466299201,15cyclictest3592854-21taskset20:56:573
345466299201,15cyclictest3586875-21taskset20:54:363
345466299201,15cyclictest3580463-21taskset20:46:223
345466299201,15cyclictest3568004-21taskset20:35:323
345466299201,15cyclictest3524332-21taskset20:00:213
345466299201,15cyclictest3506372-21taskset19:49:203
345466299201,15cyclictest3493458-21taskset19:35:423
345466299200,16cyclictest3838315-21taskset23:35:153
345466299191,15cyclictest3836712-21taskset23:33:143
345466299191,15cyclictest3828451-21chrt23:27:253
345466299191,15cyclictest3662010-21taskset21:45:023
345466299191,15cyclictest3598992-21chrt21:01:063
345466299191,15cyclictest3568810-21taskset20:40:143
345466299191,15cyclictest3537341-21taskset20:14:173
345466299191,15cyclictest3530968-21taskset20:06:363
345466299191,15cyclictest3468903-21chrt19:18:063
345466299191,14cyclictest3918131-21taskset00:25:163
345466299191,14cyclictest3795875-21taskset23:05:483
345466299191,14cyclictest3764819-21taskset22:48:023
345466299190,16cyclictest3788344-21taskset23:01:523
345466299190,16cyclictest3788344-21taskset23:01:513
345466299190,15cyclictest3749009-21chrt22:38:283
345466299190,15cyclictest3717732-21turbostat22:19:593
345466299190,15cyclictest3550005-21taskset20:25:103
345466299190,15cyclictest3475111-21chrt19:23:283
345466299181,14cyclictest3899886-21taskset00:11:043
345466299181,14cyclictest3812113-21taskset23:16:263
345466299181,14cyclictest3803838-21taskset23:10:463
345466299181,14cyclictest3733722-21taskset22:29:573
345466299181,14cyclictest3708398-21taskset22:11:553
345466299181,14cyclictest3622047-21taskset21:20:043
345466299181,14cyclictest3543530-21taskset20:19:183
345466299181,14cyclictest3518737-21taskset19:58:583
345466299181,14cyclictest3512365-21taskset19:51:523
345466299181,13cyclictest3718102-21taskset22:20:133
345466299181,13cyclictest3561856-21taskset20:31:043
345466299181,13cyclictest3549549-21taskset20:22:313
345466299180,15cyclictest3931627-21taskset00:30:333
345466299180,15cyclictest3867771-21chrt23:50:373
345466299180,15cyclictest3630094-21chrt21:25:143
345466299180,14cyclictest3820420-21taskset23:22:083
345466299171,13cyclictest3481382-21taskset19:29:333
345466299170,14cyclictest3916410-21taskset00:22:163
345466299170,14cyclictest3875830-21taskset23:55:553
345466299170,14cyclictest3693283-21taskset22:03:453
36191822151,10sleep23619289-21ntpq21:15:202
38784642130,11sleep23878965-21irqrtprio00:00:192
38321992131,9sleep23834207-21tr23:30:182
37823442131,2sleep20-21swapper/223:00:162
37823442131,2sleep20-21swapper/223:00:152
36193502131,9sleep00-21swapper/021:15:220
345466299131,9cyclictest3454707-21chrt19:10:003
147598130,9rtkit-daemon3935203-21irqstats00:35:152
147598130,8rtkit-daemon3732670-21ssh22:27:292
147598130,2rtkit-daemon214-21plymouthd22:00:062
39334482121,2sleep10-21swapper/100:34:411
39175522120,8sleep21-21systemd00:25:002
38799622120,9sleep03881939-21tr00:00:210
37100292121,8sleep23710033-21ssh22:15:072
34829972120,9sleep23484376-21cut19:30:162
147598120,8rtkit-daemon3892091-21ssh00:06:340
147598120,8rtkit-daemon3685379-21ssh21:58:522
147598120,7rtkit-daemon1540-21sendmail-mta19:26:282
147598120,5rtkit-daemon3460969-21munin-node22:53:350
147598120,3rtkit-daemon3460969-21munin-node21:36:202
39172792110,1sleep10-21swapper/100:24:151
38859492111,7sleep23886086-21cpuspeed_turbos00:05:122
38222822110,8sleep23822487-21cut23:25:172
37012542110,1sleep20-21swapper/222:08:532
36464502110,8sleep03646719-21cut21:35:170
36449682110,2sleep20-21swapper/221:33:082
35500082110,8sleep10-21swapper/120:25:131
35373902110,8sleep03537391-21cat20:14:550
34551402110,8sleep23455826-21cut19:10:172
147598110,7rtkit-daemon474-21systemd-logind23:03:430
147598110,7rtkit-daemon474-21systemd-logind23:03:430
147598110,4rtkit-daemon3908547-21ssh00:17:390
39080132101,7sleep23908019-21ssh00:16:222
38998702100,7sleep13899878-21ssh00:11:041
38620362100,7sleep03862168-21expr23:50:150
38605512100,7sleep13860557-21ssh23:47:241
37675862100,7sleep23769502-21grep22:50:172
37643272100,8sleep20-21swapper/222:46:542
34816752101,4sleep13481708-21apt-get19:30:111
147598100,6rtkit-daemon730-21ntpd21:44:272
147598100,6rtkit-daemon474-21systemd-logind19:08:422
147598100,6rtkit-daemon3812542-21ssh23:17:300
147598100,6rtkit-daemon3652801-21kworker/u16:3+nfsiod22:37:482
147598100,6rtkit-daemon1540-21sendmail-mta19:42:172
147598100,5rtkit-daemon3670355-21apt-get21:50:152
3940818291,5sleep03940826-21ssh00:38:290
3934837290,6sleep13935971-21kthreadcore00:35:161
3932824290,1sleep20-21swapper/200:33:182
3869947291,4sleep03869981-21apt-get23:55:120
3867857295,2sleep2214-21plymouthd23:50:492
3851879290,1sleep10-21swapper/123:41:221
3741351290,7sleep23741359-21ssh22:34:072
3741270290,3sleep1214-21plymouthd22:33:501
3668693290,1sleep00-21swapper/021:47:350
3643126290,6sleep03643269-21/usr/sbin/munin21:30:210
3636321290,6sleep23636327-21ssh21:26:342
3631455291,5sleep13633052-21grep21:25:211
3630104291,4sleep03630139-21apt-get21:25:160
3598959290,1sleep00-21swapper/021:00:390
3586093290,6sleep03586218-21/usr/sbin/munin20:50:250
3580489290,1sleep20-21swapper/220:46:432
3537687291,5sleep23537694-21grep20:15:122
3530996290,1sleep00-21swapper/020:06:570
14759890,6rtkit-daemon3712550-21kworker/0:2+kacpid22:21:510
14759890,5rtkit-daemon3781401-21diskmemload22:59:280
14759890,5rtkit-daemon3603570-21grep21:05:202
14759890,5rtkit-daemon22110-21packagekitd23:24:540
14759890,4rtkit-daemon3488060-21diskstats19:35:152
14759890,1rtkit-daemon0-21swapper/221:20:462
3860968286,1sleep244-21ksoftirqd/223:48:212
3819261280,5sleep23819405-21ssh23:20:242
3797215280,1sleep20-21swapper/223:08:572
3787856280,1sleep10-21swapper/123:00:511
3787856280,1sleep10-21swapper/123:00:501
3757178280,6sleep13605761-21diskmemload22:43:511
3724637280,2sleep23724645-21ssh22:22:212
3718049280,5sleep13718069-21acpi22:20:121
3699869280,6sleep13699874-21sh22:05:321
3685427280,6sleep13685433-21ssh21:58:581
3679170280,5sleep03680247-21kthreadcore21:55:170
3612094280,1sleep10-21swapper/121:11:011
3605467280,1sleep10-21swapper/121:06:261
3593375281,3sleep23593408-21apt-get21:00:172
3579848280,5sleep13579904-21ntp_states20:45:221
3568629281,3sleep13568663-21apt-get20:40:131
3568126280,1sleep00-21swapper/020:37:070
3555670280,6sleep00-21swapper/020:26:210
3543436280,1sleep10-21swapper/120:18:041
3537851280,5sleep00-21swapper/020:15:140
3531002280,1sleep10-21swapper/120:07:021
3518542280,1sleep10-21swapper/119:56:291
3500519281,4sleep13500552-21apt-get19:45:111
3481172280,1sleep00-21swapper/019:26:500
3475192280,1sleep00-21swapper/019:24:290
3474942280,1sleep20-21swapper/219:21:192
3472563280,5sleep13469331-21/usr/sbin/munin19:20:211
14759880,4rtkit-daemon3859361-21plymouth23:45:230
14759880,4rtkit-daemon3822202-21cut23:25:150
14759880,4rtkit-daemon3798417-21grep23:10:150
14759880,4rtkit-daemon3701246-21ssh22:08:520
14759880,4rtkit-daemon3662290-21df_inode21:45:142
14759880,4rtkit-daemon3586983-21sed20:55:012
14759880,4rtkit-daemon3460969-21munin-node23:35:320
14759880,4rtkit-daemon3460969-21munin-node22:40:042
14759880,4rtkit-daemon3460969-21munin-node21:14:282
14759880,4rtkit-daemon3460969-21munin-node20:20:032
14759880,4rtkit-daemon3460969-21munin-node19:50:112
14759880,4rtkit-daemon214-21plymouthd23:32:500
14759880,4rtkit-daemon214-21plymouthd22:15:230
14759880,4rtkit-daemon214-21plymouthd22:12:430
14759880,4rtkit-daemon214-21plymouthd20:11:362
14759880,4rtkit-daemon1-21systemd23:42:110
14759880,4rtkit-daemon1-21systemd00:18:211
14759880,1rtkit-daemon0-21swapper/220:25:062
3931123270,4sleep03931206-21/usr/sbin/munin00:30:270
3916698270,5sleep00-21swapper/000:22:580
3900698270,5sleep2214-21plymouthd00:13:062
3885905271,3sleep13885944-21apt-get00:05:111
3884870270,1sleep1214-21plymouthd00:03:231
3873221270,5sleep23874924-21latency23:55:182
3845324272,1sleep2171rcu_preempt23:39:472
3844041270,1sleep10-21swapper/123:36:381
3836944270,4sleep13652801-21kworker/u16:3+rpciod23:33:441
3819816271,4sleep13812683-21kworker/u16:0+events_power_efficient23:20:421
3813125271,3sleep13813131-21ssh23:18:511
3804401273,2sleep1331ktimers/123:12:051
3798022271,3sleep23798174-21df23:10:142
3797645271,3sleep13797876-21latency_hist23:10:011
3757454270,5sleep00-21swapper/022:44:260
3748957270,1sleep00-21swapper/022:38:220
3734402271,3sleep0214-21plymouthd22:30:150
3712603270,4sleep13714677-21cut22:15:191
3692551271,4sleep13652801-21kworker/u16:3+nfsiod22:02:041
3669589271,4sleep13669595-21ssh21:49:371
3629584270,1sleep10-21swapper/121:24:411
3613374270,4sleep03613382-21ssh21:14:020
3574796270,5sleep00-21swapper/020:45:130
3537397271,3sleep13537407-21ls20:15:001
3463423270,1sleep00-21swapper/019:15:160
14759870,4rtkit-daemon3460969-21munin-node19:47:062
14759870,4rtkit-daemon3460969-21munin-node19:18:022
14759870,4rtkit-daemon1-21systemd00:11:130
14759870,3rtkit-daemon729-21lldpd23:06:490
14759870,3rtkit-daemon3605761-21diskmemload22:02:010
14759870,3rtkit-daemon3522057-21kthreadcore20:00:172
14759870,3rtkit-daemon3460969-21munin-node20:36:392
14759870,3rtkit-daemon295-21systemd-journal19:55:352
14759870,3rtkit-daemon214-21plymouthd20:41:102
14759870,3rtkit-daemon214-21plymouthd00:26:341
14759870,3rtkit-daemon1-21systemd21:30:161
14759870,3rtkit-daemon1-21systemd00:28:522
14759870,2rtkit-daemon3558804-21cut20:30:182
14759870,1rtkit-daemon0-21swapper/220:55:142
3925294260,1sleep00-21swapper/000:29:260
3868021260,1sleep10-21swapper/123:51:161
3848637260,3sleep23850715-21tr23:40:192
3780693260,4sleep10-21swapper/122:57:431
3779688260,1sleep20-21swapper/222:55:272
3766035261,2sleep13766057-21apt-get22:50:121
3765583260,1sleep00-21swapper/022:49:480
3733384260,1sleep10-21swapper/122:29:091
3708880260,4sleep13652801-21kworker/u16:3+nfsiod22:13:061
3708356260,1sleep20-21swapper/222:11:512
3676126260,4sleep10-21swapper/121:50:461
3675498260,3sleep03675625-21mailstats21:50:250
3646721261,2sleep10-21swapper/121:35:191
3625270261,3sleep03627042-21latency21:20:190
3593059260,1sleep10-21swapper/120:59:361
3573838261,2sleep00-21swapper/020:40:230
3568087260,4sleep10-21swapper/120:36:361
3561874260,5sleep00-21swapper/020:31:180
3558437260,4sleep1214-21plymouthd20:30:191
3512262260,1sleep00-21swapper/019:50:350
3494324260,4sleep00-21swapper/019:40:050
3493710260,4sleep10-21swapper/119:38:571
3487477260,4sleep0214-21plymouthd19:33:220
3456989261,2sleep00-21swapper/019:10:180
3454286260,1sleep10-21swapper/119:07:201
3449297260,3sleep03450189-21kthreadcore19:05:180
14759860,3rtkit-daemon720-21snmpd22:25:210
14759860,3rtkit-daemon3525351-21cpu20:05:142
3871332251,2sleep10-21swapper/123:55:171
3829528250,2sleep1214-21plymouthd23:30:011
3813399251,1sleep20-21swapper/223:19:362
3763326250,3sleep13763441-21ssh22:45:241
3743001250,2sleep10-21swapper/122:35:181
3660379250,3sleep00-21swapper/021:41:470
3654404250,2sleep10-21swapper/121:40:161
3618617250,3sleep13619181-21ssh21:15:191
3588853250,3sleep03590857-21tr20:55:190
3548979250,3sleep13549076-21ls20:20:251
3518817251,2sleep03468715-21kworker/u16:3+ext4-rsv-conversion19:59:590
3507558250,2sleep10-21swapper/119:50:201
3504890251,2sleep00-21swapper/019:45:180
3463119251,1sleep10-21swapper/119:15:131
3601648240,2sleep00-21swapper/021:05:200
3598578240,2sleep10-21swapper/121:00:281
3519193240,2sleep13519297-21awk20:00:131
3519130240,2sleep00-21swapper/020:00:120
3459026240,2sleep10-21swapper/119:10:201
3580993230,1sleep10-21swapper/120:50:071
3543852230,1sleep00-21swapper/020:20:040
3493795230,1sleep10-21swapper/119:40:011
3487855230,1sleep0214-21plymouthd19:35:040
3478569230,1sleep10-21swapper/119:25:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional