You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-28 - 14:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4.osadl.org (updated Wed Feb 28, 2024 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4039091992620,5cyclictest0-21swapper/123:10:001
4039098992220,1cyclictest4173662-21cron21:15:003
4039098992214,1cyclictest191rcu_preempt00:35:313
4039096992118,2cyclictest131957-21irqcore22:55:262
4039096992118,2cyclictest130987-21latency_hist22:55:002
4039091992117,3cyclictest10646-21latency_hist21:35:001
4039091992117,2cyclictest519-21in:imuxsock22:10:001
4039088992114,5cyclictest0-21swapper/019:40:000
403909899206,3cyclictest0-21swapper/320:10:243
4039098992014,4cyclictest503-21dbus-daemon22:15:003
4039098992014,1cyclictest191rcu_preempt23:50:263
4039098992013,1cyclictest191rcu_preempt21:15:423
4039096992017,2cyclictest4128992-21systemd-run20:35:002
4039096992017,2cyclictest4099524-21kthreadcore20:05:262
4039096992016,3cyclictest153794-21sh23:10:002
4039088992016,2cyclictest0-21swapper/023:35:010
403909899197,1cyclictest191rcu_preempt22:40:473
403909899196,6cyclictest0-21swapper/322:30:343
4039098991916,2cyclictest11447-21hddtemp_smartct21:35:303
4039098991912,1cyclictest191rcu_preempt22:03:243
4039096991916,2cyclictest4113704-21irqcore20:20:312
4039096991910,3cyclictest4150271-21awk20:55:012
4039088991914,4cyclictest59150irq/127-enp0s29f1:rx-023:15:570
403909899189,5cyclictest0-21swapper/323:20:013
403909899189,5cyclictest0-21swapper/319:35:343
403909899184,4cyclictest243159-21unixbench_multi00:05:373
4039098991815,2cyclictest4156268-21irqcore21:00:253
4039098991812,1cyclictest191rcu_preempt21:40:393
4039098991811,1cyclictest191rcu_preempt23:38:013
4039098991811,1cyclictest191rcu_preempt21:20:323
4039096991815,2cyclictest213569-21sshd23:46:562
403909199186,3cyclictest124526-21cut22:50:311
4039091991814,3cyclictest258120-21ntp_offset00:15:331
4039088991815,2cyclictest48262-21latency_hist22:00:010
403909899179,2cyclictest0-21swapper/320:50:303
403909899178,4cyclictest0-21swapper/300:31:123
403909899177,5cyclictest50-21ksoftirqd/322:15:313
403909899177,5cyclictest50-21ksoftirqd/322:15:303
403909899177,1cyclictest191rcu_preempt23:42:143
403909899177,1cyclictest191rcu_preempt22:23:163
403909899175,3cyclictest526-21gdbus23:05:013
4039098991715,1cyclictest481rcuc/323:15:003
403909899171,3cyclictest48253-21systemd-run22:00:003
4039098991713,2cyclictest0-21swapper/323:55:223
4039098991713,1cyclictest191rcu_preempt22:38:573
4039098991713,1cyclictest191rcu_preempt19:55:313
4039098991712,2cyclictest191rcu_preempt23:48:553
4039098991712,2cyclictest191rcu_preempt23:06:323
4039098991711,1cyclictest191rcu_preempt22:56:183
4039096991713,3cyclictest172573-21kthreadcore23:20:292
4039096991712,3cyclictest0-21swapper/222:00:262
4039096991712,3cyclictest0-21swapper/221:05:292
4039091991714,2cyclictest4189326-21apt-key21:25:001
4039091991714,2cyclictest4065598-21idleruntime-cro19:35:001
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-023:00:260
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-022:15:430
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-022:15:420
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-021:40:060
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-021:19:110
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-020:21:000
4039088991714,2cyclictest59150irq/127-enp0s29f1:rx-000:27:330
4039088991713,3cyclictest59150irq/127-enp0s29f1:rx-023:08:560
4039088991713,3cyclictest59150irq/127-enp0s29f1:rx-021:45:310
4039088991712,2cyclictest59150irq/127-enp0s29f1:rx-022:00:550
4039088991710,5cyclictest4136259-21cut20:40:280
4039088991710,3cyclictest59150irq/127-enp0s29f1:rx-022:35:270
403909899169,1cyclictest191rcu_preempt19:25:353
403909899166,4cyclictest0-21swapper/320:35:323
403909899166,3cyclictest0-21swapper/319:50:393
403909899165,2cyclictest0-21swapper/320:50:003
403909899164,7cyclictest86895-21irqcore22:25:283
403909899164,1cyclictest191rcu_preempt21:46:593
403909899162,4cyclictest50-21ksoftirqd/323:20:493
403909899162,3cyclictest0-21swapper/320:30:283
4039098991615,1cyclictest481rcuc/319:20:013
4039098991614,1cyclictest0-21swapper/320:25:013
403909899161,3cyclictest0-21swapper/300:26:513
4039098991613,2cyclictest481rcuc/319:15:003
4039098991612,3cyclictest481rcuc/319:30:323
4039098991612,3cyclictest181806-21nvmesmart_nvme023:25:353
4039098991612,1cyclictest191rcu_preempt22:05:533
4039098991612,1cyclictest191rcu_preempt00:15:293
4039098991611,4cyclictest124481-21grep22:50:313
4039098991611,3cyclictest4071256-21fschecks_count19:40:253
4039098991611,1cyclictest191rcu_preempt21:29:083
4039098991610,1cyclictest191rcu_preempt00:03:123
403909899160,3cyclictest246453-21fw_conntrack00:10:243
403909899160,3cyclictest185085-21apt-get23:30:183
403909699161,6cyclictest19624-21irqstats21:40:302
4039096991613,2cyclictest4186468-21unixbench_multi21:20:332
4039096991613,2cyclictest215989-21ssh23:50:242
4039096991613,2cyclictest0-21swapper/223:00:372
4039096991612,3cyclictest235517-21unixbench-2d00:00:352
4039096991612,2cyclictest0-21swapper/219:25:012
4039096991612,2cyclictest0-21swapper/200:40:002
4039096991612,2cyclictest0-21swapper/200:20:282
4039096991610,4cyclictest149712-21systemd-journal22:22:562
4039091991613,2cyclictest4086445-21sh19:55:011
4039091991613,2cyclictest158825-21needreboot23:10:301
4039091991613,2cyclictest149712-21systemd-journal22:02:121
4039091991612,2cyclictest0-21swapper/123:20:231
4039091991611,3cyclictest208392-21cat23:45:251
403908899169,2cyclictest59150irq/127-enp0s29f1:rx-023:45:260
4039088991614,1cyclictest59150irq/127-enp0s29f1:rx-021:20:480
4039088991614,1cyclictest59150irq/127-enp0s29f1:rx-000:07:430
4039088991613,2cyclictest59950irq/128-enp0s29f1:tx-023:40:300
4039088991613,2cyclictest59950irq/128-enp0s29f1:tx-000:20:280
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-023:37:210
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-023:10:020
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-022:55:250
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-022:43:240
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-022:30:320
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-021:51:470
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-021:25:370
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-021:10:370
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-021:04:170
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-020:35:250
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-020:31:050
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-020:08:180
4039088991613,2cyclictest59150irq/127-enp0s29f1:rx-000:30:190
4039088991612,3cyclictest257177-21kthreadcore00:15:300
4039088991612,2cyclictest0-21swapper/023:25:330
4039088991611,4cyclictest59150irq/127-enp0s29f1:rx-022:25:290
4039088991611,1cyclictest59150irq/127-enp0s29f1:rx-022:24:590
403909899156,6cyclictest0-21swapper/321:50:563
403909899155,5cyclictest50-21ksoftirqd/300:21:383
403909899152,7cyclictest50-21ksoftirqd/319:10:013
403909899152,3cyclictest0-21swapper/320:40:243
4039098991512,2cyclictest4118265-21systemd20:25:013
4039098991511,1cyclictest191rcu_preempt20:07:433
4039096991512,2cyclictest60150irq/132-enp0s29f1:tx-223:30:222
4039096991512,2cyclictest4158203-21kthreadcore21:00:272
4039096991512,2cyclictest401ktimers/223:25:302
4039096991512,2cyclictest0-21swapper/219:35:002
4039096991511,3cyclictest4086853-21cpuspeed_turbos19:55:282
4039096991511,3cyclictest4055561-21fschecks_count19:25:242
4039096991511,2cyclictest0-21swapper/223:20:012
4039096991510,3cyclictest0-21swapper/200:05:032
403909699150,14cyclictest4070816-21munin-plugin-st19:40:012
403909199158,5cyclictest223048-21sed23:55:001
4039091991512,2cyclictest514-21systemd-logind21:14:131
4039091991512,2cyclictest311ktimers/100:23:571
4039091991512,2cyclictest149712-21systemd-journal21:44:131
4039091991512,2cyclictest14415-21kthreadcore21:35:341
4039091991512,2cyclictest0-21swapper/123:05:001
4039091991512,2cyclictest0-21swapper/120:40:001
4039091991512,1cyclictest0-21swapper/119:15:001
4039091991511,3cyclictest4155846-21cpuspeed_turbos21:00:211
4039091991511,3cyclictest4055878-21irqcore19:25:281
4039091991511,2cyclictest0-21swapper/122:35:001
4039091991511,2cyclictest0-21swapper/121:18:551
4039091991511,2cyclictest0-21swapper/100:10:201
4039088991513,2cyclictest59150irq/127-enp0s29f1:rx-000:35:310
4039088991513,1cyclictest59150irq/127-enp0s29f1:rx-022:53:530
4039088991513,1cyclictest59150irq/127-enp0s29f1:rx-022:46:250
4039088991513,1cyclictest59150irq/127-enp0s29f1:rx-022:05:070
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-023:50:260
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-022:10:230
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-021:30:180
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-020:57:470
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-019:15:080
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-019:12:280
4039088991512,2cyclictest59150irq/127-enp0s29f1:rx-000:01:550
4039088991512,2cyclictest0-21swapper/020:55:000
4039088991511,3cyclictest59150irq/127-enp0s29f1:rx-000:10:200
4039088991511,3cyclictest4108113-21cpuspeed_turbos20:15:230
4039088991510,4cyclictest149712-21systemd-journal19:53:560
403909899148,2cyclictest191rcu_preempt20:55:293
4039098991411,2cyclictest481rcuc/320:00:313
4039098991411,2cyclictest4161020-21/usr/sbin/munin21:05:293
403909899140,4cyclictest4040738-21cut19:10:313
403909699149,3cyclictest40616-21cron21:55:002
403909699144,2cyclictest191rcu_preempt22:05:252
403909699141,5cyclictest0-21swapper/222:26:552
403909699141,4cyclictest4145050-21awk20:50:002
4039096991413,1cyclictest1301304-21kworker/2:0+events_power_efficient22:11:382
4039096991410,3cyclictest4092224-21gltestperf20:00:292
4039096991410,3cyclictest258548-21unixbench_singl00:15:372
4039096991410,3cyclictest109435-21sed22:40:282
4039096991410,2cyclictest4085965-21grep19:50:412
4039096991410,2cyclictest0-21swapper/220:40:012
403909199143,2cyclictest1202422cut22:45:271
4039091991411,2cyclictest86484-21cpuspeed_turbos22:25:231
4039091991411,2cyclictest238656-21cpuspeed_turbos00:05:221
4039091991411,2cyclictest0-21swapper/121:55:311
4039091991411,2cyclictest0-21swapper/119:15:001
4039091991410,2cyclictest0-21swapper/122:45:001
4039091991410,2cyclictest0-21swapper/100:00:241
403909199140,5cyclictest818-21runrttasks23:55:061
403909199140,5cyclictest4069958-21ntpq19:35:361
4039088991411,2cyclictest59150irq/127-enp0s29f1:rx-019:43:360
4039088991411,2cyclictest59150irq/127-enp0s29f1:rx-019:23:180
4039098991311,1cyclictest0-21swapper/320:15:303
4039098991310,2cyclictest4076277-21apt-get19:45:193
403909699139,2cyclictest0-21swapper/221:30:382
403909699138,2cyclictest0-21swapper/222:35:372
403909699137,5cyclictest27960-21cut21:45:242
403909699137,4cyclictest94130-21perl22:30:262
403909699136,5cyclictest15598-21cat21:35:402
4039096991310,2cyclictest284092-21sh00:35:002
403909199139,3cyclictest4126449-21sort20:30:341
403909199138,3cyclictest0-21swapper/121:25:251
403909199138,3cyclictest0-21swapper/120:05:001
4039091991310,2cyclictest4165046-21needreboot21:05:331
4039091991310,2cyclictest285086-21cpuspeed_turbos00:35:191
4039091991310,2cyclictest0-21swapper/122:40:011
403908899139,3cyclictest4055648-21fschecks_time19:25:260
403908899138,4cyclictest149712-21systemd-journal20:00:010
4039088991310,2cyclictest59150irq/127-enp0s29f1:rx-021:07:200
4039088991310,2cyclictest4091997-21cpuspeed_turbos20:00:260
403909699129,2cyclictest4118914-21cpuspeed_turbos20:25:232
403909699129,2cyclictest223705-21gltestperf23:55:262
403909699128,3cyclictest4076567-21gltestperf19:45:232
403909699128,2cyclictest0-21swapper/222:15:242
403909699128,2cyclictest0-21swapper/222:15:232
403909699128,2cyclictest0-21swapper/220:15:232
403909699127,4cyclictest401ktimers/223:41:532
403909699126,4cyclictest149712-21systemd-journal23:40:012
403909699124,6cyclictest154812-21df23:10:212
403909699122,8cyclictest4173676-21munin-run21:15:002
403909699121,4cyclictest4049893-21latency_hist19:20:002
403909699120,9cyclictest4181519-21cat21:20:002
403909199129,2cyclictest4076764-21sed19:45:261
403909199129,2cyclictest177647-21cpuspeed_turbos23:25:251
403909199129,2cyclictest0-21swapper/120:50:221
403909199128,3cyclictest4054639-21unixbench_multi19:20:391
403909199128,3cyclictest311ktimers/121:50:261
403909199128,2cyclictest135826-21sensors22:55:341
403909199127,3cyclictest4112921-21sh20:20:001
403909199126,5cyclictest65008-21cut22:10:311
403909199126,4cyclictest277835-21cut00:30:251
403909199125,5cyclictest503-21dbus-daemon20:10:001
403909199120,5cyclictest271316-21cat00:25:281
403909199120,4cyclictest4091654-21awk20:00:011
403908899129,2cyclictest59950irq/128-enp0s29f1:tx-020:10:230
403908899129,2cyclictest41400822sleep020:45:210
403908899128,2cyclictest0-21swapper/019:30:300
4039088991211,1cyclictest59150irq/127-enp0s29f1:rx-019:45:370
403909699117,3cyclictest4155529-21systemctl21:00:002
403909699117,2cyclictest0-21swapper/220:10:242
403909699116,4cyclictest4043465-21tr19:10:352
403909699116,4cyclictest269844-21sed00:25:252
403909699112,2cyclictest4165968-21diskmemload00:13:382
403909699111,2cyclictest4139839-21latency_hist20:45:012
403909699110,4cyclictest0-21swapper/221:57:072
403909199119,1cyclictest0-21swapper/123:40:291
403909199118,2cyclictest4144144-21cut20:45:311
403909199118,2cyclictest4122844-21needreboot20:25:311
403909199118,2cyclictest193145-21gltestperf23:35:281
403909199118,2cyclictest164321-21sh23:15:321
403909199118,2cyclictest0-21swapper/122:15:311
403909199118,2cyclictest0-21swapper/122:15:301
403909199118,2cyclictest0-21swapper/120:10:291
403909199117,3cyclictest4113538-21gltestperf20:20:271
403909199117,3cyclictest311ktimers/122:23:061
403909199117,3cyclictest311ktimers/121:45:181
403909199117,3cyclictest189743-21unixbench_multi23:30:341
403908899115,4cyclictest4119125-21fw_forwarded_lo20:25:250
403909699105,4cyclictest4075042-21sort19:40:322
403909199105,4cyclictest4071602-21sed19:40:281
403909199100,3cyclictest0-21swapper/120:55:251
40390919993,4cyclictest4134889-21apt-get20:40:221
40390889964,1cyclictest0-21swapper/019:10:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional