You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 11:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7.osadl.org (updated Sun Nov 16, 2025 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3380688996033,6cyclictest14-21ksoftirqd/021:48:380
3380688995513,30cyclictest0-21swapper/000:07:030
3380688995032,3cyclictest0-21swapper/021:23:010
338068899414,14cyclictest3603882-21ssh00:37:590
3380688994020,11cyclictest3557779-21ntp_states23:43:020
3380688993725,3cyclictest0-21swapper/021:43:030
3380688993614,19cyclictest151ktimers/019:15:530
338068899353,7cyclictest3527334-21latency_hist23:10:530
338069099315,19cyclictest3578151-21sendmail-msp00:06:022
338068899312,8cyclictest3460012-21ntpq21:58:020
3380688993121,1cyclictest0-21swapper/000:45:530
338068899311,23cyclictest0-21swapper/000:22:560
338068899311,22cyclictest111788-21systemd-journal21:52:480
338068899303,2cyclictest0-21swapper/023:45:580
3380689992915,10cyclictest1008-21mta-sts-daemon00:07:021
338068899293,7cyclictest3529690-21ntpq23:13:030
3380688992919,3cyclictest0-21swapper/019:42:590
3380688992918,3cyclictest0-21swapper/000:12:580
338068899291,21cyclictest3570918-21grep23:57:580
338068899272,2cyclictest171rcu_preempt22:13:100
3380688992714,2cyclictest0-21swapper/022:22:560
338068899271,21cyclictest3392480-21latency_hist19:50:550
338068899271,20cyclictest0-21swapper/021:37:560
338068899270,20cyclictest3505673-21/usr/sbin/munin22:48:090
3380690992617,6cyclictest0-21swapper/223:35:572
338068899268,15cyclictest1008-21mta-sts-daemon00:20:150
338068899262,20cyclictest111788-21systemd-journal23:53:090
3380688992615,5cyclictest0-21swapper/022:32:570
3380688992615,5cyclictest0-21swapper/022:32:570
338068999253,16cyclictest3478832-21ssh22:18:011
338068899253,4cyclictest171rcu_preempt22:28:290
3380688992517,6cyclictest0-21swapper/019:51:000
338068899251,18cyclictest0-21swapper/022:42:590
338068899250,4cyclictest191rcuc/022:40:520
338068899250,3cyclictest171rcu_preempt22:53:080
338068899250,1cyclictest0-21swapper/021:15:530
338069099248,12cyclictest111788-21systemd-journal22:45:532
3380690992414,4cyclictest3560035-21sendmail-msp23:45:582
338068999247,13cyclictest259-21jbd2/mmcblk0p2-820:30:571
338068899249,8cyclictest0-21swapper/023:33:020
338068899248,5cyclictest3436786-21ntpq21:33:030
338068899246,3cyclictest0-21swapper/023:28:010
338068899246,3cyclictest0-21swapper/023:28:010
338068899243,4cyclictest11050irq/107-eth%d23:40:470
3380690992314,5cyclictest0-21swapper/222:58:042
338068999233,5cyclictest3380687-21cyclictest19:15:531
3380688992313,9cyclictest11050irq/107-eth%d21:27:120
112699230,1rtkit-daemon1125-21rtkit-daemon23:22:142
338069099224,16cyclictest3454787-21/usr/sbin/munin21:52:572
3380690992216,3cyclictest3482930-21fschecks_time22:22:582
338068899223,4cyclictest171rcu_preempt23:03:000
3380688992222,0cyclictest0-21swapper/023:20:520
112699220,3rtkit-daemon1125-21rtkit-daemon20:09:062
338069099213,10cyclictest11050irq/107-eth%d21:46:062
3380690992116,4cyclictest0-21swapper/223:08:002
3380690992114,5cyclictest3430052-21latency_hist21:25:532
338068999213,14cyclictest111788-21systemd-journal20:08:341
338068999211,5cyclictest3604714-21grep00:38:121
338068899217,9cyclictest0-21swapper/020:18:050
338068899212,13cyclictest3469537-21ssh22:08:060
3380688992120,1cyclictest0-21swapper/019:38:050
3380688992118,2cyclictest0-21swapper/022:18:000
3380688992110,8cyclictest0-21swapper/000:28:040
338068899210,20cyclictest3399666-21fschecks_count20:12:590
338069099205,12cyclictest3596136-21tail00:27:562
3380690992017,2cyclictest3403042-21fschecks_time20:22:582
3380690992014,3cyclictest3566377-21fschecks_count23:53:022
338068999205,6cyclictest3427660-21ntpq21:23:011
338069099195,11cyclictest3518538-21ssh23:01:172
338069099194,11cyclictest3457813-21ssh21:55:532
338069099191,6cyclictest0-21swapper/222:12:552
3380690991916,2cyclictest0-21swapper/219:27:592
3380690991915,3cyclictest3408337-21fschecks_time20:37:592
3380690991913,4cyclictest3419830-21fschecks_time21:12:572
338069099190,6cyclictest0-21swapper/223:59:072
338068999196,8cyclictest3504241-21sendmail-msp22:45:531
338068999194,4cyclictest3492735-21ntpq22:33:031
338068999194,4cyclictest3492735-21ntpq22:33:031
338068999192,9cyclictest3550313-21apt-get23:35:541
3380689991913,5cyclictest3518871-21ssh23:01:501
338068899194,11cyclictest762-21in:imuxsock21:05:530
338068899191,15cyclictest3578145-21dpkg00:05:530
338069099189,6cyclictest3423278-21ntpq21:18:072
338069099182,9cyclictest1008-21mta-sts-daemon23:27:112
338069099182,9cyclictest1008-21mta-sts-daemon23:27:112
338069099181,6cyclictest3522723-21sendmail-msp23:13:022
3380690991814,3cyclictest3402345-21unixbench_multi20:18:092
338069099181,3cyclictest3557847-21ssh23:43:032
338068999188,1cyclictest3486321-21ssh22:26:261
338068999186,10cyclictest3448415-21sendmail-msp21:45:561
338068999185,9cyclictest3423239-21/usr/sbin/munin21:18:061
338068999185,7cyclictest3459656-21/usr/sbin/munin21:57:591
338068999182,8cyclictest3448415-21sendmail-msp21:53:031
338068999182,12cyclictest3541460-21sendmail-msp23:30:531
338068999182,11cyclictest3472852-21ssh22:12:201
3380689991813,4cyclictest1006-21mta-sts-daemon19:23:021
338068899184,11cyclictest3595006-21sendmail-msp00:32:540
3380688991816,1cyclictest3423511-21switchtime21:18:110
3380688991815,2cyclictest1006-21mta-sts-daemon20:08:020
338068899181,10cyclictest0-21swapper/023:22:510
338069199173,8cyclictest0-21swapper/300:33:073
338069199173,7cyclictest0-21swapper/321:52:573
338069199173,7cyclictest0-21swapper/300:06:263
338069099174,10cyclictest3504241-21sendmail-msp22:53:342
338069099174,10cyclictest3419180-21/usr/sbin/munin21:08:092
338069099173,12cyclictest3492148-21/usr/sbin/munin22:32:572
338069099173,12cyclictest3492148-21/usr/sbin/munin22:32:572
338069099172,9cyclictest3441078-21/usr/sbin/munin21:37:572
338069099172,8cyclictest111788-21systemd-journal23:30:532
3380690991712,3cyclictest259-21jbd2/mmcblk0p2-820:13:002
3380689991714,2cyclictest3380687-21cyclictest23:07:561
3380689991712,2cyclictest764-21rs:main1
338068899178,7cyclictest3418206-21fschecks_time21:07:580
338068899174,10cyclictest14-21ksoftirqd/022:58:360
3380688991710,5cyclictest3379309-21kworker/u8:2+events_unbound19:17:570
112699170,0rtkit-daemon0-21swapper/100:41:311
338069199163,6cyclictest0-21swapper/321:40:563
338069099164,8cyclictest619-21mta-sts-daemon22:38:022
338069099164,8cyclictest3479051-21switchtime22:18:052
338069099163,8cyclictest11050irq/107-eth%d00:12:422
338069099163,4cyclictest3588894-21ntpq00:18:042
338069099162,9cyclictest3437076-21ssh21:33:092
338069099162,11cyclictest3607385-21ntp_states00:43:072
3380690991612,3cyclictest3444130-21packagekitd21:40:562
3380690991611,4cyclictest0-21swapper/200:22:582
338068999165,3cyclictest11050irq/107-eth%d22:43:021
338068999163,11cyclictest3429949-21sendmail-msp21:27:121
338068999162,11cyclictest261rcuc/123:18:031
3380689991612,3cyclictest3482930-21fschecks_time22:22:581
338068999160,12cyclictest111788-21systemd-journal23:46:041
338068899162,5cyclictest3405747-21apt-get20:30:570
338068899162,13cyclictest1561943-1kworker/0:1H+kblockd19:33:070
112699160,1rtkit-daemon0-21swapper/220:01:352
338069199155,4cyclictest0-21swapper/321:05:533
338069199153,7cyclictest0-21swapper/321:33:003
338069199153,5cyclictest0-21swapper/320:33:093
338069199152,7cyclictest0-21swapper/322:32:573
338069199152,7cyclictest0-21swapper/322:32:573
338069199152,7cyclictest0-21swapper/321:45:563
338069099154,8cyclictest3383412-21/usr/sbin/munin19:23:012
338069099154,6cyclictest3575809-21ntpq00:03:012
338069099152,9cyclictest3430924-21ssh21:27:122
338069099152,10cyclictest3390581-21sendmail-msp19:51:002
3380690991511,3cyclictest3600364-21gltestperf00:33:042
3380690991510,3cyclictest0-21swapper/222:07:562
338069099150,8cyclictest11050irq/107-eth%d22:29:012
338068999156,2cyclictest0-21swapper/121:33:031
338068999153,8cyclictest3566183-21ssh23:52:571
338068999151,4cyclictest3465002-21ssh22:03:091
3380689991512,2cyclictest3570827-21fschecks_time23:57:581
3380689991512,2cyclictest3516374-21unixbench_multi22:58:101
3380689991511,3cyclictest3391081-21cut19:47:591
338068899153,1cyclictest171rcu_preempt20:02:590
338068899152,12cyclictest3464688-21grep22:03:030
3380688991511,2cyclictest3412628-21unixbench_singl20:48:120
338068899150,9cyclictest3409638-21cut20:42:560
338068899150,9cyclictest3409638-21cut20:42:560
112699150,2rtkit-daemon1125-21rtkit-daemon20:57:190
338069199146,4cyclictest0-21swapper/323:13:033
338069199144,5cyclictest0-21swapper/300:18:103
338069199143,7cyclictest0-21swapper/300:28:043
338069199143,5cyclictest0-21swapper/323:53:033
338069199142,8cyclictest0-21swapper/323:28:043
338069199142,8cyclictest0-21swapper/323:28:043
338069199142,6cyclictest0-21swapper/319:51:003
338069199141,8cyclictest0-21swapper/300:05:523
338069099148,4cyclictest3404114-21sendmail-msp20:30:582
3380690991411,2cyclictest3414256-21unixbench_singl20:53:092
3380690991411,2cyclictest3388856-21unixbench_singl19:38:092
3380690991411,2cyclictest0-21swapper/222:42:582
338069099141,11cyclictest1008-21mta-sts-daemon19:50:532
338069099141,10cyclictest3534336-21cut23:18:002
338069099141,10cyclictest3394491-21aten_r2power_en19:57:582
338068999149,4cyclictest3504241-21sendmail-msp22:52:571
338068999146,7cyclictest3594982-21munin-run00:25:531
338068999145,8cyclictest3388434-21/usr/sbin/munin19:38:041
338068999145,4cyclictest0-21swapper/122:08:101
338068999143,8cyclictest3539114-21ssh23:23:021
338068999142,9cyclictest3584720-21/usr/sbin/munin00:13:041
338068999142,9cyclictest3557378-21/usr/sbin/munin23:42:581
338068999142,8cyclictest3405111-21/usr/sbin/munin20:28:031
338068999142,10cyclictest3440541-21aten_r2power_fr21:37:541
338068999142,10cyclictest111788-21systemd-journal19:28:061
338068999141,8cyclictest3600236-21tune2fs00:33:031
338068999141,4cyclictest3578123-21munin-run00:05:531
3380689991412,1cyclictest3409366-21cron20:40:521
338068999141,10cyclictest3412629-21sed20:48:121
338068899144,3cyclictest171rcu_preempt20:52:560
338068899141,11cyclictest3408249-21fschecks_count20:37:590
338068899141,10cyclictest3394522-21/usr/sbin/munin19:57:580
3380688991410,3cyclictest3384550-21fschecks_time19:28:020
112699140,2rtkit-daemon1125-21rtkit-daemon20:22:361
338069199134,5cyclictest0-21swapper/323:35:543
338069199134,5cyclictest0-21swapper/321:18:073
338069199134,4cyclictest0-21swapper/323:48:383
338069199132,6cyclictest0-21swapper/323:10:533
338069199132,6cyclictest0-21swapper/323:03:063
338069099133,9cyclictest3416521-21fschecks_time21:02:572
338069099133,8cyclictest3405762-21idleruntime-cro20:30:522
338069099132,9cyclictest3381267-21fschecks_time19:17:592
338069099132,5cyclictest3604715-21grep00:38:122
338069099132,10cyclictest3383997-21sendmail-msp19:33:022
338069099131,9cyclictest3415016-21tail20:58:012
338069099131,8cyclictest3389417-21/usr/sbin/munin19:42:592
338068999138,3cyclictest259-21jbd2/mmcblk0p2-819:32:581
338068999135,6cyclictest0-21swapper/123:12:571
338068999133,7cyclictest3419802-21/usr/sbin/munin21:12:561
338068999132,9cyclictest3543445-21/usr/sbin/munin23:28:041
338068999132,9cyclictest3543445-21/usr/sbin/munin23:28:041
338068999132,8cyclictest3413174-21/usr/sbin/munin20:52:561
338068999132,8cyclictest3396330-21/usr/sbin/munin20:02:541
338068999132,8cyclictest3394790-21grep19:58:001
338068899131,10cyclictest3382553-21/usr/sbin/munin19:22:590
338069199125,4cyclictest0-21swapper/320:18:053
338069199124,5cyclictest0-21swapper/322:48:023
338069199124,5cyclictest0-21swapper/320:48:073
338069199124,5cyclictest0-21swapper/320:28:023
338069199123,6cyclictest0-21swapper/300:12:423
338069199123,5cyclictest0-21swapper/323:22:573
338069199123,5cyclictest0-21swapper/322:17:543
338069199122,7cyclictest0-21swapper/320:03:043
338069199122,6cyclictest0-21swapper/322:58:043
338069199122,6cyclictest0-21swapper/300:22:583
338069199122,4cyclictest0-21swapper/323:17:553
338069199122,4cyclictest0-21swapper/321:57:593
338069199122,4cyclictest0-21swapper/321:36:493
338069199121,7cyclictest0-21swapper/322:50:533
338069199121,7cyclictest0-21swapper/321:22:573
338069199121,6cyclictest0-21swapper/322:22:563
338069199121,6cyclictest0-21swapper/320:43:013
338069199121,6cyclictest0-21swapper/320:43:013
338069099125,5cyclictest259-21jbd2/mmcblk0p2-820:47:062
338069099123,6cyclictest3380651-21munin-run19:15:532
338069099121,4cyclictest3464434-21/usr/sbin/munin22:03:012
338068999125,6cyclictest3445444-21aten_r2power_en21:42:571
338068999123,2cyclictest3401389-21fschecks_time20:17:591
338068999122,9cyclictest3392783-21apt-get19:50:591
338068999122,6cyclictest3411008-21cron20:45:521
338068999122,6cyclictest3411008-21cron20:45:521
338068999121,9cyclictest3496872-21fschecks_time22:37:581
338068999121,8cyclictest3414669-21/usr/sbin/munin20:57:591
338068999121,7cyclictest111788-21systemd-journal21:08:091
338068999121,6cyclictest3390677-21grep19:45:531
338068999120,9cyclictest3416519-21cut21:02:571
338068999120,10cyclictest3589410-21grep00:18:101
338068899129,1cyclictest0-21swapper/020:27:580
338069199114,4cyclictest0-21swapper/300:43:073
338069199113,5cyclictest0-21swapper/322:05:533
338069199113,5cyclictest0-21swapper/321:27:123
338069199113,5cyclictest0-21swapper/319:57:573
338069199113,4cyclictest0-21swapper/322:03:033
338069199112,6cyclictest0-21swapper/323:45:533
338069199112,6cyclictest0-21swapper/321:12:583
338069199112,5cyclictest0-21swapper/323:57:563
338069199112,5cyclictest0-21swapper/323:30:533
338069199112,5cyclictest0-21swapper/322:28:293
338069199112,5cyclictest0-21swapper/321:07:593
338069199112,3cyclictest0-21swapper/322:11:163
338069199111,6cyclictest0-21swapper/319:18:053
338069199111,4cyclictest0-21swapper/322:36:433
338069099113,6cyclictest1005-21mta-sts-daemon20:43:022
338069099113,6cyclictest1005-21mta-sts-daemon20:43:022
338068999113,4cyclictest3400141-21/usr/sbin/munin20:13:031
338068999113,4cyclictest1006-21mta-sts-daemon19:18:091
338068899111,7cyclictest3403453-21/usr/sbin/munin20:23:020
338069199104,5cyclictest0-21swapper/320:13:003
338069199103,6cyclictest0-21swapper/319:22:563
338069199102,6cyclictest0-21swapper/320:57:593
338069199102,5cyclictest0-21swapper/319:38:003
338069199102,4cyclictest0-21swapper/320:37:573
338069199102,3cyclictest0-21swapper/320:52:553
338069199102,3cyclictest0-21swapper/320:25:533
338069199101,6cyclictest0-21swapper/322:43:033
338069199101,6cyclictest0-21swapper/320:08:023
33806919995,3cyclictest0-21swapper/319:45:533
33806919992,3cyclictest0-21swapper/319:32:543
33806919991,4cyclictest0-21swapper/300:36:223
33806919990,4cyclictest0-21swapper/319:47:573
33806919983,3cyclictest0-21swapper/319:26:093
33806919951,2cyclictest0-21swapper/319:15:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional