You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 10:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7.osadl.org (updated Sun Nov 30, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
377290999671,41cyclictest3920411-21ssh23:16:110
377290999542,14cyclictest3965568-40gzip00:07:570
3772910994014,19cyclictest111788-21systemd-journal22:33:111
377290999402,27cyclictest3917391-21ssh23:13:070
3772911993931,3cyclictest0-21swapper/200:07:532
377291099396,20cyclictest111788-21systemd-journal20:41:071
377290999367,20cyclictest0-21swapper/022:01:080
3772911993414,15cyclictest111788-21systemd-journal23:26:082
3772909993427,5cyclictest3772908-21cyclictest21:31:100
377291099335,12cyclictest3965969-21ssh00:07:361
3772910993315,16cyclictest3912401-21basename23:08:031
377290999339,8cyclictest259-21jbd2/mmcblk0p2-800:32:390
377290999336,19cyclictest3803623-21sendmail-msp20:48:110
377290999333,20cyclictest0-21swapper/023:31:210
3772909993326,5cyclictest0-21swapper/023:23:140
3772909993316,11cyclictest764-21rs:main0
3772910993228,2cyclictest0-21swapper/122:54:411
3772909993219,8cyclictest0-21swapper/020:26:060
377290999321,4cyclictest0-21swapper/000:03:110
377291199314,24cyclictest3879453-21latency_hist22:31:052
3772911993122,4cyclictest3826820-21apt-get21:31:102
377290999313,7cyclictest171rcu_preempt19:16:040
377290999311,2cyclictest0-21swapper/021:48:090
3772911993018,7cyclictest3782805-21sendmail-msp19:54:152
3772911993016,11cyclictest3981882-21sendmail-msp00:30:212
3772911993010,7cyclictest36-21ksoftirqd/221:43:092
377291099309,5cyclictest0-21swapper/100:31:101
377291099308,12cyclictest1005-21mta-sts-daemon20:30:351
377290999300,6cyclictest3852916-21cat22:01:050
377291099290,4cyclictest11050irq/107-eth%d22:01:081
377291099289,5cyclictest1007-21mta-sts-daemon23:49:431
377291099283,11cyclictest3820296-21ntpq21:23:091
377291099282,16cyclictest3857438-21sendmail-msp22:06:111
377290999287,16cyclictest111788-21systemd-journal00:12:310
377290999284,18cyclictest111788-21systemd-journal00:28:140
377290999283,2cyclictest0-21swapper/000:43:010
3772909992823,2cyclictest0-21swapper/023:51:050
377290999282,20cyclictest0-21swapper/021:18:080
377290999280,19cyclictest0-21swapper/021:53:190
377291199277,8cyclictest1007-21mta-sts-daemon23:58:082
3772910992712,12cyclictest762-21in:imuxsock20:32:451
377290999277,7cyclictest11050irq/107-eth%d21:36:440
377290999274,21cyclictest0-21swapper/023:49:110
377290999273,20cyclictest3913122-21ntpq23:08:100
3772909992724,2cyclictest3857698-21ssh22:06:110
377290999271,25cyclictest0-21swapper/023:58:070
3772910992622,3cyclictest3985594-21awk00:31:051
377291099261,12cyclictest3830632-21ssh21:34:331
377290999262,23cyclictest14-21ksoftirqd/022:36:050
377290999262,11cyclictest171rcu_preempt00:38:090
3772911992519,4cyclictest3840030-21sendmail-msp21:46:072
377291099254,12cyclictest3951324-21ssh23:51:051
377290999254,16cyclictest3796335-21sendmail-msp20:33:100
3772909992524,1cyclictest0-21swapper/023:43:040
377290999252,12cyclictest3819397-21ssh21:23:020
377291199242,16cyclictest111788-21systemd-journal00:13:112
3772911992410,9cyclictest0-21swapper/223:52:212
377291099243,20cyclictest3887280-21apt-get22:39:201
377290999243,13cyclictest3808391-21unixbench-2d20:58:140
377290999242,21cyclictest3877014-21ntpq22:28:110
3772909992420,1cyclictest0-21swapper/019:58:130
3772909992411,7cyclictest3824665-21ntpq21:28:090
377291199238,11cyclictest3799356-21ntpq20:33:102
377291199233,16cyclictest3910624-21sendmail-msp23:11:112
3772911992317,5cyclictest11050irq/107-eth%d23:43:102
3772911992310,10cyclictest3881586-21ntp_states22:33:112
377291099236,3cyclictest11050irq/107-eth%d22:58:071
377291099236,3cyclictest11050irq/107-eth%d22:58:071
377291099233,12cyclictest111788-21systemd-journal00:15:121
3772910992316,5cyclictest3808390-21unixbench-2d20:58:141
377290999237,9cyclictest3890260-21ssh22:43:070
377290999231,20cyclictest3892544-21sendmail-msp22:53:000
377290999231,19cyclictest3894867-21ntpq22:48:120
377291199226,12cyclictest619-21mta-sts-daemon00:03:112
3772911992213,7cyclictest1005-21mta-sts-daemon20:26:062
377291099224,13cyclictest3874858-21sendmail-msp22:26:111
377291099224,13cyclictest1005-21mta-sts-daemon20:36:551
377291099223,15cyclictest3857438-21sendmail-msp22:14:141
3772910992217,4cyclictest3929328-21ssh23:26:131
377290999225,15cyclictest3806254-21/usr/sbin/munin20:53:090
3772909992218,3cyclictest0-21swapper/019:43:060
377290999221,15cyclictest3979596-21missed_timers00:23:070
377290999221,11cyclictest0-21swapper/022:40:080
377291199216,10cyclictest111788-21systemd-journal23:07:542
377291199212,14cyclictest111788-21systemd-journal23:47:262
3772911992117,2cyclictest3775290-21run-parts19:23:042
377290999214,12cyclictest3784180-21/usr/sbin/munin19:48:110
377290999212,6cyclictest171rcu_preempt20:38:070
377290999212,16cyclictest3939998-21ntpq23:38:060
3772909992116,4cyclictest3910786-21latency_hist23:06:050
377290999210,17cyclictest0-21swapper/023:28:130
3772911992016,3cyclictest0-21swapper/200:35:152
3772911992015,5cyclictest36-21ksoftirqd/221:56:142
3772911992013,5cyclictest0-21swapper/219:48:112
377291099208,9cyclictest762-21in:imuxsock23:31:581
377291099203,14cyclictest3975625-21/usr/sbin/munin00:18:091
112699200,1rtkit-daemon0-21swapper/022:17:070
377291199198,4cyclictest3832701-21/usr/sbin/munin21:38:012
377291199196,11cyclictest111788-21systemd-journal20:48:112
377291199193,11cyclictest3929048-21sendmail-msp23:34:432
3772911991914,2cyclictest3920433-21ssh23:16:132
3772911991913,4cyclictest3887278-21ssh22:39:202
3772911991913,4cyclictest3801003-21ntpq20:38:072
377291099196,9cyclictest3944450-21ssh23:43:101
3772910991915,3cyclictest3960283-21latency_hist00:01:051
377290999199,8cyclictest3789739-21sendmail-msp20:08:170
377290999192,11cyclictest3975985-21ssh00:18:140
3772909991914,4cyclictest3839989-21cron21:46:050
377290999191,17cyclictest0-21swapper/020:13:100
377291299183,8cyclictest0-21swapper/321:33:103
377291299183,8cyclictest0-21swapper/300:06:293
377291199187,9cyclictest3897805-21sh22:51:562
377291199187,10cyclictest3786983-21grep19:58:042
377291199185,10cyclictest3892802-21ssh22:46:102
377291199182,11cyclictest3811325-21gcc21:08:132
3772911991815,2cyclictest351ktimers/222:03:092
377291099185,7cyclictest3837747-21/usr/sbin/munin21:43:071
377291099183,9cyclictest3826984-21latency_hist21:31:061
377291099183,12cyclictest3811323-21ntpq21:08:131
3772910991814,2cyclictest764-21rs:main1
3772910991811,5cyclictest1006-21mta-sts-daemon20:03:121
377290999180,5cyclictest3872256-21gltestperf22:23:040
377291299173,7cyclictest0-21swapper/323:54:303
377291199177,7cyclictest3795252-21fschecks_time20:23:052
377291199174,3cyclictest36-21ksoftirqd/221:23:092
3772911991713,3cyclictest3826809-21cron21:31:052
3772911991710,4cyclictest0-21swapper/222:43:072
377291199170,5cyclictest3780516-21ntpq19:38:082
377291099177,5cyclictest3874858-21sendmail-msp22:26:041
377291099177,4cyclictest3920444-21apt-get23:16:141
377291099175,10cyclictest1007-21mta-sts-daemon19:28:111
377291099174,10cyclictest271ktimers/122:16:081
377291099172,13cyclictest3927284-21grep23:23:141
3772910991715,1cyclictest0-21swapper/119:23:031
377290999173,2cyclictest3785367-21grep19:53:050
377290999171,14cyclictest3811753-21grep21:08:180
377291299164,7cyclictest0-21swapper/321:23:093
377291299163,6cyclictest0-21swapper/300:33:153
377291199165,7cyclictest3990297-21ssh00:38:022
377291199164,8cyclictest3907867-21cut23:03:062
377291199164,10cyclictest3805498-21aten_r2power_cu20:53:012
377291199163,8cyclictest3815403-21/usr/sbin/munin21:18:052
377291199163,3cyclictest11050irq/107-eth%d23:23:102
377291199162,12cyclictest3870983-21ssh22:21:312
377291199162,12cyclictest0-21swapper/222:16:082
377291199161,3cyclictest0-21swapper/200:16:342
3772911991612,3cyclictest3792526-21/usr/sbin/munin20:13:122
3772911991612,3cyclictest3782800-21sendmail19:46:042
3772911991610,4cyclictest3809134-21cut21:03:052
377291099167,7cyclictest3917094-21/usr/sbin/munin23:13:051
377291099163,9cyclictest764-21rs:main1
377291099162,4cyclictest11050irq/107-eth%d21:54:381
377291099162,10cyclictest3815368-21if_end021:18:051
3772909991613,3cyclictest3794025-21ntp_kernel_err20:18:070
112699160,3rtkit-daemon1125-21rtkit-daemon21:58:111
377291299153,8cyclictest0-21swapper/323:50:093
377291299153,7cyclictest0-21swapper/322:06:113
377291299153,7cyclictest0-21swapper/300:31:053
377291299153,6cyclictest0-21swapper/321:58:023
377291299152,8cyclictest0-21swapper/322:28:113
377291299151,8cyclictest0-21swapper/323:07:543
377291199159,5cyclictest3994148-21sort00:43:042
377291199156,7cyclictest3939470-21grep23:38:032
377291199155,8cyclictest3845952-21grep21:53:082
377291199155,5cyclictest3807473-21tune2fs20:58:052
377291199154,8cyclictest0-21swapper/222:08:032
377291199153,9cyclictest3788995-21/usr/sbin/munin20:03:082
377291199153,10cyclictest3903511-21/usr/sbin/munin22:58:062
377291199153,10cyclictest3903511-21/usr/sbin/munin22:58:062
377291199151,11cyclictest3862981-21munin-node22:13:002
377291199150,2cyclictest3778980-21ntpq19:33:122
377291199150,1cyclictest69-21kswapd020:41:072
377291099159,3cyclictest0-21swapper/120:26:051
377291099153,9cyclictest3890675-21/usr/sbin/munin22:43:131
377291099153,6cyclictest3783093-21apt-key19:46:051
377291099152,9cyclictest3832991-21ssh21:38:031
377291099152,7cyclictest3841635-21fschecks_count21:48:041
377291099152,11cyclictest3939436-21/usr/sbin/munin23:38:031
377291099152,11cyclictest3778449-21cut19:33:071
377291099152,10cyclictest633-21gpm19:46:421
377291099151,8cyclictest3909030-21/usr/sbin/munin23:03:171
377291099151,11cyclictest3779996-21/usr/sbin/munin19:38:051
377290999152,10cyclictest3803593-21munin-run20:46:050
377291299145,5cyclictest0-21swapper/320:48:113
377291299144,6cyclictest0-21swapper/321:53:103
377291299143,6cyclictest0-21swapper/323:16:133
377291299143,6cyclictest0-21swapper/300:17:213
377291299142,6cyclictest0-21swapper/320:33:103
377291299141,6cyclictest0-21swapper/322:58:063
377291299141,6cyclictest0-21swapper/322:58:063
377291199146,3cyclictest3979255-21fschecks_time00:23:052
377291199142,8cyclictest3794018-21/usr/sbin/munin20:18:072
377291099144,8cyclictest3803623-21sendmail-msp20:46:301
377291099142,8cyclictest1-21systemd19:16:041
377291099142,7cyclictest3991250-21ssh00:38:121
377291099141,11cyclictest3978848-21/usr/sbin/munin00:23:021
3772910991411,1cyclictest0-21swapper/119:53:061
377291099140,12cyclictest3787498-21ntpq19:58:081
377290999143,7cyclictest3795261-21fschecks_time20:23:050
377290999142,9cyclictest3781320-21latency_hist19:41:050
377290999141,5cyclictest3789750-21munin-run20:06:050
377291299134,5cyclictest0-21swapper/319:48:113
377291299134,4cyclictest0-21swapper/322:16:083
377291299134,4cyclictest0-21swapper/322:01:083
377291299133,5cyclictest0-21swapper/323:26:083
377291299133,5cyclictest0-21swapper/322:13:103
377291299133,4cyclictest0-21swapper/321:38:083
377291299132,6cyclictest0-21swapper/322:53:113
377291299132,6cyclictest0-21swapper/321:46:063
377291299132,5cyclictest0-21swapper/322:43:103
377291299132,5cyclictest0-21swapper/320:58:103
377291199139,3cyclictest3772971-21ls19:16:052
377291199136,6cyclictest3776952-21/usr/sbin/munin19:28:062
377291199136,6cyclictest341rcuc/220:08:162
377291199131,9cyclictest3812357-21grep21:13:062
377291099134,7cyclictest3962753-21/usr/sbin/munin00:03:161
377291099134,7cyclictest3773514-21cut19:18:061
377291099133,6cyclictest3813520-21grep21:16:051
377290999133,8cyclictest3773560-21/usr/sbin/munin19:18:060
377290999132,5cyclictest3776218-21sendmail-msp19:28:090
377290999131,9cyclictest3812407-21cut21:13:070
377290999131,9cyclictest3775755-21/usr/sbin/munin19:23:090
377290999130,1cyclictest3903851-21ntpq22:58:090
377290999130,1cyclictest3903851-21ntpq22:58:090
377291299127,3cyclictest0-21swapper/323:11:103
377291299125,3cyclictest0-21swapper/320:13:123
377291299124,5cyclictest0-21swapper/320:28:093
377291299123,5cyclictest0-21swapper/319:18:113
377291299122,7cyclictest0-21swapper/322:46:083
377291299122,6cyclictest0-21swapper/321:26:233
377291299122,5cyclictest0-21swapper/323:37:223
377291299122,5cyclictest0-21swapper/319:53:023
377291299121,6cyclictest0-21swapper/323:32:583
377291299121,6cyclictest0-21swapper/300:03:113
377291299121,5cyclictest0-21swapper/323:43:063
377291299120,7cyclictest0-21swapper/300:38:023
377291199123,8cyclictest3773125-21/usr/sbin/munin19:18:032
377291099126,4cyclictest3809710-21proc_pri21:03:111
377291099124,5cyclictest3790333-21cut20:08:071
377291099122,8cyclictest3793609-21/usr/sbin/munin20:18:041
377291099122,6cyclictest3994138-21fschecks_time00:43:041
377290999123,7cyclictest3810382-21grep21:06:050
377290999120,8cyclictest3778078-21/usr/sbin/munin19:33:090
377291299113,6cyclictest0-21swapper/300:12:313
377291299113,5cyclictest0-21swapper/322:36:523
377291299113,4cyclictest0-21swapper/321:13:033
377291299113,4cyclictest0-21swapper/321:08:123
377291299113,4cyclictest0-21swapper/320:41:073
377291299113,4cyclictest0-21swapper/300:23:093
377291299112,6cyclictest0-21swapper/321:43:133
377291299112,5cyclictest0-21swapper/320:53:053
377291299112,5cyclictest0-21swapper/319:58:083
377291299112,5cyclictest0-21swapper/300:43:013
377291299112,4cyclictest0-21swapper/323:58:013
377291299112,4cyclictest0-21swapper/323:03:023
377291299111,4cyclictest0-21swapper/321:03:043
377291299110,6cyclictest0-21swapper/320:23:053
377291099113,7cyclictest870701-1kworker/1:2H+kblockd20:11:351
377291299104,3cyclictest0-21swapper/322:23:083
377291299104,3cyclictest0-21swapper/319:38:103
377291299103,4cyclictest0-21swapper/319:26:183
377291299103,3cyclictest0-21swapper/319:35:103
377291299102,4cyclictest0-21swapper/323:23:013
377291299101,5cyclictest0-21swapper/320:03:083
37729129995,3cyclictest0-21swapper/320:38:403
37729129992,5cyclictest0-21swapper/320:08:073
37729129992,4cyclictest0-21swapper/320:18:023
37729129991,6cyclictest0-21swapper/322:33:093
37729129991,4cyclictest0-21swapper/321:17:283
37729129983,3cyclictest0-21swapper/319:23:033
37729129980,5cyclictest0-21swapper/319:43:043
37729129961,4cyclictest0-21swapper/319:16:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional