You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 16:10
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Mon Sep 15, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8618562350,1chrt861857-21ssh11:14:001
8535222330,1sleep00-21swapper/011:08:200
6281812280,1chrt628184-21cut08:40:191
509935992623,1cyclictest0-21swapper/310:25:003
509924992620,4cyclictest789823-21systemd-run10:30:000
509935992421,1cyclictest0-21swapper/308:05:203
50992899241,15cyclictest838160-21munin-plugin-st11:00:001
509924992418,4cyclictest666721-21unixbench-2d09:10:270
509935992320,1cyclictest0-21swapper/312:40:003
50993599230,1cyclictest838155-21turbostat11:00:003
509928992317,4cyclictest954927-21grep12:20:251
509928992314,8cyclictest0-21swapper/111:55:241
509928992312,10cyclictest969272-21kernelversion12:35:171
509924992319,2cyclictest0-21swapper/012:40:000
50993599227,4cyclictest693407-21taskset09:30:123
509935992211,8cyclictest0-21swapper/310:30:013
509935992210,1cyclictest0-21swapper/311:00:273
509932992220,1cyclictest668858-21munin-run09:15:012
509932992219,2cyclictest789831-21systemd-run10:30:012
509928992215,6cyclictest635717-21cron08:50:011
509928992213,8cyclictest548062-21ntpq07:35:221
509928992211,9cyclictest914000-21sed11:45:181
509928992211,3cyclictest1-21systemd09:06:231
509928992211,2cyclictest813980-21/usr/sbin/munin10:45:011
50993599219,1cyclictest0-21swapper/309:35:173
509935992118,1cyclictest0-21swapper/311:45:003
509935992112,7cyclictest0-21swapper/312:06:173
509935992111,1cyclictest0-21swapper/311:39:333
509935992111,1cyclictest0-21swapper/309:30:003
509935992110,1cyclictest0-21swapper/311:45:243
509935992110,1cyclictest0-21swapper/309:40:013
509935992110,1cyclictest0-21swapper/309:00:193
509935992110,1cyclictest0-21swapper/308:55:253
509935992110,1cyclictest0-21swapper/307:45:013
509928992113,6cyclictest0-21swapper/108:25:171
509928992112,8cyclictest723183-21timerandwakeup09:45:241
509928992111,9cyclictest790628-21irqcore10:30:161
509924992118,2cyclictest955424-21latency_hist12:25:010
509924992114,6cyclictest750-21sshd09:45:570
509924992112,8cyclictest0-21swapper/010:00:010
509935992017,1cyclictest0-21swapper/311:54:593
509935992017,1cyclictest0-21swapper/311:10:153
509935992017,1cyclictest0-21swapper/310:45:173
509935992011,7cyclictest0-21swapper/310:15:243
509935992010,1cyclictest0-21swapper/310:07:553
509932992016,2cyclictest924689-21irqcore11:55:172
50992899209,2cyclictest746980-21sshd10:00:231
50992899209,2cyclictest0-21swapper/112:10:171
50992899208,4cyclictest3818356-21systemd-journal08:35:001
50992899206,5cyclictest749956-21seq10:05:161
50992899206,4cyclictest579077-21cut08:00:201
509928992012,7cyclictest880736-21kthreadcore11:25:181
509928992012,7cyclictest706032-21kthreadcore09:35:201
509928992011,8cyclictest711600-21kthreadcore09:40:191
509928992011,7cyclictest0-21swapper/107:45:001
509928992010,5cyclictest819528-21ntpq10:45:271
509924992016,3cyclictest887427-21irqcore11:30:160
50993599199,1cyclictest0-21swapper/311:29:593
50993599199,1cyclictest0-21swapper/307:15:193
509935991911,1cyclictest0-21swapper/310:35:003
509935991910,1cyclictest0-21swapper/309:20:493
509935991910,1cyclictest0-21swapper/308:15:213
509932991916,2cyclictest813982-21cut10:45:012
509932991916,2cyclictest598988-21latency_hist08:20:012
509932991916,2cyclictest0-21swapper/210:55:012
509932991916,2cyclictest0-21swapper/207:25:012
509932991915,3cyclictest0-21swapper/210:25:002
509932991915,2cyclictest0-21swapper/209:10:002
50992899199,2cyclictest0-21swapper/112:15:221
50992899197,4cyclictest846276-21pluginstate11:05:011
50992899195,5cyclictest877249-21sshd11:22:231
509928991916,2cyclictest0-21swapper/108:40:011
509928991911,3cyclictest798715-21irqcore10:35:151
509928991911,2cyclictest0-21swapper/110:00:011
50992899190,2cyclictest0-21swapper/111:15:171
50992899190,17cyclictest524340-21idleruntime-cro07:20:001
509924991916,2cyclictest761-21cups-browsed11:15:000
509924991915,3cyclictest635719-21idleruntime-cro08:50:000
509924991915,2cyclictest937256-21cpuspeed_turbos12:10:140
50993599189,1cyclictest0-21swapper/307:10:173
50993599188,1cyclictest0-21swapper/310:05:013
50993599188,1cyclictest0-21swapper/309:15:013
509935991816,1cyclictest0-21swapper/310:40:013
509935991815,1cyclictest0-21swapper/308:45:203
509935991814,2cyclictest878462-21turbostat11:25:013
509935991812,4cyclictest0-21swapper/309:05:173
509935991811,5cyclictest0-21swapper/308:40:183
509935991811,5cyclictest0-21swapper/308:10:133
509932991814,3cyclictest954742-21timerandwakeup12:20:242
509932991814,2cyclictest0-21swapper/211:50:002
509932991810,6cyclictest943059-21grep12:14:592
50992899189,2cyclictest601198-21kthreadcore08:20:191
50992899189,2cyclictest601198-21kthreadcore08:20:181
50992899189,2cyclictest0-21swapper/110:10:211
50992899188,3cyclictest773744-21tr10:20:011
50992899188,2cyclictest0-21swapper/111:35:151
50992899186,4cyclictest663440-21cut09:10:191
50992899181,5cyclictest574136-21cat08:00:011
509928991815,2cyclictest789790-21cron10:30:011
509928991810,2cyclictest892488-21users11:30:251
509924991815,2cyclictest514-21systemd-logind11:44:590
509924991815,2cyclictest0-21swapper/009:40:140
509924991814,2cyclictest917856-21latency_hist11:50:010
509924991814,2cyclictest0-21swapper/007:15:020
509924991812,5cyclictest0-21swapper/009:25:120
509935991714,1cyclictest0-21swapper/312:20:253
509935991714,1cyclictest0-21swapper/312:19:243
509935991712,4cyclictest0-21swapper/307:50:013
509935991712,3cyclictest0-21swapper/312:30:133
509932991714,2cyclictest930744-21cron12:04:592
509932991714,2cyclictest733504-21cron09:55:002
509932991714,2cyclictest564814-21kthreadcore07:50:192
509932991713,3cyclictest591485-21latency08:10:182
509932991713,2cyclictest574645-21tr08:00:162
50992899179,2cyclictest585971-21sshd08:07:231
50992899179,1cyclictest0-21swapper/112:07:221
50992899178,2cyclictest908013-21needreboot11:40:201
50992899177,3cyclictest530482-21grep07:25:021
50992899176,2cyclictest19850irq/126-ahci[0000:00:17.0]07:34:591
50992899175,3cyclictest642714-21cut08:55:171
50992899171,8cyclictest967646-21latency_hist12:35:011
509928991712,3cyclictest593006-21cpuspeed_turbos08:15:151
509928991712,3cyclictest592135-21grep08:10:241
509924991714,2cyclictest791100-21irqrtprio10:30:170
509924991714,2cyclictest660778-21latency_hist09:10:000
509924991714,2cyclictest547-21gdbus08:50:250
509924991713,3cyclictest873216-21kthreadcore11:20:170
509924991713,3cyclictest3818356-21systemd-journal09:54:590
509924991713,2cyclictest946258-21kthreadcore12:15:170
509924991713,2cyclictest924307-21cpuspeed_turbos11:55:130
509924991713,2cyclictest524674-21cpuspeed_turbos07:20:120
509935991613,1cyclictest0-21swapper/311:58:243
509935991613,1cyclictest0-21swapper/311:20:003
509935991613,1cyclictest0-21swapper/307:29:223
50993299169,5cyclictest501-21dbus-daemon10:15:002
50993299163,2cyclictest471rcuog/211:00:002
509932991613,2cyclictest854996-21gltestperf11:10:172
509932991613,2cyclictest623546-21latency_hist08:40:012
509932991612,2cyclictest0-21swapper/209:25:152
50992899166,3cyclictest501-21dbus-daemon07:50:011
509928991613,2cyclictest0-21swapper/109:25:151
509928991611,3cyclictest693702-21grep09:30:141
509928991611,3cyclictest0-21swapper/112:05:001
50992499166,8cyclictest0-21swapper/007:15:000
509924991613,2cyclictest757813-21latency_hist10:10:010
509924991612,3cyclictest827353-21needreboot10:50:170
509924991612,3cyclictest588098-21kthreadcore08:10:170
509924991612,2cyclictest0-21swapper/008:20:230
509924991612,2cyclictest0-21swapper/008:20:220
509935991513,1cyclictest0-21swapper/309:55:153
509935991512,1cyclictest0-21swapper/312:25:223
509935991512,1cyclictest0-21swapper/311:32:253
509935991512,1cyclictest0-21swapper/310:41:343
509935991512,1cyclictest0-21swapper/310:13:083
509935991512,1cyclictest0-21swapper/309:45:163
509935991512,1cyclictest0-21swapper/308:39:443
509935991512,1cyclictest0-21swapper/308:00:183
509935991512,1cyclictest0-21swapper/307:20:233
50993299158,5cyclictest653722-21sshd09:00:232
509932991512,2cyclictest878481-21latency_hist11:25:012
509932991512,2cyclictest660570-21diskmemload11:00:212
509932991512,2cyclictest637373-21kthreadcore08:50:182
509932991512,2cyclictest0-21swapper/212:30:202
509932991511,3cyclictest795652-21unixbench-2d10:30:232
509932991511,3cyclictest765958-21acpi10:15:132
509932991511,3cyclictest537167-21gltestperf07:30:172
509932991511,3cyclictest1-21systemd09:40:242
509932991511,2cyclictest611848-21grep08:30:152
50992899156,2cyclictest0-21swapper/109:25:011
509928991511,3cyclictest923067-21ntp_kernel_pll_11:50:201
50992499159,4cyclictest501-21dbus-daemon11:30:000
50992499158,5cyclictest3818356-21systemd-journal10:05:000
509924991512,2cyclictest771377-21timerandwakeup10:15:240
509924991512,2cyclictest761706-21kworker/0:0+kacpid10:35:330
509924991512,2cyclictest0-21swapper/010:20:140
509924991511,3cyclictest674272-21ntp_kernel_pll_09:15:200
509924991511,3cyclictest647105-21ntp_kernel_pll_08:55:210
509935991411,1cyclictest0-21swapper/312:10:203
509935991411,1cyclictest0-21swapper/312:04:053
509935991411,1cyclictest0-21swapper/311:07:403
509935991411,1cyclictest0-21swapper/310:52:143
509935991411,1cyclictest0-21swapper/309:15:193
509935991411,1cyclictest0-21swapper/308:31:103
509935991411,1cyclictest0-21swapper/307:58:103
50993299146,6cyclictest847023-21if_eno211:05:142
50993299143,3cyclictest676938-21systemctl09:20:002
509932991411,2cyclictest894853-21/usr/sbin/munin11:35:182
509932991411,2cyclictest867834-21sendmail_mailqu11:15:212
509932991411,2cyclictest660570-21diskmemload09:30:152
509932991411,2cyclictest524-21rs:main2
509932991411,2cyclictest0-21swapper/209:20:262
509932991411,2cyclictest0-21swapper/208:10:012
509932991410,3cyclictest972990-21sendmail_mailqu12:35:232
509932991410,3cyclictest888316-21kthreadcore11:30:172
509932991410,3cyclictest723342-21unixbench_multi09:45:242
509932991410,3cyclictest3818356-21systemd-journal07:17:002
509932991410,2cyclictest0-21swapper/211:40:142
50992899148,4cyclictest648558-21apt-get09:00:131
50992899147,3cyclictest780970-21ssh10:23:111
509928991412,1cyclictest0-21swapper/109:15:141
509924991412,1cyclictest917395-21kworker/0:0+kacpid12:01:410
509924991412,1cyclictest800457-21kworker/0:1+kacpid11:37:230
509924991412,1cyclictest696571-21kworker/0:1+kacpid10:11:340
509924991412,1cyclictest497409-21kworker/0:1+kacpid09:00:220
509924991411,2cyclictest581160-21irqcore08:05:150
509924991411,2cyclictest552807-21kthreadcore07:40:170
509924991411,2cyclictest508-21polkitd09:30:220
509924991411,2cyclictest497409-21kworker/0:1+kacpid07:38:020
509924991410,3cyclictest616438-21proc_pri08:30:210
509924991410,2cyclictest702799-21kthreadcore09:35:190
509935991311,1cyclictest0-21swapper/309:50:043
509935991311,1cyclictest0-21swapper/308:50:153
509935991311,1cyclictest0-21swapper/307:50:163
509935991310,1cyclictest0-21swapper/308:21:593
509935991310,1cyclictest0-21swapper/308:21:583
50993299139,3cyclictest936465-21unixbench_singl12:05:202
50993299137,4cyclictest634869-21ntpq08:45:192
509932991311,1cyclictest0-21swapper/211:25:162
509932991310,2cyclictest815628-21irqrtprio10:45:192
509932991310,2cyclictest801234-21kthreadcore10:35:192
509932991310,2cyclictest625226-21kthreadcore08:40:172
509932991310,2cyclictest610258-21sshd08:25:232
509932991310,2cyclictest500-21avahi-daemon07:55:252
50993299130,4cyclictest6461261kthreadcore08:55:202
50992899136,3cyclictest331ktimers/111:05:111
509928991310,2cyclictest825356-21sed10:50:161
50992499139,3cyclictest171ktimers/008:25:130
50992499138,4cyclictest3818356-21systemd-journal08:20:000
50992499131,3cyclictest814533-21fschecks_count10:45:160
509924991310,2cyclictest867203-21kthreadcore11:15:190
509924991310,2cyclictest568363-21cut07:55:170
509924991310,2cyclictest537811-21irqrtprio07:30:170
509924991310,2cyclictest501-21dbus-daemon08:35:230
509924991310,2cyclictest497409-21kworker/0:1+kacpid08:42:180
509924991310,2cyclictest1976426-21kworker/u8:0-nfsiod09:20:260
509924991310,2cyclictest0-21swapper/010:40:170
509924991310,1cyclictest0-21swapper/008:00:240
50992499130,1cyclictest0-21swapper/007:25:260
50993599129,1cyclictest0-21swapper/307:35:243
50993599129,1cyclictest0-21swapper/307:30:223
509935991210,1cyclictest0-21swapper/308:25:223
509935991210,1cyclictest0-21swapper/307:10:013
50993299129,2cyclictest754768-21kthreadcore10:05:222
50993299129,2cyclictest660570-21diskmemload09:35:192
50993299129,2cyclictest547-21gdbus11:50:002
50993299129,2cyclictest543237-21fschecks_count07:35:162
50993299129,2cyclictest510256-21cpuspeed_turbos07:10:152
50993299129,2cyclictest0-21swapper/208:20:162
50993299129,2cyclictest0-21swapper/208:20:162
50993299128,2cyclictest0-21swapper/207:45:132
50993299126,4cyclictest735998-21cut09:55:192
509932991210,1cyclictest945573-21kthreadcore12:15:172
50992899129,2cyclictest535738-21ntp_kernel_pll_07:25:241
50992899129,2cyclictest516056-21sh07:15:011
50992899128,2cyclictest0-21swapper/107:10:011
50992899121,3cyclictest641850-21latency_hist08:55:001
509928991210,1cyclictest660570-21diskmemload09:50:141
50992499129,2cyclictest259603-21kworker/u8:1-rpciod10:55:130
50992499127,4cyclictest956569-21sed12:25:180
50993299117,3cyclictest742240-21gltestperf10:00:162
50993299115,4cyclictest956299-21sed12:25:182
50992899118,2cyclictest955913-21gltestperf12:25:171
50992899110,1cyclictest0-21swapper/107:51:001
50992499119,1cyclictest800457-21kworker/0:1+kacpid11:00:160
50992499119,1cyclictest0-21swapper/012:35:010
50992499118,2cyclictest918360-21gltestperf11:50:140
50992499118,2cyclictest0-21swapper/012:10:000
50992499114,6cyclictest566785-21grep07:50:210
50993299104,4cyclictest553052-21tr07:40:172
50992499103,5cyclictest509929-21wc07:10:000
5099329995,3cyclictest509938-21sort07:10:002
5099249994,3cyclictest555675-21gdbus07:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional