You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-10 - 09:21
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Thu Jul 10, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15520822530,1sleep01552081-21sshd21:15:250
16766032400,1sleep2191rcu_preempt22:35:162
14896982350,1chrt0-21swapper/020:30:160
15002892310,1sleep21500290-21readlink20:35:232
1387075992926,2cyclictest0-21swapper/221:35:012
1387065992822,5cyclictest1740695-21grep23:15:000
1387065992621,4cyclictest1-21systemd19:45:010
1387065992519,5cyclictest1803740-21kworker/u8:1-rpciod00:00:010
1387065992519,5cyclictest1420587-21kworker/u8:1-rpciod22:39:590
1387075992420,2cyclictest1414226-21systemd-run19:30:012
1387075992419,3cyclictest1543541-21strings21:10:232
138707099241,2cyclictest0-21swapper/100:05:011
1387065992419,4cyclictest0-21swapper/023:50:010
1387065992418,5cyclictest0-21swapper/021:10:000
1387077992320,1cyclictest0-21swapper/321:55:013
1387077992320,1cyclictest0-21swapper/319:45:013
1387070992320,2cyclictest501-21dbus-daemon00:15:001
1387070992320,2cyclictest1393537-21latency_hist19:15:011
1387075992219,2cyclictest1586619-21apt21:40:012
1387075992218,3cyclictest761-21cups-browsed23:35:012
1387075992217,3cyclictest0-21swapper/219:14:592
1387065992217,4cyclictest0-21swapper/021:25:000
1387065992216,5cyclictest1854361-21sh00:25:000
1387065992216,5cyclictest1-21systemd20:30:000
1387077992118,1cyclictest0-21swapper/321:40:013
1387075992118,2cyclictest1805638-21cron23:55:012
1387077992017,1cyclictest0-21swapper/323:00:263
1387075992017,2cyclictest1519601-21sh20:55:002
1387075992016,3cyclictest1554268-21turbostat.cron21:20:002
1387075992015,3cyclictest0-21swapper/220:19:592
1387070992017,2cyclictest277-21plymouthd22:35:221
1387070992017,2cyclictest1619000-21sh22:00:011
1387065992015,4cyclictest1842101-21kthreadcore00:15:190
1387065992014,5cyclictest1742230-21irqrtprio23:15:200
1387065992014,5cyclictest1387151-21latency_hist19:10:010
1387065992014,5cyclictest0-21swapper/022:25:000
1387065992013,6cyclictest1393227-21latency_hist19:15:010
1387077991916,1cyclictest0-21swapper/319:15:013
1387075991916,2cyclictest1773757-21gltestperf23:35:152
1387075991916,2cyclictest1749569-21grep23:20:152
1387075991915,3cyclictest508-21polkitd20:05:012
1387075991915,3cyclictest1705860-21grep22:50:252
1387075991915,3cyclictest1532105-21latency_hist21:05:012
1387075991915,3cyclictest1532105-21latency_hist21:05:012
1387075991915,2cyclictest0-21swapper/200:25:002
1387070991915,3cyclictest522-21in:imuxsock20:45:001
1387070991915,3cyclictest1768028-21kthreadcore23:30:181
138706599199,9cyclictest1694915-21kthreadcore22:45:180
1387065991914,4cyclictest0-21swapper/022:20:000
1387065991913,5cyclictest1570601-21latency_hist21:30:010
1387065991913,5cyclictest1408727-21irqcore19:25:180
1387065991913,5cyclictest0-21swapper/021:35:010
1387065991913,5cyclictest0-21swapper/019:50:000
1387065991913,4cyclictest0-21swapper/021:50:010
1387065991912,5cyclictest1771007-21kworker/u8:2-nfsiod23:34:360
1387065991911,3cyclictest1757420-21tune2fs23:25:130
138706599190,4cyclictest1605149-21kthreadcore21:50:180
1387075991815,2cyclictest1643526-21systemd-run22:14:592
1387075991814,3cyclictest1-21systemd19:24:252
1387075991814,3cyclictest0-21swapper/220:50:002
1387075991814,3cyclictest0-21swapper/200:40:002
1387075991814,2cyclictest1736137-21kthreadcore23:10:162
1387070991815,2cyclictest1781851-21gltestperf23:40:161
1387070991815,2cyclictest1635414-21sh22:10:001
1387070991815,2cyclictest0-21swapper/120:54:591
1387070991814,3cyclictest0-21swapper/122:20:001
1387070991812,5cyclictest0-21swapper/123:20:001
138706599186,6cyclictest1-21systemd00:39:590
1387065991813,4cyclictest0-21swapper/023:45:010
1387065991812,5cyclictest1665553-21unixbench-2d22:25:230
1387065991812,5cyclictest1520069-21fschecks_count20:55:150
1387065991812,5cyclictest1470532-21cpuspeed_turbos20:15:150
1387065991812,5cyclictest1445090-21munin-plugin-st19:55:010
1387065991812,5cyclictest0-21swapper/023:20:120
1387065991812,5cyclictest0-21swapper/020:15:000
1387065991811,6cyclictest1613846-21kthreadcore21:55:200
1387065991811,6cyclictest1495440-21tail20:35:150
1387065991811,6cyclictest1393758-21gltestperf19:15:010
1387065991811,5cyclictest0-21swapper/020:05:010
1387065991811,2cyclictest1835056-21kthreadcore00:10:200
1387065991810,3cyclictest1709322-21irqcore22:55:190
1387065991810,2cyclictest0-21swapper/019:20:150
138706599180,5cyclictest16274652grep22:05:010
1387077991714,1cyclictest0-21swapper/321:35:013
1387077991714,1cyclictest0-21swapper/319:55:013
1387077991713,1cyclictest0-21swapper/320:10:013
1387075991714,2cyclictest1838157-21systemd-run00:15:002
1387075991714,2cyclictest1627295-21cron22:05:002
1387075991714,2cyclictest0-21swapper/222:00:012
1387075991713,3cyclictest1746616-21unixbench_multi23:15:282
1387075991713,3cyclictest1395627-21timerwakeupswit19:15:032
1387075991713,2cyclictest1445080-21cron19:55:012
1387070991714,2cyclictest501-21dbus-daemon19:40:191
1387070991714,2cyclictest1855163-21sed00:25:171
1387070991714,2cyclictest1758166-21irqrtprio23:25:151
1387070991714,2cyclictest1525851-21latency_hist21:00:011
1387070991713,3cyclictest1608279-21ntp_kernel_pll_21:50:201
1387070991713,2cyclictest0-21swapper/122:55:011
138706599179,6cyclictest1865783-21kthreadcore00:30:190
138706599179,2cyclictest1538123-21diskmemload21:10:150
138706599179,2cyclictest0-21swapper/000:05:010
138706599176,2cyclictest1643509-21turbostat22:15:000
1387065991712,4cyclictest1380780-21kworker/0:1+kacpid23:54:330
1387065991712,4cyclictest1380780-21kworker/0:1+kacpid19:30:250
1387065991711,5cyclictest1716896-21cpuspeed_turbos23:00:130
1387065991711,5cyclictest1519613-21latency_hist20:55:000
1387065991711,4cyclictest0-21swapper/022:50:160
1387065991711,1cyclictest1673167-21grep22:30:240
1387065991710,6cyclictest1628388-21irqcore22:05:170
1387065991710,2cyclictest1823464-21irqrtprio00:05:170
1387065991710,2cyclictest1725010-21cpuspeed_turbos23:05:140
1387065991710,2cyclictest1684542-21fschecks_time22:40:150
1387065991710,2cyclictest1582656-21kthreadcore21:35:210
1387065991710,2cyclictest1528473-21kthreadcore21:00:200
1387065991710,2cyclictest1528473-21kthreadcore21:00:200
1387065991710,2cyclictest1450804-21mailstats19:55:240
1387065991710,2cyclictest0-21swapper/020:50:000
1387065991710,1cyclictest0-21swapper/023:35:130
138706599170,4cyclictest1854698-21ssh00:25:140
138706599170,11cyclictest231rcuc/020:10:010
1387077991614,1cyclictest0-21swapper/300:35:003
1387077991613,1cyclictest0-21swapper/323:50:013
1387077991613,1cyclictest0-21swapper/322:24:303
1387077991613,1cyclictest0-21swapper/322:05:013
1387077991613,1cyclictest0-21swapper/320:10:463
1387077991612,1cyclictest0-21swapper/321:15:003
138707799160,13cyclictest0-21swapper/322:10:003
138707599169,5cyclictest3561480-21systemd-journal00:01:242
1387075991613,2cyclictest508-21polkitd00:05:222
1387075991613,2cyclictest1685656-21kthreadcore22:40:182
1387075991613,2cyclictest0-21swapper/222:10:002
1387075991613,2cyclictest0-21swapper/220:40:232
1387075991612,3cyclictest1718179-21kernelversion23:00:172
1387075991612,2cyclictest0-21swapper/223:40:222
1387075991612,2cyclictest0-21swapper/222:55:202
138707099169,2cyclictest1643515-21cut22:15:001
1387070991613,2cyclictest508-21polkitd00:30:241
1387070991613,2cyclictest1879026-21grep00:40:001
1387070991613,2cyclictest1830058-21latency_hist00:10:001
1387070991613,2cyclictest1749585-21irqcore23:20:151
1387070991613,2cyclictest1531939-21cron21:05:001
1387070991613,2cyclictest1531939-21cron21:04:591
1387070991613,2cyclictest1500161-21ntp_kernel_pll_20:35:211
1387070991612,3cyclictest1803471-21unixbench-2d23:50:211
1387070991612,3cyclictest1733705-21irqcore23:10:141
1387070991612,3cyclictest1450634-21ntp_kernel_pll_19:55:211
1387070991610,5cyclictest1716416-21cron22:59:591
1387065991611,4cyclictest1591208-21kthreadcore21:40:180
1387065991611,4cyclictest1380780-21kworker/0:1+kacpid19:38:240
1387077991513,1cyclictest0-21swapper/322:10:013
1387077991513,1cyclictest0-21swapper/300:05:193
1387077991512,1cyclictest0-21swapper/323:22:173
1387077991512,1cyclictest0-21swapper/322:00:013
1387077991512,1cyclictest0-21swapper/320:29:593
1387077991511,1cyclictest0-21swapper/322:30:003
1387077991510,2cyclictest1807444-21chrt23:55:173
138707599158,5cyclictest547-21gdbus21:00:002
1387075991513,1cyclictest0-21swapper/200:00:012
1387075991512,2cyclictest1858866-21kthreadcore00:25:192
1387075991512,2cyclictest1598199-21kthreadcore21:45:182
1387075991512,2cyclictest1562525-21latency_hist21:25:012
1387075991512,2cyclictest1387087-21latency_hist19:10:012
1387075991512,2cyclictest0-21swapper/220:00:012
1387075991511,3cyclictest1864248-21kernelversion00:30:182
1387075991511,3cyclictest1792169-21kthreadcore23:45:212
1387075991511,3cyclictest0-21swapper/222:33:572
1387075991511,2cyclictest0-21swapper/220:30:152
1387075991511,2cyclictest0-21swapper/219:35:272
1387075991510,3cyclictest1476716-21cpuspeed_turbos20:20:162
1387075991510,3cyclictest0-21swapper/222:25:122
138707099159,5cyclictest3561480-21systemd-journal21:25:001
1387070991512,2cyclictest3561480-21systemd-journal21:45:011
1387070991512,2cyclictest1721969-21/usr/sbin/munin23:00:211
1387070991512,2cyclictest1563070-21gltestperf21:25:151
1387070991512,2cyclictest0-21swapper/123:10:001
1387070991511,3cyclictest1600220-21ntp_offset21:45:221
1387070991511,3cyclictest1573683-21kthreadcore21:30:211
138707099150,3cyclictest1396513-21cut19:15:171
138706599159,2cyclictest1479581-21tr20:20:210
138706599158,3cyclictest1505741-21kthreadcore20:40:180
1387077991412,1cyclictest0-21swapper/322:30:203
1387077991411,1cyclictest0-21swapper/323:33:153
1387077991411,1cyclictest0-21swapper/323:28:253
1387077991411,1cyclictest0-21swapper/322:55:003
1387077991411,1cyclictest0-21swapper/322:45:183
1387077991411,1cyclictest0-21swapper/322:40:253
1387077991411,1cyclictest0-21swapper/322:15:193
1387077991411,1cyclictest0-21swapper/321:40:133
1387077991411,1cyclictest0-21swapper/320:45:193
1387077991411,1cyclictest0-21swapper/300:30:003
1387077991411,1cyclictest0-21swapper/300:20:253
1387077991411,1cyclictest0-21swapper/300:10:193
138707599149,3cyclictest1725862-21irqrtprio23:05:162
138707599147,5cyclictest3561480-21systemd-journal21:55:012
1387075991411,2cyclictest1759256-21kthreadcore23:25:162
1387075991411,2cyclictest1438915-21sh19:50:012
1387075991411,2cyclictest0-21swapper/221:40:152
1387075991410,3cyclictest1644048-21fschecks_count22:15:152
1387075991410,2cyclictest1635694-21kworker/u8:4-rpciod22:20:182
1387075991410,2cyclictest1420587-21kworker/u8:1-rpciod22:45:182
138707099146,5cyclictest1482799-21apt-get20:25:131
1387070991411,2cyclictest1777350-21sh23:35:181
1387070991411,2cyclictest1454789-21kthreadcore20:00:191
1387070991411,2cyclictest0-21swapper/122:05:011
1387070991410,3cyclictest1406568-21kthreadcore19:20:191
138707099140,13cyclictest1699974-21cron22:49:591
138707799139,1cyclictest0-21swapper/323:40:013
138707799139,1cyclictest0-21swapper/320:55:003
1387077991311,1cyclictest0-21swapper/319:25:183
1387077991310,1cyclictest0-21swapper/323:14:083
1387077991310,1cyclictest0-21swapper/322:59:243
1387077991310,1cyclictest0-21swapper/321:23:133
1387077991310,1cyclictest0-21swapper/320:59:093
1387077991310,1cyclictest0-21swapper/320:40:203
1387077991310,1cyclictest0-21swapper/320:40:003
1387077991310,1cyclictest0-21swapper/319:49:283
1387077991310,1cyclictest0-21swapper/300:03:303
138707599139,3cyclictest1483866-21irqrtprio20:25:172
138707599139,2cyclictest0-21swapper/221:05:242
138707599138,3cyclictest1839898-21sed00:15:172
1387075991310,2cyclictest1459399-21kthreadcore20:05:162
1387075991310,2cyclictest0-21swapper/221:25:152
138707099139,3cyclictest1553090-21sshd21:17:251
138707099139,3cyclictest1467922-21kthreadcore20:10:181
138707099139,2cyclictest0-21swapper/120:25:001
138707099138,3cyclictest3561480-21systemd-journal22:20:251
1387070991311,1cyclictest1538123-21diskmemload00:15:531
1387070991311,1cyclictest0-21swapper/122:40:201
1387070991310,2cyclictest1672429-21kthreadcore22:30:231
1387070991310,2cyclictest1539485-21irqrtprio21:10:191
1387070991310,2cyclictest1538123-21diskmemload00:24:031
1387070991310,2cyclictest1507598-21cpuspeed_turbos20:45:131
1387070991310,2cyclictest1493068-21kthreadcore20:30:181
1387070991310,2cyclictest1445422-21latency_hist19:55:011
1387070991310,2cyclictest0-21swapper/119:35:011
138707099130,4cyclictest1426542-21cron19:40:011
138707099130,12cyclictest0-21swapper/120:15:161
138707799129,1cyclictest0-21swapper/323:40:173
138707799129,1cyclictest0-21swapper/323:05:253
138707799129,1cyclictest0-21swapper/322:35:183
138707799129,1cyclictest0-21swapper/321:15:163
138707799129,1cyclictest0-21swapper/320:20:203
138707799129,1cyclictest0-21swapper/300:15:163
138707799128,1cyclictest0-21swapper/321:25:173
138707799128,1cyclictest0-21swapper/320:18:263
1387077991210,1cyclictest0-21swapper/319:15:433
1387077991210,1cyclictest0-21swapper/300:38:073
138707599129,2cyclictest1428075-21irqrtprio19:40:132
138707599126,4cyclictest1416287-21cat19:30:202
138707099129,2cyclictest1794318-21kthreadcore23:45:221
138707099129,2cyclictest1662678-21kthreadcore22:25:171
138707099129,2cyclictest1463177-21wc20:05:221
138707099129,2cyclictest1438429-21unixbench-2d19:45:251
138707099129,2cyclictest1411866-21kthreadcore19:25:201
138707099123,7cyclictest1582257-21tr21:35:201
1387070991210,1cyclictest0-21swapper/121:10:001
138707099120,11cyclictest0-21swapper/123:56:201
17411312110,1sleep30-21swapper/323:15:173
138707799119,1cyclictest0-21swapper/321:45:173
138707799118,1cyclictest0-21swapper/323:50:113
138707799118,1cyclictest0-21swapper/321:00:163
138707799118,1cyclictest0-21swapper/321:00:163
138707799118,1cyclictest0-21swapper/320:30:183
138707599118,2cyclictest1465596-21kthreadcore20:10:172
138707799108,1cyclictest0-21swapper/321:05:133
138707799108,1cyclictest0-21swapper/319:58:423
138707799107,1cyclictest0-21swapper/319:35:193
138707799107,1cyclictest0-21swapper/319:30:183
13870779997,1cyclictest0-21swapper/319:20:183
13870779986,1cyclictest0-21swapper/320:00:193
1386937280,2sleep30-21swapper/319:09:563
13870709974,2cyclictest19850irq/126-ahci[0000:00:17.0]19:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional