You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-20 - 00:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Sun Oct 19, 2025 12:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
192353421080,1sleep016-21ksoftirqd/010:16:170
16663402970,2sleep0241rcuc/007:15:030
21464602860,2sleep30-21swapper/312:21:243
18570712830,1sleep10-21swapper/109:36:221
1659474992724,2cyclictest1635-21dbus-daemon12:20:010
165947499251,9cyclictest0-21swapper/010:31:120
2564091245,6phc2sys0-21swapper/310:31:123
1659474992320,2cyclictest260-21systemd-journal11:00:010
1659474992320,2cyclictest2158206-21grep12:30:020
1659474992320,2cyclictest1-21systemd12:15:010
1659474992314,5cyclictest1917774-21kthreadcore10:11:200
1659474992314,5cyclictest1826543-21cstates09:21:140
1659480992219,1cyclictest0-21swapper/207:35:002
1659479992219,2cyclictest1659468-21cyclictest09:20:001
1659479992215,3cyclictest1835898-21if_err_enp1s0.809:26:201
2564091211,6phc2sys0-21swapper/308:26:133
1659474992110,6cyclictest0-21swapper/009:26:140
2564091200,2phc2sys0-21swapper/307:11:183
1659480992017,1cyclictest0-21swapper/207:50:012
1659479992017,2cyclictest1692150-21apt-key07:35:011
1659474992017,2cyclictest2088029-21systemd11:50:010
1659474992015,3cyclictest260-21systemd-journal07:24:550
1659474992014,5cyclictest2094097-21sh11:51:260
1659474992014,5cyclictest1659468-21cyclictest10:46:160
1659474992013,2cyclictest1931202-21/usr/sbin/munin10:21:260
1659474992011,5cyclictest2071323-21/usr/sbin/munin11:41:140
1659474992011,5cyclictest1907180-21kthreadcore10:06:230
1659474992010,6cyclictest1659468-21cyclictest11:21:190
165947499200,19cyclictest241rcuc/009:36:200
2564091190,3phc2sys0-21swapper/308:01:113
1659480991916,2cyclictest0-21swapper/210:55:022
1659480991916,1cyclictest0-21swapper/208:00:012
165948099190,8cyclictest0-21swapper/208:45:012
1659479991916,2cyclictest2157996-21gpgconf12:30:001
1659479991912,2cyclictest170550irq/121-eno111:35:021
1659479991912,2cyclictest0-21swapper/111:00:011
1659479991911,3cyclictest0-21swapper/110:15:011
165947499199,6cyclictest0-21swapper/011:01:230
165947499199,5cyclictest0-21swapper/011:16:300
1659474991916,2cyclictest1807851-21munin-plugin-st09:10:010
1659474991916,2cyclictest1700-21systemd-logind08:15:000
2564091180,2phc2sys0-21swapper/312:35:283
2564091180,2phc2sys0-21swapper/312:35:003
2564091180,2phc2sys0-21swapper/312:35:003
2564091180,2phc2sys0-21swapper/311:12:533
2564091180,2phc2sys0-21swapper/310:01:303
2564091180,1phc2sys0-21swapper/310:35:533
165948099188,3cyclictest0-21swapper/210:37:462
165948099181,9cyclictest0-21swapper/211:00:012
1659480991816,1cyclictest0-21swapper/212:05:012
1659480991815,1cyclictest0-21swapper/212:20:022
1659480991815,1cyclictest0-21swapper/210:48:112
1659480991815,1cyclictest0-21swapper/210:15:012
1659480991815,1cyclictest0-21swapper/209:45:002
1659480991815,1cyclictest0-21swapper/209:35:022
1659480991814,2cyclictest0-21swapper/211:05:002
1659480991814,2cyclictest0-21swapper/209:39:562
1659479991811,6cyclictest2123824-21aten2.4-expect12:11:121
165947499183,2cyclictest0-21swapper/009:11:200
1659474991815,2cyclictest1851662-21sh09:35:020
1659474991815,2cyclictest1652-1atopacctd10:26:190
2564091171,2phc2sys0-21swapper/312:11:203
2564091170,2phc2sys0-21swapper/312:07:433
2564091170,2phc2sys0-21swapper/310:18:043
1659480991715,1cyclictest0-21swapper/212:40:002
1659480991714,2cyclictest0-21swapper/210:10:012
1659480991714,1cyclictest0-21swapper/211:50:002
1659480991714,1cyclictest0-21swapper/211:20:002
1659480991714,1cyclictest0-21swapper/210:16:172
1659480991713,2cyclictest0-21swapper/208:55:002
165947999179,3cyclictest0-21swapper/109:35:011
1659479991714,2cyclictest2009210-21turbostat.cron11:05:001
1659479991714,2cyclictest1861629-21tr09:41:141
1659479991714,2cyclictest1833843-21cron09:25:011
1659479991714,2cyclictest1771110-21apt-key08:40:011
1659479991714,2cyclictest1659468-21cyclictest10:31:191
1659479991711,2cyclictest1941304-21kthreadcore10:26:201
1659479991710,5cyclictest1974670-21sshd10:45:401
1659479991710,5cyclictest1-21systemd11:55:011
1659479991710,3cyclictest1878876-21aten_repower_fr09:51:131
1659479991710,3cyclictest1635-21dbus-daemon10:20:001
165947499179,7cyclictest0-21swapper/010:36:130
165947499178,7cyclictest0-21swapper/012:25:020
165947499178,1cyclictest0-21swapper/007:41:140
165947499175,2cyclictest241rcuc/008:26:240
1659474991714,2cyclictest260-21systemd-journal11:35:020
1659474991714,2cyclictest1820406-21kthreadcore09:16:220
1659474991714,2cyclictest1702-21in:imuxsock08:05:520
1659474991714,2cyclictest1635-21dbus-daemon08:50:010
1659474991711,5cyclictest2167668-21/usr/sbin/munin12:36:150
1659474991711,5cyclictest2052639-21latency_hist11:30:020
1659474991711,5cyclictest1880158-21kthreadcore09:51:210
1659474991711,5cyclictest0-21swapper/009:50:010
2564091160,2phc2sys0-21swapper/311:57:003
2564091160,2phc2sys0-21swapper/311:09:313
2564091160,2phc2sys0-21swapper/309:40:033
2564091160,1phc2sys0-21swapper/308:33:303
19957192160,2sleep30-21swapper/310:56:193
165948099165,4cyclictest0-21swapper/212:00:012
165948099161,9cyclictest0-21swapper/208:06:122
165948099161,8cyclictest0-21swapper/209:50:012
1659480991614,1cyclictest0-21swapper/210:25:012
1659480991613,1cyclictest0-21swapper/212:11:252
1659480991613,1cyclictest0-21swapper/209:20:002
1659480991612,2cyclictest0-21swapper/211:53:302
1659480991612,2cyclictest0-21swapper/210:41:182
165947999169,3cyclictest2159810-21hddtemp_smartct12:31:181
165947999169,3cyclictest2159810-21hddtemp_smartct12:31:181
165947999168,7cyclictest170550irq/121-eno111:06:101
165947999168,3cyclictest0-21swapper/111:30:021
1659479991614,1cyclictest2009-21runrttasks09:01:131
1659479991614,1cyclictest2009-21runrttasks09:01:131
1659479991613,2cyclictest2039004-21kthreadcore11:21:191
1659479991613,2cyclictest1804895-21kthreadcore09:06:231
1659479991613,2cyclictest1740755-21grep08:15:001
1659479991613,2cyclictest1635-21dbus-daemon07:20:011
1659479991612,3cyclictest2102301-21latency11:56:211
1659479991612,3cyclictest2061483-21kworker/u16:1+nfsiod11:37:461
1659479991612,3cyclictest2018678-21kworker/u16:3+nfsiod12:36:521
1659479991612,3cyclictest1886637-21kworker/u16:2+nfsiod11:42:041
1659479991612,3cyclictest1825520-21kworker/u16:3+nfsiod10:03:081
1659479991612,3cyclictest1679839-21kworker/u16:0+nfsiod10:37:301
1659479991610,1cyclictest191rcu_preempt08:41:111
165947499169,6cyclictest2062796-21cstates11:36:140
165947499167,5cyclictest0-21swapper/010:55:020
165947499167,4cyclictest1679839-21kworker/u16:0-xprtiod10:03:470
165947499165,7cyclictest0-21swapper/011:11:120
1659474991614,1cyclictest1862171-21if_enp1s0.809:41:170
1659474991613,2cyclictest2097917-21tr11:56:140
1659474991613,2cyclictest1971238-21processes10:41:220
1659474991613,2cyclictest1892673-21mailstats09:56:250
1659474991613,2cyclictest1652-1atopacctd07:51:250
2564091150,3phc2sys0-21swapper/310:22:573
2564091150,3phc2sys0-21swapper/309:18:053
2564091150,2phc2sys0-21swapper/311:49:273
165948099151,4cyclictest20105932sleep211:06:132
1659480991513,1cyclictest0-21swapper/209:25:022
1659480991513,1cyclictest0-21swapper/207:10:022
1659480991512,1cyclictest0-21swapper/212:30:002
1659480991512,1cyclictest0-21swapper/211:26:272
1659480991512,1cyclictest0-21swapper/211:23:372
1659480991512,1cyclictest0-21swapper/208:11:592
1659480991512,1cyclictest0-21swapper/207:30:012
1659480991512,1cyclictest0-21swapper/207:30:012
1659480991511,3cyclictest0-21swapper/211:11:202
1659480991511,2cyclictest0-21swapper/212:35:012
1659480991511,2cyclictest0-21swapper/212:35:012
1659480991511,2cyclictest0-21swapper/211:36:192
1659480991511,2cyclictest0-21swapper/211:32:342
1659480991511,2cyclictest0-21swapper/207:40:022
165948099150,13cyclictest0-21swapper/210:01:112
165947999151,9cyclictest1710558-21gpg-connect-age07:50:011
1659479991512,2cyclictest2144239-21kthreadcore12:21:191
1659479991512,2cyclictest1988714-21kthreadcore10:51:221
1659479991512,2cyclictest1974104-21latency_hist10:45:011
1659479991511,3cyclictest1825520-21kworker/u16:3+nfsiod10:08:351
1659479991511,3cyclictest1684428-21kthreadcore07:26:231
1659479991511,3cyclictest1684428-21kthreadcore07:26:231
165947499156,2cyclictest0-21swapper/012:06:150
165947499154,2cyclictest491rcuog/212:01:180
165947499153,3cyclictest1771022-21awk08:40:010
1659474991513,1cyclictest1659468-21cyclictest09:01:250
1659474991513,1cyclictest1659468-21cyclictest09:01:250
1659474991512,2cyclictest1912808-21kworker/u16:1-events_power_efficient11:10:000
1659474991512,2cyclictest1705242-21uniq07:46:150
165947499150,8cyclictest1669249-21irqrtprio07:16:170
2564091140,2phc2sys0-21swapper/310:49:563
2564091140,2phc2sys0-21swapper/309:49:123
21141702140,3sleep30-21swapper/312:05:013
165948099141,6cyclictest0-21swapper/207:20:012
165948099141,5cyclictest0-21swapper/209:10:012
1659480991412,1cyclictest0-21swapper/210:31:202
1659480991412,1cyclictest0-21swapper/209:30:012
1659480991412,1cyclictest0-21swapper/207:15:022
1659480991411,2cyclictest0-21swapper/210:25:342
1659480991411,2cyclictest0-21swapper/207:41:182
1659480991411,1cyclictest0-21swapper/212:10:012
1659480991411,1cyclictest0-21swapper/211:41:192
1659480991411,1cyclictest0-21swapper/209:56:142
1659480991411,1cyclictest0-21swapper/209:11:182
1659480991410,2cyclictest0-21swapper/212:24:282
1659480991410,2cyclictest0-21swapper/209:53:422
1659480991410,2cyclictest0-21swapper/208:36:402
1659480991410,2cyclictest0-21swapper/208:26:242
165948099140,11cyclictest1716619-21turbostat07:55:012
165947999148,5cyclictest1729749-21sort08:06:161
165947999148,5cyclictest0-21swapper/109:00:011
1659479991412,1cyclictest8351-21rtkit-daemon08:26:161
1659479991411,2cyclictest2133160-21fschecks_time12:16:171
1659479991411,2cyclictest2115694-21uniq12:06:181
1659479991411,2cyclictest2106975-21fschecks_count12:01:141
1659479991411,2cyclictest2021846-21kthreadcore11:11:201
1659479991411,2cyclictest1890414-21kthreadcore09:56:211
1659479991411,2cyclictest1752881-21cut08:25:011
1659479991411,2cyclictest1744357-21kthreadcore08:16:211
1659479991411,2cyclictest1700-21systemd-logind07:55:011
1659479991411,2cyclictest1-21systemd08:35:011
1659479991410,3cyclictest1781644-21kthreadcore08:46:201
1659479991410,2cyclictest0-21swapper/110:25:011
165947499149,4cyclictest1704-21rs:main0
165947499145,5cyclictest0-21swapper/008:43:550
165947499145,5cyclictest0-21swapper/008:17:540
165947499145,5cyclictest0-21swapper/007:28:330
165947499145,5cyclictest0-21swapper/007:28:330
165947499143,6cyclictest0-21swapper/007:35:410
1659474991411,2cyclictest2163148-21kthreadcore12:31:220
1659474991411,2cyclictest2163148-21kthreadcore12:31:220
1659474991411,2cyclictest1794000-21kthreadcore08:56:190
1659474991411,2cyclictest1723884-21if_enp1s008:01:160
1659474991411,2cyclictest0-21swapper/007:35:010
165947499140,7cyclictest1716995-21/usr/sbin/munin07:56:220
2564091134,2phc2sys0-21swapper/308:21:163
2564091130,6phc2sys0-21swapper/307:26:123
2564091130,6phc2sys0-21swapper/307:26:123
2564091130,2phc2sys0-21swapper/310:52:123
2564091130,2phc2sys0-21swapper/308:54:213
2564091130,1phc2sys0-21swapper/308:43:363
165948099139,2cyclictest0-21swapper/207:20:332
1659480991311,1cyclictest0-21swapper/208:56:192
1659480991310,1cyclictest0-21swapper/208:46:182
165948099130,10cyclictest1752855-21turbostat08:25:002
165947999139,3cyclictest1717687-21gltestperf07:56:131
1659479991310,2cyclictest2079992-21aten_repower_en11:46:141
1659479991310,2cyclictest2028046-21cpuspeed_turbos11:16:191
1659479991310,2cyclictest1872921-21kthreadcore09:46:201
1659479991310,2cyclictest1789518-21latency_hist08:55:011
1659479991310,2cyclictest1699362-21gltestperf07:41:181
1659479991310,2cyclictest1660144-21cpuspeed_turbos07:11:171
1659479991310,2cyclictest1631-21gmain08:00:021
165947999130,12cyclictest0-21swapper/109:11:141
1659474991310,2cyclictest1785001-21kernelversion08:51:160
1659474991310,2cyclictest1748260-21irqcore08:21:200
2564091121,2phc2sys0-21swapper/311:01:143
2564091120,2phc2sys0-21swapper/309:10:123
2564091120,1phc2sys0-21swapper/311:42:383
2564091120,1phc2sys0-21swapper/311:29:043
2564091120,1phc2sys0-21swapper/307:44:553
2564091120,1phc2sys0-21swapper/307:21:123
165948099129,1cyclictest0-21swapper/208:15:502
165948099128,2cyclictest0-21swapper/208:01:192
1659480991210,1cyclictest0-21swapper/208:31:182
165948099120,8cyclictest0-21swapper/209:01:252
165948099120,8cyclictest0-21swapper/209:01:252
165947999129,2cyclictest1692886-21tr07:36:161
165947999129,2cyclictest1674279-21tail07:21:151
165947999129,2cyclictest1659534-21munin-plugin-st07:10:011
2564091113,2phc2sys0-21swapper/309:01:193
2564091113,2phc2sys0-21swapper/309:01:193
2564091112,2phc2sys0-21swapper/311:16:333
2564091112,2phc2sys0-21swapper/310:06:233
2564091111,4phc2sys0-21swapper/308:36:133
2564091111,2phc2sys0-21swapper/309:30:163
2564091110,2phc2sys0-21swapper/311:40:023
2564091110,2phc2sys0-21swapper/310:41:183
2564091110,2phc2sys0-21swapper/309:36:133
2564091110,2phc2sys0-21swapper/309:29:193
2564091110,1phc2sys0-21swapper/308:46:403
2564091110,1phc2sys0-21swapper/308:14:143
165947499111,4cyclictest1659313-21latency_hist07:10:010
2564091102,3phc2sys0-21swapper/311:20:053
2564091102,2phc2sys0-21swapper/310:10:483
2564091101,2phc2sys0-21swapper/312:25:063
2564091101,2phc2sys0-21swapper/311:51:123
2564091101,2phc2sys0-21swapper/309:50:523
2564091101,2phc2sys0-21swapper/308:56:143
2564091101,2phc2sys0-21swapper/307:31:143
2564091100,2phc2sys0-21swapper/310:26:433
2564091100,2phc2sys0-21swapper/309:56:133
2564091100,2phc2sys0-21swapper/308:06:163
2564091100,2phc2sys0-21swapper/307:05:373
2564091100,1phc2sys0-21swapper/307:58:313
256409190,2phc2sys0-21swapper/312:15:483
256409190,2phc2sys0-21swapper/311:30:123
256409190,2phc2sys0-21swapper/309:20:063
256409190,2phc2sys0-21swapper/309:06:203
256409190,2phc2sys0-21swapper/308:16:193
256409190,2phc2sys0-21swapper/307:46:163
256409190,2phc2sys0-21swapper/307:16:113
256409180,1phc2sys0-21swapper/307:50:333
256409180,1phc2sys0-21swapper/307:35:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional