You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 13:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Aug 17, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
172884722530,11sleep00-21swapper/019:09:360
180201621200,2sleep13056-21PK-Backend22:35:161
174801721120,2sleep00-21swapper/020:00:210
178725921060,3sleep33056-21PK-Backend21:55:003
17754942970,3sleep00-21swapper/021:20:120
17885662920,1sleep21788567-21sshd21:55:352
172876027240,26sleep10-21swapper/119:08:291
172829326743,16sleep30-21swapper/319:05:263
172880826543,14sleep20-21swapper/219:09:072
18389002550,4sleep00-21swapper/000:20:010
18139602550,2sleep2331rcuc/223:10:062
18227582510,3sleep30-21swapper/323:35:013
17308912510,4sleep33056-21PK-Backend19:15:003
18052702500,4sleep00-21swapper/022:45:010
17980702500,4sleep20-21swapper/222:25:152
17913232360,2sleep20-21swapper/222:05:142
17742822360,3sleep30-21swapper/321:15:183
172901599317,22cyclictest883-21dbus-daemon20:50:010
1729015992913,14cyclictest1828833-21awk23:50:170
139099290,4rtkit-daemon1389-21rtkit-daemon20:29:293
172901599280,3cyclictest1788980-21awk22:00:000
172901599277,18cyclictest1798529-21sed22:25:200
139099270,4rtkit-daemon1389-21rtkit-daemon23:52:433
139099270,4rtkit-daemon1389-21rtkit-daemon19:59:293
18336152260,4sleep00-21swapper/000:05:130
18318652260,12sleep013-21ksoftirqd/000:00:120
172901599268,16cyclictest1773780-21cstates21:15:130
172901599267,17cyclictest1810317-21date23:00:000
172901599267,17cyclictest1740273-21cpuspeed_turbos19:40:130
172901599266,18cyclictest1735274-21expr19:25:150
172901599265,19cyclictest1799784-21expr22:30:140
1729015992610,14cyclictest1738963-21cut19:35:160
172901599260,11cyclictest1831620-40ogrotate00:00:000
139099260,4rtkit-daemon1389-21rtkit-daemon23:55:363
139099260,4rtkit-daemon1389-21rtkit-daemon23:03:192
139099260,4rtkit-daemon1389-21rtkit-daemon23:03:192
17700992250,4sleep33056-21PK-Backend21:05:123
172901599255,17cyclictest1790199-21timerandwakeup22:00:230
1729015992517,7cyclictest0-21swapper/020:54:410
172901599251,10cyclictest1792449-21awk22:10:000
1729015992510,13cyclictest1836798-21sh00:15:000
172901599250,1cyclictest0-21swapper/020:57:310
139099250,3rtkit-daemon1389-21rtkit-daemon23:25:042
172902299241,3cyclictest1095-21runrttasks00:08:082
172901599248,14cyclictest1095-21runrttasks22:54:520
172901599242,18cyclictest1733191-21apt19:20:130
1729015992419,4cyclictest0-21swapper/020:05:540
1729015992419,4cyclictest0-21swapper/019:19:300
172901599240,2cyclictest1793791-21awk22:10:240
18004282230,3chrt3056-21PK-Backend22:30:213
172902799231,7cyclictest1759181-21munin-run20:35:003
1729022992321,1cyclictest0-21swapper/222:58:392
1729022992321,1cyclictest0-21swapper/222:54:412
1729022992315,6cyclictest1844623-21df00:35:152
172901999230,2cyclictest141rcu_preempt22:50:011
172901599237,14cyclictest1762829-21cut20:45:010
172901599236,15cyclictest1819300-21cpuspeed_turbos23:25:160
172901599236,15cyclictest1769955-21latency_hist21:05:000
172901599233,5cyclictest1782234-21cat21:40:100
172901599233,5cyclictest1754045-21cut20:20:000
172901599232,18cyclictest0-21swapper/021:05:120
172901599232,17cyclictest0-21swapper/023:00:140
172901599232,17cyclictest0-21swapper/023:00:140
139099230,4rtkit-daemon1389-21rtkit-daemon21:37:133
17364132220,5chrt3056-21PK-Backend19:30:003
172902799220,20cyclictest1836831-21munin-run00:14:593
1729022992220,1cyclictest0-21swapper/223:35:232
1729022992220,1cyclictest0-21swapper/221:01:182
1729022992220,1cyclictest0-21swapper/220:44:492
172902299220,21cyclictest0-21swapper/219:31:142
1729019992220,1cyclictest3056-21PK-Backend23:27:041
1729019992220,1cyclictest0-21swapper/122:59:331
1729019992220,1cyclictest0-21swapper/122:21:481
1729019992214,6cyclictest1771839-21apt21:10:121
172901999220,20cyclictest3056-21PK-Backend22:43:171
172901599229,11cyclictest1842723-21tr00:30:140
172901599228,12cyclictest1771842-21apt21:10:130
172901599226,14cyclictest1827043-21forks23:45:130
172901599225,15cyclictest1729387-21cpuspeed_turbos19:10:130
172901599225,15cyclictest0-21swapper/000:39:140
172901599222,16cyclictest1796175-21apt22:20:120
172901599222,16cyclictest1794438-21apt22:15:120
172901599221,7cyclictest1746064-21grep19:55:180
1729015992214,6cyclictest1779651-21python321:30:220
172901599221,17cyclictest0-21swapper/020:10:120
1729027992117,3cyclictest42-21ksoftirqd/323:48:583
1729027992115,4cyclictest1776776-21munin-run21:25:003
172902799211,18cyclictest1843598-21/usr/sbin/munin00:30:253
172902799210,20cyclictest3056-21PK-Backend21:41:503
172902799210,20cyclictest0-21swapper/320:23:493
172902799210,19cyclictest0-21swapper/323:03:363
172902799210,19cyclictest0-21swapper/323:03:363
1729022992119,1cyclictest0-21swapper/223:43:482
1729022992119,1cyclictest0-21swapper/223:32:192
1729022992119,1cyclictest0-21swapper/223:07:112
1729022992119,1cyclictest0-21swapper/222:19:532
1729022992119,1cyclictest0-21swapper/221:28:222
1729022992119,1cyclictest0-21swapper/220:39:212
1729022992119,1cyclictest0-21swapper/220:01:292
1729022992119,1cyclictest0-21swapper/219:55:412
1729022992119,1cyclictest0-21swapper/200:19:022
1729022992113,6cyclictest1804518-21http22:41:342
172902299210,20cyclictest0-21swapper/222:21:352
172902299210,20cyclictest0-21swapper/221:21:052
172902299210,20cyclictest0-21swapper/200:03:562
1729019992119,1cyclictest3056-21PK-Backend21:49:371
1729019992119,1cyclictest3056-21PK-Backend20:24:271
1729019992119,1cyclictest0-21swapper/122:12:481
1729019992119,1cyclictest0-21swapper/121:57:011
1729019992114,6cyclictest883-21dbus-daemon22:34:191
1729019992112,7cyclictest26-21ksoftirqd/123:00:131
1729019992112,7cyclictest26-21ksoftirqd/123:00:121
172901999210,20cyclictest0-21swapper/121:05:501
172901999210,20cyclictest0-21swapper/121:04:421
172901999210,19cyclictest0-21swapper/120:44:481
172901599217,12cyclictest0-21swapper/023:35:180
172901599216,13cyclictest1786537-21processes21:50:200
172901599215,5cyclictest1737400-21ntpq19:30:190
172901599215,4cyclictest1837382-21expr00:15:130
172901599214,5cyclictest1788610-21gdbus22:41:330
172901599214,2cyclictest0-21swapper/023:08:560
172901599210,8cyclictest1781374-21grep21:35:210
172901599210,7cyclictest436-20systemd-journal21:50:010
172901599210,7cyclictest1814839-21awk23:10:220
172901599210,6cyclictest1758154-21awk20:30:170
172901599210,5cyclictest1745310-21cron19:55:000
172901599210,19cyclictest0-21swapper/022:35:190
1729027992018,1cyclictest3056-21PK-Backend20:45:053
1729027992017,2cyclictest3056-21PK-Backend19:40:093
1729027992016,3cyclictest1778882-21cpuspeed21:30:143
1729027992015,3cyclictest42-21ksoftirqd/319:55:013
1729027992015,3cyclictest1839803-21ntp_states00:20:213
1729027992014,5cyclictest42-21ksoftirqd/323:40:193
1729027992014,4cyclictest1812995-21ntp_states23:05:173
1729027992013,5cyclictest42-21ksoftirqd/321:26:273
1729027992013,5cyclictest1845723-21cron00:40:003
172902799201,17cyclictest1748765-21cron20:05:003
1729027992011,6cyclictest1834467-21/usr/sbin/munin00:05:223
172902799201,16cyclictest1807038-21/usr/sbin/munin22:50:153
172902799200,1cyclictest3056-21PK-Backend22:16:243
172902799200,19cyclictest3056-21PK-Backend22:12:393
172902799200,19cyclictest3056-21PK-Backend22:05:433
172902799200,19cyclictest3056-21PK-Backend19:32:253
172902799200,19cyclictest0-21swapper/323:24:523
172902799200,19cyclictest0-21swapper/321:45:563
172902799200,18cyclictest3056-21PK-Backend21:14:123
1729022992018,1cyclictest3056-21PK-Backend19:25:262
1729022992018,1cyclictest0-21swapper/221:51:542
1729022992018,1cyclictest0-21swapper/221:41:402
1729022992018,1cyclictest0-21swapper/220:46:312
1729022992018,1cyclictest0-21swapper/220:23:192
1729022992018,1cyclictest0-21swapper/220:10:492
1729022992018,1cyclictest0-21swapper/219:48:182
1729022992018,1cyclictest0-21swapper/219:35:542
1729022992018,1cyclictest0-21swapper/219:23:302
1729022992018,1cyclictest0-21swapper/200:26:472
1729022992018,1cyclictest0-21swapper/200:10:042
1729022992017,2cyclictest34-21ksoftirqd/222:10:062
1729022992011,8cyclictest1830831-21ntpq23:55:202
172902299200,19cyclictest0-21swapper/223:53:352
172902299200,19cyclictest0-21swapper/223:21:032
172902299200,19cyclictest0-21swapper/222:34:592
172902299200,19cyclictest0-21swapper/219:11:202
172902299200,19cyclictest0-21swapper/200:33:092
1729019992018,1cyclictest3056-21PK-Backend23:35:231
1729019992018,1cyclictest3056-21PK-Backend23:33:391
1729019992018,1cyclictest3056-21PK-Backend23:20:421
1729019992018,1cyclictest3056-21PK-Backend23:11:091
1729019992018,1cyclictest3056-21PK-Backend20:39:481
1729019992018,1cyclictest3056-21PK-Backend00:02:331
1729019992018,1cyclictest0-21swapper/122:04:041
1729019992017,2cyclictest3056-21PK-Backend00:22:201
1729019992014,4cyclictest26-21ksoftirqd/122:55:001
1729019992012,6cyclictest26-21ksoftirqd/119:10:121
1729019992012,2cyclictest26-21ksoftirqd/120:45:031
1729019992012,2cyclictest26-21ksoftirqd/119:35:211
1729019992012,2cyclictest26-21ksoftirqd/119:26:091
1729019992011,3cyclictest26-21ksoftirqd/120:15:001
172901999200,19cyclictest0-21swapper/121:53:291
172901599204,4cyclictest1821901-21awk23:30:200
1729015992018,1cyclictest0-21swapper/019:46:210
1729015992017,2cyclictest3056-21PK-Backend20:37:450
172901599201,6cyclictest1841472-21uname00:25:190
172901599200,6cyclictest1756472-21sed20:25:170
172901599200,5cyclictest1824751-21cat23:40:120
172901599200,5cyclictest1818979-21sort23:25:010
139099200,16rtkit-daemon1729009-21cyclictest21:30:182
18308332190,4sleep13056-21PK-Backend23:55:191
1729027991917,1cyclictest3056-21PK-Backend20:12:543
1729027991917,1cyclictest0-21swapper/322:36:113
1729027991917,1cyclictest0-21swapper/320:37:013
1729027991917,1cyclictest0-21swapper/319:49:153
1729027991917,1cyclictest0-21swapper/300:25:353
1729027991916,2cyclictest3056-21PK-Backend19:17:043
1729027991914,3cyclictest1805332-21apt22:45:133
1729027991912,5cyclictest1749522-21irqstats20:05:173
172902799190,6cyclictest1814668-21uname23:10:193
172902799190,1cyclictest3056-21PK-Backend20:50:333
172902799190,1cyclictest0-21swapper/320:40:123
172902799190,18cyclictest3056-21PK-Backend19:23:333
1729022991917,1cyclictest0-21swapper/222:37:512
1729022991917,1cyclictest0-21swapper/221:47:492
1729022991917,1cyclictest0-21swapper/220:08:052
1729022991917,1cyclictest0-21swapper/200:20:592
172902299190,1cyclictest0-21swapper/220:25:292
172902299190,1cyclictest0-21swapper/220:16:162
172902299190,1cyclictest0-21swapper/219:43:382
1729019991917,1cyclictest3056-21PK-Backend21:38:011
1729019991917,1cyclictest3056-21PK-Backend19:50:061
1729019991917,1cyclictest0-21swapper/120:15:351
1729019991917,1cyclictest0-21swapper/120:07:161
1729019991916,2cyclictest3056-21PK-Backend23:53:071
1729019991912,5cyclictest1759246-21sed20:35:011
1729019991911,2cyclictest26-21ksoftirqd/119:15:021
1729019991910,7cyclictest1773571-21apt21:15:121
172901999190,1cyclictest3056-21PK-Backend00:08:351
172901999190,1cyclictest0-21swapper/120:53:471
172901599193,3cyclictest1777210-21cut21:25:150
172901599191,8cyclictest1816121-21hddtemp_smartct23:15:170
172901599190,7cyclictest1754247-21apt20:20:120
172902799189,6cyclictest1838762-21latency_hist00:20:013
172902799182,12cyclictest1831877-21apt00:00:133
1729027991816,1cyclictest3056-21PK-Backend22:02:323
1729027991816,1cyclictest3056-21PK-Backend20:15:383
1729027991812,4cyclictest2254-21gdbus22:41:333
172902799180,16cyclictest17888212sleep321:58:173
1729022991816,1cyclictest0-21swapper/221:35:172
172901999189,7cyclictest1733933-21ntp_states19:20:211
172901999189,2cyclictest26-21ksoftirqd/121:30:221
1729019991816,1cyclictest0-21swapper/121:40:041
1729019991816,1cyclictest0-21swapper/100:10:171
1729019991811,6cyclictest1792650-21cat22:10:011
1729019991811,2cyclictest26-21ksoftirqd/119:40:221
1729019991810,6cyclictest1748772-21perf20:05:001
1729019991810,2cyclictest26-21ksoftirqd/122:25:241
1729019991810,2cyclictest26-21ksoftirqd/119:30:131
1729019991810,2cyclictest26-21ksoftirqd/100:25:161
172902799179,6cyclictest1823641-21ntpq23:35:203
1729027991711,4cyclictest2156-21gnome-shell23:28:093
172902299179,6cyclictest1774434-21python321:15:212
1729022991715,1cyclictest0-21swapper/222:04:382
1729022991715,1cyclictest0-21swapper/219:51:012
1729022991712,4cyclictest978-21snmpd23:49:372
1729022991712,3cyclictest1816605-21switchtime23:15:232
1729022991711,5cyclictest1758613-21sed20:30:222
1729022991710,6cyclictest883-21dbus-daemon20:58:032
172902299170,7cyclictest1731628-21cpuspeed_turbos19:15:112
172901999179,2cyclictest26-21ksoftirqd/121:24:091
172901999179,2cyclictest26-21ksoftirqd/119:55:221
172901999173,13cyclictest1813732-21grep23:10:001
1729019991715,1cyclictest3056-21PK-Backend21:25:511
1729019991715,1cyclictest3056-21PK-Backend00:18:561
1729019991715,1cyclictest0-21swapper/122:16:001
1729019991715,1cyclictest0-21swapper/120:58:401
1729027991614,1cyclictest3056-21PK-Backend22:28:273
1729027991614,1cyclictest3056-21PK-Backend22:22:263
172902799161,13cyclictest1815701-21/usr/sbin/munin23:15:183
172902299169,5cyclictest1806826-21pluginstate22:50:012
172902299164,10cyclictest1770956-21grep21:05:212
1729019991614,1cyclictest3056-21PK-Backend19:45:541
1729019991614,1cyclictest3056-21PK-Backend00:37:551
172902799151,5cyclictest141rcu_preempt21:03:353
1729027991513,1cyclictest3056-21PK-Backend22:55:183
1729027991513,1cyclictest3056-21PK-Backend19:35:433
172902799150,1cyclictest141rcu_preempt21:00:003
1729022991513,1cyclictest0-21swapper/221:13:272
1729019991513,1cyclictest3056-21PK-Backend20:28:191
172901999150,14cyclictest3056-21PK-Backend23:49:421
172901999150,14cyclictest3056-21PK-Backend23:41:241
172902299148,5cyclictest1765852-21awk20:50:212
172901999140,5cyclictest3056-21PK-Backend00:30:231
172901999140,1cyclictest3056-21PK-Backend23:16:571
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional