You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-16 - 08:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Sep 16, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
188532723120,17sleep20-21swapper/219:05:192
196124221010,2sleep00-21swapper/022:40:190
188548626844,15sleep00-21swapper/019:05:210
188622126445,13sleep30-21swapper/319:10:003
188591226343,13sleep10-21swapper/119:05:591
19090622530,4sleep13056-21PK-Backend20:10:241
19281752500,1chrt1927432-21/usr/sbin/munin21:05:182
19152922470,1sleep20-21swapper/220:30:142
1886385994124,9cyclictest0-21swapper/321:44:223
188638599388,9cyclictest1923522-21dpkg20:55:013
188638599336,16cyclictest1909083-21sed20:10:233
188638599316,13cyclictest0-21swapper/320:15:093
188638599315,8cyclictest0-21swapper/322:25:023
188638599315,14cyclictest1-21systemd23:05:003
188638599314,9cyclictest0-21swapper/323:50:133
188638599309,19cyclictest1905284-21cat20:00:193
188638599306,8cyclictest1948068-21systemctl22:05:003
188638599305,13cyclictest1888516-21sed19:15:023
139099300,21rtkit-daemon1927167-21dump-pmu-power21:04:552
188637799292,3cyclictest26-21ksoftirqd/120:00:001
188638599284,14cyclictest0-21swapper/300:00:223
188638599284,14cyclictest0-21swapper/300:00:223
188638599284,13cyclictest0-21swapper/321:05:113
188637799283,6cyclictest26-21ksoftirqd/122:35:001
1886377992812,8cyclictest3056-21PK-Backend19:21:361
139099280,18rtkit-daemon1987070-21cut23:55:002
18937342270,2sleep03056-21PK-Backend19:30:010
188638599276,8cyclictest1943307-21cstates21:50:143
188638599274,9cyclictest0-21swapper/321:16:443
188637799274,10cyclictest26-21ksoftirqd/119:50:211
188637799272,4cyclictest26-21ksoftirqd/122:05:001
188637799271,8cyclictest26-21ksoftirqd/123:35:001
188637799263,3cyclictest26-21ksoftirqd/120:35:001
188638599254,8cyclictest0-21swapper/323:25:173
188638599253,9cyclictest0-21swapper/319:55:043
1886385992512,11cyclictest1997781-21cpuspeed_turbos00:25:123
1886385992511,5cyclictest1978267-21acpi23:30:113
1886385992510,2cyclictest1959072-21df_inode22:35:123
1886385992510,13cyclictest1929710-21ls21:10:173
188637799257,9cyclictest26-21ksoftirqd/120:55:001
188637799254,7cyclictest26-21ksoftirqd/122:15:011
188637799252,4cyclictest26-21ksoftirqd/123:10:011
1886377992511,4cyclictest26-21ksoftirqd/120:05:191
1886377992511,4cyclictest26-21ksoftirqd/120:05:181
139099250,17rtkit-daemon0-21swapper/219:15:182
139099250,16rtkit-daemon0-21swapper/219:50:132
188638599249,13cyclictest1984451-21ntp_kernel_err23:45:193
188638599249,13cyclictest1973899-21users23:15:243
188638599249,13cyclictest1956101-21python322:25:193
188638599249,13cyclictest1894732-21ntp_states19:30:203
188638599248,14cyclictest1936900-21ntpq21:30:193
188638599241,21cyclictest1938783-21python321:35:233
1886385992412,10cyclictest1095-21runrttasks00:30:193
1886385992410,4cyclictest1935291-21/usr/sbin/munin21:25:223
1886385992410,13cyclictest1886933-21df_inode19:10:133
139099240,15rtkit-daemon0-21swapper/200:10:152
19008322230,4sleep03056-21PK-Backend19:50:010
188638599239,13cyclictest0-21swapper/300:05:153
188638599239,12cyclictest436-20systemd-journal22:15:373
188638599239,12cyclictest1958048-21users22:30:223
188638599237,14cyclictest0-21swapper/320:55:143
1886385992315,6cyclictest1916507-21cron20:35:013
1886385992314,7cyclictest1994157-21apt00:15:123
188638599230,20cyclictest436-20systemd-journal20:29:383
1886381992314,6cyclictest436-20systemd-journal20:38:062
188637799236,6cyclictest26-21ksoftirqd/122:45:181
188637799235,9cyclictest26-21ksoftirqd/123:10:171
188637799235,6cyclictest26-21ksoftirqd/120:15:201
188637799235,3cyclictest26-21ksoftirqd/121:55:001
188637799234,6cyclictest26-21ksoftirqd/100:20:211
188637799230,7cyclictest26-21ksoftirqd/120:30:001
188637799230,7cyclictest1961918-21idleruntime-cro22:45:011
188637799230,5cyclictest3056-21PK-Backend23:05:001
1886375992320,2cyclictest0-21swapper/021:17:260
188637599230,2cyclictest3056-21PK-Backend20:55:010
188637599230,21cyclictest1788620-21WebKitWebProces23:46:040
139099230,15rtkit-daemon0-21swapper/220:10:212
139099230,14rtkit-daemon0-21swapper/200:30:192
188638599229,11cyclictest1963589-21sshd22:49:373
188638599228,12cyclictest1896159-21basename19:35:143
1886385992220,1cyclictest0-21swapper/323:57:393
188638599221,8cyclictest1950903-21python322:10:213
1886385992211,9cyclictest1907865-21cut20:09:593
1886385992211,9cyclictest1907865-21cut20:09:593
1886385992211,9cyclictest1902441-21apt-config19:55:013
1886385992211,9cyclictest1900658-21sh19:50:003
1886385992211,9cyclictest1890999-21taskset19:20:163
188638599220,20cyclictest0-21swapper/321:20:103
1886381992220,1cyclictest0-21swapper/223:47:422
1886381992220,1cyclictest0-21swapper/200:16:222
1886381992220,1cyclictest0-21swapper/200:06:282
188638199220,20cyclictest0-21swapper/220:46:402
188637799223,9cyclictest1955818-21memory22:25:161
188637799222,6cyclictest26-21ksoftirqd/120:55:211
188637799221,7cyclictest1988712-40updatedb.mlocat00:00:001
188637799221,6cyclictest26-21ksoftirqd/123:25:001
1886377992212,3cyclictest26-21ksoftirqd/122:17:251
188637799220,5cyclictest3056-21PK-Backend21:59:591
188637799220,20cyclictest3056-21PK-Backend22:55:231
188637799220,20cyclictest3056-21PK-Backend22:20:351
1886375992218,2cyclictest13-21ksoftirqd/022:04:590
188637599220,5cyclictest1956339-21chrt22:25:310
188637599220,21cyclictest0-21swapper/022:48:320
188638599215,5cyclictest0-21swapper/323:43:093
1886385992119,1cyclictest0-21swapper/323:37:103
1886385992119,1cyclictest0-21swapper/320:45:073
1886385992119,1cyclictest0-21swapper/300:20:233
1886385992118,2cyclictest0-21swapper/321:45:103
1886385992112,7cyclictest1918719-21tr20:40:123
1886385992111,8cyclictest1965776-21grep22:55:173
1886385992111,8cyclictest1949756-21cron22:09:593
1886385992111,8cyclictest1917262-21/usr/sbin/munin20:35:163
1886385992111,8cyclictest1911965-21taskset20:20:163
1886385992110,8cyclictest0-21swapper/319:25:133
188638599210,20cyclictest0-21swapper/321:00:563
1886381992119,1cyclictest0-21swapper/223:44:452
1886381992119,1cyclictest0-21swapper/223:33:102
1886381992119,1cyclictest0-21swapper/222:37:462
1886381992119,1cyclictest0-21swapper/222:17:042
1886381992119,1cyclictest0-21swapper/221:36:362
1886381992119,1cyclictest0-21swapper/220:22:002
1886381992119,1cyclictest0-21swapper/219:46:382
1886381992119,1cyclictest0-21swapper/219:44:492
1886381992119,1cyclictest0-21swapper/219:31:582
188638199210,20cyclictest0-21swapper/222:12:252
188638199210,20cyclictest0-21swapper/219:27:452
188638199210,20cyclictest0-21swapper/219:14:142
188637799213,6cyclictest26-21ksoftirqd/121:10:011
188637799213,4cyclictest1993288-21python300:10:201
188637799212,7cyclictest26-21ksoftirqd/120:21:581
188637799212,7cyclictest26-21ksoftirqd/119:40:241
188637799211,8cyclictest26-21ksoftirqd/119:39:461
188637799211,6cyclictest1974299-21munin-run23:20:001
188637799210,8cyclictest1964767-21python322:50:211
188637799210,8cyclictest1940111-21cat21:40:171
188637799210,7cyclictest1942123-21/usr/sbin/munin21:45:201
188637799210,6cyclictest3056-21PK-Backend00:40:001
188637799210,6cyclictest19058852sleep120:02:091
188637799210,6cyclictest1893721-21munin-run19:30:001
188637799210,20cyclictest3056-21PK-Backend23:51:061
188637799210,20cyclictest3056-21PK-Backend21:01:401
188637799210,20cyclictest0-21swapper/121:15:201
188637799210,19cyclictest3056-21PK-Backend19:15:221
188637599213,2cyclictest141rcu_preempt19:20:100
188637599212,2cyclictest141rcu_preempt23:12:270
188637599212,2cyclictest141rcu_preempt22:58:030
188637599211,5cyclictest141rcu_preempt23:16:450
188637599210,20cyclictest1887230-21strings19:10:180
188637599210,20cyclictest0-21swapper/020:34:130
188637599210,20cyclictest0-21swapper/020:27:310
139099210,15rtkit-daemon0-21swapper/222:01:232
139099210,15rtkit-daemon0-21swapper/220:58:582
139099210,14rtkit-daemon0-21swapper/200:23:122
139099210,13rtkit-daemon0-21swapper/223:55:232
139099210,13rtkit-daemon0-21swapper/221:55:162
188638599209,9cyclictest1961036-21cat22:40:173
1886385992018,1cyclictest0-21swapper/323:20:423
1886385992018,1cyclictest0-21swapper/323:12:033
1886385992012,6cyclictest1964343-21if_enp2s022:50:163
1886385992011,7cyclictest1945702-21chrt21:55:203
1886385992010,8cyclictest2001283-21/usr/sbin/munin00:35:133
1886385992010,8cyclictest1993125-21/usr/sbin/munin00:10:183
1886381992018,1cyclictest0-21swapper/223:11:332
1886381992018,1cyclictest0-21swapper/223:05:042
1886381992018,1cyclictest0-21swapper/221:53:252
1886381992018,1cyclictest0-21swapper/221:44:132
1886381992018,1cyclictest0-21swapper/221:16:352
1886381992018,1cyclictest0-21swapper/221:14:112
1886381992018,1cyclictest0-21swapper/200:36:572
1886381992018,1cyclictest0-21swapper/200:01:562
1886381992018,1cyclictest0-21swapper/200:01:552
1886381992014,4cyclictest34-21ksoftirqd/223:00:012
188638199200,1cyclictest0-21swapper/222:45:372
188638199200,1cyclictest0-21swapper/222:44:492
188638199200,19cyclictest0-21swapper/220:05:582
188638199200,19cyclictest0-21swapper/220:05:582
188638199200,19cyclictest0-21swapper/200:26:152
188637799202,7cyclictest26-21ksoftirqd/119:46:571
188637799202,6cyclictest26-21ksoftirqd/122:09:351
188637799202,3cyclictest26-21ksoftirqd/123:45:001
188637799202,1cyclictest141rcu_preempt23:50:001
1886377992018,1cyclictest3056-21PK-Backend21:14:321
188637799201,7cyclictest26-21ksoftirqd/119:13:201
188637799201,7cyclictest26-21ksoftirqd/100:00:131
188637799201,7cyclictest26-21ksoftirqd/100:00:121
1886377992012,6cyclictest1920004-21munin-plugin-st20:45:011
188637799200,7cyclictest26-21ksoftirqd/119:32:321
188637799200,7cyclictest26-21ksoftirqd/100:30:361
188637799200,1cyclictest3056-21PK-Backend23:26:461
188637799200,1cyclictest3056-21PK-Backend21:30:421
188637799200,19cyclictest3056-21PK-Backend22:39:091
188637799200,19cyclictest3056-21PK-Backend21:23:031
188637799200,19cyclictest3056-21PK-Backend20:39:571
188637599203,2cyclictest141rcu_preempt21:46:510
188637599203,2cyclictest141rcu_preempt00:22:570
188637599202,2cyclictest141rcu_preempt21:40:360
1886375992018,1cyclictest0-21swapper/022:39:180
1886375992018,1cyclictest0-21swapper/021:38:550
1886375992018,1cyclictest0-21swapper/019:56:560
1886375992018,1cyclictest0-21swapper/019:50:410
1886375992018,1cyclictest0-21swapper/019:38:230
1886375992018,1cyclictest0-21swapper/000:25:580
1886375992018,1cyclictest0-21swapper/000:16:050
188637599200,8cyclictest0-21swapper/021:30:190
188637599200,5cyclictest2156-21gnome-shell20:48:090
188637599200,5cyclictest2156-21gnome-shell20:39:080
188637599200,1cyclictest0-21swapper/022:17:160
188637599200,1cyclictest0-21swapper/020:10:540
188637599200,19cyclictest3056-21PK-Backend00:30:310
188637599200,19cyclictest0-21swapper/023:57:530
188637599200,19cyclictest0-21swapper/023:35:150
188637599200,19cyclictest0-21swapper/023:30:560
188637599200,19cyclictest0-21swapper/022:09:170
188637599200,19cyclictest0-21swapper/021:58:420
188637599200,19cyclictest0-21swapper/021:06:030
139099200,14rtkit-daemon0-21swapper/219:36:182
139099200,13rtkit-daemon0-21swapper/222:55:322
1886385991911,6cyclictest1969538-21grep23:05:123
1886385991911,6cyclictest1897569-21cpuspeed_turbos19:40:123
1886381991917,1cyclictest0-21swapper/223:24:302
1886381991917,1cyclictest0-21swapper/222:23:062
1886381991917,1cyclictest0-21swapper/221:46:222
1886381991916,2cyclictest1935181-21ntp_states21:25:202
1886381991912,6cyclictest1904745-21ls20:00:122
188638199190,1cyclictest0-21swapper/223:39:382
188638199190,1cyclictest0-21swapper/221:30:072
188638199190,1cyclictest0-21swapper/220:52:012
1886377991917,1cyclictest3056-21PK-Backend20:48:501
1886377991917,1cyclictest3056-21PK-Backend00:27:501
188637799190,1cyclictest3056-21PK-Backend00:07:291
1886375991917,1cyclictest3056-21PK-Backend21:01:300
1886375991917,1cyclictest3056-21PK-Backend00:09:560
1886375991917,1cyclictest0-21swapper/023:51:100
1886375991917,1cyclictest0-21swapper/023:01:290
1886375991917,1cyclictest0-21swapper/020:22:240
1886375991917,1cyclictest0-21swapper/020:07:090
1886375991917,1cyclictest0-21swapper/020:07:090
1886375991917,1cyclictest0-21swapper/019:43:370
1886375991917,1cyclictest0-21swapper/019:34:590
1886375991917,1cyclictest0-21swapper/000:36:400
1886375991917,1cyclictest0-21swapper/000:10:450
1886375991917,1cyclictest0-21swapper/000:02:260
1886375991917,1cyclictest0-21swapper/000:02:260
188637599191,2cyclictest141rcu_preempt21:10:010
1886375991912,5cyclictest1974984-21expr23:20:110
188637599190,6cyclictest1969365-21taskset23:05:110
188637599190,1cyclictest0-21swapper/022:23:170
188637599190,1cyclictest0-21swapper/022:10:390
188637599190,1cyclictest0-21swapper/021:50:530
188637599190,1cyclictest0-21swapper/020:00:340
188637599190,18cyclictest0-21swapper/023:41:450
188637599190,18cyclictest0-21swapper/023:26:370
139099190,13rtkit-daemon0-21swapper/223:15:362
1886381991816,1cyclictest0-21swapper/221:20:332
1886381991816,1cyclictest0-21swapper/220:27:542
1886381991816,1cyclictest0-21swapper/219:20:352
1886381991814,2cyclictest1977135-21memory23:25:192
1886381991811,5cyclictest1955427-21grep22:25:122
1886381991811,5cyclictest1095-21runrttasks20:15:112
1886381991810,6cyclictest436-20systemd-journal20:42:362
1886377991816,1cyclictest3056-21PK-Backend23:36:481
1886377991816,1cyclictest3056-21PK-Backend21:26:141
1886377991816,1cyclictest0-21swapper/121:35:551
1886375991816,1cyclictest0-21swapper/020:17:500
1886375991816,1cyclictest0-21swapper/019:15:380
1886381991715,1cyclictest0-21swapper/222:34:162
188638199171,12cyclictest1963904-21apt22:50:112
188637799178,3cyclictest26-21ksoftirqd/100:18:081
188637599178,3cyclictest141rcu_preempt22:30:220
1886375991715,1cyclictest0-21swapper/021:20:100
1886375991715,1cyclictest0-21swapper/020:43:190
1886375991711,4cyclictest13-21ksoftirqd/020:56:130
188637599170,2cyclictest141rcu_preempt22:50:390
1886381991614,1cyclictest0-21swapper/222:05:502
1886381991611,4cyclictest34-21ksoftirqd/219:55:242
188637599169,2cyclictest13-21ksoftirqd/021:25:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional