You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-15 - 06:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Nov 15, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
198973721410,3sleep20-21swapper/200:10:212
189442921300,3sleep03056-21PK-Backend19:40:110
139099800,61rtkit-daemon0-21swapper/319:05:103
188224926740,21sleep00-21swapper/019:08:570
188085426444,14sleep20-21swapper/219:05:092
188205526344,13sleep10-21swapper/119:06:291
19842692540,4sleep33056-21PK-Backend23:55:183
19554722500,2chrt0-21swapper/122:35:131
19154712450,4sleep33056-21PK-Backend20:40:173
19293862400,7sleep2341ksoftirqd/221:20:112
19398022290,2sleep1251rcuc/121:50:131
188254799299,18cyclictest1940405-21timerandwakeup21:50:220
188254799297,20cyclictest1918080-21cron20:50:000
188254799297,20cyclictest1918080-21cron20:50:000
188254799293,24cyclictest1928687-21sed21:20:010
188254799291,10cyclictest1948069-21/usr/sbin/munin22:15:240
1882560992810,5cyclictest1950760-21cat22:20:213
188254799288,18cyclictest1923300-21sh21:04:590
188254799283,23cyclictest1892556-21cstates19:35:100
188254799282,22cyclictest0-21swapper/022:10:110
19845012270,7sleep2341ksoftirqd/223:55:222
188254799279,16cyclictest1884401-21latency_hist19:15:010
188254799278,18cyclictest1904938-21cat20:10:160
188254799278,17cyclictest1935268-21timerwakeupswit21:35:230
188254799278,17cyclictest1889029-21cut19:25:140
188254799278,17cyclictest1095-21runrttasks23:09:280
188254799277,18cyclictest82332-21update-manager20:25:280
188254799277,18cyclictest1975902-21sed23:30:210
188254799277,18cyclictest1941737-21cat21:55:180
188254799277,18cyclictest0-21swapper/023:03:590
188254799276,19cyclictest1913267-21sort20:35:110
188254799275,20cyclictest1970339-21needreboot23:15:150
188254799272,20cyclictest0-21swapper/019:55:120
188254799271,21cyclictest1884388-21apt-get19:15:040
1882547992711,14cyclictest1925038-21latency_hist21:10:010
188254799270,4cyclictest1897931-21df_inode19:50:140
188254799270,4cyclictest1897931-21df_inode19:50:140
139099270,4rtkit-daemon1389-21rtkit-daemon00:26:193
188254799268,16cyclictest1903506-21open_files20:05:190
188254799267,17cyclictest1989170-21idleruntime00:10:160
188254799267,17cyclictest1952055-21sed22:25:160
188254799267,17cyclictest0-21swapper/022:50:160
188254799266,18cyclictest1943114-21ls22:00:140
188254799265,5cyclictest1959039-21expr22:45:120
188254799262,4cyclictest0-21swapper/022:05:110
188254799261,7cyclictest1901886-21python320:00:210
188254799261,23cyclictest1983705-21cut23:55:110
188254799261,20cyclictest1955254-21apt-get22:35:040
188254799261,11cyclictest1985670-21diskstats00:00:160
1882547992610,14cyclictest1930493-21sh21:25:000
188254799260,4cyclictest1963222-21ntp_states22:55:170
188256099253,20cyclictest1922927-21nm-dispatcher21:00:283
188255099250,23cyclictest1897955-21hddtemp_smartct19:50:161
188255099250,23cyclictest1897955-21hddtemp_smartct19:50:161
188255099250,23cyclictest1788620-21WebKitWebProces21:55:351
188254799258,15cyclictest1968018-21cpuspeed23:10:120
188254799258,15cyclictest1954872-21apt-key22:35:010
188254799258,15cyclictest1954872-21apt-key22:35:010
188254799257,16cyclictest2156-21gnome-shell21:27:090
188254799257,16cyclictest1938210-21latency21:45:160
188254799257,16cyclictest1938210-21latency21:45:160
188254799257,15cyclictest0-21swapper/021:30:110
188254799256,4cyclictest1915191-21cat20:40:140
188254799256,17cyclictest1999330-21cat00:40:010
188254799254,5cyclictest1925652-21cat21:10:120
188254799254,4cyclictest1935974-21grep21:40:120
188254799252,5cyclictest1987889-21python300:05:220
188254799252,5cyclictest1974001-21python323:25:180
1882547992519,5cyclictest0-21swapper/020:23:540
1882547992519,5cyclictest0-21swapper/000:18:360
188254799251,8cyclictest1978070-21perf23:40:000
188254799251,8cyclictest1912912-21awk20:35:010
188254799251,8cyclictest1896188-21/usr/sbin/munin19:45:130
188254799251,7cyclictest1994402-21awk00:25:140
188254799251,5cyclictest1891486-21python319:30:220
188254799251,21cyclictest1918366-21apt20:50:110
1882547992510,13cyclictest19204742sleep020:55:160
188254799250,9cyclictest1957673-21grep22:40:200
188254799250,7cyclictest0-21swapper/023:20:110
139099250,4rtkit-daemon1389-21rtkit-daemon20:03:112
139099250,3rtkit-daemon1389-21rtkit-daemon21:47:272
139099250,3rtkit-daemon1389-21rtkit-daemon21:47:262
188256099241,16cyclictest1882937-21chrt19:10:123
188254799247,15cyclictest1993772-21grep00:25:000
188254799246,16cyclictest0-21swapper/023:51:230
188254799245,2cyclictest0-21swapper/023:48:420
188254799245,17cyclictest0-21swapper/022:22:410
1882547992418,5cyclictest3056-21PK-Backend19:21:010
1882560992318,3cyclictest1902473-21cron20:05:003
1882560992313,7cyclictest1944344-21cron22:04:593
188256099230,21cyclictest3056-21PK-Backend21:10:213
188255099230,21cyclictest1788620-21WebKitWebProces22:48:061
188254799233,3cyclictest0-21swapper/020:17:300
188254799230,6cyclictest1997343-21apt-config00:35:010
1882560992220,1cyclictest3056-21PK-Backend23:22:233
1882560992219,2cyclictest3056-21PK-Backend21:42:253
1882560992219,2cyclictest3056-21PK-Backend20:48:523
1882560992219,2cyclictest3056-21PK-Backend20:48:523
1882560992218,2cyclictest3056-21PK-Backend21:23:263
1882560992213,7cyclictest1964842-21df_inode23:00:133
188256099220,20cyclictest3056-21PK-Backend23:12:153
188256099220,20cyclictest3056-21PK-Backend22:15:033
188255699220,21cyclictest0-21swapper/219:32:362
1882550992220,1cyclictest0-21swapper/122:14:551
1882550992220,1cyclictest0-21swapper/120:08:271
1882550992218,2cyclictest436-20systemd-journal19:49:551
188255099220,5cyclictest0-21swapper/121:35:131
188254799220,6cyclictest1979481-21chrt23:41:040
1882560992119,1cyclictest3056-21PK-Backend21:49:283
1882560992119,1cyclictest3056-21PK-Backend21:49:283
1882560992119,1cyclictest3056-21PK-Backend20:37:023
1882560992119,1cyclictest0-21swapper/323:48:253
1882560992119,1cyclictest0-21swapper/323:42:103
1882560992119,1cyclictest0-21swapper/322:25:243
1882560992119,1cyclictest0-21swapper/319:41:053
1882560992119,1cyclictest0-21swapper/300:22:113
1882560992119,1cyclictest0-21swapper/300:14:273
1882560992113,4cyclictest1970533-21python323:15:193
188256099210,20cyclictest0-21swapper/300:18:193
188256099210,19cyclictest3056-21PK-Backend21:38:053
1882556992119,1cyclictest0-21swapper/223:50:172
1882556992119,1cyclictest0-21swapper/222:40:282
1882556992119,1cyclictest0-21swapper/222:30:412
1882556992119,1cyclictest0-21swapper/222:30:412
1882556992119,1cyclictest0-21swapper/222:12:082
1882556992119,1cyclictest0-21swapper/222:06:342
1882556992119,1cyclictest0-21swapper/221:38:292
1882556992119,1cyclictest0-21swapper/220:49:082
1882556992119,1cyclictest0-21swapper/220:49:082
1882556992119,1cyclictest0-21swapper/219:11:072
1882556992119,1cyclictest0-21swapper/200:32:152
1882556992119,1cyclictest0-21swapper/200:28:232
1882556992118,1cyclictest3056-21PK-Backend20:55:232
1882556992117,3cyclictest978-21snmpd23:36:452
1882556992114,5cyclictest1896721-21sed19:45:222
188255699210,20cyclictest0-21swapper/223:28:272
188255699210,20cyclictest0-21swapper/223:21:582
188255699210,20cyclictest0-21swapper/223:05:562
188255699210,20cyclictest0-21swapper/200:05:102
188255699210,19cyclictest3056-21PK-Backend22:15:202
188255699210,19cyclictest3056-21PK-Backend19:55:142
1882550992119,1cyclictest0-21swapper/123:50:121
1882550992119,1cyclictest0-21swapper/123:35:321
1882550992119,1cyclictest0-21swapper/123:16:191
1882550992119,1cyclictest0-21swapper/122:53:411
1882550992119,1cyclictest0-21swapper/122:44:081
1882550992119,1cyclictest0-21swapper/122:20:281
1882550992119,1cyclictest0-21swapper/122:18:061
1882550992119,1cyclictest0-21swapper/122:02:381
1882550992119,1cyclictest0-21swapper/121:18:581
1882550992119,1cyclictest0-21swapper/121:10:461
1882550992119,1cyclictest0-21swapper/120:50:041
1882550992119,1cyclictest0-21swapper/120:41:191
1882550992119,1cyclictest0-21swapper/120:38:011
1882550992119,1cyclictest0-21swapper/120:30:311
1882550992119,1cyclictest0-21swapper/120:28:561
188255099210,20cyclictest3056-21PK-Backend21:41:291
188255099210,20cyclictest3056-21PK-Backend00:13:101
188255099210,20cyclictest1981396-21taskset23:48:221
188255099210,20cyclictest0-21swapper/120:45:591
188255099210,20cyclictest0-21swapper/120:45:591
188255099210,19cyclictest3056-21PK-Backend19:25:351
188255099210,19cyclictest0-21swapper/100:35:141
139099210,4rtkit-daemon1389-21rtkit-daemon21:55:382
139099210,18rtkit-daemon1997-21gvfs-afc-volume20:15:483
188256099203,4cyclictest1888962-21cat19:25:133
1882560992018,1cyclictest3056-21PK-Backend20:28:303
1882560992018,1cyclictest0-21swapper/323:50:143
1882560992018,1cyclictest0-21swapper/322:45:323
1882560992018,1cyclictest0-21swapper/320:14:513
1882560992018,1cyclictest0-21swapper/319:19:293
1882560992013,5cyclictest42-21ksoftirqd/319:20:133
1882560992012,6cyclictest1997798-21cat00:35:123
1882560992012,6cyclictest1927483-21cat21:15:163
1882560992012,5cyclictest1997207-21dpkg00:34:593
188256099201,16cyclictest1908323-21df_inode20:20:143
188256099200,1cyclictest3056-21PK-Backend22:12:403
188256099200,1cyclictest0-21swapper/300:03:523
188256099200,19cyclictest3056-21PK-Backend23:32:153
188256099200,19cyclictest3056-21PK-Backend23:07:363
188256099200,19cyclictest3056-21PK-Backend21:08:113
188256099200,19cyclictest0-21swapper/319:47:203
188256099200,19cyclictest0-21swapper/319:39:163
1882556992018,1cyclictest3056-21PK-Backend00:04:092
1882556992018,1cyclictest0-21swapper/223:40:222
1882556992018,1cyclictest0-21swapper/223:17:182
1882556992018,1cyclictest0-21swapper/221:50:052
1882556992018,1cyclictest0-21swapper/221:28:292
1882556992018,1cyclictest0-21swapper/221:13:012
1882556992018,1cyclictest0-21swapper/221:01:392
1882556992018,1cyclictest0-21swapper/220:41:442
1882556992018,1cyclictest0-21swapper/220:20:012
1882556992018,1cyclictest0-21swapper/220:14:412
1882556992018,1cyclictest0-21swapper/220:08:122
1882556992018,1cyclictest0-21swapper/219:28:242
188255699200,1cyclictest3056-21PK-Backend19:35:132
188255699200,19cyclictest1882540-21cyclictest22:58:122
188255699200,19cyclictest0-21swapper/223:02:042
188255699200,19cyclictest0-21swapper/222:20:062
188255699200,19cyclictest0-21swapper/222:00:402
188255699200,19cyclictest0-21swapper/221:42:482
188255699200,19cyclictest0-21swapper/220:53:142
188255699200,19cyclictest0-21swapper/220:37:052
188255699200,19cyclictest0-21swapper/220:18:262
188255699200,19cyclictest0-21swapper/200:37:552
188255099208,10cyclictest1939321-21latency_hist21:50:011
188255099208,10cyclictest1939321-21latency_hist21:50:011
1882550992018,1cyclictest3056-21PK-Backend23:01:321
1882550992018,1cyclictest0-21swapper/123:28:151
1882550992018,1cyclictest0-21swapper/121:30:541
1882550992018,1cyclictest0-21swapper/120:03:341
1882550992018,1cyclictest0-21swapper/100:21:291
1882550992013,5cyclictest1987870-21rt-features00:05:211
188255099200,1cyclictest3056-21PK-Backend21:05:051
188255099200,19cyclictest3056-21PK-Backend23:41:341
188255099200,19cyclictest3056-21PK-Backend20:14:091
188255099200,19cyclictest0-21swapper/123:55:531
188255099200,19cyclictest0-21swapper/123:13:281
188255099200,19cyclictest0-21swapper/123:05:101
188255099200,19cyclictest0-21swapper/122:30:361
188255099200,19cyclictest0-21swapper/122:30:361
188255099200,19cyclictest0-21swapper/100:03:371
139099200,3rtkit-daemon1389-21rtkit-daemon00:22:342
19552512190,4chrt3056-21PK-Backend22:35:033
1882560991917,1cyclictest3056-21PK-Backend23:26:013
1882560991917,1cyclictest3056-21PK-Backend20:30:063
1882560991917,1cyclictest0-21swapper/320:51:353
1882560991916,2cyclictest3056-21PK-Backend21:57:533
1882560991916,2cyclictest3056-21PK-Backend21:51:383
188256099191,15cyclictest1963949-21perf23:00:003
188256099190,1cyclictest3056-21PK-Backend20:06:133
188256099190,1cyclictest0-21swapper/319:55:383
188256099190,17cyclictest1958096-21uptime22:40:253
1882556991917,1cyclictest0-21swapper/221:07:202
1882556991917,1cyclictest0-21swapper/220:25:222
1882556991913,5cyclictest34-21ksoftirqd/223:30:122
1882556991912,5cyclictest1962408-21latency_hist22:55:012
1882556991912,5cyclictest1894632-21ls19:40:162
188255699190,1cyclictest0-21swapper/223:11:362
188255699190,1cyclictest0-21swapper/221:19:162
188255699190,1cyclictest0-21swapper/220:33:542
188255699190,1cyclictest0-21swapper/200:17:482
1882550991917,1cyclictest3056-21PK-Backend19:37:111
1882550991917,1cyclictest3056-21PK-Backend00:16:001
1882550991917,1cyclictest0-21swapper/123:33:431
1882550991917,1cyclictest0-21swapper/121:28:441
1882550991917,1cyclictest0-21swapper/119:33:461
188255099190,1cyclictest3056-21PK-Backend22:26:371
188255099190,1cyclictest3056-21PK-Backend19:17:031
188255099190,1cyclictest0-21swapper/122:56:521
188255099190,1cyclictest0-21swapper/121:01:001
188255099190,1cyclictest0-21swapper/119:23:121
188255099190,1cyclictest0-21swapper/119:13:181
19873432180,4sleep33056-21PK-Backend00:05:133
1882560991816,1cyclictest3056-21PK-Backend22:05:443
1882560991816,1cyclictest3056-21PK-Backend21:33:133
1882560991816,1cyclictest3056-21PK-Backend19:32:273
1882560991816,1cyclictest0-21swapper/323:35:543
1882560991816,1cyclictest0-21swapper/320:55:413
1882560991815,2cyclictest3056-21PK-Backend21:29:413
1882550991816,1cyclictest0-21swapper/123:20:581
188255099180,1cyclictest0-21swapper/120:16:181
139099180,15rtkit-daemon1882540-21cyclictest19:52:122
139099180,15rtkit-daemon1882540-21cyclictest19:52:122
1882560991715,1cyclictest3056-21PK-Backend19:52:003
1882560991715,1cyclictest3056-21PK-Backend19:52:003
188255699179,6cyclictest1886275-21python319:15:202
1882556991715,1cyclictest0-21swapper/223:48:282
1882556991713,3cyclictest34-21ksoftirqd/222:50:012
1882556991711,4cyclictest1933177-21cut21:30:182
188255099179,2cyclictest26-21ksoftirqd/122:05:161
1882550991715,1cyclictest3056-21PK-Backend20:23:351
1882550991715,1cyclictest3056-21PK-Backend00:26:081
1882550991715,1cyclictest0-21swapper/121:21:061
1882550991713,3cyclictest1920181-21cpuspeed20:55:131
1882560991614,1cyclictest3056-21PK-Backend22:50:183
1882556991612,3cyclictest1887547-21sh19:20:142
1882556991610,5cyclictest34-21ksoftirqd/222:25:112
1882550991614,1cyclictest0-21swapper/119:55:561
1882550991612,2cyclictest26-21ksoftirqd/100:35:001
139099160,13rtkit-daemon0-21swapper/322:30:033
139099160,13rtkit-daemon0-21swapper/322:30:023
188255099149,4cyclictest1894480-21hddtemp_smartct19:40:151
188255699127,3cyclictest436-20systemd-journal22:35:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional