You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 10:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Mar 22, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190822722610,9sleep00-21swapper/019:05:130
201154421180,3sleep33056-21PK-Backend23:55:123
196757221140,3sleep10-21swapper/121:50:151
199427321030,2sleep33056-21PK-Backend23:05:173
139099850,52rtkit-daemon0-21swapper/319:06:523
190931326740,21sleep20-21swapper/219:05:322
190957026344,13sleep10-21swapper/119:08:441
19568902520,4sleep10-21swapper/121:20:161
139099480,9rtkit-daemon3056-21PK-Backend23:38:533
19528262440,4sleep00-21swapper/021:10:010
19528262440,4sleep00-21swapper/021:10:010
19126372420,2sleep20-21swapper/219:15:142
19479422360,5chrt3056-21PK-Backend20:55:163
19854332310,1sleep30-21swapper/322:40:173
1909819992818,8cyclictest436-20systemd-journal23:25:003
19498072240,4chrt0-21swapper/221:00:172
1909809992416,7cyclictest0-21swapper/019:45:120
1909815992321,1cyclictest0-21swapper/222:17:442
1909815992320,2cyclictest341ksoftirqd/219:34:382
1909815992319,3cyclictest2019-21gmain20:55:012
190981599231,3cyclictest1973777-21munin-run22:10:002
190981399231,20cyclictest1931351-21munin-run20:10:001
139099230,16rtkit-daemon3056-21PK-Backend19:27:253
139099230,16rtkit-daemon0-21swapper/320:15:213
19397612220,10sleep234-21ksoftirqd/220:30:212
190981999222,6cyclictest1936205-21python320:20:203
1909819992220,1cyclictest3056-21PK-Backend22:21:323
1909819992219,2cyclictest3056-21PK-Backend19:45:383
1909819992217,3cyclictest2015889-21users00:05:213
1909819992212,9cyclictest978-21snmpd22:00:393
1909815992220,1cyclictest3056-21PK-Backend21:30:252
1909815992220,1cyclictest0-21swapper/223:13:062
1909815992220,1cyclictest0-21swapper/222:28:322
1909815992220,1cyclictest0-21swapper/221:29:242
1909815992220,1cyclictest0-21swapper/219:57:102
1909815992219,2cyclictest0-21swapper/222:42:382
1909815992213,7cyclictest1931338-21perf20:09:592
190981599220,5cyclictest0-21swapper/219:35:332
1909809992211,9cyclictest1918475-21cat19:30:210
139099220,15rtkit-daemon3056-21PK-Backend21:41:093
139099220,15rtkit-daemon3056-21PK-Backend19:11:353
139099220,15rtkit-daemon0-21swapper/321:52:143
139099220,15rtkit-daemon0-21swapper/219:40:102
19705932210,12sleep013-21ksoftirqd/022:00:130
190981999219,11cyclictest42-21ksoftirqd/322:40:013
1909819992119,1cyclictest3056-21PK-Backend23:51:023
1909819992119,1cyclictest0-21swapper/322:53:093
1909819992119,1cyclictest0-21swapper/320:28:233
1909819992119,1cyclictest0-21swapper/320:11:063
1909819992119,1cyclictest0-21swapper/300:38:403
1909819992119,1cyclictest0-21swapper/300:30:433
1909819992117,2cyclictest3056-21PK-Backend22:55:123
1909819992113,6cyclictest436-20systemd-journal22:10:003
190981999210,20cyclictest3056-21PK-Backend20:08:433
190981999210,20cyclictest0-21swapper/322:14:373
1909815992119,1cyclictest0-21swapper/223:18:272
1909815992119,1cyclictest0-21swapper/223:09:492
1909815992119,1cyclictest0-21swapper/222:23:392
1909815992119,1cyclictest0-21swapper/222:11:012
1909815992119,1cyclictest0-21swapper/221:35:052
1909815992119,1cyclictest0-21swapper/221:12:482
1909815992119,1cyclictest0-21swapper/221:12:482
1909815992119,1cyclictest0-21swapper/221:07:202
1909815992119,1cyclictest0-21swapper/221:07:202
1909815992119,1cyclictest0-21swapper/220:26:592
1909815992119,1cyclictest0-21swapper/220:21:442
1909815992119,1cyclictest0-21swapper/220:04:552
1909815992119,1cyclictest0-21swapper/200:09:312
1909815992118,2cyclictest0-21swapper/220:45:442
190981599210,20cyclictest0-21swapper/222:55:432
190981599210,20cyclictest0-21swapper/221:51:082
190981599210,20cyclictest0-21swapper/221:45:262
190981599210,20cyclictest0-21swapper/220:38:272
1909813992113,6cyclictest436-20systemd-journal23:35:001
1909813992112,7cyclictest1986205-21cron22:45:001
190980999210,20cyclictest0-21swapper/020:28:450
139099210,15rtkit-daemon0-21swapper/222:37:462
139099210,15rtkit-daemon0-21swapper/221:42:362
19378242200,4sleep13056-21PK-Backend20:25:181
1909819992018,1cyclictest3056-21PK-Backend23:48:593
1909819992018,1cyclictest3056-21PK-Backend21:19:203
1909819992018,1cyclictest3056-21PK-Backend19:43:283
1909819992018,1cyclictest3056-21PK-Backend19:19:353
1909819992018,1cyclictest0-21swapper/322:33:083
1909819992018,1cyclictest0-21swapper/319:31:043
1909819992018,1cyclictest0-21swapper/300:24:343
1909819992015,3cyclictest1940917-21df20:35:133
190981999200,19cyclictest3056-21PK-Backend23:44:543
190981999200,19cyclictest3056-21PK-Backend23:10:533
190981999200,19cyclictest3056-21PK-Backend21:32:523
190981999200,19cyclictest3056-21PK-Backend21:12:183
190981999200,19cyclictest3056-21PK-Backend21:12:173
190981999200,19cyclictest3056-21PK-Backend21:05:143
190981999200,19cyclictest3056-21PK-Backend21:05:143
190981999200,19cyclictest3056-21PK-Backend20:52:583
190981999200,19cyclictest3056-21PK-Backend19:58:013
190981999200,19cyclictest0-21swapper/322:16:393
190981999200,19cyclictest0-21swapper/300:26:373
1909815992018,1cyclictest0-21swapper/222:30:352
1909815992018,1cyclictest0-21swapper/221:56:482
1909815992018,1cyclictest0-21swapper/220:13:532
1909815992018,1cyclictest0-21swapper/219:48:522
1909815992018,1cyclictest0-21swapper/200:02:212
1909815992012,6cyclictest1986652-21expr22:45:102
1909815992011,8cyclictest2027545-21sed00:40:012
190981599200,1cyclictest0-21swapper/200:14:382
190981599200,19cyclictest0-21swapper/222:52:382
190981599200,19cyclictest0-21swapper/220:58:152
190981599200,19cyclictest0-21swapper/220:44:422
190981599200,19cyclictest0-21swapper/219:29:192
190981599200,19cyclictest0-21swapper/219:11:082
190981399203,12cyclictest1995425-21apt23:10:091
1909813992018,1cyclictest3056-21PK-Backend20:39:341
1909813992018,1cyclictest0-21swapper/123:56:111
1909813992018,1cyclictest0-21swapper/123:48:261
1909813992018,1cyclictest0-21swapper/121:31:461
1909813992018,1cyclictest0-21swapper/100:30:171
190981399200,19cyclictest3056-21PK-Backend22:27:571
190981399200,19cyclictest3056-21PK-Backend22:17:141
190981399200,19cyclictest0-21swapper/123:28:541
190981399200,19cyclictest0-21swapper/123:00:131
190981399200,19cyclictest0-21swapper/121:07:191
190981399200,19cyclictest0-21swapper/121:07:181
190981399200,19cyclictest0-21swapper/120:04:051
190981399200,19cyclictest0-21swapper/119:49:451
190981399200,19cyclictest0-21swapper/119:28:021
190981399200,19cyclictest0-21swapper/100:15:101
1909809992018,1cyclictest0-21swapper/022:38:360
1909809992018,1cyclictest0-21swapper/019:44:240
1909809992018,1cyclictest0-21swapper/000:14:090
1909809992015,3cyclictest1927863-21sh20:00:000
1909809992015,3cyclictest1919660-21kernelversion19:35:170
1909809992012,6cyclictest1965682-21memory21:45:170
1909809992012,6cyclictest1962490-21python321:35:190
190980999200,19cyclictest3056-21PK-Backend21:51:040
190980999200,19cyclictest0-21swapper/023:55:090
190980999200,19cyclictest0-21swapper/023:30:210
190980999200,19cyclictest0-21swapper/023:10:270
190980999200,19cyclictest0-21swapper/020:42:510
190980999200,19cyclictest0-21swapper/020:30:340
190980999200,19cyclictest0-21swapper/020:23:310
190980999200,19cyclictest0-21swapper/020:17:300
190980999200,19cyclictest0-21swapper/019:10:510
190980999200,19cyclictest0-21swapper/000:39:160
190980999200,19cyclictest0-21swapper/000:32:190
139099200,14rtkit-daemon3056-21PK-Backend23:27:302
139099200,14rtkit-daemon0-21swapper/221:21:162
1909819991917,1cyclictest3056-21PK-Backend21:21:093
1909819991917,1cyclictest3056-21PK-Backend21:02:373
1909819991917,1cyclictest3056-21PK-Backend19:20:243
1909819991917,1cyclictest0-21swapper/322:28:153
1909819991915,2cyclictest2018023-21/usr/sbin/munin00:15:003
1909819991914,4cyclictest42-21ksoftirqd/322:50:013
1909819991912,5cyclictest2001948-21cat23:25:213
1909819991911,6cyclictest1998910-21cron23:20:003
190981999190,1cyclictest3056-21PK-Backend21:27:243
190981999190,1cyclictest3056-21PK-Backend19:52:083
190981999190,1cyclictest0-21swapper/320:31:003
190981999190,18cyclictest3056-21PK-Backend21:48:263
190981999190,18cyclictest3056-21PK-Backend20:47:433
190981999190,18cyclictest0-21swapper/321:55:573
1909815991917,1cyclictest0-21swapper/223:45:452
1909815991917,1cyclictest0-21swapper/223:42:402
1909815991917,1cyclictest0-21swapper/223:03:402
1909815991917,1cyclictest0-21swapper/222:02:292
1909815991917,1cyclictest0-21swapper/221:18:152
1909815991917,1cyclictest0-21swapper/219:52:382
1909815991912,5cyclictest1933904-21cat20:15:122
190981599190,1cyclictest0-21swapper/219:20:472
190981599190,18cyclictest0-21swapper/200:28:572
190981399193,14cyclictest1925326-21tail19:50:171
1909813991917,1cyclictest0-21swapper/122:23:301
1909813991917,1cyclictest0-21swapper/121:15:431
1909813991917,1cyclictest0-21swapper/119:36:541
1909813991917,1cyclictest0-21swapper/119:31:471
1909813991913,4cyclictest1911472-21sh19:14:591
1909813991912,5cyclictest1912623-21awk19:15:151
1909813991911,6cyclictest883-21dbus-daemon22:50:031
1909813991911,6cyclictest883-21dbus-daemon22:35:001
1909813991911,6cyclictest1983288-21expr22:35:141
1909813991911,6cyclictest1963114-21/usr/sbin/munin21:40:001
1909813991911,6cyclictest1926802-21idleruntime19:55:141
190981399190,1cyclictest3056-21PK-Backend22:46:411
190981399190,1cyclictest3056-21PK-Backend00:22:131
190981399190,1cyclictest0-21swapper/122:57:241
190981399190,1cyclictest0-21swapper/121:12:181
190981399190,1cyclictest0-21swapper/121:12:181
1909809991917,1cyclictest3056-21PK-Backend00:25:230
1909809991917,1cyclictest0-21swapper/023:50:490
1909809991917,1cyclictest0-21swapper/023:15:480
1909809991917,1cyclictest0-21swapper/022:47:350
1909809991917,1cyclictest0-21swapper/022:29:370
1909809991917,1cyclictest0-21swapper/021:59:230
1909809991917,1cyclictest0-21swapper/021:27:120
1909809991917,1cyclictest0-21swapper/021:18:060
1909809991917,1cyclictest0-21swapper/019:18:080
1909809991916,2cyclictest3056-21PK-Backend00:05:090
1909809991916,2cyclictest1956224-21kworker/0:122:20:000
1909809991912,5cyclictest1956697-21expr21:20:130
1909809991911,6cyclictest1993897-21cat23:05:110
1909809991911,6cyclictest1946891-21sshd20:50:560
1909809991911,6cyclictest1928713-21memory20:00:180
190980999190,1cyclictest0-21swapper/022:40:390
190980999190,1cyclictest0-21swapper/021:41:450
190980999190,1cyclictest0-21swapper/021:07:240
190980999190,1cyclictest0-21swapper/021:07:240
190980999190,1cyclictest0-21swapper/020:57:440
190980999190,1cyclictest0-21swapper/020:38:250
190980999190,1cyclictest0-21swapper/000:19:150
190980999190,18cyclictest0-21swapper/022:22:550
1909819991816,1cyclictest3056-21PK-Backend21:38:193
1909819991816,1cyclictest0-21swapper/300:00:143
190981999180,1cyclictest3056-21PK-Backend23:01:543
1909815991816,1cyclictest0-21swapper/223:56:122
1909815991814,3cyclictest2009590-21apt23:50:122
1909815991811,5cyclictest2005089-21awk23:35:182
1909815991810,6cyclictest2000127-21sed23:20:212
1909813991816,1cyclictest3056-21PK-Backend20:14:331
1909813991816,1cyclictest0-21swapper/123:43:331
1909813991816,1cyclictest0-21swapper/123:09:331
1909813991811,5cyclictest2005232-21gzip23:35:201
1909813991811,5cyclictest1935787-21awk20:20:141
190981399180,1cyclictest3056-21PK-Backend21:57:341
190981399180,1cyclictest0-21swapper/119:20:451
1909809991816,1cyclictest0-21swapper/023:45:420
1909809991816,1cyclictest0-21swapper/023:36:300
1909809991816,1cyclictest0-21swapper/022:05:050
1909809991816,1cyclictest0-21swapper/020:49:550
1909809991816,1cyclictest0-21swapper/020:14:520
1909809991816,1cyclictest0-21swapper/000:00:020
1909809991814,2cyclictest1931336-21sh20:09:590
1909809991813,4cyclictest1992441-21kernelversion23:00:170
1909809991813,3cyclictest1990398-21tr22:55:110
1909809991812,4cyclictest2002575-21cat23:30:010
1909809991812,4cyclictest1915449-21basename19:25:010
1909809991811,5cyclictest1982786-21date22:35:010
1909809991811,5cyclictest1916001-21expr19:25:130
1909809991810,6cyclictest1949228-21apt21:00:110
190980999180,17cyclictest0-21swapper/019:53:500
190981999179,6cyclictest1942289-21apt20:40:083
1909819991715,1cyclictest3056-21PK-Backend20:03:223
1909819991715,1cyclictest3056-21PK-Backend19:35:033
1909815991711,5cyclictest2020435-21mktemp00:20:022
1909815991710,5cyclictest436-20systemd-journal00:33:562
1909815991710,5cyclictest2020027-21/usr/sbin/munin00:20:002
190981399179,6cyclictest1999156-21cat23:20:111
190981399179,6cyclictest1946007-21expr20:50:111
1909813991715,1cyclictest3056-21PK-Backend20:16:561
1909813991715,1cyclictest0-21swapper/123:50:291
1909813991715,1cyclictest0-21swapper/121:45:381
1909813991714,2cyclictest3056-21PK-Backend00:03:271
1909813991711,5cyclictest1998912-21/usr/sbin/munin23:20:001
1909813991711,5cyclictest1947866-21diskstats20:55:141
1909813991710,4cyclictest1922734-21cat19:45:021
1909809991716,1cyclictest0-21swapper/022:53:300
1909809991715,1cyclictest0-21swapper/000:23:200
1909809991710,5cyclictest2006706-21sed23:40:160
1909809991710,5cyclictest1960126-21expr21:30:110
1909819991610,4cyclictest2018477-21touch00:15:013
1909815991614,1cyclictest0-21swapper/223:30:102
190981399169,5cyclictest1958733-21cut21:25:181
190981399161,9cyclictest1949600-21/usr/sbin/munin21:00:131
1909813991612,3cyclictest2022922-21needreboot00:25:171
1909813991611,4cyclictest1972052-21munin-plugin-st22:05:001
1909809991614,1cyclictest0-21swapper/022:11:330
1909809991611,3cyclictest1999142-21/usr/sbin/munin23:20:130
1909819991511,2cyclictest42-21ksoftirqd/323:30:423
190981399158,5cyclictest1963958-21wc21:40:161
1909813991512,2cyclictest3056-21PK-Backend00:37:201
1909813991412,1cyclictest3056-21PK-Backend22:13:021
1909813991412,1cyclictest3056-21PK-Backend00:07:051
1909813991410,3cyclictest3056-21PK-Backend20:40:111
1909813991410,3cyclictest2016920-21df00:10:141
190981399139,3cyclictest3056-21PK-Backend22:06:431
190981399137,5cyclictest1945013-21cat20:45:201
190981399116,3cyclictest19402202sleep120:34:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional