You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 12:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Wed Jun 03, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
128074523070,10sleep00-21swapper/019:05:150
139099870,75rtkit-daemon0-21swapper/319:08:213
128049726342,14sleep20-21swapper/219:05:142
128170226041,14sleep10-21swapper/119:06:581
13721402510,4sleep10-21swapper/123:30:221
13518222460,3sleep20-21swapper/222:35:022
13915112450,2sleep00-21swapper/000:25:180
13472002450,4sleep13056-21PK-Backend22:20:211
13570662410,4sleep13056-21PK-Backend22:50:111
12870082370,2sleep00-21swapper/019:25:010
139099330,2rtkit-daemon13-21ksoftirqd/022:40:510
13074922300,7sleep0131ksoftirqd/020:30:010
139099270,23rtkit-daemon953-21gmain20:22:570
13787702260,10sleep126-21ksoftirqd/123:50:141
1282153992615,9cyclictest1344385-21cron22:15:002
139099250,4rtkit-daemon1389-21rtkit-daemon23:15:500
13171202250,4chrt3056-21PK-Backend20:55:161
128215599250,7cyclictest3056-21PK-Backend21:51:473
128215599250,6cyclictest3056-21PK-Backend21:49:223
128215599250,6cyclictest3056-21PK-Backend21:49:213
139099240,4rtkit-daemon1389-21rtkit-daemon23:08:220
139099240,4rtkit-daemon1389-21rtkit-daemon22:36:360
139099240,4rtkit-daemon1389-21rtkit-daemon21:23:010
1282155992420,3cyclictest411rcuc/322:13:583
128215599240,6cyclictest3056-21PK-Backend23:44:073
128215599240,6cyclictest3056-21PK-Backend23:44:073
128215599240,6cyclictest0-21swapper/320:30:033
139099230,4rtkit-daemon1389-21rtkit-daemon21:57:521
128215599234,11cyclictest42-21ksoftirqd/320:36:163
128215599232,6cyclictest42-21ksoftirqd/323:50:153
139099220,3rtkit-daemon1389-21rtkit-daemon20:51:200
139099220,3rtkit-daemon1389-21rtkit-daemon20:51:200
13872592220,4sleep13056-21PK-Backend00:15:011
1282155992219,2cyclictest411rcuc/323:05:433
1282155992218,3cyclictest411rcuc/323:04:523
1282155992218,3cyclictest411rcuc/322:32:443
1282155992218,3cyclictest411rcuc/322:18:003
1282155992218,3cyclictest411rcuc/321:39:313
1282155992218,3cyclictest411rcuc/320:03:393
1282155992218,3cyclictest411rcuc/319:43:543
1282155992218,3cyclictest411rcuc/319:17:043
1282155992217,4cyclictest411rcuc/321:06:353
128215599220,6cyclictest3056-21PK-Backend23:26:223
128215599220,6cyclictest3056-21PK-Backend21:57:283
1282153992214,6cyclictest1292297-21if_enp2s019:45:152
1282147992220,1cyclictest3056-21PK-Backend21:37:481
1282147992220,1cyclictest3056-21PK-Backend19:26:071
1282147992219,2cyclictest3056-21PK-Backend22:38:181
1282147992219,2cyclictest3056-21PK-Backend21:34:161
1282147992219,2cyclictest3056-21PK-Backend00:29:241
1282147992214,6cyclictest1374399-21apt-get23:40:001
128214799220,20cyclictest3056-21PK-Backend22:19:051
139099210,4rtkit-daemon1389-21rtkit-daemon23:13:340
139099210,18rtkit-daemon945-21thermald21:33:430
139099210,18rtkit-daemon2637-21gmain00:16:450
139099210,14rtkit-daemon0-21swapper/220:01:002
13438602210,3sleep00-21swapper/022:10:220
12844452210,2sleep00-21swapper/019:15:020
128215599219,3cyclictest42-21ksoftirqd/323:30:223
128215599216,8cyclictest42-21ksoftirqd/319:26:593
128215599214,6cyclictest42-21ksoftirqd/300:20:243
128215599213,7cyclictest42-21ksoftirqd/300:35:233
128215599213,7cyclictest42-21ksoftirqd/300:35:233
1282155992119,1cyclictest0-21swapper/323:14:573
1282155992119,1cyclictest0-21swapper/322:38:393
1282155992119,1cyclictest0-21swapper/320:23:333
1282155992119,1cyclictest0-21swapper/319:22:163
1282155992119,1cyclictest0-21swapper/300:30:263
1282155992119,1cyclictest0-21swapper/300:11:483
1282155992118,2cyclictest411rcuc/322:24:543
1282155992118,2cyclictest411rcuc/320:28:583
1282155992118,2cyclictest411rcuc/320:05:563
1282155992118,2cyclictest411rcuc/319:55:533
1282155992118,2cyclictest411rcuc/300:04:423
128215599211,7cyclictest42-21ksoftirqd/323:37:233
1282155992117,3cyclictest411rcuc/323:17:473
1282155992117,3cyclictest411rcuc/319:32:483
1282155992117,3cyclictest411rcuc/300:25:123
1282155992112,7cyclictest1301251-21cat20:10:163
128215599210,6cyclictest3056-21PK-Backend23:20:333
128215599210,6cyclictest3056-21PK-Backend22:01:213
128215599210,20cyclictest0-21swapper/320:55:183
128215599210,20cyclictest0-21swapper/319:50:073
1282153992114,5cyclictest1321329-21cron21:10:002
128215399210,20cyclictest0-21swapper/223:35:112
1282147992119,1cyclictest3056-21PK-Backend23:56:251
1282147992119,1cyclictest3056-21PK-Backend21:48:231
1282147992119,1cyclictest3056-21PK-Backend21:48:231
1282147992119,1cyclictest3056-21PK-Backend21:08:011
1282147992119,1cyclictest3056-21PK-Backend20:25:011
1282147992119,1cyclictest3056-21PK-Backend19:54:261
1282147992119,1cyclictest0-21swapper/123:46:171
1282147992119,1cyclictest0-21swapper/122:47:441
1282147992119,1cyclictest0-21swapper/121:20:381
1282147992119,1cyclictest0-21swapper/120:31:171
128214799210,20cyclictest3056-21PK-Backend21:28:011
128214799210,19cyclictest3056-21PK-Backend22:30:271
128214799210,19cyclictest3056-21PK-Backend19:48:311
128214799210,19cyclictest3056-21PK-Backend00:08:011
1282143992119,1cyclictest0-21swapper/019:51:540
1282143992114,5cyclictest1-21systemd23:45:010
1282143992114,5cyclictest1-21systemd23:45:010
139099200,3rtkit-daemon1389-21rtkit-daemon22:33:130
139099200,14rtkit-daemon0-21swapper/219:18:442
128215599209,3cyclictest42-21ksoftirqd/323:45:353
128215599208,4cyclictest42-21ksoftirqd/321:15:533
1282155992018,1cyclictest0-21swapper/321:31:413
1282155992017,2cyclictest411rcuc/322:59:193
1282155992017,2cyclictest411rcuc/321:26:563
1282155992017,2cyclictest411rcuc/320:15:103
1282155992017,2cyclictest411rcuc/319:45:593
1282155992017,2cyclictest411rcuc/319:11:023
1282155992017,2cyclictest411rcuc/300:05:153
1282155992017,1cyclictest0-21swapper/321:20:193
1282155992016,3cyclictest411rcuc/320:50:353
1282155992016,3cyclictest411rcuc/320:50:343
1282155992014,5cyclictest42-21ksoftirqd/322:50:123
128215599201,17cyclictest1387610-21expr00:15:153
128215599200,8cyclictest42-21ksoftirqd/322:27:123
128215599200,7cyclictest42-21ksoftirqd/320:46:543
128215599200,1cyclictest3056-21PK-Backend20:40:583
128215599200,1cyclictest0-21swapper/321:41:543
128215599200,19cyclictest0-21swapper/322:45:233
128215599200,19cyclictest0-21swapper/321:14:383
1282153992014,4cyclictest34-21ksoftirqd/220:05:172
1282153992011,7cyclictest1358528-21apt-get22:55:002
128215399200,1cyclictest0-21swapper/222:16:082
128215399200,19cyclictest0-21swapper/223:07:202
128215399200,19cyclictest0-21swapper/221:51:352
128215399200,19cyclictest0-21swapper/219:38:192
1282147992018,1cyclictest3056-21PK-Backend21:17:541
1282147992018,1cyclictest3056-21PK-Backend20:46:171
1282147992018,1cyclictest3056-21PK-Backend20:06:501
1282147992018,1cyclictest3056-21PK-Backend19:59:061
1282147992018,1cyclictest3056-21PK-Backend19:35:541
1282147992018,1cyclictest3056-21PK-Backend00:15:581
1282147992018,1cyclictest0-21swapper/123:40:371
1282147992018,1cyclictest0-21swapper/123:40:371
1282147992018,1cyclictest0-21swapper/120:40:571
1282147992017,2cyclictest3056-21PK-Backend23:19:211
1282147992017,2cyclictest3056-21PK-Backend22:58:381
128214799200,1cyclictest0-21swapper/123:25:081
128214799200,19cyclictest3056-21PK-Backend20:12:441
128214799200,19cyclictest3056-21PK-Backend00:03:561
128214799200,19cyclictest0-21swapper/122:40:281
128214799200,19cyclictest0-21swapper/122:14:121
128214799200,19cyclictest0-21swapper/120:39:201
1282143992018,1cyclictest0-21swapper/023:30:540
1282143992018,1cyclictest0-21swapper/022:00:100
1282143992018,1cyclictest0-21swapper/021:49:560
1282143992018,1cyclictest0-21swapper/021:49:560
1282143992018,1cyclictest0-21swapper/021:01:430
1282143992018,1cyclictest0-21swapper/000:34:080
1282143992017,2cyclictest0-21swapper/022:19:170
1282143992012,6cyclictest1310336-21python320:35:220
128214399200,1cyclictest3056-21PK-Backend21:25:230
128214399200,1cyclictest0-21swapper/022:26:330
128214399200,19cyclictest0-21swapper/023:28:520
128214399200,19cyclictest0-21swapper/021:36:440
128214399200,19cyclictest0-21swapper/020:47:090
128214399200,19cyclictest0-21swapper/020:44:470
128214399200,19cyclictest0-21swapper/020:15:130
128214399200,19cyclictest0-21swapper/019:14:360
128215599194,4cyclictest42-21ksoftirqd/322:42:003
1282155991916,2cyclictest411rcuc/323:57:133
1282155991916,2cyclictest411rcuc/319:36:253
128215599190,6cyclictest3056-21PK-Backend22:05:053
128215599190,1cyclictest0-21swapper/321:01:063
1282153991917,1cyclictest0-21swapper/221:29:452
1282153991917,1cyclictest0-21swapper/219:30:002
1282153991915,3cyclictest978-21snmpd00:35:492
1282153991915,3cyclictest978-21snmpd00:35:492
1282153991914,4cyclictest978-21snmpd20:21:482
1282153991914,3cyclictest34-21ksoftirqd/220:45:002
1282153991914,3cyclictest1330913-21diskstats21:35:132
1282153991914,3cyclictest1330279-21ls21:35:012
1282153991913,5cyclictest1354038-21cat22:40:152
1282153991912,5cyclictest1317481-21awk20:55:202
1282153991910,7cyclictest436-20systemd-journal22:35:002
128215399190,1cyclictest0-21swapper/222:48:192
128215399190,1cyclictest0-21swapper/220:25:242
128215399190,1cyclictest0-21swapper/219:53:202
128215399190,1cyclictest0-21swapper/219:43:472
128215399190,17cyclictest0-21swapper/223:12:392
1282147991917,1cyclictest3056-21PK-Backend21:41:201
1282147991917,1cyclictest3056-21PK-Backend20:00:071
1282147991917,1cyclictest3056-21PK-Backend19:43:371
1282147991917,1cyclictest3056-21PK-Backend19:16:071
1282147991917,1cyclictest3056-21PK-Backend00:33:161
1282147991917,1cyclictest0-21swapper/122:04:041
1282147991913,4cyclictest1367857-21cpuspeed_turbos23:20:121
1282147991911,6cyclictest1284002-21sh19:14:591
1282143991917,1cyclictest0-21swapper/021:50:440
1282143991917,1cyclictest0-21swapper/021:16:240
1282143991917,1cyclictest0-21swapper/021:10:550
1282143991917,1cyclictest0-21swapper/020:00:530
1282143991917,1cyclictest0-21swapper/019:56:000
1282143991917,1cyclictest0-21swapper/000:39:290
1282143991917,1cyclictest0-21swapper/000:39:290
128214399190,7cyclictest12996462cut20:05:240
128214399190,1cyclictest3056-21PK-Backend00:01:230
128214399190,1cyclictest0-21swapper/023:47:110
128214399190,1cyclictest0-21swapper/021:57:270
128214399190,18cyclictest0-21swapper/023:25:000
128214399190,18cyclictest0-21swapper/019:37:340
13409602180,3sleep10-21swapper/122:05:011
1282153991816,1cyclictest0-21swapper/222:22:592
1282153991816,1cyclictest0-21swapper/200:00:062
1282153991815,2cyclictest0-21swapper/223:32:342
1282153991815,2cyclictest0-21swapper/223:26:332
1282153991812,5cyclictest1359579-21sed22:55:182
1282153991812,5cyclictest1309195-21cron20:35:002
1282153991812,5cyclictest1282946-21basename19:10:152
1282153991811,6cyclictest1381019-21sed23:55:222
1282153991811,6cyclictest1303490-21sed20:15:222
1282153991811,5cyclictest1390812-21dpkg00:25:122
1282153991811,5cyclictest1313091-21expr20:45:142
1282153991810,6cyclictest1387739-21which00:15:182
128215399180,6cyclictest0-21swapper/221:50:002
128215399180,6cyclictest0-21swapper/221:49:592
128215399180,1cyclictest0-21swapper/223:17:122
128215399180,17cyclictest0-21swapper/220:13:342
128215399180,16cyclictest1390530-21cron00:25:012
1282147991816,1cyclictest3056-21PK-Backend21:01:441
1282147991816,1cyclictest0-21swapper/121:52:081
1282143991816,1cyclictest3056-21PK-Backend19:30:250
1282143991816,1cyclictest0-21swapper/022:06:120
1282143991816,1cyclictest0-21swapper/021:44:010
1282143991816,1cyclictest0-21swapper/000:22:060
1282143991813,4cyclictest13-21ksoftirqd/022:55:010
1282143991812,4cyclictest2156-21gnome-shell20:14:090
1282143991811,5cyclictest1290833-21awk19:40:140
128215399179,6cyclictest1318461-21expr21:00:142
128215399179,6cyclictest1310216-21ntp_states20:35:202
128215399178,8cyclictest1295794-21cat19:55:172
128215399178,7cyclictest1316156-21basename20:55:002
128215399178,7cyclictest1316156-21basename20:54:592
128215399177,8cyclictest0-21swapper/223:44:592
128215399177,8cyclictest0-21swapper/223:44:592
128215399172,13cyclictest1095-21runrttasks21:12:072
1282153991715,1cyclictest0-21swapper/222:03:252
1282153991712,3cyclictest2156-21gnome-shell22:26:092
1282153991712,3cyclictest1326660-21cat21:25:002
1282153991712,3cyclictest1287498-21latency_hist19:30:012
128215399171,12cyclictest1378397-21apt23:50:132
1282153991710,5cyclictest1392699-21expr00:30:142
1282153991710,5cyclictest1384843-21sed00:05:252
128215399170,16cyclictest0-21swapper/221:16:002
1282147991715,1cyclictest3056-21PK-Backend21:13:141
1282147991715,1cyclictest3056-21PK-Backend00:22:401
128214799171,11cyclictest1395758-21perf00:40:001
128214799171,11cyclictest1395758-21perf00:40:001
128214399179,6cyclictest436-20systemd-journal23:35:010
1282143991715,1cyclictest0-21swapper/022:48:030
1282143991715,1cyclictest0-21swapper/022:20:040
1282143991715,1cyclictest0-21swapper/019:27:200
1282143991713,3cyclictest1321343-21latency_hist21:10:000
1282143991710,5cyclictest1362407-21grep23:05:000
13775002160,4sleep20-21swapper/223:45:192
128215399169,5cyclictest1386626-21cat00:10:222
128215399169,5cyclictest1332676-21which21:40:152
1282153991614,1cyclictest0-21swapper/219:20:552
1282153991612,3cyclictest1341176-21/usr/sbin/munin22:05:212
128214799162,8cyclictest1095-21runrttasks22:25:171
1282147991614,1cyclictest3056-21PK-Backend23:05:151
1282147991614,1cyclictest3056-21PK-Backend19:20:261
1282143991614,1cyclictest0-21swapper/022:59:250
1282143991614,1cyclictest0-21swapper/020:34:190
1282143991612,3cyclictest1317058-21tail20:55:150
1282143991610,5cyclictest1381589-21apt-get23:59:590
1282143991610,4cyclictest1293492-21grep19:50:000
139099150,12rtkit-daemon3056-21PK-Backend20:53:041
139099150,12rtkit-daemon3056-21PK-Backend20:53:041
128215399159,4cyclictest1361491-21ntpq23:00:182
128215399150,1cyclictest0-21swapper/221:55:072
1282147991513,1cyclictest3056-21PK-Backend20:20:561
1282147991513,1cyclictest3056-21PK-Backend19:32:021
128214799150,6cyclictest3056-21PK-Backend23:00:121
1282143991513,1cyclictest0-21swapper/023:51:030
1282143991513,1cyclictest0-21swapper/000:10:500
1282143991410,3cyclictest1095-21runrttasks00:05:310
128215399131,10cyclictest1368394-21needreboot23:20:192
1282147991311,1cyclictest3056-21PK-Backend20:15:491
128214799128,3cyclictest3056-21PK-Backend23:14:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional