You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 09:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Apr 06, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
203804521260,3sleep20-21swapper/222:20:212
205344521060,3sleep13056-21PK-Backend23:05:191
196936428139,14sleep30-21swapper/319:09:043
139099740,64rtkit-daemon0-21swapper/119:08:591
196941527338,9sleep20-21swapper/219:09:442
196795426243,13sleep00-21swapper/019:05:030
20067492560,19sleep234-21ksoftirqd/220:55:002
20484902470,2sleep2331rcuc/222:50:202
19845512420,4sleep10-21swapper/119:50:111
20782972340,1sleep02078299-21grep00:15:190
20387302340,3sleep20-21swapper/222:25:102
139099340,2rtkit-daemon0-21swapper/119:57:031
20145862280,3chrt0-21swapper/221:15:172
1969575992611,13cyclictest2052405-21unin-run23:05:002
196957099254,10cyclictest2047174-21perf22:54:560
196957099240,22cyclictest2052982-21expr23:05:120
1969579992315,6cyclictest2045143-21sshd22:41:203
1969570992317,4cyclictest1982408-21basename19:44:590
196957099230,21cyclictest2072180-21latency_hist00:00:020
139099230,4rtkit-daemon1389-21rtkit-daemon19:10:081
1969579992220,1cyclictest0-21swapper/322:38:293
1969579992220,1cyclictest0-21swapper/322:05:103
1969579992219,2cyclictest0-21swapper/323:27:113
1969579992215,5cyclictest436-20systemd-journal22:25:003
196957999220,20cyclictest3056-21PK-Backend19:45:453
1969575992220,1cyclictest0-21swapper/221:22:122
1969575992220,1cyclictest0-21swapper/221:14:292
1969575992220,1cyclictest0-21swapper/220:07:172
1969575992220,1cyclictest0-21swapper/219:59:252
1969575992220,1cyclictest0-21swapper/219:32:142
1969573992220,1cyclictest3056-21PK-Backend23:46:351
1969573992220,1cyclictest3056-21PK-Backend22:52:261
1969573992220,1cyclictest3056-21PK-Backend20:49:521
1969573992220,1cyclictest3056-21PK-Backend20:49:521
196957399220,20cyclictest3056-21PK-Backend00:31:161
1969570992211,8cyclictest1971234-21apt-get19:15:000
139099220,4rtkit-daemon1389-21rtkit-daemon20:40:211
139099220,15rtkit-daemon3056-21PK-Backend23:14:061
19763412210,5sleep10-21swapper/119:25:171
196957999214,8cyclictest141rcu_preempt21:25:213
196957999214,15cyclictest2076389-21head00:10:203
1969579992119,1cyclictest3056-21PK-Backend19:18:353
1969579992119,1cyclictest0-21swapper/322:11:053
1969579992119,1cyclictest0-21swapper/321:57:073
1969579992119,1cyclictest0-21swapper/320:41:563
1969579992119,1cyclictest0-21swapper/320:36:423
1969579992119,1cyclictest0-21swapper/320:24:053
1969579992119,1cyclictest0-21swapper/320:09:043
1969579992119,1cyclictest0-21swapper/319:58:033
1969579992119,1cyclictest0-21swapper/319:32:073
1969579992119,1cyclictest0-21swapper/300:29:563
1969579992119,1cyclictest0-21swapper/300:04:013
1969579992118,2cyclictest3056-21PK-Backend00:16:583
1969579992112,7cyclictest883-21dbus-daemon00:00:003
196957999210,20cyclictest0-21swapper/323:22:433
196957999210,20cyclictest0-21swapper/321:19:343
196957999210,19cyclictest3056-21PK-Backend19:11:523
1969575992119,1cyclictest0-21swapper/223:13:392
1969575992119,1cyclictest0-21swapper/222:17:562
1969575992119,1cyclictest0-21swapper/221:50:312
1969575992119,1cyclictest0-21swapper/221:45:312
1969575992119,1cyclictest0-21swapper/221:45:312
1969575992119,1cyclictest0-21swapper/220:56:302
1969575992119,1cyclictest0-21swapper/219:47:222
1969575992119,1cyclictest0-21swapper/200:29:232
1969575992115,4cyclictest886-21NetworkManager20:15:002
196957599210,20cyclictest0-21swapper/223:24:202
196957599210,20cyclictest0-21swapper/220:43:332
196957599210,20cyclictest0-21swapper/219:44:522
196957599210,20cyclictest0-21swapper/219:21:332
196957599210,20cyclictest0-21swapper/200:10:442
1969573992119,1cyclictest3056-21PK-Backend23:50:131
1969573992119,1cyclictest3056-21PK-Backend23:37:281
1969573992119,1cyclictest3056-21PK-Backend23:27:481
1969573992119,1cyclictest3056-21PK-Backend22:40:511
1969573992119,1cyclictest3056-21PK-Backend21:17:501
1969573992119,1cyclictest3056-21PK-Backend20:22:471
1969573992119,1cyclictest3056-21PK-Backend20:17:481
1969573992119,1cyclictest0-21swapper/121:28:311
1969573992119,1cyclictest0-21swapper/120:56:201
1969573992119,1cyclictest0-21swapper/119:34:071
196957399210,20cyclictest3056-21PK-Backend22:47:271
196957399210,20cyclictest3056-21PK-Backend22:24:281
196957399210,20cyclictest3056-21PK-Backend22:00:151
196957399210,20cyclictest3056-21PK-Backend20:37:341
196957399210,20cyclictest0-21swapper/123:30:251
196957399210,20cyclictest0-21swapper/121:21:361
196957399210,20cyclictest0-21swapper/119:21:231
196957399210,19cyclictest3056-21PK-Backend20:33:221
1969570992113,6cyclictest2164-21gdbus20:55:580
1969570992113,6cyclictest2010028-21awk21:00:230
1969570992110,8cyclictest883-21dbus-daemon19:55:570
196957099210,20cyclictest0-21swapper/021:50:510
196957099210,20cyclictest0-21swapper/020:39:320
139099210,4rtkit-daemon1389-21rtkit-daemon20:06:100
139099210,17rtkit-daemon1969565-21cyclictest21:41:551
139099210,14rtkit-daemon3056-21PK-Backend23:43:591
196957999203,15cyclictest0-21swapper/320:30:003
1969579992018,1cyclictest3056-21PK-Backend20:58:533
1969579992018,1cyclictest3056-21PK-Backend19:25:453
1969579992018,1cyclictest0-21swapper/320:15:403
1969579992018,1cyclictest0-21swapper/319:35:243
1969579992018,1cyclictest0-21swapper/319:20:383
1969579992018,1cyclictest0-21swapper/300:36:383
1969579992017,2cyclictest2063044-21munin-plugin-st23:35:003
1969579992015,4cyclictest42-21ksoftirqd/322:05:003
1969579992015,3cyclictest2053310-21grep23:05:173
196957999200,19cyclictest3056-21PK-Backend21:24:553
196957999200,19cyclictest0-21swapper/323:12:023
196957999200,19cyclictest0-21swapper/323:03:103
196957999200,19cyclictest0-21swapper/322:28:303
196957999200,19cyclictest0-21swapper/320:46:223
196957999200,19cyclictest0-21swapper/320:46:223
196957999200,18cyclictest2041789-21sshd22:33:283
1969575992018,1cyclictest0-21swapper/223:57:322
1969575992018,1cyclictest0-21swapper/222:11:132
1969575992018,1cyclictest0-21swapper/221:59:582
1969575992018,1cyclictest0-21swapper/200:33:152
1969575992018,1cyclictest0-21swapper/200:06:042
1969575992013,5cyclictest2156-21gnome-shell20:39:102
1969575992013,5cyclictest2064327-21cat23:35:222
1969575992012,6cyclictest1987881-21apt20:00:122
196957599200,1cyclictest0-21swapper/223:53:402
196957599200,19cyclictest0-21swapper/223:40:442
196957599200,19cyclictest0-21swapper/220:49:002
196957599200,19cyclictest0-21swapper/220:49:002
196957599200,19cyclictest0-21swapper/200:38:562
196957599200,19cyclictest0-21swapper/200:00:232
1969573992018,1cyclictest3056-21PK-Backend23:55:401
1969573992018,1cyclictest3056-21PK-Backend22:57:271
1969573992018,1cyclictest3056-21PK-Backend22:26:511
1969573992018,1cyclictest0-21swapper/122:17:451
1969573992018,1cyclictest0-21swapper/121:30:071
1969573992018,1cyclictest0-21swapper/119:15:291
196957399200,1cyclictest3056-21PK-Backend00:01:071
196957399200,19cyclictest3056-21PK-Backend23:22:341
196957399200,19cyclictest3056-21PK-Backend21:11:071
196957399200,19cyclictest3056-21PK-Backend21:01:411
196957399200,18cyclictest3056-21PK-Backend21:08:101
1969570992018,1cyclictest0-21swapper/022:47:140
1969570992018,1cyclictest0-21swapper/022:20:380
1969570992018,1cyclictest0-21swapper/020:53:250
1969570992018,1cyclictest0-21swapper/020:42:560
1969570992014,3cyclictest2019388-21sh21:30:010
196957099200,1cyclictest0-21swapper/023:42:300
196957099200,19cyclictest3056-21PK-Backend22:32:070
196957099200,19cyclictest3056-21PK-Backend21:10:480
196957099200,19cyclictest0-21swapper/023:21:140
196957099200,19cyclictest0-21swapper/022:39:160
196957099200,19cyclictest0-21swapper/022:27:200
196957099200,19cyclictest0-21swapper/021:20:140
196957099200,19cyclictest0-21swapper/020:46:210
196957099200,19cyclictest0-21swapper/020:46:210
196957099200,19cyclictest0-21swapper/020:22:020
196957099200,19cyclictest0-21swapper/020:10:520
196957099200,19cyclictest0-21swapper/019:37:330
196957099200,19cyclictest0-21swapper/019:23:350
196957099200,19cyclictest0-21swapper/019:19:280
196957099200,19cyclictest0-21swapper/000:02:380
139099200,14rtkit-daemon0-21swapper/000:09:100
1969579991917,1cyclictest0-21swapper/323:43:323
1969579991917,1cyclictest0-21swapper/323:38:523
1969579991917,1cyclictest0-21swapper/323:18:173
1969579991917,1cyclictest0-21swapper/321:06:093
1969579991917,1cyclictest0-21swapper/321:03:133
1969579991917,1cyclictest0-21swapper/320:34:063
1969579991917,1cyclictest0-21swapper/320:00:393
1969579991916,2cyclictest0-21swapper/322:45:193
1969579991914,4cyclictest42-21ksoftirqd/323:50:133
1969579991914,4cyclictest42-21ksoftirqd/321:50:183
1969579991914,4cyclictest42-21ksoftirqd/300:25:013
1969579991911,7cyclictest978-21snmpd00:31:153
196957999190,1cyclictest3056-21PK-Backend19:51:273
196957999190,1cyclictest3056-21PK-Backend19:42:143
196957999190,1cyclictest0-21swapper/321:47:123
196957999190,1cyclictest0-21swapper/321:47:123
196957999190,1cyclictest0-21swapper/321:12:043
1969575991917,1cyclictest0-21swapper/223:27:582
1969575991917,1cyclictest0-21swapper/222:48:172
1969575991917,1cyclictest0-21swapper/222:35:402
1969575991917,1cyclictest0-21swapper/222:31:272
1969575991917,1cyclictest0-21swapper/222:00:452
1969575991917,1cyclictest0-21swapper/221:25:232
1969575991917,1cyclictest0-21swapper/220:28:522
1969575991917,1cyclictest0-21swapper/219:28:572
1969575991917,1cyclictest0-21swapper/219:18:292
1969575991917,1cyclictest0-21swapper/200:18:142
1969575991914,4cyclictest1995189-21diskstats20:20:122
1969575991911,6cyclictest883-21dbus-daemon21:31:012
196957599190,1cyclictest0-21swapper/223:15:282
196957599190,1cyclictest0-21swapper/222:06:412
196957599190,1cyclictest0-21swapper/219:12:342
196957599190,18cyclictest0-21swapper/223:45:222
196957599190,18cyclictest0-21swapper/220:16:152
1969573991917,1cyclictest3056-21PK-Backend00:22:101
1969573991917,1cyclictest0-21swapper/100:18:321
1969573991917,1cyclictest0-21swapper/100:11:151
1969573991916,2cyclictest3056-21PK-Backend23:16:341
196957399190,1cyclictest3056-21PK-Backend23:01:391
196957399190,1cyclictest3056-21PK-Backend20:09:361
196957399190,1cyclictest3056-21PK-Backend19:37:251
196957399190,18cyclictest3056-21PK-Backend20:02:121
1969570991917,1cyclictest0-21swapper/023:39:530
1969570991917,1cyclictest0-21swapper/022:08:140
1969570991917,1cyclictest0-21swapper/021:38:400
1969570991917,1cyclictest0-21swapper/020:26:010
1969570991917,1cyclictest0-21swapper/019:29:570
1969570991917,1cyclictest0-21swapper/000:33:540
1969570991913,5cyclictest1984938-21sed19:50:160
1969570991911,6cyclictest2068233-21cron23:50:000
1969570991911,6cyclictest2050628-21cron23:00:000
196957099190,1cyclictest0-21swapper/023:17:020
196957099190,1cyclictest0-21swapper/023:13:370
196957099190,1cyclictest0-21swapper/020:02:070
196957099190,1cyclictest0-21swapper/000:13:190
196957099190,18cyclictest0-21swapper/021:19:270
139099190,15rtkit-daemon1969565-21cyclictest21:48:520
139099190,15rtkit-daemon1969565-21cyclictest21:48:520
139099190,13rtkit-daemon0-21swapper/023:50:570
20052702180,3chrt0-21swapper/320:50:123
196957999182,14cyclictest2049305-21cut22:55:123
1969579991816,1cyclictest3056-21PK-Backend22:17:483
1969579991813,3cyclictest2073957-21/usr/sbin/munin00:05:103
1969579991813,3cyclictest2066917-21cpuspeed_turbos23:45:123
1969575991816,1cyclictest0-21swapper/222:44:532
1969575991816,1cyclictest0-21swapper/221:37:202
1969575991816,1cyclictest0-21swapper/221:01:512
1969575991816,1cyclictest0-21swapper/200:21:052
1969575991812,4cyclictest2049358-21cpuspeed_turbos22:55:112
1969575991811,5cyclictest1998275-21awk20:30:102
1969573991816,1cyclictest3056-21PK-Backend22:34:151
1969573991816,1cyclictest3056-21PK-Backend22:14:491
1969573991816,1cyclictest3056-21PK-Backend22:07:311
1969573991816,1cyclictest3056-21PK-Backend21:50:141
1969573991816,1cyclictest3056-21PK-Backend00:37:441
1969573991816,1cyclictest3056-21PK-Backend00:25:481
1969573991816,1cyclictest0-21swapper/119:40:581
1969570991816,1cyclictest0-21swapper/023:00:120
1969570991816,1cyclictest0-21swapper/022:43:150
1969570991816,1cyclictest0-21swapper/022:19:500
1969570991816,1cyclictest0-21swapper/022:02:330
1969570991816,1cyclictest0-21swapper/021:56:170
1969570991816,1cyclictest0-21swapper/021:08:380
1969570991810,6cyclictest436-20systemd-journal23:35:000
196957099180,1cyclictest0-21swapper/000:27:400
1969579991712,3cyclictest2048173-21ntp_offset22:50:173
196957999171,14cyclictest2020015-21diskstats21:30:143
196957999170,3cyclictest2021391-21dpkg21:35:113
1969575991715,1cyclictest0-21swapper/219:39:052
1969575991711,5cyclictest2011708-21sed21:05:222
196957399179,6cyclictest2025838-21cat21:45:221
196957399179,6cyclictest2025838-21cat21:45:211
1969573991711,4cyclictest2006710-21systemctl20:55:001
1969570991715,1cyclictest0-21swapper/000:37:320
1969570991710,5cyclictest436-20systemd-journal21:43:270
196957099170,6cyclictest2061314-21munin-run23:30:000
196957999161,13cyclictest1991320-21/usr/sbin/munin20:10:143
196957599169,5cyclictest2053644-21cat23:05:222
196957399169,6cyclictest2029300-21sort21:55:191
196957399168,6cyclictest1983966-21pool-udisksd19:48:041
196957099162,9cyclictest1982668-21apt19:45:110
1969570991614,1cyclictest0-21swapper/019:33:140
1969570991610,5cyclictest2080771-21grep00:25:010
139099160,13rtkit-daemon0-21swapper/121:38:041
1969579991513,1cyclictest3056-21PK-Backend21:40:023
196957599157,5cyclictest2156-21gnome-shell21:42:102
1969575991511,3cyclictest1985870-21dump-pmu-power19:54:592
1969575991510,4cyclictest2156-21gnome-shell23:34:092
196957399159,5cyclictest2043248-21sed22:35:211
196957099158,5cyclictest436-20systemd-journal22:15:000
196957099155,9cyclictest2021112-21cron21:35:010
1969570991510,3cyclictest1999939-21date20:35:010
1969573991412,1cyclictest3056-21PK-Backend00:05:471
1969573991410,3cyclictest3056-21PK-Backend20:27:401
196957399139,3cyclictest3056-21PK-Backend20:14:181
196957099139,3cyclictest3056-21PK-Backend20:15:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional