You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 17:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 10, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33770923230,9sleep30-21swapper/307:10:013
3365582930,4sleep20-21swapper/207:05:142
4147992660,4sleep2331rcuc/210:45:162
4477212650,3sleep33056-21PK-Backend12:20:013
33740926445,13sleep00-21swapper/007:06:580
33688026442,14sleep10-21swapper/107:05:211
4320972540,5sleep13056-21PK-Backend11:35:001
4229462490,2sleep00-21swapper/011:10:010
4107442450,4sleep30-21swapper/310:35:113
139099430,4rtkit-daemon1389-21rtkit-daemon07:48:453
139099400,3rtkit-daemon1389-21rtkit-daemon09:02:203
139099390,4rtkit-daemon1389-21rtkit-daemon12:08:233
4397642380,4sleep2439768-21kernelversion11:55:142
4267132370,4sleep20-21swapper/211:20:012
139099290,20rtkit-daemon3056-21PK-Backend07:10:090
337804992511,12cyclictest440726-21cron12:00:013
33780499250,23cyclictest1389-21rtkit-daemon08:43:323
33780499240,23cyclictest1389-21rtkit-daemon07:44:233
337800992417,6cyclictest359336-21idleruntime-cro08:10:002
139099240,3rtkit-daemon1389-21rtkit-daemon08:45:562
139099240,16rtkit-daemon3056-21PK-Backend08:47:461
139099240,15rtkit-daemon3056-21PK-Backend07:52:051
337800992317,4cyclictest34-21ksoftirqd/208:15:012
33780099231,20cyclictest1317-21systemd08:36:242
337790992314,8cyclictest379582-21sed09:05:150
139099230,4rtkit-daemon1389-21rtkit-daemon11:00:212
139099230,18rtkit-daemon337785-21cyclictest07:42:512
3444992220,3sleep3411rcuc/307:25:213
337804992214,6cyclictest346948-21anacron07:32:463
337804992214,6cyclictest342811-21python307:20:203
33780499220,21cyclictest1389-21rtkit-daemon12:01:583
33780499220,21cyclictest1389-21rtkit-daemon11:37:163
33780499220,21cyclictest1389-21rtkit-daemon11:37:163
337800992220,1cyclictest0-21swapper/211:06:502
337800992220,1cyclictest0-21swapper/210:18:442
337800992220,1cyclictest0-21swapper/210:00:472
337800992220,1cyclictest0-21swapper/209:26:192
337800992220,1cyclictest0-21swapper/208:54:022
337800992220,1cyclictest0-21swapper/208:25:092
337800992220,1cyclictest0-21swapper/208:17:382
337800992220,1cyclictest0-21swapper/208:02:042
337800992219,2cyclictest0-21swapper/211:48:482
33780099221,10cyclictest435551-21perf11:44:592
337795992214,6cyclictest387974-21df09:30:131
337790992220,1cyclictest0-21swapper/007:19:280
139099220,17rtkit-daemon978-21snmpd11:27:382
139099220,15rtkit-daemon3056-21PK-Backend09:19:061
139099220,15rtkit-daemon0-21swapper/009:34:210
337804992116,3cyclictest0-21swapper/309:05:223
337804992114,5cyclictest1095-21runrttasks09:44:593
337804992113,6cyclictest436-20systemd-journal09:40:003
337804992113,6cyclictest395392-21expr09:50:163
33780499210,20cyclictest3056-21PK-Backend08:10:213
33780499210,20cyclictest1389-21rtkit-daemon10:53:573
33780499210,19cyclictest3056-21PK-Backend10:12:273
33780499210,19cyclictest3056-21PK-Backend08:55:093
33780499210,19cyclictest3056-21PK-Backend08:33:323
33780099212,18cyclictest1095-21runrttasks07:32:112
337800992119,1cyclictest0-21swapper/212:27:152
337800992119,1cyclictest0-21swapper/211:20:302
337800992119,1cyclictest0-21swapper/210:41:492
337800992119,1cyclictest0-21swapper/210:41:492
337800992119,1cyclictest0-21swapper/210:34:202
337800992119,1cyclictest0-21swapper/210:23:242
337800992119,1cyclictest0-21swapper/210:10:402
337800992119,1cyclictest0-21swapper/209:52:412
337800992119,1cyclictest0-21swapper/209:45:382
337800992119,1cyclictest0-21swapper/209:23:422
337800992119,1cyclictest0-21swapper/208:40:372
337800992116,3cyclictest381413-21ntp_states09:10:162
337800992112,7cyclictest883-21dbus-daemon07:15:012
33780099210,20cyclictest0-21swapper/212:30:052
33780099210,20cyclictest0-21swapper/212:30:042
33780099210,20cyclictest0-21swapper/211:30:172
33780099210,20cyclictest0-21swapper/207:58:242
33780099210,19cyclictest0-21swapper/209:15:172
33779599219,3cyclictest4237412ls11:10:171
337795992114,5cyclictest345179-21systemd-run07:30:011
337795992113,6cyclictest436-20systemd-journal10:34:351
33779599210,20cyclictest3056-21PK-Backend09:57:061
33779599210,20cyclictest3056-21PK-Backend08:44:001
33779599210,20cyclictest0-21swapper/109:45:521
33779599210,19cyclictest3056-21PK-Backend11:52:321
33779599210,19cyclictest3056-21PK-Backend10:47:021
33779599210,19cyclictest3056-21PK-Backend09:51:531
337790992119,1cyclictest0-21swapper/012:30:010
337790992119,1cyclictest0-21swapper/012:30:000
337790992119,1cyclictest0-21swapper/012:28:590
337790992119,1cyclictest0-21swapper/011:46:070
337790992119,1cyclictest0-21swapper/011:27:340
337790992119,1cyclictest0-21swapper/010:52:200
337790992119,1cyclictest0-21swapper/010:40:100
337790992119,1cyclictest0-21swapper/010:40:100
337790992119,1cyclictest0-21swapper/010:39:080
337790992119,1cyclictest0-21swapper/010:19:280
337790992119,1cyclictest0-21swapper/010:03:320
337790992119,1cyclictest0-21swapper/009:55:420
337790992119,1cyclictest0-21swapper/007:41:440
337790992117,3cyclictest361581-21find08:15:110
337790992114,5cyclictest454280-21chrt12:35:420
337790992113,6cyclictest1-21systemd08:10:000
33779099210,20cyclictest0-21swapper/012:21:290
33779099210,20cyclictest0-21swapper/011:41:140
33779099210,20cyclictest0-21swapper/011:03:070
33779099210,20cyclictest0-21swapper/008:38:080
33779099210,20cyclictest0-21swapper/008:02:540
33779099210,20cyclictest0-21swapper/007:46:590
139099210,4rtkit-daemon1389-21rtkit-daemon12:10:272
139099210,3rtkit-daemon1389-21rtkit-daemon12:08:532
139099210,13rtkit-daemon0-21swapper/008:40:240
3917932200,5sleep20-21swapper/209:40:172
337804992018,1cyclictest0-21swapper/311:33:393
337804992018,1cyclictest0-21swapper/311:24:553
337804992018,1cyclictest0-21swapper/309:33:013
337804992017,2cyclictest3056-21PK-Backend10:25:133
337804992017,2cyclictest3056-21PK-Backend08:26:433
337804992013,6cyclictest436743-21sed11:45:213
337804992012,6cyclictest353737-21awk07:50:223
33780499200,1cyclictest3056-21PK-Backend10:20:333
33780499200,19cyclictest3056-21PK-Backend10:08:013
33780499200,19cyclictest3056-21PK-Backend09:26:123
33780499200,19cyclictest3056-21PK-Backend08:08:313
33780499200,19cyclictest0-21swapper/311:44:153
33780499200,19cyclictest0-21swapper/309:49:293
33780499200,19cyclictest0-21swapper/309:19:223
33780499200,19cyclictest0-21swapper/308:01:283
33780499200,19cyclictest0-21swapper/307:57:023
33780499200,19cyclictest0-21swapper/307:38:173
337800992018,1cyclictest0-21swapper/211:36:112
337800992018,1cyclictest0-21swapper/211:36:102
337800992018,1cyclictest0-21swapper/210:07:022
337800992018,1cyclictest0-21swapper/209:03:282
337800992018,1cyclictest0-21swapper/208:20:282
337800992018,1cyclictest0-21swapper/207:37:502
337800992013,5cyclictest343682-21apt07:25:122
337800992012,6cyclictest338734-21ntpq07:10:182
33780099200,1cyclictest0-21swapper/208:57:262
33780099200,19cyclictest0-21swapper/210:57:042
33780099200,19cyclictest0-21swapper/209:36:472
33780099200,19cyclictest0-21swapper/209:06:052
33780099200,19cyclictest0-21swapper/207:46:432
33779599209,3cyclictest377863-21tr09:00:171
33779599208,3cyclictest141rcu_preempt08:40:001
337795992018,1cyclictest3056-21PK-Backend11:46:501
337795992012,6cyclictest1-21systemd11:05:011
33779599200,1cyclictest3056-21PK-Backend11:42:591
33779599200,19cyclictest3056-21PK-Backend11:25:411
33779599200,19cyclictest3056-21PK-Backend09:12:121
33779599200,19cyclictest0-21swapper/112:29:291
33779599200,19cyclictest0-21swapper/111:39:071
33779599200,19cyclictest0-21swapper/111:39:071
33779599200,19cyclictest0-21swapper/110:16:271
33779599200,19cyclictest0-21swapper/110:12:011
33779599200,19cyclictest0-21swapper/108:56:231
33779599200,18cyclictest3056-21PK-Backend07:42:491
337790992018,1cyclictest3056-21PK-Backend07:35:430
337790992018,1cyclictest0-21swapper/011:18:500
337790992018,1cyclictest0-21swapper/011:10:050
337790992018,1cyclictest0-21swapper/010:20:300
337790992018,1cyclictest0-21swapper/010:05:210
337790992018,1cyclictest0-21swapper/009:40:000
337790992018,1cyclictest0-21swapper/009:22:100
337790992018,1cyclictest0-21swapper/009:16:220
337790992018,1cyclictest0-21swapper/007:53:140
33779099200,19cyclictest3056-21PK-Backend08:50:330
33779099200,19cyclictest0-21swapper/012:17:370
33779099200,19cyclictest0-21swapper/012:11:560
33779099200,19cyclictest0-21swapper/012:00:330
33779099200,19cyclictest0-21swapper/011:36:210
33779099200,19cyclictest0-21swapper/011:36:200
33779099200,19cyclictest0-21swapper/010:57:130
33779099200,19cyclictest0-21swapper/010:46:390
33779099200,19cyclictest0-21swapper/010:26:590
33779099200,19cyclictest0-21swapper/009:51:160
33779099200,19cyclictest0-21swapper/008:55:330
33779099200,19cyclictest0-21swapper/008:22:410
4444742190,4sleep33056-21PK-Backend12:10:113
337804991917,1cyclictest3056-21PK-Backend12:27:063
337804991917,1cyclictest3056-21PK-Backend11:00:543
337804991917,1cyclictest0-21swapper/312:34:373
337804991917,1cyclictest0-21swapper/312:34:363
337804991917,1cyclictest0-21swapper/310:41:493
337804991917,1cyclictest0-21swapper/310:41:483
337804991917,1cyclictest0-21swapper/309:11:593
337804991917,1cyclictest0-21swapper/308:50:093
337804991917,1cyclictest0-21swapper/307:11:133
337804991916,2cyclictest3056-21PK-Backend12:37:283
33780499190,1cyclictest3056-21PK-Backend08:23:533
33780499190,1cyclictest0-21swapper/309:58:223
33780499190,1cyclictest0-21swapper/308:45:563
33780499190,18cyclictest3056-21PK-Backend10:02:493
33780499190,18cyclictest0-21swapper/311:51:593
33780499190,18cyclictest0-21swapper/311:25:563
33780499190,18cyclictest0-21swapper/310:58:523
33780499190,18cyclictest0-21swapper/309:23:353
337800991917,1cyclictest0-21swapper/212:35:462
337800991917,1cyclictest0-21swapper/211:11:432
337800991917,1cyclictest0-21swapper/207:52:092
337800991913,4cyclictest343440-21systemd-run07:25:002
33780099190,1cyclictest0-21swapper/212:01:262
33780099190,1cyclictest0-21swapper/210:51:092
33780099190,1cyclictest0-21swapper/209:57:092
337795991917,1cyclictest3056-21PK-Backend07:21:331
337795991917,1cyclictest0-21swapper/111:09:131
337795991917,1cyclictest0-21swapper/110:35:061
337795991917,1cyclictest0-21swapper/110:06:471
337795991917,1cyclictest0-21swapper/108:26:361
337795991915,2cyclictest447939-21apt12:20:101
337795991912,6cyclictest446907-21ntp_kernel_err12:15:161
337795991912,5cyclictest445337-21python312:10:201
33779599190,1cyclictest3056-21PK-Backend09:20:221
33779599190,1cyclictest3056-21PK-Backend07:48:031
33779599190,1cyclictest0-21swapper/112:06:451
33779599190,1cyclictest0-21swapper/108:23:461
33779599190,1cyclictest0-21swapper/108:11:011
33779599190,18cyclictest3056-21PK-Backend08:51:231
33779599190,18cyclictest3056-21PK-Backend07:34:101
33779599190,18cyclictest0-21swapper/111:21:491
33779599190,18cyclictest0-21swapper/110:27:351
33779599190,18cyclictest0-21swapper/110:21:061
33779599190,18cyclictest0-21swapper/110:02:201
337790991917,1cyclictest0-21swapper/010:34:280
337790991917,1cyclictest0-21swapper/009:48:390
337790991917,1cyclictest0-21swapper/009:04:450
337790991917,1cyclictest0-21swapper/008:30:590
337790991917,1cyclictest0-21swapper/007:22:520
33779099190,1cyclictest0-21swapper/011:52:490
33779099190,1cyclictest0-21swapper/011:31:260
33779099190,1cyclictest0-21swapper/010:10:360
33779099190,1cyclictest0-21swapper/009:10:340
33779099190,1cyclictest0-21swapper/007:58:280
33779099190,18cyclictest0-21swapper/012:06:140
139099190,13rtkit-daemon0-21swapper/007:29:060
337804991816,1cyclictest3056-21PK-Backend08:18:243
337804991813,3cyclictest426360-21munin-run11:20:013
337804991812,5cyclictest414668-21if_err_enp2s010:45:163
337800991816,1cyclictest0-21swapper/212:24:242
337800991816,1cyclictest0-21swapper/210:38:592
337800991816,1cyclictest0-21swapper/209:34:102
337800991812,5cyclictest367796-21sed08:30:212
337800991810,5cyclictest437681-21cpuspeed_turbos11:50:122
33779599184,5cyclictest0-21swapper/111:55:101
337795991816,1cyclictest3056-21PK-Backend11:17:571
337795991816,1cyclictest3056-21PK-Backend07:39:251
337795991816,1cyclictest3056-21PK-Backend07:16:261
337795991816,1cyclictest0-21swapper/108:06:341
337795991812,4cyclictest386670-21wc09:25:191
337795991811,5cyclictest453443-21expr12:35:111
337795991810,6cyclictest362071-21awk08:15:171
33779599180,1cyclictest0-21swapper/110:55:331
337790991817,1cyclictest0-21swapper/007:30:290
337790991816,1cyclictest0-21swapper/011:57:420
337790991816,1cyclictest0-21swapper/009:25:340
337790991816,1cyclictest0-21swapper/008:25:320
33779099180,1cyclictest0-21swapper/011:22:420
337804991715,1cyclictest3056-21PK-Backend12:24:163
337804991715,1cyclictest3056-21PK-Backend11:14:203
337804991715,1cyclictest3056-21PK-Backend11:07:373
337804991715,1cyclictest0-21swapper/307:17:133
337795991715,1cyclictest3056-21PK-Backend10:54:321
337795991715,1cyclictest3056-21PK-Backend07:56:551
337790991715,1cyclictest0-21swapper/009:36:350
337790991712,3cyclictest13-21ksoftirqd/008:10:110
337804991614,1cyclictest3056-21PK-Backend10:31:413
337804991614,1cyclictest3056-21PK-Backend10:16:553
33780499160,14cyclictest369331-21ps08:35:193
337800991611,3cyclictest446178-21/usr/sbin/munin12:15:152
33780099160,15cyclictest0-21swapper/210:28:522
337795991614,1cyclictest3056-21PK-Backend12:33:221
337795991614,1cyclictest3056-21PK-Backend12:33:221
337795991614,1cyclictest3056-21PK-Backend12:02:061
337795991614,1cyclictest3056-21PK-Backend10:41:351
337795991614,1cyclictest3056-21PK-Backend10:41:351
337795991612,3cyclictest357135-21timerwakeupswit08:00:211
337795991612,3cyclictest1095-21runrttasks08:30:101
337795991513,1cyclictest3056-21PK-Backend09:35:381
33779599150,5cyclictest0-21swapper/107:10:181
337795991412,1cyclictest3056-21PK-Backend09:42:271
33779099139,2cyclictest13-21ksoftirqd/008:49:060
33779599128,3cyclictest1095-21runrttasks09:08:361
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional