You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 23:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Dec 29, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
125658922570,11sleep10-21swapper/107:05:061
197399950,8rtkit-daemon1972-21rtkit-daemon12:18:541
197399900,10rtkit-daemon4497-21PK-Backend08:54:331
197399800,66rtkit-daemon0-21swapper/307:07:593
125790927644,22sleep00-21swapper/007:07:560
13105512750,4sleep21310552-21cat09:35:302
13105512750,4sleep21310552-21cat09:35:302
197399720,4rtkit-daemon4497-21PK-Backend12:10:021
125789826748,13sleep20-21swapper/207:07:502
197399630,9rtkit-daemon1972-21rtkit-daemon12:15:130
1258239995916,18cyclictest1361187-21taskset11:58:043
1258239995716,15cyclictest1269072-21taskset07:39:403
1258239995716,15cyclictest1269072-21taskset07:39:403
197399560,9rtkit-daemon1972-21rtkit-daemon07:54:290
1258239995618,16cyclictest1279259-21taskset08:07:103
1258239995615,14cyclictest0-21swapper/312:03:003
197399550,8rtkit-daemon1972-21rtkit-daemon09:01:140
197399550,8rtkit-daemon1972-21rtkit-daemon08:21:160
197399550,8rtkit-daemon1972-21rtkit-daemon08:02:260
197399550,8rtkit-daemon1972-21rtkit-daemon08:02:250
13298752550,4sleep00-21swapper/010:30:220
1258239995514,15cyclictest0-21swapper/310:18:453
1258239995513,18cyclictest1354142-21cut11:40:023
1258239995417,15cyclictest1354875-21ls11:40:243
1258239995414,13cyclictest0-21swapper/312:29:403
1258239995411,21cyclictest1054-21snmpd12:18:313
197399530,8rtkit-daemon1972-21rtkit-daemon08:28:380
1258239995311,19cyclictest1316401-21runrttasks09:53:453
125822899537,12cyclictest1321167-21needreboot10:05:210
1258239995213,16cyclictest1280789-21expr08:10:313
1258239995213,16cyclictest1265609-21awk07:30:003
1258239995212,15cyclictest0-21swapper/308:47:543
125822899524,23cyclictest438-20systemd-journal12:32:090
197399510,8rtkit-daemon1972-21rtkit-daemon08:25:401
1258239995113,16cyclictest1033-21gdbus12:32:423
1258239995112,17cyclictest1314501-21awk09:50:003
1258239995112,11cyclictest438-20systemd-journal07:46:323
1258239995110,15cyclictest1320325-21latency_hist10:05:023
1258239995013,15cyclictest1330343-21python310:30:303
1258239994938,7cyclictest924-21NetworkManager11:02:053
1258239994920,24cyclictest1365825-21ntp_states12:10:253
1258239994912,13cyclictest1348384-21find11:25:013
1258239994817,12cyclictest1349977-31apt-check11:25:533
125822899489,15cyclictest1353379-21sort11:35:300
197399470,31rtkit-daemon1355414-21cat11:40:332
1258239994713,13cyclictest1137-21runrttasks07:31:003
1258239994712,13cyclictest0-21swapper/312:24:523
1258239994711,15cyclictest0-21swapper/310:37:353
1258235994721,22cyclictest1265580-21cron07:30:002
125822899479,15cyclictest1325689-21sed10:20:020
1258228994711,12cyclictest1371059-21ntp_states12:25:230
1258239994622,19cyclictest42-21ksoftirqd/311:50:133
1258239994619,7cyclictest1351497-21python311:30:283
1258239994619,23cyclictest1309370-21latency_hist09:35:023
1258239994619,22cyclictest1286040-21users08:25:313
125822899467,15cyclictest1326294-21sed10:20:190
125822899467,14cyclictest1269031-21taskset07:39:090
125822899467,14cyclictest1269031-21taskset07:39:090
125822899464,19cyclictest1356841-21chrt11:45:230
1258228994611,11cyclictest1335822-21users10:45:310
125823999457,16cyclictest1033-21gdbus09:09:273
1258239994519,22cyclictest1311873-21memory09:40:223
1258239994519,22cyclictest1259202-21ntp_states07:10:253
1258239994519,22cyclictest1259202-21ntp_states07:10:253
1258239994518,22cyclictest1283622-21hddtemp_smartct08:20:183
1258239994516,24cyclictest1273796-21timerandwakeup07:50:293
125822899459,12cyclictest1323133-21grep10:10:260
197399440,29rtkit-daemon1342537-21python311:05:262
1258239994420,20cyclictest1337924-21sh10:55:003
1258239994419,21cyclictest1356303-21cpuspeed_turbos11:45:133
1258239994418,6cyclictest1338608-21hddtemp_smartct10:55:213
1258239994417,23cyclictest2257-21gsd-housekeepin07:59:093
1258239994415,25cyclictest1309933-21sh09:35:173
1258239994415,25cyclictest1309933-21sh09:35:173
125822899449,13cyclictest1282968-21perf08:20:000
125822899448,19cyclictest2069-21gnome-shell12:21:590
125822899447,10cyclictest1266124-21expr07:30:150
125822899446,15cyclictest1270547-21taskset07:41:220
125822899445,16cyclictest1359433-40lsb_release11:53:020
125822899445,14cyclictest1332316-21users10:35:290
125822899444,17cyclictest2069-21gnome-shell08:48:590
125822899444,17cyclictest1338625-21sort10:55:210
13217532430,21sleep234-21ksoftirqd/210:07:302
1258239994319,19cyclictest1342037-21df_inode11:05:153
1258239994318,9cyclictest1364842-21systemd-run12:10:013
1258239994316,22cyclictest1345365-21expr11:15:133
125822899438,9cyclictest1362339-21awk12:00:250
125822899438,9cyclictest1320095-21date10:05:010
125822899438,10cyclictest1261847-21taskset07:16:310
125822899437,13cyclictest1344112-21awk11:10:230
1258228994312,17cyclictest13-21ksoftirqd/008:35:290
125822899430,18cyclictest1315733-21awk09:50:240
197399420,34rtkit-daemon4497-21PK-Backend08:35:041
1258239994217,20cyclictest1306499-21smartctl09:25:183
1258239994216,22cyclictest12638892sleep307:25:013
1258239994215,23cyclictest13181572chrt09:59:063
125822899428,9cyclictest1258949-21taskset07:10:210
125822899428,9cyclictest1258949-21taskset07:10:210
125822899427,14cyclictest1288338-21taskset08:33:420
125822899424,14cyclictest1328588-21chrt10:25:470
125823999417,13cyclictest0-21swapper/308:40:333
1258239994120,16cyclictest1303365-21ntp_states09:15:253
1258239994120,16cyclictest1298255-21python309:00:283
1258239994118,20cyclictest1320763-21expr10:05:143
1258239994116,22cyclictest2190-21gvfs-udisks2-vo10:49:573
1258239994116,21cyclictest1260795-21sort07:15:143
1258235994123,13cyclictest438-20systemd-journal09:15:012
1258235994117,20cyclictest1373622-21unin-run12:35:012
125823599411,35cyclictest1356122-21cpu11:45:132
125822899417,17cyclictest1291162-21taskset08:40:260
125822899414,12cyclictest1294866-21grep08:50:280
125822899413,16cyclictest1275128-21latency07:55:210
125822899413,14cyclictest1305667-21chrt09:23:090
1258228994110,11cyclictest13-21ksoftirqd/007:20:270
1258239994025,11cyclictest1270833-21cron07:45:003
1258239994021,15cyclictest1326486-21ntp_states10:20:233
1258239994016,20cyclictest1334134-21taskset10:41:233
1258239994016,20cyclictest1334134-21taskset10:41:233
1258239994015,22cyclictest1295270-21chrt08:53:493
1258235994023,12cyclictest438-20systemd-journal10:05:002
125823399401,15cyclictest13278902sort10:25:201
125822899407,9cyclictest1302352-21perf09:15:010
125822899407,11cyclictest1364801-21cron12:10:000
125822899406,18cyclictest1351498-21smartctl11:30:290
125822899405,9cyclictest1346834-21/usr/sbin/munin11:20:270
125822899403,15cyclictest1137-21runrttasks10:50:260
125822899403,14cyclictest1342932-21chrt11:07:400
1258228994025,11cyclictest1265613-21perf07:30:000
125822899401,13cyclictest1300163-21timerwakeupswit09:05:290
197399390,28rtkit-daemon4497-21PK-Backend10:23:532
197399390,28rtkit-daemon4497-21PK-Backend09:28:202
12631492390,5chrt0-21swapper/207:20:252
1258239993929,6cyclictest1282953-21cron08:20:003
1258239993920,15cyclictest1322707-21/usr/sbin/munin10:10:193
1258239993919,15cyclictest1374179-21cstates12:35:153
1258239993918,16cyclictest1276849-21wc08:00:203
1258239993918,16cyclictest1276849-21wc08:00:203
1258239993917,17cyclictest1343507-21grep11:10:133
1258239993917,17cyclictest1288705-21basename08:35:123
1258239993916,20cyclictest1300883-21awk09:10:113
1258239993915,20cyclictest1286893-21test08:30:023
125823599394,31cyclictest1353825-21munin-run11:39:592
1258235993928,7cyclictest1336193-21cron10:50:012
1258235993923,11cyclictest1284678-21cron08:25:012
1258235993921,13cyclictest1274712-21tr07:55:142
1258233993922,13cyclictest26-21ksoftirqd/111:50:011
1258233993922,13cyclictest1276886-21memory08:00:211
1258233993922,13cyclictest1276886-21memory08:00:211
125822899398,9cyclictest1296309-21head08:55:240
125822899395,14cyclictest1340066-21cut11:00:160
125822899393,14cyclictest1317590-21cut09:55:260
125822899393,13cyclictest1360612-21awk11:55:230
125822899391,12cyclictest1355970-21latency_hist11:45:020
1258228993911,11cyclictest13-21ksoftirqd/009:15:190
197399380,30rtkit-daemon1258222-21cyclictest08:08:130
197399380,24rtkit-daemon0-21swapper/211:17:462
1258239993820,14cyclictest1327886-21/usr/sbin/munin10:25:213
1258239993817,17cyclictest1304663-21cat09:20:133
1258239993816,17cyclictest1295655-21grep08:55:143
1258235993827,8cyclictest1137-21runrttasks08:01:482
1258235993827,8cyclictest1137-21runrttasks08:01:482
1258235993810,7cyclictest13373462swap10:50:272
1258233993828,6cyclictest1081-21ntpd08:40:281
1258233993827,6cyclictest26-21ksoftirqd/107:35:211
1258233993827,6cyclictest26-21ksoftirqd/107:35:211
125822899385,12cyclictest1313538-21cat09:45:210
125822899383,10cyclictest1349937-21users11:25:300
125822899382,31cyclictest1312373-21users09:40:300
197399370,27rtkit-daemon4497-21PK-Backend07:17:292
197399370,25rtkit-daemon0-21swapper/210:28:232
1258235993723,10cyclictest1272605-21systemd-run07:50:002
1258235993721,12cyclictest1347603-21ntp_states11:20:252
1258235993721,12cyclictest1-21systemd11:33:302
1258235993720,13cyclictest937-21polkitd12:12:462
1258235993720,12cyclictest1299984-21python309:05:282
197399360,26rtkit-daemon34-21ksoftirqd/212:29:362
197399360,25rtkit-daemon0-21swapper/212:39:432
197399360,24rtkit-daemon0-21swapper/210:13:562
13724152360,5sleep14497-21PK-Backend12:30:141
13495452360,7sleep14497-21PK-Backend11:25:241
1258235993624,7cyclictest438-20systemd-journal08:30:092
1258235993623,9cyclictest1333645-21grep10:40:242
1258235993623,9cyclictest1333645-21grep10:40:232
125823599362,30cyclictest1137-21runrttasks12:22:252
1258235993621,11cyclictest1364800-21cron12:10:002
1258233993627,5cyclictest26-21ksoftirqd/109:30:001
1258233993620,11cyclictest438-20systemd-journal08:35:001
125823399361,31cyclictest4497-21PK-Backend11:00:021
125822899365,16cyclictest13-21ksoftirqd/010:41:480
125822899365,16cyclictest13-21ksoftirqd/010:41:480
125822899364,17cyclictest13-21ksoftirqd/012:12:300
125822899364,16cyclictest13-21ksoftirqd/011:18:030
125822899364,16cyclictest13-21ksoftirqd/008:11:330
197399350,23rtkit-daemon0-21swapper/209:54:472
1258235993521,10cyclictest34-21ksoftirqd/212:00:192
1258235993519,12cyclictest1338943-21cat10:55:262
1258233993519,12cyclictest1326764-21sed10:20:281
125823399351,14cyclictest4497-21PK-Backend07:54:111
125823399351,10cyclictest4497-21PK-Backend09:22:001
125822899357,9cyclictest13-21ksoftirqd/012:35:120
125822899354,16cyclictest13-21ksoftirqd/009:35:470
125822899354,16cyclictest13-21ksoftirqd/009:35:460
125823599344,4cyclictest0-21swapper/209:21:222
1258235993420,10cyclictest1282709-21cron08:17:002
1258235993420,10cyclictest1281212-21cron08:14:592
1258235993419,11cyclictest1359855-21seq11:55:052
1258235993419,11cyclictest1329812-21expr10:30:172
1258235993418,12cyclictest1294319-21which08:50:192
1258235993418,12cyclictest1265864-21awk07:30:122
1258235993417,13cyclictest1297375-21awk09:00:102
1258235993416,13cyclictest1358060-21dpkg11:50:032
125823599341,30cyclictest0-21swapper/208:38:312
1258233993425,5cyclictest1348349-21http11:25:001
125823399341,9cyclictest4497-21PK-Backend10:17:541
125823399341,9cyclictest4497-21PK-Backend09:39:151
125823399341,9cyclictest4497-21PK-Backend09:39:141
125823399341,9cyclictest4497-21PK-Backend09:33:111
125823399341,9cyclictest4497-21PK-Backend07:33:461
1258233993418,12cyclictest923-21dbus-daemon10:45:031
125823399341,29cyclictest4497-21PK-Backend09:05:351
125823399341,14cyclictest4497-21PK-Backend11:17:571
125823399341,13cyclictest4497-21PK-Backend11:05:391
125823399341,13cyclictest4497-21PK-Backend08:14:401
125823399341,10cyclictest4497-21PK-Backend12:06:551
125823399341,10cyclictest4497-21PK-Backend11:37:181
125822899345,14cyclictest13-21ksoftirqd/007:50:000
1258228993416,13cyclictest1306965-21python309:25:270
125823599332,5cyclictest0-21swapper/211:25:312
1258235993319,10cyclictest1292315-21tr08:45:132
1258235993318,11cyclictest1307808-21cat09:30:132
1258235993318,11cyclictest1268468-21python307:35:272
1258235993318,11cyclictest1268468-21python307:35:272
1258235993317,12cyclictest1-21systemd09:50:012
1258235993317,11cyclictest956-21systemd-logind07:53:412
1258235993316,13cyclictest1296015-21df_inode08:55:172
1258233993323,6cyclictest26-21ksoftirqd/108:57:261
1258233993323,5cyclictest26-21ksoftirqd/112:37:271
1258233993322,8cyclictest26-21ksoftirqd/110:45:021
1258233993322,8cyclictest26-21ksoftirqd/110:45:021
1258233993322,7cyclictest1358784-21needreboot11:50:241
1258233993322,6cyclictest26-21ksoftirqd/111:40:151
1258233993322,6cyclictest26-21ksoftirqd/110:31:561
125823399331,9cyclictest4497-21PK-Backend12:03:181
125823399331,9cyclictest4497-21PK-Backend11:34:041
125823399331,9cyclictest4497-21PK-Backend11:00:401
125823399331,9cyclictest4497-21PK-Backend09:49:171
125823399331,9cyclictest4497-21PK-Backend07:27:511
125823399331,4cyclictest4497-21PK-Backend09:40:361
125823399331,4cyclictest4497-21PK-Backend09:16:131
125823399331,29cyclictest4497-21PK-Backend08:45:151
125823399331,29cyclictest0-21swapper/107:43:201
125823399331,28cyclictest4497-21PK-Backend11:12:331
125823399331,15cyclictest4497-21PK-Backend10:53:071
125823399331,14cyclictest4497-21PK-Backend12:25:511
125823399331,13cyclictest4497-21PK-Backend12:22:091
125823399331,13cyclictest4497-21PK-Backend10:01:081
125823399331,13cyclictest4497-21PK-Backend08:18:251
1258235993220,8cyclictest1339989-21cpu11:00:162
1258235993217,11cyclictest1331966-21awk10:35:232
1258235993217,11cyclictest1312349-21cat09:40:312
1258235993216,11cyclictest1317508-21ntpq09:55:242
1258235993216,11cyclictest1290822-21df08:40:172
125823599321,28cyclictest0-21swapper/211:10:482
125823599321,28cyclictest0-21swapper/210:17:202
125823599321,28cyclictest0-21swapper/207:44:252
1258233993223,5cyclictest26-21ksoftirqd/107:21:281
1258233993222,5cyclictest26-21ksoftirqd/110:35:341
125823399321,9cyclictest4497-21PK-Backend11:58:391
125823399321,9cyclictest4497-21PK-Backend08:20:431
125823399321,8cyclictest4497-21PK-Backend10:08:201
1258233993215,5cyclictest26-21ksoftirqd/108:05:231
125823399321,4cyclictest4497-21PK-Backend07:49:351
125823399321,28cyclictest4497-21PK-Backend09:58:071
125823399321,28cyclictest4497-21PK-Backend09:13:041
125823399321,27cyclictest4497-21PK-Backend09:51:331
125823399321,13cyclictest4497-21PK-Backend09:02:041
1258228993217,11cyclictest1307954-21tr09:30:150
1258235993124,4cyclictest0-21swapper/207:12:122
1258235993124,4cyclictest0-21swapper/207:12:122
1258235993121,6cyclictest1286408-21cron08:30:002
1258235993120,7cyclictest1278039-21cpu08:05:132
125823599311,4cyclictest0-21swapper/209:18:142
1258233993122,5cyclictest26-21ksoftirqd/110:11:431
1258233993122,5cyclictest26-21ksoftirqd/107:15:291
1258233993120,7cyclictest1276031-21sh08:00:001
1258233993019,6cyclictest26-21ksoftirqd/107:10:141
1258233993019,6cyclictest26-21ksoftirqd/107:10:141
1258235992914,11cyclictest1367231-21df12:15:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional