You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 21:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Sun Sep 06, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099860,72rtkit-daemon0-21swapper/207:05:252
73122428443,33sleep00-21swapper/007:05:140
7430002770,1sleep00-21swapper/007:36:460
73213327640,13sleep10-21swapper/107:07:481
73154227646,21sleep30-21swapper/307:05:213
8276052510,3sleep00-21swapper/011:35:170
7922662490,2sleep00-21swapper/009:55:130
7975342440,2sleep20-21swapper/210:10:142
8158262360,2sleep23056-21PK-Backend11:00:232
7475252290,9sleep234-21ksoftirqd/207:50:152
8118302270,4sleep126-21ksoftirqd/110:50:151
732442992723,3cyclictest161rcuc/009:05:460
7433572240,8chrt26-21ksoftirqd/107:40:001
732450992415,7cyclictest823271-21unin-run11:25:012
73244299242,6cyclictest13-21ksoftirqd/012:21:500
73244299242,6cyclictest13-21ksoftirqd/012:21:500
732442992412,10cyclictest765704-21tr08:40:170
732450992314,7cyclictest436-20systemd-journal12:05:002
732442992314,7cyclictest1-21systemd10:49:590
139099230,16rtkit-daemon3056-21PK-Backend07:22:003
8045162220,3sleep33056-21PK-Backend10:30:123
8014442220,4sleep30-21swapper/310:20:173
732450992220,1cyclictest0-21swapper/208:53:522
732450992220,1cyclictest0-21swapper/207:22:212
732450992214,6cyclictest791646-21apt-key09:55:012
732447992220,1cyclictest0-21swapper/112:26:321
732447992220,1cyclictest0-21swapper/112:07:121
732447992220,1cyclictest0-21swapper/111:22:031
732447992220,1cyclictest0-21swapper/110:46:201
732447992220,1cyclictest0-21swapper/107:14:091
732447992219,2cyclictest3056-21PK-Backend11:18:041
732447992219,2cyclictest0-21swapper/108:02:151
732447992214,6cyclictest791993-21apt09:55:111
732447992210,3cyclictest846199-21tr12:30:121
732447992210,3cyclictest846199-21tr12:30:111
73244799220,21cyclictest7985662sleep110:13:151
73244299224,7cyclictest13-21ksoftirqd/007:30:150
73244299224,10cyclictest13-21ksoftirqd/008:00:000
73244299223,6cyclictest13-21ksoftirqd/008:00:130
732442992220,1cyclictest0-21swapper/009:00:420
732442992220,1cyclictest0-21swapper/008:26:220
73244299220,6cyclictest3056-21PK-Backend11:50:140
139099220,3rtkit-daemon0-21swapper/210:45:122
73245299215,5cyclictest3056-21PK-Backend10:59:493
732452992119,1cyclictest3056-21PK-Backend07:31:513
732452992119,1cyclictest0-21swapper/307:43:333
732452992118,2cyclictest3056-21PK-Backend08:08:213
73245299211,7cyclictest845384-21users12:25:233
732452992114,6cyclictest764896-21http08:39:443
73245299210,20cyclictest3056-21PK-Backend09:25:473
73245299210,20cyclictest3056-21PK-Backend07:58:003
73245299210,19cyclictest3056-21PK-Backend10:45:513
73245299210,19cyclictest3056-21PK-Backend09:01:543
73245299210,19cyclictest3056-21PK-Backend08:10:243
732450992119,1cyclictest0-21swapper/212:11:172
732450992119,1cyclictest0-21swapper/211:50:222
732450992119,1cyclictest0-21swapper/211:42:512
732450992119,1cyclictest0-21swapper/211:10:342
732450992119,1cyclictest0-21swapper/210:22:412
732450992119,1cyclictest0-21swapper/209:39:082
732450992119,1cyclictest0-21swapper/209:33:542
732450992119,1cyclictest0-21swapper/209:18:262
732450992119,1cyclictest0-21swapper/208:30:472
732450992119,1cyclictest0-21swapper/208:06:002
73245099210,20cyclictest0-21swapper/211:25:412
73245099210,20cyclictest0-21swapper/209:01:092
73245099210,1cyclictest0-21swapper/207:37:482
73245099210,19cyclictest3056-21PK-Backend07:10:242
73245099210,19cyclictest0-21swapper/212:36:252
732447992119,1cyclictest0-21swapper/110:41:541
732447992119,1cyclictest0-21swapper/109:52:471
732447992119,1cyclictest0-21swapper/108:59:201
732447992119,1cyclictest0-21swapper/108:54:541
732447992119,1cyclictest0-21swapper/108:30:271
732447992113,6cyclictest784002-21cat09:30:221
732447992110,8cyclictest1075-21whoopsie08:24:441
73244799210,20cyclictest3056-21PK-Backend07:34:371
73244799210,20cyclictest0-21swapper/111:27:371
73244799210,20cyclictest0-21swapper/110:28:361
73244799210,20cyclictest0-21swapper/110:23:091
73244799210,20cyclictest0-21swapper/109:47:191
73244799210,20cyclictest0-21swapper/109:40:161
73244799210,20cyclictest0-21swapper/108:09:111
73244299218,7cyclictest0-21swapper/011:02:360
73244299212,7cyclictest13-21ksoftirqd/012:35:230
732442992119,1cyclictest0-21swapper/008:55:210
732442992117,3cyclictest161rcuc/010:12:120
732442992117,3cyclictest161rcuc/009:45:000
73244299211,6cyclictest13-21ksoftirqd/010:04:120
732442992116,3cyclictest736367-21unin-run07:20:000
732442992114,6cyclictest883-21dbus-daemon08:24:430
732442992110,3cyclictest141rcu_preempt09:33:200
732442992110,2cyclictest141rcu_preempt11:16:060
139099210,14rtkit-daemon3056-21PK-Backend09:32:373
139099210,13rtkit-daemon0-21swapper/210:05:042
732452992018,1cyclictest3056-21PK-Backend09:35:483
732452992018,1cyclictest3056-21PK-Backend08:42:213
732452992018,1cyclictest0-21swapper/309:11:143
732452992018,1cyclictest0-21swapper/308:47:143
732452992014,4cyclictest2156-21gnome-shell10:15:093
732452992013,5cyclictest807537-21date10:40:003
732452992013,5cyclictest807537-21date10:40:003
732452992011,7cyclictest746400-21users07:45:223
732452992011,7cyclictest746400-21users07:45:223
73245299200,1cyclictest0-21swapper/311:55:263
73245299200,19cyclictest3056-21PK-Backend07:35:363
73245299200,19cyclictest3056-21PK-Backend07:12:173
73245299200,19cyclictest0-21swapper/312:19:533
73245299200,19cyclictest0-21swapper/311:54:383
73245299200,19cyclictest0-21swapper/311:19:433
73245299200,19cyclictest0-21swapper/310:13:473
732450992018,1cyclictest3056-21PK-Backend08:25:272
732450992018,1cyclictest0-21swapper/210:50:462
732450992018,1cyclictest0-21swapper/209:20:292
732450992018,1cyclictest0-21swapper/208:47:512
732450992018,1cyclictest0-21swapper/208:41:562
732450992018,1cyclictest0-21swapper/207:55:532
732450992018,1cyclictest0-21swapper/207:25:582
73245099200,1cyclictest0-21swapper/212:09:422
73245099200,1cyclictest0-21swapper/208:01:542
73245099200,19cyclictest0-21swapper/212:23:202
73245099200,19cyclictest0-21swapper/212:23:192
73245099200,19cyclictest0-21swapper/210:00:102
73245099200,19cyclictest0-21swapper/209:57:202
73245099200,19cyclictest0-21swapper/209:26:172
73245099200,19cyclictest0-21swapper/209:12:312
73245099200,19cyclictest0-21swapper/208:18:032
732447992018,1cyclictest3056-21PK-Backend12:22:461
732447992018,1cyclictest3056-21PK-Backend12:22:461
732447992018,1cyclictest3056-21PK-Backend11:56:101
732447992018,1cyclictest3056-21PK-Backend07:42:351
732447992018,1cyclictest0-21swapper/112:19:011
732447992018,1cyclictest0-21swapper/109:37:391
732447992018,1cyclictest0-21swapper/109:00:211
732447992018,1cyclictest0-21swapper/107:46:471
732447992018,1cyclictest0-21swapper/107:46:471
732447992018,1cyclictest0-21swapper/107:29:301
732447992017,2cyclictest0-21swapper/111:45:561
732447992013,5cyclictest755292-21sshd08:13:071
732447992012,6cyclictest737310-21ntp_kernel_err07:20:181
732447992011,7cyclictest768653-21cron08:50:001
73244799200,1cyclictest3056-21PK-Backend09:10:291
73244799200,1cyclictest0-21swapper/108:27:091
73244799200,19cyclictest3056-21PK-Backend10:36:411
73244799200,19cyclictest3056-21PK-Backend10:36:411
73244799200,19cyclictest3056-21PK-Backend10:19:301
73244799200,19cyclictest0-21swapper/112:10:091
73244799200,19cyclictest0-21swapper/111:00:401
73244799200,19cyclictest0-21swapper/108:42:231
73244799200,19cyclictest0-21swapper/107:56:131
73244299205,9cyclictest13-21ksoftirqd/008:05:070
73244299203,9cyclictest13-21ksoftirqd/011:59:000
73244299203,9cyclictest13-21ksoftirqd/007:44:330
73244299202,5cyclictest13-21ksoftirqd/012:14:120
732442992017,2cyclictest161rcuc/012:08:060
732442992017,2cyclictest161rcuc/010:55:590
732442992010,4cyclictest13-21ksoftirqd/008:45:140
73244299200,19cyclictest0-21swapper/009:48:070
73244299200,19cyclictest0-21swapper/009:21:580
7622052190,4sleep342-21ksoftirqd/308:30:183
732452991917,1cyclictest3056-21PK-Backend09:45:143
732452991917,1cyclictest3056-21PK-Backend08:00:373
732452991917,1cyclictest0-21swapper/311:25:183
732452991917,1cyclictest0-21swapper/310:00:353
732452991917,1cyclictest0-21swapper/307:25:013
732452991916,2cyclictest3056-21PK-Backend09:51:293
732452991913,4cyclictest769056-21cpuspeed_turbos08:50:123
732452991912,5cyclictest835802-21cpuspeed_turbos12:00:133
73245299190,1cyclictest3056-21PK-Backend12:07:163
73245299190,1cyclictest3056-21PK-Backend11:43:373
73245299190,1cyclictest3056-21PK-Backend11:21:193
73245299190,1cyclictest3056-21PK-Backend11:07:133
73245299190,1cyclictest3056-21PK-Backend11:00:583
73245299190,1cyclictest0-21swapper/312:23:383
73245299190,1cyclictest0-21swapper/312:23:373
73245299190,18cyclictest3056-21PK-Backend09:42:373
73245299190,18cyclictest3056-21PK-Backend08:24:023
73245299190,18cyclictest0-21swapper/312:10:133
73245299190,18cyclictest0-21swapper/310:08:053
73245299190,18cyclictest0-21swapper/309:59:343
732450991918,1cyclictest0-21swapper/207:44:242
732450991917,1cyclictest0-21swapper/212:31:522
732450991917,1cyclictest0-21swapper/212:31:522
732450991917,1cyclictest0-21swapper/212:25:442
732450991917,1cyclictest0-21swapper/212:18:472
732450991917,1cyclictest0-21swapper/211:46:372
732450991917,1cyclictest0-21swapper/211:06:212
732450991917,1cyclictest0-21swapper/210:37:152
732450991917,1cyclictest0-21swapper/210:37:152
732450991917,1cyclictest0-21swapper/210:32:472
732450991917,1cyclictest0-21swapper/207:46:132
732450991917,1cyclictest0-21swapper/207:46:132
732450991917,1cyclictest0-21swapper/207:32:272
732450991917,1cyclictest0-21swapper/207:17:472
73245099190,1cyclictest0-21swapper/210:28:092
73245099190,1cyclictest0-21swapper/209:05:492
73245099190,1cyclictest0-21swapper/208:55:422
732447991917,1cyclictest3056-21PK-Backend11:53:121
732447991917,1cyclictest3056-21PK-Backend08:39:061
732447991917,1cyclictest0-21swapper/112:37:471
732447991917,1cyclictest0-21swapper/111:10:201
732447991917,1cyclictest0-21swapper/109:29:151
732447991916,2cyclictest3056-21PK-Backend12:02:051
732447991912,6cyclictest829486-21awk11:40:211
73244799190,1cyclictest3056-21PK-Backend10:02:541
73244799190,1cyclictest3056-21PK-Backend09:17:111
73244799190,1cyclictest0-21swapper/111:34:471
73244799190,1cyclictest0-21swapper/110:57:021
73244799190,1cyclictest0-21swapper/109:05:491
73244799190,18cyclictest0-21swapper/107:15:241
73244299198,10cyclictest13-21ksoftirqd/007:55:000
73244299197,5cyclictest13-21ksoftirqd/009:12:550
73244299197,4cyclictest13-21ksoftirqd/009:19:250
73244299193,8cyclictest736608-21apt07:20:120
73244299192,7cyclictest13-21ksoftirqd/008:15:030
73244299192,6cyclictest13-21ksoftirqd/010:30:140
732442991916,2cyclictest732649-21apt07:10:110
73244299190,6cyclictest821783-21apt11:20:130
73244299190,1cyclictest0-21swapper/007:48:090
73244299190,1cyclictest0-21swapper/007:48:090
73244299190,18cyclictest0-21swapper/012:32:200
73244299190,18cyclictest0-21swapper/012:32:200
139099190,13rtkit-daemon3056-21PK-Backend11:46:173
732452991816,1cyclictest3056-21PK-Backend10:43:143
732452991816,1cyclictest0-21swapper/311:39:383
732452991816,1cyclictest0-21swapper/310:54:433
732452991810,6cyclictest780652-21cat09:20:223
73245299180,17cyclictest777948-21expr09:15:133
732450991816,1cyclictest0-21swapper/210:19:032
732450991816,1cyclictest0-21swapper/209:49:022
732450991815,2cyclictest331rcuc/208:15:022
732450991813,3cyclictest833887-21acpi11:55:122
732450991810,6cyclictest436-20systemd-journal08:40:002
732447991816,1cyclictest3056-21PK-Backend10:08:351
732447991816,1cyclictest0-21swapper/111:06:081
732447991811,5cyclictest756360-21diskstats08:15:131
73244299187,3cyclictest13-21ksoftirqd/008:35:210
73244299186,5cyclictest13-21ksoftirqd/010:06:240
73244299186,3cyclictest13-21ksoftirqd/012:15:030
73244299186,3cyclictest13-21ksoftirqd/010:17:110
73244299183,6cyclictest13-21ksoftirqd/009:52:070
73244299182,7cyclictest13-21ksoftirqd/011:29:070
73244299181,7cyclictest13-21ksoftirqd/010:42:070
732442991816,1cyclictest3056-21PK-Backend10:50:190
732442991815,2cyclictest161rcuc/007:26:060
732442991814,3cyclictest161rcuc/012:28:370
732442991814,3cyclictest161rcuc/011:47:430
732442991812,5cyclictest754264-21ls08:10:120
73244299180,8cyclictest13-21ksoftirqd/010:25:070
73244299180,6cyclictest3056-21PK-Backend08:33:410
732452991715,1cyclictest3056-21PK-Backend12:37:503
732452991715,1cyclictest0-21swapper/309:08:233
732450991715,1cyclictest3056-21PK-Backend10:56:142
732450991713,3cyclictest978-21snmpd09:41:042
732450991711,5cyclictest828619-21sed11:40:012
732450991711,5cyclictest820575-21sed11:15:162
732450991711,4cyclictest808362-21ntp_kernel_err10:40:192
73244799179,6cyclictest804771-21memory10:30:171
732447991715,1cyclictest0-21swapper/107:53:021
73244299179,6cyclictest770004-21awk08:50:220
73244299176,3cyclictest13-21ksoftirqd/012:01:410
73244299171,6cyclictest13-21ksoftirqd/010:35:170
73244299171,6cyclictest13-21ksoftirqd/010:35:170
732442991713,3cyclictest161rcuc/011:40:490
732442991713,3cyclictest13-21ksoftirqd/011:10:010
732442991712,3cyclictest826093-21rt-features11:30:200
732442991711,5cyclictest786405-21sed09:40:000
73244299171,14cyclictest819170-21sensors_temp11:10:200
732442991710,5cyclictest801088-21df10:20:120
73245299169,5cyclictest802899-21awk10:25:153
732452991614,1cyclictest3056-21PK-Backend11:12:483
732452991612,3cyclictest759703-21/usr/sbin/munin08:25:113
732450991614,1cyclictest0-21swapper/211:34:402
732447991614,1cyclictest0-21swapper/109:20:161
732442991614,1cyclictest0-21swapper/009:25:290
732452991513,1cyclictest3056-21PK-Backend08:18:083
732452991510,4cyclictest3056-21PK-Backend08:55:413
732450991511,3cyclictest1095-21runrttasks08:22:382
73244799158,5cyclictest827278-21expr11:35:131
73245299148,4cyclictest734634-21rt-features07:15:033
732452991412,1cyclictest3056-21PK-Backend11:30:053
732452991412,1cyclictest3056-21PK-Backend07:50:363
139099140,5rtkit-daemon846039-21seq12:30:113
139099140,5rtkit-daemon846039-21seq12:30:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional