You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 20:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Apr 19, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
126827121150,4sleep0161rcuc/007:35:000
125786221120,19sleep20-21swapper/207:05:142
131336721080,12sleep30-21swapper/309:40:193
125867427039,25sleep00-21swapper/007:06:420
125784927044,16sleep10-21swapper/107:05:141
125872026343,13sleep30-21swapper/307:07:173
13159862610,3sleep342-21ksoftirqd/309:50:013
13284382510,1sleep20-21swapper/210:25:012
12860522490,8sleep234-21ksoftirqd/208:25:112
12705672470,1sleep30-21swapper/307:40:133
13612192420,3sleep10-21swapper/111:55:221
12861042290,3sleep33056-21PK-Backend08:25:113
139099260,4rtkit-daemon1389-21rtkit-daemon07:26:350
139099240,4rtkit-daemon1389-21rtkit-daemon07:21:310
139099240,15rtkit-daemon0-21swapper/208:45:242
1259093992314,7cyclictest978-21snmpd07:25:002
1259089992310,9cyclictest883-21dbus-daemon08:36:521
1259085992314,7cyclictest1291653-21df_inode08:40:120
13057802220,4sleep13056-21PK-Backend09:20:111
125909699225,15cyclictest1261224-21df07:15:033
1259096992214,6cyclictest1297963-21cron09:00:003
1259093992220,1cyclictest0-21swapper/212:28:402
1259093992220,1cyclictest0-21swapper/211:49:142
1259093992220,1cyclictest0-21swapper/211:34:192
1259093992220,1cyclictest0-21swapper/209:16:102
1259093992220,1cyclictest0-21swapper/208:31:422
1259093992220,1cyclictest0-21swapper/208:15:202
125909399220,21cyclictest0-21swapper/208:12:092
125909399220,21cyclictest0-21swapper/207:27:202
1259089992220,1cyclictest0-21swapper/112:33:281
1259089992220,1cyclictest0-21swapper/110:16:471
1259089992219,1cyclictest3056-21PK-Backend11:00:351
125908999220,20cyclictest3056-21PK-Backend11:15:141
125908999220,20cyclictest3056-21PK-Backend11:15:141
125908999220,20cyclictest3056-21PK-Backend10:22:491
125908999220,20cyclictest3056-21PK-Backend08:48:591
125908999220,20cyclictest3056-21PK-Backend08:02:371
1259085992213,7cyclictest1270931-21awk07:40:180
139099210,4rtkit-daemon1389-21rtkit-daemon07:20:081
139099210,14rtkit-daemon0-21swapper/211:02:102
13096002210,3sleep30-21swapper/309:30:153
1259096992114,5cyclictest1365210-21cron12:10:003
125909699210,1cyclictest3056-21PK-Backend08:22:193
1259093992119,1cyclictest0-21swapper/212:33:272
1259093992119,1cyclictest0-21swapper/211:35:342
1259093992119,1cyclictest0-21swapper/211:08:502
1259093992119,1cyclictest0-21swapper/210:05:162
1259093992119,1cyclictest0-21swapper/209:04:482
1259093992119,1cyclictest0-21swapper/208:53:052
1259093992119,1cyclictest0-21swapper/208:43:442
1259093992119,1cyclictest0-21swapper/207:47:482
1259093992116,4cyclictest34-21ksoftirqd/212:15:002
1259093992115,4cyclictest34-21ksoftirqd/209:30:012
1259093992110,8cyclictest883-21dbus-daemon09:41:542
125909399210,20cyclictest0-21swapper/212:22:402
125909399210,20cyclictest0-21swapper/210:45:332
125909399210,20cyclictest0-21swapper/209:34:352
125909399210,20cyclictest0-21swapper/207:55:532
125909399210,1cyclictest3056-21PK-Backend12:15:232
1259089992119,1cyclictest3056-21PK-Backend11:46:171
1259089992119,1cyclictest3056-21PK-Backend11:11:221
1259089992119,1cyclictest3056-21PK-Backend10:03:431
1259089992119,1cyclictest3056-21PK-Backend09:49:161
1259089992119,1cyclictest3056-21PK-Backend08:57:191
1259089992119,1cyclictest0-21swapper/110:36:211
1259089992119,1cyclictest0-21swapper/107:55:541
1259089992119,1cyclictest0-21swapper/107:25:261
1259089992118,2cyclictest3056-21PK-Backend10:59:331
1259089992118,2cyclictest3056-21PK-Backend10:05:321
1259089992118,2cyclictest3056-21PK-Backend09:02:391
125908999210,19cyclictest3056-21PK-Backend09:19:081
125908999210,19cyclictest3056-21PK-Backend09:14:211
125908999210,19cyclictest3056-21PK-Backend08:13:171
125908999210,19cyclictest3056-21PK-Backend07:45:121
1259085992119,1cyclictest0-21swapper/011:33:310
1259085992113,6cyclictest1373563-21python312:30:220
125908599210,20cyclictest0-21swapper/008:10:250
139099200,14rtkit-daemon3056-21PK-Backend12:27:283
139099200,14rtkit-daemon3056-21PK-Backend08:38:193
1259096992018,1cyclictest0-21swapper/311:42:283
1259096992018,1cyclictest0-21swapper/311:34:233
1259096992018,1cyclictest0-21swapper/310:35:293
1259096992018,1cyclictest0-21swapper/310:18:193
1259096992018,1cyclictest0-21swapper/307:46:033
1259096992018,1cyclictest0-21swapper/307:39:193
1259096992014,5cyclictest1264362-21sshd07:20:553
1259096992013,5cyclictest1-21systemd10:45:013
125909699201,16cyclictest1331164-21python310:30:213
125909699200,1cyclictest0-21swapper/309:03:213
125909699200,1cyclictest0-21swapper/308:16:453
125909699200,19cyclictest3056-21PK-Backend08:40:313
125909699200,19cyclictest0-21swapper/311:23:003
125909699200,19cyclictest0-21swapper/311:17:063
125909699200,19cyclictest0-21swapper/311:17:063
125909699200,19cyclictest0-21swapper/311:08:073
125909699200,19cyclictest0-21swapper/309:39:173
125909699200,19cyclictest0-21swapper/309:08:093
125909699200,19cyclictest0-21swapper/308:53:153
1259093992018,1cyclictest0-21swapper/211:54:482
1259093992018,1cyclictest0-21swapper/211:44:412
1259093992018,1cyclictest0-21swapper/210:44:302
1259093992018,1cyclictest0-21swapper/210:14:092
1259093992018,1cyclictest0-21swapper/209:22:322
1259093992018,1cyclictest0-21swapper/207:41:052
1259093992018,1cyclictest0-21swapper/207:38:152
1259093992016,3cyclictest34-21ksoftirqd/210:30:012
1259093992014,5cyclictest1331793-21systemctl10:35:002
125909399200,1cyclictest0-21swapper/210:36:462
125909399200,19cyclictest3056-21PK-Backend08:35:062
125909399200,19cyclictest3056-21PK-Backend08:20:062
125909399200,19cyclictest0-21swapper/211:17:502
125909399200,19cyclictest0-21swapper/211:17:502
125909399200,19cyclictest0-21swapper/209:57:132
125909399200,19cyclictest0-21swapper/209:36:102
125909399200,19cyclictest0-21swapper/208:09:442
125909399200,19cyclictest0-21swapper/207:10:502
1259089992018,1cyclictest3056-21PK-Backend12:37:001
1259089992018,1cyclictest3056-21PK-Backend11:44:001
1259089992018,1cyclictest3056-21PK-Backend09:41:531
1259089992018,1cyclictest0-21swapper/112:13:341
1259089992017,2cyclictest3056-21PK-Backend11:37:241
1259089992017,2cyclictest3056-21PK-Backend10:10:451
1259089992017,2cyclictest0-21swapper/108:41:031
1259089992012,6cyclictest1283825-21/usr/sbin/munin08:19:591
125908999200,1cyclictest3056-21PK-Backend10:51:351
125908999200,19cyclictest3056-21PK-Backend11:32:521
125908999200,19cyclictest3056-21PK-Backend11:23:471
125908999200,19cyclictest3056-21PK-Backend07:36:551
125908999200,19cyclictest0-21swapper/112:21:531
125908999200,19cyclictest0-21swapper/107:33:161
125908999200,19cyclictest0-21swapper/107:10:591
1259085992018,1cyclictest0-21swapper/012:01:360
1259085992018,1cyclictest0-21swapper/011:58:040
1259085992018,1cyclictest0-21swapper/011:18:170
1259085992018,1cyclictest0-21swapper/011:18:160
1259085992018,1cyclictest0-21swapper/010:18:410
1259085992018,1cyclictest0-21swapper/009:49:550
1259085992018,1cyclictest0-21swapper/008:33:440
1259085992018,1cyclictest0-21swapper/007:17:260
1259085992018,1cyclictest0-21swapper/007:12:390
1259085992017,2cyclictest0-21swapper/010:25:430
1259085992014,5cyclictest1-21systemd08:24:590
125908599200,1cyclictest0-21swapper/011:48:100
125908599200,19cyclictest3056-21PK-Backend10:47:140
125908599200,19cyclictest0-21swapper/012:19:400
125908599200,19cyclictest0-21swapper/011:04:240
125908599200,19cyclictest0-21swapper/010:37:410
125908599200,19cyclictest0-21swapper/009:56:440
125908599200,19cyclictest0-21swapper/008:08:020
125908599200,19cyclictest0-21swapper/007:52:470
125909699192,12cyclictest1328560-21apt10:25:123
1259096991917,1cyclictest3056-21PK-Backend07:31:153
1259096991917,1cyclictest3056-21PK-Backend07:29:273
1259096991917,1cyclictest0-21swapper/311:55:523
1259096991917,1cyclictest0-21swapper/311:53:353
1259096991917,1cyclictest0-21swapper/309:57:563
1259096991917,1cyclictest0-21swapper/308:06:383
1259096991915,3cyclictest0-21swapper/309:50:203
1259096991914,3cyclictest1345367-21cut11:10:173
1259096991913,3cyclictest1301956-21apt09:10:103
1259096991912,5cyclictest1362508-21irqstats12:00:183
1259096991911,6cyclictest1323712-21df10:10:133
1259096991910,7cyclictest1336138-21df_inode10:45:143
125909699190,1cyclictest3056-21PK-Backend12:21:003
125909699190,1cyclictest3056-21PK-Backend08:31:173
125909699190,1cyclictest3056-21PK-Backend07:50:563
125909699190,1cyclictest0-21swapper/310:20:553
125909699190,18cyclictest0-21swapper/312:35:073
125909699190,18cyclictest0-21swapper/310:56:033
125909699190,18cyclictest0-21swapper/310:06:503
125909699190,18cyclictest0-21swapper/307:13:523
125909699190,17cyclictest1337503-21/usr/sbin/munin10:50:103
125909399192,15cyclictest1338176-21cut10:50:182
1259093991917,1cyclictest0-21swapper/211:57:032
1259093991917,1cyclictest0-21swapper/210:03:282
1259093991917,1cyclictest0-21swapper/207:30:592
1259093991913,4cyclictest978-21snmpd11:24:292
125909399190,1cyclictest0-21swapper/209:08:452
125909399190,18cyclictest0-21swapper/212:38:142
1259089991917,1cyclictest3056-21PK-Backend10:43:511
1259089991917,1cyclictest3056-21PK-Backend10:33:311
1259089991917,1cyclictest3056-21PK-Backend09:32:341
1259089991916,2cyclictest3056-21PK-Backend10:28:031
1259089991916,2cyclictest3056-21PK-Backend09:53:281
125908999190,1cyclictest3056-21PK-Backend11:51:381
125908999190,1cyclictest3056-21PK-Backend11:05:281
125908999190,1cyclictest3056-21PK-Backend10:49:321
125908599192,12cyclictest1289547-21apt08:35:120
1259085991917,1cyclictest0-21swapper/012:37:190
1259085991917,1cyclictest0-21swapper/011:53:440
1259085991917,1cyclictest0-21swapper/011:12:220
1259085991917,1cyclictest0-21swapper/010:21:170
1259085991917,1cyclictest0-21swapper/010:05:370
1259085991917,1cyclictest0-21swapper/009:23:530
1259085991917,1cyclictest0-21swapper/007:57:190
1259085991916,2cyclictest3056-21PK-Backend10:10:030
1259085991911,6cyclictest436-20systemd-journal12:15:000
1259085991911,6cyclictest1286795-21ntp_kernel_pll_08:25:180
125908599190,1cyclictest0-21swapper/012:08:530
125908599190,1cyclictest0-21swapper/008:18:090
125908599190,1cyclictest0-21swapper/007:46:530
125908599190,1cyclictest0-21swapper/007:36:320
125908599190,18cyclictest0-21swapper/008:01:050
13501792180,2sleep00-21swapper/011:25:130
1259096991816,1cyclictest3056-21PK-Backend12:33:523
1259096991816,1cyclictest3056-21PK-Backend07:55:563
1259096991816,1cyclictest0-21swapper/311:37:553
1259096991816,1cyclictest0-21swapper/311:00:093
1259096991816,1cyclictest0-21swapper/308:00:283
1259096991815,2cyclictest3056-21PK-Backend12:15:123
1259096991813,3cyclictest1304431-21ntp_states09:15:193
1259096991812,5cyclictest1281573-21sed08:10:213
125909699181,13cyclictest1365496-21apt12:10:123
125909399187,3cyclictest34-21ksoftirqd/211:00:012
1259093991816,1cyclictest0-21swapper/211:28:312
1259093991816,1cyclictest0-21swapper/211:11:552
1259093991816,1cyclictest0-21swapper/210:18:362
1259093991816,1cyclictest0-21swapper/209:50:242
1259093991816,1cyclictest0-21swapper/209:46:582
1259093991816,1cyclictest0-21swapper/208:55:282
1259093991813,4cyclictest978-21snmpd09:10:242
1259093991813,4cyclictest1362735-21ntp_kernel_err12:00:192
1259093991812,4cyclictest1364288-21cat12:05:162
125908999182,10cyclictest1367231-21apt12:15:141
1259089991811,5cyclictest1285836-21rm08:25:011
1259089991810,6cyclictest1362612-21memory12:00:191
125908999180,1cyclictest3056-21PK-Backend09:35:311
1259085991816,1cyclictest3056-21PK-Backend10:55:240
1259085991816,1cyclictest0-21swapper/010:03:470
1259085991816,1cyclictest0-21swapper/009:36:440
1259085991816,1cyclictest0-21swapper/009:27:370
1259085991816,1cyclictest0-21swapper/009:16:560
1259085991816,1cyclictest0-21swapper/009:01:010
1259085991816,1cyclictest0-21swapper/008:52:300
1259085991816,1cyclictest0-21swapper/008:47:570
1259085991815,2cyclictest0-21swapper/011:37:020
1259085991811,5cyclictest1334628-21python310:40:200
125909699178,7cyclictest1305885-21cat09:20:153
1259096991715,1cyclictest3056-21PK-Backend11:27:333
1259096991712,3cyclictest978-21snmpd10:00:283
1259096991710,5cyclictest1294227-21sshd08:46:553
125909699170,15cyclictest1357411-21memory11:45:173
125909699170,15cyclictest1308228-21users09:25:223
1259093991715,1cyclictest0-21swapper/208:02:012
125908999179,6cyclictest1274351-21grep07:50:171
125908999177,8cyclictest436-20systemd-journal08:33:471
125908999175,9cyclictest3056-21PK-Backend11:30:001
125908999174,3cyclictest12962312sleep08:55:001
125908999173,12cyclictest1372043-21vmstat12:25:231
1259089991715,1cyclictest3056-21PK-Backend12:08:471
1259089991715,1cyclictest3056-21PK-Backend07:15:461
1259089991710,4cyclictest1285587-21latency_hist08:25:011
1259085991715,1cyclictest0-21swapper/012:24:270
1259085991715,1cyclictest0-21swapper/011:20:180
1259085991715,1cyclictest0-21swapper/011:07:280
1259085991715,1cyclictest0-21swapper/010:50:170
1259085991713,3cyclictest1330968-21grep10:30:190
1259093991611,4cyclictest34-21ksoftirqd/207:20:012
1259093991610,4cyclictest436-20systemd-journal07:55:002
1259089991614,1cyclictest3056-21PK-Backend09:57:421
1259089991614,1cyclictest3056-21PK-Backend09:05:031
1259089991614,1cyclictest3056-21PK-Backend08:09:331
1259089991614,1cyclictest3056-21PK-Backend07:43:241
125908599168,6cyclictest1313889-21python309:42:350
1259085991614,1cyclictest0-21swapper/009:53:200
1259085991614,1cyclictest0-21swapper/008:58:370
1259085991611,3cyclictest1371695-21sh12:25:190
125908599161,12cyclictest1355073-21/usr/sbin/munin11:40:240
1259089991513,1cyclictest3056-21PK-Backend09:29:361
125908599159,5cyclictest1302249-21expr09:10:120
125908599149,4cyclictest1300329-21cat09:05:120
125908599148,5cyclictest1309042-21mktemp09:30:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional