You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 00:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu Aug 06, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099790,66rtkit-daemon0-21swapper/307:08:083
419078027853,16sleep10-21swapper/107:05:211
419114126343,14sleep20-21swapper/207:05:352
419134626243,14sleep00-21swapper/007:08:140
882832560,6sleep30-21swapper/311:25:123
509912470,2sleep13056-21PK-Backend09:40:001
1037492470,4sleep0103750-21cron12:10:000
320742450,3sleep30-21swapper/308:45:003
747562400,3sleep33056-21PK-Backend10:45:183
1005542380,2sleep10-21swapper/112:00:111
872272350,1sleep287231-21sed11:20:222
295302280,3chrt0-21swapper/208:35:212
4191633992612,12cyclictest7336-21unin-run07:35:000
419164299250,23cyclictest20239-21cat08:10:122
419163899250,23cyclictest42900-21cstates09:15:101
419163399241,21cyclictest402222sleep009:06:030
774092230,4sleep33056-21PK-Backend10:55:013
4191647992317,5cyclictest1609-21seq07:20:003
419163899230,21cyclictest21513-21munin-run08:15:011
4191633992314,7cyclictest436-20systemd-journal10:20:000
939192220,3sleep20-21swapper/211:40:192
4191647992218,2cyclictest70752chrt07:32:143
4191647992215,5cyclictest90911-21chrt11:32:383
4191647992215,5cyclictest35161-21sshd08:52:493
4191642992220,1cyclictest0-21swapper/212:24:482
4191642992220,1cyclictest0-21swapper/210:18:292
419164299220,20cyclictest0-21swapper/207:25:582
4191638992220,1cyclictest0-21swapper/112:28:291
4191638992219,2cyclictest3056-21PK-Backend08:27:171
4191638992216,4cyclictest19748-21sh08:10:001
419163899220,20cyclictest3056-21PK-Backend08:03:521
4191633992220,1cyclictest0-21swapper/011:54:270
4191633992220,1cyclictest0-21swapper/010:07:000
4191633992220,1cyclictest0-21swapper/009:59:430
4191633992220,1cyclictest0-21swapper/009:36:240
4191633992220,1cyclictest0-21swapper/008:37:510
4191633992220,1cyclictest0-21swapper/008:27:160
4191633992219,2cyclictest52940-21sort09:45:100
4191633992213,4cyclictest107262-21systemd-run12:20:010
419163399220,21cyclictest0-21swapper/009:43:550
419163399220,20cyclictest3056-21PK-Backend07:40:180
419163399220,10cyclictest939222processes11:40:180
16102220,4chrt1599-21perf07:20:000
4191647992119,1cyclictest3056-21PK-Backend10:57:503
4191647992119,1cyclictest0-21swapper/309:40:583
4191647992119,1cyclictest0-21swapper/309:08:473
4191647992119,1cyclictest0-21swapper/308:48:393
4191647992118,2cyclictest3056-21PK-Backend11:07:583
4191647992117,2cyclictest1788620-21WebKitWebProces09:35:583
4191647992112,7cyclictest53698-21ntp_states09:45:193
419164799210,20cyclictest3056-21PK-Backend10:42:023
419164799210,19cyclictest3056-21PK-Backend12:28:423
419164799210,19cyclictest3056-21PK-Backend12:06:043
4191642992119,1cyclictest0-21swapper/212:17:052
4191642992119,1cyclictest0-21swapper/212:02:252
4191642992119,1cyclictest0-21swapper/211:37:582
4191642992119,1cyclictest0-21swapper/211:37:582
4191642992119,1cyclictest0-21swapper/211:16:152
4191642992119,1cyclictest0-21swapper/211:07:022
4191642992119,1cyclictest0-21swapper/211:01:282
4191642992119,1cyclictest0-21swapper/210:43:022
4191642992119,1cyclictest0-21swapper/210:27:072
4191642992119,1cyclictest0-21swapper/210:20:592
4191642992119,1cyclictest0-21swapper/210:11:462
4191642992119,1cyclictest0-21swapper/209:51:252
4191642992119,1cyclictest0-21swapper/209:02:312
4191642992119,1cyclictest0-21swapper/208:33:172
4191642992119,1cyclictest0-21swapper/207:34:092
4191642992119,1cyclictest0-21swapper/207:23:142
4191642992113,6cyclictest24938-21cron08:25:002
419164299210,20cyclictest0-21swapper/211:57:582
419164299210,20cyclictest0-21swapper/211:47:582
419164299210,20cyclictest0-21swapper/211:28:322
419164299210,20cyclictest0-21swapper/209:33:402
419164299210,20cyclictest0-21swapper/209:09:072
419164299210,20cyclictest0-21swapper/208:50:072
419164299210,20cyclictest0-21swapper/207:52:352
4191638992119,1cyclictest3056-21PK-Backend07:56:281
4191638992119,1cyclictest3056-21PK-Backend07:31:271
4191638992113,5cyclictest1095-21runrttasks07:15:161
4191638992112,7cyclictest1-21systemd11:10:001
419163899210,20cyclictest3056-21PK-Backend08:16:221
419163899210,20cyclictest0-21swapper/112:20:111
419163899210,1cyclictest3056-21PK-Backend08:43:061
419163899210,19cyclictest3056-21PK-Backend08:57:461
4191633992119,1cyclictest0-21swapper/012:13:190
4191633992119,1cyclictest0-21swapper/011:20:480
4191633992119,1cyclictest0-21swapper/011:10:270
4191633992119,1cyclictest0-21swapper/010:44:450
4191633992119,1cyclictest0-21swapper/010:32:420
4191633992119,1cyclictest0-21swapper/010:20:240
4191633992119,1cyclictest0-21swapper/010:10:040
4191633992119,1cyclictest0-21swapper/009:54:430
4191633992119,1cyclictest0-21swapper/008:55:350
4191633992119,1cyclictest0-21swapper/008:52:170
4191633992119,1cyclictest0-21swapper/008:16:070
4191633992119,1cyclictest0-21swapper/008:00:050
4191633992119,1cyclictest0-21swapper/007:10:370
4191633992113,6cyclictest23852-21df_inode08:20:130
419163399210,20cyclictest0-21swapper/011:36:570
419163399210,20cyclictest0-21swapper/011:36:570
419163399210,20cyclictest0-21swapper/009:31:240
419163399210,20cyclictest0-21swapper/007:54:240
419163399210,1cyclictest0-21swapper/007:29:090
139099210,15rtkit-daemon3056-21PK-Backend09:17:133
139099210,15rtkit-daemon0-21swapper/210:47:032
139099210,15rtkit-daemon0-21swapper/207:37:382
139099210,14rtkit-daemon3056-21PK-Backend10:04:373
139099210,14rtkit-daemon3056-21PK-Backend07:35:593
4191647992018,1cyclictest3056-21PK-Backend08:09:393
4191647992018,1cyclictest3056-21PK-Backend07:47:493
4191647992018,1cyclictest0-21swapper/309:55:243
4191647992018,1cyclictest0-21swapper/309:21:453
4191647992018,1cyclictest0-21swapper/308:55:493
4191647992017,2cyclictest3056-21PK-Backend12:16:243
4191647992017,2cyclictest3056-21PK-Backend11:02:373
4191647992017,1cyclictest3056-21PK-Backend09:10:093
4191647992013,5cyclictest37663-21cat09:00:133
4191647992012,6cyclictest46382-21df_inode09:25:133
419164799201,17cyclictest96576-21/usr/sbin/munin11:50:103
419164799200,19cyclictest3056-21PK-Backend12:13:213
419164799200,19cyclictest3056-21PK-Backend11:14:263
419164799200,19cyclictest0-21swapper/311:59:563
419164799200,19cyclictest0-21swapper/311:24:343
419164799200,19cyclictest0-21swapper/309:50:583
419164799200,19cyclictest0-21swapper/309:30:233
419164799200,19cyclictest0-21swapper/308:17:033
4191642992018,1cyclictest0-21swapper/212:26:112
4191642992018,1cyclictest0-21swapper/212:08:472
4191642992018,1cyclictest0-21swapper/211:32:572
4191642992018,1cyclictest0-21swapper/210:32:422
4191642992018,1cyclictest0-21swapper/209:35:362
4191642992018,1cyclictest0-21swapper/208:29:462
4191642992018,1cyclictest0-21swapper/207:15:312
4191642992013,6cyclictest1-21systemd12:35:002
4191642992012,6cyclictest46590-21memory09:25:162
4191642992012,6cyclictest36005-21fw_packets08:55:142
419164299200,19cyclictest0-21swapper/212:13:132
419164299200,19cyclictest0-21swapper/211:13:452
419164299200,19cyclictest0-21swapper/210:58:582
419164299200,19cyclictest0-21swapper/210:00:512
419164299200,19cyclictest0-21swapper/207:48:172
419163899202,12cyclictest74030-21apt10:45:111
4191638992018,1cyclictest3056-21PK-Backend09:41:401
4191638992017,2cyclictest3056-21PK-Backend09:25:581
4191638992017,2cyclictest3056-21PK-Backend09:06:041
4191638992012,6cyclictest883-21dbus-daemon12:20:011
419163899200,1cyclictest3056-21PK-Backend09:34:571
419163899200,19cyclictest0-21swapper/111:17:181
419163899200,19cyclictest0-21swapper/110:35:201
419163899200,19cyclictest0-21swapper/110:35:201
419163899200,19cyclictest0-21swapper/110:15:471
419163899200,19cyclictest0-21swapper/107:28:451
419163899200,18cyclictest3056-21PK-Backend08:33:191
419163399209,9cyclictest75525-21systemd10:50:000
4191633992018,1cyclictest0-21swapper/012:21:380
4191633992018,1cyclictest0-21swapper/011:56:230
4191633992018,1cyclictest0-21swapper/011:00:400
4191633992018,1cyclictest0-21swapper/010:35:460
4191633992018,1cyclictest0-21swapper/010:35:460
4191633992018,1cyclictest0-21swapper/009:25:500
4191633992018,1cyclictest0-21swapper/009:23:200
4191633992018,1cyclictest0-21swapper/007:56:340
4191633992013,6cyclictest110231-21sed12:25:220
4191633992013,5cyclictest19966-21date08:10:010
419163399200,1cyclictest0-21swapper/009:02:450
419163399200,1cyclictest0-21swapper/008:44:540
419163399200,19cyclictest0-21swapper/011:46:570
419163399200,19cyclictest0-21swapper/011:18:180
419163399200,19cyclictest0-21swapper/010:27:070
419163399200,19cyclictest0-21swapper/009:15:220
139099200,14rtkit-daemon3056-21PK-Backend08:20:283
139099200,13rtkit-daemon3056-21PK-Backend07:43:353
4191647991917,1cyclictest3056-21PK-Backend12:21:593
4191647991917,1cyclictest3056-21PK-Backend07:23:433
4191647991917,1cyclictest0-21swapper/311:35:083
4191647991917,1cyclictest0-21swapper/311:35:083
4191647991914,3cyclictest95916-21users11:45:203
4191647991914,3cyclictest113580-21python312:35:203
4191647991911,6cyclictest15569-21ntpq07:55:213
419164799190,7cyclictest16765-21cat08:00:123
419164799190,1cyclictest3056-21PK-Backend10:22:493
419164799190,1cyclictest3056-21PK-Backend10:10:383
419164799190,1cyclictest3056-21PK-Backend07:25:393
419164799190,18cyclictest0-21swapper/311:40:433
419164799190,18cyclictest0-21swapper/308:35:353
4191642991917,1cyclictest0-21swapper/210:08:082
4191642991917,1cyclictest0-21swapper/209:17:522
4191642991917,1cyclictest0-21swapper/208:43:182
4191642991917,1cyclictest0-21swapper/208:19:592
4191642991913,5cyclictest4192490-21grep07:10:162
4191642991911,2cyclictest34-21ksoftirqd/210:55:002
419164299190,1cyclictest0-21swapper/209:55:512
419164299190,1cyclictest0-21swapper/209:47:332
419164299190,1cyclictest0-21swapper/208:01:002
419164299190,18cyclictest0-21swapper/208:45:272
4191638991917,1cyclictest0-21swapper/112:32:201
4191638991917,1cyclictest0-21swapper/110:43:101
4191638991917,1cyclictest0-21swapper/110:31:421
4191638991917,1cyclictest0-21swapper/107:22:421
4191638991916,2cyclictest3056-21PK-Backend09:51:261
4191638991913,4cyclictest66816-21cron10:25:001
4191638991912,5cyclictest11776-21basename07:45:161
419163899190,1cyclictest3056-21PK-Backend11:54:021
419163899190,1cyclictest3056-21PK-Backend11:46:111
419163899190,1cyclictest3056-21PK-Backend11:38:541
419163899190,1cyclictest3056-21PK-Backend11:38:531
419163899190,1cyclictest3056-21PK-Backend11:21:301
419163899190,1cyclictest3056-21PK-Backend09:45:181
419163899190,1cyclictest3056-21PK-Backend08:45:501
419163899190,1cyclictest0-21swapper/112:13:491
419163899190,1cyclictest0-21swapper/112:09:231
419163899190,1cyclictest0-21swapper/110:00:051
419163899190,1cyclictest0-21swapper/109:10:301
419163899190,1cyclictest0-21swapper/107:52:571
419163899190,18cyclictest0-21swapper/110:56:361
4191633991917,1cyclictest0-21swapper/012:38:270
4191633991917,1cyclictest0-21swapper/010:54:320
4191633991917,1cyclictest0-21swapper/010:03:210
4191633991913,5cyclictest90217-21load11:30:160
419163399190,1cyclictest0-21swapper/012:33:480
419163399190,1cyclictest0-21swapper/012:04:280
419163399190,1cyclictest0-21swapper/008:48:590
419163399190,1cyclictest0-21swapper/007:21:460
419163399190,18cyclictest0-21swapper/008:30:480
139099190,13rtkit-daemon3056-21PK-Backend10:30:413
139099190,13rtkit-daemon3056-21PK-Backend10:19:483
4191647991813,3cyclictest4191828-21apt07:10:113
419164799180,1cyclictest3056-21PK-Backend12:03:003
419164799180,1cyclictest3056-21PK-Backend08:11:013
419164299182,12cyclictest96578-21apt11:50:112
4191642991816,1cyclictest0-21swapper/209:11:502
4191642991816,1cyclictest0-21swapper/208:06:412
4191642991810,6cyclictest51624-21which09:40:152
419163899189,7cyclictest30119-21sshd08:39:491
4191638991811,5cyclictest68549-21cron10:30:011
4191638991811,5cyclictest113248-21cat12:35:161
419163899180,9cyclictest89377-21perf11:30:011
4191633991816,1cyclictest0-21swapper/009:12:040
4191633991816,1cyclictest0-21swapper/007:47:210
419164799174,11cyclictest3056-21PK-Backend10:05:043
4191647991715,1cyclictest3056-21PK-Backend12:34:503
4191647991715,1cyclictest3056-21PK-Backend10:28:083
419164799170,15cyclictest26080-21python308:25:203
419164299179,6cyclictest44508-21expr09:20:132
4191642991711,4cyclictest16252-21sshd07:59:492
4191642991710,5cyclictest114194-21perf12:40:002
419164299170,7cyclictest10283-21switchtime07:40:202
4191638991715,1cyclictest3056-21PK-Backend09:01:381
4191638991715,1cyclictest0-21swapper/110:08:301
4191638991715,1cyclictest0-21swapper/107:13:501
4191638991714,2cyclictest3056-21PK-Backend11:57:061
4191638991712,3cyclictest76794-21timerwakeupswit10:50:211
4191638991712,3cyclictest1095-21runrttasks11:10:171
4191638991711,4cyclictest11037-21apt-key07:45:001
4191638991710,5cyclictest436-20systemd-journal09:59:551
419163899170,1cyclictest3056-21PK-Backend11:42:321
4191633991715,1cyclictest0-21swapper/011:29:260
4191633991712,3cyclictest78527-21timerwakeupswit10:55:220
4191633991710,5cyclictest81100-21grep11:05:120
4191647991614,1cyclictest3056-21PK-Backend11:19:473
4191647991614,1cyclictest3056-21PK-Backend07:50:333
419164799160,14cyclictest27688-21ntp_states08:30:173
4191642991614,1cyclictest0-21swapper/210:35:122
4191642991614,1cyclictest0-21swapper/210:35:112
419163899169,6cyclictest79203-21find11:00:111
419163899169,5cyclictest1-21systemd10:15:001
4191638991614,1cyclictest0-21swapper/107:35:331
419163899161,11cyclictest89648-21apt11:30:111
4191633991612,3cyclictest1095-21runrttasks07:35:160
4191633991611,3cyclictest20002-21acpi08:10:110
419164799159,5cyclictest3056-21PK-Backend10:40:003
419164799159,5cyclictest3056-21PK-Backend10:40:003
4191638991513,1cyclictest3056-21PK-Backend09:21:461
4191638991513,1cyclictest3056-21PK-Backend08:52:461
419163899147,5cyclictest23611-21cut08:20:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional