You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 01:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri Aug 28, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
402554222280,5sleep23056-21PK-Backend07:05:122
405498921020,5sleep24054506-21/usr/sbin/munin08:25:182
139099860,71rtkit-daemon0-21swapper/007:05:340
402655726445,13sleep30-21swapper/307:06:063
402537926240,16sleep10-21swapper/107:05:051
40682832500,4sleep00-21swapper/009:05:010
40563932490,3sleep30-21swapper/308:30:113
40563932490,3sleep30-21swapper/308:30:113
41231992470,3sleep00-21swapper/011:40:150
4026978992515,9cyclictest3056-21PK-Backend09:36:563
402697899250,3cyclictest4053536-21ntp_states08:20:183
402697899250,19cyclictest3056-21PK-Backend07:45:163
402697899250,19cyclictest0-21swapper/310:25:353
402697899250,15cyclictest4087943-21cstates10:00:133
402697899250,14cyclictest4065654-21/usr/sbin/munin08:55:183
402697899240,3cyclictest4061840-21df08:45:133
402697899240,18cyclictest0-21swapper/312:38:043
40829152230,5sleep00-21swapper/009:45:150
4026978992320,2cyclictest3056-21PK-Backend10:36:523
4026978992316,1cyclictest0-21swapper/311:23:013
402697899230,15cyclictest4063212-21apt08:50:123
4026973992321,1cyclictest0-21swapper/211:23:202
402697399230,22cyclictest0-21swapper/208:12:112
4026968992317,5cyclictest26-21ksoftirqd/111:00:111
4026964992315,6cyclictest4028663-21cron07:15:010
402697899221,8cyclictest4028061-21python307:10:193
4026978992217,3cyclictest4030286-21python307:15:213
402697899220,20cyclictest3056-21PK-Backend11:53:433
402697899220,20cyclictest3056-21PK-Backend08:12:463
402697899220,20cyclictest0-21swapper/309:55:413
4026973992220,1cyclictest0-21swapper/212:12:342
4026973992220,1cyclictest0-21swapper/211:50:112
4026973992220,1cyclictest0-21swapper/210:57:382
4026973992220,1cyclictest0-21swapper/210:53:322
4026973992220,1cyclictest0-21swapper/210:40:412
4026973992220,1cyclictest0-21swapper/208:56:312
4026973992220,1cyclictest0-21swapper/207:42:092
402697399220,21cyclictest0-21swapper/207:26:412
402696899223,17cyclictest4132417-21users12:05:211
4026968992220,1cyclictest3056-21PK-Backend08:58:451
4026968992220,1cyclictest0-21swapper/112:29:541
4026968992211,8cyclictest436-20systemd-journal12:15:001
402696499220,20cyclictest4127620-21dump-pmu-power11:55:000
139099220,16rtkit-daemon3056-21PK-Backend07:18:091
139099220,15rtkit-daemon3056-21PK-Backend10:16:421
139099220,14rtkit-daemon3056-21PK-Backend12:36:241
4026978992119,1cyclictest3056-21PK-Backend12:14:383
4026978992119,1cyclictest3056-21PK-Backend11:58:163
4026978992119,1cyclictest3056-21PK-Backend08:04:413
4026978992119,1cyclictest0-21swapper/312:34:533
4026978992119,1cyclictest0-21swapper/311:49:583
4026978992119,1cyclictest0-21swapper/311:41:263
4026978992119,1cyclictest0-21swapper/310:20:213
4026978992119,1cyclictest0-21swapper/310:06:493
4026978992119,1cyclictest0-21swapper/308:40:523
4026978992119,1cyclictest0-21swapper/308:38:073
4026978992118,2cyclictest3056-21PK-Backend10:53:203
402697899210,20cyclictest3056-21PK-Backend07:30:003
402697899210,20cyclictest0-21swapper/311:28:423
402697899210,20cyclictest0-21swapper/310:11:503
402697899210,19cyclictest3056-21PK-Backend10:40:083
402697899210,19cyclictest3056-21PK-Backend09:26:413
402697899210,19cyclictest3056-21PK-Backend09:26:413
402697899210,19cyclictest3056-21PK-Backend07:43:063
4026973992119,1cyclictest3056-21PK-Backend11:40:032
4026973992119,1cyclictest0-21swapper/211:47:472
4026973992119,1cyclictest0-21swapper/211:32:462
4026973992119,1cyclictest0-21swapper/211:03:052
4026973992119,1cyclictest0-21swapper/210:22:362
4026973992119,1cyclictest0-21swapper/210:05:472
4026973992119,1cyclictest0-21swapper/210:01:212
4026973992119,1cyclictest0-21swapper/209:56:202
4026973992119,1cyclictest0-21swapper/209:20:052
4026973992119,1cyclictest0-21swapper/209:10:382
4026973992119,1cyclictest0-21swapper/209:01:252
4026973992119,1cyclictest0-21swapper/208:54:232
4026973992119,1cyclictest0-21swapper/207:57:112
4026973992119,1cyclictest0-21swapper/207:14:312
4026973992113,6cyclictest4122023-21awk11:35:232
402697399210,20cyclictest0-21swapper/211:29:492
402697399210,20cyclictest0-21swapper/210:25:542
402697399210,20cyclictest0-21swapper/209:44:182
402697399210,20cyclictest0-21swapper/209:36:202
402697399210,20cyclictest0-21swapper/209:34:582
402697399210,20cyclictest0-21swapper/209:34:582
4026968992119,1cyclictest3056-21PK-Backend12:33:061
4026968992119,1cyclictest3056-21PK-Backend11:45:331
4026968992119,1cyclictest3056-21PK-Backend10:03:191
4026968992119,1cyclictest3056-21PK-Backend08:49:451
4026968992119,1cyclictest0-21swapper/112:15:481
4026968992119,1cyclictest0-21swapper/110:50:441
4026968992119,1cyclictest0-21swapper/110:43:211
4026968992119,1cyclictest0-21swapper/110:38:411
4026968992119,1cyclictest0-21swapper/109:34:191
4026968992119,1cyclictest0-21swapper/109:34:191
4026968992119,1cyclictest0-21swapper/109:26:021
4026968992119,1cyclictest0-21swapper/109:26:011
4026968992119,1cyclictest0-21swapper/108:34:591
4026968992119,1cyclictest0-21swapper/108:34:581
4026968992118,2cyclictest3056-21PK-Backend11:50:061
4026968992118,2cyclictest3056-21PK-Backend08:20:041
4026968992115,4cyclictest4121622-21grep11:35:181
4026968992115,3cyclictest4113728-21sh11:15:001
4026968992114,5cyclictest4041612-21idleruntime-cro07:50:001
4026968992111,8cyclictest883-21dbus-daemon10:00:001
402696899210,20cyclictest3056-21PK-Backend10:08:541
402696899210,19cyclictest3056-21PK-Backend10:28:481
402696899210,19cyclictest3056-21PK-Backend10:21:041
4026964992119,1cyclictest0-21swapper/009:07:460
4026964992113,6cyclictest4055243-21awk08:25:190
4026964992112,7cyclictest4076805-21apt-get09:30:000
4026964992112,7cyclictest4076805-21apt-get09:30:000
402696499210,20cyclictest3056-21PK-Backend08:20:130
402696499210,20cyclictest0-21swapper/011:26:290
402696499210,20cyclictest0-21swapper/011:24:200
402696499210,20cyclictest0-21swapper/010:20:520
402696499210,20cyclictest0-21swapper/007:57:080
139099210,14rtkit-daemon3056-21PK-Backend08:50:181
139099210,14rtkit-daemon0-21swapper/010:18:220
139099210,14rtkit-daemon0-21swapper/007:15:110
4026978992018,1cyclictest3056-21PK-Backend12:22:223
4026978992018,1cyclictest3056-21PK-Backend11:11:383
4026978992018,1cyclictest3056-21PK-Backend09:40:473
4026978992018,1cyclictest3056-21PK-Backend09:24:453
4026978992018,1cyclictest3056-21PK-Backend09:05:453
4026978992018,1cyclictest3056-21PK-Backend07:26:433
4026978992018,1cyclictest0-21swapper/312:15:263
4026978992018,1cyclictest0-21swapper/310:30:153
4026978992018,1cyclictest0-21swapper/310:18:593
4026978992017,2cyclictest3056-21PK-Backend12:05:463
4026978992017,2cyclictest3056-21PK-Backend11:04:353
4026978992014,1cyclictest3056-21PK-Backend07:35:283
402697899200,19cyclictest3056-21PK-Backend11:05:583
402697899200,19cyclictest3056-21PK-Backend09:31:083
402697899200,19cyclictest3056-21PK-Backend09:31:073
402697899200,19cyclictest3056-21PK-Backend09:14:453
402697899200,19cyclictest3056-21PK-Backend08:19:433
402697899200,19cyclictest3056-21PK-Backend07:23:253
402697899200,19cyclictest0-21swapper/308:28:143
4026973992018,1cyclictest0-21swapper/209:25:522
4026973992018,1cyclictest0-21swapper/209:25:522
4026973992018,1cyclictest0-21swapper/209:08:072
4026973992018,1cyclictest0-21swapper/208:16:032
4026973992016,3cyclictest4054254-21dump-pmu-power08:24:592
402697399200,1cyclictest0-21swapper/212:04:502
402697399200,1cyclictest0-21swapper/209:18:422
402697399200,19cyclictest0-21swapper/212:06:262
402697399200,19cyclictest0-21swapper/211:57:542
402697399200,19cyclictest0-21swapper/211:18:142
402697399200,19cyclictest0-21swapper/210:48:062
402697399200,19cyclictest0-21swapper/210:36:362
402697399200,19cyclictest0-21swapper/209:54:242
402697399200,19cyclictest0-21swapper/208:41:512
402697399200,19cyclictest0-21swapper/208:03:182
402697399200,19cyclictest0-21swapper/207:35:402
402697399200,19cyclictest0-21swapper/207:30:472
402697399200,19cyclictest0-21swapper/207:22:032
402696899207,7cyclictest41278832chrt11:55:121
4026968992018,1cyclictest3056-21PK-Backend11:28:511
4026968992018,1cyclictest3056-21PK-Backend08:38:161
4026968992018,1cyclictest0-21swapper/111:34:311
4026968992018,1cyclictest0-21swapper/111:08:151
4026968992018,1cyclictest0-21swapper/109:01:291
4026968992018,1cyclictest0-21swapper/108:16:121
4026968992018,1cyclictest0-21swapper/107:53:211
4026968992018,1cyclictest0-21swapper/107:41:241
4026968992018,1cyclictest0-21swapper/107:25:371
4026968992014,4cyclictest2175-21JS10:27:481
4026968992013,5cyclictest26-21ksoftirqd/107:30:221
4026968992012,6cyclictest918-21rs:main1
402696899200,19cyclictest3056-21PK-Backend08:27:481
402696899200,19cyclictest0-21swapper/110:58:431
402696899200,19cyclictest0-21swapper/109:16:351
402696899200,19cyclictest0-21swapper/109:06:141
402696899200,19cyclictest0-21swapper/107:37:051
402696899200,18cyclictest3056-21PK-Backend08:44:171
402696499209,5cyclictest436-20systemd-journal09:20:180
4026964992018,1cyclictest0-21swapper/012:12:270
4026964992018,1cyclictest0-21swapper/011:11:350
4026964992018,1cyclictest0-21swapper/010:31:540
4026964992018,1cyclictest0-21swapper/010:10:380
4026964992018,1cyclictest0-21swapper/007:22:270
4026964992012,6cyclictest4036718-21df07:35:130
402696499200,1cyclictest0-21swapper/010:55:470
402696499200,1cyclictest0-21swapper/008:31:150
402696499200,1cyclictest0-21swapper/008:31:150
402696499200,1cyclictest0-21swapper/008:16:210
402696499200,19cyclictest0-21swapper/012:33:280
402696499200,19cyclictest0-21swapper/010:35:120
402696499200,19cyclictest0-21swapper/008:06:210
139099200,14rtkit-daemon0-21swapper/011:45:410
139099200,14rtkit-daemon0-21swapper/008:03:490
40743472190,3sleep10-21swapper/109:20:171
4026978991917,1cyclictest3056-21PK-Backend12:02:013
4026978991917,1cyclictest3056-21PK-Backend11:35:453
4026978991917,1cyclictest3056-21PK-Backend11:15:103
402697899190,1cyclictest3056-21PK-Backend07:56:513
4026973991917,1cyclictest0-21swapper/210:19:192
4026973991917,1cyclictest0-21swapper/208:39:082
4026973991917,1cyclictest0-21swapper/207:52:582
4026973991911,6cyclictest4061472-21awk08:45:122
402697399190,1cyclictest0-21swapper/211:13:062
402697399190,1cyclictest0-21swapper/207:48:042
4026968991917,1cyclictest3056-21PK-Backend11:44:451
4026968991917,1cyclictest0-21swapper/109:13:311
4026968991917,1cyclictest0-21swapper/108:09:511
4026968991917,1cyclictest0-21swapper/108:01:121
402696899190,1cyclictest3056-21PK-Backend10:13:541
4026964991917,1cyclictest0-21swapper/011:55:500
4026964991917,1cyclictest0-21swapper/011:17:170
4026964991917,1cyclictest0-21swapper/011:05:540
4026964991917,1cyclictest0-21swapper/009:57:000
4026964991917,1cyclictest0-21swapper/009:32:130
4026964991917,1cyclictest0-21swapper/009:32:130
4026964991917,1cyclictest0-21swapper/008:49:200
4026964991917,1cyclictest0-21swapper/008:37:520
4026964991917,1cyclictest0-21swapper/007:51:470
4026964991917,1cyclictest0-21swapper/007:42:220
4026964991917,1cyclictest0-21swapper/007:33:080
4026964991912,5cyclictest4105322-21awk10:50:110
4026964991911,6cyclictest4134813-21apt12:15:130
402696499190,1cyclictest0-21swapper/012:35:510
402696499190,1cyclictest0-21swapper/012:03:200
402696499190,1cyclictest0-21swapper/011:30:010
402696499190,1cyclictest0-21swapper/010:41:480
402696499190,1cyclictest0-21swapper/010:08:420
402696499190,1cyclictest0-21swapper/008:55:350
402696499190,1cyclictest0-21swapper/008:42:310
402696499190,1cyclictest0-21swapper/008:14:260
402696499190,1cyclictest0-21swapper/007:49:380
402696499190,18cyclictest0-21swapper/012:07:530
402696499190,18cyclictest0-21swapper/010:46:280
402696499190,18cyclictest0-21swapper/009:17:320
4026978991816,1cyclictest3056-21PK-Backend09:49:533
4026978991816,1cyclictest0-21swapper/310:58:003
402697399185,12cyclictest4030879-21cron07:20:002
4026973991816,1cyclictest0-21swapper/210:31:562
4026973991812,5cyclictest4137585-21which12:20:202
402696899183,13cyclictest4081480-21cut09:40:161
4026968991816,1cyclictest3056-21PK-Backend11:18:161
4026968991816,1cyclictest3056-21PK-Backend10:30:101
4026968991816,1cyclictest0-21swapper/109:35:411
4026968991816,1cyclictest0-21swapper/108:11:461
402696899181,12cyclictest4031158-21apt07:20:111
402696899180,1cyclictest0-21swapper/109:51:241
4026964991817,1cyclictest0-21swapper/009:10:500
4026964991816,1cyclictest0-21swapper/010:28:370
4026964991810,6cyclictest4082257-21cron09:45:000
4026978991715,1cyclictest3056-21PK-Backend12:25:333
4026978991715,1cyclictest3056-21PK-Backend11:34:233
4026978991715,1cyclictest3056-21PK-Backend09:51:153
4026973991714,2cyclictest0-21swapper/210:14:062
4026973991712,3cyclictest978-21snmpd12:32:432
402697399171,12cyclictest4142026-21apt12:35:102
402696899179,6cyclictest4103952-21df10:45:131
402696899179,6cyclictest4082898-21df09:45:131
4026968991715,1cyclictest3056-21PK-Backend11:20:251
4026964991715,1cyclictest0-21swapper/009:52:330
4026964991712,3cyclictest4122414-21latency_hist11:40:010
4026964991712,3cyclictest4109836-21timerwakeupswit11:00:210
4026964991711,5cyclictest4138826-21expr12:25:120
4026964991711,5cyclictest4033417-21sed07:25:160
4026978991614,1cyclictest3056-21PK-Backend08:09:423
402697899160,3cyclictest4066676-21apt09:00:113
4026973991614,1cyclictest0-21swapper/211:08:472
4026973991614,1cyclictest0-21swapper/209:48:242
4026973991614,1cyclictest0-21swapper/208:33:062
4026973991614,1cyclictest0-21swapper/208:33:062
4026968991614,1cyclictest0-21swapper/112:03:461
402696499169,6cyclictest4079289-21expr09:35:090
4026964991611,3cyclictest4088708-21timerandwakeup10:00:230
4026978991513,1cyclictest3056-21PK-Backend10:48:063
4026978991513,1cyclictest3056-21PK-Backend09:16:403
402697399159,4cyclictest0-21swapper/208:05:092
402696899151,10cyclictest4027200-21apt07:10:101
4026964991511,3cyclictest4136890-21cpuspeed12:20:120
4026964991510,3cyclictest1-21systemd08:55:000
402697899140,13cyclictest3056-21PK-Backend07:53:223
402697399147,5cyclictest1009-21lldpd12:18:232
402697399140,12cyclictest4138531-21awk12:25:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional