You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 00:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Sep 04, 2026 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
380008027839,33sleep10-21swapper/107:08:551
379936727853,16sleep30-21swapper/307:05:223
379987126344,14sleep00-21swapper/007:06:120
379929726343,14sleep20-21swapper/207:05:212
38585902550,4sleep30-21swapper/309:55:013
38850932510,4sleep00-21swapper/011:10:160
38031012500,1sleep30-21swapper/307:15:143
38483182330,2sleep33842834-21kworker/u8:109:25:023
3800304993210,6cyclictest3853466-21ls09:40:002
380030499313,8cyclictest3914604-21apt12:35:102
380030499311,11cyclictest3822133-21apt08:10:122
3800304993110,6cyclictest3855033-21sshd09:44:412
380030499308,7cyclictest883-21dbus-daemon09:35:002
380030499299,4cyclictest3860469-21latency_hist10:00:012
380030499298,6cyclictest3847416-21ntp_states09:20:182
380030499292,6cyclictest3884208-21python311:05:192
3800304992910,4cyclictest3866465-21ntp_kernel_pll_10:15:202
3800306992823,3cyclictest42-21ksoftirqd/311:20:003
380030499289,3cyclictest978-21snmpd07:38:362
380030499288,6cyclictest3849267-21sensors_temp09:25:192
380030499288,5cyclictest3843096-21date09:10:012
380030499287,6cyclictest3820623-21expr08:05:122
380030499287,5cyclictest3806188-21cat07:25:122
380030499287,4cyclictest3911269-21grep12:25:132
380030499287,4cyclictest3911269-21grep12:25:132
380030499287,4cyclictest0-21swapper/209:10:212
380030499286,6cyclictest3856230-21expr09:45:202
380030499286,5cyclictest3885448-21df11:10:172
380030499282,5cyclictest0-21swapper/210:05:102
380030499282,4cyclictest0-21swapper/209:00:112
380030499282,4cyclictest0-21swapper/207:15:112
380030499278,4cyclictest3864661-21needreboot10:10:222
380030499277,6cyclictest3829238-21cpuspeed_turbos08:30:122
380030499271,9cyclictest3890278-21apt11:25:112
380030499271,9cyclictest3816663-21apt07:55:092
380030499268,3cyclictest3820251-21latency_hist08:05:012
380030499266,5cyclictest3838327-21expr08:55:112
380030499261,8cyclictest82332-21update-manager07:33:372
3800304992611,2cyclictest3832556-21cpu08:40:122
3800304992611,2cyclictest3832556-21cpu08:40:112
380030499260,10cyclictest3877798-21cat10:50:012
380030499259,4cyclictest978-21snmpd10:39:032
380030499257,4cyclictest3812940-21munin-plugin-st07:45:002
380030499252,6cyclictest2156-21gnome-shell08:35:092
380030499251,9cyclictest3857942-21python309:50:222
380030499250,8cyclictest3868964-21idleruntime-cro10:25:002
380030499250,8cyclictest3815411-21irqstats07:50:162
380030499250,19cyclictest0-21swapper/212:10:472
380030499249,2cyclictest3895217-21munin-plugin-st11:40:002
380030499243,7cyclictest3890016-21munin-run11:25:002
380030499243,7cyclictest3890016-21munin-run11:25:002
380030499241,9cyclictest3892594-21/usr/sbin/munin11:30:212
380030499241,8cyclictest3905003-21python312:05:202
380030499241,8cyclictest3828193-21python308:25:202
380030499241,5cyclictest3901536-21python311:55:222
380030499240,7cyclictest3860705-21tr10:00:122
139099240,20rtkit-daemon1788620-21WebKitWebProces07:22:370
380030499238,4cyclictest34-21ksoftirqd/208:25:012
380030499231,7cyclictest3870690-21dump-pmu-power10:30:002
380030499231,7cyclictest3801971-21perf07:15:002
380030499231,6cyclictest3907658-21apt12:15:112
380030499231,6cyclictest3877621-21apt-get10:50:042
380030499230,8cyclictest3874425-21apt10:40:112
139099230,3rtkit-daemon1389-21rtkit-daemon10:48:300
38488062220,4chrt0-21swapper/009:25:140
380030699227,13cyclictest3816596-21ls07:55:003
3800306992220,1cyclictest0-21swapper/308:40:073
3800306992220,1cyclictest0-21swapper/308:40:073
3800306992219,2cyclictest3056-21PK-Backend11:43:053
3800306992218,3cyclictest42-21ksoftirqd/310:20:003
3800306992218,3cyclictest0-21swapper/308:25:133
3800306992216,4cyclictest42-21ksoftirqd/309:50:003
3800306992214,6cyclictest3849351-21/usr/sbin/munin09:25:203
380030699220,21cyclictest3056-21PK-Backend08:14:183
380030499224,6cyclictest0-21swapper/208:50:112
3800304992219,2cyclictest0-21swapper/208:17:212
380030499220,21cyclictest0-21swapper/212:02:432
3800300992220,1cyclictest0-21swapper/112:01:521
3800300992219,2cyclictest3905984-21cpuspeed_turbos12:10:111
3800300992212,8cyclictest3872413-21cron10:35:001
3800300992210,11cyclictest3835784-21unin-run08:50:001
3800297992220,1cyclictest0-21swapper/012:12:420
3800297992220,1cyclictest0-21swapper/011:58:230
3800297992220,1cyclictest0-21swapper/010:13:320
3800297992220,1cyclictest0-21swapper/009:48:310
3800297992220,1cyclictest0-21swapper/008:31:330
3800297992219,2cyclictest3056-21PK-Backend09:20:460
3800297992216,4cyclictest2156-21gnome-shell07:31:090
3800297992214,5cyclictest3819529-21python308:00:200
380029799220,21cyclictest0-21swapper/010:04:330
139099220,4rtkit-daemon1389-21rtkit-daemon11:07:510
139099220,4rtkit-daemon1389-21rtkit-daemon09:53:060
139099220,4rtkit-daemon1389-21rtkit-daemon09:37:250
139099220,4rtkit-daemon1389-21rtkit-daemon08:16:580
3800306992119,1cyclictest3056-21PK-Backend11:27:583
3800306992119,1cyclictest3056-21PK-Backend11:00:263
3800306992119,1cyclictest3056-21PK-Backend10:42:353
3800306992119,1cyclictest3056-21PK-Backend10:39:043
3800306992119,1cyclictest3056-21PK-Backend10:27:553
3800306992119,1cyclictest3056-21PK-Backend08:52:373
3800306992119,1cyclictest0-21swapper/311:06:083
3800306992119,1cyclictest0-21swapper/310:05:513
3800306992119,1cyclictest0-21swapper/309:35:433
3800306992119,1cyclictest0-21swapper/309:09:003
3800306992119,1cyclictest0-21swapper/309:03:393
3800306992119,1cyclictest0-21swapper/307:28:353
3800306992118,2cyclictest3056-21PK-Backend12:08:473
3800306992116,4cyclictest42-21ksoftirqd/310:25:003
380030699210,20cyclictest0-21swapper/307:31:123
380030499210,20cyclictest0-21swapper/211:52:292
3800300992119,1cyclictest3056-21PK-Backend09:31:181
3800300992119,1cyclictest3056-21PK-Backend09:29:561
3800300992119,1cyclictest3056-21PK-Backend09:05:431
3800300992119,1cyclictest0-21swapper/112:25:381
3800300992119,1cyclictest0-21swapper/112:25:371
3800300992119,1cyclictest0-21swapper/112:09:431
3800300992119,1cyclictest0-21swapper/110:45:061
3800300992119,1cyclictest0-21swapper/110:41:351
3800300992119,1cyclictest0-21swapper/108:56:511
3800300992119,1cyclictest0-21swapper/108:40:011
3800300992119,1cyclictest0-21swapper/108:40:011
3800300992118,2cyclictest3056-21PK-Backend10:11:541
380030099210,20cyclictest3056-21PK-Backend09:22:471
380030099210,20cyclictest0-21swapper/112:24:501
380030099210,20cyclictest0-21swapper/111:15:421
380030099210,20cyclictest0-21swapper/107:59:061
380030099210,19cyclictest3056-21PK-Backend11:05:411
380030099210,19cyclictest3056-21PK-Backend08:32:101
380030099210,19cyclictest3056-21PK-Backend08:05:201
3800297992119,1cyclictest0-21swapper/009:16:210
3800297992119,1cyclictest0-21swapper/009:14:440
3800297992119,1cyclictest0-21swapper/008:22:190
3800297992119,1cyclictest0-21swapper/007:58:260
3800297992119,1cyclictest0-21swapper/007:27:440
380029799210,20cyclictest0-21swapper/012:21:210
380029799210,20cyclictest0-21swapper/011:44:030
380029799210,20cyclictest0-21swapper/008:35:030
380029799210,20cyclictest0-21swapper/007:47:250
380029799210,20cyclictest0-21swapper/007:47:240
380029799210,1cyclictest0-21swapper/010:19:340
139099210,3rtkit-daemon1389-21rtkit-daemon12:31:330
139099210,3rtkit-daemon1389-21rtkit-daemon08:10:500
139099210,18rtkit-daemon1095-21runrttasks10:56:521
38255462200,4sleep33056-21PK-Backend08:20:013
380030699209,9cyclictest3846443-21unin-run09:20:013
3800306992018,1cyclictest3056-21PK-Backend10:34:103
3800306992018,1cyclictest0-21swapper/312:39:023
3800306992018,1cyclictest0-21swapper/308:59:543
3800306992018,1cyclictest0-21swapper/308:49:063
3800306992017,2cyclictest3056-21PK-Backend10:58:173
3800306992017,2cyclictest3056-21PK-Backend08:23:033
380030699200,1cyclictest0-21swapper/310:50:263
380030699200,19cyclictest3056-21PK-Backend12:10:573
380030699200,19cyclictest3056-21PK-Backend12:00:563
380030699200,19cyclictest3056-21PK-Backend08:02:223
380030699200,19cyclictest3056-21PK-Backend07:38:493
380030699200,19cyclictest0-21swapper/311:20:413
380030699200,19cyclictest0-21swapper/311:20:413
380030699200,19cyclictest0-21swapper/311:14:463
380030699200,19cyclictest0-21swapper/310:02:543
380030699200,19cyclictest0-21swapper/309:58:353
380030699200,19cyclictest0-21swapper/308:39:193
380030699200,19cyclictest0-21swapper/308:06:013
380030699200,19cyclictest0-21swapper/307:14:363
3800304992018,1cyclictest0-21swapper/211:40:532
3800304992017,2cyclictest0-21swapper/209:19:182
380030499201,5cyclictest0-21swapper/210:30:162
380030499200,1cyclictest0-21swapper/211:47:212
380030499200,1cyclictest0-21swapper/210:55:582
380030499200,19cyclictest0-21swapper/212:30:412
380030499200,19cyclictest0-21swapper/212:21:492
3800300992018,1cyclictest3056-21PK-Backend11:13:321
3800300992018,1cyclictest3056-21PK-Backend09:57:481
3800300992018,1cyclictest3056-21PK-Backend09:50:111
3800300992018,1cyclictest3056-21PK-Backend09:37:541
3800300992018,1cyclictest3056-21PK-Backend08:26:021
3800300992018,1cyclictest3056-21PK-Backend07:33:231
3800300992018,1cyclictest3056-21PK-Backend07:24:581
3800300992018,1cyclictest0-21swapper/108:23:111
3800300992013,6cyclictest3881847-21ls11:00:121
380030099200,1cyclictest3056-21PK-Backend09:04:551
380030099200,19cyclictest3056-21PK-Backend10:09:101
380030099200,19cyclictest3056-21PK-Backend08:03:311
380030099200,19cyclictest3056-21PK-Backend07:29:581
380030099200,19cyclictest0-21swapper/111:55:231
380030099200,19cyclictest0-21swapper/111:54:351
380030099200,19cyclictest0-21swapper/111:46:441
380030099200,19cyclictest0-21swapper/111:31:371
380030099200,19cyclictest0-21swapper/111:20:011
380030099200,19cyclictest0-21swapper/111:20:011
380030099200,19cyclictest0-21swapper/109:14:491
380030099200,19cyclictest0-21swapper/108:52:321
380030099200,19cyclictest0-21swapper/108:14:261
380030099200,19cyclictest0-21swapper/107:43:371
3800297992018,1cyclictest0-21swapper/011:52:420
3800297992018,1cyclictest0-21swapper/011:34:030
3800297992018,1cyclictest0-21swapper/010:59:280
3800297992018,1cyclictest0-21swapper/009:33:440
3800297992018,1cyclictest0-21swapper/009:02:410
3800297992018,1cyclictest0-21swapper/007:13:250
380029799200,1cyclictest0-21swapper/011:49:440
380029799200,1cyclictest0-21swapper/011:01:380
380029799200,19cyclictest0-21swapper/011:38:220
380029799200,19cyclictest0-21swapper/010:51:380
380029799200,19cyclictest0-21swapper/010:36:100
380029799200,19cyclictest0-21swapper/010:31:160
380029799200,19cyclictest0-21swapper/010:06:430
380029799200,19cyclictest0-21swapper/008:55:250
380029799200,19cyclictest0-21swapper/008:42:060
380029799200,19cyclictest0-21swapper/008:42:060
380029799200,19cyclictest0-21swapper/007:18:450
139099200,3rtkit-daemon1389-21rtkit-daemon09:42:520
38680242190,4sleep13056-21PK-Backend10:20:171
380030699192,2cyclictest141rcu_preempt11:50:203
3800306991917,1cyclictest3056-21PK-Backend12:21:453
3800306991917,1cyclictest3056-21PK-Backend12:19:353
3800306991917,1cyclictest3056-21PK-Backend07:45:053
3800306991917,1cyclictest3056-21PK-Backend07:45:053
3800306991917,1cyclictest0-21swapper/310:48:163
3800306991917,1cyclictest0-21swapper/310:10:113
3800306991917,1cyclictest0-21swapper/309:41:113
3800306991917,1cyclictest0-21swapper/308:30:073
380030699190,1cyclictest3056-21PK-Backend11:55:153
380030699190,1cyclictest3056-21PK-Backend11:35:143
380030699190,1cyclictest0-21swapper/312:26:523
380030699190,1cyclictest0-21swapper/312:26:523
380030699190,18cyclictest3056-21PK-Backend11:48:463
380030699190,17cyclictest3913004-21expr12:30:103
3800304991917,1cyclictest0-21swapper/208:48:092
3800304991913,5cyclictest0-21swapper/211:01:532
3800304991912,1cyclictest0-21swapper/207:24:272
380030499190,1cyclictest3056-21PK-Backend07:45:222
380030499190,1cyclictest3056-21PK-Backend07:45:222
380030499190,1cyclictest0-21swapper/211:16:262
3800300991917,1cyclictest3056-21PK-Backend12:32:071
3800300991917,1cyclictest3056-21PK-Backend07:47:151
3800300991917,1cyclictest3056-21PK-Backend07:47:151
3800300991916,2cyclictest3056-21PK-Backend12:35:381
3800300991912,5cyclictest3824223-21df08:15:111
380030099190,1cyclictest3056-21PK-Backend10:01:201
380030099190,1cyclictest3056-21PK-Backend09:16:241
380030099190,1cyclictest0-21swapper/111:36:441
380030099190,1cyclictest0-21swapper/111:28:401
380030099190,18cyclictest0-21swapper/112:18:211
3800297991917,1cyclictest0-21swapper/012:27:500
3800297991917,1cyclictest0-21swapper/012:27:490
3800297991917,1cyclictest0-21swapper/012:19:110
3800297991917,1cyclictest0-21swapper/012:08:230
3800297991917,1cyclictest0-21swapper/011:24:360
3800297991917,1cyclictest0-21swapper/011:24:360
3800297991917,1cyclictest0-21swapper/010:43:130
3800297991917,1cyclictest0-21swapper/009:06:260
3800297991917,1cyclictest0-21swapper/008:26:040
3800297991913,4cyclictest3820536-21cut08:05:120
3800297991911,6cyclictest3869961-21ntpq10:25:180
380029799190,1cyclictest0-21swapper/010:23:390
380029799190,1cyclictest0-21swapper/008:54:380
380029799190,1cyclictest0-21swapper/008:46:120
38042042180,3sleep13056-21PK-Backend07:20:001
3800306991816,1cyclictest3056-21PK-Backend07:59:453
3800306991816,1cyclictest3056-21PK-Backend07:43:153
3800306991813,3cyclictest3851642-21munin-run09:35:003
3800306991813,3cyclictest3843186-21/usr/sbin/munin09:10:133
3800306991811,5cyclictest3804777-21expr07:20:123
380030099189,7cyclictest3879016-21python310:50:241
3800300991816,1cyclictest3056-21PK-Backend08:37:511
3800300991816,1cyclictest0-21swapper/110:39:251
3800300991811,5cyclictest3869769-21ls10:25:161
3800300991810,7cyclictest886-21NetworkManager11:40:121
3800300991810,6cyclictest3801257-21ntpq07:10:181
380030099180,1cyclictest3056-21PK-Backend09:43:561
380030099180,1cyclictest3056-21PK-Backend07:51:281
380029799185,11cyclictest3811206-21grep07:39:590
3800306991715,1cyclictest3056-21PK-Backend11:30:553
3800300991715,1cyclictest0-21swapper/107:36:491
380029799179,6cyclictest3812329-21python307:40:200
3800297991715,1cyclictest0-21swapper/011:25:240
3800297991711,4cyclictest3815322-21forks07:50:140
3800297991710,5cyclictest3914816-21ls12:35:110
3800300991614,1cyclictest3056-21PK-Backend10:16:001
380029799168,6cyclictest3902623-21expr12:00:130
380029799163,8cyclictest3886808-21apt11:15:090
3800297991610,5cyclictest3859358-21ls09:55:160
139099160,13rtkit-daemon3056-21PK-Backend09:49:401
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional