You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 17:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jul 21, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
370767521060,3sleep10-21swapper/110:55:111
362594928047,22sleep00-21swapper/007:05:120
139099770,66rtkit-daemon0-21swapper/207:06:162
362734527539,13sleep30-21swapper/307:09:053
139099720,58rtkit-daemon0-21swapper/107:05:021
36479512580,4sleep23647956-21cpuspeed08:05:142
37160742440,3sleep03716070-21munin-run11:20:010
37077702420,3sleep33056-21PK-Backend10:55:143
36595982380,3sleep03056-21PK-Backend08:35:240
36482072380,2sleep33056-21PK-Backend08:05:163
37252372330,2sleep10-21swapper/111:45:111
37067402290,3sleep10-21swapper/110:50:221
36771682290,12sleep234-21ksoftirqd/209:25:222
362756499270,25cyclictest1389-21rtkit-daemon09:32:243
139099270,4rtkit-daemon1389-21rtkit-daemon09:33:532
362756499260,24cyclictest1389-21rtkit-daemon11:41:463
139099260,4rtkit-daemon1389-21rtkit-daemon12:17:312
362756499250,23cyclictest1389-21rtkit-daemon07:39:133
139099250,4rtkit-daemon1389-21rtkit-daemon10:11:372
3627562992415,7cyclictest3663518-21unin-run08:50:012
3627559992421,2cyclictest3056-21PK-Backend10:10:541
3627559992421,2cyclictest0-21swapper/111:22:521
362755799240,3cyclictest3662942-21ls08:45:230
36639922230,9sleep234-21ksoftirqd/208:50:142
3627564992319,2cyclictest3056-21PK-Backend11:17:093
362756499230,22cyclictest82332-21update-manager09:24:003
3627559992321,1cyclictest0-21swapper/112:27:341
3627559992321,1cyclictest0-21swapper/109:38:221
3627559992321,1cyclictest0-21swapper/109:30:251
3627559992321,1cyclictest0-21swapper/108:13:111
3627559992320,2cyclictest0-21swapper/111:15:011
362755799239,6cyclictest13-21ksoftirqd/008:20:230
3627557992312,9cyclictest3684619-21unin-run09:50:000
139099230,4rtkit-daemon1389-21rtkit-daemon10:47:573
3627564992211,8cyclictest436-20systemd-journal10:34:053
3627562992220,1cyclictest0-21swapper/212:09:252
3627562992220,1cyclictest0-21swapper/211:46:002
3627562992220,1cyclictest0-21swapper/211:34:452
3627562992220,1cyclictest0-21swapper/211:15:112
3627562992220,1cyclictest0-21swapper/210:33:212
3627562992220,1cyclictest0-21swapper/209:42:102
3627562992214,6cyclictest3658034-21perf08:34:592
362756299220,20cyclictest0-21swapper/212:10:132
362756299220,20cyclictest0-21swapper/208:17:412
3627559992220,1cyclictest0-21swapper/110:36:501
3627559992220,1cyclictest0-21swapper/109:28:021
3627559992220,1cyclictest0-21swapper/108:44:011
3627559992220,1cyclictest0-21swapper/108:26:031
3627559992220,1cyclictest0-21swapper/108:17:241
3627559992219,2cyclictest3056-21PK-Backend11:38:141
3627559992211,9cyclictest3712361-21unin-run11:10:011
362755999220,21cyclictest3056-21PK-Backend10:19:121
362755999220,21cyclictest3056-21PK-Backend10:03:501
362755999220,21cyclictest0-21swapper/112:21:051
362755999220,21cyclictest0-21swapper/112:16:461
362755999220,21cyclictest0-21swapper/110:25:191
362755999220,21cyclictest0-21swapper/109:16:321
362755999220,21cyclictest0-21swapper/108:51:171
362755999220,21cyclictest0-21swapper/108:33:061
362755799228,12cyclictest3717779-21cron11:25:010
3627557992220,1cyclictest0-21swapper/011:53:270
3627557992220,1cyclictest0-21swapper/010:35:130
3627557992213,7cyclictest3740206-21users12:25:240
3627557992213,7cyclictest3707771-21/usr/sbin/munin10:55:140
362755799220,6cyclictest3742329-21dump-pmu-power12:34:590
362755799220,21cyclictest3056-21PK-Backend09:24:020
362755799220,21cyclictest0-21swapper/010:33:380
362755799220,21cyclictest0-21swapper/010:07:150
362755799220,21cyclictest0-21swapper/009:56:000
362755799220,21cyclictest0-21swapper/009:34:580
362755799220,21cyclictest0-21swapper/008:56:240
362755799220,21cyclictest0-21swapper/008:27:380
139099220,4rtkit-daemon1389-21rtkit-daemon08:36:403
139099220,19rtkit-daemon3629631-21munin-node07:28:242
362756499218,11cyclictest42-21ksoftirqd/308:34:593
3627564992114,6cyclictest3629912-21users07:15:033
362756499210,19cyclictest3056-21PK-Backend12:35:433
362756499210,19cyclictest3056-21PK-Backend12:16:383
362756499210,19cyclictest3056-21PK-Backend11:04:453
362756499210,19cyclictest3056-21PK-Backend10:54:593
362756499210,19cyclictest3056-21PK-Backend08:20:333
362756499210,19cyclictest3056-21PK-Backend07:54:383
3627562992119,1cyclictest3056-21PK-Backend12:28:122
3627562992119,1cyclictest0-21swapper/211:59:462
3627562992119,1cyclictest0-21swapper/211:52:302
3627562992119,1cyclictest0-21swapper/211:00:312
3627562992119,1cyclictest0-21swapper/210:50:302
3627562992119,1cyclictest0-21swapper/209:07:492
3627562992119,1cyclictest0-21swapper/208:21:192
3627562992119,1cyclictest0-21swapper/207:52:062
3627562992119,1cyclictest0-21swapper/207:12:302
362756299210,20cyclictest0-21swapper/208:13:142
362756299210,1cyclictest0-21swapper/208:39:092
3627559992119,1cyclictest3056-21PK-Backend11:42:331
3627559992119,1cyclictest3056-21PK-Backend11:27:111
3627559992119,1cyclictest3056-21PK-Backend10:20:481
3627559992119,1cyclictest0-21swapper/111:12:521
3627559992119,1cyclictest0-21swapper/109:57:491
3627559992119,1cyclictest0-21swapper/109:47:351
3627559992119,1cyclictest0-21swapper/109:43:221
3627559992119,1cyclictest0-21swapper/108:58:001
3627559992119,1cyclictest0-21swapper/107:47:091
3627559992119,1cyclictest0-21swapper/107:19:311
3627559992113,6cyclictest436-20systemd-journal09:10:011
362755999210,20cyclictest0-21swapper/111:01:501
362755999210,20cyclictest0-21swapper/109:51:481
362755999210,20cyclictest0-21swapper/107:55:551
362755999210,1cyclictest3056-21PK-Backend09:21:051
362755999210,1cyclictest0-21swapper/112:01:531
362755999210,1cyclictest0-21swapper/111:30:091
362755999210,1cyclictest0-21swapper/110:49:541
362755999210,1cyclictest0-21swapper/109:11:591
3627557992119,1cyclictest0-21swapper/012:22:280
3627557992119,1cyclictest0-21swapper/011:09:540
3627557992119,1cyclictest0-21swapper/010:29:060
362755799210,20cyclictest0-21swapper/009:25:370
362755799210,20cyclictest0-21swapper/009:18:420
362755799210,20cyclictest0-21swapper/009:11:120
362755799210,20cyclictest0-21swapper/009:00:570
362755799210,20cyclictest0-21swapper/008:33:540
362755799210,1cyclictest0-21swapper/011:43:070
362755799210,1cyclictest0-21swapper/011:28:330
362755799210,1cyclictest0-21swapper/010:47:310
139099210,4rtkit-daemon1389-21rtkit-daemon12:39:132
139099210,4rtkit-daemon1389-21rtkit-daemon09:45:422
3627564992018,1cyclictest3056-21PK-Backend08:59:263
3627564992018,1cyclictest3056-21PK-Backend07:57:143
3627564992018,1cyclictest3056-21PK-Backend07:45:393
3627564992017,2cyclictest3056-21PK-Backend08:53:183
362756499200,1cyclictest3056-21PK-Backend11:22:163
362756499200,1cyclictest3056-21PK-Backend09:51:383
362756499200,19cyclictest3056-21PK-Backend12:11:313
362756499200,19cyclictest3056-21PK-Backend08:46:363
362756499200,19cyclictest0-21swapper/311:52:383
362756499200,19cyclictest0-21swapper/310:03:553
362756499200,19cyclictest0-21swapper/307:40:523
362756499200,18cyclictest3056-21PK-Backend12:27:393
362756499200,18cyclictest3056-21PK-Backend12:08:323
362756499200,18cyclictest3056-21PK-Backend09:11:303
362756499200,18cyclictest3056-21PK-Backend08:11:073
3627562992018,1cyclictest0-21swapper/212:32:452
3627562992018,1cyclictest0-21swapper/210:55:302
3627562992018,1cyclictest0-21swapper/207:46:312
3627562992018,1cyclictest0-21swapper/207:31:242
3627562992017,2cyclictest0-21swapper/211:08:292
3627562992012,6cyclictest3630062-21ntp_kernel_err07:15:032
362756299200,1cyclictest0-21swapper/212:23:392
362756299200,19cyclictest0-21swapper/210:26:582
362756299200,19cyclictest0-21swapper/209:51:032
362756299200,19cyclictest0-21swapper/209:37:432
362756299200,19cyclictest0-21swapper/207:58:072
362755999203,13cyclictest3635454-21apt07:30:131
3627559992018,1cyclictest0-21swapper/112:30:331
3627559992018,1cyclictest0-21swapper/112:13:151
3627559992018,1cyclictest0-21swapper/112:07:341
3627559992018,1cyclictest0-21swapper/110:42:581
3627559992018,1cyclictest0-21swapper/108:07:241
3627559992018,1cyclictest0-21swapper/107:28:581
3627559992016,2cyclictest3056-21PK-Backend10:32:161
3627559992010,7cyclictest3643931-21unin-run07:55:001
362755999200,19cyclictest0-21swapper/108:20:481
362755999200,19cyclictest0-21swapper/107:24:381
3627557992018,1cyclictest0-21swapper/012:12:280
3627557992018,1cyclictest0-21swapper/012:05:250
3627557992018,1cyclictest0-21swapper/011:37:250
3627557992018,1cyclictest0-21swapper/009:52:340
3627557992013,5cyclictest3725718-21interrupts11:45:170
362755799200,1cyclictest0-21swapper/010:22:090
362755799200,1cyclictest0-21swapper/010:17:360
362755799200,19cyclictest0-21swapper/011:12:050
362755799200,19cyclictest0-21swapper/010:40:340
139099200,3rtkit-daemon1389-21rtkit-daemon07:22:462
3627564991917,1cyclictest3056-21PK-Backend09:56:523
3627564991917,1cyclictest3056-21PK-Backend09:43:003
3627564991917,1cyclictest0-21swapper/311:31:213
3627564991916,2cyclictest3056-21PK-Backend08:17:563
3627564991916,2cyclictest3056-21PK-Backend07:28:273
3627564991913,4cyclictest3695905-21needreboot10:20:173
3627564991911,6cyclictest3697040-21awk10:25:143
362756499190,7cyclictest3725618-21df11:45:143
362756499190,1cyclictest3056-21PK-Backend12:03:253
362756499190,1cyclictest3056-21PK-Backend09:46:243
362756499190,1cyclictest3056-21PK-Backend09:36:113
362756499190,1cyclictest3056-21PK-Backend09:06:573
362756499190,1cyclictest3056-21PK-Backend09:00:143
362756499190,1cyclictest3056-21PK-Backend08:00:383
362756499190,1cyclictest3056-21PK-Backend07:13:213
362756499190,18cyclictest3056-21PK-Backend08:26:483
362756499190,18cyclictest0-21swapper/311:26:023
362756499190,18cyclictest0-21swapper/310:15:103
362756499190,18cyclictest0-21swapper/310:07:333
362756499190,18cyclictest0-21swapper/308:41:493
362756499190,18cyclictest0-21swapper/307:32:473
3627562991917,1cyclictest0-21swapper/211:14:232
3627562991917,1cyclictest0-21swapper/210:15:012
3627562991917,1cyclictest0-21swapper/208:01:462
3627562991917,1cyclictest0-21swapper/207:41:442
3627562991911,6cyclictest3690050-21awk10:05:122
362756299190,1cyclictest0-21swapper/212:02:442
362756299190,1cyclictest0-21swapper/211:21:532
362756299190,18cyclictest0-21swapper/211:29:242
362755999195,12cyclictest3691525-21munin-run10:10:001
3627559991917,1cyclictest3056-21PK-Backend09:03:551
3627559991917,1cyclictest0-21swapper/107:11:481
3627559991916,2cyclictest0-21swapper/108:01:091
362755799193,2cyclictest141rcu_preempt08:09:180
3627557991917,1cyclictest3056-21PK-Backend10:01:140
3627557991917,1cyclictest0-21swapper/012:18:090
3627557991917,1cyclictest0-21swapper/012:03:140
3627557991917,1cyclictest0-21swapper/010:13:500
3627557991915,3cyclictest3635185-21turbostat.cron07:29:590
3627557991912,5cyclictest3682843-21cron09:45:000
3627557991911,6cyclictest883-21dbus-daemon07:15:020
362755799190,18cyclictest0-21swapper/008:50:290
37130512180,4chrt3056-21PK-Backend11:10:163
362756499184,12cyclictest1095-21runrttasks11:39:013
3627564991816,1cyclictest3056-21PK-Backend11:55:353
3627564991815,2cyclictest3056-21PK-Backend12:20:233
3627564991814,3cyclictest3700467-21apt10:35:103
362756499181,15cyclictest3676122-21cnf-update-db09:25:013
3627562991816,1cyclictest0-21swapper/210:04:352
3627562991816,1cyclictest0-21swapper/209:56:292
3627562991816,1cyclictest0-21swapper/208:42:142
3627562991816,1cyclictest0-21swapper/207:35:562
3627562991813,3cyclictest978-21snmpd10:24:022
3627562991813,3cyclictest978-21snmpd09:14:412
3627562991812,4cyclictest886-21NetworkManager11:40:002
3627562991811,5cyclictest34-21ksoftirqd/211:40:122
3627562991810,6cyclictest3672601-21tr09:15:152
362755999188,8cyclictest436-20systemd-journal12:00:001
3627559991815,2cyclictest3056-21PK-Backend11:50:091
3627559991815,2cyclictest3056-21PK-Backend07:41:211
3627559991811,5cyclictest3742753-21grep12:35:131
3627559991810,5cyclictest1095-21runrttasks08:45:111
3627557991816,1cyclictest0-21swapper/012:38:560
3627557991816,1cyclictest0-21swapper/009:08:140
3627557991816,1cyclictest0-21swapper/008:43:130
3627557991816,1cyclictest0-21swapper/008:19:000
3627557991813,3cyclictest3681229-21cat09:40:010
3627557991811,6cyclictest3709709-21sshd11:00:170
3627557991811,5cyclictest3705945-21expr10:50:130
3627557991810,6cyclictest3632688-21ntpq07:20:190
3627564991715,1cyclictest3056-21PK-Backend07:20:173
362756499171,14cyclictest3692148-21df10:10:133
362756299177,8cyclictest3668698-21sh09:05:012
3627562991715,1cyclictest0-21swapper/208:55:332
3627562991715,1cyclictest0-21swapper/208:29:372
3627562991711,5cyclictest2164-21gdbus09:24:012
3627559991715,1cyclictest3056-21PK-Backend08:35:571
362755799179,6cyclictest436-20systemd-journal08:03:190
3627557991715,1cyclictest0-21swapper/011:55:370
3627557991715,1cyclictest0-21swapper/011:32:180
3627557991711,5cyclictest3642381-21sed07:50:010
3627557991711,4cyclictest2156-21gnome-shell07:33:080
3627557991710,6cyclictest3645627-21apt-get08:00:000
3627557991710,5cyclictest883-21dbus-daemon07:40:000
3627557991710,5cyclictest3642539-21cat07:50:110
3627557991710,5cyclictest3629070-21sshd07:13:180
3627564991614,1cyclictest3056-21PK-Backend11:08:303
3627564991612,3cyclictest42-21ksoftirqd/309:16:563
362756299169,5cyclictest3704770-21processes10:45:182
3627562991614,1cyclictest0-21swapper/210:44:422
362755799169,5cyclictest1020-21ntpd08:10:190
362755999159,4cyclictest3637239-21cpuspeed07:35:131
3627557991510,3cyclictest3639087-21cpuspeed_turbos07:40:140
3627564991412,1cyclictest3056-21PK-Backend12:31:243
362756299149,4cyclictest978-21snmpd10:35:482
362756499139,3cyclictest3056-21PK-Backend10:40:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional