You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Apr 17, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12370422480,6sleep0123687-21munin-run07:10:010
22559521300,4sleep13056-21PK-Backend11:55:201
12343227539,9sleep30-21swapper/307:07:113
12301126843,17sleep20-21swapper/207:05:252
12341526242,14sleep10-21swapper/107:06:571
1777762560,3sleep13056-21PK-Backend09:40:141
1493782480,1sleep30-21swapper/308:20:133
1986942440,3sleep00-21swapper/010:40:130
2099292310,9sleep013-21ksoftirqd/011:10:200
1728152300,3sleep0172821-21needreboot09:25:170
12379299299,4cyclictest183428-21ls09:55:202
123792992911,16cyclictest134944-21cpu07:40:122
1516822280,8sleep126-21ksoftirqd/108:25:201
12379299287,4cyclictest193189-21find10:25:102
12379299283,21cyclictest154029-21apt-get08:35:032
12379299282,5cyclictest0-21swapper/208:25:112
12379299282,4cyclictest0-21swapper/208:55:132
123792992811,15cyclictest177325-21/usr/sbin/munin09:40:212
123792992811,15cyclictest144069-21cat08:05:142
12379299278,5cyclictest236932-21sed12:30:002
12379299277,6cyclictest189482-21basename10:14:592
12379299277,4cyclictest221606-21smartctl11:45:152
12379299272,6cyclictest0-21swapper/209:10:112
12379299272,5cyclictest138418-21apt07:50:122
12379299272,5cyclictest0-21swapper/211:50:112
12379299272,4cyclictest0-21swapper/210:20:112
12379299271,9cyclictest199894-21perf-exec10:45:002
12379799263,14cyclictest0-21swapper/309:39:283
12379799263,12cyclictest0-21swapper/310:33:403
12379299269,5cyclictest174607-21tr09:30:182
12379299269,14cyclictest202052-21cut10:50:132
12379299262,9cyclictest168682-21apt09:15:122
12379299262,8cyclictest226402-21apt12:00:092
12379299262,5cyclictest0-21swapper/211:00:102
12379299262,5cyclictest0-21swapper/207:55:132
12379299261,9cyclictest133124-21/usr/sbin/munin07:35:142
12379299261,5cyclictest0-21swapper/207:30:092
12379299259,5cyclictest210067-21cat11:10:212
12379299259,14cyclictest2156-21gnome-shell08:34:082
12379299258,5cyclictest124107-21ls07:10:112
12379299257,4cyclictest187938-21latency_hist10:10:002
12379299255,5cyclictest128058-21cat07:20:122
12379299255,5cyclictest128058-21cat07:20:122
12379299252,5cyclictest0-21swapper/211:40:112
12379299252,5cyclictest0-21swapper/207:25:132
12379299251,9cyclictest232703-21awk12:15:192
12379299251,9cyclictest217452-21cron11:35:002
12379299251,9cyclictest200782-21grep10:45:162
12379299251,7cyclictest436-20systemd-journal08:50:012
12379299251,6cyclictest166695-21idleruntime-cro09:09:592
12379299251,4cyclictest0-21swapper/209:35:102
12379299251,10cyclictest218806-21taskset11:35:212
123792992510,2cyclictest127682-21latency_hist07:20:012
139099240,3rtkit-daemon1389-21rtkit-daemon10:59:012
12379799249,13cyclictest131179-21unin-run07:29:593
12379799245,17cyclictest186938-21uname10:05:173
12379799245,17cyclictest166268-21sed09:05:223
12379799244,18cyclictest144372-21/usr/sbin/munin08:05:183
12379799241,19cyclictest172151-21apt09:25:123
12379299247,5cyclictest228907-21proc_pri12:05:192
12379299247,5cyclictest228907-21proc_pri12:05:192
12379299247,3cyclictest0-21swapper/209:50:222
12379299247,2cyclictest173690-21latency_hist09:30:012
12379299242,4cyclictest0-21swapper/207:45:102
12379299241,8cyclictest214445-21cpuspeed_turbos11:25:132
12379299241,8cyclictest149233-21grep08:20:132
12379299241,8cyclictest1-21systemd09:25:002
12379299241,7cyclictest233724-21grep12:20:112
12379299241,7cyclictest2156-21gnome-shell08:50:092
12379299241,7cyclictest142555-21tail08:00:162
12379299240,9cyclictest189861-21tr10:15:122
12379299240,8cyclictest163461-21/usr/sbin/munin09:00:112
12379799237,14cyclictest190280-21latency10:15:173
12379799236,15cyclictest165099-21date09:05:003
123797992315,6cyclictest171120-21ntp_kernel_err09:20:183
12379799230,4cyclictest155073-21latency08:35:173
12379299238,3cyclictest197457-21proc_pri10:35:162
12379299238,3cyclictest0-21swapper/212:35:162
12379299234,5cyclictest213521-21timerandwakeup11:20:222
12379299234,16cyclictest195607-21needreboot10:30:172
12379299232,6cyclictest225531-21python311:55:192
12379299231,9cyclictest210798-21apt11:15:112
12379299231,6cyclictest237654-21/usr/sbin/munin12:30:182
12379299231,6cyclictest206802-21apt-get11:05:022
12379299231,5cyclictest231322-21cron12:15:002
12379299231,5cyclictest147065-21/usr/sbin/munin08:15:162
12379299231,4cyclictest185403-21python310:00:192
139099220,3rtkit-daemon1389-21rtkit-daemon08:56:033
139099220,15rtkit-daemon3056-21PK-Backend08:17:231
139099220,13rtkit-daemon0-21swapper/007:41:540
12379799226,14cyclictest202618-21ntpq10:50:203
12379799223,8cyclictest2387912acpi12:35:113
12379299221,6cyclictest179567-21kernelversion09:45:172
12379299220,8cyclictest145567-21/usr/sbin/munin08:10:132
123790992220,1cyclictest3056-21PK-Backend09:10:201
123786992220,1cyclictest3056-21PK-Backend10:46:250
123786992220,1cyclictest3056-21PK-Backend10:30:350
123786992214,6cyclictest436-20systemd-journal11:10:000
12378699220,3cyclictest1980-21gvfs-udisks2-vo12:08:040
12378699220,3cyclictest1980-21gvfs-udisks2-vo12:08:030
139099210,14rtkit-daemon3056-21PK-Backend08:22:241
139099210,14rtkit-daemon0-21swapper/007:57:430
139099210,14rtkit-daemon0-21swapper/007:22:220
139099210,14rtkit-daemon0-21swapper/007:22:210
139099210,13rtkit-daemon0-21swapper/108:05:491
12379799216,2cyclictest131995-21memory07:30:163
12379799216,13cyclictest224010-21users11:50:213
12379799211,8cyclictest157154-21python308:40:203
123797992118,2cyclictest42-21ksoftirqd/308:30:513
123797992113,6cyclictest205326-21awk11:00:093
123797992113,6cyclictest203883-21expr10:55:113
123797992113,6cyclictest141055-21sed07:55:223
12379799210,20cyclictest1389-21rtkit-daemon12:11:293
12379299211,6cyclictest156658-21taskset08:40:132
123790992118,2cyclictest3056-21PK-Backend10:55:171
123790992113,6cyclictest220569-21fwupdmgr11:40:341
12379099210,20cyclictest0-21swapper/107:50:091
12379099210,19cyclictest3056-21PK-Backend12:11:351
123786992119,1cyclictest3056-21PK-Backend09:31:010
123786992119,1cyclictest3056-21PK-Backend08:05:230
123786992119,1cyclictest0-21swapper/012:37:230
123786992119,1cyclictest0-21swapper/012:16:210
123786992119,1cyclictest0-21swapper/012:10:530
123786992119,1cyclictest0-21swapper/011:57:090
123786992119,1cyclictest0-21swapper/011:48:500
123786992119,1cyclictest0-21swapper/011:35:050
123786992119,1cyclictest0-21swapper/011:22:540
123786992119,1cyclictest0-21swapper/010:54:420
123786992119,1cyclictest0-21swapper/010:17:390
123786992119,1cyclictest0-21swapper/009:19:460
123786992119,1cyclictest0-21swapper/008:53:220
123786992119,1cyclictest0-21swapper/008:44:170
123786992119,1cyclictest0-21swapper/008:17:270
123786992117,2cyclictest197406-21ntp_states10:35:160
12378699210,20cyclictest0-21swapper/007:32:320
12378699210,20cyclictest0-21swapper/007:16:020
12379799207,9cyclictest0-21swapper/307:35:113
12379799205,13cyclictest177338-21cat09:40:103
12379799205,13cyclictest142752-21python308:00:203
123797992018,1cyclictest0-21swapper/310:12:583
123797992018,1cyclictest0-21swapper/308:26:313
123797992018,1cyclictest0-21swapper/308:18:073
123797992010,7cyclictest212536-21taskset11:20:113
12379799200,19cyclictest42-21ksoftirqd/311:19:433
12379799200,19cyclictest0-21swapper/312:27:233
12379799200,19cyclictest0-21swapper/312:24:313
12379799200,19cyclictest0-21swapper/312:15:263
12379799200,19cyclictest0-21swapper/311:35:453
12379799200,19cyclictest0-21swapper/311:31:063
12379799200,19cyclictest0-21swapper/310:40:573
12379799200,19cyclictest0-21swapper/310:39:553
12379799200,19cyclictest0-21swapper/308:47:483
12379799200,18cyclictest191471-21apt10:20:113
12379799200,18cyclictest0-21swapper/307:51:433
12379099202,16cyclictest181160-21ls09:50:121
123790992018,1cyclictest3056-21PK-Backend12:36:351
123790992018,1cyclictest3056-21PK-Backend09:08:101
123790992017,2cyclictest3056-21PK-Backend11:45:131
12379099200,19cyclictest3056-21PK-Backend07:16:081
123786992018,1cyclictest3056-21PK-Backend08:03:210
123786992018,1cyclictest0-21swapper/012:23:380
123786992018,1cyclictest0-21swapper/011:15:240
123786992018,1cyclictest0-21swapper/010:55:430
123786992018,1cyclictest0-21swapper/010:12:110
123786992018,1cyclictest0-21swapper/009:45:070
123786992018,1cyclictest0-21swapper/008:47:550
123786992018,1cyclictest0-21swapper/008:29:560
123786992018,1cyclictest0-21swapper/007:25:480
123786992013,5cyclictest222930-21awk11:50:100
12378699200,19cyclictest3056-21PK-Backend09:36:150
12378699200,19cyclictest0-21swapper/012:27:170
12378699200,19cyclictest0-21swapper/011:32:140
12378699200,19cyclictest0-21swapper/011:25:450
12378699200,19cyclictest0-21swapper/011:00:240
12378699200,19cyclictest0-21swapper/010:29:340
12378699200,19cyclictest0-21swapper/009:55:480
12378699200,19cyclictest0-21swapper/009:52:090
12378699200,19cyclictest0-21swapper/009:23:570
12378699200,19cyclictest0-21swapper/008:32:000
12378699200,19cyclictest0-21swapper/007:46:240
2033692190,4sleep13056-21PK-Backend10:55:001
12379799198,3cyclictest2146562fw_packets11:25:153
12379799197,10cyclictest186011-21basename10:04:593
12379799194,13cyclictest160018-21apt08:50:093
12379799193,14cyclictest201597-21sh10:50:013
123797991917,1cyclictest0-21swapper/309:55:333
123797991917,1cyclictest0-21swapper/309:45:413
123797991917,1cyclictest0-21swapper/307:15:483
123797991914,3cyclictest167528-21memory09:10:173
123797991913,5cyclictest168815-21expr09:15:133
123797991913,4cyclictest229768-21date12:10:013
123797991913,4cyclictest229768-21date12:10:013
123797991912,5cyclictest221478-21cat11:45:113
123797991912,5cyclictest123985-21awk07:10:113
123797991910,6cyclictest136670-21apt07:45:103
12379799190,1cyclictest3056-21PK-Backend09:50:073
12379799190,1cyclictest0-21swapper/311:05:303
12379799190,1cyclictest0-21swapper/310:25:553
12379799190,1cyclictest0-21swapper/307:43:333
123790991917,1cyclictest3056-21PK-Backend10:07:251
123790991917,1cyclictest3056-21PK-Backend09:29:121
12379099190,1cyclictest3056-21PK-Backend12:35:001
12379099190,1cyclictest3056-21PK-Backend11:37:141
12379099190,1cyclictest3056-21PK-Backend10:25:161
12379099190,1cyclictest3056-21PK-Backend08:04:221
12379099190,1cyclictest0-21swapper/107:20:011
12379099190,1cyclictest0-21swapper/107:20:001
123786991917,1cyclictest3056-21PK-Backend07:10:210
123786991917,1cyclictest0-21swapper/012:30:070
123786991917,1cyclictest0-21swapper/012:01:480
123786991917,1cyclictest0-21swapper/010:01:160
123786991917,1cyclictest0-21swapper/009:40:410
123786991917,1cyclictest0-21swapper/007:39:280
12378699190,1cyclictest0-21swapper/008:12:200
12379799186,10cyclictest0-21swapper/312:00:103
12379799185,11cyclictest237050-21/usr/sbin/munin12:30:183
123797991812,5cyclictest128165-21seq07:20:123
123797991812,5cyclictest128165-21seq07:20:123
123797991812,4cyclictest436-20systemd-journal08:10:243
12379099188,2cyclictest141rcu_preempt11:20:001
123790991816,1cyclictest0-21swapper/108:10:231
123790991815,2cyclictest3056-21PK-Backend09:39:401
123790991813,3cyclictest196391-21cron10:35:001
123790991811,6cyclictest185996-21cron10:05:001
12379099180,1cyclictest0-21swapper/109:18:091
123786991816,1cyclictest0-21swapper/010:20:280
123786991816,1cyclictest0-21swapper/010:06:440
123786991816,1cyclictest0-21swapper/008:56:120
123786991810,6cyclictest167693-21ntp_states09:10:190
12379799176,8cyclictest0-21swapper/311:10:113
123797991712,4cyclictest225441-21pluginstate11:55:173
123797991712,3cyclictest219828-21df11:40:133
12379099178,7cyclictest197036-21df_inode10:35:111
123790991715,1cyclictest3056-21PK-Backend12:05:261
123790991715,1cyclictest3056-21PK-Backend12:05:261
123790991715,1cyclictest3056-21PK-Backend09:32:571
123790991715,1cyclictest3056-21PK-Backend08:56:271
123790991715,1cyclictest3056-21PK-Backend08:47:551
123790991715,1cyclictest3056-21PK-Backend07:45:081
123790991715,1cyclictest0-21swapper/109:45:471
123790991710,5cyclictest199871-21grep10:45:001
123790991710,5cyclictest156049-21NetworkManager08:40:001
12379099170,4cyclictest3056-21PK-Backend08:30:091
12378699172,13cyclictest163542-21sort09:00:120
123786991715,1cyclictest3056-21PK-Backend07:50:290
123786991710,5cyclictest156220-21wc08:40:010
12379799160,4cyclictest174469-21missed_timers09:30:163
12379099169,5cyclictest156273-21cat08:40:101
12379099168,6cyclictest206808-21cron11:05:001
12379099167,8cyclictest3056-21PK-Backend07:44:591
123790991614,1cyclictest3056-21PK-Backend12:25:411
123790991614,1cyclictest3056-21PK-Backend07:35:571
123790991613,2cyclictest3056-21PK-Backend11:30:191
123790991613,2cyclictest3056-21PK-Backend09:57:181
123790991613,2cyclictest3056-21PK-Backend09:23:171
123790991611,3cyclictest200798-21grep10:45:151
123790991610,4cyclictest140953-21cut07:55:211
12378699162,12cyclictest165396-21cut09:05:130
123786991612,3cyclictest220723-21taskset11:42:270
12379099154,3cyclictest141rcu_preempt07:25:241
123790991513,1cyclictest3056-21PK-Backend10:12:441
123790991513,1cyclictest3056-21PK-Backend08:54:041
123790991513,1cyclictest0-21swapper/112:15:341
123790991511,3cyclictest124147-21tr07:10:121
123790991510,4cyclictest3056-21PK-Backend10:22:281
12379099150,14cyclictest3056-21PK-Backend12:03:041
12379099150,14cyclictest3056-21PK-Backend11:52:091
12379099142,3cyclictest3056-21PK-Backend11:14:591
123790991412,1cyclictest3056-21PK-Backend09:01:141
12379099140,5cyclictest3056-21PK-Backend07:31:591
12379099140,5cyclictest26-21ksoftirqd/111:25:181
12379099140,1cyclictest3056-21PK-Backend10:18:531
12378699141,10cyclictest148549-21apt-get08:20:030
12379099131,4cyclictest26-21ksoftirqd/112:20:181
12379099120,5cyclictest3056-21PK-Backend11:20:191
12379099113,2cyclictest141rcu_preempt11:05:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional