You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 05:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Apr 21, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211162722480,5sleep21-21systemd19:10:012
215922521540,3sleep13056-21PK-Backend21:20:001
220625721100,4sleep2341ksoftirqd/223:30:212
211050528743,35sleep10-21swapper/119:05:191
139099830,72rtkit-daemon0-21swapper/019:09:020
22058422640,3sleep10-21swapper/123:30:161
211133826241,14sleep30-21swapper/319:07:273
21379042520,7sleep126-21ksoftirqd/120:20:141
21679562410,3sleep30-21swapper/321:45:003
21449512310,4sleep20-21swapper/220:40:112
2111738992421,2cyclictest13-21ksoftirqd/021:04:570
22279852230,2sleep13056-21PK-Backend00:35:001
2111750992310,9cyclictest978-21snmpd00:15:103
2111745992315,6cyclictest2213396-21users23:50:232
2111738992321,1cyclictest0-21swapper/021:09:230
2111738992320,2cyclictest3056-21PK-Backend20:35:300
2111750992215,5cyclictest2117819-21idleruntime-cro19:24:593
2111750992215,4cyclictest82332-21update-manager23:22:203
2111750992215,4cyclictest82332-21update-manager23:22:193
2111750992215,4cyclictest2173647-21apt22:00:113
2111750992213,7cyclictest2165584-21python321:35:203
211175099220,19cyclictest2180585-21perf22:20:003
2111745992220,1cyclictest0-21swapper/200:06:052
2111745992214,6cyclictest2131494-21awk20:00:192
211174599220,21cyclictest0-21swapper/221:37:222
211174599220,20cyclictest0-21swapper/200:00:452
2111738992220,1cyclictest0-21swapper/023:22:260
2111738992220,1cyclictest0-21swapper/023:22:260
2111738992220,1cyclictest0-21swapper/022:25:210
2111738992220,1cyclictest0-21swapper/020:44:020
2111738992220,1cyclictest0-21swapper/020:26:110
2111738992220,1cyclictest0-21swapper/019:47:100
2111738992219,2cyclictest0-21swapper/022:24:210
211173899220,21cyclictest0-21swapper/023:55:320
139099220,3rtkit-daemon1389-21rtkit-daemon22:50:041
21415382210,4sleep03056-21PK-Backend20:30:160
21415382210,4sleep03056-21PK-Backend20:30:150
2111750992115,4cyclictest2126741-21grep19:50:003
2111750992113,6cyclictest2184048-21apt-get22:30:003
2111750992112,7cyclictest2121293-21apt-get19:35:013
211175099210,19cyclictest3056-21PK-Backend00:00:353
2111745992119,1cyclictest0-21swapper/223:35:032
2111745992119,1cyclictest0-21swapper/223:15:562
2111745992119,1cyclictest0-21swapper/222:37:442
2111745992119,1cyclictest0-21swapper/222:12:362
2111745992119,1cyclictest0-21swapper/222:07:222
2111745992119,1cyclictest0-21swapper/221:56:072
2111745992119,1cyclictest0-21swapper/221:45:252
2111745992119,1cyclictest0-21swapper/221:15:242
2111745992119,1cyclictest0-21swapper/221:11:462
2111745992119,1cyclictest0-21swapper/221:05:322
2111745992119,1cyclictest0-21swapper/220:59:152
2111745992119,1cyclictest0-21swapper/219:55:412
2111745992119,1cyclictest0-21swapper/219:35:472
2111745992113,6cyclictest2152924-21python321:00:202
211174599210,20cyclictest978-21snmpd23:45:442
211174599210,20cyclictest0-21swapper/222:53:122
211174599210,20cyclictest0-21swapper/222:19:532
211174599210,20cyclictest0-21swapper/220:54:012
211174599210,20cyclictest0-21swapper/219:16:352
211174599210,20cyclictest0-21swapper/219:16:352
211174599210,20cyclictest0-21swapper/200:15:322
211174599210,19cyclictest0-21swapper/223:25:102
2111739992119,1cyclictest3056-21PK-Backend21:23:471
2111739992119,1cyclictest0-21swapper/123:27:441
2111739992119,1cyclictest0-21swapper/122:03:351
2111739992119,1cyclictest0-21swapper/121:41:521
2111739992119,1cyclictest0-21swapper/120:01:211
2111739992118,2cyclictest3056-21PK-Backend22:37:561
2111739992117,2cyclictest3056-21PK-Backend00:20:301
211173999210,19cyclictest3056-21PK-Backend22:56:141
211173999210,19cyclictest3056-21PK-Backend20:39:541
211173999210,19cyclictest3056-21PK-Backend19:29:231
211173999210,19cyclictest3056-21PK-Backend00:17:121
2111738992119,1cyclictest0-21swapper/023:42:470
2111738992119,1cyclictest0-21swapper/022:30:010
2111738992119,1cyclictest0-21swapper/022:13:110
2111738992119,1cyclictest0-21swapper/021:27:550
2111738992119,1cyclictest0-21swapper/021:11:120
2111738992119,1cyclictest0-21swapper/020:11:510
2111738992119,1cyclictest0-21swapper/020:02:110
2111738992119,1cyclictest0-21swapper/019:41:020
2111738992119,1cyclictest0-21swapper/019:37:580
2111738992116,3cyclictest2179248-21tr22:15:120
211173899210,20cyclictest0-21swapper/020:45:030
211173899210,20cyclictest0-21swapper/019:24:530
211173899210,20cyclictest0-21swapper/019:15:540
211173899210,20cyclictest0-21swapper/019:15:540
139099210,18rtkit-daemon884-21gmain19:47:331
2111750992018,1cyclictest3056-21PK-Backend21:49:143
2111750992018,1cyclictest0-21swapper/323:50:073
2111750992018,1cyclictest0-21swapper/323:42:443
2111750992018,1cyclictest0-21swapper/322:37:413
2111750992018,1cyclictest0-21swapper/321:18:533
2111750992018,1cyclictest0-21swapper/319:43:423
2111750992018,1cyclictest0-21swapper/300:28:063
2111750992018,1cyclictest0-21swapper/300:24:493
2111750992018,1cyclictest0-21swapper/300:11:033
2111750992015,3cyclictest2141174-21apt-config20:30:003
2111750992015,3cyclictest2130715-21/usr/sbin/munin20:00:153
2111750992013,4cyclictest2141547-21grep20:30:163
2111750992013,4cyclictest2141547-21grep20:30:153
211175099200,19cyclictest3056-21PK-Backend23:29:463
211175099200,19cyclictest3056-21PK-Backend23:07:153
211175099200,19cyclictest3056-21PK-Backend22:09:083
211175099200,19cyclictest0-21swapper/322:57:013
211175099200,19cyclictest0-21swapper/321:55:313
211175099200,19cyclictest0-21swapper/320:36:073
211175099200,19cyclictest0-21swapper/319:37:333
211175099200,19cyclictest0-21swapper/319:20:003
211175099200,19cyclictest0-21swapper/319:20:003
2111745992018,1cyclictest3056-21PK-Backend00:30:322
2111745992018,1cyclictest0-21swapper/223:20:022
2111745992018,1cyclictest0-21swapper/223:20:022
2111745992018,1cyclictest0-21swapper/222:33:042
2111745992018,1cyclictest0-21swapper/221:51:412
2111745992018,1cyclictest0-21swapper/221:26:532
2111745992018,1cyclictest0-21swapper/220:37:402
2111745992018,1cyclictest0-21swapper/220:31:582
2111745992018,1cyclictest0-21swapper/220:31:582
2111745992018,1cyclictest0-21swapper/220:19:012
2111745992018,1cyclictest0-21swapper/220:05:352
2111745992015,3cyclictest2128023-21ntp_states19:50:212
211174599200,19cyclictest3056-21PK-Backend20:45:232
211174599200,19cyclictest0-21swapper/223:41:252
211174599200,19cyclictest0-21swapper/222:25:342
211174599200,19cyclictest0-21swapper/222:20:542
211174599200,19cyclictest0-21swapper/222:02:082
211174599200,19cyclictest0-21swapper/221:20:522
211174599200,19cyclictest0-21swapper/220:20:492
211174599200,19cyclictest0-21swapper/220:12:462
211174599200,19cyclictest0-21swapper/219:21:412
211174599200,19cyclictest0-21swapper/219:12:012
211174599200,19cyclictest0-21swapper/200:26:132
211174599200,19cyclictest0-21swapper/200:20:522
2111739992018,1cyclictest3056-21PK-Backend23:43:461
2111739992018,1cyclictest3056-21PK-Backend22:15:521
2111739992018,1cyclictest3056-21PK-Backend20:58:261
2111739992018,1cyclictest3056-21PK-Backend19:58:031
2111739992018,1cyclictest0-21swapper/123:05:271
2111739992018,1cyclictest0-21swapper/122:49:041
2111739992018,1cyclictest0-21swapper/122:27:491
2111739992018,1cyclictest0-21swapper/121:55:581
2111739992018,1cyclictest0-21swapper/120:46:371
2111739992018,1cyclictest0-21swapper/120:11:001
2111739992017,2cyclictest3056-21PK-Backend21:27:121
2111739992017,2cyclictest3056-21PK-Backend21:04:271
2111739992017,2cyclictest3056-21PK-Backend00:11:041
2111739992015,3cyclictest2168198-21grep21:45:131
2111739992012,6cyclictest2176937-21pool-udisksd22:08:031
211173999200,19cyclictest3056-21PK-Backend23:53:131
211173999200,19cyclictest3056-21PK-Backend21:54:091
211173999200,19cyclictest3056-21PK-Backend21:09:411
211173999200,19cyclictest3056-21PK-Backend20:16:141
211173999200,19cyclictest3056-21PK-Backend20:09:591
211173999200,19cyclictest3056-21PK-Backend19:40:251
211173999200,19cyclictest3056-21PK-Backend19:21:401
211173999200,18cyclictest3056-21PK-Backend23:37:251
211173999200,18cyclictest3056-21PK-Backend23:00:201
211173999200,18cyclictest3056-21PK-Backend20:28:461
2111738992018,1cyclictest0-21swapper/023:06:170
2111738992018,1cyclictest0-21swapper/023:00:090
2111738992018,1cyclictest0-21swapper/022:49:420
2111738992018,1cyclictest0-21swapper/021:59:190
2111738992018,1cyclictest0-21swapper/020:18:540
2111738992018,1cyclictest0-21swapper/019:58:540
2111738992018,1cyclictest0-21swapper/000:35:470
2111738992018,1cyclictest0-21swapper/000:02:550
211173899200,19cyclictest0-21swapper/023:29:500
211173899200,19cyclictest0-21swapper/022:38:190
211173899200,19cyclictest0-21swapper/021:48:510
211173899200,19cyclictest0-21swapper/021:44:250
211173899200,19cyclictest0-21swapper/020:23:200
211173899200,19cyclictest0-21swapper/019:14:390
211175099194,13cyclictest2150450-21tr20:55:133
211175099192,11cyclictest2148593-21apt20:50:093
2111750991917,1cyclictest3056-21PK-Backend20:45:403
2111750991917,1cyclictest0-21swapper/323:37:233
2111750991917,1cyclictest0-21swapper/322:20:033
2111750991917,1cyclictest0-21swapper/321:01:083
2111750991917,1cyclictest0-21swapper/320:24:583
2111750991917,1cyclictest0-21swapper/300:39:493
2111750991917,1cyclictest0-21swapper/300:34:283
2111750991916,2cyclictest0-21swapper/322:54:593
2111750991911,6cyclictest2214605-21cat23:55:153
211175099191,16cyclictest2163735-21ntp_states21:30:193
211175099191,13cyclictest2126755-21apt-get19:50:043
211175099190,1cyclictest3056-21PK-Backend23:33:043
211175099190,1cyclictest3056-21PK-Backend23:04:113
211175099190,1cyclictest3056-21PK-Backend22:45:593
211175099190,18cyclictest0-21swapper/323:10:193
211175099190,18cyclictest0-21swapper/321:06:223
211175099190,18cyclictest0-21swapper/320:07:223
211175099190,18cyclictest0-21swapper/319:57:273
211175099190,18cyclictest0-21swapper/319:28:213
211175099190,18cyclictest0-21swapper/319:13:013
2111745991917,1cyclictest0-21swapper/223:10:492
2111745991917,1cyclictest0-21swapper/221:40:592
2111745991917,1cyclictest0-21swapper/221:32:072
2111745991917,1cyclictest0-21swapper/219:31:422
2111745991916,2cyclictest3056-21PK-Backend22:40:352
2111745991913,4cyclictest28650irq/123-ahci[0000:10:212
2111745991910,6cyclictest436-20systemd-journal23:00:002
211174599190,1cyclictest0-21swapper/222:46:142
211174599190,1cyclictest0-21swapper/200:35:532
211174599190,18cyclictest0-21swapper/219:45:012
2111739991917,1cyclictest3056-21PK-Backend23:18:321
2111739991917,1cyclictest3056-21PK-Backend20:31:351
2111739991917,1cyclictest3056-21PK-Backend20:31:351
2111739991917,1cyclictest3056-21PK-Backend00:29:561
2111739991917,1cyclictest0-21swapper/123:58:331
2111739991917,1cyclictest0-21swapper/100:05:571
2111739991914,3cyclictest2145450-21missed_timers20:40:161
2111739991912,5cyclictest2113226-21seq19:10:231
211173999190,1cyclictest3056-21PK-Backend19:50:401
211173999190,1cyclictest0-21swapper/122:43:231
211173999190,18cyclictest0-21swapper/121:37:261
2111738991917,1cyclictest3056-21PK-Backend20:50:180
2111738991917,1cyclictest0-21swapper/023:48:080
2111738991917,1cyclictest0-21swapper/023:37:270
2111738991917,1cyclictest0-21swapper/023:32:060
2111738991917,1cyclictest0-21swapper/020:59:430
2111738991916,2cyclictest3056-21PK-Backend19:30:000
2111738991913,4cyclictest2156-21gnome-shell22:01:090
2111738991911,6cyclictest2176502-21python322:05:210
211173899190,1cyclictest0-21swapper/023:18:210
211173899190,1cyclictest0-21swapper/021:15:390
211173899190,1cyclictest0-21swapper/020:05:290
2111750991816,1cyclictest3056-21PK-Backend22:42:213
2111750991816,1cyclictest0-21swapper/321:51:033
2111750991816,1cyclictest0-21swapper/321:23:193
2111750991816,1cyclictest0-21swapper/320:17:423
211175099181,5cyclictest2200150-21/usr/sbin/munin23:15:153
2111750991812,4cyclictest2219016-21munin-plugin-st00:10:013
2111750991811,5cyclictest2185503-21cat22:30:213
2111750991811,5cyclictest2178084-21awk22:10:183
211175099180,16cyclictest2156262-21irqstats21:10:163
2111745991816,1cyclictest0-21swapper/219:28:372
2111745991813,3cyclictest2195886-21switchtime23:00:222
2111745991811,5cyclictest2214689-21sed23:55:162
2111739991816,1cyclictest3056-21PK-Backend23:20:341
2111739991816,1cyclictest3056-21PK-Backend23:20:341
2111739991816,1cyclictest3056-21PK-Backend19:39:241
2111739991816,1cyclictest3056-21PK-Backend19:17:471
2111739991816,1cyclictest3056-21PK-Backend19:17:471
2111739991816,1cyclictest0-21swapper/123:10:341
2111739991811,5cyclictest2120337-21wc19:30:161
2111738991816,1cyclictest0-21swapper/023:10:230
2111738991816,1cyclictest0-21swapper/000:13:230
2111738991811,6cyclictest2194712-21perf23:00:000
211173899180,1cyclictest0-21swapper/000:09:040
21200852170,4sleep03056-21PK-Backend19:30:130
2111750991715,1cyclictest3056-21PK-Backend20:42:503
211175099171,14cyclictest978-21snmpd21:25:193
2111745991715,1cyclictest0-21swapper/223:05:422
2111745991715,1cyclictest0-21swapper/220:25:292
2111739991715,1cyclictest3056-21PK-Backend22:30:391
2111739991715,1cyclictest3056-21PK-Backend22:11:271
211173899179,2cyclictest13-21ksoftirqd/023:50:120
2111738991712,4cyclictest2160183-21needreboot21:20:190
2111738991711,5cyclictest2166112-21sshd21:39:150
2111738991710,5cyclictest2223340-21expr00:20:130
211175099160,4cyclictest2210796-21cstates23:45:123
211175099160,3cyclictest2134580-21diskstats20:10:143
211173999168,6cyclictest2163146-21cut21:30:131
211173999167,7cyclictest2156-21gnome-shell21:10:091
2111739991614,1cyclictest3056-21PK-Backend00:39:241
2111739991614,1cyclictest0-21swapper/120:50:011
2111739991610,4cyclictest2216451-21needreboot00:00:191
2111739991610,4cyclictest2212172-21grep23:50:011
2111738991614,1cyclictest0-21swapper/019:54:340
211174599158,5cyclictest2126595-21sshd19:48:152
211173999159,4cyclictest2181493-21tr22:20:181
211173899158,5cyclictest2188827-21proc_pri22:40:200
211173899156,8cyclictest2170657-21grep21:50:180
2111738991513,1cyclictest0-21swapper/021:33:090
2111738991513,1cyclictest0-21swapper/000:29:250
2111738991513,1cyclictest0-21swapper/000:19:460
2111738991511,3cyclictest1095-21runrttasks00:30:280
211173899138,3cyclictest2192973-21grep22:55:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional