You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 10:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon Dec 15, 2025 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13605802980,8sleep14497-21PK-Backend22:15:011
197399820,9rtkit-daemon1972-21rtkit-daemon22:31:171
197399820,9rtkit-daemon1972-21rtkit-daemon22:31:171
13349202800,6sleep10-21swapper/121:00:211
1294637997969,7cyclictest13-21ksoftirqd/020:25:580
129464899763,68cyclictest2069-21gnome-shell20:59:583
129426627639,13sleep30-21swapper/319:07:003
13662032750,4sleep20-21swapper/222:30:152
13662032750,4sleep20-21swapper/222:30:152
197399740,6rtkit-daemon4497-21PK-Backend20:26:133
197399740,6rtkit-daemon4497-21PK-Backend19:33:103
129464899732,66cyclictest2069-21gnome-shell21:53:583
197399720,4rtkit-daemon0-21swapper/123:42:061
13281332720,4sleep04497-21PK-Backend20:40:240
129464899722,29cyclictest2069-21gnome-shell20:17:593
197399710,5rtkit-daemon4497-21PK-Backend20:42:203
129464899712,28cyclictest2069-21gnome-shell21:48:583
197399690,56rtkit-daemon0-21swapper/219:09:202
129464899692,63cyclictest2069-21gnome-shell22:03:593
13749142680,7chrt4497-21PK-Backend22:55:182
13749142680,7chrt4497-21PK-Backend22:55:182
129464899682,27cyclictest0-21swapper/322:41:583
1294648996819,28cyclictest1137-21runrttasks23:04:343
197399640,9rtkit-daemon1972-21rtkit-daemon23:15:590
129464899642,58cyclictest0-21swapper/320:24:593
129464899641,40cyclictest1372729-21/usr/sbin/munin22:50:213
129427826444,14sleep00-21swapper/019:07:090
197399630,9rtkit-daemon1972-21rtkit-daemon22:16:300
129464899632,21cyclictest0-21swapper/323:16:583
129464899631,58cyclictest0-21swapper/321:04:583
129302326342,14sleep10-21swapper/119:05:041
129464899622,57cyclictest0-21swapper/300:24:593
129464899622,56cyclictest0-21swapper/322:49:583
129464899622,56cyclictest0-21swapper/319:35:593
129463799612,56cyclictest161rcuc/019:25:590
197399600,9rtkit-daemon1972-21rtkit-daemon21:22:370
197399600,9rtkit-daemon1972-21rtkit-daemon21:03:040
13129402600,14sleep3411rcuc/320:00:103
129463799602,54cyclictest2069-21gnome-shell23:55:590
197399590,9rtkit-daemon1972-21rtkit-daemon20:48:450
197399590,10rtkit-daemon1972-21rtkit-daemon22:58:110
197399590,10rtkit-daemon1972-21rtkit-daemon22:58:110
1294648995934,21cyclictest4497-21PK-Backend21:08:593
129464099591,54cyclictest2069-21gnome-shell00:39:591
197399580,9rtkit-daemon1972-21rtkit-daemon22:30:350
197399580,9rtkit-daemon1972-21rtkit-daemon22:30:350
129464599582,52cyclictest2069-21gnome-shell22:40:592
129463799582,52cyclictest2069-21gnome-shell21:07:590
197399570,9rtkit-daemon1972-21rtkit-daemon23:38:330
197399570,9rtkit-daemon1972-21rtkit-daemon21:47:560
197399570,8rtkit-daemon1972-21rtkit-daemon00:17:101
1294648995731,21cyclictest2069-21gnome-shell22:33:593
1294648995731,21cyclictest2069-21gnome-shell22:33:593
129464899571,5cyclictest4497-21PK-Backend20:33:583
129464099572,51cyclictest2069-21gnome-shell23:06:581
197399560,9rtkit-daemon1972-21rtkit-daemon00:06:160
1294648995620,30cyclictest2069-21gnome-shell21:40:593
1294648995620,30cyclictest2069-21gnome-shell21:40:593
129463799562,50cyclictest1372308-21taskset22:47:580
197399550,8rtkit-daemon1972-21rtkit-daemon23:53:150
129464099552,11cyclictest2069-21gnome-shell00:31:581
129464099552,11cyclictest2069-21gnome-shell00:31:581
1294637995519,29cyclictest13-21ksoftirqd/000:18:580
1294648995445,6cyclictest42-21ksoftirqd/322:39:593
129464099547,5cyclictest0-21swapper/120:32:591
129464099542,48cyclictest2069-21gnome-shell19:14:581
129464099542,42cyclictest2069-21gnome-shell22:54:591
129464099542,10cyclictest2069-21gnome-shell23:57:581
129463799542,48cyclictest2069-21gnome-shell23:22:590
129463799542,47cyclictest1312308-40lsb_release19:55:590
129463799541,10cyclictest2069-21gnome-shell19:38:580
13810782530,5sleep00-21swapper/023:14:320
13328782530,7sleep24497-21PK-Backend20:55:152
129464599533,41cyclictest2069-21gnome-shell21:51:592
129464099532,10cyclictest2069-21gnome-shell20:28:591
129463799532,47cyclictest2069-21gnome-shell20:30:580
129463799532,47cyclictest2069-21gnome-shell00:00:580
129463799531,10cyclictest2069-21gnome-shell19:50:590
1294648995227,20cyclictest4497-21PK-Backend23:05:583
129464099529,38cyclictest2069-21gnome-shell20:44:581
129464099525,6cyclictest26-21ksoftirqd/122:59:581
129464099525,6cyclictest26-21ksoftirqd/122:59:581
129464099522,46cyclictest2069-21gnome-shell20:47:581
1294648995131,17cyclictest0-21swapper/300:32:593
1294648995131,17cyclictest0-21swapper/300:32:583
1294648995125,22cyclictest1355134-21cron22:00:003
129464899512,45cyclictest2069-21gnome-shell22:06:593
129464899512,45cyclictest2069-21gnome-shell22:06:593
129464599518,41cyclictest34-21ksoftirqd/222:28:582
1294645995127,8cyclictest34-21ksoftirqd/221:20:152
129464599512,45cyclictest2142-21evolution-calen19:54:442
129464099512,45cyclictest2069-21gnome-shell19:27:591
129463799512,8cyclictest2069-21gnome-shell00:14:580
129463799512,46cyclictest2069-21gnome-shell23:47:580
129463799512,45cyclictest2069-21gnome-shell23:26:590
129464899502,43cyclictest1294972-21tr19:10:103
1294648995018,27cyclictest2069-21gnome-shell21:39:593
129464599503,44cyclictest331rcuc/222:17:592
1294645995021,11cyclictest34-21ksoftirqd/220:40:002
1294640995029,16cyclictest2069-21gnome-shell21:21:581
129464099501,5cyclictest4497-21PK-Backend23:10:591
1294648994937,8cyclictest2069-21gnome-shell22:22:583
1294648994932,14cyclictest4497-21PK-Backend23:39:583
129464899492,43cyclictest0-21swapper/322:26:583
129464899492,43cyclictest0-21swapper/321:32:583
129464899492,43cyclictest0-21swapper/321:12:593
1294648994915,29cyclictest1339782-21expr21:15:153
129464599492,44cyclictest0-21swapper/223:10:582
129464099491,8cyclictest2069-21gnome-shell00:23:581
1294637994939,6cyclictest2069-21gnome-shell00:28:590
129464899486,20cyclictest42-21ksoftirqd/319:20:343
1294648994831,13cyclictest4497-21PK-Backend19:41:583
1294648994831,13cyclictest4497-21PK-Backend19:41:583
129464899482,6cyclictest0-21swapper/320:10:583
129464899482,42cyclictest0-21swapper/323:10:583
129464899482,42cyclictest0-21swapper/320:46:593
1294648994818,26cyclictest1391532-21basename23:45:003
129464899481,43cyclictest3039-21gdbus00:17:073
1294648994814,29cyclictest1392836-21cat23:45:313
129464599482,6cyclictest0-21swapper/222:22:582
129464599482,6cyclictest0-21swapper/220:54:592
129464099481,5cyclictest4497-21PK-Backend22:20:591
129464099481,43cyclictest4497-21PK-Backend23:30:581
129464099481,43cyclictest4497-21PK-Backend21:32:591
1294637994837,7cyclictest2069-21gnome-shell20:04:580
129463799482,5cyclictest0-21swapper/022:44:580
129463799482,44cyclictest0-21swapper/021:39:590
129463799482,43cyclictest0-21swapper/021:30:580
1294637994812,31cyclictest13-21ksoftirqd/019:32:590
129463799480,44cyclictest1972-21rtkit-daemon23:05:170
1294648994739,5cyclictest0-21swapper/321:20:593
1294648994738,6cyclictest0-21swapper/300:28:593
1294648994738,5cyclictest4497-21PK-Backend22:10:583
129464899472,6cyclictest0-21swapper/323:32:593
129464899472,6cyclictest0-21swapper/319:52:593
129464899472,6cyclictest0-21swapper/300:11:583
129464899472,42cyclictest4497-21PK-Backend20:37:583
129464899472,42cyclictest0-21swapper/319:19:583
129464899472,42cyclictest0-21swapper/319:19:583
129464899472,41cyclictest4497-21PK-Backend19:46:593
1294645994738,7cyclictest331rcuc/222:01:582
129464599472,6cyclictest0-21swapper/223:52:582
129464599472,6cyclictest0-21swapper/221:33:582
129464099471,5cyclictest4497-21PK-Backend23:46:591
129464099471,42cyclictest4497-21PK-Backend20:07:591
1294648994638,5cyclictest0-21swapper/322:57:593
1294648994638,5cyclictest0-21swapper/322:57:593
1294648994638,5cyclictest0-21swapper/320:07:583
1294648994638,5cyclictest0-21swapper/300:05:583
129464899462,41cyclictest0-21swapper/323:58:593
129464899462,41cyclictest0-21swapper/322:18:583
129464899462,41cyclictest0-21swapper/319:58:593
1294648994620,21cyclictest1393854-21sort23:50:133
1294645994634,8cyclictest2069-21gnome-shell22:38:592
129464599462,6cyclictest0-21swapper/221:40:592
129464599462,6cyclictest0-21swapper/221:40:582
129464599462,41cyclictest0-21swapper/219:43:592
129464599462,41cyclictest0-21swapper/219:43:592
129464599462,41cyclictest0-21swapper/200:28:582
1294640994638,5cyclictest231irq_work/119:54:441
1294640994628,14cyclictest2069-21gnome-shell20:37:591
129464099462,42cyclictest0-21swapper/121:09:591
129464099462,42cyclictest0-21swapper/119:59:581
129464099461,42cyclictest4497-21PK-Backend23:29:581
1294637994640,4cyclictest0-21swapper/022:26:590
1294637994640,4cyclictest0-21swapper/022:24:580
1294637994639,5cyclictest0-21swapper/022:35:580
1294637994630,12cyclictest2069-21gnome-shell19:13:580
129463799462,42cyclictest0-21swapper/022:09:590
129463799462,42cyclictest0-21swapper/022:09:580
129463799462,42cyclictest0-21swapper/021:28:580
129463799462,42cyclictest0-21swapper/020:56:580
14068882450,16sleep126-21ksoftirqd/100:25:201
1294648994538,4cyclictest0-21swapper/300:39:583
129464899451,4cyclictest0-21swapper/319:25:593
1294648994514,27cyclictest1398389-21seq00:00:373
1294645994538,5cyclictest0-21swapper/223:38:592
1294645994537,5cyclictest0-21swapper/221:47:592
129464599452,41cyclictest0-21swapper/220:20:592
129464599452,36cyclictest0-21swapper/220:00:592
129464599451,41cyclictest4497-21PK-Backend20:46:582
129464599451,41cyclictest4497-21PK-Backend20:43:582
129464599451,41cyclictest4497-21PK-Backend19:47:582
129464099457,34cyclictest26-21ksoftirqd/120:23:581
129463799452,4cyclictest141rcu_preempt23:43:580
129463799452,4cyclictest141rcu_preempt19:24:590
129463799452,4cyclictest141rcu_preempt00:32:580
129463799452,4cyclictest141rcu_preempt00:32:580
129463799451,5cyclictest141rcu_preempt21:12:590
129463799451,4cyclictest141rcu_preempt21:16:590
1294648994436,5cyclictest0-21swapper/321:29:583
1294645994436,5cyclictest0-21swapper/223:28:582
1294645994436,5cyclictest0-21swapper/220:16:582
1294645994436,5cyclictest0-21swapper/200:32:592
1294645994436,5cyclictest0-21swapper/200:32:592
129464599441,22cyclictest1317081-21taskset20:10:232
1294640994437,4cyclictest0-21swapper/123:38:581
1294637994438,4cyclictest0-21swapper/021:40:590
1294637994438,4cyclictest0-21swapper/021:40:580
1294648994310,28cyclictest1383288-21expr23:20:133
1294645994335,5cyclictest0-21swapper/200:19:582
1294645994330,9cyclictest1354778-40lsb_release21:55:582
14102112420,5sleep2331rcuc/200:35:232
129464599426,20cyclictest34-21ksoftirqd/200:10:132
1294645994235,4cyclictest4497-21PK-Backend21:03:582
1294645994235,4cyclictest4497-21PK-Backend19:12:592
129463799422,5cyclictest4497-21PK-Backend21:58:590
129463799421,37cyclictest0-21swapper/023:04:580
1294645994135,4cyclictest0-21swapper/200:22:582
1294645994134,5cyclictest0-21swapper/219:37:592
1294645994133,5cyclictest0-21swapper/222:05:592
1294645994133,5cyclictest0-21swapper/222:05:582
1294645994123,8cyclictest34-21ksoftirqd/222:47:582
129464599412,18cyclictest34-21ksoftirqd/221:15:212
1294645994113,5cyclictest141rcu_preempt19:56:082
129464099417,8cyclictest141rcu_preempt19:24:591
129464099413,16cyclictest141rcu_preempt21:16:581
1294640994126,11cyclictest2069-21gnome-shell19:47:591
1294648994014,22cyclictest1384974-21cpuspeed_turbos23:25:153
1294648994014,22cyclictest1331085-21tr20:50:153
129464599409,16cyclictest34-21ksoftirqd/219:30:262
129464599408,18cyclictest34-21ksoftirqd/222:50:502
129464599407,17cyclictest34-21ksoftirqd/223:35:022
129464599401,16cyclictest1393347-21apt-config23:50:012
129464099401,34cyclictest2069-21gnome-shell23:52:591
1294637994028,8cyclictest2069-21gnome-shell23:31:580
129463799401,34cyclictest2069-21gnome-shell22:50:590
197399390,31rtkit-daemon4497-21PK-Backend22:16:461
1294645993924,10cyclictest2069-21gnome-shell23:08:592
1294645993923,12cyclictest923-21dbus-daemon22:15:012
1294645993913,9cyclictest34-21ksoftirqd/220:09:552
1294640993924,10cyclictest2069-21gnome-shell23:16:591
1294640993916,20cyclictest1338587-21/usr/sbin/munin21:10:241
1294637993922,13cyclictest13-21ksoftirqd/022:00:210
1294637993921,6cyclictest13-21ksoftirqd/022:10:280
1294637993914,6cyclictest141rcu_preempt00:40:000
197399380,33rtkit-daemon1972-21rtkit-daemon21:56:501
129464599385,17cyclictest1398194-21switchtime00:00:292
1294645993831,4cyclictest4497-21PK-Backend21:11:582
1294645993831,4cyclictest4497-21PK-Backend20:26:592
1294645993826,8cyclictest2069-21gnome-shell21:35:582
1294645993822,12cyclictest1376928-21ntpq23:00:262
1294645993815,8cyclictest34-21ksoftirqd/221:25:142
129464599381,20cyclictest34-21ksoftirqd/219:30:012
1294640993820,13cyclictest438-20systemd-journal20:56:331
1294637993823,11cyclictest1321156-21sshd20:22:330
1294637993821,12cyclictest1332179-21sshd20:52:570
1294637993817,17cyclictest141rcu_preempt19:40:220
1294637993817,17cyclictest141rcu_preempt19:40:220
129463799381,32cyclictest2069-21gnome-shell20:14:590
1294645993728,5cyclictest13911812sleep223:40:592
129464599372,14cyclictest0-21swapper/223:23:592
129464599371,19cyclictest34-21ksoftirqd/221:09:402
129464599371,10cyclictest4497-21PK-Backend19:15:222
129464599371,10cyclictest4497-21PK-Backend19:15:222
1294640993729,5cyclictest4497-21PK-Backend20:10:591
1294640993718,14cyclictest2069-21gnome-shell23:01:581
129463799373,30cyclictest0-21swapper/020:39:590
1294637993720,13cyclictest13-21ksoftirqd/021:55:010
197399360,28rtkit-daemon1941-21systemd22:47:341
1294645993615,6cyclictest141rcu_preempt23:55:142
1294640993625,7cyclictest231irq_work/121:51:591
1294640993620,13cyclictest1304243-21cat19:35:141
129464099361,9cyclictest0-21swapper/119:31:131
129464099361,14cyclictest4497-21PK-Backend23:20:071
1294637993619,13cyclictest13-21ksoftirqd/019:45:300
129464599354,18cyclictest34-21ksoftirqd/220:30:282
129464599351,7cyclictest1054-21snmpd00:10:012
129464599351,14cyclictest4497-21PK-Backend23:15:132
1294640993525,6cyclictest4497-21PK-Backend22:36:581
129464099351,8cyclictest2069-21gnome-shell19:43:591
129464099351,8cyclictest2069-21gnome-shell19:43:591
1294640993513,17cyclictest2069-21gnome-shell19:19:591
1294640993513,17cyclictest2069-21gnome-shell19:19:591
129464099351,14cyclictest4497-21PK-Backend22:00:311
1294637993524,6cyclictest13-21ksoftirqd/019:15:020
1294637993524,6cyclictest13-21ksoftirqd/019:15:020
1294637993519,12cyclictest1315337-21head20:05:220
1294640993425,5cyclictest4497-21PK-Backend21:45:581
1294640993424,6cyclictest4497-21PK-Backend21:35:591
1294640993419,11cyclictest1331018-21grep20:50:131
129464099341,29cyclictest4497-21PK-Backend22:41:581
129464099341,29cyclictest4497-21PK-Backend22:28:591
129464099341,29cyclictest4497-21PK-Backend20:16:581
129464099341,10cyclictest4497-21PK-Backend20:04:461
1294637993425,5cyclictest13-21ksoftirqd/000:20:210
1294637993423,6cyclictest13-21ksoftirqd/020:20:010
129464599331,15cyclictest4497-21PK-Backend19:20:302
129464099331,8cyclictest4497-21PK-Backend00:01:531
129464099331,29cyclictest4497-21PK-Backend21:40:211
129464099331,29cyclictest4497-21PK-Backend21:40:211
129464099331,29cyclictest4497-21PK-Backend21:26:361
129464099331,29cyclictest4497-21PK-Backend00:10:101
129464099321,8cyclictest4497-21PK-Backend00:09:011
129464099311,4cyclictest4497-21PK-Backend22:08:111
129464099311,4cyclictest4497-21PK-Backend22:08:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional