You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 10:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Dec 16, 2025 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
186896122610,12sleep013-21ksoftirqd/019:08:120
197399980,5rtkit-daemon0-21swapper/023:40:500
197399970,14rtkit-daemon4497-21PK-Backend19:16:220
1869287999122,50cyclictest2069-21gnome-shell00:36:593
197399880,26rtkit-daemon4497-21PK-Backend22:30:380
197399870,11rtkit-daemon4497-21PK-Backend21:18:570
197399830,10rtkit-daemon1977842-21df_inode00:15:160
197399810,5rtkit-daemon0-21swapper/019:45:380
197399800,5rtkit-daemon0-21swapper/000:02:240
197399790,5rtkit-daemon0-21swapper/023:18:300
1869287997921,54cyclictest2069-21gnome-shell22:20:593
197399780,6rtkit-daemon4497-21PK-Backend21:10:300
197399780,5rtkit-daemon0-21swapper/022:40:490
197399780,5rtkit-daemon0-21swapper/020:32:150
197399770,6rtkit-daemon4497-21PK-Backend23:51:450
197399770,6rtkit-daemon4497-21PK-Backend23:14:120
197399770,5rtkit-daemon0-21swapper/023:27:430
197399770,5rtkit-daemon0-21swapper/021:01:010
197399760,6rtkit-daemon4497-21PK-Backend20:40:310
197399760,5rtkit-daemon4497-21PK-Backend20:56:480
197399760,5rtkit-daemon0-21swapper/020:13:380
1869287997622,44cyclictest2069-21gnome-shell23:44:583
186927999762,68cyclictest2069-21gnome-shell23:05:581
197399750,6rtkit-daemon4497-21PK-Backend21:31:120
197399750,6rtkit-daemon4497-21PK-Backend00:35:220
197399750,5rtkit-daemon4497-21PK-Backend23:49:440
197399750,5rtkit-daemon4497-21PK-Backend00:05:450
197399750,5rtkit-daemon0-21swapper/020:15:090
186927999752,68cyclictest2069-21gnome-shell23:04:591
186882127539,14sleep30-21swapper/319:06:313
197399740,6rtkit-daemon4497-21PK-Backend21:40:200
197399740,61rtkit-daemon0-21swapper/119:07:141
197399740,5rtkit-daemon4497-21PK-Backend19:50:550
197399740,5rtkit-daemon0-21swapper/023:08:410
197399740,4rtkit-daemon0-21swapper/020:09:210
197399730,6rtkit-daemon4497-21PK-Backend22:08:480
186927999731,30cyclictest1869270-21cyclictest22:00:581
197399720,5rtkit-daemon0-21swapper/020:26:500
1869287997257,10cyclictest2069-21gnome-shell23:52:583
186927999722,65cyclictest2069-21gnome-shell19:47:581
18851362710,4chrt1885135-21sshd19:52:573
1869287997156,11cyclictest2069-21gnome-shell22:17:593
1869287997122,8cyclictest2069-21gnome-shell20:36:593
1869279997160,7cyclictest2069-21gnome-shell20:13:581
186927999712,27cyclictest2069-21gnome-shell21:31:581
197399700,5rtkit-daemon0-21swapper/021:51:090
1869287997056,10cyclictest2069-21gnome-shell23:45:583
197399690,5rtkit-daemon4497-21PK-Backend21:07:120
1869287996919,44cyclictest2069-21gnome-shell21:20:593
197399680,6rtkit-daemon4497-21PK-Backend23:30:190
197399680,6rtkit-daemon4497-21PK-Backend23:30:190
197399680,5rtkit-daemon4497-21PK-Backend19:29:520
18959512680,8sleep01895953-21latency_hist20:25:020
18959512680,8sleep01895953-21latency_hist20:25:020
1869287996858,6cyclictest2069-21gnome-shell23:55:593
186927999682,61cyclictest2069-21gnome-shell19:37:591
186927999642,58cyclictest0-21swapper/121:23:591
186927999642,55cyclictest0-21swapper/123:52:591
1869279996416,25cyclictest1137-21runrttasks21:42:551
186927999632,57cyclictest0-21swapper/123:48:591
1869279996210,29cyclictest1955804-21taskset23:11:421
1869287996113,45cyclictest4497-21PK-Backend20:32:593
186927999611,54cyclictest2069-21gnome-shell19:31:591
1869279996114,27cyclictest1137-21runrttasks20:52:021
186927999611,31cyclictest1887480-21cpuspeed_turbos20:00:141
186881526140,14sleep20-21swapper/219:06:272
186928799602,51cyclictest2069-21gnome-shell23:16:583
186927999602,32cyclictest1934253-21ntp_states22:10:271
1869277996038,5cyclictest141rcu_preempt19:21:590
1869277996038,5cyclictest141rcu_preempt19:21:580
1869287995941,13cyclictest2069-21gnome-shell00:32:593
186928799592,5cyclictest0-21swapper/323:02:593
186928799592,15cyclictest2069-21gnome-shell22:59:583
186927999591,34cyclictest1957212-21sensors_temp23:15:291
186927999591,34cyclictest1943881-21taskset22:39:031
186928799589,28cyclictest1976941-21taskset00:11:543
1869287995815,20cyclictest1973475-21taskset00:02:033
186927999582,14cyclictest2069-21gnome-shell21:59:591
186927999582,14cyclictest2069-21gnome-shell21:59:581
186927999581,14cyclictest2069-21gnome-shell21:05:581
1869279995746,8cyclictest26-21ksoftirqd/123:28:591
1869279995726,26cyclictest2069-21gnome-shell00:12:591
186927999572,32cyclictest1973455-40lsb_release00:01:581
186927999572,29cyclictest1914561-21hddtemp_smartct21:15:211
186927999571,51cyclictest2069-21gnome-shell19:26:591
186927999571,33cyclictest1927348-21timerwakeupswit21:50:301
186927799571,52cyclictest2069-21gnome-shell19:14:590
186927799571,52cyclictest2069-21gnome-shell19:14:580
186928799569,25cyclictest1940456-21taskset22:29:333
1869287995641,9cyclictest2069-21gnome-shell21:47:583
1869287995615,17cyclictest1903141-40apt-check20:41:053
1869287995611,43cyclictest0-21swapper/322:48:593
1869287995611,43cyclictest0-21swapper/322:48:593
1869284995636,7cyclictest34-21ksoftirqd/221:49:552
186927999561,32cyclictest1913918-21awk21:15:011
186928799551,14cyclictest2069-21gnome-shell21:38:593
186928799551,14cyclictest2069-21gnome-shell21:38:583
1869287995511,38cyclictest2069-21gnome-shell22:11:583
186928499556,17cyclictest34-21ksoftirqd/219:25:232
186927999552,27cyclictest0-21swapper/123:40:011
197399540,39rtkit-daemon4497-21PK-Backend23:55:140
186928799542,11cyclictest2069-21gnome-shell21:50:583
186928799542,10cyclictest2069-21gnome-shell20:55:583
1869287995414,19cyclictest1947665-21taskset22:50:023
1869287995414,16cyclictest1906385-21/usr/sbin/munin20:50:263
186927999542,48cyclictest1938584-21kworker/1:222:42:591
186927999542,46cyclictest1974070-21sort00:05:131
186927999542,10cyclictest2069-21gnome-shell23:21:591
186927999542,10cyclictest2069-21gnome-shell22:05:591
186928799532,47cyclictest2069-21gnome-shell00:20:583
186928799532,10cyclictest2069-21gnome-shell21:32:593
1869284995344,6cyclictest331rcuc/221:15:402
1869284995338,11cyclictest2069-21gnome-shell22:21:582
186928499532,10cyclictest2069-21gnome-shell00:17:582
186927999532,46cyclictest2069-21gnome-shell19:15:581
1869284995244,5cyclictest331rcuc/222:06:512
1869284995231,8cyclictest34-21ksoftirqd/221:29:552
186928499522,10cyclictest2069-21gnome-shell23:15:592
186928499522,10cyclictest2069-21gnome-shell22:46:592
186928499522,10cyclictest2069-21gnome-shell22:46:582
186927999522,47cyclictest231irq_work/123:34:581
186927999522,47cyclictest231irq_work/123:34:581
186927999522,45cyclictest2069-21gnome-shell19:10:591
186927999522,45cyclictest2069-21gnome-shell19:10:591
1869279995219,28cyclictest1977612-21grep00:15:141
186928799515,6cyclictest0-21swapper/319:47:593
186928499515,43cyclictest0-21swapper/220:27:582
1869284995144,5cyclictest331rcuc/221:42:572
1869284995143,5cyclictest331rcuc/200:01:332
1869284995142,6cyclictest331rcuc/223:57:212
1869284995138,8cyclictest2069-21gnome-shell22:54:592
1869279995140,8cyclictest231irq_work/120:34:591
186927999513,45cyclictest231irq_work/119:23:591
186927999513,45cyclictest231irq_work/119:23:581
186927999512,8cyclictest231irq_work/120:17:581
186927999512,7cyclictest0-21swapper/120:38:581
186927999512,45cyclictest0-21swapper/120:58:591
186927999511,8cyclictest2069-21gnome-shell19:43:581
186928799509,19cyclictest2069-21gnome-shell20:26:073
186928799508,17cyclictest1953638-21ntp_states23:05:273
186928799506,38cyclictest2069-21gnome-shell20:47:583
1869287995010,16cyclictest1979160-21sed00:20:013
1869284995043,5cyclictest331rcuc/223:24:222
1869284995043,5cyclictest331rcuc/222:41:192
1869284995042,5cyclictest331rcuc/220:08:472
186928499502,6cyclictest4497-21PK-Backend22:59:582
186928499502,6cyclictest0-21swapper/200:12:592
186927999502,7cyclictest231irq_work/122:57:591
186927999502,7cyclictest0-21swapper/122:30:581
186927999502,45cyclictest2069-21gnome-shell19:58:591
186927999502,44cyclictest231irq_work/120:07:581
186927999502,44cyclictest231irq_work/100:20:591
186927999502,44cyclictest0-21swapper/122:15:591
186927999502,44cyclictest0-21swapper/100:30:581
186927799502,44cyclictest4497-21PK-Backend22:52:580
1869277995016,6cyclictest13-21ksoftirqd/023:35:580
186928799497,17cyclictest1886327-21ntp_states19:55:243
186928799495,16cyclictest1914541-21diskstats21:15:183
186928499494,22cyclictest34-21ksoftirqd/219:25:022
186928499494,22cyclictest34-21ksoftirqd/219:25:022
1869284994940,6cyclictest331rcuc/220:12:062
1869284994928,16cyclictest2069-21gnome-shell20:53:582
186928499492,7cyclictest0-21swapper/219:36:582
186928499492,44cyclictest0-21swapper/221:09:592
186928499492,38cyclictest0-21swapper/200:27:582
1869284994920,17cyclictest34-21ksoftirqd/223:14:562
1869279994940,6cyclictest231irq_work/122:50:591
186927999492,6cyclictest0-21swapper/119:54:591
186927999492,44cyclictest0-21swapper/123:55:581
186927999492,44cyclictest0-21swapper/123:41:591
186927999492,44cyclictest0-21swapper/100:38:581
186927999492,43cyclictest0-21swapper/120:47:591
186927999492,38cyclictest0-21swapper/100:28:581
186927999491,44cyclictest231irq_work/120:20:591
186927999491,44cyclictest231irq_work/120:20:591
197399480,32rtkit-daemon1981308-21diskstats00:25:180
186928799489,15cyclictest1875743-21/usr/sbin/munin19:25:213
186928799488,16cyclictest1877372-21irqstats19:30:193
186928799482,6cyclictest0-21swapper/322:04:583
186928799482,6cyclictest0-21swapper/319:41:593
186928799482,42cyclictest0-21swapper/321:05:593
186928799482,37cyclictest0-21swapper/320:04:593
186928499487,21cyclictest34-21ksoftirqd/222:30:312
186928499484,5cyclictest331rcuc/221:20:582
186928499482,7cyclictest0-21swapper/222:13:582
186928499482,7cyclictest0-21swapper/220:49:592
186928499482,6cyclictest4497-21PK-Backend23:39:592
186928499482,43cyclictest0-21swapper/223:43:592
186928499482,43cyclictest0-21swapper/220:40:592
186928499482,43cyclictest0-21swapper/200:30:582
186928499481,45cyclictest331rcuc/223:07:582
1869279994839,6cyclictest231irq_work/122:27:581
186927999482,43cyclictest0-21swapper/122:49:581
186927999482,43cyclictest0-21swapper/122:49:581
186927999482,43cyclictest0-21swapper/121:35:581
186927999482,43cyclictest0-21swapper/121:35:581
1869279994819,24cyclictest1896269-21cpu20:25:151
186927999481,4cyclictest0-21swapper/121:47:591
186927799482,5cyclictest4497-21PK-Backend22:55:580
186927799482,43cyclictest0-21swapper/022:24:590
186927799482,43cyclictest0-21swapper/021:29:590
186927799481,6cyclictest4497-21PK-Backend19:56:590
186927799481,5cyclictest4497-21PK-Backend19:32:590
19368352470,6sleep24497-21PK-Backend22:20:012
186928799478,16cyclictest1889622-21memory20:05:223
186928799477,15cyclictest1981679-21ntp_states00:25:253
186928799472,6cyclictest0-21swapper/323:35:583
186928799472,5cyclictest0-21swapper/322:05:583
186928799472,5cyclictest0-21swapper/319:35:593
186928799472,42cyclictest0-21swapper/320:16:593
186928799472,42cyclictest0-21swapper/319:10:593
186928799472,42cyclictest0-21swapper/319:10:583
186928799472,42cyclictest0-21swapper/300:08:583
186928799472,41cyclictest0-21swapper/321:59:593
186928799472,41cyclictest0-21swapper/321:59:593
186928799471,6cyclictest4497-21PK-Backend23:10:593
186928799471,42cyclictest4497-21PK-Backend22:41:583
186928799471,42cyclictest4497-21PK-Backend22:37:583
1869284994739,5cyclictest331rcuc/220:24:502
1869284994739,5cyclictest331rcuc/220:24:502
1869284994738,6cyclictest331rcuc/222:26:552
1869284994737,7cyclictest2069-21gnome-shell22:37:582
186928499472,6cyclictest0-21swapper/221:30:582
186928499472,43cyclictest0-21swapper/200:23:582
186928499472,42cyclictest0-21swapper/223:45:582
186928499472,42cyclictest0-21swapper/221:13:592
186928499472,42cyclictest0-21swapper/220:58:582
186928499472,42cyclictest0-21swapper/220:36:582
186928499472,42cyclictest0-21swapper/219:53:592
1869284994716,18cyclictest34-21ksoftirqd/221:04:312
1869279994739,5cyclictest0-21swapper/122:20:581
1869279994725,18cyclictest0-21swapper/121:29:581
1869279994721,22cyclictest1137-21runrttasks21:00:151
1869277994739,5cyclictest4497-21PK-Backend23:23:580
186927799472,43cyclictest0-21swapper/020:45:580
186927799471,43cyclictest4497-21PK-Backend21:24:590
1869287994638,5cyclictest4497-21PK-Backend21:27:593
1869287994628,13cyclictest0-21swapper/323:29:583
186928799462,5cyclictest0-21swapper/320:12:593
186928799462,42cyclictest0-21swapper/319:17:593
186928799462,41cyclictest0-21swapper/323:30:593
186928799462,41cyclictest0-21swapper/323:30:593
186928799462,41cyclictest0-21swapper/320:24:583
186928799462,41cyclictest0-21swapper/320:24:583
186928799461,41cyclictest4497-21PK-Backend23:20:583
186928799461,41cyclictest4497-21PK-Backend21:04:593
1869284994639,4cyclictest0-21swapper/219:46:592
1869284994638,5cyclictest0-21swapper/222:00:592
1869284994637,6cyclictest331rcuc/221:53:022
186928499462,5cyclictest0-21swapper/219:57:582
186928499462,41cyclictest4497-21PK-Backend00:06:592
186928499462,41cyclictest0-21swapper/220:31:582
186928499461,5cyclictest4497-21PK-Backend21:38:592
186928499461,5cyclictest4497-21PK-Backend21:38:592
186928499461,42cyclictest4497-21PK-Backend21:57:592
186928499461,42cyclictest4497-21PK-Backend21:57:582
186928499461,37cyclictest4497-21PK-Backend19:44:582
186927799461,5cyclictest4497-21PK-Backend20:01:590
186928799455,17cyclictest1913177-21ntp_states21:10:253
1869284994537,5cyclictest0-21swapper/223:52:582
1869284994537,5cyclictest0-21swapper/220:04:582
1869277994539,4cyclictest0-21swapper/022:19:590
1869287994436,5cyclictest0-21swapper/322:30:583
1869284994413,8cyclictest34-21ksoftirqd/219:30:132
1869277994437,4cyclictest4497-21PK-Backend19:42:590
1869284994334,6cyclictest331rcuc/223:27:152
197399420,28rtkit-daemon4497-21PK-Backend21:37:130
197399420,28rtkit-daemon4497-21PK-Backend21:37:130
186928799427,13cyclictest1874868-21taskset19:24:553
186928799427,13cyclictest1874868-21taskset19:24:543
186928499429,17cyclictest34-21ksoftirqd/223:01:172
186928499429,12cyclictest34-21ksoftirqd/220:16:112
1869284994233,6cyclictest331rcuc/219:10:442
1869284994233,6cyclictest331rcuc/219:10:432
186927799429,5cyclictest13-21ksoftirqd/021:59:590
186927799429,5cyclictest13-21ksoftirqd/021:59:590
1869277994235,4cyclictest4497-21PK-Backend22:35:580
186927799421,38cyclictest0-21swapper/019:38:590
19760052410,17sleep013-21ksoftirqd/000:10:140
186928799411,35cyclictest2069-21gnome-shell21:44:583
186928499415,18cyclictest34-21ksoftirqd/223:35:012
186928499415,18cyclictest34-21ksoftirqd/223:35:012
1869284994132,6cyclictest0-21swapper/200:37:592
197399400,26rtkit-daemon4497-21PK-Backend00:33:290
1869284994034,4cyclictest0-21swapper/219:17:582
1869277994033,4cyclictest4497-21PK-Backend20:54:580
197399390,26rtkit-daemon4497-21PK-Backend22:02:220
197399390,26rtkit-daemon4497-21PK-Backend21:45:420
197399390,25rtkit-daemon4497-21PK-Backend23:01:350
186927999391,9cyclictest0-21swapper/120:41:121
1869277993932,4cyclictest4497-21PK-Backend22:25:580
186927799381,21cyclictest1947124-21chrt22:46:160
186927799381,21cyclictest1947124-21chrt22:46:150
197399370,25rtkit-daemon4497-21PK-Backend00:23:340
1869277993727,6cyclictest13-21ksoftirqd/022:10:190
1869277993423,6cyclictest13-21ksoftirqd/020:40:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional