You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 18:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue Dec 16, 2025 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22523462900,7sleep14497-21PK-Backend11:40:021
22380032900,5sleep34497-21PK-Backend11:00:003
197399780,10rtkit-daemon2746-21gmain09:30:221
2156119997741,9cyclictest13-21ksoftirqd/008:40:540
215449127751,16sleep10-21swapper/107:05:091
197399770,62rtkit-daemon0-21swapper/207:09:012
2156119997623,50cyclictest161rcuc/008:35:580
215612699726,15cyclictest923-21dbus-daemon08:35:032
215612699675,40cyclictest2250333-21systemd-udevd11:33:562
215457026743,16sleep30-21swapper/307:05:113
215583926545,14sleep00-21swapper/007:08:260
22697062640,24sleep126-21ksoftirqd/112:25:301
22697062640,24sleep126-21ksoftirqd/112:25:301
2156129996433,16cyclictest42-21ksoftirqd/309:05:583
2156126996451,9cyclictest2069-21gnome-shell11:55:592
2156126996310,29cyclictest2069-21gnome-shell08:53:082
2156126996310,29cyclictest2069-21gnome-shell08:53:082
2156126996151,6cyclictest2069-21gnome-shell12:33:582
22127832600,13sleep1261ksoftirqd/109:45:251
2156126996050,6cyclictest2069-21gnome-shell08:41:582
215611999602,54cyclictest2069-21gnome-shell10:33:590
215611999602,54cyclictest2069-21gnome-shell10:33:590
215611999602,15cyclictest2069-21gnome-shell09:46:580
215612699597,10cyclictest2246283-21users11:20:312
2156123995938,16cyclictest2069-21gnome-shell09:25:581
215611999592,16cyclictest2193950-40lsb_release08:52:580
215611999592,16cyclictest2193950-40lsb_release08:52:580
215612999582,15cyclictest2069-21gnome-shell12:38:593
2156126995748,6cyclictest2069-21gnome-shell10:57:592
2156123995727,6cyclictest141rcu_preempt09:23:581
2156123995725,5cyclictest141rcu_preempt11:14:591
215611999579,45cyclictest2069-21gnome-shell09:24:580
215611999571,52cyclictest2069-21gnome-shell11:37:590
215611999571,52cyclictest2069-21gnome-shell09:09:580
215612999562,50cyclictest2069-21gnome-shell10:53:593
215612999562,50cyclictest2069-21gnome-shell10:53:593
215612699564,11cyclictest2179098-21systemd-run08:15:002
2156123995626,6cyclictest141rcu_preempt09:18:581
215612699557,15cyclictest2179594-21cstates08:15:152
215612699556,20cyclictest2217131-21date10:00:022
215612699554,24cyclictest0-21swapper/210:30:032
215612699554,24cyclictest0-21swapper/210:30:032
2156126995514,15cyclictest2234173-21taskset10:45:302
215611999552,11cyclictest2250141-40lsb_release11:31:590
2156123995426,6cyclictest141rcu_preempt09:35:581
215612399542,10cyclictest2069-21gnome-shell07:32:581
215611999542,48cyclictest2069-21gnome-shell09:19:580
215611999542,10cyclictest2069-21gnome-shell08:16:580
215612999532,41cyclictest4497-21PK-Backend09:32:583
215612699539,18cyclictest2165315-21sed07:35:022
215612699538,21cyclictest2160664-21idleruntime07:20:182
215612699538,21cyclictest2160664-21idleruntime07:20:182
215612699537,18cyclictest2238558-21expr11:00:132
215612699536,24cyclictest2177424-21latency_hist08:10:022
215612699536,24cyclictest2177424-21latency_hist08:10:022
215612699536,18cyclictest2240339-21expr11:05:152
215612699534,21cyclictest2259910-21cstates12:00:142
215612699532,10cyclictest2069-21gnome-shell12:13:592
215612699532,10cyclictest2069-21gnome-shell12:07:582
2156123995325,5cyclictest141rcu_preempt08:00:581
215612399532,47cyclictest2069-21gnome-shell07:47:581
215612399532,10cyclictest2069-21gnome-shell07:44:581
215611999532,41cyclictest2069-21gnome-shell10:42:590
215611999532,10cyclictest2069-21gnome-shell10:26:590
2156119995311,28cyclictest141rcu_preempt09:54:580
215612999522,46cyclictest2069-21gnome-shell12:27:593
215612999522,46cyclictest2069-21gnome-shell12:27:593
215612999521,10cyclictest2069-21gnome-shell11:26:593
215612699527,18cyclictest2256340-21cat11:50:152
215612699526,22cyclictest1-21systemd10:30:022
215612699526,17cyclictest2244939-21munin-run11:20:002
215612699526,17cyclictest2244939-21munin-run11:20:002
215612699525,25cyclictest0-21swapper/208:55:272
215612699524,30cyclictest2247533-21cat11:25:232
215612399522,45cyclictest2069-21gnome-shell09:53:581
2156119995245,5cyclictest2156112-21cyclictest09:28:580
215611999522,9cyclictest2069-21gnome-shell09:30:580
22529532510,5sleep0161rcuc/011:40:210
2156129995144,5cyclictest4497-21PK-Backend08:36:583
215612699517,18cyclictest2069-21gnome-shell08:47:082
215612699517,16cyclictest2242603-21sensors11:10:262
215612699516,27cyclictest2201391-21munin-run09:15:002
215612699516,18cyclictest2172734-21cat07:55:142
2156123995144,4cyclictest4497-21PK-Backend12:19:591
215612399512,5cyclictest0-21swapper/110:29:591
215612399512,45cyclictest2069-21gnome-shell07:22:581
215612399512,45cyclictest2069-21gnome-shell07:22:581
215611999515,43cyclictest0-21swapper/012:03:590
2156119995139,8cyclictest2069-21gnome-shell07:52:580
2156119995137,10cyclictest2069-21gnome-shell07:20:580
2156119995137,10cyclictest2069-21gnome-shell07:20:580
215611999512,45cyclictest2069-21gnome-shell10:14:590
215611999512,45cyclictest2069-21gnome-shell08:02:580
215612699506,16cyclictest1-21systemd10:10:002
215612399502,45cyclictest231irq_work/108:07:581
215612399502,45cyclictest231irq_work/108:07:581
215611999507,38cyclictest2069-21gnome-shell07:38:580
215611999501,45cyclictest2069-21gnome-shell11:14:590
215612699497,16cyclictest2273750-21apt-config12:40:022
215612699495,20cyclictest2174786-21cat08:00:222
215612699495,20cyclictest2171666-21sed07:50:292
215612699494,42cyclictest0-21swapper/208:32:582
215612699491,33cyclictest0-21swapper/209:21:112
215612399492,5cyclictest4497-21PK-Backend08:21:581
2156119994939,7cyclictest2069-21gnome-shell08:59:580
215611999492,6cyclictest0-21swapper/010:08:590
215611999492,44cyclictest0-21swapper/008:22:580
215612699485,22cyclictest2266894-21/usr/sbin/munin12:20:282
215612699482,7cyclictest0-21swapper/212:25:592
215612699482,7cyclictest0-21swapper/212:25:592
215612699482,6cyclictest0-21swapper/209:25:582
215612699482,43cyclictest0-21swapper/210:21:592
215612699482,43cyclictest0-21swapper/207:44:582
215612699482,42cyclictest0-21swapper/209:32:582
2156123994840,5cyclictest231irq_work/111:26:591
215612399482,6cyclictest0-21swapper/111:51:591
215612399482,44cyclictest231irq_work/110:30:591
215612399482,44cyclictest231irq_work/110:30:591
215612399482,43cyclictest231irq_work/109:07:581
215612399481,6cyclictest4497-21PK-Backend10:43:591
215612399481,5cyclictest4497-21PK-Backend10:11:591
215612399481,43cyclictest4497-21PK-Backend12:23:591
215611999486,5cyclictest141rcu_preempt07:15:000
215611999482,5cyclictest0-21swapper/007:48:580
215611999482,44cyclictest0-21swapper/010:53:590
215611999482,44cyclictest0-21swapper/010:53:590
215611999482,43cyclictest0-21swapper/011:04:580
215611999482,43cyclictest0-21swapper/010:48:590
2156129994736,7cyclictest2069-21gnome-shell12:16:593
215612999472,6cyclictest0-21swapper/310:14:593
215612999472,42cyclictest4497-21PK-Backend11:44:593
215612999472,42cyclictest4497-21PK-Backend10:06:593
2156129994717,14cyclictest42-21ksoftirqd/311:20:313
215612999471,6cyclictest4497-21PK-Backend11:12:593
215612999471,42cyclictest4497-21PK-Backend08:00:583
215612699478,15cyclictest2201901-21cat09:15:142
215612699477,17cyclictest2169276-21expr07:45:142
2156126994739,5cyclictest0-21swapper/211:41:592
2156126994738,6cyclictest0-21swapper/211:36:582
2156126994738,6cyclictest0-21swapper/210:50:592
2156126994738,6cyclictest0-21swapper/210:50:592
215612699472,5cyclictest0-21swapper/209:38:582
2156126994710,15cyclictest2184099-21taskset08:28:112
215612399472,5cyclictest0-21swapper/111:06:591
215612399471,6cyclictest4497-21PK-Backend12:00:591
215612399471,42cyclictest4497-21PK-Backend10:54:591
215612399471,42cyclictest4497-21PK-Backend10:54:591
215612399471,42cyclictest4497-21PK-Backend10:15:591
215612399471,42cyclictest4497-21PK-Backend08:28:581
215612399471,42cyclictest4497-21PK-Backend07:12:581
215611999472,5cyclictest0-21swapper/011:46:590
215611999472,5cyclictest0-21swapper/011:20:590
215611999472,42cyclictest0-21swapper/010:20:590
215611999472,41cyclictest0-21swapper/012:35:590
2156119994715,17cyclictest13-21ksoftirqd/008:10:220
2156119994713,19cyclictest141rcu_preempt08:45:140
2156129994638,5cyclictest4497-21PK-Backend07:20:583
2156129994638,5cyclictest4497-21PK-Backend07:20:583
2156129994638,5cyclictest411rcuc/312:01:583
215612999462,5cyclictest0-21swapper/308:22:583
215612999462,41cyclictest4497-21PK-Backend10:24:593
215612999462,41cyclictest0-21swapper/308:25:583
215612999462,37cyclictest0-21swapper/309:40:583
215612999462,37cyclictest0-21swapper/309:40:583
215612999462,37cyclictest0-21swapper/309:27:583
2156129994617,25cyclictest2197914-21cron09:05:003
215612999461,5cyclictest4497-21PK-Backend11:04:593
215612999461,5cyclictest4497-21PK-Backend07:51:583
215612999461,5cyclictest4497-21PK-Backend07:48:593
215612999461,42cyclictest4497-21PK-Backend11:31:593
215612699467,16cyclictest2254507-21cpu11:45:132
2156126994639,5cyclictest0-21swapper/207:37:582
2156126994638,5cyclictest0-21swapper/210:43:592
2156126994638,5cyclictest0-21swapper/208:23:582
215612699462,5cyclictest0-21swapper/209:00:582
215612699462,41cyclictest0-21swapper/207:13:582
2156126994610,15cyclictest2223766-21taskset10:18:542
2156123994639,5cyclictest231irq_work/108:51:591
2156123994639,5cyclictest231irq_work/108:51:581
2156119994629,13cyclictest2069-21gnome-shell09:13:580
215611999462,41cyclictest0-21swapper/009:36:580
215611999461,23cyclictest2215585-21chrt09:55:140
215611999461,21cyclictest13-21ksoftirqd/008:00:010
2156129994539,4cyclictest0-21swapper/308:59:593
215612999452,41cyclictest0-21swapper/310:48:593
215612999451,41cyclictest4497-21PK-Backend10:25:593
215612999451,41cyclictest4497-21PK-Backend09:36:583
215612999451,41cyclictest4497-21PK-Backend07:59:583
215612999451,41cyclictest4497-21PK-Backend07:41:583
215612999451,40cyclictest4497-21PK-Backend09:21:583
215612699457,14cyclictest2198559-21df09:05:162
2156126994537,5cyclictest0-21swapper/210:02:592
215612699453,24cyclictest0-21swapper/209:45:312
215612699452,41cyclictest0-21swapper/210:10:592
2156123994539,4cyclictest0-21swapper/111:19:591
2156123994539,4cyclictest0-21swapper/111:19:591
2156123994539,4cyclictest0-21swapper/111:01:591
2156123994538,4cyclictest4497-21PK-Backend07:17:591
2156123994537,5cyclictest231irq_work/110:38:591
2156123994523,18cyclictest2191826-31apt-check08:46:521
215611999455,36cyclictest2069-21gnome-shell10:18:590
215611999455,35cyclictest2069-21gnome-shell09:03:580
215611999454,36cyclictest2069-21gnome-shell07:31:580
2156119994529,12cyclictest2069-21gnome-shell08:07:580
2156119994529,12cyclictest2069-21gnome-shell08:07:580
2156129994437,4cyclictest4497-21PK-Backend08:47:583
215612999442,40cyclictest0-21swapper/311:38:593
215612999442,40cyclictest0-21swapper/307:19:583
215612999441,40cyclictest4497-21PK-Backend11:56:593
215612999441,40cyclictest4497-21PK-Backend08:13:593
2156129994412,9cyclictest42-21ksoftirqd/307:40:003
2156126994438,4cyclictest0-21swapper/209:44:582
2156126994438,4cyclictest0-21swapper/209:44:582
2156123994429,10cyclictest2225645-21kworker/1:110:47:591
215611999443,36cyclictest2069-21gnome-shell07:16:580
2156119994414,9cyclictest13-21ksoftirqd/012:30:000
2156119994414,9cyclictest13-21ksoftirqd/012:30:000
2156119994414,14cyclictest141rcu_preempt11:07:250
2156129994331,8cyclictest2069-21gnome-shell12:21:593
2156129994328,11cyclictest2157798-21systemd-run07:15:013
215612999431,15cyclictest4497-21PK-Backend07:35:013
215612399431,7cyclictest231irq_work/109:13:581
215612399431,38cyclictest2131625-21kworker/1:107:56:581
2156119994316,6cyclictest141rcu_preempt12:15:010
2156119994316,14cyclictest141rcu_preempt10:39:040
215611999431,37cyclictest2069-21gnome-shell10:03:590
2156129994236,4cyclictest0-21swapper/310:44:593
215612999421,8cyclictest937-21polkitd08:40:543
2156126994227,11cyclictest0-21swapper/212:19:592
2156123994235,4cyclictest4497-21PK-Backend12:05:591
2156119994236,4cyclictest0-21swapper/012:16:590
215611999423,16cyclictest13-21ksoftirqd/007:40:240
215611999422,11cyclictest958-21thermald12:00:010
2156119994215,14cyclictest141rcu_preempt09:41:220
2156119994215,14cyclictest141rcu_preempt09:41:220
2156119994213,16cyclictest13-21ksoftirqd/011:18:200
2156119994213,16cyclictest13-21ksoftirqd/011:18:200
215611999421,11cyclictest2257689-21cron11:55:000
215612999411,10cyclictest2216953-21sh10:00:003
2156126994134,5cyclictest0-21swapper/207:18:582
2156126994113,23cyclictest2230071-21if_enp2s010:35:202
2156123994134,5cyclictest231irq_work/108:17:581
2156123994134,4cyclictest4497-21PK-Backend12:30:591
2156123994133,5cyclictest231irq_work/109:43:591
2156123994133,5cyclictest231irq_work/109:43:591
2156123994131,6cyclictest4497-21PK-Backend08:32:581
2156123994115,6cyclictest141rcu_preempt09:58:591
2156119994114,15cyclictest13-21ksoftirqd/010:58:480
2156119994114,14cyclictest141rcu_preempt08:33:010
2156119994112,9cyclictest13-21ksoftirqd/011:28:430
2156119994112,8cyclictest13-21ksoftirqd/012:30:130
2156119994112,8cyclictest13-21ksoftirqd/007:29:580
215612999403,19cyclictest42-21ksoftirqd/308:35:023
215612999402,34cyclictest2069-21gnome-shell12:08:593
2156129994010,6cyclictest141rcu_preempt11:55:023
2156123994025,11cyclictest2131625-21kworker/1:108:13:581
215612399402,34cyclictest26-21ksoftirqd/110:04:591
215612999397,13cyclictest42-21ksoftirqd/311:05:143
2156129993932,4cyclictest4497-21PK-Backend09:54:593
2156129993916,6cyclictest141rcu_preempt10:20:023
2156129993910,6cyclictest141rcu_preempt12:31:253
2156126993913,21cyclictest2214505-21ntp_states09:50:242
2156123993930,6cyclictest231irq_work/108:55:581
2156123993930,6cyclictest231irq_work/108:36:581
2156123993928,7cyclictest4497-21PK-Backend11:42:591
2156119993914,6cyclictest141rcu_preempt08:25:220
2156119993912,6cyclictest141rcu_preempt12:20:130
2156119993911,6cyclictest141rcu_preempt12:05:050
2156129993813,6cyclictest141rcu_preempt09:48:193
2156129993813,6cyclictest141rcu_preempt08:18:573
215612999381,16cyclictest4497-21PK-Backend11:20:003
215612999381,16cyclictest4497-21PK-Backend11:20:003
215612999381,15cyclictest4497-21PK-Backend10:30:143
215612999381,15cyclictest4497-21PK-Backend10:30:143
2156123993828,7cyclictest231irq_work/110:23:591
2156123993826,7cyclictest231irq_work/112:12:591
215612999378,5cyclictest141rcu_preempt11:45:263
215612999378,5cyclictest141rcu_preempt08:05:133
215612999378,5cyclictest141rcu_preempt08:05:133
2156129993727,7cyclictest411rcuc/310:35:153
215612999371,6cyclictest4497-21PK-Backend09:15:003
2156129993714,6cyclictest141rcu_preempt10:00:153
2156126993727,6cyclictest1137-21runrttasks07:29:582
2156123993726,7cyclictest4497-21PK-Backend12:37:591
2156123993726,7cyclictest0-21swapper/111:47:581
2156123993725,8cyclictest231irq_work/111:21:591
2156123993722,11cyclictest923-21dbus-daemon09:05:001
215612399371,13cyclictest2259064-21taskset11:55:321
2156129993629,4cyclictest0-21swapper/312:13:593
2156129993620,5cyclictest42-21ksoftirqd/308:53:213
2156129993620,5cyclictest42-21ksoftirqd/308:53:213
2156129993619,6cyclictest42-21ksoftirqd/309:15:513
215612399363,4cyclictest141rcu_preempt10:05:221
2156123993629,5cyclictest231irq_work/111:33:591
2156123993628,5cyclictest0-21swapper/107:52:581
2156123993625,7cyclictest231irq_work/107:39:591
2156123993625,7cyclictest231irq_work/107:26:581
215612399361,31cyclictest0-21swapper/108:41:581
215612999352,29cyclictest2162363-21diskstats07:25:163
215612399351,10cyclictest4497-21PK-Backend10:55:461
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional