You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 06:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sun Jul 19, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220176222950,10sleep10-21swapper/119:05:231
227149621280,3sleep00-21swapper/022:25:010
228242021050,3sleep13056-21PK-Backend22:55:131
22019962970,15sleep013-21ksoftirqd/019:05:260
139099830,71rtkit-daemon0-21swapper/219:06:292
220240727839,13sleep30-21swapper/319:09:323
22774072600,2sleep33056-21PK-Backend22:40:163
22423542500,2sleep30-21swapper/321:00:153
22439232480,3sleep0161rcuc/021:05:130
2202626992817,7cyclictest1095-21runrttasks20:30:201
22733082250,3sleep3411rcuc/322:30:013
220262999241,4cyclictest1095-21runrttasks23:17:282
2202626992412,9cyclictest2211922-21anacron19:34:461
2202622992417,5cyclictest2252080-21cron21:30:000
220263499232,17cyclictest2213703-21munin-plugin-st19:40:003
2202629992315,6cyclictest436-20systemd-journal22:30:002
220262999230,21cyclictest0-21swapper/221:45:212
220262999230,21cyclictest0-21swapper/221:45:212
2202626992313,8cyclictest2299397-21cron23:45:001
2202626992313,8cyclictest2299397-21cron23:45:001
220262299230,22cyclictest3056-21PK-Backend22:56:520
2202634992220,1cyclictest3056-21PK-Backend20:28:313
2202634992213,7cyclictest436-20systemd-journal23:00:003
2202634992213,6cyclictest2255543-21cron21:39:593
220263499220,3cyclictest2291332-21memory23:20:193
2202629992216,4cyclictest2287007-21sh23:10:002
2202626992215,4cyclictest2310286-21cstates00:15:121
2202626992213,6cyclictest436-20systemd-journal22:15:001
2202622992220,1cyclictest0-21swapper/023:05:100
2202622992220,1cyclictest0-21swapper/020:55:380
2202622992220,1cyclictest0-21swapper/000:34:120
2202622992219,2cyclictest0-21swapper/023:49:240
2202622992214,6cyclictest2309221-21python300:10:210
220262299221,20cyclictest2250369-21perf21:25:010
220262299220,20cyclictest3056-21PK-Backend22:30:220
139099220,14rtkit-daemon0-21swapper/200:14:252
2202634992119,1cyclictest3056-21PK-Backend22:50:043
2202634992119,1cyclictest3056-21PK-Backend00:16:233
2202634992119,1cyclictest0-21swapper/322:30:583
2202634992119,1cyclictest0-21swapper/322:17:123
2202634992119,1cyclictest0-21swapper/321:28:333
2202634992119,1cyclictest0-21swapper/320:55:283
220263499211,8cyclictest2245907-21/usr/sbin/munin21:10:173
2202634992118,2cyclictest3056-21PK-Backend00:32:383
2202634992118,2cyclictest3056-21PK-Backend00:27:183
220263499211,19cyclictest42-21ksoftirqd/321:17:103
220263499210,20cyclictest0-21swapper/320:37:223
220263499210,19cyclictest3056-21PK-Backend20:40:463
2202629992119,1cyclictest0-21swapper/220:54:042
2202629992113,6cyclictest883-21dbus-daemon00:35:012
2202629992113,6cyclictest2255821-21awk21:40:082
2202629992113,6cyclictest2255821-21awk21:40:082
220262999210,20cyclictest0-21swapper/223:29:312
220262999210,1cyclictest0-21swapper/219:30:572
220262999210,19cyclictest0-21swapper/219:20:012
2202626992119,1cyclictest0-21swapper/100:10:201
2202626992113,6cyclictest2205996-21awk19:15:191
220262699210,20cyclictest0-21swapper/123:46:411
220262699210,19cyclictest3056-21PK-Backend23:01:591
2202622992119,1cyclictest0-21swapper/023:52:010
2202622992119,1cyclictest0-21swapper/023:42:340
2202622992119,1cyclictest0-21swapper/023:42:340
2202622992119,1cyclictest0-21swapper/023:37:210
2202622992119,1cyclictest0-21swapper/023:11:520
2202622992119,1cyclictest0-21swapper/022:26:370
2202622992119,1cyclictest0-21swapper/021:51:280
2202622992119,1cyclictest0-21swapper/021:45:070
2202622992119,1cyclictest0-21swapper/021:45:060
2202622992119,1cyclictest0-21swapper/021:17:010
2202622992119,1cyclictest0-21swapper/020:52:010
2202622992119,1cyclictest0-21swapper/020:41:530
2202622992119,1cyclictest0-21swapper/020:17:190
2202622992119,1cyclictest0-21swapper/020:10:090
2202622992119,1cyclictest0-21swapper/019:58:330
2202622992119,1cyclictest0-21swapper/019:25:340
2202622992119,1cyclictest0-21swapper/000:21:140
2202622992119,1cyclictest0-21swapper/000:08:500
2202622992118,2cyclictest0-21swapper/022:05:000
2202622992116,3cyclictest2291165-21idleruntime23:20:170
220262299210,20cyclictest0-21swapper/023:32:540
220262299210,20cyclictest0-21swapper/021:34:320
220262299210,20cyclictest0-21swapper/021:14:100
220262299210,20cyclictest0-21swapper/020:01:170
139099210,8rtkit-daemon0-21swapper/319:20:133
139099210,15rtkit-daemon3056-21PK-Backend20:20:363
139099210,15rtkit-daemon3056-21PK-Backend20:08:083
139099210,14rtkit-daemon0-21swapper/221:29:572
2202634992018,1cyclictest3056-21PK-Backend22:12:123
2202634992018,1cyclictest3056-21PK-Backend21:59:423
2202634992018,1cyclictest3056-21PK-Backend20:10:593
2202634992018,1cyclictest0-21swapper/321:06:563
2202634992018,1cyclictest0-21swapper/320:50:403
2202634992018,1cyclictest0-21swapper/319:41:323
2202634992015,3cyclictest2295938-21dump-pmu-power23:35:003
220263499200,19cyclictest3056-21PK-Backend19:28:473
220263499200,19cyclictest0-21swapper/321:48:533
220263499200,19cyclictest0-21swapper/321:48:533
220263499200,19cyclictest0-21swapper/321:41:233
220263499200,19cyclictest0-21swapper/321:41:233
220263499200,19cyclictest0-21swapper/321:32:103
2202629992018,1cyclictest0-21swapper/223:21:542
2202629992018,1cyclictest0-21swapper/222:49:432
2202629992018,1cyclictest0-21swapper/222:08:472
2202629992018,1cyclictest0-21swapper/220:40:532
2202629992018,1cyclictest0-21swapper/219:56:522
2202629992017,2cyclictest0-21swapper/221:35:222
2202629992013,5cyclictest34-21ksoftirqd/221:50:142
2202629992012,6cyclictest2242608-21grep21:00:172
220262999200,1cyclictest2202615-21cyclictest20:32:142
220262999200,1cyclictest0-21swapper/220:57:082
220262999200,1cyclictest0-21swapper/219:17:312
220262999200,19cyclictest0-21swapper/223:49:522
220262999200,19cyclictest0-21swapper/223:42:022
220262999200,19cyclictest0-21swapper/223:42:022
220262999200,19cyclictest0-21swapper/223:04:092
220262999200,19cyclictest0-21swapper/222:15:222
220262999200,19cyclictest0-21swapper/222:10:092
220262999200,19cyclictest0-21swapper/220:13:212
220262999200,19cyclictest0-21swapper/220:03:412
220262999200,19cyclictest0-21swapper/219:51:182
220262999200,19cyclictest0-21swapper/200:19:472
220262999200,19cyclictest0-21swapper/200:07:092
2202626992016,3cyclictest0-21swapper/122:40:171
2202626992016,2cyclictest2218906-21apt-get19:55:021
2202626992011,5cyclictest2297675-21turbostat.cron23:40:001
220262699200,2cyclictest141rcu_preempt00:40:001
220262699200,19cyclictest3056-21PK-Backend22:16:231
220262699200,19cyclictest3056-21PK-Backend21:09:311
220262699200,19cyclictest3056-21PK-Backend20:45:181
220262699200,19cyclictest0-21swapper/123:52:151
220262699200,19cyclictest0-21swapper/123:20:591
220262699200,19cyclictest0-21swapper/120:03:001
2202622992018,1cyclictest0-21swapper/023:17:400
2202622992018,1cyclictest0-21swapper/022:16:360
2202622992018,1cyclictest0-21swapper/022:12:440
2202622992018,1cyclictest0-21swapper/020:21:590
2202622992018,1cyclictest0-21swapper/020:05:430
2202622992018,1cyclictest0-21swapper/019:47:380
2202622992018,1cyclictest0-21swapper/019:38:320
2202622992018,1cyclictest0-21swapper/019:16:080
220262299200,19cyclictest0-21swapper/021:55:200
220262299200,19cyclictest0-21swapper/020:28:000
220262299200,19cyclictest0-21swapper/019:54:350
220262299200,19cyclictest0-21swapper/019:11:220
139099200,12rtkit-daemon0-21swapper/223:50:272
139099200,12rtkit-daemon0-21swapper/220:45:452
22932432190,2sleep10-21swapper/123:25:171
22855522190,4chrt3056-21PK-Backend23:05:121
2202634991917,1cyclictest3056-21PK-Backend23:50:413
2202634991917,1cyclictest3056-21PK-Backend22:09:423
2202634991917,1cyclictest3056-21PK-Backend00:20:223
2202634991917,1cyclictest0-21swapper/323:15:193
2202634991917,1cyclictest0-21swapper/323:00:183
2202634991917,1cyclictest0-21swapper/319:45:173
2202634991916,2cyclictest3056-21PK-Backend20:31:473
2202634991914,3cyclictest2301150-21turbostat23:50:003
2202634991914,3cyclictest2286413-21python323:05:213
2202634991911,6cyclictest2288739-21cron23:15:003
220263499190,1cyclictest3056-21PK-Backend22:49:163
220263499190,1cyclictest3056-21PK-Backend22:01:383
2202629991917,1cyclictest0-21swapper/223:12:282
2202629991917,1cyclictest0-21swapper/222:34:152
2202629991917,1cyclictest0-21swapper/221:32:512
2202629991917,1cyclictest0-21swapper/221:11:212
2202629991917,1cyclictest0-21swapper/221:07:102
2202629991917,1cyclictest0-21swapper/220:18:012
2202629991917,1cyclictest0-21swapper/219:40:362
2202629991914,3cyclictest2305545-21needreboot00:00:182
2202629991913,5cyclictest2213176-21expr19:35:202
2202629991910,6cyclictest34-21ksoftirqd/221:15:142
220262999190,1cyclictest3056-21PK-Backend20:05:382
220262999190,1cyclictest0-21swapper/222:22:122
220262999190,1cyclictest0-21swapper/220:25:252
220262999190,1cyclictest0-21swapper/219:48:542
220262999190,18cyclictest0-21swapper/223:55:262
220262999190,18cyclictest0-21swapper/223:37:352
2202626991917,1cyclictest3056-21PK-Backend21:33:171
2202626991917,1cyclictest3056-21PK-Backend21:18:171
2202626991917,1cyclictest0-21swapper/100:29:331
2202626991916,2cyclictest3056-21PK-Backend21:35:341
220262699191,4cyclictest2208487-21turbostat19:25:001
2202626991912,4cyclictest2262760-21turbostat.cron22:00:001
2202626991911,6cyclictest2305522-21cat00:00:181
2202626991911,6cyclictest2304616-21/usr/sbin/munin00:00:001
2202626991911,6cyclictest2280721-21df_inode22:50:141
2202626991911,6cyclictest2271426-21cron22:25:001
220262699190,1cyclictest0-21swapper/122:04:061
220262699190,1cyclictest0-21swapper/120:28:211
220262699190,1cyclictest0-21swapper/120:19:301
220262699190,1cyclictest0-21swapper/120:10:171
2202622991917,1cyclictest0-21swapper/022:47:460
2202622991917,1cyclictest0-21swapper/022:06:560
2202622991917,1cyclictest0-21swapper/000:04:380
2202622991914,3cyclictest2277426-21latency22:40:160
2202622991913,4cyclictest2280935-21missed_timers22:50:180
2202622991912,6cyclictest2232219-21grep20:30:170
2202622991912,5cyclictest2311306-21cron00:17:000
220262299190,1cyclictest0-21swapper/023:00:370
220262299190,1cyclictest0-21swapper/021:43:100
220262299190,1cyclictest0-21swapper/021:43:100
220262299190,1cyclictest0-21swapper/019:34:470
220262299190,18cyclictest0-21swapper/022:36:170
220262299190,17cyclictest2236923-21grep20:45:110
220263499189,2cyclictest42-21ksoftirqd/320:00:213
220263499182,11cyclictest2269959-21apt22:20:133
2202634991815,2cyclictest3056-21PK-Backend00:10:483
2202634991815,2cyclictest2275618-21cat22:35:163
2202634991811,5cyclictest886-21NetworkManager19:20:003
2202634991810,2cyclictest42-21ksoftirqd/321:50:153
2202634991810,2cyclictest42-21ksoftirqd/321:20:153
2202634991810,2cyclictest42-21ksoftirqd/300:35:223
220263499180,16cyclictest2237263-21/usr/sbin/munin20:45:153
2202629991816,1cyclictest0-21swapper/223:33:092
2202629991816,1cyclictest0-21swapper/222:42:472
2202629991816,1cyclictest0-21swapper/220:24:032
2202629991816,1cyclictest0-21swapper/219:28:472
2202629991816,1cyclictest0-21swapper/219:13:192
2202629991816,1cyclictest0-21swapper/200:23:112
2202629991810,6cyclictest2282083-21apt22:55:112
2202626991816,1cyclictest0-21swapper/120:20:511
2202626991813,3cyclictest1095-21runrttasks00:30:221
2202626991812,5cyclictest26-21ksoftirqd/100:20:201
2202626991812,5cyclictest2223441-21cat20:05:171
2202626991812,4cyclictest26-21ksoftirqd/119:10:211
2202626991812,4cyclictest2208898-21tr19:25:141
2202626991811,5cyclictest2295185-21ntpq23:30:191
220262699180,1cyclictest0-21swapper/119:53:271
220262299188,8cyclictest2234784-21unin-run20:40:010
2202622991816,1cyclictest0-21swapper/023:58:500
2202622991816,1cyclictest0-21swapper/019:22:180
2202622991812,5cyclictest2318547-21dpkg00:40:000
2202622991812,4cyclictest2214579-21grep19:40:160
2202634991710,2cyclictest42-21ksoftirqd/320:15:403
220263499170,16cyclictest3056-21PK-Backend19:58:433
220263499170,16cyclictest22974922sleep323:37:543
220263499170,15cyclictest2218008-21memory19:50:173
2202629991715,1cyclictest0-21swapper/222:52:062
2202629991711,5cyclictest2264474-21cron22:05:002
2202629991711,4cyclictest2156-21gnome-shell22:35:092
2202629991710,6cyclictest2249063-21expr21:20:122
2202629991710,5cyclictest2262019-21ntpq21:55:192
220262699178,7cyclictest2279288-21strings22:45:191
2202626991715,1cyclictest3056-21PK-Backend21:26:071
2202626991714,2cyclictest3056-21PK-Backend20:37:001
2202626991710,6cyclictest2265740-21sed22:05:211
220262299179,6cyclictest2314298-21ntp_states00:25:200
2202622991715,1cyclictest0-21swapper/021:36:280
2202622991715,1cyclictest0-21swapper/021:01:470
2202622991711,5cyclictest883-21dbus-daemon23:30:010
22584042160,5chrt3056-21PK-Backend21:45:201
22584042160,5chrt3056-21PK-Backend21:45:201
2202634991614,1cyclictest3056-21PK-Backend00:00:283
220262699166,8cyclictest2235195-21cpuspeed_turbos20:40:131
220262699161,5cyclictest141rcu_preempt20:55:201
2202626991614,1cyclictest3056-21PK-Backend19:40:231
2202626991614,1cyclictest0-21swapper/100:09:321
2202626991611,4cyclictest2276167-21expr22:35:221
220262699160,6cyclictest141rcu_preempt22:30:001
220262699160,14cyclictest3056-21PK-Backend21:22:561
220263499159,4cyclictest42-21ksoftirqd/300:00:003
2202634991512,2cyclictest3056-21PK-Backend19:33:413
2202634991512,2cyclictest3056-21PK-Backend00:06:503
2202634991511,3cyclictest42-21ksoftirqd/319:10:043
2202634991511,2cyclictest42-21ksoftirqd/323:40:303
2202634991511,2cyclictest42-21ksoftirqd/323:40:303
220262999152,3cyclictest34-21ksoftirqd/200:25:132
2202629991513,1cyclictest0-21swapper/200:38:462
2202626991513,1cyclictest3056-21PK-Backend21:12:081
2202626991510,3cyclictest2273859-21if_enp2s022:30:161
220262699150,14cyclictest3056-21PK-Backend23:15:381
220262699150,13cyclictest3056-21PK-Backend20:52:281
139099150,7rtkit-daemon2233281-21acpi20:35:132
220262699149,3cyclictest1095-21runrttasks21:02:091
220262699147,5cyclictest2212433-21cstates19:35:111
220262699141,2cyclictest141rcu_preempt19:45:181
220262699140,5cyclictest141rcu_preempt23:10:171
220262699140,5cyclictest141rcu_preempt21:50:221
2202634991310,2cyclictest3056-21PK-Backend23:25:123
220262699130,2cyclictest141rcu_preempt21:40:151
220262699130,2cyclictest141rcu_preempt21:40:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional