You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 15:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sat Dec 06, 2025 12:39:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2126831995338,11cyclictest2220650-21ssh10:33:481
2126830995343,7cyclictest1315733-21cups-browsed11:34:230
2126830995339,11cyclictest2306962-21true12:30:010
2126830995335,3cyclictest171rcu_preempt09:24:180
2126831994738,5cyclictest2209575-21sshd10:17:411
2126831994736,7cyclictest422-21dbus-daemon11:10:011
212683199463,34cyclictest2214408-21ssh10:24:191
2126830994537,5cyclictest2147832-21sendmail08:20:010
2126830994534,8cyclictest2249877-21sshd11:14:410
2126831994436,5cyclictest2194447-21latency_hist09:55:031
2126830994437,4cyclictest2138836-21systemd-run07:50:010
2126830994436,4cyclictest2135808-21sendmail07:40:010
2126830994427,14cyclictest12550irq/33-eth012:14:210
2126830994422,13cyclictest12450irq/32-eth009:14:230
2126832994335,5cyclictest2158318-21systemd-run08:55:022
2126832994333,6cyclictest0-21swapper/212:00:012
2126831994335,5cyclictest2306961-21systemd-run12:30:011
2126831994335,5cyclictest1315733-21cups-browsed11:19:191
2126831994334,6cyclictest2242559-21unixbench-2d11:04:361
2126830994334,6cyclictest2275145-21systemd-run11:50:010
2126830994334,5cyclictest2298850-21sendmail12:20:010
2126830994334,5cyclictest2298850-21sendmail12:20:010
2126830994332,9cyclictest0-21swapper/012:34:370
2126832994235,4cyclictest2239363-21idleruntime-cro11:00:002
2126832994235,4cyclictest0-21swapper/210:33:472
2126832994233,6cyclictest2245465-21missed_timers11:09:182
2126832994231,6cyclictest421-21avahi-daemon09:49:212
2126831994233,6cyclictest2301319-21aten2.4-expect12:24:091
2126830994232,7cyclictest2240681-21sshd11:02:400
2126830994133,5cyclictest422-21dbus-daemon09:30:010
212683099413,32cyclictest2186224-21ssh09:44:210
2126830994130,8cyclictest2235909-21kworker/u8:2+xprtiod11:40:020
2126830994127,10cyclictest0-21swapper/008:40:010
2126830994114,23cyclictest2193444-21ssh09:54:240
212683099411,32cyclictest2169310-21ssh09:19:210
2126832994032,5cyclictest2260609-21ls11:30:012
2126832994029,8cyclictest698-21cron11:15:012
2126831994033,4cyclictest2158287-21cron08:55:011
2126831994032,5cyclictest2141843-21systemd-run08:00:021
2126831994031,7cyclictest2135808-21sendmail07:40:011
2126831994031,6cyclictest2267243-21unixbench-2d11:39:341
2126831994028,8cyclictest421-21avahi-daemon11:54:251
2126830994032,5cyclictest422-21dbus-daemon11:30:010
2126830994031,6cyclictest422-21dbus-daemon11:00:010
2126832993934,2cyclictest0-21swapper/208:00:022
2126832993933,3cyclictest0-21swapper/210:00:012
212683299392,27cyclictest2200653-21ssh10:04:232
2126831993931,5cyclictest2290390-21grep12:09:341
2126831993931,5cyclictest2260627-21systemd-run11:30:011
2126831993931,5cyclictest2147802-21cron08:20:011
212683099393,27cyclictest2245597-21ssh11:09:220
2126830993925,10cyclictest0-21swapper/009:10:010
2126832993831,5cyclictest2232517-21latency_hist10:50:022
212683299382,30cyclictest2293799-21ssh12:14:182
2126831993830,5cyclictest2189698-21missed_timers09:49:191
2126831993827,8cyclictest2196728-21latency09:59:161
2126831993826,8cyclictest2186579-21switchtime09:44:321
2126830993831,4cyclictest12450irq/32-eth012:24:210
2126830993829,5cyclictest2253640-21awk11:20:010
2126832993730,4cyclictest957-21mta-sts-daemon09:42:012
2126831993732,2cyclictest0-21swapper/107:20:011
2126831993729,5cyclictest2309573-21cpuspeed12:34:131
2126831993729,5cyclictest2161347-21latency_hist09:05:021
2126831993729,5cyclictest2131867-21fschecks_time07:29:131
2126831993728,6cyclictest422-21dbus-daemon07:15:011
2126831993727,6cyclictest422-21dbus-daemon11:57:241
2126832993628,5cyclictest266-21systemd-journal08:40:012
2126832993628,5cyclictest2298848-21idleruntime-cro12:20:012
2126832993628,5cyclictest2298848-21idleruntime-cro12:20:012
2126832993628,5cyclictest2257170-21systemd-run11:25:012
2126832993628,5cyclictest2241407-21aten_r2power_fr11:04:122
2126832993626,6cyclictest2146235-21sed08:14:392
2126832993625,8cyclictest2160643-21needreboot09:04:202
2126831993628,5cyclictest2201465-21ssh10:05:011
2126831993623,9cyclictest1315733-21cups-browsed11:34:231
2126830993628,6cyclictest1315733-21cups-browsed10:50:390
2126830993628,6cyclictest1315733-21cups-browsed10:50:390
2126830993625,7cyclictest0-21swapper/010:10:020
212683099362,28cyclictest2189804-21ssh09:49:220
2126830993617,15cyclictest2132852-21latency_hist07:30:020
2126830993614,16cyclictest12450irq/32-eth010:24:240
2126832993526,6cyclictest1315733-21cups-browsed12:24:322
2126832993526,5cyclictest2140245-21unixbench_singl07:54:322
2126832993524,8cyclictest2278203-21proc_pri11:54:302
212683199354,25cyclictest2234882-21ssh10:54:221
212683199354,25cyclictest2234882-21ssh10:54:221
2126831993530,2cyclictest0-21swapper/111:00:011
2126831993527,5cyclictest491-21gdbus09:00:011
2126831993527,5cyclictest2275122-21munin-run11:50:011
2126831993526,6cyclictest1-21systemd07:35:011
2126831993524,8cyclictest2179437-21mailstats09:34:341
212683099354,23cyclictest657-21snmpd07:59:270
2126830993527,6cyclictest2227390-21kworker/u8:7+xprtiod10:49:180
2126830993527,5cyclictest2222168-21unixbench_singl10:34:340
2126830993527,5cyclictest2162806-21diskmemload11:55:010
2126830993525,7cyclictest1315733-21cups-browsed11:56:510
2126830993522,9cyclictest2211417-21unixbench_singl10:19:330
2126832993429,2cyclictest0-21swapper/212:35:012
2126832993427,4cyclictest0-21swapper/210:45:002
2126832993423,8cyclictest2235783-21/usr/sbin/munin10:55:012
2126832993423,8cyclictest2235783-21/usr/sbin/munin10:55:012
212683199346,21cyclictest437-21mta-sts-daemon12:14:181
2126831993426,5cyclictest2143431-21latency_hist08:05:031
2126831993426,4cyclictest0-21swapper/110:27:401
2126831993425,6cyclictest1315733-21cups-browsed12:00:531
2126831993425,6cyclictest1315733-21cups-browsed11:25:011
2126831993423,6cyclictest2231502-21sh10:49:191
212683099345,26cyclictest2218029-21ssh10:29:260
2126830993427,5cyclictest2224068-21aten_r2power_en10:39:110
2126830993427,5cyclictest2224068-21aten_r2power_en10:39:110
2126830993426,5cyclictest2131317-21cron07:25:010
2126830993426,4cyclictest2178488-21aten_r2power_cu09:34:140
2126830993425,6cyclictest2158183-21unixbench_multi08:54:460
2126830993425,6cyclictest1315733-21cups-browsed11:22:500
2126830993425,6cyclictest1315733-21cups-browsed10:10:380
2126830993425,6cyclictest0-21swapper/009:40:010
212683099342,27cyclictest2270450-21ssh11:44:220
2126830993419,11cyclictest0-21swapper/008:59:320
2126830993417,9cyclictest12450irq/32-eth012:04:220
2126832993326,4cyclictest2135803-21cron07:40:012
2126832993325,6cyclictest1-21systemd09:40:012
2126832993325,5cyclictest422-21dbus-daemon12:28:122
2126832993325,5cyclictest2315022-21sendmail12:40:012
2126832993325,5cyclictest2176187-21grep09:29:282
2126832993325,5cyclictest2156807-21idleruntime-cro08:50:012
2126832993324,3cyclictest0-21swapper/210:19:232
2126832993321,8cyclictest2173045-21unixbench_multi09:24:322
212683299331,24cyclictest2266716-21ssh11:39:232
212683199332,28cyclictest2176121-21ssh09:29:261
212683099332,26cyclictest2196909-21ssh09:59:190
2126830993319,10cyclictest0-21swapper/010:01:410
2126832993224,5cyclictest2143228-21unixbench_multi08:04:322
2126832993223,6cyclictest266-21systemd-journal10:15:012
2126832993221,8cyclictest2165613-21latency09:14:172
2126832993221,8cyclictest2162140-21needreboot09:09:192
2126832993218,7cyclictest956-21mta-sts-daemon09:18:162
212683199323,24cyclictest2249268-21ssh11:14:251
2126831993225,4cyclictest2161436-21kworker/u8:15+rpciod09:10:011
2126831993224,5cyclictest2204993-21latency_hist10:10:021
2126831993224,5cyclictest2165374-21ssh09:14:131
2126831993224,5cyclictest0-21swapper/112:20:011
2126831993224,5cyclictest0-21swapper/112:20:011
2126831993223,6cyclictest2152320-21systemd-run08:35:011
2126832993123,5cyclictest2218479-21unixbench-2d10:29:342
2126832993121,7cyclictest2178940-21kernelversion09:34:222
2126832993119,7cyclictest1315733-21cups-browsed07:24:212
2126831993125,3cyclictest0-21swapper/110:14:201
2126831993123,5cyclictest2173323-21cron09:25:011
212683099314,18cyclictest2314275-21ssh12:39:400
2126830993126,3cyclictest959-21mta-sts-daemon07:34:270
2126830993125,3cyclictest2127480-21kernelversion07:14:180
2126830993123,4cyclictest0-21swapper/008:14:100
212683299308,13cyclictest684-21munin-node11:49:262
2126832993022,5cyclictest2204964-21idleruntime-cro10:10:012
2126832993022,4cyclictest2227390-21kworker/u8:7-xprtiod11:35:022
2126832993021,6cyclictest2149343-21systemd-run08:25:012
2126832993019,8cyclictest698-21cron09:00:012
2126832993019,6cyclictest957-21mta-sts-daemon09:50:202
2126831993025,2cyclictest0-21swapper/109:20:011
2126831993024,4cyclictest2149773-21cpu08:29:121
2126831993024,4cyclictest2149773-21cpu08:29:121
2126831993024,3cyclictest0-21swapper/110:39:141
2126831993024,3cyclictest0-21swapper/110:39:141
2126831993023,4cyclictest2315006-21latency_hist12:40:011
2126831993022,5cyclictest959-21mta-sts-daemon07:44:241
2126831993022,5cyclictest1315733-21cups-browsed08:24:241
2126831993021,7cyclictest2270326-21sh11:44:181
2126830993023,4cyclictest2289238-21aten_r2power_vo12:09:120
2126830993021,6cyclictest0-21swapper/007:20:010
2126830993019,8cyclictest2140298-21cron07:55:000
2126832992923,3cyclictest0-21swapper/208:29:212
2126832992923,3cyclictest0-21swapper/208:29:212
2126832992922,3cyclictest0-21swapper/207:35:012
2126832992918,8cyclictest2162806-21diskmemload12:09:222
2126832992913,12cyclictest351ktimers/210:39:252
2126832992913,12cyclictest351ktimers/210:39:252
2126831992921,4cyclictest0-21swapper/109:39:161
2126831992911,5cyclictest59950lldpd10:43:181
2126830992920,6cyclictest2142366-21fschecks_time08:04:150
2126830992920,4cyclictest0-21swapper/010:40:110
2126830992919,4cyclictest2137345-21munin-run07:45:010
212683099291,4cyclictest171rcu_preempt08:24:250
2126832992821,4cyclictest0-21swapper/210:24:132
2126832992820,5cyclictest2147839-21sendmail-msp08:20:012
2126832992820,5cyclictest1315733-21cups-browsed07:16:302
2126832992817,8cyclictest2285543-21latency12:04:182
2126831992822,4cyclictest2155554-21/usr/sbin/munin08:49:341
2126831992822,3cyclictest959-21mta-sts-daemon07:54:251
2126831992821,4cyclictest2155259-21unixbench_singl08:44:321
212683099289,16cyclictest14-21ksoftirqd/008:49:230
2126830992818,3cyclictest0-21swapper/007:10:010
2126830992814,9cyclictest266-21systemd-journal09:05:010
2126832992722,2cyclictest0-21swapper/207:15:012
2126832992720,4cyclictest2139023-21latency_hist07:50:022
2126832992719,3cyclictest0-21swapper/211:44:102
2126831992719,6cyclictest422-21dbus-daemon08:40:021
212683099271,11cyclictest2151370-21grep08:34:150
2126832992618,4cyclictest0-21swapper/211:19:222
2126830992621,2cyclictest0-21swapper/008:29:230
2126830992621,2cyclictest0-21swapper/008:29:230
2126830992614,8cyclictest422-21dbus-daemon08:45:020
2126832992519,3cyclictest0-21swapper/208:45:012
2126832992518,3cyclictest351ktimers/208:09:182
2126832992517,5cyclictest1-21systemd07:45:012
2126831992520,2cyclictest0-21swapper/107:25:011
2126832992316,4cyclictest2131710-21aten_r2power_po07:29:102
2126832992315,5cyclictest2152297-21cron08:35:002
2126831992317,3cyclictest0-21swapper/108:14:131
2126831992311,9cyclictest2137895-21awk07:49:131
2126830992314,6cyclictest0-21swapper/008:09:310
2126831992217,2cyclictest0-21swapper/108:10:011
212683399202,10cyclictest0-21swapper/311:04:263
2126832991913,3cyclictest2067185-21kworker/u8:0-nfsiod07:10:012
212683399181,9cyclictest0-21swapper/309:39:283
212683399181,8cyclictest0-21swapper/312:09:223
2126831991810,5cyclictest2126971-21sort07:10:021
212683399173,9cyclictest0-21swapper/309:29:253
212683399173,7cyclictest0-21swapper/310:14:313
212683399173,5cyclictest0-21swapper/309:09:263
212683399172,8cyclictest0-21swapper/311:39:243
212683399171,8cyclictest0-21swapper/310:59:273
212683399164,8cyclictest0-21swapper/310:24:193
212683399162,8cyclictest0-21swapper/310:54:253
212683399162,8cyclictest0-21swapper/310:54:253
212683399162,7cyclictest0-21swapper/311:44:223
212683399161,8cyclictest0-21swapper/311:19:223
212683399152,7cyclictest0-21swapper/309:54:233
212683399152,6cyclictest0-21swapper/309:34:293
212683399151,8cyclictest0-21swapper/310:44:273
212683399151,8cyclictest0-21swapper/310:09:283
212683399143,7cyclictest0-21swapper/311:24:233
212683399143,7cyclictest0-21swapper/310:39:243
212683399143,7cyclictest0-21swapper/310:39:243
212683399143,7cyclictest0-21swapper/309:44:223
212683399142,7cyclictest0-21swapper/311:59:133
212683399141,7cyclictest0-21swapper/310:19:223
212683399141,7cyclictest0-21swapper/308:44:193
212683399141,6cyclictest0-21swapper/308:24:223
212683399133,6cyclictest0-21swapper/311:54:253
212683399133,6cyclictest0-21swapper/310:29:333
212683399132,8cyclictest0-21swapper/310:33:463
212683399132,7cyclictest0-21swapper/311:34:233
212683399132,7cyclictest0-21swapper/311:29:383
212683399132,6cyclictest0-21swapper/312:39:423
212683399132,5cyclictest0-21swapper/312:14:183
212683399131,8cyclictest0-21swapper/312:19:203
212683399131,8cyclictest0-21swapper/312:19:203
212683399131,7cyclictest0-21swapper/312:29:223
212683399131,7cyclictest0-21swapper/311:12:403
212683399131,7cyclictest0-21swapper/307:44:233
212683399131,6cyclictest0-21swapper/308:49:213
212683399131,6cyclictest0-21swapper/308:29:193
212683399131,6cyclictest0-21swapper/308:29:193
212683399122,6cyclictest0-21swapper/312:34:273
212683399122,5cyclictest0-21swapper/311:49:233
212683399121,7cyclictest0-21swapper/311:09:243
212683399121,7cyclictest0-21swapper/309:59:193
212683399121,7cyclictest0-21swapper/309:49:213
212683399121,7cyclictest0-21swapper/309:19:213
212683399121,6cyclictest0-21swapper/308:54:403
212683399113,5cyclictest0-21swapper/307:54:243
212683399112,5cyclictest0-21swapper/307:14:263
212683399111,7cyclictest0-21swapper/309:14:103
212683399111,7cyclictest0-21swapper/308:39:223
212683399111,7cyclictest0-21swapper/307:59:213
212683399111,7cyclictest0-21swapper/307:19:223
212683399111,6cyclictest0-21swapper/312:24:153
212683399111,6cyclictest0-21swapper/310:04:223
212683399111,6cyclictest0-21swapper/309:24:193
212683399111,6cyclictest0-21swapper/308:09:263
212683399111,5cyclictest0-21swapper/307:34:153
212683399102,5cyclictest0-21swapper/312:04:133
212683399102,5cyclictest0-21swapper/310:49:133
212683399102,5cyclictest0-21swapper/308:20:013
212683399101,6cyclictest0-21swapper/308:59:213
212683399101,5cyclictest0-21swapper/308:34:203
212683399101,5cyclictest0-21swapper/307:24:193
212683399100,3cyclictest0-21swapper/309:00:033
21268339991,5cyclictest0-21swapper/308:14:183
21268339991,5cyclictest0-21swapper/308:04:113
21268339991,5cyclictest0-21swapper/307:50:023
21268339991,5cyclictest0-21swapper/307:29:193
21268339991,4cyclictest0-21swapper/307:39:213
21268339982,3cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional