You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 23:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Dec 28, 2025 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22746092140119,10sleep10-21swapper/107:05:241
2275432212166,51sleep30-21swapper/307:09:263
227529828453,10sleep20-21swapper/207:07:412
227517927455,13sleep00-21swapper/007:06:110
25525352260,4sleep0171rcu_preempt11:30:160
25525352260,4sleep0171rcu_preempt11:30:150
2275588992623,1cyclictest0-21swapper/309:00:253
2275588992522,1cyclictest0-21swapper/308:35:233
2275588992421,1cyclictest0-21swapper/309:50:133
2275588992321,1cyclictest0-21swapper/307:15:003
2275588992320,1cyclictest0-21swapper/308:00:013
2275588992320,1cyclictest0-21swapper/307:19:593
24177102220,10sleep1331ktimers/109:20:181
227558899221,7cyclictest2456740-21taskset09:59:173
2275588992117,2cyclictest0-21swapper/311:05:013
227558899211,14cyclictest0-21swapper/310:10:233
25878892200,3sleep20-21swapper/212:02:292
2275588992017,1cyclictest0-21swapper/312:00:203
2275588992017,1cyclictest0-21swapper/311:55:123
2275588992016,2cyclictest0-21swapper/312:25:003
227558899201,14cyclictest0-21swapper/309:30:133
25660352191,12sleep02566124-21ntpq11:40:190
2275588991916,1cyclictest0-21swapper/309:45:163
2275588991916,1cyclictest0-21swapper/308:00:193
2275588991915,2cyclictest0-21swapper/308:50:263
227558899191,15cyclictest0-21swapper/311:30:133
227558899191,15cyclictest0-21swapper/311:30:123
227558899191,14cyclictest0-21swapper/309:25:113
2275588991910,4cyclictest22882722sleep307:20:123
23833292181,12sleep12383358-21uniq08:50:141
2275588991815,1cyclictest0-21swapper/312:30:193
2275588991815,1cyclictest0-21swapper/312:30:183
2275588991815,1cyclictest0-21swapper/310:35:163
2275588991815,1cyclictest0-21swapper/310:30:133
227558899181,3cyclictest0-21swapper/308:20:123
227558899181,13cyclictest0-21swapper/308:55:103
227558899181,12cyclictest2324802-21sleep07:55:003
227558899181,12cyclictest0-21swapper/311:25:123
227558899180,11cyclictest2535264-21taskset11:10:273
227558899180,11cyclictest2461863-21taskset10:02:323
2275588991715,1cyclictest0-21swapper/307:40:173
2275588991714,1cyclictest0-21swapper/312:20:003
2275588991714,1cyclictest0-21swapper/307:35:193
227558899171,13cyclictest0-21swapper/311:50:103
227558899171,13cyclictest0-21swapper/311:35:213
227558899171,13cyclictest0-21swapper/309:05:153
227558899171,12cyclictest2567143-21turbostat11:45:003
227558899171,12cyclictest0-21swapper/312:10:123
227558899171,11cyclictest2414328-21taskset09:20:003
227558899171,10cyclictest2609642-21taskset12:25:063
227558899170,14cyclictest0-21swapper/308:40:223
227558899170,10cyclictest2466943-21taskset10:05:343
25051352161,4sleep12505484-21tr10:45:161
23935432161,4sleep22393562-21apt-get09:00:132
227558899169,4cyclictest2419437-21sleep09:25:003
227558899162,9cyclictest2519044-21taskset10:55:203
227558899161,9cyclictest2341120-21taskset08:10:053
2275588991614,1cyclictest0-21swapper/310:50:013
227558899161,3cyclictest0-21swapper/312:35:153
2275588991613,1cyclictest0-21swapper/310:50:153
2275588991613,1cyclictest0-21swapper/309:35:163
2275588991612,2cyclictest0-21swapper/308:10:003
227558899161,12cyclictest0-21swapper/308:15:153
227558899161,10cyclictest2529533-21taskset11:05:213
227558899160,13cyclictest0-21swapper/310:25:133
227558899160,13cyclictest0-21swapper/308:25:243
227558899160,12cyclictest0-21swapper/310:15:133
227558899160,10cyclictest2593339-21taskset12:09:553
26031252150,11sleep22603308-21timerwakeupswit12:15:282
25365732151,9sleep22537639-21grep11:15:152
24254472151,9sleep22425517-21cat09:30:142
24239402151,5sleep02423980-21ntpq09:25:200
23839692150,1sleep00-21swapper/008:50:160
23713352151,10sleep10-21swapper/108:35:221
23565472151,4sleep12356571-21munin-run08:25:001
227558899152,9cyclictest2540011-21taskset11:15:193
227558899151,9cyclictest2403981-21taskset09:10:073
2275588991512,1cyclictest0-21swapper/308:45:133
2275588991512,1cyclictest0-21swapper/308:30:293
2275588991511,2cyclictest15850irq/127-ahci[0000:00:17.0]09:40:153
227558899151,12cyclictest0-21swapper/307:45:133
227558899151,11cyclictest0-21swapper/310:40:123
227558899151,11cyclictest0-21swapper/310:20:203
227558899151,10cyclictest2567487-21chrt11:45:103
227558899150,12cyclictest0-21swapper/307:30:193
227558899150,12cyclictest0-21swapper/307:25:193
24348422141,4sleep22434924-21grep09:35:242
23397082141,9sleep12339828-21netstat08:05:191
23306002141,9sleep00-21swapper/008:00:140
227558899141,10cyclictest2546248-21turbostat11:25:003
25923702131,6sleep10-21swapper/112:05:211
25612262131,8sleep02561485-21head11:35:250
24681102136,3sleep22468804-21tr10:10:182
24365962131,8sleep22437434-21grep09:40:162
24242682131,8sleep22424434-21wc09:25:262
23253222131,8sleep22325390-21fschecks_time07:55:152
23098462131,7sleep12309989-21kthreadcore07:40:141
25347322121,7sleep02534776-21/usr/sbin/munin11:10:210
25095982121,7sleep02509697-21latency_hist10:50:000
24471982121,7sleep02448694-21cut09:50:160
24413032121,7sleep02441536-21expr09:45:120
24191022122,7sleep0800-21snmpd09:21:060
24078062121,7sleep22407947-21ntpq09:10:192
24051612121,7sleep02406829-21kthreadcore09:10:160
24037252120,2sleep00-21swapper/009:10:010
23992542121,6sleep12399326-21irqstats09:05:161
23923662121,7sleep02392457-21ps08:55:220
23361472121,7sleep02336188-21fschecks_count08:05:140
23093592121,7sleep22309570-21cstates07:40:112
22937992121,7sleep22293525-21/usr/sbin/munin07:25:132
22798282121,5sleep12279875-21ntpq07:10:201
25948202111,6sleep02596492-21kthreadcore12:10:180
25732022110,7sleep22573894-21cat11:50:152
25034572111,7sleep02503571-21/usr/sbin/munin10:40:230
24666002111,4sleep22466671-21find10:05:262
24042712114,3sleep12404373-21cut09:10:131
23766102112,4sleep22376658-21ntpq08:40:212
23469082111,6sleep02347947-21kthreadcore08:15:160
23079262112,4sleep12307972-21ntpq07:35:221
22757792111,4sleep02275798-21apt-get07:10:110
26206482101,6sleep00-21swapper/012:35:180
25451422101,4sleep02545223-21ntpq11:20:190
25260852101,5sleep22527088-21grep11:05:172
25199952101,4sleep22520068-21munin-run11:00:002
25150652101,5sleep12515081-21awk10:55:101
24307792101,4sleep02431035-21df09:35:140
24131822101,4sleep22413228-21ntpq09:15:192
23568072101,3sleep02356826-21apt-get08:25:100
23189702101,4sleep02319174-21users07:45:260
23089292101,4sleep02308880-31find07:38:340
23041632101,4sleep22304286-21diskstats07:35:152
22993242102,4sleep02300086-21tr07:30:150
22942632100,6sleep12295897-21kthreadcore07:25:161
22869692101,4sleep02287015-21ntpq07:15:210
2624710290,1sleep10-21swapper/112:35:441
2613503291,3sleep12613594-21ntpq12:25:211
2604005291,3sleep22603832-21apt-get12:20:022
2603817292,4sleep12604001-21dpkg12:20:001
2599394291,4sleep02599979-21grep12:15:180
2597423291,4sleep22597564-21ntpq12:10:202
2588884291,4sleep02589218-21grep12:05:160
2567409291,4sleep22567486-21expr11:45:102
2560901291,4sleep22560949-21ntpq11:35:202
2510433291,4sleep12511824-21grep10:50:181
2492859291,3sleep02492913-21ntpq10:30:210
2492696291,4sleep22492809-21ntpq10:30:182
2484151291,4sleep22484377-21grep10:25:142
2478383291,2sleep20-21swapper/210:20:122
2477140291,4sleep02477185-21ntpq10:15:210
2462315291,3sleep02462334-21apt-get10:05:110
2453601291,4sleep02455359-21cut09:55:180
2450980291,4sleep22451196-21users09:50:262
2439906291,4sleep12440019-21ntpq09:40:191
2382496290,1sleep20-21swapper/208:46:172
2345103291,3sleep02345146-21ntpq08:10:200
2339829291,3sleep22339870-21ntpq08:05:202
2326201291,5sleep10-21swapper/107:55:171
2314594291,3sleep12314613-21apt-get07:45:121
2293524291,3sleep00-21swapper/007:25:110
2292614291,4sleep22292857-21grep07:20:272
2280558290,1sleep20-21swapper/207:11:532
2576488281,3sleep10-21swapper/111:50:181
2572027280,1sleep00-21swapper/011:45:270
2566174281,2sleep20-21swapper/211:40:202
2552010280,1sleep10-21swapper/111:30:141
2552010280,1sleep10-21swapper/111:30:131
2546521280,1sleep10-21swapper/111:25:121
2545224281,3sleep12545270-21ntpq11:20:201
2540982281,3sleep12541013-21munin-run11:19:591
2535743281,2sleep20-21swapper/211:15:002
2509859281,2sleep02509878-21apt-get10:50:130
2504340280,5sleep12504486-21latency_hist10:45:011
2499083281,3sleep20-21swapper/210:40:012
2494088280,3sleep02494107-21apt-get10:35:130
2467574281,3sleep12467730-21cstates10:10:131
2467552280,3sleep02467571-21apt-get10:10:130
2462891281,3sleep12462923-21kthreadcore10:05:171
2452324281,3sleep22452398-21irqstats09:55:162
2425245281,3sleep12425446-21cstates09:30:131
2425223280,3sleep00-21swapper/009:30:120
2373134281,3sleep12373656-21kthreadcore08:40:161
2371578281,3sleep02371593-21plymouth08:35:250
2361539280,1sleep10-21swapper/108:26:561
2352760281,3sleep02354414-21kthreadcore08:20:180
2351950280,1sleep20-21swapper/208:20:162
2335481281,3sleep12335482-21sshd08:03:281
2314616280,1sleep20-21swapper/207:45:132
2289577281,3sleep02291213-21kthreadcore07:20:170
2619769270,1sleep10-21swapper/112:34:111
2619769270,1sleep10-21swapper/112:34:101
2619634271,3sleep22619635-21sshd12:32:282
2619634271,3sleep22619635-21sshd12:32:272
2619511270,4sleep00-21swapper/012:30:540
2619511270,4sleep00-21swapper/012:30:530
2609051271,1sleep00-21swapper/012:20:440
2604378270,2sleep10-21swapper/112:20:131
2583716270,3sleep02585117-21kthreadcore12:00:190
2583129270,1sleep10-21swapper/112:00:041
2577881271,1sleep12577900-21apt-get11:55:101
2566802270,1sleep10-21swapper/111:40:481
2556921270,3sleep12556940-21apt-get11:35:111
2550428271,2sleep02550476-21ntpq11:25:200
2545546271,2sleep22545550-21mailstats11:20:252
2539381270,3sleep02539895-21sed11:15:170
2514613270,1sleep20-21swapper/210:52:322
2509530270,1sleep20-21swapper/210:49:102
2499962271,3sleep22501423-21cut10:40:172
2494992270,3sleep12496146-21kthreadcore10:35:181
2477827270,1sleep20-21swapper/210:16:362
2456562270,1sleep10-21swapper/109:56:591
2446004270,1sleep10-21swapper/109:46:291
2439148270,3sleep02439905-21missed_timers09:40:180
2421020271,2sleep10-21swapper/109:25:171
2413957270,1sleep00-21swapper/009:16:580
2392258270,2sleep22392307-21ntpq08:55:192
2388284270,2sleep10-21swapper/108:55:101
2381813270,4sleep10-21swapper/108:45:201
2377840270,2sleep00-21swapper/008:45:110
2350355270,4sleep10-21swapper/108:15:211
2325592271,2sleep00-21swapper/007:55:160
2314010270,1sleep00-21swapper/007:40:530
2303656270,1sleep20-21swapper/207:34:022
2300087271,2sleep10-21swapper/107:30:161
2612746261,2sleep00-21swapper/012:25:210
2581972261,2sleep00-21swapper/011:55:210
2577286260,3sleep00-21swapper/011:50:460
2567387261,1sleep10-21swapper/111:45:081
2546677260,3sleep20-21swapper/211:25:132
2532157260,2sleep12533860-21grep11:10:181
2530251260,4sleep10-21swapper/111:06:411
2524982260,1sleep20-21swapper/211:01:332
2521198261,2sleep00-21swapper/011:00:190
2519339261,2sleep00-21swapper/010:55:250
2493754260,1sleep10-21swapper/110:34:071
2488264260,1sleep00-21swapper/010:25:510
2446191260,1sleep20-21swapper/209:48:142
2432807261,2sleep10-21swapper/109:35:191
2419376260,1sleep20-21swapper/209:24:382
2414112260,1sleep10-21swapper/109:18:551
2397550260,1sleep00-21swapper/009:00:220
2377365260,1sleep00-21swapper/008:42:190
2367302261,3sleep2431ktimers/208:35:112
2366878260,4sleep20-21swapper/208:32:592
2366736260,1sleep10-21swapper/108:31:101
2357382261,2sleep20-21swapper/208:25:162
2351183260,1sleep20-21swapper/208:18:232
2335555260,4sleep20-21swapper/208:04:242
2321476261,2sleep10-21swapper/107:50:161
2320376261,2sleep00-21swapper/007:50:160
2287844260,4sleep20-21swapper/207:18:162
2281141261,2sleep10-21swapper/107:15:021
2622709250,1sleep20-21swapper/212:35:212
2593888250,2sleep10-21swapper/112:10:151
2592655250,1sleep20-21swapper/212:05:262
2578463250,1sleep20-21swapper/211:55:152
2528818250,1sleep00-21swapper/011:05:190
2483378250,1sleep10-21swapper/110:25:001
2481153250,1sleep00-21swapper/010:20:190
2476270250,2sleep1321rcuc/110:15:191
2461070250,2sleep00-21swapper/010:00:220
2457377251,1sleep20-21swapper/210:00:162
2450639250,1sleep10-21swapper/109:50:211
2399327250,2sleep20-21swapper/209:05:172
2398366250,1sleep10-21swapper/109:03:281
2387159250,1sleep20-21swapper/208:50:232
2362046250,1sleep00-21swapper/008:30:120
2345385250,2sleep10-21swapper/108:10:251
2319698250,1sleep20-21swapper/207:50:012
2293139250,1sleep10-21swapper/107:23:271
2609658240,1sleep20-21swapper/212:25:142
2552035240,1sleep20-21swapper/211:30:152
2552035240,1sleep20-21swapper/211:30:142
2520560240,1sleep10-21swapper/111:00:161
2478733240,1sleep10-21swapper/110:20:161
2457262240,1sleep10-21swapper/110:00:151
2341426240,1sleep20-21swapper/208:10:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional