You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 13:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Fri Dec 19, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2296538213156,70sleep30-21swapper/319:08:363
229632628759,20sleep00-21swapper/019:05:550
229660028554,10sleep20-21swapper/219:09:232
229633227964,9sleep10-21swapper/119:06:011
2296780992624,1cyclictest0-21swapper/320:40:253
2296780992623,1cyclictest0-21swapper/322:35:163
2296780992623,1cyclictest0-21swapper/319:20:153
2296780992616,7cyclictest2604006-21turbostat23:59:593
2296780992616,7cyclictest2604006-21turbostat00:00:003
2296780992523,1cyclictest0-21swapper/300:00:203
2296780992421,1cyclictest0-21swapper/321:30:203
2296780992421,1cyclictest0-21swapper/321:10:003
2296780992320,1cyclictest0-21swapper/323:15:153
2296780992320,1cyclictest0-21swapper/320:35:123
2296780992320,1cyclictest0-21swapper/300:35:013
2296780992319,2cyclictest0-21swapper/320:15:133
2296780992219,1cyclictest0-21swapper/322:15:203
2296780992219,1cyclictest0-21swapper/321:55:113
2296780992219,1cyclictest0-21swapper/320:20:163
2296780992219,1cyclictest0-21swapper/320:10:173
229678099219,8cyclictest2472021-21chrt21:50:273
229678099219,8cyclictest2472021-21chrt21:50:263
229678099218,9cyclictest2414633-21taskset20:57:393
2296780992119,1cyclictest0-21swapper/323:45:133
2296780992118,1cyclictest0-21swapper/322:50:133
2296780992118,1cyclictest0-21swapper/321:10:233
2296780992118,1cyclictest0-21swapper/320:54:173
2296780992118,1cyclictest0-21swapper/320:25:183
2296780992118,1cyclictest0-21swapper/300:25:193
229678099208,1cyclictest0-21swapper/319:40:023
229678099208,1cyclictest0-21swapper/319:40:023
2296780992018,1cyclictest0-21swapper/323:30:133
2296780992018,1cyclictest0-21swapper/320:30:473
2296780992018,1cyclictest0-21swapper/320:05:203
2296780992018,1cyclictest0-21swapper/319:55:153
2296780992017,2cyclictest0-21swapper/319:45:203
2296780992017,1cyclictest0-21swapper/323:25:013
2296780992017,1cyclictest0-21swapper/323:00:143
2296780992017,1cyclictest0-21swapper/322:25:383
2296780992017,1cyclictest0-21swapper/321:20:003
2296780992017,1cyclictest0-21swapper/300:20:003
2296780992017,1cyclictest0-21swapper/300:05:253
2296780991916,1cyclictest0-21swapper/323:28:433
2296780991916,1cyclictest0-21swapper/322:57:123
2296780991916,1cyclictest0-21swapper/322:08:573
2296780991916,1cyclictest0-21swapper/320:04:393
2296780991916,1cyclictest0-21swapper/300:14:373
229678099191,13cyclictest0-21swapper/322:10:123
23822662181,12sleep02382316-21ntpq20:25:190
229678099188,7cyclictest0-21swapper/322:30:123
2296780991816,1cyclictest0-21swapper/323:50:183
2296780991816,1cyclictest0-21swapper/322:45:493
2296780991816,1cyclictest0-21swapper/321:45:123
2296780991816,1cyclictest0-21swapper/321:44:593
2296780991815,2cyclictest0-21swapper/319:30:173
2296780991815,1cyclictest0-21swapper/322:01:423
2296780991815,1cyclictest0-21swapper/321:36:123
2296780991815,1cyclictest0-21swapper/320:45:183
2296780991815,1cyclictest0-21swapper/319:51:233
2296780991815,1cyclictest0-21swapper/300:40:003
2296780991814,2cyclictest0-21swapper/322:22:493
2296780991814,2cyclictest0-21swapper/319:10:193
229678099181,14cyclictest2319586-21taskset19:28:153
229678099181,12cyclictest0-21swapper/323:40:123
2296780991715,1cyclictest0-21swapper/323:35:223
2296780991715,1cyclictest0-21swapper/319:35:063
2296780991715,1cyclictest0-21swapper/300:20:233
229678099171,4cyclictest0-21swapper/321:20:123
2296780991714,1cyclictest0-21swapper/323:12:213
2296780991713,2cyclictest0-21swapper/321:00:123
229678099170,15cyclictest0-21swapper/321:25:193
229678099170,10cyclictest2546617-21taskset23:05:003
25136932161,10sleep22513741-21ntpq22:30:222
23789122161,6sleep12380365-21kthreadcore20:25:161
229678099162,9cyclictest2520213-21taskset22:40:123
229678099161,9cyclictest2304168-21taskset19:15:083
25295032151,10sleep12529622-21find22:45:241
24661912151,9sleep22466234-21ntpq21:45:222
25938112141,2sleep00-21swapper/023:50:110
23609822141,9sleep02361027-21/usr/sbin/munin20:05:190
23301942141,9sleep22330432-21cat19:40:002
26311442131,7sleep12632511-21kthreadcore00:25:161
25945762131,7sleep12596233-21cut23:50:161
24674452131,8sleep02467559-21cpuspeed_turbos21:50:130
24674452131,8sleep02467559-21cpuspeed_turbos21:50:120
24405242131,6sleep22440623-21users21:20:282
23998672130,8sleep02400669-21grep20:45:150
23122682130,2sleep10-21swapper/119:20:141
23085062131,8sleep02308774-21sed19:15:250
26423392121,7sleep02644089-21cut00:35:170
25358732122,5sleep02535892-21apt-get22:55:120
25149322120,7sleep02514995-21grep22:35:130
25050122121,3sleep12505486-21kthreadcore22:25:171
24307552122,5sleep22430774-21apt-get21:15:122
23937632121,4sleep20-21swapper/220:40:002
23366182121,7sleep12338259-21kthreadcore19:45:171
25096772111,5sleep10-21swapper/122:30:131
24836592111,6sleep12483754-21kthreadcore22:05:171
24045642111,4sleep12404586-21apt-get20:50:141
23990432111,5sleep12399092-21munin-run20:45:011
23717842111,5sleep02371826-21ntpq20:15:210
23557252111,4sleep12355782-21ntpq20:00:191
23514652111,5sleep22351492-21munin-run20:00:012
23255362111,5sleep12325745-21hddtemp_smartct19:35:141
23246812110,1sleep10-21swapper/119:31:141
23155302111,6sleep02316784-21kthreadcore19:25:150
26346022101,4sleep22634659-21ntp_states00:25:202
25786112104,3sleep134-21ksoftirqd/123:35:181
25777782101,5sleep02577835-21grep23:35:000
25731142101,5sleep22573290-21head23:30:142
25719782101,4sleep02572195-21users23:25:270
25466312101,4sleep12546650-21apt-get23:05:111
25306352101,4sleep02530654-21apt-get22:50:120
25199082100,1sleep20-21swapper/222:40:002
24871582101,4sleep22487205-21ntpq22:05:222
24625402102,4sleep02462726-21sort21:45:150
24587322101,5sleep02460519-21tr21:40:190
24490842101,4sleep22450349-21memory21:30:182
24465552101,4sleep12446811-21df21:30:131
23980992101,4sleep02398153-21ntpq20:40:230
23945512101,4sleep22394622-21irqstats20:40:172
23399542101,4sleep02339987-21ntpq19:45:190
23199852101,4sleep22320003-21apt-get19:30:112
2640091291,3sleep02640194-21timerandwakeup00:30:270
2636380292,3sleep12637731-21kthreadcore00:30:161
2630308291,4sleep22630477-21latency_hist00:25:012
2613872291,4sleep22613994-21sed00:05:282
2609567291,3sleep02609586-21apt-get00:05:140
2599019291,3sleep22599038-21apt-get23:55:112
2599019291,3sleep22599038-21apt-get23:55:102
2589077291,3sleep02589148-21irqstats23:45:150
2583035291,3sleep22583107-21munin-run23:39:592
2572897291,4sleep12573113-21diskstats23:30:121
2566389291,4sleep12566403-21ntpq23:20:231
2566333291,4sleep02566382-21ntpq23:20:210
2561127291,3sleep02561182-21ntpq23:15:210
2555840291,4sleep12555881-21ntpq23:10:211
2550629291,4sleep22550680-21ntpq23:05:212
2546928291,3sleep02547138-21hddtemp_smartct23:05:140
2545498291,4sleep12545621-21find23:00:241
2505488291,4sleep22507194-21kthreadcore22:25:182
2493920290,4sleep12493427-21apt-get22:15:031
2477630291,4sleep22477671-21awk22:00:012
2467564290,1sleep10-21swapper/121:50:141
2467564290,1sleep10-21swapper/121:50:131
2462211291,2sleep10-21swapper/121:45:121
2461950291,3sleep12461999-21munin-run21:45:001
2452056291,4sleep22452180-21perl21:35:162
2429819291,4sleep02430078-21unixbench_singl21:10:280
2424608291,4sleep02424859-21unixbench_singl21:05:280
2418954291,4sleep12419044-21ldconfig21:00:191
2410115291,3sleep22410325-21hddtemp_smartct20:55:152
2362585291,3sleep12362841-21df20:10:141
2357603291,4sleep12358846-21cut20:05:161
2357184291,4sleep22357320-21diskstats20:05:142
2345239291,3sleep22345280-21ntpq19:50:222
2334773291,3sleep22334814-21ntpq19:40:212
2334773291,3sleep22334814-21ntpq19:40:202
2304171291,3sleep12304190-21apt-get19:15:101
2297520291,4sleep02298787-21cut19:10:170
2624392280,5sleep244-21ksoftirqd/200:15:262
2620650281,3sleep02621939-21kthreadcore00:15:170
2614842281,3sleep22614923-21cpuspeed_turbos00:10:142
2603127281,4sleep12603265-21mailstats23:55:221
2603127281,4sleep12603265-21mailstats23:55:211
2597825281,3sleep22597878-21ntp_states23:50:212
2589149281,3sleep12590461-21grep23:45:161
2583599281,3sleep02583638-21fschecks_count23:40:130
2568186281,3sleep22569677-21grep23:25:162
2552188281,3sleep02552400-21smartctl23:10:160
2541891281,3sleep20-21swapper/223:00:162
2541352281,3sleep02540832-21apt-get23:00:020
2536873281,3sleep12538563-21cut22:55:181
2530657281,3sleep12530873-21cstates22:50:141
2525723281,3sleep02525935-21smartctl22:45:150
2524947280,1sleep10-21swapper/122:42:161
2524476280,4sleep02524533-21cat22:40:270
2520191280,2sleep20-21swapper/222:40:122
2514996281,3sleep12515202-21diskstats22:35:141
2499206281,2sleep20-21swapper/222:20:122
2498700281,3sleep02498701-21sshd22:17:140
2488671280,1sleep20-21swapper/222:10:142
2477913280,3sleep10-21swapper/122:00:111
2471513281,3sleep22471593-21wc21:50:222
2471513281,3sleep22471593-21wc21:50:222
2446533281,2sleep00-21swapper/021:30:120
2446271281,3sleep02446473-21latency_hist21:30:010
2425546281,3sleep12425693-21cstates21:10:131
2424864280,4sleep10-21swapper/121:05:291
2420310281,2sleep20-21swapper/221:05:132
2415662281,3sleep22416500-21kthreadcore21:00:172
2415381280,5sleep02415487-21/usr/sbin/munin21:00:140
2410426281,4sleep02411918-21tr20:55:180
2377765280,1sleep00-21swapper/020:21:270
2341809281,3sleep12341842-21kthreadcore19:50:171
2335742281,3sleep12335789-21systemd-run19:45:011
2335742281,3sleep12335789-21systemd-run19:45:001
2323031281,3sleep02319984-21/usr/sbin/munin19:30:170
2645130271,3sleep12645192-21pmu-power00:35:221
2630568270,2sleep00-21swapper/000:25:110
2619627270,4sleep00-21swapper/000:12:360
2605090271,3sleep10-21swapper/100:00:191
2590462271,2sleep20-21swapper/223:45:172
2583319271,2sleep20-21swapper/223:40:112
2572255270,1sleep10-21swapper/123:26:081
2552504271,2sleep20-21swapper/223:10:182
2540747270,1sleep20-21swapper/222:59:032
2530627270,1sleep20-21swapper/222:50:062
2530272270,1sleep20-21swapper/222:48:412
2514302270,1sleep00-21swapper/022:30:370
2503905270,1sleep00-21swapper/022:21:140
2493950270,2sleep20-21swapper/222:15:122
2493373270,4sleep10-21swapper/122:14:291
2488395271,2sleep00-21swapper/022:10:120
2487917270,1sleep00-21swapper/022:07:160
2456557270,1sleep10-21swapper/121:37:391
2441251270,2sleep10-21swapper/121:25:131
2435548270,1sleep00-21swapper/021:17:380
2404589271,2sleep20-21swapper/220:50:152
2399587271,3sleep20-21swapper/220:45:132
2388398270,4sleep10-21swapper/120:33:141
2384143271,2sleep20-21swapper/220:30:192
2383566270,2sleep00-21swapper/020:30:130
2377930270,1sleep10-21swapper/120:23:321
2371882270,3sleep22372001-21gzip20:15:232
2356150270,1sleep00-21swapper/020:00:270
2351225270,1sleep10-21swapper/119:56:591
2351154270,1sleep00-21swapper/019:56:050
2319465270,1sleep10-21swapper/119:26:441
2304756271,2sleep22306233-21ps19:15:162
2301778270,1sleep20-21swapper/219:12:502
2298788271,3sleep12300589-21kthreadcore19:10:171
2640704260,1sleep20-21swapper/200:34:072
2628834260,2sleep12629260-21strings00:20:181
2627032261,2sleep00-21swapper/000:20:170
2614820260,2sleep10-21swapper/100:10:131
2610157261,2sleep10-21swapper/100:05:191
2604840261,2sleep00-21swapper/000:00:180
2566642261,2sleep20-21swapper/223:20:292
2561752260,1sleep20-21swapper/223:15:432
2560417260,2sleep10-21swapper/123:15:171
2509303260,1sleep00-21swapper/022:28:350
2482661260,1sleep00-21swapper/022:02:100
2479079261,2sleep20-21swapper/222:00:162
2477305260,1sleep10-21swapper/121:55:551
2461613260,4sleep20-21swapper/221:40:442
2452338261,2sleep00-21swapper/021:35:170
2441827260,2sleep20-21swapper/221:25:182
2437878261,2sleep00-21swapper/021:20:180
2414626260,4sleep10-21swapper/120:57:341
2408873261,2sleep02408944-21find20:50:280
2393470260,4sleep00-21swapper/020:36:170
2382991260,1sleep20-21swapper/220:26:052
2377767260,1sleep20-21swapper/220:21:292
2362563261,1sleep00-21swapper/020:10:130
2362555260,4sleep20-21swapper/220:10:062
2352333261,2sleep20-21swapper/220:00:162
2341234261,2sleep00-21swapper/019:50:120
2340924260,1sleep20-21swapper/219:49:232
2318839260,3sleep20-21swapper/219:25:212
2621942250,2sleep10-21swapper/100:15:171
2609122250,1sleep20-21swapper/200:02:452
2583313250,3sleep10-21swapper/123:40:061
2504081250,3sleep10-21swapper/122:23:271
2436016250,3sleep10-21swapper/121:20:111
2429495250,1sleep20-21swapper/221:10:222
2400671250,2sleep10-21swapper/120:45:161
2392824250,2sleep10-21swapper/120:35:221
2371828251,1sleep10-21swapper/120:15:221
2329505250,1sleep00-21swapper/019:35:250
2313877250,1sleep20-21swapper/219:20:242
2309466250,1sleep00-21swapper/019:20:110
2641425240,1sleep20-21swapper/200:35:142
2598798240,1sleep00-21swapper/023:55:010
2598798240,1sleep00-21swapper/023:55:000
2578060240,1sleep00-21swapper/023:35:130
2472438240,1sleep00-21swapper/021:55:020
2431059240,1sleep10-21swapper/121:15:151
2330742240,1sleep00-21swapper/019:40:030
2330742240,1sleep00-21swapper/019:40:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional