You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 22:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Mon Dec 22, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22903119911640,1159cyclictest2363867-21sleep08:05:003
22903119911141,1109cyclictest26410412sleep311:39:553
2290311992541,250cyclictest2531345-21turbostat10:19:553
23748462960,2sleep2421rcuc/208:10:192
2290311992017,1cyclictest0-21swapper/309:50:003
229031199181,7cyclictest0-21swapper/311:40:133
23516992171,5sleep02352279-21grep07:55:160
229031199177,7cyclictest2428259-21turbostat08:55:003
229031199177,7cyclictest2351058-21turbostat07:55:013
2290311991714,1cyclictest0-21swapper/310:45:003
2290311991613,1cyclictest0-21swapper/311:00:163
2290311991613,1cyclictest0-21swapper/309:05:143
2290311991613,1cyclictest0-21swapper/308:11:133
2290311991613,1cyclictest0-21swapper/307:30:263
229031199161,12cyclictest2312202-21taskset07:25:003
229031199161,11cyclictest2467751-21taskset09:25:173
229031199155,7cyclictest2305454-21turbostat07:20:013
2290311991513,1cyclictest0-21swapper/311:15:013
2290311991512,1cyclictest0-21swapper/311:56:093
2290311991512,1cyclictest0-21swapper/309:30:273
2290311991512,1cyclictest0-21swapper/307:38:273
229031199151,11cyclictest2551178-21taskset10:30:123
229031199151,11cyclictest2296564-21chrt07:11:343
25958382141,4sleep12595917-21latency_hist11:05:011
22961532140,11sleep12296392-21sed07:10:251
229031199147,5cyclictest2673371-21turbostat12:04:593
229031199144,7cyclictest2692881-21taskset12:20:313
229031199144,7cyclictest2686541-21taskset12:16:463
229031199144,7cyclictest2492512-21turbostat09:45:013
229031199143,9cyclictest2363857-21turbostat08:09:563
2290311991412,1cyclictest0-21swapper/310:20:183
2290311991412,1cyclictest0-21swapper/309:35:103
2290311991412,1cyclictest0-21swapper/309:21:543
2290311991412,1cyclictest0-21swapper/309:03:573
2290311991412,1cyclictest0-21swapper/308:31:473
2290311991411,1cyclictest0-21swapper/311:17:383
2290311991411,1cyclictest0-21swapper/310:54:173
2290311991411,1cyclictest0-21swapper/308:35:193
229031199141,10cyclictest2634359-21chrt11:31:463
229031199141,10cyclictest2531060-21taskset10:11:283
229031199141,10cyclictest2453768-21taskset09:12:233
229031199141,10cyclictest2409295-21taskset08:40:133
229031199140,11cyclictest2505418-21turbostat09:55:013
229031199140,11cyclictest2421815-21taskset08:49:513
26160442130,2sleep10-21swapper/111:20:151
229031199137,4cyclictest2563768-21turbostat10:40:013
229031199135,6cyclictest2622227-21taskset11:25:143
229031199131,9cyclictest2673644-21chrt12:05:413
229031199131,9cyclictest2654290-21taskset11:50:143
229031199131,9cyclictest2653501-21taskset11:45:323
229031199131,9cyclictest2588738-21taskset10:55:243
229031199131,9cyclictest2544736-21taskset10:25:033
229031199131,9cyclictest2454681-21taskset09:15:163
229031199131,9cyclictest2351281-21taskset07:55:013
229031199131,9cyclictest2337847-21taskset07:40:433
229031199131,8cyclictest2378282-21taskset08:15:173
2290311991312,0cyclictest0-21swapper/309:55:223
2290311991311,1cyclictest0-21swapper/312:38:233
2290311991311,1cyclictest0-21swapper/307:28:023
2290311991311,0cyclictest0-21swapper/310:47:223
229031199130,10cyclictest2703544-21taskset12:30:213
229031199130,10cyclictest2703544-21taskset12:30:213
229031199130,10cyclictest2680074-21chrt12:10:493
229031199130,10cyclictest2621598-21chrt11:22:403
229031199130,10cyclictest2596081-21chrt11:05:013
229031199130,10cyclictest2518928-21taskset10:05:163
229031199130,10cyclictest2434650-21chrt08:59:563
301798120,7rtkit-daemon2654297-21sh11:50:131
301798120,7rtkit-daemon2383977-21hddtemp_smartct08:20:161
301798120,5rtkit-daemon3205-21sendmail-mta09:05:011
27015372120,4sleep22703543-21cut12:30:212
27015372120,4sleep22703543-21cut12:30:202
26998822120,1sleep02699915-21apt-get12:30:150
26998822120,1sleep02699915-21apt-get12:30:140
23513312121,8sleep12351458-21tr07:55:131
229031199121,8cyclictest2693486-21taskset12:25:113
229031199121,8cyclictest25124162sleep310:00:153
229031199120,9cyclictest2395826-21taskset08:25:343
229031199120,9cyclictest2383760-21taskset08:20:143
229031199120,9cyclictest2344266-21chrt07:45:373
229031199120,9cyclictest2344266-21chrt07:45:363
301798110,3rtkit-daemon667-21lldpd10:18:581
25757132111,3sleep22575761-21proc_pri10:45:232
24668402111,7sleep1297-21systemd-journal09:24:591
24397872111,7sleep12440188-21grep09:00:201
229031199118,1cyclictest0-21swapper/307:10:013
301798100,7rtkit-daemon636-21snmpd07:15:341
301798100,6rtkit-daemon636-21snmpd11:07:161
301798100,5rtkit-daemon448-21wpa_supplicant10:35:231
301798100,5rtkit-daemon2283984-21apt-get07:05:131
26676002101,6sleep02668181-21sed12:00:170
25708282101,7sleep02571206-21sed10:45:160
30179890,5rtkit-daemon3205-21sendmail-mta09:16:261
30179890,5rtkit-daemon3205-21sendmail-mta08:39:521
30179890,5rtkit-daemon2297054-21munin-node07:41:151
30179890,4rtkit-daemon636-21snmpd11:45:131
30179890,4rtkit-daemon2392268-21cut08:25:161
30179890,4rtkit-daemon2338471-21apt-get07:45:141
30179890,4rtkit-daemon2338471-21apt-get07:45:131
30179890,1rtkit-daemon0-21swapper/112:30:011
30179890,1rtkit-daemon0-21swapper/112:30:001
30179890,1rtkit-daemon0-21swapper/109:35:061
2632165291,6sleep22628429-21/usr/sbin/munin11:30:182
2553090290,7sleep00-21swapper/010:30:170
2551342291,4sleep12551541-21hddtemp_smartct10:30:141
2460727291,5sleep02460883-21diskstats09:20:140
2409727290,5sleep22410714-21irqstats08:40:172
2350387290,6sleep12350626-21grep07:50:231
2343701291,4sleep02343764-21ntp_states07:45:230
2343701291,4sleep02343764-21ntp_states07:45:230
2325454291,4sleep22325487-21apt-get07:35:102
2305726291,4sleep02305748-21apt-get07:20:140
2299446290,1sleep02299576-21gltestperf07:15:160
30179880,5rtkit-daemon2297054-21munin-node09:11:071
30179880,4rtkit-daemon667-21lldpd12:20:321
30179880,4rtkit-daemon667-21lldpd11:41:011
30179880,4rtkit-daemon667-21lldpd09:25:571
30179880,4rtkit-daemon638-21ntpd08:58:071
30179880,4rtkit-daemon636-21snmpd12:39:361
30179880,4rtkit-daemon636-21snmpd11:36:131
30179880,4rtkit-daemon636-21snmpd10:40:351
30179880,4rtkit-daemon636-21snmpd10:10:021
30179880,4rtkit-daemon636-21snmpd09:48:211
30179880,4rtkit-daemon636-21snmpd08:45:011
30179880,4rtkit-daemon636-21snmpd07:29:081
30179880,4rtkit-daemon492-21gmain08:00:121
30179880,4rtkit-daemon432-21NetworkManager10:04:221
30179880,4rtkit-daemon3200908-21packagekitd08:32:281
30179880,4rtkit-daemon2297054-21munin-node12:26:371
30179880,4rtkit-daemon2297054-21munin-node12:06:001
30179880,4rtkit-daemon2297054-21munin-node11:31:271
30179880,4rtkit-daemon2297054-21munin-node10:54:311
30179880,4rtkit-daemon2297054-21munin-node09:59:211
30179880,4rtkit-daemon2297054-21munin-node07:32:361
30179880,4rtkit-daemon0-21swapper/108:41:011
30179880,1rtkit-daemon0-21swapper/111:19:161
30179880,1rtkit-daemon0-21swapper/110:26:061
30179880,1rtkit-daemon0-21swapper/110:20:141
30179880,1rtkit-daemon0-21swapper/108:15:361
30179880,1rtkit-daemon0-21swapper/107:24:101
2674440280,6sleep02674639-21hddtemp_smartct12:10:150
2674240281,3sleep12674273-21apt-get12:10:121
2662381281,4sleep22664422-21cut11:55:192
2622046280,2sleep10-21swapper/111:25:121
2602534280,2sleep10-21swapper/111:10:131
2596296281,4sleep02596495-21hddtemp_smartct11:05:160
2395488281,4sleep22395730-21grep08:25:222
2337264281,4sleep02337335-21ntp_states07:40:210
30179870,4rtkit-daemon2297054-21munin-node12:00:401
30179870,4rtkit-daemon2297054-21munin-node11:59:301
30179870,3rtkit-daemon638-21ntpd09:40:241
30179870,3rtkit-daemon636-21snmpd07:35:101
30179870,3rtkit-daemon2524324-21vmstat10:05:271
30179870,3rtkit-daemon2297054-21munin-node10:56:071
30179870,3rtkit-daemon2297054-21munin-node08:10:551
30179870,2rtkit-daemon2570463-21apt-get10:45:121
30179870,1rtkit-daemon0-21swapper/109:31:341
2707761271,4sleep02709436-21kthreadcore12:35:190
2681061271,3sleep12681655-21irqrtprio12:15:161
2652949271,4sleep02653012-21ntp_states11:45:230
2647887270,4sleep22648040-21gltestperf11:45:162
2558033271,3sleep22559014-21kernelversion10:35:172
2557299271,3sleep22557471-21ls10:35:002
2544787271,3sleep22544930-21df10:25:152
2531389270,4sleep22531349-21apt-get10:15:012
2526656270,4sleep22528392-21tr10:10:162
2519555271,4sleep02520017-21kthreadcore10:05:170
2499193271,3sleep12499225-21apt-get09:50:121
2473063270,2sleep0151ktimers/009:27:310
2447002271,3sleep02447166-21vmstat09:05:240
2435122271,3sleep02435237-21perl09:00:140
2422087271,3sleep12422120-21apt-get08:50:151
2404400270,4sleep22406416-21grep08:35:212
2396466270,4sleep02396565-21cut08:30:130
2369309270,4sleep02369386-21ntp_states08:05:210
2344870271,3sleep20-21swapper/207:50:122
2290623271,3sleep20-21swapper/207:10:142
2290048271,3sleep02290058-21cron07:10:010
30179860,3rtkit-daemon3205-21sendmail-mta08:07:161
2687483261,2sleep20-21swapper/212:20:152
2686794260,1sleep00-21swapper/012:20:000
2673631260,1sleep00-21swapper/012:05:300
2669235260,4sleep22671283-21kthreadcore12:00:192
2642783261,3sleep02644419-21kthreadcore11:40:170
2634242260,1sleep00-21swapper/011:30:250
2623526260,3sleep22625419-21cut11:25:182
2610198261,2sleep00-21swapper/011:15:180
2608609260,1sleep00-21swapper/011:14:010
2589739260,3sleep20-21swapper/211:00:142
2582798260,1sleep00-21swapper/010:52:340
2577218260,4sleep22577307-21irqcore10:50:152
2544373260,1sleep00-21swapper/010:23:280
2532250260,4sleep02532748-21irqcore10:15:150
2525198260,3sleep02525200-21/usr/sbin/munin10:10:110
2524681260,3sleep22524844-21dpkg10:10:012
2517346260,3sleep02517423-21ntp_states10:00:200
2507049260,4sleep22507145-21kthreadcore09:55:192
2505060260,1sleep00-21swapper/009:50:430
2441549262,2sleep22441580-21fschecks_count09:05:132
2440648260,1sleep20-21swapper/209:00:252
2421828261,2sleep00-21swapper/008:50:010
2383343260,3sleep22383060-21apt-get08:20:022
2371606261,3sleep02372739-21cut08:10:190
2359563260,3sleep22361610-21cut08:00:182
2353322261,3sleep22355404-21grep07:55:182
2331322260,1sleep00-21swapper/007:36:050
2312828261,2sleep20-21swapper/207:25:162
2308859260,3sleep22310796-21latency07:20:202
2285409261,2sleep20-21swapper/207:05:182
2699292250,1sleep00-21swapper/012:25:500
2686700250,1sleep00-21swapper/012:18:480
2686650250,1sleep20-21swapper/212:18:102
2675221251,2sleep22676196-21kthreadcore12:10:162
2666043250,2sleep02666285-21users11:55:250
2659916250,1sleep00-21swapper/011:50:510
2655325251,2sleep20-21swapper/211:50:172
2641045250,3sleep22641069-21munin-run11:39:592
2640852250,1sleep00-21swapper/011:37:290
2616521250,3sleep22617356-21grep11:20:162
2603686251,2sleep20-21swapper/211:10:182
2597077251,2sleep20-21swapper/211:05:172
2585930250,2sleep20-21swapper/210:55:202
2570034250,1sleep20-21swapper/210:43:142
2569834250,1sleep00-21swapper/010:40:390
2512301250,2sleep20-21swapper/210:00:142
2505094250,3sleep20-21swapper/209:51:092
2492755250,2sleep20-21swapper/209:45:122
2479380250,1sleep20-21swapper/209:31:342
2475260250,2sleep00-21swapper/009:30:200
2473110250,3sleep22473111-21sshd09:28:102
2461084250,2sleep20-21swapper/209:20:162
2453710250,1sleep20-21swapper/209:11:382
2434540250,1sleep20-21swapper/208:58:322
2415134250,1sleep00-21swapper/008:41:230
2403196250,3sleep02403484-21irqcore08:35:190
2402374250,1sleep20-21swapper/208:32:522
2396078250,1sleep00-21swapper/008:28:500
2379418250,3sleep02381539-21kthreadcore08:15:190
2377331251,2sleep20-21swapper/208:15:172
2337415251,2sleep20-21swapper/207:40:232
2318172250,1sleep00-21swapper/007:25:300
2296628250,1sleep00-21swapper/007:12:240
2699433240,3sleep20-21swapper/212:27:352
2647127241,2sleep2638-21ntpd11:41:042
2583242240,3sleep00-21swapper/010:55:100
2560809240,2sleep00-21swapper/010:35:190
2544979240,2sleep00-21swapper/010:25:180
2538724240,2sleep22539308-21ps10:20:192
2505905240,1sleep00-21swapper/009:55:170
2487177240,2sleep00-21swapper/009:40:170
2480891240,2sleep22481988-21kthreadcore09:35:142
2480104240,2sleep02480304-21smartctl09:35:120
2460410240,2sleep22460474-21systemd-run09:20:012
2447793240,1sleep00-21swapper/009:10:120
2429069240,2sleep02429990-21kthreadcore08:55:170
2428051240,3sleep00-21swapper/008:52:340
2423085240,2sleep22424235-21kthreadcore08:50:192
2417846240,2sleep22419930-21kthreadcore08:45:192
2383806240,2sleep00-21swapper/008:20:160
2369464240,2sleep20-21swapper/208:05:242
2357731240,1sleep00-21swapper/008:00:120
2324127240,1sleep00-21swapper/007:30:230
2318782240,1sleep20-21swapper/207:30:142
2305175240,3sleep20-21swapper/207:16:432
2706600230,1sleep20-21swapper/212:35:172
2673978230,1sleep20-21swapper/212:09:572
2627946230,1sleep00-21swapper/011:27:070
2621498230,1sleep00-21swapper/011:21:220
2615186230,1sleep20-21swapper/211:17:412
2589931230,1sleep00-21swapper/011:00:150
2492945230,1sleep00-21swapper/009:45:150
2486537230,1sleep20-21swapper/209:40:162
2454264230,1sleep00-21swapper/009:15:140
2348035230,1sleep00-21swapper/007:50:180
2338630230,1sleep20-21swapper/207:45:172
2338630230,1sleep20-21swapper/207:45:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional