You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 12:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Wed Nov 19, 2025 00:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26289782260,7sleep2421rcuc/223:20:002
27459812250,2sleep0221rcuc/000:30:190
2269641992521,1cyclictest0-21swapper/320:50:013
2269641992119,0cyclictest0-21swapper/323:15:273
2269641992017,1cyclictest0-21swapper/320:15:203
2269641991916,1cyclictest0-21swapper/300:15:143
2269641991916,1cyclictest0-21swapper/300:00:183
226964199191,13cyclictest2557091-21taskset22:35:013
2269641991816,1cyclictest0-21swapper/319:15:003
2269641991815,1cyclictest0-21swapper/322:40:013
2269641991815,1cyclictest0-21swapper/320:35:193
2269641991815,1cyclictest0-21swapper/319:50:263
2269641991815,1cyclictest0-21swapper/319:30:213
226964199181,13cyclictest2749989-21taskset00:35:153
226964199181,13cyclictest2743968-21chrt00:30:183
226964199181,13cyclictest2715965-21taskset00:11:433
226964199181,13cyclictest2701576-21taskset00:05:133
226964199181,13cyclictest2533438-21taskset22:20:143
226964199181,13cyclictest2376278-21taskset20:30:213
226964199178,6cyclictest25168492sleep322:09:253
226964199176,7cyclictest23633142sleep320:20:213
2269641991714,1cyclictest0-21swapper/323:55:263
2269641991714,1cyclictest0-21swapper/323:50:213
2269641991714,1cyclictest0-21swapper/323:50:003
2269641991714,1cyclictest0-21swapper/322:55:193
2269641991714,1cyclictest0-21swapper/321:50:233
2269641991714,1cyclictest0-21swapper/319:15:193
2269641991713,2cyclictest2661002-21turbostat23:40:003
2269641991713,1cyclictest0-21swapper/319:25:153
226964199171,13cyclictest2306272-21taskset19:35:193
226964199171,12cyclictest2741373-21turbostat00:30:003
226964199171,12cyclictest2451559-21chrt21:26:483
226964199171,12cyclictest2290182-21turbostat19:25:003
226964199170,14cyclictest2628935-21turbostat23:24:553
226964199170,13cyclictest2611738-21chrt23:07:113
226964199170,12cyclictest2661242-21taskset23:40:013
25474752161,4sleep22547483-21ssh22:26:232
2269641991613,1cyclictest0-21swapper/323:00:193
2269641991613,1cyclictest0-21swapper/322:28:053
226964199161,12cyclictest2613356-21chrt23:10:193
226964199161,12cyclictest2579408-21chrt22:46:273
226964199161,12cyclictest2531189-21taskset22:15:373
226964199161,12cyclictest2523631-21chrt22:11:363
226964199161,12cyclictest2508148-21taskset22:02:473
226964199161,12cyclictest2500899-21taskset21:59:343
226964199161,12cyclictest2483763-21chrt21:46:533
226964199161,12cyclictest2475598-21taskset21:41:233
226964199161,12cyclictest2459859-21chrt21:32:333
226964199161,12cyclictest2402347-21taskset20:55:063
226964199161,12cyclictest2401989-21taskset20:53:383
226964199161,12cyclictest2389128-21chrt20:40:473
226964199161,12cyclictest2352007-21taskset20:14:513
226964199161,12cyclictest2340713-21taskset20:05:183
226964199161,12cyclictest2339345-21chrt20:00:323
226964199161,12cyclictest2333347-21chrt19:57:523
226964199161,12cyclictest2314075-21chrt19:40:203
226964199161,11cyclictest2461283-21chrt21:35:133
226964199161,11cyclictest2315194-21taskset19:45:133
226964199161,11cyclictest2315194-21taskset19:45:133
226964199160,13cyclictest2435388-21chrt21:16:263
226964199160,12cyclictest2731814-21taskset00:21:253
226964199160,12cyclictest2650685-21taskset23:30:233
226964199160,12cyclictest2644969-21taskset23:29:553
226964199160,12cyclictest2414507-21taskset21:05:003
25014022151,6sleep12501409-21sh22:00:051
226964199151,11cyclictest2581601-21taskset22:50:153
226964199151,11cyclictest2565779-21taskset22:40:193
226964199151,11cyclictest2420560-21taskset21:05:283
226964199150,12cyclictest2437255-21chrt21:20:113
147598150,8rtkit-daemon2516870-21ssh22:09:260
24672942140,5sleep10-21swapper/121:36:091
23824652141,2sleep00-21swapper/020:35:240
226964199141,11cyclictest2427071-21taskset21:10:413
2269641991411,1cyclictest0-21swapper/319:10:003
226964199141,10cyclictest2364713-21taskset20:25:133
147598140,4rtkit-daemon2275940-21munin-node21:45:160
147598140,10rtkit-daemon214-21plymouthd22:41:040
23114402130,6sleep02313529-21kthreadcore19:40:180
25481472121,8sleep12548154-21sh22:27:541
147598120,8rtkit-daemon2603671-21ssh23:02:010
26427572111,7sleep22643009-21ssh23:25:262
26289012110,1sleep12628926-21/usr/sbin/munin23:20:001
26117212110,8sleep02611729-21ssh23:07:090
25677122111,3sleep10-21swapper/122:40:211
24936542111,7sleep02493777-21/usr/sbin/munin21:55:150
24851092110,1sleep22485134-21munin-run21:50:002
23903642112,2sleep0171rcu_preempt20:45:180
23695732111,2sleep10-21swapper/120:25:201
147598110,7rtkit-daemon730-21ntpd20:30:272
147598110,7rtkit-daemon2585784-21kthreadcore22:50:191
27423132100,7sleep12742930-21irqrtprio00:30:171
26771742100,4sleep12603964-21kworker/u16:4+nfsiod23:49:541
25754952100,7sleep12577462-21kthreadcore22:45:211
24207252100,7sleep12420726-21sshd21:07:351
24023672102,4sleep22402389-21apt-get20:55:142
24013382101,6sleep22401463-21cat20:50:252
23093492100,7sleep22310299-21irqrtprio19:40:162
147598100,7rtkit-daemon729-21lldpd21:14:442
147598100,7rtkit-daemon1540-21sendmail-mta23:22:440
147598100,6rtkit-daemon456-21avahi-daemon21:33:502
147598100,6rtkit-daemon2263830-21fschecks_count19:05:172
147598100,6rtkit-daemon22110-21packagekitd00:26:421
147598100,6rtkit-daemon214-21plymouthd22:13:000
147598100,6rtkit-daemon214-21plymouthd21:15:192
147598100,6rtkit-daemon1540-21sendmail-mta00:14:441
147598100,5rtkit-daemon2700061-21kworker/u16:3-rpciod00:04:361
147598100,5rtkit-daemon2614730-21irqstats23:10:260
147598100,4rtkit-daemon2668450-21ssh23:43:470
2735282295,1sleep00-21swapper/000:25:190
2733345290,6sleep12733373-21idleruntime-cro00:25:001
2699565291,5sleep22420928-21diskmemload00:01:132
2557601291,5sleep22364184-21kworker/u16:0+rpciod22:35:172
2541442291,4sleep02541481-21apt-get22:25:130
2491689290,7sleep22491696-21sh21:51:402
2485791291,6sleep02485975-21irqcore21:50:180
2469946291,5sleep12470703-21cut21:40:171
2364677290,4sleep22364710-21apt-get20:25:132
2287569290,6sleep22289125-21/usr/sbin/munin19:20:212
2263648291,4sleep02263726-21grep19:05:160
14759890,6rtkit-daemon515-21gmain21:07:002
14759890,5rtkit-daemon730-21ntpd23:49:550
14759890,5rtkit-daemon476-21wpa_supplicant21:28:512
14759890,5rtkit-daemon2275940-21munin-node23:57:361
14759890,5rtkit-daemon22110-21packagekitd19:51:122
14759890,5rtkit-daemon1540-21sendmail-mta20:42:402
14759890,3rtkit-daemon2643105-21ssh23:25:370
14759890,2rtkit-daemon2756809-21ssh00:38:102
14759890,2rtkit-daemon214-21plymouthd21:04:362
14759890,1rtkit-daemon0-21swapper/219:35:252
2716584280,1sleep20-21swapper/200:13:112
2679878281,4sleep12681925-21cut23:50:181
2653314281,4sleep12653355-21apt-get23:35:151
2620116280,1sleep20-21swapper/223:13:042
2610443281,4sleep22610489-21ntp_states23:05:222
2602527281,4sleep22602650-21python323:00:252
2555519280,5sleep02555527-21ssh22:31:340
2532370281,4sleep1720-21snmpd22:18:201
2493450280,4sleep22493493-21ssh21:55:132
2493381280,2sleep12493389-21ssh21:55:041
2492104280,1sleep10-21swapper/121:52:401
2453318280,2sleep10-21swapper/121:30:151
2442381281,4sleep12442440-21ntp_states21:20:221
2413432281,4sleep02413582-21ntp_kernel_pll_21:00:210
2407669281,4sleep12407913-21users20:55:251
2395676280,1sleep20-21swapper/220:47:052
2338694280,5sleep22338820-21sed20:00:242
2335748280,5sleep02337799-21kthreadcore20:00:220
2302813281,3sleep12302834-21apt-get19:35:141
2269968281,4sleep12270065-21sort19:10:141
14759880,4rtkit-daemon720-21snmpd21:22:132
14759880,4rtkit-daemon476-21wpa_supplicant19:26:362
14759880,4rtkit-daemon456-21avahi-daemon00:32:292
14759880,4rtkit-daemon2275940-21munin-node22:56:071
14759880,4rtkit-daemon2275940-21munin-node22:18:010
14759880,4rtkit-daemon214-21plymouthd19:18:342
14759880,4rtkit-daemon1-21systemd23:52:480
14759880,4rtkit-daemon1-21systemd22:22:322
14759880,2rtkit-daemon0-21swapper/220:23:142
14759880,2rtkit-daemon0-21swapper/023:33:460
14759880,1rtkit-daemon0-21swapper/219:33:442
14759880,1rtkit-daemon0-21swapper/100:15:241
2732928270,1sleep00-21swapper/000:23:590
2691999270,1sleep00-21swapper/023:57:300
2675870270,4sleep22675878-21ssh23:46:542
2652753270,5sleep22652759-21ssh23:34:282
2643165270,5sleep10-21swapper/123:25:451
2612671270,4sleep12612679-21ssh23:09:141
2579668270,1sleep00-21swapper/022:47:010
2565304270,1sleep20-21swapper/222:40:052
2549369271,3sleep12549404-21apt-get22:30:151
2519797270,4sleep22521813-21tr22:10:202
2510472273,2sleep22511557-21kthreadcore22:05:192
2477446271,3sleep1214-21plymouthd21:45:141
2477090271,4sleep02477094-21ssh21:44:530
2431024270,4sleep02432991-21tr21:15:200
2427283270,5sleep12427291-21ssh21:11:081
2423336270,5sleep02425335-21kthreadcore21:10:190
2383569270,3sleep02383601-21apt-get20:40:170
2346458271,3sleep00-21swapper/020:10:200
2326521271,3sleep02326638-21python319:50:240
2320229271,3sleep02320300-21ntp_states19:45:230
2320229271,3sleep02320300-21ntp_states19:45:230
2278056271,3sleep12278089-21apt-get19:15:151
2276144270,4sleep02277889-21df_inode19:15:020
2269819270,1sleep00-21swapper/019:10:130
2269280270,1sleep10-21swapper/119:07:291
14759870,4rtkit-daemon214-21plymouthd20:35:572
14759870,4rtkit-daemon1-21systemd23:00:041
14759870,3rtkit-daemon2708049-21ssh00:07:211
14759870,3rtkit-daemon2555622-21ssh22:31:462
14759870,3rtkit-daemon2532826-21ssh22:19:242
14759870,3rtkit-daemon2469484-21ssh21:40:032
14759870,3rtkit-daemon2231651-21munin-node19:14:152
14759870,3rtkit-daemon214-21plymouthd22:00:110
14759870,3rtkit-daemon1-21systemd23:16:160
14759870,3rtkit-daemon1-21systemd21:35:292
14759870,3rtkit-daemon1-21systemd20:16:402
14759870,3rtkit-daemon1-21systemd19:56:482
14759870,2rtkit-daemon214-21plymouthd23:35:350
14759870,2rtkit-daemon214-21plymouthd20:06:072
14759870,1rtkit-daemon0-21swapper/022:35:330
2756233260,1sleep10-21swapper/100:36:501
2732717260,3sleep22732725-21ssh00:23:332
2717699260,1sleep00-21swapper/000:15:090
2709805260,3sleep02709957-21df00:10:160
2708124260,1sleep00-21swapper/000:07:320
2701543261,2sleep20-21swapper/200:05:132
2693495260,3sleep02693656-21diskstats00:00:140
2691634260,1sleep20-21swapper/223:56:362
2683733260,1sleep2214-21plymouthd23:51:392
2668272260,1sleep20-21swapper/223:43:252
2667937260,1sleep10-21swapper/123:42:361
2660557260,1sleep20-21swapper/223:39:092
2651478261,3sleep12651484-21ssh23:31:301
2619807260,1sleep10-21swapper/123:12:191
2595827260,1sleep20-21swapper/222:57:252
2588849260,3sleep22588857-21ssh22:54:482
2558784260,3sleep12560578-21grep22:35:201
2540266260,4sleep00-21swapper/022:23:080
2537549260,4sleep12538430-21sed22:20:211
2517460260,3sleep12517611-21df22:10:151
2508209260,3sleep22508217-21ssh22:02:552
2444392260,1sleep00-21swapper/021:23:450
2420873260,1sleep00-21swapper/021:09:310
2414216260,1sleep10-21swapper/121:01:501
2389120260,1sleep10-21swapper/120:40:421
2377070260,1sleep00-21swapper/020:34:200
2376785260,1sleep10-21swapper/120:30:421
2364194260,1sleep10-21swapper/120:22:111
2358878260,3sleep02359809-21irqstats20:20:190
2352320260,3sleep12352475-21diskstats20:15:181
2352279260,1sleep00-21swapper/020:15:130
2333483260,1sleep00-21swapper/019:59:380
2314639260,1sleep10-21swapper/119:41:161
2302305260,1sleep10-21swapper/119:32:021
2296640261,2sleep00-21swapper/019:30:170
2296132260,1sleep10-21swapper/119:26:531
14759860,3rtkit-daemon1-21systemd20:10:112
14759860,2rtkit-daemon214-21plymouthd19:45:052
14759860,2rtkit-daemon214-21plymouthd19:45:052
2756447250,3sleep00-21swapper/000:37:170
2723472251,2sleep2431ktimers/200:15:352
2635428250,3sleep20-21swapper/223:21:252
2595392250,3sleep00-21swapper/022:56:240
2587561251,1sleep00-21swapper/022:51:500
2578593250,3sleep20-21swapper/222:45:262
2515400250,2sleep12515408-21ssh22:06:061
2459937251,2sleep00-21swapper/021:32:430
2451989250,3sleep10-21swapper/121:27:431
2451373250,3sleep0214-21plymouthd21:26:180
2429487251,2sleep12414264-21kworker/u16:3+xprtiod21:15:171
2402095250,3sleep12402183-21latency_hist20:55:011
2345522250,3sleep0214-21plymouthd20:05:590
2327009250,1sleep10-21swapper/119:50:571
2302800250,2sleep02302812-21/usr/sbin/munin19:35:130
2296376250,3sleep02296577-21cut19:30:010
2285639250,2sleep10-21swapper/119:20:201
2739091240,2sleep22739344-21unixbench_singl00:25:252
2468895240,1sleep00-21swapper/021:39:560
2396973240,2sleep00-21swapper/020:50:190
2391302240,2sleep10-21swapper/120:45:191
2369856240,2sleep00-21swapper/020:25:230
2345835240,2sleep10-21swapper/120:10:011
2284285240,2sleep00-21swapper/019:20:150
2629200230,1sleep10-21swapper/123:20:041
2402344230,1sleep00-21swapper/020:55:040
2377378230,1sleep10-21swapper/120:35:091
2346081230,1sleep10-21swapper/120:10:051
2337801230,1sleep10-21swapper/120:00:231
2327573230,1sleep10-21swapper/119:55:021
2315182230,1sleep10-21swapper/119:45:051
2315182230,1sleep10-21swapper/119:45:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional