You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 18:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Thu Dec 04, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2271650993835,1cyclictest0-21swapper/312:30:003
2271650993125,3cyclictest2292563-21turbostat07:25:003
2271650992320,1cyclictest0-21swapper/311:45:003
2271650992219,1cyclictest0-21swapper/311:50:153
227165099219,9cyclictest2286297-21sleep07:20:003
2271650992118,1cyclictest0-21swapper/310:20:203
2271650992017,1cyclictest0-21swapper/309:45:193
2271650991916,1cyclictest0-21swapper/311:20:003
2271650991916,1cyclictest0-21swapper/308:35:013
2271650991915,1cyclictest0-21swapper/309:35:003
227165099191,14cyclictest2575224-21taskset10:40:173
24879662180,7sleep134-21ksoftirqd/109:46:291
2271650991815,1cyclictest0-21swapper/312:15:183
2271650991815,1cyclictest0-21swapper/310:00:003
2271650991814,1cyclictest0-21swapper/309:41:243
227165099181,13cyclictest2756067-21taskset12:34:103
227165099181,13cyclictest2432748-21taskset09:14:493
227165099177,7cyclictest25684422sleep310:35:383
2271650991714,1cyclictest0-21swapper/310:29:383
2271650991714,1cyclictest0-21swapper/308:55:193
2271650991713,1cyclictest0-21swapper/311:00:263
2271650991713,1cyclictest0-21swapper/307:50:133
227165099171,13cyclictest2586394-21chrt10:49:513
227165099171,13cyclictest2471404-21taskset09:35:293
227165099171,12cyclictest2738545-21taskset12:21:083
227165099171,12cyclictest2719242-21taskset12:10:183
227165099171,12cyclictest2292844-21taskset07:25:123
227165099167,6cyclictest24963682sleep309:52:133
227165099166,7cyclictest25606742sleep310:31:173
227165099166,7cyclictest25606742sleep310:31:163
2271650991613,1cyclictest0-21swapper/311:07:553
2271650991613,1cyclictest0-21swapper/310:10:223
2271650991613,1cyclictest0-21swapper/308:28:553
227165099161,12cyclictest2761189-21taskset12:35:193
227165099161,12cyclictest2708439-21taskset12:05:143
227165099161,12cyclictest2706207-21taskset12:00:493
227165099161,12cyclictest2667400-21turbostat11:39:593
227165099161,12cyclictest2650164-21taskset11:27:253
227165099161,12cyclictest2624906-21taskset11:10:393
227165099161,12cyclictest2520895-21taskset10:07:273
227165099161,12cyclictest2512741-21chrt10:02:503
227165099161,12cyclictest2453095-21taskset09:25:193
227165099161,12cyclictest2448122-21taskset09:23:033
227165099161,12cyclictest2438931-21taskset09:15:263
227165099161,12cyclictest2418192-21chrt09:00:353
227165099161,12cyclictest2405640-21chrt08:53:013
227165099161,12cyclictest2393564-21chrt08:45:113
227165099161,12cyclictest2392935-21chrt08:40:433
227165099161,12cyclictest2367974-21taskset08:22:313
227165099161,12cyclictest2360928-21taskset08:15:223
227165099161,12cyclictest2341937-21taskset08:00:193
227165099161,12cyclictest2323849-21chrt07:47:333
227165099161,12cyclictest2305291-21taskset07:35:033
227165099161,12cyclictest2277579-21chrt07:11:193
227165099161,11cyclictest2602544-21turbostat11:00:013
227165099161,11cyclictest2591900-21taskset10:50:183
227165099161,11cyclictest2424797-21taskset09:09:563
227165099160,13cyclictest2691830-21chrt11:55:043
227165099160,13cyclictest2641347-21chrt11:21:253
23750872151,2sleep10-21swapper/108:30:161
2271650991512,1cyclictest0-21swapper/307:40:193
227165099151,11cyclictest2681910-21chrt11:46:143
227165099151,11cyclictest2386623-21taskset08:35:383
227165099151,11cyclictest2351962-21taskset08:10:173
227165099151,11cyclictest2343030-21taskset08:05:053
227165099151,11cyclictest2336169-21chrt07:55:313
227165099150,12cyclictest2651522-21taskset11:30:073
227165099150,12cyclictest2536328-21chrt10:15:533
227165099150,12cyclictest2305021-21taskset07:34:433
27459822141,10sleep22746186-21grep12:25:252
22659742140,1sleep12266280-21irqcore07:05:161
26585402130,10sleep22658548-21ssh11:33:162
27545052121,8sleep22754513-21ssh12:30:382
26024942121,8sleep0720-21snmpd10:59:500
24464262120,9sleep02446491-21ntp_states09:20:200
147598120,8rtkit-daemon2764494-21latency_hist12:40:001
147598120,8rtkit-daemon2325283-21sed07:50:150
147598120,7rtkit-daemon1540-21sendmail-mta08:16:120
26762962113,5sleep22677225-21kernelversion11:45:182
26429242110,7sleep12643049-21apt-key11:25:011
25622772111,6sleep22562323-21unin-run10:35:012
25622772111,6sleep22562323-21unin-run10:35:002
24557822114,4sleep12399519-21kworker/u16:3+nfsiod09:26:391
23418772111,7sleep22341936-21ntp_states08:00:192
147598110,7rtkit-daemon2665857-21ssh11:36:230
147598110,7rtkit-daemon1540-21sendmail-mta08:23:330
147598110,6rtkit-daemon2398695-21ntp_states08:45:210
147598110,3rtkit-daemon2730573-21ssh12:16:192
27055812100,7sleep02705665-21/usr/sbin/munin12:00:230
25787422101,5sleep02578779-21apt-get10:45:110
25362262100,1sleep22536231-21sh10:15:412
24331352102,4sleep12433172-21apt-get09:15:131
24323842100,2sleep10-21swapper/109:14:001
23923762101,2sleep20-21swapper/208:40:232
22729312100,7sleep02274062-21grep07:10:160
147598100,6rtkit-daemon456-21avahi-daemon08:55:150
147598100,6rtkit-daemon1540-21sendmail-mta09:16:330
2674214290,1sleep00-21swapper/011:42:060
2650405290,1sleep20-21swapper/211:27:582
2633102290,6sleep12633110-21ssh11:16:021
2561360290,3sleep02561368-21ssh10:32:500
2561360290,3sleep02561368-21ssh10:32:500
2543982290,6sleep22544236-21sed10:20:262
2536553291,6sleep0214-21plymouthd10:16:220
2520656290,6sleep12520662-21ssh10:06:581
2515905290,6sleep22517873-21kthreadcore10:05:192
2498393291,5sleep22499299-21sed09:55:172
2470906290,1sleep20-21swapper/209:35:252
2457615291,4sleep02457647-21apt-get09:30:140
2426132290,6sleep02426618-21kthreadcore09:10:180
2380934291,4sleep02380967-21apt-get08:35:110
2323861290,1sleep00-21swapper/007:47:430
14759890,6rtkit-daemon730-21ntpd10:52:540
14759890,6rtkit-daemon1540-21sendmail-mta08:34:030
14759890,5rtkit-daemon2455426-21kworker/u16:0-rpciod09:59:531
14759890,5rtkit-daemon214-21plymouthd12:01:122
14759890,5rtkit-daemon1540-21sendmail-mta09:07:530
14759890,4rtkit-daemon2675257-21ssh11:44:352
14759890,4rtkit-daemon2611007-21apt-get11:05:121
14759890,4rtkit-daemon2552914-21ssh10:26:521
14759890,4rtkit-daemon2399519-21kworker/u16:3-nfsiod09:34:191
14759890,4rtkit-daemon2316930-21ntp_states07:40:220
14759890,2rtkit-daemon214-21plymouthd10:46:241
14759890,2rtkit-daemon214-21plymouthd09:39:491
14759890,1rtkit-daemon0-21swapper/212:21:442
14759890,1rtkit-daemon0-21swapper/008:41:050
2704774280,6sleep12705373-21expr12:00:191
2665740281,4sleep22665748-21ssh11:36:082
2513506280,1sleep10-21swapper/110:04:381
2500459281,5sleep00-21swapper/009:55:180
2481725281,3sleep00-21swapper/009:45:130
2425366281,4sleep22425571-21hddtemp_smartct09:10:172
2418790280,1sleep10-21swapper/109:05:071
2405497280,1sleep10-21swapper/108:51:121
2401635280,5sleep02403642-21kthreadcore08:50:190
2349301281,4sleep12349334-21apt-get08:10:131
2343233281,4sleep12343349-21tr08:05:161
2329352280,5sleep12329429-21ntp_states07:50:201
2325694280,2sleep22327275-21kthreadcore07:50:172
2299035281,4sleep02299068-21apt-get07:30:110
2266282281,4sleep22267212-21cut07:05:172
14759880,5rtkit-daemon2573878-21kthreadcore10:40:171
14759880,5rtkit-daemon1540-21sendmail-mta09:03:270
14759880,4rtkit-daemon730-21ntpd11:21:490
14759880,4rtkit-daemon729-21lldpd10:39:371
14759880,4rtkit-daemon729-21lldpd10:28:212
14759880,4rtkit-daemon476-21wpa_supplicant12:12:082
14759880,4rtkit-daemon295-21systemd-journal07:17:010
14759880,4rtkit-daemon2697178-21ntp_states11:55:202
14759880,4rtkit-daemon2631048-21cut11:15:210
14759880,4rtkit-daemon2560932-21ssh10:31:481
14759880,4rtkit-daemon2560932-21ssh10:31:471
14759880,4rtkit-daemon2479593-21ssh09:40:481
14759880,4rtkit-daemon2424828-21diskmemload11:32:010
14759880,4rtkit-daemon2373665-21/usr/sbin/munin08:25:200
14759880,4rtkit-daemon2278024-21munin-node12:19:021
14759880,4rtkit-daemon2278024-21munin-node11:11:060
14759880,4rtkit-daemon2278024-21munin-node08:13:000
14759880,4rtkit-daemon214-21plymouthd11:06:172
14759880,4rtkit-daemon214-21plymouthd07:27:330
14759880,4rtkit-daemon214-21plymouthd07:07:160
14759880,4rtkit-daemon1-21systemd12:05:232
14759880,4rtkit-daemon1-21systemd11:25:570
14759880,4rtkit-daemon1-21systemd08:04:080
14759880,3rtkit-daemon214-21plymouthd10:22:051
14759880,1rtkit-daemon0-21swapper/111:45:131
14759880,1rtkit-daemon0-21swapper/110:13:371
2698978270,1sleep00-21swapper/011:58:260
2659827271,3sleep12659881-21fschecks_count11:35:151
2652050271,3sleep10-21swapper/111:30:181
2634760271,4sleep22634768-21ssh11:19:562
2602189270,5sleep12602196-21sh10:59:111
2570691270,3sleep02570730-21apt-get10:40:120
2563104270,4sleep02564041-21irqstats10:35:190
2545536270,1sleep00-21swapper/010:23:240
2522616270,4sleep22522789-21/usr/sbin/munin10:10:162
2522339271,3sleep02522494-21diskstats10:10:140
2489451271,4sleep22489490-21basename09:50:002
2478978270,4sleep22473648-21/usr/sbin/munin09:40:212
2421183270,5sleep20-21swapper/209:05:202
2418441270,1sleep20-21swapper/209:03:472
2369724270,4sleep22370821-21grep08:25:162
2368425271,2sleep10-21swapper/108:25:101
2355626270,4sleep12355724-21tr08:15:131
2311781270,3sleep10-21swapper/107:40:141
2280273270,4sleep22280428-21diskstats07:15:142
22716509971,3cyclictest0-21swapper/307:10:003
14759870,4rtkit-daemon729-21lldpd09:26:202
14759870,4rtkit-daemon456-21avahi-daemon07:55:010
14759870,3rtkit-daemon729-21lldpd09:53:061
14759870,3rtkit-daemon2758184-21grep12:35:170
14759870,3rtkit-daemon2604293-21irqstats11:00:170
14759870,3rtkit-daemon2536146-21ssh10:15:291
14759870,3rtkit-daemon2278024-21munin-node08:06:300
14759870,3rtkit-daemon1-21systemd07:37:550
14759870,3rtkit-daemon1-21systemd07:20:060
14759870,1rtkit-daemon0-21swapper/211:51:082
2763181260,1sleep20-21swapper/212:37:002
2755148260,1sleep00-21swapper/012:32:090
2746567260,1sleep10-21swapper/112:26:001
2745805260,3sleep02745862-21ps12:25:220
2724344260,1sleep00-21swapper/012:15:050
2713874260,3sleep12714082-21users12:05:251
2698830260,1sleep10-21swapper/111:58:071
2683872260,3sleep02684023-21df11:50:120
2682910262,2sleep02424828-21diskmemload11:48:360
2651085260,1sleep10-21swapper/111:29:421
2635084260,4sleep20-21swapper/211:20:052
2608688260,1sleep20-21swapper/211:00:292
2596766260,3sleep22598529-21ssh10:55:202
2594464260,1sleep20-21swapper/210:55:002
2592530260,1sleep10-21swapper/110:50:301
2573191260,3sleep22575223-21cut10:40:172
2569054260,4sleep22424828-21diskmemload10:37:012
2546463260,1sleep00-21swapper/010:25:040
2519745260,1sleep00-21swapper/010:05:240
2513540260,1sleep20-21swapper/210:04:432
2511199260,1sleep00-21swapper/010:00:200
2489838261,3sleep02489990-21df09:50:140
2464515260,1sleep20-21swapper/209:33:162
2449523261,2sleep02449562-21apt-get09:25:130
2447266260,1sleep20-21swapper/209:21:072
2439327260,4sleep244-21ksoftirqd/209:16:262
2413978260,3sleep12415560-21kthreadcore09:00:181
2406057260,1sleep20-21swapper/208:55:132
2397359260,3sleep12398501-21memory08:45:181
2393042260,1sleep10-21swapper/108:42:061
2368017260,1sleep10-21swapper/108:23:071
2361478260,1sleep20-21swapper/208:18:112
2348961260,1sleep20-21swapper/208:08:572
2342690260,1sleep10-21swapper/108:03:501
2336256260,1sleep10-21swapper/107:56:381
2336173260,1sleep20-21swapper/207:55:352
2323748260,1sleep20-21swapper/207:46:172
2313171260,3sleep22314891-21kthreadcore07:40:202
2305010260,1sleep20-21swapper/207:34:352
2299071260,3sleep12299216-21sh07:30:131
2298462260,1sleep20-21swapper/207:25:572
2298094260,1sleep10-21swapper/107:25:241
2287893260,3sleep12288981-21cut07:20:191
2277976261,2sleep10-21swapper/107:15:021
2277516260,1sleep20-21swapper/207:10:312
2274064260,3sleep12276102-21tr07:10:171
14759860,3rtkit-daemon2738372-21ssh12:20:471
14759860,3rtkit-daemon214-21plymouthd11:08:000
14759860,2rtkit-daemon476-21wpa_supplicant11:04:461
2755708250,3sleep10-21swapper/112:33:261
2732732250,3sleep02732853-21perl12:20:140
2721297250,3sleep02721487-21cat12:10:210
2716410250,2sleep10-21swapper/112:10:121
2713740250,2sleep00-21swapper/012:05:240
2691153251,2sleep1331ktimers/111:54:011
2626496250,3sleep20-21swapper/211:14:242
2620580251,2sleep10-21swapper/111:10:181
2581374250,2sleep20-21swapper/210:45:192
2478833251,2sleep02478897-21ntp_states09:40:200
2471947250,3sleep00-21swapper/009:36:480
2448499250,3sleep10-21swapper/109:24:011
2405724250,1sleep20-21swapper/208:54:062
2374720250,3sleep22374755-21grep08:30:132
2368050250,3sleep20-21swapper/208:23:322
2288982251,2sleep20-21swapper/207:20:202
14759850,1rtkit-daemon0-21swapper/111:41:271
2495237240,2sleep22495328-21ssh09:50:232
2408003240,2sleep10-21swapper/108:55:211
2398981241,2sleep2431ktimers/208:45:232
2386737240,2sleep2214-21plymouthd08:37:062
2382334240,2sleep12383794-21grep08:35:161
2319427240,2sleep1214-21plymouthd07:45:161
2311046240,1sleep10-21swapper/107:36:261
2349086230,1sleep20-21swapper/208:10:012
2305290230,1sleep20-21swapper/207:35:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional