You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 21:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rack5slot0.osadl.org (updated Thu Sep 18, 2025 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1661021420,1sleep22500699cyclictest12:21:502
2920321390,0sleep02499799cyclictest12:23:000
2357321380,4sleep02499799cyclictest11:27:040
2328421310,1sleep32500799cyclictest12:22:273
868021260,0sleep00-21swapper/011:31:340
620521200,2sleep16201-21sshd11:57:401
1967921180,2sleep219678-21open_files09:10:232
1626121090,2sleep10-21swapper/110:00:481
1967221080,4sleep12500299cyclictest11:53:031
1386321050,25sleep0135650irq/27-eth009:48:120
279192980,1sleep027916-21sshd10:17:260
205062960,1sleep120501-21ls09:10:261
2461529529,46sleep10-21swapper/107:08:261
60442940,4sleep32500799cyclictest10:38:583
60442940,4sleep32500799cyclictest10:38:583
2469629333,47sleep30-21swapper/307:09:273
116272930,2sleep20-21swapper/210:19:002
72622920,2sleep27258-21sshd12:29:472
34672920,2sleep12500299cyclictest09:28:171
323982910,2sleep10-21swapper/112:02:571
323982910,2sleep10-21swapper/112:02:571
40162900,2sleep10-21swapper/109:50:181
259582890,1sleep025943-21sshd11:21:270
208932890,7sleep30-21swapper/310:07:393
270202880,3sleep30-21swapper/309:46:193
213792880,1sleep00-21swapper/009:26:480
259752850,3sleep10-21swapper/112:16:511
253382830,1sleep025337-21sshd12:10:570
166302830,2sleep11848-21nfsd12:39:241
129202830,4sleep30-21swapper/311:08:443
156902820,2sleep215692-21bash09:45:142
147632820,1sleep30-21swapper/309:10:003
312102800,1sleep331208-21sshd10:14:573
228872800,1sleep20-21swapper/212:05:032
233842790,4sleep22500699cyclictest12:31:072
224502770,4sleep32500799cyclictest10:57:533
21782770,2sleep30-21swapper/311:42:433
158952770,3sleep215893-21bash11:46:502
317632750,2sleep10-21swapper/109:24:411
317092750,1sleep30-21swapper/311:57:013
272222740,1sleep327220-21sshd10:40:463
267062740,1sleep30-21swapper/311:15:413
303382720,2sleep30-21swapper/309:59:093
176432720,2sleep217615-21sshd10:25:262
136912720,2sleep113689-21sshd11:49:381
197212710,2sleep319712-21sshd10:51:443
189542700,1sleep10-21swapper/110:22:441
141572700,4sleep32500799cyclictest09:26:033
271722690,2sleep22500699cyclictest10:23:292
106652680,4sleep12500299cyclictest12:21:161
191872670,1sleep10-21swapper/110:37:111
191872670,1sleep10-21swapper/110:37:111
89232640,2sleep20-21swapper/209:28:502
222402640,3sleep22500699cyclictest09:39:412
22182620,0sleep12219-21sshd10:09:011
126382570,2sleep10-21swapper/107:51:281
293992530,2sleep229398-21sshd11:56:482
2438924933,12sleep20-21swapper/207:05:342
265522470,1sleep326551-21sshd09:36:533
280272450,1sleep128026-21bash11:12:591
2458224336,4sleep00-21swapper/007:07:590
2499799420,2cyclictest14248-21sshd10:57:080
207972400,3sleep10-21swapper/110:51:501
25007993420,11cyclictest14536-21sshd09:16:253
25007993324,8cyclictest0-21swapper/311:30:073
25007993322,8cyclictest7241-21sshd10:03:103
25007993225,4cyclictest2853-21sshd11:04:563
25007993220,7cyclictest28723-21sshd12:11:163
25007993126,3cyclictest12485-21sshd12:27:163
25007993123,5cyclictest0-21swapper/309:42:243
224942310,4sleep20-21swapper/211:26:582
199472310,5sleep20-21swapper/211:06:222
62942300,0sleep041ktimersoftd/011:45:560
25007993025,3cyclictest412-21sshd10:47:083
25007993025,3cyclictest2480-21sshd11:25:153
25007993024,4cyclictest0-21swapper/310:19:453
25007993023,5cyclictest16934-21bash09:10:113
25007993023,4cyclictest5125-21sshd11:13:513
25007993023,4cyclictest135650irq/27-eth011:24:133
25007993022,6cyclictest25779-21sshd10:29:083
25007993021,5cyclictest15975-21sshd09:54:423
33882290,2sleep13372-21sshd11:36:551
25007992919,7cyclictest3829-21sshd10:21:203
25007992918,7cyclictest21062-21sshd11:50:153
25007992917,8cyclictest705-21sshd11:45:253
25007992915,9cyclictest3352-21sshd12:17:463
24997992914,7cyclictest8-21rcu_preempt09:54:150
318262280,4sleep32500799cyclictest12:05:433
25007992822,4cyclictest1848-21nfsd12:37:573
25007992821,5cyclictest14941-21switchtime12:30:223
25007992821,4cyclictest958-21sshd11:36:423
25007992819,6cyclictest17267-21sshd09:32:513
2499799285,10cyclictest0-21swapper/012:33:330
2499799281,20cyclictest385-21sshd10:21:010
25007992720,5cyclictest25806-21sshd10:34:573
25007992720,5cyclictest25806-21sshd10:34:573
25007992719,4cyclictest1847-21nfsd09:21:263
2499799278,10cyclictest22195-21sshd11:59:080
2499799270,23cyclictest0-21swapper/010:33:360
2499799270,23cyclictest0-21swapper/010:33:360
240872270,2sleep10-21swapper/110:49:181
73532260,1sleep20-21swapper/210:59:292
61192260,1sleep06113-21bash10:35:580
61192260,1sleep06113-21bash10:35:580
25007992616,7cyclictest0-21swapper/312:02:123
25007992616,7cyclictest0-21swapper/312:02:113
2499799260,16cyclictest0-21swapper/011:39:550
220712260,3sleep20-21swapper/212:13:402
103762260,2sleep110375-21bash09:15:591
54502250,2sleep20-21swapper/209:18:482
2499799255,12cyclictest0-21swapper/009:11:360
24997992521,2cyclictest0-21swapper/009:32:380
2499799251,14cyclictest6922-21sshd10:09:290
2499799251,13cyclictest10216-21bash10:50:510
2499799250,1cyclictest3999-21bash11:13:450
2499799250,14cyclictest30346-21sshd10:41:040
177982250,2sleep10-21swapper/109:32:541
24997992414,6cyclictest3-21ksoftirqd/012:01:300
24997992414,6cyclictest3-21ksoftirqd/012:01:300
2499799240,19cyclictest0-21swapper/011:16:300
2499799240,18cyclictest816-21sshd09:43:490
2499799240,18cyclictest10-21rcuc/009:38:590
2499799240,17cyclictest21770-21sshd10:10:490
2499799240,16cyclictest16978-21sed11:00:170
2499799240,16cyclictest0-21swapper/010:45:330
2499799240,13cyclictest14659-21sshd11:08:530
2499799240,12cyclictest11452-21sshd09:16:060
2499799238,4cyclictest9771-21sshd12:09:410
2499799230,15cyclictest10-21rcuc/009:21:530
25006992212,7cyclictest14320-21sshd10:07:002
25002992217,4cyclictest1848-21nfsd12:32:031
2499799229,7cyclictest3-21ksoftirqd/012:36:310
2499799221,19cyclictest0-21swapper/012:15:310
2499799220,15cyclictest0-21swapper/009:57:050
231922220,2sleep20-21swapper/211:12:312
84552210,17sleep1135650irq/27-eth011:17:001
72592210,1sleep22500699cyclictest11:16:522
34972210,19sleep2135650irq/27-eth009:44:062
25006992115,5cyclictest25-21rcuop/110:37:052
25006992115,5cyclictest25-21rcuop/110:37:052
25006992114,5cyclictest1927-21snmpd09:52:082
2500299217,3cyclictest10950irq/23-ehci_hcd10:27:331
2500299213,6cyclictest10950irq/23-ehci_hcd09:59:411
25002992113,5cyclictest31591-21cp11:04:361
2500299211,10cyclictest32283-21sshd10:55:481
2500299210,5cyclictest22606-21sshd09:36:291
2499799210,14cyclictest0-21swapper/011:53:010
2499799210,12cyclictest0-21swapper/010:00:320
290592200,0sleep00-21swapper/010:26:280
25006992015,3cyclictest135650irq/27-eth012:39:142
2499799208,4cyclictest32599-21sshd11:42:310
2499799200,14cyclictest0-21swapper/007:20:440
2500699196,6cyclictest135650irq/27-eth012:17:572
2500699196,6cyclictest135650irq/27-eth010:48:372
25006991916,2cyclictest23381-21sshd10:31:452
25006991916,2cyclictest23381-21sshd10:31:452
25006991915,3cyclictest14643-21bash12:01:162
25006991915,3cyclictest14643-21bash12:01:162
2500299198,7cyclictest1959-21sshd12:14:281
2500299194,1cyclictest0-21swapper/111:30:401
25002991913,2cyclictest11999-21bash11:29:011
25002991911,5cyclictest3269-21sshd10:18:081
2499799198,5cyclictest0-21swapper/008:37:080
2499799197,6cyclictest0-21swapper/007:47:350
2499799191,6cyclictest0-21swapper/012:25:300
249702192,15sleep210950irq/23-ehci_hcd11:50:332
2500699189,1cyclictest21156-21bash11:00:382
2500699185,6cyclictest11445-21sshd11:20:132
25006991812,4cyclictest3755-21sshd09:56:322
25006991812,2cyclictest135650irq/27-eth008:00:132
2500699181,15cyclictest20700-21sshd10:14:002
2500299181,6cyclictest1846-21nfsd10:34:001
2500299181,6cyclictest1846-21nfsd10:34:001
2500299181,5cyclictest31354-21sshd10:44:081
25002991813,3cyclictest0-21swapper/110:13:081
25002991811,4cyclictest27003-21sshd11:21:331
25002991811,3cyclictest0-21swapper/109:40:141
112092180,16sleep2135650irq/27-eth010:42:182
8252170,0sleep2826-21bash09:24:502
25007991710,4cyclictest0-21swapper/308:05:143
2500799170,2cyclictest0-21swapper/307:54:383
2500699179,3cyclictest6857-21bash09:31:462
2500699178,5cyclictest32148-21sshd10:52:572
2500699176,6cyclictest135650irq/27-eth010:03:032
2500699170,14cyclictest27951-21sshd11:39:122
2500299172,5cyclictest10950irq/23-ehci_hcd11:08:491
2500299171,14cyclictest27620-21sshd11:42:001
2500299170,5cyclictest0-21swapper/112:08:231
2499799171,7cyclictest0-21swapper/008:25:130
2499799170,7cyclictest0-21swapper/008:15:130
2499799170,1cyclictest30507-21munin-node08:30:210
2499799170,1cyclictest0-21swapper/008:23:120
2499799170,1cyclictest0-21swapper/007:41:260
2499799170,16cyclictest0-21swapper/008:56:360
2500799168,6cyclictest10259-21munin-node07:45:243
2500799162,12cyclictest1459-21crond07:30:003
2500699161,7cyclictest21474-21nscd11:42:572
2500699160,8cyclictest25759-21id11:30:132
2500299169,4cyclictest0-21swapper/112:26:101
2499799161,1cyclictest0-21swapper/008:49:110
2499799160,1cyclictest0-21swapper/008:40:250
2499799160,1cyclictest0-21swapper/008:12:230
2499799160,1cyclictest0-21swapper/008:01:570
2499799160,15cyclictest0-21swapper/009:00:490
2499799160,15cyclictest0-21swapper/007:50:540
2499799160,15cyclictest0-21swapper/007:15:350
2499799160,11cyclictest0-21swapper/008:50:180
320342150,2sleep10-21swapper/107:25:171
2500799157,6cyclictest1915-21snmpd08:22:523
2500799155,1cyclictest44-21ksoftirqd/308:02:003
2500799153,10cyclictest13999-21latency07:55:193
25007991511,2cyclictest30649-21grep07:20:183
2500799150,2cyclictest135650irq/27-eth008:50:523
2500299155,0cyclictest0-21swapper/109:45:261
2500299155,0cyclictest0-21swapper/109:00:211
2500299153,11cyclictest10950irq/23-ehci_hcd08:41:121
2499799150,1cyclictest0-21swapper/008:06:170
2499799150,1cyclictest0-21swapper/007:31:120
2499799150,14cyclictest0-21swapper/009:06:180
2500799149,3cyclictest23878-21grep08:15:223
2500799148,5cyclictest0-21swapper/307:16:173
2500799142,11cyclictest144550irq/28-eth1-rx-07:34:483
2500799140,5cyclictest30646-21grep08:30:213
2500799140,11cyclictest135650irq/27-eth007:42:263
2500699148,4cyclictest14806-21users07:55:262
2500699146,6cyclictest30416-21sendmail_mailtr08:30:202
2500699142,11cyclictest135650irq/27-eth009:06:232
2500699141,4cyclictest2405-21runrttasks07:51:502
2500699141,12cyclictest135650irq/27-eth007:45:282
2500699141,11cyclictest31675-21df08:35:132
2500699141,11cyclictest25354-21diskstats07:10:122
2500699140,13cyclictest0-21swapper/208:15:142
2500299148,1cyclictest0-21swapper/108:20:001
2500299146,1cyclictest25790-21iostat_ios07:10:151
2500299144,0cyclictest0-21swapper/108:05:161
2500299143,10cyclictest10950irq/23-ehci_hcd08:23:081
25002991410,3cyclictest10950irq/23-ehci_hcd08:28:061
2499799141,10cyclictest0-21swapper/007:55:030
2499799140,8cyclictest0-21swapper/007:25:200
2499799140,7cyclictest0-21swapper/007:10:060
2499799140,1cyclictest0-21swapper/007:35:290
2500799138,3cyclictest0-21swapper/309:00:133
2500799135,6cyclictest135650irq/27-eth008:10:253
2500799131,3cyclictest32547-21munin-node08:35:193
2500799131,11cyclictest0-21swapper/308:47:113
2500799130,9cyclictest26371-21switchtime07:10:213
2500799130,11cyclictest4370-21diskstats07:35:133
2500699138,3cyclictest4240-21perl08:45:142
2500699137,4cyclictest27422-21fschecks_time08:25:112
2500699131,11cyclictest135650irq/27-eth007:31:342
25006991311,1cyclictest0-21swapper/207:21:592
2500699131,10cyclictest7080-21iostat07:40:142
2500699130,4cyclictest20162-21cpu08:10:122
2500699130,12cyclictest0-21swapper/209:00:122
2500699130,12cyclictest0-21swapper/208:55:282
2500699130,12cyclictest0-21swapper/208:50:302
2500699130,12cyclictest0-21swapper/208:43:242
2500699130,12cyclictest0-21swapper/207:27:182
2500699130,11cyclictest18406-21hddtemp_smartct08:05:162
2500299136,6cyclictest135650irq/27-eth008:10:431
2500299133,1cyclictest10662-21munin-run07:50:001
2500299132,10cyclictest135650irq/27-eth008:55:061
2500299131,4cyclictest135650irq/27-eth008:00:591
2500299131,10cyclictest14950-21latency_hist09:10:001
2500799126,5cyclictest1848-21nfsd08:25:033
2500799120,3cyclictest8153-21munin-node08:55:113
2500799120,10cyclictest0-21swapper/308:40:123
2500699120,3cyclictest28958-21users07:15:222
2500699120,3cyclictest25832-21rt-features08:20:222
2500699120,11cyclictest0-21swapper/207:35:162
2500299126,5cyclictest0-21swapper/108:45:361
2500299126,2cyclictest0-21swapper/107:55:271
2500299124,1cyclictest31678-21df08:35:121
2500299124,1cyclictest28085-21irqstats07:15:141
2500299123,3cyclictest144550irq/28-eth1-rx-07:20:171
2500299121,1cyclictest0-21swapper/107:40:221
25002991210,1cyclictest0-21swapper/107:38:551
25002991210,1cyclictest0-21swapper/107:32:331
2500299117,3cyclictest1848-21nfsd08:50:021
2500299116,3cyclictest30772-21users08:30:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional