You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-03 - 11:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Thu Jul 03, 2025 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3176821580,2sleep01257299cyclictest22:36:140
2951621550,2sleep329501-21sshd21:12:433
2575021460,0sleep31848-21nfsd23:00:433
1874221450,0sleep318743-21sshd22:04:103
883521430,4sleep21258199cyclictest23:05:232
367921410,5sleep31258599cyclictest23:32:543
2085621360,4sleep21258199cyclictest22:54:042
1135221320,3sleep11257699cyclictest22:56:111
550921300,3sleep01257299cyclictest22:09:040
54521300,2sleep30-21swapper/323:10:483
1677521280,2sleep10-21swapper/123:53:071
36221240,2sleep2361-21sshd00:17:382
1125321240,7sleep111254-21needreboot19:05:171
305221230,2sleep23051-21sshd22:30:232
2354421210,2sleep10-21swapper/121:30:331
161021210,2sleep30-21swapper/321:31:353
3154421200,0sleep21848-21nfsd22:45:302
3055721200,1sleep20-21swapper/222:11:222
292821200,3sleep01257299cyclictest00:11:100
1397521140,3sleep113972-21sshd23:15:161
2824721130,4sleep31258599cyclictest21:49:393
2273221100,0sleep00-21swapper/021:30:280
1052621100,2sleep21848-21nfsd23:27:192
636521090,1sleep30-21swapper/321:16:413
2739321070,1sleep30-21swapper/323:22:473
1981021060,4sleep31258599cyclictest00:09:413
1449821060,3sleep21258199cyclictest23:46:312
1449821060,3sleep21258199cyclictest23:46:312
1912321050,0sleep10-21swapper/123:37:321
1912321050,0sleep10-21swapper/123:37:321
3180421030,4sleep01257299cyclictest23:51:230
37221020,3sleep21847-21nfsd00:24:192
749121010,4sleep01257299cyclictest23:24:000
2943821010,2sleep329431-21sshd22:57:573
1700021010,2sleep116977-21sshd23:30:591
1428121000,3sleep01257299cyclictest23:37:030
1428121000,3sleep01257299cyclictest23:37:030
12108210034,45sleep30-21swapper/319:07:163
201872990,1sleep11257699cyclictest21:39:351
296192970,2sleep229616-21sshd21:18:552
282182960,2sleep11257699cyclictest22:45:181
22432960,2sleep12242-21bash22:36:311
69462950,2sleep11847-21nfsd22:43:121
157542950,2sleep01257299cyclictest22:28:330
102962950,2sleep10-21swapper/100:12:011
76472940,5sleep21848-21nfsd23:39:392
76472940,5sleep21848-21nfsd23:39:382
159412940,1sleep30-21swapper/322:13:083
187482920,1sleep20-21swapper/221:27:062
183032920,0sleep00-21swapper/023:12:340
50062910,3sleep21258199cyclictest21:35:032
292922900,3sleep329285-21cp23:44:563
292922900,3sleep329285-21cp23:44:563
139692900,1sleep20-21swapper/221:54:272
216182890,5sleep31258599cyclictest22:47:473
197022890,1sleep019701-21bash23:28:140
116932880,2sleep10-21swapper/100:05:261
248252870,2sleep20-21swapper/223:03:542
200412870,0sleep00-21swapper/000:29:510
36732850,2sleep23674-21bash21:13:222
291102850,2sleep229112-21timerwakeupswit22:20:282
80892840,1sleep30-21swapper/300:11:453
273942840,1sleep20-21swapper/223:10:202
157932830,2sleep30-21swapper/321:39:083
1216128335,11sleep20-21swapper/219:07:562
121102810,3sleep01257299cyclictest22:00:220
145072800,1sleep10-21swapper/123:12:111
103722800,1sleep21848-21nfsd21:47:532
95602790,1sleep20-21swapper/200:32:172
169142790,2sleep10-21swapper/119:20:001
182652780,1sleep10-21swapper/122:22:361
284322760,3sleep2135650irq/27-eth021:31:032
219962760,2sleep321998-21sshd22:35:213
13612760,1sleep31848-21nfsd21:59:263
213992730,2sleep10-21swapper/120:43:321
197522730,2sleep11848-21nfsd23:06:251
73112720,1sleep20-21swapper/221:22:562
247942720,2sleep32405-21runrttasks19:35:173
134062710,2sleep313405-21sshd23:36:593
134062710,2sleep313405-21sshd23:36:583
88242700,1sleep10-21swapper/100:32:131
41872700,4sleep31258599cyclictest21:22:373
267852700,2sleep20-21swapper/222:29:382
188972700,3sleep01257299cyclictest23:40:360
188972700,3sleep01257299cyclictest23:40:360
144612700,2sleep10-21swapper/122:31:301
13202700,2sleep10-21swapper/100:27:441
67212690,0sleep26724-21sshd00:11:372
171652640,3sleep01257299cyclictest23:34:150
171242630,3sleep21258199cyclictest23:21:462
166142600,2sleep12405-21runrttasks20:30:371
233812540,3sleep30-21swapper/322:44:513
127122540,0sleep212707-21cp22:03:342
111432520,2sleep11848-21nfsd21:44:571
182132490,0sleep218214-21bash22:19:322
1257299490,40cyclictest0-21swapper/023:08:480
165402480,1sleep30-21swapper/300:22:363
1209524235,4sleep00-21swapper/019:07:050
12585993834,2cyclictest0-21swapper/322:23:463
65332362,32sleep2135650irq/27-eth020:10:152
186592350,1sleep11848-21nfsd22:28:491
12585993513,2cyclictest135650irq/27-eth022:07:283
204142340,2sleep120406-21sshd22:13:351
12585993417,16cyclictest44-21ksoftirqd/322:28:333
12576993430,3cyclictest22927-21sshd21:52:091
79182330,30sleep226950irq/26-ahci[00021:41:252
1257299330,31cyclictest0-21swapper/021:37:340
263112320,30sleep1135650irq/27-eth000:23:401
194822320,2sleep319464-21sshd21:42:333
12585993015,4cyclictest32366-21bash21:53:053
12581993025,3cyclictest15726-21sshd21:57:392
293032287,19sleep110950irq/23-ehci_hcd23:01:051
224372280,2sleep322424-21sshd23:56:513
1258599288,6cyclictest0-21swapper/321:28:183
1257299289,16cyclictest0-21swapper/022:44:270
1258599271,4cyclictest31309-21sshd22:30:053
223902260,2sleep20-21swapper/222:57:162
1258599263,20cyclictest44-21ksoftirqd/300:16:553
12585992619,3cyclictest23129-21bash23:06:463
12572992622,3cyclictest135650irq/27-eth021:27:080
1257299262,16cyclictest19357-21sshd23:18:560
1257299260,17cyclictest26056-21sshd21:15:270
296122252,2sleep041ktimersoftd/000:20:330
193492250,3sleep219347-21sshd00:37:062
12585992520,3cyclictest14198-21sshd00:02:253
1257699256,15cyclictest22-21ksoftirqd/122:17:071
204462240,2sleep120445-21sshd21:58:071
196702240,1sleep30-21swapper/322:50:403
1258599243,19cyclictest44-21ksoftirqd/300:31:223
12585992419,3cyclictest11021-21sshd23:27:233
1258199242,3cyclictest4759-21sshd00:28:052
12581992412,10cyclictest25645-21bash23:31:512
1258199240,18cyclictest33-21ksoftirqd/222:09:392
1257299244,3cyclictest12982-21sshd22:53:160
1257299244,11cyclictest9291-21nscd22:15:390
12572992422,1cyclictest10140-21bash21:54:040
1257299241,16cyclictest14137-21sshd22:31:280
1258599232,19cyclictest44-21ksoftirqd/323:46:273
1258599232,19cyclictest44-21ksoftirqd/323:46:273
12585992314,1cyclictest135650irq/27-eth023:53:003
12581992315,5cyclictest19690-21sshd23:59:512
1258199230,18cyclictest33-21ksoftirqd/223:44:112
1258199230,18cyclictest33-21ksoftirqd/223:44:112
12576992320,2cyclictest0-21swapper/121:24:091
12576992311,9cyclictest0-21swapper/122:07:521
1257299235,16cyclictest0-21swapper/021:41:130
12572992312,4cyclictest3-21ksoftirqd/021:12:310
1257299230,21cyclictest6912-21sshd00:35:390
1257299230,21cyclictest10364-21sshd22:56:060
1257299230,15cyclictest9440-21sshd22:24:540
12585992218,2cyclictest0-21swapper/322:16:033
12585992215,5cyclictest10733-21sshd23:18:043
12581992217,3cyclictest15588-21sshd23:15:232
12581992215,5cyclictest13598-21sshd22:37:372
1257699224,16cyclictest22-21ksoftirqd/121:25:231
12576992219,2cyclictest23868-21rm21:49:131
12576992218,2cyclictest29722-21sshd23:26:011
1257699220,19cyclictest22-21ksoftirqd/121:12:011
12572992220,1cyclictest0-21swapper/019:33:550
12572992211,3cyclictest11752-21sshd23:49:300
12572992211,3cyclictest11752-21sshd23:49:300
1257299220,1cyclictest22092-21sshd22:13:440
1257299220,13cyclictest0-21swapper/023:55:060
12585992116,3cyclictest7330-21sshd00:25:013
12585992114,4cyclictest1848-21nfsd00:35:093
12581992114,5cyclictest4052-21bash23:51:512
12576992118,2cyclictest1848-21nfsd00:35:331
12576992115,4cyclictest32638-21rm23:42:011
12576992115,4cyclictest32638-21rm23:42:001
12576992115,4cyclictest31149-21bash00:00:491
12576992114,5cyclictest26007-21sshd00:16:561
1257699210,1cyclictest19074-21sshd21:17:541
1257299219,5cyclictest22069-21sshd22:47:500
1257299213,1cyclictest6528-21sshd00:05:040
1257299211,10cyclictest10247-21bash00:18:410
12572992110,2cyclictest0-21swapper/021:45:060
1257299210,16cyclictest0-21swapper/023:00:050
1257299210,13cyclictest27020-21sshd21:58:450
12581992012,2cyclictest0-21swapper/200:04:192
1258199200,4cyclictest22565-21sshd00:06:342
1258199200,16cyclictest19290-21cp22:44:282
1257699206,11cyclictest2911-21sshd23:20:241
12576992015,4cyclictest1846-21nfsd23:46:361
12576992015,4cyclictest1846-21nfsd23:46:361
12576992015,3cyclictest1847-21nfsd22:01:021
12576992014,5cyclictest2405-21runrttasks22:52:011
1257699200,16cyclictest22-21ksoftirqd/123:55:101
1257299200,7cyclictest28585-21sshd00:00:340
1257299200,13cyclictest0-21swapper/021:20:080
1257299190,10cyclictest12379-21bash00:32:390
1258599170,14cyclictest0-21swapper/320:58:273
1258199170,9cyclictest135650irq/27-eth020:00:122
1257699175,4cyclictest10950irq/23-ehci_hcd19:28:311
12585991612,3cyclictest10950irq/23-ehci_hcd20:11:523
12585991611,0cyclictest0-21swapper/320:45:213
1258199163,8cyclictest135650irq/27-eth021:06:482
1258599154,10cyclictest10950irq/23-ehci_hcd19:34:253
1258599154,10cyclictest10950irq/23-ehci_hcd19:22:103
1258599153,11cyclictest10950irq/23-ehci_hcd19:57:553
1258199157,6cyclictest1959-21sshd19:11:012
1258199152,7cyclictest0-21swapper/220:37:322
1258199151,11cyclictest135650irq/27-eth020:21:422
1258199150,13cyclictest0-21swapper/220:50:322
1257699150,14cyclictest0-21swapper/120:55:181
1257299152,10cyclictest0-21swapper/019:15:100
1257299151,13cyclictest0-21swapper/020:10:140
1258599148,4cyclictest626-21bash21:10:003
12585991411,2cyclictest20617-21tr20:40:183
12585991411,2cyclictest0-21swapper/321:02:293
1258599140,13cyclictest1953-21gltestperf20:00:163
1258599140,13cyclictest0-21swapper/320:35:153
1258199147,4cyclictest3351-21dump-pmu-power20:09:552
1258199147,4cyclictest3351-21dump-pmu-power20:09:552
1258199143,11cyclictest0-21swapper/221:01:122
1258199141,9cyclictest24218-21df19:35:122
1258199141,6cyclictest135650irq/27-eth020:59:522
1258199141,6cyclictest135650irq/27-eth020:25:122
1258199141,6cyclictest135650irq/27-eth019:52:522
1258199140,9cyclictest0-21swapper/219:16:262
1258199140,12cyclictest26950irq/26-ahci[00020:40:232
1257699149,4cyclictest1927-21snmpd19:53:111
1257699147,3cyclictest135650irq/27-eth019:55:101
1257699143,3cyclictest10950irq/23-ehci_hcd21:02:351
1257699143,3cyclictest10950irq/23-ehci_hcd20:03:051
1257699143,10cyclictest10950irq/23-ehci_hcd20:24:261
12576991410,3cyclictest10950irq/23-ehci_hcd20:10:301
1257699140,13cyclictest0-21swapper/120:26:041
1257699140,12cyclictest0-21swapper/119:10:161
1257299147,5cyclictest0-21swapper/021:07:160
1257299147,1cyclictest0-21swapper/020:35:560
12572991412,1cyclictest0-21swapper/020:55:130
1257299140,1cyclictest20250-21iostat_ios20:40:150
1257299140,1cyclictest0-21swapper/019:26:210
1257299140,13cyclictest0-21swapper/021:00:220
1257299140,13cyclictest0-21swapper/020:21:210
1257299140,12cyclictest0-21swapper/020:30:430
1258599137,5cyclictest10950irq/23-ehci_hcd19:45:093
1258599137,5cyclictest0-21swapper/320:25:243
1258599137,5cyclictest0-21swapper/320:08:083
1258599137,5cyclictest0-21swapper/320:08:083
1258599132,10cyclictest135650irq/27-eth019:10:383
12585991310,1cyclictest0-21swapper/319:51:213
1258599130,12cyclictest15829-21munin-node20:30:213
1258599130,12cyclictest0-21swapper/320:50:573
1258599130,12cyclictest0-21swapper/320:20:173
1258599130,12cyclictest0-21swapper/319:42:233
1258599130,11cyclictest9764-21users20:15:223
1258599130,11cyclictest19407-21munin-node19:25:183
1258599130,11cyclictest15541-21hddtemp_smartct19:15:123
1258199137,5cyclictest0-21swapper/219:46:562
1258199137,1cyclictest508-21grep19:55:172
1258199132,7cyclictest0-21swapper/220:33:162
12581991311,1cyclictest0-21swapper/219:25:362
1258199130,12cyclictest0-21swapper/219:41:582
1258199130,12cyclictest0-21swapper/219:31:342
1258199130,12cyclictest0-21swapper/219:20:482
1258199130,11cyclictest8900-21irqstats20:15:162
1258199130,11cyclictest22749-21munin-node20:45:202
1257699139,3cyclictest10950irq/23-ehci_hcd19:45:071
1257699138,4cyclictest135650irq/27-eth019:35:581
1257699136,3cyclictest10950irq/23-ehci_hcd19:30:161
1257699130,12cyclictest32004-21ntp_states21:05:221
1257699130,12cyclictest0-21swapper/120:53:241
1257699130,12cyclictest0-21swapper/120:45:591
1257699130,12cyclictest0-21swapper/120:36:011
1257699130,12cyclictest0-21swapper/120:06:461
1257699130,12cyclictest0-21swapper/120:06:461
1257299136,4cyclictest0-21swapper/019:55:280
12572991311,1cyclictest0-21swapper/019:11:230
1257299130,3cyclictest0-21swapper/020:00:010
1257299130,1cyclictest0-21swapper/020:25:140
1257299130,1cyclictest0-21swapper/020:06:020
1257299130,1cyclictest0-21swapper/020:06:020
1257299130,13cyclictest0-21swapper/019:42:370
1257299130,12cyclictest0-21swapper/019:50:540
1257299130,12cyclictest0-21swapper/019:47:100
1257299130,12cyclictest0-21swapper/019:20:560
1257299130,10cyclictest24708-21hddtemp_smartct20:50:140
1257699120,11cyclictest0-21swapper/120:15:211
1257699120,11cyclictest0-21swapper/119:40:031
1257699120,11cyclictest0-21swapper/119:21:201
1257299121,1cyclictest0-21swapper/019:39:110
1257299120,3cyclictest0-21swapper/020:15:210
1257299120,2cyclictest0-21swapper/020:45:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional