You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-29 - 02:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Sun Jun 29, 2025 00:44:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2844721460,4sleep02341399cyclictest00:38:300
2580421400,1sleep30-21swapper/323:04:233
106221380,2sleep11848-21nfsd23:32:391
1149621320,3sleep22341599cyclictest00:07:472
1571321280,4sleep32341699cyclictest00:33:453
1236721110,1sleep20-21swapper/221:40:332
23050210934,46sleep10-21swapper/119:08:391
120652990,0sleep00-21swapper/000:23:230
236792980,2sleep02341399cyclictest23:37:540
2307929729,39sleep30-21swapper/319:09:013
298672950,2sleep041ktimersoftd/000:09:320
177782950,1sleep017780-21sshd22:11:290
177782950,1sleep017780-21sshd22:11:290
49972930,1sleep04995-21bash22:01:070
287872920,1sleep028774-21sshd22:30:450
154182910,5sleep02341399cyclictest22:47:520
32422900,5sleep02341399cyclictest21:12:150
259082890,2sleep30-21swapper/323:16:373
78262880,2sleep10-21swapper/123:51:511
110572860,3sleep20-21swapper/222:47:272
314602850,1sleep231458-21sshd22:55:322
231052850,2sleep20-21swapper/222:27:112
145452840,7sleep03-21ksoftirqd/023:55:350
80342810,2sleep30-21swapper/321:31:003
208322810,2sleep10-21swapper/122:02:411
150362810,4sleep32341699cyclictest23:58:543
62212800,1sleep30-21swapper/323:27:003
4422790,4sleep20-21swapper/223:08:062
324942790,1sleep30-21swapper/319:30:003
321402790,1sleep21847-21nfsd23:47:592
237842790,2sleep10-21swapper/122:42:281
185422790,2sleep218541-21sshd23:37:242
12972790,1sleep00-21swapper/022:25:130
88952780,2sleep38886-21sshd22:25:483
28092770,1sleep10-21swapper/122:49:501
2278527735,11sleep20-21swapper/219:05:322
167612770,0sleep00-21swapper/023:43:280
136932770,2sleep213690-21bash21:16:172
227302740,2sleep322728-21sshd00:11:533
124732730,4sleep12341499cyclictest00:14:031
151892700,4sleep12341499cyclictest23:03:201
78172690,1sleep12341499cyclictest21:52:191
12152670,2sleep11205-21sshd21:36:321
71452660,1sleep10-21swapper/123:08:461
295192660,4sleep210950irq/23-ehci_hcd22:43:012
295932640,2sleep30-21swapper/300:38:373
276732630,2sleep327672-21sshd00:06:113
2289726237,13sleep00-21swapper/019:06:540
224892590,1sleep30-21swapper/300:18:023
23413994115,15cyclictest3-21ksoftirqd/022:41:300
2341399384,30cyclictest3-21ksoftirqd/023:48:390
23413993614,19cyclictest3-21ksoftirqd/000:14:110
23413993611,20cyclictest3-21ksoftirqd/023:31:330
108952360,2sleep210896-21sshd23:58:282
2341399359,20cyclictest3-21ksoftirqd/023:03:140
129592350,1sleep10-21swapper/122:53:551
238762340,2sleep223869-21sshd22:21:092
2341499335,18cyclictest22-21ksoftirqd/122:31:381
124632330,1sleep00-21swapper/023:15:220
23414993230,1cyclictest1848-21nfsd21:58:271
23413993213,17cyclictest3-21ksoftirqd/022:55:440
23413993213,17cyclictest3-21ksoftirqd/000:28:280
189122320,1sleep00-21swapper/023:28:130
239552310,5sleep30-21swapper/321:50:443
23413993012,15cyclictest3-21ksoftirqd/000:19:050
23413993011,18cyclictest3-21ksoftirqd/022:07:440
23413993011,16cyclictest3-21ksoftirqd/023:07:470
23415992921,6cyclictest1927-21snmpd21:33:452
2341399292,26cyclictest5956-21sshd22:16:280
2341399292,26cyclictest5956-21sshd22:16:270
23413992917,10cyclictest3-21ksoftirqd/022:20:030
23413992912,13cyclictest3-21ksoftirqd/023:52:000
200192290,2sleep220016-21sshd22:51:232
75112280,26sleep1135650irq/27-eth000:39:451
52922280,4sleep22341599cyclictest00:32:362
23413992810,12cyclictest3-21ksoftirqd/021:26:210
223392280,2sleep10-21swapper/122:21:011
23413992712,12cyclictest3-21ksoftirqd/023:11:430
2341499260,6cyclictest7397-21sshd21:15:391
2341399261,2cyclictest6098-21sshd00:01:030
23413992612,8cyclictest3-21ksoftirqd/021:34:430
23413992610,13cyclictest3-21ksoftirqd/022:39:090
152972260,24sleep3135650irq/27-eth021:44:013
138202260,3sleep10-21swapper/122:14:161
138202260,3sleep10-21swapper/122:14:151
23416992512,6cyclictest10950irq/23-ehci_hcd23:21:063
2341399255,17cyclictest7923-21sshd21:15:410
2341399254,18cyclictest3-21ksoftirqd/021:22:050
2341399252,2cyclictest3-21ksoftirqd/000:30:040
2341399251,3cyclictest0-21swapper/023:23:070
2341499242,7cyclictest2619-21sshd00:19:141
23414992411,3cyclictest22-21ksoftirqd/123:15:191
2341399247,14cyclictest3-21ksoftirqd/021:40:290
104332240,21sleep3135650irq/27-eth022:16:553
104332240,21sleep3135650irq/27-eth022:16:543
103382240,1sleep210330-21sshd21:49:362
23416992311,2cyclictest10950irq/23-ehci_hcd21:21:223
2341499234,17cyclictest22-21ksoftirqd/122:55:361
2341499233,17cyclictest22-21ksoftirqd/123:48:031
23414992315,2cyclictest135650irq/27-eth023:28:281
2341399230,2cyclictest41ktimersoftd/022:50:200
219972230,2sleep221995-21bash23:19:262
98242220,0sleep00-21swapper/021:46:240
2341699224,7cyclictest1847-21nfsd23:11:483
2341499221,9cyclictest26950irq/26-ahci[00000:23:581
23414992216,3cyclictest135650irq/27-eth022:16:271
23414992216,3cyclictest135650irq/27-eth022:16:271
23414992212,7cyclictest26845-21sshd23:10:301
23413992214,5cyclictest3-21ksoftirqd/021:51:510
23413992213,7cyclictest3-21ksoftirqd/021:58:520
252782214,15sleep310950irq/23-ehci_hcd21:38:513
2341499212,16cyclictest22-21ksoftirqd/122:28:391
23414992110,3cyclictest8569-21rm21:34:141
2341499210,15cyclictest22-21ksoftirqd/123:41:531
2341399210,15cyclictest20063-21sshd21:38:200
59562200,2sleep25938-21sshd21:21:412
23416992010,2cyclictest29372-21sshd22:06:303
2341599201,9cyclictest27989-21sshd00:25:012
23415992012,5cyclictest31493-21sshd22:15:482
23415992012,5cyclictest31493-21sshd22:15:472
23415992012,2cyclictest135650irq/27-eth000:35:082
2341499206,3cyclictest18006-21bash00:02:131
2341499201,18cyclictest22-21ksoftirqd/121:21:431
223192200,2sleep222309-21sshd22:02:492
201862200,18sleep2135650irq/27-eth022:39:102
23416991912,5cyclictest30929-21bash23:38:383
2341599196,6cyclictest135650irq/27-eth023:44:322
2341599195,3cyclictest3102-21bash21:12:132
2341499192,3cyclictest24709-21sshd21:29:381
218002190,17sleep1135650irq/27-eth023:37:451
82722181,15sleep1135650irq/27-eth023:24:151
23416991814,2cyclictest9348-21bash00:23:063
2341699180,16cyclictest31040-21bash22:24:593
2341599187,5cyclictest135650irq/27-eth000:20:272
2341599184,3cyclictest27129-21sshd22:09:252
23415991812,3cyclictest1959-21sshd23:22:262
23415991812,2cyclictest0-21swapper/223:03:082
2341599181,15cyclictest24255-21sshd00:12:012
2341499189,3cyclictest0-21swapper/122:06:321
23414991812,4cyclictest11539-21sshd23:55:201
2341399180,17cyclictest0-21swapper/019:21:050
2341399180,14cyclictest0-21swapper/020:17:360
2341399180,11cyclictest0-21swapper/020:20:180
220362180,2sleep222016-21sshd21:35:242
306682170,15sleep1135650irq/27-eth021:48:271
2341699178,4cyclictest135650irq/27-eth023:09:373
2341699173,12cyclictest12427-21sshd22:10:583
2341699173,12cyclictest12427-21sshd22:10:583
23416991714,2cyclictest20663-21sshd22:33:053
23416991713,3cyclictest0-21swapper/323:42:133
23416991711,1cyclictest135650irq/27-eth023:50:553
2341699170,5cyclictest29684-21sshd22:40:033
2341699170,14cyclictest22189-21sshd22:39:203
2341699170,13cyclictest0-21swapper/300:01:393
2341699170,11cyclictest14246-21id21:59:043
2341599179,4cyclictest10950irq/23-ehci_hcd23:34:512
2341599178,4cyclictest23514-21sshd21:56:502
2341599178,1cyclictest10950irq/23-ehci_hcd22:11:542
2341599178,1cyclictest10950irq/23-ehci_hcd22:11:532
2341599177,8cyclictest0-21swapper/200:17:022
2341599175,6cyclictest10950irq/23-ehci_hcd00:03:042
2341599173,9cyclictest5740-21bash23:51:372
2341599172,4cyclictest16389-21sshd22:32:422
2341499179,3cyclictest10950irq/23-ehci_hcd00:05:091
2341499171,15cyclictest22-21ksoftirqd/121:14:391
2341499171,14cyclictest22-21ksoftirqd/121:40:081
2341399178,8cyclictest0-21swapper/020:54:350
2341399177,9cyclictest17345-21sensors20:05:250
2341399177,9cyclictest0-21swapper/021:09:030
2341399170,4cyclictest0-21swapper/019:16:590
2341399170,16cyclictest0-21swapper/021:00:240
17982172,13sleep3144550irq/28-eth1-rx-19:31:053
23416991610,4cyclictest4840-21bash22:56:063
2341699160,8cyclictest17045-21sshd21:19:473
2341699160,7cyclictest433-21bash22:00:403
2341699160,14cyclictest16840-21bash23:46:273
2341699160,14cyclictest1340-21sshd21:45:363
2341699160,14cyclictest0-21swapper/321:14:183
2341699160,11cyclictest6962-21sshd23:30:103
2341599169,6cyclictest135650irq/27-eth021:25:412
2341599163,11cyclictest29559-21sshd21:51:162
23415991611,3cyclictest20882-21memory23:25:212
23415991610,1cyclictest1848-21nfsd23:11:192
2341499169,4cyclictest0-21swapper/100:25:201
2341499167,7cyclictest0-21swapper/120:53:231
2341499162,6cyclictest4939-21sshd00:32:331
2341499162,12cyclictest1959-21sshd22:35:121
2341499160,14cyclictest0-21swapper/119:40:161
2341399161,3cyclictest0-21swapper/020:31:090
2341399160,1cyclictest0-21swapper/020:04:020
2341399160,1cyclictest0-21swapper/019:26:270
2341399160,15cyclictest0-21swapper/020:59:070
2341399160,15cyclictest0-21swapper/020:40:120
2341399160,15cyclictest0-21swapper/019:50:360
2341399160,12cyclictest0-21swapper/020:46:390
2341699157,6cyclictest4691-21munin-node19:40:163
2341699157,3cyclictest0-21swapper/322:51:063
2341699153,9cyclictest30792-21sshd21:27:073
2341699153,1cyclictest0-21swapper/320:33:183
2341699151,5cyclictest30908-21fschecks_count00:25:183
23416991511,3cyclictest10950irq/23-ehci_hcd20:17:133
2341699150,6cyclictest0-21swapper/321:03:463
2341699150,13cyclictest0-21swapper/320:54:433
2341699150,10cyclictest0-21swapper/322:45:093
23415991511,3cyclictest10950irq/23-ehci_hcd19:31:082
2341599150,13cyclictest0-21swapper/220:15:182
2341499157,6cyclictest0-21swapper/119:45:181
2341499154,3cyclictest10950irq/23-ehci_hcd19:16:221
2341499150,6cyclictest0-21swapper/119:55:211
2341399159,5cyclictest0-21swapper/019:58:230
2341399157,7cyclictest1131-21jbd2/dm-2-819:41:360
23413991511,3cyclictest0-21swapper/020:13:560
2341399150,5cyclictest0-21swapper/019:38:470
2341399150,4cyclictest0-21swapper/019:10:170
2341399150,1cyclictest0-21swapper/020:36:390
2341399150,12cyclictest0-21swapper/019:30:290
2341699143,10cyclictest10950irq/23-ehci_hcd20:36:013
23416991410,3cyclictest10950irq/23-ehci_hcd20:27:023
23416991410,3cyclictest10950irq/23-ehci_hcd20:27:013
2341699140,13cyclictest0-21swapper/320:45:123
2341699140,10cyclictest0-21swapper/320:55:113
2341599143,10cyclictest10950irq/23-ehci_hcd20:49:362
2341599141,4cyclictest10463-21ntpq21:05:212
2341599141,3cyclictest16928-21ls20:05:222
23415991410,3cyclictest10950irq/23-ehci_hcd19:55:272
2341499149,4cyclictest0-21swapper/119:50:141
2341499143,10cyclictest10950irq/23-ehci_hcd20:32:011
2341499143,10cyclictest10950irq/23-ehci_hcd20:14:291
2341499143,10cyclictest10950irq/23-ehci_hcd19:36:491
2341499140,5cyclictest9072-21chrt21:04:531
2341499140,13cyclictest0-21swapper/120:05:271
2341499140,12cyclictest1804-21munin-node20:45:201
23413991410,2cyclictest0-21swapper/019:46:080
2341399140,1cyclictest0-21swapper/020:26:000
2341399140,1cyclictest0-21swapper/020:25:590
231992141,11sleep1135650irq/27-eth020:20:151
2341699132,10cyclictest10950irq/23-ehci_hcd21:08:533
23416991311,1cyclictest0-21swapper/320:41:263
23416991311,1cyclictest0-21swapper/319:20:523
2341699130,12cyclictest0-21swapper/320:12:263
2341699130,12cyclictest0-21swapper/320:06:523
2341699130,12cyclictest0-21swapper/320:00:233
2341699130,12cyclictest0-21swapper/319:59:403
2341699130,11cyclictest8341-21munin-node19:45:233
2341699130,11cyclictest3180-21iostat_ios19:35:193
2341699130,10cyclictest10950irq/23-ehci_hcd19:16:453
2341599139,3cyclictest135650irq/27-eth020:25:032
2341599139,3cyclictest135650irq/27-eth020:25:032
2341599138,3cyclictest30446-21iostat20:35:172
2341599137,5cyclictest0-21swapper/220:12:342
2341599137,5cyclictest0-21swapper/219:14:022
2341599133,9cyclictest0-21swapper/220:54:392
2341599131,3cyclictest30575-21diskstats19:25:172
2341599131,3cyclictest26845-21ntp_states19:15:202
23415991311,1cyclictest0-21swapper/219:48:422
2341599130,4cyclictest31963-21munin-node20:40:162
2341599130,3cyclictest135650irq/27-eth019:20:582
2341599130,12cyclictest28115-21hwlatdetect20:30:162
2341599130,12cyclictest158072chrt20:04:522
2341599130,12cyclictest0-21swapper/221:00:212
2341599130,12cyclictest0-21swapper/220:55:282
2341599130,12cyclictest0-21swapper/219:42:252
2341599130,12cyclictest0-21swapper/219:36:002
2341599130,11cyclictest10747-21unixbench-2d19:50:232
2341499139,3cyclictest1848-21nfsd20:27:211
2341499139,3cyclictest1848-21nfsd20:27:201
2341499139,3cyclictest135650irq/27-eth019:26:291
2341499139,1cyclictest23412-21cyclictest19:21:051
2341499131,11cyclictest367-21munin-node20:40:191
2341499130,11cyclictest5440-21hddtemp_smartct20:55:121
2341499130,11cyclictest0-21swapper/121:05:181
2341699120,4cyclictest9536-21munin-node19:50:133
2341699120,11cyclictest135650irq/27-eth020:20:143
2341699120,11cyclictest0-21swapper/319:10:113
2341599128,3cyclictest23001-21cpu20:20:142
2341499124,4cyclictest10950irq/23-ehci_hcd20:00:181
2341499121,2cyclictest819-21munin-node19:30:141
2341499120,3cyclictest30833-21ntp_states20:35:211
2341499120,11cyclictest24189-21iostat19:10:151
2341499120,11cyclictest20857-21df20:15:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional