You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 10:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Sun Jul 13, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2341421680,4sleep3863499cyclictest23:31:003
3200121590,1sleep3863499cyclictest00:31:133
418421450,3sleep0862899cyclictest22:38:590
418421450,3sleep0862899cyclictest22:38:590
924721430,3sleep2863399cyclictest23:20:172
1528321420,0sleep215280-21sshd21:49:352
922821360,0sleep19213-21sshd00:35:521
191821340,3sleep31847-21nfsd23:10:123
1419921330,1sleep014200-21bash23:17:390
970521310,2sleep19702-21sshd23:17:121
3009121270,4sleep1863299cyclictest21:28:371
1673621250,1sleep016730-21sshd23:08:290
704221220,4sleep1863299cyclictest00:18:121
3062221220,2sleep2863399cyclictest23:54:282
2509421190,0sleep00-21swapper/021:34:370
2488021130,2sleep10-21swapper/123:31:111
1792421110,1sleep03-21ksoftirqd/000:36:510
1067421060,2sleep10-21swapper/123:36:151
2361521050,4sleep1863299cyclictest23:27:581
580921000,1sleep20-21swapper/221:23:052
225522970,1sleep20-21swapper/200:37:212
65232930,4sleep1863299cyclictest21:51:501
53492910,1sleep00-21swapper/021:29:250
265012900,1sleep0862899cyclictest23:34:420
820628835,49sleep10-21swapper/119:07:511
809828739,44sleep30-21swapper/319:06:403
34092870,2sleep20-21swapper/222:07:142
315442870,0sleep21844-21nfsd23:44:542
191082870,3sleep20-21swapper/223:05:302
265932860,2sleep30-21swapper/323:03:113
314182810,2sleep231419-21bash22:32:122
148622810,1sleep20-21swapper/221:52:402
99672800,4sleep3863499cyclictest22:20:303
278502800,1sleep20-21swapper/223:28:232
177252800,2sleep0862899cyclictest00:02:480
42622790,1sleep10-21swapper/100:21:171
193542790,2sleep219353-21bash00:26:192
274382780,2sleep20-21swapper/222:03:182
264742780,0sleep00-21swapper/023:47:330
205172780,2sleep120501-21sshd00:03:051
193202780,1sleep21848-21nfsd22:24:462
89312770,1sleep12405-21runrttasks22:51:551
275012770,1sleep227500-21sshd00:30:442
174372770,4sleep3863499cyclictest21:24:153
60232760,0sleep00-21swapper/021:39:070
301012760,2sleep130099-21sshd22:16:111
23472760,2sleep32343-21sshd22:10:203
225172760,2sleep322511-21sshd00:16:183
253952730,2sleep10-21swapper/123:57:041
231612730,1sleep023154-21sshd23:12:140
322212720,2sleep10-21swapper/120:00:181
302912700,4sleep2863399cyclictest21:35:082
209502690,2sleep10-21swapper/123:08:541
161982690,3sleep1863299cyclictest22:11:431
93932660,2sleep39392-21bash22:39:303
93932660,2sleep39392-21bash22:39:303
97672650,3sleep3863499cyclictest23:29:483
249392650,1sleep20-21swapper/222:53:322
48922640,2sleep34890-21sshd23:42:113
634425833,4sleep00-21swapper/019:05:130
212772530,0sleep00-21swapper/000:26:330
836125136,10sleep20-21swapper/219:09:482
322272440,1sleep132218-21rm22:35:231
322272440,1sleep132218-21rm22:35:221
862899390,31cyclictest10611-21sshd23:26:410
220532370,34sleep1135650irq/27-eth023:40:371
28652350,0sleep00-21swapper/022:29:290
28652350,0sleep00-21swapper/022:29:280
862899340,33cyclictest0-21swapper/000:18:440
292652320,27sleep1135650irq/27-eth023:12:511
288232322,28sleep2135650irq/27-eth023:15:532
157842320,2sleep11848-21nfsd22:27:341
157842320,2sleep11848-21nfsd22:27:341
863399306,22cyclictest33-21ksoftirqd/223:37:072
10922300,28sleep3135650irq/27-eth021:48:093
233012290,27sleep3135650irq/27-eth023:05:563
8633992811,6cyclictest22264-21sshd00:19:502
863399280,26cyclictest11051-21sshd22:42:422
100252280,26sleep3135650irq/27-eth022:52:023
8634992724,2cyclictest0-21swapper/323:45:413
863299270,25cyclictest23895-21sshd22:47:031
862899272,14cyclictest0-21swapper/023:38:090
862899271,19cyclictest9534-21cp22:20:280
312322270,1sleep031234-21sshd00:13:540
8634992623,1cyclictest0-21swapper/321:56:523
863399265,19cyclictest33-21ksoftirqd/222:37:172
863399265,19cyclictest33-21ksoftirqd/222:37:172
863399260,23cyclictest33-21ksoftirqd/223:11:222
862899269,9cyclictest0-21swapper/022:15:220
8633992513,6cyclictest25-21rcuop/100:07:042
8632992511,5cyclictest542-21sshd22:44:521
309652250,1sleep00-21swapper/021:54:180
863399245,2cyclictest10950irq/23-ehci_hcd23:45:232
863399240,21cyclictest33-21ksoftirqd/200:00:032
8632992418,3cyclictest0-21swapper/122:06:591
862899240,17cyclictest0-21swapper/023:59:170
257402240,2sleep225741-21sshd21:18:482
144952240,22sleep3135650irq/27-eth000:25:493
863499238,3cyclictest10950irq/23-ehci_hcd00:20:053
863499237,8cyclictest10950irq/23-ehci_hcd21:34:413
863399233,17cyclictest33-21ksoftirqd/223:55:522
8633992315,4cyclictest8263-21sshd00:11:322
8633992310,11cyclictest28847-21rm22:57:062
863299235,9cyclictest2405-21runrttasks00:06:541
863299230,9cyclictest0-21swapper/100:28:211
117762230,3sleep1863299cyclictest21:36:241
8634992216,3cyclictest0-21swapper/323:57:173
8634992211,7cyclictest135650irq/27-eth023:23:193
863399227,1cyclictest33-21ksoftirqd/221:11:372
8633992216,4cyclictest21573-21sshd21:40:272
8633992216,4cyclictest12132-21sshd23:30:012
8633992214,2cyclictest0-21swapper/221:29:042
8633992213,7cyclictest13565-21sshd22:11:252
8633992210,8cyclictest1959-21sshd21:31:052
862899222,12cyclictest23877-21sshd23:24:560
862899220,2cyclictest19388-21sshd00:33:320
30712220,19sleep3135650irq/27-eth021:16:393
176612220,3sleep0862899cyclictest22:05:240
863499219,0cyclictest0-21swapper/323:37:463
863499216,12cyclictest10950irq/23-ehci_hcd21:28:113
8634992111,3cyclictest10950irq/23-ehci_hcd00:05:263
863399212,18cyclictest33-21ksoftirqd/200:22:132
8633992117,2cyclictest17446-21sshd23:02:162
8633992116,3cyclictest32186-21sshd22:29:112
8633992116,3cyclictest32186-21sshd22:29:102
863399210,19cyclictest33-21ksoftirqd/221:55:392
863299214,4cyclictest26580-21sshd00:30:371
8632992114,5cyclictest14494-21sshd00:12:111
30212210,19sleep1135650irq/27-eth023:51:401
12982210,4sleep2863399cyclictest22:19:492
863499207,3cyclictest27103-21id00:00:243
8634992014,4cyclictest26628-21bash22:41:013
8634992012,4cyclictest0-21swapper/323:52:363
8634992010,1cyclictest0-21swapper/322:19:223
8633992014,2cyclictest10950irq/23-ehci_hcd19:30:182
863299208,2cyclictest20902-21bash21:12:081
8632992015,3cyclictest9451-21sshd21:45:401
8632992013,5cyclictest28886-21sshd21:44:301
862899200,2cyclictest0-21swapper/021:57:110
862899200,19cyclictest0-21swapper/023:54:590
160522200,1sleep016051-21seq23:02:080
863499195,5cyclictest135650irq/27-eth023:15:203
863499190,8cyclictest23009-21bash00:37:253
8633991914,3cyclictest17794-21sshd22:46:252
863299198,4cyclictest1288-21sshd21:22:381
8632991915,3cyclictest1848-21nfsd23:48:031
863299190,9cyclictest0-21swapper/123:20:121
862899191,8cyclictest30627-21kworker/0:021:11:460
8628991913,5cyclictest3-21ksoftirqd/023:43:210
862899190,1cyclictest0-21swapper/019:45:160
278692190,16sleep3135650irq/27-eth022:00:143
223332190,2sleep122322-21sshd22:59:401
8634991813,4cyclictest1893-21nscd21:35:163
863299188,2cyclictest10950irq/23-ehci_hcd22:30:511
8632991814,2cyclictest0-21swapper/122:21:461
862899183,13cyclictest0-21swapper/022:12:520
862899180,12cyclictest10985-21perl20:25:140
863499178,7cyclictest28174-21sshd21:41:083
863499177,4cyclictest135650irq/27-eth000:13:193
863499176,1cyclictest0-21swapper/322:29:403
863499176,1cyclictest0-21swapper/322:29:393
863499173,7cyclictest1846-21nfsd22:06:463
863499170,15cyclictest0-21swapper/320:00:233
863299178,7cyclictest29701-21sshd21:31:481
863299178,4cyclictest22492-21sshd21:18:311
863299171,11cyclictest1959-21sshd22:00:221
862899171,12cyclictest21772-21sshd21:43:460
8628991710,1cyclictest479-21sshd22:03:490
862899170,13cyclictest0-21swapper/022:41:360
214092170,1sleep20-21swapper/220:49:092
214092170,1sleep20-21swapper/220:49:092
863499161,5cyclictest1846-21nfsd22:56:433
8634991612,3cyclictest14612-21sshd22:30:303
8634991611,3cyclictest0-21swapper/322:45:413
863399163,11cyclictest0-21swapper/220:11:132
8633991611,4cyclictest10950irq/23-ehci_hcd19:22:342
863299168,5cyclictest135650irq/27-eth023:00:511
8632991612,3cyclictest135650irq/27-eth021:55:201
862899166,2cyclictest23444-21sshd21:47:070
8628991612,2cyclictest18127-21sshd21:15:050
862899160,9cyclictest0-21swapper/022:31:080
862899160,9cyclictest0-21swapper/019:15:240
862899160,15cyclictest6820-21id22:54:590
862899160,12cyclictest7605-21sshd00:05:040
862899160,10cyclictest0-21swapper/000:20:040
12672160,14sleep310950irq/23-ehci_hcd21:13:253
123292161,13sleep310950irq/23-ehci_hcd19:15:263
863499155,2cyclictest21192-21bash21:53:183
8634991511,1cyclictest0-21swapper/320:18:293
863399153,11cyclictest10950irq/23-ehci_hcd19:44:172
863399151,11cyclictest0-21swapper/220:17:422
8632991511,3cyclictest10950irq/23-ehci_hcd21:05:181
8632991511,3cyclictest10950irq/23-ehci_hcd19:56:311
862899156,1cyclictest0-21swapper/022:57:440
862899155,9cyclictest7952-21sshd22:45:250
862899150,12cyclictest0-21swapper/020:59:420
8634991410,3cyclictest10950irq/23-ehci_hcd21:04:273
8634991410,3cyclictest10950irq/23-ehci_hcd20:53:443
863499140,1cyclictest1848-21nfsd19:21:523
863399143,3cyclictest10950irq/23-ehci_hcd20:24:502
863399141,12cyclictest135650irq/27-eth021:01:432
863399141,12cyclictest10950irq/23-ehci_hcd20:29:222
8633991410,3cyclictest10950irq/23-ehci_hcd19:19:212
8633991410,2cyclictest0-21swapper/220:55:322
863399140,12cyclictest32535-21pmu-power20:00:222
863299143,10cyclictest10950irq/23-ehci_hcd20:39:581
8632991410,3cyclictest10950irq/23-ehci_hcd19:42:091
862899147,5cyclictest0-21swapper/020:31:430
862899147,2cyclictest0-21swapper/020:12:100
862899140,7cyclictest1959-21sshd21:22:350
862899140,1cyclictest21293-21smart_sda19:35:240
862899140,13cyclictest0-21swapper/020:52:390
862899140,13cyclictest0-21swapper/020:05:170
862899140,12cyclictest0-21swapper/021:01:460
862899140,10cyclictest0-21swapper/020:15:190
862899140,10cyclictest0-21swapper/019:20:160
863499139,3cyclictest135650irq/27-eth019:51:493
863499137,5cyclictest0-21swapper/320:10:183
8634991311,1cyclictest0-21swapper/321:05:563
863499130,12cyclictest21163-21open_files19:35:223
863499130,12cyclictest1848-21nfsd19:55:163
863499130,12cyclictest0-21swapper/320:47:033
863499130,12cyclictest0-21swapper/320:47:023
863499130,12cyclictest0-21swapper/320:40:413
863499130,12cyclictest0-21swapper/320:35:333
863499130,12cyclictest0-21swapper/320:31:583
863499130,12cyclictest0-21swapper/320:25:193
863499130,12cyclictest0-21swapper/320:05:183
863499130,12cyclictest0-21swapper/319:46:123
863499130,12cyclictest0-21swapper/319:43:173
863499130,11cyclictest18941-21munin-node19:30:213
863399138,3cyclictest20269-21df19:35:142
863399137,2cyclictest10950irq/23-ehci_hcd19:45:152
863399132,3cyclictest135650irq/27-eth020:50:232
8633991311,1cyclictest0-21swapper/221:07:372
8633991311,1cyclictest0-21swapper/220:38:332
863399131,11cyclictest0-21swapper/220:30:192
863399130,4cyclictest8653-21munin-node19:10:232
863399130,12cyclictest0-21swapper/220:05:272
863399130,12cyclictest0-21swapper/219:57:042
863399130,11cyclictest17344-21grep20:40:142
863399130,11cyclictest15788-21diskstats19:25:142
863299139,3cyclictest144550irq/28-eth1-rx-19:21:131
863299138,3cyclictest0-21swapper/120:10:221
863299137,5cyclictest1847-21nfsd21:01:511
863299132,10cyclictest0-21swapper/119:50:421
863299130,5cyclictest0-21swapper/120:32:001
863299130,4cyclictest9243-21grep19:10:171
863299130,12cyclictest18842-21sshd20:40:311
863299130,12cyclictest1848-21nfsd20:55:121
863299130,12cyclictest0-21swapper/120:25:481
863299130,12cyclictest0-21swapper/120:16:301
863299130,12cyclictest0-21swapper/120:05:021
863299130,11cyclictest21085-21users20:45:251
863299130,11cyclictest21085-21users20:45:251
863299130,11cyclictest15727-21munin-node19:25:131
863299130,11cyclictest0-21swapper/120:20:151
8628991311,1cyclictest0-21swapper/020:00:220
862899130,1cyclictest0-21swapper/020:23:190
862899130,1cyclictest0-21swapper/019:28:550
862899130,12cyclictest0-21swapper/021:06:550
862899130,12cyclictest0-21swapper/020:45:220
862899130,12cyclictest0-21swapper/020:45:220
862899130,12cyclictest0-21swapper/020:40:490
862899130,12cyclictest0-21swapper/019:31:140
863499121,10cyclictest1131-21jbd2/dm-2-820:20:063
863499120,11cyclictest8764-21atenall_r5power19:10:143
863499120,11cyclictest0-21swapper/319:25:283
863499120,10cyclictest0-21swapper/320:55:153
863399121,10cyclictest10950irq/23-ehci_hcd19:50:212
863299120,11cyclictest0-21swapper/120:50:021
863299120,10cyclictest19024-21munin-node19:30:221
863299120,10cyclictest0-21swapper/119:45:141
863299120,10cyclictest0-21swapper/119:35:151
863299120,10cyclictest0-21swapper/119:15:151
862899127,5cyclictest0-21swapper/020:37:200
862899127,4cyclictest0-21swapper/019:55:160
862899120,7cyclictest0-21swapper/019:50:250
862899120,11cyclictest0-21swapper/019:11:420
862899120,10cyclictest0-21swapper/019:40:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional