You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 13:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Sat Jul 12, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137821640,3sleep12757499cyclictest23:24:261
137821640,3sleep12757499cyclictest23:24:251
2586221430,1sleep10-21swapper/123:33:001
440921420,3sleep30-21swapper/300:33:153
1577121420,2sleep22757899cyclictest00:06:522
130221420,2sleep30-21swapper/322:50:083
1272921410,4sleep12757499cyclictest22:29:271
2318821380,1sleep22757899cyclictest21:13:402
1178821340,2sleep30-21swapper/323:03:443
1480021330,1sleep30-21swapper/321:55:423
824521230,0sleep20-21swapper/221:49:032
1819921230,0sleep00-21swapper/023:19:520
2782521220,3sleep32758399cyclictest22:00:063
944721180,3sleep22757899cyclictest22:47:442
2330221160,1sleep236-21rcuop/221:59:422
3270921150,0sleep00-21swapper/021:11:260
2979821130,3sleep20-21swapper/222:12:322
1774321120,0sleep317745-21id00:17:153
2572921100,3sleep22757899cyclictest23:55:102
2467321040,1sleep02756999cyclictest23:48:410
27185210234,47sleep30-21swapper/319:08:173
2687221000,2sleep10-21swapper/100:11:161
222392990,2sleep20-21swapper/222:55:172
163752990,1sleep20-21swapper/223:47:522
98152960,1sleep12757499cyclictest22:50:511
10942960,1sleep01083-21sshd22:25:120
135512950,1sleep013530-21sshd00:23:340
294192940,2sleep10-21swapper/123:55:251
2528529235,53sleep10-21swapper/119:05:051
113752920,1sleep041ktimersoftd/022:57:200
129462910,2sleep22405-21runrttasks21:00:132
7002890,2sleep1694-21sshd21:17:361
68432890,2sleep20-21swapper/222:38:092
201442890,2sleep20-21swapper/200:20:432
209262880,1sleep30-21swapper/323:20:083
209262880,1sleep30-21swapper/323:20:083
154722880,1sleep20-21swapper/223:28:522
154722880,1sleep20-21swapper/223:28:522
51122870,2sleep30-21swapper/322:28:413
153412860,2sleep20-21swapper/200:03:362
107252860,3sleep02756999cyclictest21:15:260
6052840,2sleep12405-21runrttasks00:25:271
30622840,1sleep20-21swapper/222:06:552
297282840,2sleep20-21swapper/221:44:592
276192820,2sleep327616-21sshd22:40:113
320372800,4sleep22757899cyclictest22:15:452
36722790,2sleep30-21swapper/321:48:363
294052780,0sleep00-21swapper/022:34:100
195822780,1sleep10-21swapper/122:39:251
315012770,2sleep20-21swapper/223:52:292
313662770,3sleep32758399cyclictest00:08:303
20112770,0sleep00-21swapper/021:39:160
254902750,0sleep00-21swapper/022:24:290
136212750,1sleep20-21swapper/222:01:462
236582740,2sleep20-21swapper/222:27:202
193662740,2sleep30-21swapper/321:28:363
77532700,5sleep12757499cyclictest22:04:221
181122700,1sleep20-21swapper/222:33:052
301952660,2sleep10-21swapper/121:11:101
175002660,2sleep30-21swapper/323:25:533
175002660,2sleep30-21swapper/323:25:523
67202650,4sleep12757499cyclictest23:03:141
65022630,2sleep21848-21nfsd23:09:282
283752630,2sleep128376-21sshd00:21:371
157672600,3sleep22757899cyclictest23:35:142
205622590,4sleep10-21swapper/123:26:111
205622590,4sleep10-21swapper/123:26:111
87412570,2sleep31847-21nfsd22:16:383
2724825539,12sleep20-21swapper/219:09:032
212912550,1sleep10-21swapper/100:07:271
241902500,44sleep126950irq/26-ahci[00022:58:381
27569994835,5cyclictest16976-21bash00:06:590
2731024834,11sleep00-21swapper/019:09:500
206992410,11sleep12757499cyclictest22:45:341
286412400,2sleep128629-21sshd23:52:121
59442390,4sleep05933-21sshd23:12:260
27569993832,2cyclictest0-21swapper/021:23:450
255952380,0sleep00-21swapper/022:40:000
27569993715,12cyclictest3-21ksoftirqd/023:35:160
27569993634,1cyclictest0-21swapper/023:57:580
2756999350,3cyclictest11571-21sshd22:54:180
27574993414,7cyclictest20227-21sshd22:11:371
27569993416,16cyclictest3-21ksoftirqd/022:47:190
27569993216,9cyclictest3-21ksoftirqd/022:04:480
38582310,21sleep1135650irq/27-eth022:19:211
175252310,3sleep317516-21bash21:43:473
68692300,28sleep1135650irq/27-eth023:18:461
2756999300,27cyclictest1847-21nfsd00:16:330
27578992924,4cyclictest18585-21sshd21:37:442
27574992924,4cyclictest25786-21sshd21:23:041
152422290,0sleep315243-21bash23:35:113
278582280,26sleep3135650irq/27-eth023:30:073
27583992712,5cyclictest135650irq/27-eth022:08:353
27569992715,6cyclictest3-21ksoftirqd/023:06:050
152912270,3sleep30-21swapper/300:00:243
27574992624,1cyclictest0-21swapper/121:30:351
27574992612,2cyclictest4538-21sshd22:40:571
2756999268,11cyclictest3-21ksoftirqd/021:57:100
2756999265,13cyclictest1893-21nscd21:50:160
42462250,1sleep30-21swapper/322:34:533
278482250,0sleep227847-21sshd00:25:042
27574992521,3cyclictest17703-21id00:34:491
2756999259,7cyclictest3-21ksoftirqd/000:03:070
27569992515,5cyclictest3-21ksoftirqd/021:25:420
27569992513,11cyclictest3-21ksoftirqd/023:52:000
27569992512,6cyclictest3-21ksoftirqd/023:24:540
27569992512,6cyclictest3-21ksoftirqd/023:24:540
27569992511,7cyclictest3-21ksoftirqd/021:40:230
27569992511,4cyclictest3-21ksoftirqd/022:43:270
263562250,23sleep2135650irq/27-eth021:26:072
27583992411,4cyclictest135650irq/27-eth023:07:333
2757899240,3cyclictest29486-21sshd00:15:072
2757499249,6cyclictest16972-21sshd23:38:281
27574992413,8cyclictest1959-21sshd23:47:451
27569992415,7cyclictest3-21ksoftirqd/000:25:160
27569992414,8cyclictest3-21ksoftirqd/022:12:240
27569992413,5cyclictest3-21ksoftirqd/021:48:550
27569992413,3cyclictest3-21ksoftirqd/000:14:370
2756999240,23cyclictest30288-21id23:42:550
2756999240,15cyclictest0-21swapper/023:26:230
2756999240,15cyclictest0-21swapper/023:26:220
27578992312,2cyclictest10950irq/23-ehci_hcd21:54:182
27574992315,1cyclictest22-21ksoftirqd/123:07:581
2756999235,17cyclictest29603-21sshd00:36:040
27569992312,3cyclictest3-21ksoftirqd/021:34:460
27569992310,3cyclictest3-21ksoftirqd/023:02:470
27583992212,5cyclictest10950irq/23-ehci_hcd21:24:043
27583992210,3cyclictest2648-21sshd21:54:423
2757899229,5cyclictest20383-21sshd22:20:492
2757499227,9cyclictest135650irq/27-eth021:53:231
27574992211,3cyclictest24844-21sshd21:41:211
27569992216,5cyclictest3-21ksoftirqd/023:31:060
27569992211,2cyclictest28181-21sshd22:06:110
2756999220,17cyclictest7628-21sshd00:33:380
2756999220,12cyclictest2013-21bash22:19:100
257162220,2sleep31847-21nfsd00:21:203
124002220,1sleep20-21swapper/223:44:242
80742210,1sleep30-21swapper/300:37:203
2758399219,9cyclictest1848-21nfsd23:16:543
2758399219,3cyclictest15455-21bash23:41:263
2758399217,5cyclictest135650irq/27-eth022:22:053
2758399210,12cyclictest1847-21nfsd22:38:253
2757899214,11cyclictest1844-21nfsd21:24:352
27578992111,4cyclictest31154-21sshd23:02:262
2757499217,6cyclictest10950irq/23-ehci_hcd21:29:381
27574992115,4cyclictest18573-21sshd22:08:271
27574992114,3cyclictest15697-21bash00:00:251
27574992113,6cyclictest1959-21sshd21:57:551
269362210,1sleep20-21swapper/223:20:352
269362210,1sleep20-21swapper/223:20:342
65972200,3sleep20-21swapper/222:53:492
2758399208,3cyclictest0-21swapper/322:10:163
2758399203,14cyclictest22116-21sshd22:49:013
27583992015,4cyclictest10950irq/23-ehci_hcd21:33:313
27583992014,1cyclictest4531-21cp23:46:393
27583992012,2cyclictest10950irq/23-ehci_hcd23:50:163
27583992011,4cyclictest26950irq/26-ahci[00023:58:203
2757499209,5cyclictest10950irq/23-ehci_hcd23:41:121
27574992015,3cyclictest23435-21sshd21:47:231
27574992014,4cyclictest6296-21sshd00:19:261
27574992014,4cyclictest16567-21sshd00:38:191
27574992010,7cyclictest25639-21sshd22:30:361
2758399199,4cyclictest10950irq/23-ehci_hcd00:25:033
2758399196,2cyclictest9845-21sshd22:57:103
27583991910,4cyclictest10950irq/23-ehci_hcd21:15:233
27578991913,3cyclictest4663-21hwlatdetect21:30:142
2757899191,11cyclictest5286-21sshd22:41:022
27578991910,4cyclictest32274-21bash00:11:502
27574991913,4cyclictest0-21swapper/122:22:171
2757499190,15cyclictest2956-21sshd23:12:091
279212180,2sleep30-21swapper/321:38:383
27583991810,5cyclictest2235-21sshd23:12:053
2757899181,8cyclictest14377-21sshd23:19:302
2757899180,9cyclictest0-21swapper/200:37:232
2757499180,16cyclictest28717-21bash21:35:311
2758399170,7cyclictest0-21swapper/300:10:573
2758399170,14cyclictest25024-21sshd21:10:393
2757899173,3cyclictest13702-21sshd00:30:312
27578991711,2cyclictest10950irq/23-ehci_hcd23:10:212
2757899170,15cyclictest135650irq/27-eth021:17:082
27583991610,5cyclictest144550irq/28-eth1-rx-20:12:093
2758399160,14cyclictest0-21swapper/319:47:273
2757899160,14cyclictest27916-21sshd23:30:052
2757499160,14cyclictest0-21swapper/120:45:231
2758399150,14cyclictest0-21swapper/320:47:473
2758399150,13cyclictest0-21swapper/321:00:163
27578991511,3cyclictest10950irq/23-ehci_hcd20:10:212
2757899151,11cyclictest0-21swapper/220:44:462
2757499159,5cyclictest10950irq/23-ehci_hcd20:25:431
2756999157,7cyclictest0-21swapper/019:45:240
2756999157,5cyclictest0-21swapper/021:05:450
27569991513,1cyclictest0-21swapper/020:13:410
2756999150,12cyclictest0-21swapper/019:20:460
2758399146,6cyclictest0-21swapper/319:20:173
2758399143,10cyclictest10950irq/23-ehci_hcd20:39:423
2758399143,10cyclictest10950irq/23-ehci_hcd20:15:463
2758399143,10cyclictest10950irq/23-ehci_hcd19:40:083
27583991412,1cyclictest0-21swapper/319:27:253
2758399140,13cyclictest0-21swapper/319:51:243
2758399140,12cyclictest0-21swapper/319:35:223
2757899148,4cyclictest32146-21munin-node19:20:162
2757899147,6cyclictest0-21swapper/220:06:252
2757899143,10cyclictest10950irq/23-ehci_hcd20:49:032
2757899143,10cyclictest10950irq/23-ehci_hcd20:03:572
27578991412,1cyclictest0-21swapper/220:36:202
2757499149,4cyclictest135650irq/27-eth020:32:371
2757499144,9cyclictest2456-21strings20:35:191
2757499143,3cyclictest10950irq/23-ehci_hcd19:48:311
27574991410,3cyclictest10950irq/23-ehci_hcd19:54:331
2757499140,5cyclictest25505-21if_eth020:15:151
2757499140,5cyclictest0-21swapper/120:50:151
2757499140,13cyclictest17081-21runrttasks21:09:001
2757499140,13cyclictest0-21swapper/119:12:341
2757499140,12cyclictest29975-21atenall_r5power19:15:141
2757499140,12cyclictest27366-21df20:20:161
27569991412,1cyclictest0-21swapper/020:40:180
2756999141,10cyclictest0-21swapper/019:56:240
2756999140,6cyclictest3424-21find19:25:230
2756999140,13cyclictest0-21swapper/020:25:160
2756999140,13cyclictest0-21swapper/019:50:140
2756999140,13cyclictest0-21swapper/019:10:180
2758399136,6cyclictest10950irq/23-ehci_hcd20:53:153
2758399131,3cyclictest27307-21df20:20:153
27583991311,1cyclictest0-21swapper/320:57:123
2758399130,2cyclictest10950irq/23-ehci_hcd20:05:213
2758399130,12cyclictest0-21swapper/320:30:043
2758399130,12cyclictest0-21swapper/320:25:033
2758399130,11cyclictest16001-21df_inode19:55:153
2758399130,10cyclictest135650irq/27-eth019:32:403
2758399130,10cyclictest135650irq/27-eth019:20:003
2757899138,4cyclictest10950irq/23-ehci_hcd19:30:112
2757899138,3cyclictest2726-21hddtemp_smartct19:25:152
2757899131,5cyclictest0-21swapper/219:45:192
2757899130,4cyclictest8867-21munin-node19:40:252
2757899130,12cyclictest0-21swapper/221:05:182
2757899130,12cyclictest0-21swapper/220:55:462
2757899130,12cyclictest0-21swapper/220:31:512
2757899130,12cyclictest0-21swapper/220:25:282
2757899130,12cyclictest0-21swapper/219:55:382
2757899130,12cyclictest0-21swapper/219:37:382
2757899130,12cyclictest0-21swapper/219:19:442
2757899130,11cyclictest27929-21memory20:20:192
2757499139,3cyclictest10950irq/23-ehci_hcd19:41:071
2757499138,4cyclictest2405-21runrttasks19:34:521
2757499133,9cyclictest0-21swapper/119:25:111
2757499132,1cyclictest0-21swapper/119:58:001
2757499131,11cyclictest0-21swapper/121:00:241
27574991311,1cyclictest0-21swapper/119:36:341
2757499130,5cyclictest0-21swapper/120:40:141
2757499130,5cyclictest0-21swapper/119:20:331
2757499130,12cyclictest0-21swapper/120:55:061
2757499130,12cyclictest0-21swapper/120:14:031
2757499130,12cyclictest0-21swapper/120:06:131
2757499130,12cyclictest0-21swapper/120:01:541
2756999137,5cyclictest0-21swapper/020:05:260
2756999132,10cyclictest438-21kworker/0:020:56:370
27569991311,1cyclictest0-21swapper/020:31:490
2756999130,9cyclictest6910-21df_inode19:35:130
2756999130,3cyclictest0-21swapper/020:00:450
2756999130,1cyclictest0-21swapper/020:45:210
2756999130,12cyclictest0-21swapper/020:35:430
2756999130,12cyclictest0-21swapper/020:20:590
2756999130,12cyclictest0-21swapper/019:32:580
2756999130,12cyclictest0-21swapper/019:16:190
2756999130,11cyclictest0-21swapper/019:40:200
2758399122,9cyclictest0-21swapper/320:41:453
2758399120,11cyclictest10950irq/23-ehci_hcd21:05:203
2758399120,11cyclictest0-21swapper/320:00:253
2758399120,11cyclictest0-21swapper/319:13:483
2757899120,1cyclictest28465-21latency19:10:162
2757899120,11cyclictest25521-21iostat20:15:142
2757899120,11cyclictest14197-21grep19:50:182
2757899120,11cyclictest0-21swapper/220:50:082
2756999127,5cyclictest0-21swapper/021:01:130
2756999120,4cyclictest135650irq/27-eth020:50:400
2756999120,11cyclictest0-21swapper/020:15:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional