You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-31 - 14:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Fri Oct 31, 2025 12:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2138521810,2sleep321380-21sshd09:51:123
2199321570,1sleep30-21swapper/309:31:123
1951921420,2sleep30-21swapper/311:45:423
1293721410,1sleep12024599cyclictest11:32:181
1851021400,1sleep30-21swapper/310:04:143
2715921380,1sleep327156-21sshd10:30:563
254921370,3sleep12024599cyclictest10:31:461
2190321240,2sleep10-21swapper/112:36:191
2576421220,0sleep00-21swapper/010:57:380
2005921130,2sleep120056-21sshd09:10:541
200372940,2sleep30-21swapper/312:21:513
155292880,1sleep12024599cyclictest09:50:341
55032870,2sleep20-21swapper/211:02:052
325632870,4sleep02024499cyclictest11:17:500
295332870,1sleep229535-21id12:11:112
28482870,2sleep20-21swapper/210:48:392
248782870,2sleep10-21swapper/110:37:361
248782870,2sleep10-21swapper/110:37:351
1998628632,49sleep30-21swapper/307:09:573
1890128633,43sleep10-21swapper/107:05:221
59402840,1sleep00-21swapper/011:34:580
51032840,2sleep35095-21sshd11:15:053
174372840,1sleep20-21swapper/212:33:072
169722840,2sleep10-21swapper/111:48:411
268712830,2sleep226869-21sshd09:25:142
174052830,1sleep10-21swapper/110:07:181
105862830,3sleep32024799cyclictest12:09:353
288372820,5sleep10-21swapper/111:20:311
178032820,2sleep10-21swapper/109:20:481
202642790,2sleep10-21swapper/110:50:221
162602780,0sleep00-21swapper/011:39:140
67292770,2sleep20-21swapper/211:11:552
232812770,2sleep223273-21sshd11:39:562
174242770,1sleep10-21swapper/112:13:021
142572760,2sleep10-21swapper/111:15:501
1987427333,11sleep20-21swapper/207:08:522
192722730,2sleep30-21swapper/309:24:263
184402730,3sleep02024499cyclictest10:40:150
313062720,4sleep32024799cyclictest10:15:083
149732720,1sleep010-21rcuc/011:09:370
295572700,2sleep20-21swapper/207:30:142
92512680,2sleep12024599cyclictest11:44:521
22112670,1sleep31847-21nfsd12:34:423
249852610,2sleep30-21swapper/312:01:463
1974425933,14sleep00-21swapper/007:07:130
20244994239,2cyclictest5134-21sshd12:23:280
52242360,34sleep3135650irq/27-eth010:22:123
20245993313,17cyclictest22-21ksoftirqd/112:22:371
20244993312,16cyclictest3-21ksoftirqd/011:41:210
20245993112,18cyclictest22-21ksoftirqd/109:15:321
185262310,29sleep3135650irq/27-eth008:15:203
241732300,3sleep02024499cyclictest09:54:540
20244993012,12cyclictest3-21ksoftirqd/012:33:550
110162300,4sleep02024499cyclictest10:06:360
2024599291,19cyclictest22-21ksoftirqd/110:57:561
2024599291,18cyclictest22-21ksoftirqd/110:13:021
20244992923,5cyclictest3-21ksoftirqd/012:39:510
20244992911,5cyclictest3-21ksoftirqd/010:31:060
114482290,1sleep00-21swapper/011:54:210
63462280,2sleep26345-21sshd09:36:162
20246992823,4cyclictest14101-21id10:13:222
2024599288,17cyclictest22-21ksoftirqd/110:46:451
20245992820,4cyclictest1848-21nfsd12:16:291
20245992819,7cyclictest1959-21sshd11:37:351
20245992819,5cyclictest3545-21sshd12:34:481
2024499280,15cyclictest0-21swapper/011:47:130
20244992717,6cyclictest8-21rcu_preempt09:42:340
20244992713,6cyclictest3-21ksoftirqd/010:27:390
20244992713,5cyclictest3-21ksoftirqd/011:04:090
20244992712,10cyclictest3-21ksoftirqd/012:05:050
20245992617,6cyclictest1848-21nfsd09:35:101
20245992614,7cyclictest305-21sshd12:25:481
20245992613,6cyclictest22-21ksoftirqd/111:06:461
20244992612,5cyclictest3-21ksoftirqd/012:28:010
20244992612,4cyclictest3-21ksoftirqd/012:01:210
2024499261,18cyclictest0-21swapper/009:30:540
124932260,2sleep30-21swapper/311:51:163
306872250,1sleep10-21swapper/110:28:011
20247992512,4cyclictest20891-21sshd11:42:453
2024599257,16cyclictest22-21ksoftirqd/112:07:201
20245992514,6cyclictest9024-21sshd09:43:121
20245992513,2cyclictest22-21ksoftirqd/109:27:551
2024599250,11cyclictest22-21ksoftirqd/111:50:311
20244992511,6cyclictest3-21ksoftirqd/009:28:140
2024499250,13cyclictest28639-21sshd10:01:560
2024499250,10cyclictest13252-21cp09:20:200
148482250,23sleep2135650irq/27-eth012:15:372
20245992420,2cyclictest1847-21nfsd10:43:591
20245992417,4cyclictest22-21ksoftirqd/110:02:551
2024599241,3cyclictest27724-21sshd11:01:031
20244992413,4cyclictest3-21ksoftirqd/012:15:010
305462230,2sleep20-21swapper/210:08:382
20247992322,0cyclictest0-21swapper/312:19:243
20247992320,2cyclictest9374-21sshd10:42:353
20247992315,7cyclictest0-21swapper/310:35:073
20247992315,7cyclictest0-21swapper/310:35:073
2024599237,14cyclictest22-21ksoftirqd/112:00:521
20247992215,4cyclictest1959-21sshd10:14:463
2024799220,19cyclictest6922-21sshd10:52:193
2024699222,3cyclictest9268-21sshd10:55:572
2024599224,12cyclictest22-21ksoftirqd/110:22:451
20245992216,4cyclictest26319-21sshd09:45:001
2024599220,17cyclictest22-21ksoftirqd/109:59:341
296162210,19sleep2135650irq/27-eth012:05:222
2024799215,8cyclictest47-21rcuop/312:35:193
20247992116,3cyclictest30260-21sshd09:45:193
20247992113,4cyclictest0-21swapper/310:49:233
2024699211,8cyclictest1848-21nfsd11:55:472
20245992112,5cyclictest12970-21sshd11:28:551
2024599211,19cyclictest22-21ksoftirqd/111:11:461
20244992115,4cyclictest3-21ksoftirqd/012:11:190
20244992111,3cyclictest3-21ksoftirqd/009:11:470
195332210,1sleep21848-21nfsd09:47:502
240992200,0sleep041ktimersoftd/009:58:060
2024799204,13cyclictest36-21rcuop/209:40:393
20247992015,4cyclictest5573-21sshd11:21:323
20247992013,5cyclictest1848-21nfsd10:09:213
20247992011,7cyclictest135650irq/27-eth009:17:353
2024699204,7cyclictest1820-21sshd12:23:092
2024699202,8cyclictest0-21swapper/209:21:432
20246992016,2cyclictest1843-21nfsd10:01:392
20246992014,4cyclictest29676-21sshd10:54:412
2024599209,3cyclictest10950irq/23-ehci_hcd10:16:281
20245992014,4cyclictest5639-21runrttasks09:32:571
2024499209,3cyclictest3-21ksoftirqd/010:10:580
2024499202,16cyclictest8-21rcu_preempt11:14:530
20244992016,3cyclictest3-21ksoftirqd/010:21:570
20244992012,3cyclictest3-21ksoftirqd/010:54:270
20244992012,2cyclictest3-21ksoftirqd/010:36:350
20244992012,2cyclictest3-21ksoftirqd/010:36:350
2024499200,19cyclictest0-21swapper/010:15:090
2024499200,18cyclictest10-21rcuc/011:24:420
2024499200,16cyclictest0-21swapper/011:27:360
6822190,0sleep0678-21bash09:15:380
2024799199,7cyclictest0-21swapper/309:55:393
2024799198,2cyclictest26950irq/26-ahci[00011:05:143
20247991915,3cyclictest1846-21nfsd11:32:493
20247991915,3cyclictest0-21swapper/309:35:393
20247991914,3cyclictest2769-21bash09:26:013
2024699198,4cyclictest10950irq/23-ehci_hcd11:18:482
2024699194,4cyclictest7344-21bash12:37:582
2024599193,1cyclictest3946-21sshd11:56:361
20244991914,3cyclictest3-21ksoftirqd/010:45:590
20244991914,3cyclictest3-21ksoftirqd/009:37:140
118352190,17sleep2135650irq/27-eth009:33:362
24892180,2sleep22482-21sshd10:41:532
2024799189,3cyclictest5697-21sshd12:26:173
20247991814,3cyclictest1847-21nfsd09:11:113
20247991811,4cyclictest27727-21sshd12:11:013
20247991810,5cyclictest1791-21bash10:28:233
20247991810,4cyclictest1847-21nfsd11:13:533
20247991810,3cyclictest18599-21sshd11:36:053
2024699187,5cyclictest10950irq/23-ehci_hcd09:13:282
2024699187,4cyclictest135650irq/27-eth010:36:542
2024699187,4cyclictest135650irq/27-eth010:36:542
2024699186,6cyclictest10950irq/23-ehci_hcd10:31:162
2024699186,5cyclictest4940-21bash12:26:142
2024699184,7cyclictest14398-21sshd10:19:552
20246991811,5cyclictest29937-21id11:20:402
20246991810,6cyclictest21349-21sshd09:41:072
2024699180,10cyclictest0-21swapper/212:01:092
2024799178,3cyclictest28072-21bash11:01:053
2024699177,5cyclictest10950irq/23-ehci_hcd11:40:562
2024699176,4cyclictest18035-21sshd09:50:512
2024699174,4cyclictest31106-21bash11:27:252
2024699173,2cyclictest0-21swapper/207:24:142
20246991711,4cyclictest30493-21bash11:07:532
2024699170,15cyclictest0-21swapper/208:55:182
2024599170,15cyclictest22-21ksoftirqd/108:20:151
2024499175,8cyclictest3-21ksoftirqd/011:59:070
179822170,4sleep22024699cyclictest09:17:362
113092170,15sleep3135650irq/27-eth010:59:303
9262160,1sleep0928-21bash09:45:360
2024699163,8cyclictest9227-21bash11:50:562
2024699163,3cyclictest26680-21sshd11:30:212
2024699162,4cyclictest144550irq/28-eth1-rx-08:28:592
2024699161,8cyclictest1848-21nfsd08:51:272
2024699160,14cyclictest4267-21bash09:56:032
2024699160,14cyclictest13641-21sshd11:45:162
2024699160,14cyclictest0-21swapper/207:54:282
2024599160,14cyclictest0-21swapper/108:15:201
20247991511,3cyclictest0-21swapper/311:27:353
2024799150,6cyclictest0-21swapper/308:00:133
2024699157,5cyclictest16482-21sshd10:20:072
2024699153,2cyclictest0-21swapper/208:12:102
2024699152,7cyclictest1959-21sshd10:27:232
2024699152,1cyclictest0-21swapper/207:56:112
2024699151,2cyclictest0-21swapper/207:26:512
2024599156,8cyclictest22-21ksoftirqd/107:25:171
2024599150,14cyclictest22-21ksoftirqd/108:55:271
2024599150,13cyclictest0-21swapper/107:56:561
2024499154,10cyclictest0-21swapper/008:10:140
20244991510,4cyclictest0-21swapper/007:43:510
2024499150,12cyclictest0-21swapper/007:23:200
2024799142,11cyclictest10950irq/23-ehci_hcd07:30:593
2024799140,13cyclictest0-21swapper/308:11:393
2024799140,12cyclictest19180-21sshd11:55:063
2024699148,5cyclictest25160-21kernelversion08:30:172
2024699148,5cyclictest0-21swapper/208:45:182
2024699145,8cyclictest144550irq/28-eth1-rx-08:02:152
2024699141,2cyclictest0-21swapper/208:06:592
2024699141,12cyclictest1848-21nfsd08:41:032
2024699141,12cyclictest1848-21nfsd08:41:032
2024699141,12cyclictest144550irq/28-eth1-rx-07:13:272
2024699141,12cyclictest135650irq/27-eth008:22:302
2024699140,12cyclictest6374-21hwlatdetect09:00:152
2024699140,11cyclictest0-21swapper/209:06:522
2024599145,8cyclictest22-21ksoftirqd/107:10:181
2024599143,2cyclictest0-21swapper/108:00:151
2024599142,11cyclictest25-21rcuop/108:36:021
2024599142,10cyclictest15258-21munin-node08:10:121
2024599141,5cyclictest22-21ksoftirqd/108:45:031
2024599141,12cyclictest36-21rcuop/207:17:471
2024599140,13cyclictest36-21rcuop/207:36:241
2024599140,12cyclictest8949-21netstat09:05:181
2024599140,12cyclictest3168-21unixbench_singl07:40:221
2024499140,13cyclictest0-21swapper/008:05:210
2024799138,3cyclictest1849-21iostat_ios08:50:153
2024799137,5cyclictest0-21swapper/308:25:493
2024799137,1cyclictest2697-21tr07:40:173
2024799132,10cyclictest10950irq/23-ehci_hcd08:46:553
2024799132,10cyclictest10950irq/23-ehci_hcd07:46:473
2024799131,11cyclictest135650irq/27-eth008:20:143
20247991311,1cyclictest0-21swapper/309:02:413
2024799130,12cyclictest135650irq/27-eth007:51:563
2024799130,12cyclictest0-21swapper/309:09:013
2024799130,12cyclictest0-21swapper/308:32:333
2024799130,12cyclictest0-21swapper/307:59:053
2024799130,12cyclictest0-21swapper/307:26:493
2024799130,12cyclictest0-21swapper/307:22:053
2024699134,8cyclictest144550irq/28-eth1-rx-08:15:152
2024699133,4cyclictest144550irq/28-eth1-rx-07:35:172
20246991311,1cyclictest0-21swapper/207:40:102
2024699130,5cyclictest0-21swapper/208:36:062
2024699130,12cyclictest0-21swapper/207:17:042
2024599139,3cyclictest36-21rcuop/207:50:041
2024599137,5cyclictest0-21swapper/108:30:021
2024599137,5cyclictest0-21swapper/107:46:201
20245991311,1cyclictest0-21swapper/108:52:541
2024599130,12cyclictest36-21rcuop/207:30:271
2024599130,12cyclictest0-21swapper/108:44:071
2024599130,12cyclictest0-21swapper/108:44:071
2024599130,12cyclictest0-21swapper/107:20:021
2024599130,11cyclictest22773-21hddtemp_smartct08:25:141
2024599130,11cyclictest1848-21nfsd09:00:231
2024599130,11cyclictest13242-21df08:05:131
2024499137,5cyclictest11466-21irqstats08:00:160
2024499137,5cyclictest0-21swapper/007:50:290
2024499134,4cyclictest0-21swapper/008:35:260
20244991311,1cyclictest0-21swapper/007:55:590
20244991311,1cyclictest0-21swapper/007:38:590
2024499130,9cyclictest0-21swapper/008:50:220
2024499130,3cyclictest0-21swapper/007:30:230
2024499130,3cyclictest0-21swapper/007:15:080
2024499130,1cyclictest4551-21sh08:55:200
2024499130,12cyclictest0-21swapper/009:05:250
2024499130,12cyclictest0-21swapper/008:45:190
2024499130,12cyclictest0-21swapper/008:31:130
20247991210,1cyclictest0-21swapper/308:05:013
2024799120,3cyclictest32539-21memory07:35:183
2024799120,3cyclictest20530-21df07:10:133
2024799120,11cyclictest0-21swapper/308:36:513
2024799120,10cyclictest29260-21df_inode08:40:133
2024799120,10cyclictest29260-21df_inode08:40:133
2024799120,10cyclictest22841-21munin-node07:15:123
2024799120,10cyclictest0-21swapper/308:55:133
2024699128,3cyclictest1848-21nfsd07:45:092
2024499127,4cyclictest0-21swapper/007:45:120
2024499127,2cyclictest0-21swapper/008:20:100
2024499120,4cyclictest23240-21ntp_states08:25:190
2024499120,2cyclictest0-21swapper/007:10:170
2024499120,12cyclictest0-21swapper/009:03:520
2024499120,11cyclictest0-21swapper/008:40:020
2024499120,11cyclictest0-21swapper/008:40:020
2024499120,11cyclictest0-21swapper/008:15:550
2024499110,1cyclictest0-21swapper/007:25:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional