You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 20:46
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Wed Nov 19, 2025 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
825921580,4sleep33127199cyclictest09:46:433
211421470,1sleep32103-21sshd12:37:293
2477621380,0sleep124756-21sshd09:10:181
427221370,3sleep23126699cyclictest12:10:192
105321350,1sleep11052-21sshd12:03:121
873721260,4sleep23126699cyclictest12:20:582
400121250,4sleep13126199cyclictest09:53:071
400121250,4sleep13126199cyclictest09:53:071
1337021180,0sleep20-21swapper/210:11:352
807021170,2sleep10-21swapper/111:22:441
1275421100,4sleep33127199cyclictest09:12:253
3245221020,33sleep3135650irq/27-eth011:00:543
58512960,3sleep15845-21sshd09:15:031
237892950,0sleep00-21swapper/009:48:250
31432930,1sleep10-21swapper/109:35:541
286572930,1sleep028658-21bash12:09:330
265862900,0sleep00-21swapper/009:31:370
214292880,0sleep23126699cyclictest07:59:272
21472860,2sleep22143-21rm12:06:422
214052860,0sleep00-21swapper/010:09:010
257452840,0sleep00-21swapper/011:21:030
3091928333,46sleep30-21swapper/307:08:443
48722820,3sleep20-21swapper/211:50:152
124202820,2sleep10-21swapper/112:21:201
78352810,2sleep30-21swapper/312:17:343
3084228133,44sleep10-21swapper/107:07:581
197032810,0sleep00-21swapper/012:29:000
164462810,1sleep016440-21sshd10:39:230
116842810,2sleep03125599cyclictest11:57:300
96332800,3sleep13126199cyclictest12:27:571
277762790,4sleep13126199cyclictest09:24:531
28212770,2sleep22819-21bash09:35:522
232512770,2sleep123248-21sshd10:40:041
153282750,2sleep30-21swapper/309:27:003
190492730,2sleep20-21swapper/210:36:162
26242710,1sleep10-21swapper/111:08:061
164012700,0sleep11848-21nfsd10:25:331
3069926733,11sleep20-21swapper/207:06:142
202442660,2sleep11848-21nfsd11:55:071
150822650,63sleep126950irq/26-ahci[00012:35:241
209482640,2sleep33127199cyclictest10:22:393
151772630,1sleep10-21swapper/111:54:351
84012620,2sleep30-21swapper/310:17:543
17672620,2sleep11765-21bash11:32:221
311362600,0sleep31848-21nfsd10:34:133
92312570,1sleep03125599cyclictest09:53:420
92312570,1sleep03125599cyclictest09:53:420
3079125633,19sleep00-21swapper/007:07:250
210712530,10sleep20-21swapper/209:24:062
310532510,2sleep31846-21nfsd11:39:143
176562450,2sleep20-21swapper/209:54:362
176562450,2sleep20-21swapper/209:54:362
31255993831,5cyclictest0-21swapper/010:50:410
3127199367,22cyclictest44-21ksoftirqd/311:45:103
3127199361,14cyclictest0-21swapper/311:31:533
3125599360,1cyclictest0-21swapper/009:22:370
139422350,0sleep00-21swapper/007:40:210
3127199316,6cyclictest135650irq/27-eth011:14:193
222612310,0sleep022259-21sshd10:15:540
188882300,28sleep1135650irq/27-eth010:36:161
52962290,2sleep15290-21rm10:48:101
327672290,1sleep00-21swapper/012:03:070
3125599294,17cyclictest0-21swapper/011:03:520
3127199281,20cyclictest44-21ksoftirqd/310:52:363
31266992825,2cyclictest0-21swapper/209:44:102
31266992821,5cyclictest1915-21snmpd12:17:542
89022270,2sleep30-21swapper/311:05:193
3272270,0sleep032645-21sshd09:11:050
3127199273,16cyclictest44-21ksoftirqd/312:25:583
31271992723,3cyclictest184750nfsd10:25:443
3127199271,2cyclictest25446-21sshd09:59:003
3127199270,3cyclictest11080-21sshd11:54:083
104422270,1sleep00-21swapper/012:10:570
98032260,24sleep1135650irq/27-eth010:14:481
3127199267,16cyclictest44-21ksoftirqd/312:02:593
31271992612,6cyclictest21353-21bash10:46:273
227772260,1sleep00-21swapper/009:58:430
3127199254,19cyclictest44-21ksoftirqd/310:56:293
31271992520,3cyclictest29657-21sshd09:42:073
31271992516,5cyclictest18596-21sshd12:21:593
3127199250,18cyclictest44-21ksoftirqd/311:55:593
270912250,3sleep127090-21bash11:42:121
270912250,3sleep127090-21bash11:42:121
24672250,3sleep20-21swapper/210:51:072
243702250,2sleep20-21swapper/211:24:342
31271992416,5cyclictest11960-21sshd12:31:343
31271992413,7cyclictest23730-21sshd10:09:163
3126199249,3cyclictest13274-21sshd12:07:551
3125599247,15cyclictest3-21ksoftirqd/010:58:410
230792240,2sleep20-21swapper/211:10:152
56212230,3sleep15610-21sshd12:17:191
3127199234,17cyclictest44-21ksoftirqd/309:19:083
3127199232,19cyclictest44-21ksoftirqd/309:50:303
3127199232,19cyclictest44-21ksoftirqd/309:50:303
31271992313,4cyclictest10950irq/23-ehci_hcd07:59:543
31266992316,5cyclictest1927-21snmpd11:55:082
3125599235,15cyclictest17077-21bash10:32:370
31255992311,6cyclictest3-21ksoftirqd/010:44:490
3125599230,15cyclictest12238-21bash10:04:300
311332230,4sleep20-21swapper/212:37:082
206962230,3sleep120688-21sshd11:10:031
147172230,3sleep11845-21nfsd10:01:121
31271992215,3cyclictest1848-21nfsd12:11:333
31271992214,5cyclictest2801-21sshd11:22:093
31271992214,5cyclictest10962-21sshd10:04:203
31271992213,4cyclictest27979-21bash11:17:473
3127199220,19cyclictest44-21ksoftirqd/309:39:453
3127199220,18cyclictest44-21ksoftirqd/310:10:463
31266992217,4cyclictest1846-21nfsd10:20:482
3126199229,3cyclictest10950irq/23-ehci_hcd12:30:271
3125599225,11cyclictest1847-21nfsd10:10:390
3125599224,3cyclictest32465-21cp12:20:110
289782220,1sleep20-21swapper/210:44:022
162672220,1sleep20-21swapper/211:02:372
113122220,16sleep3135650irq/27-eth009:30:093
95552210,0sleep00-21swapper/012:31:170
72612210,5sleep20-21swapper/209:29:432
326792212,17sleep1135650irq/27-eth012:13:271
31271992114,4cyclictest12210-21bash10:38:553
3126699218,3cyclictest10950irq/23-ehci_hcd09:13:042
31266992118,2cyclictest21618-21bash11:06:382
31261992115,3cyclictest10312-21sshd10:58:411
3126199211,14cyclictest0-21swapper/111:00:201
3125599210,8cyclictest26950irq/26-ahci[00012:19:210
191422210,3sleep12405-21runrttasks10:05:131
142562210,2sleep314255-21bash11:44:233
142562210,2sleep314255-21bash11:44:233
118012210,1sleep031249-21cyclictest09:36:550
94602201,16sleep2135650irq/27-eth010:07:402
67942200,18sleep2135650irq/27-eth012:31:002
31271992010,3cyclictest21937-21sshd09:20:293
31266992015,3cyclictest24424-21sshd11:48:522
3126199201,4cyclictest21717-21sshd09:27:411
3126199200,12cyclictest0-21swapper/111:48:571
3125599208,8cyclictest0-21swapper/010:45:090
3125599205,2cyclictest0-21swapper/011:05:180
31255992014,1cyclictest3-21ksoftirqd/010:22:170
31271991911,5cyclictest23156-21sshd10:40:033
31266991913,4cyclictest21257-21bash09:16:532
3126199198,4cyclictest1844-21nfsd09:31:051
3126199198,3cyclictest0-21swapper/109:43:141
31261991913,3cyclictest7922-21sshd09:46:411
31261991912,5cyclictest1845-21nfsd10:21:071
31261991911,5cyclictest20749-21sshd10:15:441
3125599192,2cyclictest19096-21sshd11:44:530
3125599192,2cyclictest19096-21sshd11:44:530
3125599191,15cyclictest0-21swapper/009:25:340
3125599190,4cyclictest30967-21sshd11:39:130
3125599190,13cyclictest30081-21sshd09:42:090
3125599190,12cyclictest16091-21bash11:51:190
3125599190,11cyclictest25-21rcuop/109:15:290
3125599190,11cyclictest11234-21bash11:33:260
31271991812,4cyclictest20367-21sshd12:05:133
31271991810,5cyclictest32687-21munin-node11:25:203
3126699187,6cyclictest135650irq/27-eth011:31:312
3126699186,1cyclictest26950irq/26-ahci[00011:36:102
3126699183,4cyclictest1848-21nfsd10:29:352
31266991812,3cyclictest12362-21sshd12:00:522
31266991810,4cyclictest0-21swapper/210:58:192
3126199187,3cyclictest0-21swapper/107:40:161
31261991811,4cyclictest27780-21sshd09:55:341
3126199180,16cyclictest1622-21sshd11:25:281
3125599180,7cyclictest0-21swapper/010:25:430
3126699175,4cyclictest0-21swapper/211:15:512
3126699172,6cyclictest27716-21sshd09:31:452
3126699170,15cyclictest0-21swapper/209:57:162
3126199171,4cyclictest0-21swapper/110:50:561
31261991710,4cyclictest0-21swapper/110:33:081
3125599171,12cyclictest0-21swapper/008:51:070
3125599170,16cyclictest15784-21bash11:47:550
3127199161,13cyclictest44-21ksoftirqd/307:50:203
3126699169,3cyclictest10950irq/23-ehci_hcd11:42:092
3126699169,3cyclictest10950irq/23-ehci_hcd11:42:092
3126699167,3cyclictest16896-21sshd10:18:492
3126699166,4cyclictest10950irq/23-ehci_hcd10:32:532
31266991613,2cyclictest12102-21sshd10:45:292
3126699160,14cyclictest135650irq/27-eth010:01:562
3126199165,5cyclictest0-21swapper/107:50:111
3126199161,6cyclictest0-21swapper/111:35:111
3126199161,5cyclictest28445-21sshd11:17:501
31261991611,4cyclictest10950irq/23-ehci_hcd07:37:271
31255991613,2cyclictest3-21ksoftirqd/012:39:140
31255991612,2cyclictest0-21swapper/011:29:580
3125599160,8cyclictest19362-21sshd11:16:490
3125599160,5cyclictest0-21swapper/011:12:580
3125599160,13cyclictest0-21swapper/007:29:380
31271991511,2cyclictest8932-21missed_timers07:30:203
3127199150,14cyclictest0-21swapper/308:10:123
3127199150,13cyclictest22869-21ntpq08:00:223
3127199150,13cyclictest0-21swapper/308:26:113
3127199150,13cyclictest0-21swapper/308:15:153
3126699156,7cyclictest135650irq/27-eth012:27:342
31266991510,1cyclictest26369-21bash11:28:082
3126699150,13cyclictest0-21swapper/207:42:262
3126699150,10cyclictest5728-21sshd09:46:262
3126199158,6cyclictest0-21swapper/108:55:221
3126199158,5cyclictest0-21swapper/108:06:411
3126199152,12cyclictest0-21swapper/108:16:061
31261991511,3cyclictest10950irq/23-ehci_hcd07:18:381
3126199150,4cyclictest135650irq/27-eth008:02:261
3126199150,13cyclictest0-21swapper/108:44:391
3126199150,13cyclictest0-21swapper/108:25:141
3127199140,12cyclictest16721-21munin-node09:00:223
3126699141,12cyclictest144550irq/28-eth1-rx-09:03:232
3126699141,11cyclictest8356-21iostat08:40:182
3126699141,11cyclictest12159-21munin-node08:50:122
3126699140,12cyclictest10499-21fschecks_time08:45:142
3126199143,3cyclictest10950irq/23-ehci_hcd07:29:101
3126199143,10cyclictest10950irq/23-ehci_hcd08:52:011
3126199141,11cyclictest0-21swapper/107:30:171
3126199140,5cyclictest0-21swapper/107:45:121
3126199140,5cyclictest0-21swapper/107:45:111
3126199140,13cyclictest0-21swapper/108:45:321
3126199140,10cyclictest19272-21df09:05:131
3125599140,4cyclictest0-21swapper/007:35:180
3125599140,3cyclictest0-21swapper/008:15:140
3125599140,13cyclictest0-21swapper/008:49:590
3125599140,13cyclictest0-21swapper/008:30:100
3127199137,5cyclictest0-21swapper/307:46:333
3127199137,5cyclictest0-21swapper/307:46:333
31271991311,1cyclictest0-21swapper/308:49:273
31271991311,1cyclictest0-21swapper/307:15:363
31271991311,1cyclictest0-21swapper/307:10:243
31271991310,2cyclictest135650irq/27-eth009:05:133
3127199130,4cyclictest5238-21taskset07:21:413
3127199130,12cyclictest2405-21runrttasks08:52:043
3127199130,12cyclictest0-21swapper/308:21:393
3127199130,12cyclictest0-21swapper/308:09:293
3127199130,12cyclictest0-21swapper/307:43:343
3127199130,12cyclictest0-21swapper/307:29:153
3127199130,11cyclictest5661-21grep08:35:143
3127199130,11cyclictest15662-21ps08:55:233
3127199130,11cyclictest11606-21munin-node07:35:233
3126699138,3cyclictest19881-21memory09:05:182
3126699137,5cyclictest1927-21snmpd07:47:052
3126699137,5cyclictest1927-21snmpd07:47:042
3126699137,5cyclictest0-21swapper/208:05:162
3126699131,3cyclictest2841-21users07:15:262
3126699131,3cyclictest1864-21ntp_states08:25:192
31266991311,1cyclictest0-21swapper/207:35:032
3126699130,12cyclictest135650irq/27-eth008:30:452
3126699130,12cyclictest0-21swapper/208:38:252
3126699130,12cyclictest0-21swapper/208:00:082
3126699130,12cyclictest0-21swapper/207:51:572
3126699130,12cyclictest0-21swapper/207:30:172
3126699130,12cyclictest0-21swapper/207:25:132
3126699130,11cyclictest5135-21munin-node07:20:252
3126699130,11cyclictest14438-21munin-node08:55:212
3126199139,3cyclictest135650irq/27-eth008:36:431
3126199131,3cyclictest4364-21memory07:20:181
3126199130,12cyclictest0-21swapper/109:04:531
3126199130,12cyclictest0-21swapper/108:20:531
3126199130,12cyclictest0-21swapper/107:56:501
31255991311,1cyclictest0-21swapper/008:35:170
3125599130,5cyclictest0-21swapper/009:06:330
3125599130,5cyclictest0-21swapper/008:57:350
3125599130,5cyclictest0-21swapper/007:15:250
3125599130,12cyclictest0-21swapper/009:00:130
3125599130,12cyclictest0-21swapper/008:10:130
3125599130,12cyclictest0-21swapper/007:46:260
3125599130,12cyclictest0-21swapper/007:46:260
3125599130,11cyclictest0-21swapper/007:57:010
3125599130,10cyclictest0-21swapper/007:20:510
3127199120,3cyclictest3370-21diskstats08:30:153
3127199120,11cyclictest8834-21processes08:40:233
3126699128,3cyclictest31616-21iostat_ios08:20:172
3126699128,3cyclictest27063-21irqstats08:10:172
3126699120,11cyclictest0-21swapper/208:15:192
3126699120,11cyclictest0-21swapper/207:10:062
3126199128,3cyclictest135650irq/27-eth008:10:121
3126199120,11cyclictest0-21swapper/107:10:071
3126199120,10cyclictest3709-21fw_packets08:30:161
3125599127,3cyclictest0-21swapper/008:06:410
31255991210,1cyclictest0-21swapper/007:50:200
3125599120,1cyclictest0-21swapper/008:21:150
3125599120,12cyclictest0-21swapper/008:25:390
3125599120,11cyclictest0-21swapper/008:41:060
3125599120,11cyclictest0-21swapper/007:30:240
3125599120,11cyclictest0-21swapper/007:10:220
3125599110,10cyclictest0-21swapper/008:01:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional