You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-21 - 22:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Thu Aug 21, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1299221780,2sleep12043899cyclictest10:59:241
1338321540,4sleep22043999cyclictest10:45:182
148421300,4sleep32044099cyclictest12:17:023
2201121280,0sleep10-21swapper/110:39:281
1909321260,1sleep019090-21sshd12:33:060
2010921250,3sleep21848-21nfsd11:00:092
2010921250,3sleep21848-21nfsd11:00:092
435321220,1sleep20-21swapper/211:38:282
264421210,1sleep10-21swapper/110:06:361
475821150,4sleep22043999cyclictest11:48:072
1218421110,1sleep21848-21nfsd09:26:422
1231421080,1sleep012313-21sshd11:45:310
20060210532,44sleep30-21swapper/307:08:283
2258721020,2sleep322586-21sshd11:33:563
877321010,1sleep30-21swapper/309:16:213
174342990,3sleep317433-21sshd11:13:253
325222980,1sleep30-21swapper/310:02:473
186142980,1sleep21848-21nfsd10:42:252
111452980,3sleep30-21swapper/310:14:243
205282960,1sleep00-21swapper/011:52:300
8392950,1sleep00-21swapper/010:02:520
57042950,2sleep20-21swapper/209:56:352
300522930,2sleep30-21swapper/310:30:043
214652930,1sleep1417-21jbd2/dm-0-809:44:531
80012920,1sleep00-21swapper/011:25:360
50662910,2sleep21848-21nfsd10:51:152
243762910,2sleep30-21swapper/311:24:113
175172910,0sleep00-21swapper/010:28:440
307382900,2sleep00-21swapper/009:35:330
1989329027,50sleep10-21swapper/107:06:221
14322890,2sleep20-21swapper/210:33:482
163272880,2sleep12043899cyclictest09:30:411
280592870,1sleep128052-21sshd09:55:311
86982860,1sleep00-21swapper/011:02:160
86982860,1sleep00-21swapper/011:02:150
300892850,0sleep041ktimersoftd/009:45:360
240422850,4sleep124043-21bash10:32:431
189192850,5sleep10-21swapper/109:54:391
163222850,3sleep116315-21sshd12:35:401
108522850,4sleep22043999cyclictest12:38:092
107792850,1sleep30-21swapper/310:59:103
211002840,1sleep30-21swapper/310:46:073
228132830,1sleep30-21swapper/312:13:093
178452830,2sleep30-21swapper/309:20:293
106812830,1sleep010680-21sshd12:12:020
243502820,2sleep10-21swapper/112:19:071
225422820,2sleep10-21swapper/112:24:451
3082810,2sleep10-21swapper/111:50:381
160342810,2sleep20-21swapper/209:47:352
7632800,4sleep32044099cyclictest09:52:443
227112800,2sleep20-21swapper/209:31:222
112412790,0sleep011239-21sshd10:45:080
229762780,2sleep10-21swapper/112:01:281
242552710,1sleep224184-21sshd11:24:102
144542710,1sleep10-21swapper/112:12:231
201652700,1sleep020167-21id12:24:310
212762690,0sleep00-21swapper/009:27:450
94482600,0sleep00-21swapper/010:14:130
43622600,1sleep10-21swapper/111:56:501
1991125810,11sleep20-21swapper/207:06:352
75452490,1sleep10-21swapper/112:32:031
246482450,1sleep30-21swapper/311:55:413
2004324437,4sleep00-21swapper/007:08:170
58012420,0sleep00-21swapper/010:58:390
28782410,0sleep12879-21bash11:44:441
229192370,34sleep3135650irq/27-eth012:24:483
144752370,4sleep20-21swapper/210:00:522
39592350,32sleep3135650irq/27-eth011:41:353
132652330,1sleep20-21swapper/212:15:122
255742320,1sleep025573-21bash11:14:180
2043799327,18cyclictest3-21ksoftirqd/010:05:180
20437993210,18cyclictest3-21ksoftirqd/011:23:080
246602310,28sleep1135650irq/27-eth009:48:301
20437993113,11cyclictest3-21ksoftirqd/012:06:430
20437993013,4cyclictest3-21ksoftirqd/009:56:490
111372300,28sleep1135650irq/27-eth011:45:261
2044099298,17cyclictest44-21ksoftirqd/309:29:303
2043899290,27cyclictest7367-21sshd11:02:071
2043899290,27cyclictest7367-21sshd11:02:061
20440992811,5cyclictest0-21swapper/309:12:563
20437992814,13cyclictest3-21ksoftirqd/010:23:350
199752271,24sleep3135650irq/27-eth009:44:443
87902260,3sleep20-21swapper/212:06:022
20440992615,6cyclictest25-21rcuop/112:06:473
20437992623,2cyclictest0-21swapper/011:31:170
20437992613,7cyclictest3-21ksoftirqd/009:10:220
20437992613,4cyclictest3-21ksoftirqd/010:35:060
266642250,2sleep326663-21sshd10:39:573
20437992511,11cyclictest3-21ksoftirqd/012:00:590
2043799250,24cyclictest3-21ksoftirqd/009:51:430
151512250,23sleep3135650irq/27-eth011:09:533
306362240,2sleep10-21swapper/111:08:061
252152240,18sleep3135650irq/27-eth010:26:043
2044099249,1cyclictest0-21swapper/312:35:203
20439992420,2cyclictest1848-21nfsd11:58:422
20437992412,6cyclictest3-21ksoftirqd/012:36:520
2044099237,6cyclictest26659-21rm10:05:413
20440992315,6cyclictest25-21rcuop/112:27:483
20440992312,7cyclictest22921-21bash12:01:283
20440992311,4cyclictest19621-21sshd10:18:483
2044099230,18cyclictest44-21ksoftirqd/310:24:093
20438992310,3cyclictest24882-21sshd12:27:501
2043799232,20cyclictest3-21ksoftirqd/010:44:410
20437992315,7cyclictest3-21ksoftirqd/011:41:310
20437992312,7cyclictest3-21ksoftirqd/011:37:360
2043799230,2cyclictest10-21rcuc/011:06:520
20440992214,5cyclictest14325-21sshd11:36:113
20440992211,8cyclictest671-21sshd11:15:063
2043999225,8cyclictest20036-21sshd09:41:172
20439992214,5cyclictest17245-21sshd12:12:372
20438992212,6cyclictest8631-21iostat_ios10:00:201
2043899220,17cyclictest16028-21ls11:20:001
2043899220,11cyclictest0-21swapper/111:35:241
2043799228,11cyclictest3-21ksoftirqd/009:20:280
2043799224,16cyclictest3-21ksoftirqd/011:16:030
20437992214,2cyclictest3-21ksoftirqd/012:16:480
20437992211,5cyclictest3-21ksoftirqd/009:33:480
20437992211,4cyclictest3-21ksoftirqd/011:58:540
20437992211,4cyclictest3-21ksoftirqd/010:52:110
2043799220,1cyclictest2733-21sshd10:33:560
213742210,19sleep1135650irq/27-eth010:46:091
20440992111,5cyclictest1848-21nfsd09:55:433
20440992110,5cyclictest10950irq/23-ehci_hcd11:26:013
20440992110,1cyclictest0-21swapper/310:51:013
2043999215,8cyclictest8608-21sshd12:03:092
2043999214,3cyclictest10097-21sshd09:36:502
20438992115,4cyclictest1846-21nfsd12:08:321
20437992114,5cyclictest3-21ksoftirqd/009:44:160
2043799211,16cyclictest0-21swapper/012:25:040
202392210,19sleep1135650irq/27-eth011:10:191
2044099209,6cyclictest20308-21bash12:30:183
2044099208,4cyclictest10186-21sshd10:41:323
2043999208,3cyclictest4828-21sshd09:15:562
2043999204,7cyclictest0-21swapper/210:39:252
2043899208,5cyclictest10950irq/23-ehci_hcd11:25:431
20437992016,3cyclictest3-21ksoftirqd/009:18:590
20437992016,2cyclictest3-21ksoftirqd/010:15:560
2044099199,5cyclictest11004-21sshd09:36:543
2044099198,2cyclictest24930-21id11:04:073
2044099198,2cyclictest24930-21id11:04:073
20440991912,4cyclictest8688-21bash09:46:483
2044099190,18cyclictest0-21swapper/309:32:043
2044099190,13cyclictest17204-21sshd11:52:123
2043999194,1cyclictest28857-21sshd12:31:012
20439991911,2cyclictest22034-21sshd10:05:172
20439991911,2cyclictest14400-21sshd10:59:322
2043899193,12cyclictest2735-21sshd10:54:571
2043899190,6cyclictest0-21swapper/110:22:231
2044099182,4cyclictest16391-21sshd11:45:563
2043999182,3cyclictest11267-21sshd10:24:452
2043999181,7cyclictest0-21swapper/207:58:512
20439991812,4cyclictest26426-21sshd11:34:202
2043999181,16cyclictest0-21swapper/208:13:322
20439991810,2cyclictest135650irq/27-eth010:15:202
2043899183,5cyclictest1846-21nfsd11:23:541
20438991812,4cyclictest14980-21sshd11:30:011
2043899180,15cyclictest135650irq/27-eth009:39:201
2043899180,14cyclictest0-21swapper/110:19:351
297562170,15sleep210950irq/23-ehci_hcd09:21:472
208002170,15sleep2135650irq/27-eth011:27:082
20439991712,3cyclictest1848-21nfsd10:26:032
2043999171,13cyclictest7639-21sshd12:26:132
2043899176,5cyclictest31055-21cp10:26:421
2043899176,3cyclictest0-21swapper/110:40:541
2043899175,2cyclictest23158-21id09:21:041
195162170,2sleep119513-21sshd09:27:341
2043999169,6cyclictest36-21rcuop/209:52:372
2043999168,7cyclictest1893-21nscd11:15:502
2043999164,11cyclictest1848-21nfsd09:11:352
2043999162,7cyclictest0-21swapper/207:52:322
2043999162,13cyclictest135650irq/27-eth008:55:552
20439991613,2cyclictest4322-21ipmi_temp11:05:182
20439991610,4cyclictest25244-21sshd11:52:552
20439991610,2cyclictest10950irq/23-ehci_hcd10:13:352
2043999160,14cyclictest0-21swapper/212:20:092
2043999160,14cyclictest0-21swapper/211:14:002
2043899168,7cyclictest1131-21jbd2/dm-2-809:15:121
20438991612,3cyclictest1959-21sshd09:11:281
2043899160,8cyclictest0-21swapper/110:10:031
2043899160,14cyclictest0-21swapper/108:30:151
2043799160,15cyclictest0-21swapper/007:25:170
2043799160,12cyclictest241562chrt08:27:540
2044099150,6cyclictest0-21swapper/307:35:143
2044099150,13cyclictest936-21sshd08:49:233
2044099150,13cyclictest0-21swapper/307:18:103
2043999155,2cyclictest0-21swapper/207:43:472
2043999154,6cyclictest47-21rcuop/311:41:072
2043999154,6cyclictest0-21swapper/207:49:402
2043999152,7cyclictest0-21swapper/209:03:282
2043999152,7cyclictest0-21swapper/208:02:562
2043999152,7cyclictest0-21swapper/207:22:512
2043999152,6cyclictest0-21swapper/209:09:242
2043899157,6cyclictest0-21swapper/109:00:151
20438991511,3cyclictest10950irq/23-ehci_hcd07:30:111
2043899150,6cyclictest9381-21proc_pri09:05:201
20437991512,2cyclictest3-21ksoftirqd/007:15:010
2043799151,10cyclictest6643-21gawk09:00:170
2043799150,14cyclictest0-21swapper/007:35:180
2043799150,13cyclictest0-21swapper/007:30:170
7062140,12sleep2135650irq/27-eth008:46:282
2044099143,3cyclictest10950irq/23-ehci_hcd08:09:283
20440991410,3cyclictest10950irq/23-ehci_hcd08:44:243
20440991410,3cyclictest10950irq/23-ehci_hcd07:30:453
20440991410,3cyclictest10950irq/23-ehci_hcd07:30:003
2044099140,12cyclictest0-21swapper/307:50:183
2043999149,4cyclictest135650irq/27-eth008:31:452
2043999145,1cyclictest0-21swapper/208:36:282
2043999145,1cyclictest0-21swapper/208:29:152
2043999143,9cyclictest22919-21cpuspeed_turbos07:15:142
2043999142,7cyclictest0-21swapper/207:10:512
2043899145,5cyclictest10950irq/23-ehci_hcd07:10:191
2043899143,10cyclictest10950irq/23-ehci_hcd08:48:001
2043899140,13cyclictest8590-21munin-node07:55:191
2043899140,13cyclictest2405-21runrttasks08:08:311
2043899140,13cyclictest0-21swapper/108:24:381
2043799140,10cyclictest0-21swapper/008:24:590
2044099139,3cyclictest15710-21munin-node08:10:133
2044099136,6cyclictest0-21swapper/307:20:183
20440991311,1cyclictest0-21swapper/308:00:133
2044099130,12cyclictest1848-21nfsd08:26:293
2044099130,12cyclictest1576-21avahi-daemon08:21:343
2044099130,12cyclictest135650irq/27-eth007:55:553
2044099130,12cyclictest0-21swapper/309:01:353
2044099130,12cyclictest0-21swapper/308:15:533
2044099130,11cyclictest4816-21irqstats07:45:193
2044099130,11cyclictest4369-21irqstats08:55:183
2044099130,11cyclictest27108-21df08:35:113
2044099130,11cyclictest21046-21perl07:10:143
2043999138,4cyclictest135650irq/27-eth007:37:182
2043999138,3cyclictest29539-21tune2fs08:40:142
2043999130,4cyclictest13165-21munin-node08:05:142
2043999130,12cyclictest0-21swapper/208:50:512
2043999130,12cyclictest0-21swapper/208:17:002
2043899134,3cyclictest135650irq/27-eth008:15:141
2043899132,10cyclictest135650irq/27-eth008:01:111
2043899132,10cyclictest135650irq/27-eth007:25:041
2043899132,10cyclictest10950irq/23-ehci_hcd07:38:021
2043899130,12cyclictest0-21swapper/108:54:081
2043899130,12cyclictest0-21swapper/108:40:281
2043899130,12cyclictest0-21swapper/108:25:381
2043899130,12cyclictest0-21swapper/107:54:361
2043899130,12cyclictest0-21swapper/107:23:161
2043899130,11cyclictest3594-21munin-node08:55:151
2043899130,11cyclictest23410-21hddtemp_smartct07:15:151
2043899130,11cyclictest16206-21iostat_ios08:10:161
2043899130,10cyclictest0-21swapper/108:35:581
2043799138,4cyclictest0-21swapper/008:35:560
2043799137,5cyclictest0-21swapper/008:53:540
2043799137,3cyclictest2405-21runrttasks08:45:520
20437991312,1cyclictest0-21swapper/007:24:490
20437991311,1cyclictest0-21swapper/007:16:260
2043799130,9cyclictest0-21swapper/007:55:170
2043799130,1cyclictest0-21swapper/008:15:130
2043799130,1cyclictest0-21swapper/007:40:070
2043799130,12cyclictest12107-21ps08:00:230
2043799130,10cyclictest0-21swapper/008:30:500
2044099127,4cyclictest8341-21munin-node09:05:123
2044099120,4cyclictest1848-21nfsd07:40:133
2044099120,11cyclictest2686-21munin-node08:50:223
2044099120,11cyclictest24565-21munin-node08:30:193
2043999128,3cyclictest29747-21cpuspeed_turbos07:30:142
2043999120,3cyclictest27558-21df07:25:132
2043999120,11cyclictest0-21swapper/208:20:292
2043899120,3cyclictest2512-21iostat07:40:171
2043899120,10cyclictest0-21swapper/107:45:141
2043799127,2cyclictest0-21swapper/008:06:520
2043799126,5cyclictest0-21swapper/008:11:500
2043799120,11cyclictest0-21swapper/008:56:230
2043799120,11cyclictest0-21swapper/008:40:070
2043799120,11cyclictest0-21swapper/007:50:300
2043799120,11cyclictest0-21swapper/007:48:250
2043799118,2cyclictest0-21swapper/009:05:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional