You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-18 - 00:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Thu Jul 17, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2406121660,2sleep30-21swapper/310:01:093
2148521380,0sleep22877599cyclictest10:57:522
31021360,2sleep10-21swapper/110:08:321
682921350,0sleep36828-21sshd11:54:153
1115521350,3sleep22877599cyclictest09:53:352
2821721340,6sleep02876699cyclictest09:32:570
2821721340,6sleep02876699cyclictest09:32:570
1994921320,1sleep30-21swapper/311:40:543
413521270,1sleep22877599cyclictest12:34:402
655321250,1sleep30-21swapper/311:39:483
2286021240,2sleep20-21swapper/211:18:082
2985821230,2sleep129857-21sshd11:38:591
1188721130,2sleep10-21swapper/109:21:551
1188721130,2sleep10-21swapper/109:21:541
302621110,2sleep10-21swapper/110:34:141
2221221110,1sleep10-21swapper/112:13:021
28291210533,44sleep10-21swapper/107:07:131
2049321040,2sleep20-21swapper/211:40:562
1819321010,1sleep01846-21nfsd10:13:320
2872121000,0sleep00-21swapper/010:52:450
197782930,3sleep30-21swapper/312:06:593
176592910,2sleep317658-21sshd12:32:573
2824629029,49sleep30-21swapper/307:06:393
265022890,2sleep226501-21bash11:30:022
69152880,1sleep06914-21bash12:26:070
105602880,0sleep00-21swapper/011:31:250
226732850,2sleep30-21swapper/312:01:263
4552840,5sleep02876699cyclictest10:05:140
107952840,1sleep10-21swapper/109:31:091
107952840,1sleep10-21swapper/109:31:081
321322830,2sleep12405-21runrttasks11:27:361
71652780,3sleep10-21swapper/111:54:181
263182770,2sleep10-21swapper/109:45:231
113482770,2sleep30-21swapper/312:15:013
111132760,3sleep32877999cyclictest09:47:063
89402750,3sleep38937-21id11:31:163
64582750,0sleep00-21swapper/009:15:040
247042750,4sleep12877099cyclictest11:12:291
173912750,2sleep317389-21sshd10:32:283
47012740,2sleep20-21swapper/212:28:552
289682740,2sleep30-21swapper/309:55:163
213882710,1sleep32877999cyclictest11:29:353
15332700,2sleep20-21swapper/212:37:162
94072680,2sleep22405-21runrttasks11:28:302
309002670,3sleep02876699cyclictest11:44:520
279512670,1sleep30-21swapper/312:39:423
31362640,1sleep30-21swapper/309:43:133
2848725734,12sleep00-21swapper/007:09:410
2840825334,11sleep20-21swapper/207:08:442
224172530,0sleep122416-21sshd09:26:041
224172530,0sleep122416-21sshd09:26:031
110492470,1sleep21848-21nfsd09:12:212
125372460,1sleep32877999cyclictest09:15:313
28779994116,3cyclictest0-21swapper/311:10:063
28766993912,18cyclictest3-21ksoftirqd/011:15:280
148362380,1sleep014822-21sshd10:38:260
28770993633,2cyclictest31040-21bash12:08:001
2876699366,28cyclictest3-21ksoftirqd/011:06:030
28766993611,14cyclictest3-21ksoftirqd/010:43:360
28766993514,11cyclictest3-21ksoftirqd/011:27:480
309802330,1sleep10-21swapper/110:01:501
2876699339,16cyclictest3-21ksoftirqd/011:54:590
2877999328,1cyclictest0-21swapper/310:48:593
2877999320,11cyclictest0-21swapper/310:19:093
32602310,3sleep20-21swapper/211:56:482
2876699312,1cyclictest41ktimersoftd/012:16:130
107572310,2sleep10-21swapper/112:26:291
72242300,2sleep27220-21sshd09:40:192
28766993015,8cyclictest3-21ksoftirqd/010:04:170
28766993013,7cyclictest3-21ksoftirqd/009:21:340
28766993013,7cyclictest3-21ksoftirqd/009:21:330
28766993013,15cyclictest3-21ksoftirqd/012:32:220
28766993010,10cyclictest3-21ksoftirqd/010:56:170
288642290,5sleep02876699cyclictest11:12:530
2876699299,13cyclictest3-21ksoftirqd/010:33:400
28766992913,10cyclictest3-21ksoftirqd/012:24:370
2876699290,27cyclictest0-21swapper/009:48:080
280982290,1sleep028084-21sshd11:56:060
97142280,3sleep20-21swapper/209:56:302
41222280,2sleep31847-21nfsd12:28:513
2877999287,1cyclictest0-21swapper/311:17:053
2877999286,0cyclictest0-21swapper/309:25:243
2877999286,0cyclictest0-21swapper/309:25:233
28779992810,1cyclictest0-21swapper/310:59:133
28770992817,7cyclictest22719-21sshd09:57:501
2876699289,18cyclictest3-21ksoftirqd/009:59:400
2876699286,12cyclictest3-21ksoftirqd/012:06:070
28766992818,5cyclictest3-21ksoftirqd/011:45:220
28766992814,8cyclictest3-21ksoftirqd/012:00:510
28766992814,6cyclictest3-21ksoftirqd/010:26:150
28766992813,5cyclictest3-21ksoftirqd/011:35:060
28766992812,6cyclictest3-21ksoftirqd/010:48:190
28766992811,11cyclictest3-21ksoftirqd/009:37:330
28779992713,4cyclictest10950irq/23-ehci_hcd11:08:373
28766992710,7cyclictest3-21ksoftirqd/010:18:360
41742260,2sleep10-21swapper/110:59:141
291552260,2sleep229149-21sshd10:08:102
2876699264,8cyclictest3-21ksoftirqd/009:14:290
28766992612,6cyclictest3-21ksoftirqd/012:35:320
28766992610,15cyclictest3-21ksoftirqd/011:03:200
208642260,2sleep30-21swapper/311:55:253
28779992516,2cyclictest10950irq/23-ehci_hcd11:49:013
2876699256,8cyclictest3-21ksoftirqd/012:12:480
179532250,23sleep2135650irq/27-eth009:38:172
28779992414,2cyclictest8787-21id10:25:183
2877599245,6cyclictest10950irq/23-ehci_hcd09:47:482
28775992415,6cyclictest33-21ksoftirqd/210:03:162
28770992419,3cyclictest7242-21sshd10:53:441
2877099240,13cyclictest10950irq/23-ehci_hcd09:15:011
2876699248,7cyclictest3-21ksoftirqd/010:20:010
28766992416,5cyclictest3-21ksoftirqd/011:22:490
28766992412,6cyclictest3-21ksoftirqd/009:29:240
28766992412,6cyclictest3-21ksoftirqd/009:29:230
269612240,21sleep1135650irq/27-eth009:13:581
86572230,2sleep38652-21sshd09:34:163
86572230,2sleep38652-21sshd09:34:163
53222230,2sleep20-21swapper/212:23:092
28779992315,3cyclictest9109-21sshd10:09:263
28779992314,6cyclictest1846-21nfsd10:37:003
28766992311,11cyclictest3-21ksoftirqd/009:52:040
258882230,2sleep21847-21nfsd09:29:342
258882230,2sleep21847-21nfsd09:29:342
2877999229,1cyclictest15331-21sshd09:50:393
28779992213,3cyclictest1845-21nfsd10:10:093
2877599223,5cyclictest33-21ksoftirqd/212:13:232
2877599223,17cyclictest33-21ksoftirqd/209:34:172
2877599223,17cyclictest33-21ksoftirqd/209:34:162
28775992210,9cyclictest24330-21rm10:39:242
2877099221,9cyclictest0-21swapper/110:37:091
28770992217,3cyclictest25867-21sshd12:33:421
28770992215,5cyclictest20951-21rm11:32:211
318032210,3sleep20-21swapper/207:15:152
28779992112,4cyclictest1847-21nfsd09:22:433
28779992112,4cyclictest1847-21nfsd09:22:423
28779992112,3cyclictest0-21swapper/310:41:223
28779992110,9cyclictest135650irq/27-eth010:51:093
2877599219,3cyclictest23584-21bash10:20:212
28775992111,8cyclictest908-21sshd11:07:302
2877099216,5cyclictest10950irq/23-ehci_hcd11:03:081
2877099211,6cyclictest1845-21nfsd11:15:061
2876699210,4cyclictest8-21rcu_preempt09:42:220
165882214,15sleep210950irq/23-ehci_hcd10:41:462
317182200,18sleep1135650irq/27-eth012:02:171
2877999205,2cyclictest22507-21sshd11:20:553
28779992012,2cyclictest10950irq/23-ehci_hcd10:22:343
28779992010,1cyclictest16396-21sshd12:21:093
2877999200,4cyclictest0-21swapper/312:10:483
2877599202,6cyclictest135650irq/27-eth010:18:582
28775992016,3cyclictest24730-21sshd11:38:312
28775992011,5cyclictest7759-21sshd11:45:332
2877099205,4cyclictest26950irq/26-ahci[00012:36:411
28770992015,4cyclictest1848-21nfsd10:24:081
28770992014,1cyclictest31816-21sshd09:36:231
171222201,17sleep1135650irq/27-eth012:18:241
2877999198,10cyclictest135650irq/27-eth007:55:193
2877999194,5cyclictest1959-21sshd11:03:033
28779991912,1cyclictest4487-21bash09:11:433
2877599199,8cyclictest7290-21sshd10:12:262
2877599198,3cyclictest1848-21nfsd12:05:022
28775991912,3cyclictest25596-21sshd11:04:002
2877599191,15cyclictest0-21swapper/211:13:052
2877099192,7cyclictest13994-21sshd11:20:141
28770991915,2cyclictest3513-21id10:18:281
28770991914,3cyclictest0-21swapper/111:07:331
28770991914,2cyclictest5066-21sshd10:12:121
28770991913,4cyclictest28126-21sshd12:22:141
28770991910,2cyclictest135650irq/27-eth010:29:021
2877999185,1cyclictest12463-21bash09:37:413
2877999182,0cyclictest0-21swapper/308:42:093
2877599189,4cyclictest1959-21sshd12:16:522
28770991813,3cyclictest16137-21sshd11:46:191
28770991812,5cyclictest10950irq/23-ehci_hcd11:43:531
2877099180,13cyclictest25853-21sshd11:55:531
2876699187,10cyclictest0-21swapper/008:45:330
2876699180,16cyclictest6281-21iostat_ios07:30:170
2876699180,15cyclictest0-21swapper/008:41:550
2876699180,14cyclictest0-21swapper/007:44:160
2877999172,0cyclictest0-21swapper/308:56:023
28779991713,0cyclictest0-21swapper/308:50:223
2877599177,5cyclictest1848-21nfsd11:22:192
2877599175,9cyclictest32640-21sshd11:53:402
28775991711,4cyclictest1848-21nfsd12:02:352
28775991711,2cyclictest135650irq/27-eth010:34:272
2877099177,5cyclictest24915-21sshd09:54:561
2877099176,5cyclictest135650irq/27-eth009:43:211
28770991712,4cyclictest18524-21bash10:45:121
2876699179,3cyclictest0-21swapper/008:24:230
2876699171,2cyclictest10950irq/23-ehci_hcd08:25:310
2876699170,16cyclictest0-21swapper/009:04:030
2876699170,16cyclictest0-21swapper/007:55:160
2876699170,14cyclictest0-21swapper/007:28:590
2876699170,12cyclictest0-21swapper/008:35:180
2877999160,10cyclictest0-21swapper/307:30:133
2877099167,2cyclictest135650irq/27-eth010:40:271
2877099161,6cyclictest0-21swapper/107:14:171
2876699167,7cyclictest0-21swapper/008:15:260
2876699166,9cyclictest0-21swapper/009:06:460
2876699160,15cyclictest0-21swapper/008:55:130
2876699160,15cyclictest0-21swapper/008:30:200
2876699160,15cyclictest0-21swapper/008:06:540
2876699160,15cyclictest0-21swapper/007:52:430
2877999158,6cyclictest0-21swapper/309:05:243
2877999158,0cyclictest0-21swapper/307:41:093
2877999155,2cyclictest10950irq/23-ehci_hcd08:25:223
2877999150,13cyclictest0-21swapper/307:20:203
2877599159,3cyclictest135650irq/27-eth008:55:332
2877599158,5cyclictest1848-21nfsd10:45:052
2877599155,7cyclictest607-21sshd10:27:422
2877599153,8cyclictest31143-21sshd10:52:582
28775991512,2cyclictest1079-21sshd09:17:432
28775991510,4cyclictest10950irq/23-ehci_hcd08:48:202
28775991510,2cyclictest24715-21sshd09:20:012
28775991510,2cyclictest24715-21sshd09:20:012
2877099158,6cyclictest24432-21cat08:10:161
28770991512,1cyclictest23462-21munin-run08:10:001
2877099151,13cyclictest135650irq/27-eth008:30:231
2877099151,13cyclictest0-21swapper/108:55:251
2877099151,13cyclictest0-21swapper/107:49:541
2877099150,3cyclictest514-21sshd07:18:331
2876699158,2cyclictest0-21swapper/007:35:040
2876699150,2cyclictest10950irq/23-ehci_hcd08:50:130
2876699150,1cyclictest0-21swapper/008:00:490
2876699150,10cyclictest0-21swapper/007:45:520
2877999143,3cyclictest10950irq/23-ehci_hcd07:51:433
28779991410,3cyclictest1814-21timerwakeupswit08:30:213
2877999140,11cyclictest0-21swapper/308:06:073
2877599149,4cyclictest1959-21sshd08:21:412
2877599142,11cyclictest10950irq/23-ehci_hcd08:07:312
2877599141,12cyclictest1848-21nfsd08:42:032
2877099147,6cyclictest2405-21runrttasks08:17:401
2877099141,11cyclictest0-21swapper/108:40:171
28770991410,3cyclictest2659-21chrt08:35:021
2877099140,2cyclictest0-21swapper/108:00:411
2877099140,13cyclictest0-21swapper/108:47:121
2877099140,13cyclictest0-21swapper/108:26:441
28766991412,1cyclictest0-21swapper/007:23:140
2876699140,13cyclictest0-21swapper/008:10:150
2876699140,13cyclictest0-21swapper/007:14:160
2876699140,10cyclictest32072-21nfsd407:15:190
2877999137,5cyclictest0-21swapper/308:35:553
2877999137,5cyclictest0-21swapper/307:29:543
2877999137,0cyclictest0-21swapper/307:35:383
2877999136,3cyclictest10950irq/23-ehci_hcd07:45:143
2877999135,4cyclictest135650irq/27-eth007:10:223
2877999132,10cyclictest135650irq/27-eth008:45:153
2877999130,5cyclictest0-21swapper/308:10:363
2877999130,12cyclictest0-21swapper/309:00:303
2877999130,12cyclictest0-21swapper/308:15:483
2877999130,12cyclictest0-21swapper/308:00:543
2877999130,12cyclictest0-21swapper/307:15:233
2877599139,3cyclictest0-21swapper/208:36:102
2877599133,9cyclictest0-21swapper/207:30:082
2877599133,9cyclictest0-21swapper/207:13:542
2877599131,11cyclictest1848-21nfsd08:50:402
2877599131,11cyclictest1848-21nfsd08:25:232
2877599131,11cyclictest1848-21nfsd08:01:152
2877599131,11cyclictest1848-21nfsd07:25:112
2877599131,11cyclictest135650irq/27-eth009:01:252
2877599131,11cyclictest135650irq/27-eth007:36:012
2877599131,11cyclictest135650irq/27-eth007:20:572
28775991311,1cyclictest0-21swapper/207:50:432
2877599130,12cyclictest0-21swapper/208:10:152
2877599130,12cyclictest0-21swapper/207:57:032
2877599130,12cyclictest0-21swapper/207:45:502
2877099138,4cyclictest0-21swapper/107:38:231
2877099137,5cyclictest0-21swapper/107:50:051
2877099136,2cyclictest0-21swapper/107:56:241
2877099130,12cyclictest0-21swapper/107:20:351
2877099130,11cyclictest5697-21munin-node07:30:121
2877099130,11cyclictest3544-21sed07:25:111
2877099130,11cyclictest29039-21munin-node08:20:141
2877099130,11cyclictest17196-21memory09:05:201
2877099130,11cyclictest11354-21munin-node07:40:211
2877099130,11cyclictest10375-21memory08:50:191
2877999128,3cyclictest135650irq/27-eth008:20:013
2877599129,2cyclictest0-21swapper/208:15:182
2877599121,10cyclictest17596-21sendmail_mailst09:05:252
2877599121,10cyclictest135650irq/27-eth007:40:322
2877599120,4cyclictest0-21swapper/208:30:022
2877099129,2cyclictest0-21swapper/109:00:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional