You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 21:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Tue Nov 18, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2687721520,2sleep00-21swapper/009:22:170
2415121400,4sleep11641699cyclictest11:52:401
1014921370,0sleep310150-21id12:14:223
570721340,1sleep30-21swapper/309:26:533
1292621300,4sleep31641899cyclictest12:08:333
1217721300,1sleep21641799cyclictest12:31:452
2203221240,3sleep21847-21nfsd11:15:572
2565321210,3sleep01641599cyclictest12:35:480
1104321190,3sleep21641799cyclictest09:13:502
1600621140,2sleep116003-21sshd10:21:221
2249421090,3sleep11641699cyclictest11:45:431
1201321040,4sleep11641699cyclictest10:37:371
2135321020,2sleep10-21swapper/112:21:021
315792950,1sleep10-21swapper/110:09:321
216702940,1sleep00-21swapper/010:55:360
147502940,2sleep114748-21sshd12:08:431
34122920,1sleep01641599cyclictest12:04:130
290072920,2sleep20-21swapper/209:25:562
93422910,2sleep00-21swapper/011:44:320
73502910,1sleep20-21swapper/212:22:442
30862910,2sleep33088-21id10:26:413
4622860,2sleep01641599cyclictest09:56:280
265662850,1sleep20-21swapper/209:22:142
232302840,1sleep023224-21sshd10:18:490
20272840,2sleep32025-21sshd10:36:353
167452840,2sleep1135650irq/27-eth011:28:551
1597328431,49sleep30-21swapper/307:07:573
263942830,1sleep10-21swapper/111:33:081
1596628331,48sleep10-21swapper/107:07:521
88852820,0sleep00-21swapper/010:27:180
233132810,2sleep10-21swapper/111:02:261
146762810,2sleep214673-21sshd09:54:452
5272800,4sleep21641799cyclictest12:13:292
46942800,2sleep14692-21sshd10:10:041
316622800,2sleep30-21swapper/309:46:283
314572790,4sleep21641799cyclictest10:53:222
188362790,2sleep318833-21bash12:32:223
184892790,1sleep10-21swapper/111:12:201
254902780,1sleep30-21swapper/310:08:543
86622770,2sleep18663-21bash09:34:011
176012770,2sleep117599-21netstat11:55:211
302052760,1sleep10-21swapper/110:42:541
259102760,4sleep11641699cyclictest10:19:051
150052750,4sleep21641799cyclictest11:51:412
36642740,2sleep20-21swapper/210:23:322
297762720,0sleep00-21swapper/012:21:490
313282710,0sleep131331-21id10:53:211
1605627133,9sleep20-21swapper/207:09:022
195282690,3sleep01641599cyclictest11:45:260
75052670,2sleep30-21swapper/310:54:153
181752670,2sleep118174-21bash12:02:101
32682640,2sleep13267-21sshd09:19:501
324372610,1sleep30-21swapper/310:23:073
288132480,2sleep20-21swapper/210:56:222
196692460,0sleep219668-21bash11:12:292
1599524235,4sleep00-21swapper/007:08:140
1641599390,37cyclictest10644-21cp09:10:210
296532350,33sleep226950irq/26-ahci[00009:36:182
304582340,28sleep2135650irq/27-eth012:00:072
251882340,7sleep10-21swapper/111:23:051
16418993412,9cyclictest29512-21bash11:13:323
16416993329,3cyclictest0-21swapper/112:29:171
16416993329,3cyclictest0-21swapper/112:29:161
248592300,1sleep00-21swapper/012:12:390
16416993016,2cyclictest22-21ksoftirqd/112:12:521
1641699290,21cyclictest22-21ksoftirqd/112:39:051
213042280,2sleep20-21swapper/211:29:252
1641699286,20cyclictest22-21ksoftirqd/109:57:301
1641899270,19cyclictest44-21ksoftirqd/310:31:543
16416992713,4cyclictest22-21ksoftirqd/112:17:541
16416992711,13cyclictest22-21ksoftirqd/109:28:031
24452260,2sleep22437-21rm09:40:112
1641699269,16cyclictest22-21ksoftirqd/110:49:431
1641699263,6cyclictest28437-21sshd12:33:151
1641699263,4cyclictest31116-21munin-node11:40:241
1641699261,19cyclictest22-21ksoftirqd/109:50:431
66912250,23sleep3135650irq/27-eth010:57:273
27062251,22sleep2135650irq/27-eth011:37:192
16418992519,4cyclictest1847-21nfsd10:10:043
16415992512,12cyclictest3-21ksoftirqd/009:40:270
1641599250,19cyclictest9728-21sshd10:00:400
71372240,1sleep07131-21sshd09:27:020
16418992421,2cyclictest1848-21nfsd10:15:143
16417992410,5cyclictest10950irq/23-ehci_hcd11:41:022
1641699242,15cyclictest22-21ksoftirqd/110:02:021
16416992417,5cyclictest2513-21sshd09:23:111
1641699241,21cyclictest22-21ksoftirqd/109:11:451
1641699240,22cyclictest22-21ksoftirqd/109:35:451
1641699240,21cyclictest22-21ksoftirqd/109:48:501
313622230,4sleep20-21swapper/210:29:402
16418992316,5cyclictest25737-21sshd12:27:133
16418992316,5cyclictest25737-21sshd12:27:133
1641699235,16cyclictest22-21ksoftirqd/110:28:431
1641699230,20cyclictest22-21ksoftirqd/111:08:121
1641599231,2cyclictest1845-21nfsd11:58:180
16415992311,5cyclictest3-21ksoftirqd/012:07:110
1641599230,14cyclictest7723-21sshd10:40:300
16418992217,3cyclictest12557-21sshd09:20:403
16418992216,4cyclictest36-21rcuop/209:32:513
16418992215,4cyclictest0-21swapper/309:55:313
16418992210,3cyclictest6667-21sshd12:19:503
1641799222,14cyclictest28147-21sshd11:33:192
16417992215,4cyclictest20527-21sshd12:06:022
1641699221,19cyclictest22-21ksoftirqd/111:16:431
1641599222,12cyclictest5156-21sshd09:16:350
1641899217,5cyclictest1115-21bash09:43:243
16418992117,2cyclictest12719-21sshd12:37:363
16418992116,3cyclictest1959-21sshd12:20:153
16418992114,6cyclictest135650irq/27-eth010:04:063
16418992113,4cyclictest0-21swapper/312:01:303
16418992112,7cyclictest47-21rcuop/311:25:073
16418992112,7cyclictest14183-21sshd11:15:143
16418992112,6cyclictest1845-21nfsd09:14:383
1641799215,7cyclictest19561-21sshd10:15:072
16417992112,1cyclictest1848-21nfsd12:18:352
16416992114,6cyclictest22-21ksoftirqd/109:40:411
1641599217,6cyclictest3190-21sshd10:30:100
16415992111,4cyclictest3-21ksoftirqd/009:53:140
16415992110,5cyclictest3-21ksoftirqd/009:33:430
1641599210,20cyclictest0-21swapper/012:30:410
1641599210,19cyclictest31407-21sshd10:12:480
103752210,1sleep041ktimersoftd/011:24:540
1641899209,4cyclictest8835-21sshd11:57:523
1641899207,11cyclictest1959-21sshd09:35:223
16418992017,2cyclictest1847-21nfsd11:53:163
16418992015,3cyclictest3285-21cp09:16:193
16418992015,3cyclictest1959-21sshd11:30:373
16418992014,4cyclictest1846-21nfsd09:51:023
1641899200,17cyclictest44-21ksoftirqd/311:41:023
1641799201,9cyclictest23475-21sshd09:45:352
16417992015,3cyclictest2127-21sshd09:16:082
16416992011,6cyclictest19486-21id10:31:441
16415992017,2cyclictest3-21ksoftirqd/010:50:250
16415992013,5cyclictest3-21ksoftirqd/012:15:120
16415992011,4cyclictest3-21ksoftirqd/010:22:210
1641599200,19cyclictest0-21swapper/009:49:560
96862190,1sleep010-21rcuc/011:31:230
16418991915,2cyclictest184750nfsd10:46:403
16418991914,4cyclictest22334-21sshd10:42:053
1641799197,6cyclictest10950irq/23-ehci_hcd10:47:332
16417991912,3cyclictest1848-21nfsd10:00:152
1641799190,8cyclictest13736-21id10:31:072
16415991912,5cyclictest3-21ksoftirqd/009:35:370
1641599190,17cyclictest0-21swapper/010:38:210
1641599190,16cyclictest4592-21bash11:14:170
26432180,2sleep20-21swapper/211:07:122
16418991810,5cyclictest14013-21sshd11:08:253
1641799188,4cyclictest10950irq/23-ehci_hcd12:39:302
1641799185,3cyclictest31668-21sshd10:36:172
1641799181,16cyclictest1847-21nfsd11:02:442
16417991810,1cyclictest5153-21grep12:25:212
16417991810,1cyclictest5153-21grep12:25:212
1641699188,9cyclictest22-21ksoftirqd/110:56:331
1641699184,9cyclictest22-21ksoftirqd/111:37:491
1641699182,6cyclictest135650irq/27-eth007:15:241
16416991810,3cyclictest0-21swapper/107:15:011
1641599188,4cyclictest3-21ksoftirqd/011:15:200
1641599187,2cyclictest3-21ksoftirqd/011:29:290
16415991813,3cyclictest3-21ksoftirqd/010:45:290
1641599180,12cyclictest20508-21sshd11:02:090
1641599180,12cyclictest0-21swapper/012:27:070
1641599180,12cyclictest0-21swapper/012:27:070
1641599180,11cyclictest10501-21sshd11:51:110
16418991712,3cyclictest1845-21nfsd11:46:563
16418991711,3cyclictest135650irq/27-eth011:00:053
16418991710,5cyclictest9108-21sshd11:24:463
16417991710,5cyclictest32269-21sshd10:09:352
1641799170,6cyclictest1845-21nfsd11:22:522
16415991714,2cyclictest3-21ksoftirqd/010:07:380
1641899161,12cyclictest0-21swapper/307:42:593
16418991610,3cyclictest0-21swapper/307:29:303
1641799167,1cyclictest0-21swapper/210:41:382
1641799166,1cyclictest25849-21sshd09:55:442
16417991610,3cyclictest1959-21sshd09:30:442
1641799160,7cyclictest32544-21sshd11:56:552
1641699160,15cyclictest22-21ksoftirqd/108:00:211
1641699160,14cyclictest25438-21munin-node08:40:151
1641599160,14cyclictest0-21swapper/009:05:230
1641899158,3cyclictest1592-21users07:45:233
1641899150,3cyclictest28156-21diskstats07:35:133
1641899150,13cyclictest0-21swapper/308:00:173
1641799158,3cyclictest26012-21sshd11:46:052
1641799156,6cyclictest1847-21nfsd10:10:222
1641699154,9cyclictest0-21swapper/108:15:231
16416991511,3cyclictest10950irq/23-ehci_hcd07:24:551
1641699150,6cyclictest0-21swapper/107:56:381
1641699150,14cyclictest0-21swapper/107:50:151
1641699150,13cyclictest0-21swapper/107:40:181
1641599154,7cyclictest0-21swapper/011:08:020
1641599151,2cyclictest0-21swapper/011:35:570
16418991410,3cyclictest10950irq/23-ehci_hcd09:00:593
1641899140,13cyclictest0-21swapper/308:52:453
1641899140,12cyclictest0-21swapper/311:35:093
1641799143,3cyclictest10950irq/23-ehci_hcd08:34:212
1641799143,10cyclictest10950irq/23-ehci_hcd08:54:432
1641799142,4cyclictest144550irq/28-eth1-rx-07:29:302
1641799142,11cyclictest135650irq/27-eth008:26:022
1641799141,12cyclictest144550irq/28-eth1-rx-07:44:042
16417991410,3cyclictest10950irq/23-ehci_hcd07:48:052
1641699149,3cyclictest24119-21swap08:35:221
1641699147,5cyclictest15978-21munin-node08:20:171
1641699142,11cyclictest144550irq/28-eth1-rx-09:07:391
1641699140,13cyclictest0-21swapper/107:25:561
1641599146,5cyclictest0-21swapper/007:37:450
16415991410,3cyclictest8142-21kworker/0:008:29:410
1641599140,4cyclictest0-21swapper/007:18:280
1641899137,1cyclictest17952-21grep07:10:263
1641899137,1cyclictest13914-21cut08:15:153
1641899136,5cyclictest0-21swapper/308:10:143
1641899136,3cyclictest10950irq/23-ehci_hcd08:40:143
1641899132,3cyclictest135650irq/27-eth009:06:393
1641899130,5cyclictest0-21swapper/307:21:543
1641899130,4cyclictest2405-21runrttasks08:06:433
1641899130,4cyclictest23770-21cut08:35:193
1641899130,12cyclictest0-21swapper/308:25:483
1641899130,12cyclictest0-21swapper/308:20:413
1641899130,12cyclictest0-21swapper/307:52:013
1641899130,12cyclictest0-21swapper/307:15:033
1641799139,3cyclictest135650irq/27-eth009:05:572
1641799138,4cyclictest0-21swapper/208:46:542
1641799138,3cyclictest25545-21munin-node07:30:132
1641799131,11cyclictest8873-21sshd08:04:422
1641799131,10cyclictest9464-21df_inode08:05:152
1641799131,10cyclictest23964-21ntp_states08:35:212
1641799130,12cyclictest0-21swapper/209:04:562
1641799130,12cyclictest0-21swapper/208:58:252
1641799130,12cyclictest0-21swapper/207:56:512
1641799130,12cyclictest0-21swapper/207:38:012
1641799130,12cyclictest0-21swapper/207:20:032
1641799130,12cyclictest0-21swapper/207:12:392
1641699130,5cyclictest0-21swapper/108:51:031
1641699130,4cyclictest135650irq/27-eth008:55:231
1641699130,12cyclictest28103-21df_abs07:35:131
1641699130,12cyclictest22250-21runrttasks08:31:091
1641699130,12cyclictest1848-21nfsd08:45:051
1641699130,12cyclictest0-21swapper/108:25:141
1641699130,12cyclictest0-21swapper/108:11:541
1641699130,12cyclictest0-21swapper/107:46:321
1641699130,12cyclictest0-21swapper/107:30:071
1641699130,11cyclictest9140-21munin-node08:05:141
1641699130,11cyclictest2161-21df_inode09:00:141
1641599139,3cyclictest0-21swapper/007:53:200
1641599137,5cyclictest0-21swapper/008:12:000
1641599136,6cyclictest0-21swapper/008:56:460
1641599136,4cyclictest0-21swapper/008:51:500
1641599131,4cyclictest0-21swapper/007:47:420
16415991311,1cyclictest0-21swapper/008:47:360
16415991311,1cyclictest0-21swapper/008:35:100
16415991311,1cyclictest0-21swapper/008:05:410
1641599130,1cyclictest0-21swapper/008:41:190
1641599130,12cyclictest0-21swapper/008:33:110
1641599130,12cyclictest0-21swapper/008:21:330
1641599130,12cyclictest0-21swapper/007:41:090
1641599130,10cyclictest0-21swapper/008:17:340
1641899127,4cyclictest4741-21cpuspeed_turbos07:55:113
1641899120,3cyclictest32273-21munin-node08:55:163
1641899120,3cyclictest28381-21nfsd408:45:193
1641899120,11cyclictest22269-21runrttasks08:31:253
1641899120,11cyclictest0-21swapper/307:30:033
1641799128,3cyclictest14478-21irqstats08:15:182
1641799128,2cyclictest0-21swapper/208:40:142
1641799121,10cyclictest3072-21irqstats07:50:152
1641799120,3cyclictest16113-21grep08:20:122
1641799120,11cyclictest4062-21kworker/dying08:11:132
1641799120,11cyclictest0-21swapper/207:15:132
1641599125,1cyclictest0-21swapper/007:10:270
1641599120,11cyclictest0-21swapper/009:00:230
1641599120,11cyclictest0-21swapper/008:01:240
1641599120,11cyclictest0-21swapper/007:56:480
1641599120,11cyclictest0-21swapper/007:25:040
1641599120,11cyclictest0-21swapper/007:20:560
1641599117,3cyclictest0-21swapper/007:30:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional