You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-10 - 08:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Fri Oct 10, 2025 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2427921430,0sleep024265-21sshd21:58:060
1503121420,1sleep0900999cyclictest22:06:190
375621370,2sleep33753-21sshd22:29:483
1914821270,125sleep126950irq/26-ahci[00000:22:501
233621230,1sleep122-21ksoftirqd/122:54:231
874721220,2sleep20-21swapper/223:35:042
818021180,3sleep1901399cyclictest23:16:341
137721180,4sleep3902599cyclictest00:34:093
137721180,4sleep3902599cyclictest00:34:093
2091921130,1sleep30-21swapper/321:30:203
1248521100,1sleep00-21swapper/023:32:170
2024721090,2sleep10-21swapper/121:54:421
3129221070,0sleep331290-21cpuspeed_turbos21:25:133
2583621050,4sleep30-21swapper/323:36:373
8683210432,40sleep30-21swapper/319:08:563
3053921040,2sleep31848-21nfsd23:46:223
8743210333,40sleep10-21swapper/119:09:421
55452990,2sleep10-21swapper/123:37:461
137182960,1sleep013714-21sshd21:44:580
73562950,2sleep37353-21rm23:07:153
277622920,1sleep027761-21awk23:15:220
24932920,1sleep10-21swapper/122:02:071
54802900,1sleep05465-21sshd22:48:240
47392900,0sleep1901399cyclictest23:28:331
215342900,1sleep00-21swapper/023:45:270
138532900,4sleep10-21swapper/122:33:461
311962860,2sleep331189-21sshd22:41:303
311962860,2sleep331189-21sshd22:41:303
120932860,2sleep212091-21sshd23:57:032
184862850,4sleep3902599cyclictest00:19:333
84962840,1sleep10-21swapper/122:39:281
270742830,2sleep127071-21sshd23:52:181
138792830,2sleep213877-21sshd00:03:342
192052820,1sleep20-21swapper/200:29:222
128712820,2sleep112872-21bash00:09:451
139922810,2sleep213991-21sshd21:38:522
74232790,2sleep37420-21sshd22:39:213
44962790,1sleep10-21swapper/100:11:581
319722780,2sleep31846-21nfsd21:22:133
260382780,2sleep126039-21bash23:55:191
129642780,1sleep212959-21sshd22:52:112
279492760,3sleep2902099cyclictest21:24:572
145862760,2sleep114581-21bash22:55:271
187752750,2sleep30-21swapper/323:26:463
46292730,3sleep2902099cyclictest00:11:592
323232730,1sleep22405-21runrttasks23:43:292
173402730,3sleep10-21swapper/100:25:491
862327233,9sleep20-21swapper/219:08:112
195052720,2sleep10-21swapper/122:19:031
68132710,1sleep06796-21sshd23:07:120
262192690,1sleep126214-21sshd23:12:091
255482690,2sleep125539-21sshd23:02:481
139912690,1sleep10-21swapper/122:12:221
215862680,0sleep00-21swapper/023:54:580
128262670,4sleep1901399cyclictest21:20:241
18142580,2sleep20-21swapper/221:28:332
308252540,1sleep330824-21sshd21:43:293
848524934,11sleep00-21swapper/019:06:440
81512470,3sleep30-21swapper/322:30:113
9025994642,2cyclictest1848-21nfsd23:42:133
9025994540,3cyclictest16672-21sshd21:45:133
98552370,2sleep20-21swapper/223:22:532
900999360,25cyclictest10-21rcuc/023:42:130
313142360,0sleep00-21swapper/022:01:470
9009993416,10cyclictest3-21ksoftirqd/022:31:500
289622340,32sleep2135650irq/27-eth023:18:372
170392340,32sleep1135650irq/27-eth021:30:011
9013993214,9cyclictest0-21swapper/123:08:271
900999320,2cyclictest135650irq/27-eth000:23:350
152892321,29sleep1135650irq/27-eth021:42:001
900999318,15cyclictest3-21ksoftirqd/000:36:300
9009993121,9cyclictest135650irq/27-eth000:34:310
9009993121,9cyclictest135650irq/27-eth000:34:310
900999311,27cyclictest6185-21cp23:56:290
900999310,12cyclictest26535-21sshd22:56:370
900999304,22cyclictest14823-21sshd23:01:460
9009993013,11cyclictest3-21ksoftirqd/021:26:410
9009993010,10cyclictest0-21swapper/000:03:350
9025992923,4cyclictest1847-21nfsd00:06:343
900999299,17cyclictest3-21ksoftirqd/022:22:410
900999295,7cyclictest10-21rcuc/021:36:140
900999293,5cyclictest10974-21sshd22:12:030
900999291,9cyclictest27422-21sshd23:12:170
900999291,7cyclictest25166-21bash23:36:330
9009992913,15cyclictest17550-21sshd00:10:110
900999290,28cyclictest1005-21sshd22:51:010
161562290,2sleep20-21swapper/221:42:052
900999287,5cyclictest3-21ksoftirqd/021:34:390
900999285,16cyclictest0-21swapper/000:06:430
9009992814,13cyclictest3-21ksoftirqd/021:45:200
902599270,20cyclictest44-21ksoftirqd/300:13:113
9020992722,4cyclictest15393-21sshd21:14:402
901399271,6cyclictest20077-21bash22:28:131
900999272,18cyclictest587-21bash21:22:190
9009992719,7cyclictest3-21ksoftirqd/022:18:130
9009992715,10cyclictest3-21ksoftirqd/022:25:130
9009992713,8cyclictest3-21ksoftirqd/023:22:180
900999271,11cyclictest0-21swapper/000:16:360
9009992711,12cyclictest3-21ksoftirqd/022:37:350
900999271,10cyclictest0-21swapper/022:40:330
900999271,10cyclictest0-21swapper/022:40:330
304282270,25sleep3135650irq/27-eth022:20:073
268452270,2sleep326844-21sshd22:50:243
259722270,1sleep31848-21nfsd21:18:413
180612270,4sleep1901399cyclictest19:30:001
156392270,2sleep115636-21bash00:16:071
114092270,17sleep1135650irq/27-eth021:26:221
9009992618,7cyclictest3-21ksoftirqd/000:25:430
9009992610,3cyclictest3-21ksoftirqd/023:25:400
9009992610,15cyclictest3-21ksoftirqd/021:52:020
142322260,1sleep214223-21sshd22:40:012
9013992523,1cyclictest0-21swapper/122:43:331
9013992523,1cyclictest0-21swapper/122:43:331
900999250,5cyclictest0-21swapper/021:12:250
196822250,2sleep30-21swapper/321:51:313
9025992417,4cyclictest135650irq/27-eth023:54:093
901399241,2cyclictest135650irq/27-eth021:14:311
900999240,18cyclictest0-21swapper/021:00:160
210752240,2sleep321062-21sshd23:02:213
200682240,3sleep320065-21sshd23:14:453
182672240,3sleep30-21swapper/323:57:413
9020992319,3cyclictest4991-21sshd00:34:312
9020992319,3cyclictest4991-21sshd00:34:312
9013992313,8cyclictest5655-21gawk22:45:151
901399230,8cyclictest1844-21nfsd00:38:111
900999230,9cyclictest0-21swapper/019:43:490
900999230,19cyclictest14049-21cp21:17:310
9025992218,2cyclictest135650irq/27-eth021:55:423
9025992214,5cyclictest8297-21sshd23:31:533
902099224,6cyclictest9989-21rm22:45:362
9020992217,3cyclictest5086-21bash22:14:392
902099221,16cyclictest33-21ksoftirqd/221:48:062
9013992216,4cyclictest7151-21id21:56:261
901399221,5cyclictest30033-21sshd23:30:501
9013992213,8cyclictest0-21swapper/122:05:291
900999227,11cyclictest0-21swapper/019:48:070
49402220,2sleep24941-21missed_timers20:10:172
9025992118,2cyclictest1845-21nfsd22:16:013
9025992117,3cyclictest30212-21sshd22:07:473
9025992116,3cyclictest987-21sshd21:37:383
9025992111,8cyclictest7310-21sshd23:16:303
9025992110,9cyclictest135650irq/27-eth022:12:203
9020992117,3cyclictest10682-21sshd22:18:122
9020992112,7cyclictest0-21swapper/200:23:132
9013992116,3cyclictest4595-21sshd00:34:281
9013992116,3cyclictest4595-21sshd00:34:271
9013992116,3cyclictest3204-21cp21:46:571
9013992116,3cyclictest18396-21sshd23:48:241
9013992116,3cyclictest135650irq/27-eth023:41:311
9013992113,5cyclictest36-21rcuop/223:24:361
9013992111,7cyclictest32645-21sshd22:23:181
901399210,7cyclictest1959-21sshd21:19:591
900999210,12cyclictest0-21swapper/020:05:240
34192210,2sleep20-21swapper/220:05:352
9025992017,2cyclictest0-21swapper/322:57:383
9025992012,4cyclictest27355-21bash23:24:353
9020992016,3cyclictest29128-21sshd23:09:252
9020992015,3cyclictest15428-21bash23:01:482
9020992015,3cyclictest12333-21rm22:27:292
9020992014,5cyclictest31261-21sshd21:55:362
9020992014,5cyclictest0-21swapper/222:31:512
9020992014,4cyclictest27075-21sshd23:30:322
901399205,4cyclictest27601-21sshd21:37:031
9013992013,5cyclictest28579-21rm00:01:471
900999200,1cyclictest0-21swapper/020:10:170
900999200,1cyclictest0-21swapper/019:30:360
900999200,19cyclictest0-21swapper/020:40:400
900999200,15cyclictest20760-21gawk20:45:170
900999200,12cyclictest0-21swapper/021:05:190
900999200,12cyclictest0-21swapper/021:05:180
900999200,12cyclictest0-21swapper/019:25:240
159272200,1sleep20-21swapper/222:21:402
9025991916,2cyclictest893-21bash00:27:313
9025991916,2cyclictest1848-21nfsd00:01:583
9025991916,2cyclictest0-21swapper/300:35:523
9025991915,3cyclictest26158-21cp22:01:173
9025991913,3cyclictest5654-21ipmitool22:45:163
9025991912,4cyclictest6334-21sshd00:24:513
902099192,3cyclictest0-21swapper/222:58:312
9020991915,3cyclictest1848-21nfsd23:50:512
9020991914,4cyclictest7780-21sshd00:06:012
9020991913,4cyclictest1660-21sshd23:25:132
902099190,17cyclictest1848-21nfsd22:05:102
900999198,3cyclictest0-21swapper/019:16:440
900999191,11cyclictest22262-21kworker/0:120:30:570
900999190,7cyclictest0-21swapper/020:37:130
900999190,18cyclictest0-21swapper/020:54:070
900999190,18cyclictest0-21swapper/020:24:180
900999190,11cyclictest0-21swapper/020:00:320
9020991814,2cyclictest135650irq/27-eth000:38:422
9020991813,3cyclictest7378-21sshd00:15:202
9020991813,3cyclictest24683-21sshd22:01:072
9020991812,5cyclictest30583-21sshd21:31:112
9020991812,5cyclictest20276-21sshd23:11:362
9020991812,3cyclictest1848-21nfsd21:51:432
902099180,16cyclictest33-21ksoftirqd/223:45:282
900999187,4cyclictest0-21swapper/020:15:390
900999180,6cyclictest0-21swapper/019:22:420
900999180,1cyclictest0-21swapper/019:55:590
900999180,1cyclictest0-21swapper/019:10:370
900999180,17cyclictest0-21swapper/020:55:300
900999180,14cyclictest0-21swapper/019:50:060
900999180,14cyclictest0-21swapper/019:35:330
9020991712,4cyclictest25-21rcuop/121:18:432
900999170,12cyclictest0-21swapper/020:25:520
234372170,9sleep20-21swapper/222:40:442
234372170,9sleep20-21swapper/222:40:442
902599161,6cyclictest0-21swapper/321:10:233
902599161,10cyclictest135650irq/27-eth021:09:493
902599161,10cyclictest135650irq/27-eth021:09:483
902099164,11cyclictest10950irq/23-ehci_hcd20:23:082
902599151,13cyclictest0-21swapper/320:01:333
902099151,13cyclictest1927-21snmpd20:35:142
902099151,13cyclictest0-21swapper/220:55:152
902099150,13cyclictest0-21swapper/220:25:142
901399158,3cyclictest135650irq/27-eth019:35:141
901399152,1cyclictest10348-21swap19:10:231
9013991511,3cyclictest10950irq/23-ehci_hcd19:56:021
902599148,5cyclictest0-21swapper/320:23:043
902599142,11cyclictest1848-21nfsd20:53:233
902599142,10cyclictest1848-21nfsd20:40:133
902599141,5cyclictest0-21swapper/320:26:333
902599141,5cyclictest0-21swapper/319:16:593
902599141,12cyclictest1848-21nfsd20:46:013
902599141,12cyclictest0-21swapper/321:01:403
902599141,12cyclictest0-21swapper/320:57:353
902599141,12cyclictest0-21swapper/320:37:303
902599141,12cyclictest0-21swapper/320:13:003
902599141,12cyclictest0-21swapper/319:20:353
902599141,12cyclictest0-21swapper/319:14:273
902099148,5cyclictest0-21swapper/219:56:322
902099148,5cyclictest0-21swapper/219:47:082
902099147,6cyclictest0-21swapper/219:40:042
902099144,9cyclictest0-21swapper/220:54:062
902099141,5cyclictest0-21swapper/220:44:002
9020991412,1cyclictest0-21swapper/219:27:262
901399147,6cyclictest0-21swapper/119:40:191
901399147,6cyclictest0-21swapper/119:23:091
901399143,10cyclictest10950irq/23-ehci_hcd20:14:371
9013991410,3cyclictest135650irq/27-eth021:06:481
9013991410,3cyclictest135650irq/27-eth021:06:481
9013991410,3cyclictest10950irq/23-ehci_hcd19:46:511
901399140,13cyclictest1848-21nfsd19:15:081
901399140,13cyclictest0-21swapper/120:40:161
901399140,12cyclictest0-21swapper/120:50:171
274442140,0sleep127447-21cpuspeed_turbos19:50:121
902599139,3cyclictest1847-21nfsd19:51:103
902599138,4cyclictest0-21swapper/320:06:333
902599131,11cyclictest23856-21ntpq19:40:203
902599131,11cyclictest0-21swapper/320:15:123
902599131,11cyclictest0-21swapper/319:55:123
9025991311,1cyclictest0-21swapper/319:45:503
9025991311,1cyclictest0-21swapper/319:30:433
902099137,5cyclictest0-21swapper/219:32:452
902099135,4cyclictest10950irq/23-ehci_hcd21:00:152
902099134,3cyclictest135650irq/27-eth020:30:152
902099131,3cyclictest29397-21df21:05:142
902099131,3cyclictest29397-21df21:05:142
9020991312,0cyclictest0-21swapper/220:46:432
902099131,11cyclictest7988-21runrttasks20:15:472
902099131,11cyclictest1848-21nfsd19:20:142
9020991311,1cyclictest0-21swapper/220:03:592
9020991311,1cyclictest0-21swapper/219:50:072
9020991311,1cyclictest0-21swapper/219:35:332
901399132,10cyclictest135650irq/27-eth021:02:031
901399130,5cyclictest135650irq/27-eth019:30:221
901399130,5cyclictest0-21swapper/120:05:171
901399130,12cyclictest0-21swapper/120:57:311
901399130,12cyclictest0-21swapper/120:48:141
901399130,12cyclictest0-21swapper/120:35:161
901399130,12cyclictest0-21swapper/120:27:231
901399130,12cyclictest0-21swapper/120:24:411
901399130,12cyclictest0-21swapper/120:00:131
902599120,11cyclictest17577-21runrttasks19:25:593
902599120,11cyclictest0-21swapper/320:30:203
902599120,10cyclictest20699-21df19:35:123
902099127,4cyclictest0-21swapper/219:11:182
902099120,11cyclictest11319-21runrttasks19:15:102
901399120,3cyclictest6984-21hddtemp_smartct20:15:151
901399120,10cyclictest0-21swapper/120:30:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional