You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-28 - 19:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack5slot0.osadl.org (updated Tue Oct 28, 2025 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
614221250,2sleep26141-21sshd11:01:052
1375421210,3sleep01657499cyclictest09:54:340
753221080,3sleep21657699cyclictest10:17:152
3119921030,1sleep31657799cyclictest09:15:443
1543529734,43sleep30-21swapper/307:05:273
287212950,2sleep30-21swapper/310:02:393
43242940,0sleep00-21swapper/010:00:150
21672910,3sleep01657499cyclictest12:34:130
278842900,2sleep20-21swapper/210:05:502
302362880,0sleep21847-21nfsd11:58:122
62382870,2sleep20-21swapper/211:18:162
274822870,2sleep20-21swapper/210:26:092
155722870,2sleep115570-21sshd11:42:551
56322860,2sleep35623-21cp11:38:293
281182840,2sleep128117-21sshd10:19:361
1605628436,44sleep10-21swapper/107:07:011
244352830,2sleep324431-21bash12:21:063
194102830,1sleep019409-21bash09:31:320
162392830,2sleep00-21swapper/010:58:550
91632820,2sleep30-21swapper/310:20:493
260482820,3sleep10-21swapper/112:33:261
39892810,1sleep23979-21rm11:45:142
124482800,1sleep312203-21sshd10:07:403
71342790,0sleep00-21swapper/011:15:010
307292790,2sleep21848-21nfsd09:42:502
221382780,1sleep30-21swapper/310:11:593
196702780,0sleep019667-21seq11:53:410
129022780,1sleep012887-21sshd11:59:470
74602760,2sleep30-21swapper/312:25:463
275022760,2sleep20-21swapper/211:44:182
51472750,1sleep20-21swapper/212:25:342
320062750,2sleep132001-21bash09:26:001
164392750,1sleep20-21swapper/210:14:562
85342740,2sleep20-21swapper/210:34:282
315132740,0sleep31847-21nfsd11:51:283
1615127232,11sleep20-21swapper/207:08:162
270642670,3sleep01657499cyclictest12:11:360
93922660,3sleep21657699cyclictest10:47:592
281602660,3sleep11657599cyclictest09:52:421
180292650,2sleep21848-21nfsd12:23:412
1607525633,11sleep00-21swapper/007:07:160
255982390,2sleep22405-21runrttasks12:11:272
237522390,1sleep01848-21nfsd10:25:440
82832360,34sleep3135650irq/27-eth010:58:063
106022350,1sleep10-21swapper/111:56:071
9372320,4sleep31657799cyclictest09:43:063
16575993225,5cyclictest19316-21sshd11:19:391
137302320,1sleep10-21swapper/109:13:581
16574993112,3cyclictest3-21ksoftirqd/009:16:290
66152300,1sleep30-21swapper/309:26:483
16574993021,7cyclictest3-21ksoftirqd/010:13:590
297812280,26sleep3135650irq/27-eth011:58:103
91112252,21sleep1135650irq/27-eth012:09:551
16575992513,4cyclictest10950irq/23-ehci_hcd10:09:361
16575992511,3cyclictest9329-21sshd10:30:551
155572250,23sleep1135650irq/27-eth011:22:291
59562240,21sleep3135650irq/27-eth012:12:563
297052240,1sleep00-21swapper/010:33:180
1657799247,0cyclictest0-21swapper/310:44:563
16574992412,11cyclictest3-21ksoftirqd/009:49:010
48562230,4sleep21657699cyclictest10:00:162
282912230,21sleep1135650irq/27-eth010:36:241
1657499237,4cyclictest3-21ksoftirqd/012:28:290
1657499235,8cyclictest0-21swapper/012:15:390
16574992312,8cyclictest3-21ksoftirqd/009:10:140
1657499230,18cyclictest0-21swapper/010:37:290
134202230,2sleep20-21swapper/209:24:092
74742220,20sleep1135650irq/27-eth012:19:321
1657799227,1cyclictest0-21swapper/311:06:453
1657799220,9cyclictest18888-21sshd10:28:433
1657799220,20cyclictest11067-21sshd09:34:103
1657599220,4cyclictest15941-21sshd12:23:281
1657599220,13cyclictest135650irq/27-eth010:40:391
1657499226,8cyclictest3-21ksoftirqd/010:42:560
16574992218,2cyclictest8-21rcu_preempt12:38:010
16574992215,3cyclictest3-21ksoftirqd/012:23:250
16574992211,4cyclictest3-21ksoftirqd/010:46:400
1657499220,13cyclictest0-21swapper/010:19:260
295502210,1sleep229551-21missed_timers12:15:192
267102211,18sleep2135650irq/27-eth012:08:192
1657799214,1cyclictest0-21swapper/312:32:473
16577992118,2cyclictest0-21swapper/312:05:103
1657699215,2cyclictest0-21swapper/211:20:412
16575992118,2cyclictest17290-21bash11:39:441
16575992111,2cyclictest10950irq/23-ehci_hcd09:33:351
1657599210,16cyclictest135650irq/27-eth012:10:521
16574992111,8cyclictest3-21ksoftirqd/009:41:330
1657499210,15cyclictest0-21swapper/010:07:140
59642203,15sleep110950irq/23-ehci_hcd10:24:011
21032200,2sleep12405-21runrttasks09:16:111
1657799207,5cyclictest10950irq/23-ehci_hcd11:40:343
1657799207,4cyclictest22211-21sshd09:58:463
1657799202,7cyclictest417-21jbd2/dm-0-809:47:163
16577992011,6cyclictest1847-21nfsd11:46:153
1657799200,8cyclictest1846-21nfsd09:12:293
1657599209,2cyclictest135650irq/27-eth011:46:021
16575992012,5cyclictest3370-21sshd11:28:031
1657599200,16cyclictest31305-21dump-pmu-power10:30:001
1657499209,3cyclictest2889-21sshd12:09:140
1657499204,2cyclictest10728-21bash12:03:150
16574992016,3cyclictest3-21ksoftirqd/011:25:150
16574992015,3cyclictest3-21ksoftirqd/009:35:470
16574992012,6cyclictest3-21ksoftirqd/011:42:560
1657499200,15cyclictest11641-21sshd11:12:000
1657799194,14cyclictest0-21swapper/309:07:033
1657799192,15cyclictest18434-21sshd09:24:433
1657799191,3cyclictest9289-21nscd10:34:163
16577991912,4cyclictest0-21swapper/310:49:593
1657799190,4cyclictest29516-21sshd10:16:093
1657799190,2cyclictest0-21swapper/309:36:303
1657699196,7cyclictest5896-21kworker/u8:210:41:362
1657699194,5cyclictest3043-21sshd09:16:182
1657599199,1cyclictest5717-21sshd12:25:371
1657599196,6cyclictest10950irq/23-ehci_hcd11:51:571
16575991914,3cyclictest0-21swapper/110:11:461
1657599190,7cyclictest135650irq/27-eth009:40:501
1657599190,15cyclictest1902-21nscd12:36:581
16574991915,2cyclictest3-21ksoftirqd/011:30:300
16574991915,2cyclictest3-21ksoftirqd/011:07:450
16574991914,4cyclictest3-21ksoftirqd/010:50:320
16574991914,3cyclictest3-21ksoftirqd/010:24:500
166742182,14sleep1135650irq/27-eth008:20:151
1657799180,6cyclictest20620-21sshd10:38:593
1657799180,5cyclictest30267-21rm12:18:363
1657799180,5cyclictest23962-21sshd12:36:053
1657799180,3cyclictest0-21swapper/311:17:273
1657799180,15cyclictest1901-21nscd11:30:083
1657699187,5cyclictest135650irq/27-eth012:30:132
16576991810,6cyclictest22009-21sshd09:52:032
1657599180,16cyclictest1848-21nfsd11:30:051
1657599180,10cyclictest0-21swapper/110:48:391
16574991814,3cyclictest3-21ksoftirqd/011:47:230
16574991814,3cyclictest3-21ksoftirqd/009:27:350
1657799172,3cyclictest1959-21sshd11:24:573
16577991713,3cyclictest3313-21sshd11:28:043
1657799170,5cyclictest5978-21sshd10:50:573
1657699173,6cyclictest144550irq/28-eth1-rx-11:25:542
1657699171,5cyclictest0-21swapper/208:42:182
1657699171,4cyclictest0-21swapper/209:36:512
16576991711,4cyclictest27427-21sshd09:45:492
1657699170,5cyclictest21496-21sshd12:35:512
1657699170,4cyclictest14699-21sshd11:32:362
1657599179,4cyclictest20461-21sshd09:58:361
1657599170,8cyclictest1845-21nfsd10:01:371
1657499171,9cyclictest2405-21runrttasks09:59:450
16574991714,2cyclictest3-21ksoftirqd/011:36:050
16574991714,2cyclictest3-21ksoftirqd/011:03:500
16574991713,3cyclictest3-21ksoftirqd/011:21:230
107862170,2sleep30-21swapper/309:50:523
1657799169,4cyclictest4998-21sshd11:11:123
1657799169,2cyclictest1847-21nfsd12:03:143
1657799160,14cyclictest0-21swapper/308:25:253
1657799160,14cyclictest0-21swapper/307:45:143
1657699168,6cyclictest26795-21sshd10:22:502
16576991613,2cyclictest13053-21sshd09:30:512
16576991611,3cyclictest1848-21nfsd09:56:512
1657699160,5cyclictest25428-21bash10:59:512
1657699160,15cyclictest0-21swapper/211:38:072
1657699160,14cyclictest0-21swapper/209:10:092
1657599168,6cyclictest29634-21sshd10:50:101
1657599168,0cyclictest0-21swapper/109:49:501
16575991610,1cyclictest15109-21sshd10:55:201
1657599160,5cyclictest16546-21sshd11:12:331
1657599160,4cyclictest914-21bash09:39:491
1657499160,15cyclictest0-21swapper/007:44:400
1657799158,6cyclictest0-21swapper/308:30:243
1657799153,11cyclictest10950irq/23-ehci_hcd07:22:103
1657799150,13cyclictest1927-21snmpd11:03:023
1657799150,13cyclictest1927-21snmpd07:43:433
1657699159,1cyclictest16805-21id10:38:352
1657699151,13cyclictest0-21swapper/208:54:042
1657699151,10cyclictest0-21swapper/211:51:332
1657699150,7cyclictest30137-21sshd11:10:262
1657699150,14cyclictest0-21swapper/210:50:542
1657699150,13cyclictest22361-21sshd09:25:082
1657699150,13cyclictest17299-21sshd12:00:122
1657699150,13cyclictest0-21swapper/211:07:362
1657699150,13cyclictest0-21swapper/208:05:182
1657599154,3cyclictest10950irq/23-ehci_hcd12:02:171
1657599150,6cyclictest19789-21sshd11:02:321
1657599150,3cyclictest6610-21sshd09:20:031
1657599150,13cyclictest28149-21cpu07:35:131
1657499150,8cyclictest21066-21sshd09:25:000
1657499150,13cyclictest0-21swapper/008:35:170
1657499150,12cyclictest0-21swapper/009:04:310
1657799143,3cyclictest30068-21cpuspeed_turbos08:50:133
1657799143,10cyclictest10950irq/23-ehci_hcd08:15:423
16577991410,3cyclictest10950irq/23-ehci_hcd08:10:263
1657699149,3cyclictest3726-21munin-node07:50:202
1657699143,10cyclictest10950irq/23-ehci_hcd08:20:232
1657699141,12cyclictest1848-21nfsd07:48:282
1657699141,11cyclictest0-21swapper/207:55:192
1657699140,6cyclictest0-21swapper/207:13:422
1657699140,5cyclictest0-21swapper/208:55:292
1657699140,13cyclictest0-21swapper/209:00:352
1657699140,12cyclictest26032-21sh07:30:152
1657699140,10cyclictest135650irq/27-eth007:25:152
16575991412,1cyclictest0-21swapper/107:45:131
16575991410,3cyclictest10950irq/23-ehci_hcd07:33:251
1657599140,5cyclictest23752-21diskstats07:25:141
1657599140,13cyclictest18021-21sshd11:05:441
1657599140,13cyclictest0-21swapper/108:15:261
1657599140,13cyclictest0-21swapper/107:57:351
1657599140,10cyclictest0-21swapper/108:51:431
1657499148,2cyclictest0-21swapper/009:05:190
1657499147,5cyclictest0-21swapper/007:31:100
1657499140,4cyclictest0-21swapper/008:28:230
1657499140,1cyclictest17117-21strings08:20:180
1657499140,10cyclictest0-21swapper/008:49:220
1657499140,10cyclictest0-21swapper/008:15:160
1657799132,9cyclictest0-21swapper/307:30:273
16577991311,1cyclictest0-21swapper/309:01:463
16577991310,1cyclictest0-21swapper/307:15:303
1657799130,4cyclictest24621-21users08:35:243
1657799130,12cyclictest0-21swapper/308:21:513
1657799130,12cyclictest0-21swapper/307:51:543
1657799130,12cyclictest0-21swapper/307:38:323
1657799130,12cyclictest0-21swapper/307:29:283
1657799130,11cyclictest7337-21df_inode08:00:143
1657799130,11cyclictest620-21needreboot08:55:203
1657799130,11cyclictest27862-21df_inode08:45:123
1657799130,11cyclictest17354-21iostat_ios07:10:163
1657699136,6cyclictest0-21swapper/208:15:542
1657699131,3cyclictest0-21swapper/208:28:382
1657699131,11cyclictest1848-21nfsd07:40:172
1657699131,11cyclictest0-21swapper/208:35:222
16576991311,1cyclictest0-21swapper/207:16:062
1657699130,12cyclictest5249-21grep09:05:192
1657699130,12cyclictest0-21swapper/208:46:132
1657699130,12cyclictest0-21swapper/208:30:572
1657699130,12cyclictest0-21swapper/208:00:512
1657699130,11cyclictest12726-21ntp_states08:10:192
1657599137,5cyclictest0-21swapper/108:01:251
1657599132,3cyclictest10950irq/23-ehci_hcd09:07:481
16575991311,1cyclictest0-21swapper/108:55:011
16575991311,1cyclictest0-21swapper/108:10:101
1657599130,4cyclictest20396-21taskset08:29:211
1657599130,4cyclictest17595-21awk07:10:201
1657599130,12cyclictest3837-21threads07:50:231
1657599130,12cyclictest0-21swapper/108:48:181
1657599130,12cyclictest0-21swapper/108:35:281
1657599130,12cyclictest0-21swapper/108:07:241
1657599130,12cyclictest0-21swapper/107:45:001
1657599130,12cyclictest0-21swapper/107:24:211
1657599130,11cyclictest2405-21runrttasks07:15:091
1657499139,3cyclictest27722-21kworker/0:208:31:470
1657499138,2cyclictest0-21swapper/007:52:160
1657499137,5cyclictest0-21swapper/008:00:400
1657499137,4cyclictest0-21swapper/008:44:250
1657499133,1cyclictest0-21swapper/007:56:190
16574991311,1cyclictest0-21swapper/007:22:430
1657499130,12cyclictest0-21swapper/008:05:230
1657499130,12cyclictest0-21swapper/007:12:160
1657499130,10cyclictest0-21swapper/007:36:420
1657799127,3cyclictest9386-21atenall_r5power08:05:123
1657799120,3cyclictest26021-21iostat08:40:173
1657799120,3cyclictest0-21swapper/307:55:153
1657699121,10cyclictest0-21swapper/207:35:152
1657699120,10cyclictest0-21swapper/207:20:112
1657599120,4cyclictest0-21swapper/108:31:101
1657599120,11cyclictest0-21swapper/109:00:301
1657599120,10cyclictest0-21swapper/108:40:131
1657499126,2cyclictest30552-21if_err_eth008:50:160
1657499125,1cyclictest0-21swapper/007:25:220
16574991210,1cyclictest0-21swapper/008:14:020
1657499120,11cyclictest0-21swapper/008:55:450
1657499120,11cyclictest0-21swapper/007:47:560
1657499120,11cyclictest0-21swapper/007:15:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional