You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 06:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot2.osadl.org (updated Thu Apr 25, 2024 00:57:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9516994001229191,10819cyclictest0-21swapper/322:25:473
951599399494440,24697cyclictest35392irq/88-enp1s0-T22:25:482
951399398894373,35514cyclictest9511-21cyclictest22:25:471
9516993857134468,1262cyclictest0-21swapper/300:35:593
9515993850634405,1255cyclictest0-21swapper/200:35:592
9512993763334489,11cyclictest31955-21kworker/0:100:36:000
9513993748434338,1261cyclictest0-21swapper/100:36:001
9512993633636324,10cyclictest20700-21kworker/0:000:07:560
951299355140,24702cyclictest14565-21kworker/0:122:25:470
9513993384014195,19641cyclictest0-21swapper/100:04:551
9512993379114147,9cyclictest20700-21kworker/0:000:04:550
9516993377814126,19650cyclictest0-21swapper/300:04:553
9515993371214062,19647cyclictest11690-21systemd-udevd00:04:552
9516993291632890,23cyclictest24651-21systemd-udevd00:07:563
951599320871863,21440cyclictest33-21ksoftirqd/200:11:332
9512993093330918,12cyclictest121rcu_preempt23:08:490
9515993084230832,7cyclictest321ktimersoftd/223:08:492
9513993069330680,11cyclictest0-21swapper/100:25:481
9512993064430632,9cyclictest5627-21kworker/0:300:25:480
9516993061830610,5cyclictest243-21systemd-udevd00:25:483
9515993055830547,8cyclictest2740-21systemd-udevd00:25:472
951299301717684,13709cyclictest111rcuc/000:11:330
951699293641201,28158cyclictest0-21swapper/300:42:373
951599292981137,6102cyclictest27801-21systemd-udevd00:42:372
951399280721,28068cyclictest0-21swapper/122:17:451
9515992794410596,17345cyclictest2160-21upowerd22:17:462
951599269617176,19782cyclictest33-21ksoftirqd/223:26:422
951299267014587,22111cyclictest23497-21kworker/0:222:50:060
951599266144502,14813cyclictest71152sleep222:50:062
9515992562514568,11054cyclictest29153-21systemd-udevd00:33:342
951399255703170,22014cyclictest0-21swapper/100:41:181
9513992555614505,9177cyclictest1126-21runrttasks00:33:331
951699253217665,8876cyclictest391rcuc/300:11:333
951299251463121,22013cyclictest31955-21kworker/0:100:41:180
9513992482424812,10cyclictest0-21swapper/123:02:161
9512992477624764,10cyclictest1807-21kworker/0:023:02:160
9516992475924743,13cyclictest0-21swapper/323:02:173
9513992475018698,6049cyclictest0-21swapper/100:10:551
951299247141222,5772cyclictest23497-21kworker/0:222:46:550
9515992469824680,16cyclictest0-21swapper/223:02:172
951699246941201,23489cyclictest0-21swapper/322:46:543
951599246271137,12995cyclictest0-21swapper/222:46:542
951599245634053,20507cyclictest0-21swapper/200:07:222
951399245591070,5771cyclictest0-21swapper/122:46:551
951399244973987,20507cyclictest243-21systemd-udevd00:07:221
9512992402410088,13933cyclictest5627-21kworker/0:300:20:340
9512992387512928,10944cyclictest2701-21kworker/0:200:31:180
9516992385612906,10947cyclictest0-21swapper/300:31:183
9515992351711381,12127cyclictest21503-21systemd-udevd00:22:472
9516992338011244,16cyclictest26664-21apache200:22:473
951399227524314,15844cyclictest0-21swapper/122:39:271
951299227064265,18438cyclictest23497-21kworker/0:222:39:270
951699226378315,14319cyclictest0-21swapper/323:19:103
9516992258818050,4535cyclictest243-21systemd-udevd00:18:483
951599225748252,14310cyclictest30165-21systemd-udevd23:19:102
9515992251917987,4072cyclictest0-21swapper/200:18:482
951399225058186,14316cyclictest0-21swapper/123:19:101
951399222399938,12298cyclictest12932-21systemd-udevd00:20:341
9512992214118072,4067cyclictest2701-21kworker/0:200:18:480
9513992199117920,4069cyclictest0-21swapper/100:18:471
951299214211731,10073cyclictest23497-21kworker/0:223:19:100
951299211099344,11752cyclictest23497-21kworker/0:222:55:400
951699210869323,11760cyclictest0-21swapper/322:55:403
951599210279259,10cyclictest0-21swapper/222:55:412
951399209549192,11753cyclictest17104-21systemd-udevd22:55:411
95159920664485,15469cyclictest21247-21systemd-udevd23:50:062
951699206144567,16045cyclictest0-21swapper/322:50:063
9513992024718088,2156cyclictest0-21swapper/123:42:501
9512992019918040,2150cyclictest0-21swapper/023:42:500
951399201830,15468cyclictest35492irq/89-enp1s0-T23:50:071
9516992018018018,2159cyclictest0-21swapper/323:42:493
951299201741,4cyclictest5627-21kworker/0:323:50:060
9515992011217954,11cyclictest26344-21systemd-udevd23:42:492
951699200177602,12412cyclictest25835-21systemd-udevd23:31:163
951599199732371,1465cyclictest12323-21apache223:35:102
951599199487538,12407cyclictest0-21swapper/223:31:162
951399199092304,1466cyclictest5066-21apt-get23:35:111
951399198777471,3603cyclictest0-21swapper/123:31:161
951699198382235,1466cyclictest0-21swapper/323:35:113
951399196935961,3989cyclictest0-21swapper/122:50:061
951299196507509,12137cyclictest11399-21kworker/0:023:26:420
951299194542110,17341cyclictest0-21swapper/022:17:450
951699194352088,17344cyclictest0-21swapper/322:17:453
951399192654775,14488cyclictest0-21swapper/123:21:041
9516991923917533,17cyclictest0-21swapper/322:39:273
951699191984699,14489cyclictest0-21swapper/323:21:043
9512991911719104,11cyclictest25239-21kworker/0:123:31:440
951399188222723,16096cyclictest0-21swapper/123:27:561
951399188035538,13257cyclictest231rcuc/122:33:211
951299187545493,13259cyclictest111rcuc/022:33:210
9512991870218312,387cyclictest25239-21kworker/0:123:45:110
9516991869218679,11cyclictest0-21swapper/323:45:113
9513991866110098,11cyclictest0-21swapper/100:49:181
9515991863118611,18cyclictest33-21ksoftirqd/223:45:112
9512991861510049,10cyclictest2391-21kworker/0:300:49:180
9516991859410027,12cyclictest0-21swapper/300:49:183
9513991857818569,7cyclictest25-21ksoftirqd/123:08:491
9516991851518501,12cyclictest7056-21chrt23:08:483
951699184921208,3cyclictest0-21swapper/322:33:213
9516991843511873,6181cyclictest11658-21systemd-udevd21:26:133
951599184201144,2cyclictest0-21swapper/222:33:212
9513991837318361,10cyclictest0-21swapper/123:45:121
9515991822511402,6820cyclictest6137-21systemd-udevd00:45:252
9512991770317691,10cyclictest29975-21kworker/0:021:12:510
9515991748817469,16cyclictest2160-21upowerd22:39:272
951699173143444,13862cyclictest401ktimersoftd/321:59:353
951599171272322,14787cyclictest11859-21systemd-udevd21:27:202
9512991707617063,11cyclictest11399-21kworker/0:023:35:030
951399170562255,3cyclictest0-21swapper/121:27:211
951599169628868,8091cyclictest0-21swapper/219:17:122
951399168998801,8094cyclictest243-21systemd-udevd19:17:121
951699168575166,11688cyclictest0-21swapper/323:29:543
951299168548752,8099cyclictest31162-21kworker/0:219:17:130
951699168318731,8097cyclictest0-21swapper/319:17:133
9512991608415691,13cyclictest23967-21kworker/0:221:59:340
9516991603316018,13cyclictest0-21swapper/323:50:063
9513991575115739,10cyclictest0-21swapper/121:59:351
9515991562015606,12cyclictest0-21swapper/221:59:352
951299155750,14684cyclictest9124-21kworker/0:221:22:270
9516991556814662,903cyclictest0-21swapper/321:22:273
9515991550314599,901cyclictest2160-21upowerd21:22:272
9513991542214532,888cyclictest0-21swapper/121:22:271
9513991514210735,4405cyclictest241ktimersoftd/122:12:121
951299150206145,8871cyclictest111rcuc/022:12:120
9512991461014596,12cyclictest9124-21kworker/0:221:27:200
9516991458314571,10cyclictest401ktimersoftd/323:14:193
951399142461,14242cyclictest0-21swapper/121:39:501
9512991419513534,658cyclictest0-21swapper/021:39:500
9516991417613513,661cyclictest0-21swapper/321:39:513
9515991411613449,663cyclictest17965-21systemd-udevd21:39:512
9516991406313374,686cyclictest0-21swapper/322:40:143
951399138192150,9595cyclictest19110-21systemd-udevd23:14:191
9515991380913793,14cyclictest0-21swapper/223:57:562
9513991374413727,14cyclictest16304-21systemd-udevd23:57:561
9513991345913443,13cyclictest28552-21systemd-udevd22:40:141
9512991340613395,9cyclictest23497-21kworker/0:222:40:140
9515991332613310,13cyclictest28507-21cstates22:40:132
951599131841,6593cyclictest0-21swapper/223:24:162
951299130646480,6581cyclictest0-21swapper/023:24:160
9516991266010710,1948cyclictest0-21swapper/321:42:153
9515991259410646,1945cyclictest0-21swapper/221:42:152
9513991252810579,1946cyclictest0-21swapper/121:42:141
951599122625914,11cyclictest19111-21systemd-udevd23:14:192
9515991213812128,7cyclictest8100-21ls22:15:012
951699116810,1521cyclictest2160-21upowerd22:12:133
951299116670,9595cyclictest7578-21kworker/0:123:14:190
951299106242731,12cyclictest0-21swapper/023:58:400
951699106102709,7898cyclictest19126-21systemd-udevd23:58:403
951399104372105,6cyclictest2225-21systemd-udevd22:05:141
95159999247316,2604cyclictest0-21swapper/221:46:082
95129998381620,4cyclictest9124-21kworker/0:221:42:140
95159996261972,5cyclictest2224-21cstates22:05:142
95129992822930,6349cyclictest17773-21kworker/0:020:41:130
95169992732908,6361cyclictest0-21swapper/320:41:143
95159992042845,6355cyclictest22040-21systemd-udevd20:41:132
95139991292779,6347cyclictest0-21swapper/120:41:131
95139991011576,7521cyclictest0-21swapper/121:12:521
95169990351507,7525cyclictest0-21swapper/321:12:523
95169990188888,128cyclictest0-21swapper/322:05:153
95159989661442,7521cyclictest0-21swapper/221:12:522
95129985858572,11cyclictest15680-21kworker/0:119:24:450
95169985678551,13cyclictest0-21swapper/319:24:463
95159984978487,8cyclictest0-21swapper/219:24:452
95169984380,2cyclictest0-21swapper/322:21:153
95159983766717,1656cyclictest0-21swapper/222:21:162
95139983136650,1660cyclictest0-21swapper/122:21:151
95129982666601,1662cyclictest6200-21kworker/0:222:21:150
95139982128201,9cyclictest231rcuc/121:31:191
95129981618153,6cyclictest111rcuc/021:31:190
95139976854,7660cyclictest0-21swapper/121:46:081
95129976267615,9cyclictest0-21swapper/021:46:080
95169976087594,12cyclictest0-21swapper/321:46:083
95129973767247,126cyclictest30093-21kworker/0:022:05:140
95169967614280,2478cyclictest0-21swapper/321:50:483
95159966954217,2475cyclictest0-21swapper/221:50:492
95139966304151,2476cyclictest0-21swapper/121:50:481
95169962805,6272cyclictest243-21systemd-udevd21:17:293
95159962196205,12cyclictest0-21swapper/221:17:282
95129962046189,12cyclictest21106-21kworker/0:121:50:480
95139961536134,15cyclictest1437-21avahi-daemon21:17:281
95129961016086,13cyclictest29975-21kworker/0:021:17:280
95139959555937,15cyclictest17451-21/usr/sbin/munin19:20:251
95169948684847,18cyclictest0-21swapper/321:31:193
95159948044783,18cyclictest243-21systemd-udevd21:31:202
95139944962898,1595cyclictest0-21swapper/119:31:061
95129944502849,1598cyclictest19384-21kworker/0:019:31:060
95169944222829,1590cyclictest0-21swapper/319:31:073
95159943572765,1589cyclictest0-21swapper/219:31:072
95169941654146,16cyclictest243-21systemd-udevd21:09:143
95159940944082,9cyclictest0-21swapper/221:09:142
95139940264015,9cyclictest0-21swapper/121:09:141
95139940173625,389cyclictest0-21swapper/119:12:331
95129939663576,10cyclictest28556-21kworker/0:019:12:330
95169939493555,391cyclictest0-21swapper/319:12:323
95159938823491,388cyclictest0-21swapper/219:12:332
9515992552359,9cyclictest33-21ksoftirqd/220:36:432
95169921442129,12cyclictest0-21swapper/320:36:433
12369121292112,10phc2sys18697-21systemd-udevd19:05:023
95139920752071,2cyclictest0-21swapper/120:36:441
95129920272021,4cyclictest17773-21kworker/0:020:36:430
95159919143,1909cyclictest311rcuc/220:53:552
95129918061801,3cyclictest111rcuc/020:53:550
95169917831778,3cyclictest391rcuc/320:53:553
95169916420,1639cyclictest42150irq/95-enp4s0-T19:40:293
95159915811576,3cyclictest0-21swapper/219:40:292
95139915171510,4cyclictest23818-21/usr/sbin/munin19:40:291
95129914661461,3cyclictest15680-21kworker/0:119:40:290
95139914541439,12cyclictest243-21systemd-udevd20:53:541
95129912561248,5cyclictest19384-21kworker/0:019:29:360
9515991253865,384cyclictest17779-21latency_hist20:35:012
95129912381233,3cyclictest0-21swapper/020:45:420
95169912321227,3cyclictest0-21swapper/319:29:363
95159912321225,4cyclictest23622-21ls20:45:232
95159912201215,3cyclictest0-21swapper/219:29:372
95169912151210,3cyclictest0-21swapper/320:45:433
9513991187798,386cyclictest17780-21latency_hist20:35:011
95139911751157,15cyclictest23630-21systemd-udevd20:45:231
95139911571149,4cyclictest0-21swapper/119:29:361
9512991135750,382cyclictest32713-21kworker/0:120:35:010
9516991114729,382cyclictest0-21swapper/320:35:003
951699965952,9cyclictest0-21swapper/320:22:013
9512999580,955cyclictest24656-21kworker/0:221:03:340
951699945938,4cyclictest0-21swapper/321:03:343
951599939934,3cyclictest0-21swapper/220:56:272
951599901888,10cyclictest12641-21systemd-udevd20:22:002
951599879874,3cyclictest0-21swapper/221:03:332
951399877867,6cyclictest2160-21upowerd20:56:271
951299855377,475cyclictest0-21swapper/021:09:140
951399828821,5cyclictest0-21swapper/120:22:001
951299824819,3cyclictest0-21swapper/020:56:280
951399812807,3cyclictest0-21swapper/121:03:331
951699803798,3cyclictest0-21swapper/320:56:273
951299674669,3cyclictest101ktimersoftd/019:40:000
9515996593,2cyclictest0-21swapper/222:00:072
951399600595,3cyclictest0-21swapper/122:00:071
951599589582,4cyclictest0-21swapper/219:40:002
951299552546,4cyclictest30093-21kworker/0:022:00:070
951699532525,4cyclictest31627-21systemd-udevd22:00:073
951399521515,4cyclictest0-21swapper/119:40:001
234842459448,5sleep20-21swapper/219:05:462
9515994110,407cyclictest2160-21upowerd20:05:452
951699407382,22cyclictest23570-21cron19:40:003
9515994050,2cyclictest0-21swapper/219:49:402
951399404378,22cyclictest32704-21systemd-udevd19:55:331
9513994020,398cyclictest243-21systemd-udevd19:52:041
951699401389,9cyclictest0-21swapper/319:49:403
951599401378,19cyclictest7793-21systemd-udevd20:10:382
951599400380,17cyclictest0-21swapper/219:52:042
951599397376,18cyclictest32702-21ls19:55:332
951699396372,21cyclictest0-21swapper/320:05:023
9516993951,390cyclictest0-21swapper/320:15:053
9516993950,391cyclictest243-21systemd-udevd19:55:353
2506023942,387sleep10-21swapper/119:06:041
951299393386,4cyclictest25564-21kworker/0:219:52:040
951699392387,3cyclictest0-21swapper/319:52:143
9516993922,386cyclictest7818-21systemd-udevd20:10:543
9513993921,387cyclictest243-21systemd-udevd19:49:401
951399391386,3cyclictest0-21swapper/120:05:451
9513993910,388cyclictest0-21swapper/120:10:391
9512993910,389cyclictest25564-21kworker/0:219:46:350
9512993910,387cyclictest11022-21cstates20:20:160
951399389383,4cyclictest0-21swapper/120:15:051
951299388381,4cyclictest32713-21kworker/0:120:10:390
9512993870,3cyclictest32713-21kworker/0:119:58:390
9512993810,2cyclictest0-21swapper/020:07:070
951599363359,2cyclictest0-21swapper/220:00:382
951299356348,5cyclictest0-21swapper/020:00:380
951299340335,3cyclictest0-21swapper/020:15:060
951599337331,3cyclictest8379-21systemd-udevd20:15:052
951699333327,3cyclictest0-21swapper/320:00:373
49742314294,15sleep031162-21kworker/0:219:09:060
951399298292,4cyclictest0-21swapper/120:00:381
9516997814,19cyclictest13354-21sort20:25:133
9516997814,19cyclictest13354-21sort20:25:133
9515994842,3cyclictest33-21ksoftirqd/220:30:002
9515994842,3cyclictest33-21ksoftirqd/220:30:002
9513993831,5cyclictest25-21ksoftirqd/120:26:021
9513993831,5cyclictest25-21ksoftirqd/120:26:021
9512993511,5cyclictest14673-21timerandwakeup20:25:320
9512993511,5cyclictest14673-21timerandwakeup20:25:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional