You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 18:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat Apr 25, 2026 12:55:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
194019963280,381cyclictest0-21swapper/211:02:282
194049960917,461cyclictest121rcu_preempt09:37:545
19402995997,2cyclictest0-21swapper/309:48:123
194069959440,126cyclictest0-21swapper/709:45:027
194069959440,126cyclictest0-21swapper/709:45:027
194069959381,380cyclictest0-21swapper/709:38:117
194039958476,84cyclictest121rcu_preempt10:43:454
1940199583114,211cyclictest33-21ksoftirqd/209:45:082
1940199582499,2cyclictest311rcuc/209:44:502
1940199582499,2cyclictest311rcuc/209:44:502
1940199580497,3cyclictest311rcuc/210:25:102
1940199577494,3cyclictest311rcuc/210:00:412
194039957668,126cyclictest0-21swapper/409:47:004
194049956961,84cyclictest0-21swapper/510:00:035
194039956717,130cyclictest0-21swapper/411:28:074
19406995639,380cyclictest6615-21ssh11:00:567
194059956112,126cyclictest0-21swapper/609:49:296
19405995606,84cyclictest0-21swapper/612:43:446
1940599557513,42cyclictest631rcuc/611:19:166
19399995512,169cyclictest4361-21ssh10:43:440
19400995480,84cyclictest0-21swapper/109:38:131
1939999548207,126cyclictest121rcu_preempt11:04:140
1940699547206,338cyclictest32190-1kworker/u17:012:42:017
194009954476,126cyclictest0-21swapper/111:03:141
1940499541286,253cyclictest561ktimersoftd/510:43:445
194009953932,259cyclictest0-21swapper/110:04:351
1940499538196,338cyclictest24616-21nscd09:40:355
1940499538196,338cyclictest24616-21nscd09:40:355
1940699535196,84cyclictest19674-21dump-pmu-power10:00:017
193999953528,83cyclictest121rcu_preempt09:40:130
193999953528,83cyclictest121rcu_preempt09:40:130
1940199534193,338cyclictest1651-21ntpd11:27:152
1940699533450,3cyclictest721ktimersoftd/707:49:007
194039953364,126cyclictest0-21swapper/411:00:384
194019953263,211cyclictest33-21ksoftirqd/212:42:382
1940399531105,295cyclictest0-21swapper/411:57:544
1940099527232,84cyclictest0-21swapper/112:17:571
194049952616,507cyclictest0-21swapper/511:04:165
193999952699,211cyclictest101ktimersoftd/009:35:050
1940599525227,125cyclictest0-21swapper/610:24:196
194039952254,337cyclictest0-21swapper/412:44:354
194019952214,84cyclictest0-21swapper/209:38:122
194009952215,8cyclictest0-21swapper/112:43:571
194009952114,3cyclictest0-21swapper/109:48:121
193999952014,253cyclictest0-21swapper/010:24:130
1940299519220,126cyclictest5536-21rm10:26:483
194049951345,84cyclictest1000-21ssh12:42:455
19405995113,210cyclictest0-21swapper/610:00:016
19404995110,507cyclictest0-21swapper/510:02:565
194039951143,253cyclictest0-21swapper/410:24:174
19405995090,84cyclictest0-21swapper/609:43:176
19405995090,84cyclictest0-21swapper/609:43:176
19406995080,83cyclictest0-21swapper/711:29:497
1940199508169,84cyclictest19397-21cyclictest12:20:112
1940299507207,127cyclictest0-21swapper/309:43:333
1940299507207,127cyclictest0-21swapper/309:43:333
1939999507123,253cyclictest19691-21awk10:00:010
194049950637,43cyclictest0-21swapper/510:12:575
194039950540,84cyclictest121rcu_preempt10:02:134
1940699502418,3cyclictest721ktimersoftd/710:24:097
1940399500417,3cyclictest481ktimersoftd/409:34:444
193999950034,211cyclictest0-21swapper/009:33:260
1939999500199,126cyclictest121rcu_preempt12:20:170
1940699498414,3cyclictest721ktimersoftd/712:19:417
194049949830,465cyclictest0-21swapper/512:09:555
194049949733,380cyclictest31792-21ssh09:48:125
194049949671,422cyclictest0-21swapper/512:18:115
1940499496195,295cyclictest19397-21cyclictest10:25:155
1940399496239,254cyclictest64450irq/122-eth009:35:394
194049949569,253cyclictest0-21swapper/507:48:395
193999949569,211cyclictest9-21ksoftirqd/011:56:400
1940399494156,169cyclictest0-21swapper/412:09:534
194059949370,338cyclictest0-21swapper/610:25:056
194029949371,84cyclictest0-21swapper/311:00:573
1940299493113,84cyclictest18193-21sh09:58:233
193999949367,253cyclictest32356-21ssh12:41:200
194069949265,126cyclictest0-21swapper/709:48:397
1940699492161,120cyclictest121rcu_preempt07:50:167
1940299492237,84cyclictest20338-1kworker/u17:211:30:263
1940299492237,84cyclictest20338-1kworker/u17:211:30:263
194019949222,254cyclictest0-21swapper/212:10:072
194029949168,84cyclictest0-21swapper/312:42:083
1940099491446,42cyclictest231rcuc/109:33:531
1940399489110,295cyclictest0-21swapper/412:19:304
194059948819,340cyclictest0-21swapper/612:09:086
194049948826,120cyclictest0-21swapper/511:27:235
194059948626,81cyclictest0-21swapper/611:05:056
194059948519,169cyclictest0-21swapper/612:20:016
1939999485272,84cyclictest3539-21rm10:25:250
193999948520,380cyclictest0-21swapper/009:48:320
194059948464,294cyclictest121rcu_preempt11:05:006
1940499484229,253cyclictest561ktimersoftd/509:33:265
194039948420,297cyclictest0-21swapper/411:20:254
194029948357,295cyclictest0-21swapper/310:43:453
194039948299,125cyclictest49-21ksoftirqd/410:26:184
194049948118,459cyclictest0-21swapper/510:24:325
194009948116,3cyclictest0-21swapper/112:09:081
194069948056,169cyclictest0-21swapper/710:26:027
194019948053,211cyclictest33-21ksoftirqd/210:42:372
193999948015,295cyclictest0-21swapper/011:29:230
194039947916,251cyclictest0-21swapper/411:39:004
194029947911,211cyclictest0-21swapper/312:20:233
1940199478433,42cyclictest311rcuc/210:00:052
1939999476176,126cyclictest121rcu_preempt10:45:070
19404994756,466cyclictest0-21swapper/510:45:105
194029947591,211cyclictest0-21swapper/312:19:573
1940099475180,126cyclictest121rcu_preempt09:50:051
19404994739,7cyclictest0-21swapper/511:20:255
194049947314,456cyclictest0-21swapper/510:59:255
19406994728,2cyclictest0-21swapper/710:02:307
19404994726,211cyclictest0-21swapper/511:57:415
194019947249,84cyclictest0-21swapper/210:24:542
1940099470214,126cyclictest12948-21kworker/u16:209:59:311
19400994691,84cyclictest0-21swapper/111:19:261
19404994680,465cyclictest0-21swapper/511:05:265
19401994671,84cyclictest0-21swapper/211:30:112
19401994671,84cyclictest0-21swapper/211:30:112
194019946682,169cyclictest33-21ksoftirqd/212:19:572
194009946639,8cyclictest0-21swapper/109:43:171
194009946639,8cyclictest0-21swapper/109:43:171
194039946541,253cyclictest0-21swapper/407:50:114
194029946580,296cyclictest0-21swapper/309:39:453
1940099465124,338cyclictest0-21swapper/110:43:451
1939999465124,338cyclictest0-21swapper/012:10:190
1940399464124,338cyclictest0-21swapper/409:44:004
1940399464124,338cyclictest0-21swapper/409:44:004
1940299464123,339cyclictest19397-21cyclictest10:00:113
1940599463122,338cyclictest30186-21nscd09:34:206
1940399463123,337cyclictest0-21swapper/411:19:554
194019946341,256cyclictest0-21swapper/207:48:102
194059946240,3cyclictest0-21swapper/607:50:156
1940399462121,338cyclictest0-21swapper/412:20:104
194009946276,211cyclictest25-21ksoftirqd/109:25:111
1940099462380,3cyclictest241ktimersoftd/110:26:411
1939999462121,338cyclictest0-21swapper/011:20:460
194059946177,211cyclictest641ktimersoftd/610:43:396
194039946182,84cyclictest0-21swapper/409:57:464
1940399461162,127cyclictest121rcu_preempt10:12:544
1940199461378,3cyclictest311rcuc/212:09:592
193999946038,2cyclictest0-21swapper/010:02:580
1940199459376,4cyclictest311rcuc/211:19:172
194039945874,211cyclictest698-21ptp4l07:49:514
194029945878,82cyclictest0-21swapper/309:33:083
1939999458116,339cyclictest0-21swapper/011:19:550
194069945782,372cyclictest0-21swapper/711:20:427
194069945734,256cyclictest0-21swapper/709:33:337
1940599457117,338cyclictest0-21swapper/610:02:006
1940299457159,126cyclictest121rcu_preempt10:59:253
1940299457115,211cyclictest0-21swapper/310:24:313
194019945773,211cyclictest0-21swapper/211:20:302
1940099457117,337cyclictest0-21swapper/111:05:171
1940199456114,169cyclictest0-21swapper/211:58:302
194059945533,255cyclictest0-21swapper/611:56:496
1940199455115,338cyclictest26996-21diskmemload09:33:442
1940099455157,126cyclictest1173-21ssh10:24:461
1940699454113,338cyclictest19397-21cyclictest09:25:087
194019945331,261cyclictest0-21swapper/210:13:012
1940599452111,338cyclictest0-21swapper/609:25:236
1940099452111,338cyclictest1703-21sendmail-mta07:49:351
194049945136,124cyclictest698-21ptp4l11:39:035
194049945124,424cyclictest0-21swapper/512:20:185
194019945126,259cyclictest0-21swapper/209:25:182
194049945069,85cyclictest0-21swapper/511:19:565
194069944969,295cyclictest0-21swapper/711:58:277
194069944927,255cyclictest0-21swapper/712:09:087
194059944868,211cyclictest0-21swapper/609:39:366
1940299448149,126cyclictest121rcu_preempt11:05:263
1940099448109,336cyclictest0-21swapper/111:30:251
1940099448109,336cyclictest0-21swapper/111:30:251
193999944867,84cyclictest0-21swapper/011:39:360
1940699447108,126cyclictest121rcu_preempt11:19:567
194049944725,342cyclictest0-21swapper/507:28:515
194009944725,176cyclictest0-21swapper/111:20:251
194069944421,256cyclictest0-21swapper/710:44:447
194059944321,2cyclictest0-21swapper/610:58:546
1940099443102,338cyclictest0-21swapper/110:05:051
1940099443102,338cyclictest0-21swapper/110:05:051
194069944219,256cyclictest0-21swapper/712:20:237
194029944119,2cyclictest0-21swapper/309:25:173
1939999441402,37cyclictest111rcuc/012:20:020
194049944018,341cyclictest3955-21diskmemload12:10:165
194029944018,2cyclictest1505-21runrttasks12:09:293
194009944099,126cyclictest121rcu_preempt11:28:171
1940099440183,84cyclictest0-21swapper/107:50:211
194069943998,295cyclictest23248-21kworker/u16:311:30:117
194069943998,295cyclictest23248-21kworker/u16:311:30:117
194019943959,42cyclictest121rcu_preempt11:05:112
194029943851,84cyclictest0-21swapper/311:39:353
194019943714,256cyclictest0-21swapper/209:24:592
194069943513,256cyclictest0-21swapper/711:05:117
1940599435179,126cyclictest19397-21cyclictest11:27:006
194049943416,118cyclictest0-21swapper/510:05:035
194049943416,118cyclictest0-21swapper/510:05:035
19400994314,211cyclictest25-21ksoftirqd/111:58:111
19399994315,85cyclictest10773-21ssh10:12:390
194059943093,253cyclictest0-21swapper/611:20:486
194039942848,84cyclictest0-21swapper/410:59:444
1940099428131,167cyclictest121rcu_preempt11:39:301
193999942711,297cyclictest0-21swapper/009:25:060
194039942641,84cyclictest49-21ksoftirqd/411:05:204
1940299425168,84cyclictest121rcu_preempt12:10:183
194019942583,169cyclictest0-21swapper/209:50:052
194069942483,338cyclictest0-21swapper/712:10:187
194059942483,338cyclictest0-21swapper/612:10:176
194009942481,127cyclictest25-21ksoftirqd/112:10:261
194039942283,84cyclictest0-21swapper/407:28:504
194039942282,84cyclictest0-21swapper/409:50:094
1940299422166,125cyclictest121rcu_preempt07:49:023
194019942282,338cyclictest0-21swapper/211:39:362
194019942238,253cyclictest0-21swapper/210:59:562
194049942179,338cyclictest0-21swapper/509:25:185
194019942180,338cyclictest0-21swapper/210:05:052
194019942180,338cyclictest0-21swapper/210:05:052
194069942080,338cyclictest0-21swapper/710:12:407
194059942079,338cyclictest0-21swapper/610:05:056
194059942079,338cyclictest0-21swapper/610:05:056
194069941939,296cyclictest0-21swapper/710:58:547
194039941979,84cyclictest0-21swapper/411:30:254
194039941979,84cyclictest0-21swapper/411:30:254
194029941980,84cyclictest0-21swapper/311:28:253
194069941877,338cyclictest0-21swapper/709:50:157
194029941879,84cyclictest13088-21idleruntime07:50:233
194059941779,3cyclictest1553-21runrttasks07:49:466
194029941778,84cyclictest0-21swapper/311:20:433
194039941678,84cyclictest0-21swapper/412:10:264
1940299416203,46cyclictest0-21swapper/311:57:283
193999941678,84cyclictest0-21swapper/012:09:570
193999941677,84cyclictest0-21swapper/007:28:510
193999941676,84cyclictest0-21swapper/011:05:170
194029941577,84cyclictest0-21swapper/311:19:443
193999941576,84cyclictest0-21swapper/009:50:090
194049941373,337cyclictest26019-21ssh11:30:325
194049941373,337cyclictest26019-21ssh11:30:325
193999941374,84cyclictest0-21swapper/010:05:050
193999941374,84cyclictest0-21swapper/010:05:050
193999941172,84cyclictest0-21swapper/011:30:250
193999941172,84cyclictest0-21swapper/011:30:250
193999941172,84cyclictest0-21swapper/007:48:220
193999941132,42cyclictest0-21swapper/010:58:540
194069941069,338cyclictest0-21swapper/711:39:357
194059941032,84cyclictest0-21swapper/611:30:266
194059941032,84cyclictest0-21swapper/611:30:266
194029940970,84cyclictest0-21swapper/310:05:053
194029940970,84cyclictest0-21swapper/310:05:053
194059940867,338cyclictest0-21swapper/612:20:106
194039940870,84cyclictest0-21swapper/410:05:054
194039940870,84cyclictest0-21swapper/410:05:054
194049940719,385cyclictest0-21swapper/507:50:025
194029940364,84cyclictest0-21swapper/310:12:533
194019940221,214cyclictest12062-21cpuspeed_turbos07:50:152
194009940265,171cyclictest0-21swapper/110:12:401
194009940017,168cyclictest0-21swapper/110:59:341
194039939617,176cyclictest0-21swapper/410:45:094
194069939352,338cyclictest0-21swapper/710:05:117
194069939352,338cyclictest0-21swapper/710:05:117
194009939354,84cyclictest0-21swapper/112:20:101
19405993833,127cyclictest65-21ksoftirqd/609:50:106
194059938283,296cyclictest0-21swapper/607:28:516
19404993817,119cyclictest0-21swapper/509:50:065
1940599379295,3cyclictest631rcuc/610:45:066
194029937878,126cyclictest41-21ksoftirqd/310:45:113
193999937637,84cyclictest0-21swapper/007:50:120
194059937576,84cyclictest0-21swapper/611:39:006
194039937579,84cyclictest0-21swapper/409:25:044
194029937435,84cyclictest0-21swapper/309:50:063
1940199371288,3cyclictest311rcuc/212:39:572
194069937032,172cyclictest0-21swapper/709:24:597
194059937032,169cyclictest11183-21ssh10:13:006
193999936730,47cyclictest26996-21diskmemload12:39:580
194019935762,215cyclictest698-21ptp4l07:28:502
194069935455,296cyclictest0-21swapper/707:28:517
194019935316,79cyclictest0-21swapper/210:45:072
19405993431,211cyclictest65-21ksoftirqd/612:39:486
194029934080,84cyclictest31962-21runrttasks07:28:513
194069933635,126cyclictest73-21ksoftirqd/710:45:127
194009932872,253cyclictest0-21swapper/110:45:111
194009932872,253cyclictest0-21swapper/107:28:261
194069932164,254cyclictest0-21swapper/712:39:587
194049932125,212cyclictest0-21swapper/512:39:575
194009932126,2cyclictest0-21swapper/112:39:491
194029931621,211cyclictest0-21swapper/312:39:573
194049931113,295cyclictest25821-21ssh09:24:595
194039930611,126cyclictest49-21ksoftirqd/409:25:004
194059929845,171cyclictest0-21swapper/609:24:596
194009929538,84cyclictest25-21ksoftirqd/109:24:591
194029928770,87cyclictest0-21swapper/309:24:593
1940399272102,168cyclictest481ktimersoftd/412:39:584
1939999267100,86cyclictest0-21swapper/009:24:590
16477210196,3sleep30-21swapper/307:05:143
1874929062,24sleep40-21swapper/407:06:504
240032890,1sleep724008-21sh09:05:007
1882328453,27sleep20-21swapper/207:07:542
1636628453,28sleep70-21swapper/707:05:027
1867128352,28sleep10-21swapper/107:05:441
1895828249,29sleep60-21swapper/607:09:506
1892328241,37sleep00-21swapper/007:09:200
1881728150,9sleep50-21swapper/507:07:495
242492710,0sleep20-21swapper/211:46:382
260142690,0sleep20-21swapper/210:55:002
201262670,0sleep10-21swapper/111:10:081
240912650,1sleep00-21swapper/010:53:540
1940499606,41cyclictest0-21swapper/512:01:045
1940499532,36cyclictest13405-21ssh11:40:595
1940499511,38cyclictest0-21swapper/510:30:425
121232500,1sleep60-21swapper/611:40:256
19404994515,17cyclictest0-21swapper/507:30:455
1939999439,23cyclictest9-21ksoftirqd/011:40:280
19403994110,18cyclictest0-21swapper/411:40:414
1939999405,23cyclictest9-21ksoftirqd/012:00:360
1940399391,25cyclictest0-21swapper/412:00:324
19406993830,4cyclictest0-21swapper/711:40:217
19400993810,14cyclictest25-21ksoftirqd/111:40:131
1940299372,31cyclictest0-21swapper/312:00:073
1940099372,31cyclictest0-21swapper/112:00:111
19400993711,15cyclictest25-21ksoftirqd/110:30:181
19399993713,11cyclictest121rcu_preempt10:30:150
1940199362,4cyclictest0-21swapper/212:00:142
1940199351,31cyclictest11049-21diskstats11:40:162
1939999351,20cyclictest9-21ksoftirqd/007:30:040
1940599341,30cyclictest0-21swapper/612:00:086
1940399349,13cyclictest0-21swapper/407:30:134
1940299346,16cyclictest41-21ksoftirqd/310:30:273
1940299341,30cyclictest0-21swapper/311:40:033
19400993325,5cyclictest2258-21sed07:30:301
19403993225,4cyclictest11368-21ntpq10:30:244
19406993124,4cyclictest11310-21cut10:30:237
19406993123,5cyclictest1033-21tail07:30:177
1940699311,27cyclictest0-21swapper/712:00:107
19405993123,5cyclictest11291-21grep10:30:236
1940299312,18cyclictest0-21swapper/307:30:233
19402992910,0cyclictest0-21swapper/308:25:063
19401992922,4cyclictest1119-21basename07:30:182
19401992922,4cyclictest10794-21sed10:30:182
19406992828,0cyclictest0-21swapper/708:10:037
19406992828,0cyclictest0-21swapper/708:09:457
19406992810,18cyclictest0-21swapper/709:12:337
19406992810,18cyclictest0-21swapper/708:17:307
19406992810,18cyclictest0-21swapper/708:02:127
19405992810,18cyclictest0-21swapper/608:17:586
19404992810,17cyclictest0-21swapper/509:12:005
19402992810,18cyclictest0-21swapper/309:12:393
19402992810,18cyclictest0-21swapper/307:19:223
19402992810,0cyclictest0-21swapper/308:54:363
19402992810,0cyclictest0-21swapper/308:38:133
19401992810,0cyclictest0-21swapper/207:14:152
19400992810,18cyclictest0-21swapper/109:12:141
19400992810,18cyclictest0-21swapper/107:14:231
19406992727,0cyclictest0-21swapper/708:31:577
19406992727,0cyclictest0-21swapper/707:18:507
19406992710,17cyclictest0-21swapper/710:39:417
19406992710,17cyclictest0-21swapper/708:54:047
19406992710,17cyclictest0-21swapper/708:38:187
19406992710,17cyclictest0-21swapper/707:37:217
19406992710,0cyclictest0-21swapper/708:45:487
19406992710,0cyclictest0-21swapper/708:25:117
19406992710,0cyclictest0-21swapper/708:24:307
19405992727,0cyclictest0-21swapper/608:54:036
19405992721,4cyclictest1666-21tr07:30:256
19405992710,17cyclictest0-21swapper/610:39:336
19405992710,17cyclictest0-21swapper/609:11:596
19405992710,17cyclictest0-21swapper/609:04:126
19405992710,17cyclictest0-21swapper/608:45:496
19405992710,17cyclictest0-21swapper/608:38:156
19405992710,17cyclictest0-21swapper/608:32:016
19405992710,17cyclictest0-21swapper/608:25:016
19405992710,17cyclictest0-21swapper/608:24:546
19405992710,17cyclictest0-21swapper/608:09:386
19405992710,17cyclictest0-21swapper/608:02:116
19405992710,17cyclictest0-21swapper/607:14:156
19405992710,0cyclictest0-21swapper/607:37:166
19405992710,0cyclictest0-21swapper/607:19:186
19404992727,0cyclictest0-21swapper/507:37:155
19404992710,17cyclictest0-21swapper/509:04:255
19404992710,17cyclictest0-21swapper/508:54:135
19404992710,17cyclictest0-21swapper/508:45:555
19404992710,17cyclictest0-21swapper/508:31:325
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional