You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 04:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Fri Sep 04, 2026 00:54:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
167159979631,762cyclictest13461-21missed_timers19:55:187
167109979328,507cyclictest0-21swapper/220:10:142
167159967579,508cyclictest0-21swapper/720:05:537
167089966833,129cyclictest0-21swapper/000:31:330
167159965925,549cyclictest0-21swapper/723:26:417
1671399651101,126cyclictest0-21swapper/522:29:235
167089965055,380cyclictest0-21swapper/023:48:040
167159964893,381cyclictest0-21swapper/721:55:147
16710996370,127cyclictest9610-21ssh21:57:332
16715996360,127cyclictest10931-21sh21:39:537
16715996360,127cyclictest10931-21sh21:39:537
1671399628117,126cyclictest0-21swapper/521:25:065
167089962434,42cyclictest121rcu_preempt21:46:200
1670899624116,126cyclictest0-21swapper/021:39:400
1670899624116,126cyclictest0-21swapper/021:39:400
1670999620536,3cyclictest231rcuc/121:39:571
1670999620536,3cyclictest231rcuc/121:39:571
1671399616190,253cyclictest0-21swapper/521:52:225
167159961115,592cyclictest0-21swapper/720:55:527
167149960854,381cyclictest0-21swapper/620:10:126
16713995940,84cyclictest0-21swapper/523:05:175
16712995940,84cyclictest0-21swapper/423:05:174
16710995940,84cyclictest0-21swapper/223:05:172
16708995930,84cyclictest0-21swapper/022:30:130
167099959079,211cyclictest25-21ksoftirqd/121:58:171
1671299587122,211cyclictest0-21swapper/422:02:044
1671299586292,47cyclictest0-21swapper/423:39:094
1670999586502,3cyclictest241ktimersoftd/121:00:291
1670999586502,3cyclictest241ktimersoftd/121:00:291
167159958532,550cyclictest0-21swapper/723:37:047
1671299585117,339cyclictest0-21swapper/400:15:144
167159958431,550cyclictest0-21swapper/721:26:517
167099958231,126cyclictest0-21swapper/120:10:121
167129957729,84cyclictest0-21swapper/420:55:464
1671099577108,296cyclictest0-21swapper/221:39:402
1671099577108,296cyclictest0-21swapper/221:39:402
167089957771,42cyclictest1551-21snmpd21:34:350
167159957059,466cyclictest0-21swapper/723:59:577
167159957016,339cyclictest0-21swapper/719:47:027
167129956861,126cyclictest0-21swapper/422:05:184
167109956760,296cyclictest0-21swapper/221:31:402
1670899567310,126cyclictest0-21swapper/019:47:490
167149956411,126cyclictest0-21swapper/622:29:506
16713995629,253cyclictest0-21swapper/523:04:005
167089956053,344cyclictest0-21swapper/021:58:160
1671299555216,337cyclictest64450irq/122-eth021:26:464
167159955383,296cyclictest0-21swapper/721:31:427
1671199550209,126cyclictest5488-21grep21:35:213
1671199550209,126cyclictest5488-21grep21:35:213
1671099550124,126cyclictest0-21swapper/223:27:172
167139954577,211cyclictest57-21ksoftirqd/521:47:195
1671199545461,4cyclictest391rcuc/321:46:203
1671199545121,126cyclictest4487-21rm00:15:133
1671599544119,126cyclictest121rcu_preempt22:29:097
1671399544123,126cyclictest24900-21kworker/u16:222:03:455
1670899544288,84cyclictest0-21swapper/022:37:590
167159954332,508cyclictest0-21swapper/721:23:347
167109954336,84cyclictest0-21swapper/221:52:412
167089954335,126cyclictest0-21swapper/021:27:410
167149954173,465cyclictest16706-21cyclictest00:25:286
167089954074,126cyclictest0-21swapper/022:02:170
1670899539244,84cyclictest0-21swapper/023:03:310
167129953875,167cyclictest0-21swapper/423:29:404
167129953872,211cyclictest0-21swapper/421:51:154
167159953628,429cyclictest11019-21sh22:50:367
167159953528,422cyclictest0-21swapper/723:30:257
167119953528,126cyclictest121rcu_preempt21:27:043
167119953523,296cyclictest0-21swapper/320:10:263
1670899535192,295cyclictest0-21swapper/000:25:280
167159953465,253cyclictest0-21swapper/722:00:087
167149953465,211cyclictest65-21ksoftirqd/621:00:326
167149953465,211cyclictest65-21ksoftirqd/621:00:326
167149953434,118cyclictest0-21swapper/622:00:166
167129953426,84cyclictest0-21swapper/422:28:014
167099953162,296cyclictest0-21swapper/100:31:331
167129953065,84cyclictest121rcu_preempt21:30:254
167119953061,381cyclictest0-21swapper/323:29:343
1671099530103,253cyclictest33-21ksoftirqd/222:29:192
1670999529408,37cyclictest231rcuc/120:58:311
167099952918,338cyclictest0-21swapper/122:00:431
167129952620,42cyclictest0-21swapper/400:00:024
167119952657,296cyclictest0-21swapper/321:32:453
167099952614,127cyclictest0-21swapper/123:49:481
1671299525441,4cyclictest481ktimersoftd/423:03:524
167089952517,84cyclictest121rcu_preempt00:35:360
167159952312,508cyclictest0-21swapper/719:15:107
167129952353,253cyclictest0-21swapper/419:57:494
16711995219,84cyclictest0-21swapper/322:04:553
167139952014,126cyclictest0-21swapper/523:26:545
167119952011,127cyclictest0-21swapper/321:58:153
167099951911,126cyclictest0-21swapper/100:15:161
167089951954,84cyclictest121rcu_preempt21:52:400
167089951794,295cyclictest0-21swapper/021:00:020
167089951794,295cyclictest0-21swapper/021:00:020
16711995156,126cyclictest0-21swapper/321:52:223
16710995158,84cyclictest0-21swapper/221:22:412
16715995121,42cyclictest0-21swapper/721:00:277
16715995121,42cyclictest0-21swapper/721:00:277
16710995114,338cyclictest0-21swapper/222:53:152
167159950941,465cyclictest0-21swapper/723:03:537
16714995090,84cyclictest3689-21irqstats23:05:176
16713995090,8cyclictest17687-21df_abs22:20:175
16710995080,84cyclictest0-21swapper/223:24:282
167139950785,84cyclictest0-21swapper/521:58:435
167119950680,211cyclictest0-21swapper/323:47:113
167109950683,84cyclictest0-21swapper/222:55:242
167129950578,169cyclictest49-21ksoftirqd/421:49:034
1670899505460,42cyclictest111rcuc/022:28:490
1670899505121,381cyclictest6535-21munin-run00:00:020
167159950478,211cyclictest73-21ksoftirqd/700:25:297
1671199504124,296cyclictest18216-21runrttasks20:59:103
167099950477,211cyclictest25-21ksoftirqd/122:51:301
1670899504124,377cyclictest0-21swapper/020:59:350
1671499501202,89cyclictest121rcu_preempt22:51:206
1671299501291,47cyclictest0-21swapper/421:39:414
1671299501291,47cyclictest0-21swapper/421:39:414
167129950077,337cyclictest0-21swapper/423:49:244
167099950030,211cyclictest25-21ksoftirqd/123:05:021
1670999500119,126cyclictest0-21swapper/122:29:461
167089950076,85cyclictest121rcu_preempt22:20:200
167159949935,381cyclictest0-21swapper/721:48:117
167129949834,84cyclictest0-21swapper/400:37:014
167129949830,464cyclictest0-21swapper/420:06:344
167099949835,132cyclictest0-21swapper/123:38:461
1670999498114,211cyclictest25-21ksoftirqd/121:24:271
167139949732,261cyclictest0-21swapper/521:38:115
167139949732,261cyclictest0-21swapper/521:38:115
167159949631,3cyclictest0-21swapper/722:30:237
1670999495111,126cyclictest0-21swapper/123:25:351
1671499494152,339cyclictest24272-21diskmemload23:03:096
167109949471,84cyclictest0-21swapper/223:49:082
1670999494109,211cyclictest25-21ksoftirqd/123:04:171
1670899494111,126cyclictest9-21ksoftirqd/022:52:440
167119949327,84cyclictest0-21swapper/300:25:283
1671099493151,338cyclictest704-21nscd22:03:412
167159949223,465cyclictest0-21swapper/722:55:247
167089949270,256cyclictest5721-21diskmemload23:23:260
167089949121,127cyclictest0-21swapper/023:39:270
167159949026,385cyclictest0-21swapper/723:48:237
167159949025,386cyclictest0-21swapper/721:40:057
167129948968,84cyclictest0-21swapper/422:50:584
167119948923,296cyclictest0-21swapper/323:23:173
1670899489106,295cyclictest0-21swapper/021:22:240
167149948866,84cyclictest0-21swapper/623:48:046
1671499487147,338cyclictest24759-21kworker/u16:021:39:246
1671499487147,338cyclictest24759-21kworker/u16:021:39:246
1671299487108,295cyclictest0-21swapper/400:24:334
167109948761,295cyclictest0-21swapper/219:47:442
167109948661,208cyclictest0-21swapper/221:28:342
167129948563,84cyclictest121rcu_preempt23:30:254
1671099485229,254cyclictest321ktimersoftd/222:39:552
167089948563,50cyclictest0-21swapper/022:55:230
1671199484104,211cyclictest0-21swapper/322:39:403
167129948321,85cyclictest0-21swapper/422:30:434
167099948398,211cyclictest26941-21taskset21:47:581
1671599482354,126cyclictest711rcuc/700:16:077
167099948214,296cyclictest0-21swapper/100:24:411
167089948257,295cyclictest0-21swapper/023:28:020
167159948116,386cyclictest0-21swapper/722:20:167
167129948155,211cyclictest0-21swapper/421:59:054
167099948157,84cyclictest121rcu_preempt00:00:151
167089948116,89cyclictest0-21swapper/023:14:310
167149948058,211cyclictest65-21ksoftirqd/621:29:416
167149948014,126cyclictest0-21swapper/621:59:046
167149947956,338cyclictest0-21swapper/619:15:106
167149947914,256cyclictest0-21swapper/621:34:406
167109947955,84cyclictest121rcu_preempt21:45:592
167099947913,84cyclictest0-21swapper/119:55:431
167089947953,253cyclictest121rcu_preempt20:05:390
167119947813,218cyclictest0-21swapper/320:09:553
167149947619,36cyclictest0-21swapper/620:59:466
167139947692,211cyclictest1007-21ssh23:39:295
16713994741,47cyclictest0-21swapper/522:30:105
167119947452,213cyclictest0-21swapper/322:28:483
1671499472175,126cyclictest30396-21kworker/6:122:39:396
16715994715,386cyclictest0-21swapper/721:54:517
16709994715,84cyclictest0-21swapper/121:25:481
16708994717,43cyclictest121rcu_preempt20:50:140
16712994683,48cyclictest0-21swapper/423:24:554
167089946885,380cyclictest0-21swapper/023:50:060
16713994671,86cyclictest0-21swapper/522:51:435
16713994671,211cyclictest0-21swapper/522:39:545
167119946783,84cyclictest0-21swapper/323:50:063
167099946740,254cyclictest0-21swapper/122:39:291
1671199466125,338cyclictest0-21swapper/322:54:243
167099946639,295cyclictest0-21swapper/123:21:571
1670899466209,169cyclictest6764-21ssh00:15:530
167139946581,211cyclictest0-21swapper/520:59:405
1671099465124,338cyclictest13569-21ntp_kernel_err19:55:212
16708994650,48cyclictest121rcu_preempt19:50:130
1671399464124,337cyclictest0-21swapper/519:50:025
1671299464123,338cyclictest0-21swapper/422:37:594
167119946480,254cyclictest1551-21snmpd23:03:193
1670899464122,211cyclictest9-21ksoftirqd/000:05:020
167159946379,381cyclictest0-21swapper/723:50:067
167159946340,262cyclictest0-21swapper/723:14:397
167149946378,211cyclictest65-21ksoftirqd/620:50:176
167139946383,42cyclictest0-21swapper/500:04:515
167139946379,254cyclictest121rcu_preempt20:07:075
1671099463122,338cyclictest0-21swapper/221:00:272
1671099463122,338cyclictest0-21swapper/221:00:272
167089946341,41cyclictest0-21swapper/022:05:070
1671599462120,169cyclictest73-21ksoftirqd/722:19:007
167149946240,255cyclictest0-21swapper/621:54:136
1671499462121,338cyclictest16706-21cyclictest21:45:396
1671499462121,338cyclictest0-21swapper/622:30:176
167139946233,126cyclictest0-21swapper/521:30:065
167129946240,3cyclictest0-21swapper/422:20:184
167089946282,84cyclictest0-21swapper/020:10:120
167099946179,126cyclictest121rcu_preempt20:00:331
167149946038,256cyclictest0-21swapper/623:38:236
167149946038,256cyclictest0-21swapper/619:57:166
167149945976,210cyclictest13826-1kworker/6:2H19:50:136
167139945979,296cyclictest0-21swapper/523:49:085
167099945975,211cyclictest25-21ksoftirqd/121:52:051
1671299458124,204cyclictest0-21swapper/423:50:134
167119945880,40cyclictest0-21swapper/300:04:503
167119945876,84cyclictest24272-21diskmemload22:20:113
1670999458160,296cyclictest16706-21cyclictest22:19:311
1670899458118,126cyclictest9-21ksoftirqd/022:18:250
167129945735,256cyclictest0-21swapper/421:23:354
1671299457116,338cyclictest0-21swapper/421:00:174
1671299457116,338cyclictest0-21swapper/421:00:164
167159945630,423cyclictest0-21swapper/700:04:567
167109945634,2cyclictest0-21swapper/223:39:432
167099945671,211cyclictest25-21ksoftirqd/121:33:531
167139945532,85cyclictest20587-21ssh23:14:415
167109945570,169cyclictest0-21swapper/200:04:562
1670999454113,339cyclictest0-21swapper/120:07:171
167159945325,127cyclictest0-21swapper/719:50:137
1671599453109,296cyclictest22099-21grep20:10:197
167149945328,258cyclictest0-21swapper/623:14:156
167099945331,256cyclictest0-21swapper/119:49:031
167099945331,211cyclictest1551-21snmpd22:55:321
167089945331,2cyclictest0-21swapper/023:30:250
167149945230,258cyclictest16302-21idleruntime-cro22:19:596
167129945230,171cyclictest0-21swapper/419:50:004
167089945230,300cyclictest0-21swapper/019:55:350
167139945167,211cyclictest57-21ksoftirqd/500:31:345
167139945124,338cyclictest0-21swapper/522:19:075
167119945128,3cyclictest0-21swapper/323:14:143
167109945171,84cyclictest0-21swapper/200:24:402
167089945167,296cyclictest16706-21cyclictest20:00:250
167149945029,255cyclictest0-21swapper/623:24:166
167119945026,84cyclictest0-21swapper/322:18:243
167109945028,7cyclictest0-21swapper/223:01:492
167159944823,422cyclictest0-21swapper/722:39:567
167129944869,296cyclictest0-21swapper/423:14:334
1671299448108,337cyclictest0-21swapper/420:10:114
167139944725,84cyclictest0-21swapper/523:22:045
167119944561,211cyclictest0-21swapper/319:57:173
1671199445103,127cyclictest8589-21taskset19:48:393
167159944422,2cyclictest0-21swapper/722:05:237
167159944416,169cyclictest0-21swapper/723:24:217
1671399444102,339cyclictest0-21swapper/522:05:185
167099944465,295cyclictest0-21swapper/123:14:331
167159944321,256cyclictest0-21swapper/700:31:097
167129944220,3cyclictest1553-21runrttasks20:00:064
167139944120,42cyclictest0-21swapper/521:22:245
1671399441100,338cyclictest0-21swapper/519:58:535
167139944057,380cyclictest0-21swapper/523:50:065
1671199440102,172cyclictest0-21swapper/321:00:293
1671199440102,172cyclictest0-21swapper/321:00:293
167109943896,339cyclictest13760-1kworker/u17:223:30:092
167109943855,211cyclictest33-21ksoftirqd/200:25:282
167109943715,84cyclictest121rcu_preempt22:05:172
16710994358,211cyclictest33-21ksoftirqd/222:19:082
167099943552,168cyclictest1553-21runrttasks22:05:331
1671299434351,3cyclictest471rcuc/400:04:454
167099943412,84cyclictest0-21swapper/100:00:011
167149943310,256cyclictest0-21swapper/623:25:146
167099943394,336cyclictest0-21swapper/100:25:261
16708994328,127cyclictest0-21swapper/000:04:570
16711994290,84cyclictest3688-21ssh23:05:173
16711994286,127cyclictest0-21swapper/321:24:053
16715994271,423cyclictest0-21swapper/720:50:167
16713994275,84cyclictest0-21swapper/521:00:015
16713994275,84cyclictest0-21swapper/521:00:015
167139942683,340cyclictest0-21swapper/500:25:285
167149942483,338cyclictest0-21swapper/619:49:246
167149942483,338cyclictest0-21swapper/600:44:406
167149942480,341cyclictest17955-21ssh21:22:416
167099942482,339cyclictest0-21swapper/120:50:141
167159942340,84cyclictest16706-21cyclictest00:24:337
167149942342,296cyclictest0-21swapper/622:05:106
16714994231,256cyclictest3167-21sh00:31:096
1671399423125,296cyclictest24272-21diskmemload00:16:085
167129942382,338cyclictest0-21swapper/400:05:184
167099942383,338cyclictest0-21swapper/100:05:021
167099942382,338cyclictest0-21swapper/122:20:181
167149942282,338cyclictest0-21swapper/620:07:246
167149942281,338cyclictest21391-21ssh00:24:396
167129942281,338cyclictest0-21swapper/422:40:034
1671299422378,42cyclictest471rcuc/419:15:104
167159942141,297cyclictest0-21swapper/700:36:597
167149942180,338cyclictest0-21swapper/622:40:036
167129942134,41cyclictest121rcu_preempt22:19:274
167109942182,84cyclictest0-21swapper/220:00:172
167159942079,338cyclictest0-21swapper/720:00:017
167119942082,83cyclictest0-21swapper/320:00:163
167119941980,84cyclictest0-21swapper/300:00:013
1670999419162,126cyclictest121rcu_preempt22:30:161
167139941881,173cyclictest17453-21ssh22:55:155
167139941877,338cyclictest24272-21diskmemload22:40:035
167139941877,338cyclictest0-21swapper/500:44:395
167129941839,2cyclictest64450irq/122-eth000:31:264
167109941879,84cyclictest0-21swapper/222:40:032
167109941879,84cyclictest0-21swapper/220:07:122
167149941776,338cyclictest0-21swapper/600:16:356
167119941774,169cyclictest41-21ksoftirqd/323:39:283
167099941738,256cyclictest0-21swapper/121:40:071
1671099416334,80cyclictest311rcuc/200:31:222
1671099416121,89cyclictest121rcu_preempt20:50:132
167099941677,83cyclictest0-21swapper/100:35:561
167149941575,338cyclictest0-21swapper/600:05:026
167149941574,338cyclictest0-21swapper/620:00:176
167139941574,126cyclictest0-21swapper/519:15:105
167119941576,84cyclictest0-21swapper/322:40:033
167119941576,84cyclictest0-21swapper/300:05:173
167119941528,90cyclictest0-21swapper/319:15:103
167109941576,84cyclictest0-21swapper/223:14:342
167139941435,127cyclictest0-21swapper/500:00:015
1671099414115,168cyclictest0-21swapper/220:55:262
1670999414119,253cyclictest20564-21chrt19:15:101
1670899414119,213cyclictest0-21swapper/000:24:330
167159941270,339cyclictest0-21swapper/723:05:157
167159941228,381cyclictest0-21swapper/700:05:017
167139941272,338cyclictest0-21swapper/500:05:025
167119941231,42cyclictest0-21swapper/322:05:073
167109941273,84cyclictest0-21swapper/200:35:482
167109941172,126cyclictest0-21swapper/200:15:102
167149941030,256cyclictest0-21swapper/622:20:176
167139941069,168cyclictest0-21swapper/520:10:225
1671199410113,85cyclictest10846-1kworker/u17:000:31:333
167119940970,84cyclictest0-21swapper/320:50:163
167099940969,338cyclictest0-21swapper/123:30:241
167119940822,85cyclictest0-21swapper/322:30:123
167109940869,84cyclictest0-21swapper/222:20:162
167089940827,84cyclictest0-21swapper/023:05:160
167109940664,211cyclictest33-21ksoftirqd/219:15:102
167149940564,338cyclictest0-21swapper/621:40:056
167129940565,84cyclictest0-21swapper/400:25:264
167139940424,2cyclictest0-21swapper/500:35:435
167109940465,84cyclictest0-21swapper/200:05:022
167109940424,44cyclictest0-21swapper/222:30:132
167089940425,84cyclictest0-21swapper/000:44:400
167129940365,83cyclictest0-21swapper/421:40:054
167089939616,84cyclictest0-21swapper/022:40:030
167149939513,82cyclictest0-21swapper/622:55:066
167119939313,211cyclictest0-21swapper/322:55:153
1671099393310,3cyclictest321ktimersoftd/200:00:022
167149939112,256cyclictest0-21swapper/623:30:256
167129939113,84cyclictest0-21swapper/400:44:404
16711993894,169cyclictest0-21swapper/300:35:433
167129938649,84cyclictest16706-21cyclictest22:55:064
167149938545,338cyclictest0-21swapper/623:50:126
167149938184,295cyclictest0-21swapper/600:04:516
16713993811,257cyclictest0-21swapper/523:30:245
16713993811,177cyclictest0-21swapper/520:00:215
167109938139,127cyclictest33-21ksoftirqd/219:50:132
167149938082,126cyclictest12263-21taskset00:35:566
167159937736,338cyclictest0-21swapper/700:44:397
167159937638,212cyclictest0-21swapper/722:40:027
167119937676,169cyclictest41-21ksoftirqd/323:30:093
167099937433,338cyclictest0-21swapper/123:50:101
167099937131,338cyclictest0-21swapper/100:44:401
167089937132,84cyclictest0-21swapper/021:40:050
1670899371203,84cyclictest0-21swapper/019:15:100
167109937031,84cyclictest0-21swapper/223:50:122
167139936932,7cyclictest0-21swapper/500:24:335
167149936225,84cyclictest6568-21awk00:00:016
167119936264,84cyclictest0-21swapper/319:50:063
167119936065,84cyclictest0-21swapper/300:24:353
167139935520,86cyclictest0-21swapper/521:40:035
1670999355141,87cyclictest121rcu_preempt22:40:021
16712993401,84cyclictest0-21swapper/420:50:164
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional