You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 16:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Fri Jun 26, 2026 12:56:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2232999673118,296cyclictest0-21swapper/710:41:197
2232399671161,508cyclictest0-21swapper/111:08:121
223229963178,380cyclictest0-21swapper/012:13:380
2232699630331,169cyclictest30538-21diskmemload11:43:564
223229963037,127cyclictest0-21swapper/011:34:200
223289962875,550cyclictest0-21swapper/612:03:036
2232899620109,508cyclictest0-21swapper/611:08:116
2232899620108,381cyclictest0-21swapper/611:34:206
223299961564,84cyclictest121rcu_preempt10:50:557
2232599614103,508cyclictest0-21swapper/310:02:293
223289960510,41cyclictest4249-21cstates10:55:146
223249960192,507cyclictest0-21swapper/210:02:292
22328995993,127cyclictest0-21swapper/610:54:316
223239959888,507cyclictest0-21swapper/110:02:281
2232599589124,381cyclictest0-21swapper/312:12:293
223289958734,550cyclictest0-21swapper/611:17:166
223269958133,380cyclictest0-21swapper/412:32:394
223259958127,423cyclictest0-21swapper/311:34:493
2232699580116,126cyclictest49-21ksoftirqd/410:55:104
2232399579322,126cyclictest0-21swapper/110:50:551
223289957623,550cyclictest0-21swapper/612:42:426
223259957222,2cyclictest0-21swapper/310:44:143
223279957060,337cyclictest0-21swapper/512:31:005
2232599569231,84cyclictest4459-1kworker/u17:210:27:293
2232399564477,84cyclictest241ktimersoftd/111:30:461
223269956313,84cyclictest0-21swapper/411:31:154
223229956212,260cyclictest0-21swapper/012:15:230
223239955948,338cyclictest0-21swapper/112:05:031
223289955647,127cyclictest0-21swapper/610:02:286
223299955182,254cyclictest121rcu_preempt12:15:277
223249955082,465cyclictest0-21swapper/211:34:562
223299954941,126cyclictest0-21swapper/711:08:117
223249954778,254cyclictest121rcu_preempt10:31:252
223229954740,126cyclictest0-21swapper/010:02:280
2232799546162,381cyclictest0-21swapper/511:08:115
223269954639,126cyclictest0-21swapper/410:02:284
223289954434,507cyclictest0-21swapper/610:42:596
223269954477,464cyclictest0-21swapper/411:09:124
223239954474,296cyclictest0-21swapper/112:30:471
223239954175,126cyclictest0-21swapper/111:17:461
223289953929,507cyclictest12990-21kworker/u16:311:10:186
223289953929,507cyclictest12990-21kworker/u16:311:10:176
223269953830,84cyclictest121rcu_preempt12:04:374
223289953528,126cyclictest0-21swapper/611:44:046
223259953522,297cyclictest0-21swapper/311:06:563
223229953225,339cyclictest0-21swapper/011:06:030
223249953120,253cyclictest0-21swapper/211:35:142
223229953166,296cyclictest1553-21runrttasks11:42:350
223229953124,253cyclictest0-21swapper/011:17:480
223299952860,465cyclictest0-21swapper/712:04:047
223289952618,428cyclictest0-21swapper/612:32:416
223239952520,82cyclictest121rcu_preempt08:00:021
2232699524102,211cyclictest49-21ksoftirqd/410:30:584
223229952416,84cyclictest0-21swapper/011:35:120
223279952316,260cyclictest0-21swapper/510:04:185
22327995136,337cyclictest0-21swapper/512:02:395
22326995121,507cyclictest0-21swapper/410:54:214
22325995090,84cyclictest0-21swapper/312:04:343
223279950842,211cyclictest0-21swapper/508:00:155
22328995071,84cyclictest22766-21ssh11:45:156
223269950681,422cyclictest0-21swapper/410:05:104
22324995060,2cyclictest0-21swapper/211:40:332
223299950577,211cyclictest0-21swapper/711:17:577
223289950538,423cyclictest4459-1kworker/u17:210:27:236
223269950434,338cyclictest0-21swapper/410:41:354
223249950481,169cyclictest0-21swapper/211:08:592
223249950440,2cyclictest0-21swapper/210:45:202
2232999503247,254cyclictest721ktimersoftd/712:10:407
223289950342,458cyclictest7464-21ssh12:13:426
223289950234,465cyclictest0-21swapper/607:57:276
223269950137,219cyclictest0-21swapper/412:40:474
223279949935,41cyclictest0-21swapper/512:41:235
2232399499115,211cyclictest0-21swapper/111:10:251
2232399499115,211cyclictest0-21swapper/111:10:251
223249949829,211cyclictest33-21ksoftirqd/210:41:422
2232299498115,126cyclictest0-21swapper/010:31:200
223289949728,339cyclictest0-21swapper/610:30:456
223269949771,423cyclictest0-21swapper/412:15:224
2232599496282,84cyclictest0-21swapper/310:54:123
2232399495451,42cyclictest231rcuc/110:43:291
223269949367,423cyclictest0-21swapper/412:10:194
223279949266,295cyclictest0-21swapper/511:34:435
2232999491111,296cyclictest0-21swapper/710:55:097
223269949166,422cyclictest0-21swapper/411:16:214
223259949121,253cyclictest0-21swapper/312:33:063
223229949122,338cyclictest0-21swapper/010:43:560
223229948925,85cyclictest0-21swapper/010:27:160
223279948718,260cyclictest0-21swapper/511:18:165
2232999486106,126cyclictest73-21ksoftirqd/711:45:087
223289948515,466cyclictest0-21swapper/611:35:176
223229948319,253cyclictest0-21swapper/010:54:320
223229948317,339cyclictest0-21swapper/008:00:210
223229948218,219cyclictest0-21swapper/012:31:140
2232799480105,78cyclictest0-21swapper/512:12:375
2232499480142,127cyclictest25286-21basename12:05:012
2232399480104,78cyclictest0-21swapper/112:12:381
223239948010,211cyclictest0-21swapper/111:35:151
223299947611,127cyclictest25552-21sshd12:41:227
223259947694,84cyclictest0-21swapper/312:42:113
223259947653,254cyclictest0-21swapper/311:35:143
223249947650,423cyclictest0-21swapper/212:10:352
223279947511,295cyclictest0-21swapper/511:41:125
2232499474132,339cyclictest22320-21cyclictest10:52:262
223269947292,84cyclictest0-21swapper/410:26:554
223259947215,331cyclictest0-21swapper/307:57:073
2232699470128,126cyclictest49-21ksoftirqd/407:56:534
223239946989,257cyclictest10135-21runrttasks12:15:251
2232699465124,338cyclictest0-21swapper/411:10:324
2232699465124,338cyclictest0-21swapper/411:10:324
2232499465124,339cyclictest0-21swapper/211:16:452
2232499465124,338cyclictest704-21nscd10:55:222
2232399465124,338cyclictest0-21swapper/111:40:361
2232799464421,41cyclictest551rcuc/510:42:045
223249946480,211cyclictest321ktimersoftd/212:33:052
2232399464124,337cyclictest0-21swapper/112:41:161
2232999463164,168cyclictest13737-21ssh11:20:147
2232999463124,84cyclictest0-21swapper/710:03:527
223279946382,84cyclictest0-21swapper/510:27:265
2232599462119,212cyclictest10087-21kworker/u16:011:15:503
223279946140,255cyclictest0-21swapper/512:15:245
223299945976,380cyclictest0-21swapper/712:33:537
2232799459160,253cyclictest0-21swapper/508:00:025
223269945934,422cyclictest0-21swapper/411:55:264
2232399459118,338cyclictest0-21swapper/110:55:131
223299945836,256cyclictest0-21swapper/710:31:267
223259945878,84cyclictest0-21swapper/311:41:583
2232399458117,338cyclictest0-21swapper/110:32:221
2232799457116,338cyclictest0-21swapper/510:51:185
2232799457116,338cyclictest0-21swapper/510:31:505
223269945775,338cyclictest0-21swapper/408:00:164
223259945776,84cyclictest0-21swapper/308:00:233
223239945735,256cyclictest0-21swapper/108:00:211
223289945634,2cyclictest0-21swapper/612:15:186
223249945671,211cyclictest9919-21gltestperf12:15:142
223239945635,255cyclictest0-21swapper/110:26:561
223229945672,217cyclictest0-21swapper/012:05:050
223299945533,2cyclictest5135-21kworker/u16:111:41:147
223289945432,343cyclictest14688-21ssh11:00:136
223259945474,84cyclictest0-21swapper/310:32:113
223229945372,84cyclictest0-21swapper/011:45:300
223239945272,256cyclictest12946-21sh11:20:061
223229945230,253cyclictest25554-21ssh12:41:230
2232499451110,338cyclictest0-21swapper/212:42:512
223299944966,380cyclictest0-21swapper/707:57:297
223269944825,126cyclictest7839-21ssh11:35:144
223249944825,256cyclictest0-21swapper/211:10:322
223249944825,256cyclictest0-21swapper/211:10:312
223299944725,256cyclictest0-21swapper/710:26:567
223299944665,84cyclictest0-21swapper/711:32:037
223299944623,256cyclictest0-21swapper/710:45:217
2232599444103,338cyclictest0-21swapper/312:15:223
223299944119,2cyclictest0-21swapper/708:00:217
223249943816,256cyclictest0-21swapper/210:26:562
223239943757,169cyclictest0-21swapper/111:45:141
223259943655,84cyclictest1551-21snmpd11:45:303
223249943613,256cyclictest0-21swapper/208:00:222
223239943252,212cyclictest0-21swapper/110:45:201
22328994316,41cyclictest0-21swapper/608:00:166
223269942584,338cyclictest21341-21tail10:45:174
22327994241,126cyclictest0-21swapper/511:45:115
223299942180,338cyclictest0-21swapper/711:35:127
223229942180,168cyclictest0-21swapper/011:00:190
223299942080,338cyclictest0-21swapper/711:10:207
223299942080,338cyclictest0-21swapper/711:10:207
223289942082,84cyclictest0-21swapper/611:20:146
223279942081,84cyclictest0-21swapper/510:45:175
223229942082,84cyclictest0-21swapper/010:55:110
223299941938,296cyclictest0-21swapper/711:55:267
223259941975,341cyclictest0-21swapper/310:05:063
223249941979,338cyclictest0-21swapper/211:00:202
223299941879,84cyclictest0-21swapper/710:05:077
223289941877,338cyclictest0-21swapper/611:50:036
223279941879,84cyclictest0-21swapper/511:10:275
223279941879,84cyclictest0-21swapper/511:10:275
223229941879,84cyclictest0-21swapper/010:45:210
223259941634,84cyclictest4288-21expr10:55:153
223299941576,84cyclictest0-21swapper/711:50:137
223279941578,84cyclictest3948-21tr10:55:125
223279941576,84cyclictest0-21swapper/511:00:195
223279941576,50cyclictest0-21swapper/511:50:145
223269941575,338cyclictest0-21swapper/411:20:094
223289941472,339cyclictest0-21swapper/610:05:026
223269941431,257cyclictest0-21swapper/411:00:154
223239941430,211cyclictest0-21swapper/110:05:091
223289941373,338cyclictest12503-21ssh11:55:246
223259941172,84cyclictest0-21swapper/311:00:203
223249941131,41cyclictest0-21swapper/208:00:022
2232299411116,171cyclictest0-21swapper/011:10:230
2232299411116,171cyclictest0-21swapper/011:10:230
2232799410115,5cyclictest11775-21awk11:55:185
223259941030,84cyclictest0-21swapper/311:10:323
223259941030,84cyclictest0-21swapper/311:10:323
223229940723,295cyclictest0-21swapper/007:58:030
223229940668,84cyclictest0-21swapper/010:05:060
223269940564,338cyclictest0-21swapper/411:50:144
223249940563,339cyclictest0-21swapper/211:55:242
223299940465,84cyclictest0-21swapper/711:00:137
223259940466,83cyclictest0-21swapper/310:45:173
223229940465,84cyclictest0-21swapper/011:50:130
223269940219,253cyclictest24745-21python11:45:264
223249940223,2cyclictest0-21swapper/211:45:302
223289939756,338cyclictest0-21swapper/610:45:226
223259939498,84cyclictest0-21swapper/311:50:133
223279939012,289cyclictest0-21swapper/511:20:145
223239939047,126cyclictest11777-21proc_pri11:55:181
223279938512,205cyclictest0-21swapper/511:35:085
2232599381337,42cyclictest391rcuc/307:15:113
2232799376118,126cyclictest0-21swapper/510:05:115
2232399374117,254cyclictest15650-21ssh11:00:191
223239937276,84cyclictest0-21swapper/107:15:101
2232599371113,126cyclictest41-21ksoftirqd/311:55:183
223259936467,84cyclictest0-21swapper/311:20:083
223229936325,177cyclictest0-21swapper/011:20:070
223249936122,84cyclictest0-21swapper/210:05:112
223229936021,84cyclictest0-21swapper/011:55:230
2232499355102,84cyclictest121rcu_preempt11:20:132
223249935417,127cyclictest0-21swapper/211:50:032
223299933338,87cyclictest0-21swapper/707:15:117
223229933379,84cyclictest0-21swapper/007:15:100
223269933175,253cyclictest0-21swapper/407:15:124
223239933075,84cyclictest865-21df_inode11:50:131
223289932934,86cyclictest0-21swapper/607:15:116
223249930043,254cyclictest0-21swapper/207:15:112
223279928977,84cyclictest0-21swapper/507:15:115
20642210095,2sleep00-21swapper/007:05:190
2165128657,26sleep60-21swapper/607:06:376
2174028446,8sleep10-21swapper/107:07:531
2179428244,34sleep70-21swapper/707:08:397
2180828151,27sleep30-21swapper/307:08:503
2157528141,26sleep50-21swapper/507:05:325
40172790,0sleep40-21swapper/410:17:334
2175327848,27sleep40-21swapper/407:08:034
2163827850,25sleep20-21swapper/207:06:242
147782780,0sleep70-21swapper/709:17:407
180492750,1sleep30-21swapper/309:19:523
35132730,0sleep40-21swapper/409:11:524
22322994218,13cyclictest9-21ksoftirqd/010:35:140
2232399418,25cyclictest25-21ksoftirqd/110:35:041
22326993831,4cyclictest0-21swapper/410:35:034
2232899372,31cyclictest0-21swapper/610:35:026
2232799374,20cyclictest57-21ksoftirqd/510:35:185
2232999362,30cyclictest0-21swapper/710:35:067
22325993530,3cyclictest0-21swapper/310:35:063
22329993226,5cyclictest0-21swapper/709:45:587
22329993226,5cyclictest0-21swapper/709:45:587
22324993122,6cyclictest4987-21netstat10:35:242
22329992810,18cyclictest0-21swapper/708:36:307
22329992810,18cyclictest0-21swapper/708:20:047
22328992810,0cyclictest0-21swapper/607:42:346
22325992810,18cyclictest0-21swapper/309:46:353
22325992810,18cyclictest0-21swapper/309:46:353
22325992810,0cyclictest0-21swapper/309:04:393
22325992810,0cyclictest0-21swapper/307:42:483
22322992827,1cyclictest0-21swapper/009:46:180
22322992827,1cyclictest0-21swapper/009:46:180
22329992727,0cyclictest0-21swapper/709:04:127
22329992727,0cyclictest0-21swapper/708:19:377
22329992710,17cyclictest0-21swapper/707:31:077
22329992710,17cyclictest0-21swapper/707:12:387
22329992710,0cyclictest0-21swapper/707:42:467
22328992727,0cyclictest0-21swapper/607:13:106
22328992710,17cyclictest0-21swapper/609:46:026
22328992710,17cyclictest0-21swapper/609:46:026
22328992710,17cyclictest0-21swapper/609:04:126
22328992710,17cyclictest0-21swapper/608:20:016
22328992710,17cyclictest0-21swapper/607:31:116
22327992710,17cyclictest0-21swapper/508:19:595
22327992710,17cyclictest0-21swapper/507:42:175
22327992710,17cyclictest0-21swapper/507:31:425
22327992710,0cyclictest0-21swapper/509:46:325
22327992710,0cyclictest0-21swapper/509:46:325
22327992710,0cyclictest0-21swapper/509:04:325
22327992710,0cyclictest0-21swapper/508:36:245
22327992710,0cyclictest0-21swapper/507:12:505
22326992727,0cyclictest0-21swapper/407:42:524
22326992727,0cyclictest0-21swapper/407:12:444
22326992710,17cyclictest0-21swapper/409:45:564
22326992710,17cyclictest0-21swapper/409:45:564
22326992710,17cyclictest0-21swapper/409:04:274
22326992710,17cyclictest0-21swapper/408:20:004
22326992710,0cyclictest0-21swapper/408:36:244
22326992710,0cyclictest0-21swapper/408:19:364
22326992710,0cyclictest0-21swapper/407:31:044
22325992727,0cyclictest0-21swapper/308:36:453
22325992727,0cyclictest0-21swapper/307:31:273
22325992710,0cyclictest0-21swapper/308:19:373
22325992710,0cyclictest0-21swapper/307:12:383
22324992710,17cyclictest0-21swapper/209:04:092
22324992710,17cyclictest0-21swapper/208:20:062
22324992710,17cyclictest0-21swapper/208:19:412
22324992710,0cyclictest0-21swapper/209:45:542
22324992710,0cyclictest0-21swapper/209:45:542
22324992710,0cyclictest0-21swapper/208:36:292
22324992710,0cyclictest0-21swapper/207:42:202
22324992710,0cyclictest0-21swapper/207:31:092
22324992710,0cyclictest0-21swapper/207:12:412
22323992727,0cyclictest0-21swapper/107:42:481
22323992710,17cyclictest0-21swapper/108:36:561
22323992710,0cyclictest0-21swapper/109:45:521
22323992710,0cyclictest0-21swapper/109:45:521
22323992710,0cyclictest0-21swapper/109:04:091
22323992710,0cyclictest0-21swapper/108:19:551
22322992726,1cyclictest0-21swapper/007:13:070
22322992710,0cyclictest0-21swapper/009:04:470
22322992710,0cyclictest0-21swapper/008:20:000
22322992710,0cyclictest0-21swapper/008:19:340
22322992710,0cyclictest0-21swapper/007:42:180
22322992710,0cyclictest0-21swapper/007:31:380
22328992626,0cyclictest0-21swapper/608:37:016
22328992626,0cyclictest0-21swapper/608:20:006
22325992626,0cyclictest0-21swapper/308:20:113
22323992626,0cyclictest0-21swapper/108:20:081
22323992626,0cyclictest0-21swapper/107:31:471
22323992626,0cyclictest0-21swapper/107:13:091
22327992424,0cyclictest0-21swapper/508:20:125
22322992424,0cyclictest0-21swapper/008:36:460
2232299230,22cyclictest0-21swapper/012:05:110
2232299218,7cyclictest0-21swapper/008:10:010
2232299218,7cyclictest0-21swapper/008:10:000
2232299207,8cyclictest9-21ksoftirqd/012:20:220
2232299206,9cyclictest9-21ksoftirqd/012:25:110
2232299206,9cyclictest9-21ksoftirqd/010:10:140
272392190,0sleep60-21swapper/612:05:176
2232899190,1cyclictest0-21swapper/610:10:126
2232499190,18cyclictest0-21swapper/207:55:002
2232299197,7cyclictest9-21ksoftirqd/009:15:110
2232299197,7cyclictest4733-21latency08:25:160
2232299196,8cyclictest9-21ksoftirqd/011:27:110
2232299196,8cyclictest9-21ksoftirqd/009:57:400
164482190,0sleep60-21swapper/612:36:266
2232299188,6cyclictest9953-21ls10:20:160
2232299187,6cyclictest9-21ksoftirqd/009:20:200
2232299186,7cyclictest9-21ksoftirqd/009:05:120
2232299185,8cyclictest9-21ksoftirqd/009:40:090
2232299184,7cyclictest9-21ksoftirqd/008:55:000
22322991816,1cyclictest101ktimersoftd/009:32:130
22322991810,2cyclictest9-21ksoftirqd/009:39:020
22329991715,1cyclictest0-21swapper/707:55:017
22325991717,0cyclictest0-21swapper/307:55:023
22323991716,1cyclictest0-21swapper/107:55:001
2232299179,1cyclictest9-21ksoftirqd/009:30:010
2232299178,4cyclictest8558-21perf07:40:000
2232299177,7cyclictest9-21ksoftirqd/012:37:520
2232299177,5cyclictest9-21ksoftirqd/009:10:260
2232299176,7cyclictest9-21ksoftirqd/009:51:140
2232299176,6cyclictest9-21ksoftirqd/008:45:140
2232299175,7cyclictest9-21ksoftirqd/010:15:130
2232299175,7cyclictest9-21ksoftirqd/008:55:000
22326991616,0cyclictest0-21swapper/407:55:024
2232299168,4cyclictest29434-21cron08:15:000
2232299167,5cyclictest0-21swapper/007:50:210
2232299166,6cyclictest9-21ksoftirqd/008:40:150
2232299166,6cyclictest9-21ksoftirqd/007:25:130
2232299165,6cyclictest6542-21grep08:30:120
22329991513,1cyclictest0-21swapper/712:35:537
2232999151,12cyclictest25966-21tr12:05:137
2232799150,14cyclictest0-21swapper/512:36:025
2232299156,5cyclictest0-21swapper/007:20:140
2232299154,5cyclictest9-21ksoftirqd/007:45:110
263552140,0sleep70-21swapper/712:23:597
22327991412,1cyclictest14727-21/usr/sbin/munin09:50:225
22327991412,1cyclictest0-21swapper/507:25:175
22327991411,2cyclictest561ktimersoftd/511:26:365
2232799140,1cyclictest0-21swapper/512:05:165
22326991412,1cyclictest23821-21ssh09:38:284
22326991412,1cyclictest0-21swapper/408:25:174
2232599143,7cyclictest41-21ksoftirqd/311:30:003
22325991412,1cyclictest401ktimersoftd/309:30:423
2232599140,1cyclictest0-21swapper/312:36:063
22324991411,2cyclictest321ktimersoftd/208:45:242
2232499141,12cyclictest0-21swapper/209:16:212
2232999139,3cyclictest27887-21head09:40:137
2232999139,3cyclictest20663-21cut10:10:127
22329991311,1cyclictest0-21swapper/710:16:347
22329991311,1cyclictest0-21swapper/709:27:327
22329991310,2cyclictest29470-21ntpq09:05:177
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional