You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 14:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Mon Apr 13, 2026 12:55:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2889499734141,126cyclictest121rcu_preempt11:41:304
2889199659278,127cyclictest121rcu_preempt09:22:191
288929965823,384cyclictest0-21swapper/209:27:402
288949965724,2cyclictest0-21swapper/409:27:394
288919965256,339cyclictest121rcu_preempt07:34:501
28891996373,3cyclictest0-21swapper/109:27:401
28890996374,382cyclictest0-21swapper/009:27:400
28890996371,126cyclictest0-21swapper/011:35:030
28896996362,3cyclictest0-21swapper/609:25:076
28895996351,383cyclictest0-21swapper/509:27:395
288949963237,380cyclictest0-21swapper/412:42:214
2889299630121,84cyclictest0-21swapper/207:53:562
288959962734,126cyclictest0-21swapper/512:39:455
2889099626285,211cyclictest4305-21diskmemload12:24:310
2889099626285,211cyclictest4305-21diskmemload12:24:300
288969962530,592cyclictest0-21swapper/609:18:286
288909962231,422cyclictest0-21swapper/009:30:190
288909962231,422cyclictest0-21swapper/009:30:180
288909961928,338cyclictest0-21swapper/012:40:170
288919960354,338cyclictest0-21swapper/108:23:561
28892995971,380cyclictest0-21swapper/209:21:482
28893995950,84cyclictest0-21swapper/312:42:003
28893995940,84cyclictest0-21swapper/309:29:493
28895995930,84cyclictest0-21swapper/512:42:015
28894995930,84cyclictest0-21swapper/412:15:094
2889399590332,85cyclictest0-21swapper/311:33:263
288929958532,340cyclictest0-21swapper/209:19:242
2889299584162,254cyclictest28957-21kworker/u16:312:42:112
288909957927,126cyclictest0-21swapper/011:52:510
2889799572489,3cyclictest711rcuc/709:37:567
288959957260,296cyclictest0-21swapper/511:33:425
288929956716,126cyclictest0-21swapper/211:33:402
288949956658,84cyclictest0-21swapper/409:21:484
288909956515,126cyclictest0-21swapper/009:24:490
288969956411,550cyclictest0-21swapper/609:20:296
288979956211,126cyclictest0-21swapper/709:17:447
2889499561135,295cyclictest0-21swapper/412:39:454
288969956047,297cyclictest0-21swapper/610:49:486
28896995598,42cyclictest0-21swapper/611:44:176
2889799555171,126cyclictest121rcu_preempt12:44:477
2889399549295,84cyclictest0-21swapper/312:38:403
2889499547252,42cyclictest14578-21ssh11:32:354
2889399547291,126cyclictest121rcu_preempt12:13:163
2889699544118,254cyclictest23759-21chrt07:53:576
288959954437,84cyclictest0-21swapper/509:17:265
2889699541116,83cyclictest0-21swapper/612:39:036
288929954175,211cyclictest0-21swapper/211:55:102
288969954033,388cyclictest28888-21cyclictest08:52:536
288969954032,428cyclictest0-21swapper/611:52:146
2889299540115,211cyclictest14253-21ssh10:57:562
288939953827,84cyclictest0-21swapper/307:33:003
288909953830,126cyclictest121rcu_preempt08:22:200
2889399537152,254cyclictest0-21swapper/308:23:323
2889299535113,338cyclictest0-21swapper/210:49:142
288909953423,296cyclictest0-21swapper/012:03:140
288959953267,211cyclictest0-21swapper/509:21:465
288979953124,256cyclictest0-21swapper/708:22:587
288939953165,84cyclictest0-21swapper/309:22:063
288969953020,507cyclictest0-21swapper/612:11:336
2889099528106,50cyclictest5842-21kworker/u16:310:07:150
288969952716,170cyclictest0-21swapper/612:41:186
288919952657,211cyclictest0-21swapper/108:53:101
288969952459,126cyclictest121rcu_preempt08:21:376
2889199519221,126cyclictest19639-1kworker/u17:112:14:581
288979951810,126cyclictest0-21swapper/712:13:227
288919951810,126cyclictest0-21swapper/111:30:281
288909951812,41cyclictest0-21swapper/011:45:270
288969951547,465cyclictest0-21swapper/611:31:346
28890995154,48cyclictest0-21swapper/012:38:480
288979951445,211cyclictest73-21ksoftirqd/711:31:237
288969951444,253cyclictest0-21swapper/612:24:366
288969951444,253cyclictest0-21swapper/612:24:366
28897995131,84cyclictest1553-21runrttasks11:42:117
28890995115,338cyclictest0-21swapper/012:11:340
288959951083,211cyclictest561ktimersoftd/511:44:355
28893995090,84cyclictest0-21swapper/312:15:093
2889699506124,84cyclictest0-21swapper/610:58:036
288929950439,380cyclictest0-21swapper/211:12:202
2889299504125,84cyclictest0-21swapper/211:45:182
2889099504164,126cyclictest28704-21kworker/u16:211:44:030
2889099504124,126cyclictest9-21ksoftirqd/008:51:370
288949950238,213cyclictest0-21swapper/411:45:154
2889099502122,84cyclictest0-21swapper/010:50:030
288929950136,296cyclictest0-21swapper/211:53:262
288929950035,296cyclictest0-21swapper/210:50:212
288919950034,126cyclictest0-21swapper/109:16:451
2889099500124,80cyclictest121rcu_preempt08:55:200
288949949972,211cyclictest0-21swapper/407:35:004
2889599498242,254cyclictest561ktimersoftd/512:25:045
288919949731,84cyclictest0-21swapper/112:24:121
288919949731,84cyclictest0-21swapper/112:24:121
2889599495411,4cyclictest551rcuc/510:50:025
288969949425,338cyclictest0-21swapper/611:45:146
288949949469,422cyclictest0-21swapper/412:23:404
288949949469,422cyclictest0-21swapper/412:23:394
288919949433,3cyclictest121rcu_preempt10:58:211
288969949371,2cyclictest1553-21runrttasks08:25:116
288969949126,385cyclictest0-21swapper/610:50:286
2889599490106,126cyclictest0-21swapper/512:12:305
288909949025,126cyclictest10483-21ssh12:25:260
288979948920,84cyclictest121rcu_preempt11:45:207
288959948920,295cyclictest121rcu_preempt08:55:005
288949948925,212cyclictest0-21swapper/408:23:474
288939948923,126cyclictest0-21swapper/309:19:303
2889199489150,84cyclictest121rcu_preempt11:42:271
288959948665,83cyclictest0-21swapper/511:45:185
288949948660,253cyclictest0-21swapper/412:13:224
2889299486187,126cyclictest0-21swapper/212:24:132
2889299486187,126cyclictest0-21swapper/212:24:122
288949948519,211cyclictest0-21swapper/412:02:464
288909948517,337cyclictest0-21swapper/009:18:460
288969948318,388cyclictest0-21swapper/607:34:156
288979948111,127cyclictest121rcu_preempt12:23:297
288979948111,127cyclictest121rcu_preempt12:23:287
288939948155,253cyclictest0-21swapper/309:30:233
288939948155,253cyclictest0-21swapper/309:30:223
288979948055,169cyclictest0-21swapper/709:29:487
2889599479142,218cyclictest28704-21kworker/u16:211:55:015
28892994767,296cyclictest33-21ksoftirqd/208:25:012
28896994747,464cyclictest0-21swapper/611:35:136
288919947495,84cyclictest0-21swapper/111:52:121
288909947494,84cyclictest0-21swapper/011:35:190
2889599469170,126cyclictest0-21swapper/510:49:505
288979946883,254cyclictest0-21swapper/709:30:137
288979946883,254cyclictest0-21swapper/709:30:137
2889099467210,127cyclictest121rcu_preempt09:59:560
28896994651,42cyclictest6904-21proc_pri09:30:306
28896994651,42cyclictest6904-21proc_pri09:30:296
2889699465125,84cyclictest0-21swapper/608:55:096
2889499465125,338cyclictest0-21swapper/411:35:194
2889299464124,338cyclictest0-21swapper/209:30:142
2889299464124,338cyclictest0-21swapper/209:30:132
288979946378,211cyclictest73-21ksoftirqd/712:02:097
2889699463124,84cyclictest0-21swapper/612:02:226
288949946380,126cyclictest121rcu_preempt07:53:534
288949946379,211cyclictest21162-21ssh09:17:434
2889499463125,171cyclictest481ktimersoftd/411:53:564
2889399463122,338cyclictest0-21swapper/308:55:183
288929946341,260cyclictest0-21swapper/211:41:292
288939946239,3cyclictest0-21swapper/311:42:443
2889699461122,84cyclictest0-21swapper/607:35:056
288929946139,303cyclictest0-21swapper/212:15:022
288929946139,2cyclictest0-21swapper/208:55:212
288959946077,211cyclictest0-21swapper/512:21:425
288959946077,210cyclictest0-21swapper/507:34:155
288929946038,261cyclictest0-21swapper/212:38:242
288909946077,84cyclictest0-21swapper/007:55:080
288909946037,219cyclictest0-21swapper/008:25:110
288969945933,423cyclictest0-21swapper/610:33:236
288969945933,423cyclictest0-21swapper/610:33:226
2889699459120,84cyclictest0-21swapper/612:05:266
288919945937,2cyclictest0-21swapper/108:55:211
288979945838,336cyclictest0-21swapper/709:25:017
288959945873,211cyclictest57-21ksoftirqd/511:55:095
288959945836,3cyclictest0-21swapper/509:59:575
288969945731,423cyclictest0-21swapper/609:37:566
288939945734,126cyclictest0-21swapper/307:54:473
288929945734,256cyclictest0-21swapper/209:59:562
2889299457117,84cyclictest0-21swapper/212:05:262
288909945731,295cyclictest0-21swapper/007:55:010
288929945634,256cyclictest0-21swapper/208:54:512
288909945634,2cyclictest0-21swapper/010:00:190
2889799455115,337cyclictest0-21swapper/712:25:247
2889299455114,338cyclictest0-21swapper/212:11:332
288919945528,212cyclictest25-21ksoftirqd/109:38:051
2889799454121,248cyclictest0-21swapper/707:54:487
2889399454113,211cyclictest0-21swapper/312:22:513
2889399454113,211cyclictest0-21swapper/312:22:513
288919945471,380cyclictest0-21swapper/112:02:571
2889199454114,338cyclictest9514-21sh11:29:371
288949945335,44cyclictest0-21swapper/409:30:414
288949945335,44cyclictest0-21swapper/409:30:414
288939945230,2cyclictest0-21swapper/310:50:303
288919945230,256cyclictest0-21swapper/112:45:001
288919945230,219cyclictest0-21swapper/112:38:491
288919945225,211cyclictest0-21swapper/107:54:221
2889299451109,84cyclictest0-21swapper/212:02:222
2889199451112,84cyclictest0-21swapper/112:25:201
288919945028,2cyclictest0-21swapper/110:50:151
2889799449276,84cyclictest0-21swapper/712:38:307
288969944929,79cyclictest12453-21ssh10:07:156
288949944927,2cyclictest0-21swapper/410:00:194
288909944826,256cyclictest0-21swapper/010:58:120
288979944763,212cyclictest0-21swapper/711:54:567
2889099447108,84cyclictest0-21swapper/012:05:250
2889399446107,126cyclictest41-21ksoftirqd/311:45:053
2889299446107,84cyclictest0-21swapper/207:34:502
288919944663,211cyclictest0-21swapper/110:07:151
2889199446107,127cyclictest9447-1kworker/u17:211:12:511
288919944519,211cyclictest0-21swapper/110:49:551
288969944460,211cyclictest65-21ksoftirqd/612:15:166
288949944422,256cyclictest0-21swapper/408:55:024
288959944320,126cyclictest121rcu_preempt09:30:145
288959944320,126cyclictest121rcu_preempt09:30:145
2889399443101,254cyclictest0-21swapper/312:25:263
2889099443185,88cyclictest121rcu_preempt10:49:500
2889499442100,339cyclictest0-21swapper/410:49:134
2889399441273,2cyclictest401ktimersoftd/311:12:313
288909944161,3cyclictest25053-21expr12:15:160
288959943953,211cyclictest57-21ksoftirqd/508:55:095
288979943715,256cyclictest0-21swapper/707:33:317
288949943614,2cyclictest0-21swapper/408:25:114
288979943512,340cyclictest31046-21ssh10:50:157
288929943513,2cyclictest0-21swapper/208:25:122
288939943310,260cyclictest0-21swapper/309:38:073
288919943311,256cyclictest0-21swapper/109:59:561
28896994319,256cyclictest0-21swapper/609:59:566
288969943192,171cyclictest121rcu_preempt10:00:126
2889799430391,37cyclictest711rcuc/710:49:037
28897994297,256cyclictest0-21swapper/712:15:117
288939942845,127cyclictest41-21ksoftirqd/308:53:193
28892994262,4cyclictest0-21swapper/210:00:202
288979942582,211cyclictest73-21ksoftirqd/708:55:037
288949942483,338cyclictest0-21swapper/412:25:114
288979942383,338cyclictest0-21swapper/710:58:497
28897994231,256cyclictest0-21swapper/709:59:567
2889599423251,84cyclictest0-21swapper/512:02:595
288939942382,338cyclictest0-21swapper/310:58:343
288929942383,338cyclictest0-21swapper/211:29:372
288919942382,338cyclictest0-21swapper/110:00:141
288919942382,338cyclictest0-21swapper/109:30:131
288919942382,338cyclictest0-21swapper/109:30:131
28890994231,297cyclictest0-21swapper/009:37:160
288959942239,380cyclictest0-21swapper/508:22:525
288939942281,338cyclictest0-21swapper/311:29:373
288949942140,256cyclictest0-21swapper/410:33:184
288949942140,256cyclictest0-21swapper/410:33:184
288929942182,84cyclictest9833-21/usr/sbin/munin12:25:142
288919942181,338cyclictest0-21swapper/110:33:231
288919942181,338cyclictest0-21swapper/110:33:231
288909942182,84cyclictest0-21swapper/011:55:140
288959942036,127cyclictest121rcu_preempt07:53:035
288929942036,211cyclictest23845-21apt-get07:55:042
2889199420121,211cyclictest25233-21taskset12:15:191
288979941978,338cyclictest0-21swapper/711:29:547
288919941980,84cyclictest0-21swapper/111:45:291
288909941980,84cyclictest0-21swapper/010:33:240
288909941980,84cyclictest0-21swapper/010:33:230
288969941839,295cyclictest0-21swapper/612:25:156
288969941834,126cyclictest0-21swapper/611:12:216
288949941878,337cyclictest0-21swapper/410:58:344
288959941779,83cyclictest0-21swapper/508:25:115
2889399417201,84cyclictest0-21swapper/311:53:253
288909941734,125cyclictest0-21swapper/007:33:580
288959941679,84cyclictest0-21swapper/511:35:195
288939941679,253cyclictest0-21swapper/310:00:203
288939941635,126cyclictest0-21swapper/312:03:223
288939941574,338cyclictest0-21swapper/307:35:063
288949941434,258cyclictest30223-21/usr/sbin/munin10:50:214
288949941374,336cyclictest0-21swapper/411:29:364
288929941334,295cyclictest0-21swapper/210:33:182
288929941334,295cyclictest0-21swapper/210:33:172
288909941376,42cyclictest0-21swapper/011:12:220
2889799412328,3cyclictest711rcuc/707:35:127
2889499412113,126cyclictest121rcu_preempt07:55:034
288919941232,296cyclictest0-21swapper/111:55:081
288979941070,338cyclictest0-21swapper/711:12:457
288939941069,84cyclictest0-21swapper/307:55:093
288959940970,84cyclictest0-21swapper/512:25:105
288949940969,338cyclictest0-21swapper/411:12:454
288959940870,83cyclictest0-21swapper/510:33:245
288959940870,83cyclictest0-21swapper/510:33:235
288939940869,84cyclictest0-21swapper/310:49:143
2889299408113,84cyclictest18135-21ssh11:35:192
288909940870,84cyclictest0-21swapper/007:35:050
288959940770,82cyclictest0-21swapper/507:35:065
2889299407111,84cyclictest18384-21ssh09:37:162
288909940766,338cyclictest0-21swapper/011:29:370
288969940564,338cyclictest0-21swapper/611:29:556
288959940323,42cyclictest0-21swapper/510:07:155
288959940115,84cyclictest0-21swapper/507:55:085
2889499401115,206cyclictest28888-21cyclictest09:37:324
288979939756,338cyclictest0-21swapper/710:00:047
288949939756,126cyclictest29448-21cat08:55:224
288969939655,339cyclictest0-21swapper/611:55:146
288969939554,339cyclictest0-21swapper/607:55:026
288959939455,84cyclictest0-21swapper/510:58:355
288959939254,254cyclictest0-21swapper/510:00:195
2889799389131,255cyclictest721ktimersoftd/711:55:107
28893993862,211cyclictest28704-21kworker/u16:212:05:253
28891993851,253cyclictest0-21swapper/107:55:091
288979938183,85cyclictest0-21swapper/711:35:137
2889799378124,84cyclictest0-21swapper/710:07:157
288949937840,171cyclictest0-21swapper/411:55:114
288959937783,170cyclictest0-21swapper/509:37:415
288929937740,253cyclictest0-21swapper/210:07:162
288939937634,127cyclictest0-21swapper/311:55:133
288939937534,126cyclictest0-21swapper/310:07:153
288919937535,337cyclictest0-21swapper/112:05:251
288919937534,338cyclictest0-21swapper/111:35:151
288949937479,211cyclictest28888-21cyclictest07:35:114
2889799370113,84cyclictest0-21swapper/708:55:177
288939937029,338cyclictest0-21swapper/310:33:243
288939937029,338cyclictest0-21swapper/310:33:233
2889599366320,43cyclictest551rcuc/511:12:525
288919936629,2cyclictest9956-21apt-get08:25:131
288959936573,119cyclictest0-21swapper/512:15:025
288949936569,84cyclictest0-21swapper/412:05:264
288959936325,83cyclictest0-21swapper/511:29:555
288959935518,169cyclictest0-21swapper/512:05:265
28897993529,211cyclictest0-21swapper/712:05:257
288979935011,84cyclictest0-21swapper/710:33:247
288979935011,84cyclictest0-21swapper/710:33:237
28897993478,84cyclictest0-21swapper/707:55:187
28894993469,42cyclictest0-21swapper/410:07:164
28894993467,84cyclictest0-21swapper/409:59:574
288939934345,295cyclictest0-21swapper/308:25:133
288939934344,126cyclictest41-21ksoftirqd/311:35:173
288929933780,254cyclictest0-21swapper/207:35:052
288939933540,84cyclictest0-21swapper/309:59:563
288919932468,253cyclictest0-21swapper/107:35:051
288979926512,2cyclictest0-21swapper/708:25:137
2830428858,27sleep00-21swapper/007:07:500
2694728447,32sleep60-21swapper/607:05:146
2817128353,27sleep70-21swapper/707:05:557
2823228251,28sleep30-21swapper/307:06:483
2841128149,28sleep40-21swapper/407:09:224
2827028150,27sleep20-21swapper/207:07:202
2822528050,27sleep50-21swapper/507:06:425
241342800,0sleep20-21swapper/211:03:432
2582527947,28sleep10-21swapper/107:05:021
249922790,1sleep10-21swapper/111:20:171
135192720,4sleep013523-21cpuspeed_turbos11:15:130
202292520,2sleep30-21swapper/310:28:113
28894993928,7cyclictest0-21swapper/410:26:224
2889799382,4cyclictest0-21swapper/710:26:227
2889199372,32cyclictest0-21swapper/110:26:241
2889699362,31cyclictest0-21swapper/610:26:216
28895993632,2cyclictest0-21swapper/510:26:355
2889099361,32cyclictest0-21swapper/010:26:530
2889599359,14cyclictest57-21ksoftirqd/510:35:175
2889599356,18cyclictest57-21ksoftirqd/511:15:205
28894993423,7cyclictest17201-21grep10:10:184
28893993426,5cyclictest22505-21ssh09:40:123
2889299342,29cyclictest0-21swapper/210:26:312
28890993427,4cyclictest24606-21ls09:40:290
2889799331,2cyclictest0-21swapper/711:00:057
2889699332,28cyclictest0-21swapper/611:00:126
28895993324,6cyclictest22535-21grep09:40:145
28893993325,5cyclictest17621-21sh10:10:223
28893993325,5cyclictest17248-21tr11:00:113
2889199331,28cyclictest14212-21cut11:15:151
28896993225,4cyclictest1721-21grep10:35:246
28896993123,5cyclictest22386-21grep09:40:136
28896993123,5cyclictest16357-21tr10:10:136
28895993124,4cyclictest18831-21ssh11:00:235
28895993124,4cyclictest16295-21cut10:10:145
28894993123,5cyclictest24978-21ssh09:40:374
28894993123,5cyclictest18706-21ssh11:00:224
28894993123,5cyclictest13932-21ssh11:15:144
28894993122,4cyclictest32237-21sh10:35:004
28893993124,4cyclictest3044-21ssh10:35:533
28892993124,4cyclictest22425-21cut09:40:122
28891993123,5cyclictest17996-21ssh10:10:251
2889799301,26cyclictest17712-21ssh10:10:227
28896993022,5cyclictest16341-21sh11:15:556
2889399301,18cyclictest41-21ksoftirqd/311:15:223
28891993023,4cyclictest23582-21ssh09:40:191
28891993022,5cyclictest20003-21ssh11:00:501
2889199301,25cyclictest836-21cat10:35:171
238302300,2sleep70-21swapper/709:40:217
28897992923,4cyclictest2759-21ssh10:35:387
28897992922,4cyclictest14534-21if_enp3s011:15:187
28896992910,18cyclictest0-21swapper/607:14:006
28894992927,1cyclictest64450irq/122-eth009:12:054
28890992922,4cyclictest32713-21tr10:35:150
2889099291,25cyclictest17432-21expr11:00:130
28897992810,18cyclictest0-21swapper/707:14:177
28897992810,0cyclictest0-21swapper/710:43:207
28897992810,0cyclictest0-21swapper/710:43:207
28897992810,0cyclictest0-21swapper/707:21:287
28896992826,1cyclictest0-21swapper/610:42:596
28896992826,1cyclictest0-21swapper/610:42:586
28896992810,17cyclictest0-21swapper/610:16:386
28896992810,17cyclictest0-21swapper/608:09:326
28896992810,0cyclictest0-21swapper/608:49:546
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional