You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 12:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat Jul 11, 2026 00:58:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308809975128,720cyclictest0-21swapper/223:06:542
308809972040,677cyclictest0-21swapper/200:33:242
308859966829,381cyclictest0-21swapper/719:48:407
308809966426,550cyclictest0-21swapper/223:43:122
3088299656403,92cyclictest0-21swapper/400:14:174
308849965572,118cyclictest0-21swapper/623:17:526
308819965214,84cyclictest0-21swapper/321:48:383
308859965096,126cyclictest0-21swapper/722:30:207
30882996405,126cyclictest0-21swapper/419:48:394
308819963985,126cyclictest0-21swapper/321:18:383
3088599636125,296cyclictest0-21swapper/721:18:367
30879996361,126cyclictest0-21swapper/122:00:531
3087899635169,84cyclictest121rcu_preempt00:13:590
308809963438,593cyclictest0-21swapper/223:59:222
3088299632163,338cyclictest121rcu_preempt23:19:564
3088299630122,84cyclictest0-21swapper/400:22:004
308809963034,127cyclictest0-21swapper/200:14:062
3088299627506,36cyclictest481ktimersoftd/423:06:374
308839962576,126cyclictest121rcu_preempt23:19:235
308799962173,126cyclictest0-21swapper/120:37:411
308829961969,212cyclictest0-21swapper/422:01:084
3088199616147,296cyclictest0-21swapper/319:18:073
308829961118,380cyclictest0-21swapper/421:47:314
308819959887,126cyclictest0-21swapper/321:22:063
30880995982,550cyclictest0-21swapper/200:08:132
308829959547,338cyclictest0-21swapper/421:23:444
308809959585,380cyclictest19416-21rm22:26:492
3088399592122,296cyclictest0-21swapper/521:16:125
308839959141,337cyclictest1847-21munin-run22:55:025
308839959141,337cyclictest1847-21munin-run22:55:025
3088499590507,3cyclictest631rcuc/623:10:146
3087999590122,126cyclictest121rcu_preempt23:46:421
308809958936,550cyclictest0-21swapper/221:46:292
3088099589121,254cyclictest121rcu_preempt00:00:082
3088399588122,84cyclictest0-21swapper/523:47:525
308819958129,127cyclictest0-21swapper/323:30:143
308809958069,338cyclictest0-21swapper/220:37:252
3088599579241,173cyclictest721ktimersoftd/721:25:307
308819957968,296cyclictest0-21swapper/300:35:213
308809957622,551cyclictest0-21swapper/222:30:052
308809957524,126cyclictest0-21swapper/222:00:572
308789957264,126cyclictest0-21swapper/023:10:120
3088199571186,296cyclictest0-21swapper/323:48:043
308809957117,126cyclictest0-21swapper/221:17:142
3088099570229,339cyclictest25358-21rm21:20:172
308849956914,339cyclictest65-21ksoftirqd/621:19:446
308809956815,380cyclictest0-21swapper/222:53:372
308809956815,380cyclictest0-21swapper/222:53:362
308799956860,126cyclictest0-21swapper/100:00:251
3087899567440,42cyclictest101ktimersoftd/022:27:440
308789956717,84cyclictest0-21swapper/021:23:170
3088299566355,42cyclictest0-21swapper/419:50:144
308819956552,339cyclictest0-21swapper/323:20:223
308829956255,210cyclictest0-21swapper/423:50:174
308829956211,295cyclictest0-21swapper/423:27:184
308859956151,126cyclictest0-21swapper/720:38:177
308809956113,126cyclictest0-21swapper/223:17:232
30883995609,169cyclictest0-21swapper/500:15:135
308829956052,84cyclictest0-21swapper/423:48:034
30878995596,423cyclictest0-21swapper/022:30:190
3088299555260,85cyclictest0-21swapper/400:31:514
30881995551,338cyclictest0-21swapper/322:02:373
30879995545,126cyclictest14776-21ssh22:43:161
3088399551125,253cyclictest0-21swapper/522:29:485
30878995501,211cyclictest0-21swapper/023:19:180
308859954983,126cyclictest0-21swapper/723:47:177
308839954983,126cyclictest0-21swapper/500:13:185
308829954678,465cyclictest0-21swapper/419:20:014
308839954534,126cyclictest30876-21cyclictest21:25:175
308819954478,84cyclictest121rcu_preempt20:38:063
308809954476,465cyclictest28130-21tr22:15:132
308809954476,465cyclictest28130-21tr22:15:122
308789954335,84cyclictest29337-21ssh23:45:550
3088199542114,211cyclictest722-21nscd21:50:323
308809954232,507cyclictest0-21swapper/219:49:242
308839954134,84cyclictest0-21swapper/523:30:165
308819954175,84cyclictest0-21swapper/322:27:013
308799954133,126cyclictest0-21swapper/123:07:271
308839954075,84cyclictest0-21swapper/523:06:235
308829954071,295cyclictest0-21swapper/422:53:244
308829954071,295cyclictest0-21swapper/422:53:244
308809954035,163cyclictest6086-21diskmemload23:46:282
308799954034,126cyclictest0-21swapper/121:20:451
308799953928,338cyclictest0-21swapper/123:29:191
308789953931,84cyclictest0-21swapper/023:58:460
308839953630,253cyclictest0-21swapper/523:41:595
308809953627,84cyclictest0-21swapper/222:43:592
308799953566,126cyclictest0-21swapper/123:17:571
308849953364,338cyclictest12796-21ssh22:40:396
308829953325,126cyclictest0-21swapper/423:20:214
308859953165,84cyclictest0-21swapper/700:09:187
308819953123,126cyclictest0-21swapper/323:10:033
308839953019,380cyclictest12955-21apt-get00:35:115
3088199530104,423cyclictest0-21swapper/300:25:033
308809953020,507cyclictest0-21swapper/200:42:572
308789952822,253cyclictest0-21swapper/000:08:520
308829952758,84cyclictest121rcu_preempt23:58:084
308839952657,296cyclictest0-21swapper/521:49:385
308819952618,3cyclictest0-21swapper/323:42:133
3088499525397,126cyclictest641ktimersoftd/622:04:076
308799952557,253cyclictest0-21swapper/122:53:371
308799952557,253cyclictest0-21swapper/122:53:371
308809952417,253cyclictest0-21swapper/221:25:292
308859952315,84cyclictest121rcu_preempt23:44:597
308789952210,339cyclictest0-21swapper/021:19:380
308799952110,84cyclictest0-21swapper/100:22:261
30878995198,380cyclictest0-21swapper/022:41:270
308799951810,84cyclictest121rcu_preempt00:09:391
30878995187,127cyclictest0-21swapper/000:32:000
308829951545,296cyclictest0-21swapper/421:25:134
308819951589,337cyclictest698-21ptp4l22:43:173
30881995143,338cyclictest0-21swapper/323:18:193
30878995148,385cyclictest0-21swapper/023:07:130
30880995136,428cyclictest0-21swapper/220:40:052
30882995126,84cyclictest0-21swapper/400:08:274
3088599511171,338cyclictest30876-21cyclictest21:46:477
30880995111,86cyclictest0-21swapper/223:10:112
308799951145,126cyclictest0-21swapper/119:18:081
308859951083,339cyclictest0-21swapper/700:11:147
308829951087,256cyclictest0-21swapper/419:20:074
30881995101,127cyclictest0-21swapper/300:11:503
30884995090,84cyclictest0-21swapper/623:58:096
30881995091,252cyclictest0-21swapper/322:53:553
30881995091,252cyclictest0-21swapper/322:53:553
30879995090,84cyclictest0-21swapper/100:10:531
3088499508124,211cyclictest24235-21ssh23:42:596
308789950842,127cyclictest0-21swapper/023:40:200
3087899507122,211cyclictest101ktimersoftd/021:49:280
3088599505461,42cyclictest711rcuc/721:24:177
308839950578,296cyclictest0-21swapper/520:20:275
308839950536,346cyclictest0-21swapper/522:42:075
308859950478,423cyclictest0-21swapper/722:40:377
308839950481,126cyclictest0-21swapper/523:29:545
308809950436,465cyclictest0-21swapper/221:50:192
308819950237,84cyclictest121rcu_preempt23:29:193
3087999502291,79cyclictest0-21swapper/123:20:251
308859950179,126cyclictest121rcu_preempt23:30:067
3088099501117,381cyclictest9122-21/usr/sbin/munin22:05:172
3087999501245,168cyclictest0-21swapper/123:40:071
308789950175,84cyclictest9-21ksoftirqd/022:45:170
3087999500203,84cyclictest6086-21diskmemload21:45:551
308819949983,248cyclictest0-21swapper/319:50:203
3088499498159,126cyclictest30876-21cyclictest23:48:166
308839949834,42cyclictest0-21swapper/521:24:325
308829949873,295cyclictest121rcu_preempt00:25:074
3088599497155,127cyclictest121rcu_preempt23:29:237
308829949733,219cyclictest0-21swapper/423:40:394
308829949673,168cyclictest0-21swapper/400:42:514
308799949628,341cyclictest0-21swapper/120:20:271
308849949568,296cyclictest0-21swapper/620:20:286
3088499495110,253cyclictest0-21swapper/600:11:296
308839949531,7cyclictest0-21swapper/523:40:005
308819949527,296cyclictest30876-21cyclictest00:33:333
308829949467,338cyclictest0-21swapper/421:50:274
308849949366,296cyclictest0-21swapper/600:32:556
308799949325,210cyclictest19195-21awk23:40:021
3088599492453,37cyclictest711rcuc/700:21:567
308849949269,84cyclictest0-21swapper/620:38:376
308859949166,210cyclictest0-21swapper/719:19:227
308839949123,214cyclictest0-21swapper/523:58:565
308829949128,124cyclictest121rcu_preempt22:40:444
308839949024,85cyclictest0-21swapper/522:03:345
3088299490446,41cyclictest471rcuc/422:26:024
308799949025,126cyclictest121rcu_preempt21:19:131
308809948920,466cyclictest0-21swapper/200:21:382
308859948864,127cyclictest121rcu_preempt22:52:087
308859948864,127cyclictest121rcu_preempt22:52:087
308849948866,84cyclictest0-21swapper/623:08:336
3088499488108,127cyclictest0-21swapper/619:17:536
308839948824,84cyclictest1395-21munin-node00:33:315
308859948720,127cyclictest29567-21/usr/sbin/munin22:15:217
308859948720,127cyclictest29567-21/usr/sbin/munin22:15:217
308819948663,130cyclictest23577-21fschecks_count22:30:193
308789948620,84cyclictest0-21swapper/023:28:340
3088499485229,253cyclictest641ktimersoftd/621:47:096
308799948517,126cyclictest0-21swapper/122:29:401
308849948421,83cyclictest121rcu_preempt19:45:346
308859948359,84cyclictest121rcu_preempt23:19:127
308839948370,329cyclictest0-21swapper/519:19:435
308819948398,211cyclictest32070-21sed23:10:323
308849948015,84cyclictest0-21swapper/621:21:026
308829948056,84cyclictest0-21swapper/422:30:194
308819948057,168cyclictest0-21swapper/300:46:083
308819948057,168cyclictest0-21swapper/300:46:083
308849947914,296cyclictest0-21swapper/600:00:236
308849947813,253cyclictest0-21swapper/600:23:276
308839947813,253cyclictest25988-21ssh00:23:435
3088199478136,339cyclictest24563-1kworker/u17:200:07:473
308799947852,253cyclictest121rcu_preempt19:48:221
308829947711,84cyclictest0-21swapper/420:37:244
308809947751,423cyclictest0-21swapper/219:50:142
3088599475436,3cyclictest721ktimersoftd/722:27:387
308859947493,126cyclictest29591-21ssh23:10:087
30878994748,84cyclictest0-21swapper/021:50:270
308859947291,211cyclictest121rcu_preempt21:50:177
308799947288,135cyclictest121rcu_preempt00:35:151
308789947248,126cyclictest0-21swapper/023:50:220
30879994716,253cyclictest0-21swapper/123:56:491
30878994691,126cyclictest24794-21ssh00:22:240
308849946840,211cyclictest0-21swapper/600:15:086
308849946783,253cyclictest0-21swapper/621:50:116
308809946681,211cyclictest33-21ksoftirqd/223:28:112
308789946682,211cyclictest9-21ksoftirqd/022:01:090
30884994651,84cyclictest0-21swapper/623:28:116
308829946539,125cyclictest121rcu_preempt23:30:034
30879994651,41cyclictest0-21swapper/100:43:011
308849946485,126cyclictest16660-21ntp_kernel_pll_23:20:256
308849946443,337cyclictest0-21swapper/622:53:596
308849946443,337cyclictest0-21swapper/622:53:596
308819946480,211cyclictest41-21ksoftirqd/322:45:113
3088099464120,126cyclictest0-21swapper/200:25:062
3087999464211,84cyclictest121rcu_preempt00:32:061
308849946339,126cyclictest1853-21ssh00:09:066
3088199463120,212cyclictest0-21swapper/319:49:493
3088499462121,338cyclictest0-21swapper/622:30:186
308819946240,256cyclictest0-21swapper/323:56:493
308839946183,81cyclictest0-21swapper/522:30:135
308839946037,219cyclictest2650-21perf00:10:005
3088399459118,338cyclictest0-21swapper/522:45:125
308849945836,2cyclictest1834-21rm23:50:236
308859945734,257cyclictest0-21swapper/700:31:247
3088599457161,84cyclictest121rcu_preempt23:06:397
3088399457117,338cyclictest0-21swapper/521:50:195
3088399457116,338cyclictest18892-21grep00:00:215
308789945734,84cyclictest0-21swapper/022:15:250
308789945734,84cyclictest0-21swapper/022:15:250
3088499456115,338cyclictest0-21swapper/622:29:176
308789945533,254cyclictest0-21swapper/022:55:030
308789945533,254cyclictest0-21swapper/022:55:030
308859945430,262cyclictest0-21swapper/723:59:387
3088599454113,338cyclictest3032-21ssh00:46:257
3088599454113,338cyclictest3032-21ssh00:46:257
308849945427,84cyclictest0-21swapper/600:42:316
308859945331,256cyclictest0-21swapper/700:41:107
308809945331,343cyclictest0-21swapper/200:46:362
308809945331,343cyclictest0-21swapper/200:46:362
308799945331,8cyclictest0-21swapper/119:50:091
308789945332,2cyclictest0-21swapper/023:30:140
308829945231,2cyclictest0-21swapper/400:45:354
308829945231,2cyclictest0-21swapper/400:45:354
308789945230,42cyclictest0-21swapper/019:45:240
308789945130,41cyclictest0-21swapper/019:19:320
308839945027,43cyclictest0-21swapper/520:35:285
3088299450195,253cyclictest481ktimersoftd/421:19:354
308819944927,126cyclictest0-21swapper/323:50:103
308809944922,424cyclictest0-21swapper/219:18:492
308789944927,85cyclictest0-21swapper/000:42:490
3087899449110,126cyclictest6086-21diskmemload00:46:040
3087899449110,126cyclictest6086-21diskmemload00:46:040
3087999448107,338cyclictest1346-21diskmemload21:25:361
3087999448107,126cyclictest1878-21sed23:50:241
308789944825,126cyclictest19291-21cat00:00:230
308809944721,423cyclictest0-21swapper/223:50:092
308839944624,2cyclictest0-21swapper/500:46:145
308839944624,2cyclictest0-21swapper/500:46:145
3088399446109,295cyclictest22040-21mailstats19:45:235
3088299446277,42cyclictest16787-21ssh22:45:124
308799944665,126cyclictest0-21swapper/122:45:101
308799944624,253cyclictest0-21swapper/122:15:221
308799944624,253cyclictest0-21swapper/122:15:221
308859944561,211cyclictest0-21swapper/722:01:037
3088299445235,46cyclictest0-21swapper/422:15:254
3088299445235,46cyclictest0-21swapper/422:15:254
308819944563,84cyclictest121rcu_preempt00:42:533
308859944315,84cyclictest0-21swapper/700:00:327
308819944216,211cyclictest0-21swapper/320:40:153
308799944221,255cyclictest0-21swapper/122:30:041
308789944259,168cyclictest0-21swapper/020:37:050
308789944125,205cyclictest0-21swapper/021:25:320
308789944119,3cyclictest0-21swapper/023:20:220
308819944018,256cyclictest0-21swapper/321:25:303
308809944056,253cyclictest121rcu_preempt23:30:152
308789944017,168cyclictest0-21swapper/000:35:120
3087899440141,126cyclictest121rcu_preempt22:55:150
308839943797,338cyclictest29599-21runrttasks23:10:105
308839943715,338cyclictest0-21swapper/523:50:205
3088299437137,84cyclictest121rcu_preempt22:05:164
3088599436182,84cyclictest721ktimersoftd/723:50:177
308829943614,253cyclictest19923-21rm00:00:294
308819943613,3cyclictest0-21swapper/300:22:263
308839943594,84cyclictest0-21swapper/500:25:395
308809943513,210cyclictest6086-21diskmemload23:20:372
30879994359,255cyclictest121rcu_preempt22:05:181
308799943513,380cyclictest0-21swapper/121:50:101
308789943412,261cyclictest0-21swapper/000:25:060
308799943351,210cyclictest0-21swapper/100:45:511
308799943351,210cyclictest0-21swapper/100:45:511
308829942887,338cyclictest0-21swapper/423:10:144
30881994274,211cyclictest0-21swapper/300:00:053
308859942583,126cyclictest0-21swapper/719:50:267
308849942481,211cyclictest0-21swapper/619:20:106
30879994241,84cyclictest0-21swapper/100:25:151
3088499423124,127cyclictest19138-21ssh23:40:006
308859942282,338cyclictest0-21swapper/723:20:087
3088499422123,127cyclictest31052-21ssh21:25:036
308789942283,84cyclictest9150-21tail22:05:170
308819942180,338cyclictest0-21swapper/320:20:273
3088599420163,84cyclictest121rcu_preempt20:20:337
308849942082,84cyclictest0-21swapper/622:15:136
308849942082,84cyclictest0-21swapper/622:15:136
308839942040,211cyclictest0-21swapper/523:20:185
308809942040,301cyclictest0-21swapper/200:35:172
308799942078,169cyclictest25-21ksoftirqd/123:10:151
308799942041,2cyclictest0-21swapper/123:30:151
308849941981,84cyclictest0-21swapper/600:46:486
308849941981,84cyclictest0-21swapper/600:46:486
308839941982,84cyclictest0-21swapper/519:20:175
308839941778,337cyclictest27342-1kworker/u17:000:42:045
308859941676,338cyclictest0-21swapper/722:05:047
308839941534,85cyclictest0-21swapper/520:40:085
3088199415200,84cyclictest0-21swapper/322:55:153
308809941535,296cyclictest0-21swapper/220:20:222
308789941476,84cyclictest0-21swapper/019:20:150
308839941374,84cyclictest0-21swapper/522:15:235
308839941374,84cyclictest0-21swapper/522:15:235
308839941334,42cyclictest0-21swapper/522:05:175
308809941271,338cyclictest0-21swapper/222:45:042
308809941227,253cyclictest0-21swapper/200:15:142
308859941170,338cyclictest0-21swapper/700:25:037
308849941172,84cyclictest0-21swapper/623:30:156
3087999411112,126cyclictest25-21ksoftirqd/119:20:091
308829941034,295cyclictest2350-21ssh22:55:144
308809941030,84cyclictest19156-21ssh23:40:022
308849940970,84cyclictest0-21swapper/622:05:206
3088499409114,49cyclictest0-21swapper/600:25:046
308849940869,84cyclictest0-21swapper/600:35:226
308859940765,339cyclictest0-21swapper/700:15:057
308829940766,338cyclictest0-21swapper/420:20:274
308859940565,338cyclictest0-21swapper/722:45:137
308819940565,338cyclictest0-21swapper/322:05:043
308829940463,338cyclictest0-21swapper/400:35:224
308789940324,83cyclictest0-21swapper/000:15:060
3087899403148,84cyclictest21582-21df_abs20:40:140
308859940222,261cyclictest0-21swapper/700:35:137
308789940218,127cyclictest9-21ksoftirqd/020:20:270
308809940115,131cyclictest0-21swapper/222:55:132
3088499400105,126cyclictest21655-21/usr/sbin/munin20:40:156
308799939814,127cyclictest9865-21ssh00:15:081
308799939715,85cyclictest0-21swapper/122:55:131
308859939655,338cyclictest15850irq/123-ahci[0023:40:017
308819939655,338cyclictest0-21swapper/322:15:203
308819939655,338cyclictest0-21swapper/322:15:203
308819939615,3cyclictest0-21swapper/323:40:003
308819939555,338cyclictest0-21swapper/319:20:163
308839939111,296cyclictest0-21swapper/522:55:165
3087899387132,253cyclictest0-21swapper/019:50:250
30883993851,253cyclictest0-21swapper/520:04:405
308819938442,254cyclictest0-21swapper/300:15:143
308829937942,3cyclictest0-21swapper/423:39:594
3087899379122,168cyclictest19156-21ssh23:40:000
308839937777,253cyclictest0-21swapper/519:50:095
308849937578,85cyclictest0-21swapper/622:55:126
308829937536,84cyclictest131rcu_sched00:15:154
308809937436,254cyclictest0-21swapper/219:20:152
308829937335,3cyclictest0-21swapper/420:40:054
308859936830,171cyclictest0-21swapper/720:40:057
308799936870,168cyclictest0-21swapper/120:40:111
308859936524,213cyclictest0-21swapper/719:20:177
3088499357104,87cyclictest16761-21ssh22:45:086
30884993475,254cyclictest0-21swapper/619:50:206
30885993412,84cyclictest3104-21ssh22:55:167
308829932632,211cyclictest0-21swapper/420:04:414
308789929340,3cyclictest0-21swapper/019:57:340
308849928834,8cyclictest121rcu_preempt19:57:346
308799928431,85cyclictest0-21swapper/120:04:411
308789928128,172cyclictest0-21swapper/020:04:400
308859928028,171cyclictest1553-21runrttasks20:04:417
308819927621,84cyclictest0-21swapper/320:04:413
308809927114,254cyclictest0-21swapper/220:04:412
3088299270103,86cyclictest0-21swapper/419:57:354
308849926813,127cyclictest0-21swapper/620:04:406
30881992562,42cyclictest0-21swapper/319:57:353
3087999254128,45cyclictest241ktimersoftd/119:57:341
308809925342,129cyclictest0-21swapper/219:57:352
308859925140,129cyclictest0-21swapper/719:57:347
3088399241158,3cyclictest561ktimersoftd/519:57:355
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional