You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 09:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Fri Jun 19, 2026 00:52:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
297799974565,677cyclictest0-21swapper/100:02:501
297799970727,677cyclictest0-21swapper/100:10:451
297799964048,126cyclictest29776-21cyclictest23:51:521
29783996361,382cyclictest0-21swapper/500:31:125
29778996362,84cyclictest0-21swapper/023:41:120
297839962172,127cyclictest0-21swapper/500:01:525
297839961263,465cyclictest0-21swapper/523:41:125
297799961015,592cyclictest0-21swapper/123:01:561
297819960210,383cyclictest23397-21ssh00:31:123
297799959483,84cyclictest0-21swapper/122:51:541
29782995930,84cyclictest0-21swapper/400:02:544
29778995931,3cyclictest0-21swapper/000:31:120
29784995921,382cyclictest0-21swapper/600:31:136
2978499583118,84cyclictest0-21swapper/623:41:126
297799957020,338cyclictest0-21swapper/122:07:561
297859956917,126cyclictest121rcu_preempt00:10:277
297799954438,338cyclictest0-21swapper/100:05:111
2978299538413,41cyclictest481ktimersoftd/422:36:334
2977999537113,126cyclictest0-21swapper/121:56:191
297829953421,296cyclictest0-21swapper/423:52:164
297799953424,507cyclictest0-21swapper/100:22:071
2978199528146,84cyclictest5405-21diskmemload00:01:523
2977899527445,3cyclictest111rcuc/000:04:440
297849952617,126cyclictest0-21swapper/622:07:556
2978299522100,257cyclictest0-21swapper/400:31:124
297799952013,257cyclictest0-21swapper/100:31:131
29784995179,126cyclictest0-21swapper/623:01:366
297799951446,337cyclictest0-21swapper/122:21:461
297799951446,337cyclictest0-21swapper/122:21:461
297859951145,211cyclictest29776-21cyclictest23:02:557
29780995091,262cyclictest0-21swapper/200:31:202
297849950578,423cyclictest0-21swapper/600:03:596
2978099499288,8cyclictest5405-21diskmemload00:22:222
2978199498157,126cyclictest121rcu_preempt23:44:283
297789949829,253cyclictest0-21swapper/000:12:050
297819949430,41cyclictest0-21swapper/300:21:433
2977899493110,253cyclictest2964-21taskset19:15:210
297809949269,254cyclictest0-21swapper/223:02:092
297829949167,82cyclictest0-21swapper/400:12:104
297839948923,91cyclictest121rcu_preempt23:54:275
297789948923,84cyclictest0-21swapper/022:07:080
2977899486152,123cyclictest121rcu_preempt23:02:020
297859948456,211cyclictest0-21swapper/700:32:137
297799948357,338cyclictest0-21swapper/123:41:121
29781994744,339cyclictest0-21swapper/300:10:503
29780994717,126cyclictest0-21swapper/200:12:122
2978399467123,338cyclictest0-21swapper/500:11:015
29780994671,84cyclictest0-21swapper/200:03:502
2978599465123,339cyclictest0-21swapper/700:21:327
2978399465124,338cyclictest0-21swapper/523:02:205
29779994650,42cyclictest0-21swapper/122:36:341
297799946483,126cyclictest0-21swapper/123:11:551
297859946339,84cyclictest121rcu_preempt19:15:217
297849946239,85cyclictest0-21swapper/623:51:326
2978499462207,253cyclictest641ktimersoftd/621:56:176
2978199462122,338cyclictest0-21swapper/322:36:453
2978599460329,85cyclictest0-21swapper/700:05:147
297839946075,211cyclictest6611-21sh00:21:385
2978199460120,338cyclictest0-21swapper/322:21:383
2978199460120,338cyclictest0-21swapper/322:21:383
2977999460120,338cyclictest29776-21cyclictest19:15:221
2978199459117,126cyclictest41-21ksoftirqd/323:11:543
297829945837,2cyclictest0-21swapper/423:02:094
297809945873,211cyclictest33-21ksoftirqd/223:52:242
2978499456116,85cyclictest30738-21diskmemload23:22:076
297789945633,211cyclictest0-21swapper/022:21:470
297789945633,211cyclictest0-21swapper/022:21:460
297839945570,254cyclictest0-21swapper/521:56:195
297849945432,256cyclictest0-21swapper/623:11:546
297829945428,84cyclictest0-21swapper/423:41:154
297789945431,42cyclictest0-21swapper/023:55:100
297859945271,84cyclictest0-21swapper/700:01:527
2978299451111,337cyclictest32710-21munin-node22:07:024
297849944927,2cyclictest0-21swapper/622:37:236
2978099448108,337cyclictest0-21swapper/222:07:002
297819944725,343cyclictest23311-21ssh22:07:573
297789944726,293cyclictest0-21swapper/022:51:590
297809944662,126cyclictest17406-21rm22:21:262
297809944662,126cyclictest17406-21rm22:21:262
297859944561,211cyclictest0-21swapper/723:11:547
297849944522,338cyclictest0-21swapper/622:51:436
297829944522,256cyclictest0-21swapper/400:22:314
2978199444103,339cyclictest0-21swapper/323:54:273
297809944315,84cyclictest0-21swapper/222:36:502
297839944227,245cyclictest0-21swapper/522:51:565
297789944158,210cyclictest0-21swapper/000:22:060
297839943816,334cyclictest0-21swapper/522:07:225
297839943816,256cyclictest12397-21ssh23:11:555
29778994319,41cyclictest0-21swapper/022:36:330
297849942989,338cyclictest29776-21cyclictest00:11:026
29783994261,168cyclictest13331-21seq22:36:475
297849942483,338cyclictest0-21swapper/622:21:426
297849942483,338cyclictest0-21swapper/622:21:426
29778994241,3cyclictest0-21swapper/023:52:420
297849942383,337cyclictest0-21swapper/600:21:166
297819942382,339cyclictest0-21swapper/323:02:013
297859942282,337cyclictest0-21swapper/722:36:457
297849942241,297cyclictest0-21swapper/600:05:146
297809942281,126cyclictest33-21ksoftirqd/223:41:182
297849942181,338cyclictest0-21swapper/619:15:226
297819942181,338cyclictest0-21swapper/319:15:223
297789942181,338cyclictest0-21swapper/023:22:070
297829942082,84cyclictest0-21swapper/422:21:234
297829942082,84cyclictest0-21swapper/422:21:234
297829942040,85cyclictest0-21swapper/423:12:104
297829941980,84cyclictest0-21swapper/419:15:154
297829941977,83cyclictest0-21swapper/400:05:044
2978399417121,83cyclictest0-21swapper/500:05:155
297829941779,84cyclictest0-21swapper/421:56:304
297789941777,169cyclictest0-21swapper/023:12:030
297859941674,211cyclictest73-21ksoftirqd/723:53:077
297819941675,339cyclictest0-21swapper/322:51:563
297819941635,296cyclictest0-21swapper/323:22:043
297859941576,84cyclictest0-21swapper/722:07:067
297819941574,338cyclictest0-21swapper/300:05:143
297789941474,338cyclictest0-21swapper/021:56:310
297859941273,84cyclictest0-21swapper/722:51:567
297819941270,339cyclictest0-21swapper/321:56:183
297789941031,295cyclictest0-21swapper/000:05:120
297859940970,84cyclictest0-21swapper/722:21:317
297859940970,84cyclictest0-21swapper/722:21:307
297809940970,84cyclictest0-21swapper/223:12:012
297859940869,84cyclictest0-21swapper/723:44:287
297839940829,255cyclictest0-21swapper/523:55:105
297829940870,84cyclictest29776-21cyclictest23:22:044
297829940765,253cyclictest0-21swapper/422:51:544
297819940765,126cyclictest0-21swapper/323:55:103
297839940262,338cyclictest0-21swapper/522:22:045
297839940262,338cyclictest0-21swapper/522:22:045
297809939960,84cyclictest0-21swapper/222:51:562
297799939916,168cyclictest0-21swapper/123:22:041
297809939755,213cyclictest0-21swapper/219:15:212
297809939698,126cyclictest121rcu_preempt00:05:132
297799939355,169cyclictest0-21swapper/123:55:091
297859939253,84cyclictest0-21swapper/723:22:077
297809939253,84cyclictest0-21swapper/223:22:082
297829939017,296cyclictest0-21swapper/423:55:064
297809938043,82cyclictest0-21swapper/221:56:312
297859937842,82cyclictest0-21swapper/721:56:317
297839937738,258cyclictest0-21swapper/519:15:135
297849937534,338cyclictest0-21swapper/623:55:086
297839936929,259cyclictest0-21swapper/523:21:495
297859936324,84cyclictest0-21swapper/723:55:077
29780993421,126cyclictest0-21swapper/223:55:082
27957210398,2sleep40-21swapper/419:05:174
27491210196,2sleep20-21swapper/219:05:152
2933428353,27sleep10-21swapper/119:09:471
2927928353,27sleep30-21swapper/319:09:013
2908528353,27sleep60-21swapper/619:06:156
230462830,2sleep52978399cyclictest19:50:115
66202820,0sleep70-21swapper/723:09:487
2923528252,27sleep50-21swapper/519:08:225
2906628252,27sleep70-21swapper/719:06:007
2909727949,27sleep00-21swapper/019:06:250
13772780,0sleep60-21swapper/622:48:256
208942770,3sleep773-21ksoftirqd/721:33:157
119242730,0sleep10-21swapper/122:19:381
10682730,0sleep60-21swapper/600:38:246
105672730,1sleep2321ktimersoftd/221:28:062
105672730,1sleep2321ktimersoftd/221:28:052
222472670,1sleep60-21swapper/621:34:106
2978099641,59cyclictest15826-21ssh22:55:322
29784996128,29cyclictest21537-21ssh22:25:046
2978199601,35cyclictest0-21swapper/322:55:343
2978199582,33cyclictest0-21swapper/300:25:023
2978199572,5cyclictest0-21swapper/322:40:203
2978499561,51cyclictest9685-21apt-get00:25:026
2978499522,46cyclictest4083-21ssh23:25:276
131802520,1sleep00-21swapper/023:30:180
29783995145,3cyclictest14859-21date22:55:235
2978099486,32cyclictest121rcu_preempt00:25:112
2978099485,33cyclictest121rcu_preempt22:10:032
2978099485,33cyclictest121rcu_preempt22:10:032
29785994726,10cyclictest73-21ksoftirqd/700:25:047
29783994413,5cyclictest17089-21sh23:15:135
2978099417,21cyclictest33-21ksoftirqd/200:35:132
2978099411,31cyclictest121rcu_preempt22:00:042
2978399401,34cyclictest17212-21/usr/sbin/munin22:40:175
2978299406,31cyclictest0-21swapper/423:45:184
2978499395,19cyclictest0-21swapper/622:40:206
2978599386,23cyclictest73-21ksoftirqd/723:25:227
2978599382,4cyclictest0-21swapper/700:15:127
2978599382,32cyclictest0-21swapper/723:45:167
29784993830,5cyclictest17801-21irqstats23:15:196
2978299387,4cyclictest481ktimersoftd/400:25:034
2978299382,32cyclictest0-21swapper/423:25:024
2978299382,32cyclictest0-21swapper/423:05:114
2978299381,34cyclictest0-21swapper/422:00:044
2978199382,33cyclictest0-21swapper/323:15:133
2978199381,34cyclictest0-21swapper/323:45:023
29780993832,4cyclictest321ktimersoftd/223:25:012
29780993830,5cyclictest0-21swapper/223:15:192
2977999382,4cyclictest17324-21ssh23:15:161
2977899382,32cyclictest0-21swapper/022:55:150
2978599374,25cyclictest73-21ksoftirqd/722:26:297
2978599373,4cyclictest17071-21ssh22:40:027
2978499372,5cyclictest7775-21sh23:45:136
2978499372,31cyclictest0-21swapper/622:10:216
2978499372,31cyclictest0-21swapper/622:10:216
29783993729,3cyclictest0-21swapper/522:25:035
2978399372,4cyclictest27538-21ssh22:10:225
2978399372,4cyclictest27538-21ssh22:10:215
2978399372,31cyclictest7589-21ssh23:45:035
2978299378,16cyclictest49-21ksoftirqd/422:55:014
29782993730,4cyclictest27108-21diskmemload00:35:054
2978299371,33cyclictest0-21swapper/423:15:184
2978099379,21cyclictest121rcu_preempt00:15:272
2978099372,32cyclictest0-21swapper/223:45:202
2977999372,4cyclictest0-21swapper/123:45:091
29778993732,2cyclictest26067-21ssh22:10:130
29778993732,2cyclictest26067-21ssh22:10:120
29778993730,4cyclictest0-21swapper/023:45:020
2977899372,4cyclictest0-21swapper/023:15:020
2977899372,31cyclictest21543-21ssh22:25:050
2977899372,31cyclictest17514-21tr22:40:120
2978599362,31cyclictest0-21swapper/700:35:067
2978599362,30cyclictest0-21swapper/723:15:197
2978499362,4cyclictest0-21swapper/622:55:036
29782993625,8cyclictest0-21swapper/422:25:104
2978299361,32cyclictest0-21swapper/400:15:054
2978299361,2cyclictest0-21swapper/422:10:024
2978299361,2cyclictest0-21swapper/422:10:024
29780993610,17cyclictest33-21ksoftirqd/222:40:162
2977999363,30cyclictest18272-21grep22:40:161
2977999362,4cyclictest0-21swapper/123:05:051
2978499352,30cyclictest0-21swapper/600:15:026
2978199351,3cyclictest0-21swapper/322:10:223
2978199351,3cyclictest0-21swapper/322:10:223
2978199351,31cyclictest30536-21ssh23:05:063
29779993529,3cyclictest21650-21apt-get22:25:111
2977999352,30cyclictest0-21swapper/100:25:011
2977999351,30cyclictest0-21swapper/122:10:151
2977999351,30cyclictest0-21swapper/122:10:151
29778993527,5cyclictest2001-21ssh23:25:110
29785993427,4cyclictest0-21swapper/722:00:067
2978599341,31cyclictest0-21swapper/722:55:067
2978599341,2cyclictest0-21swapper/722:10:037
2978599341,2cyclictest0-21swapper/722:10:027
2978199341,30cyclictest0-21swapper/322:25:053
2978099349,15cyclictest33-21ksoftirqd/222:25:332
2978099341,3cyclictest30590-21ssh23:05:082
29779993427,4cyclictest0-21swapper/122:55:221
2977999342,29cyclictest0-21swapper/100:15:021
2978499331,2cyclictest0-21swapper/623:05:026
29783993325,5cyclictest3728-21ntpq23:25:225
2978399331,29cyclictest0-21swapper/523:05:025
2978299331,29cyclictest0-21swapper/422:40:024
2978199331,30cyclictest0-21swapper/300:15:033
29778993324,5cyclictest8458-21grep22:00:120
2977899331,29cyclictest0-21swapper/000:35:100
29784993224,5cyclictest9902-21cut22:00:216
29781993224,5cyclictest2285-21tr23:25:123
29783993125,3cyclictest27272-21tr00:15:245
29783993123,5cyclictest9423-21smartctl22:00:175
29783993122,6cyclictest28099-21uniq00:35:175
29781993123,5cyclictest30006-21ssh00:35:433
29778993124,4cyclictest14391-21ssh00:26:470
29778993021,6cyclictest32307-21sh23:05:240
29783992921,6cyclictest10789-21seq00:25:145
29781992920,6cyclictest11045-21sh22:00:423
29779992922,5cyclictest28767-21cat00:35:221
29778992922,4cyclictest29294-21ssh00:16:400
29779992822,3cyclictest3626-21cat23:25:211
29779992820,5cyclictest241ktimersoftd/122:00:351
29778992810,18cyclictest0-21swapper/019:12:270
29785992727,0cyclictest0-21swapper/719:12:277
29784992710,0cyclictest0-21swapper/619:12:206
29783992710,0cyclictest0-21swapper/519:12:315
29782992710,17cyclictest0-21swapper/419:12:244
29781992727,0cyclictest0-21swapper/319:12:213
29780992727,0cyclictest0-21swapper/219:12:432
149962270,0sleep20-21swapper/220:30:142
29779992626,0cyclictest0-21swapper/119:12:211
29782992017,2cyclictest49-21ksoftirqd/419:20:104
2978399190,18cyclictest0-21swapper/521:25:135
2978399190,18cyclictest0-21swapper/521:25:135
29782991917,1cyclictest0-21swapper/420:35:174
2977899195,6cyclictest0-21swapper/021:40:010
2977899173,6cyclictest0-21swapper/022:50:000
2977999160,2cyclictest0-21swapper/121:37:231
29782991510,3cyclictest30449-21cut20:55:174
29781991512,1cyclictest29741-21sh21:53:553
2977899152,7cyclictest0-21swapper/021:00:010
199042150,0sleep20-21swapper/221:16:122
29785991411,2cyclictest22169-21ls20:40:197
29783991412,1cyclictest1802-21ssh22:48:415
29783991411,2cyclictest3512-21ntpq21:40:175
29782991412,1cyclictest14849-21/usr/sbin/munin19:35:174
29781991412,1cyclictest20902-21ssh21:33:163
29780991412,1cyclictest19451-21ssh21:32:182
29779991410,2cyclictest241ktimersoftd/120:30:131
29785991311,1cyclictest2793-21df_abs20:10:127
29785991311,1cyclictest0-21swapper/719:25:137
29785991310,2cyclictest721ktimersoftd/719:45:177
29784991311,1cyclictest88502sleep622:17:356
2978399139,3cyclictest21858-21cut20:40:175
29783991311,1cyclictest32739-21ssh21:39:265
29783991311,1cyclictest29693-21ssh23:39:105
29783991311,1cyclictest15155-21ssh21:14:465
29783991311,1cyclictest0-21swapper/523:32:175
29783991311,1cyclictest0-21swapper/522:20:005
29783991311,1cyclictest0-21swapper/521:49:555
29783991310,2cyclictest561ktimersoftd/520:55:215
2978299139,3cyclictest7056-21sh22:34:164
2978299139,3cyclictest49-21ksoftirqd/421:12:274
2978299139,3cyclictest20908-21grep23:35:004
2978299139,3cyclictest16740-21cat19:40:124
2978299138,4cyclictest6413-21grep21:25:244
2978299138,4cyclictest6413-21grep21:25:244
29782991311,1cyclictest49-21ksoftirqd/422:48:334
2978199139,3cyclictest10472-21rm21:11:393
29781991311,1cyclictest30027-21ssh22:45:563
29781991311,1cyclictest21868-21ssh21:17:303
29781991311,1cyclictest11810-21ssh21:45:073
29781991311,1cyclictest0-21swapper/322:34:443
29781991311,1cyclictest0-21swapper/322:15:223
29781991311,1cyclictest0-21swapper/321:38:103
29781991310,2cyclictest401ktimersoftd/320:55:143
29781991310,2cyclictest401ktimersoftd/320:40:123
2978099139,2cyclictest5283-21grep20:15:112
29780991311,1cyclictest30366-21ssh21:21:242
29780991311,1cyclictest16616-21ssh21:46:522
29780991311,1cyclictest0-21swapper/221:13:572
2977999139,3cyclictest30241-21sed20:55:151
29779991311,1cyclictest20137-21cat20:40:001
29779991311,1cyclictest1056-21timerandwakeup20:05:221
2977899134,8cyclictest0-21swapper/020:15:000
2977899131,6cyclictest0-21swapper/021:25:000
29778991311,1cyclictest24894-21ssh21:50:390
29778991311,1cyclictest10780-21sh21:28:150
29778991311,1cyclictest10780-21sh21:28:140
29778991311,1cyclictest0-21swapper/023:38:060
29778991311,1cyclictest0-21swapper/022:18:580
29778991311,1cyclictest0-21swapper/020:05:010
29778991310,2cyclictest23345-21head19:50:130
2978599129,2cyclictest8160-21ls20:20:107
2978599129,2cyclictest5418-21sh21:41:067
2978599129,2cyclictest4737-21grep21:25:147
2978599129,2cyclictest4737-21grep21:25:147
2978599129,2cyclictest29444-21cut20:55:137
2978599129,2cyclictest21614-21egrep23:35:127
2978599129,2cyclictest19332-21cat20:35:227
2978599129,2cyclictest12958-21chrt19:34:197
2978599129,2cyclictest10800-21ssh21:11:527
29785991210,1cyclictest31540-21ssh21:22:117
29785991210,1cyclictest26686-21cstates20:50:137
29785991210,1cyclictest0-21swapper/723:31:357
29785991210,1cyclictest0-21swapper/722:45:207
29785991210,1cyclictest0-21swapper/722:30:107
29785991210,1cyclictest0-21swapper/722:15:057
29785991210,1cyclictest0-21swapper/721:50:547
29785991210,1cyclictest0-21swapper/721:45:057
29785991210,1cyclictest0-21swapper/721:35:117
29785991210,1cyclictest0-21swapper/721:15:137
29785991210,1cyclictest0-21swapper/721:05:067
29785991210,1cyclictest0-21swapper/720:45:087
29785991210,1cyclictest0-21swapper/720:25:227
29785991210,1cyclictest0-21swapper/720:15:187
29785991210,1cyclictest0-21swapper/720:02:277
29785991210,1cyclictest0-21swapper/719:20:137
2978499129,2cyclictest8305-21sed21:10:236
2978499129,2cyclictest7245-21grep20:15:216
2978499129,2cyclictest641ktimersoftd/620:20:116
2978499129,2cyclictest5026-21sh21:40:496
2978499129,2cyclictest24735-21expr21:35:126
2978499128,3cyclictest948-21sed22:30:226
2978499128,3cyclictest16285-21tr21:15:116
29784991210,1cyclictest315642sleep620:56:386
29784991210,1cyclictest0-21swapper/623:30:116
29784991210,1cyclictest0-21swapper/621:50:046
29784991210,1cyclictest0-21swapper/621:25:106
29784991210,1cyclictest0-21swapper/621:25:096
29784991210,1cyclictest0-21swapper/621:20:206
29784991210,1cyclictest0-21swapper/621:05:196
29784991210,1cyclictest0-21swapper/621:00:136
29784991210,1cyclictest0-21swapper/620:50:146
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional