You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-12 - 07:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Thu Mar 12, 2026 00:53:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
221189963140,423cyclictest0-21swapper/400:20:044
221149959684,296cyclictest9-21ksoftirqd/023:46:440
221159959482,126cyclictest121rcu_preempt00:16:541
2211999579237,167cyclictest121rcu_preempt23:46:435
221219957822,469cyclictest0-21swapper/722:10:247
221199957622,423cyclictest0-21swapper/523:17:505
221179954837,508cyclictest0-21swapper/300:43:273
221219953217,508cyclictest4951-21cron00:20:007
221209953061,254cyclictest121rcu_preempt23:17:376
221209952417,343cyclictest0-21swapper/622:59:336
221199951850,465cyclictest4949-21cron00:20:005
22114995121,126cyclictest0-21swapper/023:00:020
22116995090,84cyclictest0-21swapper/221:49:202
221169950637,344cyclictest0-21swapper/222:59:532
2211599504124,338cyclictest121rcu_preempt21:48:421
2211799502159,127cyclictest41-21ksoftirqd/321:14:553
2211899501417,3cyclictest471rcuc/422:11:384
221199950032,42cyclictest0-21swapper/521:45:325
221149949934,126cyclictest0-21swapper/021:46:060
221149949834,298cyclictest0-21swapper/000:43:140
221219949732,219cyclictest0-21swapper/721:49:317
2211999497116,84cyclictest8497-21ssh00:03:335
2211599496157,84cyclictest121rcu_preempt23:01:551
221199949531,380cyclictest0-21swapper/521:15:055
2211899495112,253cyclictest0-21swapper/400:01:304
221159949530,42cyclictest0-21swapper/100:41:171
2212099493449,42cyclictest641ktimersoftd/600:40:166
2212099491111,84cyclictest631-21ptp4l23:47:116
221199949024,85cyclictest0-21swapper/500:44:385
2211499487146,338cyclictest21445-21kworker/u16:023:00:340
221179948659,211cyclictest1556-21runrttasks23:46:433
221169948562,84cyclictest0-21swapper/200:02:122
221189948462,256cyclictest6341-21ssh21:45:324
221189948054,423cyclictest0-21swapper/421:44:284
221159947813,3cyclictest6551-21sh21:14:071
221169947713,301cyclictest0-21swapper/223:18:392
2211499475134,338cyclictest623-21nscd00:02:140
22114994739,83cyclictest121rcu_preempt23:33:000
2211499471388,3cyclictest101ktimersoftd/021:44:410
22117994701,254cyclictest0-21swapper/322:12:243
22120994691,465cyclictest0-21swapper/600:18:566
22116994681,211cyclictest0-21swapper/222:10:222
22114994681,84cyclictest121rcu_preempt22:51:430
221169946741,253cyclictest0-21swapper/223:01:142
22118994661,254cyclictest0-21swapper/422:59:324
2212099465124,338cyclictest0-21swapper/600:00:266
221189946581,211cyclictest0-21swapper/400:42:004
2211699465123,127cyclictest33-21ksoftirqd/221:15:122
2211599465123,84cyclictest121rcu_preempt00:01:471
2211499464124,338cyclictest0-21swapper/022:11:300
221179946341,256cyclictest0-21swapper/321:48:433
2211499463121,211cyclictest0-21swapper/000:17:040
2211899462121,338cyclictest0-21swapper/423:00:404
221179946240,256cyclictest0-21swapper/322:58:443
221169946039,7cyclictest0-21swapper/200:42:172
221209945936,256cyclictest0-21swapper/623:02:216
2212099459118,339cyclictest0-21swapper/623:33:026
2211999459119,337cyclictest0-21swapper/523:01:465
2211599459120,125cyclictest0-21swapper/123:19:331
221219945836,256cyclictest0-21swapper/700:40:067
2212199458117,338cyclictest8182-21cpu21:15:187
221209945836,255cyclictest0-21swapper/622:11:476
221149945878,214cyclictest21466-21diskmemload23:19:480
2211699457117,337cyclictest25117-21munin-node23:33:022
221169945634,2cyclictest0-21swapper/200:16:282
2211499456115,338cyclictest0-21swapper/021:14:520
221179945571,211cyclictest0-21swapper/323:19:273
221179945533,256cyclictest0-21swapper/323:01:053
221159945531,123cyclictest25-21ksoftirqd/121:43:471
221219945433,255cyclictest0-21swapper/700:00:467
221199945432,302cyclictest0-21swapper/522:12:465
221179945469,211cyclictest0-21swapper/300:15:383
221209945331,255cyclictest0-21swapper/621:49:396
221219945273,84cyclictest0-21swapper/723:00:127
221209945128,256cyclictest0-21swapper/621:15:026
221169945129,255cyclictest0-21swapper/223:46:232
2211999450109,84cyclictest0-21swapper/522:59:085
221179945028,256cyclictest0-21swapper/321:15:133
221219944965,126cyclictest0-21swapper/722:59:487
2211699448107,338cyclictest0-21swapper/200:29:402
221149944826,212cyclictest0-21swapper/000:20:190
221159944725,85cyclictest0-21swapper/122:10:511
221199944320,210cyclictest0-21swapper/521:44:455
221189944013,296cyclictest0-21swapper/423:19:034
221169944016,2cyclictest0-21swapper/200:20:152
221219943810,296cyclictest0-21swapper/723:47:367
221209943816,3cyclictest0-21swapper/621:14:086
221169943811,295cyclictest6940-21ssh21:14:272
221149943818,256cyclictest0-21swapper/021:15:020
221189943612,126cyclictest0-21swapper/421:14:584
221219943513,338cyclictest0-21swapper/700:20:137
221179943251,296cyclictest0-21swapper/300:20:203
2211999428260,3cyclictest561ktimersoftd/523:31:415
221219942647,295cyclictest16250irq/123-ahci[0000:39:547
22115994251,84cyclictest0-21swapper/100:29:571
22121994241,257cyclictest0-21swapper/723:18:057
221179942483,338cyclictest0-21swapper/300:03:313
2211799423128,211cyclictest11361-21cat00:39:553
221219942241,296cyclictest0-21swapper/721:44:407
22118994221,255cyclictest0-21swapper/421:15:144
221179942281,338cyclictest0-21swapper/300:30:043
221219942180,338cyclictest0-21swapper/723:31:407
221169942141,296cyclictest0-21swapper/221:44:332
221199942082,84cyclictest0-21swapper/521:14:105
221159942080,84cyclictest0-21swapper/122:59:431
221199941939,295cyclictest0-21swapper/500:30:045
221189941978,338cyclictest0-21swapper/400:29:404
221219941777,338cyclictest0-21swapper/721:14:187
221199941736,296cyclictest0-21swapper/500:20:145
221209941675,338cyclictest0-21swapper/600:20:216
221209941636,296cyclictest0-21swapper/600:29:406
221189941675,338cyclictest0-21swapper/400:30:044
221199941535,295cyclictest0-21swapper/500:29:405
221159941472,175cyclictest121rcu_preempt23:32:291
221169941170,338cyclictest0-21swapper/200:30:052
221219941030,296cyclictest0-21swapper/700:29:407
221219940969,338cyclictest0-21swapper/700:30:047
221189940867,338cyclictest0-21swapper/400:20:094
221209940726,260cyclictest0-21swapper/600:30:066
221189940768,337cyclictest0-21swapper/423:46:444
221159940669,82cyclictest0-21swapper/100:20:091
221189940526,256cyclictest15047-21ssh23:31:584
221159940424,85cyclictest0-21swapper/123:47:371
2212099403107,124cyclictest0-21swapper/621:44:216
221179940361,339cyclictest0-21swapper/321:44:283
221159940366,171cyclictest0-21swapper/121:15:121
221149939919,126cyclictest0-21swapper/000:29:560
221179939554,339cyclictest0-21swapper/323:32:573
2211899394100,84cyclictest0-21swapper/422:52:034
22115993845,127cyclictest0-21swapper/122:52:031
2211699381125,254cyclictest0-21swapper/222:52:032
221179937941,254cyclictest0-21swapper/300:29:403
221179937437,254cyclictest0-21swapper/322:51:433
221149937373,84cyclictest121rcu_preempt00:30:050
221209937235,255cyclictest0-21swapper/621:58:356
221209937235,255cyclictest0-21swapper/621:58:356
221209937134,254cyclictest0-21swapper/622:51:436
221219936626,296cyclictest0-21swapper/722:51:437
221159936267,211cyclictest0-21swapper/100:39:551
221159936225,126cyclictest0-21swapper/100:30:041
221199935821,254cyclictest0-21swapper/500:39:555
221169935518,253cyclictest0-21swapper/221:58:322
221169935518,253cyclictest0-21swapper/221:58:322
2211499352308,42cyclictest101ktimersoftd/000:39:550
22119993509,338cyclictest0-21swapper/522:52:025
221199934041,296cyclictest0-21swapper/521:58:335
221199934041,296cyclictest0-21swapper/521:58:325
2211599340168,83cyclictest25-21ksoftirqd/121:58:341
2211599340168,83cyclictest25-21ksoftirqd/121:58:341
22114993392,7cyclictest121rcu_preempt21:58:360
22114993392,7cyclictest121rcu_preempt21:58:350
221209933841,212cyclictest0-21swapper/600:39:556
221219933338,213cyclictest0-21swapper/721:58:357
221219933338,213cyclictest0-21swapper/721:58:357
221189932831,253cyclictest0-21swapper/421:58:324
221189932831,253cyclictest0-21swapper/421:58:324
221169932729,295cyclictest0-21swapper/200:39:552
221179931720,42cyclictest0-21swapper/321:58:333
221179931720,42cyclictest0-21swapper/321:58:333
22118992961,211cyclictest0-21swapper/400:39:544
2167828454,27sleep30-21swapper/319:09:543
2148428352,27sleep50-21swapper/519:07:085
2166728249,29sleep10-21swapper/119:09:441
2161028252,27sleep00-21swapper/019:08:550
260592810,1sleep70-21swapper/723:38:277
2164528151,27sleep60-21swapper/619:09:256
2163028148,29sleep20-21swapper/219:09:142
2153328151,27sleep70-21swapper/719:07:507
2139027947,28sleep40-21swapper/419:05:464
315612750,1sleep40-21swapper/422:16:434
24632750,0sleep40-21swapper/421:27:584
167082740,0sleep40-21swapper/421:35:114
38962650,3sleep665-21ksoftirqd/619:30:146
22116996129,29cyclictest31187-21ssh22:32:542
2212199561,51cyclictest31242-21sh22:32:577
2211599556,37cyclictest25-21ksoftirqd/122:33:021
22114995113,18cyclictest0-21swapper/022:35:000
22121994518,23cyclictest16537-21basename23:50:197
22121994221,18cyclictest26893-21ssh23:05:137
22121994221,18cyclictest26893-21ssh23:05:137
22121994011,25cyclictest27500-21cut22:15:127
2212099382,32cyclictest0-21swapper/622:32:546
22117993729,5cyclictest31203-21ssh22:32:543
2211599379,20cyclictest25-21ksoftirqd/100:05:301
2211999362,30cyclictest0-21swapper/522:32:545
22119993526,5cyclictest27192-21chrt22:15:085
2211499351,30cyclictest12278-21ntpq00:05:220
22121993410,23cyclictest0-21swapper/721:32:157
22121993410,23cyclictest0-21swapper/719:50:187
2211999342,29cyclictest0-21swapper/500:05:025
2211899341,2cyclictest0-21swapper/422:33:254
22117993425,5cyclictest965-21wc22:00:283
2212199335,24cyclictest646-21nscd00:05:307
22121993326,6cyclictest0-21swapper/722:42:567
22121993310,22cyclictest0-21swapper/722:48:557
22121993310,22cyclictest0-21swapper/722:23:007
22121993310,22cyclictest0-21swapper/721:38:217
22121993310,22cyclictest0-21swapper/721:26:477
22121993310,22cyclictest0-21swapper/720:27:407
22121993310,22cyclictest0-21swapper/720:13:597
22119993324,5cyclictest20680-21ls23:35:215
2211799331,2cyclictest0-21swapper/323:50:113
22116993324,6cyclictest738-21tr22:00:252
22121993226,5cyclictest0-21swapper/720:37:517
22121993224,5cyclictest1607-21ssh22:00:387
22120993224,5cyclictest13323-21ssh00:05:426
22119993223,6cyclictest31452-21cut22:00:135
2211999321,28cyclictest0-21swapper/523:50:105
22116993224,5cyclictest15305-21chrt23:50:112
2211599327,8cyclictest0-21swapper/122:15:241
22114993221,8cyclictest16313-21ssh23:50:160
22121993127,4cyclictest0-21swapper/719:41:247
22121993110,3cyclictest0-21swapper/719:23:287
22121993110,20cyclictest0-21swapper/719:32:327
22120993124,5cyclictest21811-21ssh23:35:346
22120993123,5cyclictest917-21ssh22:00:266
22118993125,3cyclictest15113-21ssh23:50:014
22118993123,5cyclictest1837-21ssh22:00:504
2211699311,27cyclictest0-21swapper/200:05:022
2211599319,15cyclictest25-21ksoftirqd/123:05:131
2211599319,15cyclictest25-21ksoftirqd/123:05:131
22115993112,12cyclictest0-21swapper/123:50:081
22121993026,4cyclictest0-21swapper/720:42:307
22121993025,5cyclictest0-21swapper/720:47:187
22121993025,5cyclictest0-21swapper/719:36:547
22121993010,3cyclictest0-21swapper/720:59:067
22121993010,19cyclictest0-21swapper/721:04:157
22120993025,3cyclictest0-21swapper/623:50:116
22120993021,6cyclictest27534-21tr22:15:126
22118993022,5cyclictest26612-21cstates23:05:124
22118993022,5cyclictest26612-21cstates23:05:124
22118993022,5cyclictest2456-21sh22:35:154
22118993022,5cyclictest20240-21tail23:35:184
22117993022,5cyclictest28348-21tail22:15:173
22117993022,5cyclictest26898-21expr23:05:133
22117993022,5cyclictest26898-21expr23:05:133
2211599309,13cyclictest25-21ksoftirqd/122:00:041
2211599307,17cyclictest25-21ksoftirqd/123:35:251
22115993023,4cyclictest1512-21ssh22:35:041
22114993023,4cyclictest2092-21ssh22:01:040
22114993023,4cyclictest1609-21ssh22:35:090
22120992922,4cyclictest27064-21sed23:05:156
22120992922,4cyclictest27064-21sed23:05:146
22119992922,4cyclictest2775-21ssh22:35:165
22119992922,4cyclictest26272-21rm23:05:085
22119992922,4cyclictest26272-21rm23:05:085
22119992910,18cyclictest0-21swapper/522:48:585
22117992922,4cyclictest12523-21taskset00:05:243
22117992910,19cyclictest0-21swapper/323:29:003
22117992910,18cyclictest0-21swapper/323:42:553
22117992910,0cyclictest0-21swapper/322:49:013
22116992921,5cyclictest19755-21cat23:35:142
22116992921,5cyclictest1517-21sh22:35:042
22114992927,1cyclictest0-21swapper/021:31:400
22114992921,5cyclictest28972-21tr22:15:220
22114992910,18cyclictest0-21swapper/022:49:130
22114992910,18cyclictest0-21swapper/021:09:510
22114992910,18cyclictest0-21swapper/020:52:360
2212199287,17cyclictest1885-21cpuspeed_turbos22:35:137
22121992828,0cyclictest0-21swapper/723:28:597
22121992825,3cyclictest0-21swapper/720:22:467
22121992810,1cyclictest0-21swapper/719:45:467
22121992810,18cyclictest0-21swapper/723:42:387
22121992810,18cyclictest0-21swapper/723:14:037
22121992810,18cyclictest0-21swapper/720:07:527
22121992810,18cyclictest0-21swapper/720:07:527
22121992810,18cyclictest0-21swapper/719:15:137
22121992810,0cyclictest0-21swapper/720:53:117
22121992810,0cyclictest0-21swapper/720:03:337
22120992827,1cyclictest0-21swapper/621:26:316
22120992820,5cyclictest2845-21rm22:35:176
22120992810,0cyclictest0-21swapper/623:28:566
22120992810,0cyclictest0-21swapper/622:23:406
22120992810,0cyclictest0-21swapper/620:52:546
22119992810,0cyclictest0-21swapper/523:42:385
22119992810,0cyclictest0-21swapper/523:29:045
22118992827,1cyclictest0-21swapper/422:42:464
22118992820,5cyclictest11992-21cat00:05:184
22118992810,0cyclictest0-21swapper/423:29:274
22117992828,0cyclictest0-21swapper/321:38:263
22117992822,4cyclictest20522-21cut23:35:203
22117992821,4cyclictest1658-21ssh22:35:113
22117992810,18cyclictest0-21swapper/323:13:433
22117992810,18cyclictest0-21swapper/322:23:153
22117992810,18cyclictest0-21swapper/321:32:023
22117992810,18cyclictest0-21swapper/321:26:093
22117992810,18cyclictest0-21swapper/320:59:113
22117992810,18cyclictest0-21swapper/320:42:013
22117992810,18cyclictest0-21swapper/319:46:123
22117992810,17cyclictest0-21swapper/320:18:183
22117992810,17cyclictest0-21swapper/319:23:083
22117992810,17cyclictest0-21swapper/319:15:103
22117992810,0cyclictest0-21swapper/322:43:013
22117992810,0cyclictest0-21swapper/320:38:113
22117992810,0cyclictest0-21swapper/320:22:533
22117992810,0cyclictest0-21swapper/320:03:073
22117992810,0cyclictest0-21swapper/319:36:503
22117992810,0cyclictest0-21swapper/319:31:543
22116992810,18cyclictest0-21swapper/223:42:392
22116992810,18cyclictest0-21swapper/222:42:532
22116992810,0cyclictest0-21swapper/223:29:222
22116992810,0cyclictest0-21swapper/221:32:232
22116992810,0cyclictest0-21swapper/221:26:202
22115992810,18cyclictest0-21swapper/122:48:521
22114992827,1cyclictest0-21swapper/023:13:320
22114992827,1cyclictest0-21swapper/021:04:450
22114992827,1cyclictest0-21swapper/019:50:200
22114992826,1cyclictest0-21swapper/021:38:500
22114992810,18cyclictest0-21swapper/020:42:330
22114992810,17cyclictest0-21swapper/023:28:550
22114992810,17cyclictest0-21swapper/019:15:010
22121992710,17cyclictest0-21swapper/720:32:057
22121992710,17cyclictest0-21swapper/720:18:177
22121992710,0cyclictest0-21swapper/719:27:357
22120992727,0cyclictest0-21swapper/621:04:166
22120992727,0cyclictest0-21swapper/620:07:476
22120992727,0cyclictest0-21swapper/620:07:466
22120992727,0cyclictest0-21swapper/620:03:036
22120992726,0cyclictest0-21swapper/622:42:316
22120992710,17cyclictest0-21swapper/623:42:366
22120992710,17cyclictest0-21swapper/623:13:366
22120992710,17cyclictest0-21swapper/621:37:576
22120992710,17cyclictest0-21swapper/621:09:566
22120992710,17cyclictest0-21swapper/620:46:566
22120992710,17cyclictest0-21swapper/620:27:436
22120992710,17cyclictest0-21swapper/620:22:286
22120992710,17cyclictest0-21swapper/620:18:186
22120992710,17cyclictest0-21swapper/619:50:196
22120992710,17cyclictest0-21swapper/619:45:396
22120992710,17cyclictest0-21swapper/619:41:176
22120992710,17cyclictest0-21swapper/619:27:406
22120992710,17cyclictest0-21swapper/619:23:096
22120992710,17cyclictest0-21swapper/619:15:126
22120992710,0cyclictest0-21swapper/622:48:456
22120992710,0cyclictest0-21swapper/621:31:366
22120992710,0cyclictest0-21swapper/620:59:116
22120992710,0cyclictest0-21swapper/620:42:006
22120992710,0cyclictest0-21swapper/620:37:546
22120992710,0cyclictest0-21swapper/620:32:026
22120992710,0cyclictest0-21swapper/620:13:206
22120992710,0cyclictest0-21swapper/619:36:136
22119992727,0cyclictest0-21swapper/522:42:485
22119992727,0cyclictest0-21swapper/521:09:535
22119992727,0cyclictest0-21swapper/519:50:135
22119992727,0cyclictest0-21swapper/519:23:075
22119992727,0cyclictest0-21swapper/519:15:145
22119992710,17cyclictest0-21swapper/523:13:515
22119992710,17cyclictest0-21swapper/522:23:125
22119992710,17cyclictest0-21swapper/521:37:565
22119992710,17cyclictest0-21swapper/521:31:285
22119992710,17cyclictest0-21swapper/521:25:575
22119992710,17cyclictest0-21swapper/520:42:135
22119992710,17cyclictest0-21swapper/520:31:595
22119992710,17cyclictest0-21swapper/520:27:195
22119992710,17cyclictest0-21swapper/520:22:325
22119992710,17cyclictest0-21swapper/520:18:205
22119992710,17cyclictest0-21swapper/520:13:265
22119992710,17cyclictest0-21swapper/520:03:025
22119992710,17cyclictest0-21swapper/519:49:555
22119992710,17cyclictest0-21swapper/519:36:215
22119992710,17cyclictest0-21swapper/519:32:215
22119992710,17cyclictest0-21swapper/519:27:385
22119992710,0cyclictest0-21swapper/521:04:185
22119992710,0cyclictest0-21swapper/520:59:335
22119992710,0cyclictest0-21swapper/520:47:055
22119992710,0cyclictest0-21swapper/520:37:575
22119992710,0cyclictest0-21swapper/520:07:575
22119992710,0cyclictest0-21swapper/520:07:575
22118992727,0cyclictest0-21swapper/423:13:344
22118992727,0cyclictest0-21swapper/422:23:224
22118992727,0cyclictest0-21swapper/420:27:554
22118992727,0cyclictest0-21swapper/419:50:204
22118992727,0cyclictest0-21swapper/419:23:114
22118992726,0cyclictest0-21swapper/419:45:224
22118992710,17cyclictest0-21swapper/423:42:424
22118992710,17cyclictest0-21swapper/421:31:484
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional