You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-12 - 10:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot2.osadl.org (updated Thu Sep 12, 2024 00:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
356943225519,31sleep70-21swapper/718:56:047
356954623611,7sleep40-21swapper/418:57:374
3569980993533,2cyclictest3815981-21kworker/u16:1+flush-8:021:39:365
3569980993432,2cyclictest4166922-21kworker/u16:3+flush-8:000:04:485
3569980993431,3cyclictest3550379-21kworker/u16:1+events_unbound19:19:335
3569980993230,2cyclictest4001861-21kworker/u16:4+flush-8:022:49:365
356948223221,7sleep60-21swapper/618:56:436
3569980993129,2cyclictest4071428-21kworker/u16:0+events_unbound23:54:465
3569980993129,2cyclictest3680586-21kworker/u16:2+flush-8:020:14:165
3569980993028,2cyclictest4099024-21kworker/u16:2+flush-8:023:39:415
3569980993028,2cyclictest3815981-21kworker/u16:1+flush-8:021:09:435
3569980993028,2cyclictest3815981-21kworker/u16:1+flush-8:021:09:435
3569980993028,2cyclictest3787898-21kworker/u16:1+flush-8:020:49:385
3569980993028,2cyclictest3757356-21kworker/u16:5+flush-8:020:59:475
3569980993028,2cyclictest3660765-21kworker/u16:0+flush-8:020:44:445
356950822919,7sleep50-21swapper/518:57:065
3569980992826,2cyclictest4099024-21kworker/u16:2+flush-8:023:29:445
3569984992723,3cyclictest3865312-21latency_hist21:29:216
3569980992725,2cyclictest4071428-21kworker/u16:0+flush-8:000:14:475
3569980992725,2cyclictest4049148-21kworker/u16:3+flush-8:023:29:155
3569980992725,2cyclictest3962995-21kworker/u16:2+events_unbound22:19:425
3569980992725,2cyclictest3894898-21kworker/u16:4+flush-8:021:54:165
3569980992725,2cyclictest3660765-21kworker/u16:0+flush-8:021:09:205
3569980992725,2cyclictest3660765-21kworker/u16:0+flush-8:020:04:485
3569980992624,2cyclictest4138626-21kworker/u16:1+flush-8:023:49:465
3569980992623,2cyclictest3836568-21kworker/u16:2+flush-8:021:36:195
3569980992523,2cyclictest4001861-21kworker/u16:4+flush-8:023:14:415
3569980992523,2cyclictest3914349-21kworker/u16:0+events_unbound22:09:385
3569980992523,2cyclictest3815981-21kworker/u16:1+flush-8:022:04:315
3569980992523,2cyclictest3798838-21kworker/u16:2+flush-8:020:54:415
3569980992523,2cyclictest3660765-21kworker/u16:0+flush-8:020:19:465
3569980992523,2cyclictest3550379-21kworker/u16:1+flush-8:019:54:425
3569980992523,2cyclictest3530997-21kworker/u16:3+flush-8:019:39:435
3569978992523,2cyclictest4127538-21kworker/u16:5+flush-8:023:54:404
3569978992523,2cyclictest3739238-21kworker/u16:3+flush-8:020:29:464
3569980992422,2cyclictest4001861-21kworker/u16:4+events_unbound22:39:345
3569980992422,2cyclictest3757356-21kworker/u16:5+flush-8:020:34:365
356998799239,5cyclictest0-21swapper/721:18:327
3569980992321,2cyclictest4071428-21kworker/u16:0+events_unbound00:09:305
3569980992321,2cyclictest4049148-21kworker/u16:3+flush-8:023:44:415
3569980992321,2cyclictest3952616-21kworker/u16:3+flush-8:022:29:155
3569980992321,2cyclictest3815981-21kworker/u16:1+flush-8:021:54:425
3569980992320,2cyclictest3815981-21kworker/u16:1+events_unbound21:44:455
3569980992315,8cyclictest0-21swapper/521:16:085
3569978992320,2cyclictest4049148-21kworker/u16:3+flush-8:023:44:204
3569978992315,7cyclictest0-21swapper/421:41:594
3569980992220,2cyclictest4176945-21kworker/u16:2+events_unbound00:19:455
3569980992220,2cyclictest3952616-21kworker/u16:3+flush-8:022:14:455
3569980992220,2cyclictest3827201-21kworker/u16:2+flush-8:019:11:305
3569980992220,2cyclictest3815981-21kworker/u16:1+events_unbound21:59:485
3569980992119,2cyclictest3962995-21kworker/u16:2+events_unbound22:29:425
3569980992119,2cyclictest3739230-21kworker/u16:2+flush-8:020:29:335
3569980992119,2cyclictest3530997-21kworker/u16:3+flush-8:019:24:365
3569978992118,2cyclictest4049148-21kworker/u16:3+flush-8:023:24:434
3569987992018,1cyclictest0-21swapper/721:36:217
356998799200,10cyclictest0-21swapper/720:15:487
3569980992018,2cyclictest4049148-21kworker/u16:3+events_unbound23:19:345
3569980992018,2cyclictest4049148-21kworker/u16:3+events_unbound23:19:345
3569980992018,2cyclictest3670531-21kworker/u16:4+flush-8:019:50:315
3569980992018,2cyclictest3550379-21kworker/u16:1+flush-8:019:34:435
3569978992018,2cyclictest3660765-21kworker/u16:0+flush-8:020:24:204
3569987991917,1cyclictest0-21swapper/720:28:127
3569980991917,2cyclictest3739230-21kworker/u16:2+flush-8:020:24:385
3569980991917,2cyclictest3550379-21kworker/u16:1+flush-8:019:29:455
3569980991917,2cyclictest3550379-21kworker/u16:1+flush-8:018:59:425
3569980991917,2cyclictest3530997-21kworker/u16:3+flush-8:019:16:315
356998799184,6cyclictest0-21swapper/723:38:477
356998799180,2cyclictest0-21swapper/720:09:457
3569984991816,2cyclictest3827201-21kworker/u16:2+flush-8:019:09:476
3569980991816,2cyclictest3952616-21kworker/u16:3+flush-8:022:34:495
3569980991816,2cyclictest3836568-21kworker/u16:2+events_unbound21:33:295
3569980991816,2cyclictest3530997-21kworker/u16:3+flush-8:019:44:435
3569978991816,2cyclictest3660765-21kworker/u16:0+flush-8:020:24:484
3569978991816,2cyclictest3660765-21kworker/u16:0+events_unbound20:14:354
3569978991815,2cyclictest3914349-21kworker/u16:0+flush-8:022:09:334
356998799174,6cyclictest0-21swapper/721:55:087
356998799174,6cyclictest0-21swapper/718:59:557
356998799174,5cyclictest0-21swapper/723:51:427
356998799174,5cyclictest0-21swapper/722:07:037
356998499170,16cyclictest3639922-21kthreadcore19:29:426
356998499170,14cyclictest0-21swapper/622:39:406
3569980991715,2cyclictest4071428-21kworker/u16:0+events_unbound23:59:325
3569980991715,2cyclictest4001767-21kworker/u16:1+events_unbound22:44:295
3569980991715,2cyclictest3962995-21kworker/u16:2+events_unbound22:54:435
356998099170,3cyclictest3590354-21kthreadcore19:04:395
3569978991715,2cyclictest3660765-21kworker/u16:0+flush-8:020:44:404
3569978991714,2cyclictest3815981-21kworker/u16:1+flush-8:021:34:314
3569978991713,3cyclictest3757356-21kworker/u16:5+flush-8:021:19:474
356998799167,8cyclictest0-21swapper/721:00:437
356998799162,6cyclictest0-21swapper/720:24:007
356998499168,4cyclictest3604006-21cat19:14:366
3569984991612,4cyclictest0-21swapper/600:28:436
3569984991612,3cyclictest4050131-21cat23:04:206
3569984991611,4cyclictest3738846-21latency_hist20:24:206
356998499160,13cyclictest0-21swapper/622:29:446
3569980991614,2cyclictest3962995-21kworker/u16:2+flush-8:023:00:255
356997899162,14cyclictest0-21swapper/423:46:414
3569978991614,2cyclictest4138626-21kworker/u16:1+events_unbound23:59:454
3569978991614,2cyclictest4049148-21kworker/u16:3+events_unbound23:18:294
3569978991614,2cyclictest3827201-21kworker/u16:2+events_unbound18:59:394
3569978991614,2cyclictest3660765-21kworker/u16:0+flush-8:020:59:454
3569978991614,2cyclictest3660765-21kworker/u16:0+events_unbound20:49:344
3569978991614,2cyclictest3660765-21kworker/u16:0+events_unbound20:36:294
3569978991614,2cyclictest3550379-21kworker/u16:1+flush-8:019:04:504
3569978991613,2cyclictest4071428-21kworker/u16:0+flush-8:000:24:334
356998799156,8cyclictest0-21swapper/720:01:157
356998799155,9cyclictest0-21swapper/719:30:327
356998799152,6cyclictest0-21swapper/721:27:567
356998799151,6cyclictest0-21swapper/723:16:057
356998799151,6cyclictest0-21swapper/721:41:387
3569984991513,2cyclictest3914349-21kworker/u16:0+flush-8:022:09:466
3569984991510,4cyclictest4148677-21cat23:54:346
356998499150,14cyclictest0-21swapper/619:54:366
3569980991513,2cyclictest3815981-21kworker/u16:1+events_unbound21:24:455
3569980991513,2cyclictest3550379-21kworker/u16:1+events_unbound20:00:255
3569978991513,2cyclictest3670531-21kworker/u16:4+events_unbound20:09:344
3569978991513,2cyclictest3550379-21kworker/u16:1+flush-8:019:19:354
3569978991512,2cyclictest4138626-21kworker/u16:1+flush-8:000:04:364
356997899150,1cyclictest0-21swapper/419:39:384
356998799141,6cyclictest0-21swapper/722:29:217
3569987991413,0cyclictest0-21swapper/723:13:467
3569987991413,0cyclictest0-21swapper/720:39:457
3569987991411,2cyclictest0-21swapper/722:36:257
356998799140,6cyclictest0-21swapper/723:19:477
356998799140,6cyclictest0-21swapper/723:19:477
356998799140,1cyclictest0-21swapper/700:04:027
356998499140,13cyclictest0-21swapper/622:04:476
356998499140,13cyclictest0-21swapper/619:44:316
356998499140,12cyclictest0-21swapper/622:36:506
3569980991412,2cyclictest4176945-21kworker/u16:2+flush-8:000:27:175
3569980991412,2cyclictest3815981-21kworker/u16:1+events_unbound21:19:305
3569980991412,2cyclictest3660765-21kworker/u16:0+flush-8:020:14:365
356998099140,14cyclictest0-21swapper/520:39:335
356997899149,4cyclictest4186528-21latency_hist00:14:214
3569978991412,2cyclictest3815981-21kworker/u16:1+events_unbound21:14:354
3569978991412,2cyclictest3798838-21kworker/u16:2+flush-8:020:54:384
3569978991412,2cyclictest3550379-21kworker/u16:1+flush-8:019:54:444
3569978991411,2cyclictest3952616-21kworker/u16:3+events_unbound22:14:354
356997899140,14cyclictest0-21swapper/422:44:344
356998799134,8cyclictest0-21swapper/721:19:417
356998799130,6cyclictest0-21swapper/722:10:397
356998799130,6cyclictest0-21swapper/720:46:347
356998799130,12cyclictest0-21swapper/722:25:397
356998499139,3cyclictest3962995-21kworker/u16:2+flush-8:023:04:356
356998499136,7cyclictest0-21swapper/621:51:596
3569984991312,1cyclictest0-21swapper/623:39:166
3569984991312,1cyclictest0-21swapper/621:36:176
3569984991312,1cyclictest0-21swapper/620:37:206
356998499130,12cyclictest0-21swapper/620:29:446
3569980991311,2cyclictest4001767-21kworker/u16:1+events_unbound23:04:235
356997899139,3cyclictest3660765-21kworker/u16:0+flush-8:019:59:414
3569978991311,2cyclictest4071428-21kworker/u16:0+events_unbound23:34:354
3569978991311,2cyclictest3952616-21kworker/u16:3+flush-8:022:29:364
3569978991311,2cyclictest3660765-21kworker/u16:0+events_unbound20:39:404
3569978991310,2cyclictest3660765-21kworker/u16:0+events_unbound19:44:394
356997899130,1cyclictest829-21avahi-daemon23:06:294
356998799129,2cyclictest0-21swapper/723:58:097
356998799129,2cyclictest0-21swapper/719:13:377
356998799124,7cyclictest0-21swapper/723:32:387
356998799124,7cyclictest0-21swapper/720:34:307
356998799123,8cyclictest0-21swapper/722:54:317
356998799123,8cyclictest0-21swapper/721:29:477
356998799122,9cyclictest0-21swapper/722:59:377
3569987991211,0cyclictest0-21swapper/722:52:017
356998799120,4cyclictest0-21swapper/719:05:527
356998499129,2cyclictest26800-21mission-control23:49:166
356998499129,2cyclictest26692-21JS00:51:086
356998499129,2cyclictest1135-21snmpd20:44:496
356998499128,3cyclictest4031820-21cat22:54:356
356998499128,3cyclictest3959188-21grep22:14:406
356998499124,8cyclictest0-21swapper/623:15:146
3569984991211,1cyclictest0-21swapper/620:49:286
3569984991211,1cyclictest0-21swapper/620:10:326
3569984991211,1cyclictest0-21swapper/600:12:326
356998499120,1cyclictest0-21swapper/621:07:456
356998499120,1cyclictest0-21swapper/619:29:136
356998499120,12cyclictest0-21swapper/623:27:176
356998499120,11cyclictest0-21swapper/623:19:356
356998499120,11cyclictest0-21swapper/623:19:356
3569980991210,2cyclictest4099024-21kworker/u16:2+flush-8:023:34:585
3569980991210,2cyclictest4001767-21kworker/u16:1+events_unbound23:09:325
356997899129,2cyclictest3962995-21kworker/u16:2+flush-8:022:49:214
356997899129,2cyclictest1135-21snmpd21:27:474
356997899124,8cyclictest0-21swapper/422:21:214
356997899124,5cyclictest4001489-21idleruntime-cro22:39:204
3569978991211,1cyclictest0-21swapper/400:17:354
3569978991210,2cyclictest3660765-21kworker/u16:0+events_unbound20:04:434
356998799119,1cyclictest0-21swapper/723:44:367
356998799118,2cyclictest0-21swapper/720:29:467
356998799113,7cyclictest0-21swapper/719:20:357
356998799112,8cyclictest0-21swapper/719:43:507
356998799110,9cyclictest0-21swapper/722:17:097
356998799110,9cyclictest0-21swapper/722:01:217
356998799110,5cyclictest0-21swapper/721:09:197
356998499119,2cyclictest27326-21gmain22:48:126
356998499119,1cyclictest1135-21snmpd20:07:306
356998499118,2cyclictest1135-21snmpd21:41:086
356998499117,3cyclictest3895689-21cat21:44:356
356998499110,4cyclictest3656212-21cut19:39:386
356998499110,10cyclictest3869052-21kthreadcore21:29:386
356998499110,10cyclictest3589215-21kthreadcore19:04:386
356997899119,2cyclictest3660765-21kworker/u16:0+events_unbound21:12:304
356997899119,2cyclictest3660765-21kworker/u16:0+events_unbound21:12:304
356997899119,2cyclictest3550379-21kworker/u16:1+flush-8:019:49:474
356997899119,2cyclictest3550379-21kworker/u16:1+flush-8:019:34:464
356997899119,2cyclictest11596-21kworker/u16:3+flush-8:000:19:534
356997899118,2cyclictest3974905-21kthreadcore22:24:364
356997899118,2cyclictest3815981-21kworker/u16:1+flush-8:021:59:524
356997899118,2cyclictest1135-21snmpd23:20:054
356997899118,2cyclictest1135-21snmpd23:20:054
356997899114,7cyclictest0-21swapper/421:53:024
3569978991110,1cyclictest0-21swapper/423:31:044
3569978991110,1cyclictest0-21swapper/423:03:224
3569978991110,1cyclictest0-21swapper/421:44:304
3569978991110,1cyclictest0-21swapper/421:05:374
3569978991110,1cyclictest0-21swapper/419:30:584
3569978991110,1cyclictest0-21swapper/419:18:314
356997899110,11cyclictest0-21swapper/422:44:054
356998799109,0cyclictest0-21swapper/722:44:557
356998799109,0cyclictest0-21swapper/722:39:437
356998799108,1cyclictest0-21swapper/723:44:207
356998799100,9cyclictest0-21swapper/722:19:437
356998799100,9cyclictest0-21swapper/700:19:277
356998799100,9cyclictest0-21swapper/700:07:577
356998799100,8cyclictest0-21swapper/700:24:377
356998799100,5cyclictest0-21swapper/721:46:507
356998799100,5cyclictest0-21swapper/700:17:547
356998499109,1cyclictest0-21swapper/621:19:256
356998499109,1cyclictest0-21swapper/621:14:026
356998499109,1cyclictest0-21swapper/621:14:026
356998499109,1cyclictest0-21swapper/600:02:266
356998499108,2cyclictest86-21ksoftirqd/622:24:436
356998499100,9cyclictest3646341-21kthreadcore19:34:386
356998499100,9cyclictest3571632-21grep18:59:366
356998499100,9cyclictest0-21swapper/619:19:346
356998499100,3cyclictest3920292-21kthreadcore21:54:406
356997899109,1cyclictest0-21swapper/419:09:224
356997899108,2cyclictest3622667-21kworker/u16:0+events_unbound19:24:454
356997899107,2cyclictest1135-21snmpd23:11:504
356997899107,2cyclictest1135-21snmpd21:56:054
356997899106,3cyclictest546-21jbd2/sda3-821:34:214
35699879997,1cyclictest0-21swapper/723:24:397
35699879997,1cyclictest0-21swapper/720:04:377
35699879997,1cyclictest0-21swapper/719:54:377
35699879997,1cyclictest0-21swapper/719:49:447
35699879996,2cyclictest0-21swapper/723:09:207
35699849997,2cyclictest86-21ksoftirqd/600:14:356
35699849997,1cyclictest0-21swapper/623:49:326
35699849996,2cyclictest3809191-21kthreadcore20:59:366
35699849996,2cyclictest3768449-21/usr/sbin/munin20:39:426
35699849996,2cyclictest0-21swapper/620:29:206
35699849990,8cyclictest4177021-21cut00:09:216
35699849990,8cyclictest4119338-21hddtemp_smartct23:39:326
35699849990,8cyclictest4021310-21/usr/sbin/munin22:49:326
35699849990,8cyclictest0-21swapper/623:09:326
35699849990,8cyclictest0-21swapper/600:19:386
35699849990,2cyclictest3801481-21grep20:54:386
35699849990,2cyclictest3671804-21tune2fs19:49:346
35699789998,1cyclictest0-21swapper/423:51:534
35699789993,5cyclictest26683-21gdbus22:58:284
35699789990,9cyclictest0-21swapper/422:07:484
35699879987,0cyclictest0-21swapper/720:57:197
35699879987,0cyclictest0-21swapper/720:49:347
35699879987,0cyclictest0-21swapper/700:09:367
35699879986,1cyclictest0-21swapper/721:49:387
35699879986,1cyclictest0-21swapper/719:14:427
35699879980,3cyclictest0-21swapper/719:34:357
35699849985,2cyclictest1135-21snmpd21:14:356
35699849980,7cyclictest3719839-21/usr/sbin/munin20:14:476
35699849980,7cyclictest3690667-21/usr/sbin/munin19:59:356
35699849980,7cyclictest0-21swapper/623:29:376
35699879976,0cyclictest0-21swapper/719:24:247
35699879975,1cyclictest0-21swapper/721:09:367
35699879975,1cyclictest0-21swapper/721:09:367
35699879975,1cyclictest0-21swapper/719:44:407
35699849975,2cyclictest86-21ksoftirqd/622:19:326
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional