You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 10:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sat Apr 27, 2024 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39312172426,31sleep70-21swapper/719:08:177
3931572993936,3cyclictest3964337-21kworker/u16:0+flush-8:020:14:276
3931567993634,2cyclictest142705-21kworker/u16:1+flush-8:023:34:004
393096523625,8sleep40-21swapper/419:04:454
3931567993432,2cyclictest863851-21kworker/u16:1+flush-8:019:28:334
3931569993329,3cyclictest4111505-21latency_hist20:38:375
3931567993331,2cyclictest4159423-21kworker/u16:1+flush-8:021:49:064
3931567993230,2cyclictest298902-21kworker/u16:5+flush-8:000:13:494
3931567993230,2cyclictest172573-21kworker/u16:2+events_unbound22:54:024
3931567993230,2cyclictest142705-21kworker/u16:1+flush-8:023:03:514
393113123222,7sleep60-21swapper/619:07:046
3931567993028,2cyclictest4000610-21kworker/u16:4+flush-8:019:59:064
3931567993028,2cyclictest280098-21kworker/u16:0+flush-8:000:04:034
3931567993026,3cyclictest367632-21latency_hist00:28:364
3931567992826,2cyclictest4190973-21kworker/u16:2+events_unbound21:23:474
3931567992826,2cyclictest3964337-21kworker/u16:0+events_unbound22:00:264
3931567992826,2cyclictest172573-21kworker/u16:2+flush-8:023:48:324
3931567992826,2cyclictest172573-21kworker/u16:2+flush-8:022:48:504
393121522818,7sleep50-21swapper/519:08:155
3931567992725,2cyclictest3964337-21kworker/u16:0+events_unbound21:34:084
3931567992725,2cyclictest280098-21kworker/u16:0+flush-8:023:58:324
3931567992725,2cyclictest172573-21kworker/u16:2+events_unbound23:23:524
3931567992624,2cyclictest4000610-21kworker/u16:4+events_unbound19:53:514
3931567992624,2cyclictest3964337-21kworker/u16:0+flush-8:022:34:104
3931567992624,2cyclictest280098-21kworker/u16:0+events_unbound00:18:554
3931567992624,2cyclictest142705-21kworker/u16:1+events_unbound23:19:114
3931567992624,2cyclictest113878-21kworker/u16:3+flush-8:022:59:074
3931567992624,2cyclictest113878-21kworker/u16:3+events_unbound23:49:074
3931567992623,2cyclictest75200-21kworker/u16:4+events_unbound22:08:564
3931567992623,2cyclictest3964337-21kworker/u16:0+events_unbound19:28:544
3931577992524,0cyclictest0-21swapper/719:13:387
3931567992422,2cyclictest329250-21kworker/u16:3+events_unbound00:09:074
3931567992321,2cyclictest4092804-21kworker/u16:4+flush-8:020:33:324
3931567992321,2cyclictest3974287-21kworker/u16:2+events_unbound19:33:494
3931567992321,2cyclictest329250-21kworker/u16:3+flush-8:000:31:444
3931567992321,2cyclictest104133-21kworker/u16:2+flush-8:022:18:474
3931572992214,8cyclictest0-21swapper/623:25:596
3931567992220,2cyclictest863851-21kworker/u16:1+flush-8:019:09:054
3931567992220,2cyclictest4159423-21kworker/u16:1+flush-8:021:39:024
3931567992220,2cyclictest4090990-21kworker/u16:2+events_unbound20:33:574
3931567992220,2cyclictest4063443-21kworker/u16:5+flush-8:021:03:474
3931567992220,2cyclictest4042670-21kworker/u16:3+flush-8:020:18:534
3931567992219,2cyclictest4042651-21kworker/u16:2+events_unbound20:08:524
3931577992120,0cyclictest0-21swapper/719:48:507
3931572992118,3cyclictest75200-21kworker/u16:4+flush-8:022:09:046
3931567992119,2cyclictest863851-21kworker/u16:1+events_unbound19:38:584
3931567992119,2cyclictest4092804-21kworker/u16:4+flush-8:021:13:324
3931567992119,2cyclictest4092804-21kworker/u16:4+flush-8:021:13:324
3931567992119,2cyclictest4092804-21kworker/u16:4+flush-8:020:49:074
3931567992119,2cyclictest4063443-21kworker/u16:5+flush-8:020:44:094
3931577992018,1cyclictest0-21swapper/700:28:367
3931567992018,2cyclictest172573-21kworker/u16:2+flush-8:023:08:384
3931567992018,2cyclictest172573-21kworker/u16:2+events_unbound23:39:074
3931577991918,0cyclictest0-21swapper/722:23:547
3931577991917,1cyclictest0-21swapper/720:39:027
3931577991917,1cyclictest0-21swapper/700:09:057
393156999190,14cyclictest0-21swapper/523:49:215
3931567991917,2cyclictest4000610-21kworker/u16:4+events_unbound20:03:554
3931567991917,2cyclictest3964337-21kworker/u16:0+flush-8:022:15:424
3931567991917,2cyclictest113878-21kworker/u16:3+events_unbound22:29:014
3931569991815,2cyclictest4063443-21kworker/u16:5+flush-8:021:09:085
3931569991815,2cyclictest4063443-21kworker/u16:5+flush-8:021:09:085
393156799189,8cyclictest0-21swapper/419:16:224
3931567991816,2cyclictest4159423-21kworker/u16:1+flush-8:022:23:574
3931567991816,2cyclictest4159423-21kworker/u16:1+flush-8:021:21:174
3931567991816,2cyclictest4063443-21kworker/u16:5+flush-8:021:32:544
3931567991816,2cyclictest4063443-21kworker/u16:5+flush-8:021:17:324
3931567991816,2cyclictest4063443-21kworker/u16:5+events_unbound20:38:544
3931567991816,2cyclictest3964337-21kworker/u16:0+flush-8:020:14:284
3931567991816,2cyclictest172573-21kworker/u16:2+flush-8:023:18:224
3931577991716,0cyclictest0-21swapper/719:13:427
3931577991715,1cyclictest0-21swapper/720:54:087
393157799170,9cyclictest0-21swapper/722:46:167
3931572991715,2cyclictest4092804-21kworker/u16:4+events_unbound20:59:396
3931572991714,2cyclictest4042670-21kworker/u16:3+flush-8:020:19:086
3931572991714,2cyclictest3964337-21kworker/u16:0+events_unbound22:33:516
3931567991715,2cyclictest3984544-21kworker/u16:3+flush-8:019:43:584
3931567991715,2cyclictest377340-21kworker/u16:4+events_unbound00:34:004
3931567991715,2cyclictest309397-21kworker/u16:1+events_unbound23:58:554
3931567991715,2cyclictest1159923-21kworker/u16:3+events_unbound19:18:504
3931577991615,0cyclictest0-21swapper/723:38:557
3931577991615,0cyclictest0-21swapper/700:14:017
3931572991613,2cyclictest3974287-21kworker/u16:2+events_unbound19:39:106
393156999162,5cyclictest0-21swapper/519:36:375
3931569991614,2cyclictest172573-21kworker/u16:2+flush-8:022:48:545
3931569991613,2cyclictest4000610-21kworker/u16:4+flush-8:020:14:035
3931567991614,2cyclictest4090990-21kworker/u16:2+flush-8:020:58:594
3931567991614,2cyclictest152969-21kworker/u16:5+flush-8:022:45:464
393157799153,5cyclictest0-21swapper/723:18:177
393157799152,5cyclictest0-21swapper/720:03:177
393157799151,6cyclictest0-21swapper/723:45:147
3931577991514,0cyclictest0-21swapper/722:19:097
3931577991514,0cyclictest0-21swapper/720:49:067
3931577991513,1cyclictest0-21swapper/722:38:597
393157799150,7cyclictest0-21swapper/721:53:567
393157799150,1cyclictest0-21swapper/719:39:047
393157799150,14cyclictest0-21swapper/723:04:007
393157299157,7cyclictest0-21swapper/621:38:536
3931572991513,2cyclictest280098-21kworker/u16:0+events_unbound23:53:506
3931569991512,2cyclictest1135-21snmpd20:48:305
393156999150,7cyclictest0-21swapper/520:11:595
3931567991514,1cyclictest0-21swapper/420:54:044
3931567991513,2cyclictest75200-21kworker/u16:4+flush-8:022:04:094
3931567991513,2cyclictest65109-21kworker/u16:2+flush-8:021:53:434
3931567991513,2cyclictest113878-21kworker/u16:3+flush-8:022:40:084
3931567991510,4cyclictest54962-21latency_hist21:48:374
393157799146,7cyclictest0-21swapper/700:36:017
393157799145,8cyclictest0-21swapper/722:28:397
3931577991413,0cyclictest0-21swapper/723:33:577
3931577991412,1cyclictest0-21swapper/719:24:027
3931577991411,2cyclictest0-21swapper/721:19:287
393157799140,7cyclictest0-21swapper/720:20:387
393157799140,6cyclictest0-21swapper/721:36:047
393157799140,2cyclictest0-21swapper/723:29:477
393157799140,2cyclictest0-21swapper/723:29:477
393157799140,1cyclictest0-21swapper/719:34:027
393157799140,12cyclictest0-21swapper/700:21:277
393157299149,5cyclictest0-21swapper/619:12:276
393157299147,7cyclictest0-21swapper/623:46:596
393157299147,7cyclictest0-21swapper/619:30:316
3931572991413,1cyclictest0-21swapper/623:28:476
3931572991413,1cyclictest0-21swapper/623:28:476
3931572991412,2cyclictest4092804-21kworker/u16:4+flush-8:020:34:046
3931572991412,2cyclictest4090990-21kworker/u16:2+events_unbound20:39:066
3931572991412,2cyclictest3964337-21kworker/u16:0+flush-8:022:04:076
3931572991412,2cyclictest3964337-21kworker/u16:0+flush-8:019:23:556
3931572991412,2cyclictest142705-21kworker/u16:1+flush-8:023:08:526
393157299141,13cyclictest0-21swapper/622:02:476
393157299140,14cyclictest0-21swapper/623:37:086
393157299140,14cyclictest0-21swapper/619:23:386
393156999147,7cyclictest0-21swapper/521:49:355
3931569991410,3cyclictest35412-21latency_hist21:38:375
3931567991412,2cyclictest3964337-21kworker/u16:0+flush-8:020:27:464
3931577991312,0cyclictest0-21swapper/723:19:127
3931577991312,0cyclictest0-21swapper/722:52:277
3931577991312,0cyclictest0-21swapper/721:03:327
3931577991311,1cyclictest0-21swapper/719:43:537
393157799130,5cyclictest0-21swapper/723:10:047
393157799130,5cyclictest0-21swapper/721:43:447
393157799130,5cyclictest0-21swapper/720:23:597
393157799130,11cyclictest0-21swapper/722:12:417
393157299139,3cyclictest4190973-21kworker/u16:2+flush-8:021:23:526
393157299139,3cyclictest4013576-21latency_hist19:48:376
3931572991311,2cyclictest142705-21kworker/u16:1+flush-8:022:39:066
3931572991310,2cyclictest329250-21kworker/u16:3+flush-8:000:09:076
393157299130,13cyclictest0-21swapper/622:14:036
393156999134,8cyclictest0-21swapper/500:18:515
3931569991312,1cyclictest0-21swapper/519:08:565
3931567991311,2cyclictest4013916-21kworker/u16:1+events_unbound19:48:464
3931567991311,2cyclictest142705-21kworker/u16:1+events_unbound23:29:424
3931567991311,2cyclictest142705-21kworker/u16:1+events_unbound23:29:424
3931577991211,0cyclictest0-21swapper/723:49:057
3931577991211,0cyclictest0-21swapper/723:24:027
3931577991211,0cyclictest0-21swapper/722:13:387
3931577991211,0cyclictest0-21swapper/722:04:007
3931577991211,0cyclictest0-21swapper/720:45:527
393157299128,3cyclictest113878-21kworker/u16:3+flush-8:022:59:076
393157299125,7cyclictest0-21swapper/600:29:046
3931572991211,1cyclictest0-21swapper/623:42:396
3931572991211,1cyclictest0-21swapper/600:24:546
3931572991210,2cyclictest4092804-21kworker/u16:4+events_unbound21:13:496
3931572991210,2cyclictest3974287-21kworker/u16:2+flush-8:019:48:476
3931572991210,2cyclictest142705-21kworker/u16:1+flush-8:023:14:056
393157299120,1cyclictest0-21swapper/621:12:346
393157299120,1cyclictest0-21swapper/621:12:346
393157299120,1cyclictest0-21swapper/600:21:456
393157299120,12cyclictest0-21swapper/622:22:246
393156999129,2cyclictest1135-21snmpd20:51:335
393156999128,3cyclictest142705-21kworker/u16:1+flush-8:023:14:035
393156999128,3cyclictest104133-21kworker/u16:2+flush-8:022:18:485
3931569991211,1cyclictest0-21swapper/522:12:465
3931569991211,1cyclictest0-21swapper/521:21:015
393156999120,1cyclictest0-21swapper/522:30:355
393156999120,12cyclictest0-21swapper/523:58:245
393157799118,2cyclictest0-21swapper/721:30:527
393157799118,2cyclictest0-21swapper/719:30:287
393157799111,9cyclictest0-21swapper/723:02:437
3931577991110,0cyclictest0-21swapper/721:58:517
3931577991110,0cyclictest0-21swapper/721:15:277
3931577991110,0cyclictest0-21swapper/721:04:267
393157799110,6cyclictest0-21swapper/721:42:107
393157299119,2cyclictest3964337-21kworker/u16:0+flush-8:021:28:586
393157299118,2cyclictest1135-21snmpd00:07:156
393157299117,3cyclictest124431-21cat22:23:496
393157299116,4cyclictest289854-21/usr/sbin/munin23:48:496
393157299114,7cyclictest0-21swapper/600:00:106
393157299113,1cyclictest0-21swapper/600:33:536
3931572991110,1cyclictest0-21swapper/621:45:596
3931572991110,1cyclictest0-21swapper/620:57:266
3931572991110,1cyclictest0-21swapper/620:26:326
3931572991110,1cyclictest0-21swapper/619:58:476
393157299110,1cyclictest0-21swapper/620:52:146
393157299110,11cyclictest0-21swapper/623:08:086
393157299110,11cyclictest0-21swapper/619:37:586
393157299110,10cyclictest187104-21kthreadcore22:53:586
393156999119,1cyclictest0-21swapper/519:53:375
393156999118,2cyclictest1135-21snmpd20:54:365
393156999118,2cyclictest1135-21snmpd20:42:005
393156999118,2cyclictest1135-21snmpd20:20:355
393156999117,3cyclictest65109-21kworker/u16:2+flush-8:021:54:035
393156999116,4cyclictest191815-21latency_hist22:58:375
3931569991110,1cyclictest0-21swapper/523:45:095
3931569991110,1cyclictest0-21swapper/523:22:465
3931569991110,1cyclictest0-21swapper/523:07:575
3931569991110,1cyclictest0-21swapper/522:44:465
3931569991110,1cyclictest0-21swapper/522:35:195
3931569991110,1cyclictest0-21swapper/519:30:425
3931569991110,1cyclictest0-21swapper/519:16:215
3931569991110,1cyclictest0-21swapper/500:13:415
3931569991110,1cyclictest0-21swapper/500:01:335
393156999110,11cyclictest0-21swapper/522:27:265
393156999110,11cyclictest0-21swapper/522:00:475
393156999110,11cyclictest0-21swapper/521:39:585
393156999110,11cyclictest0-21swapper/520:30:555
393156999110,11cyclictest0-21swapper/500:12:075
393157799109,0cyclictest0-21swapper/719:53:547
393157799109,0cyclictest0-21swapper/700:29:027
393157799108,1cyclictest0-21swapper/721:23:567
393157799108,1cyclictest0-21swapper/720:09:087
393157799106,2cyclictest0-21swapper/721:09:367
393157799106,2cyclictest0-21swapper/721:09:367
393157799100,8cyclictest0-21swapper/700:05:397
393157799100,4cyclictest0-21swapper/720:37:337
393157299109,1cyclictest0-21swapper/621:19:216
393157299107,2cyclictest65119-21/usr/sbin/munin21:54:056
393157299107,2cyclictest231201-21/usr/sbin/munin23:19:056
393157299107,2cyclictest1135-21snmpd22:45:346
393157299105,5cyclictest0-21swapper/620:07:386
393157299100,9cyclictest3953996-21ps19:14:046
393157299100,9cyclictest0-21swapper/622:29:016
393157299100,3cyclictest4031712-21kthreadcore19:54:016
393156999109,1cyclictest0-21swapper/523:12:405
393156999109,1cyclictest0-21swapper/519:55:465
393156999109,1cyclictest0-21swapper/519:46:095
393156999107,2cyclictest1135-21snmpd23:29:145
393156999107,2cyclictest1135-21snmpd23:29:145
393156999107,2cyclictest1135-21snmpd22:05:115
393156999107,2cyclictest1135-21snmpd21:06:395
393156999107,2cyclictest1135-21snmpd19:41:135
393156999106,3cyclictest280098-21kworker/u16:0+flush-8:000:34:075
393156999100,0cyclictest0-21swapper/500:24:155
39315779998,0cyclictest0-21swapper/700:03:257
39315779990,8cyclictest0-21swapper/721:48:587
39315779990,4cyclictest0-21swapper/720:03:537
39315779990,0cyclictest0-21swapper/722:54:467
39315729996,2cyclictest58501-21kthreadcore21:48:576
39315729996,2cyclictest0-21swapper/621:03:576
39315729995,3cyclictest4122253-21tune2fs20:43:506
39315729995,3cyclictest34570-21proc_pri21:34:046
39315729990,8cyclictest0-21swapper/622:48:556
39315729990,8cyclictest0-21swapper/600:14:006
39315699998,1cyclictest0-21swapper/523:38:135
39315699998,1cyclictest0-21swapper/522:40:025
39315699998,1cyclictest0-21swapper/520:05:415
39315699996,2cyclictest1135-21snmpd21:15:075
39315699995,3cyclictest3964337-21kworker/u16:0+flush-8:021:44:085
39315699990,8cyclictest0-21swapper/522:14:045
39315699990,5cyclictest0-21swapper/520:25:595
39315699990,1cyclictest0-21swapper/500:28:505
39315779987,0cyclictest0-21swapper/719:19:007
39315779986,1cyclictest0-21swapper/720:13:487
39315779980,7cyclictest0-21swapper/723:53:497
39315779980,7cyclictest0-21swapper/720:32:557
39315729986,1cyclictest0-21swapper/620:08:486
39315729983,4cyclictest4095772-21cut20:28:546
39315699986,1cyclictest0-21swapper/522:59:005
39315699985,2cyclictest0-21swapper/521:23:555
39315699983,4cyclictest4038377-21cut19:58:575
39315699983,4cyclictest327801-21swap00:04:025
39315699983,4cyclictest27404-21pool-gsd-smartc19:23:475
39315699983,4cyclictest247385-21cut23:23:585
39315699983,4cyclictest20499-21grep21:28:555
39315699981,5cyclictest4159435-21ntpq20:59:045
39315779970,6cyclictest0-21swapper/722:33:587
39315699975,1cyclictest0-21swapper/523:38:545
39315699975,1cyclictest0-21swapper/519:18:565
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional