You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-16 - 08:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot2.osadl.org (updated Tue Apr 16, 2024 00:43:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
74787825622,29sleep70-21swapper/719:07:257
74797123726,8sleep60-21swapper/619:08:456
74788523222,7sleep50-21swapper/519:07:315
74785722817,8sleep40-21swapper/419:07:064
748326992621,4cyclictest771532-40dnf19:15:145
748324992624,2cyclictest801331-21kworker/u16:3+flush-8:019:44:414
748324992624,2cyclictest1066600-21kworker/u16:4+flush-8:021:49:414
748324992623,2cyclictest1263227-21kworker/u16:3+flush-8:023:34:214
748324992622,3cyclictest849762-21kworker/u16:1+flush-8:020:04:214
748324992422,2cyclictest988879-21kworker/u16:2+flush-8:023:04:464
748324992321,2cyclictest1312302-21kworker/u16:4+flush-8:000:19:204
748324992321,2cyclictest1312302-21kworker/u16:4+flush-8:000:04:514
748324992320,2cyclictest770954-21kworker/u16:1+flush-8:019:44:214
748324992320,2cyclictest1312302-21kworker/u16:4+flush-8:000:34:204
74833299229,5cyclictest0-21swapper/719:20:037
748324992220,2cyclictest937527-21kworker/u16:2+flush-8:020:49:374
748324992220,2cyclictest1008021-21kworker/u16:3+flush-8:021:19:324
748324992219,3cyclictest1312302-21kworker/u16:4+flush-8:000:24:444
748324992119,2cyclictest834298-21kworker/u16:2+flush-8:020:04:364
748324992119,2cyclictest834298-21kworker/u16:2+flush-8:020:04:364
748324992118,2cyclictest1302500-21kworker/u16:2+events_unbound00:19:394
74833299201,5cyclictest0-21swapper/722:19:417
74832999200,20cyclictest0-21swapper/620:37:036
748324992018,2cyclictest988879-21kworker/u16:2+events_unbound22:44:444
748324992018,2cyclictest953282-21kworker/u16:0+flush-8:021:14:414
748324992018,2cyclictest737964-21kworker/u16:0+events_unbound19:14:254
74833299195,6cyclictest0-21swapper/720:35:087
74833299190,6cyclictest0-21swapper/721:56:207
748329991915,3cyclictest737964-21kworker/u16:0+events_unbound19:34:446
74832999190,18cyclictest830438-21taskset19:44:506
748324991917,2cyclictest880293-21kworker/u16:3+events_unbound20:39:354
748324991917,2cyclictest849762-21kworker/u16:1+flush-8:020:34:434
748324991917,2cyclictest737964-21kworker/u16:0+flush-8:019:24:374
748324991917,2cyclictest737964-21kworker/u16:0+events_unbound19:09:494
748324991917,2cyclictest1262862-21kworker/u16:1+events_unbound23:26:454
748324991917,2cyclictest1145705-21kworker/u16:0+events_unbound22:29:434
748324991916,2cyclictest1008021-21kworker/u16:3+flush-8:021:39:494
748329991816,2cyclictest1262862-21kworker/u16:1+flush-8:023:49:396
748329991816,2cyclictest1086675-21kworker/u16:1+flush-8:022:09:476
748329991815,2cyclictest1302500-21kworker/u16:2+flush-8:000:09:476
748324991816,2cyclictest953282-21kworker/u16:0+flush-8:021:29:474
748324991816,2cyclictest1292516-21kworker/u16:0+flush-8:023:49:464
748324991816,2cyclictest1183631-21kworker/u16:4+flush-8:022:59:394
748324991816,2cyclictest1183631-21kworker/u16:4+flush-8:022:59:394
74833299174,6cyclictest0-21swapper/719:30:497
74833299174,5cyclictest0-21swapper/720:05:257
74833299174,5cyclictest0-21swapper/720:05:257
74833299170,1cyclictest0-21swapper/723:14:457
748329991715,2cyclictest1086675-21kworker/u16:1+flush-8:022:04:466
74832999170,17cyclictest0-21swapper/621:26:126
74832999170,17cyclictest0-21swapper/621:18:006
74832999170,17cyclictest0-21swapper/621:04:426
74832999170,17cyclictest0-21swapper/620:53:266
74832999170,17cyclictest0-21swapper/620:31:566
74832999170,17cyclictest0-21swapper/620:27:506
74832999170,17cyclictest0-21swapper/620:12:286
74832999170,17cyclictest0-21swapper/620:12:286
74832999170,17cyclictest0-21swapper/620:01:136
74832999170,16cyclictest893491-21kthreadcore20:19:396
74832999170,16cyclictest0-21swapper/620:54:276
74832999170,16cyclictest0-21swapper/620:14:316
748326991714,2cyclictest1135-21snmpd00:04:535
748324991715,2cyclictest988879-21kworker/u16:2+flush-8:022:14:464
748324991715,2cyclictest988879-21kworker/u16:2+events_unbound21:38:334
748324991715,2cyclictest834298-21kworker/u16:2+flush-8:019:54:364
748324991715,2cyclictest737964-21kworker/u16:0+events_unbound19:34:394
748324991715,2cyclictest1222720-21kworker/u16:3+flush-8:023:09:394
74833299167,8cyclictest0-21swapper/720:46:097
74833299160,1cyclictest0-21swapper/719:54:437
74833299160,14cyclictest0-21swapper/721:09:477
74832999160,16cyclictest0-21swapper/621:49:456
74832999160,16cyclictest0-21swapper/621:44:386
74832999160,16cyclictest0-21swapper/621:39:316
74832999160,16cyclictest0-21swapper/621:38:296
74832999160,16cyclictest0-21swapper/621:30:186
74832999160,16cyclictest0-21swapper/621:21:056
74832999160,16cyclictest0-21swapper/621:12:536
74832999160,16cyclictest0-21swapper/620:59:356
74832999160,16cyclictest0-21swapper/620:45:146
74832999160,16cyclictest0-21swapper/620:40:076
74832999160,16cyclictest0-21swapper/619:55:046
74832999160,16cyclictest0-21swapper/619:49:576
748326991611,5cyclictest27326-21gmain23:09:465
74832699160,13cyclictest0-21swapper/523:29:005
748324991614,2cyclictest880293-21kworker/u16:3+flush-8:020:59:464
748324991614,2cyclictest880293-21kworker/u16:3+flush-8:020:24:474
748324991614,2cyclictest849762-21kworker/u16:1+flush-8:020:19:344
748324991614,2cyclictest849762-21kworker/u16:1+events_unbound20:33:424
748324991614,2cyclictest1183631-21kworker/u16:4+flush-8:022:49:494
74833299157,7cyclictest0-21swapper/721:22:467
748329991513,2cyclictest737964-21kworker/u16:0+flush-8:019:14:436
748329991513,2cyclictest1263227-21kworker/u16:3+flush-8:023:59:166
748329991513,2cyclictest1183631-21kworker/u16:4+flush-8:022:59:216
748326991512,2cyclictest1135-21snmpd22:35:005
748324991513,2cyclictest849762-21kworker/u16:1+flush-8:020:09:444
748324991513,2cyclictest849762-21kworker/u16:1+flush-8:020:09:444
748324991513,2cyclictest1145705-21kworker/u16:0+flush-8:022:34:484
748324991513,2cyclictest1096523-21kworker/u16:3+flush-8:022:26:364
748324991513,2cyclictest1096523-21kworker/u16:3+flush-8:022:09:484
748324991513,2cyclictest1066600-21kworker/u16:4+events_unbound21:56:334
74833299145,8cyclictest0-21swapper/722:33:007
74833299145,8cyclictest0-21swapper/721:31:077
74833299145,8cyclictest0-21swapper/720:13:327
74833299145,8cyclictest0-21swapper/720:13:327
74833299141,5cyclictest0-21swapper/723:13:077
748332991412,1cyclictest0-21swapper/721:24:467
748332991411,2cyclictest0-21swapper/722:15:447
74833299140,6cyclictest0-21swapper/723:48:267
74833299140,6cyclictest0-21swapper/721:01:277
74833299140,6cyclictest0-21swapper/719:50:377
74833299140,6cyclictest0-21swapper/719:40:047
74833299140,13cyclictest0-21swapper/719:34:447
74832999140,14cyclictest0-21swapper/620:05:186
74832999140,14cyclictest0-21swapper/620:05:186
74832999140,14cyclictest0-21swapper/600:29:336
74832699147,6cyclictest0-21swapper/519:19:315
748326991412,1cyclictest0-21swapper/520:09:445
748326991412,1cyclictest0-21swapper/520:09:445
74832699140,14cyclictest0-21swapper/522:59:415
74832699140,14cyclictest0-21swapper/522:59:415
74832699140,14cyclictest0-21swapper/519:50:395
748324991412,2cyclictest849762-21kworker/u16:1+flush-8:021:04:384
748324991412,2cyclictest1234919-21kworker/u16:4+flush-8:023:20:114
748324991412,2cyclictest1076833-21kworker/u16:0+events_unbound22:04:304
74832499140,8cyclictest0-21swapper/422:40:544
74833299134,8cyclictest0-21swapper/721:16:137
74833299133,8cyclictest0-21swapper/720:29:597
748332991312,0cyclictest0-21swapper/723:29:087
748332991310,2cyclictest0-21swapper/720:55:127
748329991311,2cyclictest1312302-21kworker/u16:4+flush-8:000:19:516
748329991311,2cyclictest1262862-21kworker/u16:1+flush-8:023:59:526
748329991311,2cyclictest1262862-21kworker/u16:1+events_unbound23:44:496
748329991311,2cyclictest1234099-21kworker/u16:0+flush-8:023:14:456
748329991311,2cyclictest1066600-21kworker/u16:4+flush-8:021:54:526
748329991310,2cyclictest737964-21kworker/u16:0+flush-8:019:24:426
74832999130,13cyclictest0-21swapper/623:39:436
74832699138,4cyclictest1135-21snmpd22:32:005
74832699136,7cyclictest0-21swapper/519:43:225
74832699135,8cyclictest0-21swapper/519:11:395
748326991312,1cyclictest0-21swapper/523:55:295
748326991312,1cyclictest0-21swapper/520:24:425
748326991310,2cyclictest0-21swapper/521:24:585
748326991310,2cyclictest0-21swapper/519:54:305
74832699130,0cyclictest0-21swapper/500:21:115
748324991311,2cyclictest937527-21kworker/u16:2+flush-8:020:44:524
748324991311,2cyclictest770954-21kworker/u16:1+flush-8:019:49:264
748324991311,2cyclictest737964-21kworker/u16:0+flush-8:020:14:424
748324991311,2cyclictest1312302-21kworker/u16:4+events_unbound00:11:534
748324991311,2cyclictest1263227-21kworker/u16:3+flush-8:023:58:254
748324991311,2cyclictest1263227-21kworker/u16:3+events_unbound23:44:384
748324991311,2cyclictest1145705-21kworker/u16:0+flush-8:022:57:364
748324991311,2cyclictest1086675-21kworker/u16:1+flush-8:022:21:394
748324991311,2cyclictest1076833-21kworker/u16:0+events_unbound22:03:224
748324991311,2cyclictest1062467-21kworker/u16:2+flush-8:019:29:414
748324991311,2cyclictest1008021-21kworker/u16:3+flush-8:021:24:504
74832499130,1cyclictest0-21swapper/423:39:074
74833299129,2cyclictest0-21swapper/723:52:327
74833299129,2cyclictest0-21swapper/720:25:387
74833299129,2cyclictest0-21swapper/719:25:397
74833299129,2cyclictest0-21swapper/700:38:067
74833299123,8cyclictest0-21swapper/723:01:287
74833299123,8cyclictest0-21swapper/723:01:287
74833299123,8cyclictest0-21swapper/720:00:467
748332991211,0cyclictest0-21swapper/722:58:147
748332991211,0cyclictest0-21swapper/722:47:427
74833299120,11cyclictest0-21swapper/719:46:017
74833299120,10cyclictest0-21swapper/721:53:387
74832999129,3cyclictest988879-21kworker/u16:2+flush-8:023:24:486
74832999129,2cyclictest1135-21snmpd22:50:036
74832999125,7cyclictest0-21swapper/623:07:406
748329991211,1cyclictest0-21swapper/623:00:476
748329991211,1cyclictest0-21swapper/623:00:476
748329991210,1cyclictest1096144-21turbostat.cron22:04:206
74832999120,12cyclictest0-21swapper/622:24:506
74832699129,3cyclictest0-21swapper/520:35:445
74832699128,3cyclictest1115239-21sort22:09:485
74832699127,4cyclictest1135-21snmpd20:54:395
74832699126,2cyclictest1195301-21df22:54:335
74832699125,6cyclictest1135-21snmpd20:47:065
74832699124,7cyclictest976973-21cat20:59:435
74832699124,7cyclictest1097748-21grep22:04:395
74832699123,8cyclictest792947-21irqcore19:29:315
748326991211,1cyclictest0-21swapper/520:50:245
748326991210,2cyclictest0-21swapper/500:10:335
74832699120,1cyclictest0-21swapper/520:30:425
74832699120,12cyclictest0-21swapper/523:41:375
74832699120,12cyclictest0-21swapper/523:06:465
74832699120,12cyclictest0-21swapper/522:52:355
748324991210,2cyclictest1302500-21kworker/u16:2+events_unbound00:37:354
748324991210,2cyclictest1263227-21kworker/u16:3+flush-8:023:39:404
748324991210,2cyclictest1234099-21kworker/u16:0+flush-8:023:15:314
748324991210,2cyclictest1008021-21kworker/u16:3+events_unbound21:46:394
74832499120,12cyclictest0-21swapper/419:22:564
74833299118,2cyclictest0-21swapper/722:35:337
74833299110,9cyclictest0-21swapper/700:06:307
74833299110,6cyclictest0-21swapper/723:40:237
74833299110,6cyclictest0-21swapper/720:40:057
74832999116,4cyclictest1244749-21cat23:19:356
74832999114,7cyclictest0-21swapper/600:08:216
748329991110,1cyclictest0-21swapper/622:44:046
748329991110,1cyclictest0-21swapper/622:34:276
748329991110,1cyclictest0-21swapper/619:31:066
74832999110,1cyclictest0-21swapper/622:31:566
74832999110,11cyclictest0-21swapper/623:37:126
74832999110,11cyclictest0-21swapper/619:44:116
74832999110,11cyclictest0-21swapper/600:37:446
74832999110,11cyclictest0-21swapper/600:19:186
74832999110,0cyclictest0-21swapper/622:46:366
74832699119,2cyclictest953282-21kworker/u16:0+flush-8:021:14:335
74832699119,2cyclictest0-21swapper/520:21:425
74832699113,7cyclictest980849-21cut21:04:415
74832699113,7cyclictest1391683-21expr00:34:325
74832699113,7cyclictest1378664-21cut00:24:405
74832699113,7cyclictest1274032-21cut23:34:335
74832699113,7cyclictest1235895-21cut23:14:385
74832699113,7cyclictest1175866-21hwlatdetect22:44:375
74832699113,7cyclictest1173657-21cat22:39:455
74832699113,7cyclictest1049217-21grep21:39:415
74832699113,7cyclictest0-21swapper/522:27:455
74832699113,3cyclictest1059013-21grep21:44:405
74832699112,4cyclictest1080660-21cut21:54:425
74832699112,4cyclictest1074752-21cut21:49:435
74832699112,4cyclictest1041198-21cut21:34:405
74832699112,4cyclictest1008284-21grep21:19:325
74832699112,3cyclictest1354272-21tr00:14:395
74832699112,3cyclictest1031933-21cut21:29:395
74832699112,2cyclictest0-21swapper/500:30:335
748326991110,1cyclictest0-21swapper/523:49:495
748326991110,1cyclictest0-21swapper/523:34:035
748326991110,1cyclictest0-21swapper/522:18:415
748326991110,1cyclictest0-21swapper/520:01:415
74832699110,6cyclictest0-21swapper/520:39:445
74832699110,10cyclictest0-21swapper/523:44:365
74832499119,2cyclictest953282-21kworker/u16:0+events_unbound21:14:034
74832499119,2cyclictest880293-21kworker/u16:3+events_unbound20:55:584
74832499119,2cyclictest1312302-21kworker/u16:4+flush-8:000:01:364
74833299109,0cyclictest0-21swapper/721:41:287
74833299107,2cyclictest0-21swapper/720:51:067
74833299100,9cyclictest0-21swapper/723:59:147
74833299100,9cyclictest0-21swapper/721:45:317
74833299100,5cyclictest0-21swapper/700:30:487
74833299100,4cyclictest1145227-21taskset22:28:097
74833299100,2cyclictest0-21swapper/722:49:457
74833299100,2cyclictest0-21swapper/719:14:417
74832999108,2cyclictest988879-21kworker/u16:2+flush-8:023:29:356
74832999108,2cyclictest770954-21kworker/u16:1+flush-8:019:19:446
74832999108,2cyclictest1312302-21kworker/u16:4+flush-8:000:24:356
74832999106,3cyclictest1126865-21cat22:19:396
74832999102,7cyclictest1116745-21df22:14:356
74832699109,1cyclictest0-21swapper/521:59:275
74832699107,2cyclictest1135-21snmpd22:23:505
74832699106,3cyclictest0-21swapper/521:09:395
74832699102,3cyclictest880867-21cut20:14:345
74832699102,3cyclictest863488-21kthreadcore20:04:335
74832699102,3cyclictest863488-21kthreadcore20:04:335
74832699102,3cyclictest822030-21grep19:44:305
74832699102,3cyclictest1326678-21tr23:59:425
74832699100,4cyclictest809765-21grep19:34:405
74832699100,4cyclictest782580-21/usr/sbin/munin19:24:305
74832699100,4cyclictest1246117-21grep23:19:395
7483329990,8cyclictest0-21swapper/723:38:567
7483329990,8cyclictest0-21swapper/721:38:377
7483329990,8cyclictest0-21swapper/700:28:217
7483329990,8cyclictest0-21swapper/700:19:217
7483329990,2cyclictest0-21swapper/722:04:367
7483329990,2cyclictest0-21swapper/719:09:337
7483299997,2cyclictest1062467-21kworker/u16:2+flush-8:019:09:426
7483299990,8cyclictest0-21swapper/623:09:396
7483329987,0cyclictest0-21swapper/721:08:127
7483329980,7cyclictest0-21swapper/720:14:427
7483329980,1cyclictest13260362sleep723:59:437
7483329976,1cyclictest0-21swapper/723:29:497
7483329974,2cyclictest899553-21turbostat20:24:217
7483329971,5cyclictest1223857-21turbostat23:09:217
7483329970,6cyclictest0-21swapper/722:09:357
7483329970,6cyclictest0-21swapper/700:14:407
7483329970,2cyclictest0-21swapper/700:09:457
7483329963,2cyclictest0-21swapper/722:39:477
7483329960,5cyclictest0-21swapper/723:19:327
7483329960,1cyclictest0-21swapper/722:00:447
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional