You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-08 - 23:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3 (updated Fri Dec 08, 2023 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1853155997248,22cyclictest1997198-21kworker/1:211:20:221
185315599681,65cyclictest0-21swapper/110:50:181
185315599661,2cyclictest1962502-21fschecks_count10:35:221
185315599650,63cyclictest0-21swapper/111:26:101
185315599640,2cyclictest0-21swapper/111:32:371
1853155996360,2cyclictest0-21swapper/108:03:031
185315599630,1cyclictest0-21swapper/108:41:371
1853155996239,15cyclictest2031824-21kworker/1:112:05:071
1853155996143,16cyclictest1934533-21kworker/1:010:30:211
1853155996139,20cyclictest1871353-21kworker/1:108:20:201
1853155996045,14cyclictest1909077-21kworker/1:009:39:561
1853155996044,15cyclictest4013347-1kworker/1:0H12:17:061
1853155996041,17cyclictest4013347-1kworker/1:0H07:59:591
1853155995943,15cyclictest4013347-1kworker/1:0H09:29:441
1853155995942,16cyclictest1992822-21kworker/1:011:15:221
1853155995942,16cyclictest1934533-21kworker/1:010:10:301
1853155995941,17cyclictest1889875-21kworker/1:209:20:241
1853155995941,16cyclictest1853589-21kworker/1:207:10:301
1853155995843,14cyclictest4013347-1kworker/1:0H11:00:061
1853155995843,14cyclictest4013347-1kworker/1:0H08:33:081
1853155995843,14cyclictest2017029-21kworker/1:011:55:221
1853155995843,14cyclictest2017029-21kworker/1:011:55:221
1853155995843,14cyclictest1978468-21kworker/1:111:05:221
1853155995843,14cyclictest1909077-21kworker/1:009:45:201
1853155995843,14cyclictest1889875-21kworker/1:209:15:221
1853155995843,14cyclictest1871353-21kworker/1:108:45:231
1853155995842,15cyclictest4013347-1kworker/1:0H08:10:061
1853155995842,15cyclictest4013347-1kworker/1:0H07:40:261
1853155995842,15cyclictest2051063-21kworker/1:112:35:011
1853155995842,15cyclictest2051063-21kworker/1:112:30:161
1853155995842,15cyclictest2017029-21kworker/1:011:45:231
1853155995842,15cyclictest1887784-21kworker/1:009:05:061
1853155995754,2cyclictest0-21swapper/112:24:361
1853155995742,14cyclictest2031824-21kworker/1:112:10:241
1853155995742,14cyclictest2011424-21kworker/1:211:35:071
1853155995742,14cyclictest1929369-21kworker/1:210:00:331
1853155995742,14cyclictest1929369-21kworker/1:209:55:221
1853155995742,14cyclictest1853589-21kworker/1:207:53:371
1853155995742,14cyclictest1853589-21kworker/1:207:20:241
1853155995741,15cyclictest4013347-1kworker/1:0H10:40:251
1853155995741,15cyclictest1871353-21kworker/1:108:35:211
1853155995641,14cyclictest4013347-1kworker/1:0H10:15:221
1853155995641,14cyclictest4013347-1kworker/1:0H10:15:221
1853155995641,14cyclictest4013347-1kworker/1:0H08:15:221
1853155995641,14cyclictest2017029-21kworker/1:011:53:081
1853155995641,14cyclictest1974031-21kworker/1:210:55:021
1853155995641,14cyclictest1889875-21kworker/1:209:30:011
1853155995641,14cyclictest1887784-21kworker/1:009:00:271
1853155995641,14cyclictest1871353-21kworker/1:108:50:011
1853155995641,14cyclictest1871353-21kworker/1:108:29:331
1853155995641,14cyclictest1853589-21kworker/1:207:32:371
1853155995641,14cyclictest1853589-21kworker/1:207:17:121
1853155995640,15cyclictest2011424-21kworker/1:211:40:291
1853155995639,16cyclictest1978468-21kworker/1:111:10:231
1853162995550,3cyclictest3209943-1kworker/3:2H09:52:363
1853155995540,14cyclictest1934533-21kworker/1:010:20:011
1853155995540,14cyclictest1909077-21kworker/1:009:40:231
1853155995540,14cyclictest1889875-21kworker/1:209:10:011
1853155995539,15cyclictest1871353-21kworker/1:108:58:071
1853155995537,16cyclictest4013347-1kworker/1:0H10:25:221
1853155995439,14cyclictest1871353-21kworker/1:108:09:281
1853155995439,14cyclictest1853589-21kworker/1:207:49:241
1853155995438,15cyclictest1853589-21kworker/1:207:25:231
1853155995437,16cyclictest1923913-21kworker/1:109:50:211
185315599540,4cyclictest331-21jbd2/mmcblk0p2-07:37:361
1853155995338,14cyclictest1934533-21kworker/1:010:45:251
1853155995337,15cyclictest2031824-21kworker/1:112:00:221
1853155995335,17cyclictest4013347-1kworker/1:0H10:10:001
185315599531,1cyclictest0-21swapper/112:25:541
1853158995249,2cyclictest1861271-21kworker/2:107:55:292
1853158995247,3cyclictest1839978-21kworker/2:007:28:082
1853158995246,4cyclictest64099-1kworker/2:1H12:09:362
1853158995246,4cyclictest64099-1kworker/2:1H09:20:062
1853162995147,3cyclictest3209943-1kworker/3:2H09:35:233
1853162995146,2cyclictest3209943-1kworker/3:2H11:40:003
1853162995145,3cyclictest3209943-1kworker/3:2H07:55:223
1853158995147,3cyclictest64099-1kworker/2:1H12:35:292
1853158995147,2cyclictest1911472-21kworker/2:011:26:372
1853158995146,3cyclictest1839978-21kworker/2:007:23:382
1853158995144,4cyclictest64099-1kworker/2:1H08:04:262
1853158995047,2cyclictest1861271-21kworker/2:108:23:072
1853158995046,3cyclictest1861271-21kworker/2:107:43:372
1853158995045,3cyclictest64099-1kworker/2:1H11:00:072
1853158995045,3cyclictest1880003-21kworker/2:209:09:072
1853158995044,4cyclictest1911472-21kworker/2:011:39:022
1853152995045,3cyclictest3041382-1kworker/0:1H11:31:050
1853152995045,3cyclictest1940750-21kworker/0:311:54:050
1853162994945,2cyclictest1839931-21kworker/3:007:38:073
1853158994946,2cyclictest1911472-21kworker/2:010:58:372
1853158994946,2cyclictest1861271-21kworker/2:108:30:122
1853158994945,3cyclictest1839978-21kworker/2:007:39:122
1853158994944,3cyclictest64099-1kworker/2:1H10:23:022
1853152994944,3cyclictest3041382-1kworker/0:1H12:25:060
1853152994944,3cyclictest3041382-1kworker/0:1H10:55:280
1853152994944,3cyclictest3041382-1kworker/0:1H08:50:230
1853162994845,2cyclictest3209943-1kworker/3:2H10:32:373
1853162994845,2cyclictest3209943-1kworker/3:2H08:39:033
1853162994845,2cyclictest1875443-21kworker/3:009:02:373
1853162994845,2cyclictest1839931-21kworker/3:007:54:083
1853162994844,3cyclictest1894999-21kworker/3:209:35:013
1853158994845,2cyclictest1911472-21kworker/2:010:27:032
1853158994844,3cyclictest1911472-21kworker/2:010:52:372
1853158994844,2cyclictest1861271-21kworker/2:108:07:372
1853158994843,3cyclictest64099-1kworker/2:1H10:40:062
1853158994843,3cyclictest1911472-21kworker/2:009:54:012
1853158994843,3cyclictest1861271-21kworker/2:107:45:372
1853158994843,3cyclictest1839978-21kworker/2:007:14:372
1853152994844,2cyclictest1996901-21kworker/0:111:45:290
1853162994742,3cyclictest1875443-21kworker/3:009:15:003
1853158994744,2cyclictest1911472-21kworker/2:011:30:052
1853158994743,3cyclictest1861271-21kworker/2:107:53:022
1853158994742,3cyclictest2031841-21kworker/2:112:20:282
1853158994742,3cyclictest1911472-21kworker/2:011:46:072
1853158994742,3cyclictest1880003-21kworker/2:208:51:362
1853158994742,3cyclictest1880003-21kworker/2:208:39:332
1853152994743,3cyclictest3041382-1kworker/0:1H12:05:470
1853152994740,4cyclictest3041382-1kworker/0:1H09:40:210
1853162994643,2cyclictest1875443-21kworker/3:008:21:033
1853162994642,3cyclictest1894999-21kworker/3:210:17:323
1853162994642,3cyclictest1894999-21kworker/3:210:17:323
1853162994642,2cyclictest3209943-1kworker/3:2H07:45:283
1853162994641,3cyclictest1894999-21kworker/3:211:06:363
1853158994642,2cyclictest64099-1kworker/2:1H10:40:002
1853158994642,2cyclictest1861271-21kworker/2:108:11:432
1853158994641,3cyclictest1839978-21kworker/2:007:31:352
1853158994640,4cyclictest1880003-21kworker/2:209:04:442
1853158994638,5cyclictest64099-1kworker/2:1H11:15:222
1853152994643,2cyclictest1940750-21kworker/0:310:52:020
1853152994643,2cyclictest1889529-21kworker/0:309:07:070
1853152994643,2cyclictest1865866-21kworker/0:007:49:530
1853152994642,3cyclictest3041382-1kworker/0:1H10:38:360
1853152994642,2cyclictest3041382-1kworker/0:1H12:00:220
1853152994641,3cyclictest3041382-1kworker/0:1H11:15:170
1853152994641,3cyclictest3041382-1kworker/0:1H10:05:280
1853162994542,2cyclictest3209943-1kworker/3:2H08:54:423
1853162994542,2cyclictest3209943-1kworker/3:2H07:13:123
1853162994541,3cyclictest2044028-21kworker/3:012:25:003
1853162994541,2cyclictest1894999-21kworker/3:209:20:003
1853158994540,4cyclictest1880003-21kworker/2:208:46:372
1853158994539,4cyclictest2031841-21kworker/2:112:18:362
1853152994543,1cyclictest1848283-21kworker/0:208:20:270
1853152994542,2cyclictest1940750-21kworker/0:310:24:550
1853152994542,2cyclictest1848283-21kworker/0:208:19:370
1853152994542,2cyclictest1848283-21kworker/0:207:52:080
1853152994539,3cyclictest3041382-1kworker/0:1H12:30:160
1853162994440,3cyclictest2025965-21kworker/3:112:00:373
1853162994440,2cyclictest1894999-21kworker/3:210:13:363
1853162994439,3cyclictest3209943-1kworker/3:2H09:50:013
1853162994439,3cyclictest3209943-1kworker/3:2H07:40:213
1853158994439,3cyclictest1880003-21kworker/2:208:55:172
1853158994439,3cyclictest1839978-21kworker/2:007:18:372
1853158994438,4cyclictest64099-1kworker/2:1H11:44:592
1853158994438,4cyclictest64099-1kworker/2:1H10:05:332
1853152994441,2cyclictest1854532-21kworker/0:307:19:540
1853152994441,2cyclictest1848283-21kworker/0:208:28:020
1853152994440,3cyclictest3041382-1kworker/0:1H11:40:170
1853152994440,3cyclictest3041382-1kworker/0:1H09:35:540
1853152994440,3cyclictest3041382-1kworker/0:1H09:30:110
1853152994440,2cyclictest2027023-21kworker/0:211:59:030
1853152994440,2cyclictest2027023-21kworker/0:211:59:020
1853162994339,3cyclictest1894999-21kworker/3:210:29:373
1853162994338,3cyclictest1894999-21kworker/3:211:33:363
1853162994338,2cyclictest2044028-21kworker/3:012:19:553
1853158994338,4cyclictest64099-1kworker/2:1H10:10:362
1853158994338,3cyclictest64099-1kworker/2:1H11:22:022
1853152994341,1cyclictest1854532-21kworker/0:307:43:080
1853152994340,2cyclictest1923420-21kworker/0:210:15:170
1853152994340,2cyclictest1923420-21kworker/0:210:15:170
1853152994339,3cyclictest3041382-1kworker/0:1H09:11:090
1853152994339,2cyclictest3041382-1kworker/0:1H12:20:230
1853152994338,2cyclictest3041382-1kworker/0:1H11:20:210
1853162994239,2cyclictest3209943-1kworker/3:2H12:31:293
1853162994239,2cyclictest1839931-21kworker/3:007:29:513
1853162994239,2cyclictest1839931-21kworker/3:007:15:283
1853162994238,3cyclictest3209943-1kworker/3:2H09:44:553
1853162994237,3cyclictest3209943-1kworker/3:2H08:29:393
1853158994237,3cyclictest64099-1kworker/2:1H09:35:072
1853158994237,3cyclictest1911472-21kworker/2:010:47:022
1853158994236,4cyclictest64099-1kworker/2:1H08:30:002
1853152994239,2cyclictest1940750-21kworker/0:311:29:010
1853152994239,2cyclictest1867406-21kworker/0:108:02:020
1853152994238,3cyclictest3041382-1kworker/0:1H10:31:370
1853152994238,2cyclictest1867406-21kworker/0:108:35:040
1853152994236,4cyclictest3041382-1kworker/0:1H07:20:230
1853162994138,2cyclictest2025965-21kworker/3:112:14:553
1853162994138,2cyclictest2025965-21kworker/3:111:58:073
1853162994138,2cyclictest2025965-21kworker/3:111:58:073
1853162994136,3cyclictest3209943-1kworker/3:2H09:59:553
1853162994136,2cyclictest3209943-1kworker/3:2H11:40:233
1853162994135,3cyclictest3209943-1kworker/3:2H12:35:243
1853158994138,2cyclictest64099-1kworker/2:1H11:05:342
1853158994137,3cyclictest1880003-21kworker/2:209:26:422
1853158994136,3cyclictest1911472-21kworker/2:009:40:592
1853158994134,4cyclictest64099-1kworker/2:1H10:30:192
1853152994138,2cyclictest1964593-21kworker/0:110:41:360
1853152994138,2cyclictest1923420-21kworker/0:210:29:080
1853152994138,2cyclictest1848283-21kworker/0:208:12:360
1853152994137,3cyclictest3041382-1kworker/0:1H08:49:030
1853152994136,3cyclictest3041382-1kworker/0:1H08:05:260
1853162994037,2cyclictest3209943-1kworker/3:2H08:43:433
1853162994036,3cyclictest1894999-21kworker/3:210:55:533
1853162994036,2cyclictest3209943-1kworker/3:2H10:35:223
1853162994036,2cyclictest3209943-1kworker/3:2H07:20:053
1853158994037,2cyclictest2031841-21kworker/2:112:25:232
1853158994035,3cyclictest64099-1kworker/2:1H09:59:562
1853152994037,2cyclictest3041382-1kworker/0:1H10:47:370
1853152994037,2cyclictest3041382-1kworker/0:1H09:00:330
1853152994037,2cyclictest3041382-1kworker/0:1H08:43:370
1853162993936,2cyclictest3209943-1kworker/3:2H08:34:553
1853162993935,3cyclictest1894999-21kworker/3:210:54:373
1853162993935,2cyclictest1875443-21kworker/3:008:57:073
1853158993935,3cyclictest64099-1kworker/2:1H09:12:372
1853158993935,2cyclictest64099-1kworker/2:1H09:30:222
1853158993934,3cyclictest64099-1kworker/2:1H08:40:052
1853152993935,2cyclictest1867406-21kworker/0:107:59:250
1853162993833,3cyclictest3209943-1kworker/3:2H11:50:003
1853162993833,3cyclictest3209943-1kworker/3:2H08:15:243
1853162993833,3cyclictest1894999-21kworker/3:211:15:273
1853152993835,2cyclictest3041382-1kworker/0:1H07:35:180
1853152993835,2cyclictest1923420-21kworker/0:209:55:200
1853152993835,2cyclictest1892686-21kworker/0:009:25:000
1853152993834,2cyclictest3041382-1kworker/0:1H11:05:220
1853162993734,2cyclictest3209943-1kworker/3:2H08:12:423
1853162993734,2cyclictest1894999-21kworker/3:210:49:373
1853162993733,2cyclictest1894999-21kworker/3:211:12:373
1853158993733,3cyclictest2025914-21kworker/2:211:57:012
1853158993733,3cyclictest2025914-21kworker/2:211:57:012
1853158993733,2cyclictest64099-1kworker/2:1H12:00:012
1853152993734,2cyclictest3041382-1kworker/0:1H08:30:590
1853152993733,3cyclictest3041382-1kworker/0:1H12:16:370
1853152993733,3cyclictest3041382-1kworker/0:1H12:10:110
1853162993633,2cyclictest1894999-21kworker/3:210:40:543
1853162993532,2cyclictest3209943-1kworker/3:2H11:00:023
1853162993532,2cyclictest2044028-21kworker/3:012:29:563
1853162993531,2cyclictest2025965-21kworker/3:112:09:073
1853158993532,2cyclictest64099-1kworker/2:1H11:50:432
1853158993532,2cyclictest1911472-21kworker/2:009:49:022
1853158993530,3cyclictest2031841-21kworker/2:112:30:322
1853152993530,3cyclictest3041382-1kworker/0:1H09:45:230
1853162993431,2cyclictest3209943-1kworker/3:2H08:45:413
1853162993431,2cyclictest3209943-1kworker/3:2H07:32:073
1853162993431,2cyclictest1867474-21kworker/3:108:04:333
1853162993430,3cyclictest1894999-21kworker/3:211:24:563
1853162993429,3cyclictest3209943-1kworker/3:2H09:27:023
1853158993432,1cyclictest1861271-21kworker/2:108:15:292
1853158993431,2cyclictest1911472-21kworker/2:010:00:592
1853158993430,3cyclictest64099-1kworker/2:1H09:15:332
1853162993329,2cyclictest3209943-1kworker/3:2H08:05:253
1853152993330,2cyclictest3041382-1kworker/0:1H07:25:580
1853152993329,2cyclictest3041382-1kworker/0:1H09:15:190
1853162993228,2cyclictest3209943-1kworker/3:2H10:04:513
1853152993228,2cyclictest3041382-1kworker/0:1H10:00:240
1853152993227,3cyclictest3041382-1kworker/0:1H09:25:040
1853152993227,3cyclictest3041382-1kworker/0:1H07:30:180
1853162993128,2cyclictest3209943-1kworker/3:2H10:20:253
1853162993127,3cyclictest1894999-21kworker/3:210:05:543
1853158993128,2cyclictest64099-1kworker/2:1H12:14:022
1853158993027,2cyclictest1911472-21kworker/2:010:15:212
1853158993027,2cyclictest1911472-21kworker/2:010:15:212
185315599300,28cyclictest0-21swapper/107:09:591
1853152993026,3cyclictest3041382-1kworker/0:1H11:03:360
1853162992926,2cyclictest1894999-21kworker/3:211:29:363
1853162992925,3cyclictest1875443-21kworker/3:009:09:553
1853152992824,3cyclictest3041382-1kworker/0:1H12:39:340
1853158992719,2cyclictest151rcu_preempt11:10:242
1853152992721,5cyclictest2009899-21kworker/u8:111:39:370
1853162992623,2cyclictest1894999-21kworker/3:209:21:043
185315299261,23cyclictest1941934-21/usr/sbin/munin10:10:240
185315299260,23cyclictest1887200-21/usr/sbin/munin08:55:260
185316299250,23cyclictest2025220-21grep11:50:313
1853152992522,2cyclictest1917617-21kworker/u8:009:51:330
185315299251,21cyclictest1890845-21diskmemload11:11:060
185315299241,3cyclictest1853269-21apt-get07:10:190
185315899100,3cyclictest1853198-21latency_hist07:09:592
18531629990,6cyclictest1853170-21cat07:10:003
18531529980,2cyclictest151rcu_preempt07:09:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional