You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-19 - 04:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3.osadl.org (updated Fri Apr 19, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3071996997250,21cyclictest3068997-21kworker/1:219:19:591
3071996997241,29cyclictest712879-1kworker/1:1H19:37:071
3071996996937,30cyclictest3230697-21kworker/1:000:10:001
3071996996841,26cyclictest3198480-21kworker/1:223:34:571
3071996996537,27cyclictest712879-1kworker/1:1H22:15:151
3071996996447,16cyclictest3100288-21kworker/1:121:05:151
3071996996435,27cyclictest3137543-21kworker/1:222:20:131
3071996996339,23cyclictest3108608-21kworker/1:221:27:241
3071996996333,29cyclictest3137543-21kworker/1:221:50:151
3071996996233,28cyclictest3257154-21kworker/1:200:27:571
3071996996233,27cyclictest712879-1kworker/1:1H20:15:141
3071996996143,17cyclictest712879-1kworker/1:1H20:14:571
3071996996142,18cyclictest3257154-21kworker/1:200:35:131
3071996996141,18cyclictest3230697-21kworker/1:000:00:231
3071996996141,18cyclictest3198480-21kworker/1:223:40:131
3071996996043,16cyclictest3179117-21kworker/1:222:40:001
3071996996043,16cyclictest3100288-21kworker/1:120:42:581
3071996996043,16cyclictest3097802-21kworker/1:220:33:291
3071996996041,17cyclictest3198480-21kworker/1:223:30:011
3071996996032,27cyclictest3086403-21kworker/1:020:20:151
3071996996031,27cyclictest3230697-21kworker/1:023:55:001
3071996995943,15cyclictest712879-1kworker/1:1H20:05:291
3071996995943,15cyclictest3257154-21kworker/1:200:20:341
3071996995943,15cyclictest3198480-21kworker/1:223:45:151
3071996995943,15cyclictest3198480-21kworker/1:223:07:011
3071996995943,15cyclictest3191809-21kworker/1:122:55:141
3071996995943,15cyclictest3100288-21kworker/1:121:00:031
3071996995943,15cyclictest3086403-21kworker/1:020:02:111
3071996995943,15cyclictest3068997-21kworker/1:219:20:031
3071996995942,16cyclictest712879-1kworker/1:1H21:47:021
3071996995942,16cyclictest3129290-21kworker/1:021:37:591
3071996995942,16cyclictest3100288-21kworker/1:121:00:001
3071996995942,16cyclictest3097802-21kworker/1:220:35:271
3071996995942,15cyclictest3100288-21kworker/1:120:45:271
3071996995941,17cyclictest3068997-21kworker/1:219:25:231
3071996995931,26cyclictest3198480-21kworker/1:223:15:161
3071996995930,27cyclictest3068997-21kworker/1:219:50:141
3071996995842,15cyclictest3257154-21kworker/1:200:32:581
3071996995842,15cyclictest3230697-21kworker/1:023:55:141
3071996995842,15cyclictest3137543-21kworker/1:222:00:001
3071996995842,15cyclictest3086403-21kworker/1:020:25:131
3071996995842,15cyclictest3068997-21kworker/1:219:47:071
3071996995841,16cyclictest712879-1kworker/1:1H22:10:261
3071996995841,16cyclictest3198480-21kworker/1:223:10:351
3071996995841,16cyclictest3187487-21kworker/1:022:50:141
3071996995840,17cyclictest3230697-21kworker/1:000:15:291
3071996995840,17cyclictest3230697-21kworker/1:000:15:281
3071996995831,26cyclictest3191809-21kworker/1:123:00:121
3071996995741,15cyclictest3137543-21kworker/1:222:25:001
3071996995740,16cyclictest3198480-21kworker/1:223:22:291
3071996995740,16cyclictest3108608-21kworker/1:221:16:021
3071996995740,16cyclictest3108608-21kworker/1:221:16:011
3071996995718,36cyclictest3137543-21kworker/1:222:00:011
3071996995640,15cyclictest3170445-21kworker/1:022:37:361
3071996995640,15cyclictest3170445-21kworker/1:022:32:361
3071996995640,15cyclictest3133688-21kworker/1:121:40:131
3071996995640,15cyclictest3068997-21kworker/1:219:40:001
3071996995640,15cyclictest3068997-21kworker/1:219:30:151
3071996995640,15cyclictest3068997-21kworker/1:219:10:151
3071996995637,18cyclictest3108608-21kworker/1:221:30:131
3071996995628,27cyclictest712879-1kworker/1:1H22:45:211
3071996995552,2cyclictest0-21swapper/121:13:291
3071996995539,15cyclictest3086403-21kworker/1:019:58:581
307199699550,54cyclictest0-21swapper/120:51:051
307199699550,20cyclictest3198480-21kworker/1:223:35:091
3071991995550,3cyclictest3125440-21kworker/0:221:41:260
3071996995438,15cyclictest3137543-21kworker/1:222:09:561
3071996995438,15cyclictest3108608-21kworker/1:221:20:591
3071991995450,3cyclictest3081766-21kworker/0:120:30:590
3072006995348,3cyclictest3297998-1kworker/3:0H20:45:163
3072006995348,3cyclictest3089553-21kworker/3:120:12:583
3071996995337,15cyclictest3230697-21kworker/1:000:10:151
3072006995247,3cyclictest3297998-1kworker/3:0H22:44:583
3071991995249,2cyclictest3205660-21kworker/0:023:38:290
3072006995147,3cyclictest3113518-21kworker/3:222:00:233
3072006995147,3cyclictest3113518-21kworker/3:221:29:593
3072006995146,3cyclictest3297998-1kworker/3:0H21:32:233
3072006995146,3cyclictest3113518-21kworker/3:221:22:573
3072006995146,3cyclictest3067613-21kworker/3:220:00:583
3071991995147,2cyclictest3087931-21kworker/0:319:59:230
3072006995047,2cyclictest3166635-21kworker/3:022:30:503
3072006995047,2cyclictest3113518-21kworker/3:221:55:563
3072006995046,3cyclictest3297998-1kworker/3:0H23:41:563
3071991995047,2cyclictest3201754-21kworker/0:323:23:520
3071991995047,2cyclictest3116079-21kworker/0:122:04:210
3071991995047,2cyclictest1093705-1kworker/0:1H21:39:230
3071991995046,3cyclictest1093705-1kworker/0:1H22:51:240
3071991995046,2cyclictest3125440-21kworker/0:223:05:250
3071991995045,3cyclictest1093705-1kworker/0:1H22:06:540
3072006994946,2cyclictest3258095-21kworker/3:200:26:203
3072006994946,2cyclictest3089553-21kworker/3:120:40:343
3072006994945,2cyclictest3089553-21kworker/3:121:01:573
3072006994945,2cyclictest3067613-21kworker/3:219:22:183
3072006994944,3cyclictest3297998-1kworker/3:0H23:35:093
3072006994943,4cyclictest3297998-1kworker/3:0H22:18:573
3072000994944,3cyclictest1074229-1kworker/2:2H23:08:492
3071991994945,3cyclictest1093705-1kworker/0:1H20:25:190
3071991994944,3cyclictest1093705-1kworker/0:1H22:39:580
3071991994944,3cyclictest1093705-1kworker/0:1H22:15:240
3071991994944,3cyclictest1093705-1kworker/0:1H20:40:160
3072006994845,2cyclictest3297998-1kworker/3:0H23:50:233
3072006994845,2cyclictest3166635-21kworker/3:022:23:573
3072006994845,2cyclictest3067613-21kworker/3:219:11:153
3072006994843,3cyclictest3089553-21kworker/3:120:51:573
3071991994845,2cyclictest3097211-21kworker/0:221:15:580
3071991994845,2cyclictest3097211-21kworker/0:221:15:570
3071991994844,2cyclictest3236497-21kworker/0:300:18:580
3071991994844,2cyclictest3236497-21kworker/0:300:18:580
3071991994843,3cyclictest3097211-21kworker/0:221:02:580
3071991994843,3cyclictest3077148-21kworker/0:019:38:170
3072006994744,2cyclictest3258095-21kworker/3:200:36:583
3072006994744,2cyclictest3113518-21kworker/3:222:11:013
3072006994743,2cyclictest3166635-21kworker/3:023:02:243
3072006994742,3cyclictest3297998-1kworker/3:0H23:20:283
3071991994744,2cyclictest3125440-21kworker/0:221:45:340
3071991994744,2cyclictest3097211-21kworker/0:220:57:170
3071991994744,2cyclictest3071710-21kworker/0:219:46:160
3071991994743,3cyclictest1093705-1kworker/0:1H00:14:220
3071991994743,2cyclictest3071710-21kworker/0:219:29:030
3071991994742,3cyclictest3236497-21kworker/0:300:25:220
3071991994742,3cyclictest1093705-1kworker/0:1H23:48:170
3072006994643,2cyclictest3297998-1kworker/3:0H00:34:583
3072006994643,2cyclictest3166635-21kworker/3:023:32:463
3072006994643,2cyclictest3166635-21kworker/3:000:16:033
3072006994643,2cyclictest3166635-21kworker/3:000:16:033
3072006994642,3cyclictest3297998-1kworker/3:0H22:25:583
3072006994642,3cyclictest3297998-1kworker/3:0H21:15:213
3072006994642,3cyclictest3297998-1kworker/3:0H21:15:213
3072006994641,3cyclictest3297998-1kworker/3:0H22:51:563
3071991994643,2cyclictest3097211-21kworker/0:220:54:270
3071991994641,3cyclictest1093705-1kworker/0:1H23:27:460
3072006994542,2cyclictest3166635-21kworker/3:023:55:053
3072006994542,2cyclictest3166635-21kworker/3:023:25:283
3072006994542,2cyclictest3166635-21kworker/3:000:10:583
3072006994542,2cyclictest3166635-21kworker/3:000:03:223
3072006994541,3cyclictest3297998-1kworker/3:0H22:05:233
3072006994541,3cyclictest3166635-21kworker/3:022:37:433
3072006994541,3cyclictest3067613-21kworker/3:219:51:203
3072006994540,3cyclictest3297998-1kworker/3:0H23:09:553
3072006994540,3cyclictest3297998-1kworker/3:0H20:30:113
3072000994542,2cyclictest1028066-21kworker/2:019:52:082
3072000994540,3cyclictest1074229-1kworker/2:2H22:45:132
3071991994543,1cyclictest3256772-21kworker/0:100:30:040
3071991994542,2cyclictest3071710-21kworker/0:219:33:580
3071991994542,2cyclictest3071710-21kworker/0:219:16:270
3071991994541,3cyclictest1093705-1kworker/0:1H22:27:230
3071991994541,3cyclictest1093705-1kworker/0:1H21:59:580
3071991994541,3cyclictest1093705-1kworker/0:1H21:25:270
3071991994540,3cyclictest1093705-1kworker/0:1H22:40:160
3072006994441,2cyclictest3089553-21kworker/3:120:55:433
3072006994441,2cyclictest3067613-21kworker/3:219:19:223
3072006994440,3cyclictest3297998-1kworker/3:0H23:45:083
3071991994441,2cyclictest3081766-21kworker/0:119:52:150
3071991994440,3cyclictest1093705-1kworker/0:1H21:10:300
3071991994439,3cyclictest3201754-21kworker/0:323:13:280
3072006994339,2cyclictest3089553-21kworker/3:121:06:003
3072000994340,2cyclictest1028066-21kworker/2:020:36:572
3072000994339,3cyclictest1074229-1kworker/2:2H20:00:002
3071991994340,2cyclictest3201754-21kworker/0:323:41:300
3071991994340,2cyclictest3162693-21kworker/0:023:02:290
3071991994340,2cyclictest1093705-1kworker/0:1H22:20:240
3072006994239,2cyclictest3113518-21kworker/3:221:47:333
3072006994239,2cyclictest3089553-21kworker/3:121:11:203
3072006994239,2cyclictest3089553-21kworker/3:120:07:153
3072006994239,2cyclictest3067613-21kworker/3:219:57:033
3072006994238,3cyclictest3297998-1kworker/3:0H21:37:553
3072006994238,3cyclictest3067613-21kworker/3:219:48:583
3072006994238,2cyclictest3297998-1kworker/3:0H20:35:153
3072000994239,2cyclictest3226825-21kworker/2:123:49:262
3072000994239,2cyclictest1074229-1kworker/2:2H20:45:022
3072000994236,4cyclictest1074229-1kworker/2:2H23:55:152
3071991994239,2cyclictest3236497-21kworker/0:323:56:200
3071991994239,2cyclictest3236497-21kworker/0:300:07:570
3071991994238,2cyclictest3087931-21kworker/0:320:17:580
3071991994238,2cyclictest1093705-1kworker/0:1H22:55:140
3071991994238,2cyclictest1093705-1kworker/0:1H20:00:140
3072006994138,2cyclictest3089553-21kworker/3:120:27:193
3072006994138,2cyclictest3089553-21kworker/3:120:21:573
3072006994137,3cyclictest3297998-1kworker/3:0H21:50:153
3072006994137,3cyclictest3297998-1kworker/3:0H00:21:103
3072000994138,2cyclictest1074229-1kworker/2:2H22:50:152
3072000994138,2cyclictest1074229-1kworker/2:2H00:30:352
3072000994137,2cyclictest1074229-1kworker/2:2H22:09:272
3072000994137,2cyclictest1074229-1kworker/2:2H19:12:052
3071991994137,2cyclictest3081766-21kworker/0:120:37:270
3071991994137,2cyclictest1093705-1kworker/0:1H23:17:450
3071991994137,2cyclictest1093705-1kworker/0:1H20:45:170
3072006994036,3cyclictest3297998-1kworker/3:0H21:40:393
3072006994036,3cyclictest3297998-1kworker/3:0H20:15:163
3072006994035,3cyclictest3297998-1kworker/3:0H22:46:583
3071991994037,2cyclictest1093705-1kworker/0:1H22:14:570
3072006993936,2cyclictest3067613-21kworker/3:219:28:223
3072000993936,2cyclictest1074229-1kworker/2:2H00:15:172
3072000993936,2cyclictest1074229-1kworker/2:2H00:15:162
3072000993936,2cyclictest1074229-1kworker/2:2H00:02:292
3072000993936,2cyclictest1028066-21kworker/2:021:53:572
3072000993936,2cyclictest1028066-21kworker/2:020:20:142
3072000993935,2cyclictest1074229-1kworker/2:2H21:49:262
3071991993936,2cyclictest3236497-21kworker/0:300:04:280
3071991993936,2cyclictest3081766-21kworker/0:120:06:330
3071991993936,2cyclictest3071710-21kworker/0:219:40:090
3071991993935,2cyclictest1093705-1kworker/0:1H19:10:150
3071991993836,1cyclictest3097211-21kworker/0:221:05:150
3071991993835,2cyclictest3081766-21kworker/0:120:20:380
3071991993834,2cyclictest1093705-1kworker/0:1H19:20:140
3072006993733,3cyclictest3297998-1kworker/3:0H22:55:023
3072006993733,3cyclictest3166635-21kworker/3:023:12:493
3072000993734,2cyclictest3180227-21kworker/2:223:37:522
3072000993734,2cyclictest1074229-1kworker/2:2H21:57:572
3072000993734,2cyclictest1028066-21kworker/2:021:38:042
3072000993733,2cyclictest1074229-1kworker/2:2H20:16:572
3072000993732,3cyclictest1074229-1kworker/2:2H21:40:132
3071991993734,2cyclictest51950irq/161-enp0s2921:30:130
3071991993734,2cyclictest3162693-21kworker/0:022:30:440
3071991993732,3cyclictest1093705-1kworker/0:1H23:30:340
3072006993631,3cyclictest3297998-1kworker/3:0H19:34:233
3072000993633,2cyclictest1074229-1kworker/2:2H23:03:572
3072000993632,2cyclictest1074229-1kworker/2:2H22:00:162
3072000993632,2cyclictest1028066-21kworker/2:021:07:202
307200099361,24cyclictest3080810-21cut19:35:142
3071991993633,2cyclictest3236497-21kworker/0:300:39:030
3071991993632,2cyclictest1093705-1kworker/0:1H23:55:000
3072006993532,2cyclictest3067613-21kworker/3:219:43:223
3072000993533,1cyclictest3254248-21kworker/2:100:35:212
3072000993532,2cyclictest1074229-1kworker/2:2H00:23:232
3072000993530,3cyclictest1074229-1kworker/2:2H19:30:162
3071991993530,3cyclictest3116079-21kworker/0:121:50:040
3072000993430,2cyclictest1074229-1kworker/2:2H19:25:132
307200099342,5cyclictest0-21swapper/223:15:482
3072000993420,4cyclictest3110345-21sh21:10:172
3071991993431,2cyclictest3087931-21kworker/0:320:14:220
3072006993329,3cyclictest3067613-21kworker/3:219:37:293
3071991993331,1cyclictest3236497-21kworker/0:300:24:570
3071991993328,3cyclictest1093705-1kworker/0:1H22:47:390
3072006993228,3cyclictest3297998-1kworker/3:0H23:19:543
3072006993228,3cyclictest3297998-1kworker/3:0H00:05:043
3072000993128,2cyclictest1074229-1kworker/2:2H00:10:132
3072000993127,2cyclictest1028066-21kworker/2:021:30:052
3072000993126,3cyclictest1074229-1kworker/2:2H23:12:442
3072000993121,5cyclictest0-21swapper/223:21:542
3072000993025,3cyclictest3096951-21grep20:25:192
307200099294,23cyclictest0-21swapper/222:31:382
3072000992927,2cyclictest33-21ksoftirqd/221:15:142
3072000992927,2cyclictest33-21ksoftirqd/221:15:132
3072000992923,4cyclictest0-21swapper/220:07:522
3072000992922,5cyclictest3266087-21munin-run00:30:002
307200099285,20cyclictest3108168-21cat21:05:002
3072000992824,2cyclictest1074229-1kworker/2:2H19:42:022
3072000992823,3cyclictest3226825-21kworker/2:123:50:222
3072000992823,3cyclictest3165149-21apt-get22:20:102
3072000992823,3cyclictest3104585-21swap20:50:192
3072000992823,3cyclictest3074848-21irqstats19:15:142
3072000992823,3cyclictest1028066-21kworker/2:020:15:012
3072000992822,5cyclictest0-21swapper/222:27:082
307200099280,4cyclictest0-21swapper/223:43:542
3071991992824,2cyclictest1093705-1kworker/0:1H21:25:010
307200099276,19cyclictest3088678-21idleruntime20:00:162
3072000992724,2cyclictest1074229-1kworker/2:2H21:20:582
3072000992723,2cyclictest3160969-21systemd-run22:15:002
3072000992723,2cyclictest0-21swapper/200:07:052
307200099270,6cyclictest3084393-21/usr/sbin/munin19:45:182
307200099270,24cyclictest3122971-21/usr/sbin/munin21:25:202
307200099264,20cyclictest3181627-21switchtime22:40:222
307200099263,21cyclictest3193324-21grep22:55:212
3072000992622,2cyclictest3077036-21/usr/sbin/munin19:20:232
3072000992621,2cyclictest3109586-21diskmemload22:15:462
3072000992618,5cyclictest3106490-21munin-plugin-st21:00:002
307200099253,11cyclictest3219130-21latency_hist23:30:012
3072000992521,2cyclictest3109586-21diskmemload22:39:282
3072000992518,4cyclictest3220671-21grep23:30:202
3072000992419,3cyclictest3098004-21/usr/sbin/munin20:30:152
307199699240,2cyclictest3072001-21apt-config19:10:001
3072000992318,4cyclictest3100866-21/usr/sbin/munin20:40:152
3072006992217,3cyclictest0-21swapper/319:10:003
307200099156,7cyclictest3071998-21cat19:10:002
307199199133,2cyclictest151rcu_preempt19:10:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional