You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-25 - 12:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack0slot7s.osadl.org (updated Mon Sep 25, 2023 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
548683210666,17sleep00-21swapper/019:09:520
548641210175,20sleep20-21swapper/219:09:222
548562210069,19sleep10-21swapper/119:08:211
54849129974,19sleep30-21swapper/319:07:283
548862996362,1cyclictest1036-21snmpd19:40:361
548867996160,1cyclictest582084-21sensors20:25:262
548867996160,1cyclictest0-21swapper/223:31:152
548867996153,4cyclictest755926-21sensors00:15:302
548862996059,1cyclictest1036-21snmpd23:16:131
548869995857,1cyclictest0-21swapper/321:55:283
548867995856,1cyclictest649487-21kworker/2:022:35:392
548867995856,1cyclictest649487-21kworker/2:022:35:392
548862995857,1cyclictest0-21swapper/122:15:271
548858995857,1cyclictest709888-21sensors23:20:330
54886299570,57cyclictest0-21swapper/121:05:311
54886299570,1cyclictest0-21swapper/119:15:301
54885899570,56cyclictest0-21swapper/019:45:280
548869995654,1cyclictest1036-21snmpd00:01:193
548869995554,1cyclictest626190-21sensors21:40:273
548862995554,1cyclictest557031-21sensors19:25:271
548862995554,1cyclictest0-21swapper/122:55:271
548858995554,1cyclictest0-21swapper/019:40:320
548869995453,1cyclictest726560-21sensors23:40:263
548862995453,1cyclictest714753-21ssh23:26:141
548858995453,1cyclictest772665-21sensors00:35:300
548858995453,1cyclictest0-21swapper/000:15:300
548869995352,1cyclictest0-21swapper/320:00:383
548862995352,1cyclictest0-21swapper/119:10:231
548858995251,1cyclictest0-21swapper/019:20:280
548869995150,1cyclictest0-21swapper/321:25:543
548862995150,1cyclictest0-21swapper/122:30:341
548862995150,1cyclictest0-21swapper/119:30:261
548869995049,1cyclictest0-21swapper/319:25:273
548869994948,1cyclictest592519-21sensors20:50:303
548862994948,1cyclictest0-21swapper/121:40:281
548862994948,1cyclictest0-21swapper/121:20:311
548858994948,1cyclictest0-21swapper/022:40:270
548869994847,1cyclictest0-21swapper/322:30:343
548869994847,1cyclictest0-21swapper/322:20:303
548862994847,1cyclictest450880-21kworker/u8:220:15:011
548858994847,1cyclictest638736-21sensors21:55:280
548869994746,1cyclictest0-21swapper/322:11:023
548869994746,1cyclictest0-21swapper/300:30:303
548867994746,1cyclictest584168-21sensors20:30:282
548867994746,1cyclictest0-21swapper/223:20:332
548862994746,1cyclictest567455-21sensors19:50:271
548858994746,1cyclictest743445-21sensors00:00:390
548862994645,1cyclictest0-21swapper/122:50:301
548862994645,1cyclictest0-21swapper/120:00:271
548858994544,1cyclictest0-21swapper/023:30:260
548869994443,1cyclictest0-21swapper/319:50:363
548858994443,1cyclictest609449-21ssh21:20:310
548862994342,1cyclictest561200-21sensors19:35:281
548862994241,1cyclictest0-21swapper/121:35:271
548862994241,1cyclictest0-21swapper/100:16:221
548858994241,1cyclictest0-21swapper/023:15:270
548869994140,1cyclictest0-21swapper/319:10:233
548867994140,1cyclictest0-21swapper/223:35:252
548867994140,1cyclictest0-21swapper/219:30:272
548867994140,1cyclictest0-21swapper/200:30:302
548862994140,1cyclictest0-21swapper/122:05:271
548858994140,1cyclictest0-21swapper/020:40:280
548869994039,1cyclictest1036-21snmpd22:46:083
548862994039,1cyclictest760084-21sensors00:20:271
548858994039,1cyclictest0-21swapper/021:37:370
548858993938,1cyclictest0-21swapper/023:35:250
548867993837,1cyclictest622001-21sensors21:35:282
548858993837,1cyclictest0-21swapper/000:10:390
548867993736,1cyclictest0-21swapper/223:06:122
548867993736,1cyclictest0-21swapper/222:40:262
548862993736,1cyclictest0-21swapper/100:25:291
548858993635,1cyclictest596691-21sensors21:00:260
548858993635,1cyclictest0-21swapper/020:55:260
548858993635,1cyclictest0-21swapper/019:30:270
548867993534,1cyclictest0-21swapper/223:45:272
548862993510,25cyclictest0-21swapper/119:21:171
548858993534,1cyclictest0-21swapper/022:20:290
548858993534,1cyclictest0-21swapper/019:10:230
548869993433,1cyclictest693043-21sensors23:00:243
165999340,3ntpd33-21ksoftirqd/219:35:282
548869993332,1cyclictest565374-21sensors19:45:283
548869993332,1cyclictest1036-21snmpd21:05:503
548869993332,1cyclictest0-21swapper/320:10:273
548867993332,1cyclictest0-21swapper/223:00:232
548862993331,2cyclictest743256-40dnf00:00:391
548858993332,1cyclictest684728-21sensors22:50:300
548869993231,1cyclictest739127-21sensors23:55:263
548869993231,1cyclictest0-21swapper/320:20:423
548869993231,1cyclictest0-21swapper/300:36:273
548858993231,1cyclictest0-21swapper/000:05:290
548869993130,1cyclictest0-21swapper/319:40:323
548862993028,2cyclictest0-21swapper/121:25:281
548862993028,1cyclictest680523-21sensors22:45:271
165999300,2ntpd33-21ksoftirqd/220:10:302
548869992928,1cyclictest0-21swapper/320:25:273
548858992928,1cyclictest714029-21sensors23:25:280
548869992827,1cyclictest709881-21sensors23:20:323
548869992827,1cyclictest0-21swapper/321:45:273
548858992827,1cyclictest0-21swapper/022:25:260
548858992827,1cyclictest0-21swapper/021:30:310
54885899280,4cyclictest0-21swapper/000:33:320
548869992726,1cyclictest0-21swapper/300:21:243
548858992726,1cyclictest0-21swapper/021:10:290
548858992726,1cyclictest0-21swapper/020:10:270
548858992726,0cyclictest0-21swapper/020:07:100
548869992625,1cyclictest0-21swapper/323:05:283
548858992626,0cyclictest0-21swapper/023:05:520
548862992524,1cyclictest0-21swapper/120:05:281
54885899250,24cyclictest0-21swapper/020:39:230
548862992423,1cyclictest601067-21sensors21:10:291
548862992423,1cyclictest0-21swapper/123:50:271
548858992423,1cyclictest0-21swapper/020:15:260
548858992423,1cyclictest0-21swapper/000:25:290
548867992322,1cyclictest0-21swapper/222:08:382
548862992322,1cyclictest651264-21sensors22:10:281
165999230,2ntpd33-21ksoftirqd/223:41:292
54885899228,5cyclictest0-21swapper/023:58:110
54885899221,1cyclictest0-21swapper/023:03:150
165999220,3ntpd33-21ksoftirqd/223:59:152
548869992120,1cyclictest0-21swapper/319:15:263
548867992120,1cyclictest551660-21sensors19:15:012
548867992120,1cyclictest0-21swapper/223:50:282
548862992120,1cyclictest1036-21snmpd20:55:491
165999210,2ntpd33-21ksoftirqd/222:30:472
165999210,2ntpd33-21ksoftirqd/221:20:092
548867992019,1cyclictest550122-21sensors19:10:232
548867992019,1cyclictest0-21swapper/223:25:272
54886799200,9cyclictest0-21swapper/200:21:092
548858992019,1cyclictest0-21swapper/022:15:270
548858992019,1cyclictest0-21swapper/021:45:170
548858992018,1cyclictest1031-21runrttasks00:20:050
54886999190,4cyclictest0-21swapper/323:25:333
54886799190,4cyclictest0-21swapper/220:44:072
54886799190,4cyclictest0-21swapper/200:08:412
548862991918,1cyclictest672196-21sensors22:35:311
548862991918,1cyclictest672196-21sensors22:35:301
54886299190,4cyclictest0-21swapper/100:34:141
548858991919,0cyclictest0-21swapper/020:31:140
548858991918,1cyclictest0-21swapper/023:13:120
54885899190,4cyclictest0-21swapper/019:56:040
165999190,2ntpd33-21ksoftirqd/222:12:532
548869991817,1cyclictest617843-21sensors21:30:323
54886999180,17cyclictest0-21swapper/322:52:543
54886999180,17cyclictest0-21swapper/322:00:213
54886999180,17cyclictest0-21swapper/320:40:213
54886799180,3cyclictest0-21swapper/222:03:282
548862991814,1cyclictest0-21swapper/120:47:351
548858991814,1cyclictest0-21swapper/020:53:080
54885899180,3cyclictest0-21swapper/023:50:070
54885899180,3cyclictest0-21swapper/019:39:420
54885899180,3cyclictest0-21swapper/019:27:280
5794182170,0sleep20-21swapper/220:20:212
548869991716,1cyclictest655439-21sensors22:15:273
54886999170,17cyclictest0-21swapper/320:57:253
54886999170,17cyclictest0-21swapper/320:16:343
54886999170,16cyclictest0-21swapper/321:18:243
54886999170,16cyclictest0-21swapper/320:49:353
54886999170,16cyclictest0-21swapper/320:37:053
54886999170,16cyclictest0-21swapper/300:16:233
548867991716,1cyclictest0-21swapper/200:35:302
548867991715,1cyclictest1026-21nmbd21:25:312
54886799170,17cyclictest0-21swapper/222:23:142
54886799170,16cyclictest0-21swapper/220:45:412
54886799170,16cyclictest0-21swapper/220:16:162
54886799170,12cyclictest0-21swapper/221:11:592
548862991716,1cyclictest659639-21sensors22:20:301
548862991713,1cyclictest0-21swapper/120:52:411
548862991712,1cyclictest0-21swapper/123:58:011
548858991716,1cyclictest0-21swapper/021:40:040
548858991713,1cyclictest0-21swapper/023:49:100
54885899170,8cyclictest0-21swapper/021:05:150
54885899170,16cyclictest0-21swapper/022:10:050
54885899170,16cyclictest0-21swapper/021:51:280
54885899170,16cyclictest0-21swapper/019:19:420
548869991615,1cyclictest0-21swapper/322:26:053
54886999160,16cyclictest0-21swapper/323:35:353
54886999160,16cyclictest0-21swapper/323:32:133
54886999160,16cyclictest0-21swapper/321:10:353
548867991615,1cyclictest594600-21sensors20:55:252
548867991613,2cyclictest1036-21snmpd22:18:042
548867991612,1cyclictest0-21swapper/223:16:512
54886799160,7cyclictest0-21swapper/222:52:152
54886799160,16cyclictest0-21swapper/222:56:232
548862991615,1cyclictest1036-21snmpd23:44:581
54886299160,16cyclictest703957-21ssh23:14:451
54886299160,16cyclictest0-21swapper/123:39:201
54886299160,16cyclictest0-21swapper/123:32:401
54886299160,16cyclictest0-21swapper/123:20:371
548858991615,1cyclictest0-21swapper/020:45:340
54885899160,16cyclictest0-21swapper/022:58:000
54885899160,16cyclictest0-21swapper/022:49:470
54886999154,10cyclictest427-21plymouthd23:15:183
548869991513,1cyclictest1036-21snmpd21:51:133
548869991512,2cyclictest1036-21snmpd22:36:483
548869991512,2cyclictest1036-21snmpd22:36:483
548869991512,2cyclictest1036-21snmpd21:22:093
54886999150,6cyclictest0-21swapper/323:50:413
54886799150,7cyclictest0-21swapper/222:28:112
54886799150,6cyclictest0-21swapper/220:05:282
54886299150,7cyclictest0-21swapper/123:02:361
54886299150,7cyclictest0-21swapper/121:45:031
54886299150,7cyclictest0-21swapper/120:26:321
54886299150,15cyclictest0-21swapper/122:29:051
548869991413,1cyclictest0-21swapper/320:05:283
54886999140,6cyclictest0-21swapper/322:58:353
548862991413,1cyclictest1036-21snmpd21:50:581
548862991413,1cyclictest0-21swapper/120:35:451
54886299140,6cyclictest0-21swapper/100:12:561
54886799139,1cyclictest0-21swapper/222:47:092
54886799133,10cyclictest0-21swapper/221:48:212
548867991310,2cyclictest1036-21snmpd21:34:062
54886799130,5cyclictest0-21swapper/200:03:152
548862991312,1cyclictest0-21swapper/122:40:241
54886299130,13cyclictest0-21swapper/121:00:251
54885899133,9cyclictest0-21swapper/020:23:220
165999130,5ntpd0-21swapper/023:43:490
54886999129,2cyclictest1036-21snmpd23:46:493
54886999120,12cyclictest0-21swapper/319:35:283
54886799129,2cyclictest1036-21snmpd00:11:172
54886799122,4cyclictest591985-21fschecks_time20:50:232
548867991210,2cyclictest1036-21snmpd00:26:592
54886799120,12cyclictest0-21swapper/221:15:332
54886299129,2cyclictest564938-21cat19:45:221
54886299129,2cyclictest1036-21snmpd00:35:591
54886299123,9cyclictest0-21swapper/122:00:271
54885899129,2cyclictest1036-21snmpd21:29:090
548858991210,1cyclictest1036-21snmpd19:50:580
54885899120,12cyclictest0-21swapper/022:35:270
54885899120,12cyclictest0-21swapper/022:35:260
165999120,8ntpd571455-21ntpq20:00:252
54886999119,2cyclictest1036-21snmpd21:35:493
54886999110,1cyclictest0-21swapper/300:05:243
54886999110,10cyclictest0-21swapper/322:40:193
54886799119,1cyclictest1036-21snmpd23:10:452
54886799119,1cyclictest1036-21snmpd21:44:582
54886799118,2cyclictest1036-21snmpd19:45:222
54886299117,1cyclictest0-21swapper/120:40:291
548862991111,0cyclictest1036-21snmpd21:58:161
54886299110,11cyclictest0-21swapper/119:55:201
54885899119,1cyclictest6434262sleep022:00:490
54885899119,1cyclictest1036-21snmpd20:00:040
54886999109,1cyclictest1036-21snmpd21:03:063
54886999108,2cyclictest1036-21snmpd23:11:213
54886999108,2cyclictest1036-21snmpd19:21:003
54886999107,2cyclictest569104-21cat19:55:183
54886799108,2cyclictest1036-21snmpd21:09:392
54886799108,1cyclictest1036-21snmpd19:56:192
54886799108,1cyclictest1036-21snmpd19:27:212
54886799100,9cyclictest0-21swapper/220:35:242
54886799100,10cyclictest0-21swapper/221:54:102
54886299107,2cyclictest1036-21snmpd23:07:281
54886299107,2cyclictest1036-21snmpd20:20:131
54886299100,10cyclictest604759-21ssh21:15:201
54886299100,10cyclictest0-21swapper/123:49:221
54886299100,10cyclictest0-21swapper/121:33:591
54885899109,1cyclictest1036-21snmpd22:05:560
54885899107,2cyclictest1036-21snmpd22:30:030
54885899100,9cyclictest0-21swapper/021:16:230
165999100,1ntpd33-21ksoftirqd/221:55:432
5488699997,1cyclictest1036-21snmpd22:06:083
5488699996,2cyclictest1036-21snmpd00:10:293
5488679990,9cyclictest0-21swapper/221:01:112
5488679990,9cyclictest0-21swapper/219:41:512
5488629997,1cyclictest1036-21snmpd20:10:031
5488629990,9cyclictest0-21swapper/120:31:411
5488629990,1cyclictest0-21swapper/100:05:241
5488699986,2cyclictest1036-21snmpd20:31:563
5488699985,2cyclictest1036-21snmpd19:32:513
16599980,1ntpd567302-21ntpq19:50:252
16599980,0ntpd0-21swapper/219:20:252
16599980,0ntpd0-21swapper/020:25:240
5488699973,2cyclictest1038-21lldpd00:25:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional