You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-29 - 01:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Tue May 28, 2024 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28784210486,11sleep30-21swapper/307:05:303
28932210180,16sleep00-21swapper/007:07:070
28827210089,6sleep20-21swapper/207:05:452
2885129980,14sleep10-21swapper/107:06:041
28162770,0sleep10-21swapper/109:33:441
162862720,1sleep00-21swapper/009:47:580
88362350,1sleep38834-21ssh10:15:153
324052250,1sleep10-21swapper/110:40:251
29194992219,2cyclictest502-21kworker/u8:210:22:042
29194992217,4cyclictest14956-21kworker/u8:010:01:032
2919599212,2cyclictest0-21swapper/308:08:043
2919599211,8cyclictest0-21swapper/308:04:043
29195992118,2cyclictest6912-21kworker/u8:111:26:043
29195992118,2cyclictest2582-21kworker/u8:012:03:053
29195992118,2cyclictest14214-21kworker/u8:112:08:043
2919599210,4cyclictest0-21swapper/307:48:043
2918999211,18cyclictest0-21swapper/010:45:090
2918999210,19cyclictest0-21swapper/012:25:120
2919599202,2cyclictest0-21swapper/310:57:033
29195992017,2cyclictest7567-21kworker/u8:010:51:033
29195992017,2cyclictest658-21kworker/u8:210:45:043
29195992017,2cyclictest25357-21kworker/u8:107:14:033
29195992017,2cyclictest12234-21kworker/u8:010:25:023
29194992017,2cyclictest6912-21kworker/u8:111:36:042
29194992017,2cyclictest502-21kworker/u8:210:08:042
29194992017,2cyclictest24988-21kworker/u8:208:23:042
29195991916,2cyclictest20028-21kworker/u8:011:44:043
29195991916,2cyclictest11242-21kworker/u8:009:26:033
2919599190,2cyclictest0-21swapper/312:16:033
2919599190,14cyclictest0-21swapper/311:45:053
29194991916,2cyclictest658-21kworker/u8:210:47:042
29194991916,2cyclictest5778-21kworker/u8:110:16:032
29194991916,2cyclictest27822-21kworker/u8:110:54:042
29194991916,2cyclictest14214-21kworker/u8:112:15:032
29194991916,2cyclictest12234-21kworker/u8:010:26:042
2919499190,2cyclictest0-21swapper/210:57:032
2919499190,2cyclictest0-21swapper/208:08:042
2919499190,2cyclictest0-21swapper/207:48:042
29192991914,3cyclictest22531-21ssh09:20:111
29195991815,2cyclictest6729-21kworker/u8:107:42:043
29195991815,2cyclictest31431-21kworker/u8:207:20:043
29195991815,2cyclictest30075-21kworker/u8:211:51:043
29195991815,2cyclictest26468-21kworker/u8:109:48:043
29195991815,2cyclictest26468-21kworker/u8:109:38:043
29195991815,2cyclictest24988-21kworker/u8:208:37:033
29195991815,2cyclictest24988-21kworker/u8:208:28:043
29195991815,2cyclictest14956-21kworker/u8:010:04:043
29195991815,2cyclictest11242-21kworker/u8:009:17:043
2919599181,2cyclictest0-21swapper/311:16:033
2919599181,2cyclictest0-21swapper/307:25:043
2919599180,2cyclictest0-21swapper/311:03:033
2919599180,16cyclictest0-21swapper/309:34:043
29194991815,2cyclictest2582-21kworker/u8:012:00:042
29194991815,2cyclictest25133-21kworker/u8:111:50:042
29194991815,2cyclictest12481-21kworker/u8:211:31:042
29194991815,2cyclictest11242-21kworker/u8:009:27:032
29195991714,2cyclictest4508-21kworker/u8:008:48:043
29195991714,2cyclictest27822-21kworker/u8:110:39:043
29195991714,2cyclictest2228-21kworker/u8:012:39:053
29195991714,2cyclictest14214-21kworker/u8:112:25:043
29195991714,2cyclictest13262-21kworker/u8:008:14:033
29195991714,2cyclictest12234-21kworker/u8:010:32:043
29195991714,2cyclictest1175-21kworker/u8:108:41:043
2919599170,2cyclictest22457-21ssh09:20:043
2919599170,16cyclictest0-21swapper/309:41:043
29194991714,2cyclictest6730-21kworker/u8:307:58:042
29194991714,2cyclictest658-21kworker/u8:210:44:042
29194991714,2cyclictest5778-21kworker/u8:110:12:042
29194991714,2cyclictest17152-21kworker/u8:108:16:042
2919499170,2cyclictest13038-21diskmemload11:16:032
2919499170,2cyclictest0-21swapper/211:03:032
2918999173,2cyclictest101ktimersoftd/009:16:010
2918999170,15cyclictest0-21swapper/011:55:080
29195991613,2cyclictest25357-21kworker/u8:107:15:043
29195991613,2cyclictest12773-21kworker/u8:211:22:043
29195991613,2cyclictest12234-21kworker/u8:010:20:043
29195991613,2cyclictest11242-21kworker/u8:009:07:043
2919599160,2cyclictest13038-21diskmemload09:12:043
2919599160,14cyclictest18820-21ps09:50:213
29194991613,2cyclictest6730-21kworker/u8:307:52:042
29194991613,2cyclictest4508-21kworker/u8:008:52:042
29194991613,2cyclictest29491-21kworker/u8:212:26:042
29194991613,2cyclictest25357-21kworker/u8:107:12:042
29194991613,2cyclictest21048-21kworker/u8:012:23:042
29194991613,2cyclictest19626-21kworker/u8:209:23:042
29194991613,2cyclictest14956-21kworker/u8:009:47:042
29194991613,2cyclictest14214-21kworker/u8:112:33:042
29194991613,2cyclictest12234-21kworker/u8:010:30:042
2919499160,2cyclictest0-21swapper/207:25:042
2919299160,14cyclictest0-21swapper/110:35:121
2918999163,2cyclictest101ktimersoftd/009:52:340
2918999161,3cyclictest0-21swapper/011:20:220
29189991613,2cyclictest101ktimersoftd/010:23:240
29189991612,2cyclictest14349-21ntp_states09:10:240
2918999161,13cyclictest28964-21runrttasks10:36:480
29195991512,2cyclictest27481-21kworker/u8:008:31:033
29195991512,2cyclictest14956-21kworker/u8:009:55:043
29195991512,2cyclictest1175-21kworker/u8:109:00:033
29195991512,2cyclictest0-21swapper/311:35:313
2919599150,2cyclictest0-21swapper/308:20:043
2919599150,10cyclictest23782-21ssh11:06:043
29194991512,2cyclictest7341-21kworker/u8:209:02:032
29194991512,2cyclictest6912-21kworker/u8:111:27:042
29194991512,2cyclictest31431-21kworker/u8:207:19:032
29194991512,2cyclictest28058-21kworker/u8:111:12:042
29194991512,2cyclictest27822-21kworker/u8:111:05:042
29194991512,2cyclictest27481-21kworker/u8:008:40:042
29194991512,2cyclictest26468-21kworker/u8:109:43:032
29194991512,2cyclictest2582-21kworker/u8:011:57:042
29194991512,2cyclictest24988-21kworker/u8:208:27:042
29194991512,2cyclictest20028-21kworker/u8:011:48:042
29194991512,2cyclictest14956-21kworker/u8:009:57:042
29194991512,2cyclictest13262-21kworker/u8:008:02:032
29194991512,2cyclictest0-21swapper/209:39:072
2919499150,2cyclictest0-21swapper/210:38:042
2919499150,2cyclictest0-21swapper/209:12:042
2919299150,13cyclictest26609-21swap08:20:361
2918999152,12cyclictest101ktimersoftd/011:32:310
2918999152,12cyclictest0-21swapper/011:29:010
2918999151,3cyclictest1429-21ssh10:06:390
2918999151,12cyclictest101ktimersoftd/012:31:010
2918999150,3cyclictest0-21swapper/009:43:390
2918999150,13cyclictest28072-21ssh11:10:380
2918999150,13cyclictest0-21swapper/010:58:460
2919599142,2cyclictest0-21swapper/312:20:043
29195991411,2cyclictest8042-21kworker/3:011:34:043
29195991411,2cyclictest6730-21kworker/u8:307:38:043
2919599140,2cyclictest0-21swapper/310:10:043
2919599140,13cyclictest0-21swapper/312:12:043
2919599140,12cyclictest0-21swapper/311:11:013
2919599140,10cyclictest0-21swapper/310:06:043
2919599140,10cyclictest0-21swapper/307:56:043
29194991411,2cyclictest6730-21kworker/u8:307:32:042
29194991411,2cyclictest6729-21kworker/u8:107:41:042
29194991411,2cyclictest4508-21kworker/u8:008:47:042
29194991411,2cyclictest27481-21kworker/u8:008:32:042
29194991411,2cyclictest1175-21kworker/u8:108:58:042
2919499140,2cyclictest330350irq/24-eth112:07:032
2919499140,13cyclictest13038-21diskmemload09:32:072
2919499140,13cyclictest0-21swapper/209:15:272
2919299149,3cyclictest0-21swapper/110:51:381
2919299142,7cyclictest0-21swapper/107:15:001
2919299141,3cyclictest3840-21ssh11:55:271
2919299141,11cyclictest3168-21cpu07:25:121
2919299141,11cyclictest13038-21diskmemload10:13:171
2919299140,3cyclictest0-21swapper/111:04:391
2919299140,2cyclictest0-21swapper/108:15:061
2919299140,2cyclictest0-21swapper/107:40:001
2919299140,12cyclictest0-21swapper/110:01:321
2918999149,3cyclictest1401-21ssh10:42:160
2918999149,3cyclictest0-21swapper/008:30:320
2918999142,6cyclictest0-21swapper/012:20:010
2918999142,6cyclictest0-21swapper/010:10:000
2918999142,2cyclictest9-21ksoftirqd/010:00:030
2918999142,11cyclictest0-21swapper/011:39:010
2918999141,2cyclictest101ktimersoftd/010:29:420
2918999141,1cyclictest0-21swapper/010:34:310
29189991411,2cyclictest9-21ksoftirqd/012:35:160
2918999141,12cyclictest9-21ksoftirqd/011:17:000
29189991411,2cyclictest11-21rcuc/011:49:450
2918999140,3cyclictest23358-21ssh11:41:010
2918999140,12cyclictest17314-21ssh12:10:200
2918999140,12cyclictest0-21swapper/011:50:250
29195991310,2cyclictest0-21swapper/308:15:123
2919599130,2cyclictest0-21swapper/312:32:043
2919599130,2cyclictest0-21swapper/311:58:043
2919599130,2cyclictest0-21swapper/310:40:013
2919599130,2cyclictest0-21swapper/307:34:043
29194991310,2cyclictest9374-21kworker/u8:212:10:042
29194991310,2cyclictest6730-21kworker/u8:307:35:032
29194991310,2cyclictest12773-21kworker/u8:211:20:042
2919499130,12cyclictest0-21swapper/212:35:182
2919499130,11cyclictest0-21swapper/208:10:182
2919299138,3cyclictest5501-21diskstats07:30:151
2919299132,6cyclictest0-21swapper/108:45:001
2919299132,6cyclictest0-21swapper/107:45:001
29192991310,2cyclictest0-21swapper/112:00:161
29192991310,2cyclictest0-21swapper/111:43:591
29192991310,2cyclictest0-21swapper/111:25:301
29192991310,2cyclictest0-21swapper/110:57:521
29192991310,2cyclictest0-21swapper/110:34:021
29192991310,2cyclictest0-21swapper/110:15:201
2919299130,3cyclictest25152-21ssh09:57:271
2919299130,2cyclictest0-21swapper/112:16:401
2919299130,12cyclictest0-21swapper/111:45:221
2919299130,12cyclictest0-21swapper/111:15:221
2919299130,11cyclictest13038-21diskmemload11:11:391
2919299130,11cyclictest0-21swapper/111:52:511
2918999133,7cyclictest5883-21gdm-simple-gree08:10:590
2918999132,6cyclictest0-21swapper/009:05:000
2918999131,10cyclictest31216-21turbostat07:15:000
29189991310,2cyclictest101ktimersoftd/007:58:450
29189991310,2cyclictest0-21swapper/012:00:530
29189991310,2cyclictest0-21swapper/011:07:150
29189991310,2cyclictest0-21swapper/009:35:370
2918999130,2cyclictest101ktimersoftd/010:15:340
2918999130,2cyclictest0-21swapper/012:06:070
2918999130,2cyclictest0-21swapper/011:02:270
2918999130,2cyclictest0-21swapper/010:51:090
2918999130,2cyclictest0-21swapper/009:27:260
2918999130,2cyclictest0-21swapper/009:24:410
2919599120,2cyclictest0-21swapper/308:53:043
2919599120,2cyclictest0-21swapper/307:50:043
2919499129,2cyclictest0-21swapper/209:51:382
2919499122,7cyclictest5883-21gdm-simple-gree08:35:592
2919499120,11cyclictest0-21swapper/211:40:192
2919499120,11cyclictest0-21swapper/207:20:202
2919299129,2cyclictest1142-21ssh10:06:111
2919299129,2cyclictest0-21swapper/112:36:201
2919299129,2cyclictest0-21swapper/112:25:211
2919299129,2cyclictest0-21swapper/111:38:571
2919299129,2cyclictest0-21swapper/111:20:371
2919299129,2cyclictest0-21swapper/111:07:201
2919299129,2cyclictest0-21swapper/110:28:381
2919299129,2cyclictest0-21swapper/109:46:121
2919299129,2cyclictest0-21swapper/108:55:221
2919299127,3cyclictest14304-21ntp_kernel_err09:10:241
2919299127,3cyclictest14214-21ssh10:20:241
2919299123,5cyclictest0-21swapper/107:24:131
2919299122,5cyclictest0-21swapper/108:24:591
2919299121,6cyclictest0-21swapper/108:10:001
2919299120,2cyclictest13038-21diskmemload12:33:111
2919299120,2cyclictest0-21swapper/112:11:081
2919299120,2cyclictest0-21swapper/111:30:101
2919299120,2cyclictest0-21swapper/110:48:581
2919299120,2cyclictest0-21swapper/109:16:371
2919299120,2cyclictest0-21swapper/107:45:201
2919299120,11cyclictest0-21swapper/112:20:471
2919299120,11cyclictest0-21swapper/109:50:311
2919299120,11cyclictest0-21swapper/109:25:501
2919299120,11cyclictest0-21swapper/109:05:141
2919299120,11cyclictest0-21swapper/108:30:121
2918999129,2cyclictest13038-21diskmemload09:55:090
2918999129,2cyclictest0-21swapper/008:50:110
2918999129,2cyclictest0-21swapper/008:45:150
2918999129,2cyclictest0-21swapper/008:25:370
2918999129,2cyclictest0-21swapper/008:05:180
2918999122,7cyclictest0-21swapper/007:51:010
2918999122,7cyclictest0-21swapper/007:40:000
29189991210,1cyclictest0-21swapper/009:05:030
2918999120,11cyclictest26755-21ssh12:20:270
2918999120,11cyclictest0-21swapper/009:32:050
2918999120,11cyclictest0-21swapper/008:55:340
2918999120,11cyclictest0-21swapper/008:35:390
2918999120,11cyclictest0-21swapper/007:45:160
2919599111,6cyclictest0-21swapper/308:59:303
2919299118,2cyclictest15577-21ssh12:08:581
2919299112,6cyclictest0-21swapper/108:14:021
2919299110,1cyclictest0-21swapper/108:45:191
2919299110,10cyclictest0-21swapper/109:43:371
2919299110,10cyclictest0-21swapper/109:37:181
2918999118,2cyclictest101ktimersoftd/008:20:340
2918999118,2cyclictest0-21swapper/008:40:210
2918999118,2cyclictest0-21swapper/007:20:250
2918999110,9cyclictest3813-21missed_timers07:25:180
2918999110,2cyclictest0-21swapper/008:15:080
2918999110,1cyclictest0-21swapper/007:35:250
2919299102,6cyclictest0-21swapper/108:38:591
2919299101,6cyclictest0-21swapper/108:54:281
2918999102,5cyclictest0-21swapper/007:17:020
291949994,3cyclictest11672-21grep09:05:182
291929994,3cyclictest31527-21aten2.4_r3power07:15:101
291929994,3cyclictest18351-21open_inodes08:00:221
291929994,3cyclictest13382-21aten2.4_r3power07:50:131
291899997,1cyclictest101ktimersoftd/007:30:120
291899994,3cyclictest18727-21grep08:00:330
291929983,3cyclictest9311-21aten2.4_r3power09:00:151
291929981,5cyclictest16102-21cut07:55:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional