You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-15 - 11:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Sat Feb 15, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991960,86ptp4l0-21swapper/119:08:211
2823328271,9sleep110-21swapper/1119:07:363
130712730,0sleep613072-21cpuspeed_turbos22:05:388
294142710,0sleep10-21swapper/119:10:131
2826126756,9sleep00-21swapper/019:08:010
2808826655,9sleep100-21swapper/1019:05:342
2830726553,10sleep50-21swapper/519:08:427
2814526453,9sleep70-21swapper/719:06:199
2825226251,9sleep40-21swapper/419:07:536
2812926251,9sleep60-21swapper/619:06:078
44792580,0sleep90-21swapper/923:20:1311
113462560,0sleep50-21swapper/522:00:007
53012200,0sleep60-21swapper/620:50:188
2875299202,14cyclictest0-21swapper/419:20:016
2874199204,11cyclictest0-21swapper/220:20:014
2873299203,11cyclictest0-21swapper/023:40:000
2874199193,10cyclictest0-21swapper/220:35:014
2874199190,17cyclictest0-21swapper/222:06:364
2874199183,8cyclictest0-21swapper/200:20:004
2874199183,8cyclictest0-21swapper/200:20:004
2877099172,9cyclictest0-21swapper/823:00:0010
2876699174,8cyclictest0-21swapper/722:25:009
2874199174,7cyclictest0-21swapper/223:25:004
2874199173,9cyclictest0-21swapper/223:00:004
2874199173,9cyclictest0-21swapper/221:10:004
2874199173,8cyclictest0-21swapper/222:50:004
2878599161,2cyclictest0-21swapper/1122:35:153
28780991615,1cyclictest0-21swapper/1000:35:222
28780991615,1cyclictest0-21swapper/1000:35:222
2877099162,8cyclictest0-21swapper/822:35:0010
28761991614,1cyclictest0-21swapper/620:15:158
2875799162,8cyclictest0-21swapper/519:15:017
2874899164,7cyclictest0-21swapper/323:45:005
2874899163,8cyclictest0-21swapper/300:25:015
2874899163,8cyclictest0-21swapper/300:25:005
2874899162,8cyclictest0-21swapper/319:20:015
2874199163,8cyclictest0-21swapper/221:25:004
2873699163,7cyclictest0-21swapper/123:20:001
2873699162,8cyclictest0-21swapper/121:30:001
2873299163,8cyclictest0-21swapper/020:20:010
2873299162,9cyclictest0-21swapper/020:50:000
275991160,0ptp4l0-21swapper/122:15:421
91462150,0sleep30-21swapper/320:55:155
90662150,0sleep90-21swapper/920:10:1611
2877599152,8cyclictest0-21swapper/923:30:0011
2877599152,8cyclictest0-21swapper/921:10:0111
2877599152,8cyclictest0-21swapper/900:15:0011
2877599152,8cyclictest0-21swapper/900:15:0011
2877599152,8cyclictest0-21swapper/900:10:0011
28775991513,1cyclictest0-21swapper/900:04:5511
2877099153,7cyclictest0-21swapper/820:40:0110
2877099153,7cyclictest0-21swapper/820:00:0110
2877099152,8cyclictest0-21swapper/800:10:2210
2877099152,8cyclictest0-21swapper/800:10:2210
2876699152,8cyclictest0-21swapper/721:00:239
2876199153,8cyclictest0-21swapper/600:30:008
2876199153,8cyclictest0-21swapper/600:30:008
2875799152,7cyclictest0-21swapper/522:10:007
2875799151,10cyclictest0-21swapper/520:05:017
2875299152,12cyclictest0-21swapper/421:05:006
2874899152,7cyclictest0-21swapper/321:15:005
2874199153,8cyclictest0-21swapper/221:20:004
2874199153,7cyclictest0-21swapper/219:35:014
2873299153,7cyclictest0-21swapper/022:15:000
275991150,0ptp4l0-21swapper/120:35:121
191491150,0getstats0-21swapper/121:05:281
94980140,2rtkit-daemon0-21swapper/321:37:355
92662140,1sleep11801rcuc/1119:25:013
2878599143,7cyclictest0-21swapper/1100:00:003
2878599142,7cyclictest0-21swapper/1120:50:003
2878599142,7cyclictest0-21swapper/1100:40:013
2878599142,7cyclictest0-21swapper/1100:40:003
2877599143,7cyclictest0-21swapper/919:25:0111
2877099143,7cyclictest0-21swapper/821:50:0010
2877099143,6cyclictest0-21swapper/822:05:0010
2877099142,8cyclictest0-21swapper/800:30:0110
2877099142,8cyclictest0-21swapper/800:30:0010
2877099142,7cyclictest0-21swapper/820:55:2410
2876699142,7cyclictest0-21swapper/723:10:009
2876699142,7cyclictest0-21swapper/720:00:279
2876699142,7cyclictest0-21swapper/700:15:009
2876699142,7cyclictest0-21swapper/700:15:009
2876199142,8cyclictest15745-21kworker/u24:223:15:008
2876199142,8cyclictest15745-21kworker/u24:219:40:018
2876199142,8cyclictest0-21swapper/623:05:008
2876199142,8cyclictest0-21swapper/619:35:278
2875799142,7cyclictest0-21swapper/523:40:167
2874899143,6cyclictest0-21swapper/319:30:015
2874899141,9cyclictest0-21swapper/323:10:005
2874199142,8cyclictest0-21swapper/221:50:004
2874199142,8cyclictest0-21swapper/200:30:004
2874199142,8cyclictest0-21swapper/200:30:004
2873699143,7cyclictest0-21swapper/122:15:011
2873699142,7cyclictest0-21swapper/120:45:011
2873699140,8cyclictest0-21swapper/119:40:011
2873299142,9cyclictest0-21swapper/022:00:000
2873299141,8cyclictest0-21swapper/022:55:000
275991140,1ptp4l21-21ksoftirqd/120:50:241
275991140,0ptp4l0-21swapper/100:00:191
2877599133,6cyclictest0-21swapper/922:26:2711
2876699133,6cyclictest0-21swapper/720:20:279
2876699132,7cyclictest0-21swapper/719:30:029
2876699132,6cyclictest0-21swapper/719:25:019
2876699131,7cyclictest0-21swapper/720:45:009
2876699130,1cyclictest0-21swapper/719:40:019
2876199132,7cyclictest0-21swapper/623:20:208
28761991311,1cyclictest0-21swapper/621:24:558
28761991310,2cyclictest0-21swapper/620:25:018
2875799132,7cyclictest0-21swapper/519:30:027
2874899133,7cyclictest0-21swapper/322:45:005
2874899132,7cyclictest0-21swapper/323:35:005
2873699131,7cyclictest0-21swapper/121:35:001
276702130,0sleep30-21swapper/300:10:005
275991130,1ptp4l21-21ksoftirqd/120:44:371
275991130,1ptp4l21-21ksoftirqd/119:59:441
275991130,1ptp4l21-21ksoftirqd/119:30:241
275991130,0ptp4l0-21swapper/123:55:161
275991130,0ptp4l0-21swapper/123:42:491
275991130,0ptp4l0-21swapper/122:27:121
275991130,0ptp4l0-21swapper/120:10:211
275991130,0ptp4l0-21swapper/119:40:241
2878099120,7cyclictest0-21swapper/1023:20:012
2877599122,6cyclictest0-21swapper/920:35:2411
2876699123,7cyclictest0-21swapper/721:10:289
2876199122,7cyclictest0-21swapper/622:25:008
2875799122,6cyclictest0-21swapper/523:50:007
2875799122,6cyclictest0-21swapper/500:15:007
2875799122,6cyclictest0-21swapper/500:15:007
2875799121,6cyclictest0-21swapper/523:00:007
2875799121,6cyclictest0-21swapper/521:30:007
2875299122,6cyclictest0-21swapper/423:16:016
2873699122,7cyclictest0-21swapper/121:55:001
2873699122,6cyclictest0-21swapper/120:30:191
2873699122,6cyclictest0-21swapper/119:25:531
2873299123,5cyclictest0-21swapper/023:55:000
275991120,2ptp4l712-21fw_conntrack19:15:011
275991120,0ptp4l0-21swapper/123:33:311
275991120,0ptp4l0-21swapper/100:35:131
275991120,0ptp4l0-21swapper/100:35:131
275991120,0ptp4l0-21swapper/100:20:141
275991120,0ptp4l0-21swapper/100:20:141
275991120,0ptp4l0-21swapper/100:10:141
275991120,0ptp4l0-21swapper/100:10:141
94980110,2rtkit-daemon0-21swapper/319:05:535
2878599119,1cyclictest0-21swapper/1121:59:553
2877599113,7cyclictest0-21swapper/922:10:0011
2876699111,6cyclictest0-21swapper/720:10:019
2876199110,10cyclictest0-21swapper/621:05:008
2875299112,6cyclictest14850irq/20-ata_piix23:20:006
2874899112,5cyclictest0-21swapper/323:25:005
2874899112,5cyclictest0-21swapper/320:00:015
2874899111,6cyclictest0-21swapper/321:25:015
2874899111,6cyclictest0-21swapper/320:40:015
2873299111,6cyclictest15745-21kworker/u24:222:05:000
2873299110,10cyclictest9785-21cat23:35:000
283492111,7sleep80-21swapper/819:09:2110
275991110,2ptp4l13331-21cat21:00:161
275991110,0ptp4l0-21swapper/122:49:501
275991110,0ptp4l0-21swapper/122:05:371
275991110,0ptp4l0-21swapper/122:02:071
275991110,0ptp4l0-21swapper/120:15:011
275991110,0ptp4l0-21swapper/100:15:121
275991110,0ptp4l0-21swapper/100:15:121
2878099100,9cyclictest0-21swapper/1000:10:002
2877599107,2cyclictest0-21swapper/923:50:1111
2877599102,7cyclictest0-21swapper/921:05:0011
2876699107,2cyclictest11422-21munin-run00:35:009
2876699107,2cyclictest11422-21munin-run00:35:009
2876699101,5cyclictest0-21swapper/722:20:019
2876699100,9cyclictest0-21swapper/722:06:389
2876199102,7cyclictest0-21swapper/600:15:248
2876199102,7cyclictest0-21swapper/600:15:248
2875799102,4cyclictest0-21swapper/522:40:007
2875299107,2cyclictest0-21swapper/420:40:176
2874199100,9cyclictest0-21swapper/220:15:134
2874199100,9cyclictest0-21swapper/219:10:014
2873299102,6cyclictest0-21swapper/021:55:000
2873299102,5cyclictest0-21swapper/023:05:000
275991100,1ptp4l17296-21irqstats23:50:171
275991100,0ptp4l0-21swapper/121:10:211
275991100,0ptp4l0-21swapper/120:26:001
275991100,0ptp4l0-21swapper/120:20:181
287859990,9cyclictest0-21swapper/1119:55:203
287859990,9cyclictest0-21swapper/1100:30:073
287859990,9cyclictest0-21swapper/1100:30:073
287859990,1cyclictest0-21swapper/1119:35:183
287809998,1cyclictest0-21swapper/1020:10:262
287759990,1cyclictest0-21swapper/920:20:2711
287759990,1cyclictest0-21swapper/920:00:2111
287709990,9cyclictest63-21ksoftirqd/820:00:2710
287709990,2cyclictest2466-21diskstats19:15:1510
287669997,1cyclictest2620-21smartctl20:45:229
287669996,2cyclictest0-21swapper/722:35:119
287619990,1cyclictest0-21swapper/620:25:598
287619990,1cyclictest0-21swapper/620:05:268
287579992,6cyclictest0-21swapper/523:25:007
287529997,1cyclictest0-21swapper/420:25:016
287529992,6cyclictest12112-21crond20:15:016
287529990,9cyclictest0-21swapper/422:11:216
287489998,1cyclictest0-21swapper/319:10:255
287489996,2cyclictest22490-21if_enp1s0.120:30:135
287419996,2cyclictest22300-21unixbench_singl23:55:284
287419990,9cyclictest0-21swapper/221:20:244
287419990,3cyclictest8236-21/usr/sbin/munin19:20:254
287329992,5cyclictest0-21swapper/021:50:000
287329991,5cyclictest0-21swapper/020:45:000
287329990,9cyclictest0-21swapper/000:35:200
287329990,9cyclictest0-21swapper/000:35:190
27599190,3ptp4l21-21ksoftirqd/123:07:421
27599190,1ptp4l21-21ksoftirqd/123:04:531
27599190,1ptp4l21-21ksoftirqd/122:44:151
27599190,1ptp4l21-21ksoftirqd/120:05:151
27599190,1ptp4l21-21ksoftirqd/100:31:541
27599190,1ptp4l21-21ksoftirqd/100:31:541
27599190,0ptp4l0-21swapper/123:45:001
27599190,0ptp4l0-21swapper/123:35:001
27599190,0ptp4l0-21swapper/123:27:161
27599190,0ptp4l0-21swapper/123:20:091
27599190,0ptp4l0-21swapper/122:35:101
27599190,0ptp4l0-21swapper/121:50:011
27599190,0ptp4l0-21swapper/121:45:001
27599190,0ptp4l0-21swapper/121:20:001
27599190,0ptp4l0-21swapper/121:15:001
27599190,0ptp4l0-21swapper/120:55:131
27599190,0ptp4l0-21swapper/120:00:161
27599190,0ptp4l0-21swapper/119:50:141
27599190,0ptp4l0-21swapper/119:20:141
287859987,1cyclictest0-21swapper/1122:50:003
287859987,1cyclictest0-21swapper/1100:28:253
287859987,1cyclictest0-21swapper/1100:28:243
287859985,2cyclictest1657-21/usr/sbin/munin23:16:013
287859980,8cyclictest0-21swapper/1121:50:113
287859980,8cyclictest0-21swapper/1121:30:313
287859980,8cyclictest0-21swapper/1120:50:273
287859980,8cyclictest0-21swapper/1120:20:293
287859980,8cyclictest0-21swapper/1119:40:193
287809981,5cyclictest0-21swapper/1019:50:012
287809980,1cyclictest0-21swapper/1023:50:132
287809980,1cyclictest0-21swapper/1021:00:202
287809980,1cyclictest0-21swapper/1019:10:172
287759987,1cyclictest0-21swapper/922:35:1511
287759987,1cyclictest0-21swapper/921:20:1111
287759987,1cyclictest0-21swapper/900:30:1211
287759987,1cyclictest0-21swapper/900:30:1211
287759987,1cyclictest0-21swapper/900:20:1511
287759987,1cyclictest0-21swapper/900:20:1511
287759980,8cyclictest0-21swapper/921:50:1111
287759980,8cyclictest0-21swapper/921:30:3411
287759980,8cyclictest0-21swapper/919:55:1911
287759980,3cyclictest0-21swapper/921:00:0011
287759980,1cyclictest0-21swapper/923:55:2311
287759980,1cyclictest0-21swapper/920:54:5811
287709986,1cyclictest0-21swapper/820:45:2610
287709980,7cyclictest19077-21diskmemload21:59:1010
287709980,7cyclictest0-21swapper/821:07:2010
287709980,1cyclictest0-21swapper/823:55:2310
287669987,1cyclictest0-21swapper/720:50:239
287669980,8cyclictest0-21swapper/723:15:569
287669980,1cyclictest0-21swapper/700:35:159
287669980,1cyclictest0-21swapper/700:35:159
287669980,1cyclictest0-21swapper/700:20:159
287669980,1cyclictest0-21swapper/700:20:159
287619987,1cyclictest0-21swapper/621:00:008
287619987,1cyclictest0-21swapper/600:10:128
287619987,1cyclictest0-21swapper/600:10:128
287619986,1cyclictest0-21swapper/619:20:148
287619985,2cyclictest2439-21turbostat23:20:008
287619980,8cyclictest0-21swapper/623:40:138
287619980,7cyclictest0-21swapper/619:25:238
287619980,1cyclictest0-21swapper/623:55:188
287619980,1cyclictest0-21swapper/620:30:168
287619980,1cyclictest0-21swapper/619:30:248
287579985,2cyclictest2899-21turbostat21:39:567
287579984,3cyclictest3904-21gdbus21:45:077
287579980,8cyclictest0-21swapper/521:20:177
287579980,7cyclictest1020-21runrttasks20:05:387
287529985,2cyclictest7169-21turbostat21:54:556
287529985,2cyclictest24448-21turbostat20:39:566
287529980,1cyclictest0-21swapper/420:30:586
287489987,1cyclictest0-21swapper/320:50:175
287489980,8cyclictest0-21swapper/320:05:265
287489980,1cyclictest17690-21/usr/sbin/munin22:15:385
287489980,1cyclictest0-21swapper/321:10:205
287489980,1cyclictest0-21swapper/321:00:245
287489980,1cyclictest0-21swapper/319:35:275
287419987,1cyclictest0-21swapper/200:35:174
287419987,1cyclictest0-21swapper/200:35:174
287419987,1cyclictest0-21swapper/200:00:154
287419980,8cyclictest0-21swapper/200:15:124
287419980,8cyclictest0-21swapper/200:15:124
287419980,7cyclictest0-21swapper/220:35:134
287419980,2cyclictest0-21swapper/220:05:184
287419980,1cyclictest0-21swapper/223:00:074
287419980,1cyclictest0-21swapper/221:00:184
287419980,1cyclictest0-21swapper/220:00:184
27599180,1ptp4l21-21ksoftirqd/122:30:201
27599180,1ptp4l21-21ksoftirqd/100:05:011
27599180,0ptp4l0-21swapper/123:10:001
27599180,0ptp4l0-21swapper/122:55:001
27599180,0ptp4l0-21swapper/122:50:001
27599180,0ptp4l0-21swapper/122:20:001
27599180,0ptp4l0-21swapper/121:40:001
27599180,0ptp4l0-21swapper/121:35:041
27599180,0ptp4l0-21swapper/119:45:011
27599180,0ptp4l0-21swapper/100:26:361
27599180,0ptp4l0-21swapper/100:26:361
25099281,5sleep90-21swapper/919:05:0911
19159180,0getstats0-21swapper/222:31:354
287859976,1cyclictest0-21swapper/1123:40:413
287859976,1cyclictest0-21swapper/1122:55:103
287859976,1cyclictest0-21swapper/1122:51:213
287859976,1cyclictest0-21swapper/1122:15:283
287859976,1cyclictest0-21swapper/1122:12:443
287859976,1cyclictest0-21swapper/1122:06:243
287859976,1cyclictest0-21swapper/1121:48:583
287859976,1cyclictest0-21swapper/1121:20:123
287859976,1cyclictest0-21swapper/1121:11:133
287859976,1cyclictest0-21swapper/1121:00:543
287859976,1cyclictest0-21swapper/1120:57:383
287859976,1cyclictest0-21swapper/1120:27:303
287859976,1cyclictest0-21swapper/1119:31:593
287859976,1cyclictest0-21swapper/1119:14:463
287859976,1cyclictest0-21swapper/1100:10:183
287859976,1cyclictest0-21swapper/1100:10:183
287859976,1cyclictest0-21swapper/1100:00:523
287859974,2cyclictest32561-21entropy00:15:143
287859974,2cyclictest32561-21entropy00:15:143
287859973,3cyclictest5087-21cut00:20:183
287859973,3cyclictest5087-21cut00:20:183
287859970,7cyclictest0-21swapper/1123:46:283
287859970,7cyclictest0-21swapper/1123:35:023
287859970,7cyclictest0-21swapper/1123:30:143
287859970,7cyclictest0-21swapper/1123:12:173
287859970,7cyclictest0-21swapper/1123:00:283
287859970,7cyclictest0-21swapper/1122:25:173
287859970,7cyclictest0-21swapper/1122:21:043
287859970,7cyclictest0-21swapper/1121:38:053
287859970,7cyclictest0-21swapper/1120:43:243
287859970,7cyclictest0-21swapper/1120:35:183
287859970,7cyclictest0-21swapper/1120:31:303
287859970,7cyclictest0-21swapper/1119:50:123
287859970,1cyclictest0-21swapper/1123:26:233
287859970,1cyclictest0-21swapper/1123:23:553
287859970,1cyclictest0-21swapper/1123:05:313
287859970,1cyclictest0-21swapper/1122:41:003
287859970,1cyclictest0-21swapper/1122:03:003
287859970,1cyclictest0-21swapper/1121:42:023
287859970,1cyclictest0-21swapper/1121:27:103
287859970,1cyclictest0-21swapper/1121:15:283
287859970,1cyclictest0-21swapper/1121:05:083
287859970,1cyclictest0-21swapper/1120:13:253
287859970,1cyclictest0-21swapper/1120:06:243
287859970,1cyclictest0-21swapper/1120:00:143
287859970,1cyclictest0-21swapper/1119:47:493
287859970,1cyclictest0-21swapper/1119:25:403
287859970,1cyclictest0-21swapper/1119:16:133
287859970,1cyclictest0-21swapper/1100:05:033
287809976,1cyclictest0-21swapper/1023:46:092
287809976,1cyclictest0-21swapper/1023:42:572
287809976,1cyclictest0-21swapper/1023:30:532
287809976,1cyclictest0-21swapper/1023:10:502
287809976,1cyclictest0-21swapper/1023:06:242
287809976,1cyclictest0-21swapper/1022:36:242
287809976,1cyclictest0-21swapper/1022:32:122
287809976,1cyclictest0-21swapper/1022:25:012
287809976,1cyclictest0-21swapper/1022:10:272
287809976,1cyclictest0-21swapper/1022:05:432
287809976,1cyclictest0-21swapper/1021:40:152
287809976,1cyclictest0-21swapper/1021:35:102
287809976,1cyclictest0-21swapper/1020:50:222
287809976,1cyclictest0-21swapper/1020:36:202
287809976,1cyclictest0-21swapper/1020:30:132
287809976,1cyclictest0-21swapper/1020:25:442
287809976,1cyclictest0-21swapper/1020:24:092
287809976,1cyclictest0-21swapper/1020:05:512
287809976,1cyclictest0-21swapper/1019:42:572
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional