You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-16 - 16:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Thu Oct 16, 2025 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911010,92ptp4l0-21swapper/107:09:171
94980780,61rtkit-daemon0-21swapper/707:07:489
484826755,10sleep50-21swapper/507:07:287
484926554,9sleep60-21swapper/607:07:298
151326554,9sleep00-21swapper/007:05:120
91932640,5sleep10-21swapper/107:15:011
493426453,9sleep110-21swapper/1107:08:463
168526450,11sleep40-21swapper/407:05:156
481226252,8sleep100-21swapper/1007:06:582
88462590,0sleep80-21swapper/808:35:1710
538599243,16cyclictest0-21swapper/412:40:016
539499224,9cyclictest0-21swapper/609:40:248
538599200,7cyclictest0-21swapper/408:05:016
202432200,2sleep757-21ksoftirqd/708:10:009
537699194,10cyclictest0-21swapper/210:29:594
275991190,1ptp4l21-21ksoftirqd/111:20:271
537699184,9cyclictest0-21swapper/207:20:014
537699183,10cyclictest0-21swapper/210:45:004
540399173,9cyclictest0-21swapper/812:20:0010
539999172,9cyclictest0-21swapper/708:55:249
539499173,8cyclictest0-21swapper/608:20:018
537699173,8cyclictest0-21swapper/211:09:594
537699172,11cyclictest14757-21kworker/u24:109:00:004
537299171,10cyclictest0-21swapper/110:25:011
536799174,8cyclictest0-21swapper/010:55:000
536799173,8cyclictest0-21swapper/012:00:000
275991170,0ptp4l0-21swapper/111:50:121
182952170,0sleep100-21swapper/1010:55:112
541299160,2cyclictest0-21swapper/1012:15:172
540399160,10cyclictest0-21swapper/807:55:0110
539499160,2cyclictest0-21swapper/607:20:168
538199163,8cyclictest0-21swapper/310:15:015
537699160,10cyclictest0-21swapper/211:55:004
537299164,7cyclictest0-21swapper/107:15:001
537299162,9cyclictest0-21swapper/110:40:001
536799162,9cyclictest0-21swapper/009:40:000
275991160,1ptp4l21-21ksoftirqd/111:38:581
275991160,1ptp4l12525-21grep12:10:141
275991160,0ptp4l0-21swapper/109:45:241
275991160,0ptp4l0-21swapper/109:15:501
541299150,1cyclictest0-21swapper/1011:30:222
540899153,8cyclictest0-21swapper/909:15:0211
540899152,8cyclictest0-21swapper/912:20:0011
539499152,8cyclictest0-21swapper/611:45:008
539499152,8cyclictest0-21swapper/608:15:018
538199153,7cyclictest0-21swapper/312:40:015
538199152,8cyclictest0-21swapper/309:55:005
538199152,8cyclictest0-21swapper/308:40:025
537699153,8cyclictest0-21swapper/212:20:014
537699152,8cyclictest0-21swapper/208:30:014
537299153,7cyclictest0-21swapper/111:35:001
537299152,9cyclictest0-21swapper/109:10:011
537299150,9cyclictest0-21swapper/111:15:001
536799153,8cyclictest0-21swapper/008:45:000
536799153,7cyclictest0-21swapper/009:55:000
275991150,1ptp4l8484-21fschecks_count08:35:141
275991150,1ptp4l21-21ksoftirqd/109:50:161
275991150,1ptp4l21-21ksoftirqd/109:25:121
275991150,0ptp4l0-21swapper/112:24:011
275991150,0ptp4l0-21swapper/109:10:391
275991150,0ptp4l0-21swapper/108:45:181
275991150,0ptp4l0-21swapper/107:20:121
123302150,0sleep100-21swapper/1009:20:232
541799141,3cyclictest0-21swapper/1110:50:193
541299143,7cyclictest0-21swapper/1012:25:252
541299142,7cyclictest0-21swapper/1012:20:242
540899142,8cyclictest0-21swapper/911:15:0011
540899142,7cyclictest0-21swapper/912:15:0011
540899142,7cyclictest0-21swapper/910:00:2511
540899142,7cyclictest0-21swapper/907:50:0111
540899140,13cyclictest10843-21awk12:10:0011
540399142,8cyclictest0-21swapper/811:40:2510
540399142,8cyclictest0-21swapper/809:10:0010
540399141,8cyclictest0-21swapper/811:00:2510
539999141,7cyclictest0-21swapper/711:25:019
539499142,7cyclictest0-21swapper/610:30:018
539499142,7cyclictest0-21swapper/609:55:008
539499141,7cyclictest0-21swapper/612:10:018
539499141,7cyclictest0-21swapper/611:35:018
538199143,6cyclictest0-21swapper/308:30:015
538199142,7cyclictest0-21swapper/310:50:275
538199142,7cyclictest0-21swapper/307:10:005
5376991412,2cyclictest0-21swapper/208:40:184
537299142,8cyclictest0-21swapper/110:14:591
537299142,7cyclictest0-21swapper/110:35:001
536799142,7cyclictest0-21swapper/007:25:010
275991140,0ptp4l0-21swapper/107:35:201
540899132,6cyclictest0-21swapper/912:35:0011
540899132,6cyclictest0-21swapper/908:30:2511
540899130,12cyclictest0-21swapper/908:09:5611
539999133,6cyclictest0-21swapper/707:15:249
539999132,7cyclictest0-21swapper/711:50:019
539999132,7cyclictest0-21swapper/709:50:019
539999131,7cyclictest0-21swapper/708:25:019
539499130,8cyclictest0-21swapper/611:50:018
538199132,8cyclictest0-21swapper/309:30:005
538199132,7cyclictest0-21swapper/311:48:265
538199132,7cyclictest0-21swapper/310:05:015
538199132,6cyclictest0-21swapper/309:45:005
537299132,6cyclictest0-21swapper/107:50:011
536799132,7cyclictest0-21swapper/009:00:010
275991130,1ptp4l21-21ksoftirqd/112:35:141
275991130,0ptp4l0-21swapper/110:45:121
275991130,0ptp4l0-21swapper/108:25:181
541799128,3cyclictest3985-21timerandwakeup07:45:253
540399122,7cyclictest0-21swapper/807:35:2310
539999122,6cyclictest0-21swapper/710:40:019
539999122,6cyclictest0-21swapper/707:30:019
539499122,6cyclictest0-21swapper/611:25:008
539099123,5cyclictest0-21swapper/511:50:017
538599121,7cyclictest0-21swapper/409:25:006
538599120,7cyclictest0-21swapper/409:30:006
538199122,7cyclictest0-21swapper/310:45:245
538199122,7cyclictest0-21swapper/309:35:255
536799123,5cyclictest0-21swapper/010:00:000
275991120,1ptp4l21-21ksoftirqd/110:00:141
275991120,0ptp4l0-21swapper/112:25:131
275991120,0ptp4l0-21swapper/112:05:141
275991120,0ptp4l0-21swapper/111:40:221
275991120,0ptp4l0-21swapper/111:16:091
275991120,0ptp4l0-21swapper/110:43:131
275991120,0ptp4l0-21swapper/109:35:151
275991120,0ptp4l0-21swapper/107:45:251
541299112,6cyclictest0-21swapper/1007:30:012
540899110,10cyclictest26581-21latency_hist12:30:0111
539999113,6cyclictest0-21swapper/707:15:019
539999112,6cyclictest0-21swapper/708:15:029
539499119,1cyclictest0-21swapper/612:30:008
539499119,1cyclictest0-21swapper/609:30:008
539099113,5cyclictest0-21swapper/507:25:007
538599112,5cyclictest0-21swapper/408:47:286
538199119,1cyclictest0-21swapper/307:25:015
537699112,6cyclictest0-21swapper/212:10:004
537699110,10cyclictest1834-21cat09:10:024
536799118,2cyclictest22944-21switchtime11:40:250
536799110,10cyclictest0-21swapper/007:30:160
28682112,7sleep80-21swapper/807:05:2010
275991110,2ptp4l28068-21latency_hist08:20:001
275991110,2ptp4l22629-21cstates07:30:111
275991110,2ptp4l19358-21fschecks_time07:25:151
275991110,2ptp4l19251-21perl10:55:131
275991110,2ptp4l1924-21taskset11:55:191
275991110,2ptp4l15219-21df10:50:151
275991110,2ptp4l10103-21perl11:25:141
275991110,1ptp4l21-21ksoftirqd/108:55:241
275991110,1ptp4l0-21swapper/111:05:001
275991110,1ptp4l0-21swapper/110:15:141
275991110,1ptp4l0-21swapper/107:55:101
275991110,0ptp4l1020-21runrttasks11:46:181
275991110,0ptp4l0-21swapper/108:40:241
275991110,0ptp4l0-21swapper/108:00:211
541799107,2cyclictest0-21swapper/1111:35:173
541299107,2cyclictest0-21swapper/1009:45:202
541299100,9cyclictest0-21swapper/1009:55:242
541299100,10cyclictest0-21swapper/1012:30:222
540399106,2cyclictest0-21swapper/810:27:3510
540399102,6cyclictest0-21swapper/808:40:2410
539999107,2cyclictest576-21turbostat11:14:599
539499108,1cyclictest0-21swapper/610:05:008
539499107,2cyclictest0-21swapper/609:15:448
539099106,3cyclictest15896-21date10:10:017
539099102,5cyclictest0-21swapper/508:05:007
539099101,5cyclictest0-21swapper/507:30:007
539099100,9cyclictest19597-21nfs4_client09:30:197
539099100,9cyclictest0-21swapper/509:25:007
538199102,4cyclictest0-21swapper/310:20:005
538199101,5cyclictest0-21swapper/311:00:015
537699101,6cyclictest0-21swapper/211:30:004
537699100,9cyclictest2737-21irqrtprio11:15:194
537299100,9cyclictest0-21swapper/109:55:161
536799102,5cyclictest0-21swapper/007:20:000
536799100,5cyclictest0-21swapper/008:25:000
275991100,2ptp4l21-21ksoftirqd/109:30:161
275991100,1ptp4l4514-21dnf08:30:141
275991100,1ptp4l29974-21/usr/sbin/munin09:05:001
275991100,1ptp4l26133-21perl09:40:151
275991100,1ptp4l20565-21/usr/sbin/munin08:10:151
275991100,1ptp4l13395-21smartctl10:05:181
275991100,1ptp4l0-21swapper/112:00:111
275991100,0ptp4l21999-21sshd08:50:291
275991100,0ptp4l0-21swapper/109:24:591
247692100,0sleep80-21swapper/810:20:1710
54179997,1cyclictest0-21swapper/1108:40:233
54179997,1cyclictest0-21swapper/1107:30:013
54179994,4cyclictest0-21swapper/1112:05:013
54179992,6cyclictest0-21swapper/1109:05:243
54129998,1cyclictest0-21swapper/1012:05:132
54129996,2cyclictest1856-21users10:30:262
54129990,8cyclictest0-21swapper/1010:00:142
54129990,1cyclictest0-21swapper/1008:10:212
54089996,2cyclictest0-21swapper/911:34:5911
54089990,9cyclictest0-21swapper/907:10:2611
54089990,1cyclictest0-21swapper/910:45:1911
54089990,1cyclictest0-21swapper/908:10:2411
54039990,2cyclictest12124-21sshd07:56:1110
53999997,1cyclictest0-21swapper/708:25:009
53949997,1cyclictest0-21swapper/607:40:218
53909998,1cyclictest0-21swapper/510:10:247
53909996,2cyclictest18326-21grep10:55:107
53909992,4cyclictest0-21swapper/508:15:007
53909992,4cyclictest0-21swapper/508:10:007
53909991,5cyclictest0-21swapper/512:10:247
53909991,5cyclictest0-21swapper/508:35:247
53909990,8cyclictest27344-21users08:15:287
53909990,8cyclictest0-21swapper/509:35:197
53909990,6cyclictest0-21swapper/507:45:007
53859996,2cyclictest13101-21kernelversion12:10:186
53859990,8cyclictest0-21swapper/411:00:146
53819992,4cyclictest0-21swapper/311:45:005
53819991,5cyclictest0-21swapper/307:50:235
53819990,8cyclictest17864-21fschecks_time08:05:145
53769991,6cyclictest0-21swapper/212:10:004
53769990,8cyclictest0-21swapper/211:40:244
53769990,8cyclictest0-21swapper/211:20:194
53769990,8cyclictest0-21swapper/210:30:204
53769990,8cyclictest0-21swapper/209:35:184
53769990,8cyclictest0-21swapper/208:15:204
53769990,8cyclictest0-21swapper/207:45:144
53769990,3cyclictest13552-21/usr/sbin/munin10:45:254
53679998,1cyclictest0-21swapper/011:20:220
53679997,1cyclictest5027-21netstat10:35:200
53679996,2cyclictest16446-21hddtemp_smartct08:45:150
53679990,1cyclictest0-21swapper/009:10:120
53679990,1cyclictest0-21swapper/008:00:200
53679990,1cyclictest0-21swapper/007:45:200
2973291,5sleep90-21swapper/907:05:2111
27599190,0ptp4l0-21swapper/112:30:001
27599190,0ptp4l0-21swapper/112:15:121
27599190,0ptp4l0-21swapper/111:05:121
27599190,0ptp4l0-21swapper/110:25:151
27599190,0ptp4l0-21swapper/108:20:161
27599190,0ptp4l0-21swapper/108:05:121
27599190,0ptp4l0-21swapper/107:40:131
54179987,1cyclictest0-21swapper/1111:50:193
54179987,1cyclictest0-21swapper/1110:20:183
54179987,1cyclictest0-21swapper/1108:00:003
54179985,2cyclictest0-21swapper/1108:35:123
54179984,2cyclictest11614-21latency_hist10:04:593
54179982,5cyclictest0-21swapper/1110:50:003
54179981,6cyclictest0-21swapper/1107:55:003
54179980,8cyclictest0-21swapper/1110:25:243
54179980,8cyclictest0-21swapper/1109:40:003
54179980,8cyclictest0-21swapper/1109:25:123
54179980,7cyclictest0-21swapper/1112:07:253
54179980,7cyclictest0-21swapper/1110:59:113
54179980,1cyclictest0-21swapper/1112:11:073
54179980,1cyclictest0-21swapper/1110:05:163
54179980,1cyclictest0-21swapper/1108:00:233
54129987,1cyclictest0-21swapper/1009:50:192
54129987,1cyclictest0-21swapper/1009:10:152
54129985,2cyclictest0-21swapper/1010:50:152
54129981,4cyclictest0-21swapper/1011:10:002
54089987,1cyclictest0-21swapper/909:35:1511
54089987,1cyclictest0-21swapper/908:20:2111
54089987,1cyclictest0-21swapper/907:25:1411
54089986,1cyclictest0-21swapper/909:45:1411
54089985,2cyclictest10644-21diskstats09:20:1311
54089980,1cyclictest0-21swapper/910:14:3211
54089980,1cyclictest0-21swapper/908:45:2311
54039986,1cyclictest0-21swapper/811:45:2910
54039986,1cyclictest0-21swapper/810:15:1310
54039980,8cyclictest0-21swapper/812:30:2310
54039980,8cyclictest0-21swapper/808:10:1510
54039980,1cyclictest0-21swapper/812:25:1410
54039980,1cyclictest0-21swapper/809:20:2010
54039980,1cyclictest0-21swapper/808:50:2310
54039980,1cyclictest0-21swapper/807:10:1610
53999986,1cyclictest0-21swapper/710:14:599
53999980,8cyclictest0-21swapper/707:25:169
53999980,1cyclictest0-21swapper/710:15:149
53949980,7cyclictest0-21swapper/609:57:068
53949980,1cyclictest0-21swapper/609:01:018
53909987,1cyclictest0-21swapper/510:45:257
53909987,1cyclictest0-21swapper/510:20:207
53909987,1cyclictest0-21swapper/509:10:187
53909985,2cyclictest23338-21hddtemp_smartct07:30:147
53909982,4cyclictest0-21swapper/508:50:007
53909980,8cyclictest0-21swapper/510:35:137
53909980,8cyclictest0-21swapper/508:20:287
53909980,5cyclictest0-21swapper/512:40:017
53909980,1cyclictest0-21swapper/511:25:187
53909980,1cyclictest0-21swapper/509:50:167
53859987,1cyclictest0-21swapper/412:30:156
53859981,5cyclictest0-21swapper/408:35:006
53859980,8cyclictest0-21swapper/408:25:166
53859980,7cyclictest0-21swapper/409:10:246
53859980,1cyclictest0-21swapper/412:00:106
53859980,1cyclictest0-21swapper/411:05:146
53859980,1cyclictest0-21swapper/410:27:306
53859980,1cyclictest0-21swapper/410:15:176
53859980,1cyclictest0-21swapper/409:55:006
53859980,1cyclictest0-21swapper/407:20:026
53819986,1cyclictest0-21swapper/309:48:505
53819980,7cyclictest0-21swapper/311:00:245
53819980,1cyclictest0-21swapper/308:55:195
53819980,1cyclictest0-21swapper/308:15:145
53769980,8cyclictest0-21swapper/212:30:244
53769980,8cyclictest0-21swapper/211:30:144
53769980,8cyclictest0-21swapper/211:10:274
53769980,8cyclictest0-21swapper/209:40:144
53769980,8cyclictest0-21swapper/209:15:444
53769980,8cyclictest0-21swapper/209:00:144
53769980,8cyclictest0-21swapper/208:30:124
53769980,8cyclictest0-21swapper/207:20:244
53769980,7cyclictest22588-21/usr/sbin/munin11:00:134
53769980,7cyclictest12775-21sshd09:20:244
53769980,7cyclictest0-21swapper/210:35:164
53769980,7cyclictest0-21swapper/210:10:174
53769980,2cyclictest31479-21irqstats07:40:194
53769980,1cyclictest19256-21timerandwakeup11:35:274
53769980,1cyclictest0-21swapper/211:48:264
53769980,1cyclictest0-21swapper/208:35:174
53679987,1cyclictest0-21swapper/012:10:240
53679987,1cyclictest0-21swapper/009:20:170
53679987,1cyclictest0-21swapper/009:20:020
53679987,1cyclictest0-21swapper/008:50:200
53679986,1cyclictest30722-21turbostat10:30:000
53679985,2cyclictest0-21swapper/011:45:210
53679980,8cyclictest0-21swapper/010:45:160
53679980,8cyclictest0-21swapper/009:25:170
53679980,8cyclictest0-21swapper/007:10:070
53679980,1cyclictest0-21swapper/012:05:130
53679980,1cyclictest0-21swapper/010:30:220
27609180,6phc2sys4153-21switchtime07:05:254
9458270,0sleep90-21swapper/911:25:1211
54179976,1cyclictest0-21swapper/1111:55:233
54179976,1cyclictest0-21swapper/1111:31:193
54179976,1cyclictest0-21swapper/1111:20:293
54179976,1cyclictest0-21swapper/1111:11:123
54179976,1cyclictest0-21swapper/1110:10:313
54179976,1cyclictest0-21swapper/1109:45:053
54179976,1cyclictest0-21swapper/1109:20:123
54179976,1cyclictest0-21swapper/1109:00:383
54179976,1cyclictest0-21swapper/1108:30:123
54179976,1cyclictest0-21swapper/1108:20:173
54179976,1cyclictest0-21swapper/1108:05:193
54179976,1cyclictest0-21swapper/1107:35:183
54179974,2cyclictest1020-21runrttasks11:05:223
54179970,7cyclictest0-21swapper/1112:35:553
54179970,7cyclictest0-21swapper/1111:45:013
54179970,7cyclictest0-21swapper/1111:25:463
54179970,7cyclictest0-21swapper/1109:15:493
54179970,7cyclictest0-21swapper/1108:46:323
54179970,7cyclictest0-21swapper/1107:20:283
54179970,1cyclictest0-21swapper/1112:31:373
54179970,1cyclictest0-21swapper/1112:25:453
54179970,1cyclictest0-21swapper/1112:22:423
54179970,1cyclictest0-21swapper/1112:15:113
54179970,1cyclictest0-21swapper/1111:40:063
54179970,1cyclictest0-21swapper/1111:15:473
54179970,1cyclictest0-21swapper/1111:00:103
54179970,1cyclictest0-21swapper/1110:42:143
54179970,1cyclictest0-21swapper/1110:35:273
54179970,1cyclictest0-21swapper/1110:30:073
54179970,1cyclictest0-21swapper/1110:15:443
54179970,1cyclictest0-21swapper/1109:55:013
54179970,1cyclictest0-21swapper/1109:50:493
54179970,1cyclictest0-21swapper/1109:35:553
54179970,1cyclictest0-21swapper/1109:33:103
54179970,1cyclictest0-21swapper/1109:10:113
54179970,1cyclictest0-21swapper/1108:55:223
54179970,1cyclictest0-21swapper/1108:52:033
54179970,1cyclictest0-21swapper/1108:26:553
54179970,1cyclictest0-21swapper/1108:15:023
54179970,1cyclictest0-21swapper/1107:11:443
54129976,1cyclictest0-21swapper/1012:36:192
54129976,1cyclictest0-21swapper/1012:00:182
54129976,1cyclictest0-21swapper/1011:50:172
54129976,1cyclictest0-21swapper/1011:20:162
54129976,1cyclictest0-21swapper/1010:40:152
54129976,1cyclictest0-21swapper/1010:35:322
54129976,1cyclictest0-21swapper/1010:13:012
54129976,1cyclictest0-21swapper/1010:05:342
54129976,1cyclictest0-21swapper/1009:40:212
54129976,1cyclictest0-21swapper/1009:27:002
54129976,1cyclictest0-21swapper/1009:15:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional