You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 14:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Wed Dec 03, 2025 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991920,82ptp4l0-21swapper/107:05:401
275991920,82ptp4l0-21swapper/107:05:391
15432800,0sleep70-21swapper/709:35:159
269952660,2sleep10131rcu_preempt12:15:002
1655326554,8sleep100-21swapper/1007:05:272
1655326554,8sleep100-21swapper/1007:05:262
23902640,0sleep70-21swapper/707:30:139
1696826454,8sleep00-21swapper/007:07:290
1696826454,8sleep00-21swapper/007:07:290
1695126456,6sleep110-21swapper/1107:07:153
1695126456,6sleep110-21swapper/1107:07:153
1683926452,10sleep50-21swapper/507:05:337
1683926452,10sleep50-21swapper/507:05:337
1704226350,11sleep70-21swapper/707:08:379
1704226350,11sleep70-21swapper/707:08:369
1690726352,9sleep60-21swapper/607:06:358
1690726352,9sleep60-21swapper/607:06:348
1683826353,8sleep40-21swapper/407:05:326
1683826353,8sleep40-21swapper/407:05:326
116912620,0sleep00-21swapper/007:40:190
276091410,39phc2sys0-21swapper/207:09:144
276091410,39phc2sys0-21swapper/207:09:144
1748999212,9cyclictest0-21swapper/611:30:018
1748999192,7cyclictest0-21swapper/609:35:008
1746699193,11cyclictest0-21swapper/209:25:014
1750399183,9cyclictest0-21swapper/912:25:0011
1747199180,2cyclictest0-21swapper/307:20:185
1746699183,9cyclictest0-21swapper/207:20:004
1746699183,9cyclictest0-21swapper/207:20:004
1746699183,10cyclictest0-21swapper/209:55:024
275991170,0ptp4l0-21swapper/108:30:261
275991170,0ptp4l0-21swapper/108:15:151
1750399172,9cyclictest0-21swapper/911:05:0011
1749999173,9cyclictest0-21swapper/810:50:0010
1749999173,8cyclictest0-21swapper/811:45:0110
1746699173,9cyclictest0-21swapper/207:40:004
275991160,0ptp4l0-21swapper/109:15:171
1750399161,2cyclictest0-21swapper/909:50:1611
1749999162,8cyclictest0-21swapper/808:30:0010
1749399162,8cyclictest0-21swapper/712:30:239
1748999163,9cyclictest0-21swapper/610:50:018
1748999162,9cyclictest0-21swapper/611:40:008
1748999162,8cyclictest0-21swapper/610:20:248
1748999162,8cyclictest0-21swapper/609:25:018
1747199164,8cyclictest0-21swapper/307:55:005
1747199163,8cyclictest0-21swapper/308:30:025
1747199161,10cyclictest0-21swapper/307:20:005
1747199161,10cyclictest0-21swapper/307:20:005
1746699163,9cyclictest0-21swapper/209:35:024
1746699162,9cyclictest0-21swapper/208:45:004
1745699162,9cyclictest0-21swapper/009:00:240
1745699162,8cyclictest0-21swapper/011:05:010
275991150,1ptp4l21-21ksoftirqd/110:50:191
275991150,1ptp4l21-21ksoftirqd/107:15:031
275991150,1ptp4l21-21ksoftirqd/107:15:031
275991150,0ptp4l0-21swapper/110:15:211
275991150,0ptp4l0-21swapper/108:10:221
232552150,0sleep100-21swapper/1009:20:162
1749999152,7cyclictest0-21swapper/812:00:0010
1748999153,7cyclictest0-21swapper/607:25:018
1748999151,9cyclictest0-21swapper/610:14:598
1747199152,8cyclictest0-21swapper/310:55:015
1746699152,9cyclictest0-21swapper/208:30:314
1746699152,8cyclictest0-21swapper/208:05:004
1745699154,7cyclictest0-21swapper/008:35:000
1745699152,8cyclictest0-21swapper/012:25:000
275991140,0ptp4l0-21swapper/108:45:001
260502140,0sleep00-21swapper/012:10:250
208172140,0sleep30-21swapper/310:00:145
1750399142,7cyclictest0-21swapper/907:15:0011
1750399142,7cyclictest0-21swapper/907:15:0011
17503991411,2cyclictest0-21swapper/910:25:1611
1750399140,9cyclictest0-21swapper/907:45:0011
1750399140,3cyclictest0-21swapper/907:45:1711
1749999143,7cyclictest0-21swapper/811:00:0110
1749999142,7cyclictest0-21swapper/812:05:0110
1749999142,7cyclictest0-21swapper/810:30:0010
1749399143,7cyclictest0-21swapper/707:55:009
1748999143,6cyclictest0-21swapper/609:20:008
1748999142,8cyclictest0-21swapper/611:05:268
1748999142,8cyclictest0-21swapper/609:50:008
1748999141,9cyclictest0-21swapper/611:00:008
1747199143,7cyclictest0-21swapper/311:15:015
1747199142,7cyclictest0-21swapper/308:50:015
1746299143,6cyclictest0-21swapper/109:55:001
1745699142,8cyclictest0-21swapper/007:40:010
1745699140,8cyclictest0-21swapper/012:25:010
170152141,11sleep80-21swapper/807:08:1410
170152141,11sleep80-21swapper/807:08:1410
275991130,5ptp4l21-21ksoftirqd/109:30:131
275991130,4ptp4l21-21ksoftirqd/111:00:251
275991130,1ptp4l21-21ksoftirqd/112:17:531
275991130,1ptp4l21-21ksoftirqd/109:20:141
275991130,1ptp4l17468-21/usr/sbin/munin11:20:151
275991130,0ptp4l0-21swapper/112:00:241
275991130,0ptp4l0-21swapper/111:50:181
275991130,0ptp4l0-21swapper/110:10:261
275991130,0ptp4l0-21swapper/109:25:231
275991130,0ptp4l0-21swapper/108:05:231
261982130,0sleep80-21swapper/810:05:1910
1749999131,7cyclictest0-21swapper/808:05:0010
1749399132,6cyclictest0-21swapper/710:45:019
1749399132,6cyclictest0-21swapper/708:55:009
1748999132,8cyclictest0-21swapper/610:35:008
1747199132,6cyclictest0-21swapper/308:20:015
1747199131,7cyclictest0-21swapper/310:15:295
1746699132,7cyclictest0-21swapper/211:20:004
1746299133,6cyclictest0-21swapper/107:45:001
1746299132,7cyclictest0-21swapper/107:35:011
1746299130,7cyclictest0-21swapper/110:30:001
1745699133,6cyclictest0-21swapper/008:45:010
1745699132,7cyclictest0-21swapper/012:40:000
1745699132,7cyclictest0-21swapper/012:05:010
1745699132,7cyclictest0-21swapper/008:20:010
1745699132,6cyclictest0-21swapper/010:50:000
1745699131,8cyclictest13080-21kworker/u24:008:00:010
1745699130,3cyclictest0-21swapper/009:25:160
275991120,1ptp4l21-21ksoftirqd/111:45:141
275991120,1ptp4l21-21ksoftirqd/110:35:211
275991120,1ptp4l21-21ksoftirqd/108:55:231
275991120,1ptp4l0-21swapper/110:20:141
275991120,0ptp4l0-21swapper/112:35:181
275991120,0ptp4l0-21swapper/112:30:191
275991120,0ptp4l0-21swapper/112:25:211
275991120,0ptp4l0-21swapper/111:55:191
275991120,0ptp4l0-21swapper/111:25:191
275991120,0ptp4l0-21swapper/110:40:071
275991120,0ptp4l0-21swapper/109:40:151
275991120,0ptp4l0-21swapper/108:50:201
275991120,0ptp4l0-21swapper/107:55:171
275991120,0ptp4l0-21swapper/107:45:161
1750399122,6cyclictest0-21swapper/910:05:0111
1749999122,6cyclictest0-21swapper/809:05:2310
1749999122,6cyclictest0-21swapper/807:45:0010
1745699123,6cyclictest0-21swapper/009:00:010
1745699122,6cyclictest0-21swapper/011:40:260
1745699121,7cyclictest0-21swapper/008:05:230
275991110,2ptp4l7042-21fw_forwarded_lo07:35:181
275991110,2ptp4l21400-21date07:15:001
275991110,2ptp4l21400-21date07:15:001
275991110,2ptp4l21358-21sed12:05:221
275991110,2ptp4l17409-21df09:55:171
275991110,1ptp4l27368-21df07:20:141
275991110,1ptp4l0-21swapper/112:20:161
275991110,1ptp4l0-21swapper/111:40:271
275991110,1ptp4l0-21swapper/109:45:161
275991110,0ptp4l0-21swapper/111:10:201
275991110,0ptp4l0-21swapper/111:05:251
275991110,0ptp4l0-21swapper/110:33:161
275991110,0ptp4l0-21swapper/108:35:211
275991110,0ptp4l0-21swapper/108:25:141
275991110,0ptp4l0-21swapper/108:20:191
275991110,0ptp4l0-21swapper/107:50:021
1751399119,1cyclictest0-21swapper/1108:00:203
1751399118,2cyclictest0-21swapper/1111:55:203
1751399110,10cyclictest0-21swapper/1111:10:143
1749999110,2cyclictest0-21swapper/809:20:1310
1749399110,2cyclictest12487-21head10:30:169
1748999110,10cyclictest0-21swapper/608:10:158
1747199110,10cyclictest0-21swapper/312:15:185
1746299117,3cyclictest25506-21sort10:45:261
1745699113,7cyclictest0-21swapper/007:35:000
275991100,1ptp4l7127-21tr09:00:131
275991100,1ptp4l32164-21irqstats07:25:471
275991100,1ptp4l29691-21df_inode11:35:141
275991100,1ptp4l26671-21cut11:30:181
275991100,1ptp4l21594-21perl10:00:161
275991100,1ptp4l15128-21diskstats09:10:151
275991100,0ptp4l0-21swapper/112:10:321
275991100,0ptp4l0-21swapper/110:55:191
275991100,0ptp4l0-21swapper/108:00:181
1749999109,1cyclictest0-21swapper/807:10:1310
1749999109,1cyclictest0-21swapper/807:10:1310
1749399102,6cyclictest0-21swapper/707:55:279
1748999101,8cyclictest0-21swapper/610:50:168
1748999100,9cyclictest521-21perl10:15:198
1748999100,9cyclictest15359-21munin-run12:00:008
1747599100,9cyclictest12063-21swap07:40:246
1747599100,9cyclictest0-21swapper/411:00:136
1747199102,5cyclictest0-21swapper/311:35:015
1747199102,5cyclictest0-21swapper/307:40:005
1747199100,9cyclictest13798-21diskmemload10:44:505
1747199100,9cyclictest10375-21users10:25:255
1746299102,5cyclictest0-21swapper/108:50:001
1745699108,1cyclictest0-21swapper/012:15:250
1745699106,3cyclictest21207-21taskset09:16:320
1745699100,9cyclictest0-21swapper/010:10:200
27599190,1ptp4l24393-21wget10:05:131
27599190,1ptp4l21-21ksoftirqd/111:15:151
27599190,1ptp4l21-21ksoftirqd/109:35:161
27599190,0ptp4l0-21swapper/109:05:011
19259290,0chrt0-21swapper/1112:05:003
175139998,1cyclictest0-21swapper/1108:50:163
175139997,1cyclictest0-21swapper/1109:35:023
175139990,8cyclictest0-21swapper/1110:55:173
175039991,5cyclictest0-21swapper/911:20:2511
174999990,9cyclictest0-21swapper/809:15:2510
174999990,1cyclictest0-21swapper/812:20:0110
174939998,1cyclictest0-21swapper/709:20:029
174939990,9cyclictest0-21swapper/709:25:179
174939990,9cyclictest0-21swapper/708:30:019
174899998,1cyclictest0-21swapper/607:15:008
174899996,2cyclictest7929-21hddtemp_smartct12:30:158
174839997,1cyclictest0-21swapper/510:34:597
174839990,1cyclictest0-21swapper/508:05:127
174759990,1cyclictest0-21swapper/411:05:266
174719998,1cyclictest0-21swapper/309:25:255
174719996,2cyclictest12308-21cpuspeed_turbos11:55:135
174719992,6cyclictest0-21swapper/310:55:235
174719990,1cyclictest0-21swapper/310:10:005
174669998,1cyclictest0-21swapper/211:30:194
174669996,2cyclictest28829-21/usr/sbin/munin10:50:214
174669992,6cyclictest5668-21grep10:20:184
174669990,8cyclictest5677-21if_enp1s008:15:164
174569998,1cyclictest28798-21irqrtprio08:45:170
174569998,1cyclictest0-21swapper/009:40:210
174569990,9cyclictest0-21swapper/007:15:000
174569990,9cyclictest0-21swapper/007:15:000
174569990,7cyclictest0-21swapper/010:20:180
174569990,1cyclictest0-21swapper/012:05:290
174569990,1cyclictest0-21swapper/010:50:150
174569990,1cyclictest0-21swapper/010:15:100
9498080,1rtkit-daemon14238-21processes08:25:214
19159180,0getstats0-21swapper/210:10:344
175139987,1cyclictest0-21swapper/1112:10:153
175139987,1cyclictest0-21swapper/1111:30:003
175139987,1cyclictest0-21swapper/1109:40:143
175139987,1cyclictest0-21swapper/1107:35:263
175139987,1cyclictest0-21swapper/1107:10:193
175139987,1cyclictest0-21swapper/1107:10:183
175139980,8cyclictest0-21swapper/1111:52:243
175139980,8cyclictest0-21swapper/1111:40:153
175139980,1cyclictest0-21swapper/1112:39:523
175139980,1cyclictest0-21swapper/1111:45:243
175139980,1cyclictest0-21swapper/1110:15:263
175099987,1cyclictest0-21swapper/1011:50:242
175099987,1cyclictest0-21swapper/1008:50:152
175099986,1cyclictest0-21swapper/1012:35:272
175099986,1cyclictest0-21swapper/1009:10:272
175099986,1cyclictest0-21swapper/1007:45:192
175099985,2cyclictest0-21swapper/1009:30:162
175099980,7cyclictest0-21swapper/1008:41:422
175099980,1cyclictest0-21swapper/1012:30:192
175099980,1cyclictest0-21swapper/1009:55:292
175099980,1cyclictest0-21swapper/1008:00:182
175039987,1cyclictest0-21swapper/912:30:1311
175039987,1cyclictest0-21swapper/910:45:1511
175039980,8cyclictest0-21swapper/910:10:1411
175039980,1cyclictest0-21swapper/909:55:1611
175039980,1cyclictest0-21swapper/909:00:1411
174999987,1cyclictest0-21swapper/809:00:1510
174999981,5cyclictest13635-21aten2_r3power_v07:45:1210
174999981,4cyclictest0-21swapper/810:05:0210
174999980,8cyclictest0-21swapper/811:30:0010
174999980,8cyclictest0-21swapper/810:10:1610
174999980,1cyclictest0-21swapper/812:29:4610
174999980,1cyclictest0-21swapper/807:35:2710
174939987,1cyclictest0-21swapper/711:15:219
174939987,1cyclictest0-21swapper/710:05:149
174939980,1cyclictest0-21swapper/707:20:249
174899986,1cyclictest0-21swapper/608:10:008
174899980,1cyclictest0-21swapper/612:25:168
174899980,1cyclictest0-21swapper/611:05:018
174899980,1cyclictest0-21swapper/610:30:208
174899980,1cyclictest0-21swapper/607:35:268
174839987,1cyclictest0-21swapper/509:25:157
174839987,1cyclictest0-21swapper/509:20:167
174839980,8cyclictest0-21swapper/512:25:177
174839980,8cyclictest0-21swapper/512:00:117
174839980,1cyclictest0-21swapper/512:30:197
174759987,1cyclictest0-21swapper/411:55:226
174759987,1cyclictest0-21swapper/410:00:146
174759987,1cyclictest0-21swapper/409:00:196
174759980,8cyclictest4316-21ps07:30:216
174759980,8cyclictest0-21swapper/412:00:156
174759980,8cyclictest0-21swapper/410:35:156
174759980,1cyclictest0-21swapper/410:05:216
174759980,1cyclictest0-21swapper/409:20:216
174759980,1cyclictest0-21swapper/407:45:166
174719987,1cyclictest0-21swapper/309:10:255
174719980,8cyclictest0-21swapper/312:05:235
174719980,8cyclictest0-21swapper/309:35:165
174719980,1cyclictest0-21swapper/312:25:255
174669987,1cyclictest0-21swapper/207:10:224
174669987,1cyclictest0-21swapper/207:10:214
174669985,2cyclictest18394-21turbostat10:44:544
174669980,8cyclictest0-21swapper/209:00:154
174669980,7cyclictest17379-21/usr/sbin/munin07:50:174
174669980,7cyclictest0-21swapper/211:00:244
174669980,7cyclictest0-21swapper/210:55:124
174669980,7cyclictest0-21swapper/210:05:134
174669980,2cyclictest32149-21sshd11:39:354
174669980,1cyclictest0-21swapper/211:24:344
174669980,1cyclictest0-21swapper/210:00:254
174669980,1cyclictest0-21swapper/209:40:264
174669980,1cyclictest0-21swapper/209:30:014
174669980,1cyclictest0-21swapper/208:50:154
174669980,1cyclictest0-21swapper/208:45:254
174669980,1cyclictest0-21swapper/208:20:144
174669980,1cyclictest0-21swapper/207:40:184
174569987,1cyclictest0-21swapper/012:30:240
174569987,1cyclictest0-21swapper/011:15:210
174569987,1cyclictest0-21swapper/011:12:230
174569987,1cyclictest0-21swapper/010:55:120
174569987,1cyclictest0-21swapper/008:35:170
174569987,1cyclictest0-21swapper/008:10:240
174569980,8cyclictest0-21swapper/009:30:140
174569980,1cyclictest0-21swapper/011:35:120
174569980,1cyclictest0-21swapper/010:35:250
174569980,1cyclictest0-21swapper/010:30:130
13849281,6sleep30-21swapper/307:05:075
13849281,6sleep30-21swapper/307:05:075
13366281,5sleep90-21swapper/907:05:0211
13366281,5sleep90-21swapper/907:05:0111
25459270,1sleep100-21swapper/1010:45:272
175139976,1cyclictest0-21swapper/1112:20:153
175139976,1cyclictest0-21swapper/1112:15:243
175139976,1cyclictest0-21swapper/1111:05:093
175139976,1cyclictest0-21swapper/1111:00:053
175139976,1cyclictest0-21swapper/1110:50:133
175139976,1cyclictest0-21swapper/1110:45:383
175139976,1cyclictest0-21swapper/1110:11:353
175139976,1cyclictest0-21swapper/1109:45:053
175139976,1cyclictest0-21swapper/1109:36:233
175139976,1cyclictest0-21swapper/1109:10:203
175139976,1cyclictest0-21swapper/1108:45:093
175139976,1cyclictest0-21swapper/1108:40:133
175139976,1cyclictest0-21swapper/1108:30:503
175139976,1cyclictest0-21swapper/1108:10:583
175139976,1cyclictest0-21swapper/1107:57:173
175139976,1cyclictest0-21swapper/1107:53:403
175139976,1cyclictest0-21swapper/1107:45:263
175139976,1cyclictest0-21swapper/1107:18:113
175139976,1cyclictest0-21swapper/1107:18:103
175139974,2cyclictest1020-21runrttasks09:15:043
175139970,7cyclictest0-21swapper/1112:25:153
175139970,7cyclictest0-21swapper/1112:06:273
175139970,7cyclictest0-21swapper/1111:21:193
175139970,7cyclictest0-21swapper/1111:18:083
175139970,7cyclictest0-21swapper/1110:41:113
175139970,7cyclictest0-21swapper/1110:38:433
175139970,7cyclictest0-21swapper/1110:31:263
175139970,7cyclictest0-21swapper/1110:06:133
175139970,7cyclictest0-21swapper/1109:50:343
175139970,7cyclictest0-21swapper/1109:20:543
175139970,7cyclictest0-21swapper/1109:01:143
175139970,7cyclictest0-21swapper/1108:36:083
175139970,7cyclictest0-21swapper/1108:05:583
175139970,7cyclictest0-21swapper/1107:26:003
175139970,1cyclictest0-21swapper/1112:31:203
175139970,1cyclictest0-21swapper/1111:35:383
175139970,1cyclictest0-21swapper/1111:31:073
175139970,1cyclictest0-21swapper/1110:25:583
175139970,1cyclictest0-21swapper/1110:03:213
175139970,1cyclictest0-21swapper/1109:55:533
175139970,1cyclictest0-21swapper/1109:25:323
175139970,1cyclictest0-21swapper/1109:06:163
175139970,1cyclictest0-21swapper/1108:26:133
175139970,1cyclictest0-21swapper/1108:24:363
175139970,1cyclictest0-21swapper/1108:15:113
175139970,1cyclictest0-21swapper/1107:43:573
175139970,1cyclictest0-21swapper/1107:30:183
175139970,1cyclictest0-21swapper/1107:21:463
175099976,1cyclictest0-21swapper/1012:20:502
175099976,1cyclictest0-21swapper/1012:15:152
175099976,1cyclictest0-21swapper/1012:00:032
175099976,1cyclictest0-21swapper/1011:46:512
175099976,1cyclictest0-21swapper/1011:37:132
175099976,1cyclictest0-21swapper/1011:30:142
175099976,1cyclictest0-21swapper/1011:05:472
175099976,1cyclictest0-21swapper/1010:51:582
175099976,1cyclictest0-21swapper/1010:37:322
175099976,1cyclictest0-21swapper/1010:33:122
175099976,1cyclictest0-21swapper/1009:52:152
175099976,1cyclictest0-21swapper/1009:00:142
175099976,1cyclictest0-21swapper/1008:45:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional