You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 03:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Dec 18, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
818528975,9sleep20-21swapper/219:07:122
837328572,9sleep00-21swapper/019:09:400
836127966,9sleep30-21swapper/319:09:303
820027958,6sleep10-21swapper/119:07:241
849499221,2cyclictest0-21swapper/020:05:150
8496992121,0cyclictest0-21swapper/122:55:241
850299200,15cyclictest0-21swapper/319:55:123
849499205,7cyclictest22982-21munin-run00:10:000
8494992020,0cyclictest0-21swapper/022:13:460
849999191,5cyclictest32486-21cron00:30:002
849699192,16cyclictest0-21swapper/121:10:221
8494991817,1cyclictest0-21swapper/022:30:120
850299170,17cyclictest0-21swapper/322:16:533
850299170,15cyclictest0-21swapper/319:20:143
850299161,10cyclictest0-21swapper/322:55:133
849699162,11cyclictest0-21swapper/123:45:131
849699162,11cyclictest0-21swapper/123:45:121
849499160,9cyclictest0-21swapper/023:25:130
850299159,5cyclictest27498-21cron22:05:003
8499991512,2cyclictest17928-21cron21:45:012
8496991512,2cyclictest32304-21sendmail20:00:001
8496991512,2cyclictest0-21swapper/123:21:281
8496991511,3cyclictest20683-21grep19:35:151
849499150,9cyclictest2469-21perf00:35:000
849499150,1cyclictest0-21swapper/021:35:130
849499150,10cyclictest0-21swapper/019:50:140
8502991412,1cyclictest0-21swapper/321:00:143
8502991410,3cyclictest10311-21grep21:25:273
8499991411,2cyclictest3401-21perf23:30:012
8499991411,2cyclictest229242sleep220:45:182
8499991410,2cyclictest2288-21sed21:10:172
8499991410,2cyclictest20015-21sort20:40:162
849699149,3cyclictest4659-21sleep20:10:011
8496991410,3cyclictest31167-21grep00:25:181
8494991410,3cyclictest28908-21cron20:59:590
8494991410,2cyclictest7010-21cron20:15:000
8494991410,2cyclictest4680-21sed21:15:170
8494991410,2cyclictest28664-21turbostat.cron23:14:590
850299139,3cyclictest8356-21uname22:30:203
8502991311,1cyclictest20289-21turbostat.cron21:49:553
8502991310,2cyclictest8144-21dump-pmu-power23:40:013
8502991310,2cyclictest28464-21grep20:55:273
8502991310,2cyclictest27154-21cat00:15:273
8502991310,2cyclictest23376-21grep00:10:153
849999139,3cyclictest6227-21cat21:20:002
849999139,3cyclictest10799-21sh21:30:012
849999139,2cyclictest28722-21grep00:20:162
849999135,2cyclictest225-21systemd-journal22:04:532
8499991310,2cyclictest29509-21head19:50:282
8499991310,2cyclictest27544-21grep20:55:172
8499991310,2cyclictest25166-21cron19:45:002
8499991310,2cyclictest20793-21grep00:05:162
8499991310,2cyclictest18292-21grep00:00:152
8499991310,2cyclictest1711-21grep22:15:252
8499991310,2cyclictest14409-21basename22:45:152
8499991310,2cyclictest12785-21sed22:40:172
8499991310,2cyclictest12202-21tr21:30:202
8499991310,2cyclictest11826-21grep20:25:012
849999130,12cyclictest20445-21tr22:55:202
849999130,12cyclictest17381-21sed23:55:262
849999130,12cyclictest14731-21grep19:20:222
849699139,2cyclictest25640-21basename00:15:151
849699139,2cyclictest14945-21wc21:35:261
849699136,6cyclictest25-21ksoftirqd/119:30:131
849699135,4cyclictest225-21systemd-journal00:39:591
849699131,5cyclictest225-21systemd-journal21:19:591
8496991310,2cyclictest7967-21tail21:20:261
8496991310,2cyclictest25891-21grep23:05:271
8496991310,2cyclictest25667-21grep20:50:221
8496991310,2cyclictest17736-21tail20:35:181
849699130,13cyclictest0-21swapper/100:00:161
849699130,11cyclictest28104-21grep19:50:161
849499139,2cyclictest23885-21grep00:10:160
849499139,2cyclictest15157-21sed22:45:180
849499132,6cyclictest30135-21awk00:25:000
849499130,1cyclictest0-21swapper/023:57:410
850299129,2cyclictest5523-21cut22:25:163
850299129,2cyclictest4335-21sed00:35:273
850299129,2cyclictest29951-21wc19:55:013
850299129,2cyclictest26146-21grep20:50:263
850299129,2cyclictest25376-21ls23:05:223
850299129,2cyclictest22658-21grep20:45:163
850299129,2cyclictest16629-21sed19:25:163
850299129,2cyclictest14125-21head23:50:153
850299129,2cyclictest12032-21tr23:45:223
850299129,2cyclictest12032-21tr23:45:213
850299129,2cyclictest11312-21grep20:20:263
850299128,3cyclictest18345-21cut19:30:143
850299124,4cyclictest225-21systemd-journal20:35:013
849999129,2cyclictest29528-21grep23:15:172
849999129,2cyclictest2794-21tr20:05:172
849999129,2cyclictest27716-21cron00:20:012
849999129,2cyclictest23272-21grep00:10:142
849999129,2cyclictest21329-21sed21:50:162
849999129,2cyclictest19471-21grep19:30:222
849999128,3cyclictest6653-21cut23:35:172
849999128,3cyclictest6561-21grep21:20:142
849699129,2cyclictest9196-21grep23:40:161
849699129,2cyclictest7544-21tr20:15:151
849699129,2cyclictest7474-21wc23:35:271
849699129,2cyclictest4419-21sed23:30:171
849699129,2cyclictest31319-21ls21:05:001
849699129,2cyclictest27247-21cut23:10:151
849699129,2cyclictest27066-21grep20:55:151
849699129,2cyclictest24192-21tr19:40:221
849699129,2cyclictest22318-21tr20:45:151
849699128,4cyclictest25-21ksoftirqd/121:50:211
849699128,3cyclictest15560-21cat23:55:001
849699128,2cyclictest16044-21grep22:45:281
849699128,2cyclictest15934-21wget19:25:141
849699126,5cyclictest11735-21cron22:40:001
849699125,3cyclictest225-21systemd-journal23:05:001
849499129,2cyclictest4350-21grep23:30:180
849499129,2cyclictest3736-21grep22:20:230
849499129,2cyclictest26299-21apt23:09:590
849499129,2cyclictest23787-21grep19:40:180
849499129,2cyclictest22685-21sort20:45:160
849499129,2cyclictest20201-21head20:40:170
849499129,2cyclictest11765-21grep21:30:160
849499128,3cyclictest12646-21grep22:40:160
849499128,2cyclictest32592-21tr22:15:130
849499128,2cyclictest18181-21basename00:00:150
849499128,2cyclictest11981-21cat20:25:010
849499124,7cyclictest13235-21awk19:20:000
849499124,4cyclictest225-21systemd-journal23:40:000
849499121,1cyclictest0-21swapper/023:15:150
850299119,1cyclictest12062sleep321:09:263
850299118,2cyclictest4738-21cat23:30:203
850299118,2cyclictest4165-21sed20:05:283
850299118,2cyclictest31930-21chrt22:10:563
850299118,2cyclictest30665-21grep00:25:163
850299118,2cyclictest30625-21grep23:15:273
850299118,2cyclictest30263-21cat00:25:003
850299118,2cyclictest23096-21aten2_r7power_p21:55:123
850299118,2cyclictest22524-21grep23:00:173
850299118,2cyclictest21190-21tune2fs21:50:153
850299118,2cyclictest20987-21expr19:35:163
850299118,2cyclictest1901-21expr21:10:153
850299118,2cyclictest17760-21head22:50:163
850299118,2cyclictest17491-21grep21:40:253
850299118,2cyclictest13506-21cut21:35:153
850299118,2cyclictest13502-21cat22:40:263
850299118,2cyclictest12735-21fschecks_time20:25:163
850299117,3cyclictest28161-21grep23:10:253
850299110,10cyclictest4720-21cat21:15:163
849999119,1cyclictest8901-21sed22:30:262
849999119,1cyclictest24677-21sed21:55:252
849999118,2cyclictest5299-21grep23:30:272
849999118,2cyclictest4010-21cat22:20:272
849999118,2cyclictest31595-21basename21:05:142
849999118,2cyclictest27136-21grep19:45:262
849999118,2cyclictest23455-21sed23:00:282
849999118,2cyclictest22985-21grep19:40:012
849999118,2cyclictest18823-21fschecks_count21:45:162
849999118,2cyclictest15538-21sh21:40:012
849999118,2cyclictest13088-21chrt19:18:382
849999118,2cyclictest12347-21cut20:25:142
849999118,2cyclictest10836-21munin-run19:15:002
849999117,3cyclictest7163-21cat20:15:002
849999117,3cyclictest1287-21tr20:00:212
849999117,3cyclictest11248-21grep22:35:272
849999111,7cyclictest3838-21ntp_kernel_err00:35:212
849999110,10cyclictest31048-21taskset21:01:452
849699118,2cyclictest7499-21tr22:30:141
849699118,2cyclictest3812-21gzip22:20:231
849699118,2cyclictest30766-21grep22:10:161
849699118,2cyclictest2997-21head23:25:271
849699118,2cyclictest26155-21tail19:45:161
849699118,2cyclictest24343-21head00:10:201
849699118,2cyclictest17840-21taskset23:59:511
849699118,2cyclictest14226-21grep19:20:161
849699118,2cyclictest13688-21grep20:25:261
849699118,2cyclictest1336-21head22:15:211
849699118,2cyclictest12403-21tr21:30:221
849699118,2cyclictest10202-21sort21:25:271
849699117,3cyclictest25-21ksoftirqd/100:07:561
849699117,2cyclictest1600-21wc00:30:231
849699112,7cyclictest23718-21tail21:55:161
849499118,2cyclictest9139-21expr19:10:120
849499118,2cyclictest6794-21chrt22:27:440
849499118,2cyclictest3422-21grep00:35:180
849499118,2cyclictest30223-21cut21:00:210
849499118,2cyclictest27689-21chrt00:19:440
849499118,2cyclictest26127-21tail22:00:160
849499118,2cyclictest25119-21dump-pmu-power22:00:010
849499118,2cyclictest22688-21taskset19:39:010
849499118,2cyclictest22439-21cut23:00:170
849499118,2cyclictest18974-21head21:45:160
849499118,2cyclictest17269-21grep22:50:150
849499118,2cyclictest17129-21uname21:40:220
849499118,2cyclictest15152-21sed19:20:260
849499118,2cyclictest13127-21sort23:50:000
849499118,2cyclictest13127-21sort23:50:000
849499117,3cyclictest567-21grep23:20:290
849499117,3cyclictest21300-21head21:50:150
849499115,2cyclictest225-21systemd-journal22:10:010
849499111,5cyclictest17993-21perf19:30:010
849499110,8cyclictest0-21swapper/020:15:140
849499110,5cyclictest0-21swapper/019:30:130
850299108,1cyclictest8507-21cut20:15:233
850299108,1cyclictest644-21expr00:30:143
850299108,1cyclictest5709-21hostname20:10:153
850299108,1cyclictest3313-21grep22:20:173
850299108,1cyclictest22868-21taskset00:08:513
850299108,1cyclictest17189-21cat23:55:253
850299108,1cyclictest14760-21cat20:30:153
850299108,1cyclictest1232-21taskset23:25:033
850299108,1cyclictest11270-21cat21:30:133
850299108,1cyclictest11067-21chrt19:15:113
850299107,2cyclictest6568-21tr21:20:153
850299107,2cyclictest352-21tr20:00:163
850299107,2cyclictest31411-21basename23:20:163
850299107,2cyclictest28059-21tr22:05:163
850299107,2cyclictest26134-21tune2fs19:45:163
850299107,2cyclictest19401-21cat20:40:123
850299107,2cyclictest14883-21df22:45:163
850299107,2cyclictest10422-21sh22:35:173
850299100,9cyclictest9826-21smartctl23:40:243
849999108,1cyclictest6572-21sed22:25:272
849999108,1cyclictest26797-21expr23:10:132
849999108,1cyclictest26252-21chrt23:09:412
849999108,1cyclictest24434-21apache_volume20:50:122
849999108,1cyclictest14744-21cstates20:30:152
849999107,2cyclictest404-21tr00:30:142
849999107,2cyclictest31614-21cut23:20:162
849999107,2cyclictest31447-21grep19:55:212
849999107,2cyclictest27979-21grep22:05:152
849999107,2cyclictest16440-21df19:25:162
849999107,2cyclictest15580-21cat23:55:002
849999107,2cyclictest12738-21chrt23:48:012
849999107,2cyclictest12738-21chrt23:48:002
849999105,3cyclictest17282-21tr20:35:162
849999100,9cyclictest8575-21egrep23:40:142
849999100,9cyclictest18728-21taskset22:50:312
849699108,1cyclictest277402sleep122:05:101
849699108,1cyclictest19530-21which21:45:221
849699107,2cyclictest932-21head20:00:171
849699107,2cyclictest6815-21chrt22:28:011
849699107,2cyclictest10362-21sed19:10:231
849699107,1cyclictest19625-21grep20:40:141
849699103,3cyclictest11816-21gltestperf19:15:161
849699102,4cyclictest225-21systemd-journal20:30:121
849699101,4cyclictest225-21systemd-journal22:05:001
849699101,1cyclictest0-21swapper/120:10:111
849699100,9cyclictest32117-21sed21:05:161
849699100,9cyclictest16031-21grep21:40:141
849699100,8cyclictest28282-21cpuspeed_turbos00:20:141
849699100,1cyclictest0-21swapper/120:20:181
849499108,1cyclictest9786-21sendmail_mailtr23:40:230
849499108,1cyclictest9462sleep021:06:100
849499108,1cyclictest6400-21aten2_r7power_p21:20:130
849499108,1cyclictest32353-21taskset00:28:220
849499108,1cyclictest30651-21expr19:55:140
849499108,1cyclictest272982sleep019:46:570
849499108,1cyclictest26428-21chrt20:53:400
849499108,1cyclictest20485-21ntpq22:55:210
849499108,1cyclictest2017-21taskset21:10:150
849499108,1cyclictest18478-21cat20:35:280
849499108,1cyclictest14768-21expr20:30:140
849499108,1cyclictest13770-21expr23:50:140
849499108,1cyclictest12271-21grep20:25:140
849499108,1cyclictest104632sleep021:25:510
849499107,2cyclictest2264-21cron20:05:000
849499107,2cyclictest11400-21chrt22:35:510
85029997,1cyclictest86762sleep319:10:033
85029997,1cyclictest23866-21smartctl19:40:173
85029997,1cyclictest18356-21cat00:00:163
84999997,1cyclictest7659-21expr20:15:162
84999997,1cyclictest30123-21cat22:10:112
84969997,1cyclictest29208-21cut23:15:161
84969997,1cyclictest119802sleep122:40:031
84969995,4cyclictest25-21ksoftirqd/122:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional